.. SPDX-FileCopyrightText: Copyright (c) 2026 The Newton Developers .. SPDX-License-Identifier: CC-BY-4.0 .. currentmodule:: newton.actuators Actuators ========= .. experimental:: The actuator API may change without prior notice. Feedback is welcome — please file issues or discussion threads. Actuators provide composable implementations that read physics simulation state, compute effort, and **accumulate** (scatter-add) the effort into control arrays for application to the simulation. The caller must zero the output array before stepping actuators each frame. The simulator does not need to be part of Newton: actuators are designed to be reusable anywhere the caller can provide state arrays and consume effort. Each :class:`Actuator` instance is **vectorized**: a single actuator object operates on a batch of DOF indices in global state and control arrays, allowing efficient integration into RL workflows with many parallel environments. The goal is to provide canonical actuator models with support for **differentiability** and **graphable execution** where the underlying drive implementation supports it. Actuators are designed to be easy to customize and extend for specific actuator models. Architecture ------------ An actuator is composed from three building blocks, applied in this order: .. code-block:: text Actuator ├── Delay (optional: delays command inputs by N actuator timesteps) ├── Drive (control law that computes raw effort) └── Clamping[] (clamps raw effort based on motor-limit modeling) ├── ClampingMaxEffort (±max_effort symmetric clamp) ├── ClampingDCMotor (velocity-dependent saturation) └── ClampingPositionBased (position-dependent lookup table) **Delay** Optionally delays command inputs (control targets and feedforward terms) by *N* actuator timesteps before they reach the drive, modeling communication or processing latency. The delay always produces output; when the buffer is empty or a DOF has ``delay_steps == 0``, the current command inputs are used directly. When underfilled, the lag is clamped to the available history so the oldest available entry is returned. **Drive** Computes raw actuator effort [N or N·m] from the current simulator state and control targets. This is the actuator's control law — for example PD, PID, or neural-network-based control. See the individual drive class documentation for the control-law equations. **Clamping** Clamps raw effort based on motor-limit modeling. This applies post-drive output limits to the computed effort to model motor limits such as saturation, back-EMF losses, performance envelopes, or position-dependent effort limits. Multiple clamping stages can be combined on a single actuator. The per-step pipeline is: .. code-block:: text Delay read → Drive → Clamping → Scatter-add → State updates (drive + delay write) Drives and clamping objects are pluggable: implement the :class:`DriveBase` or :class:`ClampingBase` class to add new models. .. deprecated:: 1.6 The actuator ``Controller*`` class names are retained as compatibility aliases for :class:`DriveBase`, :class:`DrivePD`, :class:`DrivePID`, :class:`DriveNeuralMLP`, and :class:`DriveNeuralLSTM`. The former ``controller`` constructor keywords and attributes are also deprecated; use ``drive``, ``drive_class``, ``drive_state``, and ``drive_kwargs``. The clamping base class is now :class:`ClampingBase`; :class:`Clamping` remains as a deprecated compatibility alias. .. note:: **Current limitations:** the first version does not include a transmission model (gear ratios / linkage transforms), supports only one DOF per actuator, and does not model actuator dynamics (inertia, friction, thermal effects). Usage ----- Actuators are registered during model construction with :meth:`~newton.ModelBuilder.add_actuator` and are instantiated automatically when the model is finalized: .. testsetup:: actuator-usage import warp as wp import newton from newton.actuators import ( Actuator, ClampingMaxEffort, DrivePD, Delay, ) builder = newton.ModelBuilder() link = builder.add_link() joint = builder.add_joint_revolute(parent=-1, child=link, axis=newton.Axis.Z) builder.add_articulation([joint]) dof_index = builder.joint_qd_start[joint] .. testcode:: actuator-usage builder.add_actuator( DrivePD, index=dof_index, kp=100.0, kd=10.0, delay_steps=5, clamping=[(ClampingMaxEffort, {"max_effort": 50.0})], ) model = builder.finalize() For manual construction (outside of :class:`~newton.ModelBuilder`), compose the components directly: .. testcode:: actuator-usage indices = wp.array([0], dtype=wp.uint32) kp = wp.array([100.0], dtype=wp.float32) kd = wp.array([10.0], dtype=wp.float32) max_e = wp.array([50.0], dtype=wp.float32) actuator = Actuator( indices, drive=DrivePD(kp=kp, kd=kd), delay=Delay(delay_steps=wp.array([5], dtype=wp.int32), max_delay=5), clamping=[ClampingMaxEffort(max_effort=max_e)], control_target_pos_attr="joint_target_q", control_target_vel_attr="joint_target_qd", ) The simulator state and control objects do not need to be a full :class:`newton.Model` / :class:`newton.Control` — any objects exposing ``joint_q``, ``joint_qd``, ``joint_target_q``, ``joint_target_qd``, ``joint_act`` (optional), and ``joint_f`` will do. This makes actuators reusable from a custom simulator or test harness: .. testcode:: actuator-usage import types sim_state = types.SimpleNamespace( joint_q=wp.array([0.0], dtype=wp.float32), joint_qd=wp.array([0.0], dtype=wp.float32), ) sim_control = types.SimpleNamespace( joint_target_q=wp.array([1.0], dtype=wp.float32), joint_target_qd=wp.array([0.0], dtype=wp.float32), joint_act=None, joint_f=wp.zeros(1, dtype=wp.float32), ) state_a = actuator.state() state_b = actuator.state() sim_control.joint_f.zero_() actuator.step(sim_state, sim_control, state_a, state_b, dt=0.01) .. _stateful-actuators: Stateful Actuators ------------------ Drives that maintain internal state (e.g. :class:`DrivePID` with an integral accumulator, :class:`DriveNeuralGRU` with hidden state, or :class:`DriveNeuralLSTM` with hidden/cell state) and actuators with a :class:`Delay` require explicit double-buffered state management. Create two state objects with :meth:`Actuator.state` and swap them after each step: .. testcode:: actuator-usage state_0 = model.actuators[0].state() state_1 = model.actuators[0].state() state = model.state() control = model.control() for step in range(3): control.joint_f.zero_() # zero output before stepping actuators model.actuators[0].step(state, control, state_0, state_1, dt=0.01) state_0, state_1 = state_1, state_0 The swap rebinds Python names, while a CUDA graph records fixed buffer addresses. A captured region with an odd number of actuator steps therefore cannot carry its final state into the next replay through the ordinary swap. A single-step region updates its destination buffer, but does not advance state across replays. Even-length regions end with the original buffer orientation and need no special handling. For an odd-length region, choose between assigning the state at the region boundary or alternating graphs for the two buffer orientations. Boundary assignment ^^^^^^^^^^^^^^^^^^^ Call :meth:`Actuator.State.assign` in place of the final swap. This uses one graph and copies the actuator state once at the boundary of each replay. .. code-block:: python with wp.ScopedCapture() as capture: for i in range(steps): control.joint_f.zero_() model.actuators[0].step(state, control, state_0, state_1, dt=0.01) if steps % 2 == 1 and i == steps - 1: state_0.assign(state_1) else: state_0, state_1 = state_1, state_0 Alternating graphs ^^^^^^^^^^^^^^^^^^ Key captured graphs by the current state buffer and alternate between them. This avoids the boundary copy and keeps at most two graphs. .. code-block:: python graphs = {} for _ in range(replays): key = id(state_0) # one entry per buffer orientation if key not in graphs: with wp.ScopedCapture() as capture: after = run_region(state_0, state_1) # ordinary swapping inside graphs[key] = (capture.graph, after) graph, (state_0, state_1) = graphs[key] wp.capture_launch(graph) Stateless actuators (e.g. a plain PD drive without delay) do not require state objects — simply omit them: .. testcode:: actuator-usage # Build a stateless actuator (no delay, stateless drive) b2 = newton.ModelBuilder() lk = b2.add_link() jt = b2.add_joint_revolute(parent=-1, child=lk, axis=newton.Axis.Z) b2.add_articulation([jt]) b2.add_actuator(DrivePD, index=b2.joint_qd_start[jt], kp=50.0) m2 = b2.finalize() m2.actuators[0].step(m2.state(), m2.control()) .. _custom-drive-inputs: Custom Drive Inputs ------------------- A drive may need an array beyond the positions, velocities and targets the actuator already reads. It lists the array names in :attr:`DriveBase.custom_inputs`. These values are caller-owned inputs for the current evaluation, so they are always read from the ``sim_control`` argument of :meth:`Actuator.step`, even when a value describes physical state such as an estimated load or measured temperature. :class:`Actuator` reads each declared value and passes it unchanged to :meth:`DriveBase.compute` in the ``custom_inputs`` mapping, keyed by attribute name. A missing attribute is passed as ``None``. The drive owns requiredness, fallback behavior, and all type, length, dtype, device, shape, and semantic validation. A drive called directly receives the same mapping explicitly; it does not look for custom values on a simulation state or control object. For actuators created by :class:`~newton.ModelBuilder`, :meth:`Actuator.register_custom_attributes` automatically registers every name declared by the drive as a ``wp.float32`` joint-DOF array on :class:`~newton.Control`. Registration allocates the array but does not populate or clear it. Write every declared value before each actuator evaluation to avoid reusing stale data. Users of another simulation engine provide the same-named arrays through their ``sim_control`` adapter instead. :meth:`Actuator.sim_state` returns an empty container with exactly the fields the actuator reads from ``sim_state``; :meth:`Actuator.sim_control` is its counterpart and includes all declared custom inputs. Passing your own object or a mapping to :meth:`Actuator.step` is also supported. .. warning:: ``sim_state`` holds references. The simulation loop swaps ``state_0`` and ``state_1`` each step, so a ``sim_state`` built once from ``state_0`` still points at that buffer after the swap and the actuator reads stale positions and velocities. Re-point its fields every step. In the loop below the drive declares one array named ``extra_input``: .. code-block:: python sim_state = actuator.sim_state() for _ in range(num_steps): extra_input = compute_extra_input(model, state_0) sim_state.joint_q = state_0.joint_q sim_state.joint_qd = state_0.joint_qd control.clear(model) control.joint_target_q.assign(target_positions) control.extra_input.assign(extra_input) actuator.step(sim_state, control, actuator_state_a, actuator_state_b, dt=dt) actuator_state_a, actuator_state_b = actuator_state_b, actuator_state_a solver.step(state_0, state_1, control, contacts, dt) state_0, state_1 = state_1, state_0 .. _neural-network-checkpoints: Neural-Network Checkpoints -------------------------- Neural-network drives support `ONNX `__ (``.onnx``), which Warp-NN runs with its own Warp kernels. :class:`DriveNeuralMLP` and :class:`DriveNeuralLSTM` additionally support Torch checkpoints through the Torch backend; :class:`DriveNeuralGRU` uses ONNX only. For :class:`DriveNeuralMLP` and :class:`DriveNeuralLSTM`, prefer ONNX exports with dynamic batch axes: axis 0 of the MLP input and axis 1 of the LSTM input, hidden state, and cell state. Existing fixed-batch exports remain supported by adapting these input declarations without changing the source checkpoint. This does not rewrite batch sizes embedded in graph operations such as ``Reshape``; re-export those models with dynamic batch axes when running a different number of actuators. Torch checkpoints must be pt2 archives saved with ``torch.export.save``. Use the ``.pt2`` extension; ``.pt`` and ``.pth`` are also accepted for these archives. Checkpoint metadata (scales and network configuration) is stored as a JSON extra file: .. code-block:: python import json import torch exported = torch.export.export(net, example_inputs) metadata = {"effort_scale": 2.0, "num_layers": 2, "hidden_size": 8} torch.export.save(exported, "policy.pt2", extra_files={"metadata.json": json.dumps(metadata)}) USD neural actuators may reference a checkpoint through ``newton:modelPath``. Importing a USD rejects Torch checkpoints (``.pt2``, ``.pt``, or ``.pth``) by default because :meth:`~newton.ModelBuilder.finalize` would load them with ``torch.export.load``, which can execute code from an untrusted checkpoint. For a trusted USD asset and checkpoint, pass ``allow_unsafe_torch_checkpoints=True`` to :meth:`newton.ModelBuilder.add_usd`. Programmatic construction of a neural drive from a checkpoint path remains the caller's explicit trust decision. :class:`DriveNeuralLSTM` requires ``num_layers`` and ``hidden_size`` in the metadata of both pt2 and ONNX checkpoints. :class:`DriveNeuralGRU` requires ``input_columns``, ``normalization`` and ``sample_dt_s`` in its ONNX checkpoint metadata, and an optional ``custom_inputs`` that marks one column of ``input_columns`` as an array the application supplies each step. .. _effort-modes: Effort Modes ------------ By default an actuator computes effort **explicitly**: the control law is evaluated at the current state and held constant over the step (zero-order hold). At stiff gains and large timesteps this can overshoot or go unstable. The **implicit** effort mode instead solves the control law against the predicted end-of-step state (a Stable-PD style solve). A key advantage of this formulation over the explicit effort mode is that it stays stable at higher gains. That stability comes at a cost: the solve reaches it by applying less effort than the control law nominally asks for. The trade-off is between stability at large timesteps with the implicit mode, and fidelity to the requested gains at small timesteps with the explicit mode. The predicted state accounts only for the actuator's own impulse. Gravity, any other applied force, other actuators driving the same articulation, and joint drive applied without the actuator are all absent from it. The implicit effort mode necessarily requires the joint-space inverse mass matrix. This is supplied by a :class:`~newton.actuators.JointSpaceResponse`, which is refreshed once per step at the current pose: .. code-block:: python from newton.actuators import JointSpaceResponse response = JointSpaceResponse(model) actuator.set_effort_mode_implicit(response=response) # Simulation loop response.refresh(sim_state) sim_control.joint_f.zero_() actuator.step(sim_state, sim_control, state_a, state_b, dt=0.01) solver.step(sim_state, next_sim_state, sim_control, contacts, dt=0.01) In this mode :meth:`Actuator.step ` evaluates the joint force for each actuated DOF. A force on one DOF changes the velocity of every DOF in its articulation, so an articulation's actuated DOFs are solved together as one coupled system. The inverse mass matrix, called *the response* below, is computed for a whole articulation. The actuator then reads only the entries for the DOFs it drives. :class:`~newton.actuators.JointSpaceResponse` is responsible for providing that matrix, and there are two ways to obtain it: compute it from scratch (:meth:`JointSpaceResponse.refresh `), or reuse what the solver already has (:meth:`JointSpaceResponse.refresh_from_solve `). :meth:`~newton.actuators.JointSpaceResponse.refresh` builds the mass matrix itself, from :func:`~newton.eval_mass_matrix` and joint armature. This comes with approximations. First, joint damping, joint limits, friction, contacts and constraint regularization are absent. All of those resist motion, so the response comes out larger than anticipated. A larger response further divides the control law and so yields a smaller effort than would have been evaluated without the simplifications listed above. Second, kinematic loop closures are also ignored. The approximations inherent in :meth:`JointSpaceResponse.refresh ` may be avoided when working with a solver that is able to evaluate the inverse mass matrix more directly, with the exception of loop closure effects. To this end, :meth:`JointSpaceResponse.refresh_from_solve ` takes a callable that computes ``x = M^-1 y``. The response recovers the matrix one column at a time, by passing unit vectors through that callable. MuJoCo is currently the only Newton solver that provides one. .. code-block:: python def solve_inverse(x, y): # x = M^-1 y, using the factorization the solver already built mujoco_warp.solve_m(solver.mjw_model, solver.mjw_data, x, y) # Simulation loop, in place of response.refresh(sim_state) response.refresh_from_solve(solve_inverse, dof_map=solver.mjc_dof_to_newton_dof) Both refresh paths launch only kernels, so the actuator, the solver step and the response update can be captured in one CUDA graph. :meth:`~newton.actuators.Actuator.set_effort_mode_explicit` switches back to explicit mode. :class:`~newton.actuators.Actuator.ImplicitOptions` sets the solve's iteration count and convergence tolerances. The following built-in drives support the implicit mode (neural drives require the ONNX backend): :class:`~newton.actuators.DrivePD`, :class:`~newton.actuators.DrivePID`, :class:`~newton.actuators.DriveNeuralMLP` and :class:`~newton.actuators.DriveNeuralLSTM`. The neural drives enter the solve as a per-step linearization of the network. Its slopes come from a Warp autodiff pass over the loaded network, so only the ONNX backend supports them (see :ref:`neural-network-checkpoints`); with a Torch checkpoint :meth:`~newton.actuators.Actuator.set_effort_mode_implicit` raises ``NotImplementedError``. :class:`~newton.actuators.DriveNeuralMLP` also needs a single-step input history (``input_idx == [0]``). Differentiability and Graph Capture ----------------------------------- Whether an actuator supports differentiability and CUDA graph capture depends on its drive. :class:`DrivePD` and :class:`DrivePID` are fully graphable. For neural-network drives it depends on the checkpoint backend: ONNX checkpoints are graphable, while Torch checkpoints are not due to framework interop overhead. :meth:`Actuator.is_graphable` returns ``True`` when all components can be captured in a CUDA graph. :meth:`Actuator.is_graphable` describes the components, not the captured region. A stateful actuator also needs the region's state exchange to be graph-safe. See :ref:`stateful-actuators` for the two patterns that keep an odd-length region correct. Torch-backed neural drives are not graphable and cannot be used in a captured region. Available Components -------------------- Delay ^^^^^ * :class:`Delay` — circular-buffer delay for control targets (stateful). Drives ^^^^^^ * :class:`DrivePD` — proportional-derivative control law (stateless). * :class:`DrivePID` — proportional-integral-derivative control law (stateful: integral accumulator with anti-windup clamp). * :class:`DriveNeuralGRU` — GRU neural-network drive (stateful: hidden state; explicit Warp-NN inference). * :class:`DriveNeuralMLP` — MLP neural-network drive (stateful: position/velocity history buffers). * :class:`DriveNeuralLSTM` — LSTM neural-network drive (stateful: hidden/cell state). See the API documentation for each drive's control-law equations. Clamping ^^^^^^^^ * :class:`ClampingMaxEffort` — symmetric clamp to ±max_effort per actuator. * :class:`ClampingDCMotor` — velocity-dependent effort saturation using the DC motor effort-speed characteristic. * :class:`ClampingPositionBased` — position-dependent effort limits via interpolated lookup table (e.g. for linkage-driven joints). Multiple clamping objects can be stacked on a single actuator; they are applied in sequence. Customization ------------- Any actuator can be assembled from the existing building blocks — mix and match drives, clamping stages, and delay to fit a specific use case. When the built-in components are not sufficient, implement new ones by subclassing :class:`DriveBase` or :class:`ClampingBase`. For example, a custom drive needs to implement :meth:`~DriveBase.compute`, :meth:`~DriveBase.resolve_arguments`, :meth:`~DriveBase.is_stateful`, and :meth:`~DriveBase.is_graphable`: .. code-block:: python :caption: Skeleton — the ``compute`` body is omitted; see existing drives for complete examples. import warp as wp from newton.actuators import DriveBase class MyDrive(DriveBase): @classmethod def resolve_arguments(cls, args): return {"gain": args.get("gain", 1.0)} def __init__(self, gain: wp.array): self.gain = gain def is_stateful(self): return False def is_graphable(self): return True def compute(self, positions, velocities, target_pos, target_vel, feedforward, pos_indices, vel_indices, target_pos_indices, target_vel_indices, forces, state, dt, device=None, *, custom_inputs=None): # Launch a Warp kernel that writes effort into `forces` ... ``resolve_arguments`` maps user-provided keyword arguments (from :meth:`~newton.ModelBuilder.add_actuator` or USD schemas) to constructor parameters, filling in defaults where needed. A stateful custom drive also defines a dataclass subclass of :class:`DriveBase.State` and implements :meth:`~DriveBase.State.reset`. The default :meth:`Actuator.State.assign` behavior copies direct Warp array and Torch tensor fields without replacing their storage. States with other field types or nested storage implement ``assign()`` to define that copy. A drive that needs additional per-step arrays declares them through :attr:`DriveBase.custom_inputs`. They arrive in the ``custom_inputs`` argument to :meth:`~DriveBase.compute` after being read from ``sim_control``; see :ref:`custom-drive-inputs`. A custom drive works in the explicit mode with the methods above. To also support the implicit mode it provides three more things, because the solve evaluates the control law inside its own kernel rather than calling :meth:`~DriveBase.compute`: * :attr:`~DriveBase.evaluate_force` — a ``@wp.func`` holding the control law. The solve calls it at the predicted state, so it must read every parameter it needs from one packed row rather than from the drive's own arrays. * :meth:`~DriveBase.bind_params` — packs those parameters into an ``(num_actuators, P)`` array and re-points the drive's public arrays at its columns, so later writes stay visible to the solve. * :meth:`~DriveBase.prepare_implicit` — optional, called once per step before the solve. Use it for parameters that depend on the current state, such as a PID integral term or a network linearization. Drives with fixed parameters do not override it. .. code-block:: python :caption: Adding implicit support to ``MyDrive``. @wp.func def _my_force(q: wp.float64, qd: wp.float64, target_q: wp.float64, target_qd: wp.float64, feedforward: wp.float64, params: wp.array2d[float], i: wp.int32) -> wp.float64: return wp.float64(params[i, 0]) * (target_q - q) class MyDrive(DriveBase): evaluate_force = _my_force def bind_params(self): pack = wp.zeros((len(self.gain), 1), dtype=float, device=self.gain.device) pack[:, 0].assign(self.gain) self.gain = pack[:, 0] # writes to self.gain now reach the solve return pack Returning ``None`` from :meth:`~DriveBase.bind_params` declares that this configuration cannot be solved implicitly. :meth:`~newton.actuators.Actuator.set_effort_mode_implicit` then raises ``NotImplementedError`` rather than falling back silently, as it does for a Torch-backed neural checkpoint. Leaving :attr:`~DriveBase.evaluate_force` as ``None`` raises the same error. Similarly, a custom clamping stage subclasses :class:`ClampingBase` and implements :meth:`~ClampingBase.modify_forces` (which reads effort from a source buffer and writes bounded effort to a destination buffer). See Also -------- * :mod:`newton.actuators` — full API reference * :meth:`newton.ModelBuilder.add_actuator` — registering actuators during model construction