{ "cells": [ { "cell_type": "markdown", "id": "05cba041", "metadata": {}, "source": [ "[![Open In Colab](https://colab.research.google.com/assets/colab-badge.svg)](https://colab.research.google.com/github/tayalmanan28/MuJoCo-Tutorial/blob/main/tutorial/04_control.ipynb)\n", "\n", "# Tutorial 4: Actuators & Control\n", "\n", "Learn how to add actuators, implement controllers, and use `data.ctrl`." ] }, { "cell_type": "code", "execution_count": null, "id": "7f50ab40", "metadata": {}, "outputs": [], "source": [ "!pip install -q mujoco mediapy matplotlib" ] }, { "cell_type": "code", "execution_count": null, "id": "89446a5a", "metadata": {}, "outputs": [], "source": [ "import mujoco\n", "import numpy as np\n", "import matplotlib.pyplot as plt\n", "import mediapy as media" ] }, { "cell_type": "markdown", "id": "7c070381", "metadata": {}, "source": [ "## Actuators in MJCF\n", "\n", "Actuators apply forces/torques to joints. Common types:\n", "- `` \u2014 direct torque\n", "- `` \u2014 position servo\n", "- `` \u2014 velocity servo" ] }, { "cell_type": "code", "execution_count": null, "id": "d2b02524", "metadata": {}, "outputs": [], "source": [ "xml = \"\"\"\n", "\n", " \n", "\"\"\"\n", "\n", "model = mujoco.MjModel.from_xml_string(xml)\n", "data = mujoco.MjData(model)\n", "print(f\"Actuators: {model.nu}, Joints: {model.njnt}\")" ] }, { "cell_type": "markdown", "id": "7c60f952", "metadata": {}, "source": [ "## PD Controller\n", "\n", "$$\\tau = k_p (q_{target} - q) - k_d \\dot{q}$$" ] }, { "cell_type": "code", "execution_count": null, "id": "6fbf5351", "metadata": {}, "outputs": [], "source": [ "target_angle = np.pi / 3 # 60 degrees\n", "kp, kd = 10.0, 2.0\n", "\n", "times, angles, controls = [], [], []\n", "mujoco.mj_resetData(model, data)\n", "\n", "while data.time < 5.0:\n", " # PD control\n", " angle = data.qpos[0]\n", " velocity = data.qvel[0]\n", " data.ctrl[0] = np.clip(kp * (target_angle - angle) - kd * velocity, -5, 5)\n", " \n", " mujoco.mj_step(model, data)\n", " times.append(data.time)\n", " angles.append(angle)\n", " controls.append(data.ctrl[0])\n", "\n", "fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(10, 6), sharex=True)\n", "ax1.plot(times, np.degrees(angles), label='Angle')\n", "ax1.axhline(np.degrees(target_angle), color='r', linestyle='--', label='Target')\n", "ax1.set_ylabel('Angle (deg)'); ax1.legend(); ax1.grid(True)\n", "ax2.plot(times, controls, color='orange')\n", "ax2.set_xlabel('Time (s)'); ax2.set_ylabel('Torque (Nm)'); ax2.grid(True)\n", "plt.suptitle('PD Control of Pendulum'); plt.tight_layout(); plt.show()" ] }, { "cell_type": "markdown", "id": "30daa598", "metadata": {}, "source": [ "## Tuning PD Gains\n", "\n", "Compare different gain settings:" ] }, { "cell_type": "code", "execution_count": null, "id": "476047eb", "metadata": {}, "outputs": [], "source": [ "plt.figure(figsize=(10, 4))\n", "\n", "for kp, kd, label in [(5, 1, 'Underdamped'), (10, 6, 'Critically damped'), (10, 15, 'Overdamped')]:\n", " mujoco.mj_resetData(model, data)\n", " times, angles = [], []\n", " while data.time < 5.0:\n", " data.ctrl[0] = np.clip(kp * (target_angle - data.qpos[0]) - kd * data.qvel[0], -5, 5)\n", " mujoco.mj_step(model, data)\n", " times.append(data.time)\n", " angles.append(data.qpos[0])\n", " plt.plot(times, np.degrees(angles), label=label)\n", "\n", "plt.axhline(np.degrees(target_angle), color='k', linestyle='--', alpha=0.5)\n", "plt.xlabel('Time (s)'); plt.ylabel('Angle (deg)')\n", "plt.title('Effect of PD Gains'); plt.legend(); plt.grid(True); plt.show()" ] }, { "cell_type": "markdown", "id": "69bc1778", "metadata": {}, "source": [ "## Position & Velocity Servos\n", "\n", "MuJoCo has built-in servo actuators that implement PD internally:" ] }, { "cell_type": "code", "execution_count": null, "id": "43033324", "metadata": {}, "outputs": [], "source": [ "xml_servo = \"\"\"\n", "\n", " \n", "\"\"\"\n", "\n", "model_s = mujoco.MjModel.from_xml_string(xml_servo)\n", "data_s = mujoco.MjData(model_s)\n", "\n", "# Just set the target \u2014 the servo does the rest\n", "times, angles = [], []\n", "while data_s.time < 5.0:\n", " # Step target over time\n", " data_s.ctrl[0] = np.pi/4 if data_s.time < 2.5 else -np.pi/4\n", " mujoco.mj_step(model_s, data_s)\n", " times.append(data_s.time)\n", " angles.append(data_s.qpos[0])\n", "\n", "plt.figure(figsize=(10, 4))\n", "plt.plot(times, np.degrees(angles))\n", "plt.xlabel('Time (s)'); plt.ylabel('Angle (deg)')\n", "plt.title('Position Servo (built-in PD)'); plt.grid(True); plt.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Velocity Servo\n", "\n", "A velocity servo applies force proportional to velocity error:\n", "$$F = k_v (v_{target} - v)$$\n", "\n", "Set `ctrl` = desired velocity. Useful for wheels, conveyors, continuous rotation." ] }, { "cell_type": "code", "metadata": {}, "source": [ "xml_vel = \"\"\"\n", "\n", " \n", "\"\"\"\n", "\n", "model_v = mujoco.MjModel.from_xml_string(xml_vel)\n", "data_v = mujoco.MjData(model_v)\n", "\n", "# Command constant velocity\n", "times, vels = [], []\n", "while data_v.time < 5.0:\n", " data_v.ctrl[0] = 2.0 # desired angular velocity (rad/s)\n", " mujoco.mj_step(model_v, data_v)\n", " times.append(data_v.time)\n", " vels.append(data_v.qvel[0])\n", "\n", "plt.figure(figsize=(10, 3))\n", "plt.plot(times, vels)\n", "plt.axhline(2.0, color='r', linestyle='--', label='Target velocity')\n", "plt.xlabel('Time (s)'); plt.ylabel('Velocity (rad/s)')\n", "plt.title('Velocity Servo'); plt.legend(); plt.grid(True); plt.show()" ], "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Actuator Types Comparison\n", "\n", "| Type | `ctrl` meaning | Internal formula | Use case |\n", "|------|---------------|-----------------|----------|\n", "| `` | Direct force/torque | $F = \\text{gear} \\cdot \\text{ctrl}$ | Full control authority (RL, custom controllers) |\n", "| `` | Target position | $F = k_p (\\text{ctrl} - q)$ | Servos, joint-space tracking |\n", "| `` | Target velocity | $F = k_v (\\text{ctrl} - \\dot{q})$ | Wheels, conveyors |\n", "| `` | Piston pressure | Pneumatic dynamics | Hydraulic/pneumatic actuators |\n", "| `` | Activation (0-1) | Force-length-velocity curves | Biomechanical models |\n", "| `` | Configurable | Customizable transmission | Advanced custom actuators |\n", "\n", "All actuators can have `ctrlrange`, `forcerange`, `gear`, and `dyntype` attributes." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## General Actuator (Custom Transmission)\n", "\n", "The `` actuator lets you configure gain, bias, and dynamics independently.\n", "This is the most flexible actuator type \u2014 all others are special cases of it." ] }, { "cell_type": "code", "metadata": {}, "source": [ "xml_general = \"\"\"\n", "\n", " \n", "\"\"\"\n", "\n", "model_g = mujoco.MjModel.from_xml_string(xml_general)\n", "data_g = mujoco.MjData(model_g)\n", "\n", "# This behaves like position servo: ctrl=target, internal PD\n", "times, angles = [], []\n", "while data_g.time < 5.0:\n", " data_g.ctrl[0] = np.pi / 3\n", " mujoco.mj_step(model_g, data_g)\n", " times.append(data_g.time)\n", " angles.append(data_g.qpos[0])\n", "\n", "plt.figure(figsize=(10, 3))\n", "plt.plot(times, np.degrees(angles))\n", "plt.axhline(60, color='r', linestyle='--', label='Target')\n", "plt.xlabel('Time (s)'); plt.ylabel('Angle (deg)')\n", "plt.title('General Actuator (custom PD)'); plt.legend(); plt.grid(True); plt.show()" ], "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Muscle Actuator\n", "\n", "MuJoCo has a biomechanical muscle model with:\n", "- **Activation dynamics** (first-order filter)\n", "- **Force-length curve** (bell-shaped, peak at optimal length)\n", "- **Force-velocity curve** (concentric weaker than eccentric)\n", "\n", "Control input is excitation \u2208 [0, 1]. The muscle filters this into activation." ] }, { "cell_type": "code", "metadata": {}, "source": [ "xml_muscle = \"\"\"\n", "\n", " \n", "\"\"\"\n", "\n", "model_m = mujoco.MjModel.from_xml_string(xml_muscle)\n", "data_m = mujoco.MjData(model_m)\n", "\n", "# Pulse excitation\n", "times, angles, activations = [], [], []\n", "while data_m.time < 3.0:\n", " # Full excitation for 1s, then release\n", " data_m.ctrl[0] = 1.0 if data_m.time < 1.0 else 0.0\n", " mujoco.mj_step(model_m, data_m)\n", " times.append(data_m.time)\n", " angles.append(np.degrees(data_m.qpos[0]))\n", " activations.append(data_m.act[0]) # internal activation state\n", "\n", "fig, (ax1, ax2) = plt.subplots(2, 1, figsize=(10, 5), sharex=True)\n", "ax1.plot(times, angles)\n", "ax1.set_ylabel('Angle (deg)'); ax1.set_title('Muscle Actuator'); ax1.grid(True)\n", "ax2.plot(times, activations, color='red', label='Activation')\n", "ax2.axvspan(0, 1, alpha=0.1, color='green', label='Excitation ON')\n", "ax2.set_xlabel('Time (s)'); ax2.set_ylabel('Activation'); ax2.legend(); ax2.grid(True)\n", "plt.tight_layout(); plt.show()\n", "print('Note: Activation lags behind excitation (first-order dynamics)')" ], "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Keyframes\n", "\n", "Keyframes store named configurations (qpos/qvel/ctrl) you can reset to.\n", "Useful for setting initial poses, test configurations, or waypoints." ] }, { "cell_type": "code", "metadata": {}, "source": [ "xml_keyframe = \"\"\"\n", "\n", " \n", "\"\"\"\n", "\n", "model_k = mujoco.MjModel.from_xml_string(xml_keyframe)\n", "data_k = mujoco.MjData(model_k)\n", "\n", "# Reset to each keyframe and render\n", "renderer = mujoco.Renderer(model_k, height=200, width=300)\n", "images = []\n", "for i in range(model_k.nkey):\n", " key_name = mujoco.mj_id2name(model_k, mujoco.mjtObj.mjOBJ_KEY, i)\n", " # Load keyframe\n", " mujoco.mj_resetDataKeyframe(model_k, data_k, i)\n", " mujoco.mj_forward(model_k, data_k)\n", " renderer.update_scene(data_k)\n", " images.append(renderer.render().copy())\n", " print(f'Keyframe \"{key_name}\": qpos = {data_k.qpos}')\n", "\n", "renderer.close()\n", "media.show_images(images, titles=['home', 'reach_up', 'reach_forward'])" ], "outputs": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Summary\n", "\n", "| Concept | Key Takeaway |\n", "|---------|-------------|\n", "| `` | Direct torque \u2014 you provide the full control signal |\n", "| `` | Built-in PD servo \u2014 just set target angle |\n", "| `` | Velocity tracking \u2014 set target speed |\n", "| `` | Fully configurable gain/bias/dynamics |\n", "| `` | Biomechanical with activation dynamics |\n", "| `data.ctrl` | Array of control inputs (one per actuator) |\n", "| `ctrlrange` | Clips control input; `forcerange` clips output force |\n", "| Keyframes | Named poses for reset (`mj_resetDataKeyframe`) |\n", "\n", "**Next:** [Tutorial 05 \u2014 Contact & Collision](05_contact.ipynb)" ] } ], "metadata": { "language_info": { "name": "python" } }, "nbformat": 4, "nbformat_minor": 5 }