{
"cells": [
{
"cell_type": "markdown",
"id": "1af9f11a",
"metadata": {},
"source": [
"[](https://colab.research.google.com/github/tayalmanan28/MuJoCo-Tutorial/blob/main/tutorial/00_what_is_mujoco.ipynb)\n",
"\n",
"# Tutorial 0: What is MuJoCo?\n",
"\n",
"**No prior physics simulation experience needed.**\n",
"\n",
"This tutorial takes you from zero to understanding what MuJoCo does, why it exists, and how to think about it — all through hands-on examples you can see."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "83a6f70b",
"metadata": {},
"outputs": [],
"source": [
"!pip install -q mujoco mediapy"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "4346ae74",
"metadata": {},
"outputs": [],
"source": [
"import mujoco\n",
"import numpy as np\n",
"import mediapy as media\n",
"print(f\"MuJoCo {mujoco.__version__} ready!\")"
]
},
{
"cell_type": "markdown",
"id": "07834e35",
"metadata": {},
"source": [
"---\n",
"## What Problem Does MuJoCo Solve?\n",
"\n",
"Imagine you're building a robot. Before building the physical robot (expensive, slow, breakable), you want to:\n",
"\n",
"1. **Test** if your robot design can actually walk/grasp/fly\n",
"2. **Train** an AI controller using millions of trial-and-error attempts\n",
"3. **Predict** what happens when the robot interacts with objects\n",
"\n",
"MuJoCo is a **physics simulator** — software that computes how objects move under forces, gravity, and contact. It answers: *\"If I apply this force, what happens next?\"*\n",
"\n",
"### Why MuJoCo specifically?\n",
"\n",
"| Feature | Why it matters |\n",
"|---------|---------------|\n",
"| **Fast** | 10-100× faster than real-time — train robots in minutes, not years |\n",
"| **Accurate contacts** | Handles touching, sliding, bouncing realistically |\n",
"| **Stable** | Doesn't \"explode\" or produce garbage even with complex robots |\n",
"| **GPU-parallel (MJX)** | Simulate 10,000 robots simultaneously on one GPU |\n",
"| **Free & open-source** | Apache 2.0 license, maintained by Google DeepMind |"
]
},
{
"cell_type": "markdown",
"id": "700bcaaf",
"metadata": {},
"source": [
"---\n",
"## The Big Idea: Describe → Simulate → Observe\n",
"\n",
"Using MuJoCo always follows three steps:\n",
"\n",
"```\n",
"1. DESCRIBE → 2. SIMULATE → 3. OBSERVE\n",
" (what exists) (advance time) (read results)\n",
"```\n",
"\n",
"Let's see each step with the simplest possible example: **a ball falling under gravity.**"
]
},
{
"cell_type": "markdown",
"id": "98cb35b3",
"metadata": {},
"source": [
"### Step 1: DESCRIBE — Tell MuJoCo what exists\n",
"\n",
"We write a description in XML (a text format called MJCF). Think of it like a recipe:\n",
"\n",
"- \"There's a flat ground\"\n",
"- \"There's a ball at height 2 meters\"\n",
"- \"Gravity pulls things down at 9.81 m/s²\""
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "3df8bd83",
"metadata": {},
"outputs": [],
"source": [
"# This XML string describes our world\n",
"world_description = \"\"\"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"\n",
"\"\"\"\n",
"\n",
"# MuJoCo \"compiles\" the description into an efficient internal format\n",
"model = mujoco.MjModel.from_xml_string(world_description)\n",
"print(\"✓ World described and compiled!\")\n",
"print(f\" - Bodies: {model.nbody} (includes the 'world' itself)\")\n",
"print(f\" - Geoms (shapes): {model.ngeom}\")\n",
"print(f\" - Gravity: {model.opt.gravity}\")"
]
},
{
"cell_type": "markdown",
"id": "4ec659dd",
"metadata": {},
"source": [
"### Step 2: SIMULATE — Advance time\n",
"\n",
"Now we create the **state** (positions, velocities at a moment in time) and ask MuJoCo to compute what happens next."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "fc05d6e3",
"metadata": {},
"outputs": [],
"source": [
"# Create the initial state (ball at height 2, not moving)\n",
"data = mujoco.MjData(model)\n",
"\n",
"print(f\"At time {data.time:.3f}s: ball height = {data.qpos[2]:.3f}m\")\n",
"\n",
"# Advance simulation by one tiny timestep (0.002 seconds by default)\n",
"mujoco.mj_step(model, data)\n",
"\n",
"print(f\"At time {data.time:.3f}s: ball height = {data.qpos[2]:.6f}m\")\n",
"print(f\" (It moved {2.0 - data.qpos[2]:.6f}m downward — gravity is working!)\")"
]
},
{
"cell_type": "markdown",
"id": "e7f0c40f",
"metadata": {},
"source": [
"### Step 3: OBSERVE — See what happened\n",
"\n",
"Let's simulate for longer and watch the ball fall and bounce:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dfee1a5e",
"metadata": {},
"outputs": [],
"source": [
"# Reset everything to the start\n",
"data = mujoco.MjData(model)\n",
"\n",
"# Create a renderer (a virtual camera)\n",
"renderer = mujoco.Renderer(model, height=400, width=500)\n",
"\n",
"# Simulate for 3 seconds, capturing frames for a video\n",
"frames = []\n",
"while data.time < 3.0:\n",
" mujoco.mj_step(model, data) # Physics step\n",
" \n",
" # Capture a frame every 1/30th of a second (30 fps video)\n",
" if len(frames) < data.time * 30:\n",
" renderer.update_scene(data)\n",
" frames.append(renderer.render().copy())\n",
"\n",
"renderer.close()\n",
"media.show_video(frames, fps=30)"
]
},
{
"cell_type": "markdown",
"id": "f9d24d3e",
"metadata": {},
"source": [
"**That's it!** You just:\n",
"1. Described a world (XML) → compiled to `model`\n",
"2. Simulated it (`mj_step` in a loop) → updated `data`\n",
"3. Observed the result (rendered frames)\n",
"\n",
"Everything else in MuJoCo builds on this pattern."
]
},
{
"cell_type": "markdown",
"id": "bf54ab68",
"metadata": {},
"source": [
"---\n",
"## The Two Core Objects\n",
"\n",
"MuJoCo has just **two** main data structures. Understanding them is the key to everything:\n",
"\n",
"### `MjModel` — The Blueprint (doesn't change)\n",
"\n",
"Contains everything that stays **constant** during simulation:\n",
"- How many objects exist, their shapes and sizes\n",
"- Mass, friction, stiffness\n",
"- Joint types and limits\n",
"- Gravity, timestep settings\n",
"\n",
"**Analogy:** The architectural plan of a building — it tells you what's there, but not where the people are right now.\n",
"\n",
"### `MjData` — The Snapshot (changes every step)\n",
"\n",
"Contains everything that **changes** over time:\n",
"- Current positions (`data.qpos`) and velocities (`data.qvel`)\n",
"- Current time (`data.time`)\n",
"- Forces, contacts, sensor readings\n",
"- Control signals (`data.ctrl`)\n",
"\n",
"**Analogy:** A security camera snapshot — tells you where everything is *right now*."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "c5e7d2c3",
"metadata": {},
"outputs": [],
"source": [
"# model = the blueprint (constant)\n",
"print(\"=== MODEL (blueprint) ===\")\n",
"print(f\" Timestep: {model.opt.timestep}s (fixed)\")\n",
"print(f\" Ball radius: {model.geom_size[1, 0]}m (fixed)\")\n",
"print(f\" Gravity: {model.opt.gravity} (fixed)\")\n",
"\n",
"print()\n",
"\n",
"# data = the current state (changes)\n",
"print(\"=== DATA (current snapshot) ===\")\n",
"print(f\" Time: {data.time:.3f}s\")\n",
"print(f\" Ball position: {data.qpos[:3]}\")\n",
"print(f\" Ball velocity: {data.qvel[:3]}\")"
]
},
{
"cell_type": "markdown",
"id": "c17ce77d",
"metadata": {},
"source": [
"---\n",
"## Building Blocks: Bodies, Joints, Geoms\n",
"\n",
"Every MuJoCo world is made of three building blocks:\n",
"\n",
"| Concept | What it is | Real-world analogy |\n",
"|---------|-----------|-------------------|\n",
"| **Body** | A rigid frame that can move | A bone in your skeleton |\n",
"| **Joint** | Connects bodies, allows specific movement | A knee or elbow |\n",
"| **Geom** | A shape attached to a body (for collision & visuals) | The flesh/skin on a bone |\n",
"\n",
"Let's build something step by step and see how they work together:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "a9fa0fc6",
"metadata": {},
"outputs": [],
"source": [
"# A pendulum: one body, connected to the world by a hinge joint\n",
"pendulum_xml = \"\"\"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"\n",
"\"\"\"\n",
"\n",
"m = mujoco.MjModel.from_xml_string(pendulum_xml)\n",
"d = mujoco.MjData(m)\n",
"\n",
"# Start the pendulum at 90 degrees\n",
"d.qpos[0] = np.pi / 2 # 90° in radians\n",
"\n",
"# Simulate\n",
"renderer = mujoco.Renderer(m, height=400, width=400)\n",
"frames = []\n",
"while d.time < 5.0:\n",
" mujoco.mj_step(m, d)\n",
" if len(frames) < d.time * 30:\n",
" renderer.update_scene(d)\n",
" frames.append(renderer.render().copy())\n",
"renderer.close()\n",
"\n",
"media.show_video(frames, fps=30)"
]
},
{
"cell_type": "markdown",
"id": "b32d8e65",
"metadata": {},
"source": [
"**What happened:**\n",
"- We created one **body** (the arm)\n",
"- We gave it a **hinge joint** — this allows rotation around one axis (like a door hinge)\n",
"- We gave it a **geom** (capsule shape) — so it has mass and is visible\n",
"- Gravity did the rest!\n",
"\n",
"### Joint Types\n",
"\n",
"| Joint type | Degrees of freedom | Example |\n",
"|-----------|-------------------|--------|\n",
"| `free` | 6 (move + rotate anywhere) | Drone, thrown ball |\n",
"| `hinge` | 1 (rotate around axis) | Door, elbow, wheel |\n",
"| `slide` | 1 (translate along axis) | Drawer, piston |\n",
"| `ball` | 3 (rotate in any direction) | Shoulder, hip |"
]
},
{
"cell_type": "markdown",
"id": "ad6b11d1",
"metadata": {},
"source": [
"---\n",
"## Adding Control: Making Things Move on Purpose\n",
"\n",
"So far, only gravity moves things. To **control** a robot, we need **actuators** — motors that apply forces to joints."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "cfd735a3",
"metadata": {},
"outputs": [],
"source": [
"controlled_xml = \"\"\"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"\n",
"\"\"\"\n",
"\n",
"m = mujoco.MjModel.from_xml_string(controlled_xml)\n",
"d = mujoco.MjData(m)\n",
"\n",
"# Our goal: hold the pendulum at 45 degrees (π/4 radians)\n",
"target = np.pi / 4\n",
"\n",
"renderer = mujoco.Renderer(m, height=400, width=400)\n",
"frames = []\n",
"\n",
"while d.time < 6.0:\n",
" # === THE CONTROLLER ===\n",
" # Read current angle and speed\n",
" current_angle = d.qpos[0]\n",
" current_speed = d.qvel[0]\n",
" \n",
" # Compute the motor command (PD controller):\n",
" # - Push toward target (proportional)\n",
" # - Slow down when moving fast (damping)\n",
" error = target - current_angle\n",
" motor_command = 10.0 * error - 3.0 * current_speed\n",
" \n",
" # Send command to the motor (clipped to ±5 Nm)\n",
" d.ctrl[0] = np.clip(motor_command, -5, 5)\n",
" # === END CONTROLLER ===\n",
" \n",
" mujoco.mj_step(m, d)\n",
" \n",
" if len(frames) < d.time * 30:\n",
" renderer.update_scene(d)\n",
" frames.append(renderer.render().copy())\n",
"\n",
"renderer.close()\n",
"media.show_video(frames, fps=30)\n",
"print(f\"Final angle: {np.degrees(d.qpos[0]):.1f}° (target was {np.degrees(target):.1f}°)\")"
]
},
{
"cell_type": "markdown",
"id": "98ca45ff",
"metadata": {},
"source": [
"**The control loop pattern** — this is how ALL robot control works in MuJoCo:\n",
"\n",
"```python\n",
"while simulating:\n",
" # 1. READ: Get current state from data\n",
" angle = data.qpos[0]\n",
" \n",
" # 2. DECIDE: Compute what force to apply\n",
" command = my_controller(angle, target)\n",
" \n",
" # 3. ACT: Set the control signal\n",
" data.ctrl[0] = command\n",
" \n",
" # 4. STEP: Let physics happen\n",
" mujoco.mj_step(model, data)\n",
"```"
]
},
{
"cell_type": "markdown",
"id": "4c863653",
"metadata": {},
"source": [
"---\n",
"## Understanding `qpos` and `qvel` (Generalized Coordinates)\n",
"\n",
"You'll see `data.qpos` and `data.qvel` everywhere. Here's the intuition:\n",
"\n",
"- **`qpos`** = all positions/angles that describe the configuration\n",
"- **`qvel`** = all velocities (how fast each position is changing)\n",
"\n",
"For a **hinge joint**: `qpos` has 1 number (the angle in radians) \n",
"For a **free joint**: `qpos` has 7 numbers (x, y, z + quaternion rotation)\n",
"\n",
"| Joint | qpos entries | qvel entries |\n",
"|-------|-------------|-------------|\n",
"| hinge | 1 (angle) | 1 (angular velocity) |\n",
"| slide | 1 (position) | 1 (linear velocity) |\n",
"| free | 7 (pos + quaternion) | 6 (linear vel + angular vel) |\n",
"| ball | 4 (quaternion) | 3 (angular velocity) |\n",
"\n",
"⚠️ **Note:** For `free` and `ball` joints, `qpos` and `qvel` have *different* sizes! This is because rotations need 4 numbers (quaternion) to represent, but only 3 numbers to describe the rate of change."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "dd4618ce",
"metadata": {},
"outputs": [],
"source": [
"# Example: a model with mixed joint types\n",
"mixed_xml = \"\"\"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"\n",
"\"\"\"\n",
"m = mujoco.MjModel.from_xml_string(mixed_xml)\n",
"d = mujoco.MjData(m)\n",
"\n",
"print(f\"Total qpos size: {m.nq} (7 from free + 1 from hinge = 8)\")\n",
"print(f\"Total qvel size: {m.nv} (6 from free + 1 from hinge = 7)\")\n",
"print(f\"\")\n",
"print(f\"qpos = {d.qpos}\") # [x,y,z, qw,qx,qy,qz, hinge_angle]\n",
"print(f\"qvel = {d.qvel}\") # [vx,vy,vz, wx,wy,wz, hinge_velocity]\")"
]
},
{
"cell_type": "markdown",
"id": "c3734a27",
"metadata": {},
"source": [
"---\n",
"## Putting It All Together: A Mini Robot\n",
"\n",
"Let's build a 2-link arm from scratch — combining everything we've learned:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "02cab5d5",
"metadata": {},
"outputs": [],
"source": [
"robot_xml = \"\"\"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"\n",
"\"\"\"\n",
"\n",
"m = mujoco.MjModel.from_xml_string(robot_xml)\n",
"d = mujoco.MjData(m)\n",
"\n",
"print(f\"Our robot has:\")\n",
"print(f\" {m.njnt} joints (shoulder + elbow)\")\n",
"print(f\" {m.nu} actuators (one motor per joint)\")\n",
"print(f\" {m.nq} degrees of freedom\")\n",
"\n",
"# Make the arm trace a circle!\n",
"renderer = mujoco.Renderer(m, height=400, width=500)\n",
"frames = []\n",
"\n",
"while d.time < 8.0:\n",
" # Move joints in a sinusoidal pattern\n",
" t = d.time\n",
" target_shoulder = 0.5 * np.sin(t * 1.5)\n",
" target_elbow = 0.8 * np.sin(t * 2.0 + 1.0)\n",
" \n",
" # PD control for both joints\n",
" d.ctrl[0] = 20 * (target_shoulder - d.qpos[0]) - 5 * d.qvel[0]\n",
" d.ctrl[1] = 10 * (target_elbow - d.qpos[1]) - 3 * d.qvel[1]\n",
" \n",
" mujoco.mj_step(m, d)\n",
" if len(frames) < d.time * 30:\n",
" renderer.update_scene(d)\n",
" frames.append(renderer.render().copy())\n",
"\n",
"renderer.close()\n",
"media.show_video(frames, fps=30)"
]
},
{
"cell_type": "markdown",
"id": "f3e3b453",
"metadata": {},
"source": [
"---\n",
"## Summary: The MuJoCo Mental Model\n",
"\n",
"```\n",
"┌─────────────────────────────────────────────────────────┐\n",
"│ YOUR CODE │\n",
"│ │\n",
"│ 1. Write XML (once) ──► model = MjModel(xml) │\n",
"│ │ │\n",
"│ 2. Create state ──► data = MjData(model) │\n",
"│ │ │\n",
"│ 3. Loop: ▼ │\n",
"│ ┌──────────────────────────────────────┐ │\n",
"│ │ Read: angle = data.qpos[0] │ │\n",
"│ │ Decide: command = controller(angle) │ │\n",
"│ │ Act: data.ctrl[0] = command │ │\n",
"│ │ Step: mj_step(model, data) │ ◄── repeat│\n",
"│ │ (Optional) Render: renderer.render() │ │\n",
"│ └──────────────────────────────────────┘ │\n",
"└─────────────────────────────────────────────────────────┘\n",
"```\n",
"\n",
"### Key vocabulary\n",
"\n",
"| Term | Meaning |\n",
"|------|--------|\n",
"| **MJCF** | The XML format for describing models |\n",
"| **MjModel** | Compiled model (constant blueprint) |\n",
"| **MjData** | Simulation state (changes each step) |\n",
"| **qpos** | Generalized positions (angles/positions of all joints) |\n",
"| **qvel** | Generalized velocities |\n",
"| **ctrl** | Control signals sent to actuators |\n",
"| **mj_step** | Advance physics by one timestep |\n",
"| **mj_forward** | Recompute derived quantities without advancing time |"
]
},
{
"cell_type": "markdown",
"id": "61c06f5f",
"metadata": {},
"source": [
"---\n",
"## What's Next?\n",
"\n",
"Now that you understand the fundamentals, continue with:\n",
"\n",
"1. **[Tutorial 01: Basics](01_basics.ipynb)** — Named access, model inspection\n",
"2. **[Tutorial 02: Rendering](02_rendering.ipynb)** — Cameras, depth, segmentation\n",
"3. **[Tutorial 03: Simulation](03_simulation.ipynb)** — Timesteps, energy, integrators\n",
"4. **[Tutorial 04: Control](04_control.ipynb)** — PD control, servos, gain tuning\n",
"\n",
"Or jump to specific topics:\n",
"- Building robots from XML → [Tutorial 06](06_xml_modeling.ipynb)\n",
"- Reinforcement Learning → [Tutorial 08](08_gymnasium.ipynb)\n",
"- GPU-parallel simulation → [Tutorial 07](07_mjx.ipynb)"
]
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 5
}