{
"cells": [
{
"cell_type": "markdown",
"id": "b32fe0bc",
"metadata": {},
"source": [
"[](https://colab.research.google.com/github/tayalmanan28/MuJoCo-Tutorial/blob/main/tutorial/09_sensors_and_estimation.ipynb)\n",
"\n",
"# Tutorial 9: Sensors & State Estimation\n",
"\n",
"Use MuJoCo's built-in sensors: IMU, force/torque, joint encoders, cameras."
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "76d87119",
"metadata": {},
"outputs": [],
"source": [
"!pip install -q mujoco mediapy matplotlib"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "ca7f7bb6",
"metadata": {},
"outputs": [],
"source": [
"import mujoco\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import mediapy as media"
]
},
{
"cell_type": "markdown",
"id": "fd9e5377",
"metadata": {},
"source": [
"## Sensor Types\n",
"\n",
"MuJoCo supports many sensor types:\n",
"- `jointpos`, `jointvel` — encoder readings\n",
"- `accelerometer`, `gyro` — IMU\n",
"- `force`, `torque` — wrench sensors\n",
"- `framepos`, `framequat` — body pose\n",
"- `touch` — contact normal force"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "0395b16f",
"metadata": {},
"outputs": [],
"source": [
"xml = \"\"\"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"\n",
"\"\"\"\n",
"\n",
"model = mujoco.MjModel.from_xml_string(xml)\n",
"data = mujoco.MjData(model)\n",
"\n",
"# List all sensors\n",
"print(f\"Number of sensors: {model.nsensor}\")\n",
"print(f\"Total sensor dimensions: {model.nsensordata}\")\n",
"print()\n",
"for i in range(model.nsensor):\n",
" name = mujoco.mj_id2name(model, mujoco.mjtObj.mjOBJ_SENSOR, i)\n",
" dim = model.sensor_dim[i]\n",
" adr = model.sensor_adr[i]\n",
" print(f\" {name}: dim={dim}, address={adr}\")"
]
},
{
"cell_type": "markdown",
"id": "f3dedbef",
"metadata": {},
"source": [
"## Reading Sensor Data\n",
"\n",
"All sensor values are in `data.sensordata` (flat array).\n",
"Use `sensor_adr` and `sensor_dim` to index, or use named access:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "529f3ebe",
"metadata": {},
"outputs": [],
"source": [
"# Simulate with a bang-bang controller and record sensors\n",
"mujoco.mj_resetData(model, data)\n",
"data.qpos[0] = 0.5 # start at 0.5 rad\n",
"\n",
"times = []\n",
"encoder_data = []\n",
"accel_data = []\n",
"gyro_data = []\n",
"tip_data = []\n",
"\n",
"while data.time < 5.0:\n",
" # Simple controller\n",
" data.ctrl[0] = -2.0 * np.sign(data.qpos[0])\n",
" mujoco.mj_step(model, data)\n",
" \n",
" times.append(data.time)\n",
" \n",
" # Read sensors by name\n",
" encoder_data.append(data.sensor('encoder').data.copy())\n",
" accel_data.append(data.sensor('accel').data.copy())\n",
" gyro_data.append(data.sensor('gyro').data.copy())\n",
" tip_data.append(data.sensor('tip_pos').data.copy())\n",
"\n",
"encoder_data = np.array(encoder_data)\n",
"accel_data = np.array(accel_data)\n",
"gyro_data = np.array(gyro_data)\n",
"tip_data = np.array(tip_data)"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "41746aa7",
"metadata": {},
"outputs": [],
"source": [
"fig, axes = plt.subplots(4, 1, figsize=(10, 10), sharex=True)\n",
"\n",
"axes[0].plot(times, np.degrees(encoder_data))\n",
"axes[0].set_ylabel('Encoder (deg)'); axes[0].set_title('Joint Position')\n",
"\n",
"axes[1].plot(times, accel_data)\n",
"axes[1].set_ylabel('m/s²'); axes[1].set_title('Accelerometer (x, y, z)')\n",
"axes[1].legend(['x', 'y', 'z'])\n",
"\n",
"axes[2].plot(times, gyro_data)\n",
"axes[2].set_ylabel('rad/s'); axes[2].set_title('Gyroscope (x, y, z)')\n",
"axes[2].legend(['x', 'y', 'z'])\n",
"\n",
"axes[3].plot(times, tip_data)\n",
"axes[3].set_ylabel('m'); axes[3].set_title('Tip Position (world frame)')\n",
"axes[3].legend(['x', 'y', 'z']); axes[3].set_xlabel('Time (s)')\n",
"\n",
"for ax in axes: ax.grid(True)\n",
"plt.tight_layout(); plt.show()"
]
},
{
"cell_type": "markdown",
"id": "e4a54199",
"metadata": {},
"source": [
"## Adding Sensor Noise\n",
"\n",
"MuJoCo supports sensor noise natively via `noise` attribute:"
]
},
{
"cell_type": "code",
"execution_count": null,
"id": "1c7bbc15",
"metadata": {},
"outputs": [],
"source": [
"xml_noisy = \"\"\"\n",
"\n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
" \n",
"\n",
"\"\"\"\n",
"\n",
"m = mujoco.MjModel.from_xml_string(xml_noisy)\n",
"d = mujoco.MjData(m)\n",
"d.qpos[0] = 1.0\n",
"\n",
"clean, noisy = [], []\n",
"for _ in range(1000):\n",
" mujoco.mj_step(m, d)\n",
" clean.append(float(d.sensor('clean').data[0]))\n",
" noisy.append(float(d.sensor('noisy').data[0]))\n",
"\n",
"plt.figure(figsize=(10, 3))\n",
"plt.plot(clean[:200], label='Clean', linewidth=2)\n",
"plt.plot(noisy[:200], label='Noisy (σ=0.05)', alpha=0.7)\n",
"plt.ylabel('Joint angle (rad)'); plt.xlabel('Step')\n",
"plt.legend(); plt.grid(True); plt.title('Sensor Noise'); plt.show()"
]
}
],
"metadata": {
"language_info": {
"name": "python"
}
},
"nbformat": 4,
"nbformat_minor": 5
}