{ "cells": [ { "cell_type": "markdown", "id": "cell-title", "metadata": {}, "source": [ "# Simplicits with Rigid Body Simulation\n", "\n", "This notebook demonstrates coupling Simplicits soft-body simulation with Newton rigid body\n", "dynamics. Two deformable cubes fall and interact with a rigid body cube driven by\n", "`SolverSemiImplicit`, all sharing the same Newton model and collision pipeline." ] }, { "cell_type": "markdown", "id": "cell-imports-md", "metadata": {}, "source": [ "## Imports\n", "\n", "We import Kaolin, Newton, Warp, and **k3d** for interactive 3D visualization in the\n", "notebook. `transform_points` from `newton._src.geometry.utils` is used to transform\n", "rigid body mesh vertices by the body's pose each frame." ] }, { "cell_type": "code", "execution_count": null, "id": "546f6f75-24a3-4447-844b-0744f2935834", "metadata": {}, "outputs": [], "source": [ "!pip install -q newton[examples]==1.2.0" ] }, { "cell_type": "code", "execution_count": null, "id": "cell-imports", "metadata": {}, "outputs": [], "source": [ "import os\n", "import sys\n", "import threading\n", "from pathlib import Path\n", "\n", "import numpy as np\n", "import torch\n", "import warp as wp\n", "from IPython.display import display\n", "from ipywidgets import Button, VBox\n", "from newton._src.geometry.utils import transform_points\n", "\n", "import kaolin\n", "from kaolin.physics.simplicits import SimplicitsObject\n", "\n", "import newton\n", "from kaolin.experimental.newton.builder import SimplicitsModelBuilder\n", "from kaolin.experimental.newton.solver import SimplicitsSolver\n", "from newton.solvers import SolverSemiImplicit\n", "from newton._src.sim.collide import CollisionPipeline\n", "\n", "sys.path.append(str(Path(\"..\")))\n", "from tutorial_common import COMMON_DATA_DIR\n", "\n", "wp.clear_kernel_cache()\n", "\n", "try:\n", " import k3d\n", "except ModuleNotFoundError:\n", " !pip install -q k3d\n", " import k3d\n", " raise RuntimeError(\"k3d has been installed. Do a hard refresh of the page (Ctrl + Shift + R) to avoid issue with k3d\")" ] }, { "cell_type": "markdown", "id": "cell-config-md", "metadata": {}, "source": [ "## Configuration\n", "\n", "Key constants:\n", "- **`MESH_SCALE`** — scales the loaded cube mesh before adding it as a Newton collision shape.\n", "- **`SOFT_YOUNGS_MODULUS` / `POISSON_RATIO` / `DENSITY`** — elastic material parameters for the\n", " Simplicits soft-body cubes.\n", "- **`DT`** — simulation timestep (seconds per frame).\n", "- **`FLOOR_PLANE`** — y-coordinate of the ground plane." ] }, { "cell_type": "code", "execution_count": null, "id": "cell-config", "metadata": {}, "outputs": [], "source": [ "# Mesh parameters\n", "MESH_SCALE = 1.0\n", "\n", "# Physics material parameters\n", "SOFT_YOUNGS_MODULUS = 1e5\n", "POISSON_RATIO = 0.45\n", "DENSITY = 500 # kg/m^3\n", "APPROX_VOLUME = 0.5 # m^3\n", "NUM_SAMPLES = 10000\n", "\n", "# Simulation parameters\n", "DT = 0.03\n", "FLOOR_PLANE = -1.0\n", "GRAVITY_ACC = 9.8\n", "\n", "# Newton solver parameters\n", "PENALTY_WEIGHT = 10000.0\n", "MAX_NEWTON_STEPS = 10\n", "MAX_LS_STEPS = 30\n", "CONV_TOL = 1e-8\n", "ELASTICITY_TYPE = \"neohookean_elasticity\"\n", "NEWTON_HESSIAN_REGULARIZER = 1e-2\n", "GRAVITY_ON = True" ] }, { "cell_type": "markdown", "id": "cell-mesh-md", "metadata": {}, "source": [ "## Mesh Loading and Simplicits Object\n", "\n", "We load a cube mesh and sample interior points for Simplicits. Material properties\n", "(Young's modulus, Poisson ratio, density) are assigned per sample point.\n", "\n", "`SimplicitsObject.create_rigid` creates a reduced-DOF elastic object with a single handle (for fast construction) in this example." ] }, { "cell_type": "code", "execution_count": null, "id": "cell-mesh", "metadata": {}, "outputs": [], "source": [ "def load_mesh():\n", " \"\"\"Load and prepare the mesh for simulation.\"\"\"\n", " mesh_path = os.path.join(COMMON_DATA_DIR, \"meshes\")\n", " mesh = kaolin.io.import_mesh(\n", " mesh_path + \"/cube.obj\", triangulate=True).to('cuda')\n", " mesh.vertices = kaolin.ops.pointcloud.center_points(\n", " mesh.vertices.unsqueeze(0), normalize=True).squeeze(0)\n", " return mesh\n", "\n", "\n", "def create_simplicits_object(mesh):\n", " \"\"\"Create a Simplicits object from mesh.\"\"\"\n", " orig_vertices = mesh.vertices.clone()\n", "\n", " # Sample points uniformly over the bounding box\n", " uniform_pts = torch.rand(NUM_SAMPLES, 3, device='cuda') * (\n", " orig_vertices.max(dim=0).values - orig_vertices.min(dim=0).values\n", " ) + orig_vertices.min(dim=0).values\n", "\n", " physics_points = kaolin.physics.simplicits.PhysicsPoints(\n", " pts=uniform_pts,\n", " yms=SOFT_YOUNGS_MODULUS,\n", " prs=POISSON_RATIO,\n", " rhos=DENSITY,\n", " appx_vol=APPROX_VOLUME\n", " ).cuda()\n", "\n", " # Create single handle Simplicits object\n", " sim_obj = SimplicitsObject.create_rigid(\n", " physics_points=physics_points\n", " ) # create_rigid creates a 1 handle object that affinely deforms. Its cheap to construct and at higher stiffnesses, this acts as a rigid object.\n", " return sim_obj\n", "\n", "\n", "mesh = load_mesh()\n", "sim_obj = create_simplicits_object(mesh)\n", "print(f\"Mesh: {mesh.vertices.shape[0]} vertices | Sim samples: {sim_obj.pts.shape[0]}\")" ] }, { "cell_type": "markdown", "id": "cell-builder-md", "metadata": {}, "source": [ "## Building the Newton Model\n", "\n", "`SimplicitsModelBuilder` extends Newton's `ModelBuilder`. We:\n", "1. Add two Simplicits objects (two soft cubes, the second offset upward by 3 m).\n", "2. Add a ground plane.\n", "3. Configure inter-object collision handling.\n", "4. Add one rigid cube body (driven by `SolverSemiImplicit`).\n", "\n", "`finalize()` compiles all geometry and physics parameters into GPU arrays." ] }, { "cell_type": "code", "execution_count": null, "id": "cell-builder", "metadata": {}, "outputs": [], "source": [ "def add_cube_mesh(builder, mesh, position):\n", " \"\"\"Add a rigid cube mesh body to the builder.\"\"\"\n", " newton_mesh = newton.Mesh(\n", " mesh.vertices.detach().cpu().numpy() * MESH_SCALE,\n", " mesh.faces.detach().cpu().numpy()\n", " )\n", " body = builder.add_body(xform=wp.transform(\n", " wp.vec3(position[0], position[1], position[2]), wp.quat_identity()\n", " ))\n", " builder.add_joint_free(body)\n", " builder.add_shape_mesh(body=body, mesh=newton_mesh)\n", "\n", "\n", "def create_builder_model(sim_obj, mesh):\n", " \"\"\"Create and configure the Simplicits model builder.\"\"\"\n", " print(\"Building model...\")\n", " builder = SimplicitsModelBuilder(up_axis=\"y\", gravity=-GRAVITY_ACC)\n", "\n", " # Add two Simplicits soft-body cubes; capture their object IDs\n", " builder.add_simplicits_object(sim_obj, num_qp=1000, renderable_pts=mesh.vertices.clone().detach())\n", " builder.add_simplicits_object(\n", " sim_obj,\n", " num_qp=1000,\n", " init_transform=torch.tensor([\n", " [1, 0, 0, 0],\n", " [0, 1, 0, 3.0],\n", " [0, 0, 1, 0],\n", " [0, 0, 0, 1]\n", " ], dtype=torch.float32, device='cuda'),\n", " renderable_pts=mesh.vertices.clone().detach()\n", " )\n", "\n", " # Add ground plane\n", " builder.add_shape_plane(\n", " plane=(*builder.up_vector, -FLOOR_PLANE),\n", " width=0.0,\n", " length=0.0,\n", " cfg=SimplicitsModelBuilder.ShapeConfig(\n", " ke=1e4, mu=0.5, kd=100.0, kf=1.0\n", " ),\n", " label=\"ground_plane\",\n", " )\n", "\n", " # Add collision handling between soft-body particles and shapes\n", " builder.add_simplicits_collisions(\n", " collision_particle_radius=0.1,\n", " detection_ratio=1.5,\n", " impenetrable_barrier_ratio=0.25,\n", " collision_penalty=1000.0,\n", " max_contact_pairs=10000,\n", " friction=0.5\n", " )\n", "\n", " # Add rigid body cube\n", " add_cube_mesh(builder, mesh, wp.vec3(0, 1.2, 0))\n", "\n", " # Configure material contact properties\n", " for i in range(len(builder.shape_material_ke)):\n", " builder.shape_material_ke[i] = 1e5\n", " builder.shape_material_kd[i] = 1000.0\n", " builder.shape_material_kf[i] = 5.0\n", " builder.shape_material_mu[i] = 0.5\n", "\n", " # Finalize model\n", " model = builder.finalize()\n", " model.soft_contact_ke = 1e5\n", " model.soft_contact_mu = 0.5\n", " model.soft_contact_kd = 1000.0\n", " model.soft_contact_kf = 5.0\n", " model.rigid_contact_ke = 1e2\n", "\n", " # Pre-compile the collision pipeline\n", " model.collide(model.state(), collision_pipeline=CollisionPipeline(model, soft_contact_margin=0.01))\n", "\n", " print(\"Model finalized\")\n", " obj_id_0=0\n", " obj_id_1=1\n", " return model, builder, obj_id_0, obj_id_1\n", "\n", "\n", "model, builder, obj_id_0, obj_id_1 = create_builder_model(sim_obj, mesh)\n", "print(f\"Model: {model.particle_count} particles, {model.body_count} bodies\")" ] }, { "cell_type": "markdown", "id": "cell-states-md", "metadata": {}, "source": [ "## States and Solvers\n", "\n", "Newton uses a double-buffer pattern (`state_0` / `state_1`). Each solver reads from\n", "`state_0` and writes results into `state_1`; states are then swapped.\n", "\n", "- **`SimplicitsSolver`** — handles the two soft-body Simplicits cubes.\n", "- **`SolverSemiImplicit`** — handles the rigid cube body.\n", "\n", "Both solvers share the same model and write into the same state buffers." ] }, { "cell_type": "code", "execution_count": null, "id": "cell-states", "metadata": {}, "outputs": [], "source": [ "state_0 = model.state()\n", "state_0.particle_q = model.sim_z_to_full(state_0.sim_z)\n", "state_1 = model.state()\n", "\n", "simplicits_solver = SimplicitsSolver(model)\n", "rigid_solver = SolverSemiImplicit(model)\n", "\n", "print(\"States and solvers ready\")" ] }, { "cell_type": "markdown", "id": "cell-simloop-md", "metadata": {}, "source": [ "## Simulation Loop\n", "\n", "Each call to `simulate()` advances the scene by one frame:\n", "\n", "1. **Simplicits solver** steps the soft cubes (`state_0 → state_1`), writing particle DOFs.\n", "2. **Rigid body solver** steps the cube body (`state_0 → state_1`), writing body DOFs.\n", " `model.particle_count` is temporarily set to 0 so the rigid solver skips particle\n", " processing.\n", "3. States are swapped for the next frame.\n", "\n", "> **Note:** CUDA graph capture is not used here because `model.particle_count` is mutated\n", "> on the host between the two solver calls, which is incompatible with graph capture." ] }, { "cell_type": "code", "execution_count": null, "id": "cell-simloop", "metadata": {}, "outputs": [], "source": [ "sim_substeps = 2\n", "sim_dt = DT / sim_substeps\n", "\n", "\n", "def simulate():\n", " global state_0, state_1\n", "\n", " for _ in range(sim_substeps):\n", " state_0.clear_forces()\n", " \n", " # Step 1: Simplicits (soft body) solver\n", " contacts = model.collide(state_0)\n", " simplicits_solver.step(state_0, state_1, None, contacts, sim_dt)\n", " \n", " # Step 2: Rigid body solver (simplicits particles disabled during this step)\n", " particle_count = model.particle_count\n", " model.particle_count = 0\n", " contacts = model.collide(state_0)\n", " rigid_solver.step(state_0, state_1, None, contacts, sim_dt)\n", " model.particle_count = particle_count\n", "\n", " state_0, state_1 = state_1, state_0\n", "\n", " " ] }, { "cell_type": "markdown", "id": "cell-viewer-md", "metadata": {}, "source": [ "## Visualization with k3d\n", "\n", "**k3d** renders all three cubes interactively in the notebook:\n", "\n", "- **`mesh_soft`** (blue) — the two deformable Simplicits cubes **with a single affine handle each**, combined into a single k3d\n", " mesh by concatenating their vertices and offsetting the face indices of the second cube by\n", " `num_verts`. Updated each frame via `get_object_deformed_pts` (linear blend skinning).\n", "- **`mesh_rigid`** (orange) — the single rigid body cube. Updated each frame by applying the\n", " body's pose (`state_0.body_q`) via `transform_points(orig_vertices_np, body_q_wp)`.\n", "\n", "Play/Stop/Reset buttons control the simulation via a background thread." ] }, { "cell_type": "code", "execution_count": null, "id": "cell-viewer", "metadata": {}, "outputs": [], "source": [ "orig_vertices_np = mesh.vertices.cpu().numpy().astype(np.float32)\n", "orig_faces_np = mesh.faces.cpu().numpy().astype(np.uint32)\n", "orig_vertices = mesh.vertices.clone() # keep GPU tensor for get_object_deformed_pts\n", "num_verts = orig_vertices_np.shape[0]\n", "\n", "# Combined mesh for the two Simplicits soft cubes\n", "combined_faces_np = np.concatenate(\n", " [orig_faces_np, orig_faces_np + num_verts]\n", ").astype(np.uint32)\n", "\n", "plot = k3d.plot(camera_auto_fit=False)\n", "plot.camera = [3, 3, 3, 0, 0, 0, 0, 1, 0]\n", "\n", "mesh_soft = k3d.mesh(\n", " np.concatenate([orig_vertices_np, orig_vertices_np]).astype(np.float32),\n", " combined_faces_np,\n", " color=0x3399ff\n", ")\n", "mesh_rigid = k3d.mesh(\n", " orig_vertices_np.copy(),\n", " orig_faces_np,\n", " color=0xff6633\n", ")\n", "plot += mesh_soft\n", "plot += mesh_rigid\n", "\n", "# Floor plane at FLOOR_PLANE height\n", "_s = 3.0 # half-extent\n", "floor_verts = np.array([\n", " [-_s, FLOOR_PLANE, -_s],\n", " [ _s, FLOOR_PLANE, -_s],\n", " [ _s, FLOOR_PLANE, _s],\n", " [-_s, FLOOR_PLANE, _s],\n", "], dtype=np.float32)\n", "floor_faces = np.array([[0, 1, 2], [0, 2, 3]], dtype=np.uint32)\n", "mesh_floor = k3d.mesh(floor_verts, floor_faces, color=0xcccccc, opacity=0.5)\n", "plot += mesh_floor\n", "\n", "plot.display()\n", "\n", "\n", "def update_vis_mesh():\n", " # Soft bodies\n", " model.simplicits_scene.sim_z = state_0.sim_z\n", " v0 = model.simplicits_scene.get_object_deformed_pts(obj_id_0, 'rendered')\n", " v1 = model.simplicits_scene.get_object_deformed_pts(obj_id_1, 'rendered')\n", " mesh_soft.vertices = np.concatenate(\n", " [v0.cpu().numpy(), v1.cpu().numpy()]\n", " ).astype(np.float32)\n", "\n", " # Rigid body\n", " body_q = state_0.body_q.numpy()\n", " for i in range(len(body_q)):\n", " body_q_wp = wp.transform(\n", " wp.vec3(body_q[i][0], body_q[i][1], body_q[i][2]),\n", " wp.quat(body_q[i][3], body_q[i][4], body_q[i][5], body_q[i][6])\n", " )\n", " mesh_rigid.vertices = transform_points(orig_vertices_np, body_q_wp).astype(np.float32)\n", "\n", "\n", "sim_running = [False]\n", "sim_thread = [None]\n", "\n", "\n", "def run_sim_loop():\n", " while sim_running[0]:\n", " simulate()\n", " update_vis_mesh()\n", "\n", "\n", "def on_play(b):\n", " if not sim_running[0]:\n", " sim_running[0] = True\n", " sim_thread[0] = threading.Thread(target=run_sim_loop, daemon=True)\n", " sim_thread[0].start()\n", "\n", "\n", "def on_stop(b):\n", " sim_running[0] = False\n", "\n", "\n", "def on_reset(b):\n", " global state_0, state_1\n", " sim_running[0] = False\n", " if sim_thread[0] is not None:\n", " sim_thread[0].join()\n", " model.simplicits_scene.reset_scene()\n", " state_0 = model.state()\n", " state_0.particle_q = model.sim_z_to_full(state_0.sim_z)\n", " state_1 = model.state()\n", " update_vis_mesh()\n", "\n", "\n", "buttons = [Button(description=x) for x in [\"Play\", \"Stop\", \"Reset\"]]\n", "buttons[0].on_click(on_play)\n", "buttons[1].on_click(on_stop)\n", "buttons[2].on_click(on_reset)\n", "\n", "update_vis_mesh()\n", "display(VBox(buttons))" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.12.3" } }, "nbformat": 4, "nbformat_minor": 5 }