{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Gradient-based optimization"
]
},
{
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"## Overview\n",
"\n",
"Mitsuba 3 can be used to solve inverse problems involving light using a technique known as *differentiable rendering*. It interprets the rendering algorithm as a function $f(\\mathbf{x})$ that converts an input $\\mathbf{x}$ (the scene description) into an output $\\mathbf{y}$ (the rendering). This function $f$ is then mathematically differentiated to obtain $\\frac{d\\mathbf{y}}{d\\mathbf{x}}$, providing a first-order approximation of how a desired change in the output $\\mathbf{y}$ (the rendering) can be achieved by changing the inputs $\\mathbf{x}$ (the scene description). Together with a differentiable *objective function* $g(\\mathbf{y})$ that quantifies the suitability of tentative scene parameters, a gradient-based optimization algorithm such as stochastic gradient descent or Adam can then be used to find a sequence of scene parameters $\\mathbf{x_0}$, $\\mathbf{x_1}$, $\\mathbf{x_2}$, etc., that successively improve the objective function. In pictures:\n",
"\n",
"\n",
"\n",
"\n",
"In this tutorial, we will build a simple example application that showcases differentiation and optimization through a light transport simulation:\n",
"\n",
"1. We will first render a reference image of the Cornell Box scene.\n",
"2. Then, we will perturb the color of one of the walls, e.g. changing it to blue.\n",
"3. Finally, we will try to recover the original color of the wall using differentiation along with the reference image generated in step 1.\n",
"\n",
"Mitsuba’s ability to automatically differentiate entire rendering algorithms builds on differentiable JIT array types provided by the Dr.Jit library. Those are explained in the [Dr.Jit documentation][1]. The linked document also discusses key differences compared to related frameworks like PyTorch and TensorFlow. For *automatic differentiation* (AD), Dr.Jit records and simplifies computation graphs and uses them to propagate derivatives in forward or reverse mode. Before getting further into this tutorial, we recommend that you familiarize yourself Dr.Jit.\n",
"\n",
"
\n",
"\n",
"🚀 **You will learn how to:**\n",
" \n",
"
\n",
"
Pass scene arguments when loading an XML file
\n",
"
Build an optimization loop using the Optimizer classes
\n",
"
Perform a gradient-based optimization using automatic differentiation
\n",
"
\n",
" \n",
"
\n",
"\n",
" \n",
"[1]: https://drjit.readthedocs.io/en/master/"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Setup\n",
"\n",
"In order to use the automatic differentiation, we need to enable a variant that supports it. Those are the ones containing `_ad` after the backend description. E.g. `cuda_ad_rgb`, `llvm_ad_rgb`, …\n",
"\n",
"\n",
"
\n",
" \n",
"If you receive an error mentionning that the requested variant is not supported, you can switch to another available `_ad` variant. If you compiled Mitsuba 3 yourself, you can also add the desired variant to your `build/mitsuba.conf` file and recompile the project ([documentation][1]).\n",
"\n",
"[1]: https://mitsuba.readthedocs.io/en/latest/src/developer_guide/compiling.html#configuring-mitsuba-conf\n",
"\n",
"
"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"ExecuteTime": {
"end_time": "2021-06-15T14:13:40.427650Z",
"start_time": "2021-06-15T14:13:40.259649Z"
}
},
"outputs": [],
"source": [
"import drjit as dr\n",
"import mitsuba as mi\n",
"\n",
"mi.set_variant('llvm_ad_rgb')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Scene loading\n",
"\n",
"Before loading the scene, let's note that in `cbox.xml`, we expose some variables at the top of the file:\n",
"\n",
"```xml\n",
" \n",
" \n",
" \n",
" \n",
"```\n",
"\n",
"Those variables are later referenced in the XML file, as explained in the [XML scene format documentation][1].\n",
"They can be given new values directly from Python when loading the scene by passing keyword arguments to the `load_file()` function. This helpful feature let us change the film resolution and integrator type for this tutorial without editing the XML file.\n",
"\n",
"For this simple differentiable rendering example, which does not involve moving objects or cameras, we recommand using the Path Replay Backpropagation integrator ([prb][2]) introduced by Vicini et al. (2021). It is essentially a path tracer, augmented with a specialized algorithm to efficiently compute the gradients in a separate adjoint pass.\n",
"\n",
"[1]: https://mitsuba.readthedocs.io/en/latest/src/key_topics/scene_format.html\n",
"[2]: https://mitsuba.readthedocs.io/en/latest/src/api_reference.html#mitsuba.ad.integrators.prb"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"ExecuteTime": {
"end_time": "2021-06-15T14:13:40.914750Z",
"start_time": "2021-06-15T14:13:40.807334Z"
}
},
"outputs": [],
"source": [
"scene = mi.load_file('../scenes/cbox.xml', res=128, integrator='prb')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Reference image\n",
"\n",
"We render a reference image of the original scene that will later be used in the objective function for the optimization. Ideally, this reference image should expose very little noise as it will pertube optimization process otherwise. For best results, we should render it with an even larger sample count."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
""
],
"text/plain": [
"Bitmap[\n",
" pixel_format = rgb,\n",
" component_format = uint8,\n",
" size = [128, 128],\n",
" srgb_gamma = 1,\n",
" struct = Struct<3>[\n",
" uint8 R; // @0, normalized, gamma, premultiplied alpha\n",
" uint8 G; // @1, normalized, gamma, premultiplied alpha\n",
" uint8 B; // @2, normalized, gamma, premultiplied alpha\n",
" ],\n",
" data = [ 48 KiB of image data ]\n",
"]"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"image_ref = mi.render(scene, spp=512)\n",
"\n",
"# Preview the reference image\n",
"mi.util.convert_to_bitmap(image_ref)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Initial state\n",
"\n",
"Using the `traverse` mechanism, we can pick the parameter that we will be optimizing and change its value away from the correct value. The goal of the optimization process will be to recover the original value of this parameter using gradient descent. \n",
"\n",
"We chose the `'red.reflectance.value'` parameter, which controls the albedo color of the red wall in the scene.\n",
"For later comparison, we also save the original value of the scene parameter."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [],
"source": [
"params = mi.traverse(scene)\n",
"\n",
"key = 'red.reflectance.value'\n",
"\n",
"# Save the original value\n",
"param_ref = mi.Color3f(params[key])\n",
"\n",
"# Set another color value and update the scene\n",
"params[key] = mi.Color3f(0.01, 0.2, 0.9)\n",
"params.update();"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As expected, when rendering the scene again, the wall has changed color."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
""
],
"text/plain": [
"Bitmap[\n",
" pixel_format = rgb,\n",
" component_format = uint8,\n",
" size = [128, 128],\n",
" srgb_gamma = 1,\n",
" struct = Struct<3>[\n",
" uint8 R; // @0, normalized, gamma, premultiplied alpha\n",
" uint8 G; // @1, normalized, gamma, premultiplied alpha\n",
" uint8 B; // @2, normalized, gamma, premultiplied alpha\n",
" ],\n",
" data = [ 48 KiB of image data ]\n",
"]"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"image_init = mi.render(scene, spp=128)\n",
"mi.util.convert_to_bitmap(image_init)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Optimization\n",
"\n",
"For gradient-based optimization, Mitsuba ships with standard optimizers including *Stochastic Gradient Descent* ([SGD][1]) with and without momentum, as well as [Adam][2] [KB14]. We will instantiate the latter and optimize our scene parameter with a learning rate of `0.05`. \n",
"\n",
"We then set the color to optimize on the optimizer, which will now hold a copy of this parameter and enable gradient tracking on it. During the optimization process, the optimizer will always perfom gradient steps on those variables. To propagate those changes to the scene, we need to call the `update()` method which will copy the values back into the `params` data structure. As always this method also notifies all objects in the scene whose parameters have changed, in case they need to update their internal state.\n",
"\n",
"This first call to `params.update()` ensures that gradient tracking with respect to our wall color parameter is propagated to the scene internal state. For more detailed explanation on how-to-use the optimizer classes, please refer to the dedicated [how-to-guide][3].\n",
"\n",
"[1]: https://mitsuba.readthedocs.io/en/latest/src/api_reference.html#mitsuba.ad.SGD\n",
"[2]: https://mitsuba.readthedocs.io/en/latest/src/api_reference.html#mitsuba.ad.Adam\n",
"[3]: https://mitsuba.readthedocs.io/en/latest/src/how_to_guides/use_optimizers.html"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"ExecuteTime": {
"end_time": "2021-06-15T14:13:51.368393Z",
"start_time": "2021-06-15T14:13:51.362695Z"
}
},
"outputs": [],
"source": [
"opt = mi.ad.Adam(lr=0.05)\n",
"opt[key] = params[key]\n",
"params.update(opt);"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"At every iteration of the gradient descent, we will compute the derivatives of the scene parameters with respect to the objective function. In this simple experiment, we use the [*mean square error*][1], or $L_2$ error, between the current image and the reference created above.\n",
"\n",
"[1]: https://en.wikipedia.org/wiki/Mean_squared_error"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [],
"source": [
"def mse(image):\n",
" return dr.mean(dr.sqr(image - image_ref))"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"In the following cell we define the hyper parameters controlling our optimization loop, such as the number of iterations:"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [],
"source": [
"iteration_count = 50"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"nbsphinx": "hidden",
"tags": []
},
"outputs": [],
"source": [
"# IGNORE THIS: When running under pytest, adjust parameters to reduce computation time\n",
"import os\n",
"if 'PYTEST_CURRENT_TEST' in os.environ:\n",
" iteration_count = 2"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"It is now time to actually perform the gradient-descent loop that executes 50 differentiable rendering iterations."
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"ExecuteTime": {
"end_time": "2021-06-15T14:14:04.664564Z",
"start_time": "2021-06-15T14:13:54.209445Z"
}
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Iteration 49: parameter error = 0.001432\n",
"Optimization complete.\n"
]
}
],
"source": [
"errors = []\n",
"for it in range(iteration_count):\n",
" # Perform a (noisy) differentiable rendering of the scene\n",
" image = mi.render(scene, params, spp=4)\n",
" \n",
" # Evaluate the objective function from the current rendered image\n",
" loss = mse(image)\n",
"\n",
" # Backpropagate through the rendering process\n",
" dr.backward(loss)\n",
"\n",
" # Optimizer: take a gradient descent step\n",
" opt.step()\n",
"\n",
" # Post-process the optimized parameters to ensure legal color values.\n",
" opt[key] = dr.clamp(opt[key], 0.0, 1.0)\n",
"\n",
" # Update the scene state to the new optimized values\n",
" params.update(opt)\n",
" \n",
" # Track the difference between the current color and the true value\n",
" err_ref = dr.sum(dr.sqr(param_ref - params[key]))\n",
" print(f\"Iteration {it:02d}: parameter error = {err_ref[0]:6f}\", end='\\r')\n",
" errors.append(err_ref)\n",
"print('\\nOptimization complete.')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Results\n",
"\n",
"We can now render the scene again to check whether the optimization process successfully recovered the color of the red wall."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"ExecuteTime": {
"end_time": "2021-06-15T14:14:07.765382Z",
"start_time": "2021-06-15T14:14:06.895491Z"
}
},
"outputs": [
{
"data": {
"text/html": [
""
],
"text/plain": [
"Bitmap[\n",
" pixel_format = rgb,\n",
" component_format = uint8,\n",
" size = [128, 128],\n",
" srgb_gamma = 1,\n",
" struct = Struct<3>[\n",
" uint8 R; // @0, normalized, gamma, premultiplied alpha\n",
" uint8 G; // @1, normalized, gamma, premultiplied alpha\n",
" uint8 B; // @2, normalized, gamma, premultiplied alpha\n",
" ],\n",
" data = [ 48 KiB of image data ]\n",
"]"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"image_final = mi.render(scene, spp=128)\n",
"mi.util.convert_to_bitmap(image_final)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"It worked!\n",
"\n",
"Note visualizing the objective value directly sometimes gives limited information, since differences between `image` and `image_ref` can be dominated by Monte Carlo noise that is not related to the parameter being optimized. \n",
"\n",
"Since we know the “true” target parameter in this scene, we can validate the convergence of the optimization by checking the difference to the true color at each iteration:"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"nbsphinx-thumbnail": {},
"tags": []
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAYIAAAEWCAYAAABrDZDcAAAAOXRFWHRTb2Z0d2FyZQBNYXRwbG90bGliIHZlcnNpb24zLjUuMSwgaHR0cHM6Ly9tYXRwbG90bGliLm9yZy/YYfK9AAAACXBIWXMAAAsTAAALEwEAmpwYAAAm3ElEQVR4nO3deZxddX3/8df7zppMMkkmGbJONkiEBFlDAIEUQRBEwLZaoNVapaK1bq2WUmutxfr7ubT+tNW2ora2oCBqsRFBQGRTGiBhiUkIEkJWErKQfZvlfn5/nDPhMkwyE5I7Z+ae9/PxuI97tnvu50wm9z3n+z33exQRmJlZfhWyLsDMzLLlIDAzyzkHgZlZzjkIzMxyzkFgZpZzDgIzs5xzEJgZkj4j6aas67BsOAisbCStkLRH0k5JL0r6jqQhWdd1MJImSwpJ1VnX0l+l/65vyroOO3IcBFZul0bEEOAUYBbwqUN5sRID5vf0cAOku+M91H06xOxQDZj/YDawRcRa4E7geEkjJN0uaaOkLen0hM5tJd0v6XOSfgXsBqZKeo+kpyXtkLRc0vtLtj9X0hpJ10raIGmdpLdJeouk30h6SdInS7YvSLpO0nOSNku6VVJTuvrB9HlreiZzZvqa96bvv0XSXZImlewvJP2ppGeBZ7s7fklnSHpY0lZJT0k6t4fjfdU+Jb1P0rL0eOZKGtfbGkrOdK6R9EL6M/rEgf69JF0maXFa7/2SjkuX3whMBH6S/nyuPdA+bACJCD/8KMsDWAG8KZ1uARYDnwVGAr8LDAaGAj8AflzyuvuBVcBMoBqoAS4BjgYE/BbJB+Yp6fbnAu3Ap9Nt3wdsBL6X7n8msAeYkm7/UWAeMAGoA74B3JyumwwEUF1Sz+XAMuC4tJ5PAQ+XrA/gHqAJGNTNz2E8sBl4C8kfXxek880HOd5X7BM4D9hEcmZVB/wz8OAh1NB5XDcDDcDr059R57/PZ4Cb0unpwK60zhrg2vT4a7v+u/pRGY/MC/Cjch/pB8ZOYCuwEviXA3xInQRsKZm/H7i+h33/GPhoOn1u+kFflc4PTT/0Ti/ZfgHwtnT6aeD8knVjgbb0Q7i7ILgTuLpkvkASRJPS+QDOO0itfwnc2GXZXcC7D3S8XfcJfBv4Ysn8kLTmyb2sofO4ji1Z9kXg2+l0aRD8DXBrl+NdC5xb8u/qIKigh5uGrNzeFhHDI2JSRHwwIvZIGizpG5JWStpO0hwzXFJVyetWl+5E0sWS5qXNIltJ/roeVbLJ5ojoSKf3pM8vlqzfQ/LhCTAJuC1t9thKEgwdwOgDHMMk4Ksl279EcmYy/kD1dvP6d3S+Pt3H2SQBdLDXly4bRxKmAETETpKzit7W0N02K9P9dtX1vYrp68Z3s61VAAeBZeHjwOtI/mJvBOaky1Wyzf5hcSXVAT8C/gEYHRHDgTu6bH8oVgMXpwHV+aiPpB+ju+F4VwPv77L9oIh4uLt6D/D6G7u8viEiPt/D60uXvUASKABIaiBpYlvbyxo6tZRMT0z321XX91L6us738pDFFcZBYFkYSvIX+ta0k/Zve9i+lqRdfCPQLuli4MLDeP9/Az7X2eErqVnS5em6jUARmNpl+7+SNDPdfpikdxzC+90EXCrpzZKqJNWnHdwTenzly24G3iPppDQY/w/wSESsOIR9APxNekY2E3gP8P1utrkVuETS+ZJqSIJ7H9AZfC/yyp+PDXAOAsvCV0g6QDeRdNr+7GAbR8QO4CMkH1BbgN8H5h7G+381ff3dknakNZyevtdu4HPAr9JmnDMi4jbgC8AtaVPWIuDi3r5ZRKwm6XD+JEnQrAb+gkP4/xcRPydpu/8RsI6k4/zK3r6+xAMkHb/3Av8QEXd3817PAO8k6ZDeBFxKchlwa7rJ/wU+lf58DnjlkQ0civBZnlmlkzQZeB6oiYj2jMuxfsZnBGZmOecgMDPLOTcNmZnlnM8IzMxybsANTjVq1KiYPHly1mWYmQ0oCxYs2BQRzd2tG3BBMHnyZObPn591GWZmA4qklQda56YhM7OccxCYmeWcg8DMLOccBGZmOecgMDPLOQeBmVnOOQjMzHIuN0GwYOVLfOFnS/GQGmZmr5SbIFi0djv/ev9zrNu2N+tSzMz6ldwEwUktwwF4cvXWTOswM+tvchMEx41tpLa64CAwM+siN0FQW11g5rhGnly1NetSzMz6ldwEASTNQwvXbqW9o5h1KWZm/UbugmBvW5Gl63dkXYqZWb+RqyA4uWUE4A5jM7NSuQqClqZBNDXUOgjMzErkKggkcVLLcAeBmVmJXAUBJP0Ez23cyfa9bVmXYmbWL+QyCCJg4eptWZdiZtYv5C4ITtz/DeMt2RZiZtZP5C4Ihg2q4ejmBvcTmJmlchcEACe1jODJ1Vs9EqmZGXkNgonD2bSzlTVb9mRdiplZ5nIZBCd7JFIzs/1yGQSvGzOUOo9EamYG5DQIaqoKvH78MAeBmRk5DQJIvk+waO022jwSqZnlXH6DYOJw9rUXWbrOI5GaWb7lNwj8xTIzMyDHQTB++CBGDanjCd+xzMxyLrdB4JFIzcwSZQ0CSRdJekbSMknXdbN+oqT7JD0haaGkt5Sznq5Onjic5Zt2sW23RyI1s/wqWxBIqgK+DlwMzACukjSjy2afAm6NiJOBK4F/KVc93dnfT7Bma1++rZlZv1LOM4LZwLKIWB4RrcAtwOVdtgmgMZ0eBrxQxnpe5YQJw5DgSfcTmFmOlTMIxgOrS+bXpMtKfQZ4p6Q1wB3Ah7vbkaRrJM2XNH/jxo1HrMCh9TUc0zzEVw6ZWa5l3Vl8FfCdiJgAvAW4UdKraoqIGyJiVkTMam5uPqIFnDxxuEciNbNcK2cQrAVaSuYnpMtKXQ3cChAR/wvUA6PKWNOrnNQygi2721j10u6+fFszs36jnEHwGDBN0hRJtSSdwXO7bLMKOB9A0nEkQXDk2n564ZRJwwGYv8LNQ2aWT2ULgohoBz4E3AU8TXJ10GJJ10u6LN3s48D7JD0F3Az8UfRxG830o4YybFANjz7/Ul++rZlZv1Fdzp1HxB0kncClyz5dMr0EOKucNfSkUBCnTW7i0RUOAjPLp6w7i/uF06c08fymXWzYvjfrUszM+pyDAJg9pQnAZwVmlksOAmDmuEYG11a5n8DMcslBAFRXFTh10ggHgZnlkoMgdfqUJpau38GWXa1Zl2Jm1qccBKnTp44E4DH3E5hZzjgIUidMGEZtdcHNQ2aWOw6CVF11FSe3DPeVQ2aWOw6CEqdPaWLR2m3s3NeedSlmZn3GQVBi9pSRFAMWrPS4Q2aWHw6CEqdMGk51QTz6/OasSzEz6zMOghKDa6s5fvwwHlnufgIzyw8HQRenT2niqTVb2dvWkXUpZmZ9wkHQxelTm2jrCJ7wfYzNLCccBF2cOqkJCX+fwMxyw0HQxbBBNRw3ppFHV7jD2MzywUHQjdlTmliwcgut7cWsSzEzKzsHQTdOn9LE3rYii17YlnUpZmZl5yDoxmnpjWp8GamZ5YGDoBujhtRxdHODv1hmZrngIDiA2VNGMn/FFjqKkXUpZmZl5SA4gDOmNrFjXztPr9uedSlmZmXlIDiAzhvaz1vu5iEzq2wOggMYO2wQU0c18Mtlm7IuxcysrBwEB3HOtFHMW77Z4w6ZWUVzEBzEnOnN7G0r+v4EZlbRHAQHccbUkdRUiQd/szHrUszMysZBcBANddWcOmkEDzgIzKyCOQh6MGd6M0vX72DD9r1Zl2JmVhYOgh7MmdYMwEPP+uohM6tMDoIezBjbyKghtTz4rJuHzKwyOQh6UCiIs48ZxUPPbqLo4SbMrAI5CHphzvRmXtrVyhIPN2FmFchB0AtnTxsF4KuHzKwilTUIJF0k6RlJyyRdd4Btfk/SEkmLJX2vnPW8VkcNree4sY3+PoGZVaSyBYGkKuDrwMXADOAqSTO6bDMN+CvgrIiYCXysXPUcrjnTR/H4qi3s3NeedSlmZkdUOc8IZgPLImJ5RLQCtwCXd9nmfcDXI2ILQERsKGM9h+W3pjXT1hHMe86jkZpZZSlnEIwHVpfMr0mXlZoOTJf0K0nzJF3U3Y4kXSNpvqT5Gzdm0zxz6uQRDKqp8mWkZlZxsu4srgamAecCVwHflDS860YRcUNEzIqIWc3NzX1bYaquuoozpja5n8DMKk6vgkBSQdLJki6RdJ6ko3rxsrVAS8n8hHRZqTXA3Ihoi4jngd+QBEO/NGd6Mys272bV5t1Zl2JmdsQcNAgkHS3pBmAZ8HmSv9o/CPw8bcp5j6QD7eMxYJqkKZJqgSuBuV22+THJ2QCSRpE0FS1/jcdSduekw024ecjMKklPZwR/D9wEHB0Rb46Id0bE2yPiBOAyYBjwru5eGBHtwIeAu4CngVsjYrGk6yVdlm52F7BZ0hLgPuAvIqLf9sYe3dzA+OGD3DxkZhWl+mArI+Kqg6zbAHylh9ffAdzRZdmnS6YD+PP00e9JYs70Udz+1DraOorUVGXdxWJmdvgOGgSd0u8EXAJMLn1NRHy5PGX1X3OmNXPzo6t5cvVWTpvclHU5ZmaHrbd/0v4E+CNgJDC05JE7bzh6FAXh5iEzqxi9OiMAJqT9Ark3bHANp0wcwS+WbuDjF74u63LMzA5bb88I7pR0YVkrGUDeNGM0i1/Yztqte7IuxczssPU2COYBt0naI2m7pB2Scjsm8wUzRgPw8yUvZlyJmdnh620QfBk4ExgcEY0RMTQiGstYV792dPMQpjY3cI+DwMwqQG+DYDWwKL3c00jOCuYt38y2PW1Zl2Jmdlh6GwTLgfsl/ZWkP+98lLOw/u7CGaNpLwb3P9NvB0w1M+uV3gbB88C9QC05v3y000ktIxg1pNbNQ2Y24PXq8tGI+LtyFzLQVBXE+ceO5qe/Xse+9g7qqquyLsnM7DXp7eijzZK+JOkOSb/ofJS7uP7uwpmj2bmvnXnLX8q6FDOz16y3TUPfBZYCU4C/A1aQjC6aa2cdM4pBNVXcs2R91qWYmb1mvQ2CkRHxbaAtIh6IiPcC55WxrgGhvqaKOdNH8fMlG/AFVWY2UPU2CDqvkVyX3pzmZMAjrgEXzBjD+u17+fXabVmXYmb2mvR2rKG/lzQM+Djwz0Aj8Gdlq2oAOe/YoygI7lnyIidMGJ51OWZmh6zHM4J0COppEbEtIhZFxBsj4tSI6Hq3sVxqaqhl1uQmX0ZqZgNWj0EQER0kt6i0A7hwxmiWrt/B6pd8L2MzG3h620fwK0lfk3SOpFM6H2WtbADpHITubp8VmNkA1Ns+gpPS5+tLlgW+cgiASSMbmD56CPcsWc/VZ0/Juhwzs0PS228Wv7HchQx0F84Yw78+8Bxbd7cyfHBt1uWYmfVar+++nl42eq2kT3c+ylnYQHPBjNF0FINfLPUgdGY2sPR2iIl/A64APgwIeAcwqYx1DTivHz+M0Y113L3Y/QRmNrD09ozgDRHxh8CWdAC6M4Hp5Str4CkUxAUzRvPAbzayu7U963LMzHqtt0HQeXPe3ZLGkXzTeGx5Shq43nrCOPa0dXDv024eMrOBo7dBcLuk4cCXgMdJBp37XplqGrBOm9zE6MY6fvLUC1mXYmbWa729auiz6eSPJN0O1EeEB9fpoqogLnn9OG6at5Lte9torK/JuiQzsx71trO4Pr095X+TnAm8V1J9eUsbmC49cSytHUV3GpvZgNHbpqH/AmaSDDj3NWAGcGO5ihrITmoZzoQRg9w8ZGYDRm+/WXx8RMwomb9P0pJyFDTQSeLSE8dxw4PLeWlXK00N/nKZmfVvvT0jeFzSGZ0zkk4H5penpIHv0hPG0VEM7ly0LutSzMx61NsgOBV4WNIKSSuA/wVOk/RrSQvLVt0AddzYoRzd3ODmITMbEHrbNHRRWauoMJ3NQ1+991le3L6X0Y3uVzez/uugZwSShgBExMoDPYDNfVLpAPPWE8YRAT9d6OYhM+vfemoa+h9J/yhpjqSGzoWSpkq6WtJd+GyhW8ccNYQZYxv5yUI3D5lZ/3bQIIiI84F7gfcDiyVtk7QZuAkYA7w7In54oNdLukjSM5KWSbruINv9rqSQNOu1HUb/dOmJ43hi1VbfuczM+rXe3Kryjoj4g4iYHBHDImJkRLwhIj4XEesP9Lr0XsdfBy4m+d7BVZJmdLPdUOCjwCOv/TD6p7eekAzHdLubh8ysH+upj+CdJdNndVn3oR72PRtYFhHLI6IVuAW4vJvtPgt8Adjbq4oHkJamwZw8cbivHjKzfq2nM4I/L5n+5y7r3tvDa8cDq0vm16TL9kvve9wSET892I4kXSNpvqT5Gzdu7OFt+5dLTxjHknXbWbZhZ9almJl1q6cg0AGmu5s/JJIKwJeBj/e0bUTcEBGzImJWc3Pz4bxtn7vkhLFIcLs7jc2sn+opCOIA093Nd7UWaCmZn5Au6zQUOB64P/2S2hnA3ErrMB7dWM/pU5r4yVMvENHTj8zMrO/1FATHSloo6dcl053zr+vhtY8B0yRNkVQLXAnM7VwZEdsiYlTaCT0ZmAdcFhEVN3TFZSeO57mNu1i4xiN3m1n/09M3i497rTuOiPa0Q/kuoAr494hYLOl6YH5EzD34HirHW08cy/W3L+aWx1ZzYsvwrMsxM3uFgwZB+s3h/SSNBOYAqyJiQU87j4g7gDu6LPv0AbY9t6f9DVSN9TVc8vpxzH1yLZ+65Dga6no7soeZWfn1dPno7ZKOT6fHAotIrha6UdLHyl9e5bhqdgu7Wjs85ISZ9Ts99RFMiYhF6fR7gHsi4lLgdHq+fNRKnDppBMccNYRbHluVdSlmZq/QUxC0lUyfT9rMExE7gGK5iqpEkrjytBYeX7WV37y4I+tyzMz26ykIVkv6sKTfBk4BfgYgaRDgO7Mfot8+eTw1VeKWR1f3vLGZWR/pKQiuJrlX8R8BV0TE1nT5GcB/lK+syjRySB0XzhzDfz+xhn3tHVmXY2YG9Dz66IaI+EBEXB4Rd5csvy8i/qH85VWeK09rYevuNu5a/GLWpZiZAT1cPirpoNf6R8RlR7acynfW0aOYMGIQ339sFZedOC7rcszMevxC2ZkkA8fdTDJM9GGNL2RQKIgrZrXwj/f8hpWbdzFpZEPPLzIzK6Oe+gjGAJ8kGRPoq8AFwKaIeCAiHih3cZXq7bMmUBDcOt+dxmaWvZ76CDoi4mcR8W6SDuJlJIPE9XQvAjuIscMG8cbXHcUP5q+hvcNX4ZpZtnq8Q5mkOkm/Q3J7yj8F/gm4rdyFVborTmthw4593PfMwLq/gplVnp46i/+LpFnoDuDvSr5lbIfpjcceRfPQOm55dBUXzBiddTlmlmM9nRG8E5hGck/hhyVtTx87JG0vf3mVq6aqwDtOncB9z2xg3bY9WZdjZjnWUx9BISKGpo/GksfQiGjsqyIr1ZWnTSSAm+at7HFbM7Ny6bGPwMpn4sjBvHnGGG6at4pd+9qzLsfMcspBkLH3zZnKtj1tvpTUzDLjIMjYqZNGMGvSCL79y+d9KamZZcJB0A+8b85U1mzZw52L1mddipnlkIOgH7jguNFMGdXADQ8uJyKyLsfMcsZB0A8UCuKPz5nCr9duY97yl7Iux8xyxkHQT/zuKRMY2VDLNx9annUpZpYzDoJ+or6mij88czK/WLqBZ30rSzPrQw6CfuRdZ06ivqbgswIz61MOgn6kqaGWd5zawo+feIEN2/dmXY6Z5YSDoJ/543Om0FYs8p2HV2RdipnlhIOgn5k0soGLZo7hpnkrPeyEmfUJB0E/dM2cqWzf287Nj67KuhQzywEHQT908sQRnHXMSP7l/ufYsbct63LMrMI5CPqpv7zoWF7a1co3H3o+61LMrMI5CPqpEyYM55ITxvKth5azYYevIDKz8nEQ9GOfuPB1tLYX+ed7l2VdiplVMAdBPzZlVANXzm7h5kdXsWLTrqzLMbMK5SDo5z5y/jRqqgr8w93PZF2KmVUoB0E/d9TQev74nCncvnAdv16zLetyzKwClTUIJF0k6RlJyyRd1836P5e0RNJCSfdKmlTOegaqa+ZMZcTgGr7ws6VZl2JmFahsQSCpCvg6cDEwA7hK0owumz0BzIqIE4AfAl8sVz0D2dD6Gj583jR+uWwTDz27MetyzKzClPOMYDawLCKWR0QrcAtweekGEXFfROxOZ+cBE8pYz4D2B2dMZMKIQXz+zqUUi76LmZkdOeUMgvHA6pL5NemyA7kauLO7FZKukTRf0vyNG/P5F3FddRUfv3A6i1/Yzk8WvpB1OWZWQfpFZ7GkdwKzgC91tz4iboiIWRExq7m5uW+L60cuP3E8x41t5PN3LvXQE2Z2xJQzCNYCLSXzE9JlryDpTcBfA5dFxL4y1jPgFQric799POu37+WLP/PlpGZ2ZJQzCB4DpkmaIqkWuBKYW7qBpJOBb5CEwIYy1lIxTpk4gve8YQo3zlvJo8/7RvdmdvjKFgQR0Q58CLgLeBq4NSIWS7pe0mXpZl8ChgA/kPSkpLkH2J2V+MSbpzNhxCD+8kcL2dvWkXU5ZjbAKWJgXYEya9asmD9/ftZlZO6Xz27ind9+hA/81tFcd/GxWZdjZv2cpAURMau7df2is9gO3dnTRnHFrBa++dByf+PYzA6Lg2AA++QlxzGyoZZrf7SQto5i1uWY2QDlIBjAhg2q4bNvO56n123nhgeXZ12OmQ1QDoIB7s0zx3DJ68fy1Z8/y7INO7Mux8wGIAdBBfjMZTMZVFvFtT98yk1EZnbIHAQVoHloHX//tuN5fNVWPvfTp7Mux8wGmOqsC7Aj49ITx/HU6q1865fPM3NcI++Y1dLzi8zM8BlBRbnu4mM565iR/PWPF/HU6q1Zl2NmA4SDoIJUVxX42lWncNTQOt5/4wI27vDQTWbWMwdBhRnRUMsN75rF1j2tfPC7C2htd+exmR2cg6ACzRjXyBfffiKPrdjCZ29fknU5ZtbPubO4Ql124jgWv7CNbzywnJnjGrly9sSsSzKzfspnBBXs2jcfyznTRvHp/1nMfc94lG8z656DoIJVFcTXrjqF6WOG8P7/WsC9T7+YdUlm1g85CCrcsME1fPfqMzh27FA+cNMC7l68PuuSzKyfcRDkwLDBNdx49enMHDeMD373cX62aF3WJZlZP+IgyIlhg2q48erZnNgynD/93hP8dKHDwMwSDoIcGVpfw3++dzanTBzOR255gv95cm3WJZlZP+AgyJkhddV85z2zmTVpBH/2/Sf51kPLGWi3KzWzI8tBkEMNddX8x3tO403Hjebvf/o0f3LT42zf25Z1WWaWEQdBTg2ureYb7zqVT11yHD9/+kXe+k+/ZNFa3/vYLI8cBDkmiT8+Zyrff/8ZtHUU+Z1/fZjvPrLSTUVmOeMgME6d1MRPP3IOZ0wdyV/ftoiPff9Jdu5rz7osM+sjDgIDoKmhlu/80Wl8/ILp/OSpFzj/H+/nhwvWUCz67MCs0jkIbL9CQXz4/Gn84ANvYExjPZ/4wVNc/vVf8cjyzVmXZmZl5CCwVzl10ghu++BZfOWKk9i0cx9X3DCPD9y4gJWbd2VdmpmVgYehtm4VCuJtJ4/nzTPH8M2HlvOv9z/HL5Zu4KrZLbzrzEkcc9TQrEs0syNEA+0KkVmzZsX8+fOzLiN3Xty+l3+8+xlue2ItbR3B7ClN/MHpE7no+DHUVVdlXZ6Z9UDSgoiY1e06B4Edik079/HDBWu4+dFVrNy8m6aGWt5+6gSuOK2Fo5uHZF2emR2Ag8COuGIx+NVzm/jeI6u4e8mLdBSDiU2DmTN9FOdMa+YNR49kaH1N1mWaWcpBYGW1Yfte7ly0noee3cjDz21md2sHVQVxysThnH1MM6+f0MixYxoZO6weSVmXa5ZLDgLrM63tRRas3MJDz27kwWc3smjt9v3rGuurOXZsI8eNGcrrxjQysWkwY4bVM3ZYPQ11vm7BrJwcBJaZbXva+M2LO1i6bjtL1+9g6fodPLN+x6u+udxYX83YYYMYM6yekUNqGTao5lWPofU1DKqpYlBtgUG11cl0TRV11QUKBZ9pmB3MwYLAf4ZZWQ0bVMNpk5s4bXLT/mURwdqte1i7ZQ/rtu1NH3v2Py/bsJNte9oOaZiLmipRU1WgtrqQPKfT1QVRVRDVVaK68PJ8fRoig2qrSqYLNNbXMKKhlpENtTQ11DKyoY4RDTUMqat2s5ZVrLIGgaSLgK8CVcC3IuLzXdbXAf8FnApsBq6IiBXlrMmyJ4kJIwYzYcTgg27X3lFk+952tu1pS4Jhbzt72zrY3dbB3tYO9rR1sLu1g71tHbR2FGlrL9LWUaS1o0hre9DaUaSjWKS9I2gvJo+OYpG2jmDrnjbWb9vLnrZkP3tbk/12HGBIjdqqAk2d4TCk9uXphlqGD65lxOBaRgyuSaYbahgxuJb6Gl9WawND2YJAUhXwdeACYA3wmKS5EbGkZLOrgS0RcYykK4EvAFeUqyYbWKpLPnz7QkSwu7WDl3a1snlXKy/t2sfmna1s2Z3O72zdv27l5t28tKv1oGcttdXJGUbjoGoa62sYWl9N46AahtRWM7iuiobS59rk7KSuuora6gJ1+x9V1FaLqpKzmc7nqoIQgvREpfOERUAxkuMpBhQjKBaT6fZikY40FJOATObbOoK2jmLJo2Q+DdXSda1p6LYXX57uXJfsu9jlOegoBh0R++vpiKCjmNQZAUHnM68YAVcSSo9PCAkKEoUCVElI6c8jXVZdSJoKu/68qguiuqqQPpecIVaJmkKB6vSssnO7ms5tqrR/+pXLXrltVaFkWbpN5/t0raOqoH51hlnOM4LZwLKIWA4g6RbgcqA0CC4HPpNO/xD4miTFQOu4sIogiYa6ahrqqmlpOvjZSqd97R1s3d3Glt2tbNnVxtbdrWxJ57fvbWP7nnZ27G1j+952tu9pY+3WPeze18Gu1nZ27WtnII/pV1PyYZg0wyUffrVV6QffKz500w/KQiH5EE8/vJPpV37Id04joDQgSqb3B1wkAfPyM+xp69h/9tdRTM4sO+LlMGrrKO5/bi++HFx9/W/RGV5VhVc+kp8NabC9HHISfOxN07n0xHFHvJZyBsF4YHXJ/Brg9ANtExHtkrYBI4FNpRtJuga4BmDixInlqtfskNVVVzG6sYrRjfWH/NqIYF97kd2tHezalzR77Wsvpo90ui2ZLqZ/PXcUi+mHXPLBFiX7enm/yYfpKz9ok+mufSbJX7HJss7+lZr0L9zaqgLV+/tdkvnS9f3pL9ojoVgM2jqbEjtenm4rObtp7Xj5DKrzbKeto7T58dXrO587z8Refi7u/zftKCbB1t453Xn2tP8sKlk2fHB5vpszIDqLI+IG4AZIrhrKuByzI0JKOq3ra6r6rPnLDqxQEHWFKvJ4JXM5Rx9dC7SUzE9Il3W7jaRqYBhJp7GZmfWRcgbBY8A0SVMk1QJXAnO7bDMXeHc6/XbgF+4fMDPrW2U7CUrb/D8E3EVy+ei/R8RiSdcD8yNiLvBt4EZJy4CXSMLCzMz6UFlbwyLiDuCOLss+XTK9F3hHOWswM7OD8x3KzMxyzkFgZpZzDgIzs5xzEJiZ5dyAG4Za0kZg5Wt8+Si6fGs5J/J63JDfY/dx50tvjntSRDR3t2LABcHhkDT/QONxV7K8Hjfk99h93PlyuMftpiEzs5xzEJiZ5VzeguCGrAvISF6PG/J77D7ufDms485VH4GZmb1a3s4IzMysCweBmVnO5SYIJF0k6RlJyyRdl3U95SLp3yVtkLSoZFmTpHskPZs+j8iyxnKQ1CLpPklLJC2W9NF0eUUfu6R6SY9Keio97r9Ll0+R9Ej6+/79dCj4iiOpStITkm5P5yv+uCWtkPRrSU9Kmp8uO6zf81wEgaQq4OvAxcAM4CpJM7Ktqmy+A1zUZdl1wL0RMQ24N52vNO3AxyNiBnAG8Kfpv3GlH/s+4LyIOBE4CbhI0hnAF4D/FxHHAFuAq7Mrsaw+CjxdMp+X435jRJxU8t2Bw/o9z0UQALOBZRGxPCJagVuAyzOuqSwi4kGSezuUuhz4z3T6P4G39WVNfSEi1kXE4+n0DpIPh/FU+LFHYmc6W5M+AjgP+GG6vOKOG0DSBOAS4FvpvMjBcR/AYf2e5yUIxgOrS+bXpMvyYnRErEun1wOjsyym3CRNBk4GHiEHx542jzwJbADuAZ4DtkZEe7pJpf6+fwW4Fiim8yPJx3EHcLekBZKuSZcd1u95Dm/TnG8REZIq9pphSUOAHwEfi4jtyR+JiUo99ojoAE6SNBy4DTg224rKT9JbgQ0RsUDSuRmX09fOjoi1ko4C7pG0tHTla/k9z8sZwVqgpWR+QrosL16UNBYgfd6QcT1lIamGJAS+GxH/nS7OxbEDRMRW4D7gTGC4pM4/9Crx9/0s4DJJK0iaes8DvkrlHzcRsTZ93kAS/LM5zN/zvATBY8C09IqCWpJ7I8/NuKa+NBd4dzr9buB/MqylLNL24W8DT0fEl0tWVfSxS2pOzwSQNAi4gKR/5D7g7elmFXfcEfFXETEhIiaT/H/+RUT8ARV+3JIaJA3tnAYuBBZxmL/nuflmsaS3kLQpVgH/HhGfy7ai8pB0M3AuybC0LwJ/C/wYuBWYSDKE9+9FRNcO5QFN0tnAQ8CvebnN+JMk/QQVe+ySTiDpHKwi+cPu1oi4XtJUkr+Um4AngHdGxL7sKi2ftGnoExHx1ko/7vT4bktnq4HvRcTnJI3kMH7PcxMEZmbWvbw0DZmZ2QE4CMzMcs5BYGaWcw4CM7OccxCYmeWcg8ByS9LO9HmypN8/wvv+ZJf5h4/k/s2OJAeBGUwGDikISr69eiCvCIKIeMMh1mTWZxwEZvB54Jx0fPc/Swdx+5KkxyQtlPR+SL64JOkhSXOBJemyH6eDfy3uHABM0ueBQen+vpsu6zz7ULrvRemY8leU7Pt+ST+UtFTSd1U6UJJZGXnQObNk7PZPRMRbAdIP9G0RcZqkOuBXku5Otz0FOD4ink/n3xsRL6XDOzwm6UcRcZ2kD0XESd281++Q3DfgRJJvfz8m6cF03cnATOAF4Fck4+n88kgfrFlXPiMwe7ULgT9Mh3Z+hGR442npukdLQgDgI5KeAuaRDGw4jYM7G7g5Ijoi4kXgAeC0kn2viYgi8CRJk5VZ2fmMwOzVBHw4Iu56xcJkTJtdXebfBJwZEbsl3Q/UH8b7lo6J04H/f1of8RmBGewAhpbM3wX8STqsNZKmpyM9djUM2JKGwLEkt8js1Nb5+i4eAq5I+yGagTnAo0fkKMxeI//FYQYLgY60iec7JOPaTwYeTztsN9L9rf9+BnxA0tPAMyTNQ51uABZKejwdHrnTbST3C3iK5E5T10bE+jRIzDLh0UfNzHLOTUNmZjnnIDAzyzkHgZlZzjkIzMxyzkFgZpZzDgIzs5xzEJiZ5dz/B6cAo6ckkIgfAAAAAElFTkSuQmCC\n",
"text/plain": [
""
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"import matplotlib.pyplot as plt\n",
"plt.plot(errors)\n",
"plt.xlabel('Iteration'); plt.ylabel('MSE(param)'); plt.title('Parameter error plot');\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## See also\n",
"\n",
"- [Detailed look at Optimizer](https://mitsuba.readthedocs.io/en/latest/src/how_to_guides/use_optimizers.html)\n",
"- API reference:\n",
" - [mitsuba.ad.Optimizer](https://mitsuba.readthedocs.io/en/latest/src/api_reference.html#mitsuba.ad.Optimizer)\n",
" - [prb plugin](https://mitsuba.readthedocs.io/en/latest/src/generated/plugins_integrators.html#path-replay-backpropagation-prb)\n",
" - [mitsuba.ad.SGD](https://mitsuba.readthedocs.io/en/latest/src/api_reference.html#mitsuba.ad.SGD)\n",
" - [mitsuba.ad.Adam](https://mitsuba.readthedocs.io/en/latest/src/api_reference.html#mitsuba.ad.Adam)\n",
" - [drjit.backward](https://drjit.readthedocs.io/en/latest/reference.html#drjit.backward)"
]
}
],
"metadata": {
"file_extension": ".py",
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.9.12"
},
"metadata": {
"interpreter": {
"hash": "31f2aee4e71d21fbe5cf8b01ff0e069b9275f58929596ceb00d14d90e3e16cd6"
}
},
"mimetype": "text/x-python",
"name": "python",
"npconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": 3
},
"nbformat": 4,
"nbformat_minor": 4
}