{
"cells": [
{
"cell_type": "markdown",
"id": "17621e06-1633-4942-b692-6e0cd9039fbb",
"metadata": {},
"source": [
"# Notebook 6: Gradient Descent [](https://colab.research.google.com/github/mattsankner/micrograd/blob/main/mg6_gradient_descent.ipynb) [](https://nbviewer.jupyter.org/github/mattsankner/micrograd/blob/main/mg6_gradient_descent.ipynb)"
]
},
{
"cell_type": "markdown",
"id": "a10e6c35-868b-47d8-83e0-e741889cbcee",
"metadata": {},
"source": [
"### Now, we will build a dataset..."
]
},
{
"cell_type": "code",
"execution_count": 10,
"id": "f20503d7-0738-44f1-b736-bd13b1875d17",
"metadata": {},
"outputs": [],
"source": [
"import math\n",
"import numpy as np\n",
"import matplotlib.pyplot as plt\n",
"import torch\n",
"import random"
]
},
{
"cell_type": "code",
"execution_count": 16,
"id": "822945fc-908e-460d-b4a1-edadd3dbdd97",
"metadata": {},
"outputs": [],
"source": [
"from graphviz import Digraph\n",
"\n",
"def trace(root):\n",
" # builds a set of all nodes and edges in a graph\n",
" nodes, edges = set(), set()\n",
" def build(v):\n",
" if v not in nodes:\n",
" nodes.add(v)\n",
" for child in v._prev:\n",
" edges.add((child, v))\n",
" build(child)\n",
" build(root)\n",
" return nodes, edges\n",
"\n",
"def draw_dot(root):\n",
" dot = Digraph(format='svg', graph_attr={'rankdir': 'LR'}) # LR = left to right\n",
" \n",
" nodes, edges = trace(root)\n",
" for n in nodes:\n",
" uid = str(id(n))\n",
" # for any value in the graph, create a rectangular ('record') node for it\n",
" dot.node(name = uid, label = \"{ %s | data %.4f | grad %.4f }\" % (n.label, n.data, n.grad), shape='record')\n",
" if n._op:\n",
" # if this value is a result of some operation, create an op node for it\n",
" dot.node(name = uid + n._op, label = n._op)\n",
" # and connect this node to it\n",
" dot.edge(uid + n._op, uid)\n",
"\n",
" for n1, n2 in edges:\n",
" # connect n1 to the op node of n2\n",
" dot.edge(str(id(n1)), str(id(n2)) + n2._op)\n",
"\n",
" return dot"
]
},
{
"cell_type": "code",
"execution_count": 19,
"id": "6e709cd7-d600-45ca-ba0a-4cbc8b5e637a",
"metadata": {},
"outputs": [],
"source": [
"class Value:\n",
" def __init__(self, data, _children=(), _op='', label=''):\n",
" self.data = data\n",
" self.grad = 0.0\n",
" self._backward = lambda: None\n",
" self._prev = set(_children)\n",
" self._op = _op\n",
" self.label = label\n",
"\n",
" def __repr__(self):\n",
" return f\"Value(data={self.data})\"\n",
"\n",
" def __add__(self, other):\n",
" other = other if isinstance(other, Value) else Value(other)\n",
" out = Value(self.data + other.data, (self, other), '+')\n",
"\n",
" def _backward():\n",
" self.grad += 1.0 * out.grad\n",
" other.grad += 1.0 * out.grad\n",
" out._backward = _backward\n",
"\n",
" return out\n",
"\n",
" def __mul__(self, other):\n",
" other = other if isinstance(other, Value) else Value(other)\n",
" out = Value(self.data * other.data, (self, other), '*')\n",
"\n",
" def _backward():\n",
" self.grad += other.data * out.grad\n",
" other.grad += self.data * out.grad\n",
" out._backward = _backward\n",
"\n",
" return out\n",
"\n",
" def __neg__(self):\n",
" return self * -1\n",
"\n",
" def __sub__(self, other):\n",
" return self + (-other)\n",
"\n",
" def __pow__(self, other):\n",
" assert isinstance(other, (int, float)), \"only supporting int/float powers for now\"\n",
" out = Value(self.data ** other, (self,), f'**{other}')\n",
"\n",
" def _backward():\n",
" self.grad += other * self.data ** (other - 1) * out.grad\n",
" out._backward = _backward\n",
" return out\n",
"\n",
" def __rmul__(self, other):\n",
" return self * other\n",
"\n",
" def __truediv__(self, other):\n",
" return self * other ** -1\n",
"\n",
" def tanh(self):\n",
" x = self.data\n",
" t = (math.exp(2 * x) - 1) / (math.exp(2 * x) + 1)\n",
" out = Value(t, (self,), 'tanh')\n",
"\n",
" def _backward():\n",
" self.grad += (1 - t ** 2) * out.grad\n",
" out._backward = _backward\n",
"\n",
" return out\n",
"\n",
" def exp(self):\n",
" x = self.data\n",
" out = Value(math.exp(x), (self,), 'exp')\n",
"\n",
" def _backward():\n",
" self.grad += out.data * out.grad\n",
" out._backward = _backward\n",
" return out\n",
"\n",
" def backward(self):\n",
" topo = []\n",
" visited = set()\n",
" def build_topo(v):\n",
" if v not in visited:\n",
" visited.add(v)\n",
" for child in v._prev:\n",
" build_topo(child)\n",
" topo.append(v)\n",
" build_topo(self)\n",
"\n",
" self.grad = 1.0\n",
" for node in reversed(topo):\n",
" node._backward()"
]
},
{
"cell_type": "code",
"execution_count": 50,
"id": "213782e6-1496-4a73-9e6f-9e39962bd40a",
"metadata": {},
"outputs": [],
"source": [
"class Neuron:\n",
" def __init__(self,nin): \n",
" self.w = [Value(random.uniform(-1,1)) for _ in range(nin)]\n",
" self.b = Value(random.uniform(-1,1))\n",
"\n",
" def __call__(self,x): \n",
" act = sum((wi*xi for wi, xi in zip(self.w, x)), self.b) \n",
" out = act.tanh()\n",
" return out\n",
"\n",
"class Layer: \n",
" def __init__(self, nin, nout): \n",
" #nin = num inputs to each each neuron in the layer\n",
" #nout = number of neurons in the layer\n",
" self.neurons = [Neuron(nin) for _ in range(nout)]\n",
"\n",
" def __call__(self, x):\n",
" outs = [n(x) for n in self.neurons]\n",
" return outs[0] if len(outs) == 1 else outs \n",
"\n",
"class MLP:\n",
" def __init__(self, nin, nouts):\n",
" sz = [nin] + nouts \n",
" self.layers = [Layer(sz[i], sz[i+1]) for i in range(len(nouts))]\n",
" \n",
" def __call__(self, x):\n",
" for layer in self.layers: \n",
" x = layer(x) \n",
" return x "
]
},
{
"cell_type": "code",
"execution_count": 21,
"id": "909ee94f-9013-4ee0-9524-20e5af9d38ed",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Value(data=-0.5900520440314407)"
]
},
"execution_count": 21,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"x = [2.0, 3.0, -1.0]\n",
"n = MLP(3,[4, 4, 1])\n",
"n(x)"
]
},
{
"cell_type": "markdown",
"id": "d24b420e-6cac-49cc-bce8-c8497d931a3d",
"metadata": {},
"source": [
"Below, we create an example dataset, with four possible inputs into the neural network and four desired targets that we want to shoot for. \n",
"\n",
"We would like the neural net to ouptut $ys[0]$ when its fed first array, $ys[1]$ for second array, etc.\n",
"\n",
"This is a very simple binary classifier neural net."
]
},
{
"cell_type": "code",
"execution_count": 33,
"id": "cae28d6f-144f-4d88-8518-3a882c300004",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Value(data=-0.5900520440314407),\n",
" Value(data=-0.6045680133303286),\n",
" Value(data=-0.5654156624843023),\n",
" Value(data=-0.49584526005845786)]"
]
},
"execution_count": 33,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"xs = [\n",
" [2.0, 3.0, -1.0],\n",
" [3.0, -1.0, 0.5],\n",
" [0.5, 1.0, 1.0],\n",
" [1.0, 1.0, -1.0],\n",
"]\n",
"ys = [1.0, -1.0, -1.0, 1.0] #desired target\n",
"\n",
"ypred = [n(x) for x in xs]\n",
"\n",
"ypred #outputs of neural net on the four examples -> predictions"
]
},
{
"cell_type": "markdown",
"id": "630c1132-3a59-4793-9f0b-46b5370fb6a2",
"metadata": {},
"source": [
"## The loss...\n",
"\n",
"Notice your output for each of the four tensors above. We want to take those values and move it towards the desired targets in ```ys```. The trick to this is to calculate a single number that measures the performance of the neural net. This is how we train the neural net to output the desired targets.\n",
"\n",
"This loss we calculate is something we want to minimize. The loss function we will use today is the ```mean squared error```.\n",
"\n",
"This means we iterate for ```y ground truth``` and ```y output``` in the ```zip``` of ```ys, ypred```. \n",
"\n",
"For each y ground truth and y output in the tuples of ```ys, ypred```, we subtract them and square them to get the difference (how far we are away from our prediction). The square will be good for the loss because it will negate negative gradients, forcing them to be positive so we can evaluate them all against the prediction. This makes it a bell curve. \n",
"\n",
"We could also take the ```abs``` (absolute value) to take away the negative.\n",
"\n",
"In short, our prediction will only be accurate when ```yout == ygt```, and we get $0$. When our prediction is not the target we will get some other number. The more off we are, the greater the loss will be. We want low loss. "
]
},
{
"cell_type": "code",
"execution_count": 30,
"id": "49292775-dfa0-4204-af8c-0eb40915f012",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Value(data=2.5282655027285625),\n",
" Value(data=0.15636645608152316),\n",
" Value(data=0.18886354641395786),\n",
" Value(data=2.2375530420393557)]"
]
},
"execution_count": 30,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"### Mean Squared Error: \n",
"[(yout - ygt) ** 2 for ygt, yout in zip (ys, ypred)]"
]
},
{
"cell_type": "markdown",
"id": "1c06ecd6-5ba5-421b-86ce-d1e69e084c70",
"metadata": {},
"source": [
"### Our final loss will be the sum of the mean squared errors of the MLP's output on each of the tensors. "
]
},
{
"cell_type": "code",
"execution_count": 36,
"id": "3ba38361-11f9-49bd-9db5-dce7857b6c15",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Value(data=5.111048547263399)"
]
},
"execution_count": 36,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# Convert ys elements to Value objects to avoid a later unsupported operand type error in the sum func\n",
"ys_value = [Value(y) for y in ys]\n",
"\n",
"loss = sum([(yout-ygt)**2 for ygt, yout in zip(ys_value,ypred)], Value(0.0))\n",
"#loss = sum((yout - ygt)**2 for ygt, yout in zip(ys, ypred)) would throw an operand type error \n",
"#between ys of type float and ypred of type Value()\n",
"\n",
"loss"
]
},
{
"cell_type": "markdown",
"id": "bce44df4-050a-48d5-95d8-9395e4232159",
"metadata": {},
"source": [
"Now, we want to make every one of our predictions == target so the loss is 0.\n",
"\n",
"\n",
"We minimize the loss with ```loss.backward()```:"
]
},
{
"cell_type": "code",
"execution_count": 37,
"id": "ca15faeb-eab7-4179-b05c-c36bd9fc1c59",
"metadata": {},
"outputs": [],
"source": [
"loss.backward() #here's where the magic happens"
]
},
{
"cell_type": "code",
"execution_count": 39,
"id": "580a401b-9eff-4878-8495-43fa4e2e4eb9",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"-3.2376296037647085"
]
},
"execution_count": 39,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#look at a weight's gradient at 0\n",
"n.layers[0].neurons[0].w[0].grad"
]
},
{
"cell_type": "markdown",
"id": "a73e55bc-317d-400b-8719-ad29ff8f4459",
"metadata": {},
"source": [
"We can manually view the gradients of these individual weights in their neurons. Their sign tells us what happens to the loss when we adjust them one way or another. We have this info for every parameter.\n",
"\n",
"If the above gradient is negative, that is telling us that if we decrease the given gradient, the loss will go up. Thus, if we increase the given gradient, the loss will go down.\n",
"\n",
"Further, this means that if a gradient is positive, that's telling us that if we increase the given gradient, the loss will go up. Thus, if we decrease it, the loss goes down. We can visualize what we have:"
]
},
{
"cell_type": "code",
"execution_count": 40,
"id": "2d3e73ef-7fb9-4368-8543-58b3846097aa",
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n",
"\n",
"\n",
"\n",
" \n",
"\n",
"\n",
"4809687120 \n",
" \n",
" \n",
" \n",
"data -0.3494 \n",
" \n",
"grad 0.3022 \n",
" \n",
"\n",
"\n",
"4809695760+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809687120->4809695760+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809687120* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809687120*->4809687120 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810735760 \n",
" \n",
" \n",
" \n",
"data -0.5654 \n",
" \n",
"grad 0.8692 \n",
" \n",
"\n",
"\n",
"4811337808+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810735760->4811337808+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810735760tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810735760tanh->4810735760 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809687184 \n",
" \n",
" \n",
" \n",
"data -0.5761 \n",
" \n",
"grad -0.0569 \n",
" \n",
"\n",
"\n",
"4809688336+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809687184->4809688336+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809687184+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809687184+->4809687184 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809687248 \n",
" \n",
" \n",
" \n",
"data 0.1233 \n",
" \n",
"grad 0.4827 \n",
" \n",
"\n",
"\n",
"4809698320tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4809687248->4809698320tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809687248+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809687248+->4809687248 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809687312 \n",
" \n",
" \n",
" \n",
"data -0.3234 \n",
" \n",
"grad -0.0569 \n",
" \n",
"\n",
"\n",
"4809687312->4809688336+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809687312* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809687312*->4809687312 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810736016 \n",
" \n",
" \n",
" \n",
"data -0.8726 \n",
" \n",
"grad -1.1778 \n",
" \n",
"\n",
"\n",
"4810736976+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810736016->4810736976+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810736016+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810736016+->4810736016 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809687440 \n",
" \n",
" \n",
" \n",
"data -0.0403 \n",
" \n",
"grad -2.0729 \n",
" \n",
"\n",
"\n",
"4809688656+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809687440->4809688656+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809687440* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809687440*->4809687440 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810736080 \n",
" \n",
" \n",
" \n",
"data 1.0000 \n",
" \n",
"grad 0.7034 \n",
" \n",
"\n",
"\n",
"4810736208* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810736080->4810736208* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809687568 \n",
" \n",
" \n",
" \n",
"data 1.6772 \n",
" \n",
"grad -0.0065 \n",
" \n",
"\n",
"\n",
"4809693200+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809687568->4809693200+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809687568* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809687568*->4809687568 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810736208 \n",
" \n",
" \n",
" \n",
"data -0.5972 \n",
" \n",
"grad -1.1778 \n",
" \n",
"\n",
"\n",
"4810736208->4810736016+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810736208*->4810736208 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809687632 \n",
" \n",
" \n",
" \n",
"data -1.2597 \n",
" \n",
"grad -0.0569 \n",
" \n",
"\n",
"\n",
"4809687632->4809687184+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809687632+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809687632+->4809687632 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810736272 \n",
" \n",
" \n",
" \n",
"data -0.1887 \n",
" \n",
"grad -1.6974 \n",
" \n",
"\n",
"\n",
"4810741392+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810736272->4810741392+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810736272* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810736272*->4810736272 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809687696 \n",
" \n",
" \n",
" \n",
"data -0.7161 \n",
" \n",
"grad -0.1168 \n",
" \n",
"\n",
"\n",
"4809687696->4809687440* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809687696tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4809687696tanh->4809687696 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810736400 \n",
" \n",
" \n",
" \n",
"data -0.2222 \n",
" \n",
"grad -1.6974 \n",
" \n",
"\n",
"\n",
"4810740880+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810736400->4810740880+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810736400* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810736400*->4810736400 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807753680 \n",
" \n",
" \n",
" \n",
"data 0.7363 \n",
" \n",
"grad -0.3015 \n",
" \n",
"\n",
"\n",
"4809690960+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4807753680->4809690960+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810742544+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4807753680->4810742544+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665424+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4807753680->4810665424+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810667152+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4807753680->4810667152+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809687824 \n",
" \n",
" \n",
" \n",
"data 0.6836 \n",
" \n",
"grad -0.0569 \n",
" \n",
"\n",
"\n",
"4809687824->4809687184+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809687824* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809687824*->4809687824 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810736464 \n",
" \n",
" \n",
" \n",
"data 0.1335 \n",
" \n",
"grad -1.1778 \n",
" \n",
"\n",
"\n",
"4810736464->4810736976+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810736464* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810736464*->4810736464 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809687952 \n",
" \n",
" \n",
" \n",
"data -0.3921 \n",
" \n",
"grad -0.0569 \n",
" \n",
"\n",
"\n",
"4809694224+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809687952->4809694224+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809687952* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809687952*->4809687952 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810736592 \n",
" \n",
" \n",
" \n",
"data -0.6408 \n",
" \n",
"grad 0.5913 \n",
" \n",
"\n",
"\n",
"4810736592->4810735760tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810736592+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810736592+->4810736592 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688016 \n",
" \n",
" \n",
" \n",
"data 0.5686 \n",
" \n",
"grad 0.7924 \n",
" \n",
"\n",
"\n",
"4809693648+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809688016->4809693648+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688016* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809688016*->4809688016 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688144 \n",
" \n",
" \n",
" \n",
"data 0.1441 \n",
" \n",
"grad -1.6785 \n",
" \n",
"\n",
"\n",
"4809689936+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809688144->4809689936+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688144* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809688144*->4809688144 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688208 \n",
" \n",
" \n",
" \n",
"data 0.3937 \n",
" \n",
"grad -2.0729 \n",
" \n",
"\n",
"\n",
"4809689424+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809688208->4809689424+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688208* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809688208*->4809688208 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688272 \n",
" \n",
" \n",
" \n",
"data -0.0245 \n",
" \n",
"grad -1.6785 \n",
" \n",
"\n",
"\n",
"4809700816+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809688272->4809700816+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688272+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809688272+->4809688272 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688336 \n",
" \n",
" \n",
" \n",
"data -0.8995 \n",
" \n",
"grad -0.0569 \n",
" \n",
"\n",
"\n",
"4809688336->4809687696tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688336+->4809688336 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810736976 \n",
" \n",
" \n",
" \n",
"data -0.7390 \n",
" \n",
"grad -1.1778 \n",
" \n",
"\n",
"\n",
"4810737360+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810736976->4810737360+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810736976+->4810736976 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688400 \n",
" \n",
" \n",
" \n",
"data -1.0000 \n",
" \n",
"grad 0.0022 \n",
" \n",
"\n",
"\n",
"4809693776* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809688400->4809693776* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810737040 \n",
" \n",
" \n",
" \n",
"data 1.0000 \n",
" \n",
"grad -0.1573 \n",
" \n",
"\n",
"\n",
"4810737040->4810736464* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808181136 \n",
" \n",
" \n",
" \n",
"data -0.4026 \n",
" \n",
"grad 2.2809 \n",
" \n",
"\n",
"\n",
"4809689488* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4808181136->4809689488* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810746640* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4808181136->4810746640* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656784* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4808181136->4810656784* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810658768* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4808181136->4810658768* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688464 \n",
" \n",
" \n",
" \n",
"data 0.0673 \n",
" \n",
"grad -0.0569 \n",
" \n",
"\n",
"\n",
"4809688464->4809687632+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688464* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809688464*->4809688464 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810737104 \n",
" \n",
" \n",
" \n",
"data -1.0000 \n",
" \n",
"grad 0.5061 \n",
" \n",
"\n",
"\n",
"4810737232* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810737104->4810737232* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688592 \n",
" \n",
" \n",
" \n",
"data 1.1487 \n",
" \n",
"grad -0.0053 \n",
" \n",
"\n",
"\n",
"4809694288tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4809688592->4809694288tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688592+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809688592+->4809688592 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810737232 \n",
" \n",
" \n",
" \n",
"data 0.4297 \n",
" \n",
"grad -1.1778 \n",
" \n",
"\n",
"\n",
"4810737232->4810737360+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810737232*->4810737232 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688656 \n",
" \n",
" \n",
" \n",
"data -0.7682 \n",
" \n",
"grad -2.0729 \n",
" \n",
"\n",
"\n",
"4809689040+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809688656->4809689040+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688656+->4809688656 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808640080 \n",
" \n",
" \n",
" \n",
"data -1.0000 \n",
" \n",
"grad -3.1801 \n",
" \n",
"\n",
"\n",
"4809210960* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4808640080->4809210960* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688720 \n",
" \n",
" \n",
" \n",
"data 0.3218 \n",
" \n",
"grad 0.7924 \n",
" \n",
"\n",
"\n",
"4809689168+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809688720->4809689168+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688720* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809688720*->4809688720 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810737360 \n",
" \n",
" \n",
" \n",
"data -0.3093 \n",
" \n",
"grad -1.1778 \n",
" \n",
"\n",
"\n",
"4810737424tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810737360->4810737424tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810737360+->4810737360 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810737424 \n",
" \n",
" \n",
" \n",
"data -0.2998 \n",
" \n",
"grad -1.2941 \n",
" \n",
"\n",
"\n",
"4810737424->4810736400* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810742288* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810737424->4810742288* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810743632* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810737424->4810743632* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810744784* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810737424->4810744784* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810737424tanh->4810737424 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688848 \n",
" \n",
" \n",
" \n",
"data -0.2573 \n",
" \n",
"grad -1.6785 \n",
" \n",
"\n",
"\n",
"4809690064+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809688848->4809690064+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688848* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809688848*->4809688848 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688912 \n",
" \n",
" \n",
" \n",
"data -0.6201 \n",
" \n",
"grad 0.7924 \n",
" \n",
"\n",
"\n",
"4809689552+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809688912->4809689552+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688912* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809688912*->4809688912 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810737552 \n",
" \n",
" \n",
" \n",
"data 1.0000 \n",
" \n",
"grad -0.0986 \n",
" \n",
"\n",
"\n",
"4810737680* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810737552->4810737680* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688976 \n",
" \n",
" \n",
" \n",
"data 0.0070 \n",
" \n",
"grad 0.7924 \n",
" \n",
"\n",
"\n",
"4809697168+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809688976->4809697168+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809688976* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809688976*->4809688976 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809623440 \n",
" \n",
" \n",
" \n",
"data -1.4698 \n",
" \n",
"grad -1.0774 \n",
" \n",
"\n",
"\n",
"4809634320+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809623440->4809634320+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809623440+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809623440+->4809623440 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689040 \n",
" \n",
" \n",
" \n",
"data -0.6777 \n",
" \n",
"grad -2.0729 \n",
" \n",
"\n",
"\n",
"4809689104tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4809689040->4809689104tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689040+->4809689040 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810737680 \n",
" \n",
" \n",
" \n",
"data -0.3885 \n",
" \n",
"grad 0.2537 \n",
" \n",
"\n",
"\n",
"4810737808+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810737680->4810737808+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810737680*->4810737680 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689104 \n",
" \n",
" \n",
" \n",
"data -0.5901 \n",
" \n",
"grad -3.1801 \n",
" \n",
"\n",
"\n",
"4809205200+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809689104->4809205200+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689104tanh->4809689104 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689168 \n",
" \n",
" \n",
" \n",
"data -0.1841 \n",
" \n",
"grad 0.7924 \n",
" \n",
"\n",
"\n",
"4809689168->4809697168+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689168+->4809689168 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810737808 \n",
" \n",
" \n",
" \n",
"data 0.0621 \n",
" \n",
"grad 0.2537 \n",
" \n",
"\n",
"\n",
"4810738128+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810737808->4810738128+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810737808+->4810737808 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689232 \n",
" \n",
" \n",
" \n",
"data 3.0000 \n",
" \n",
"grad 0.0311 \n",
" \n",
"\n",
"\n",
"4809702800* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809689232->4809702800* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810737872 \n",
" \n",
" \n",
" \n",
"data 1.0000 \n",
" \n",
"grad -0.0263 \n",
" \n",
"\n",
"\n",
"4810738000* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810737872->4810738000* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689296 \n",
" \n",
" \n",
" \n",
"data -0.1448 \n",
" \n",
"grad -2.0729 \n",
" \n",
"\n",
"\n",
"4809689680+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809689296->4809689680+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689296* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809689296*->4809689296 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689360 \n",
" \n",
" \n",
" \n",
"data -0.2246 \n",
" \n",
"grad 0.8345 \n",
" \n",
"\n",
"\n",
"4809689360->4809689488* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689360tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4809689360tanh->4809689360 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810738000 \n",
" \n",
" \n",
" \n",
"data -0.1037 \n",
" \n",
"grad 0.2537 \n",
" \n",
"\n",
"\n",
"4810738000->4810738128+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810738000*->4810738000 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689424 \n",
" \n",
" \n",
" \n",
"data -0.7278 \n",
" \n",
"grad -2.0729 \n",
" \n",
"\n",
"\n",
"4809689424->4809688656+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689424+->4809689424 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689488 \n",
" \n",
" \n",
" \n",
"data 0.0904 \n",
" \n",
"grad -2.0729 \n",
" \n",
"\n",
"\n",
"4809689488->4809689040+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689488*->4809689488 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810738128 \n",
" \n",
" \n",
" \n",
"data -0.0416 \n",
" \n",
"grad 0.2537 \n",
" \n",
"\n",
"\n",
"4810738448+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810738128->4810738448+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810738128+->4810738128 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689552 \n",
" \n",
" \n",
" \n",
"data -0.7972 \n",
" \n",
"grad 0.7924 \n",
" \n",
"\n",
"\n",
"4809689552->4809693648+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689552+->4809689552 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810738192 \n",
" \n",
" \n",
" \n",
"data -1.0000 \n",
" \n",
"grad -0.1931 \n",
" \n",
"\n",
"\n",
"4810738320* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810738192->4810738320* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689616 \n",
" \n",
" \n",
" \n",
"data -0.9842 \n",
" \n",
"grad 0.2315 \n",
" \n",
"\n",
"\n",
"4809695888* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809689616->4809695888* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654736* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809689616->4810654736* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655568* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809689616->4810655568* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660112* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809689616->4810660112* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689616tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4809689616tanh->4809689616 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689680 \n",
" \n",
" \n",
" \n",
"data -1.1215 \n",
" \n",
"grad -2.0729 \n",
" \n",
"\n",
"\n",
"4809689680->4809689424+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689680+->4809689680 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810738320 \n",
" \n",
" \n",
" \n",
"data 0.7608 \n",
" \n",
"grad 0.2537 \n",
" \n",
"\n",
"\n",
"4810738320->4810738448+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810738320*->4810738320 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689744 \n",
" \n",
" \n",
" \n",
"data -1.0000 \n",
" \n",
"grad 0.0010 \n",
" \n",
"\n",
"\n",
"4809692176* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809689744->4809692176* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810738448 \n",
" \n",
" \n",
" \n",
"data 0.7192 \n",
" \n",
"grad 0.2537 \n",
" \n",
"\n",
"\n",
"4810738512tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810738448->4810738512tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810738448+->4810738448 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689872 \n",
" \n",
" \n",
" \n",
"data -2.0670 \n",
" \n",
"grad 0.0073 \n",
" \n",
"\n",
"\n",
"4809691920+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809689872->4809691920+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689872+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809689872+->4809689872 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810738512 \n",
" \n",
" \n",
" \n",
"data 0.6164 \n",
" \n",
"grad 0.4093 \n",
" \n",
"\n",
"\n",
"4810738512->4810736272* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810742736* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810738512->4810742736* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810743888* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810738512->4810743888* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810745040* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810738512->4810745040* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810738512tanh->4810738512 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689936 \n",
" \n",
" \n",
" \n",
"data -0.1752 \n",
" \n",
"grad -1.6785 \n",
" \n",
"\n",
"\n",
"4809692752tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4809689936->4809692752tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809689936+->4809689936 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809690000 \n",
" \n",
" \n",
" \n",
"data 3.0000 \n",
" \n",
"grad 0.0502 \n",
" \n",
"\n",
"\n",
"4809691280* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809690000->4809691280* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810738640 \n",
" \n",
" \n",
" \n",
"data 1.0000 \n",
" \n",
"grad 0.0406 \n",
" \n",
"\n",
"\n",
"4810738768* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810738640->4810738768* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809690064 \n",
" \n",
" \n",
" \n",
"data -0.3194 \n",
" \n",
"grad -1.6785 \n",
" \n",
"\n",
"\n",
"4809690064->4809689936+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809690064+->4809690064 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810738768 \n",
" \n",
" \n",
" \n",
"data 0.5591 \n",
" \n",
"grad 0.0726 \n",
" \n",
"\n",
"\n",
"4810738896+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810738768->4810738896+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810738768*->4810738768 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808182864 \n",
" \n",
" \n",
" \n",
"data -0.9767 \n",
" \n",
"grad -3.2360 \n",
" \n",
"\n",
"\n",
"4808182864->4809689680+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810746000+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4808182864->4810746000+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656848+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4808182864->4810656848+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810657552+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4808182864->4810657552+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809690192 \n",
" \n",
" \n",
" \n",
"data -1.7916 \n",
" \n",
"grad 0.0073 \n",
" \n",
"\n",
"\n",
"4809690192->4809689872+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809690192* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809690192*->4809690192 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810738896 \n",
" \n",
" \n",
" \n",
"data 0.5723 \n",
" \n",
"grad 0.0726 \n",
" \n",
"\n",
"\n",
"4810739216+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810738896->4810739216+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810738896+->4810738896 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809690320 \n",
" \n",
" \n",
" \n",
"data 0.5000 \n",
" \n",
"grad -0.0031 \n",
" \n",
"\n",
"\n",
"4809691408* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809690320->4809691408* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810738960 \n",
" \n",
" \n",
" \n",
"data 1.0000 \n",
" \n",
"grad -0.0244 \n",
" \n",
"\n",
"\n",
"4810739088* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810738960->4810739088* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810739088 \n",
" \n",
" \n",
" \n",
"data -0.3363 \n",
" \n",
"grad 0.0726 \n",
" \n",
"\n",
"\n",
"4810739088->4810739216+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810739088*->4810739088 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809690512 \n",
" \n",
" \n",
" \n",
"data 0.7942 \n",
" \n",
"grad -0.0574 \n",
" \n",
"\n",
"\n",
"4809695952+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809690512->4809695952+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809690512* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809690512*->4809690512 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810739216 \n",
" \n",
" \n",
" \n",
"data 0.2361 \n",
" \n",
"grad 0.0726 \n",
" \n",
"\n",
"\n",
"4810739536+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810739216->4810739536+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810739216+->4810739216 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809690640 \n",
" \n",
" \n",
" \n",
"data -1.0000 \n",
" \n",
"grad 0.0456 \n",
" \n",
"\n",
"\n",
"4809690640->4809690512* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810739280 \n",
" \n",
" \n",
" \n",
"data -1.0000 \n",
" \n",
"grad -0.0520 \n",
" \n",
"\n",
"\n",
"4810739408* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810739280->4810739408* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809690704 \n",
" \n",
" \n",
" \n",
"data 3.0000 \n",
" \n",
"grad -0.0043 \n",
" \n",
"\n",
"\n",
"4809690704->4809690192* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809690768 \n",
" \n",
" \n",
" \n",
"data -0.8762 \n",
" \n",
"grad -0.2473 \n",
" \n",
"\n",
"\n",
"4809690768->4809687312* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809690768->4809688016* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809690768->4809688144* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691216* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809690768->4809691216* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809690768tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4809690768tanh->4809690768 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810739408 \n",
" \n",
" \n",
" \n",
"data 0.7172 \n",
" \n",
"grad 0.0726 \n",
" \n",
"\n",
"\n",
"4810739408->4810739536+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810739408*->4810739408 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809690896 \n",
" \n",
" \n",
" \n",
"data 0.9076 \n",
" \n",
"grad -0.8991 \n",
" \n",
"\n",
"\n",
"4809690896->4809688208* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809690896tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4809690896tanh->4809690896 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810739536 \n",
" \n",
" \n",
" \n",
"data 0.9533 \n",
" \n",
"grad 0.0726 \n",
" \n",
"\n",
"\n",
"4810739600tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810739536->4810739600tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810739536+->4810739536 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809690960 \n",
" \n",
" \n",
" \n",
"data 1.2839 \n",
" \n",
"grad -0.1585 \n",
" \n",
"\n",
"\n",
"4809694160+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809690960->4809694160+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809690960+->4809690960 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810739600 \n",
" \n",
" \n",
" \n",
"data 0.7413 \n",
" \n",
"grad 0.1610 \n",
" \n",
"\n",
"\n",
"4810741136* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810739600->4810741136* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810742992* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810739600->4810742992* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810744144* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810739600->4810744144* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810745296* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810739600->4810745296* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810739600tanh->4810739600 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691024 \n",
" \n",
" \n",
" \n",
"data -0.7581 \n",
" \n",
"grad -0.0791 \n",
" \n",
"\n",
"\n",
"4809692048* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809691024->4809692048* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655120* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809691024->4810655120* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655760* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809691024->4810655760* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656208* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809691024->4810656208* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691024tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4809691024tanh->4809691024 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810739728 \n",
" \n",
" \n",
" \n",
"data 1.0000 \n",
" \n",
"grad 0.0069 \n",
" \n",
"\n",
"\n",
"4810739856* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810739728->4810739856* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691152 \n",
" \n",
" \n",
" \n",
"data -2.4153 \n",
" \n",
"grad 0.0073 \n",
" \n",
"\n",
"\n",
"4809691152->4809689616tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691152+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809691152+->4809691152 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691216 \n",
" \n",
" \n",
" \n",
"data 0.0651 \n",
" \n",
"grad -0.1585 \n",
" \n",
"\n",
"\n",
"4809691792+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809691216->4809691792+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691216*->4809691216 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810739856 \n",
" \n",
" \n",
" \n",
"data 0.2028 \n",
" \n",
"grad 0.0342 \n",
" \n",
"\n",
"\n",
"4810739984+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810739856->4810739984+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810739856*->4810739856 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691280 \n",
" \n",
" \n",
" \n",
"data -2.6223 \n",
" \n",
"grad -0.0574 \n",
" \n",
"\n",
"\n",
"4809695312+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809691280->4809695312+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691280*->4809691280 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810739984 \n",
" \n",
" \n",
" \n",
"data 0.2661 \n",
" \n",
"grad 0.0342 \n",
" \n",
"\n",
"\n",
"4810740304+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810739984->4810740304+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810739984+->4810739984 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691408 \n",
" \n",
" \n",
" \n",
"data -0.2149 \n",
" \n",
"grad 0.0073 \n",
" \n",
"\n",
"\n",
"4809691408->4809691152+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691408*->4809691408 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808184144 \n",
" \n",
" \n",
" \n",
"data 0.8349 \n",
" \n",
"grad 1.0437 \n",
" \n",
"\n",
"\n",
"4808184144->4809689296* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810747088* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4808184144->4810747088* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655440* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4808184144->4810655440* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662416* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4808184144->4810662416* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810740048 \n",
" \n",
" \n",
" \n",
"data 1.0000 \n",
" \n",
"grad -0.0299 \n",
" \n",
"\n",
"\n",
"4810740176* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810740048->4810740176* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691536 \n",
" \n",
" \n",
" \n",
"data -1.1656 \n",
" \n",
"grad -0.0337 \n",
" \n",
"\n",
"\n",
"4809692560+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809691536->4809692560+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691536* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809691536*->4809691536 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810740176 \n",
" \n",
" \n",
" \n",
"data -0.8741 \n",
" \n",
"grad 0.0342 \n",
" \n",
"\n",
"\n",
"4810740176->4810740304+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810740176*->4810740176 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691664 \n",
" \n",
" \n",
" \n",
"data 0.5477 \n",
" \n",
"grad -0.1585 \n",
" \n",
"\n",
"\n",
"4809691664->4809690960+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691664* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809691664*->4809691664 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810740304 \n",
" \n",
" \n",
" \n",
"data -0.6080 \n",
" \n",
"grad 0.0342 \n",
" \n",
"\n",
"\n",
"4810740624+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810740304->4810740624+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810740304+->4810740304 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691728 \n",
" \n",
" \n",
" \n",
"data 1.5458 \n",
" \n",
"grad -0.0053 \n",
" \n",
"\n",
"\n",
"4809691728->4809688592+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691728+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809691728+->4809691728 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810740368 \n",
" \n",
" \n",
" \n",
"data -1.0000 \n",
" \n",
"grad -0.0272 \n",
" \n",
"\n",
"\n",
"4810740496* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810740368->4810740496* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809626256 \n",
" \n",
" \n",
" \n",
"data 3.0000 \n",
" \n",
"grad -0.1439 \n",
" \n",
"\n",
"\n",
"4809633488* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809626256->4809633488* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691792 \n",
" \n",
" \n",
" \n",
"data 1.5137 \n",
" \n",
"grad -0.1585 \n",
" \n",
"\n",
"\n",
"4809691792->4809690896tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691792+->4809691792 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810740496 \n",
" \n",
" \n",
" \n",
"data 0.7942 \n",
" \n",
"grad 0.0342 \n",
" \n",
"\n",
"\n",
"4810740496->4810740624+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810740496*->4810740496 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691920 \n",
" \n",
" \n",
" \n",
"data -2.2005 \n",
" \n",
"grad 0.0073 \n",
" \n",
"\n",
"\n",
"4809691920->4809691152+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809691920+->4809691920 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810740624 \n",
" \n",
" \n",
" \n",
"data 0.1861 \n",
" \n",
"grad 0.0342 \n",
" \n",
"\n",
"\n",
"4810740688tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810740624->4810740688tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810740624+->4810740624 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809692048 \n",
" \n",
" \n",
" \n",
"data 0.2321 \n",
" \n",
"grad 0.3022 \n",
" \n",
"\n",
"\n",
"4809692880+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809692048->4809692880+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809692048*->4809692048 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810740688 \n",
" \n",
" \n",
" \n",
"data 0.1840 \n",
" \n",
"grad 0.0354 \n",
" \n",
"\n",
"\n",
"4810741648* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810740688->4810741648* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810743248* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810740688->4810743248* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810744400* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810740688->4810744400* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810745552* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810740688->4810745552* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810740688tanh->4810740688 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4804744208 \n",
" \n",
" \n",
" \n",
"data 0.5484 \n",
" \n",
"grad -0.0976 \n",
" \n",
"\n",
"\n",
"4804744208->4809688464* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4804744208->4810743888* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4804744208->4810655120* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810667536* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4804744208->4810667536* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809692176 \n",
" \n",
" \n",
" \n",
"data -0.1335 \n",
" \n",
"grad 0.0073 \n",
" \n",
"\n",
"\n",
"4809692176->4809691920+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809692176*->4809692176 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4804744272 \n",
" \n",
" \n",
" \n",
"data 0.9969 \n",
" \n",
"grad -0.1190 \n",
" \n",
"\n",
"\n",
"4804744272->4809687824* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4804744272->4810744144* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810653968* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4804744272->4810653968* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810667856* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4804744272->4810667856* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809692240 \n",
" \n",
" \n",
" \n",
"data 3.0000 \n",
" \n",
"grad 0.0131 \n",
" \n",
"\n",
"\n",
"4809692240->4809691536* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810740880 \n",
" \n",
" \n",
" \n",
"data 0.1717 \n",
" \n",
"grad -1.6974 \n",
" \n",
"\n",
"\n",
"4810740880->4810741392+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810740880+->4810740880 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809692368 \n",
" \n",
" \n",
" \n",
"data 0.1037 \n",
" \n",
"grad -0.0337 \n",
" \n",
"\n",
"\n",
"4809692432+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809692368->4809692432+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809692368* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809692368*->4809692368 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809692432 \n",
" \n",
" \n",
" \n",
"data -0.6113 \n",
" \n",
"grad -0.0337 \n",
" \n",
"\n",
"\n",
"4809692688+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809692432->4809692688+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809692432+->4809692432 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810741136 \n",
" \n",
" \n",
" \n",
"data -0.2781 \n",
" \n",
"grad -1.6974 \n",
" \n",
"\n",
"\n",
"4810741328+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810741136->4810741328+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810741136*->4810741136 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809692560 \n",
" \n",
" \n",
" \n",
"data -0.7150 \n",
" \n",
"grad -0.0337 \n",
" \n",
"\n",
"\n",
"4809692560->4809692432+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809692560+->4809692560 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809692624 \n",
" \n",
" \n",
" \n",
"data 0.5000 \n",
" \n",
"grad 0.0256 \n",
" \n",
"\n",
"\n",
"4809693392* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809692624->4809693392* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809692688 \n",
" \n",
" \n",
" \n",
"data -0.9917 \n",
" \n",
"grad -0.0337 \n",
" \n",
"\n",
"\n",
"4809692688->4809691024tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809692688+->4809692688 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810741328 \n",
" \n",
" \n",
" \n",
"data -0.2952 \n",
" \n",
"grad -1.6974 \n",
" \n",
"\n",
"\n",
"4810741840+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810741328->4810741840+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810741328+->4810741328 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809102928 \n",
" \n",
" \n",
" \n",
"data -1.1944 \n",
" \n",
"grad -1.0774 \n",
" \n",
"\n",
"\n",
"4809102928->4809623440+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809102928* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809102928*->4809102928 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809692752 \n",
" \n",
" \n",
" \n",
"data -0.1735 \n",
" \n",
"grad -1.7306 \n",
" \n",
"\n",
"\n",
"4809692752->4809689296* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809692752tanh->4809692752 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810741392 \n",
" \n",
" \n",
" \n",
"data -0.0171 \n",
" \n",
"grad -1.6974 \n",
" \n",
"\n",
"\n",
"4810741392->4810741328+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810741392+->4810741392 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809692816 \n",
" \n",
" \n",
" \n",
"data -1.0000 \n",
" \n",
"grad 0.0035 \n",
" \n",
"\n",
"\n",
"4809692816->4809692368* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809692880 \n",
" \n",
" \n",
" \n",
"data -0.1034 \n",
" \n",
"grad 0.3022 \n",
" \n",
"\n",
"\n",
"4809692880->4809695760+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809692880+->4809692880 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810741648 \n",
" \n",
" \n",
" \n",
"data -0.0303 \n",
" \n",
"grad -1.6974 \n",
" \n",
"\n",
"\n",
"4810741648->4810741840+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810741648*->4810741648 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809693200 \n",
" \n",
" \n",
" \n",
"data 1.6905 \n",
" \n",
"grad -0.0065 \n",
" \n",
"\n",
"\n",
"4809693840+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809693200->4809693840+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809693200+->4809693200 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810741840 \n",
" \n",
" \n",
" \n",
"data -0.3255 \n",
" \n",
"grad -1.6974 \n",
" \n",
"\n",
"\n",
"4810742096tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810741840->4810742096tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810741840+->4810741840 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809693328 \n",
" \n",
" \n",
" \n",
"data 3.0000 \n",
" \n",
"grad -0.0037 \n",
" \n",
"\n",
"\n",
"4809693328->4809687568* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809693392 \n",
" \n",
" \n",
" \n",
"data -0.3804 \n",
" \n",
"grad -0.0337 \n",
" \n",
"\n",
"\n",
"4809693392->4809692688+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809693392*->4809693392 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809693456 \n",
" \n",
" \n",
" \n",
"data 0.9313 \n",
" \n",
"grad -0.0493 \n",
" \n",
"\n",
"\n",
"4809693456->4809687120* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810653840* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809693456->4810653840* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809693456->4810653968* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655376* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809693456->4810655376* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809693456tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4809693456tanh->4809693456 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810742096 \n",
" \n",
" \n",
" \n",
"data -0.3144 \n",
" \n",
"grad -1.8836 \n",
" \n",
"\n",
"\n",
"4810742096->4810747088* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810742096tanh->4810742096 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809693584 \n",
" \n",
" \n",
" \n",
"data 0.0295 \n",
" \n",
"grad -0.1585 \n",
" \n",
"\n",
"\n",
"4809693584->4809694160+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809693584* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809693584*->4809693584 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809693648 \n",
" \n",
" \n",
" \n",
"data -0.2285 \n",
" \n",
"grad 0.7924 \n",
" \n",
"\n",
"\n",
"4809693648->4809689360tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809693648+->4809693648 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810742288 \n",
" \n",
" \n",
" \n",
"data 0.2909 \n",
" \n",
"grad -0.2486 \n",
" \n",
"\n",
"\n",
"4810742288->4810742544+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810742288*->4810742288 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809693712 \n",
" \n",
" \n",
" \n",
"data 1.4485 \n",
" \n",
"grad -0.1585 \n",
" \n",
"\n",
"\n",
"4809693712->4809691792+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809693712+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809693712+->4809693712 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809693776 \n",
" \n",
" \n",
" \n",
"data 0.3363 \n",
" \n",
"grad -0.0065 \n",
" \n",
"\n",
"\n",
"4809693776->4809693840+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809693776*->4809693776 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809693840 \n",
" \n",
" \n",
" \n",
"data 2.0267 \n",
" \n",
"grad -0.0065 \n",
" \n",
"\n",
"\n",
"4809696144+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809693840->4809696144+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809693840+->4809693840 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810742544 \n",
" \n",
" \n",
" \n",
"data 1.0271 \n",
" \n",
"grad -0.2486 \n",
" \n",
"\n",
"\n",
"4810742864+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810742544->4810742864+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810742544+->4810742544 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809694096 \n",
" \n",
" \n",
" \n",
"data 0.1352 \n",
" \n",
"grad -0.1585 \n",
" \n",
"\n",
"\n",
"4809694096->4809693712+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809694096* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809694096*->4809694096 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810742736 \n",
" \n",
" \n",
" \n",
"data 0.1481 \n",
" \n",
"grad -0.2486 \n",
" \n",
"\n",
"\n",
"4810742736->4810742864+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810742736*->4810742736 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808186832 \n",
" \n",
" \n",
" \n",
"data -0.5059 \n",
" \n",
"grad 0.9821 \n",
" \n",
"\n",
"\n",
"4808186832->4809689168+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810744912+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4808186832->4810744912+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655696+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4808186832->4810655696+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665360+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4808186832->4810665360+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809694160 \n",
" \n",
" \n",
" \n",
"data 1.3134 \n",
" \n",
"grad -0.1585 \n",
" \n",
"\n",
"\n",
"4809694160->4809693712+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809694160+->4809694160 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809694224 \n",
" \n",
" \n",
" \n",
"data -1.3270 \n",
" \n",
"grad -0.0569 \n",
" \n",
"\n",
"\n",
"4809694224->4809687632+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809694224+->4809694224 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810742864 \n",
" \n",
" \n",
" \n",
"data 1.1752 \n",
" \n",
"grad -0.2486 \n",
" \n",
"\n",
"\n",
"4810743120+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810742864->4810743120+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810742864+->4810742864 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808186960 \n",
" \n",
" \n",
" \n",
"data -0.6490 \n",
" \n",
"grad -0.5851 \n",
" \n",
"\n",
"\n",
"4808186960->4809688016* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808186960->4810745552* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655952* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4808186960->4810655952* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810668496* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4808186960->4810668496* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809694288 \n",
" \n",
" \n",
" \n",
"data 0.8173 \n",
" \n",
"grad -0.0159 \n",
" \n",
"\n",
"\n",
"4810653904* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809694288->4810653904* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655184* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809694288->4810655184* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809694288->4810655952* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656144* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809694288->4810656144* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809694288tanh->4809694288 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810742992 \n",
" \n",
" \n",
" \n",
"data 0.1461 \n",
" \n",
"grad -0.2486 \n",
" \n",
"\n",
"\n",
"4810742992->4810743120+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810742992*->4810742992 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811332880 \n",
" \n",
" \n",
" \n",
"data -1.0000 \n",
" \n",
"grad -0.7909 \n",
" \n",
"\n",
"\n",
"4811333840* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4811332880->4811333840* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810743120 \n",
" \n",
" \n",
" \n",
"data 1.3213 \n",
" \n",
"grad -0.2486 \n",
" \n",
"\n",
"\n",
"4810743376+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810743120->4810743376+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810743120+->4810743120 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811333008 \n",
" \n",
" \n",
" \n",
"data 1.0000 \n",
" \n",
"grad 3.1801 \n",
" \n",
"\n",
"\n",
"4811333008->4809210960* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810743248 \n",
" \n",
" \n",
" \n",
"data -0.0137 \n",
" \n",
"grad -0.2486 \n",
" \n",
"\n",
"\n",
"4810743248->4810743376+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810743248*->4810743248 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809694672 \n",
" \n",
" \n",
" \n",
"data 0.5000 \n",
" \n",
"grad 0.0047 \n",
" \n",
"\n",
"\n",
"4809694736* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809694672->4809694736* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811333136 \n",
" \n",
" \n",
" \n",
"data -1.0000 \n",
" \n",
"grad -0.8692 \n",
" \n",
"\n",
"\n",
"4811338192* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4811333136->4811338192* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809694736 \n",
" \n",
" \n",
" \n",
"data -0.3586 \n",
" \n",
"grad -0.0065 \n",
" \n",
"\n",
"\n",
"4809694736->4809696144+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809694736*->4809694736 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810743376 \n",
" \n",
" \n",
" \n",
"data 1.3076 \n",
" \n",
"grad -0.2486 \n",
" \n",
"\n",
"\n",
"4810743440tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810743376->4810743440tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810743376+->4810743376 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810743440 \n",
" \n",
" \n",
" \n",
"data 0.8637 \n",
" \n",
"grad -0.9786 \n",
" \n",
"\n",
"\n",
"4810746128* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810743440->4810746128* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810743440tanh->4810743440 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810743632 \n",
" \n",
" \n",
" \n",
"data -0.2082 \n",
" \n",
"grad -0.1271 \n",
" \n",
"\n",
"\n",
"4810743760+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810743632->4810743760+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810743632*->4810743632 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810743760 \n",
" \n",
" \n",
" \n",
"data -1.1432 \n",
" \n",
"grad -0.1271 \n",
" \n",
"\n",
"\n",
"4810744016+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810743760->4810744016+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810743760+->4810743760 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811333648 \n",
" \n",
" \n",
" \n",
"data -1.0000 \n",
" \n",
"grad -0.7909 \n",
" \n",
"\n",
"\n",
"4811333648->4811333840* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810743888 \n",
" \n",
" \n",
" \n",
"data 0.3380 \n",
" \n",
"grad -0.1271 \n",
" \n",
"\n",
"\n",
"4810743888->4810744016+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810743888*->4810743888 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809695312 \n",
" \n",
" \n",
" \n",
"data -2.1534 \n",
" \n",
"grad -0.0574 \n",
" \n",
"\n",
"\n",
"4809695312->4809695952+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809695312+->4809695312 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810744016 \n",
" \n",
" \n",
" \n",
"data -0.8051 \n",
" \n",
"grad -0.1271 \n",
" \n",
"\n",
"\n",
"4810744272+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810744016->4810744272+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810744016+->4810744016 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811333840 \n",
" \n",
" \n",
" \n",
"data 1.0000 \n",
" \n",
"grad 0.7909 \n",
" \n",
"\n",
"\n",
"4811337424+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4811333840->4811337424+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811333840*->4811333840 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810744144 \n",
" \n",
" \n",
" \n",
"data 0.7390 \n",
" \n",
"grad -0.1271 \n",
" \n",
"\n",
"\n",
"4810744144->4810744272+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810744144*->4810744144 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809695568 \n",
" \n",
" \n",
" \n",
"data 0.6717 \n",
" \n",
"grad -0.0053 \n",
" \n",
"\n",
"\n",
"4809695568->4809691728+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809695568+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809695568+->4809695568 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810744272 \n",
" \n",
" \n",
" \n",
"data -0.0662 \n",
" \n",
"grad -0.1271 \n",
" \n",
"\n",
"\n",
"4810744528+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810744272->4810744528+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810744272+->4810744272 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809695696 \n",
" \n",
" \n",
" \n",
"data -0.4184 \n",
" \n",
"grad -1.6785 \n",
" \n",
"\n",
"\n",
"4809695696->4809688272+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809695696* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809695696*->4809695696 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809695760 \n",
" \n",
" \n",
" \n",
"data -0.4529 \n",
" \n",
"grad 0.3022 \n",
" \n",
"\n",
"\n",
"4810660560+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809695760->4810660560+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809695760+->4809695760 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810744400 \n",
" \n",
" \n",
" \n",
"data 0.0679 \n",
" \n",
"grad -0.1271 \n",
" \n",
"\n",
"\n",
"4810744400->4810744528+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810744400*->4810744400 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809695888 \n",
" \n",
" \n",
" \n",
"data -0.7294 \n",
" \n",
"grad 0.3022 \n",
" \n",
"\n",
"\n",
"4809703376+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809695888->4809703376+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809695888*->4809695888 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810744528 \n",
" \n",
" \n",
" \n",
"data 0.0018 \n",
" \n",
"grad -0.1271 \n",
" \n",
"\n",
"\n",
"4810744592tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810744528->4810744592tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810744528+->4810744528 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809695952 \n",
" \n",
" \n",
" \n",
"data -1.3593 \n",
" \n",
"grad -0.0574 \n",
" \n",
"\n",
"\n",
"4809695952->4809690768tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809695952+->4809695952 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810744592 \n",
" \n",
" \n",
" \n",
"data 0.0018 \n",
" \n",
"grad -0.1271 \n",
" \n",
"\n",
"\n",
"4810746448* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810744592->4810746448* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810744592tanh->4810744592 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809696144 \n",
" \n",
" \n",
" \n",
"data 1.6681 \n",
" \n",
"grad -0.0065 \n",
" \n",
"\n",
"\n",
"4809696144->4809693456tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809696144+->4809696144 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810744784 \n",
" \n",
" \n",
" \n",
"data 0.1709 \n",
" \n",
"grad 0.3319 \n",
" \n",
"\n",
"\n",
"4810744784->4810744912+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810744784*->4810744784 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810744912 \n",
" \n",
" \n",
" \n",
"data -0.3350 \n",
" \n",
"grad 0.3319 \n",
" \n",
"\n",
"\n",
"4810745168+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810744912->4810745168+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810744912+->4810744912 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809696400 \n",
" \n",
" \n",
" \n",
"data 3.0000 \n",
" \n",
"grad -0.0011 \n",
" \n",
"\n",
"\n",
"4809696656* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809696400->4809696656* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810745040 \n",
" \n",
" \n",
" \n",
"data 0.0354 \n",
" \n",
"grad 0.3319 \n",
" \n",
"\n",
"\n",
"4810745040->4810745168+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810745040*->4810745040 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810745168 \n",
" \n",
" \n",
" \n",
"data -0.2995 \n",
" \n",
"grad 0.3319 \n",
" \n",
"\n",
"\n",
"4810745424+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810745168->4810745424+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810745168+->4810745168 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809696656 \n",
" \n",
" \n",
" \n",
"data 0.6084 \n",
" \n",
"grad -0.0053 \n",
" \n",
"\n",
"\n",
"4809696656->4809695568+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809696656*->4809696656 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810745296 \n",
" \n",
" \n",
" \n",
"data -0.6704 \n",
" \n",
"grad 0.3319 \n",
" \n",
"\n",
"\n",
"4810745296->4810745424+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810745296*->4810745296 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809696720 \n",
" \n",
" \n",
" \n",
"data -1.0000 \n",
" \n",
"grad 0.0046 \n",
" \n",
"\n",
"\n",
"4809697808* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809696720->4809697808* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809205200 \n",
" \n",
" \n",
" \n",
"data -1.5901 \n",
" \n",
"grad -3.1801 \n",
" \n",
"\n",
"\n",
"4811335184**2 \n",
"\n",
"**2 \n",
" \n",
"\n",
"\n",
"4809205200->4811335184**2 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809205200+->4809205200 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811335184 \n",
" \n",
" \n",
" \n",
"data 2.5283 \n",
" \n",
"grad 1.0000 \n",
" \n",
"\n",
"\n",
"4811339088+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4811335184->4811339088+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811335184**2->4811335184 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810745424 \n",
" \n",
" \n",
" \n",
"data -0.9699 \n",
" \n",
"grad 0.3319 \n",
" \n",
"\n",
"\n",
"4810745680+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810745424->4810745680+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810745424+->4810745424 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810745552 \n",
" \n",
" \n",
" \n",
"data -0.1194 \n",
" \n",
"grad 0.3319 \n",
" \n",
"\n",
"\n",
"4810745552->4810745680+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810745552*->4810745552 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810745680 \n",
" \n",
" \n",
" \n",
"data -1.0893 \n",
" \n",
"grad 0.3319 \n",
" \n",
"\n",
"\n",
"4810745744tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810745680->4810745744tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810745680+->4810745680 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810745744 \n",
" \n",
" \n",
" \n",
"data -0.7966 \n",
" \n",
"grad 0.9083 \n",
" \n",
"\n",
"\n",
"4810745744->4810746640* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810745744tanh->4810745744 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809697168 \n",
" \n",
" \n",
" \n",
"data -0.1770 \n",
" \n",
"grad 0.7924 \n",
" \n",
"\n",
"\n",
"4809697168->4809689552+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809697168+->4809697168 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808157072 \n",
" \n",
" \n",
" \n",
"data -0.5645 \n",
" \n",
"grad -1.5814 \n",
" \n",
"\n",
"\n",
"4808157072->4809687952* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808157072->4809688720* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808157072->4809691664* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808157072->4809695696* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808157072tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4808157072tanh->4808157072 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809697232 \n",
" \n",
" \n",
" \n",
"data 2.0000 \n",
" \n",
"grad -0.0518 \n",
" \n",
"\n",
"\n",
"4809702224* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809697232->4809702224* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809697296 \n",
" \n",
" \n",
" \n",
"data -0.0375 \n",
" \n",
"grad -1.6785 \n",
" \n",
"\n",
"\n",
"4809697296->4809700816+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809697296* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809697296*->4809697296 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810746000 \n",
" \n",
" \n",
" \n",
"data -1.2392 \n",
" \n",
"grad -2.2561 \n",
" \n",
"\n",
"\n",
"4810746320+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810746000->4810746320+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810746000+->4810746000 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810746128 \n",
" \n",
" \n",
" \n",
"data 0.3746 \n",
" \n",
"grad -2.2561 \n",
" \n",
"\n",
"\n",
"4810746128->4810746320+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810746128*->4810746128 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809697552 \n",
" \n",
" \n",
" \n",
"data 0.4297 \n",
" \n",
"grad -1.0774 \n",
" \n",
"\n",
"\n",
"4809701200+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809697552->4809701200+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809697552* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809697552*->4809697552 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811336144 \n",
" \n",
" \n",
" \n",
"data 5.1110 \n",
" \n",
"grad 1.0000 \n",
" \n",
"\n",
"\n",
"4811336144+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4811336144+->4811336144 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810746320 \n",
" \n",
" \n",
" \n",
"data -0.8646 \n",
" \n",
"grad -2.2561 \n",
" \n",
"\n",
"\n",
"4810746512+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810746320->4810746512+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810746320+->4810746320 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809697808 \n",
" \n",
" \n",
" \n",
"data 0.8741 \n",
" \n",
"grad -0.0053 \n",
" \n",
"\n",
"\n",
"4809697808->4809691728+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809697808*->4809697808 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810746448 \n",
" \n",
" \n",
" \n",
"data 0.0001 \n",
" \n",
"grad -2.2561 \n",
" \n",
"\n",
"\n",
"4810746448->4810746512+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810746448*->4810746448 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810746512 \n",
" \n",
" \n",
" \n",
"data -0.8645 \n",
" \n",
"grad -2.2561 \n",
" \n",
"\n",
"\n",
"4810747472+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810746512->4810747472+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810746512+->4810746512 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810746640 \n",
" \n",
" \n",
" \n",
"data 0.3207 \n",
" \n",
"grad -2.2561 \n",
" \n",
"\n",
"\n",
"4810746640->4810747472+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810746640*->4810746640 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808190864 \n",
" \n",
" \n",
" \n",
"data 0.4337 \n",
" \n",
"grad -2.8673 \n",
" \n",
"\n",
"\n",
"4808190864->4809688208* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808190864->4810746128* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656400* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4808190864->4810656400* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662352* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4808190864->4810662352* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809698320 \n",
" \n",
" \n",
" \n",
"data 0.1226 \n",
" \n",
"grad 0.4901 \n",
" \n",
"\n",
"\n",
"4809698320->4809688464* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809698320->4809688976* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809698320->4809693584* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809698320->4809697296* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809698320tanh->4809698320 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809698384 \n",
" \n",
" \n",
" \n",
"data -1.0000 \n",
" \n",
"grad 0.4630 \n",
" \n",
"\n",
"\n",
"4809698384->4809697552* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810747088 \n",
" \n",
" \n",
" \n",
"data -0.2625 \n",
" \n",
"grad -2.2561 \n",
" \n",
"\n",
"\n",
"4810747088->4810746000+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810747088*->4810747088 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810747472 \n",
" \n",
" \n",
" \n",
"data -0.5438 \n",
" \n",
"grad -2.2561 \n",
" \n",
"\n",
"\n",
"4810747536tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810747472->4810747536tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810747472+->4810747472 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811337296 \n",
" \n",
" \n",
" \n",
"data 1.0000 \n",
" \n",
"grad 2.9917 \n",
" \n",
"\n",
"\n",
"4811338512* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4811337296->4811338512* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810747536 \n",
" \n",
" \n",
" \n",
"data -0.4958 \n",
" \n",
"grad -2.9917 \n",
" \n",
"\n",
"\n",
"4811338576+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810747536->4811338576+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810747536tanh->4810747536 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811337424 \n",
" \n",
" \n",
" \n",
"data 0.3954 \n",
" \n",
"grad 0.7909 \n",
" \n",
"\n",
"\n",
"4811337488**2 \n",
"\n",
"**2 \n",
" \n",
"\n",
"\n",
"4811337424->4811337488**2 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811337424+->4811337424 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809633488 \n",
" \n",
" \n",
" \n",
"data 0.4006 \n",
" \n",
"grad -1.0774 \n",
" \n",
"\n",
"\n",
"4809633488->4809634320+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809633488*->4809633488 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811337488 \n",
" \n",
" \n",
" \n",
"data 0.1564 \n",
" \n",
"grad 1.0000 \n",
" \n",
"\n",
"\n",
"4811339216+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4811337488->4811339216+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811337488**2->4811337488 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808192080 \n",
" \n",
" \n",
" \n",
"data 0.0563 \n",
" \n",
"grad 0.5601 \n",
" \n",
"\n",
"\n",
"4808192080->4809687440* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808192080->4810746448* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654352* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4808192080->4810654352* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810657040* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4808192080->4810657040* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811337808 \n",
" \n",
" \n",
" \n",
"data 0.4346 \n",
" \n",
"grad 0.8692 \n",
" \n",
"\n",
"\n",
"4811338256**2 \n",
"\n",
"**2 \n",
" \n",
"\n",
"\n",
"4811337808->4811338256**2 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811337808+->4811337808 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811338128 \n",
" \n",
" \n",
" \n",
"data -1.0000 \n",
" \n",
"grad -0.8692 \n",
" \n",
"\n",
"\n",
"4811338128->4811338192* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811338192 \n",
" \n",
" \n",
" \n",
"data 1.0000 \n",
" \n",
"grad 0.8692 \n",
" \n",
"\n",
"\n",
"4811338192->4811337808+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811338192*->4811338192 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811338256 \n",
" \n",
" \n",
" \n",
"data 0.1889 \n",
" \n",
"grad 1.0000 \n",
" \n",
"\n",
"\n",
"4811338832+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4811338256->4811338832+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811338256**2->4811338256 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809634320 \n",
" \n",
" \n",
" \n",
"data -1.0692 \n",
" \n",
"grad -1.0774 \n",
" \n",
"\n",
"\n",
"4809634320->4809701200+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809634320+->4809634320 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811338448 \n",
" \n",
" \n",
" \n",
"data -1.0000 \n",
" \n",
"grad -2.9917 \n",
" \n",
"\n",
"\n",
"4811338448->4811338512* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811338512 \n",
" \n",
" \n",
" \n",
"data -1.0000 \n",
" \n",
"grad -2.9917 \n",
" \n",
"\n",
"\n",
"4811338512->4811338576+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811338512*->4811338512 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811338576 \n",
" \n",
" \n",
" \n",
"data -1.4958 \n",
" \n",
"grad -2.9917 \n",
" \n",
"\n",
"\n",
"4811338704**2 \n",
"\n",
"**2 \n",
" \n",
"\n",
"\n",
"4811338576->4811338704**2 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811338576+->4811338576 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811338704 \n",
" \n",
" \n",
" \n",
"data 2.2376 \n",
" \n",
"grad 1.0000 \n",
" \n",
"\n",
"\n",
"4811338704->4811336144+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811338704**2->4811338704 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811338832 \n",
" \n",
" \n",
" \n",
"data 2.8735 \n",
" \n",
"grad 1.0000 \n",
" \n",
"\n",
"\n",
"4811338832->4811336144+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811338832+->4811338832 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811339088 \n",
" \n",
" \n",
" \n",
"data 2.5283 \n",
" \n",
"grad 1.0000 \n",
" \n",
"\n",
"\n",
"4811339088->4811339216+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811339088+->4811339088 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809700688 \n",
" \n",
" \n",
" \n",
"data 0.8399 \n",
" \n",
"grad -0.0926 \n",
" \n",
"\n",
"\n",
"4809702416tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4809700688->4809702416tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809700688+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809700688+->4809700688 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811339216 \n",
" \n",
" \n",
" \n",
"data 2.6846 \n",
" \n",
"grad 1.0000 \n",
" \n",
"\n",
"\n",
"4811339216->4811338832+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4811339216+->4811339216 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809700816 \n",
" \n",
" \n",
" \n",
"data -0.0621 \n",
" \n",
"grad -1.6785 \n",
" \n",
"\n",
"\n",
"4809700816->4809690064+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809700816+->4809700816 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809700880 \n",
" \n",
" \n",
" \n",
"data -0.5280 \n",
" \n",
"grad 0.4189 \n",
" \n",
"\n",
"\n",
"4809700880->4810655440* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809700880tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4809700880tanh->4809700880 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809701072 \n",
" \n",
" \n",
" \n",
"data -0.7771 \n",
" \n",
"grad 0.4827 \n",
" \n",
"\n",
"\n",
"4809701904+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809701072->4809701904+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809701072* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809701072*->4809701072 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809701136 \n",
" \n",
" \n",
" \n",
"data 2.0000 \n",
" \n",
"grad -0.1876 \n",
" \n",
"\n",
"\n",
"4809701136->4809701072* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809701200 \n",
" \n",
" \n",
" \n",
"data -0.6395 \n",
" \n",
"grad -1.0774 \n",
" \n",
"\n",
"\n",
"4809701200->4808157072tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809701200+->4809701200 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809701520 \n",
" \n",
" \n",
" \n",
"data 2.0000 \n",
" \n",
"grad -0.0116 \n",
" \n",
"\n",
"\n",
"4809703056* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809701520->4809703056* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809701584 \n",
" \n",
" \n",
" \n",
"data 0.7608 \n",
" \n",
"grad 0.4827 \n",
" \n",
"\n",
"\n",
"4809701584->4809687248+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809701584* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809701584*->4809701584 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809701904 \n",
" \n",
" \n",
" \n",
"data -0.3264 \n",
" \n",
"grad 0.4827 \n",
" \n",
"\n",
"\n",
"4809702288+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809701904->4809702288+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809701904+->4809701904 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809701968 \n",
" \n",
" \n",
" \n",
"data 1.1314 \n",
" \n",
"grad -0.0926 \n",
" \n",
"\n",
"\n",
"4809702672+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809701968->4809702672+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809701968+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809701968+->4809701968 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702096 \n",
" \n",
" \n",
" \n",
"data 3.0000 \n",
" \n",
"grad -0.0501 \n",
" \n",
"\n",
"\n",
"4809702544* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809702096->4809702544* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702224 \n",
" \n",
" \n",
" \n",
"data 1.1182 \n",
" \n",
"grad -0.0926 \n",
" \n",
"\n",
"\n",
"4809702224->4809701968+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702224*->4809702224 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702288 \n",
" \n",
" \n",
" \n",
"data -0.6376 \n",
" \n",
"grad 0.4827 \n",
" \n",
"\n",
"\n",
"4809702288->4809687248+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702288+->4809702288 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702352 \n",
" \n",
" \n",
" \n",
"data -1.0000 \n",
" \n",
"grad -0.3673 \n",
" \n",
"\n",
"\n",
"4809702352->4809701584* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702416 \n",
" \n",
" \n",
" \n",
"data 0.6857 \n",
" \n",
"grad -0.1747 \n",
" \n",
"\n",
"\n",
"4809702416->4809687824* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702416->4809688848* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702416->4809688912* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702416->4809694096* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702416tanh->4809702416 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809210960 \n",
" \n",
" \n",
" \n",
"data -1.0000 \n",
" \n",
"grad -3.1801 \n",
" \n",
"\n",
"\n",
"4809210960->4809205200+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809210960*->4809210960 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702544 \n",
" \n",
" \n",
" \n",
"data -0.3111 \n",
" \n",
"grad 0.4827 \n",
" \n",
"\n",
"\n",
"4809702544->4809702288+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702544*->4809702544 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702608 \n",
" \n",
" \n",
" \n",
"data -0.3971 \n",
" \n",
"grad -0.0053 \n",
" \n",
"\n",
"\n",
"4809702608->4809688592+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702608* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809702608*->4809702608 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702672 \n",
" \n",
" \n",
" \n",
"data 0.1227 \n",
" \n",
"grad -0.0926 \n",
" \n",
"\n",
"\n",
"4809702672->4809700688+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702672+->4809702672 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702800 \n",
" \n",
" \n",
" \n",
"data -1.0088 \n",
" \n",
"grad -0.0926 \n",
" \n",
"\n",
"\n",
"4809702800->4809702672+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702800*->4809702800 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702864 \n",
" \n",
" \n",
" \n",
"data -1.0000 \n",
" \n",
"grad 0.0664 \n",
" \n",
"\n",
"\n",
"4809702928* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4809702864->4809702928* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702928 \n",
" \n",
" \n",
" \n",
"data 0.7172 \n",
" \n",
"grad -0.0926 \n",
" \n",
"\n",
"\n",
"4809702928->4809700688+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809702928*->4809702928 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809703056 \n",
" \n",
" \n",
" \n",
"data 0.4056 \n",
" \n",
"grad -0.0574 \n",
" \n",
"\n",
"\n",
"4809703312+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4809703056->4809703312+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809703056*->4809703056 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809703248 \n",
" \n",
" \n",
" \n",
"data 0.5000 \n",
" \n",
"grad 0.0042 \n",
" \n",
"\n",
"\n",
"4809703248->4809702608* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809703312 \n",
" \n",
" \n",
" \n",
"data 0.4689 \n",
" \n",
"grad -0.0574 \n",
" \n",
"\n",
"\n",
"4809703312->4809695312+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809703312+->4809703312 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809703376 \n",
" \n",
" \n",
" \n",
"data -0.3355 \n",
" \n",
"grad 0.3022 \n",
" \n",
"\n",
"\n",
"4809703376->4809692880+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809703376+->4809703376 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802756688 \n",
" \n",
" \n",
" \n",
"data 0.0632 \n",
" \n",
"grad -0.0305 \n",
" \n",
"\n",
"\n",
"4802756688->4810739984+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802756688->4809695568+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802756688->4809703312+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810661840+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4802756688->4810661840+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810653840 \n",
" \n",
" \n",
" \n",
"data 0.1835 \n",
" \n",
"grad 0.0309 \n",
" \n",
"\n",
"\n",
"4810654416+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810653840->4810654416+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810653840*->4810653840 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802756816 \n",
" \n",
" \n",
" \n",
"data -0.2754 \n",
" \n",
"grad -2.1016 \n",
" \n",
"\n",
"\n",
"4802756816->4810736016+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802756816->4809623440+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802756816->4809689872+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656528+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4802756816->4810656528+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810653904 \n",
" \n",
" \n",
" \n",
"data -0.0608 \n",
" \n",
"grad 0.0309 \n",
" \n",
"\n",
"\n",
"4810654096+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810653904->4810654096+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810653904*->4810653904 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802756880 \n",
" \n",
" \n",
" \n",
"data -0.1037 \n",
" \n",
"grad 1.6556 \n",
" \n",
"\n",
"\n",
"4802756880->4810738000* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802756880->4809692368* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802756880->4809702544* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810659344* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4802756880->4810659344* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810653968 \n",
" \n",
" \n",
" \n",
"data 0.9284 \n",
" \n",
"grad 0.0157 \n",
" \n",
"\n",
"\n",
"4810667280+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810653968->4810667280+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810653968*->4810653968 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654096 \n",
" \n",
" \n",
" \n",
"data 1.6317 \n",
" \n",
"grad 0.0309 \n",
" \n",
"\n",
"\n",
"4810656720tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810654096->4810656720tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654096+->4810654096 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807737808 \n",
" \n",
" \n",
" \n",
"data 0.7411 \n",
" \n",
"grad 0.8697 \n",
" \n",
"\n",
"\n",
"4807737808->4810736400* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807737808->4809695696* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807737808->4809695888* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810663120* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4807737808->4810663120* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654224 \n",
" \n",
" \n",
" \n",
"data 1.5089 \n",
" \n",
"grad 0.0309 \n",
" \n",
"\n",
"\n",
"4810654224->4810654416+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654224+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810654224+->4810654224 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802757200 \n",
" \n",
" \n",
" \n",
"data -0.7172 \n",
" \n",
"grad -0.0146 \n",
" \n",
"\n",
"\n",
"4802757200->4810739408* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802757200->4809694736* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802757200->4809702928* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810661392* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4802757200->4810661392* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654288 \n",
" \n",
" \n",
" \n",
"data -0.8041 \n",
" \n",
"grad 0.0157 \n",
" \n",
"\n",
"\n",
"4810654544tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810654288->4810654544tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654288+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810654288+->4810654288 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654352 \n",
" \n",
" \n",
" \n",
"data -0.0558 \n",
" \n",
"grad 0.5913 \n",
" \n",
"\n",
"\n",
"4810659600+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810654352->4810659600+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654352*->4810654352 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654416 \n",
" \n",
" \n",
" \n",
"data 1.6924 \n",
" \n",
"grad 0.0309 \n",
" \n",
"\n",
"\n",
"4810654416->4810654096+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654416+->4810654416 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654544 \n",
" \n",
" \n",
" \n",
"data -0.6663 \n",
" \n",
"grad 0.0283 \n",
" \n",
"\n",
"\n",
"4810654544->4810657040* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654544tanh->4810654544 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654672 \n",
" \n",
" \n",
" \n",
"data 0.5000 \n",
" \n",
"grad -0.0874 \n",
" \n",
"\n",
"\n",
"4810657104* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810654672->4810657104* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654736 \n",
" \n",
" \n",
" \n",
"data -0.6835 \n",
" \n",
"grad 0.0157 \n",
" \n",
"\n",
"\n",
"4810654864+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810654736->4810654864+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654736*->4810654736 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654800 \n",
" \n",
" \n",
" \n",
"data -0.1260 \n",
" \n",
"grad 0.4126 \n",
" \n",
"\n",
"\n",
"4810663952+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810654800->4810663952+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654800+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810654800+->4810654800 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654864 \n",
" \n",
" \n",
" \n",
"data -1.6184 \n",
" \n",
"grad 0.0157 \n",
" \n",
"\n",
"\n",
"4810666512+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810654864->4810666512+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654864+->4810654864 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654928 \n",
" \n",
" \n",
" \n",
"data 0.0673 \n",
" \n",
"grad 0.0747 \n",
" \n",
"\n",
"\n",
"4810666384+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810654928->4810666384+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654928* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810654928*->4810654928 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654992 \n",
" \n",
" \n",
" \n",
"data -0.6046 \n",
" \n",
"grad 0.7909 \n",
" \n",
"\n",
"\n",
"4810654992->4811337424+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810654992tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810654992tanh->4810654992 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807607632 \n",
" \n",
" \n",
" \n",
"data 0.1335 \n",
" \n",
"grad -4.2710 \n",
" \n",
"\n",
"\n",
"4807607632->4810736464* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807607632->4809692176* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807607632->4809633488* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810658128* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4807607632->4810658128* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655120 \n",
" \n",
" \n",
" \n",
"data -0.4157 \n",
" \n",
"grad 0.0157 \n",
" \n",
"\n",
"\n",
"4810655120->4810666512+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655120*->4810655120 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802758096 \n",
" \n",
" \n",
" \n",
"data -0.3885 \n",
" \n",
"grad 1.0782 \n",
" \n",
"\n",
"\n",
"4802758096->4810737680* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802758096->4809691536* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802758096->4809701072* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810659024* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4802758096->4810659024* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655184 \n",
" \n",
" \n",
" \n",
"data 0.3016 \n",
" \n",
"grad 0.0157 \n",
" \n",
"\n",
"\n",
"4810655184->4810654288+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655184*->4810655184 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655248 \n",
" \n",
" \n",
" \n",
"data -0.8766 \n",
" \n",
"grad -0.2020 \n",
" \n",
"\n",
"\n",
"4810655248->4810656784* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655248tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810655248tanh->4810655248 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655376 \n",
" \n",
" \n",
" \n",
"data -0.8422 \n",
" \n",
"grad -0.0468 \n",
" \n",
"\n",
"\n",
"4810655504+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810655376->4810655504+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655376*->4810655376 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802758352 \n",
" \n",
" \n",
" \n",
"data 0.5591 \n",
" \n",
"grad -0.1479 \n",
" \n",
"\n",
"\n",
"4802758352->4809687568* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802758352->4810738768* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802758352->4809702224* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660496* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4802758352->4810660496* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655440 \n",
" \n",
" \n",
" \n",
"data -0.4408 \n",
" \n",
"grad 0.5018 \n",
" \n",
"\n",
"\n",
"4810655440->4810656848+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655440*->4810655440 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655504 \n",
" \n",
" \n",
" \n",
"data -0.8307 \n",
" \n",
"grad -0.0468 \n",
" \n",
"\n",
"\n",
"4810656080+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810655504->4810656080+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655504+->4810655504 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655568 \n",
" \n",
" \n",
" \n",
"data 0.5610 \n",
" \n",
"grad -0.0468 \n",
" \n",
"\n",
"\n",
"4810655568->4810655696+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655568*->4810655568 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655696 \n",
" \n",
" \n",
" \n",
"data 0.0551 \n",
" \n",
"grad -0.0468 \n",
" \n",
"\n",
"\n",
"4810655824+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810655696->4810655824+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655696+->4810655696 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655760 \n",
" \n",
" \n",
" \n",
"data -0.0436 \n",
" \n",
"grad -0.0468 \n",
" \n",
"\n",
"\n",
"4810655760->4810655824+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655760*->4810655760 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655824 \n",
" \n",
" \n",
" \n",
"data 0.0115 \n",
" \n",
"grad -0.0468 \n",
" \n",
"\n",
"\n",
"4810655824->4810655504+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655824+->4810655824 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655952 \n",
" \n",
" \n",
" \n",
"data -0.5304 \n",
" \n",
"grad -0.0468 \n",
" \n",
"\n",
"\n",
"4810655952->4810656080+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810655952*->4810655952 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656080 \n",
" \n",
" \n",
" \n",
"data -1.3611 \n",
" \n",
"grad -0.0468 \n",
" \n",
"\n",
"\n",
"4810656080->4810655248tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656080+->4810656080 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656144 \n",
" \n",
" \n",
" \n",
"data -0.1344 \n",
" \n",
"grad 0.3022 \n",
" \n",
"\n",
"\n",
"4810656144->4810660560+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656144*->4810656144 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807739856 \n",
" \n",
" \n",
" \n",
"data 0.3938 \n",
" \n",
"grad -2.6611 \n",
" \n",
"\n",
"\n",
"4807739856->4809688272+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807739856->4810740880+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807739856->4809703376+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807739856->4810654800+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656208 \n",
" \n",
" \n",
" \n",
"data -0.1821 \n",
" \n",
"grad 0.0309 \n",
" \n",
"\n",
"\n",
"4810656208->4810654224+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656208*->4810656208 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807739920 \n",
" \n",
" \n",
" \n",
"data -0.1645 \n",
" \n",
"grad 1.0317 \n",
" \n",
"\n",
"\n",
"4807739920->4809688144* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807739920->4810741648* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807739920->4810656144* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810664720* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4807739920->4810664720* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802759184 \n",
" \n",
" \n",
" \n",
"data -0.8741 \n",
" \n",
"grad -0.1349 \n",
" \n",
"\n",
"\n",
"4802759184->4809691280* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802759184->4810740176* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802759184->4809697808* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660816* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4802759184->4810660816* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802759248 \n",
" \n",
" \n",
" \n",
"data -0.7608 \n",
" \n",
"grad -0.8334 \n",
" \n",
"\n",
"\n",
"4802759248->4810738320* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802759248->4809693392* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802759248->4809701584* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810657168* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4802759248->4810657168* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656336 \n",
" \n",
" \n",
" \n",
"data -1.0157 \n",
" \n",
"grad 0.5018 \n",
" \n",
"\n",
"\n",
"4810656656+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810656336->4810656656+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656336+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810656336+->4810656336 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656400 \n",
" \n",
" \n",
" \n",
"data 0.4018 \n",
" \n",
"grad 0.5018 \n",
" \n",
"\n",
"\n",
"4810656400->4810656336+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656400*->4810656400 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802759440 \n",
" \n",
" \n",
" \n",
"data 0.2028 \n",
" \n",
"grad -0.0975 \n",
" \n",
"\n",
"\n",
"4802759440->4810739856* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802759440->4809696656* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802759440->4809703056* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662480* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4802759440->4810662480* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656528 \n",
" \n",
" \n",
" \n",
"data -0.5740 \n",
" \n",
"grad 0.1463 \n",
" \n",
"\n",
"\n",
"4810657680+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810656528->4810657680+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656528+->4810656528 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802759568 \n",
" \n",
" \n",
" \n",
"data -0.7942 \n",
" \n",
"grad 0.0186 \n",
" \n",
"\n",
"\n",
"4802759568->4809690512* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802759568->4810740496* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802759568->4809702608* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662736* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4802759568->4810662736* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656656 \n",
" \n",
" \n",
" \n",
"data -1.0532 \n",
" \n",
"grad 0.5018 \n",
" \n",
"\n",
"\n",
"4810656912+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810656656->4810656912+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656656+->4810656656 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656720 \n",
" \n",
" \n",
" \n",
"data 0.9263 \n",
" \n",
"grad 0.2177 \n",
" \n",
"\n",
"\n",
"4810656720->4810656400* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656720tanh->4810656720 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802366416 \n",
" \n",
" \n",
" \n",
"data 0.0000 \n",
" \n",
"grad 1.0000 \n",
" \n",
"\n",
"\n",
"4802366416->4811339088+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656784 \n",
" \n",
" \n",
" \n",
"data 0.3529 \n",
" \n",
"grad 0.5018 \n",
" \n",
"\n",
"\n",
"4810656784->4810656912+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656784*->4810656784 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656848 \n",
" \n",
" \n",
" \n",
"data -1.4175 \n",
" \n",
"grad 0.5018 \n",
" \n",
"\n",
"\n",
"4810656848->4810656336+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656848+->4810656848 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656912 \n",
" \n",
" \n",
" \n",
"data -0.7003 \n",
" \n",
"grad 0.5018 \n",
" \n",
"\n",
"\n",
"4810656912->4810654992tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656912+->4810656912 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656976 \n",
" \n",
" \n",
" \n",
"data -0.7015 \n",
" \n",
"grad 0.2880 \n",
" \n",
"\n",
"\n",
"4810662032* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810656976->4810662032* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656976->4810663120* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665616* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810656976->4810665616* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810669776* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810656976->4810669776* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810656976tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810656976tanh->4810656976 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810657040 \n",
" \n",
" \n",
" \n",
"data -0.0375 \n",
" \n",
"grad 0.5018 \n",
" \n",
"\n",
"\n",
"4810657040->4810656656+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810657040*->4810657040 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810657104 \n",
" \n",
" \n",
" \n",
"data -0.2986 \n",
" \n",
"grad 0.1463 \n",
" \n",
"\n",
"\n",
"4810657104->4810656528+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810657104*->4810657104 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810657168 \n",
" \n",
" \n",
" \n",
"data -0.7608 \n",
" \n",
"grad -0.0801 \n",
" \n",
"\n",
"\n",
"4810660432+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810657168->4810660432+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810657168*->4810657168 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810657296 \n",
" \n",
" \n",
" \n",
"data 1.0000 \n",
" \n",
"grad 0.0195 \n",
" \n",
"\n",
"\n",
"4810657296->4810658128* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802760336 \n",
" \n",
" \n",
" \n",
"data 0.0132 \n",
" \n",
"grad -0.0579 \n",
" \n",
"\n",
"\n",
"4802760336->4810738896+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802760336->4809693200+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802760336->4809701968+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660240+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4802760336->4810660240+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810657552 \n",
" \n",
" \n",
" \n",
"data -0.6384 \n",
" \n",
"grad 0.5913 \n",
" \n",
"\n",
"\n",
"4810664592+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810657552->4810664592+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810657552+->4810657552 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810657680 \n",
" \n",
" \n",
" \n",
"data -0.4404 \n",
" \n",
"grad 0.1463 \n",
" \n",
"\n",
"\n",
"4810658704+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810657680->4810658704+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810657680+->4810657680 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810657808 \n",
" \n",
" \n",
" \n",
"data 0.2809 \n",
" \n",
"grad 0.4126 \n",
" \n",
"\n",
"\n",
"4810662224+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810657808->4810662224+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810657808+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810657808+->4810657808 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810657936 \n",
" \n",
" \n",
" \n",
"data -0.4297 \n",
" \n",
"grad 0.1463 \n",
" \n",
"\n",
"\n",
"4810657936->4810658704+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810657936* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810657936*->4810657936 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810658000 \n",
" \n",
" \n",
" \n",
"data 1.0000 \n",
" \n",
"grad -0.0629 \n",
" \n",
"\n",
"\n",
"4810658000->4810657936* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810658128 \n",
" \n",
" \n",
" \n",
"data 0.1335 \n",
" \n",
"grad 0.1463 \n",
" \n",
"\n",
"\n",
"4810658128->4810657680+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810658128*->4810658128 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802761104 \n",
" \n",
" \n",
" \n",
"data -0.5972 \n",
" \n",
"grad -3.2376 \n",
" \n",
"\n",
"\n",
"4802761104->4810736208* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802761104->4809690192* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802761104->4809102928* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802761104->4810657104* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810658320 \n",
" \n",
" \n",
" \n",
"data 0.5000 \n",
" \n",
"grad 0.0311 \n",
" \n",
"\n",
"\n",
"4810658320->4810659024* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810658704 \n",
" \n",
" \n",
" \n",
"data -0.8702 \n",
" \n",
"grad 0.1463 \n",
" \n",
"\n",
"\n",
"4810658704->4810656976tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810658704+->4810658704 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810658768 \n",
" \n",
" \n",
" \n",
"data -0.3117 \n",
" \n",
"grad 0.5913 \n",
" \n",
"\n",
"\n",
"4810658768->4810736592+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810658768*->4810658768 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810658960 \n",
" \n",
" \n",
" \n",
"data -0.5429 \n",
" \n",
"grad -0.1135 \n",
" \n",
"\n",
"\n",
"4810663696* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810658960->4810663696* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665552* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810658960->4810665552* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810658960->4810667536* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810668240* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810658960->4810668240* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810658960tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810658960tanh->4810658960 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810659024 \n",
" \n",
" \n",
" \n",
"data -0.1943 \n",
" \n",
"grad -0.0801 \n",
" \n",
"\n",
"\n",
"4810660176+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810659024->4810660176+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810659024*->4810659024 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810659152 \n",
" \n",
" \n",
" \n",
"data 1.0000 \n",
" \n",
"grad 0.0609 \n",
" \n",
"\n",
"\n",
"4810659152->4810657168* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810659280 \n",
" \n",
" \n",
" \n",
"data 1.0000 \n",
" \n",
"grad 0.0083 \n",
" \n",
"\n",
"\n",
"4810659280->4810659344* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810659344 \n",
" \n",
" \n",
" \n",
"data -0.1037 \n",
" \n",
"grad -0.0801 \n",
" \n",
"\n",
"\n",
"4810659408+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810659344->4810659408+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810659344*->4810659344 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810659408 \n",
" \n",
" \n",
" \n",
"data 0.1526 \n",
" \n",
"grad -0.0801 \n",
" \n",
"\n",
"\n",
"4810659408->4810660432+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810659408+->4810659408 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810659600 \n",
" \n",
" \n",
" \n",
"data -0.3291 \n",
" \n",
"grad 0.5913 \n",
" \n",
"\n",
"\n",
"4810659600->4810736592+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810659600+->4810659600 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802762576 \n",
" \n",
" \n",
" \n",
"data 0.4506 \n",
" \n",
"grad 0.6228 \n",
" \n",
"\n",
"\n",
"4802762576->4810737808+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802762576->4809692560+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802762576->4809701904+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802762576->4810660176+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660048 \n",
" \n",
" \n",
" \n",
"data 0.5000 \n",
" \n",
"grad -0.0175 \n",
" \n",
"\n",
"\n",
"4810660048->4810660496* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660112 \n",
" \n",
" \n",
" \n",
"data 0.9547 \n",
" \n",
"grad 0.0309 \n",
" \n",
"\n",
"\n",
"4810660112->4810667152+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660112*->4810660112 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660176 \n",
" \n",
" \n",
" \n",
"data 0.2564 \n",
" \n",
"grad -0.0801 \n",
" \n",
"\n",
"\n",
"4810660176->4810659408+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660176+->4810660176 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660240 \n",
" \n",
" \n",
" \n",
"data 0.2928 \n",
" \n",
"grad -0.0313 \n",
" \n",
"\n",
"\n",
"4810661328+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810660240->4810661328+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660240+->4810660240 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660304 \n",
" \n",
" \n",
" \n",
"data -0.6415 \n",
" \n",
"grad -0.0532 \n",
" \n",
"\n",
"\n",
"4810660880* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810660304->4810660880* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810666448* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810660304->4810666448* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660304->4810667856* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810670032* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810660304->4810670032* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660304tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810660304tanh->4810660304 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660432 \n",
" \n",
" \n",
" \n",
"data -0.6082 \n",
" \n",
"grad -0.0801 \n",
" \n",
"\n",
"\n",
"4810660432->4810658960tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660432+->4810660432 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660496 \n",
" \n",
" \n",
" \n",
"data 0.2795 \n",
" \n",
"grad -0.0313 \n",
" \n",
"\n",
"\n",
"4810660496->4810660240+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660496*->4810660496 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660560 \n",
" \n",
" \n",
" \n",
"data -0.5873 \n",
" \n",
"grad 0.3022 \n",
" \n",
"\n",
"\n",
"4810660560->4809700880tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660560+->4810660560 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808006416 \n",
" \n",
" \n",
" \n",
"data 0.6945 \n",
" \n",
"grad 0.0543 \n",
" \n",
"\n",
"\n",
"4808006416->4809687952* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808006416->4810743632* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808006416->4810654736* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808006416->4810662032* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660688 \n",
" \n",
" \n",
" \n",
"data 1.0000 \n",
" \n",
"grad 0.0105 \n",
" \n",
"\n",
"\n",
"4810660752* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810660688->4810660752* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660752 \n",
" \n",
" \n",
" \n",
"data -0.3363 \n",
" \n",
"grad -0.0313 \n",
" \n",
"\n",
"\n",
"4810660752->4810661328+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660752*->4810660752 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660816 \n",
" \n",
" \n",
" \n",
"data -0.8741 \n",
" \n",
"grad -0.0020 \n",
" \n",
"\n",
"\n",
"4810662672+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810660816->4810662672+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660816*->4810660816 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660880 \n",
" \n",
" \n",
" \n",
"data 0.2407 \n",
" \n",
"grad 0.4126 \n",
" \n",
"\n",
"\n",
"4810660880->4810657808+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810660880*->4810660880 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810661008 \n",
" \n",
" \n",
" \n",
"data 1.0000 \n",
" \n",
"grad 0.0225 \n",
" \n",
"\n",
"\n",
"4810661008->4810661392* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810661072 \n",
" \n",
" \n",
" \n",
"data -0.7607 \n",
" \n",
"grad -0.0313 \n",
" \n",
"\n",
"\n",
"4810661072->4810660304tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810661072+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810661072+->4810661072 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810661328 \n",
" \n",
" \n",
" \n",
"data -0.0435 \n",
" \n",
"grad -0.0313 \n",
" \n",
"\n",
"\n",
"4810661328->4810661072+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810661328+->4810661328 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810661392 \n",
" \n",
" \n",
" \n",
"data -0.7172 \n",
" \n",
"grad -0.0313 \n",
" \n",
"\n",
"\n",
"4810661392->4810661072+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810661392*->4810661392 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4799192720 \n",
" \n",
" \n",
" \n",
"data -0.9043 \n",
" \n",
"grad 0.8070 \n",
" \n",
"\n",
"\n",
"4799192720->4809688912* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4799192720->4810745296* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4799192720->4810655376* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4799192720->4810670032* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810661648 \n",
" \n",
" \n",
" \n",
"data 0.5000 \n",
" \n",
"grad -0.0004 \n",
" \n",
"\n",
"\n",
"4810661648->4810662480* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810661840 \n",
" \n",
" \n",
" \n",
"data 0.1647 \n",
" \n",
"grad -0.0020 \n",
" \n",
"\n",
"\n",
"4810661840->4810662672+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810661840+->4810661840 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662032 \n",
" \n",
" \n",
" \n",
"data -0.4872 \n",
" \n",
"grad 0.0006 \n",
" \n",
"\n",
"\n",
"4810668560+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810662032->4810668560+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662032*->4810662032 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662096 \n",
" \n",
" \n",
" \n",
"data 1.0000 \n",
" \n",
"grad 0.0018 \n",
" \n",
"\n",
"\n",
"4810662096->4810660816* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662224 \n",
" \n",
" \n",
" \n",
"data 0.4299 \n",
" \n",
"grad 0.4126 \n",
" \n",
"\n",
"\n",
"4810664976tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810662224->4810664976tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662224+->4810662224 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662352 \n",
" \n",
" \n",
" \n",
"data 0.3651 \n",
" \n",
"grad 0.5913 \n",
" \n",
"\n",
"\n",
"4810662352->4810664592+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662352*->4810662352 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662416 \n",
" \n",
" \n",
" \n",
"data 0.3383 \n",
" \n",
"grad 0.5913 \n",
" \n",
"\n",
"\n",
"4810662416->4810657552+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662416*->4810662416 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662480 \n",
" \n",
" \n",
" \n",
"data 0.1014 \n",
" \n",
"grad -0.0020 \n",
" \n",
"\n",
"\n",
"4810662480->4810661840+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662480*->4810662480 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662672 \n",
" \n",
" \n",
" \n",
"data -0.7095 \n",
" \n",
"grad -0.0020 \n",
" \n",
"\n",
"\n",
"4810662864+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810662672->4810662864+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662672+->4810662672 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662736 \n",
" \n",
" \n",
" \n",
"data -0.7942 \n",
" \n",
"grad -0.0020 \n",
" \n",
"\n",
"\n",
"4810662736->4810662864+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662736*->4810662736 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662864 \n",
" \n",
" \n",
" \n",
"data -1.5036 \n",
" \n",
"grad -0.0020 \n",
" \n",
"\n",
"\n",
"4810663760tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810662864->4810663760tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810662864+->4810662864 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810663056 \n",
" \n",
" \n",
" \n",
"data 1.0000 \n",
" \n",
"grad 0.0016 \n",
" \n",
"\n",
"\n",
"4810663056->4810662736* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810663120 \n",
" \n",
" \n",
" \n",
"data -0.5198 \n",
" \n",
"grad 0.4126 \n",
" \n",
"\n",
"\n",
"4810663120->4810654800+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810663120*->4810663120 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810663632 \n",
" \n",
" \n",
" \n",
"data 0.8418 \n",
" \n",
"grad 0.2565 \n",
" \n",
"\n",
"\n",
"4810663632->4810662352* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810663632tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810663632tanh->4810663632 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810663696 \n",
" \n",
" \n",
" \n",
"data 0.1662 \n",
" \n",
"grad 0.4126 \n",
" \n",
"\n",
"\n",
"4810663696->4810663952+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810663696*->4810663696 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810663760 \n",
" \n",
" \n",
" \n",
"data -0.9058 \n",
" \n",
"grad -0.0113 \n",
" \n",
"\n",
"\n",
"4810663760->4810654928* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810663760->4810664720* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665296* \n",
"\n",
"* \n",
" \n",
"\n",
"\n",
"4810663760->4810665296* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810663760->4810668496* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810663760tanh->4810663760 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810663952 \n",
" \n",
" \n",
" \n",
"data 0.0402 \n",
" \n",
"grad 0.4126 \n",
" \n",
"\n",
"\n",
"4810663952->4810657808+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810663952+->4810663952 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808173968 \n",
" \n",
" \n",
" \n",
"data 0.3691 \n",
" \n",
"grad 0.0388 \n",
" \n",
"\n",
"\n",
"4808173968->4809687312* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808173968->4810744400* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808173968->4810655184* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808173968->4810665296* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810664592 \n",
" \n",
" \n",
" \n",
"data -0.2733 \n",
" \n",
"grad 0.5913 \n",
" \n",
"\n",
"\n",
"4810664592->4810659600+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810664592+->4810664592 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810664720 \n",
" \n",
" \n",
" \n",
"data 0.1490 \n",
" \n",
"grad 0.4126 \n",
" \n",
"\n",
"\n",
"4810664720->4810662224+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810664720*->4810664720 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808174544 \n",
" \n",
" \n",
" \n",
"data -0.5700 \n",
" \n",
"grad -0.4339 \n",
" \n",
"\n",
"\n",
"4808174544->4809688720* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808174544->4810744784* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808174544->4810655568* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808174544->4810669776* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810664976 \n",
" \n",
" \n",
" \n",
"data 0.4052 \n",
" \n",
"grad 0.4937 \n",
" \n",
"\n",
"\n",
"4810664976->4810662416* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810664976tanh->4810664976 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665104 \n",
" \n",
" \n",
" \n",
"data -1.7198 \n",
" \n",
"grad 0.0006 \n",
" \n",
"\n",
"\n",
"4810666704+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810665104->4810666704+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665104+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810665104+->4810665104 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665296 \n",
" \n",
" \n",
" \n",
"data -0.3343 \n",
" \n",
"grad 0.0006 \n",
" \n",
"\n",
"\n",
"4810667920+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810665296->4810667920+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665296*->4810665296 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665360 \n",
" \n",
" \n",
" \n",
"data -0.1060 \n",
" \n",
"grad -0.0954 \n",
" \n",
"\n",
"\n",
"4810669840+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810665360->4810669840+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665360+->4810665360 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665424 \n",
" \n",
" \n",
" \n",
"data 1.4168 \n",
" \n",
"grad 0.0747 \n",
" \n",
"\n",
"\n",
"4810665680+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810665424->4810665680+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665424+->4810665424 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665552 \n",
" \n",
" \n",
" \n",
"data -0.1304 \n",
" \n",
"grad 0.0747 \n",
" \n",
"\n",
"\n",
"4810665552->4810665680+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665552*->4810665552 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807749264 \n",
" \n",
" \n",
" \n",
"data 0.2402 \n",
" \n",
"grad -0.2367 \n",
" \n",
"\n",
"\n",
"4807749264->4809693584* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807749264->4810742736* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807749264->4810656208* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807749264->4810665552* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665616 \n",
" \n",
" \n",
" \n",
"data 0.6805 \n",
" \n",
"grad 0.0747 \n",
" \n",
"\n",
"\n",
"4810665616->4810665424+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665616*->4810665616 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665680 \n",
" \n",
" \n",
" \n",
"data 1.2863 \n",
" \n",
"grad 0.0747 \n",
" \n",
"\n",
"\n",
"4810665936+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810665680->4810665936+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665680+->4810665680 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665936 \n",
" \n",
" \n",
" \n",
"data 1.1599 \n",
" \n",
"grad 0.0747 \n",
" \n",
"\n",
"\n",
"4810665936->4810666384+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810665936+->4810665936 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802769168 \n",
" \n",
" \n",
" \n",
"data -0.3363 \n",
" \n",
"grad -0.2299 \n",
" \n",
"\n",
"\n",
"4802769168->4810739088* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802769168->4809693776* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802769168->4809702800* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802769168->4810660752* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810666384 \n",
" \n",
" \n",
" \n",
"data 1.2273 \n",
" \n",
"grad 0.0747 \n",
" \n",
"\n",
"\n",
"4810666384->4810663632tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810666384+->4810666384 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810666448 \n",
" \n",
" \n",
" \n",
"data -0.1264 \n",
" \n",
"grad 0.0747 \n",
" \n",
"\n",
"\n",
"4810666448->4810665936+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810666448*->4810666448 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810666512 \n",
" \n",
" \n",
" \n",
"data -2.0342 \n",
" \n",
"grad 0.0157 \n",
" \n",
"\n",
"\n",
"4810666512->4810667280+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810666512+->4810666512 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810666576 \n",
" \n",
" \n",
" \n",
"data 0.7742 \n",
" \n",
"grad -0.2380 \n",
" \n",
"\n",
"\n",
"4810666576->4810658768* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810666576tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810666576tanh->4810666576 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810666704 \n",
" \n",
" \n",
" \n",
"data -2.3593 \n",
" \n",
"grad 0.0006 \n",
" \n",
"\n",
"\n",
"4810666704->4810667920+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810666704+->4810666704 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808176720 \n",
" \n",
" \n",
" \n",
"data -0.9349 \n",
" \n",
"grad -0.1677 \n",
" \n",
"\n",
"\n",
"4808176720->4809694224+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808176720->4810743760+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808176720->4810654864+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808176720->4810668560+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810667152 \n",
" \n",
" \n",
" \n",
"data 1.6910 \n",
" \n",
"grad 0.0309 \n",
" \n",
"\n",
"\n",
"4810667152->4810654224+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810667152+->4810667152 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810667280 \n",
" \n",
" \n",
" \n",
"data -1.1058 \n",
" \n",
"grad 0.0157 \n",
" \n",
"\n",
"\n",
"4810667280->4810654288+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810667280+->4810667280 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810667536 \n",
" \n",
" \n",
" \n",
"data -0.2977 \n",
" \n",
"grad 0.0006 \n",
" \n",
"\n",
"\n",
"4810667536->4810665104+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810667536*->4810667536 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807751248 \n",
" \n",
" \n",
" \n",
"data -0.3061 \n",
" \n",
"grad -1.7052 \n",
" \n",
"\n",
"\n",
"4807751248->4810736272* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807751248->4809692048* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807751248->4809697296* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807751248->4810663696* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807751376 \n",
" \n",
" \n",
" \n",
"data -0.3752 \n",
" \n",
"grad -2.3925 \n",
" \n",
"\n",
"\n",
"4807751376->4809687120* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807751376->4809688848* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807751376->4810741136* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807751376->4810660880* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808177424 \n",
" \n",
" \n",
" \n",
"data 0.0575 \n",
" \n",
"grad 0.3890 \n",
" \n",
"\n",
"\n",
"4808177424->4809688976* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808177424->4810745040* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808177424->4810655760* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4808177424->4810668240* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810667856 \n",
" \n",
" \n",
" \n",
"data -0.6395 \n",
" \n",
"grad 0.0006 \n",
" \n",
"\n",
"\n",
"4810667856->4810666704+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810667856*->4810667856 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807751568 \n",
" \n",
" \n",
" \n",
"data 0.1971 \n",
" \n",
"grad -0.3122 \n",
" \n",
"\n",
"\n",
"4807751568->4809694096* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807751568->4810742992* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807751568->4810653840* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807751568->4810666448* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810667920 \n",
" \n",
" \n",
" \n",
"data -2.6936 \n",
" \n",
"grad 0.0006 \n",
" \n",
"\n",
"\n",
"4810668688tanh \n",
"\n",
"tanh \n",
" \n",
"\n",
"\n",
"4810667920->4810668688tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810667920+->4810667920 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810668240 \n",
" \n",
" \n",
" \n",
"data -0.0312 \n",
" \n",
"grad -0.0954 \n",
" \n",
"\n",
"\n",
"4810668240->4810669840+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810668240*->4810668240 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810668496 \n",
" \n",
" \n",
" \n",
"data 0.5878 \n",
" \n",
"grad -0.0954 \n",
" \n",
"\n",
"\n",
"4810669136+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810668496->4810669136+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810668496*->4810668496 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810668560 \n",
" \n",
" \n",
" \n",
"data -1.4221 \n",
" \n",
"grad 0.0006 \n",
" \n",
"\n",
"\n",
"4810668560->4810665104+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810668560+->4810668560 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4809062928 \n",
" \n",
" \n",
" \n",
"data 2.0000 \n",
" \n",
"grad 0.6434 \n",
" \n",
"\n",
"\n",
"4809062928->4809102928* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810668688 \n",
" \n",
" \n",
" \n",
"data -0.9909 \n",
" \n",
"grad 0.0333 \n",
" \n",
"\n",
"\n",
"4810668688->4810654352* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810668688tanh->4810668688 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810669136 \n",
" \n",
" \n",
" \n",
"data 1.0307 \n",
" \n",
"grad -0.0954 \n",
" \n",
"\n",
"\n",
"4810669136->4810666576tanh \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810669136+->4810669136 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810669200 \n",
" \n",
" \n",
" \n",
"data 0.4429 \n",
" \n",
"grad -0.0954 \n",
" \n",
"\n",
"\n",
"4810669200->4810669136+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810669200+ \n",
"\n",
"+ \n",
" \n",
"\n",
"\n",
"4810669200+->4810669200 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807752976 \n",
" \n",
" \n",
" \n",
"data -0.9701 \n",
" \n",
"grad 0.0812 \n",
" \n",
"\n",
"\n",
"4807752976->4809691664* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807752976->4810742288* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807752976->4810660112* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807752976->4810665616* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810669776 \n",
" \n",
" \n",
" \n",
"data 0.3998 \n",
" \n",
"grad -0.0954 \n",
" \n",
"\n",
"\n",
"4810669776->4810665360+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810669776*->4810669776 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802772752 \n",
" \n",
" \n",
" \n",
"data -0.4297 \n",
" \n",
"grad 2.4052 \n",
" \n",
"\n",
"\n",
"4802772752->4810737232* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802772752->4809691408* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802772752->4809697552* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4802772752->4810657936* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807753488 \n",
" \n",
" \n",
" \n",
"data -0.0743 \n",
" \n",
"grad 0.0507 \n",
" \n",
"\n",
"\n",
"4807753488->4809691216* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807753488->4810743248* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807753488->4810653904* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4807753488->4810654928* \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810669840 \n",
" \n",
" \n",
" \n",
"data -0.1372 \n",
" \n",
"grad -0.0954 \n",
" \n",
"\n",
"\n",
"4810669840->4810669200+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810669840+->4810669840 \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810670032 \n",
" \n",
" \n",
" \n",
"data 0.5801 \n",
" \n",
"grad -0.0954 \n",
" \n",
"\n",
"\n",
"4810670032->4810669200+ \n",
" \n",
" \n",
" \n",
"\n",
"\n",
"4810670032*->4810670032 \n",
" \n",
" \n",
" \n",
" \n",
" \n"
],
"text/plain": [
""
]
},
"execution_count": 40,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"draw_dot(loss)"
]
},
{
"cell_type": "markdown",
"id": "692d7822-5321-4722-9f03-4be088f8d366",
"metadata": {},
"source": [
"Since we want to decrease the loss, we have to change the inputs somehow. We could change our x values, but that tends to be fixed, becuase that's our data. The input data is a given to the problem, even though we have gradients for it. \n",
"\n",
"Some of the gradients, however, are for the neural network parameters (the ```w``` and ```b```'s.). These are what we want tot change. We want to change all of the parameters of the MLP slightly to the lower loss.\n",
"\n",
"To do this, we'll want some code that gathers up all of the parameters on the neural network and can operate on all of the parameters simulaneously. \n",
"\n",
"We'll create a parameters function that returns a list of ```self.w``` and ```self.b```:"
]
},
{
"cell_type": "code",
"execution_count": 57,
"id": "faa9ce01-28d6-4924-bc3a-15b9dfab45ce",
"metadata": {},
"outputs": [],
"source": [
"class Neuron:\n",
" def __init__(self,nin): \n",
" self.w = [Value(random.uniform(-1,1)) for _ in range(nin)]\n",
" self.b = Value(random.uniform(-1,1))\n",
"\n",
" def __call__(self,x): \n",
" act = sum((wi*xi for wi, xi in zip(self.w, x)), self.b) \n",
" out = act.tanh()\n",
" return out\n",
" \n",
" def parameters(self): #pytorch has a parameters() on every neural net module that returns param tensors (for us it's scalars)\n",
" return self.w + [self.b]\n",
"\n",
"class Layer: \n",
" def __init__(self, nin, nout): \n",
" #nin = num inputs to each each neuron in the layer\n",
" #nout = number of neurons in the layer\n",
" self.neurons = [Neuron(nin) for _ in range(nout)]\n",
"\n",
" def __call__(self, x):\n",
" outs = [n(x) for n in self.neurons]\n",
" return outs[0] if len(outs) == 1 else outs \n",
" \n",
" def parameters(self):\n",
" params = []\n",
" for neuron in self.neurons:\n",
" ps = neuron.parameters()\n",
" params.extend(ps)\n",
" return params\n",
" #can also write as : return [p for neuron in self.neurons for p in neuron.parameters()]\n",
"\n",
"class MLP:\n",
" def __init__(self, nin, nouts):\n",
" sz = [nin] + nouts \n",
" self.layers = [Layer(sz[i], sz[i+1]) for i in range(len(nouts))]\n",
" \n",
" def __call__(self, x):\n",
" for layer in self.layers: \n",
" x = layer(x) \n",
" return x \n",
" def parameters(self):\n",
" return [p for layer in self.layers for p in layer.parameters()] #get parameters for all layers in MLP"
]
},
{
"cell_type": "markdown",
"id": "4c6a1845-9875-4b3b-a932-0631a77bafa9",
"metadata": {},
"source": [
"Now that we've added this API, we reinitialize the network and test with ```n.parameters```. This will change some of the numbers in the network."
]
},
{
"cell_type": "code",
"execution_count": 63,
"id": "0aa67813-ce69-4d4b-9cff-1893415b3ff1",
"metadata": {},
"outputs": [],
"source": [
"# Neural network\n",
"n = MLP(3, [4, 4, 1])"
]
},
{
"cell_type": "code",
"execution_count": 69,
"id": "1eefa7a2-d344-4bb0-9c3a-a4f9f1d3cce1",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"parameters count = 41\n"
]
},
{
"data": {
"text/plain": [
"[Value(data=0.8087675862549752),\n",
" Value(data=0.025263971658811668),\n",
" Value(data=-0.8095723663106846),\n",
" Value(data=0.8038607738814252),\n",
" Value(data=-0.29638876451525076),\n",
" Value(data=-0.03247232926147414),\n",
" Value(data=-0.44561734574393985),\n",
" Value(data=0.06311564914043521),\n",
" Value(data=0.7567492970120597),\n",
" Value(data=-0.3446175462832468),\n",
" Value(data=-0.45540808178622627),\n",
" Value(data=0.5760849545442048),\n",
" Value(data=-0.3492314617459753),\n",
" Value(data=0.8924325636035717),\n",
" Value(data=-0.46505647936296013),\n",
" Value(data=0.6149317705883368),\n",
" Value(data=-0.5223518310937196),\n",
" Value(data=-0.11791949089726761),\n",
" Value(data=-0.8135156356899493),\n",
" Value(data=0.9913436635820159),\n",
" Value(data=-0.12121895507301739),\n",
" Value(data=-0.862460039190883),\n",
" Value(data=-0.805484213082718),\n",
" Value(data=0.19498379419998968),\n",
" Value(data=0.8092170416979758),\n",
" Value(data=0.7535190900116837),\n",
" Value(data=0.7004254761419886),\n",
" Value(data=0.1838953755012176),\n",
" Value(data=-0.3540157445416765),\n",
" Value(data=0.561899056075303),\n",
" Value(data=-0.3441460595011996),\n",
" Value(data=0.2843677810237102),\n",
" Value(data=0.9726194405716639),\n",
" Value(data=-0.7131735718769536),\n",
" Value(data=0.44689712470395704),\n",
" Value(data=0.6053467884662802),\n",
" Value(data=-0.9144193640506879),\n",
" Value(data=-0.7613067493180077),\n",
" Value(data=0.8626489273439784),\n",
" Value(data=-0.8825822569824531),\n",
" Value(data=0.3359075971184846)]"
]
},
"execution_count": 69,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"print(\"parameters count = \",len(n.parameters()))\n",
"n.parameters()"
]
},
{
"cell_type": "markdown",
"id": "678797e7-adae-450d-8194-a400ed0a4159",
"metadata": {},
"source": [
"Now, we we get all of the weights and biases in the network (41 parameters). Now we can change them."
]
},
{
"cell_type": "code",
"execution_count": 76,
"id": "105910a9-2ee2-404d-9d6c-678b52729082",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"ypred = [Value(data=0.09626161200328809), Value(data=0.8897672475073349), Value(data=-0.6677985165984013), Value(data=0.1312729219383994)]\n"
]
},
{
"data": {
"text/plain": [
"Value(data=5.253007885422014)"
]
},
"execution_count": 76,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"xs = [\n",
" [2.0, 3.0, -1.0],\n",
" [3.0, -1.0, 0.5],\n",
" [0.5, 1.0, 1.0],\n",
" [1.0, 1.0, -1.0],\n",
"]\n",
"ys = [1.0, -1.0, -1.0, 1.0]\n",
"\n",
"# Predictions\n",
"ypred = [n(x) for x in xs]\n",
"print(\"ypred = \", ypred)\n",
"\n",
"# Convert ys elements to Value objects\n",
"ys_value = [Value(y) for y in ys]\n",
"\n",
"# Calculate the loss as a single Value object\n",
"loss = sum([(yout - ygt) ** 2 for ygt, yout in zip(ys_value, ypred)], Value(0.0))\n",
"loss"
]
},
{
"cell_type": "code",
"execution_count": 78,
"id": "a9a4486c-52f0-4dad-9d80-607ee96bd4b5",
"metadata": {},
"outputs": [],
"source": [
"loss.backward()"
]
},
{
"cell_type": "markdown",
"id": "fb1ae150-beec-438e-8667-6e2871d3bec8",
"metadata": {},
"source": [
"Now, to improve the loss, we'll get ahold of a parameter's data and weight so we can track changes:"
]
},
{
"cell_type": "code",
"execution_count": 86,
"id": "8beeb03c-34bb-49a4-ab08-0e53bc1405ef",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"data: 0.8066425443125735\n",
"grad: 0.10625209712008757\n"
]
}
],
"source": [
"print(\"data: \", n.layers[0].neurons[0].w[0].data)\n",
"print(\"grad: \", n.layers[0].neurons[0].w[0].grad)"
]
},
{
"cell_type": "markdown",
"id": "36ff4a9e-6197-4c77-b3bc-096d4bc9b742",
"metadata": {},
"source": [
"## Gradient Descent: What is it?\n",
"\n",
"Gradient descent is an optimization algorithm used to minimize the loss function. The key idea is to update the parameters of the model in the direction opposite to the gradient of the loss function with respect to the parameters. The gradient points in the direction of the steepest ascent, so moving in the opposite direction reduces the loss."
]
},
{
"cell_type": "markdown",
"id": "0c82bd7f-be0b-4d08-bc87-f901ab50ab26",
"metadata": {},
"source": [
"### The ```n.layers[0].neurons[0].w[0].grad``` is positive, so we will negatively change the parameters to make loss go down. This will be a tiny update in the gradient descent scheme.\n",
"\n",
"Think of the gradient as a vector pointing in the direction of increased loss. We have to reverse it in the direction away from the loss by modifying p.data by a small step size in the opposite direction of the gradient.\n",
"\n",
"### Another way to think about it:\n",
"- Negative gradient: ```(-) loss<----d1---->d2 (+)``` (loss goes down)\n",
"- Negative gradient: ```(-)loss<----d2<----d1 (+)``` (loss goes up)\n",
"- Positive gradient: ```(-)d1---->d2---->loss (+)``` (loss goes up)\n",
"- Positive gradient: ```(-)d2<----d1---->loss (+)``` (loss goes down)\n",
"\n",
"The general formula you can use is the ```parameter update rule```: ```p.data = p.data - n * p.grad```, where:\n",
"- ```p.data``` is the current parameter value\n",
"- ```n``` is the learning rate/step size\n",
"- ```p.grad``` is the gradient of the loss with respect to the parameter\n",
"\n",
"Now, let's use this rule to update and check our grad:"
]
},
{
"cell_type": "code",
"execution_count": 126,
"id": "5497b662-91d9-45e4-9bae-5f6479d6b6ee",
"metadata": {},
"outputs": [],
"source": [
"for p in n.parameters():\n",
" p.data += -0.01 * p.grad"
]
},
{
"cell_type": "code",
"execution_count": 127,
"id": "9c1277f3-f7b2-4c1c-9c1e-1c6fa336be36",
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"data: 0.8055800233413727\n",
"grad: 0.10625209712008757\n"
]
}
],
"source": [
"#check: data decreases, which will make the loss go down\n",
"#low loss means our predictions are matching the targets. \n",
"#Now, data is slightly closer to the targers.\n",
"\n",
"print(\"data: \", n.layers[0].neurons[0].w[0].data)\n",
"print(\"grad: \", n.layers[0].neurons[0].w[0].grad)"
]
},
{
"cell_type": "code",
"execution_count": 131,
"id": "1f80a5fc-9425-492c-acb7-86fd1102408a",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Value(data=4.297124484535017)"
]
},
"execution_count": 131,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#reevaluate the loss by recalculating the forward pass of the network\n",
"#Recalculate the loss... has the loss gone down???\n",
"ypred = [n(x) for x in xs]\n",
"ys_value = [Value(y) for y in ys]\n",
"loss = sum([(yout - ygt) ** 2 for ygt, yout in zip(ys_value, ypred)], Value(0.0))\n",
"loss"
]
},
{
"cell_type": "markdown",
"id": "a5156cf3-5431-4927-8ec9-63649170b389",
"metadata": {},
"source": [
"Now we can iterate this process and keep improving the loss by doing a forward pass, backward pass, and then updating parameters. \n",
"\n",
"As we keep iterating, ```ypred``` values should keep getting closer to the targets:"
]
},
{
"cell_type": "code",
"execution_count": 132,
"id": "aba4cb31-0277-41cd-b5d7-de79a7c8f7c2",
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[Value(data=0.6823520520073336),\n",
" Value(data=0.90100678271322),\n",
" Value(data=-0.31648409938487826),\n",
" Value(data=0.6605836018167414)]"
]
},
"execution_count": 132,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ypred"
]
},
{
"cell_type": "markdown",
"id": "4a2732a7-4dcc-41a6-8d7c-1c3e9dc8f0d8",
"metadata": {},
"source": [
"Experiment by continuing to do the forward pass, backward pass, and gradient descent. Keep testing your loss and if your predictions are becoming closer to your targets. Experiment with different step sizes.\n",
"Try to get your predictions to be exactly at the targets!\n",
"\n",
"### Important:\n",
"- Be careful not to get too cocky with the size of the step... we're only aware of the local bell curve, so we can easily go up the other way on the parabola. If you set the learning rate too low, you'll take way to long to converge. But if you go too high, the whole thing gets unstable and you will explode to the other side\n",
"- The loss function has a wacky structure and curve, and we only know about the local ones. "
]
},
{
"cell_type": "markdown",
"id": "2c6e7e3c-eac1-48ef-b3d6-191875366ff7",
"metadata": {},
"source": [
"# We're almost there! Now that you've learned what we need, we'll implement an easy python training loop in the final lesson (so we don't have to do it manually) and recap what we've learned. "
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.11.4"
}
},
"nbformat": 4,
"nbformat_minor": 5
}