" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "你希望找到这样的 **参数**（权重） `a`，使得数据点和直线`x@a`之间的 *误差* 尽可能小。需要注意的是这里`a`是未知的。对于回归问题最常用的 *误差函数* 或者说 *损失函数* 是 **均方误差(MSE)** 。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def mse(y_hat, y): return ((y_hat-y)**2).mean()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Suppose we believe `a = (-1.0,1.0)` then we can compute `y_hat` which is our *prediction* and then compute our error.
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "假设我们取`a = (-1.0,1.0)`，那么我们就可以计算 *预测值* `y_hat` ，随后我们可以算出误差来。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "a = tensor(-1.,1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "tensor(7.9356)" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "y_hat = x@a\n", "mse(y_hat, y)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAW4AAAD8CAYAAABXe05zAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAALEgAACxIB0t1+/AAAADl0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uIDMuMC4yLCBodHRwOi8vbWF0cGxvdGxpYi5vcmcvOIA7rQAAIABJREFUeJzt3X+QHOV5J/Dvs6MRjES8Kzl7Z1hpg3Bx0gUkI9gj5NYVR5KDsANiLRyBz1ww5iwTUzkDjoIoCAjKlESoQ47rnLLlCz8SY0ABsRbCnEyQnFQoC3v3Vj8sQEZAQFrhIFvaNUZraVg990d37/b0dPf0z5nume+naku7Pd09r3pXz7563ud9X1FVEBFRfrQ1ugFERBQOAzcRUc4wcBMR5QwDNxFRzjBwExHlDAM3EVHOBArcItIhIk+IyCsi8rKI/H7aDSMiIndTAp73NwD+r6p+WkSmApiWYpuIiMiH1JqAIyIfALALwFnK2TpERA0XpMd9FoDDAB4UkY8AGATwZVV9z36SiKwEsBIApk+ffsG8efOSbisRUdMaHBz8hap2Bjk3SI+7B8AOAL2q+qKI/A2AX6nqX3ld09PTowMDA2HaTETU0kRkUFV7gpwbZHDyIICDqvqi+fUTAM6P2jgiIoqnZuBW1Z8DOCAic81DSwC8lGqriIjIU9Cqkj8H8IhZUfI6gGvTaxIREfkJFLhVdSeAQLkXIiJKF2dOEhHlDAM3EVHOBM1xExG1jP6hYdy3dR8OjYzhjI4SVi2di76FXY1u1gQGbiJqSV7BuX9oGLdu2oOx8jgAYHhkDLdu2gMAmQneDNxE1HL8gvN9W/dNHLeMlcdx39Z9DNxERI3iFZzvenovjh4ru15zaGSsHk0LhIOTRNRyvIKwV9AGgDM6Smk1JzQGbiJqOVGC8PDIGHrXbUP/0HAKLQqHgZuIWs6qpXNRKhZCX2flwhsdvBm4iajl9C3swtrl89HVUYIA6OoooaNUDHStNVDZSBycJKKW1Lewq6JKxFlp4qfRA5XscRMRYbIXHqTn3eiBSva4iSjXgs5yDHKe1Qu3zh0eGYMAsG83UyoWsGrpXDQSAzcR5VbQWY5hZ0Pa0yhZnP7OwE1EuRV0lmOc2ZDOXHgWMMdNRLnlNUjoPB70vLxg4Cai3PIaJHQeD3peXjBwE1FuuU2kcRs8DHpeGP1Dw+hdtw1zVj9T9xmVzHETUW5Zuecg1SJBzguq0Uu/iqrWPiuknp4eHRgYSPy+RERZ0LtuG4Zd8uNdHSW8sHpxpHuKyKCqBtrbl6kSIqKQGj3YycBNRBRSowc7GbiJiEJaNK8z1PGkBRqcFJF/A/AugHEA7wfNwxARhZXFmYpOz+x+2/X49lcO1+X9w1SVLFLVX6TWEiJqeY2u1giif2i44dubMVVCRJnhNzU9K/zaUq8cd9AetwL4gYgogG+p6gbnCSKyEsBKAOju7k6uhUTUMtKq1kgy/eLXlnqtGhi0x92rqucD+ASAG0TkD5wnqOoGVe1R1Z7Ozvok6ImouaRRrWGlX4ZHxqCIv/2YV1s6SsW6pXMCBW5VPWT++Q6ApwBcmGajiKg1pTE1Pen0i1cb1yw7J3Ibw6qZKhGR6QDaVPVd8/OLAdydesuIqOUkPTUd8E5tWLu2h32fNNoYVs0p7yJyFoxeNmAE+u+q6j1+13DKOxHVk18O22t6utvONmuXz29Y9UqiU95V9XVV/Yj5cU6toE1EVE+1cthuqQ1n0AayV73ih+WARJRrtXLY1ibAXR0lCIyFoLzyDHnZWIHLuhJRrgUpIXRuP+aVPsnLxgrscRNRrkUpIUyjeqWeGLiJKNeiBGG39EkjBybDYqqEiGJr5MJQUcvzsrh7e1AM3EQUSxYWhspzEI6CgZuIXAXtRftVdUQJpnlY1rXRGLiJqEqYXnSSC0NlofeeBxycJKIqYdb3SHJhqDws65oFDNxEBMDo7fau24Y5q59xrXEG3HvRSZbWNXoT3rxgqoSIqlIUXtx60UkuunRGRynXE2PqhYGbiFxTFE5+vegwVR1+g4+rls6t+gWSp4kx9cLATUS+qQgBEqvuqDX4mIUlU/OAgZuIPFMUXR0lvLB6cWLvE6R0sNVqsqPg4CQReS59umhestsQcvAxGQzcRIS+hV244oIuiO2YAnhycDjy3oxu0thTshUxcBMRAGD7K4dT31wg76vyZQVz3EQEIFwaI+q0dA4+JoOBm4gABK+hjjot3Rns1195HgN2REyVEBGA4GmMKNPS3faFvPHxnVh49w8SzaG3Cva4iQhA8DRGlMoQrwk+R4+VuYhUBAzcRC2kVm46SA11lGnpfkE9zhKwrSpwqkRECiIyJCJb0mwQEaXDLV1x66Y9oVMVXimVRfM6Jxap6l23reK+tcr9WMcdTpge95cBvAzgAym1hYhicutRA0aqwq2XHKW365ZSWTSvE08ODlcMWN74+E7c9fRe3HnZOVg0rxOP7HirqtzQwjrucAIFbhGZBeCPAdwD4OZUW0REkbhVe6x6YhegQPmkV8iM1tt1plR6123zzGFbbfBqAeu4wwva4/4agL8E8FteJ4jISgArAaC7uzt+y4goFLcBwPK4d8C2JNHb9Qv+fm3oYh13JDVz3CJyKYB3VHXQ7zxV3aCqPara09mZ7PoGRFRblJ5zUr3dKMFfALywejGDdgRBBid7ASwTkX8D8BiAxSLynVRbRUShhQ2eXR0lrF0+P5HA6TZgWQvz2tHVTJWo6q0AbgUAEflDAH+hqlen3C6ilhV1OrnbJgTFglTluEvFQmIB22Lda83mvRgZK1e85tUG5rWjYx03UYbE2eXcawKN27E00hPWgKVfZQvXJ0mGqNYevAirp6dHBwYGEr8vUbPrXbcttQ0NovbkqT5EZFBVe4Kcyx43UYaktdFAnJ48ZQ8XmSLKkLQ2GoiyMBRlF3vcRBmS1C7nzrSIW/oF4FTzvGLgJsqQJDYacEuLCNxnLrIkL584OEnUZLwGOJ3Bu9gmOO3UKRg5VuZgZQaEGZxkjpuoyXilPxRGdYoA6CgVATHWEomzUiA1BgM3UZPxSn9YJYVvrPtjTD9lStUaIhyszA/muIlSUs+6aft7tZeKKBakIjA7BzjTKjuk+mDgJkpBPeumne81MlZGsU0wY1rRM38dZRcbyg4GbqIYvHrVfnXTSQRu+/u2iWDcUWRQPqmYNnUKhu642PX6pMoOqTEYuIki8utVp5mKcL6vM2gHea8kyg6pcRi4iSLy61WnmYrw2jE97HsF2RiYsolVJUQR+fWqvTbUTSIVEaTXzrRHc2OPmygiv151mqkIvynsADBjWhF3XnZO1XtxdcDmwcBNFJHbAB8AHDvxPvqHhlNLRXi9r+U35ZNVx7g6YHNhqoQoor6FXVi7fL4xC9Hm6LGy7yzE/qFh9K7bhjmrn0Hvum2hZyta79vlkcN2m0jD1QGbCwM3UQx9C7sw/ZTq/7iOlcfxlY27qoKz1fMdHhmLNdW8b2EXXli9GOLxujMPzgk3zYWBmygmr+A3rloVnJPu+QZdvzutdb6pMRi4iWIKEvys4By35+tMsyya1xmoeiXNKheqPwZuopjcgqIbq5rDTZDg75ZmeXJwGFdc0DWx6l9XR8l1B3d7XtzvPMoHVpUQxeQs/XObgg5gogQv6lRzrzTL9lcOB9pImBNumgcDN5GHMHXP9qDoLL0DJoNznPpuDjCShYGbyEWcuudawTlqz5cr+pGlZo5bRE4VkR+LyC4R2Ssid9WjYUSNFLf6wyrXW3/leQCAmx7fGalm244DjGQJ0uM+DmCxqv5aRIoA/lVEnlXVHSm3jahhvNIPwyNj6F23LVCao39oGKue2DWxocHwyBhWPbELQLTZilzRjyw1A7cauwn/2vyyaH4kv8MwUYZ0TCvi6LGy62tWuqJW+uSup/dWbQ9WHlfc9fTeyMGWA4wEBCwHFJGCiOwE8A6A51T1RZdzVorIgIgMHD58OOl2EtWVxxLXVfzSJ16B/+ixcuy0CbW2QIFbVcdV9TwAswBcKCLnupyzQVV7VLWns7Mz6XYS1dXomHvQdROlqoO7qlMcoSbgqOoIgB8CuCSV1hBlRJhKDa9znYtPOXGRJ4oqSFVJp4h0mJ+XAHwcwCtpN4yokdwqOIptgmKhclknv6qONcvOQbHNaxkoA2uwKYogVSWnA3hYRAowAv1GVd2SbrOIGsurgsPtmN+kHOt8r40P2kQwZ/UzrBChUESDjsKE0NPTowMDA4nflyiPnGWBXkrFAtcPaWEiMqiqPUHO5SJTRCm7b+u+mkEbYM6bgmPgJkpZmDw2c94UBAM3UcqSqFAhsmPgJkqZa4VKQaoqTrjuCAXF1QGJUpZEhQqRHatKiIgygFUlRERNjKkSohrC7IRDVA8M3EQ+4uyEQ5QWpkqIfMTdCYcoDexxU0PkJf3ADXopixi4qe680g8Dbx7B9lcOZyqYc4NeyiKmSqjuvNIPj+x4C8MjY1B4bzTQPzSM3nXbMGf1M3XZRYYb9FIWscdNdeeVZnDOKBgrj0/sz9g/NIw1m/dixLYzTT0GCrlBL2URAzfVnVf6wc3RY2Xc3r8HTw4OV/XSgcmBwjQDKTfopaxhqoTqzi394LdPzCMvvuUatC0cKKRWw8BNdde3sAtrl89HV0cJAqCro4TPXtTteX6tVRk4UEithqkSagi39MOWXW9X5LCD4EAhtSL2uCkz1iw7pyqFUssVFzD/TK2HPW7KDLcKjveOv+/bC9/+yuF6NY8oMxi4KVOcKRTnZB0nDkxSK2LgpkyzgvhXNu7CuMsopX1gMi/T6IniqpnjFpHZIrJdRF4Wkb0i8uV6NIzI0rewC/9rxUd8ZzBaPfNaMy+JmkGQHvf7AL6iqv9PRH4LwKCIPKeqL6XWqt0bgefvBkYPAu2zgCV3AAtWpPZ2lB1eveZaMxi9ptFbMy+JmknNwK2qbwN42/z8XRF5GUAXgHQC9+6NwNP/EyibucvRA8bXAIN3k6u19rXfDEavXPfRY2X0Dw0zeFNTCZXjFpEzASwE8KLLaysBrASA7m7vyRQ1PX/3ZNC2lMeM4wzcmRI3p+y8/r3j73uufV3rvn7T6NOeEk9Ub4EDt4icBuBJADeq6q+cr6vqBgAbAGOz4MgtGj0Y7riX3RuBZ28Bxo4YX5dmAp+4l8E/IXF3hnG73suhkbGavyRWLZ2LGx/f6Xk9UTMJNAFHRIowgvYjqrop1Ra1zwp33M3ujUD/lyaDNmB8vmklsKYDWNNufNxzhnEuhRZ3Zxi36720l4o1Bx77Fnaho1R0vZ5T4qnZBKkqEQB/B+BlVb0/9RYtuQMoOv6hFUvG8aCevxs46TZpQ1GxeGj5PeCp6xm8IwizM4xzDe3b+/cEXh2wVCxABIF+SbjNvOSUeGpGQXrcvQD+O4DFIrLT/Phkai1asAK47OtA+2wAYvx52dfDpTjCpFV03Aj0Trs3AuvPNXro689lcHfw6sU6j7uV6X1nx1ue950xrVix+NTa5fMxcsx95qTzl4Tb4lVrl89nfpuaTpCqkn+F/6qbyVuwIl4uun2WUY0SlDPQ794IfO8GYPyE+foBYNMXjI/22U1dnhh0wHHV0rlVMxrderdhUiKlYgF3XnZO1fvdt3Vf4O3DuHY2tYLmXGRqyR1Am3u+05Uzf/7sLZNB28kqT9xyM3DvnMl8+b1zct8rDzOJJWjvNszAoFfvmNuHEVVqzinvVm/YXlVSmOoejKVQnT+3D2q6KY8BA39Xfc1T15vveTSXE4f8BhzdAmqQ3m3HtCKOeqQ67Lo6Sp734vZhRJWaM3AD7umWLTcDgw8CetL4ujgduOxryQVXHZ8M+qMHjCqWnKRX+oeGPQcM7b3msLXbtTZBAIL1npkCIZrUvIHbzaX3Gx+1lGbW7nUHYkatjM/+tFIkXqxc8u39e/DIjrcm6nKsVMrAm0ew/ZXDrsF8tMbGCAURnN/djvu27sNNj+9kb5oogNYK3EF94l4j7aFeg2qC6j3Ja3Cb/Wlfk6U0wzjWgDSL3wBiqVjAonmdWHj3D1xTHmPlcddgDhi95PZS0Xc97XFVvPDa5C/JeuzcTpR3zTk4GdeCFcCnvmn0vCeYhTXts4Gezxs587Ds1SvWmiyjBwCo0cMfO2J8bh8ArUNJot8A4hUXdOHJwWHfPLXzV5i9xloi1COFmchD1IrY4/ZSqySx+6LKwc/idGPw03Xij8leveK2JotdeQwYeAD1SLd4rfPR1VHC9lcOBy7ns7N+GXjVYAe9noiqsccd1YIVwC1vAGtGjY/bDgF9f2tOHAKqSt+dsz8DTRJy9GWtdAuQ6AQhv3K7qAHUyotHnW7OaepE3hi4k7RgBXDTT41AvnyD/+zPMGuv2I0erE6zWL3x3RuNjwj15adMmfxRmDGtOFFT7RdABUDvh2f61li7/VKohTXaRP6YKklLrVTLkjsq1x2v4jEA2j7Le+nbZ28Bjr9bma4ZO2LMArXa5OC2p+NvyicnPnebIQkAHaUi1iwzZjn6lQi61WAvmtdZUYXi/JpVJUT+RIMU2obU09OjAwMDid+36fhVlZx9MbDru5UBulgyeu6bViJ0VUv7bON/A473Pjl6EIdOfhB//f4KbD750YmXuzpKeGH1YgDcy5GoHkRkUFV7Ap3LwJ1hXlu4rT833FosABQCWTMyeV9Hb/+kTmblT0LQBoV0ZH/iEFGzYOBudi6BF8USjmMqTimPul7yc3TiQ2v2G1+ECPzjUsS7eio+oO/iHenEgfNX4b8s+2LcvwEROYQJ3ByczCOPpW/X6rU4rtUDgSd0Ctae+JPJAyGWvS1oGR14F20CfAiH0TP4l9AmWliLKI84OJlXLoOfD3/3GRxp+yLunPL3mCm/BgAcxWlYU/5TDH7gjyZPDLvsrU3FhJqxI5PL3UrBmGmag3VZiPKOgbuJnNFRwuaRj2LziY9WHBcA6+3ldTUrWiKwlgfI+LosRM2AqZIm4lYzLQA+e1F3ZRVIRaolBfaJQgDw8LLJuvI17cbXRBQZe9xNJNS61fZUy+6NldP3pQ3Qkzhe7EDxxAjaoux/ZOXRH14GvPHPla+98c9GALcrzTQW92IvnagmVpWQr9ce/CLmvPlY+P+aWXXjzgAdihgLegVZipco58JUlbDHTb4+fO23gN1LzHryAxO9cV/OdVkiU2OnoV/uBw4OAOX3jMPSBlxwLQM6tSz2uCm6iQlCB7yrSmL1uGsoTAUu/wbTK9QU2OPOidxPJa+1HgsAzPlYdY47KeMnKtdhcebqmTenJsXA3SDOxZ3C7vwSN+jX7ZfGNZvdByiTMn5isoLlezdUbgg9dgTo/5LxOYM3NZGaqRIReQDApQDeUdVzg9yUqZLaetdt89y8wFrcyYvbin6lYmFiKdZa4l4f25abKzeJiE38JxU5F9giyqCkp7w/BOCSWC2iKl4bFATZuMBtj8gw233FvT62S+8H1oxMbkKx/NseG1CIkWqRGut5t8/yn8YfYoo/UR7UTJWo6r+IyJnpN6W1eG0XFmTnlzhBP4nrE1crV757Y3UaxFKYagyGWoOkbqJuWrHlZmDwIWPQVQrABZ9jJQtlQmIzJ0VkpYgMiMjA4cOHk7pt0/LbLqwWr+AedLuvuNfX3YIVwF8dNnrm9g2cSzMnq0qW3OG+gXNbMVpp4pabjVJEayq/jhtf391p7jCU7gbORH4ClQOaPe4tzHEnK+oAYe5z3GlJsqrkrpmTQdtPYSow9bTJDTC4wBZFlPh63Azc2ZObqpK8ilN/XpoJnPMp4NUfVG+CQeSBgZsorqA97ii49C25SLSqREQeBfAjAHNF5KCIXBe3gUSZd8Hn0rv36AFjDfM17cYviDXtzJdTKEGqSj5Tj4YQZYpVPTL4YO21WeLgOuYUAdfjJvJy6f3AnUdtdeZi5K9LMyc/bysm937WOua7Nxo9cFaukAcuMkUUh7OSJQnFUtVG0Ljs65PrsdRa2Ityibu8NwgrNVrYREA1q0hmnhVtfRYrGDtZwTnIlnNcXCuXGLgboGlroym63RuBLTcCJ94Ldr6zp12hxnosrpeYa6ezR54LDNwNEGfRKGoRbmkOZ7rDa+p++2xzzZWo/16l+tridOCyrzGgZwTX426AzK3/QdkTZP1yoDodYu0o5LceS00uAb/8nlGWuOkL7JXnDAN3QuIsGkU0wQqc9ny5PaAGyXFHMXrAWLvcntphrjyzmCpJCHPcVBdu6ZZ6YI88dUyVNIAVnFlVQqlyS7ekUZLo5Jwg5KyiYVCvK/a4U8LSQGoI1x65y8BkVF5liYWpxmSkMtMsUbGqJGFhg3CUtInbewDswVMC7ME8tpBliQzggTFwJyhKEA5bGuj2HsU2AQQoj09+f5gzp0TZ0x2lGcDxUeBkjZx5lLLEYgn4yH8D9j5Vnc7hzkITkt5zsqVF2Z8xbGmg23uUT2pF0A7yvkShLFhhbKK8ZgS45Q2g75uVOww5WWWJYbeCK48Zuwe55eCtnYW23Bzuni2Og5M1RKnPDlsaGKbWm3XhlBrnwKffAGTSZYmDD032urnXZ00M3DVEqc9etXSua3rFaz9Jr/fwOpeoLrwmDDlrzUszgLERADGWv7XKGq29Pu3Hra8ZvCcwVVJDlE19+xZ2Ye3y+ejqKEFg5Lb9ctNu71FsExQLEup9ierGmWZZ/i2fNIt4HLefYv78Dz7k/rrX8RbFHrcHe5VHx7QiTpnShtGxcuDqjr6FXYEHEb1qwN2OcWCSMsneO3emWM6+GBj6B2D8hPf11o5DXhOK/CYaJblJdE6wqsQFZ0ESJcxrkpAzh+2116cUgDtdBjd3bzSm6p8sVx4vTAUu/0blL5OMB3eWA8bElf6IGsSZ47b0XOee415/rndNeftsI52zeyPwvRtcevwCiBhL32ZgEJRT3mPiSn9EDTKx1+dDwapKRg9638t67fm7PdI0ClgdV2sQ9Jf7gSOvZ34qPwO3C670R9RAl94fvOfrN4vTqjf3C+5O9l2LRg+Yy96uBIrTgPKxzARzVpW4cKvyEACL5nVOfN0/NIzeddswZ/Uz6F23Df1Dw3VuJRFhyR3uGzYXphqvAeEnDFVRcw0WnVxsa8vNDd3QOVDgFpFLRGSfiOwXkdVpN6rR+hZ24fzu9opjCuDJwWH0Dw1PDF4Oj4xBAQyPjOGmx3fi9v49DWkvUctasALo+9vKUsTSzMqBySV3GIE8KeUxYOABs6evkz3zh5cl9x411BycFJECgJ8B+CMABwH8BMBnVPUlr2vyPjjZPzSMmx7f6boaQ5eZLnFLpQiA9Veex8oToqxxVpW0TQVO+pQnRuU1iBpA0oOTFwLYr6qvmzd/DMDlADwDd1ZEXVr1vq37PJfQ8RugVPNaBm6ijHGbBeqcWn/mR4GDP443ld8+dT9FQQJ3FwB79v8ggN9zniQiKwGsBIDu7u5EGheHsxZ7eGQMt24yUhm1AmutdUis+4W9logyxG0QdPdG4OkbJ9cVd+WzvnmddiQKkuN2m69a1WpV3aCqPara09nZ6XJJfUVZ1c/iVT0iMAYuVy2d6zmJl5UnRDm2YAVw2yEj5WFNw5c2oDgdxlrks4Gez3tfLwXv1xIUpMd9EMBs29ezABxKpznJiVOL7bZIlAD47EXdE731gTeP4JEdb1X8BuNaIkRNolZJ4i/3V5YOWqyp+ykL0uP+CYCzRWSOiEwFcBWAzek2Kz6vnm+QHrHbIlHrrzwPX+2bP3HOV/vmY/2V5wVeSIqImsg1mx298kKsgcmwAk15F5FPAvgagAKAB1T1Hr/zs1BVwvVGiChPEp/yrqrfB/D9WK2qM+66TkTNqqmnvLstrcrd14ko75o6cDt99ts/wguvTS4NGaZEkIgoK1pmrZLb+/dUBG0LN+AlorxpmcD96IseK4iBk2aIKF9aJnCP+1TPcNIMEeVJJnPcaQwgFkQ8gzcnzRBRnmSux+22ZOqtm/bEXu/6M7832/V474dncmCSiHIlc4E7zhojfr7aNx9XX9SNghirjBREcPVF3XjkC78f675ERPWWuVRJmvs9frVvfsW0dSKiPMpc4E5qv0dOtCGiZpW5VInbfo9hV91LK09ORJQFmQvcbivzhV0YKq08ORFRFmQuVQK4rzESRpp5ciKiRstcjzsJcdbiJiLKuqYM3EnkyYmIsiqTqZK4uBY3ETWzpgzcQPw8ORFRVjVlqoSIqJkxcBMR5QwDNxFRzjBwExHlDAM3EVHOMHATEeWMqM+WXpFvKnIYwJsxbvHbAH6RUHOSxHaFk8V2ZbFNANsVVhbbFbdNv6OqnUFOTCVwxyUiA6ra0+h2OLFd4WSxXVlsE8B2hZXFdtWzTUyVEBHlDAM3EVHOZDVwb2h0AzywXeFksV1ZbBPAdoWVxXbVrU2ZzHETEZG3rPa4iYjIAwM3EVHONCxwi8ifiMheETkpIp4lNCJyiYjsE5H9IrLadnyOiLwoIq+KyOMiMjWhds0UkefM+z4nIjNczlkkIjttH78RkT7ztYdE5A3ba+fVq13meeO2995sO5748wr4rM4TkR+Z3+vdInKl7bVEn5XXz4rt9VPMv/t+81mcaXvtVvP4PhFZGqcdEdp1s4i8ZD6f50Xkd2yvuX4/69Cmz4nIYdt7/w/ba9eY3/NXReSapNoUsF3rbW36mYiM2F5L61k9ICLviMhPPV4XEfm62ebdInK+7bV0npWqNuQDwH8GMBfADwH0eJxTAPAagLMATAWwC8Dvmq9tBHCV+fk3AfxZQu36awCrzc9XA7i3xvkzARwBMM38+iEAn07heQVqF4BfexxP/HkFaROA/wTgbPPzMwC8DaAj6Wfl97NiO+dLAL5pfn4VgMfNz3/XPP8UAHPM+xTq2K5Ftp+fP7Pa5ff9rEObPgfgf3v8vL9u/jnD/HxGvdrlOP/PATyQ5rMy7/sHAM4H8FOP1z8J4FkAAuAiAC+m/awa1uNW1ZdVtda26xcC2K+qr6vqCQCPAbhcRATAYgBPmOc9DKAvoaZdbt4v6H0/DeBZVT2W0Pt7CduuCSk+r5ptUtWfqeqr5ueHALwDINDssJBcf1Z82vsEgCXms7kcwGOqelw/8qSCAAAD4ElEQVRV3wCw37xfXdqlqtttPz87AMxK6L0jt8nHUgDPqeoRVT0K4DkAlzSoXZ8B8GhC7+1JVf8FRufMy+UA/l4NOwB0iMjpSPFZZT3H3QXggO3rg+axDwIYUdX3HceT8B9V9W0AMP/8DzXOvwrVPzz3mP9lWi8ip9S5XaeKyICI7LDSN0jveYV6ViJyIYye1Gu2w0k9K6+fFddzzGcxCuPZBLk2zXbZXQej92Zx+37Wq01XmN+bJ0Rkdshr02wXzHTSHADbbIfTeFZBeLU7tWeV6tZlIvJPAD7k8tJtqvq9ILdwOaY+x2O3K+g9zPucDmA+gK22w7cC+DmMALUBwC0A7q5ju7pV9ZCInAVgm4jsAfArl/MCPa+En9U/ALhGVU+ahyM/K7e3cDnm/Dum8vNUQ+B7i8jVAHoAfMx2uOr7qaqvuV2fcJueBvCoqh4Xketh/E9lccBr02yX5SoAT6jquO1YGs8qiLr/XKUauFX14zFvcRDAbNvXswAcgrGQS4eITDF7Ttbx2O0SkX8XkdNV9W0z2Lzjc6sVAJ5S1bLt3m+bnx4XkQcB/EU922WmI6Cqr4vIDwEsBPAkIj6vJNokIh8A8AyA283/Slr3jvysXHj9rLidc1BEpgBoh/Ff4CDXptkuiMjHYfwy/JiqHreOe3w/4wajmm1S1V/avvw2gHtt1/6h49ofxmxP4HbZXAXgBvuBlJ5VEF7tTu1ZZT1V8hMAZ4tRETEVxjdrsxqZ/+0w8ssAcA2AID34IDab9wty36ocmxnArLxyHwDXkeg02iUiM6x0g4j8NoBeAC+l+LyCtGkqgKdg5AD/0fFaks/K9WfFp72fBrDNfDabAVwlRtXJHABnA/hxjLaEapeILATwLQDLVPUd23HX72ed2nS67ctlAF42P98K4GKzbTMAXIzK/3Gm2i6zbXNhDPb9yHYsrWcVxGYAf2pWl1wEYNTslKT3rNIYhQ3yAeBTMH4jHQfw7wC2msfPAPB923mfBPAzGL85b7MdPwvGP679AP4RwCkJteuDAJ4H8Kr550zzeA+A/2M770wAwwDaHNdvA7AHRhD6DoDT6tUuAP/VfO9d5p/Xpfm8ArbpagBlADttH+el8azcflZgpF6WmZ+fav7d95vP4izbtbeZ1+0D8ImEf9ZrteufzH8D1vPZXOv7WYc2rQWw13zv7QDm2a79vPkM9wO4tp7Pyvx6DYB1juvSfFaPwqiGKsOIWdcBuB7A9ebrAuAbZpv3wFYll9az4pR3IqKcyXqqhIiIHBi4iYhyhoGbiChnGLiJiHKGgZuIKGcYuImIcoaBm4goZ/4/Hxs2UrsTgccAAAAASUVORK5CYII=\n", "text/plain": [ "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "到现在我们已经指定了 *模型* 的类型（线性回归），以及 *评估标准* （或者说 *损失函数* ），接下来我们需要处理 *优化* 过程；即，我们如何才能找到最优的`a`呢？我们如何才能找到 *拟合* 最好的线性回归模型呢？" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Gradient Descent 梯度下降" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We would like to find the values of `a` that minimize `mse_loss`.
\n", "\n", "我们希望找到最小化`mse_loss`值的`a`的值。\n", "\n", "**Gradient descent** is an algorithm that minimizes functions. Given a function defined by a set of parameters, gradient descent starts with an initial set of parameter values and iteratively moves toward a set of parameter values that minimize the function. This iterative minimization is achieved by taking steps in the negative direction of the function gradient.
\n", "\n", "**梯度下降** 是一个用于优化函数的算法。给定一个由一组参数决定的函数，梯度下降从一组初始的参数值开始，不断向能够最小化函数值的参数值迭代。这个迭代式最小化的结果是，通过向函数梯度的负方向不断递进而得到的。\n", "\n", "Here is gradient descent implemented in [PyTorch](http://pytorch.org/).
\n", "\n", "这里是 [PyTorch](http://pytorch.org/)中梯度下降算法实施的细节。" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Parameter containing:\n", "tensor([-1., 1.], requires_grad=True)" ] }, "execution_count": null, "metadata": {}, "output_type": "execute_result" } ], "source": [ "a = nn.Parameter(a); a" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "def update():\n", " y_hat = x@a\n", " loss = mse(y, y_hat)\n", " if t % 10 == 0: print(loss)\n", " loss.backward()\n", " with torch.no_grad():\n", " a.sub_(lr * a.grad)\n", " a.grad.zero_()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "tensor(7.9356, grad_fn=
\n",
" ![]()
\n",
"
\n", " \n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n",
"\n",
"\n",
"\n"
],
"text/plain": [
"\n", " \n", "
\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
\n", "\n", "实际上，我们并没有立刻计算整个数据集，相反，我们采用 *mini-batches（小批次）* 的策略。" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Vocab 术语" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "- Learning rate 学习率\n", "- Epoch 轮次\n", "- Minibatch 小批次\n", "- SGD 随机梯度下降法\n", "- Model / Architecture 模型/架构\n", "- Parameters 参数\n", "- Loss function 损失函数\n", "\n", "For classification problems, we use *cross entropy loss*, also known as *negative log likelihood loss*. This penalizes incorrect confident predictions, and correct unconfident predictions.
\n", "\n", "对于分类问题，我们使用 *交叉熵损失* ，也被称为 *负对数似然损失* 。该损失函数将惩罚那些置信高的错误预测和置信低的正确预测。" ] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" } }, "nbformat": 4, "nbformat_minor": 1 }