{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# softmax回归的简洁实现\n", "\n", "我们在[“线性回归的简洁实现”](linear-regression-gluon.ipynb)一节中已经了解了使用PyTorch实现模型的便利。下面,让我们再次使用PyTorch来实现一个softmax回归模型。首先导入所需的包或模块。" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "%matplotlib inline\n", "import d2ltorch as d2lt\n", "import torch\n", "from torch import nn, optim" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 获取和读取数据\n", "\n", "我们仍然使用Fashion-MNIST数据集和上一节中设置的批量大小。" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "root = '~/dataset/'\n", "batch_size = 256\n", "train_iter, test_iter = d2lt.load_data_fashion_mnist(root, batch_size=batch_size)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 定义和初始化模型\n", "\n", "在[“softmax回归”](softmax-regression.ipynb)一节中提到,softmax回归的输出层是一个全连接层。因此,我们添加一个输出个数为10的全连接层。我们使用均值为0、标准差为0.01的正态分布随机初始化模型的权重参数。\n", "\n", "*这里我们提前使用 [“4.2. 模型参数的访问、初始化和共享”](../chapter_deep-learning-computation/parameters.ipynb) 一节中定义的初始化方法 `params_init(model, init, **kwargs)`*" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [], "source": [ "class MySoftMax(nn.Module):\n", " def __init__(self, **kwargs):\n", " super(MySoftMax, self).__init__(**kwargs)\n", " self.linear = nn.Linear(28 * 28, 10)\n", " \n", " def forward(self, x):\n", " x = x.reshape(-1, 28 * 28)\n", " return self.linear(x)\n", "\n", "net = MySoftMax()\n", "d2lt.params_init(model=net, init=nn.init.normal_, std=0.01)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## softmax和交叉熵损失函数\n", "\n", "如果做了上一节的练习,那么你可能意识到了分开定义softmax运算和交叉熵损失函数可能会造成数值不稳定。因此,PyTorch提供了一个包括softmax运算和交叉熵损失计算的函数。它的数值稳定性更好。" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [], "source": [ "loss = nn.CrossEntropyLoss()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 定义优化算法\n", "\n", "我们使用学习率为0.1的小批量随机梯度下降作为优化算法。" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [], "source": [ "optimizer = optim.SGD(net.parameters(), lr=0.1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 训练模型\n", "\n", "接下来,我们使用上一节中定义的训练函数来训练模型。" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "epoch 1, loss 0.0031, train acc 0.751, test acc 0.775\n", "epoch 2, loss 0.0022, train acc 0.813, test acc 0.795\n", "epoch 3, loss 0.0021, train acc 0.825, test acc 0.810\n", "epoch 4, loss 0.0020, train acc 0.832, test acc 0.811\n", "epoch 5, loss 0.0019, train acc 0.836, test acc 0.823\n" ] } ], "source": [ "num_epochs = 5\n", "d2lt.train_ch3(net, train_iter, test_iter, loss, num_epochs, batch_size, None, None, optimizer)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## 小结\n", "\n", "* PyTorch提供的函数往往具有更好的数值稳定性。\n", "* 可以使用PyTorch更简洁地实现softmax回归。\n", "\n", "## 练习\n", "\n", "* 尝试调一调超参数,如批量大小、迭代周期和学习率,看看结果会怎样。\n", "\n", "\n", "\n", "## 扫码直达[讨论区](https://discuss.gluon.ai/t/topic/740)\n", "\n", "![](../img/qr_softmax-regression-gluon.svg)" ] } ], "metadata": { "kernelspec": { "display_name": "Python [conda env:pytorch]", "language": "python", "name": "conda-env-pytorch-py" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.9" }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": {}, "toc_section_display": true, "toc_window_display": false } }, "nbformat": 4, "nbformat_minor": 4 }