{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "
\n", " 線形代数の使いどころ\n", "
\n", "
\n", " cc by Shigeto R. Nishitani\n", "
\n", "\n" ] }, { "cell_type": "markdown", "metadata": { "heading_collapsed": true, "slideshow": { "slide_type": "slide" }, "toc": "true" }, "source": [ "# Table of Contents\n", "

1  線形代数
2  エラー訂正符号
2.1  チェック行列
2.2  エラー検出の原理
2.3  Hammingの生成行列(generator matrix)
2.4  GF(2)
3  Deep Learning
3.1  手書き数字認識(MNIST)
3.2  load_mnist
3.3  neuralnet_mnist
" ] }, { "cell_type": "markdown", "metadata": { "hidden": true }, "source": [ "
\n", " 線形代数の使いどころ\n", "
\n", "
\n", " cc by Shigeto R. Nishitani\n", "
\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# 線形代数\n", "\n", "この頃大学でしかやらない数学の領域の代表例が線形代数です．情報科学の多くの領域で，線形代数はその基本となる概念を提供してくれます．線形代数の概念が身につくと問題の見通しがよくなるんです．\n", "\n", "講義の内容としては，\n", "* 応用\n", "* 計算アルゴリズム\n", "* 概念\n", "\n", "というのが必要なのですが，ここでは，「応用例」と「\n", "概念を視覚化」する様子を紹介します．\n", "\n", "はじめに，エラー訂正符号の計算をして見ましょう．" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# エラー訂正符号\n", "## チェック行列\n", "\n", "次の計算をしてみてください．\n", "$$\n", "\\begin{align}\n", "H & = \n", "\\left(\\begin{matrix}0 & 0 & 0 & 1 & 1 & 1 & 1\\\\0 & 1 & 1 & 0 & 0 & 1 & 1\\\\1 & 0 & 1 & 0 & 1 & 0 & 1\\end{matrix}\\right)\n", "\\\\\n", "v_1 & = \\left( \\begin{matrix}0\\\\0\\\\1\\\\0\\\\0\\\\0\\\\0\\end{matrix}\\right)\n", "\\\\\n", "H.v_1 = ?\n", "\\end{align}\n", "$$\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAANUAAABLCAMAAADQ8jKCAAAAPFBMVEX///8AAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAo1xBWAAAAE3RSTlMA\nMquZdlQQQOkwRIlmzd0i77ts7uXj/QAAAAlwSFlzAAAOxAAADsQBlSsOGwAAAvNJREFUaAXtm21z\nokAQhFdAkii+Hf//v4aBqwoDWdq292oLb/lgIs3M+MwsoJ0YDv24VeEdtusEE8Khr5thO74DVOgM\npeqN6vAWQD8QnaNq63N3jiNq8k/Rf/6bp7oMq7C93mJVNXnIery3sdy2P5nsqLq75T5f7PGXTZND\nW9VVH6dKKTuqx3gZPMZKa7K1qYmlnnqYTHZU/Uh16iNXQ03ORdX2tZU+9c3UusWjJo/Jkg1j8dKW\nyeezuvVnkw/Tj1WgJi8Lr9IPO5JBe6pxVnEqRc5GpS0xEJ2NKkyXg+P21eJV2bCSLbGxR4uHWfL5\nCgyPhx3YxS6/mmypZ4Xt6XJLJjuq6TZbb9+FX5UNIdnLXvZjkdxRhau9Y7pH3whqcjaq9jy8i49C\nBU0OdXXvL1X3W59tX0LZzypWcG/7C9V+JlZmVWaVtwNlBebtP1O9zIrpVt5j/4dZARtTHADIDmSm\nuJ8VsDFFHxJkBzJT3FEBG1PzIQPIDmSquKMCNuawBpSPfSA7kKnijgrYmFTi9WkAsgOZKj6nesIl\nEmYFsgPZmvR88TkVsDG5xHa020B2IFuqV6k2bUwusSOyJ7fJ737RQ+WKz2eVdBGsqEB2IPNUH59f\n02sALudw0POLYEUleqhU8T+fs78LAxuTSrymAtmBTBWfr8C/98mojUklXlNpHipV3FEhl1NbgSg7\n8FCZ4p4K2JiaDymapExxT7VeNvvcU6j2M7cyqzKrvB0oKzBv/5nqZVZMt/IeW2YVRB9SC2eil7Pa\n/HdK0YfUwploRwWcRM2HRC5nyuKOajjFtz7Ciz6kFk5FM1SiD6mFU9EElegCaeFcNEEl+pBaOBdN\nUUkmKHA57ba9cVJz0QQVtwhWby60cC6aoEI25XazgxhOObAMlehDauFUNEMFbEo0Ky2cimaokE25\neboPzJqNyUR7KuAkaiYocjkTFvdUqwvXTncUqv0MrsyqzCpvB6YV+I7fgm7ti8NNc8rb30TVx29B\nN034BtvLP0gXdZhuAAAAAElFTkSuQmCC\n", "text/latex": [ "$$\\left[\\begin{matrix}0 & 0 & 0 & 1 & 1 & 1 & 1\\\\0 & 1 & 1 & 0 & 0 & 1 & 1\\\\1 & 0 & 1 & 0 & 1 & 0 & 1\\end{matrix}\\right]$$" ], "text/plain": [ "⎡0 0 0 1 1 1 1⎤\n", "⎢ ⎥\n", "⎢0 1 1 0 0 1 1⎥\n", "⎢ ⎥\n", "⎣1 0 1 0 1 0 1⎦" ] }, "execution_count": 1, "metadata": {}, "output_type": "execute_result" } ], "source": [ "from sympy import *\n", "init_printing()\n", "\n", "H=Matrix([[0,0,0,1,1,1,1],[0,1,1,0,0,1,1],[1,0,1,0,1,0,1]])\n", "H" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAABoAAACvCAMAAADUkz46AAAANlBMVEX///8AAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABHL6OuAAAAEXRSTlMAMquZdlQQ\nQN0iRIlmze+7fEotVsoAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAFSSURBVFgJ7VjBloMgDIyAVAV1\n8/8/WwIVGZf17cF97bZwETMGQpwMCnUcm6LchmQh6lib0PqMkJd7xQJ1u3XveYCsdt5tzyE0h0Ht\nMCZPgPwiRjdXoDUG2bONGHhxhCZOsZaQZS2PT2y+eY3sxNalC5VeY/KqQScDUgqjr4RB6ypz+Vrw\nacm6tmQaJFHLI4llhMHswruop1emyg29slk6Hw99DEVvfEtvHin6VRZso2iZDSiUWCnWJOVBhMag\nLL/SQ/C7pvT6JYcEA1ql1UNqwqwAhXvzklCmKEYIFEXoinVptfCsfBg4tOOAyfqPoOdQtKlooQBN\nRc9VtCzzor6OZV5AV1Tle1F0K/PDtyjsRLjRQw4bRRtFQ3396e/SGUW3Hz+kaD80Fd1VDwp2N0vv\nedB7qej20X5QUUgvUhSgpqJRRX88PrJyWGTMtJM3Hh8ZQ3fxzDPpv2atnwAAAABJRU5ErkJggg==\n", "text/latex": [ "$$\\left[\\begin{matrix}0\\\\0\\\\1\\\\0\\\\0\\\\0\\\\0\\end{matrix}\\right]$$" ], "text/plain": [ "⎡0⎤\n", "⎢ ⎥\n", "⎢0⎥\n", "⎢ ⎥\n", "⎢1⎥\n", "⎢ ⎥\n", "⎢0⎥\n", "⎢ ⎥\n", "⎢0⎥\n", "⎢ ⎥\n", "⎢0⎥\n", "⎢ ⎥\n", "⎣0⎦" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "v_e=Matrix([0,0,1,0,0,0,0])\n", "v_e" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAGMAAAAUBAMAAABrMp7fAAAAMFBMVEX///8AAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAv3aB7AAAAD3RSTlMAdt3NMolEEJlmVCLv\nu6sHwGgPAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAA4ElEQVQoFWMQMmEgBXAaJTCEMTBURIYQr4sV\nrMWKoWgDhh6u4xhCDCAxsBbOAwysBujydX3f0YUYwGJgLbwPGLgw5RkxhRhAYmAt/A8Y2H5hGIlX\nS30DA9tH0rTsD2Bg+UqilgaStZDhMKD3uUj0Pu8FBk7MEMUbYsCo5MCISnAcoIcJPF4YtBhqHNg/\no8mDpOvRTEJoKRV8wsCgVICih+PN55cMTOqYYpBkCRbn2oAiDeGsxhRD0sKKKcvAEIApiKSlEFOW\ngV0AUxBJywRMWQZOLGJALUIqWMRxCzEqJQAAC1hJUUWlkpgAAAAASUVORK5CYII=\n", "text/latex": [ "$$\\left [ 0, \\quad 1, \\quad 1\\right ]$$" ], "text/plain": [ "[0, 1, 1]" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "H.dot(v_e)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## エラー検出の原理\n", "\n", "これはハミングが見つけた，エラーを見つけるチェック行列と呼ばれています．\n", "\n", "例えばAからBへコードワード$v_{code}$を送った時に，仮に一箇所エラー$v_{error}$が入ったとします．Bが受け取る信号$v_{signal}$は\n", "$\n", "v_{signal} = v_{code} + v_{error}\n", "$\n", "となるわけですが，これを$H$に通すと自動的にその場所を特定してくれるわけです．\n", "\n", "その条件は\n", "> もし，$H.v_{code}=0$ならば\n", "\n", "です．\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Hammingの生成行列(generator matrix)\n", "$H.v_{code}=0$となる$v_{code}$(コードワードと呼ばれます)を生成するHammingの生成行列(generator matrix)は\n", "$$\n", "G= \\left(\\begin{matrix}1 & 0 & 1 & 1\\\\1 & 1 & 0 & 1\\\\0 & 0 & 0 & 1\\\\1 & 1 & 1 & 0\\\\0 & 0 & 1 & 0\\\\0 & 1 & 0 & 0\\\\1 & 0 & 0 & 0\\end{matrix}\\right)\n", "$$\n", "\n", "です．例えば，送りたい信号\n", "$v_{orig}=[1,0,0,1]$だとすると，\n", "これを$G$に通して出てきたコードワード$v_{code}$を\n", "さらに$H$に通すと零ベクトルとなります．\n", "\n", "このようにある行列に作用させた時に零ベクトルとなる\n", "ベクトルの作る空間をヌルスペース(null space)と呼びます．" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAHgAAACvCAMAAAABgep3AAAANlBMVEX///8AAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABHL6OuAAAAEXRSTlMAMquZdlQQ\nQN0iRO/NZom7fEVHPZwAAAAJcEhZcwAADsQAAA7EAZUrDhsAAASYSURBVHgB7ZzbeuIwEIMNhJRD\ngc37v+za5AMsJZ7p1IeyW3NTEmX8O4oJsqG4zXR/bF2jx2HmObeZdoN/7Btx3SXQtlMAb1oxX5yL\nBTzurpdroU4yeH8eX73iZ5/+aoyHI+9+bYvVzsUygMftbjulwZdzQFw/XyB4plSTDGDfziCAb/dx\nvxeOkKqpcQt4uoNPU3r41wGP0y5Ye5qG8Gf1UQd8nK6Btpn/NAXfz7g9+MesdvPg2jcfXO52Cxf2\n0v7lNN9AdqkbiO9UnVHt3CHcMs/Czfrb4N32PH1uL6svFb9zvPq3szRXqUaZ71wpZvH9HVzc0lSD\n3eqUM8X3d6uLW5pq8L+w2pT3+YzjzL3ikiib8j6AKXMzWJFteR/AnmR4R+V+2fJ+QbAt75cDG0No\nObAx75cEm/J+OfA3rP6YPp4jNGNU2/L+H1qKyADb8n45q50t7xcE2/I+gjFzPy/844kim/I+gh+E\nBn87uIHJM6Jb3dTqcUivX1bryNEv073/enXJ03+TUa1E8jwZ/cIzViJ5niys0CuRPE+m6QCcsRLJ\n82TvdBxvAKxE8jxZACs5MU8OQyt1xkokz5NlsBjJj/NnEqkVekWWwHleKtVr4GegVyJ5nkzXGAK9\nEsnzZALDy0mJ5HmyBNYiubJCr8jpl5O6BK8kdkXG6QBYHUZeq0cHt3Laf27Ww14js4PVPdA3Mfs3\njGqcDvAZi0vwGMlXrohYjdMBAFPm5pbzZJoOANiT4gTK4DyZpgPtwDQdaAbmENoMzNOBhmCcLTQD\nr1n9DPR5w1appukABHqlNE+m6UAzq3kBvx2YZgsIxsztrcVHnox5H8HIqbrVwVXtjRvvVsduVH0e\nrI7v1VVhceN8r461qs9/++DCzL2wWpEXx+MOrEarMXNjnd9SZCXvYzWAKXMzWJGVvE/VAKbMzWBF\n9odL0wGqBjBlbgYrsgKm6hjMQZDAihyOFs6Yq2MwZ24CK7IC5moEY+ZegEVZBWN1AD8WX9gNAiuy\nAuZq+DSVMjeBM79DT43HVmtfkqdIzv3y28Lg4sYBPL/Gk1+SV2QFTNUApsy9OCXTErxSjWDM3ItS\nRVbyPlYjeIGqt6OD63lLLXeryZB6m8HqHujr+Ru1/CajGjN31L8vPVWqUcYzxsy9QjMswS+qsXEA\nU+bmUlti52pqHMCUubnUb4sRQ/6fWWocwJS5jWClmuQYzEHQBlaqWY7BnLltYKWaZQRj5raCxWr+\nRk4AfzHQh34Ig4u9pG6zfA/0j/9Xp8xNpQpYy/vU+P4QfVRfNLFzt6nx+BrzIjqX+m3Baq06J9DL\nYNt0AM44879eMbEv/EIZwYuD6+3o4HreUsvdajKk3ma3up631PK/YTVGcjoFHyjkH8FBmc+4XGJf\n9EsI9EUTO4OlQO+Pld7pKZJzyzaZrZbAFMkZbJMNYM6JBDbKBjBHcgIbZRPYlNgX/cJqA9joJYG5\n2gA2JnYCc7UFTJGcW7bJFjBFcgbbZAvYlti5X1SNYNMa+6JlTOyKjODFwfV2dHA9b6nl32r1D/2I\n4xh+VHEYTnQVqm3ef8RxGNxf8WhfhcXo6E4AAAAASUVORK5CYII=\n", "text/latex": [ "$$\\left[\\begin{matrix}1 & 0 & 1 & 1\\\\1 & 1 & 0 & 1\\\\0 & 0 & 0 & 1\\\\1 & 1 & 1 & 0\\\\0 & 0 & 1 & 0\\\\0 & 1 & 0 & 0\\\\1 & 0 & 0 & 0\\end{matrix}\\right]$$" ], "text/plain": [ "⎡1 0 1 1⎤\n", "⎢ ⎥\n", "⎢1 1 0 1⎥\n", "⎢ ⎥\n", "⎢0 0 0 1⎥\n", "⎢ ⎥\n", "⎢1 1 1 0⎥\n", "⎢ ⎥\n", "⎢0 0 1 0⎥\n", "⎢ ⎥\n", "⎢0 1 0 0⎥\n", "⎢ ⎥\n", "⎣1 0 0 0⎦" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "G=Matrix([[1,0,1,1],\n", "[1,1,0,1],\n", "[0,0,0,1],\n", "[1,1,1,0],\n", "[0,0,1,0],\n", "[0,1,0,0],\n", "[1,0,0,0]])\n", "G" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAABoAAABkCAMAAACvvNBNAAAAP1BMVEX///8AAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAADFBd4eAAAAFHRS\nTlMAMquZdlQQQO0wRO/NZondIrt8bFiOv0QAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAEISURBVEgN\n7VbbDoMgDEVuOhXErf//raNUsU1YsmwP2zJ5UOiBph5PS1UHZWhVx0IWpTowNg9XERVwrQGh7rAe\ns/A85Ca/nxOnvDYa2lDeb0+oMm/0BLMOxKLgcCf2hCQTuJJEeRND3KUsoTnr3i+pwWGY0BjnBrSW\nPHKbSoVDKNAIlE4c8mDQ1Qi2eORQgoi2jl4i+ESnWlDDYQ99ca4oDLeFceUJu664JbSCp082rU9W\nCxI1bSTy4LM55nLRppei2Z7y1LdC3ta6xENM+be9Xw+ZR8nG/0h0uBAJUqK3gdF7SpQnLNNMnkrZ\nCOyT0HnR8+LwSgPzkxf9w67SYw9p7XgItHSV1qo7NiogtJPWLEYAAAAASUVORK5CYII=\n", "text/latex": [ "$$\\left[\\begin{matrix}1\\\\0\\\\0\\\\1\\end{matrix}\\right]$$" ], "text/plain": [ "⎡1⎤\n", "⎢ ⎥\n", "⎢0⎥\n", "⎢ ⎥\n", "⎢0⎥\n", "⎢ ⎥\n", "⎣1⎦" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "v_1 = Matrix([1,0,0,1])\n", "v_1" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQQAAAAUBAMAAACZh14WAAAAMFBMVEX///8AAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAv3aB7AAAAD3RSTlMAdt3NMolEIpm7EKvv\nVGZvmWXoAAAACXBIWXMAAA7EAAAOxAGVKw4bAAAB8UlEQVRIDcWUPUsDQRRFbxI1YUwkpFGwMIhG\n0EILOwtT2hkRxU7/gWKjZUAEJa2N2Ky1hcFKRFTESgTTaSEkIHaC4BeixTqbNZnd914xIOg2O3sy\n976TSVhkRvGv130Os4hvbOeZxermFmO2QK0/sEKJ+X2zWmEPre+0XZUxv0Ah7hyGkLpkLFZRAxRK\nDPVCT+EIuKKJdgfJMwLV4alDELBX+mTsEZimUGJ+oadwDpzmSSSZRfsrYUCPwxAiXGEHqNKdEvML\nPYVxhyvEPn+h8AUsVoisxIyC3jxMpTWL8q/XI2zjp6A+tEJaNwQuiemPvULvFIC2t8DuxnKx0Fg1\n73YKcd02ScIS07VGIVpuTjGLIbNsrCwV9CkwBYHpVqOw0ZgRuCeKgYefpZ2CdOgS06VNhYgwDV3c\noJ6gmP8XoP961QrZJ7GAwi3iSySBVBErlNkqbANPDglLzCi0FBFhChNAJ2nxExQKp6BfQ2t0m8SM\nwvLM3AH2R0KheP9MKateQqyeYMxTIOFoRV3YMKMw7rrPiA6GxsVc182i1wnB0vVYjbLE8csJDaup\n7rwNQ73Qfy94g25C0/yHFPt9AImJYalQYkahICi0WjJIYVvWVFBpYdyuJZPCtuznBa0ndQjTUBOg\nxKSwLdMKmT5hzh+iTO4b4L+aM8d/WA4AAAAASUVORK5CYII=\n", "text/latex": [ "$$\\left [ 2, \\quad 2, \\quad 1, \\quad 1, \\quad 0, \\quad 0, \\quad 1\\right ]$$" ], "text/plain": [ "[2, 2, 1, 1, 0, 0, 1]" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "G.dot(v_1)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAABoAAACvCAMAAADUkz46AAAANlBMVEX///8AAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAABHL6OuAAAAEXRSTlMAMquZdlQQ\nQN0iRLuJ781mfKx/7gsAAAAJcEhZcwAADsQAAA7EAZUrDhsAAAGsSURBVFgJ7VjZloMgDI1sRQUd\n/v9nBxJBQumMD845dioP1XLNQha8AkPAIaAMSzMAQ5AqDl0QWNN/ERI07LP73cqgUcp5zs8xyCRz\nLkwkySBh0qT3HcjbNCk200xqCUmsC5Eiiw8AMCmEpuDokWfI5qg8QVKSzLNCVZDWlk7IOKIgVzig\njOpAo3XOSYELb2wtlKYelH2jK7fFsI+HPqZEH+FRMl+V6BdrWLhL9KedzSjcm0oct5tRHdwPmeA5\nrad9cYkpNEKKbceLVhkU/6tLQqxEdw+bEj3beSl8WMRKuWkDVWXsPaC7RHPK2nydUvN3ib4Kb56P\n1zbyDVTHsILaNq+gXxQe4qJGutV1uSgskeAaS+SLO78iC3UL+cLcmJEa6h6rDAhNgXhzLWUCEr0p\nxDdGHDU0EgsdNjLKIZSqodzmjUL2JiI3dMcNmGcyT++O2hbQkmVvyWBToPwWRCYFxsXvgn54can5\nh0vlWbxeCLpL9G9LNDcbL1EdP2VKL98lypl53SoX6pSSr+jff+Ki9bpOYQ6HFL4lF315fGTSYZFS\n2yFPKmA8PlIKvgHzKTKIsrgj4AAAAABJRU5ErkJggg==\n", "text/latex": [ "$$\\left[\\begin{matrix}2\\\\2\\\\1\\\\1\\\\0\\\\0\\\\1\\end{matrix}\\right]$$" ], "text/plain": [ "⎡2⎤\n", "⎢ ⎥\n", "⎢2⎥\n", "⎢ ⎥\n", "⎢1⎥\n", "⎢ ⎥\n", "⎢1⎥\n", "⎢ ⎥\n", "⎢0⎥\n", "⎢ ⎥\n", "⎢0⎥\n", "⎢ ⎥\n", "⎣1⎦" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "Matrix([2,2,1,1,0,0,1])" ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAGMAAAAUBAMAAABrMp7fAAAAMFBMVEX///8AAAAAAAAAAAAAAAAA\nAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAAv3aB7AAAAD3RSTlMAdt3NMolEIpm7EKvv\nVGZvmWXoAAAACXBIWXMAAA7EAAAOxAGVKw4bAAABE0lEQVQoFZXOLU/DUBTG8acdpaR7CZkB2SzD\ngBgh85tEzozg6DeAYJisHJmdwd3pCRYkhgWNB8ln4E3MXHpPt6Q95yC47v6e/G8uml385zROEpwB\no9upqMLxXZ8jWeCSaIGLhM9zBD+qUVI1qC35/Ai8qEZJLUb1i8/PwFOfIRkllZWS9IxMyCjJ3vJX\n7EF37RiJHbNJLgdy3f5WbZMcyRX+QmJm62QnlSvGuq2TfWX1lGec5Uk9xY2I3hBecXSWJ6fAHl+3\nUng8IaMkPBhO4uizHF0Pzx9wf1xCMkoq1toYLVOae9Z+wD+Uln+MvM6/4fS1lNClkARyBQYSC8lM\nroh2JRaSd7mioViWNNuK/01eK/kFiXRKu6We014AAAAASUVORK5CYII=\n", "text/latex": [ "$$\\left [ 2, \\quad 2, \\quad 2\\right ]$$" ], "text/plain": [ "[2, 2, 2]" ] }, "execution_count": 8, "metadata": {}, "output_type": "execute_result" } ], "source": [ "H.dot(Matrix([0,0,1,1,0,0,1]))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## GF(2)\n", "おっと，一つ計算上のルールを忘れていました．\n", "ここの計算では$GF(2)$上の算術とします．\n", "$GF(2)$での算術をまとめると\n", "\n", "|$\\times$|0|1|\n", "|:----:|:---:|:---:|\n", "|0| 0| 0|\n", "|1| 0| 1|\n", "\n", "|$+$|0|1|\n", "|:----:|:---:|:---:|\n", "|0| 0| 1|\n", "|1| 1| 0|\n", "\n", "となります．要素は0と1だけの体です．\n", "GF(2)は2元ガロア体(Galois Field 2)の略で，\n", "20歳で早世した抽象代数の基礎を築いたGaloisにちなんでいます．\n", "\n", "なので，上の行列計算で1以上が出た時には，2で割ってそのあまりを考えることになります．" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Deep Learning\n", "深層学習という手法によって\n", "「AIが目を手に入れた」\n", "と評されています．\n", "\n", "画像認識のトライアルとしてよく例示される手書き数字認識で，deep learningで何をしているかを実感してもらおうと思います．ここでの記述と関数は「ゼロから作るDeep Learning――Pythonで学ぶディープラーニングの理論と実装」斎藤 康毅　著，(Oreilly Japan 2016)によります．\n", "\n", "## 手書き数字認識(MNIST)\n", "\n", "[mnist](http://yann.lecun.com/exdb/mnist/)によると，nistが提供していたhand writingのDBとしてspecial database 3とspecial database 1というのがあって，SD-3はCensus Bureauの雇用者からSD-1はJunior high schoolの学生からあつめたもので，そこには認識率におおきな差があったのだって．それらを混ぜ合わせた，training setとtest setとして新たに提供されたのが，mixing NIST's datasetsからこの呼び名がついたって．" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true }, "source": [ "## load_mnist" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "(60000, 784)\n" ] } ], "source": [ "import sys, os\n", "sys.path.append(os.pardir)\n", "from dataset.mnist import load_mnist\n", "\n", "(x_train, t_train), (x_test, t_test) = \\\n", " load_mnist(flatten=True, normalize=False)\n", "\n", "print(x_train.shape)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "本に書いてあるcodeではjupyterで表示しなかったので，\n", "http://biopython.seesaa.net/article/451770389.html\n", "に従って修正．" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "5\n" ] }, { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAP8AAAD8CAYAAAC4nHJkAAAABHNCSVQICAgIfAhkiAAAAAlwSFlz\nAAALEgAACxIB0t1+/AAADa9JREFUeJzt3WGMVPW5x/Hf4wqGQF+gxA0RVqrITaqJEFdjDDY0Xhur\njcAbgkbDTRu2L2gjel8U7YtiqqbW297cVzVgsdQU2yZqIPXGUkitXKPEVVBZtchtFrsEWQmNtcTQ\nwj59MYdmi3v+Z5g5M+csz/eTbHbmPHPmPEz47Tkz/zPnb+4uAPGcV3UDAKpB+IGgCD8QFOEHgiL8\nQFCEHwiK8ANBEX4gKMIPBHV+NzdmZpxOCHSYu1szj2trz29mt5jZH8zsgJmta+e5AHSXtXpuv5n1\nSNov6WZJI5Jek3SHu7+TWIc9P9Bh3djzXyfpgLv/0d3/JukXkpa28XwAuqid8F8i6U/j7o9ky/6F\nmQ2Y2aCZDbaxLQAl6/gHfu6+QdIGicN+oE7a2fMfkjR33P052TIAk0A74X9N0hVm9nkzmypppaRt\n5bQFoNNaPux395Nm9k1Jv5HUI2mTuw+V1hmAjmp5qK+ljfGeH+i4rpzkA2DyIvxAUIQfCIrwA0ER\nfiAowg8ERfiBoAg/EBThB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANB\nEX4gqK5O0Q2MN3PmzGS9r6+vY9s+ePBgsn7vvfcm6/v27UvW9+/fn6y/+eabyXo3sOcHgiL8QFCE\nHwiK8ANBEX4gKMIPBEX4gaDaGuc3s2FJn0g6Jemku/eX0RQmj9tuuy1Zv/3223NrS5YsSa47f/78\nVlpqStE4/KWXXpqsX3DBBW1tv6enp631y1DGST5fcvejJTwPgC7isB8Iqt3wu6QdZva6mQ2U0RCA\n7mj3sH+xux8ys4sl/dbM3nP3l8Y/IPujwB8GoGba2vO7+6Hs96ik5yRdN8FjNrh7Px8GAvXScvjN\nbLqZfe70bUlflpT+qhOA2mjnsL9X0nNmdvp5trj7C6V0BaDjzN27tzGz7m0MkqTLL788WV+zZk2y\nvnr16mR92rRpyXq2c8AZOjnO7+5NvegM9QFBEX4gKMIPBEX4gaAIPxAU4QeC4tLd57g5c+Yk6/fc\nc0+XOum+9957L7c2NDTUxU7qiT0/EBThB4Ii/EBQhB8IivADQRF+ICjCDwTFOH8XzJo1K1kvGmt/\n+eWXk/UXXsi/jMKJEyeS63788cfJ+vHjx5P16dOnJ+vbt2/PrRVNc7179+5kfc+ePcn6p59+mlsr\n+ndFwJ4fCIrwA0ERfiAowg8ERfiBoAg/EBThB4Li0t0lKBrr3rVrV7J+9dVXJ+vLly9P1rdt25as\np8ybNy9ZHx4eTtb7+vqS9ZGRkdza2NhYcl20hkt3A0gi/EBQhB8IivADQRF+ICjCDwRF+IGgCr/P\nb2abJH1V0qi7X5Utu1DSLyXNkzQsaYW7/7lzbVZv6tSpubUtW7Yk1y0ax3/kkUeS9R07diTr7Sga\nxy/ywQcflNMIuq6ZPf9PJd1yxrJ1kna6+xWSdmb3AUwiheF395ckHTtj8VJJm7PbmyUtK7kvAB3W\n6nv+Xnc/nN3+UFJvSf0A6JK2r+Hn7p46Z9/MBiQNtLsdAOVqdc9/xMxmS1L2ezTvge6+wd373b2/\nxW0B6IBWw79N0qrs9ipJW8tpB0C3FIbfzJ6W9IqkfzOzETP7uqTvS7rZzN6X9O/ZfQCTCN/nz8yY\nMSNZv//++3Nr69alRzqPHj2arC9YsCBZL7q2PjAe3+cHkET4gaAIPxAU4QeCIvxAUIQfCIopujPL\nlqW/m5Qaziv6WuuNN96YrDOUhyqw5weCIvxAUIQfCIrwA0ERfiAowg8ERfiBoBjnz9xwww0tr7tn\nz55kPTVNNVAV9vxAUIQfCIrwA0ERfiAowg8ERfiBoAg/EBSX7s6MjuZOOiRJuuiii3JrJ06cSK77\n6KOPJutbt6bnPNm7d2+yDozHpbsBJBF+ICjCDwRF+IGgCD8QFOEHgiL8QFCF4/xmtknSVyWNuvtV\n2bL1klZL+ih72APu/r+FG6vxOH/R6zA2NtaxbRc99+OPP56sv/rqq7m1vr6+5LoHDhxI1oeGhpL1\nIldeeWVu7ZVXXkmuy3UQWlPmOP9PJd0ywfL/dveF2U9h8AHUS2H43f0lSce60AuALmrnPf+3zOwt\nM9tkZjNL6whAV7Qa/h9LukzSQkmHJf0w74FmNmBmg2Y22OK2AHRAS+F39yPufsrdxyRtlHRd4rEb\n3L3f3ftbbRJA+VoKv5nNHnd3uaR95bQDoFsKL91tZk9LWiJplpmNSPqupCVmtlCSSxqW9I0O9gig\nA/g+f+axxx5L1u+7774udRLHRx99lKy/+OKLyfrKlStL7Obcwff5ASQRfiAowg8ERfiBoAg/EBTh\nB4JiqC/T09OTrC9atCi3tmXLluS655+fPp1i7ty5yfp558X8G130f3P9+vXJ+kMPPVRiN5MHQ30A\nkgg/EBThB4Ii/EBQhB8IivADQRF+IKjC7/NHcerUqWR9cDD/KmQLFixoa9s33XRTsj5lypRkPTXe\nfe2117bSUi2YpYerr7nmmi51cm5izw8ERfiBoAg/EBThB4Ii/EBQhB8IivADQTHOXwM7d+5sa/2F\nCxfm1orG+U+ePJmsP/nkk8n6xo0bk/W1a9fm1u68887kuugs9vxAUIQfCIrwA0ERfiAowg8ERfiB\noAg/EFThOL+ZzZX0M0m9klzSBnf/HzO7UNIvJc2TNCxphbv/uXOtIs/27dtzaw8//HBy3aI5BVav\nXp2sz58/P1lfsmRJst6OkZGRjj13BM3s+U9K+k93/4Kk6yWtMbMvSFonaae7XyFpZ3YfwCRRGH53\nP+zub2S3P5H0rqRLJC2VtDl72GZJyzrVJIDyndV7fjObJ2mRpN2Set39cFb6UI23BQAmiabP7Tez\nGZKekbTW3f8y/vpq7u558/CZ2YCkgXYbBVCupvb8ZjZFjeD/3N2fzRYfMbPZWX22pNGJ1nX3De7e\n7+79ZTQMoByF4bfGLv4nkt519x+NK22TtCq7vUrS1vLbA9AphVN0m9liSbskvS1pLFv8gBrv+38l\nqU/SQTWG+o4VPFdtp+iezKZNm5Zb27RpU3LdFStWlN1O04oul/78888n63fddVeyfvz48bPu6VzQ\n7BTdhe/53f3/JOU9WfqC8wBqizP8gKAIPxAU4QeCIvxAUIQfCIrwA0EVjvOXujHG+buutzf9lYsn\nnngiWe/vT5+YefHFFyfrw8PDubWnnnoquW5q6nHka3acnz0/EBThB4Ii/EBQhB8IivADQRF+ICjC\nDwTFOD+S7r777mT9+uuvT9YffPDB3Nro6IQXf0KbGOcHkET4gaAIPxAU4QeCIvxAUIQfCIrwA0Ex\nzg+cYxjnB5BE+IGgCD8QFOEHgiL8QFCEHwiK8ANBFYbfzOaa2e/M7B0zGzKze7Ll683skJntzX5u\n7Xy7AMpSeJKPmc2WNNvd3zCzz0l6XdIySSsk/dXd/6vpjXGSD9BxzZ7kc34TT3RY0uHs9idm9q6k\nS9prD0DVzuo9v5nNk7RI0u5s0bfM7C0z22RmM3PWGTCzQTMbbKtTAKVq+tx+M5sh6feSHnb3Z82s\nV9JRSS7pe2q8NfhawXNw2A90WLOH/U2F38ymSPq1pN+4+48mqM+T9Gt3v6rgeQg/0GGlfbHHzEzS\nTyS9Oz742QeBpy2XtO9smwRQnWY+7V8saZektyWNZYsfkHSHpIVqHPYPS/pG9uFg6rnY8wMdVuph\nf1kIP9B5fJ8fQBLhB4Ii/EBQhB8IivADQRF+ICjCDwRF+IGgCD8QFOEHgiL8QFCEHwiK8ANBEX4g\nqMILeJbsqKSD4+7PypbVUV17q2tfEr21qszeLm32gV39Pv9nNm426O79lTWQUNfe6tqXRG+tqqo3\nDvuBoAg/EFTV4d9Q8fZT6tpbXfuS6K1VlfRW6Xt+ANWpes8PoCKVhN/MbjGzP5jZATNbV0UPecxs\n2MzezmYernSKsWwatFEz2zdu2YVm9lszez/7PeE0aRX1VouZmxMzS1f62tVtxuuuH/abWY+k/ZJu\nljQi6TVJd7j7O11tJIeZDUvqd/fKx4TN7IuS/irpZ6dnQzKzH0g65u7fz/5wznT3b9ekt/U6y5mb\nO9Rb3szS/6EKX7syZ7wuQxV7/uskHXD3P7r73yT9QtLSCvqoPXd/SdKxMxYvlbQ5u71Zjf88XZfT\nWy24+2F3fyO7/Ymk0zNLV/raJfqqRBXhv0TSn8bdH1G9pvx2STvM7HUzG6i6mQn0jpsZ6UNJvVU2\nM4HCmZu76YyZpWvz2rUy43XZ+MDvsxa7+0JJX5G0Jju8rSVvvGer03DNjyVdpsY0bocl/bDKZrKZ\npZ+RtNbd/zK+VuVrN0FflbxuVYT/kKS54+7PyZbVgrsfyn6PSnpOjbcpdXLk9CSp2e/Rivv5J3c/\n4u6n3H1M0kZV+NplM0s/I+nn7v5strjy126ivqp63aoI/2uSrjCzz5vZVEkrJW2roI/PMLPp2Qcx\nMrPpkr6s+s0+vE3Squz2KklbK+zlX9Rl5ua8maVV8WtXuxmv3b3rP5JuVeMT//+X9J0qesjp6zJJ\nb2Y/Q1X3JulpNQ4D/67GZyNfl3SRpJ2S3pe0Q9KFNertKTVmc35LjaDNrqi3xWoc0r8laW/2c2vV\nr12ir0peN87wA4LiAz8gKMIPBEX4gaAIPxAU4QeCIvxAUIQfCIrwA0H9A5fkhZcUZTHYAAAAAElF\nTkSuQmCC\n", "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# coding: utf-8\n", "%matplotlib inline\n", "\n", "import sys, os\n", "sys.path.append(os.pardir) \n", "import numpy as np\n", "from dataset.mnist import load_mnist\n", "from PIL import Image\n", "import matplotlib.pyplot as plt\n", "\n", "def img_show(img):\n", " pil_img = Image.fromarray(np.uint8(img))\n", " plt.imshow(np.array(pil_img))\n", "\n", "(x_train, t_train), (x_test, t_test) = load_mnist(flatten=True, normalize=False)\n", "\n", "for i in [8]:\n", " img = x_test[i]\n", " label = t_test[i]\n", " print(label)\n", " img = img.reshape(28, 28)\n", " plt.imshow(img, cmap='gray')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## neuralnet_mnist\n", "\n", "sample_weight.pklをch03からcpしておく必要あり．\n", "\n" ] }, { "cell_type": "code", "execution_count": 12, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Accuracy:0.9352\n" ] } ], "source": [ "# coding: utf-8\n", "import sys, os\n", "sys.path.append(os.pardir) # 親ディレクトリのファイルをインポートするための設定\n", "import numpy as np\n", "import pickle\n", "from dataset.mnist import load_mnist\n", "from common.functions import sigmoid, softmax\n", "\n", "\n", "def get_data():\n", " (x_train, t_train), (x_test, t_test) = load_mnist(normalize=True, flatten=True, one_hot_label=False)\n", " return x_test, t_test\n", "\n", "\n", "def init_network():\n", " with open(\"sample_weight.pkl\", 'rb') as f:\n", " network = pickle.load(f)\n", " return network\n", "\n", "\n", "def predict(network, x):\n", " W1, W2, W3 = network['W1'], network['W2'], network['W3']\n", " b1, b2, b3 = network['b1'], network['b2'], network['b3']\n", "\n", " a1 = np.dot(x, W1) + b1\n", " z1 = sigmoid(a1)\n", " a2 = np.dot(z1, W2) + b2\n", " z2 = sigmoid(a2)\n", " a3 = np.dot(z2, W3) + b3\n", " y = softmax(a3)\n", "\n", " return y\n", "\n", "\n", "x, t = get_data()\n", "network = init_network()\n", "accuracy_cnt = 0\n", "for i in range(len(x)):\n", " y = predict(network, x[i])\n", " p= np.argmax(y) # 最も確率の高い要素のインデックスを取得\n", " if p == t[i]:\n", " accuracy_cnt += 1\n", "\n", "print(\"Accuracy:\" + str(float(accuracy_cnt) / len(x)))\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "何をするかをみるため，最後尾の駆動部を変更している．\n", "最初の10データに対して，それぞれ数字の確率と，実際の数字を表示している．\n", "\n", "例えば，一つ目のデータでは，7の確率が0.9971で，実際の字も7である．\n", "\n", "predictがやっていることを見ればいいが，単に行列の積を計算しているだけ．それで予測ができるとさ．だから，中間層を表す行列の要素さえただしく決定できればそれでニューラルネットは終わり！！" ] }, { "cell_type": "code", "execution_count": 13, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "No:0, Ans:7\n", "Guess ['0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '1.00', '0.00', '0.00']\n", "No:1, Ans:2\n", "Guess ['0.00', '0.00', '0.94', '0.01', '0.00', '0.01', '0.03', '0.00', '0.00', '0.00']\n", "No:2, Ans:1\n", "Guess ['0.00', '0.99', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00']\n", "No:3, Ans:0\n", "Guess ['0.99', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00']\n", "No:4, Ans:4\n", "Guess ['0.00', '0.00', '0.00', '0.00', '0.95', '0.00', '0.00', '0.01', '0.00', '0.03']\n", "No:5, Ans:1\n", "Guess ['0.00', '0.99', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00', '0.00']\n", "No:6, Ans:4\n", "Guess ['0.00', '0.00', '0.00', '0.00', '0.97', '0.01', '0.00', '0.00', '0.01', '0.01']\n", "No:7, Ans:9\n", "Guess ['0.00', '0.00', '0.00', '0.01', '0.02', '0.00', '0.00', '0.01', '0.01', '0.95']\n", "No:8, Ans:5\n", "Guess ['0.00', '0.00', '0.01', '0.00', '0.03', '0.01', '0.95', '0.00', '0.00', '0.00']\n", "No:9, Ans:9\n", "Guess ['0.00', '0.00', '0.00', '0.00', '0.04', '0.00', '0.00', '0.04', '0.00', '0.92']\n" ] } ], "source": [ "x, t = get_data()\n", "network = init_network()\n", "accuracy_cnt = 0\n", "for i in range(10):\n", " y = predict(network, x[i])\n", " print(\"No:{}, Ans:{}\".format(i,t[i]))\n", " print(\"Guess\",['{:.2f}'.format(n) for n in y])\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.1" }, "latex_envs": { "LaTeX_envs_menu_present": true, "autocomplete": true, "bibliofile": "biblio.bib", "cite_by": "apalike", "current_citInitial": 1, "eqLabelWithNumbers": true, "eqNumInitial": 1, "hotkeys": { "equation": "Ctrl-E", "itemize": "Ctrl-I" }, "labels_anchors": false, "latex_user_defs": false, "report_style_numbering": false, "user_envs_cfg": false }, "toc": { "colors": { "hover_highlight": "#DAA520", "navigate_num": "#000000", "navigate_text": "#333333", "running_highlight": "#FF0000", "selected_highlight": "#FFD700", "sidebar_border": "#EEEEEE", "wrapper_background": "#FFFFFF" }, "moveMenuLeft": true, "nav_menu": { "height": "12px", "width": "252px" }, "navigate_menu": true, "number_sections": true, "sideBar": true, "threshold": 4, "toc_cell": true, "toc_section_display": "block", "toc_window_display": false, "widenNotebook": false } }, "nbformat": 4, "nbformat_minor": 2 }