{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 主成分分析 (2)"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"tags": [
"hide-cell"
]
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Requirement already satisfied: japanize-matplotlib in /usr/local/lib/python3.8/dist-packages (1.1.3)\n",
"Requirement already satisfied: matplotlib in /usr/local/lib/python3.8/dist-packages (from japanize-matplotlib) (3.4.2)\n",
"Requirement already satisfied: pyparsing>=2.2.1 in /usr/local/lib/python3.8/dist-packages (from matplotlib->japanize-matplotlib) (2.4.7)\n",
"Requirement already satisfied: cycler>=0.10 in /usr/local/lib/python3.8/dist-packages (from matplotlib->japanize-matplotlib) (0.10.0)\n",
"Requirement already satisfied: kiwisolver>=1.0.1 in /usr/local/lib/python3.8/dist-packages (from matplotlib->japanize-matplotlib) (1.3.1)\n",
"Requirement already satisfied: numpy>=1.16 in /home/okazaki/.local/lib/python3.8/site-packages (from matplotlib->japanize-matplotlib) (1.19.5)\n",
"Requirement already satisfied: pillow>=6.2.0 in /usr/local/lib/python3.8/dist-packages (from matplotlib->japanize-matplotlib) (8.3.1)\n",
"Requirement already satisfied: python-dateutil>=2.7 in /usr/local/lib/python3.8/dist-packages (from matplotlib->japanize-matplotlib) (2.8.1)\n",
"Requirement already satisfied: six in /home/okazaki/.local/lib/python3.8/site-packages (from cycler>=0.10->matplotlib->japanize-matplotlib) (1.15.0)\n"
]
}
],
"source": [
"!pip install japanize-matplotlib"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"import matplotlib.pyplot as plt\n",
"import japanize_matplotlib"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"$\\def\\bm{\\boldsymbol}$2次元のデータの例で主成分分析の基本的な考え方を掴んだところで、多次元のデータに対する主成分分析を考えよう。\n",
"\n",
"## データの表現\n",
"\n",
"$d$次元のベクトル$\\bm{x} \\in \\mathbb{R}^d$からなる事例を$N$件集めたデータを行列$\\bm{X} \\in \\mathbb{R}^{N \\times d}$で表す。\n",
"\n",
"\\begin{align}\n",
"\\bm{X} = \\begin{pmatrix}\\bm{x}_1^\\top \\\\ \\bm{x}_2^\\top \\\\ \\dots \\\\ \\bm{x}_N^\\top \\end{pmatrix}\n",
"\\end{align}\n",
"\n",
"また、データの平均ベクトル$\\bm{\\bar{x}} \\in \\mathbb{R}^d$を次式で定義する。\n",
"\n",
"\\begin{align}\n",
"\\bm{\\bar{x}} = \\frac{1}{N} \\sum_{i=1}^N \\bm{x}_i\n",
"\\end{align}\n",
"\n",
"### 射影したデータの分散\n",
"\n",
"あるベクトル$\\bm{u} \\in \\mathbb{R}^d$でデータ点を射影したとき、$\\bm{u}$上に射影された点の分散$J$は、\n",
"\n",
":::{margin} データの中心化\n",
"\n",
"主成分分析の説明の中には、データ点が中心化されている（データ点の平均が原点にある）ことを仮定するものがある。前章の例で用いたデータ点も中心化されていた。しかし、式{eq}`eq:pca-ptojected-variance-derivation`の定式化では、射影後の分散を求めるときに分散共分散行列を経由する（各データ点から平均を引く）ことになるため、事前にデータ点を中心化しておかなくてもよい。\n",
"\n",
":::\n",
"\n",
"$$\n",
"\\begin{align}\n",
"J &= \\frac{1}{N} \\sum_{i=1}^N \\left(\\bm{u}^\\top \\bm{x}_i - \\bm{u}^\\top \\bm{\\bar{x}}\\right)^2 \\\\\n",
"&= \\frac{1}{N} \\sum_{i=1}^N \\left\\{\\bm{u}^\\top (\\bm{x}_i - \\bm{\\bar{x}})\\right\\}^2 \\\\\n",
"&= \\frac{1}{N} \\sum_{i=1}^N \\left\\{\\bm{u}^\\top (\\bm{x}_i - \\bm{\\bar{x}})\\right\\} \\cdot \\left\\{\\bm{u}^\\top (\\bm{x}_i - \\bm{\\bar{x}})\\right\\} \\\\\n",
"&= \\frac{1}{N} \\sum_{i=1}^N \\left\\{\\bm{u}^\\top (\\bm{x}_i - \\bm{\\bar{x}})\\right\\} \\cdot \\left\\{(\\bm{x}_i - \\bm{\\bar{x}})^\\top \\bm{u}\\right\\} \\\\\n",
"&= \\bm{u}^\\top \\left[\\frac{1}{N} \\sum_{i=1}^N (\\bm{x}_i - \\bm{\\bar{x}})(\\bm{x}_i - \\bm{\\bar{x}})^\\top\\right] \\bm{u} \\\\\n",
"&= \\bm{u}^\\top \\bm{S} \\bm{u}\n",
"\\end{align}\n",
"$$ (eq:pca-ptojected-variance-derivation)\n",
"\n",
":::{margin} 二次形式\n",
"\n",
"式{eq}`eq:pca-projected-variance`は二次形式と呼ばれる。二次形式とは、二次の項のみで構成される多項式である。\n",
"\n",
"例えば、\n",
"\n",
"\\begin{align*}\n",
"\\bm{S} = \\begin{pmatrix}\n",
"1 & 1 \\\\\n",
"1 & 3\n",
"\\end{pmatrix}\n",
"\\end{align*}\n",
"\n",
"を考えると、\n",
"\n",
"\\begin{align*}\n",
"\\bm{u}^\\top\\bm{S}\\bm{u}\n",
"&=\n",
"\\begin{pmatrix}\n",
"u_1 & u_2\n",
"\\end{pmatrix}\n",
"\\begin{pmatrix}\n",
"1 & 1 \\\\\n",
"1 & 3\n",
"\\end{pmatrix}\n",
"\\begin{pmatrix}\n",
"u_1 \\\\\n",
"u_2\n",
"\\end{pmatrix} \\\\\n",
"&= \\begin{pmatrix}\n",
"u_1 + u_2 & u_1 + 3u_2\n",
"\\end{pmatrix}\n",
"\\begin{pmatrix}\n",
"u_1 \\\\\n",
"u_2\n",
"\\end{pmatrix} \\\\\n",
"&= u_1^2 + 2 u_1 u_2 + 3u_2^2\n",
"\\end{align*}\n",
"\n",
"となる。これは、$u_1$と$u_2$の二次の項のみを含む多項式であることが確認できる。\n",
"\n",
":::\n",
"\n",
":::{admonition} 射影後の点の分散\n",
":class: important\n",
"\n",
"$N$個のデータ点$\\bm{x}_1, \\dots, \\bm{x}_N \\in \\mathbb{R}^d$をベクトル$\\bm{u} \\in \\mathbb{R}^d$で射影したとき、$\\bm{u}$上に射影された点の分散は、\n",
"\n",
"$$\n",
"\\begin{align}\n",
"\\bm{u}^\\top \\bm{S} \\bm{u}\n",
"\\end{align}\n",
"$$ (eq:pca-projected-variance)\n",
"\n",
"ここで、$\\bm{S} \\in \\mathbb{R}^{d \\times d}$はデータの分散共分散行列である。\n",
"\n",
"$$\n",
"\\begin{align}\n",
"\\bm{S} &= \\frac{1}{N} \\sum_{i=1}^N (\\bm{x}_i - \\bm{\\bar{x}}) (\\bm{x}_i - \\bm{\\bar{x}})^\\top \\\\\n",
"\\end{align}\n",
"$$ (eq:variance-covariance-matrix)\n",
"\n",
":::\n",
"\n",
"なお、分散共分散行列$\\bm{S}$の各要素$S_{a,b}$は、行列$\\bm{X}$を用いると、\n",
"\n",
"\\begin{gather}\n",
"S_{a,b} = \\frac{1}{N} \\sum_{i=1}^N (X_{i,a} - \\bar{x}_a) (X_{i,b} - \\bar{x}_b) \\\\\n",
"\\bar{x}_a = \\frac{1}{N} \\sum_{i=1}^N X_{i,a}, \\; \\bar{x}_b = \\frac{1}{N} \\sum_{i=1}^N X_{i,b}\n",
"\\end{gather}\n",
"\n",
"と表される。$a$と$b$を入れ替えても同じ値が求まるので、分散共分散行列$\\bm{S}$は$d \\times d$の実対称行列である。\n",
"\n",
"(sec:eigen-and-pca)=\n",
"## 固有値問題と主成分分析\n",
"\n",
"### 第1主成分\n",
"\n",
"主成分分析では、ある単位ベクトル$\\bm{u} \\in \\mathbb{R}^d$でデータ点を射影したとき、射影された点の分散を最大化したい。前章で説明した通り、第1主成分$\\bm{u}_1$は、$\\|\\bm{u}_1\\|=1$の制約のもとで次式の目的関数を最大化する問題の解である。\n",
"\\begin{align}\n",
"J = \\bm{u}_1^\\top \\bm{S} \\bm{u}_1\n",
"\\end{align}\n",
"\n",
"これを制約なしの最大化問題に書き換えるため、ラグランジュの未定乗数法を用いる。ラグランジュ関数$\\mathcal{L}$は、\n",
"\\begin{align}\n",
"\\mathcal{L} = \\bm{u}_1^\\top \\bm{S} \\bm{u}_1 - \\lambda_1 (\\bm{u}_1^\\top\\bm{u}_1 - 1)\n",
"\\end{align}\n",
"ここで、$\\lambda_1$はラグランジュ乗数である。ラグランジュ関数$\\mathcal{L}$を最大化するため、$\\bm{u}_1$に関する偏微分を$0$とおくと、\n",
"\\begin{align}\n",
"\\frac{\\partial \\mathcal{L}}{\\partial \\bm{u}_1} = 2\\bm{S} \\bm{u}_1 - 2\\lambda_1 \\bm{u}_1 &= 0 \\\\\n",
"\\bm{S} \\bm{u}_1 = \\lambda_1 \\bm{u}_1\n",
"\\end{align}\n",
"\n",
"ゆえに、第1主成分を求める問題は分散共分散行列$\\bm{S}$の固有値問題に帰着する。なお、上式の両辺に左から$\\bm{u}_1^\\top$をかけると、左辺は最大化したい目的関数$J$となり、右辺は$\\lambda_1$となる。\n",
"\n",
"$$\n",
"\\begin{align}\n",
"J = \\bm{u}_1^\\top \\bm{S} \\bm{u}_1 = \\bm{u}_1^\\top (\\bm{S} \\bm{u}_1) = \\bm{u}_1^\\top (\\lambda_1 \\bm{u}_1) = \\lambda_1 \\bm{u}_1^\\top \\bm{u}_1 = \\lambda_1\n",
"\\end{align}\n",
"$$ (eq:j-principal-component)\n",
"\n",
"したがって、射影後の分散$J$を最大にする第1主成分ベクトル$\\bm{u}_1$は、$\\bm{S}$の固有値の中で最大のものを$\\lambda_1$として、それに対応する固有ベクトルを長さ$1$になるように正規化したものである。また、データ$\\bm{X}$を第1主成分ベクトル$\\bm{u}_1$に射影したとき、その軸における分散は$\\lambda_1$である。\n",
"\n",
"これまでの議論をまとめる。\n",
"\n",
":::{admonition} 主成分分析の第1主成分\n",
":class: important\n",
"\n",
"$N$個の事例からなるデータ$\\bm{X} = \\begin{pmatrix}\\bm{x}_1 & \\bm{x}_2 & \\dots & \\bm{x}_N \\end{pmatrix}^\\top$が与えられ、その分散共分散行列を$\\bm{S}$とする。主成分分析の第1主成分ベクトル$\\bm{u}_1$は以下の固有値問題の解である。\n",
"\n",
"$$\n",
"\\begin{align}\n",
"\\bm{S} \\bm{u}_1 = \\lambda_1 \\bm{u}_1\n",
"\\end{align}\n",
"$$ (eq:pca1-eigenvalue)\n",
"\n",
"ここで、$\\lambda_1$は$\\bm{S}$の最大の固有値で、$\\bm{u}_1$は$\\lambda_1$に対応する固有ベクトルを正規化したものである。データの平均ベクトルを$\\bar{\\bm{x}}$と書くことにすると、データ$\\bm{X}$に含まれるある事例$\\bm{x}$の第1主成分得点は、\n",
"\n",
"$$\n",
"\\begin{align}\n",
"\\bm{u}_1^\\top (\\bm{x} - \\bm{\\bar{x}})\n",
"\\end{align}\n",
"$$ (eq:pca1-score)\n",
"\n",
"データ点を第1主成分ベクトル$\\bm{u}_1$に射影したとき、その軸におけるデータの分散は$\\lambda_1$である。\n",
"\n",
":::"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### 第2主成分以降\n",
"\n",
"第2主成分以降を一般的に求めるため、第$k$主成分ベクトル$\\bm{u}_1, \\dots, \\bm{u}_k$までを求めてあると仮定し、第$k+1$主成分ベクトルを求める（$1 \\leq k < d$）。各主成分ベクトルの長さは$1$で、互いに直交するように選ぶので、主成分ベクトルは以下の制約を満たす（つまり、$\\bm{u}_1, \\dots, \\bm{u}_k$は正規直交系である）。\n",
"\n",
"$$\n",
"\\begin{align}\n",
"\\bm{u}^\\top_i \\bm{u}_j = \\begin{cases}\n",
"1 & (i = j) \\\\\n",
"0 & (i \\neq j)\n",
"\\end{cases}\n",
"\\end{align}\n",
"$$ (eq:orthonormal)\n",
"\n",
"第$k+1$主成分ベクトル$\\bm{u}_{k+1}$は、$\\|\\bm{u}_{k+1}\\|=1, \\bm{u}_1^\\top \\bm{u}_{k+1} = \\dots = \\bm{u}_k^\\top \\bm{u}_{k+1} = 0$の制約のもとで次の目的関数を最大化する問題の解である。\n",
"\n",
"\\begin{align}\n",
"J = \\bm{u}_{k+1}^\\top \\bm{S} \\bm{u}_{k+1}\n",
"\\end{align}\n",
"\n",
"これを制約なしの最大化問題に書き換えるため、ラグランジュの未定乗数法を用いる。ラグランジュ関数$\\mathcal{L}$は、\n",
"\n",
"\\begin{align}\n",
"\\mathcal{L} = \\bm{u}_{k+1}^\\top \\bm{S} \\bm{u}_{k+1} - \\lambda_{k+1} (\\bm{u}_{k+1}^\\top\\bm{u}_{k+1} - 1) - \\sum_{i=1}^k \\alpha_i \\bm{u}_i^\\top \\bm{u}_{k+1}\n",
"\\end{align}\n",
"\n",
"ただし、$\\alpha_1, \\dots, \\alpha_k, \\lambda_{k+1}$はラグランジュ乗数である。ラグランジュ関数$\\mathcal{L}$を最大化するため、$\\bm{u}_{k+1}$に関する偏微分を$0$とおくと、\n",
"\n",
"$$\n",
"\\begin{align}\n",
"\\frac{\\partial \\mathcal{L}}{\\partial \\bm{u}_{k+1}} = 2\\bm{S} \\bm{u}_{k+1} - 2\\lambda_{k+1} \\bm{u}_{k+1} - \\sum_{i=1}^k \\alpha_i \\bm{u}_i &= 0\n",
"\\end{align}\n",
"$$ (eq:pca-lagrange)\n",
"\n",
"ここで、上の等式において$\\bm{u}_1^\\top$を左からかけ、式{eq}`eq:orthonormal`に注意しながら展開すると、\n",
"\n",
"\\begin{align}\n",
"2\\bm{u}_1^\\top \\bm{S} \\bm{u}_{k+1} - 2\\lambda_{k+1} \\bm{u}_1^\\top \\bm{u}_{k+1} - \\sum_{i=1}^k \\alpha_i \\bm{u}_1^\\top \\bm{u}_i &= 0 \\\\\n",
"2\\bm{u}_1^\\top \\bm{S} \\bm{u}_{k+1} - \\alpha_1 &= 0 \\\\\n",
"\\alpha_1 &= 2\\bm{u}_1^\\top \\bm{S} \\bm{u}_{k+1} = 2\\bm{u}_{k+1}^\\top \\bm{S} \\bm{u}_1 = 2\\bm{u}_{k+1}^\\top \\lambda_1 \\bm{u}_1 = 0\n",
"\\end{align}\n",
"\n",
"同様に、全ての$j \\in \\{1, \\dots, k\\}$に対して、$\\bm{u}_j^\\top$を左からかけることで、$\\alpha_j = 0$が示される。ゆえに、式{eq}`eq:pca-lagrange`は、\n",
"\n",
"\\begin{align}\n",
"\\bm{S} \\bm{u}_{k+1} = \\lambda_{k+1} \\bm{u}_{k+1}\n",
"\\end{align}\n",
"\n",
"と整理される。したがって、第$k+1$主成分以降を求める問題も分散共分散行列$\\bm{S}$の固有値問題に帰着する。これを$k=1$から$k=d-1$まで繰り返すことで、すべての主成分ベクトルを求めることができる。\n",
"\n",
"固有値問題による主成分分析をまとめる。\n",
"\n",
":::{admonition} 主成分分析と固有値問題\n",
":class: important\n",
"\n",
"$N$個の事例からなるデータ$\\bm{X} = \\begin{pmatrix}\\bm{x}_1 & \\bm{x}_2 & \\dots & \\bm{x}_N \\end{pmatrix}^\\top$が与えられ、その分散共分散行列を$\\bm{S}$とする。$\\bm{S}$の$d$個の固有値を大きい順に並べ、\n",
"\n",
"$$\n",
"\\begin{align}\n",
"\\lambda_1 \\geq \\lambda_2 \\geq \\dots \\geq \\lambda_d \\geq 0\n",
"\\end{align}\n",
"$$ (eq:pca-lambdas)\n",
"\n",
"とする。また、$\\lambda_1, \\lambda_2, \\dots, \\lambda_d$に対応する固有ベクトルを$\\bm{u}_1, \\bm{u}_1, \\dots, \\bm{u}_d$とする。ただし、固有ベクトルは、\n",
"\n",
"\\begin{align}\n",
"\\bm{u}^\\top_i \\bm{u}_j = \\begin{cases}\n",
"1 & (i = j) \\\\\n",
"0 & (i \\neq j)\n",
"\\end{cases}\n",
"\\end{align}\n",
"\n",
"を満たすように求める。すると、$\\bm{u}_1, \\bm{u}_1, \\dots, \\bm{u}_d$はそれぞれ、第1主成分、第2主成分、……、第$d$主成分に対応する。データの平均ベクトルを$\\bar{\\bm{x}}$と書くことにすると、データ$\\bm{X}$に含まれるある事例$\\bm{x}$の第$i$主成分得点は（$i \\in \\{1, \\dots, d\\}$）、\n",
"\n",
"$$\n",
"\\begin{align}\n",
"\\bm{u}_i^\\top (\\bm{x} - \\bm{\\bar{x}})\n",
"\\end{align}\n",
"$$ (eq:pcak-score)\n",
"\n",
"データ$\\bm{X}$を各主成分に射影したとき、各軸におけるデータの分散はそれぞれ、$\\lambda_1, \\lambda_2, \\dots, \\lambda_d$である。\n",
"\n",
":::\n",
"\n",
"なお、式{eq}`eq:pca-lambdas`において、$\\bm{S}$の固有値が実数で、かつ全て非負であることは後ほど説明する。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## スペクトル分解と主成分分析\n",
"\n",
"主成分分析への理解を深めるため、スペクトル分解との関連を説明する。\n",
"\n",
"### スペクトル分解\n",
"\n",
"まず、スペクトル分解を復習する。一般に、実対象行列は以下の性質を持つ。\n",
"\n",
":::{admonition} 実対称行列のすべての固有値は実数である\n",
":class: note, dropdown\n",
"\n",
"実対称行列$\\bm{S} \\in \\mathbb{R}^{d \\times d}$のある固有値を$\\lambda \\in \\mathbb{C}$とする（この時点では複素数の可能性を排除していない）。また、この固有値に対応する固有ベクトルを$\\bm{q} \\in \\mathbb{C}^d$と書く。\n",
"\n",
"\\begin{align*}\n",
"\\bm{S} \\bm{q} = \\lambda \\bm{q}\n",
"\\end{align*}\n",
"\n",
"両辺の複素共役をとると（$\\bm{S}$は実行列なので$\\bm{\\bar{S}} = \\bm{S}$）、\n",
"\n",
"\\begin{align*}\n",
"\\bm{S} \\bm{\\bar{q}} = \\bar{\\lambda} \\bm{\\bar{q}}\n",
"\\end{align*}\n",
"\n",
"上の2つの等式、および$\\bm{S}^\\top = \\bm{S}$に注意しながら$\\bm{q}^\\top \\bm{S} \\bm{\\bar{q}}$を二通りの方法で変形する。\n",
"\n",
"\\begin{align*}\n",
"\\bm{q}^\\top \\bm{S} \\bm{\\bar{q}} &= \\bm{q}^\\top \\bm{S}^\\top \\bm{\\bar{q}} = (\\bm{S} \\bm{q})^\\top \\bm{\\bar{q}} = (\\lambda \\bm{q})^\\top \\bm{\\bar{q}} = \\lambda (\\bm{q}^\\top \\bm{\\bar{q}}) \\\\\n",
"\\bm{q}^\\top \\bm{S} \\bm{\\bar{q}} &= \\bm{q}^\\top (\\bm{S} \\bm{\\bar{q}}) = \\bm{q}^\\top (\\bar{\\lambda} \\bm{\\bar{q}}) = \\bar{\\lambda} (\\bm{q}^\\top \\bm{\\bar{q}})\n",
"\\end{align*}\n",
"\n",
"上の二つの等式の右辺は等しいので、次の等式が成り立つ。\n",
"\n",
"\\begin{align*}\n",
"\\lambda (\\bm{q}^\\top \\bm{\\bar{q}}) - \\bar{\\lambda} (\\bm{q}^\\top \\bm{\\bar{q}}) = (\\lambda - \\bar{\\lambda}) \\bm{q}^\\top \\bm{\\bar{q}} = 0 \\qquad \\dots \\mbox{ (a)}\n",
"\\end{align*}\n",
"\n",
"一般に、複素数$(a + ib)$とその複素共役$(a - ib)$の積は$(a + ib)(a - ib) = a^2 + b^2 = |a + ib|^2$であるので、\n",
"\n",
"\\begin{align*}\n",
"\\bm{q}^\\top \\bm{\\bar{q}} = \\sum_{i=1}^d q_i \\bar{q_i} = \\sum_{i=1}^d |q_i|^2 > 0\n",
"\\end{align*}\n",
"\n",
"したがって、式(a)が成り立つことから$\\lambda = \\bar{\\lambda}$。すなわち、$\\lambda$は実数である。\n",
"\n",
":::\n",
"\n",
":::{admonition} 実対称行列の固有ベクトルは正規直交系を構成する\n",
":class: note, dropdown\n",
"\n",
"実対称行列$\\bm{S} \\in \\mathbb{R}^{d \\times d}$の異なる二つの固有値を$\\lambda, \\mu$、これらに対応する固有ベクトル$\\bm{u}, \\bm{v}$とする。\n",
"\n",
"\\begin{align*}\n",
"\\bm{S} \\bm{u} = \\lambda \\bm{u}, \\; \\bm{S} \\bm{v} = \\mu \\bm{v}\n",
"\\end{align*}\n",
"\n",
"$\\bm{u}$と$\\bm{v}$を定数倍しても上の等式は成り立つので、$\\|\\bm{u}\\|=\\|\\bm{v}\\|=1$となるように正規化しておく。上の2つの等式、および$\\bm{S}^\\top = \\bm{S}$に注意しながら$\\bm{u}^\\top \\bm{S} \\bm{v}$を二通りの方法で変形する。\n",
"\n",
"\\begin{align*}\n",
"\\bm{u}^\\top \\bm{S} \\bm{v} &= \\bm{u}^\\top \\bm{S}^\\top \\bm{v} = (\\bm{S} \\bm{u})^\\top \\bm{v} = (\\lambda \\bm{u})^\\top \\bm{v} = \\lambda(\\bm{u}^\\top \\bm{v}) \\\\\n",
"\\bm{u}^\\top \\bm{S} \\bm{v} &= \\bm{u}^\\top (\\bm{S} \\bm{v}) = \\bm{u}^\\top \\mu \\bm{v} = \\mu (\\bm{u}^\\top \\bm{v})\n",
"\\end{align*}\n",
"\n",
"上の二つの等式の右辺は等しいので、次の等式が成り立つ。\n",
"\n",
"\\begin{align*}\n",
"\\lambda(\\bm{u}^\\top \\bm{v}) - \\mu (\\bm{u}^\\top \\bm{v}) = (\\lambda - \\mu) (\\bm{u}^\\top \\bm{v}) = 0\n",
"\\end{align*}\n",
"\n",
"もし、二つの固有値$\\lambda$と$\\mu$が異なるとき、$\\lambda - \\mu \\neq 0$であるから、$\\bm{u}^\\top \\bm{v} = 0$である。\n",
"\n",
"もし、二つの固有値$\\lambda$と$\\mu$が等しいとき、すなわち実対称行列が重複する固有値を持つ場合は、対応する固有ベクトルをグラム・シュミットの直交化法により直交化すればよい。\n",
"\n",
":::\n",
"\n",
"ある実対称行列$\\bm{S} \\in \\mathbb{R}^{d \\times d}$の固有値を$\\lambda_1, \\lambda_2, \\dots, \\lambda_d$とし、対応する固有ベクトルを$\\bm{q}_1, \\bm{q}_2, \\dots, \\bm{q}_d \\in \\mathbb{R}^d$とする。\n",
"\n",
"$$\n",
"\\begin{align}\n",
"\\bm{S} \\bm{q}_1 = \\lambda_1 \\bm{q}_1, \\;\n",
"\\bm{S} \\bm{q}_2 = \\lambda_2 \\bm{q}_2, \\;\n",
"\\dots, \\;\n",
"\\bm{S} \\bm{q}_d = \\lambda_d \\bm{q}_d\n",
"\\end{align}\n",
"$$ (eq:eigen-equations)\n",
"\n",
"なお、固有ベクトルを正規直交系を構成するように選ぶことができるので、$i, j \\in \\{1, \\dots, d\\}$に関して、\n",
"\n",
"$$\n",
"\\begin{align}\n",
" \\bm{q}_i^\\top\\bm{q}_j = \\begin{cases}\n",
"1 & (i = j) \\\\\n",
"0 & (i \\neq j)\n",
"\\end{cases}\n",
"\\end{align}\n",
"$$ (eq:q-is-orthogonal)\n",
"\n",
"等式{eq}`eq:eigen-equations`をベクトルとして横に並べると、次のように整理できる。\n",
"\n",
"$$\n",
"\\begin{align}\n",
"\\begin{pmatrix}\n",
"\\bm{S} \\bm{q}_1 &\n",
"\\bm{S} \\bm{q}_2 &\n",
"\\dots &\n",
"\\bm{S} \\bm{q}_d\n",
"\\end{pmatrix} &= \n",
"\\begin{pmatrix}\n",
"\\lambda_1 \\bm{q}_1 &\n",
"\\lambda_2 \\bm{q}_2 &\n",
"\\dots &\n",
"\\lambda_d \\bm{q}_d\n",
"\\end{pmatrix} \\\\\n",
"\\bm{S} \\begin{pmatrix}\n",
"\\bm{q}_1 & \\bm{q}_2 & \\dots & \\bm{q}_d\n",
"\\end{pmatrix} &= \\begin{pmatrix}\n",
"\\bm{q}_1 & \\bm{q}_2 & \\dots & \\bm{q}_d\n",
"\\end{pmatrix}\n",
"\\begin{pmatrix}\n",
"\\lambda_1 & 0 & \\dots & 0 \\\\\n",
"0 & \\lambda_2 & \\dots & 0 \\\\\n",
"0 & 0 & \\dots & 0 \\\\\n",
"0 & 0 & \\dots & \\lambda_d \\\\\n",
"\\end{pmatrix} \\\\\n",
"\\bm{S} \\bm{Q} &= \\bm{Q} \\bm{\\Lambda}\n",
"\\end{align}\n",
"$$ (eq:eigen-equation)\n",
"\n",
"ここで、$\\bm{\\Lambda} \\in \\mathbb{R}^{d \\times d}$は固有値を要素とする対角行列である。\n",
"\n",
"\\begin{align}\n",
"\\bm{\\Lambda} = {\\rm diag}(\\lambda_1, \\lambda_2, \\dots, \\lambda_d) = \\begin{pmatrix}\n",
"\\lambda_1 & 0 & \\ldots & 0 \\\\\n",
"0 & \\lambda_2 & \\ldots & 0 \\\\\n",
"\\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
"0 & 0 & \\ldots & \\lambda_d \\\\\n",
"\\end{pmatrix}\n",
"\\end{align}\n",
"\n",
"また、行列$\\bm{Q} \\in \\mathbb{R}^{d \\times d}$は列ベクトルである固有ベクトルを横方向に配置したものである。\n",
"\n",
"\\begin{align}\n",
"\\bm{Q} = \\begin{pmatrix}\n",
"\\bm{q}_1 & \\bm{q}_2 & \\dots & \\bm{q}_d\n",
"\\end{pmatrix}\n",
"\\end{align}\n",
"\n",
":::{margin} 直交行列\n",
"\n",
"ある行列$\\bm{A}$が直交行列であるとは、\n",
"\n",
"\\begin{align*}\n",
"\\bm{A}^\\top \\bm{A} = \\bm{A}\\bm{A}^\\top = \\bm{I}\n",
"\\end{align*}\n",
"\n",
":::\n",
"\n",
"以下に示すように、$\\bm{Q}^\\top \\bm{Q} = \\bm{I}$であるから、$\\bm{Q}$は直交行列である。\n",
"\n",
"$$\n",
"\\begin{align}\n",
"\\bm{Q}^\\top \\bm{Q} = \\begin{pmatrix}\n",
"\\bm{q}_1^\\top \\\\ \\bm{q}_2^\\top \\\\ \\dots \\\\ \\bm{q}_d^\\top\n",
"\\end{pmatrix} \\begin{pmatrix}\n",
"\\bm{q}_1 & \\bm{q}_2 & \\dots & \\bm{q}_d\n",
"\\end{pmatrix}\n",
"= \\begin{pmatrix}\n",
"\\bm{q}_1^\\top\\bm{q}_1 & \\bm{q}_1^\\top\\bm{q}_2 & \\ldots & \\bm{q}_1^\\top\\bm{q}_d \\\\\n",
"\\bm{q}_2^\\top\\bm{q}_1 & \\bm{q}_2^\\top\\bm{q}_2 & \\ldots & \\bm{q}_2^\\top\\bm{q}_d \\\\\n",
"\\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
"\\bm{q}_d^\\top\\bm{q}_1 & \\bm{q}_d^\\top\\bm{q}_2 & \\ldots & \\bm{q}_d^\\top\\bm{q}_d \\\\\n",
"\\end{pmatrix}\n",
"= \\begin{pmatrix}\n",
"1 & 0 & \\ldots & 0 \\\\\n",
"0 & 1 & \\ldots & 0 \\\\\n",
"\\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
"0 & 0 & \\ldots & 1 \\\\\n",
"\\end{pmatrix} = \\bm{I}\n",
"\\end{align}\n",
"$$ (eq:q-is-orthogonal-matrix)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
":::{margin} 実対称行列の対角化\n",
"\n",
"式{eq}`eq:eigen-equation`の左から$\\bm{Q}^\\top$をかけると、$\\bm{S}$は$\\bm{Q}$によって対角化可能であることを示すことができる。\n",
"\\begin{align*}\n",
"\\bm{Q}^\\top \\bm{S} \\bm{Q} = \\bm{\\Lambda}\n",
"\\end{align*}\n",
"\n",
":::\n",
"\n",
"式{eq}`eq:eigen-equation`の右から$\\bm{Q}^\\top$をかけ、左辺において$\\bm{Q}\\bm{Q}^\\top = \\bm{I}$であることに注意すると、**スペクトル分解**（spectral decomposition）が得られる。\n",
"\n",
":::{admonition} スペクトル分解\n",
":class: important\n",
"\n",
"実対称行列$\\bm{S} \\in \\mathbb{R}^{d \\times d}$は、直交行列$\\bm{Q} \\in \\mathbb{R}^{d \\times d}$、対角行列$\\bm{\\Lambda} \\in \\mathbb{R}^{d \\times d}$、直交行列$\\bm{Q}^\\top$の積に分解できる。\n",
"\n",
"\\begin{align}\n",
"\\bm{S} &= \\bm{Q} \\bm{\\Lambda} \\bm{Q}^\\top \\\\\n",
"&= \\begin{pmatrix}\n",
"\\vert & \\vert & \\ldots & \\vert \\\\\n",
"\\bm{q}_1 & \\bm{q}_2 & \\ldots & \\bm{q}_d \\\\\n",
"\\vert & \\vert & \\ldots & \\vert \\\\\n",
"\\end{pmatrix}\n",
"\\begin{pmatrix}\n",
"\\lambda_1 & 0 & \\ldots & 0 \\\\\n",
"0 & \\lambda_2 & \\ldots & 0 \\\\\n",
"\\vdots & \\vdots & \\ddots & \\vdots \\\\\n",
"0 & 0 & \\ldots & \\lambda_d \\\\\n",
"\\end{pmatrix}\n",
"\\begin{pmatrix}\n",
"\\text{―} & \\bm{q}_1^\\top & \\text{―} \\\\\n",
"\\text{―} & \\bm{q}_2^\\top & \\text{―} \\\\\n",
"\\vdots & \\vdots & \\vdots \\\\\n",
"\\text{―} & \\bm{q}_d^\\top & \\text{―}\n",
"\\end{pmatrix} \\\\\n",
"&= \\lambda_1 \\begin{pmatrix}\n",
"\\vert \\\\\n",
"\\bm{q}_1 \\\\\n",
"\\vert \\\\\n",
"\\end{pmatrix}\n",
"\\begin{pmatrix}\n",
"\\text{―} \\; \\bm{q}_1^\\top \\; \\text{―}\n",
"\\end{pmatrix} +\n",
"\\lambda_2 \\begin{pmatrix}\n",
"\\vert \\\\\n",
"\\bm{q}_2 \\\\\n",
"\\vert \\\\\n",
"\\end{pmatrix}\n",
"\\begin{pmatrix}\n",
"\\text{―} \\; \\bm{q}_2^\\top \\; \\text{―}\n",
"\\end{pmatrix} +\n",
"\\dots +\n",
"\\lambda_d \\begin{pmatrix}\n",
"\\vert \\\\\n",
"\\bm{q}_d \\\\\n",
"\\vert \\\\\n",
"\\end{pmatrix}\n",
"\\begin{pmatrix}\n",
"\\text{―} \\; \\bm{q}_d^\\top \\; \\text{―}\n",
"\\end{pmatrix} \\\\\n",
"&= \\sum_{i = 1}^{d} \\lambda_i \\bm{q}_i \\bm{q}_i^\\top\n",
"\\end{align}\n",
"\n",
"なお、\n",
"\n",
"+ $\\bm{\\Lambda}$の対角要素$\\lambda_1, \\lambda_2, \\dots, \\lambda_d$は$\\bm{S}$の固有値を並べたものである。\n",
"+ $\\bm{Q}$の列ベクトル$\\bm{q}_1, \\dots, \\bm{q}_d \\in \\mathbb{R}^d$は$\\lambda_1, \\lambda_2, \\dots, \\lambda_d$に対応する固有ベクトルであり、正規直交系をなす。\n",
"\n",
":::\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"(sec:pca-by-spectral-decomposition)=\n",
"### 主成分分析の導出\n",
"\n",
"スペクトル分解に基づいて主成分分析を導出してみよう。主成分分析では、データ行列$\\bm{X} \\in \\mathbb{R}^{N \\times d}$に対して式{eq}`eq:variance-covariance-matrix`で計算される分散共分散行列$\\bm{S} \\in \\mathbb{R}^{d \\times d}$が出発点である。以下の二つの性質により、分散共分散行列$\\bm{S}$の固有値は非負である。\n",
"\n",
":::{admonition} 分散共分散行列は半正定値対称行列である\n",
":class: note, dropdown\n",
"\n",
"行列$\\bm{X} \\in \\mathbb{R}^{N \\times d}$の分散共分散行列を$\\bm{S} \\in \\mathbb{R}^{d \\times d}$とする。分散共分散行列は対称行列であるので、半正定値行列であることを示せばよい。任意のベクトル$\\bm{u}$に対して、\n",
"\n",
"\\begin{align}\n",
"\\bm{u}^\\top \\bm{S} \\bm{u}\n",
"&= \\bm{u}^\\top \\left[\\frac{1}{N} \\sum_{i=1}^N (\\bm{x}_i - \\bm{\\bar{x})}(\\bm{x}_i - \\bm{\\bar{x}})^\\top\\right] \\bm{u} \\\\\n",
"&= \\frac{1}{N} \\sum_{i=1}^N \\left\\{\\bm{u}^\\top (\\bm{x}_i - \\bm{\\bar{x})}\\right\\} \\cdot \\left\\{(\\bm{x}_i - \\bm{\\bar{x}})^\\top \\bm{u}\\right\\} \\\\\n",
"&= \\frac{1}{N} \\sum_{i=1}^N \\left\\{\\bm{u}^\\top (\\bm{x}_i - \\bm{\\bar{x})}\\right\\} \\cdot \\left\\{\\bm{u}^\\top (\\bm{x}_i - \\bm{\\bar{x}})\\right\\}^\\top \\\\\n",
"&= \\frac{1}{N} \\sum_{i=1}^N \\left\\{\\bm{u}^\\top (\\bm{x}_i - \\bm{\\bar{x})}\\right\\} \\cdot \\left\\{\\bm{u}^\\top (\\bm{x}_i - \\bm{\\bar{x}})\\right\\} \\\\\n",
"&= \\frac{1}{N} \\sum_{i=1}^N \\left\\{\\bm{u}^\\top (\\bm{x}_i - \\bm{\\bar{x})}\\right\\}^2 \\\\\n",
"&\\geq 0\n",
"\\end{align}\n",
"\n",
"が成り立つので、分散共分散行列$\\bm{S}$は半正定値行列である。なお、この証明は式{eq}`eq:pca-objective`を逆向きに辿っていると考えればよい。\n",
"\n",
":::\n",
"\n",
":::{admonition} 半正定値対称行列の固有値はすべて非負である\n",
":class: note, dropdown\n",
"\n",
"半正定値対称行列を$\\bm{S}$とする。その行列のある固有値を$\\lambda$、対応する固有ベクトルを$\\bm{u}$とする。すなわち、\n",
"\n",
"\\begin{align}\n",
"\\bm{S} \\bm{u} = \\lambda \\bm{u}\n",
"\\end{align}\n",
"\n",
"ここで、$\\bm{S}$の二次形式を計算すると、\n",
"\n",
"\\begin{align}\n",
"\\bm{u}^\\top \\bm{S} \\bm{u} = \\bm{u}^\\top \\lambda \\bm{u} = \\lambda \\bm{u}^\\top \\bm{u}\n",
"\\end{align}\n",
"\n",
"分散共分散行列は半正定値対称行列であるから、左辺は非負であることが分かっている。右辺が非負となるためには、$\\lambda \\geq 0$が成り立たなければならない。\n",
"\n",
":::\n",
"\n",
"分散共分散行列$\\bm{S}$は実対称行列であるから、直交行列$\\bm{Q} \\in \\mathbb{R}^{d \\times d}$と対角行列$\\bm{\\Lambda} = {\\rm diag}(\\lambda_1, \\dots, \\lambda_d) \\in \\mathbb{R}^{d \\times d}$によるスペクトル分解が存在する。\n",
"\n",
"\\begin{align}\n",
"\\bm{S} = \\bm{Q} \\bm{\\Lambda} \\bm{Q}^\\top\n",
"\\end{align}\n",
"\n",
"さらに、分散共分散行列$\\bm{S}$は半正定値対称行列であり、その固有値はすべて非負である。そこで、$\\bm{S}$の固有値を大きい順に並べておく。\n",
"\n",
"$$\n",
"\\begin{align}\n",
"\\lambda_1 \\geq \\lambda_2 \\geq \\dots \\geq \\lambda_d \\geq 0\n",
"\\end{align}\n",
"$$ (eq:eigenvalue-order)\n",
"\n",
"また、これらの固有値に対応するように固有ベクトル$\\bm{Q} = \\begin{pmatrix}\\bm{q}_1 & \\bm{q}_2 & \\dots & \\bm{q}_d\\end{pmatrix}$を並び替えておく。\n",
"\n",
"さて、式{eq}`eq:pca-objective`で示したように、主成分分析における第1主成分$\\bm{u}_1 \\in \\mathbb{R}^d$は以下の最大化問題で求まる。\n",
"\n",
"\\begin{align}\n",
"\\bm{u}_1 = \\mathop{\\rm argmax}\\limits_{\\|\\bm{u}\\| = 1} \\bm{u}^\\top \\bm{S} \\bm{u} = \\mathop{\\rm argmax}\\limits_{\\|\\bm{u}\\| \\neq 0} \\frac{\\bm{u}^\\top \\bm{S} \\bm{u}}{\\bm{u}^\\top \\bm{u}}\n",
"\\end{align}\n",
"\n",
"ここで、${\\rm argmax}$の中身、\n",
"\n",
"\\begin{align}\n",
"\\frac{\\bm{u}^\\top \\bm{S} \\bm{u}}{\\bm{u}^\\top \\bm{u}}\n",
"\\end{align}\n",
"\n",
"はレイリー商（Rayleigh quotient）と呼ばる。以下で示すように、このレイリー商は$\\bm{u} = \\pm \\bm{q}_1$のとき最大値$\\lambda_1$をとる。すなわち、データ$\\bm{X}$を第1主成分ベクトルに射影するとき、その分散の最大値は$\\lambda_1$であり、その最大値を与える第1主成分ベクトルは$\\bm{u}_1 = \\pm \\bm{q}_1$である。\n",
"\n",
":::{margin} レイリー商の最小値\n",
"\n",
"同様に、レイリー商の最小値は$\\lambda_d$であることを示すことができる。\n",
":::\n",
"\n",
":::{admonition} レイリー商による最大値\n",
":class: note\n",
"\n",
"\\begin{align}\n",
"\\frac{\\bm{u}^\\top \\bm{S} \\bm{u}}{\\bm{u}^\\top \\bm{u}}\n",
"&= \\frac{\\bm{u}^\\top (\\bm{Q}\\bm{\\Lambda}\\bm{Q}^\\top) \\bm{u}}{\\bm{u}^\\top \\bm{u}} && (\\because \\bm{S} = \\bm{Q}\\bm{\\Lambda}\\bm{Q}^\\top) \\\\\n",
"&= \\frac{\\bm{u}^\\top \\bm{Q}\\bm{\\Lambda}\\bm{Q}^\\top \\bm{u}}{\\bm{u}^\\top\\bm{Q} \\bm{Q}^\\top\\bm{u}} && (\\because \\bm{Q} \\bm{Q}^\\top = \\bm{I}) \\\\\n",
"&= \\frac{(\\bm{Q}^\\top \\bm{u})^\\top\\bm{\\Lambda}\\bm{Q}^\\top \\bm{u}}{(\\bm{Q}^\\top \\bm{u})^\\top \\bm{Q}^\\top\\bm{u}} && (\\because \\bm{Q} \\bm{Q}^\\top = \\bm{I}) \\\\\n",
"&= \\frac{\\bm{z}^\\top \\bm{\\Lambda}\\bm{z}}{\\bm{z}^\\top\\bm{z}} && (\\bm{z} := \\bm{Q}^\\top \\bm{u}) \\\\\n",
"&= \\frac{\\lambda_1 z_1^2 + \\lambda_2 z_2^2 + \\dots + \\lambda_d z_d^2}{z_1^2 + z_2^2 + \\dots + z_d^2} && (\\because \\bm{\\Lambda} = {\\rm diag}(\\lambda_1, \\lambda_2, \\dots, \\lambda_d))\n",
"\\end{align}\n",
"\n",
"上式は$\\lambda_1, \\lambda_2, \\dots, \\lambda_d$の$z_1^2, z_2^2, \\dots, z_d^2$による重み付き平均である。したがって、その最大値は、\n",
"\\begin{align}\n",
"\\max_{\\|\\bm{u}\\| \\neq 0} \\frac{\\bm{u}^\\top \\bm{S} \\bm{u}}{\\bm{u}^\\top \\bm{u}}\n",
"= \\max_{\\|\\bm{z}\\| \\neq 0} \\frac{\\lambda_1 z_1^2 + \\lambda_2 z_2^2 + \\dots + \\lambda_d z_d^2}{z_1^2 + z_2^2 + \\dots + z_d^2}\n",
"\\leq \\lambda_1 \\\\\n",
"\\end{align}\n",
"等号が成立するのは、$z_1^2 = 1, z_2^2 = z_3^2 = \\dots = z_d^2 = 0$、すなわち$\\bm{u} = \\pm \\bm{q}_1$のときである。\n",
"\n",
":::"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"続いて、主成分分析で第$k$主成分までデータ$\\bm{X}$を射影する場合を考える（$1 \\leq k \\leq d$）。$k$個の直交基底ベクトルで射影した後のデータの分散の和の最大値は$\\lambda_1 + \\dots + \\lambda_k$であること、その最大値は$\\bm{q}_1, \\dots, \\bm{q}_k$で射影した場合であることを示す {cite}`Dasgupta:08`。\n",
"\n",
"$k$個の直交基底ベクトルを$\\bm{u}_1, \\dots, \\bm{u}_k$とおく。式{eq}`eq:pca-projected-variance`より、これらのベクトルでデータを射影した後の分散の和$J_k$は、分散共分散行列$\\bm{S}$のスペクトル分解$\\bm{Q}\\bm{\\Lambda}\\bm{Q}^\\top$を用いて、次のように展開できる。\n",
"\n",
"$$\n",
"\\begin{align}\n",
"J_k &= \\sum_{i=1}^k \\bm{u}_i^\\top \\bm{S} \\bm{u}_i \\\\\n",
"&= \\sum_{i=1}^k \\bm{u}_i^\\top (\\bm{Q} \\bm{\\Lambda} \\bm{Q}^\\top) \\bm{u}_i && (\\because \\bm{S} = \\bm{Q}\\bm{\\Lambda}\\bm{Q}^\\top) \\\\\n",
"&= \\sum_{i=1}^k \\bm{u}_i^\\top \\left( \\sum_{j = 1}^{d} \\lambda_j \\bm{q}_j \\bm{q}_j^\\top \\right) \\bm{u}_i && (\\mbox{スペクトル分解の別の形}) \\\\\n",
"&= \\sum_{j = 1}^{d} \\lambda_j \\sum_{i=1}^k (\\bm{u}_i^\\top\\bm{q}_j)^2 && (\\because \\bm{u}_i^\\top\\bm{q}_j = \\bm{q}_j^\\top\\bm{u}_i) \\\\\n",
"&= \\sum_{j = 1}^{d} \\lambda_j z_j && (z_j := \\sum_{i=1}^k (\\bm{u}_i^\\top\\bm{q}_j)^2) \\\\\n",
"\\end{align}\n",
"$$ (eq:pca-objective)\n",
"\n",
"ゆえに、分散の和$J_k$は$d$個の固有値$\\lambda_1, \\dots, \\lambda_d$の$z_1, \\dots, z_d$による重み付き和として表される。そこで、$j \\in \\{1, 2, \\dots, d\\}$に関して$z_j$の性質を調べる。まず、ベッセルの不等式より、\n",
"\\begin{align}\n",
"z_j = \\sum_{i=1}^k (\\bm{u}_i^\\top\\bm{q}_j)^2 \\leq \\|\\bm{q}_j\\|^2 = 1\n",
"\\end{align}\n",
"\n",
"さらに、$z_j$の和は、\n",
"\\begin{align}\n",
"\\sum_{j = 1}^{d} z_j = \\sum_{j = 1}^{d} \\sum_{i=1}^k (\\bm{u}_i^\\top\\bm{q}_j)^2 = \\sum_{i=1}^k \\sum_{j = 1}^{d} \\bm{u}_i^\\top\\bm{q}_j \\bm{q}_j^\\top\\bm{u}_i = \\sum_{i=1}^k \\bm{u}_i^\\top\\bm{Q} \\bm{Q}^\\top\\bm{u}_i = \\sum_{i=1}^k \\bm{u}_i^\\top\\bm{u}_i = k\n",
"\\end{align}\n",
"\n",
"したがって、式{eq}`eq:pca-objective`は$0 \\leq z_j \\leq 1$かつ、$\\sum_{j = 1}^{d} z_j = k$による固有値$\\lambda_1, \\dots, \\lambda_d$の重み付き和である。式{eq}`eq:eigenvalue-order`のように、分散共分散行列$\\bm{S}$の固有値を大きい順に並べてある場合、式{eq}`eq:pca-objective`が最大となるのは、$z_1 = \\dots = z_k = 1, z_{k+1} = \\dots = z_d = 0$と選んだときである。$1 \\leq j \\leq k$に対して、$z_j = 1$を満たすには、$\\bm{u}_j = \\bm{q}_j$とすればよい。\n",
"\n",
"まとめると、$k$個の直交基底ベクトルでデータ$\\bm{X}$を射影した後の分散の和$J_k$の最大値は、\n",
"\n",
"$$\n",
"\\begin{align}\n",
"\\max J_k = \\sum_{i=1}^k \\lambda_i\n",
"\\end{align}\n",
"$$ (eq:pca-maximum-variance)\n",
"\n",
"この最大値は、データを$\\bm{q}_1, \\dots, \\bm{q}_k$で射影した時に得られる。\n",
"\n",
"{ref}`sec:eigen-and-pca`では、データの分散共分散行列$\\bm{S}$の固有値を大きい順に並べ、対応する固有ベクトルを主成分ベクトルとして一つずつ選んでいた。式{eq}`eq:pca-maximum-variance`は、$k$個の直交基底ベクトルでデータを射影するとき、$\\bm{q}_1, \\bm{q}_2, \\dots, \\bm{q}_k$を射影ベクトルに選ぶことで、射影後のデータの分散の和が最大となること、その分散の和の最大値が$\\lambda_1 + \\lambda_2 + \\dots + \\lambda_k$であることを保証している。"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 主成分分析と次元圧縮"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
":::{margin} 行列のトレース\n",
"行列$\\bm{A}$の対角成分の和をトレースと呼び、$\\mathrm{tr}(\\bm{A})$で表す。\n",
":::\n",
"\n",
"データ$\\bm{X}$の共分散行列$\\bm{S}$の対角成分の和は、データ$\\bm{X}$の分散の和である。$\\bm{S} = \\bm{Q} \\bm{\\Lambda} \\bm{Q}^\\top$のスペクトル分解を用いると、$\\bm{S}$の対角成分の和は、\n",
"\n",
"\\begin{align}\n",
"\\mathrm{tr}(\\bm{S}) &= \\mathrm{tr}(\\bm{Q} \\bm{\\Lambda} \\bm{Q}^\\top) \\\\\n",
"&= \\mathrm{tr}(\\lambda_1 \\bm{q}_1 \\bm{q}_1^\\top + \\lambda_2 \\bm{q}_2 \\bm{q}_2^\\top + \\dots + \\lambda_d \\bm{q}_d \\bm{q}_d^\\top) \\\\\n",
"&= \\lambda_1 \\bm{q}_1^\\top \\bm{q}_1 + \\lambda_2 \\bm{q}_2^\\top \\bm{q}_2 + \\dots + \\lambda_d \\bm{q}_d^\\top \\bm{q}_d \\\\\n",
"&= \\lambda_1 + \\lambda_2 + \\dots + \\lambda_d \\\\\n",
"\\end{align}\n",
"\n",
"つまり、データ$\\bm{X}$の分散の和は分散共分散行列$\\bm{S}$の固有値の和に等しい"
]
},
{
"cell_type": "markdown",
"metadata": {
"jp-MarkdownHeadingCollapsed": true,
"tags": []
},
"source": [
"第$k$主成分に対応する固有値$\\lambda_k$は、元のデータ$X$を第$k$主成分の軸に射影したときの分散を表す。全ての主成分における分散の総和に対し、第$k$主成分の分散が占める割合を第$k$主成分の**寄与率**と呼ぶ。\n",
"\n",
":::{admonition} 寄与率\n",
":class: important\n",
"\n",
"\\begin{align}\n",
"\\frac{\\lambda_k}{\\sum_{i=1}^d \\lambda_i} = \\frac{\\lambda_k}{\\lambda_1 + \\lambda_2 + \\dots + \\lambda_d}\n",
"\\end{align}\n",
"\n",
":::\n",
"\n",
"また、第$k$主成分までに表現された分散の総和が、データ全体の分散の総和に占める割合を第$k$主成分までの**累積寄与率**と呼ぶ。\n",
"\n",
":::{admonition} 累積寄与率\n",
":class: important\n",
"\n",
"\\begin{align}\n",
"\\frac{\\sum_{i=1}^k \\lambda_i}{\\sum_{i=1}^d \\lambda_i} = \\frac{\\lambda_1 + \\lambda_2 \\dots + \\lambda_k}{\\lambda_1 + \\lambda_2 + \\dots + \\lambda_d}\n",
"\\end{align}\n",
"\n",
":::\n",
"\n",
"累積寄与率は、元々$d$次元のベクトルで表されていたデータ点を主成分分析で$k$次元に圧縮したとき、元データの分散をどの程度表現できているかを表す指標である。"
]
},
{
"cell_type": "markdown",
"metadata": {
"tags": []
},
"source": [
"## 応用例 (1): 麺類の支出額\n",
"\n",
"前章の冒頭で紹介した[教育用標準データセット（SSDSE: Standardized Statistical Data Set for Education）](https://www.nstac.go.jp/SSDSE/)の「C. 都道府県庁所在市別、家計消費データ」から麺類だけを抜き出したデータを2次元平面上で可視化してみよう。まず、データを読み込んで表示する。"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" 都道府県 | \n",
" 生うどん・そば | \n",
" 乾うどん・そば | \n",
" パスタ | \n",
" 中華麺 | \n",
" カップ麺 | \n",
" 即席麺 | \n",
" 他の麺類 | \n",
"
\n",
" \n",
" \n",
" \n",
" | 1 | \n",
" 北海道 | \n",
" 3162 | \n",
" 2082 | \n",
" 1266 | \n",
" 4152 | \n",
" 5189 | \n",
" 1609 | \n",
" 726 | \n",
"
\n",
" \n",
" | 2 | \n",
" 青森県 | \n",
" 2964 | \n",
" 2224 | \n",
" 1114 | \n",
" 5271 | \n",
" 6088 | \n",
" 2039 | \n",
" 554 | \n",
"
\n",
" \n",
" | 3 | \n",
" 岩手県 | \n",
" 3349 | \n",
" 2475 | \n",
" 1305 | \n",
" 5991 | \n",
" 5985 | \n",
" 1889 | \n",
" 898 | \n",
"
\n",
" \n",
" | 4 | \n",
" 宮城県 | \n",
" 3068 | \n",
" 2407 | \n",
" 1339 | \n",
" 5017 | \n",
" 5295 | \n",
" 1709 | \n",
" 842 | \n",
"
\n",
" \n",
" | 5 | \n",
" 秋田県 | \n",
" 3231 | \n",
" 3409 | \n",
" 1019 | \n",
" 5172 | \n",
" 5013 | \n",
" 1680 | \n",
" 554 | \n",
"
\n",
" \n",
" | 6 | \n",
" 山形県 | \n",
" 4478 | \n",
" 3084 | \n",
" 1288 | \n",
" 5236 | \n",
" 5875 | \n",
" 1745 | \n",
" 754 | \n",
"
\n",
" \n",
" | 7 | \n",
" 福島県 | \n",
" 2963 | \n",
" 2705 | \n",
" 1064 | \n",
" 4397 | \n",
" 5862 | \n",
" 1687 | \n",
" 919 | \n",
"
\n",
" \n",
" | 8 | \n",
" 茨城県 | \n",
" 3353 | \n",
" 2477 | \n",
" 1248 | \n",
" 4034 | \n",
" 4562 | \n",
" 1440 | \n",
" 653 | \n",
"
\n",
" \n",
" | 9 | \n",
" 栃木県 | \n",
" 3908 | \n",
" 2218 | \n",
" 1391 | \n",
" 4534 | \n",
" 4945 | \n",
" 1860 | \n",
" 742 | \n",
"
\n",
" \n",
" | 10 | \n",
" 群馬県 | \n",
" 4563 | \n",
" 1948 | \n",
" 1203 | \n",
" 4153 | \n",
" 5049 | \n",
" 1544 | \n",
" 546 | \n",
"
\n",
" \n",
" | 11 | \n",
" 埼玉県 | \n",
" 4016 | \n",
" 2256 | \n",
" 1487 | \n",
" 4512 | \n",
" 4584 | \n",
" 1568 | \n",
" 949 | \n",
"
\n",
" \n",
" | 12 | \n",
" 千葉県 | \n",
" 3389 | \n",
" 2277 | \n",
" 1441 | \n",
" 4582 | \n",
" 4513 | \n",
" 1840 | \n",
" 835 | \n",
"
\n",
" \n",
" | 13 | \n",
" 東京都 | \n",
" 3088 | \n",
" 2385 | \n",
" 1595 | \n",
" 4592 | \n",
" 3953 | \n",
" 1734 | \n",
" 974 | \n",
"
\n",
" \n",
" | 14 | \n",
" 神奈川県 | \n",
" 3410 | \n",
" 2684 | \n",
" 1472 | \n",
" 4484 | \n",
" 4035 | \n",
" 1741 | \n",
" 1049 | \n",
"
\n",
" \n",
" | 15 | \n",
" 新潟県 | \n",
" 2697 | \n",
" 3409 | \n",
" 1308 | \n",
" 4192 | \n",
" 6095 | \n",
" 2201 | \n",
" 612 | \n",
"
\n",
" \n",
" | 16 | \n",
" 富山県 | \n",
" 3294 | \n",
" 2738 | \n",
" 1184 | \n",
" 4367 | \n",
" 5576 | \n",
" 1928 | \n",
" 491 | \n",
"
\n",
" \n",
" | 17 | \n",
" 石川県 | \n",
" 3597 | \n",
" 2233 | \n",
" 1339 | \n",
" 5159 | \n",
" 4408 | \n",
" 1880 | \n",
" 607 | \n",
"
\n",
" \n",
" | 18 | \n",
" 福井県 | \n",
" 4124 | \n",
" 2549 | \n",
" 1200 | \n",
" 4247 | \n",
" 3838 | \n",
" 1875 | \n",
" 387 | \n",
"
\n",
" \n",
" | 19 | \n",
" 山梨県 | \n",
" 3870 | \n",
" 1947 | \n",
" 1123 | \n",
" 4529 | \n",
" 4786 | \n",
" 1580 | \n",
" 660 | \n",
"
\n",
" \n",
" | 20 | \n",
" 長野県 | \n",
" 4075 | \n",
" 2705 | \n",
" 1267 | \n",
" 4333 | \n",
" 4346 | \n",
" 1620 | \n",
" 808 | \n",
"
\n",
" \n",
" | 21 | \n",
" 岐阜県 | \n",
" 3154 | \n",
" 2106 | \n",
" 1239 | \n",
" 3816 | \n",
" 4566 | \n",
" 1907 | \n",
" 650 | \n",
"
\n",
" \n",
" | 22 | \n",
" 静岡県 | \n",
" 3045 | \n",
" 2414 | \n",
" 1319 | \n",
" 5106 | \n",
" 4582 | \n",
" 1682 | \n",
" 935 | \n",
"
\n",
" \n",
" | 23 | \n",
" 愛知県 | \n",
" 3991 | \n",
" 2198 | \n",
" 1394 | \n",
" 4239 | \n",
" 4490 | \n",
" 1882 | \n",
" 731 | \n",
"
\n",
" \n",
" | 24 | \n",
" 三重県 | \n",
" 3720 | \n",
" 2190 | \n",
" 1251 | \n",
" 4176 | \n",
" 4222 | \n",
" 1922 | \n",
" 595 | \n",
"
\n",
" \n",
" | 25 | \n",
" 滋賀県 | \n",
" 3835 | \n",
" 2004 | \n",
" 1295 | \n",
" 4121 | \n",
" 4397 | \n",
" 1976 | \n",
" 659 | \n",
"
\n",
" \n",
" | 26 | \n",
" 京都府 | \n",
" 3816 | \n",
" 2160 | \n",
" 1345 | \n",
" 4136 | \n",
" 3216 | \n",
" 1959 | \n",
" 596 | \n",
"
\n",
" \n",
" | 27 | \n",
" 大阪府 | \n",
" 3999 | \n",
" 1599 | \n",
" 1013 | \n",
" 3567 | \n",
" 5564 | \n",
" 2246 | \n",
" 540 | \n",
"
\n",
" \n",
" | 28 | \n",
" 兵庫県 | \n",
" 3437 | \n",
" 1988 | \n",
" 1129 | \n",
" 3914 | \n",
" 4630 | \n",
" 1984 | \n",
" 795 | \n",
"
\n",
" \n",
" | 29 | \n",
" 奈良県 | \n",
" 3543 | \n",
" 2727 | \n",
" 1229 | \n",
" 3973 | \n",
" 4067 | \n",
" 2003 | \n",
" 617 | \n",
"
\n",
" \n",
" | 30 | \n",
" 和歌山県 | \n",
" 3024 | \n",
" 2363 | \n",
" 940 | \n",
" 3603 | \n",
" 3972 | \n",
" 1921 | \n",
" 502 | \n",
"
\n",
" \n",
" | 31 | \n",
" 鳥取県 | \n",
" 3455 | \n",
" 1684 | \n",
" 1251 | \n",
" 3700 | \n",
" 5687 | \n",
" 3007 | \n",
" 436 | \n",
"
\n",
" \n",
" | 32 | \n",
" 島根県 | \n",
" 3456 | \n",
" 2314 | \n",
" 1316 | \n",
" 4166 | \n",
" 4233 | \n",
" 2086 | \n",
" 421 | \n",
"
\n",
" \n",
" | 33 | \n",
" 岡山県 | \n",
" 3384 | \n",
" 2219 | \n",
" 1425 | \n",
" 3958 | \n",
" 4655 | \n",
" 2036 | \n",
" 522 | \n",
"
\n",
" \n",
" | 34 | \n",
" 広島県 | \n",
" 3285 | \n",
" 1851 | \n",
" 1404 | \n",
" 4036 | \n",
" 4047 | \n",
" 2077 | \n",
" 585 | \n",
"
\n",
" \n",
" | 35 | \n",
" 山口県 | \n",
" 3426 | \n",
" 1840 | \n",
" 1164 | \n",
" 3342 | \n",
" 4937 | \n",
" 2533 | \n",
" 535 | \n",
"
\n",
" \n",
" | 36 | \n",
" 徳島県 | \n",
" 3692 | \n",
" 2106 | \n",
" 1107 | \n",
" 3340 | \n",
" 4149 | \n",
" 2038 | \n",
" 409 | \n",
"
\n",
" \n",
" | 37 | \n",
" 香川県 | \n",
" 6558 | \n",
" 3940 | \n",
" 972 | \n",
" 3661 | \n",
" 4412 | \n",
" 2170 | \n",
" 411 | \n",
"
\n",
" \n",
" | 38 | \n",
" 愛媛県 | \n",
" 3744 | \n",
" 1939 | \n",
" 1197 | \n",
" 3624 | \n",
" 3728 | \n",
" 1825 | \n",
" 498 | \n",
"
\n",
" \n",
" | 39 | \n",
" 高知県 | \n",
" 3333 | \n",
" 1513 | \n",
" 1134 | \n",
" 3306 | \n",
" 5126 | \n",
" 2234 | \n",
" 478 | \n",
"
\n",
" \n",
" | 40 | \n",
" 福岡県 | \n",
" 2877 | \n",
" 1745 | \n",
" 1354 | \n",
" 3376 | \n",
" 4084 | \n",
" 2128 | \n",
" 640 | \n",
"
\n",
" \n",
" | 41 | \n",
" 佐賀県 | \n",
" 2428 | \n",
" 2032 | \n",
" 1132 | \n",
" 3251 | \n",
" 4329 | \n",
" 2330 | \n",
" 517 | \n",
"
\n",
" \n",
" | 42 | \n",
" 長崎県 | \n",
" 2556 | \n",
" 1882 | \n",
" 1066 | \n",
" 3720 | \n",
" 4054 | \n",
" 1893 | \n",
" 463 | \n",
"
\n",
" \n",
" | 43 | \n",
" 熊本県 | \n",
" 2307 | \n",
" 1647 | \n",
" 1303 | \n",
" 2840 | \n",
" 4631 | \n",
" 2864 | \n",
" 501 | \n",
"
\n",
" \n",
" | 44 | \n",
" 大分県 | \n",
" 2493 | \n",
" 1597 | \n",
" 1074 | \n",
" 3164 | \n",
" 4158 | \n",
" 2384 | \n",
" 552 | \n",
"
\n",
" \n",
" | 45 | \n",
" 宮崎県 | \n",
" 2668 | \n",
" 1573 | \n",
" 1185 | \n",
" 3290 | \n",
" 4098 | \n",
" 2232 | \n",
" 463 | \n",
"
\n",
" \n",
" | 46 | \n",
" 鹿児島県 | \n",
" 2868 | \n",
" 2391 | \n",
" 1289 | \n",
" 3524 | \n",
" 3994 | \n",
" 2054 | \n",
" 462 | \n",
"
\n",
" \n",
" | 47 | \n",
" 沖縄県 | \n",
" 1753 | \n",
" 1488 | \n",
" 1012 | \n",
" 4424 | \n",
" 3342 | \n",
" 1481 | \n",
" 229 | \n",
"
\n",
" \n",
"
\n",
"
"
]
},
"metadata": {
"needs_background": "light"
},
"output_type": "display_data"
}
],
"source": [
"from pandas import plotting \n",
"plotting.scatter_matrix(df.iloc[:,1:], figsize=(10, 10), alpha=0.5)\n",
"plt.show()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[sklearn.decomposition.PCA](https://scikit-learn.org/stable/modules/generated/sklearn.decomposition.PCA.html)を用いて主成分分析を実行する。"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [],
"source": [
"from sklearn.decomposition import PCA\n",
"\n",
"pca = PCA()\n",
"P = pca.fit_transform(X)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"7個の主成分ベクトル（PC1からPC7）を表示する。第1主成分ベクトルの係数が概ね正であるので、第1主成分得点は麺類の消費の多さを反映すると考えられる。第2主成分ベクトルの係数は、「うどん・そば」では正、「中華麺」や「カップ麺」では負であるので、うどん・そばの消費の多さを反映すると考えられる。"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" 生うどん・そば | \n",
" 乾うどん・そば | \n",
" パスタ | \n",
" 中華麺 | \n",
" カップ麺 | \n",
" 即席麺 | \n",
" 他の麺類 | \n",
"
\n",
" \n",
" \n",
" \n",
" | PC1 | \n",
" 0.5050 | \n",
" 0.4089 | \n",
" 0.0169 | \n",
" 0.5472 | \n",
" 0.5031 | \n",
" -0.1329 | \n",
" 0.0858 | \n",
"
\n",
" \n",
" | PC2 | \n",
" 0.7567 | \n",
" 0.1505 | \n",
" -0.0064 | \n",
" -0.2922 | \n",
" -0.5624 | \n",
" -0.0256 | \n",
" -0.0487 | \n",
"
\n",
" \n",
" | PC3 | \n",
" -0.2577 | \n",
" 0.1220 | \n",
" 0.0783 | \n",
" 0.6099 | \n",
" -0.6228 | \n",
" -0.3794 | \n",
" 0.0938 | \n",
"
\n",
" \n",
" | PC4 | \n",
" -0.3246 | \n",
" 0.8904 | \n",
" -0.0693 | \n",
" -0.2862 | \n",
" -0.0487 | \n",
" 0.0988 | \n",
" -0.0550 | \n",
"
\n",
" \n",
" | PC5 | \n",
" 0.0214 | \n",
" 0.0239 | \n",
" 0.2031 | \n",
" 0.3552 | \n",
" -0.1849 | \n",
" 0.8891 | \n",
" -0.0829 | \n",
"
\n",
" \n",
" | PC6 | \n",
" 0.0054 | \n",
" -0.0359 | \n",
" -0.6138 | \n",
" 0.1797 | \n",
" -0.0219 | \n",
" -0.0069 | \n",
" -0.7675 | \n",
"
\n",
" \n",
" | PC7 | \n",
" -0.0093 | \n",
" 0.0255 | \n",
" 0.7555 | \n",
" -0.0536 | \n",
" 0.0759 | \n",
" -0.1934 | \n",
" -0.6184 | \n",
"
\n",
" \n",
"
\n",
"
\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" PC1 | \n",
" PC2 | \n",
" PC3 | \n",
" PC4 | \n",
" PC5 | \n",
" PC6 | \n",
" PC7 | \n",
"
\n",
" \n",
" \n",
" \n",
" | 北海道 | \n",
" 153.4222 | \n",
" -534.3034 | \n",
" -152.1148 | \n",
" -141.8253 | \n",
" -420.6236 | \n",
" -98.0002 | \n",
" 65.1171 | \n",
"
\n",
" \n",
" | 青森県 | \n",
" 1101.6032 | \n",
" -1496.8960 | \n",
" -152.2914 | \n",
" -252.7109 | \n",
" 175.4282 | \n",
" 299.5492 | \n",
" -12.7754 | \n",
"
\n",
" \n",
" | 岩手県 | \n",
" 1793.4762 | \n",
" -1334.3605 | \n",
" 386.5550 | \n",
" -402.2430 | \n",
" 341.3197 | \n",
" 43.9761 | \n",
" -95.7998 | \n",
"
\n",
" \n",
" | 宮城県 | \n",
" 763.3783 | \n",
" -877.5282 | \n",
" 352.0041 | \n",
" -76.2460 | \n",
" -33.1394 | \n",
" -91.6132 | \n",
" 0.0315 | \n",
"
\n",
" \n",
" | 秋田県 | \n",
" 1172.0693 | \n",
" -473.2078 | \n",
" 661.3547 | \n",
" 767.5094 | \n",
" 34.5863 | \n",
" 324.9647 | \n",
" -63.6768 | \n",
"
\n",
" \n",
" | 山形県 | \n",
" 2150.6186 | \n",
" -95.0627 | \n",
" -182.2755 | \n",
" -10.1633 | \n",
" 12.6520 | \n",
" 16.8480 | \n",
" 45.4275 | \n",
"
\n",
" \n",
" | 福島県 | \n",
" 783.0933 | \n",
" -1051.2636 | \n",
" -321.7877 | \n",
" 385.6438 | \n",
" -435.1023 | \n",
" -116.8548 | \n",
" -166.2046 | \n",
"
\n",
" \n",
" | 茨城県 | \n",
" 47.2661 | \n",
" 64.7670 | \n",
" 221.2223 | \n",
" 200.7490 | \n",
" -480.9328 | \n",
" -50.3860 | \n",
" 96.3663 | \n",
"
\n",
" \n",
" | 栃木県 | \n",
" 642.1218 | \n",
" 68.3046 | \n",
" -26.7517 | \n",
" -345.0909 | \n",
" 26.5946 | \n",
" -115.6424 | \n",
" 58.6243 | \n",
"
\n",
" \n",
" | 群馬県 | \n",
" 728.3322 | \n",
" 594.9992 | \n",
" -438.8713 | \n",
" -701.5308 | \n",
" -423.2849 | \n",
" 94.8302 | \n",
" 114.2694 | \n",
"
\n",
" \n",
" | 埼玉県 | \n",
" 576.7350 | \n",
" 361.9543 | \n",
" 299.1710 | \n",
" -369.3226 | \n",
" -168.5329 | \n",
" -328.2688 | \n",
" 33.3482 | \n",
"
\n",
" \n",
" | 千葉県 | \n",
" 224.5805 | \n",
" -90.9914 | \n",
" 432.7337 | \n",
" -127.3440 | \n",
" 98.5016 | \n",
" -204.3793 | \n",
" 13.7063 | \n",
"
\n",
" \n",
" | 東京都 | \n",
" -130.8999 | \n",
" 4.4330 | \n",
" 943.6409 | \n",
" 62.1463 | \n",
" 127.2547 | \n",
" -396.2840 | \n",
" 27.0850 | \n",
"
\n",
" \n",
" | 神奈川県 | \n",
" 139.5407 | \n",
" 275.5104 | \n",
" 774.9649 | \n",
" 255.8418 | \n",
" 62.7917 | \n",
" -408.6177 | \n",
" -96.9218 | \n",
"
\n",
" \n",
" | 新潟県 | \n",
" 851.1234 | \n",
" -1217.4607 | \n",
" -642.1768 | \n",
" 1196.9064 | \n",
" -7.8734 | \n",
" -103.1752 | \n",
" 157.6930 | \n",
"
\n",
" \n",
" | 富山県 | \n",
" 736.6941 | \n",
" -612.2866 | \n",
" -365.4479 | \n",
" 369.1469 | \n",
" -110.8816 | \n",
" 137.8001 | \n",
" 120.1706 | \n",
"
\n",
" \n",
" | 石川県 | \n",
" 547.9223 | \n",
" -39.0012 | \n",
" 746.5229 | \n",
" -370.5084 | \n",
" 359.9966 | \n",
" 141.6180 | \n",
" 27.9426 | \n",
"
\n",
" \n",
" | 福井県 | \n",
" 136.8892 | \n",
" 1006.1167 | \n",
" 418.3710 | \n",
" 49.8097 | \n",
" 145.8780 | \n",
" 235.9303 | \n",
" 68.7181 | \n",
"
\n",
" \n",
" | 山梨県 | \n",
" 455.0597 | \n",
" 102.5065 | \n",
" 123.4897 | \n",
" -569.4544 | \n",
" -249.6373 | \n",
" 125.8385 | \n",
" -57.3433 | \n",
"
\n",
" \n",
" | 長野県 | \n",
" 549.7100 | \n",
" 667.2875 | \n",
" 327.6183 | \n",
" 102.2576 | \n",
" -162.8597 | \n",
" -128.1230 | \n",
" -53.2867 | \n",
"
\n",
" \n",
" | 岐阜県 | \n",
" -384.6726 | \n",
" -91.9716 | \n",
" -86.3750 | \n",
" 44.1388 | \n",
" -158.5730 | \n",
" -72.7617 | \n",
" 5.4615 | \n",
"
\n",
" \n",
" | 静岡県 | \n",
" 455.8524 | \n",
" -522.6362 | \n",
" 874.5050 | \n",
" -59.6981 | \n",
" 94.2201 | \n",
" -119.2878 | \n",
" -125.8976 | \n",
"
\n",
" \n",
" | 愛知県 | \n",
" 281.7116 | \n",
" 470.1198 | \n",
" 43.7052 | \n",
" -280.6772 | \n",
" 28.3411 | \n",
" -151.0600 | \n",
" 43.4152 | \n",
"
\n",
" \n",
" | 三重県 | \n",
" -47.1050 | \n",
" 439.4803 | \n",
" 201.9089 | \n",
" -147.4071 | \n",
" 67.3410 | \n",
" 34.2168 | \n",
" -2.9096 | \n",
"
\n",
" \n",
" | 滋賀県 | \n",
" -8.0834 | \n",
" 411.3792 | \n",
" -3.9890 | \n",
" -344.3619 | \n",
" 65.1042 | \n",
" -48.7050 | \n",
" -9.2709 | \n",
"
\n",
" \n",
" | 京都府 | \n",
" -542.1400 | \n",
" 1083.4309 | \n",
" 769.0285 | \n",
" -147.7499 | \n",
" 292.4058 | \n",
" -8.0488 | \n",
" -15.5865 | \n",
"
\n",
" \n",
" | 大阪府 | \n",
" 142.2460 | \n",
" -19.1879 | \n",
" -1296.0106 | \n",
" -603.7145 | \n",
" -160.9610 | \n",
" 104.1794 | \n",
" -94.4841 | \n",
"
\n",
" \n",
" | 兵庫県 | \n",
" -203.8417 | \n",
" 31.4682 | \n",
" -178.0018 | \n",
" -176.7118 | \n",
" -98.2611 | \n",
" -95.1057 | \n",
" -188.2460 | \n",
"
\n",
" \n",
" | 奈良県 | \n",
" -115.2237 | \n",
" 529.8485 | \n",
" 255.3790 | \n",
" 462.1403 | \n",
" 98.6741 | \n",
" -23.0343 | \n",
" -34.3392 | \n",
"
\n",
" \n",
" | 和歌山県 | \n",
" -780.2281 | \n",
" 253.3868 | \n",
" 175.8894 | \n",
" 435.2915 | \n",
" -157.0167 | \n",
" 189.0887 | \n",
" -157.5332 | \n",
"
\n",
" \n",
" | 鳥取県 | \n",
" -69.1043 | \n",
" -542.0104 | \n",
" -1420.7600 | \n",
" -331.0915 | \n",
" 587.4943 | \n",
" 47.9239 | \n",
" 11.8743 | \n",
"
\n",
" \n",
" | 島根県 | \n",
" -165.2852 | \n",
" 258.9724 | \n",
" 198.6653 | \n",
" 72.2967 | \n",
" 232.5081 | \n",
" 118.8384 | \n",
" 129.0717 | \n",
"
\n",
" \n",
" | 岡山県 | \n",
" -124.8443 | \n",
" 9.2985 | \n",
" -147.0644 | \n",
" 32.0185 | \n",
" 46.0927 | \n",
" -68.8385 | \n",
" 200.0767 | \n",
"
\n",
" \n",
" | 広島県 | \n",
" -588.9009 | \n",
" 194.1220 | \n",
" 248.4702 | \n",
" -254.1786 | \n",
" 202.2912 | \n",
" -64.5546 | \n",
" 78.4911 | \n",
"
\n",
" \n",
" | 山口県 | \n",
" -523.1440 | \n",
" -6.2518 | \n",
" -963.2277 | \n",
" -90.0340 | \n",
" 154.8302 | \n",
" -25.0465 | \n",
" -56.9032 | \n",
"
\n",
" \n",
" | 徳島県 | \n",
" -623.5714 | \n",
" 697.9746 | \n",
" -338.2960 | \n",
" 61.3932 | \n",
" -129.3636 | \n",
" 118.8313 | \n",
" 18.2818 | \n",
"
\n",
" \n",
" | 香川県 | \n",
" 1861.8465 | \n",
" 2898.5926 | \n",
" -881.5167 | \n",
" 681.5892 | \n",
" 130.8813 | \n",
" 200.6197 | \n",
" -87.5708 | \n",
"
\n",
" \n",
" | 愛媛県 | \n",
" -684.5301 | \n",
" 866.4939 | \n",
" 159.5305 | \n",
" -197.1324 | \n",
" -132.0062 | \n",
" 63.2729 | \n",
" 20.4916 | \n",
"
\n",
" \n",
" | 高知県 | \n",
" -594.0772 | \n",
" -211.0039 | \n",
" -1013.0915 | \n",
" -374.2155 | \n",
" -169.9183 | \n",
" 39.8092 | \n",
" 22.3229 | \n",
"
\n",
" \n",
" | 福岡県 | \n",
" -1183.6986 | \n",
" 37.7734 | \n",
" -103.0164 | \n",
" -23.5416 | \n",
" -19.5804 | \n",
" -194.1989 | \n",
" 36.1238 | \n",
"
\n",
" \n",
" | 佐賀県 | \n",
" -1279.3688 | \n",
" -357.8023 | \n",
" -286.6590 | \n",
" 443.6874 | \n",
" 32.6912 | \n",
" -5.4547 | \n",
" -57.7901 | \n",
"
\n",
" \n",
" | 長崎県 | \n",
" -1105.4465 | \n",
" -251.6660 | \n",
" 274.9161 | \n",
" 112.1158 | \n",
" -148.1998 | \n",
" 175.8729 | \n",
" -40.7751 | \n",
"
\n",
" \n",
" | 熊本県 | \n",
" -1640.3146 | \n",
" -571.0515 | \n",
" -931.9119 | \n",
" 284.8872 | \n",
" 329.9288 | \n",
" -169.0873 | \n",
" 14.2677 | \n",
"
\n",
" \n",
" | 大分県 | \n",
" -1563.1952 | \n",
" -255.2315 | \n",
" -324.8045 | \n",
" 75.9314 | \n",
" 57.7498 | \n",
" 7.0017 | \n",
" -153.7285 | \n",
"
\n",
" \n",
" | 宮崎県 | \n",
" -1431.4404 | \n",
" -121.9727 | \n",
" -200.6135 | \n",
" -53.1888 | \n",
" 11.5342 | \n",
" 33.9756 | \n",
" 1.0191 | \n",
"
\n",
" \n",
" | 鹿児島県 | \n",
" -894.9367 | \n",
" 146.5679 | \n",
" 130.7319 | \n",
" 523.5690 | \n",
" 0.6234 | \n",
" -11.8609 | \n",
" 113.2129 | \n",
"
\n",
" \n",
" | 沖縄県 | \n",
" -1611.2403 | \n",
" -701.6404 | \n",
" 1436.6764 | \n",
" -168.9272 | \n",
" -150.9670 | \n",
" 543.4036 | \n",
" 48.4329 | \n",
"
\n",
" \n",
"
\n",
"
![\"クリエイティブ・コモンズ・ライセンス\"](\"https://i.creativecommons.org/l/by-nc-nd/4.0/80x15.png\")
ただし、作品中のコードセル部分はMITライセンスの下に提供されています。"
]
}
],
"metadata": {
"@context": {
"CreativeWork": "http://schema.org/CreativeWork",
"Organization": "http://schema.org/Organization",
"Person": "http://schema.org/Person",
"author": "http://schema.org/author",
"copyrightHolder": "http://schema.org/copyrightHolder",
"copyrightYear": "http://schema.org/copyrightYear",
"license": "http://schema.org/license",
"name": "http://schema.org/name",
"title": "http://schema.org/name",
"url": "http://schema.org/url"
},
"@type": "CreativeWork",
"author": [
{
"@type": "Person",
"name": "Naoaki Okazaki",
"url": "https://www.chokkan.org/"
}
],
"copyrightHolder": [
{
"@type": "Person",
"name": "Naoaki Okazaki",
"url": "https://www.chokkan.org/"
}
],
"copyrightYear": 2022,
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.10"
},
"license": "https://creativecommons.org/licenses/by-nc-nd/4.0/deed.ja",
"title": "機械学習帳"
},
"nbformat": 4,
"nbformat_minor": 4
}