{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Probability calibration\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"分類モデルの確率補正\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# default_exp proba_calib"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## 問題提起\n",
"\n",
"スパムメールやクレジット詐欺を見分けるタスクなどを学習した\n",
"分類モデルが出力する予測値は通常 \n",
"(0, 1) の範囲内に収まり、\n",
"予測確率とも呼ばれるので、\n",
"うっかり正例であるクラス確率だと\n",
"思い込みかねません。\n",
"\n",
"実運用では、閾値を設けて、予測値がその閾値を超えるかどうかで判断を下したりします。\n",
"予測値がクラス確率であるかどうかによって、閾値の意味も大きく変わってきます。\n",
"\n",
"スパムメール分類モデルの場合、真のクラス確率を学習したのなら、 \n",
"90% という予測値が出力されたような100通のメールのうち、\n",
"90通が本当にスパムメールだろうと期待されます。\n",
"本当のスパムメールの数が90を下回ったらモデルの自信過剰、\n",
"上回ったら自信不足と言えます。\n",
"\n",
"モデルの予測値が真の確率とどのくらい乖離しているのか図り、\n",
"予測値を真の確率に近づける補正方法を実験してみましょう。\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# exporti\n",
"import matplotlib.pyplot as plt\n",
"import japanize_matplotlib\n",
"from sklearn.calibration import CalibratedClassifierCV, calibration_curve\n",
"from sklearn.metrics import brier_score_loss, roc_auc_score"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## ライブラリの用意\n",
"\n",
"sklearn を使って信頼性曲線を書いたり確率補正します。\n",
"補正前のベースモデルとしては LightGBM を使います。\n",
"図形は Matplotlib と Plotly で作ります。\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"import numpy as np\n",
"import pandas as pd\n",
"import plotly.express as px\n",
"from lightgbm import LGBMClassifier\n",
"from sklearn import datasets\n",
"from sklearn.datasets import make_circles\n",
"from sklearn.model_selection import train_test_split"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## サンプルデータ\n",
"\n",
"簡単な実験データとして、\n",
"[sklearn](https://scikit-learn.org/stable/modules/generated/sklearn.datasets.make_circles.html#sklearn.datasets.make_circles)を使って、\n",
"円を指定し、その内側を正例として、外側を負例とします。\n",
"ノイズを投入したり、正例の割合を減らしてデータを不均衡化しています。\n",
"\n",
"実験データを学習用・補正用・試験用に分けました。\n"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"ノイズ率(ラベルが反転) 0.08449600000000002\n",
"正例率 0.20687921246278282\n",
"学習・補正・テスト用データ比率 [0.74365384 0.00634497 0.25000119]\n"
]
}
],
"source": [
"n_samples = 1_000_000\n",
"pos_rate = 0.1\n",
"radius = 0.3\n",
"\n",
"X, y = make_circles(n_samples=n_samples, factor=radius, noise=0.2)\n",
"df = pd.DataFrame(X, columns=[\"x1\", \"x2\"])\n",
"df[\"y\"] = pd.Series(y).astype(\"category\")\n",
"df[\"r\"] = df.x1 ** 2 + df.x2 ** 2\n",
"print(\"ノイズ率(ラベルが反転)\", 1 - sum((df.r <= radius) == (df.y == 1)) / n_samples)\n",
"\n",
"# ランダムに正例をドロップし、インバランスクラス化\n",
"df = pd.concat(\n",
" [\n",
" df[(df.y == 0) | (df.r >= radius)],\n",
" df[(df.y == 1) & (df.r < radius)].sample(int(pos_rate * sum(df.y))),\n",
" ]\n",
")\n",
"X = df[[\"x1\", \"x2\"]]\n",
"y = df.y\n",
"\n",
"print(\"正例率\", sum(y) / len(y))\n",
"\n",
"# 学習・補正・テストデータ分割\n",
"X_train, X_test, y_train, y_test = train_test_split(\n",
" X, y, test_size=0.25, random_state=42\n",
")\n",
"X_train_, X_calib, y_train_, y_calib = train_test_split(\n",
" X_train,\n",
" y_train,\n",
" test_size=4000 / len(X_train),\n",
" random_state=2020,\n",
")\n",
"print(\"学習・補正・テスト用データ比率\", np.array([len(X_train_), len(X_calib), len(X_test)]) / len(X))"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# exports\n",
"def plot_calibration_curve(named_classifiers, X_test, y_test):\n",
" fig = plt.figure(figsize=(10, 10))\n",
" ax1 = plt.subplot2grid((3, 1), (0, 0), rowspan=2)\n",
" ax2 = plt.subplot2grid((3, 1), (2, 0))\n",
"\n",
" ax1.plot([0, 1], [0, 1], \"k:\", label=\"完全な補正\")\n",
" for name, clf in named_classifiers.items():\n",
" prob_pos = clf.predict_proba(X_test)[:, 1]\n",
" auc = roc_auc_score(y_test, prob_pos)\n",
" brier = brier_score_loss(y_test, prob_pos)\n",
" print(\"%s:\" % name)\n",
" print(\"\\tAUC : %1.3f\" % auc)\n",
" print(\"\\tBrier: %1.3f\" % (brier))\n",
" print()\n",
"\n",
" fraction_of_positives, mean_predicted_value = calibration_curve(\n",
" y_test,\n",
" prob_pos,\n",
" n_bins=10,\n",
" )\n",
"\n",
" ax1.plot(\n",
" mean_predicted_value,\n",
" fraction_of_positives,\n",
" \"s-\",\n",
" label=\"%s (%1.3f)\" % (name, brier),\n",
" )\n",
"\n",
" ax2.hist(prob_pos, range=(0, 1), bins=10, label=name, histtype=\"step\", lw=2)\n",
"\n",
" ax1.set_ylabel(\"正例の比率\")\n",
" ax1.set_ylim([-0.05, 1.05])\n",
" ax1.legend(loc=\"lower right\")\n",
" ax1.set_title(\"信頼性曲線\")\n",
"\n",
" ax2.set_xlabel(\"予測値の平均\")\n",
" ax2.set_ylabel(\"サンプル数\")\n",
" ax2.legend(loc=\"upper center\", ncol=2)\n",
"\n",
" plt.tight_layout()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
" \n",
" "
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"application/vnd.plotly.v1+json": {
"config": {
"plotlyServerURL": "https://plot.ly"
},
"data": [
{
"customdata": [
[
0.5461556652424138
],
[
0.252653252487062
],
[
2.570393930049533
],
[
1.0773090107111591
],
[
1.306907994247422
],
[
0.4829624088841532
],
[
1.2794561310336974
],
[
2.2444261687868203
],
[
1.3280928811645365
],
[
1.3477157694907018
],
[
1.4767405520390027
],
[
0.6532279865273313
],
[
0.6300613981936767
],
[
0.6551262952173773
],
[
1.4986121298554287
],
[
1.6585720467087395
],
[
0.8927214905247108
],
[
2.123952871924093
],
[
1.4336353213075612
],
[
0.9361945680185347
],
[
1.1291153557972198
],
[
1.3764346694255722
],
[
1.398008336613556
],
[
0.8585063436504958
],
[
1.1359615964299292
],
[
0.5631007152895534
],
[
0.42656995986966106
],
[
1.7036553554172902
],
[
0.4887954412604719
],
[
0.5945379777892073
],
[
1.2318794706623213
],
[
0.9390880923663875
],
[
0.9808813999068859
],
[
1.4526857413766496
],
[
1.296997524079722
],
[
0.6993101292846846
],
[
0.4847842271951644
],
[
1.0656530511495867
],
[
0.8843957479751113
],
[
1.1131455508052341
],
[
1.2908228061696783
],
[
0.6037278919099301
],
[
0.5952785495000723
],
[
0.9380542314832108
],
[
1.0468606353283023
],
[
0.9911436082197975
],
[
0.8029431416268581
],
[
0.8611200812376716
],
[
1.2893200076030942
],
[
0.9018297314360552
],
[
1.6286022958860373
],
[
1.2255825414485624
],
[
0.704164964704143
],
[
0.44680615218564523
],
[
1.8721631580366591
],
[
1.3861276267491993
],
[
0.8535588069260541
],
[
1.4858570959354915
],
[
0.9233677545231858
],
[
1.1313865278657527
],
[
1.561154136579955
],
[
1.4231371913912536
],
[
0.8737588541401233
],
[
1.3939486766491374
],
[
0.9711947010776527
],
[
1.723135726551078
],
[
1.2627028288405864
],
[
0.1868626528946754
],
[
0.9276501406108888
],
[
1.898968297393148
],
[
0.8406554526184902
],
[
0.8156603510087991
],
[
1.6176406330832687
],
[
1.5253268807178597
],
[
0.6761751113508679
],
[
0.45820613567942486
],
[
0.5155687872420078
],
[
1.9442643721028083
],
[
0.755375398604532
],
[
1.3168209970164388
],
[
1.2153424170454112
],
[
1.1767394361066321
],
[
1.2424981633408414
],
[
1.8533152181840773
],
[
0.9497700287889967
],
[
1.1081303141186583
],
[
0.8202570826578113
],
[
1.1357967188296534
],
[
1.6454736958784917
],
[
1.3848917541842154
],
[
2.053032935629723
],
[
1.2438970243914642
],
[
1.095974765973881
],
[
0.9266430641861326
],
[
2.4112916752684095
],
[
0.6936755649348172
],
[
1.4162507148505656
],
[
1.213360213177242
],
[
0.47013411832635316
],
[
0.5479533148630666
],
[
1.3470253437100854
],
[
1.7380164992120286
],
[
1.0266031178783164
],
[
0.9932804186925358
],
[
0.6561012483081187
],
[
1.6686050517906916
],
[
0.7499789603510357
],
[
1.0733808312704358
],
[
0.6456558531443575
],
[
2.1095280676940353
],
[
1.0084690407613979
],
[
0.7414970774862685
],
[
0.3950899435769698
],
[
2.247119597297578
],
[
1.3991972377605624
],
[
1.2331208036855708
],
[
0.7856622420656044
],
[
2.313893546663363
],
[
1.0235380779397263
],
[
1.182923552313234
],
[
0.6170828778100372
],
[
1.0913425171410533
],
[
0.9415270643650312
],
[
1.2779833493238109
],
[
0.4727380055234331
],
[
0.6617422071209221
],
[
1.2275962807967604
],
[
1.6554508040109415
],
[
1.0765512942661064
],
[
0.22599907899411975
],
[
0.5367233069012239
],
[
1.1849559057680585
],
[
1.2549415678086742
],
[
0.5937071798852434
],
[
1.9297242976728741
],
[
0.7372398358348102
],
[
0.577714248288045
],
[
0.703276507309359
],
[
1.431929334726012
],
[
0.8723484111387959
],
[
0.9451763013987882
],
[
1.049497386738661
],
[
0.8890619341193163
],
[
1.0628641118125877
],
[
1.3922370089301255
],
[
1.149707697504177
],
[
0.5866125284809751
],
[
0.722133016529112
],
[
1.3995494624032372
],
[
1.3753082333782694
],
[
1.3513886836867741
],
[
1.6533453377778744
],
[
1.004734039185607
],
[
1.0009211414593642
],
[
1.0641205801882698
],
[
0.4989974743741773
],
[
0.5139505566415499
],
[
1.1438899549208812
],
[
1.1528110259283628
],
[
0.8690127821392325
],
[
0.8810680959070806
],
[
1.1752211124015863
],
[
1.5678608889875167
],
[
0.5333149858983199
],
[
0.5312957569578246
],
[
1.052287749693016
],
[
0.9489658017578375
],
[
0.40408158764163665
],
[
1.3613513186659527
],
[
1.6086603177034553
],
[
0.9048949664553648
],
[
1.1364419997715804
],
[
1.148683492056411
],
[
0.5843045779134949
],
[
1.9129704898318198
],
[
0.6805586254030324
],
[
1.5908455507879082
],
[
1.0240791659102348
],
[
1.0503842344806213
],
[
1.3602078486601943
],
[
0.6158060667645001
],
[
1.4937501818488867
],
[
0.9455485989499772
],
[
1.542870774988391
],
[
1.242741231310595
],
[
0.9395174329116511
],
[
0.9458890777110838
],
[
0.7940440210347868
],
[
1.5718842648740037
],
[
0.5631655990523228
],
[
1.1650315112497551
],
[
1.3491885285642806
],
[
1.5808615989459573
],
[
1.0762978344772274
],
[
1.378545273728775
],
[
1.616527475391276
],
[
1.2562520480748565
],
[
1.2357550189135609
],
[
1.8461095680588007
],
[
0.510011215135453
],
[
1.2940981049129767
],
[
1.8330561483790775
],
[
0.9625798327882711
],
[
1.681412448674022
],
[
0.2577412312039475
],
[
0.9323397842469625
],
[
0.9816098462014318
],
[
1.0150794455408934
],
[
1.0975568635178663
],
[
1.2595000330117685
],
[
1.5669700901200283
],
[
1.3031012140872098
],
[
0.8126212110644702
],
[
0.9897490516645175
],
[
0.5650727536617729
],
[
1.8702583343328965
],
[
1.0973019857390667
],
[
1.1165649418926356
],
[
1.8613353099760732
],
[
2.2590623680896744
],
[
1.4003314307834809
],
[
1.5071150744916895
],
[
1.9683381950397
],
[
0.703092684030591
],
[
1.4832312427574506
],
[
0.8873027627256571
],
[
1.571120164666701
],
[
1.4735144744784268
],
[
0.5375536306120027
],
[
1.381971964155014
],
[
1.86569095067229
],
[
1.3461618040245815
],
[
1.3703519137245772
],
[
1.205907462508357
],
[
1.0627930051046526
],
[
0.8615667753636393
],
[
0.6901598624480023
],
[
1.945121342368419
],
[
0.517782071144279
],
[
0.764144419140144
],
[
1.4227888532292943
],
[
0.6325384475864674
],
[
1.456361654384939
],
[
1.1842695556737484
],
[
1.8172452299754545
],
[
0.7510664789304109
],
[
1.3746664549180787
],
[
0.5449324893195199
],
[
1.4036794218930613
],
[
1.2744794275618423
],
[
0.9801639605255108
],
[
0.6874070306212964
],
[
1.307713604304729
],
[
0.3917166856225642
],
[
0.9908224815912494
],
[
0.8601789366877559
],
[
1.427201501590383
],
[
0.8717976431040794
],
[
1.1954422821034116
],
[
0.891972072248016
],
[
1.1494403714738648
],
[
1.7148643123110627
],
[
0.7048141398599845
],
[
1.3882112804254372
],
[
1.607203232580683
],
[
1.9915076144054278
],
[
0.9158368768184567
],
[
1.4240026626572873
],
[
1.4588279905869976
],
[
0.7598810307244482
],
[
1.7415190306586004
],
[
0.7054209424689986
],
[
0.8731470988604942
],
[
1.0580031804505583
],
[
0.9297083708874413
],
[
1.0636939589601488
],
[
1.215671970856336
],
[
1.3726420766604526
],
[
0.8658549275974783
],
[
0.9758142771459579
],
[
1.1917135039871314
],
[
1.2489893856866416
],
[
2.1418632468214662
],
[
1.2167734269529629
],
[
1.3197106297182168
],
[
0.8659654969520783
],
[
0.9142285529799784
],
[
1.3012640959963726
],
[
1.3494760142724629
],
[
1.5073227931403808
],
[
0.7735518516086388
],
[
0.9485561436306308
],
[
0.7685637976010137
],
[
1.0146030303395022
],
[
0.9866966983412663
],
[
1.2745781669047498
],
[
1.2917932460983699
],
[
0.6675249093648716
],
[
0.5502604128732795
],
[
1.3440564154161645
],
[
1.6598413696436412
],
[
1.1643416381429152
],
[
1.7100666431130374
],
[
1.0423858172713827
],
[
1.501013014942848
],
[
0.9829756417358251
],
[
1.2037875575292105
],
[
2.03626873153849
],
[
1.1747086156941036
],
[
0.2957551439357597
],
[
0.9991884205530585
],
[
1.3444289729272378
],
[
0.9205862597789012
],
[
0.8975388323380685
],
[
0.8470668594639256
],
[
0.9141721416703452
],
[
1.0180245830857033
],
[
0.7315564688136402
],
[
1.682725105598951
],
[
0.655143407896876
],
[
1.115250277741097
],
[
1.2093619417485988
],
[
0.7746186282257388
],
[
1.0641683531250494
],
[
0.5453204302357133
],
[
0.890609915204415
],
[
1.3111974043170627
],
[
1.804478000155973
],
[
0.7301051456172294
],
[
2.249845870119364
],
[
0.6139152026356813
],
[
1.219432014693255
],
[
0.7167196729877978
],
[
0.9559600744579924
],
[
1.0270934297441998
],
[
1.237402649110681
],
[
0.716804881313569
],
[
1.5202553654331539
],
[
0.9258676486943845
],
[
1.1848493253971042
],
[
1.0157833057079537
],
[
1.3948421001826559
],
[
0.645920689769431
],
[
1.5029004737990306
],
[
1.3110825469643268
],
[
0.5152576685428548
],
[
1.2403339871297696
],
[
0.3888013464285374
],
[
0.3450724641917416
],
[
1.3649441861969303
],
[
1.1304364161283245
],
[
0.9948224926749846
],
[
1.6339574648156683
],
[
1.5976695202071507
],
[
1.1489610698670225
],
[
0.8957576865914432
],
[
0.7741785688516597
],
[
0.7131845643455198
],
[
1.5423388549282961
],
[
1.0955637993469753
],
[
0.8176246489955148
],
[
0.8955835151494369
],
[
0.9864285164818019
],
[
1.273054019763209
],
[
1.4592700248334092
],
[
0.9485283138682999
],
[
0.5213277132123563
],
[
0.754194921743766
],
[
0.7762410269040364
],
[
0.9199257156331417
],
[
0.8677867524361825
],
[
0.9486297158814818
],
[
0.9539178343956689
],
[
1.1613433814669565
],
[
1.6012492868826615
],
[
1.1141049494783901
],
[
1.2519706657627314
],
[
0.6407093961488756
],
[
1.6174959624521714
],
[
1.8997086723885284
],
[
2.3510560650423056
],
[
1.135040725096897
],
[
0.6378037485420982
],
[
1.0594425116424402
],
[
1.3366673693868005
],
[
2.391288782215623
],
[
0.4873717831097945
],
[
1.1202426574001543
],
[
0.5161394194867208
],
[
0.7864454560194242
],
[
0.8402898994800136
],
[
0.7290280711994178
],
[
1.517728153558768
],
[
1.304467349433974
],
[
1.1040454773023745
],
[
1.244523137507654
],
[
1.4291217867443802
],
[
0.9358143831236371
],
[
0.6829870648274673
],
[
1.2700674209639178
],
[
1.2197382538804118
],
[
1.0809186764003358
],
[
1.6069382379490982
],
[
1.0065928025758388
],
[
1.4572200550362862
],
[
0.5844002838799912
],
[
0.9107073464722776
],
[
1.2060486336471188
],
[
0.7697029909322763
],
[
0.8102020493626289
],
[
0.46651291838395426
],
[
0.744494932475924
],
[
0.7433573383327283
],
[
1.8403137151137872
],
[
0.7253338828722238
],
[
1.6678024503832312
],
[
0.8357249033571201
],
[
1.347014449384418
],
[
0.42087982806401913
],
[
1.1267836833133575
],
[
1.2023108780204936
],
[
1.1750276580442702
],
[
1.3140891368942413
],
[
0.7153534912591686
],
[
1.4983644320263187
],
[
0.5949916129400511
],
[
0.8719052313979186
],
[
0.35642749328186485
],
[
0.8075035827940468
],
[
1.48559227830939
],
[
2.545915002799277
],
[
0.7372448618810721
],
[
1.0227232150950443
],
[
0.9088744535346988
],
[
0.536740733235159
],
[
0.4811564893770802
],
[
0.6049798965913182
],
[
1.929265306429484
],
[
0.6141922546016747
],
[
1.7771287460319556
],
[
0.9154460694975487
],
[
0.9725478382111192
],
[
0.7485533539268823
],
[
0.9760436844817302
],
[
0.49538734772032506
],
[
2.016029375519312
],
[
0.6124625384220285
],
[
0.7697314678524014
],
[
2.1368111381725052
],
[
1.070147609722607
],
[
0.9982205360779451
],
[
0.9642264357886028
],
[
1.4054862477089425
],
[
0.6915033155097036
],
[
1.4684249999489274
],
[
1.685620096583193
],
[
1.2568661758582116
],
[
0.8261660094890282
],
[
0.8744914004130515
],
[
0.988221215809963
],
[
0.9579072043969739
],
[
0.5365170152375961
],
[
1.1172395818838454
],
[
0.9921807963390077
],
[
1.7726322726148498
],
[
0.8227002134463417
],
[
1.732906859580981
],
[
0.46120795778956053
],
[
1.0083062722456697
],
[
1.1522727184715142
],
[
0.7242332075530438
],
[
0.5995517104294323
],
[
1.649813030498823
],
[
0.32761820308970224
],
[
0.7819232127762883
],
[
1.1423706225899495
],
[
0.7454462348966439
],
[
0.9108580839155338
],
[
0.557119770928968
],
[
0.32325795756718295
],
[
1.555675073798324
],
[
2.017433674490312
],
[
0.9180497325348306
],
[
1.0874112182909883
],
[
0.9072830201265424
],
[
1.998844495296378
],
[
1.692000374965083
],
[
1.0148861236641078
],
[
1.0704458547438058
],
[
0.3136636209976168
],
[
1.9084885541934624
],
[
0.8214807155893887
],
[
1.1720952887393794
],
[
1.1733969473457548
],
[
1.0771212139455888
],
[
1.7525783181505712
],
[
1.7444796165837864
],
[
0.8628089028967371
],
[
0.8603806830691565
],
[
1.1897481177416784
],
[
1.200580891456458
],
[
1.0612069915643225
],
[
1.4085620873700957
],
[
1.5997566746948668
],
[
0.948777428711316
],
[
0.9917106361931955
],
[
0.6885945879809706
],
[
0.3087455473815196
],
[
1.5081102057899203
],
[
0.9920778536049425
],
[
0.8673938626207445
],
[
0.7559531816375823
],
[
0.6325939737405939
],
[
1.3580771068838706
],
[
1.4675269305898393
],
[
1.2956148367096425
],
[
1.267885921883074
],
[
1.3634076621951046
],
[
1.2734808587689204
],
[
1.5853519314044147
],
[
1.746843018909172
],
[
1.0976599976433157
],
[
0.7404120832633836
],
[
1.4895582554594906
],
[
0.5919692258791425
],
[
1.2539724443056426
],
[
1.0554505935106446
],
[
1.1006683409024365
],
[
0.996451110296992
],
[
0.9852270557617457
],
[
0.6608436354498214
],
[
0.5091845350509098
],
[
1.2025803668735522
],
[
0.3253314842181734
],
[
1.0013650163166914
],
[
0.5761959204899202
],
[
0.48334896790998044
],
[
1.7842514787103119
],
[
0.6468811336269289
],
[
0.4393182057122346
],
[
1.5213588274185303
],
[
0.5143023866168277
],
[
0.796736988318047
],
[
0.6965728204889609
],
[
0.8536274028995665
],
[
2.091552634492911
],
[
1.0061628652672128
],
[
0.5426146557597299
],
[
0.6476907507276157
],
[
0.7710225811967615
],
[
0.7495847575483308
],
[
1.0641102265261637
],
[
0.6857703033417725
],
[
1.1922047542554963
],
[
0.645411133903165
],
[
0.3811254637198626
],
[
1.4495843794785463
],
[
0.9236967530883258
],
[
1.0502570476767792
],
[
0.7693633059430687
],
[
0.8983380475278447
],
[
0.7082587486912438
],
[
0.7855670388107772
],
[
0.8810815187756516
],
[
0.9243947295716943
],
[
0.7902529227562167
],
[
1.2187788479013677
],
[
0.9678715934621606
],
[
0.2793306516527633
],
[
0.9324743883957788
],
[
1.6956385865048158
],
[
0.845226010112146
],
[
1.186665108381175
],
[
1.310836597999475
],
[
1.6753886828206643
],
[
1.107885021235529
],
[
0.7004512729949354
],
[
0.9498823581341187
],
[
1.0549526870777026
],
[
1.1295714153417702
],
[
1.516257791062196
],
[
1.076655636026497
],
[
1.6248334527274246
],
[
1.2147331400590422
],
[
1.6256501595713866
],
[
1.0066789906579081
],
[
0.8574528572369227
],
[
1.2286591031824767
],
[
1.3746290699580084
],
[
0.589711616020343
],
[
1.3764933540816524
],
[
1.0038348459180875
],
[
0.6970655425583979
],
[
1.0244472528059358
],
[
0.9560365822523469
],
[
1.2381784737273678
],
[
1.7867064795671515
],
[
1.2213988996563916
],
[
0.962753763530258
],
[
0.8734800830939229
],
[
1.209792669188373
],
[
0.4520547011776859
],
[
1.1775616190262714
],
[
1.9171722311715989
],
[
2.3673373331288934
],
[
0.8515235351880208
],
[
0.6598288864902682
],
[
0.6767115586619353
],
[
1.1652676758961389
],
[
1.5217926720706663
],
[
0.6834309255788918
],
[
0.3226641604093396
],
[
1.069626295652022
],
[
1.1316632909816102
],
[
0.7284576408545342
],
[
1.3813191388484065
],
[
0.676912281542183
],
[
0.6712046156355865
],
[
0.9420205434510225
],
[
1.1230549300202837
],
[
1.3160897919080026
],
[
0.6928007596464003
],
[
1.7622895326515429
],
[
1.0777673756358161
],
[
0.8434199045418045
],
[
0.7163686603998877
],
[
0.5023886265402696
],
[
0.641773427427063
],
[
1.8433638889316222
],
[
0.9565447639759703
],
[
0.676055047878938
],
[
0.7459771568822887
],
[
1.075643815201599
],
[
1.6731102819442292
],
[
1.0557595412662502
],
[
0.9999756591523801
],
[
0.4741870926144417
],
[
1.0823726641948346
],
[
0.4891149088315905
],
[
0.6709822365931513
],
[
1.5814095661899574
],
[
0.4746341023004814
],
[
1.6146252140568973
],
[
0.8071836108682655
],
[
1.4408198212074863
],
[
1.315669402590332
],
[
1.585725816100648
],
[
0.4164933393301241
],
[
0.9042111896575614
],
[
0.7830271200076548
],
[
1.239507652429234
],
[
0.7597093853632298
],
[
0.49116907394950293
],
[
0.7879551392023758
],
[
1.0683586885613996
],
[
0.7810995766831048
],
[
1.3002351222754882
],
[
1.3651757551537873
],
[
0.9177503205480803
],
[
1.9531152471797624
],
[
0.7828905906544152
],
[
0.7381151310450862
],
[
1.2631696404194392
],
[
1.319779053640688
],
[
0.5741779281058611
],
[
1.6879220100193721
],
[
0.36608653460031904
],
[
1.6364689390423903
],
[
0.7167809951126274
],
[
1.0516247143907171
],
[
0.7203319651789797
],
[
2.0994051841653474
],
[
1.7038026367677575
],
[
1.0779033177745014
],
[
2.0126049951488
],
[
1.2072723364169489
],
[
1.014365933113332
],
[
1.124419143302412
],
[
1.085929818891605
],
[
1.1836833890575942
],
[
1.007568288389713
],
[
0.7174237124159282
],
[
0.9875831893915032
],
[
0.676870948446762
],
[
1.0469967001650784
],
[
1.3790042432258467
],
[
0.7284692286258894
],
[
1.340408204185009
],
[
1.2535487603906565
],
[
0.7975223706302258
],
[
0.8991719333131005
],
[
1.5796145815420983
],
[
1.3196329533474427
],
[
0.9415633795068677
],
[
0.597412922810964
],
[
1.5249664889188936
],
[
0.9451523014610357
],
[
1.5318147120712502
],
[
0.656964290100358
],
[
0.5381463627017379
],
[
1.1920151837120585
],
[
0.592326242322716
],
[
1.313928871677131
],
[
1.3542302584844972
],
[
1.7543463419618899
],
[
0.6810350259543043
],
[
0.3958552921752082
],
[
1.2255544252408401
],
[
0.5658740207905268
],
[
0.9485642587860639
],
[
0.8979813217764903
],
[
0.5782170908380353
],
[
1.3530665633584866
],
[
0.7234112769225501
],
[
0.6775223274576707
],
[
0.517423066924038
],
[
1.0916709458934233
],
[
1.3907559882922003
],
[
1.5011151199436656
],
[
1.2250101537412474
],
[
0.9408766933117927
],
[
0.789664702702328
],
[
1.0095048688867476
],
[
0.704738400185952
],
[
0.9190977614304082
],
[
1.6165193205666692
],
[
0.31176912124450595
],
[
1.0130890329717395
],
[
1.7321655903634001
],
[
1.0439818051832492
],
[
1.0305013365983104
],
[
0.4766509081035684
],
[
0.7335629391779309
],
[
1.4125029566304397
],
[
0.8858265282808917
],
[
0.6940311216171127
],
[
1.0646393066905764
],
[
1.1010207976201085
],
[
0.5521379188384342
],
[
0.7039333630171367
],
[
1.695332736400629
],
[
1.5563197066271797
],
[
1.311909541839214
],
[
0.14677962982475942
],
[
0.9696336960165707
],
[
0.676829879333094
],
[
1.6218711408252897
],
[
1.050768923535955
],
[
0.7858342128004803
],
[
1.1439936257615468
],
[
1.8263799331303499
],
[
1.107021597918422
],
[
2.121516074457176
],
[
1.51526008388019
],
[
0.9600080158182087
],
[
0.9664572472975348
],
[
0.7662931376670953
],
[
1.0363987083502433
],
[
0.5456426970106024
],
[
1.5235547352488494
],
[
0.6051126852358053
],
[
0.8131547273420034
],
[
1.4506091640459327
],
[
1.2207394327773051
],
[
0.6240705652463133
],
[
1.4700327467893715
],
[
1.2269843472372173
],
[
1.5424308978149082
],
[
1.037950131124497
],
[
0.5734134510381954
],
[
0.6757317579808714
],
[
1.119264081437852
],
[
1.0933199982277262
],
[
1.4610664331718564
],
[
0.9551881819114791
],
[
1.1113950009897122
],
[
1.5502714904030397
],
[
1.008780170900933
],
[
1.5127424440536843
],
[
0.9563754894390962
],
[
1.0690536353215139
],
[
0.47898218703522694
],
[
1.0121463789165452
],
[
1.076052188936277
]
],
"hovertemplate": "y=0
x1=%{x}
x2=%{y}
r=%{customdata[0]}
x1=%{x}
x2=%{y}
r=%{customdata[0]}