{
"cells": [
{
"cell_type": "raw",
"metadata": {},
"source": [
"---\n",
"title: \"Quadratic Programming\"\n",
"subtitle: Advanced Statistical Computing\n",
"author: Joong-Ho Won\n",
"date: today\n",
"date-format: \"MMMM YYYY\"\n",
"institute: Seoul National University\n",
"execute:\n",
" echo: true \n",
"format:\n",
" revealjs:\n",
" toc: false\n",
" theme: dark\n",
" code-fold: false \n",
" scrollable: true \n",
"jupyter: julia \n",
"---"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"Julia Version 1.9.3\n",
"Commit bed2cd540a1 (2023-08-24 14:43 UTC)\n",
"Build Info:\n",
" Official https://julialang.org/ release\n",
"Platform Info:\n",
" OS: macOS (x86_64-apple-darwin22.4.0)\n",
" CPU: 8 × Intel(R) Core(TM) i5-8279U CPU @ 2.40GHz\n",
" WORD_SIZE: 64\n",
" LIBM: libopenlibm\n",
" LLVM: libLLVM-14.0.6 (ORCJIT, skylake)\n",
" Threads: 2 on 8 virtual cores\n"
]
}
],
"source": [
"versioninfo()"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[32m\u001b[1m Activating\u001b[22m\u001b[39m project at `~/Dropbox/class/M1399.000200/2023/M1300_000200-2023fall/lectures/17-qp`\n"
]
},
{
"name": "stdout",
"output_type": "stream",
"text": [
"\u001b[32m\u001b[1mStatus\u001b[22m\u001b[39m `~/Dropbox/class/M1399.000200/2023/M1300_000200-2023fall/lectures/17-qp/Project.toml`\n",
" \u001b[90m[1e616198] \u001b[39mCOSMO v0.8.8\n",
" \u001b[90m[f65535da] \u001b[39mConvex v0.15.4\n",
" \u001b[90m[a93c6f00] \u001b[39mDataFrames v1.6.1\n",
" \u001b[90m[2e9cd046] \u001b[39mGurobi v1.2.0\n",
" \u001b[90m[b99e6be6] \u001b[39mHypatia v0.7.3\n",
" \u001b[90m[b6b21f68] \u001b[39mIpopt v1.5.1\n",
" \u001b[90m[b8f27783] \u001b[39mMathOptInterface v1.22.0\n",
" \u001b[90m[6405355b] \u001b[39mMosek v10.1.3\n",
" \u001b[90m[1ec41992] \u001b[39mMosekTools v0.15.1\n",
" \u001b[90m[91a5bcdd] \u001b[39mPlots v1.39.0\n",
" \u001b[90m[c946c3f1] \u001b[39mSCS v2.0.0\n"
]
}
],
"source": [
"using Pkg\n",
"Pkg.activate(pwd())\n",
"Pkg.instantiate()\n",
"Pkg.status()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Quadratic programming (QP)\n",
"\n",
"![Source: Boyd and Vandeberghe](./qp.png){width=500}\n",
"\n",
"---\n",
"\n",
"* A (linearly constrained) **quadratic program** (QP) has quadratic objective function and affine constraint functions\n",
"$$\n",
"\\begin{array}{ll}\n",
"\t&\\text{minimize}& (1/2) \\mathbf{x}^T \\mathbf{P} \\mathbf{x} + \\mathbf{q}^T \\mathbf{x} + r \\\\\n",
"\t&\\text{subject to}& \\mathbf{G} \\mathbf{x} \\leq \\mathbf{h} \\\\\n",
"\t& & \\mathbf{A} \\mathbf{x} = \\mathbf{b},\n",
"\\end{array}\n",
"$$\n",
"where we require $\\mathbf{P} \\in \\mathbb{S}_+^d$ (why?). Apparently LP is a special case of QP with $\\mathbf{P} = \\mathbf{0}_{n \\times n}$."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Example 1: least squares\n",
"\n",
"* The _least squares_ problem minimizes $\\|\\mathbf{y} - \\beta_0\\mathbf{1} - \\mathbf{X} \\beta\\|_2^2$, which obviously is an (unconstrained) QP.\n",
"$$\n",
" \\min_{\\beta} \\frac{1}{2}\\|\\mathbf{y} - \\beta_0\\mathbf{1} - \\mathbf{X} \\beta\\|_2^2 = \\frac{1}{2}\\beta^T\\mathbf{X}^T\\mathbf{X}\\beta - (\\mathbf{y - \\beta_0\\mathbf{1}})^T\\mathbf{X}\\beta + \\frac{1}{2}\\|\\mathbf{y} - \\beta_0\\mathbf{1}\\|_2^2.\n",
"$$\n",
" + If the data is centered ($\\mathbf{1}^T\\mathbf{X} = \\mathbf{0}$), then $\\beta_0 = \\frac{1}{n}\\mathbf{y}$.\n",
"\n",
"* Ridge regression.\n",
"$$\n",
" \\min_{\\beta} \\frac{1}{2}\\|\\mathbf{y} - \\beta_0\\mathbf{1} - \\mathbf{X} \\beta\\|_2^2 + \\frac{\\lambda}{2}\\|\\beta\\|_2^2\n",
"$$\n",
" + $\\mathbf{P} = \\mathbf{X}^T\\mathbf{X} + \\lambda \\mathbf{I} \\in \\mathbb{S}_{++}^d$\n",
" + Usually the intercept $\\beta_0$ is not penalized.\n",
" + If the data is centered ($\\mathbf{1}^T\\mathbf{X} = \\mathbf{0}$), then $\\beta_0 = \\frac{1}{n}\\mathbf{y}$. (Why?)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"\u001b[36m\u001b[1m[ \u001b[22m\u001b[39m\u001b[36m\u001b[1mInfo: \u001b[22m\u001b[39mPrecompiling IJuliaExt [2f4121a4-3b3a-5ce6-9c5e-1f2673ce168a]\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAGQCAIAAAD9V4nPAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nO3deVxU5eLH8WeGfd8REBTc19QEJc0lMzU1lwzsllfLUlOzbDFt9WaLLVdzKS2yrt00r7iluaC5kUuuCCguqCgqLiwi27DMMPP7Y353LgHiwjAHeD7vP+5rzjNnzvl2LvLlrKMyGAwCAABZqZUOAACAkihCoP5YtGjRhAkT0tLSlA4C1CUUIXAPtm7dansXPvroI6XiRUVFZWVlmWVpmzdvjoqKysnJMcvSgFrLWukAQF3i4eHRvXv3siP79u3TarXh4eH29vamwaCgIItHM7/58+dv27btkUcecXNzUzoLUIMoQuAedOnSZdeuXWVHfHx8MjMzly1b1rRpU6VSAagOihAws7Nnz+bm5rZt29bW1nb37t1JSUn29vbjxo27devW+fPnGzRoEBgYWHb+8+fP37p1q02bNg4ODmXHs7Kytm/fnpaW5uDgEB4e3qlTp7vPoNPptm/ffvr0aUdHx379+gUHB1ecx2AwHDly5OjRowUFBY0aNXrsscfc3d2Nb2m12sTExNzcXCFEUlKS8YVKpXrwwQdNMxw4cCAlJeXGjRs+Pj7dunVr2bLlPWwjoFYxAKgGb29vIcS5c+dMIwMGDBBC/P7776ba8PHxMRgMa9euFUK88cYb5Zbw5JNPCiHi4+NNI3q9ftasWeV68fHHH8/Ozq46zJAhQ4QQGzdubN++vemD1tbWs2fPLjfnmTNnwsLCyi7fzc1t2bJlxnevXbtW8XeFtbW18d3o6OhyB0tVKtXf/vY3jUZzX5sQUBgXywA14oUXXnByclq2bNn+/fsXLFhwT5995513Pvjgg+Dg4BUrVpw8eXLPnj2RkZFbtmx56qmnDHdx4+/48eO9vLz2799/6dKllStXent7v/322z///LNphqtXr/bs2TMuLm7KlCl79uw5derUjz/+aGdnN3r06O3btwshPD09f//9986dOwshli5d+vvvv//+++8xMTHGj9+4caNfv37R0dFHjx49derUr7/+GhYWtmLFihkzZtzTfyZQWyjdxEDddrs9wtatWxcXF5ed8y73CE+dOqVWqxs3bnzr1q2ysz3++ONCiK1bt1YRxrhHGBgYWHbn7I8//hBCBAUF6XQ648jo0aOFEP/85z/Lfvbw4cNqtTosLMw00q9fPyFEcnLyHTdCXl5e48aNHRwcCgoK7jgzUNuwRwjUiKlTp9ra2t7HB5cvX67X61955ZVyhx8nTZokhNi8efMdlzBp0qSyh1V79OjRpUuXy5cvHz58WAhRXFy8atUqFxeXV199teynQkNDu3TpcuTIkYyMjHvN7Ozs3KdPn8LCwhMnTtzrZwHFcbEMUCPatm17fx88duyYEOLgwYPljjQa++nixYt3XEKHDh3KjXTq1OnQoUNJSUnh4eGnTp0qLCz09/d/7733ys2Wk5NjMBhSU1N9fHyqXsXq1au///775OTkq1evlpSUmMYzMzPvGA+obShCoEYYD5neh1u3bgkhYmJirKysyr3l4eFRcbCiijXm6+srhMjPzzctPzMzMyoqquJnPTw8yhZbpT744IOPPvrI09Nz0KBBjRs3dnZ2FkJs3Lhx7969Op3ujvGA2oYiBCxErVYLIUpLS8uNG/vJxNgrq1atMp6iuw/p6enlRm7cuCGEcHFxMf1vhw4djEdK71VWVtZnn33m7+8fFxfn5+dnGk9ISLi/tIDiOEcIWIixNsq1lMFgOH36dNkR4/2C+/btu+8VGQ+ulnX06FEhRLt27YQQbdq0sbOzS0xMLFfAFdnY2Aghyu3kJSUlabXa3r17l21Bg8FQcaVAXUERAhYSEhKiUql27dpVVFRkGly+fPmlS5fKzjZmzBgrK6tFixaVGxdC6PV6jUZzxxUtXry4oKDANLlr166jR48GBweHhoYKIRwcHJ5++umSkpIPPvig4mfLtmNAQIAQ4sqVK2VnMB53LZftl19+KVfnQB1CEQIW4uvr26dPn2vXrg0ZMmTLli07duyYMWPG+PHjQ0JCys7WqlWrmTNnZmZmdu3adc6cObt3705MTNywYcPMmTObNGmye/fuO67I2tq6X79+27dvT05OXrp06ciRI4UQn3/+ufHYrPF1o0aNvvrqq2HDhq1evTohIWHPnj1Lly4dMWLEo48+alqO8Y77V155Zfbs2VFRUUuWLBFCNG/evGHDhvv27ZsyZUpcXNzx48dnz549bty4Jk2amG9TAZal8O0bQB13u/sIT58+XXHmCxculH0UmZOT04oVKyo+WcZgMHz77bcVr3np0KFDQkJCFWGM9xHGxMQY74U3srOzmzdvXrk509LSnnjiCZVKVXb5Li4uU6dONc2j1WonTJjg6OhofNf0ZJm9e/eWzWZtbf3FF1+89tprQoj169ff6wYEFKcy8A31QDVcuHBBp9MFBwcbz6gJIa5fv67RaIKCgkwjZZWUlOzYsePSpUseHh6PPfaYh4fHjRs3CgoKAgMDy913WFRUdODAgfPnzxsMBn9//3bt2jVu3LjqMMZVBwYGWltb792798yZM46Ojn369PH39690/tTU1AMHDmRnZ7u5uTVq1Khz585lv0PDJCMjIy8vT6VSmXZe8/Pz9+/ff/HiRXd39969e/v6+mZlZeXk5Pj5+ZmKE6grKEIAgNQ4RwgAkBpFCACQGkUIAJAaRQgAkBpFCACQGkUIAJAaRQgAkBpFCACQGkUIAJAaRQgAkFqtLsIVK1YkJycrncL8eKydeen1eqUj1CtsT/Pi37t5GR+Tbd5l1uoiXLduXf372mudTlf26+hQfWW/ew/Vx/Y0L41GQxeakU6nKykpMe8ya3URAgBQ0yhCAIDUKEIAgNQoQgCA1ChCAIDUKEJUrri4OD09XekUAFDjKEJU4j+r1ga179L+8WcfGTxCp9MpHQcAahBFiEpM+8enGVP/SJ+4OU4VHBMTo3QcAKhBFCEqUao3CGs7IUSprVNxcbHScQCgBlGEqMTrk1502/ie7bWk4LTYgQMHKh0HAGqQtdIBUBu9OWVig4fTFydb/fH2DmtrfkgA1Gf8jkPlfHx83G6U0oIA6j0OjQIApEYRAgCkRhECAKRGEQIApEYRAgCkRhECAKRGEQIApKZMEebm5u7evXvDhg2KrB0AABMFbpfesmXLsGHD/Pz8NBpNRkaG5QMAAGCiwB5hjx49bt26tXz5csuvGgCAchTYI3R2drb8SgEAqFStfpJkenr6unXrzp49a5x0cHCYPHmySqVSNlU16XQ6rVZb+5/hqdMJg0FotVqlg9yZVqutEznrCraneRm3Z13/xVV7aLVavV6vVt/t4UwrK6s7zlyrfx2Xlpbm5+dnZ2cbJ/V6vV6vr+s/T/r/UjrIHRgMKoNB6PUGpYPcWZ3YnnUI29O86scvrtrjXn+FWllZ3XGeWl2E/v7+ERERERERSgcxJ+OfJ3Z2dkoHuQMbG4NaXWpnV6t/QoxKSkpq//asQ9ie5qXT6ezs7ChCc1Gr1Xq93rw/otxHCACQmgJ/71+7du3VV1/NzMzMy8uLjIwMCgqaM2eO5WMAACAUKUIXFxfj0c6JEycKIVxdXS2fAQAAI2Vun6hnp/0AAHUX5wgBAFKjCAEAUqMIAQBSowgBAFKjCAEAUqMIAQBSowgBAFKjCAEAUqMIAQBSowgBAFKjCAEAUqMIAQBSowgBAFKjCAEAUqMIAQBSowgBS0hOTo6Ojr506ZLSQQCURxECNW79xk3dR4wdszYltN/ww0eOKB0HwF8o8A31gGy+XPRj5rP/Et7BRSHdvvrup19CQ5VOBOB/2CMEapy/r7cqM0UIYZ2e3NDPR+k4AP6CIgRq3PxPZ7Y5vECl1YSm/vr+tKlKxwHwFxQhUOMCAgIObN/k5Oj45/aNrq6uSscB8BcUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAagoUoV6v/+STTzp37tynT5+tW7daPgAAACbWll/lwoULV6xYsXz58nPnzkVGRh49erRZs2aWjwEAgFBkj/Cbb76ZNWtWhw4dRowYMXTo0O+//97yGQBU6vfft7/5/qzo1WsMBoPSWapy6NChV6a///2PS3U6ndJZqnLq1KkZ//hk7sJvioqKlM5SlStXrsz44KOPPv9nTk6O0lmqkpWVNeuzL2d+8vn169fNuFhLF2FRUdHZs2dDQ0ONk126dElMTLRwBgCV2rBp88gZ/4xS9R739fo5CxcrHee24uPjB46durC4y2trEqe89Z7ScW7rypUrjzw56uvC0Pf25Q4b9YLScW4rPz//oX5Dv0hv9tEpp54Dn1Q6zm0ZDIZujz0x+2KDL6+HhD/2RElJibmWbOlDoxkZGUIId3d346SHh0d6evrtZk5KSoqNjX3rrbeMk87Ozvv27VOrKy/vny9Y2aoMI4P15o5sZlvTxPGbhjfb5ysd5A5Ki9Vqg1V+fq3+M1YIcb1QNf2I+qcetX17FpUKe7Vtfn6tzvnz6g3Z/d8TTbrmNuq4fM3Yl8aOVjpR5VZv2JT18BTRbkBB2wGb5j9Ua7fqjh07bnWIFB2fKBJPHJsTXmtzHjx4UBMcbgh9SivE9cTVaWlpbm5uSoeqRFpa2i17n9KHRgshClJ2xMXFtWvX7o6fsre3t7a+Q9NZugiN21ej0bi6ugoh8vPzTaVYUcuWLSdOnDhw4EDjpL29vfFTlbpaUmqnVjk71/brYDNKdZeLS52d7ZQOcgcDnUWPRsLZprbnLNEZTuZpnZ1tlQ5yB85CnBspnG1rdc5unTts3rZRExxqE/9rWKcOzs7OSieqXHjnjq47V+Z2HKI6tb1ls2a1NmfHjh2dF7xV3GOCuH7K282l1uZs166dVeo7Ij9LaHJsCrMDAgJUKpXSoSrRpEkTVdZFkX1FqG3Uacdbt25trk1q6SJ0dXX19PRMTk728/MTQiQnJwcHB99uZmtr6wYNGjRp0sRy+fBfKiFcbZQOUb+41eoSFEKIKS+NO3/xg83f9AoPC5378edKx7mtwYMGvX781M9Rj7Vq0fyH7+YrHee2OnTo8OnLz365cKC/v9+Py5YoHee2GjRo8P3nH7zz8dOOjo7fLl9SO1tQCGFjY7PmX99Onj5ep9PNXfiFGXdbVZY/Jf7KK69cu3YtOjo6IyOjU6dOy5Yte+SRRyqdMzIyMiIiIiIi4m4W+/7RUju16r1OtX2P8NuTusPppT/0ru17WnXFyVuGEdu0pyJrfcnUHXl5eS4uLkqnqD8KCgocHR1rbbvUOVqtVq/X29mZ81eoArUxc+bM9PT0gICAli1b/v3vf79dCwIAYAEK3Efo5eUVGxubkZHh5OTk6Oho+QAAAJgoUIRGPj4+Sq0aAACT2n5GDQCAGkURAgCkRhECAKRGEQIApEYRAgCkRhECAKRGEQIApEYRAgCkptgN9dLKz8/XaAxC8KxRAKgVKEKL+sdnc+YmlGgD2g1dtvbX5T/yHF4AUByHRi2npKRk0dLleX3eKGo9YP/VkoSEBKUTAQAoQgsyGAxC/HcXUG1VWlqqaBwAgBAcGrUkOzu7sU8/+eM3/YWD64MBTg8++KDSiQAAFKFlffaPdyc+PyovL69du3ZKZwEACEERWl7Dhg21Wq3SKQAA/49zhAAAqVGEAACpUYQAAKlRhAAAqVGEAACpUYQAAKlRhAAAqVGEAACpUYQAAKlRhAAAqVGEAACpUYQAAKlRhAAAqVGEAACpUYQAAKlRhAAAqVGEAACpUYQAAKlRhAAAqVGEAACpUYQAAKlZm16tXr16+vTp06ZNe+mll7p27ZqZmVnpB+zt7Zs0afLGG2/07t3bQhkBAKgx/yvCgICAvn37BgcHCyF69uyZm5tb6Qd0Ot2xY8cGDRp0+vTpoKAgy6QEAKCG/K8Iu3Xr1q1bN+PrL7/8sorPlJSU+Pn57dmz55lnnqnZdAAA1LD7OUdoa2s7fvz49957z+xpAACwsPu8WKZHjx62trbmjQIAgOVx1SgAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBq91mEbdu25T5CAEA9YF3Fezk5OSkpKVlZWWUH+/btK4QIDg42PowNAIA6rfIizMzMHDNmzJYtWwwGQ7m3Ko4AAFB3VV6EEyZMOHz48A8//LB69Wo/P78hQ4Zs2rRp9erVixYtsnA+AABqVCXnCA0GQ0xMzJw5c55//nlfX18/P7+hQ4dGRUW98847s2fPtnxEAABqTiVFmJ6ertFowsLChBB2dnZ5eXnG8eeffz4xMfHcuXMWDQgAQE2qpAjd3NxUKpWx/wICAs6cOWMcLykpEULk5+dbMh8AADWqkiK0t7dv3br10aNHhRD9+/ffuXPn/Pnz9+7dO2HCBBcXlxYtWlg8JAAANaXyi2XefvvtmzdvCiG6du36wgsvTJ06VQjh4ODw3XffOTo6WjQgAAA1qfIiHDVqlOn1t99++/7771+4cKF169ZeXl6WCgYAgCVUXoS//fZbt27dTLXXsGHDhg0bZmVlrVq1KiIiwoLx7tbn876Ze1yl0mrdH3Z5efxYpeMAAOqMyh+xNm7cONM1MibJycmRkZE1H+meZWRk/HPJck34CwUPv/ThV4tMl7kCAHBH9/CsUY1GUztPEN66dUvtGShUaqFWq938cnNzlU4EAKgz/nJoNDk5OSEhQQhRVFS0c+fOtLQ001sajWbp0qVmvGS0uLi4sLDQ3d29+otq1qxZE9uC4nVvqvW61l62DRs2rP4yAQCS+EsRbtq06fXXXze+fv/998vN2qBBg6VLl1Z/lQcPHhw/fvzJkyfd3d0zMjKqv0CVSrV364YdO3ao1eo+ffpUf4EAAHn8pQjHjh07dOhQIUTXrl0XL1784IMPmt5ycnJq0KCBWVbp7++/YMECjUYzevRosyxQCGFlZdWvXz9zLQ0AII+/FKGbm5ubm5sQYseOHU2aNHF2dq6JVTZq1KhRo0Z79+6tiYUDAHBPKr994oEHHjAYDAcPHkxKSioqKpo0aZIQIi0tzd7e3pK3Emo0muPHj3t4eBgn3d3dQ0NDLbZ2AIAMKi/CmzdvDhs2bM+ePUKIhg0bGovw008/PXny5K5du+640ISEhC+//LLi+FdffeXj43P34dLS0i5evGjad7S3t//Pf/6jVt/Dla61kE6n02q1paWlSgepJzQalV5vxSNwzaigoEClUimdov7QaDR6vZ5Nai5arVav12u12ruc397e3tq6qq+gF1V8H2FKSkpMTIwQ4oUXXjAOPvPMM7169crLy3Nxcal6ob6+voMHD644fq93XzRv3jwiIqJ23sJ/34xF6ODgoHSQesJRZ1CrtTV0GF9OBoOB7WlGKpXK0dGRIjQXYxHa2dmZcZmVFKFGo1m/fv1//vOf/v37x8bGmsZbtmxZWlp6+fLlNm3aVL1Qf3//p59+2owpAQCoIZUUYXZ2tlarrdh2xr9oCgsLq7nKnJyclStXnjt3rqioKCoqysPDo57t8wEA6pBKzrd5e3vb29vHx8eXG4+NjVWr1U2aNKnmKrVabUpKilqtnjx5ckpKypUrV6q5QAAA7lsle4R2dnZPPfXU9OnTW7RoYTqufeDAgddee23QoEGmazjvm7e392effVbNhQAAYBaVXywzb968xx57rHPnzj4+Pjk5OY0bN7506VKzZs0WL15s4XwAANSoyovQy8vrzz//XLZs2datWzMzM11cXKZOnTpu3DiuJQMA1DO3vbvCzs7uhRdeMN07AQBAvVTVbYZXrlxJSEi4evWqr69vu3btmjZtarFYAABYRuVFWFhY+NJLLy1btkyv15sGhwwZsnTp0upfLAMAQO1R+ePKJkyYsGLFimnTph06dCg1NTUuLu7jjz/etWsXN/wBAOqZSvYI8/PzV6xYMXfu3ClTphhHGjVq1KlTp1atWj311FMXLlwICQmxbEgAAGpKJXuEBQUFOp2u4tf79e/fXwiRnZ1tiVwAAFhEJUXo4+PTuHHjuLi4cuNHjx51dnZu2bKlRYIBAGAJlRwaVavV33///dixY4uKikaMGOHq6qrRaDZv3jx9+vTvvvvOycnJ8ikBAKgh/9sj/OWXXzz/a+TIkRkZGWPHjnVzc3NycnJycoqIiLh8+fLLL7+sYFYAAMzuf3uELVq0GD9+vIJRAACwvP8VYWhoaGhoqIJRAACwvMrvIwQAQBIUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBqFCEAQGoUIQBAahQhAEBq1pZfZUZGxoYNGxITE52cnIYPHx4WFmb5DAAAGCmwR/jhhx9u27YtJCTExsamT58+69ats3wGAACMFNgjnDt3rq2trfG1SqVasmTJ8OHDLR8DAAChyB6hqQWFEHl5eW5ubpbPAACAkcpgMJh9oQaDQavVVhy3sbFRqVSmyePHj3fv3v2PP/7o2LFjpcvp2rWrECIwMNA46erqunDhQrW6bl/go9PptFqtg4OD0kHqidO5qr/tVh0bolc6SP2Rn5/v7OysdIr6o6CgwNHRseyvPlSHVqvV6/V2dnZ3Ob+tra219R2OfdbIodGEhITu3btXHN+1a1eXLl2Mry9cuDB48OAFCxbcrgWFEO7u7m3atAkPDzdO2tvb14P+sLKyUqvVd///IqpmY2NQqUrZnmZUUlLC9jQjnU5nZ2dHEZqLWq2+pyK8m32nGinCjh07FhQUVDHDpUuXHn300RkzZjz33HNVzObm5tatW7eIiAgz51OUwWDQ6/VWVlZKB6knrKwMKhXb05ysrKzYnmZk3J4Uobno9XqVSmXeH1EFDjOmpaX16dNn4sSJEydOtPzaAQAoS4EifPvtt9PS0lauXBkaGhoaGhoZGWn5DAAAGClw+8SHH3746quvmibrwWk/AEDdpUARhoSEhISEWH69AABUVLdvRQAAoJooQgCA1ChCAIDUKEIAgNQoQgCA1ChCAIDUKEIAgNQoQgCA1ChCAIDUKEIAgNQoQgCA1ChCAIDUKEIAgNQoQgCA1ChCAIDUKEIAgNQoQgCA1ChCAIDUKEIAgNQoQgCA1ChCAIDUKEIAgNQoQgCA1ChCAIDUKEIAgNQoQgCA1ChCAIDUKEIAgNQoQgCA1ChCAIDUKEIAgNQoQgCA1ChCAIDUKEIAgNQoQgCA1ChCAIDUKEIAgNQoQgCA1ChCAIDUKEIAgNQoQgCA1ChCAIDUKEIAgNQoQgCA1ChCAIDUKEIAgNQoQgCA1ChCAIDUKEIAgNQoQgCA1ChCAIDUKEIAgNQoQgCA1ChCAIDUKEIAgNQoQgCA1ChCAIDUKEIAgNQoQgCA1ChCAIDUKEIAgNQUKMJFixY1adLEzs4uICDg9ddf12q1ls8AAICRAkXYtWvX7du3azSa2NjYLVu2fP3115bPAACAkbXlV9m5c2fji+bNm/fu3TslJcXyGQAAMFKgCIUQqamphw4dunz58o4dO9asWaNIBgAARA0V4fHjx/fu3VtuUK1WT5gwwfg6NTU1Ojo6OTm5WbNm3t7et1tOYmLiqlWrTJMNGjQ4c+aMWl23L/DR6XRarVan0ykdpJ4oKFDp9dZ5eXlKB6k/8vPzlY5Qr2g0mtLSUpVKpXSQekKr1er1+pKSkruc397e3sbGpup5aqQIb968efLkyXKDVlZWptc9e/bs2bOnEGLChAlvvPHGL7/8UulyHnjggY8++igiIqImQirFWIQODg5KB6knnEoNarXWxcVF6SD1CtvTjNRqtaOjI0VoLsYitLOzM+Mya6QIe/Xq1atXr7uZMyws7Mcff6yJDJCBVqtdtmLtNW333bEpvXv1VDoOgDpJgcOMy5YtS0lJycvLO3To0Pz58wcMGGD5DKgfJk9796t9aTlWLiOmvH/o0CGl4wCokxQowri4uH79+jVu3Hjs2LEjR4585513LJ8B9cO2HbuLek0Wdo43e0xZH7Nd6TgA6iQFrhqdO3fu3LlzLb9e1D/t2ra+fHStvm1/1+PrHnr8GaXjAKiT6vYVmJDcT9/MHVEc2+rnYe+NeGjw4EFKxwFQJylzHyFgFl5eXtH/+jYvL4+rHAHcN/YILe3KlSuJiYlKp6g/iouLd+7cqXSKemXr1q0Gg0HpFPXHgQMHbt68qXSK+uPChQunTp0y7zIpQkvbunVrVFSU0inqj/Pnz8+YMUPpFPXK+PHjs7OzlU5Rf3z11Vf79u1TOkX9sXbt2mXLlpl3mRShpfG3NiAb/tWbUU1sTIoQACA1ihAAIDVVbd5nDw0N1Wq1vr6+Sgcxp7S0tNzc3NatWysdpJ4oKChITEx86KGHlA5Sf8TGxnbv3t3amkvKzSM+Pj4wMLCKbxfAPUlNTS0pKWnevPldzj98+PBJkyZVPU+tLsLY2Ni8vDx7e3ulg5hTfn5+YWGhj4+P0kHqCYPBkJqaGhwcrHSQ+uPChQshISFKp6g/rl696uXlZd6HRMssJyentLTU09PzLucPCbTarZAAAAvVSURBVAlp2rRp1fPU6iIEAKCmcY4QACA1ihAAIDWKEAAgNYoQACA1rpC2qBMnTiQkJDg5OfXq1cvDw0PpOPVEZmZmfHx8586d2aTVl5ubu3379vz8/Pbt23fq1EnpOHXe6dOnDx8+7Orq2qdPHx4Nf380Gs3x48fVanVYWFjZ8WPHjsXHx7du3To8PLyaq2CP0HLeeuutIUOGbN68OSoqqlmzZkeOHFE6UT3x3HPPPf744wkJCUoHqfPi4+Nbtmz53Xff7dq1a/To0UrHqfPmzZvXq1evw4cP//vf/27RosWFCxeUTlT3REVFeXh4DBw48NVXXy07Pm/evMGDBx88ePCZZ5559913q7saAywlJSWltLTU+Prll18eNmyYsnnqh59++un555/39fXdtWuX0lnqttLS0latWi1cuFDpIPVHUFDQunXrjK8HDhw4a9YsZfPURTdu3MjNzf3hhx8eeugh02B+fr6bm9uhQ4cMBsOFCxccHByuXbtWnbWwR2g5ISEhavX/b3B/f/+SkhJl89QD169f/+STT7744gulg9QHx44du3bt2ujRo//444/4+HgDdxhXm5eXl0ajMb4uKCjw8vJSNk9d5OvrW/GQ8t69e93d3Y1HSoODg9u3b79169bqrIVzhArIzMxctGjRwoULlQ5S57388suzZs3i4VVmcf78eVdX1969ezdv3jwhIaFFixbr1q2zsrJSOlcd9u9///u5555btWrVtWvXOnTo8OKLLyqdqJ5IS0sLDAw0TTZs2DAtLa06C2SP0NIKCgqGDx8+bNiw4cOHK52lbouOjtZoNCNHjlQ6SD1RWFh4+fLlefPmrVy58tixY4mJiWvWrFE6VN32008/OTo6RkZGPv3001u3bj127JjSieqJ0tJSlUplmrS2ttbpdNVZIHuEFlVYWDh06NBmzZotWLBA6Sx13pdffunn5zdhwgQhRF5e3pw5czQazcCBA5XOVVcFBARYWVk9/PDDQggHB4fw8PATJ05ERkYqnauuSk9Pnzt3bmpqalBQkBDi1q1bn3/++dq1a5XOVR/4+/unp6ebJq9fv96vX7/qLJA9QsspKSmJiIjw8vJasmSJ6WQh7tunn346evTovn379u3b19bWtnPnzjwqujq6dOni4OBgurLx7Nmzxt/guD/W1tYGg8F0KUBxcTHHmc0lPDz80qVLxp/VW7duHTlypHfv3tVZIA/dtpzXXntt8eLFo0aNMv578PPz+/DDD5UOVU80aNBg5cqV1fzHgJkzZ65fv37cuHEHDx48cOBAXFycs7Oz0qHqsJEjRyYnJ7/44ovXr1+fP3/+6tWrq7njIqETJ04sXLjwzJkzp0+fHjp0aMeOHSdOnCiEmDp16u7du59//vno6Ojg4ODly5dXZy0UoeXs3Lnz3Llzpkl3d3eOO5nLzz///OijjwYEBCgdpM7bsGHDgQMHGjVq9Oyzz3IDeDXp9fp169YZn6ExePDgtm3bKp2o7rl8+fKWLVtMk8HBwcY/JgwGQ3R09LFjx1q1ajVq1Khqfn0mRQgAkBpnqgAAUqMIAQBSowgBAFKjCAEAUqMIAQBSowgBAFKjCIG66urVq1FRUaZnTW3ZsmXnzp3KRgLqIooQqKuSk5MnTJhgeijaxx9/PG/ePLMs+V//+tfhw4fNsiig9qMIgXpi/Pjxo0aNMsuiXnnllfXr15tlUUDtx7dPAOah0Whyc3N9fHwqPls5MzNTpVLd7ntZMzIyrKysPD09b7fkjIwMe3t70wPPMjMz7e3tKz4FdMyYMZV+PDc3V61WV/rU0MLCwsLCwipWXenSNBpNgwYNyn4PDlCnsUcIVNfhw4d79uzp4uLi7+/v4OAwaNAg01urV69u2rSpj4+Pt7d3y5YtN27cWPaDK1asCAkJ8fX19fLyatOmTUxMjOmtadOmtWvXbtu2bU2bNvX19X355ZeFEElJSaGhoT4+Pq6urv37979+/XrZpQ0dOvS5554zvt66daunp+f27dv79+/v7u7u5ubWpUuXlJQU08wzZ84MDg52dHT08vLy9vZ+8803tVqtEKK4uNjT07OgoGDOnDmenp6enp6mR8Pv2LHjwQcfdHNz8/f3DwgI+P777827GQGlUIRAtSQkJPTu3Ts7O3vt2rVJSUkbN240fXd2TExMZGRkq1at9u3b98cffwQGBg4bNiw2Ntb47vr165999tkOHTrs378/NjbW29v7iSee+PPPP43vFhYWXrx4cdy4cdOnT9+/f/9LL72Um5s7YMCAmzdvbtq0KSEhoX379lOmTCmbJDMz8+bNm8bXWq02Ozt77NixDz/88IEDB5YvX37+/Pmy82dnZ3/88cdxcXEJCQnTpk1bsGDBxx9/LISwsbGJjo62t7d/8skno6Ojo6Ojn332WSHE7t27BwwYEBISsmfPnvj4+DFjxowfP37FihU1vHUBizAAqIbBgwe7u7tnZmZWfKt79+5BQUHFxcXGyYKCAl9f30cffdQ42blz56ZNm2q1WuNkbm6up6fnwIEDjZOTJ08WQvz222+mpX399ddCiEOHDplGjLueBw4cME5269btiSeeML7+7bffhBAzZswwzTxr1iyVSlVYWFjpf8XkyZObNWtmmnR2dn733XfLzhAWFtalSxedTmcaGTZsWKdOnW6/YYA6g3OEwP3T6/U7duwYOXJkxfN/BoMhPj7+xRdftLW1NY44OjoOGTLk559/1uv1BoPh+PHjU6dONX19jIuLy8CBA3/99VfTEmxsbB5//HHT5LFjx4KCgsLCwkwjI0aM2LRpUxXxyn68bdu2BoPhypUrzZo1M8aLiYk5ePDgjRs39Hp9UlJSamqqTqer9Otsbt68eeTIkb///e+7du0yDfr5+W3ZsqW0tJTvm0VdRxEC96+goKCwsNB0LLSsrKysgoICf3//soMBAQHFxcV5eXn5+fklJSUV383Pzy8qKrK3txdClLvuJi0tzc/Pr9z8Vcfz8PAwvTb2sfEL00tKSvr27XvkyJEBAwYEBgY6Ojo6OTlptVqNRuPq6lpxOenp6QaDYc2aNcYdTRNHR8fs7Gxvb++qYwC1HEUI3D8nJydbW9sbN25UfMvR0VGlUmVmZpYdzMjIsLa2dnJyMhgMQoiK79rb29vZ2Rkn1eq/nMJv0KDBmTNnyo6YbqW/V9u2bduzZ09sbGzPnj2NI//4xz+2bdt2u/mN7fjuu+++/fbb97dGoDbjYhng/qnV6h49emzatEmj0ZR7y9HR8YEHHti8ebPhv999rdPptmzZEhYWZm1t7e7u3qpVq02bNpneLS4u3rZtW3h4+O1uS2jXrl1qaurp06dNI2WvMr0nxnvwO3fubJw0GAxlvwRcCOHs7FxYWGiaDAgIaN68+Zo1a0pLS+9vjUBtRhEC1TJz5swbN24MHTo0Pj5eo9GcPXt29uzZxremTZt28uTJSZMmpaWlXb58eezYsRcvXnzrrbdM78bHx0+dOvXatWupqaljxoxJS0ubNm3a7VY0evRoNze30aNHJyUl5eXlff311+Vuxrh7HTt2FEJ8+OGH+fn5V69enTx58smTJ8vO0LZt299++y0mJubo0aNpaWlCiM8++ywuLu6ZZ545ceJEYWFhamrqypUrZ86ceX8BgNpFsct0gPpiw4YNjRo1Mv2beuCBB0xvzZ0713Qnu5ub2+LFi8t+8PPPP3dycjK+6+HhsWTJEtNbkydPDgwMLLei2NhY03nBli1bfvvtt6LKq0YTExNNnzWOJCUlGSdnzJhhOgHZv39/Y3nn5OQY301ISHj44YeN2aZPn24cXLlyZdn/TA8Pj5kzZ1Z74wHKUxn+e2QGwH3T6/VnzpzJy8sLCAgod+1MYWHhqVOnVCpVmzZtTOf/yr578uRJKyurNm3amK4vrYJWq01MTHRwcGjVqlW5k4h6vV5UOLNYhaysrJSUFD8/v6CgoLv8iMFgSE5OzsnJ8fb2btSoUaWXmAJ1DkUIAJAa5wgBAFKjCAEAUqMIAQBSowgBAFKjCAEAUqMIAQBS+z/+6s8IdznVcgAAAABJRU5ErkJggg==",
"image/svg+xml": [
"\n",
"\n"
],
"text/html": [
"\n",
"\n"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"using Statistics\n",
"using Random, LinearAlgebra, SparseArrays\n",
"\n",
"Random.seed!(123) # seed\n",
"\n",
"n, p = 100, 10\n",
"# Design matrix\n",
"X0 = randn(n, p)\n",
"X = [ones(n, 1) (X0 .- mean(X0 ,dims=1)) ./ std(X0, dims=1)] # design matrix is standardized and includes intercept\n",
"# True regression coefficients (first 5 are non-zero)\n",
"β = [1.0; randn(5); zeros(p - 5)]\n",
"# Responses\n",
"y = X * β + randn(n)\n",
"\n",
"# plot the true β\n",
"using Plots\n",
"\n",
"plt = plot(1:p, β[2:end], line=:stem, marker=2, legend=:none)\n",
"xlabel!(plt, \"coordinate\")\n",
"ylabel!(plt, \"beta_j\")\n",
"title!(plt, \"True beta\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" 1.251917 seconds (807.74 k allocations: 66.132 MiB, 1.98% gc time, 71.62% compilation time)\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAGQCAIAAAD9V4nPAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdeZxcVZ03/u855+61V+970kkgIZCEkLCvAZHNcXQEFwRxQB/xGRVEfX4/XAYHncHHcXj4DW7IuOEjjI4biCgDIrsKBMhGQvbuLL1Vd9d613PO74/bXal0ukMC3V2d7u/71a9+3bp169apXu6nzlpESgkIIYTQXEWrXQCEEEKomjAIEZpxduzYcf311997771veOTHPvaxz3/+89NQJIRmMQxChKbJZz7zGVIhHo8vWLDgQx/60CuvvDLmyN7e3u9///t/+tOf3vCc99xzzwMPPDAlxUVozlCqXQCE5pYFCxbMnz8fABzH2bhx449//OP777//17/+9WWXXVY+JpFInH/++SeccEL1ionQHIJBiNC0uvbaa7/0pS+F28Vi8cMf/vDPf/7zT3ziE5deeikhJNx/wgknPPHEE9UrI0JzCwYhQlUTiUTuvPPOn//85zt27Ni1a1dYUwSAYrG4efPmmpqaefPmVR4/MDDwyCOP9PX1zZs379JLL7Usa9zT5vP5Rx55pLu7u6Gh4dJLL62pqdm0aZNt2yeffDKlB/WG7Nmz54knnujp6UmlUuedd96iRYum5oUiNKNhECJUTU1NTaqq+r6fz+fLO9etW3fmmWdeffXVP/nJT8o7f/azn91www3lw5qamn79618fesInnnjife97X19fX3gzEon88Ic/vO222zZu3GjbtmEY4X7XdT/1qU/de++9nPNwDyHk+uuv/+Y3v6lp2lS8UoRmLBwsg1A1rV+/3vd9xlhHR8dhDnvhhReuvvpqzvndd9+9a9eul19+ec2aNe9617vGHLZr1653vvOdg4ODX/3qV7dt2/b6669/7nOfu+666/bv3195mJTy/e9//3e/+90zzjjjoYce2rx586OPPnr22Wffe++9N9100+S/SIRmOIkQmha33HILAHz5y18Ob3LOX3zxxZNPPhkArr766sojn3vuuTE7L774YgD47ne/W94jhDj//PMBoLOzs7zzox/9KADceuutlWf77Gc/G/6z27Yd7nnwwQcB4MwzzwyCoHyY67qLFy9mjG3fvn3yXjRCxwCsESI0re644450Op1Op03TXLVq1fr166+99tp77rnnMA/J5/OPP/54TU3Nhz70ofJOQsinP/3pMUc+9NBDhJAxtbqbb755zGH33XcfANx6662MsfJOTdNuuOEGzvkf/vCHN/fSEDpGYR8hQtOqubl5/vz5nPN9+/Zt2bLFsqyLLrpoomEvoc2bN3POlyxZout65f4VK1ZU3szlcvv376+vr6+rq6vc39TUlE6nBwcHy3tefvllAHj44YeffvrpyiO3bNkCALt27XqTrw2hYxMGIULTqnL6xHPPPXf55Zdfd911xx9//KmnnjrRQwqFAgDU19eP2T9mj23bAJBMJg89QyqVqgzC4eFhABh3Jn4qlTrCF4LQrIFBiFDVnHnmmXffffcHP/jBj3zkIy+++KKqquMeFo1GAaC3t3fM/jF7EokEpXTfvn1jDpNSjtkZj8cHBgbWrl07ZnoGQnMT9hEiVE0f+MAHTj311HXr1v3oRz+a6JglS5Ywxl577TXHcSr3r127tvKmYRhLly4tFAovvfRS5f6//OUvYWWxLByhEw7JQQhhECJUTYSQsKX09ttv9zxv3GOi0ejb3/72wcHB73//++WdQohvfOMbY4689tprAeALX/hCeXZgEARf+MIXxhx23XXXAcBXv/rVysmLIc/zJioGQrMVBiFCVXb55ZefeuqpXV1dh6kUfuUrX9E07ZZbbvnXf/3X11577fnnn7/qqqt2795dXpUt9IlPfGLlypW///3vTzvttDvuuONf/uVfVq9e3d3dHTaBlg++4oorrr766k2bNq1ateo73/nO888/v3bt2l/84he33HJLa2trd3f3VL5chGaeas/fQGiuGDOPsNJDDz0EAO3t7a7ryvHmEUopH3zwwcqRLG1tbS+//DIhpHIeoZQyk8lcc8014fhSxtgVV1zR1dXV2tqqKErlYUEQfOlLX4pEIpVXA0rpWWed1dfXNwWvHqGZi0j8hHqEpkUmk8lms6lUatyRmTt27ACA1tZWTdMcx+nu7o7H4w0NDZXHZLPZxx57bGBgoL29fc2aNbqu79ixQ1XVtra2MWdzHKe3t7eurs6yrGKxGI/H58+fv23btjGH5fP5Z599dvfu3bquNzU1rVixYswzIjQXYBAiNMvdddddN91000c+8pHDT9tHaM7CIERoVrnxxhsXLlx4zjnnNDY2dnd3P/zww1//+tdVVX355ZePP/74apcOoZkI5xEiNKvs2bPnO9/5TuWetra2H/zgB5iCCE0Ea4QIzSpSynXr1m3cuHFgYEDTtCVLlpxxxhn4yUoIHQYGIUIIoTkN5xEihBCa0zAIEUIIzWkYhAghhOY0DEKEEEJzGgYhQgihOQ2DECGE0Jw2o4Pw/vvvf/3114/w4HDt1CktD5qIEKLaRZij8CdfLfiTr5apuNTP6CD81a9+9eqrrx7hwZ7n+b4/peVBEymVStUuwhxVLBarXYQ5qlQq4TvvqgiCYNI/MnNGByFCCCE01TAIEUIIzWkYhAghhOY0DEKEEEJzGgYhQgihOQ2DECGE0JxWhQ/m9X1/w4YNL730UqlU+uQnPzn9BUAIIYTKqhCEDz/88M0339zS0rJly5ZJDEKn3yOEaC34AaQIIYSOQhWaRv/2b/92586dd9xxx+Se1h0Ktnx/b1Dkk3tahBBCs9vs6SNMHGell0U339ctBS73gBBC6EhVoWn0yHV1dX3729/+/e9/H940DONrX/sapeOH9+aM91Rr/Pit/Zv/7/6e5elpLCYC1+W67lS7FHOR6wr8yVeF63JNcwgh1S7InCM4P7vWX1xzpI1/mqYpyhsk3YwOwng8Pm/evJUrV4Y3E4mEaZoTHdzP6fo82bm8+fI/7d5gRvY1RKarmAh8X6rOjP5bmq3wJ18tvi8VRcEgnH5C0AUJtlxXj/D4iepOlWb0v1AymXzb29525ZVXHsnB5zcFFzQTTdPzi9oSP+havqbTSOPAmWlSKDjR6ITvUdDUyeftWAx/8lVQKDiRiIFBOP183xeCMMYm8Zyzp4+wLDbPajm/dst93ZJjZyFCCKE3UIUg7O7uXrVq1Q033DA8PLxq1aqrrrpq0p+i9fxaPaHufLBn0s+MEEJolqlC02h9ff13v/vd8s3DdPu9eQQWvb/1lTu39704XL8qOfnnRwghNFtUIQh1XT/llFOm+lmYThd/qG3Dt3dF20yrQZ/qp0MIIXSMmoV9hGWRJqPznY2v/aCLO6LaZUEIITRDzeYgBIC6U5KJTmvrz/ZWuyAIIYRmqFkehADQ+e5mZ8Db/8xgtQuCEEJoJpr9QUgVsvhDbV3/3ZffXap2WRBCCM04sz8IAcCo0Y57X8um/+ga3lqodlkQQgjNLHMiCAEgtSR2wt+3v/6TPT1/Hqp2WRBCCM0gcyUIASA2zzrx4/P3PNbf9Ye+apcFIYTQTDGHghAArAZ92ac6Bzfmt/7nXvy0JoQQQjDXghAAtJhy0v+c7+WCjd/bzV2cX4gQQnPdnAtCAGA6PeH6diOtrf/2Tr8QVLs4CCGEqmkuBiEAEEoWXtlcuyz+6l077D632sVBCCFUNXM0CEOta+raLq5b982duZ04xRAhhOaoOR2EANCwOnXc+1pe+0HXwKu5apcFIYRQFcz1IASA1JLY0o927P5dz5afdGOXIUIIzTUYhAAA0Vbz5M8uMuv0tf97W8/zuCopQgjNIVX4PMKZiSqk/e31NSfFtz6wN7Mhv/A9zXpKrXahEEIITTmsER4k0mwsv6kzscB6+Rvb9vyxH3DOPUIIzXazJwhtzh3O3/p5CCWta+pW3LRgaHNh3d04uQIhhGa52ROEv9jXc9ZTz+4uTc5ECKNWO+nG+fWrkuvu3tn1hz7JsW6IEEKz0+wJwg+2tdw4f97pjz/5zEBmcs5IoPGM9IqbF+R3l175t+2ZdTlsKUUIodln9gQhANwwr/2Hq1f+3XN/+WlX92SdU0+pSz86r+Oyhu7H+l/+xjaMQ4QQmmVm26jRtzc2PHbe2X/z7POv5wu3LV0yWadNL42ll8aGXy/s+l3v7t/3ta6prT8lCWSyTo8QQqhqZlWNMHRSIv7cmvN+19P74Rde8sRkfr5E8rjoipsWdP5t4/5nMmu/vq3vxWGsHSKE0LFuFgYhADQZxlPnn+NwsebJZ/rdSR72mTwuuhzjECGEZovZGYQAYDD209NXX1Rfd+Yfn9ycz0/6+cM4nHdFw76nM2u/vm3/s4OBMwmTNxBCCE2z2dZHWIkA3LZ0Sbtlnf+npx84/dTz62on/SnSJ8TSJ8SGtxZ6nh/a/bvemmXxxtPTsQ5z0p8IIYTQFJnNQRj6+/kd8yLW+/781y8uWXzjgvmUTP4Ql+SiaHJRNLD5wCvZrf+5ByTUr042npZWImzSnwshhNDkmrVNo5XW1Nc9fcG5D3TvOfOPT64dGp6iZ1FM1nhGeuXnFh13dauT8V/859c3/6h7+PXCFD0dQgihSTH7a4ShRdHoUxece9/ursufef5tDfV3rjipRtOm6LmirebCK82OS+v7Xhje/ov9hJG6kxO1y+NmvT5Fz4gQQuhNmxM1whABuLaj/bVLLkpp6rJHH//x7q4pHeypRpWWC2pP+X8WLXxPk18INnxn19r/vbXrD33F/c5UPi1CCKGjM1dqhGVJVb1rxbIPdbTfuPaV/9i5+1srly+Nx6fw+QjEOyPxzkjnu5pKPe7Aq9nNP+ySHNJLY7UrEvF5Fs7KRwih6ppzQRhamUo+v+a8n3R1X/Tks1e1tXz1xBOiypT/KKxGvb2xvv3i+kK3PbAu9/pP9wBA7fJ4akksPs8iDCMRIYSqYI4GIQBQQq7taL+4of4zr2446dHHv3bSie9pbZ6KMaVjEYi2m9F2c94VDYW9zuD63K7f9ti9XrzTSh4fTR0fxa5EhBCaTnM3CEONhvGT01b9qX/g1vUbb92w8eZFC6+b1xFRpmnaQ7TFiLYY7ZfUc1dktxUHN+X3PZkRvoh3RtJLY+kTYoqFEzAQQmhqzfUgDJ1fV/vcmvNeGhq+a+v2L2187YMdbZ87flGLOX3z4plOw3W9AcDuc4e2FAZeye745X6zQUt0RuKdkdg8S8VZiQghNAUwCA84JZX88amn7CgW79q6/aRHH7+iqel/LV40tUNpxmPW62a93nxOjeQyt6uU217c/2zm9f+7R40r8flWvNOKz4uYdVM19wMhhOYaDMKxOiORu1Ys+6elS364q+uSp55bHI99cuGCdzQ3Tn9JCCOJBZHEgggASCFLPW5uR2l4S6HrkT7BZXyeFe+0Yu1WpMVg2hyaBoMQQpMLg3B8CVX91KIF/6Nz3n27uz+3bsMXN266pqPtqtbWNqs664gSSiLNRqTZaDo7DQDusJ/bUcztKg28ki3td/W0Fm0zoq1mtM2MNhsUcxEhhI4YBuHhGIx9pHPe9fM7nuwf+GnXnpP/+49LE/H3tbVe2dpSq1ezcVJPqnUrk3UrkwAguSz1uIU9dqHb7n9puNTjGjVatNWMthmRZsNqMhQTOxcRQmhCGIRvjBJyQX3dBfV13zllxRN9/T/e3X3r+o1LE/FrO9rf19YaV6v8MySMRFqMSIvRcFoKACSXxR630G0X9th9L2VLPY5iMqvJiDTpVpNhNRpWg04VnLOIEEIjMAiPAiPkoob6ixrqiwF/cN/++7u7P7duw8UN9e9ta31bQ33VEzFEGAlnZQCkAAAkOENeab9b6nGGNuX3/nHAHvCMtGo1GVaDHg7MMes07GVECM1ZM+LafcyJKOz97a3vb28d8vxf7N37vZ27PvzCSyuSiYsbGy5uqF+VSk7HxPwjRMBIa0ZaC+dmQNiU2ueG0ZhZl7X7PbvPVWOKWaeb9ZpVb5j1mlmn6ym1ugVHCKHpgUH4lqQ09Yb5826YPy+Q8s+Zwd/u7/n42ld2l0oX1NVd1FB/SWN9u2VVu4xjEUYiTUakyQBIjOyS4Ax6YSIWe5yBdVm7zw1sYdRoZq1m1KhGjW7UakaNZqRVXAoOITTLYBBODoWQs2trzq6tgZOW7rOdR3v7Hu3t/cKGTfW6/raG+rNq02fUpKdzhv7RIWDUaEaNllocLe/jrnAynpPxnAGv2ONkNuScjOdlfS2uGjWaUaPqaU1PqUZa01OqpFP6SR4IITSFMAgnX7NpXDev/bp57ULKl4ezj/X23be7++NrXzUYPbOm5oya9Ok16ZOTCY3O6G45ptNwwkblTimkO+Q7A54z6DmD/tBrBXfIcwb9oBhocVVPqeVo1BKqnlT1pMqMGf0yEUIIg3AKUUJOSSVPSSXDm1sLhT9nBv+cGfrhrt3bCsXlycTp6fSZtemTk8n5kRnXgjouQklYdxyzP5/Nq4HuDvnOkOcO+rmdJTfre8O+O+wDgJ5UtYSqJ1Q9pWoJRUuoWkLV4ooWVfBTqBBCVYdBOH0WRaOLotFrOtoBoBAELwwOPZcZ/OGu3TcNr8v5wbJEfHkysSyRODmVWBqPm+xYmvxHGDESmlGjJSAy5i7uCDfru0O+l/PdIb/QbXsb827W93OBX+JqhI2EYjz8rmhxVY0qWlxRYwpO80AITQMMwuqIKko4NzG8Oeh5rwxnXx3OPpfJfHv7zs35/LyItTyRWJ5MnBCPLY7FOqMRZeaMRD0azKCWoVsN43y2lBTSzwdeNvDygZfzvVxQ2ON4ubxfCLxc4OcDohAtrqoRpsUUNa6oUUWNKlpMUSNMiShqhKkRrFMihN4qDMIZIa1pa+rr1ozmoi/Ea/n8q8PZ9dncPTt2bcnn99pOh2UticeOi0WPj0WXxGLHx2Ip7die4UAoCZtJJzqAO8LL+X6Re/nAzwV+ISjudYYLgV8I/CL3i0FQ4mp0JBHVKFOjihJRVIspEaZaTAl3Wgr2UyKEDgODcCZSKV2WSCxLJMp7PCG2Fgqbc4XXC4U/9Q18Z/vOLfmCTumCaKQzEpkfsTpHN1pNcwbNYnxrmEFNQz/MWFsppF/kQZiLowHpZDy/iwelkZ1BiYtAhumoWIpqMcVkSngz3DCZajHFGtnGKiZCcw0G4bFBo3RpPD7mM6F6HGdbobizWNxRLD3ZP/CDnbt3FEsDrttuWZ3RyPyI1WFZrabZEbFaTbPFNGb4ONU3gVCixRQt9gZ/xiKQQYkHxcAv8SD8snlQ4qVh3x/dHvlyONOpYrLyFzMP3GQGVYwDe5jBFIPixEqEjnUYhMewRsNoNIyza2sqd7pC7CwWdxZLOwrFrlJpfTbXVSp1l+z9jlOjaW2W2WqabZbZblnNptFimg263mKaEeVYGptztKhCwpE4R3Jw4HBui5GwtHng8KDEuS2cQY/bInA4d3hgi/Au7ghCiWJQZjBm0DAsmU4VgzGdjuzRGTMo02iYo1SnTKNMn21vShA6dmEQzjY6pYtjscWx2Jj9Qsoex+0qlfbY9h7b3l20/zo4tM+2exx3r21TQlpNs97QW02zwdBbTKNe1+t0vckw6nS9TtfUWVebnIhiMMVgR77CnPBE4AjucO6IwOGBI7jLuSO4K4KScDI+dzh3RfgV2Jx7QriCe0IxGdMp1WhYB2UapRoZSU2VUo2M7FRpGK5UpUw/sDGlPwSE5hQMwqMjJAx6waAXDHq8/L0QiJzPwwMCKfOj256QRS7Kj02qBzWiaZRGlJHLWUylCiEqJVGFAUBMKd+klJCEyhRCYipVCYkqTGfEYkd9HaSENJtGs2mMe28+CPbadq/j7rHtPsfdY9svD2X7XLfXcftdt99146raYOi1mtZgGA2GXqfrNZpWo2k1ulana4bvd5imdUxN+ZgsVKOaRuHIqpsHSAgrlNwTwhOBzbkrhC+4KwJHCE8EJe5k/PBe7grucuHLME3Dw8JEJCqopkIVwgxGVUIVqpiUKCMVUMoIMyhVKVUJ0xlhoJiMMMI0yjRKFKIY2CeKEAbhxLpK3vqsvT5rrx+2txXcATcY9IKsz9OaktZYWldGNjQlqtCUNpIBCiHHxUbCRiUkqhxIrGGfVy5E5nJRGo3JnM+5BF/IQuAAQC4QXEpfyEIguJQ5nwdS5n3hCVnk3OWyxIXBqMlIhDGNkoTGVELiKjMYMRmNKlQlJKUpYZRGFKpRmlSZRklUoRajOqNJlamUxBRmKVSnJKYo49Yjy/pdt9/1+l23x3H6XHfA9Tbn8xnX63fdAc8bcNxB3weAMBprNC2taSlNTWtaSlXD7ZSmpTU1pWopTU2ox/Z410lAIOxofNMn4I4QvsgN5Q3FFIHkDhe+FIEIBwcJX3BHBFzyASF8IXzJXS45BDaXXHJPcE/IQAYOBwDFYIQRptMwSplOCSOKSYEQxWSEADMYocAMGoYoVSjVCFXC4wnVKGWEajQ8SXjM5P2kEJpyGIQjhn2+IWtvyNqvDtvhhqXQExPm8oT59sb4J2JGrT4Se+M+XEhR9EsAEAhuBzYAeNx3uRtuaOwNrvsESFQLp6IrlNCIOrLQDCPMUscfNWlz4XBZ5NwTctjjvpD5gDtc2lzkA+4LOexzT8ghj+8p+Z4QQz4Pk7UYjOz3pSwEvBgIT8gxuZhQmUpJXGUGpSYjMZUphKQ0plISVYyUajXqJK4yhZCkxlRCqO+ko1EA6fIgG/gZzxv0vEHPG/L8Qc/bVigO+d6g5w953pDnD/lezg/COEyqFd81NaGqCVWJK2pSU+OKElfVmKLEVSWmqMf6XJFJxwzKDKqBEomNX8s/QlJI7goZSO6JMEq5I6SQgS1AysDmUgJ3uOQQlITkgntSBEKE330ZpmwYvSKQwhNh0IZxSCgwgwFAGPlhuDKdEkqoSqhKCQOmMQBQLAoATGeEQliFBUIUY3QngzBrAUCxGABQleJ6C2iyzPUgtLn4effQ93YMvDpsL00YJyXMkxLme9tSy5JmmHmB4Hvz+/YWtm/LFnJePu+G3/M5r5Dz8jk3n3PzRb9UTi+FMlMxAUBlqsF0AFCZ4vPg8MWQIAteMdwuZyoAcMlLvg0AOtM0pgGApVqMUEJIVI0AgMY0jamMMksxASCqRQgQUzEUqtQxxVAMqtKIZQFATIsCgKWajDBD0RVqaEzTmaZQxVSMfAAKMwoBLwXCDZNVyrzPbS4cIfM+96Uc8rgnZDFwSly4QmZ9HgiZ9bkvZc4NXAmOEHlfBFImVcbCFl1KYgrTmWqxaJSR2uhIbTWmskBIhUpfcEbB4YKAyHl82OMlXgqEKASBI/wiD+wgyAdBMfCGfT9MxKjCooqSVNWIokQUFlOUhBpOIGQxRY2rislYhLGkppqMmYylVM1k1JiTzbZHglASptTkvtEI41AK4A4HgMAe/S6BO0JKGUap5MA9DhKCkgAAd8iXAsJwBSkDRwBAWJcNsxbkyKnC3AUCihHmIqEqhdHEDeupABCGblj/HtmvUgBgBiUUyseHdVkACKu5AKNhDMC0kYHBYWW3cg+aNeZuEK4btu/dOfDT3YOn10RuOb7hiuaEQojHva7cnt25Lb/a37U7170r291T6K2P1LXFmmN6LK5FY1psXqItrsViejSuxeJ6LK7FRitzU8jlnsc9ACj6JSFFOSzdwPVFEIjADhwAyHsFALB9O5Dc537eLQgp9uT2le8q+TaX3AncQAQe91zuhY8tn3A0Ly1GqKHoKlU1pmqjeWkS2qxaABA1IsQgpmooVNGoBlxahhVWXmNaNBeAzvSSoAQUH1RfgKSazSUBrcilL2XO51zKrM8VwrI+F5IO+1wCG/I4gDbkBRC2JEvI+lyAzPsCpBSCFXzJBSv4kHGAUaAACgUhOaMBF5IS4QkJILkUnhCB5IEQjgw8LrkMLMZ0RnRKo4piMmIwFlepTmlCVWIKMxir11WVkoSqapRGFGYxplMaV1WFkKSmKoTElJG7pvrXPQsQNpqvkan8cUkIW3fD3zdAOSal8AXASOiW8zis8o7sFwAgnYwPYWy7Asr5WnFC7gnJJQBwd3RjdM+hGRm2LQMAUUbahwkFpo/2m5gUyMGHMSiPeyq3k5czGIAoJj1k54Ejy/kNGM9vzZwLwmIgHuge/N6OgX22//fza16+eEmEFJ/oevqLT766K9uVsQdbYk0dibZ5ibYL2s/uSLS1xVtUWv2fks40nWkwGlRTJ8zLMG7twAnz0uP+mLwseEUJ0vYd23c84RXsIlVoWHkdCePA5kKEWVt+VHjasHZbbg0O4zZNqamYrQAR1aIGDXMXwpvhEBDVcATRWcSTVBDKiOlJIkERoAYAXGpASIETjWkFXwChgaQOl0BYMZAAMORzT4iCzx0hXSEKPrcD2FMSgZC+kK4MpPQd4UgJlIBCAECGVxUJkoLkUgJIAVJIyYFTIAoFhYACRGVEJUSlVKegMcIIiSphhYREFaoRaig0wqjBmEqhRlcBIKmqFiOWwizGEopiKkyjpNnQASCqsLkzQHcSjFb14C30tr45hULBUEyQABUZKTwhuASAsKkZAKQA7o6OpLMFyIOiNGxwDu8NIxlgtEIMACADWxyycyTsoSK/K88JlUk52jQNYWqO9t2WcxoqaslwcLKGfcPl16uYtDyuqrJdmlBSuXJT2Lh94Mza+HdVPhHMgIbu6l/ip81LQ6Xv7Rj4effQuXXRL57QdHaaPbvnz3c+/+Trg9vPaj31ks41ncmO5lgjI3P6/X4YtEcbt4VCIRo90oeEtVspZcEvAkAYt1yIsG81DEtf+E7gHrjJg7xbAIDhUq+Qotxi7HLX4354mJSyvN8Tvhu4AOAEri98KLcJA9SoVtgoFtUiRCFAIGxkLjduC2CaEgMARzJTtYSgrqSGanqCcgka0xyhBlL6UnGlwgGKAQii+AKKXApgQkI+kIEURSD7bM4lcElcDgKkkG1vTysAACAASURBVOAKCQC+AC5BSJAAXIIEKYFIWb4QCAAgQAgISggBYFSG1wyVEkaAEKISUCgohICUhkINSgFAJcRSKAColEYVSgnojKqUGJSajBEAg5KENnK9aTQ0SgAAEqoaUxgARBRmsgMjmU02cloACAdhHdWfxFzA9PD3A29l0NNUGDcpw8bqcDvszT2w7Ytxjq+IcBhJ8ZFt7gXl0K18CITt2KO3Kp8FAAKHl88w5lHlinhopEG7fNMYe7Ph7GTzaeMsX/ymzYkg9IW8/oXdzwwUbuisffHChTsH1z7++gPf7F2/qmnFOxddekbLqrD7DU2Pcu02rk84SHXShW3CMJqsMFpzBQlhHperreWEPnAMQNEbElTCaGxDRdZW9umWfDs8eTnIYbT2PGZnGMxQEcAAoDJVZ4YnVSEJYYYglgcMQAoSFaAAQEkolKpAmCeYL6kERQgZgOIFlFLqS9rnEQDgkgTAJBBfgJQggfoSJIAAEozGrcspABACgZRhBgsJEggAIQAkfPM/ulafrPjkZUqAkZFbFEiYq5QAAdAYEBhpn9MYUQkJz8MIRJSRhf8IkIQ6clWjQEzGjLBRkRCNUosRhQIBYjAKAHGFEkIspoQniSnMYJSF2UNJrGJINiMkrh6URkmVVS41WJnr4dnGHD+bVAazOrVNSFOCu0KKA7kYdi2XOQVPS0/y7272B2EhEO95brvOyPeXOc92//QfNrxwQu3xF3ace+uZN5cvQGjWKw++neq25SNUzuPKHPW57/CRpAx7f8PtcorDaAdwebvolHRdBwA3GOlFBoBynzEcPA4LAEqBzcXoPFfulx9SWaRAKr6UdmBzYAIoAPjAS6PjvXxiMGoxygQwDoxLRWWaAA0ICbgqJQGqUkI5ZxwYB0IJFUTJOEr5UrYX9DBiJUAgqQRGAKSEAIgEKiUBQgIRFp4CwGhVgQChXI5eEiUQUtmITGjFjEhCQAIrXz0pAQFESoDRMJYAgTioLU4lQIiEkfQnRELYtVc+rUqJSoGSkScSUmiM6pSyisqKQiCqUHLw3MzaMUPNCSRUVrneIQUwGYupB9W5dUqjCtMPqYhbjOqUjNk57txihZDYeGEfV9mhnYmMkPh4nd8aJRFluhsDxqwXMabCzeJECAGTqjpBGATBhg0botHowoULp/SJ+t3giqe3HRel0ey9/7mJXzz/go+vvD5lJN74kQhNpco3YQk9fpgjDy+fz8cmnv05RZzAKYf0oTfdwPWEX77p88AdTXeoGAhdVhHtEoBXvjMYPcOBmvTIM3LX534gwR9pMwZf+MP+gTIEkrpClB/lSSqBSJCO70pCPHngwuqLwBYUALhUJBAA8KXiCV+CEIEKAD6oYX76ggfABLAwngGgBJRSnUPlZZoGUiX0oAt3AGGD00j9WAIJJIPRsCQEJBAOFORB6RRICoRU7iQgKZEAFCqCdiTSJCmXihAox4iQYaoTgAM7uSTlM5QX6Jcj+0eeiVScgctDYhOAVTQMjLk77DEkBMa8IWAENCoJEErImCjXGQlXdlAooeSgFLTYSCHDpnxCKJFwfXvq/Z2Nh5bqTatCEHZ1dV144YXJZLK3t/ecc86577776NSMDthV9N7+1NZzU35m3/+5bPHfvO+Ed5Gxv7IjIouOLHnSC8APpM+lF0AgpOsDH/3uHPi3B59Lzx97BtuDipr+GMTU4JC3eAcdYOlw6AdKECDW2FZywijoY4fBE0sb83BiqGTM+0eFkXEeeMj5dQVwzOScZyhG5dTFGVLJnmaFQiESiZRb0SuV/NKBjrJRldX0gw8eaU4fo1xBL/MlOFxWNhVIIoujfXU+DzzhAQAX4FRUdgVIx7c5gMNhDAkw5Hlj9wJ4ggRA3IPffACAI6kMBwfJwBNSyJHEcyUb83ABQkrw4aCLjAQqOBVSBhK8g++yJQ3fhQTAxrxqKUfaEjgoACBBSIAOZr2/87JDS/6mVSEIv/zlL19wwQX33HNPPp9fsWLFww8//I53vGPSn2Vj1r706W1rkv2ZfT+69YybVzWtmOhIWXT5nozoy4qCIwuOzNtjNoiukqhBVAaaShRKdBUYJYYKlBJjdLtMZSQydoIzrY0fJuoOH5MAIEtuONjs4L0gerNj93EB7iExXPLGPFw6vuQH/+8FXI7zwLH/CdINIDj4/4kQYmlSyuGKrCWqAoe0yRBdPTRER36ehzK1sW8aAYAxoo/3FzvmV1B5fkODCUZ5HC7UCSHmhN3GhFGY4OlGz6ySw7cmUXqY8x9JGQ5wfJjuCiE6YNw3AXPzncG08X1/NjSN/uxnP3vssccAIBaLvfe97/3Zz342KUEY2Fl3aJduRJgRfyYPH1g7cE7kVb2w/ttv/3pDpL58mPQCsXeQ78nwvYO8O8P3ZqTjs5Y0bUzSqEGiBq2LqzGTRI3wi0aNia6kCABASlnyisViJHJgMqX0A/DHvv+Urj82RAFkWLc+lO3JQ98ccC7d8ZYmGFMprzy/44Ez/v+MGDwk1A88TEp7nHfKI3dyARM83eiTHvI+Y+xzi8Oc/0jKUFmYocMWpoxo47w7mfDg8d7KHO54U4OjbNch4zVpHPFzvYWh9gojE6wPdRR0hQvuaJMwyG7cN4hv9ZxH87s+6pMf5d/GW3ouXYVD3lMGQUCaU1B3LI8aHRwcLBQKnZ2d4c3Ozs5nn312ooNLpdL69etTqVR4MxKJnHHGGRMdvGPj/Vs2/lQD9hJdelf06s/1fevs/Cs6Nbo2PLVPjxGPgiulFzBfAU0jhkoNnaWTpEUjukIVnSqjP9YiQBEAgGoWpYd9489Uqh3mU2PfPKroVHlL62ZNM9d1i/pk/l0CBTj0uqoATOqThFwpncPWyMuoapCR1fIOfyGYvtZjx7YN84j+CGXA4fDxXImLozgYQHocZJA9tN3iMA8BANd+w8PGeaDPx2kgOXJCyOAt1ye44IFPvZHpE2+FtDlMdv1GcgFv/TVORBzd38ZbEozzXJLAgtUXGHVnTeLzTHcQlkolANBG30kZhlEsjtPIHtq7d++uXbueeeaZ8KZpmvfff/9EHYqx1vOaFPab4cj3svOvLP1bIjWwMR7x3ZwuiWkXTCOh19fo0YRiWKpqacxUmaFIojJTVSwqiXLIdVd4JS4PtzSa5H4xm/GkzEsiCTiSuCABICcJSOITWZIEAEoAoxNhoSBJ+beaA1IeAecDsSv+taUUJc79Q3o0bUImevOfl/QN/zaLQPjRXEAEgfw4cTQeeUiP+QRKEoIZ9nkHOoBOjuzn8lauv8cgRXIF3mB1wEoxOUENGwAAGAhVHFHl9chREKqc5HNWMqRHYIJf+mS926m4ButybGfEm3SUy+UxydXDXuumGSFcExOWhxA4x7FuKCw/wrMZhqEob5B00x2E9fX1hJDBwcFEIgEAmUymsXHCwT+LFi268sorr7zyyiM5s6ouuS+X/v7evecav/n8mruad3DvTxu9ff3i9AbvwrStlmw743o51825Xm7Yy7jFnOvlw5uul/O8AlNMSlWimJJQwSIcwGemL8ElqiOIR9Vw0SUbVFdwLmVWKgqhCqVMtSQwjdBwTRCTqb5iKkANhQGAzhQlHAdFmKFFy406tYpaHryjkLGLYRpUOXSFEaN8qkNEFIVO8OZUCh4EJQAwqUKPtE1JBm6eADGPYInOgNu2XdSOrJmISV+KN27xAwAppevljuTIkOuO7TE9DM59Pyi98XEVfL8oJv7nnPhRJf6mrv5B4HD+xpdFKeW4lZI38QIPxZimKpPW5kGpqk72hCVCmK5PYYecpsXIBP9xnHM22QvY6lr8rVcx3wTGLGXyftFvHSWKNvG6lUKIjtYzjnwFjyMx3UGoadqyZcuee+65+fPnA8AzzzyzevXqSTnz3du6//31HR+PbPqs9y7x1Sfdmph+8fLoKZ1jevh8IbpKuZ3FXHcxu3PkK7ezmB3ynQaV1ahKHSMpRU3TIK6oScJjjDYRHmGEctdkikKABLZOmUIo8JHZ1p5XEAe/F3bdzKElFDzwchNWf4/QRG+5D5MYlCqaGgGAwlE8D9H1OADkj+BQVbEol5Qf0eWeMYNpRzR9hRCSjHccyZEhXT+KWTGMqapydBdlVY3Qo19sT1UtdtgG9okoisHYGzcET7Smz5t4geiohKNGq5Jbc9wsGSxz00033XrrrYlEYtu2bY899tidd945Kac9rdj9xOa9Lfujypm+9r/+ljUmw/0OD54e2Pto7+6Xhnp3FrP77WKTGemMJOdH4vMjiXc0L5gfScyPxBuNKV84exY7qiXW0CTyPWpM4wI9CM1KVQjC6667jjF27733JpPJP/7xj62trZNy2lPiC+WZteZZS8IhYRtzmUd7dj3au/vZgX3Lk3UXN3R8fslp8yOJdis2UesiQgihOag6K8tcc80111xzzeSek86vH2xNPNy347Hert/37FQIvaih4+/nn3j/6Zcn1SkYa4gQQrOAEMI5qDtZuiUp+GEOgMCT/kE9/cK1gR/UbyNKY/tVhFOCMZPlPUcGB/e780B4Y5cdkHZRVjxQCGGsPF9fcc5hXtPRmj1rjf6ke8stG54+r6714oaOzx6/amE0We0SIYRmJ+na0i4KIgkh0inJ0SEC0rHLESJ9F4KRjnMZBLJ8fZdS2Af664VrQ/khgkvnwJQS6bsyOND1LuzCgXHLnAu38kgPAq/yZmVQSe7LinSRnMvKiSuEUPOgjiGiWwetEkcpNQ7ub1Y0oh40OI7qJhw8dIhaY1vsqWHBwa1xRDOodfA8MaYo2tiZY8SMVA5Z4pyzlklem3P2BOH7Whe9r/W4iHEsTb9DCE1EurbkHMJ6A/eh8voe+NJ3AUAKIUcrK8IphhPyDuSHlMIuHjhbmDeCl+s30juQNHI0nCQPpDu6ZHnghU8EANJ1JB+pvhDdBErz4frcFbFBDPPAtqqDMjJOiigKKV/fCaHmgQ51qptQfghl1DpwF1FriHJgpBU1owfWSqSMGhXjPA9OJqKoRDvQDEaYSirShTBG9Bk0RvRozZLBMlNEIZMwuRUhVKmcPcLOV94MY+PAvU4JBJdCSLcEFQEjnCJIeaAONFqPKddghDdSbSqnnXBGakhENwljAEBUnSgaABB19HKvqETVAYBQSkYrK9SIhKvbEFUfyQ9CyrlCUnUjEUVZuX5DNL2cNGQ0nAhTiD4SG0TRyGjHSuXBgKNGZ5fZE4QIzTmCS7soiBBuaaQjR0rpFKWU0i7CaD9N2DcTVrBGakuBLzynXGGS4QG+K31fCh6GmbCLIGU5e6gZAwCijiaQGYWKmgc1LKCMkJFYIpoeJhBL1xNCD9SBGKNhXWS0BlNOF6KNrNpDdIMwvC6haYV/cAhNFymFXZSBL31Huo7kvgxveq5wbeBBWK8SdlHyQHqO9BwZ+NIpSSGkUwQpRakAIIVdACGFUwRKiW4WKAsrQ9SwgBBiRAghxIzAaD9N2DdDNIPqJlHSRNWAKVQ3y51DJDxA1YmqEsqIbgEANSPjfOYJQrMRBiFCR0R6jvRc4ZakY0vflZ4j7IL0XOm5wilKz5G+J5yS9F3pe6JUCPuxRpLPs8MeJmpFww6bsN5DzGjYe0Q1AxQ1rFdRKwpMoZpBVJ2oWthCSIzISL0q/E4JNSJQpc8jRGiWwSBEc4t0beHa0ikJtyTtonBKwilJpyRdWzgl4RSla0vPEY4t7aLwHOna0rNFqUA0nag6NSLEMImqEc2kZmRkpxklmk4jCaW2magaUXViRsINaljhwdjih9CMhf+Z6NgmSgVh56VdDDeEXRR2UTrF8TYKwi4SVaeGSXSLGhYxo9QwiRGhukl0k1pRlq6nmkl0gxoWMSJEN6hmEN3CdkKEZjEMQjTjSB6IUl6U8qIYfs+N3syJYl7YeVEqSLsYblAzSq0oMSPUjFErSs0INSLEjCi1zdSwiBmhRoSaUWJa1IxhniGEDoVBiKaVdG2eGxSFrCjleCEnCllRGObFnCjmRDErCllRyArPoVacRmLUilMrRiMxZsVIJKYlF4Q3iRkN8++gmVUIIfSmYBCiySRdmw8P8MIwzw2K/LAoDPPcEM8Pi+LIHqCMxVI0GqeRBIvEaTRBYymlsYNF4zSSoJE4jSUpLoCOEJpGGITo6Ejf48P9PDfIh/t5fpgPD4j8kJfpLZRyfHgAAFiylsVSLJ6msSSNJrV5i1ksSSPJcM+YlZkQQqjqMAjROCQP+PAAzw7wwT6eHeDDA3yoLxju58MZ6RRZspYlalmylsZSLFmrtS2kWjRa38SStcf00k0IobkJg3Buk5JnB4LBXp7pCTI9IxuDvSI/xOJplqxlqXqWrFXqWvTjV7JETVjbO/Q0olBQ8PMIEULHJgzCOUOIYLA36N8b9O8N+vYEA/uCgf18uJ9G4kpNI6tpVNIN+oKTlNUXsZpGJVkLlL3xORFCaPIEfklw33ezUga+WxDc5YE9stPLSR74Xt73nbq2cxvaTp3E58UgnJ1EKe/v2xX0dQcD+4K+PX7/Xj7YS2NJpbZFrW9R6lqMpacqtc0sVV+5jjBCCB0tKbnvFnhgC+56TjZML9/NC+EFXjHwS4J7vpsV3Av8UuAVhfB8Ny+4wwPH9wpSBL6bFdwP/BJTTKZoihanVFX1KGU6U0xFtShTVS1OKFP1OAAjZJLfpmMQzgbCKQY9Xf7+XX7Pbn//7qBnl/R9pbFDbWhV6lqsU9+m1LWEi55Uu6QIoZkl8Is8cAKvEHgFHjiBX/S9AvdtHti+m+WBe/BGjgcuD0q+m+eBw4OS7xVBSlWPMsWkTNeMRJheqhalTFe0CFNNxnRVj1OmK6qlqBZlWniTKaaiRShVVD1JmaKoRzRcHD+GCQEAgJTBwD6ve6u/Z5u/f5ff0yVKebWhXW2apzS2G0tWq40dLFlb7VIihKaDEH7glXw3y/1S4BcDv+i7uSDc9oq+m+OBzX3b93KjCVfy3Nxo1OUU1WKKoWgxRY0wxVC0qKpFmWIw1VL1OFMMVU9Y8TbGdKZaqh5jisEUS9WjjBlMjSiaRekx36qEQXgsqEg+r3ur372NWBGtdZHWtih6zt8oDe1KugHnlSN0TJMi8L2i7w4HXsH3ioFfCLyC7+Z8Lx94hdFUy/pegYfbXsF3c4FflFKoWkTR4ooaUVRL0aKqHlNUi6kRVYuqWsyI1DHVUrW4okWYYipqRNVjimKGUVft1z0jYBDOUKKYc3du8nZs8Lpe9/dsLydf7MKrtLZFNIJ/vgjNXIFX8L287+Z9Lxe4+TDPfDfre3nfLQRe3vfygVf03WHfLfhePvDygvuqHlX1hKJGFS0S1sxUPRFumJFGJRVV9biiRRU1MhJmI9U4/Y0LhA4Lg3AG4YN97o4N7o4N3o4NfHhAm7dEm780dtF7MfkQqi7PyfrusO/mfDfruTnfzRVy/RQc3837btZ3c76b872c7+Z9d9h380wxFT2majFVj498D7+0mJ6sVbWYosdULarqSVWLhnU4puAc3KrBIKwyv2e3u329t2Oju30DiECbv1TvPDF65mVqcydQWu3SITQ7SSl8N+s5w54zfGDDyfpu1nOzvjPsubkw+Txn2Hdzqh7XjKSqx1U9oWox1UgANU0rHYm3qfpS1UhUpF1C1WOTPqwRTSkMwiqQru28/rLz2ovOphcIU/QFJ+rHnRy/5INKXUu1i4bQMUxK7jnDnj3kOUNhtnnOyLbvDLvOkD+633dzmpFUjaSmJzUjoRpJzUhqelK3aqPpBZqerEi+pGYkDn2uQqEQiUQI9s3PChiE0yfI7He3vGxv/Iu79VW1ZYF54mk1139Ja1tU7XIhNNNJEbj2oOcMufagZw+6dsazhzwn3DPkOUOePeQ5Gc/J6WZKNZK6kR7JNiOlmal4zXEj20ZK0xOqkRo329CchUE4tSQP3C1rnU1/dV57UUphLFkdOePSmg/9v0Qzql00hGYEHjiePeiU+t3SgGsPenbGKfV79uBo8mU8O+O7Bc1M6WZaM9K6mdbMlG7WmtHmRN3SMOrCkNPNFABW0dBRwyCcGlK6OzeWXvyj/eozamOHsfS0mo98WW3sqHaxEJpm0i1lXDvjlgacYq9rZ9xSxin2uvZguNMtDQjh62aNbtXoVp1u1uhmWrdqY+mFmpHSzRrNTGtGSjfT1X4haDbDIJxkfk+X/cpTpRcfJ4pqrji3/tN3KTVN1S4UQlPFd3NOsdcpDTiF/W4p45T6nGKfU+p3CmHsDWhGQrdqdbPGiDSEG+nmVYZZq5lp3aoxrHpFw+XaUZVhEE4Ons3YrzxdeuVpPtRrLjsr9YHP6J1Lq10ohCaBW+p3Sv12fr9bGrAL+51Sv53f55YG7GKPW+qnTDciDYZVa0QadavWiNTH0ot0q9aMNOhmjWbVzIJlR9Csh0H41khpr3u28PSD/v5d5rKzEpdfpy84ERd5QccWHtil3F6n2GsX9tuFHrvQYxf2OcW+MPBUI2FYdWa0SY/UmdGmWGpBfdvZulVrRpt0qw5nc6NZAIPwTZKeW/zrfxf+9EsajcfO/zvjxNPxYxzQTBZ4hVJ+n53faxf2lzfC5BPct2LNRqTBjDWZ0aZocl5d25lGpN6KtehmLWX4h41mOQzCoyYK2eJf/lB46jdq68LUVZ/Qjzu52iVCaIQQvlPoKeX3lnJ7Srk9pfweO7+/lN9rF/ZLwa14ixltMqPNZqy5pnm1GWs2o01mtAkXnERzHAbhUQgG9hee+nXphceNE0+r/fi/qA3t1S4RmqME94q57lK2K9O3VfgZO7enmOsu5fa4dsaw6qx4ixVvs2KtyfrlzQsusWKtZqwZ0w6hiWAQHhFv58b8H3/h7twUPevyxs/fS6PJapcIzQlSBKX8vlKuu5jrKma7SrnuYrarmOv27EEr1hpJtKlmY6JmfsO8NVa8zYq3mJEGQvGfGqGjg/8zbyAY7M3+8tv+/t3RC/4ufc3/IhoODUBTggdOcXhXMbu7kN1dHN5VyO4qDu8u5fcYVl0k0W4l2iPxtvr28yKJ9kii3Yg0EEIBIJ/Px2KxapcdoWMbBuGEJA+Kz/w299/3R89+R/q6z+NYGDRZeOAWh3fmh3cUhnYUhncWhnYUs7tcezCS6IgkOqLJ+bH0gsb5F0aS8yLxdhyrgtBUwyAcn7v1laH/+pZS21T/6f9PSTdUuzjoWCWlKOX25Ie2FYa250dib7tb6rfibdHUgliyM914cvvid0eT841oY1jJQwhNs/GD8Ctf+co111zT0XHQkmBdXV3f+ta37rjjjmkpWNXwbCb72+97Ozcl3/1x44TV1S4OOpbwwC0MbcsPbc8Pbs0Nbi0MbssPbdetuliqM5rsjKUWNHdeHE3OtxJt+DE9CM0c4wfh3XffvWbNmjFBuHfv3q997WuzOAhH20IfiJ59ReqqTxFVq3aJ0IzGAzuX2ZrLbM5ltuQyW/KD25xibyQxL5ZeGEsvbF7w9tiq/xlLL8QPXEVohjuKptHBwcFEYtZ+dom7bd3wf32T1TTW3/x/lJrGahcHzTiC+/mhbbnMltzA5lxmSzaz2Sn0xNIL4+nj4rWLO5ddG0sviiTasaqH0DHnoCB85plnfvvb3wJAoVD49re//eCDD5bv8jzvoYceWr58+XQXcBoIMfybe5wNf06++2PG0tOrXRo0U7ilgWz/xuH+Ddn+TcP9G4vZ3Va8LVG7OF6zuH3Je+K1i6PJeRh7CM0CBwXh+vXr77nnHgCwbfs3v/mNohy41zCME0888Rvf+MZ0F3CKSdfO/Phr0nfrP3M3NXEV/LlLSl4Y2jHcvzHbv3G4f2O2f5PgbqLuxGTdCfUd5y1a9fF4+jgcwInQrHRQEN5444033ngjADQ2Nv7yl78888wzq1SqacKzmYHv/aPWujB55T8QhgNo5xYpeX5w21DvuuG+dUO9r2YHXjOs+kTdCcm6Excs/3CybqkZa652GRFC02H8q39PT880l2P6eV1bMv/xT9Fz3xm78KpqlwVNE6fQM9S3Lgy/zL6/UmakGpYl65cdv/ofappP1QxcMAihuegNqkFBEOTz+co9qVRqKsszTexXnxn6+b+n3nuTedIZ1S4LmkKunRnc/9Jgz9rMvpeGe1/VI3Wp+uWphmWLTvnY6ku/qWq4JgtCaIIg9Dzv9ttvv//++3fv3h0EQeVdUsppKdgUKjz56/wT/1X7P76itS2qdlnQJJMiyA9tH+pdl9n318y+v9r5/fHaxbUtpy1a+ZF00yrdTFe7gAihGWf8IPzc5z73rW9962Mf+9gLL7yQTqdXr179u9/9bvPmzV/84henuXyTS/Jg6D/v8vfuqL/pTpasq3Zx0OQIvEJm3wsD+/4ysOfPw30bIomOdNPKdNMpi1Z+NJZeCICfk4wQOpzxg/BHP/rRP//zP3/mM5/58Ic/3NzcfNttt/3jP/7jjTfe+MQTT3z2s5+d5iJOFlHKZ77/T9SK13/q33Dt7GOdWxoY2PfXgT1/zuz7S35oR6pheW3L6UtO/3S6caWi4ehfhNBRGCcIBwYGhoeHL7nkEgBQFKVUKgEAIeQLX/hCW1vb3r17W1papruYb1kwsG/gO583V5ybuPw6IFhFOCY5hZ6+7mcG9v55YO9f3VJfTfOptS2nLb/gq6mG5ZTixAaE0Js0ThAahgEAnHMAaGxs3LRpU7g/Ho8DQE9PzzEXhNJzMvfeFj3/3dGz31HtsqCj47u5/j3P9XU909f1lFsaqGs7q671jAUr/j5esxiXqEYITYpxgjAajba2tm7atGn58uWnnXbanXfe+fTTT69cufL2229XFKWzs3P6S/kWDf38bm3eEkzBY4UUwXD/pr6up/q6nh7qfSXVsKK+/ZzVl9ydrD8Rww8hNOnG7yO8/vrr165d+/73v/+yyy47+eSTzz333HD/rbfeesxNnyg881t/z7b6m++qdkHQX53uEQAAIABJREFUGygM7+jZ+XjPzscz+1+Kp4+r7zhn8amfrGleTRmufo4QmkLjB+Ftt90WblBKH3/88UcffXTnzp0rV64866yzpq9ok8Hrej33yI/rP/kNHB0zMwnuDex5fv+ux3t2PM6DUuP8CzuXfei0y+9R9Xi1i4YQmiveeF0xTdOuuOKKaSjKpBOlwuCP/iV11SeVhrZqlwUdxC0N9Ox6omfnf/d1PRVJdNS3n3vKxXfWtqzGqQ4Ioel3uCB0HKenp0cIUbnzmOkjlHLo/m+Yy840l59d7aKgEbnMlr1bH963/ZFSbm9Dx/lNCy5ZseYOnOSOEKqu8YNw27Ztn/zkJx955JFD7zpWVpbJPfYAzw+lr/t8tQuCYKh33b5tv9u79becuy0LL1t+/ldqmlfhBxghhGaI8YPwve997+7du2+//fbjjjuOHIOz7txt64pPP1R/y7/jZ0pUUS6zZc/rD3Vv+ZXgfvOCt698279h4ydCaAYaJyfy+fzLL7/8wAMPXHXVMfmxDDw/NHjf11IfuIUlaqpdljlIZva92L35V3u3P6IbqeaFl51+xb2J2iXVLhVCCE1onCAMp9IvWLBg2gszGQQf/OE/R866wlh8SrWLMrcUhnZ0bf5F12u/YIrRvvjd573nF9HUMdKdjBCa28YJwmQyeeGFF/7+978/5ZT/v707j4uqXNwA/p4ZdoZ9B2VxIcCFRRTcUjQJLfUq4JZezRY/LmUZ95qV3co0b6bdWxk3b2l5tdRwwZXFEleEZBFkFRm2GdlhYIAZhpnz++P0G8lQRxzmwJzn+9eZdw6Hh2GYh7MPvC6RnN5H+AaWzyxkOwhXdMokotunSnN/6mgpcx02K/jZL+zdxrEdCgDgMdwrwoaGhrKyMmb6tddeW716dXt7+7PPPmtubt79C/pzO3YWpLdnXXKK+YrwcP2RvqXsktdWXKwoiKspS3HynDokYJWnzyyKhz2yADDw3PvkOnny5MqVK7s/t23btm3btt33Bf32qFFlQ7XkyJcOr37IM8e52H2ouS5PmLO/qvikjVOAu2/UmPB/GRiaSaVStCAADFD3PrzCwsKOHDnCYpQn1FVdJnhuhZGHD9tB9JOyS1ZVfLI0Z79MWu01aumMZRdMBM5shwIA0IJ7Rejp6enp6anhl1VVVf3222/z5s3rk1C9YjwidCCe6dH/SZtKy/J+Kss7ZGXvNzxolduwmVj5AwB90stPtJs3b27atKl3RSgWi3/88ceMjAyFQhEXF9e7ANDXVEqF+E6CMPdAS0Ohh9+CsEVnzK3c2Q4FAKB9LPxrLxQKCwsLnZycDh48qPvvDo8k72i4k71PmPs/C1vvIaOXuQ6bidveAoAeY6EIJ06cOHHixCtXrqAI+5s2SfntjG8qi44P8p79dPQxC5uBeS4pAMDjwM4eIISQ5rq82xn/qRaed/eNwoEwAMApfVKE9fX1GRkZfx5/+umnTU1NNV9OXl5eSkrK3//+d+ahubn51atXeQ84R1Aul1MUZWSEm7g+nqbqjLKcPS31eYN8Fk5a8IuBkYWCJorW1sdaSFtbW789r0a/SaVStiNwVFtbm0qlwgF6uqdQKFQqVWdnp4bzm5iYGBo+YudOnxRhWVnZF1988efxgICAxyrCp556avXq1bNmzWIeWlhYWFlZPWhmIyMjFKHmaFpVLTxfcP1zlVI+fMzqwT4/PMmOQIqiBAKBFuOB5iwsLNiOwEUURZmbm6MIdY8pQmNjbd5rvU+KMDg4+MyZM0++HAMDAycnpwFzB8QBgqZVVcXx+dd2mApc/Cb8zdkzDHeEAAAuY2EfoUqlkkgkra2tNE03NTXx+XxLS1wLRkdqKy7lXv6YxzMMnP5PR/fJbMcBAGBfL4twxowZV69e7d3XlpWVBQcHM9NDhw718fG5du1a7xYFmmu4eyPvyifyjnrf0JhB3s9jLRAAgPHAIhSLxadOnRIKhU1NTd3Hv/nmG0KIkZGRnV0v7/Y3ZMiQxsbG3n0t9EJLfWFB2q7G6iyfces9Ry7GreEBALrruQhTUlJmz54tl8uNjIwMDQ3lcnlHR4epqamLi4uO88GTaG+pLEz/QnwncXjQq2MjvuLxcSQRAMD9ej4VYf369WPGjKmrq4uOjl67dm17e/v58+ddXFw++OAD3caDXuqUSbJ+ffvXH2eaW7nPfCntqbHr0IIAAD3qYY1QLpfn5eWdP3+eOVehq6uLEDJ9+vQ9e/bMnz8/OjraxMRE1zHhMdDleUduXd3mNnz2sy9eMzTGgUgAAA/TQxE2NzcrlUpXV1dCiJWVlXof4fjx41taWoqKivz9/XWaETTW2nQn+9dNCrlk/JzvbZ0D2Y4DADAA9LBp1MHBwdjYWCwWE0K8vLwuX76sVCoJIbm5uYQQMzMzHUcETSi7OvJTP7t4ZJ7LkBlhi8+iBQEANNRDEfJ4vEmTJp07d44QsmjRIqFQOHXq1HXr1s2dO3fkyJE4vb0fulualPTD022S8hl/vTAs8BUcFwoAoLmejxqNjY1ltog6OTkdP35869atJ06cCAkJ2blzJ5+PD9l+RNoszL7wrqytdtzMr+1cx7IdBwBg4Om5CIcPH66ejoiIiIiI0FUe0BSt6ipI+1fpze99QtYP9X8Rd40HAOidnk+feOWVV4qKiu4bLC4uXrBgQd9HgkeTNgtTDs9tqrn5zLJfhgW+ghYEAOi1novw1KlTDQ0N9w02NDTExcX1fSR4hPL8n1MOzR7s85eJf9lvYu7EdhwAgIHtMdYkqqqqen1ZNdAKhbwl65e3JfX5k6OOWNn7sR0HAEAf/KEIz5w588MPPxBCJBLJ5s2bu9eeVCq9du3alClTdB0Q/l9d5ZXfEte7DZs1/YUkXCYGAEBb/lCEEomktLSUEKJUKkUikUQiUT9laWm5fPnyt99+W9cBgRCVSlGY9m/hrYNjZux09pzGdhwAAL3yhyJcsmTJkiVLCCEhISGxsbFBQUEspYJ7WhtL0s+tMbMcNGPZr0YmNmzHAQDQNz3vI0xLS9NxDuhRac7+/Gufjpz0jufIJWxnAQDQTw88WKa4uDg2NjYvL8/AwODs2bOEkEOHDllYWDz33HM6jMddtKor+8K7DeIbUxedFFjjaj4AAH2l5yK8fPnyzJkzraysPD09y8vLmUGxWBwbG4si1AFFZ2v62dU0rZqy4DhuHwEA0Kd6Po9wzZo1U6ZMKSkp2bp1q3owIiKipKSkurpaV9k4qq257MJPzwmsvSb+5X9oQQCAvtbDGmFDQ8OtW7e+/fZbU1NTiqLU4+7u7oQQsVjs7Oysu4AcUy9KTzvzKnPVNLazAABwQg9FyNyJ19jY+L7xuro6QoiREc5g6ytlt368dXV7yKyvHQZPYjsLAABX9LBp1NHR0cXF5cSJE4SQ7muE+/fvt7S0fOqpp3SXjkPo/NTPin7bPSX6GFoQAECXelgjpChq48aNMTEx7e3tHh4eSqUyNTX18OHDX3311ebNmw0NDXWfUr91Kdp+O7euS9EetviskYkV23EAALil56NGX3/9dYlE8sknn8hkMkLIhAkT+Hz+2rVr33vvPd3G038d0rvX4pfbOI0OmbaHx8M/GQAAutZzEVIU9f77769du/bixYv19fUCgWDy5MmDBw/WcTi919p4+/LRBd7Ba4cFvsx2FgAAjnrY3Sfs7Ozmz5+vsyhcI20qvXx04chJ77r7RrGdBQCAux5YhE1NTfv27cvKyhKJRM7OziNHjly5ciVOnNAWaXPppaPRIyZuQgsCALCr5xPqb968OWLEiLfeeis1NVUul2dmZm7evNnX1/fixYs6zqeXpM3CSz9H+Ya86eEXzXYWAACu63mNcPny5ebm5pmZmYGBgcxIcXHx0qVLlyxZUlZWhgNHn0R7S9WVY4t8Qt7wGrWU7SwAANDTGmFtbe3Nmze//vprdQsSQry9vfft2ycWi/Py8nQYT9+0t1Rdiov0HrNmyOi/sp0FAAAI6bEIjY2NKYpydXW9b9zNzY0QYmJiootc+qi9VXQpLmr4mNVD/JeznQUAAH7XQxFaWVmFh4fv3bv3vvG9e/eOGDECV5bpnfZW0aWfI4ePWTXUfwXbWQAA4J57+wjFYnF+fj4zvWzZsjfeeCMvL2/+/PnOzs719fVnz55NTk7esWNH94uugYY6WsWX4qKGB72KS2kDAPQ394owMTFx5cqV3Z9LTExMTEzsPrJq1apXX31VR9H0RYf07qW4qGEBLw0NWPnouQEAQLfuFeGcOXNu3LjBYhS9JO9ovPRz1FD/F3HtGACA/uleEdrZ2dnZ2bEYRf/Qqq60M6vcvJ8fFvQK21kAAKBnPZ9QD1qRc+lDHo8/YsLf2Q4CAAAP9LBrjcKTqCiIqy77ddricxTFZzsLAAA8EIqwTzTX5uZc+vDpqDhDY0u2swAAwMNg06j2ydvrrp1cEThtu6UdzrkEAOjvUIRaplIprp9+1WvkC27Dn2M7CwAAPBqKUMtuXnjXyMTaJ+QNtoMAAIBGsI9Qm0pz9tdVpYYtPktR+A8DAGBgQBFqTcPdG/mpn01dGG9oZMF2FgAA0BRWXLRD1laTdmZVcPjnAmsvtrMAAMBjQBFqgbJLnnryxWEBK529prOdBQAAHg+KUAuyL7xjKnDxDl7DdhAAAHhs2Ef4pCqLjjfezZy25CwhuEEVAMDAgyJ8Ip2y5pyLH4bO/pZvYMp2FgAA6A1sGn0iORf/MfipuXYuwWwHAQCAXkIR9l5d5dW6qlQ/3FwCAGAgQxH2krKrI+N8TEDYVgNDc7azAABA76EIeynv2qd2LmNchsxgOwgAADwRHCzTG811eRUFR2cs+5XtIAAA8KSwRvjYaFVXRtKbo5/+h7GZPdtZAADgSaEIH1txRqyxmb27byTbQQAAQAuwafTxSJtLi2/ETltyju0gAACgHVgjfCx01i+bfEPeNLfyYDsJAABoB4rwMQhv/aiQtwwNXMl2EAAA0BpsGtWUrK0278onk+Yfoig+21kAAEBrsEaoqewL73qNXmrtOJLtIAAAoE0oQo1UC89L6vN9xr3JdhAAANAybBp9NEVna+YvG8fN3M03MGY7CwAAaBnWCB/tTtZ3DoMm2LuFsh0EAAC0j4U1QoVCkZKScuvWLVNT01mzZrm7u+s+g+a6FG0l2XunLDjOdhAAAOgTLKwRrl+/fvPmzSKR6Pr16yNGjLh48aLuM2juTtZ3Th5PW9gMZTsIAAD0CRbWCLds2WJnZ8dMOzg4fP7551OmTNF9DE10KdpvZ337dOQRtoMAAEBfYWGNUN2ChBATExM+v/+ellea84PD4AmW9j5sBwEAgL7C5lGjlZWVsbGxhw8fftAMd+/e/d///peens48tLKyiomJoSiqx5nlcjlFUTRNayueskt+O+Obcc9/L5fLtbVMfSWXyw0NDdlOwUVyudzIyIjtFFwkl8sNDAwe9HEEfUehUKhUKs3nNzQ05PEescrXJ0WYk5MTHh7+5/EzZ86MGTOGma6vr581a9abb745ffr0By2Hz+cLBAIbGxvmoampKY/He9A7j3nqkT+w5sryf7RxDrJ2GKGtBeoxHo+nxVceNIdXni3MK48i1D3mDa/dt32fFOGIESNu3br153Fra2tmoqmpKTw8fPbs2e++++5DluPo6Dhv3rzo6GhNvqlKpaIoSlvrJSpl553sb8bP+QErOpowNDTEC8UKvPJsYV55FCErVCqVdt/2fVKEfD7f3v6BN62VSCQRERFTpkzZtm1bX3x3rRDmHrBxCrBxGs12EAAA6Fss7COMiYnJzc318PBYsGABIWTw4ME7d+7UfYyHUKkUxRmxoc//l+0gAADQ51gowpdeeqn7HkRLS0vdZ3i4sls/Wdr52DgFsB0EAAD6HAtFGBoaGhrafy9XplIpim/sHhvxFdtBAABAF3C82f0q8n8WWHvZuY5lOwgAAOgCivAPaFpZ9NtunxDcbgkAgCtQhH9QUXDU1MLV3i2E7SAAAKAjKMJ7aFpZlP6FbyhWBwEAOARFeE9lUbyRqa3DoAlsBwEAAN1BEf6OplVF6V/6jY9hOwgAAOgUivB3otunDYzMHd2fZjsIAADoFIqQQRem/ds3dAPbMQAAQNdQhIQQUi38hcc3dPacxnYQAADQNRQhIYSU5x/xGr2M7RQAAMACFCFRdLbWVlwaNHw220EAAIAFKEJSVXzS0f1pQ+N+d+1vAADQARQhqSw4NthnPtspAACAHVwvwvZWUUtDIQ6TAQDgLK4XYWXBUTfvOTy+EdtBAACAHVwvworC4+7YLgoAwGGcLsLm2ltdijY712C2gwAAAGs4XYQVhXEeftGEUGwHAQAA1nC3CGlaVVUUj+NFAQA4jrtFWFtx2VTgamEzlO0gAADAJu4WYUVBnLtvJNspAACAZRwtQmVXx93SpEHeuKwaAADXcbQIRSXn7N1CjM0c2A4CAAAs42gRVhYcxWEyAABAuFmE8vb6xupM16HPsh0EAADYx8UirCw87jo0gm9gynYQAABgHxeLsKIQ20UBAOB3nCtCaVOprK3GYfAEtoMAAEC/wLkiLC84MtgnkqL4bAcBAIB+gWtFSFcWHnf3xXZRAAD4HbeKsF6UzjcwtbL3YzsIAAD0F9wqwoqCOA+/BWynAACAfoRDRahSKsQl5wb7zGM7CAAA9CMcKsK7wmQrBz9TgQvbQQAAoB/hUBFWFMS5++B2EwAA8AdcKcJOmaSu8qrrsJlsBwEAgP6FK0UoLjnj5DHV0NiS7SAAANC/cKUIayuuOHmGsZ0CAAD6Ha4UYb0o1WHQeLZTAABAv8OJIpQ2CwmhzK082A4CAAD9DieKsL4qFVfZBgCAHnGiCOuqUu2xXRQAAHrCiSKsF13HDkIAAOiR/hdhe0ulSqkQWA9hOwgAAPRH+l+EdVU4XhQAAB5I/4uwHjsIAQDgwfS/CLFGCAAAD6HnRdghvdvVKbWwHcZ2EAAA6Kf0vAjrKq/ZDxpPCMV2EAAA6Kf0vAjrsV0UAAAeSs+LsK4q1WEQrikDAAAPpM9FKGur6ZQ1Wdh5sx0EAAD6L30uwrqqVPtBoRSlzz8jAAA8IX0uCewgBACAR9LnIqyrSrXHDkIAAHgovS1CeUeDrK3Gyt6H7SAAANCv6W0R1lel2ruFUhSf7SAAANCv6W0R4spqAACgCf0twt+vKQMAAPAw+lmEnbLmDqnY2mEE20EAAKC/088irK9KtXMdS/EM2A4CAAD9nX4WIXYQAgCAhtgpws7OTpFIJJPJ+mj59VXYQQgAABphoQg/+ugje3v7CRMm2Nrazps3TyqVanf5CnlLm6TC2nGUdhcLAAB6iYUiXL58eV1dXXl5eU1NjUgk+vzzz7W7/HrRdVuXIB7PULuLBQAAvcTC4SQeHh7MhIWFxejRo5uamrS7/LqqVGwXBQAADbFzXOXt27eTkpKKiopyc3OPHDnyoNkUCkVNTU1paSnz0NjY2M3N7ZELr69K9Z+6RWtZAQBAr/VJEV69ejU+Pv6+QT6f/8knnzDTUqm0oqKioqKiq6uro6PjQcu5ffv21atXd+7cyTy0trZOSUnh8XrenCuXyymK4lOKlsYSA7Mhra2t2vhRQCNtbW00TbOdgou0vosdNNTW1qZSqSiKYjsI5ygUCpVK1dnZqeH8JiYmhoaP2FPWJ0VoZmbm7Ox832D3AgsMDAwMDCSEbNq0aePGjX9uTYafn98//vGP6OhoTb6pkZERRVGNosu2zoFW1na9zQ69QVGUQCBgOwVHWVhYsB2BiyiKMjc3RxHqHlOExsbGWlxmnxShuuceyd3d/erVq1r81jiDEAAAHgsL+wi3b98+duxYBweHgoKCbdu2vf3221pceH1V6sjJ72lxgQAAoN9YKEIjI6Pt27c3Nja6urpu3759yZIl2lpyl6KtpaHI1jlIWwsEAAC9x0IRbtiwYcOGDX2x5KbqTBun0XwDbW47BgAA/aZX1xptEF3HGYQAAPBY9KsIxddxpAwAADwW/SlCZZespb7A1mUM20EAAGAg0Z8ibK7JtLT35RuYsh0EAAAGEv0pQkldvv2giWynAACAAUZ/7uHuNXoF9YCrrwEAADyI/hQhxTPA5Y4AAOBxYRUKAAA4TX+KsKioqKSkhO0UXNTS0nLlyhW2U3BUQkIC2xE46tq1axKJhO0UXCQUCgsKCrS7TP0pwkOHDsXFxbGdgosyMjLUN9gCHVu8eLFSqWQ7BRd9+umnaWlpbKfgomPHjh04cEC7y9SfIiSE4JZ4AAD6rS8+5/WqCAEAAB4XihAAADiN6s+bE4ODgxUKhaOjoyYz37lzh8fjeXl59XUquE9TU1NpaemYMbi4HQt+/fXXsLAwnDike5mZmZ6enra2tmwH4Zzy8vLOzs7hw4drOP+8efPWrFnz8Hn6dRFevHixtbXVxMREk5kbGxt5PJ61tXVfp4L7dHV13b17d/DgwWwH4SKhUIh//lhRVVXl5ORkaGjIdhDOkUgkSqVS839BvLy8hg4d+vB5+nURAgAA9DXsIwQAAE5DEQIAAKehCAEAgNNQhAAAwGl6cvcJiUSSkJDA4/EiIiIsLCzYjqOHGhsbr1271tjYGBgYOGrUKGawoaEhKytLPY+/v7+DgwMzXVZWdvHiRWdn52eeeYbP57OQWF8UFhZWVVWpHz7zzDPq6dTU1MLCwqCgIH9/f/Ug/ha0JTc3t6amRv3Q2Nh48uTJhJC0tLTW1lZm0MrKauzYscy0QqFITk6ur6+fNm3aoEGDdB9YDwiFwjt37gQHB3c//r+lpSUhIYGm6YiICCsrK/V4VlZWdna2r69vaGioerC9vT0hIUEmkz377LN2dnYafl99OGpUJBKFhoaGhIR0dXXl5ORcv35dw1MPQUNFRUVjx46dNGmSk5PT6dOnV6xYsWPHDkLIuXPnFi9erP4g+Oijj8aPH08ISU5OXrRo0bx5827evOng4HDmzBmc6NZra9euTUxMVJ8jkZSUxLyYGzdujIuLCw8Pj4+Pf+edd9atW0fwt6BVW7duTUlJYaaLi4udnJzS09MJIUFBQRRFMYfv+/n5/fvf/yaEdHV1zZgxQyaT+fn5nThx4uTJkxMn4j7hj8fOzk6lUkml0gsXLkyaNIkZrK6uDgkJCQoK4vF4N27cuH79uouLCyHkX//6144dO2bPnp2UlLR48eKtW7cSQlpaWsaPHz948GA7O7vz589fuXJF09MN6YEvJiZm6dKlzHRkZOT777/Pbh79I5FIamtrmelbt24RQu7evUvT9NmzZ0NCQv48f0hIyH/+8x+aptvb293d3ZOTk3WZVs+sWbNmy5Yt9w2KRCITE5Py8nKapq9fv25ra9ve3k7TdExMzAsvvMDMg78FLQoKCtq9ezczHRgYeP78+ftmOHHihLe3t0wmo2l6165d06ZN03XEga+kpISmaXt7+8uXL6sH33nnnejoaGZ68eLFb7/9Nk3TUqnUysoqPT2dpmmhUGhqasp8Iu3atSssLEylUtE0/dprr7300ksafmt92Ed46tSpyMhIZjoyMvL06dPs5tE/lpaW6m2ezs7OFEUpFArmYXt7e1JSUnp6emdnJzNSW1ublpbG/EZMTU2fe+45/EaekEgkOnfuXGFhoXokISEhMDDQ3d2dEBISEmJmZnb16lVCyOnTp6Oioph58LegLTk5Ofn5+YsXL1aP5OfnJyUliUQi9cjp06dnz55tbGxMCImKirpw4UJbWxsLWQeyHk97P3Xq1J/f0leuXLG2tma2RXl6eo4aNSoxMZEQcvr06cjISGaTSVRUlObvf30oQpFI5Obmxky7ubl1f3eC1m3ZsmXGjBnq68golcqvvvrqr3/968iRI2/fvk0IEYvFxsbG9vb2zAz4jTwhAwODnJyc2NjYSZMmzZ8/v6urixAiEom674VSv8j4W+gL//3vf+fPn29jY8M8FAgEp06d+uyzz5566qktW7Ywg91feVdXV4qixGIxO3H1S49vaQ3f/7W1tep/0B9OHw6WUSqV6l1QfD6f+aSAvrBnz574+PjLly8zD8PDw/Py8gghKpXq5Zdf3rBhw6lTp5RKJY937x8s/Eae0K5du5ijjZqbm4ODg7/77rtVq1Z1f88TQgwMDJgXGX8LWtfZ2Xno0KHDhw+rRy5cuMD8RnJyckJCQubOnTt69Ojub3uKoiiKwouvFT2+pR/y/lf/Fvh8Pk3TKpVKk++iD2uELi4udXV1zHRNTY2rqyu7efTVDz/88PHHHycnJ6v/F1MfDsrj8RYuXMgcQers7NzR0SGVSpmnampqmJ3b0DvqF9na2nrmzJnMi+zi4lJbW6ueR/22x9+C1h07dkwgEEydOlU9ov6NjB492s/PLzs7m/zxN1JfX69UKvHia0WPb+n73v/V1dV/Hq+pqbG1tdXwUtX6UIRhYWFJSUnMdFJSUve3LGjLzz///M477yQkJAwbNqzHGTIzM5ntpa6urt7e3sxvRKVSJScnh4WF6TSrnqJpOisri9kvOGXKlPT0dIlEQgi5c+cOc7AoIWTq1KnMzhKCvwUt+e6771auXNl9I4daU1OTUChkfiNTp05NSkqiaZoQkpSUFBgY2P1Af+i1sLCwP7+lQ0NDKyoqhEIhIaS5ufnGjRvMeK+7QB9On8jLy5s4ceK6deu6urr27NmTnp7+oA9r6J28vDx/f//Jkyd7e3szI2+99Za3t/dbb73V2dnp4eFRWFh46NChY8eOhYeHE0L279+/cePGmJiYtLS0goKCzMxMXKS/16ZOnTp58mSBQJCcnFxSUpKRkcGcHbVgwYK7d+9GRUXt3bt3xowZn332GcHfgraVlZUNHz78zp07TNsRQvLz82NiYkJDQ3k83sGDBz09Pc+cOcPj8To6Ovz9/ceNG+fv779jx47Y2Fj1EXygoQ8//FAsFu/fvz/tNL+3AAAGIUlEQVQ8PNzZ2XnLli2Ojo5FRUUhISGrV6/m8Xhff/11amqqj48PIeSNN95ISUl58cUXjxw54unpefDgQUKIWCwOCAhYsmSJnZ3dzp07ExMTQ0JCNPnW+lCEhJDi4uJDhw5RFPXCCy8MGTKE7Tj6prq6+uTJk91Hnn/+eVdX11u3biUnJ9fU1Dg7O8+dO7f7/YAuXLhw/vx5R0fHFStW4F/jJ3H27NmMjAy5XD5kyJCFCxeam5sz4wqF4sCBA0VFRWPGjImKilLvMsHfghYVFRXl5ORER0erRzo6Oo4fP15YWMjj8QICAubMmaNeWWxsbNy3b19TU9PMmTNxEmEv/Pzzz01NTeqHCxcuZD46SkpKfvrpJ+b0CfV5gTRNHzlyJCsry8fHZ+nSpQYGvx/vUllZeeDAAblcHhkZqb70xyPpSRECAAD0jj7sIwQAAOg1FCEAAHAaihAAADgNRQgAAJyGIgQAAE5DEQIAAKehCAF0ITs7+8cff+yLJTc2Nu7Zs6eiokLzL9m/f//169f7IgzAQIQiBNCFo0ePrlq1qi+WXFVVtWrVqtzcXM2/JCYm5ujRo30RBmAgQhECAACnoQgBWNPW1tbc3PyQGWQyWX19ffeR1tbW9vb2hyxQfan++7S3t9fU1DzoC2marq2txZ2DgJtQhAAsePPNN93c3AQCgY2NjbOz8wcffKC+cVpra6utre3u3btffvllS0tLBweHUaNGFRcXV1VVPfPMM5aWlpaWllFRUeobXTHq6uqef/55CwsLR0fHwMDAmzdvqp+SyWQrV660trZ2dnb28vJSX8ufkZ2dPW3aNBMTEycnJzMzs5CQkBs3bujgFQDoP1CEACyQSqW7du3Kzs7OzMxctWrVRx999OWXXzJP0TTd1NS0ZcsWmqZTUlLi4+MbGhqWLVs2f/786dOnp6enf/nll/Hx8cztJtRiYmKGDBmSnZ2dkJAglUojIiIaGxuZp9atW3fgwIF//vOfeXl5H3/88UsvvdTa2qr+wrq6utDQ0MTExIKCgpMnT9I0PXPmTOYGTwBcQQNA33vvvfcEAsGDnl20aNG4ceOYaaaEJk+erH72008/JYRs2rRJPTJ37lx/f39mmln5mzZtmvrZ7OxsiqJ27NhB07RYLObz+X/729/Uz37//feEkJiYmB6TVFVVEUIOHz7cmx8SYGAyYLOEAbhKpVLFx8dnZ2fX1NTQNF1SUsLcZVSNubMjg7kN5H0jly9f7j5/97vf+fv7Dxs2jLmXfU5OjlKp7P5sZGTkihUrun+tWCw+cuRIeXk5s/fRyMiopKREGz8lwMCATaMAuiaVSoODg5cvX15QUGBsbGxjY2Nqanrf1kgbGxv1tJGR0Z9HOjs7u8/v7Ozc/aGLi0tlZSUhRCQS3fesQCCwsLBQP4yLi/Py8vr222+lUqm1tbWNjQ2Px2tpadHKTwowIGCNEEDXjh07lpWVlZubO3LkSGbk9ddfT01NfZJl3ndwaV1dHXNXUkdHR+ZZDw8P5imZTNb9QJstW7ZMmzbt7NmzzK19ZTLZjh07niQJwICDNUIAXRMKhSYmJiNGjGAednV13XckZy8kJCSop0tLS4uLi5mWHTlyJEVR3Z9NSEigu92OWygUBgUFqW9wf+7cOfXxqwAcgSIE0LWAgACZTLZ9+/aOjo7y8vIVK1aIxeInXGZycvKuXbtaWloKCwuXLl1qZma2cuVKQoinp+ecOXM+/fTTY8eOtbW1XblyJSYmhtnWqg5z8ODBmzdvymSyc+fObdiwwdDQ8AnDAAwsKEIAXZszZ87q1avfffddMzMzLy+vjo6O9evXP+Eyt23btnfvXisrK19fX6FQePz4cTc3N+apffv2jR07NjIyUiAQREREvPfee1ZWVuov3L17t4GBQUBAgKmp6QsvvLBjx47uexABuIDqvpEEAPoIc5Q2j3fvX8/a2try8vJBgwa5uLho61sUFha2t7ePGjWq+zofo6SkpLm52cfHRyAQ3PeUQqG4fft2Z2enr6+vsbGxVsIADCAoQgAA4DRsGgUAAE5DEQIAAKehCAEAgNNQhAAAwGkoQgAA4DQUIQAAcNr/AQxIAxS+onKHAAAAAElFTkSuQmCC",
"image/svg+xml": [
"\n",
"\n"
],
"text/html": [
"\n",
"\n"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"using Convex, MathOptInterface\n",
"using Mosek, MosekTools\n",
"\n",
"const MOI = MathOptInterface\n",
"\n",
"# solve at a grid of λ\n",
"λgrid = 0:20:1000\n",
"ridgepath = zeros(length(λgrid), size(X, 2)) # each row is β̂ at a λ\n",
"β̂ridge = Variable(size(X, 2))\n",
"@time for i in 1:length(λgrid)\n",
" λ = λgrid[i]\n",
" # objective\n",
" problem = minimize(sumsquares(y - X * β̂ridge) + λ * sumsquares(β̂ridge[2:end]))\n",
" solver = Mosek.Optimizer() # MOSEK this time!\n",
" MOI.set(solver, MOI.RawOptimizerAttribute(\"LOG\"), 0) # keep silent \n",
" solve!(problem, solver)\n",
" ridgepath[i, :] = β̂ridge.value\n",
"end\n",
"\n",
"plt = plot(collect(λgrid), ridgepath, legend=:none)\n",
"xlabel!(plt, \"lambda\")\n",
"ylabel!(plt, \"beta_hat\")\n",
"title!(plt, \"Ridge\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Example 2: constrained least squares\n",
"\n",
"* Least squares with linear constraints. For example, _nonnegative least squares_ (NLS)\n",
"$$\n",
"\\begin{array}{ll}\n",
"\t&\\text{minimize}& \\frac 12 \\|\\mathbf{y} - \\beta_0\\mathbf{1} - \\mathbf{X} \\beta\\|_2^2 \\\\\n",
"\t&\\text{subject to}& \\beta \\geq \\mathbf{0}.\n",
"\\end{array} \n",
"$$"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"------------------------------------------------------------------\n",
" COSMO v0.8.8 - A Quadratic Objective Conic Solver\n",
" Michael Garstka\n",
" University of Oxford, 2017 - 2022\n",
"------------------------------------------------------------------\n",
"\n",
"Problem: x ∈ R^{14},\n",
" constraints: A ∈ R^{116x14} (1116 nnz),\n",
" matrix size to factor: 130x130,\n",
" Floating-point precision: Float64\n",
"Sets: SecondOrderCone of dim: 101\n",
" Nonnegatives of dim: 11\n",
" SecondOrderCone of dim: 3\n",
" ZeroSet of dim: 1\n",
"Settings: ϵ_abs = 1.0e-05, ϵ_rel = 1.0e-05,\n",
" ϵ_prim_inf = 1.0e-04, ϵ_dual_inf = 1.0e-04,\n",
" ρ = 0.1, σ = 1e-06, α = 1.6,\n",
" max_iter = 5000,\n",
" scaling iter = 10 (on),\n",
" check termination every 25 iter,\n",
" check infeasibility every 40 iter,\n",
" KKT system solver: QDLDL\n",
"Acc: Anderson Type2{QRDecomp},\n",
" Memory size = 15, RestartedMemory,\t\n",
" Safeguarded: true, tol: 2.0\n",
"Setup Time: 0.51ms\n",
"\n",
"Iter:\tObjective:\tPrimal Res:\tDual Res:\tRho:\n",
"1\t-2.4722e+00\t7.6016e+00\t6.0000e-01\t1.0000e-01\n",
"25\t 6.3747e+01\t2.0829e+00\t8.8559e-01\t1.0000e-01\n",
"50\t 2.1884e+02\t9.9949e-01\t4.4922e-02\t1.0000e-01\n",
"75\t 2.1529e+02\t8.2760e-01\t1.2783e-02\t1.0000e-01\n",
"100\t 4.4454e+02\t7.3353e-01\t3.9757e-02\t8.2846e-03\n",
"125\t 3.6790e+02\t3.2831e-01\t8.5896e-01\t1.1684e-03\n",
"150\t 3.9757e+02\t4.0840e-01\t4.2608e-04\t1.1684e-03\n",
"175\t 3.8432e+02\t3.5502e-01\t2.1204e-05\t1.1684e-03\n",
"200\t 3.8269e+02\t3.4818e-01\t5.6533e+00\t7.7739e-03\n",
"225\t 3.7135e+02\t2.8675e-01\t1.4084e-03\t7.7739e-03\n",
"250\t 3.8831e+02\t2.7539e-01\t9.8296e-05\t7.7739e-03\n",
"275\t 4.0925e+02\t1.8601e-01\t3.1028e-04\t7.7739e-03\n",
"300\t 4.2158e+02\t2.0524e-01\t3.1091e-04\t7.7739e-03\n",
"325\t 4.2877e+02\t1.8994e-01\t3.4531e-05\t7.7739e-03\n",
"350\t 4.3495e+02\t1.4451e-01\t6.2244e-04\t7.7739e-03\n",
"375\t 4.3518e+02\t1.6097e-01\t2.8631e-04\t7.7739e-03\n",
"400\t 4.4218e+02\t1.6236e-01\t3.0478e-04\t7.7739e-03\n",
"425\t 4.6533e+02\t1.5223e-01\t5.9527e-04\t7.7739e-03\n",
"450\t 4.8040e+02\t1.1258e-01\t8.7138e-05\t7.7739e-03\n",
"475\t 4.7379e+02\t9.0149e-02\t8.9405e-05\t7.7739e-03\n",
"500\t 4.7549e+02\t8.7159e-02\t4.2263e-05\t7.7739e-03\n",
"525\t 4.8306e+02\t7.6308e-02\t4.9924e-06\t7.7739e-03\n",
"550\t 4.8484e+02\t7.3429e-02\t7.9817e-05\t7.7739e-03\n",
"575\t 5.0510e+02\t1.0148e-01\t3.5214e-03\t7.7739e-03\n",
"600\t 5.0488e+02\t5.2970e-02\t2.7751e-04\t7.7739e-03\n",
"625\t 5.1273e+02\t2.1994e-02\t2.1870e-04\t7.7739e-03\n",
"650\t 5.1248e+02\t1.8134e-02\t4.3111e-05\t7.7739e-03\n",
"675\t 5.1440e+02\t1.2543e-02\t1.5382e-05\t7.7739e-03\n",
"700\t 5.1657e+02\t1.4869e-02\t6.1672e-05\t7.7739e-03\n",
"725\t 5.1786e+02\t1.0807e-02\t1.0758e-05\t7.7739e-03\n",
"750\t 5.1774e+02\t1.0375e-02\t6.7655e-06\t7.7739e-03\n",
"775\t 5.1798e+02\t1.0285e-02\t1.2674e-05\t7.7739e-03\n",
"\n",
"------------------------------------------------------------------\n",
">>> Results\n",
"Status: \u001b[32mSolved\u001b[39m\n",
"Iterations: 831 (incl. 56 safeguarding iter)\n",
"Optimal objective: 518\n",
"Runtime: 0.019s (19.35ms)\n",
"\n",
" 0.037223 seconds (24.25 k allocations: 5.272 MiB, 22.73% compilation time)\n"
]
}
],
"source": [
"# Use COSMO solver\n",
"using COSMO\n",
"solver = COSMO.Optimizer()\n",
"## Use SCS solver\n",
"# using SCS\n",
"# solver = SCS.Optimizer()\n",
"## Use Mosek solver\n",
"# using Mosek, MosekTools\n",
"# solver = Mosek.Optimizer()\n",
"## Use Hypatia solver\n",
"# using Hypatia\n",
"# solver = Hypatia.Optimizer()\n",
"\n",
"\n",
"# Set up optimizaiton problem\n",
"β̂nonneg = Variable(size(X, 2))\n",
"problem = minimize(0.5sumsquares(y - X * β̂nonneg)) \n",
"problem.constraints += β̂nonneg[2:end] >= 0\n",
"\n",
"# Solve the problem\n",
"@time solve!(problem, solver)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"tags": []
},
"outputs": [
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAGQCAIAAAD9V4nPAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nO3deVyU5f7/8WuGHYEDoiigoKGouBwX3DXUo1mmdVKOpg/NPJ6yXMq0sjxZHS2Xr0taVi7lmo8y0zSXzF3BJZdQzAVKQVAQJJB9hmFmfn/c5zu/+YILNsPcMtfr+dfMNRfX/Znpbt7ey3WNxmw2CwAAZKVVuwAAANREEALO7+zZs2PHjt24caPahdxBYWFhQkLC0aNHk5KSiouL1S4HMiIIUSPNmjVLo9FoNJp33nmnwks6nU6j0fj6+qpSmLrOnj27YsWKS5cuVWi/du3aqlWrfv75Z4dV0rZtW3d39zNnztyjT1JS0qBBg/z9/du3b9+jR4/mzZv7+fm1b99+wYIFDqsTEAQharrFixffvHlT7SoeFj/++OO4ceOOHDlSob1u3bq9evVq2rSpwyopKyszGAwmk+luHZKSkrp27bpjx47mzZu//fbbH3300bRp0wYOHHjp0qXNmzc7rE5ACOGqdgHAnxcQEJCXl/fhhx9+8sknatfyUOvWrdvBgwfVruL/eO+99/Ly8p577rk1a9ZoNBpL++3bt48fP65iYZAQQYgabOzYsevWrVu+fPnkyZMjIiLu2//8+fPHjx8vKCgICQnp169f3bp1rV/Nzs5OT09v2LBhUFDQpUuXDh8+rNPp2rRp07t3b+tv6vz8/N9//71u3bphYWHp6el79uzJz8+PjIzs37+/m5tb5Y1mZmYeOHAgMzPTz8+vZ8+eLVq0qNwnPz9/9+7d169fr1+//oABAwICAi5cuKDT6dq3b2+96atXryYkJFy/fl2r1TZv3jwmJsbd3d3y6oULFzIyMoQQaWlplnOSzZo18/HxUWquV69egwYNlJ46na5Vq1YeHh4VKvn111/1en3r1q2tR05OTo6Pj8/JyQkODu7Tp09oaOh9P+r7iouLE0JMnjzZ+g0KIfz9/Z944gnbxwcegBmogWbOnCmEeP/99xctWiSEGDVqlOWl0tJSIYSPj491/7y8vIEDB1rv+V5eXv/zP/9j3eejjz4SQixatGjixInWPf/2t78VFxdbuu3YsUMIMWHChPnz57u6/v9/SrZq1SozM9N6QIPBMHny5ArpOGLEiJKSEutue/bsqVOnjqWDr6/v1q1bIyMjhRDl5eVKn7KyssoJGh4efvz4ccs4d4zYw4cPm83mrVu3CiFee+01peeYMWOEEBs2bKjwqd68edPV1bVBgwaW7ebm5v7973+3HtDNze3dd981mUz3/g+kFHPy5Mm7dQgPDxdCnDhx4t7jAA5AEKJGsgShXq9v3LixVqtNSEhQXqochEajMSYmRgjRp0+fI0eO/Pbbb+vXr69Xr54Q4uOPP7Z0U4KwcePGISEhX3755enTp7du3dqsWTMhxLvvvmvppgRho0aN/Pz8Fi5c+PPPP//000/du3cXQgwfPty6yNGjRwshOnbsuHXr1suXL+/fv79fv35CiJEjR1r6/Pbbb97e3m5ubnPnzr1y5UpSUtK7777r4+Pj7+9vHYQ6na5ly5YLFy48dOhQcnLysWPHpkyZ4urqWrdu3dzcXKXPsWPHxo4dK4SYPHny3v+Vl5dnrhSEhw4dEkI89thjFT7V+fPnCyHeeecdy0ajo6OFEM8888y+ffsuX768bdu2qKgoIcTChQvv/R/ovkGo5GunTp3Onj1776GA6kYQokayBKHZbP7yyy+FEIMGDVJeqhyEW7ZsEUI0bdpUp9NZGo8dOyaE8Pf3LywsVFqUIPTy8kpJSbF0S0xM1Gg0UVFRlhYlCDUaTVxcnKUxJyfHx8fH09PTaDQqLUrYtG3bVq/XW7qVl5crZzsTExOVlueff97yRiwmT56sHH5ZgvCOZsyYIYRYunSppWX27NlCiGXLllXoWSEITSZTRESEVqtNS0uz7tamTRshxOXLl5WnytG2dWybzeZbt27Vrl3b39+/qKjoHrXdNwgTEhL+8pe/KG+zYcOGw4cPX7p0aWpq6j3GBKoJd42ixhs9enRUVNT27dsPHz58xw5KEE6ZMsX6kljXrl179ep1+/bt/fv3W3ceOnRoo0aNLE9bt24dHBysfEFbd+vatWuPHj0sTwMDAzt06KDT6ZSrdEKI9evXCyGmTZtmfbHNxcVl3LhxZrP5xx9/FEKYzeYdO3a4uLhMmjTJevDXXnutKm/8qaeeEkKcPHmyKp2taTSaUaNGmUymDRs2WBrPnDmTmJjYo0cP5SDY8hYqTFCpU6fO0KFDbb+lpW3btpcuXRo7dmydOnXS09O//vrriRMnNmrUqG/fvikpKbaMDDwobpZBjefi4jJr1qwhQ4a88847yi0YFVy8eFEI0a5duwrtHTp0OHTo0MWLF59++mlLo3Jxzlq9evUyMjIKCgosRzBCCEtaWHcTQmRlZSk3pCQkJAgh9u3bd/bsWetuyrd8amqqECI3NzcnJ6dBgwa1a9e27hMWFubr61tYWFjhD+fNmxcfH3/jxo3bt29b2nNyciq/5ft6/vnnZ86cuXr16rfeektpWbt2rRBCOZ0rhDAajYmJiS4uLqtWrapwP4syT1F5C7YIDg7+4osvli9ffubMmZMnT+7du/fHH3/cv39/TEzMuXPnAgICbBwfqCKCEM5g8ODBXbp0iY+P37VrV58+fSq8WlRUJIQICgqq0K5EV4W88fb2rtBNq9UKISocEd6tm2XmnJJVmzdvrpAiQoiAgAClUTmLq1wOrNzHurDLly9369YtPz+/R48eym2lWq02JydnwYIFRqOx8p/fV3h4eExMzMGDB0+ePNmpUyeDwfDNN994eXnFxsYqHQoLC41Go1arXbly5R3Lu8ccwQfi4uLSqVOnTp06TZw4MTExMSYmJj09fdWqVVOnTrXL+MB9EYRwEnPnzu3Vq9dbb7114sSJCi8pq8xkZWU1btzYul2Zie/n51cd9fj4+AghDh8+rFx4uyMlES1nUy1MJlNmZqZ1ywcffJCXl/fZZ5+9/PLLlsb4+HhbFmEZPXr0wYMH165d26lTp507d966dWvkyJGWVK5Vq5ZGo/Hw8MjKyrrjtJDq0KZNmzFjxnz00Uf3XpIGsC+uEcJJxMTE9O3b9/z585VX1GzVqpUQ4vTp0xXaT506ZXnV7pQzscotOXdTq1atZs2a5ebmJiYmWrcfPXrUYDBYt5w7d04IMXz4cOvGymmhXI+s4jFibGysr6/vN998o9frK5wXFUK4ubm1bt26tLRUOcfrMMp1XOXwGnAM9jY4j9mzZ2s0mvfff79Cu3K6b8mSJcqpSEVcXFxcXFzt2rUrn0q1i3/+859CiPnz5+fm5lZ4yWAw6HQ65fGoUaOEEP/+978tZxoNBoNyO6g1ZaJhWlqapaWwsFCZ7WAtJCRECJGenl6VCmvVqhUbG5ubm7tmzZpdu3Y1aNCgd+/e1h2UO1pnzJhRXl5e4W+Vs822WL9+vfV/DkV+fv53330nhOjQoYON4wNVx6lROI+OHTsOGTJE+Sa19uSTT/br12/v3r39+/d/5513QkNDjx079vbbbwsh5syZU/lqn108+uijL7/88ueffx4dHT1lypS2bdvWqlUrNTX1xIkT69at27Vrl3LI+Nprr23cuHHHjh1dunQZMmSI0WjcuHGjwWBo0KDB9evXLdcXe/fufejQoREjRsybN69p06aXL19+7733Kq8L06FDB61Wu2zZMo1GExYWptVqBw0aFBwcfLciR48evXr16ilTppSVlY0ePdrFxcX61QkTJmzZsmXPnj09evQYP358VFSU0WhMSUnZv3//xo0bCwoK7vshzJ8/v/Kl2V69esXGxk6fPv3VV18dPHhwz549w8PDi4uLL126tGzZsitXroSFhSkTIgEHUXf2BvDnWM8jtJaUlKSs9lJhZZmCgoKhQ4da37fi6+trPQPP/L/zCD/66KMKYypHJ8rMdLPVyjIVuj377LPi/86cMxqNc+bMqXANUqPRdO7c2XoC361bt0aMGKGc1XRxcXn66adv3LhRr149Ly8vS5/S0tIKK+N07949Pj5eCNG/f3/rMj777LP69etbut1xZRkLZUKh0tMyfdBaUVHRuHHjKlwjdHd3r7DRyu64zI1i0qRJZrP5zTfftJ6motBqtU888YT1PE7AATRmfqEeNVBeXl5eXl5AQEDlm+zT09MNBoNWq638PXvlypXjx48XFRWFhITExMRYT4cQQuTn5//xxx+BgYEV2m/cuKHX6xs1aqRcuCopKbl586afn5/1umhCiOzs7KKiotDQ0AoHasXFxceOHUtNTXV1da1fv37btm3veIhWWlqanZ0dFBTk5eWVn5/v7+/fvHnzCj+odPbs2cTERJPJ1LJly+jo6PLy8vT0dC8vr8oDFhYW3rp1SwgREhLi6elZXFyckZEREBBQoWZL2Xf8uCxycnKOHTuWkZFRq1at0NDQDh06VPiIKktLS9Pr9Xd8yd/f37LK62+//ZacnJyZmWkymerXrx8dHa2c3QUciSAEHjoLFix44403xo8f/+mnn6pdC+D8CEJAZS+++GKLFi26d+9ev379tLS07du3L1y40NPT8+zZs02aNFG7OsD5cbMMoLK0tLQKk9bDw8PXrFlDCgKOwREhoDKTyXTu3LlLly7dunXL09OzRYsWXbt2ddgcdgAEIQBAakyoBwBIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASM2hQRgXF3fgwAFHbtGRKv9UDezCaDQyyaeasNNWE5PJZPldLdhXdey0Dg3Cw4cPHzx40JFbdKTKP64GuzAYDHxfVxN22mpiNBor/LQy7KU6dlpOjQIApEYQAgCkRhACAKRGEAIApEYQAgCkRhACAKRGEAIApEYQAgCkRhDKaO8N86RjRrWrAICHAkEoozy9OVundhEA8HAgCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAFIjCGWkLS+rXZKtdhUA8FAgCGUUkHrqyUur1a4CAB4KBKGMNCaTEGa1qwCAhwJBCACQmt2C0GAw5OXlmUwmew0IAIAD2CEIs7OzO3fu7OvrW7t27czMTNsHBADAYewQhF5eXjNnzrx48aLtQwEA4GCutg/h6+vbv3//goIC24cCAMDB7BCEVVdSUpKWlrZv3z7lqVarffTRR11dHVoDAADWHBpC169fP3r0aEZGhvJUo9GEhoaGhoY6sobqU1xcrNFo1K6iSgyGMrPJXFRUpHYhVaLT6VxcXNzc3NQuxAnVoJ22ZjEYDCaTyWAwqF2IE3rQndbb21urvc9FQIcGYWRkZHh4+KxZsxy5UYcxm80+Pj5qV1Elbm7uBq2mplTr6upKEFaTGrTT1ixKEHp4eKhdiBOqjp2WeYQAAKnZ54hw1apVynm2DRs2+Pv7/+tf/7rvoSgAAA8D+wRhSkqKwWCYNm1abm5ubm6uXcYEAMAB7BOEznrZDwDg9DiBCQCQGkEIAJAaQQgAkBpBCACQGkEIAJAaQQgAkBpBCACQGkEIAJAaQQgAkBpBCACQGkEIAJAaQQgAkBpBCACQGkEI2FPu2jnlt26oXQWAB0AQAvZUnptlKilUuwoAD4AgBABIjSAEAEiNIAQASI0gBOyp1CiMJrWLAPAgCELAnq4WmJMKzGpXAeABEISAnZnJQaBGIQgBAFIjCAEAUiMIAQBSIwgBADWG2VBm9zEJQvswG8r08T+oXQUAOLmijyebSorsOyZBaB+m4vyyEz+qXQUAOLtyg9losO+QBCEAQGoEIQBAagQhAEBqBCEAQGoEIQBAagQhAEBqBCEAQGoEIQBAagQhAEBqBCEAQGoEIQBAagQhIC9j2mW1SwDURxACsjKZir58X+0iAPURhIDMzGoXAKiPIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBACplSYeLc/NUrsKNbmqXQAAQE0lp/d7C41r7XpqF6IajggBAFIjCAHAzky6ElNJodpVoKoIQgCws9Kffyo9slXtKlBVBCEA2JvJJMwmtYtAVRGEAGoAc7nBcOOq2lXAORGEAGoAQ3ry7c2fqV0FnBNBCKAGMJuFEGa1q4BzIggBAFKzWxDm5ub+8ccf9hoNAADHsEMQGgyG4cOHR0ZGtmjR4plnntHpdLaPCQCAY9ghCNeuXXvp0qX09PTr169nZ2d//vnnto8JAIBj2CEIN2zY8MILL3h5ebm7u48bN27Dhg22jwkAgGPYYdHtlJSUyMhI5XFkZGRKSsrdepaXl+fl5V29+t/JQBqNplGjRhqN5m79c/OKagf42F6hY5i4o616lJTo3T3d3dzc1C4EKiuvOf+LGc2ivObMpzeYlJtya4bq+Ka1QxAWFhZ6eXkpj2vVqlVQUGA2m+8Yb0lJSbt37965c6fy1N3dfcuWLWFhYXcbecfSJd3Hvhzk6257kdUtq8gc59YytrBmrC7o+Rff/NoNC2tItTs373APCBzQv7vahVTJ7ToR9Tw9asZnazafD2z/lxpRqhA39O5JLo371JBqz5QHlhk9Y2pItSc1YSFl3s1qSLUJvlFeeqO7qGq13t7eLi4u9+6jMdv8L4EWLVrMnz9/4MCBQogjR44MHz78xo0bd+z5wQcf6PX6WbNmVXHksK/Ljz7l0rDWXQ8ZHx7pxeZu28rTR3DUYn+T4vWNfLVT/8pna3+FhYW+vr5qV+GEDAaDyWTy8PBQuxAnVB07rR2uEbZu3frUqVPK45MnT7Zp08b2MQEAcAw7nBodP358bGxsz5493d3dFy5cuHLlStvHBADAMewQhL169frkk09mzZplMplmz56tnCMFAKBGsEMQCiGGDx8+fPhwuwwFAIAjsdYoAEBqBCEAQGoEIQBAagQhAEBqBCEAQGoEIQBAagQhAEBqBCEAQGoEIQBAagQhAEBqBCEAQGoEIQBAagShfXhoNbVcbf2JYwCA4xGE9hHkJY7216tdBQDggRGEduPOZwkANRBf3gAAqRGEAACpEYQAAKkRhAAAqRGEAACpEYQAAKkRhAAAqRGEAACpEYQAAKkRhAAAqRGEAACpEYQAAKkRhAAAqRGEAACpEYQAAKkRhAAAqRGEAACpEYQAAKkRhAAAqRGEAACpEYQAAKkRhAAAqRGEAACpEYQAAKkRhAAAqRGEAACpEYQAAKkRhAAAqRGEAACpEYQAAKkRhAAAqRGEAACpEYQAAKkRhAAAqRGEAACpEYQAAKkRhAAAqRGEAACpEYQAAKkRhAAAqRGEAACpEYQAAKkRhAAAqRGEAACpEYQAAKkRhAAAqRGEAACpEYQAAKk91EE4va02yFOjdhUAAGfmqnYB9/JSi4c6pwEATsAOSZOfnz9hwoTOnTtHRERkZWXZPiAAAA5jhyNCg8EQGhr6t7/9bciQIeXl5bYPCACAw9ghCOvUqTN9+vSCggLbhwIAwMEceo3QbDbrdLq8vDzlqYeHh7e3tyMLAACggioFYXFx8YoVKyq3P/XUUxEREVXf2Pnz57dv3/7FF18oT7Va7ZEjR8LDw6s+wsOsuLhYo+EeV/szGIReL4qK9GoX4oTYaauJwWAwmUwGg0HtQpzQg+603t7eWu197oapUhAajcaMjIzK7TqdrurVCCHatGnTrFmzWbNmPdBf1RRms9nHx0ftKpyQm5vew0Pr4+OmdiFOiJ22mihB6OHhoXYhTqg6dtoqBaGfn9/8+fPtu2EAAB4G9rlGmJCQUFhYKIRITEy8efNm+/btOd8CAKgR7BOEM2bM0Ov1ffv2XbRokRBi9+7dLi4udhkZAIBqZZ8g3LFjh13GAQDAwVjDDAAgNYIQACA1ghAAIDWCEAAgNYIQACA1ghAAIDWCEAAgNYIQACA1ghAAIDWCEAAgNYIQACA1ghAAIDWCEAAgNYIQACA1ghAAIDWCEAAgNYIQACA1ghAAIDWCEAAgNYIQACA1ghAAIDWCEAAgNYIQACA1ghAAIDWCEAAgNYIQACA1ghAAIDWCEAAgNYIQACA1ghAAIDWCEAAgNYIQACA1ghAAIDWCEAAgNYIQACA1ghAAIDWCEAAgNYIQACA1ghAAIDWCEAAgNYIQACA1ghAAIDWCEAAgNYIQACA1ghAAIDWCEAAgNYIQACA1ghAAIDWCEAAgNYIQACA1ghAAIDWCEAAgNYIQACA1ghAAIDWCEAAgNYIQACA1ghAAIDWCEAAgNYIQACA1ghAAIDWCEAAgNYIQACA1ghAAIDWCEAAgNYIQACA1OwTh+fPnhw8f3qRJk4iIiH/96185OTm2jwkAgGPYIQhTU1N79uy5e/fuvXv33rhx44UXXrB9TAAAHMPV9iEGDRpkefz6668PHz7c9jEBAHAMO18jPHLkSNu2be07JgAA1adKR4Rms3njxo2V2zt27BgREWF5Gh8fv2TJkvj4+LuNk5iYuH379qVLlypPtVrt4cOHw8PDH7Dmh1RRUZHaJTgng0Gj14vCQp3ahTghdtpqYjAYTCZTWVmZ2oU4oQfdab29vV1cXO7dp6pBuGPHjsrt9erVswTh6dOnhwwZsnHjxlatWt1tnNatW4eHh0+fPl156u7uXqtWraoUUFP4+vqqXYITcnPTe3hofX3d1C7EObHTVgclCD08PNQuxDnZfaetUhBqtdqvvvrqHh3Onj07aNCglStX9u/f/x7dNBqNp6dnQEDAg9UIAEC1scM1wgsXLvTr12/KlCmtWrW6evXq1atXzWaz7cMCAOAAdrhrNC4uzs/Pb9myZcuWLVNakpOT73tOFgCAh4EdgvCll1566aWXbB8HAADHY4k1AIDUCEIAgNQIQgCA1AhCAIDUCEIAgNQIQgCA1AhCAIDUCEIAgNQIQgCA1AhCAIDUCEIAgNQIQgCA1AhCAIDUCEIAgNQIQgCA1AhCAIDUCEIAgNQIQgCA1AhCAIDUCEIAgNQIQgCA1AhCAIDUCEIAgNQIQgCA1AhCAIDUCEIAgNQIQgCA1AhCAIDUCEIAgNQIQgCA1AhCAIDUCEIAgNRc1S4AAJxcamqqyWRSu4oar3HjxhqNpjpGJggBoBr98ssv3bt3DwkJUbuQmi0rK+vbb78dMGBAdQxOEAJANdLr9e3atTt27JjahdRsgwcP1uv11TQ41wgBAFIjCAEAUiMIAQBSIwgBAFIjCAEAUiMIAQBSIwgBAPaxc+fO27dvq13FAyMIAUBGo0ePvnXrln3HnDBhQkpKSlV66vX6oUOHGgwG+xbw5xCEACCjrVu3FhcXq7X18vLyTZs2PSQrzxGEACCdiRMnFhcXP/XUU9HR0QcPHjxw4MALL7zwyiuvBAQELFq06LPPPlu8eLHSMzU19bHHHlMep6enDx48uGHDhpGRkcuXL7/jyPv27WvZsmVwcPCkSZPKysqUxo0bN7Zp0yY0NLRv377JyclCiDFjxgghunXrFh0dff78+fPnz/fo0SMkJCQ8PPy1114rLy+v9o/ACkusAYCjXb5tLnbgV32En8bf/f+0LF26dP369T/88EOjRo2EEJs3b169evXy5csXLVpkMBg+/PBDy3pmOp0uMTFRCGE0Gp966qmxY8du3rw5MzOzd+/ezZs3j4mJqbCtH3744fjx40KIJ554YsmSJW+88cbhw4fffvvtPXv2NGnSZO3atf/4xz8SEhJWr169adOmY8eOeXh4CCGuXbu2evXqpk2b5ufnDxkyZNWqVS+++GK1fy7/iyAEAIfSG8WYI0aDA08KDo/QTm19n/N/ERERY8eOFUK4ut45F06fPn3r1q0BAwYoVwEHDhy4ffv2ykE4ZcoUPz8/IcTrr78+Z86cN95448svvxw6dKhWq7169WrPnj3ffPPNa9euBQUFWf9VeHh4cnLyunXrsrKyfH19jx49ShACgNPycBHHn3rovnuDg4Pv3eHGjRulpaVvvfWWpaVnz573GCc4ODgrK0sIkZGRkZSUdPXqVaU9Jiam8j0y33zzzfTp00ePHh0cHOzn55efn/+n38if8ND9xwAAOICLi4v1vSrWP/Xn4+NjuaE0LS1NefDII49oNJr169crJzPv5vfff+/SpYsQ4rfffgsLCxNCNG7cODAwcO7cudbddDqdEMJSwJo1a+bMmTNs2DAhxNWrVx0chNwsAwAyatq06VdffbVv377s7OwKL3Xt2nXbtm1Hjx79+eef58yZozS2bds2Ojp6zJgxv/7665UrVzZt2hQXF1d52IULF545c+b06dMffPDBCy+8IISYPHnyypUr161bd+3atbNnz3744YdCCE9Pz4YNG3755Zf79u3Lz88PCQnZsWNHenr61q1b161bV81vvSKCEABktGbNmry8vJUrV6ampjZo0MD6al9MTMz06dPfeuutuXPnzp49e9CgQUr7999/HxUVNXny5NGjR+/du7fCdT4hxJNPPvnhhx/OnDnz9ddfnzx58nPPPSeEaNmy5aFDh/bv3z9s2LBp06ZZrkFu3rw5KSlpxYoVWVlZc+bMcXFxGTJkyO7duz/99NNu3bo55DP4L43ZbHbYxj744AO9Xj9r1iyHbdGRCgsLfX191a7CCU2K1zfy1U79q5vahTghdtpqYjAYTCaTcgrx+PHjU6dO5Yd5bTR48OBRo0Y988wz1bHTckQIAJAaQQgAkBp3jQKApOLj45OSkrp165aWltakSZOIiAgbB1QmyHfo0MEu5TkMR4QAIKOlS5dOmDAhIyOjrKxs+fLlp0+fFkJ8//338+bNs/SZOHHiqVOnqj7mt99+u2PHDvvXWs04IgQAGR04cGDy5MnKmp9btmxRGtPS0i5cuGDpc+rUqYEDB6pTnwNxRAgA0pk4ceKBAwdmzpwZHR2dlZU1duzYH3744ddff124cOGuXbuio6OHDRu2YMGCCxcuvPLKK9HR0atXrxZCHDp0qH379g0aNIiKivrpp5+UoQg4eoQAAAn/SURBVBITE9u1a1e/fv1+/frl5OSo+rb+JI4IAcDRDJmp5nLH/RSfa0CQ1ucv1i1Lly5NSUl59tlnR40aJYTIzMwsLCxs1arV1KlTz5w5Y5nSvmnTpv/85z+PP/64ECItLe3ZZ5/dvn17x44dz507179//3PnztWpU2fo0KFTpkx58cUXz5w506NHD+s12GoKghAAHMpsKMv79hNRXuawLXpH9/GJecbGQb777ru2bdtqtdozZ84IIZo0aRIXFxcZGZmXl6esINOhQ4f+/fvboVyHIwgBwKE0bu5Bry5Uu4oHlpmZmZGRsWLFCuVpy5Yt69Spk5OTU6dOHcs6pfXr11evwD+PIAQA/Jerq6vRaLR+alkXOyIiom7duhV+j/fKlSvp6emlpaVeXl5CiEuXLt33VyweQtwsAwD4r+bNmx8/fnzbtm0nTpwQQjRr1uzrr7/+6aefUlNTR44cmZKS8u9///v8+fOnTp2aP39+WlpaRERE165dJ06c+Ouvvy5ZsiQhIUHtd/BnEIQAIKPHH3+8adOmyuP+/fs3adJECNGnT58ZM2bs3btXuSl03rx5kZGRW7ZsuXTpko+Pz6lTp8rLy6dOnfqf//ynpKTE399fCLFp0yZfX99XXnnl9u3bn3/+eY2bTS84NQoAcpo0aZLl8auvvqo80Gg0Y8aMUSYXCiHq1q07Y8YMS7fAwEDr6fYKPz+/xYsXV3Ox1csOR4SnTp0aNmxY586dH330UeX3JWwfEwAAx7DDEaHJZHrmmWdatGiRm5s7derU27dvL1iwwPZhAQBwADsEYefOnTt37qw8Hjdu3Pr1620fEwAAx7DPNcKysrLr16/fvHlz9erVI0eOtMuYAAA4QJWCMD8//9y5c5Xbo6Ojvb29hRDXrl0bNmxYWlpa8+bNY2Nj7zbOhQsXdu/e/dVXXylPPTw8tmzZEhYW9qcqf+gUFxdbZpXCjgwGodeLoiKuPdsfO201UX6h3mAwCCFKS0vNZrPaFdV4ZrNZp9MVFRU96E7r7e2t1d7nbpgqBWFKSsr7779fuX316tXh4eFCiKZNm/7yyy8mk2natGmxsbHx8fF3HKdZs2aBgYFTpkxRnmo0msaNG1elgBrBbDb7+PioXYUTcnPTe3hofXzc1C7ECbHTVhMlCD08PIQQXl5e/GvDdhqNxtPT08fHpzp22ioFYdu2bQ8cOHDfblqt9tlnn126dOldN+bqGhAQ8MgjjzxAgQBQw+Xn5+/bt0/tKmq27Ozs6hvcDtcIExISoqKiPDw89Hr9F1980bFjR9vHBADn0KBBg9DQ0MrT7/BAvLy8lCn/1cEOQfjNN9989tlngYGBt27d6tKly5o1a2wfEwCcQ8OGDffs2aN2FbgXOwThvHnzZs6cmZOTExgY6OnpafuAgLURj4gAdisA1cY+0yc8PDxCQ0PtMhRQQbvaZhcXtYsA4LxYdBsAIDWCEAAgNYLQPnQ63ccff6x2Fc5p7969Z86cUbsK5zR//ny1S3BOP//8c1WmnOFPWLx4cVlZmX3HJAjt49atW8uWLVO7Cue0Z8+eo0ePql2FEzIajbNnz1a7CucUFxe3f/9+tatwTkuXLs3Ly7PvmAQhAEBqBCEAQGoEIQBAahpHLos+atSonTt3BgQEOGyLDmM0GjMyMho2bKh2IU7ojz/+cHNz8/PzU7sQJ5SamtqoUSO1q3BC+fn5RqOxdu3aahfihNLS0ho0aHDfH5SwGDFixKxZs+7dx6FBWFpaev36dRcnnR2t1+uVxeZhX+Xl5RqNxll3G3Wx01YTo9FoNptdXe2zYgmsPehOGxwc7OXlde8+Dg1CAAAeNlwjBABIjSAEAEiNIAQASI0gBABIjZuabFVeXn7w4MGjR4+WlpZ27dr16aef1mg0ahflbH744YfS0tJhw4apXYhT+eWXX7Zs2aLT6bp06RIbG6t2Oc4jPj5+9+7dJpOpd+/e/fr1U7ucmu3mzZunT5/OyMjo169f48aNLe2pqalr1qwpKSkZOnRodHS0jVvhiNBWu3fvfvPNN00mU1BQ0NSpU8ePH692Rc4mPj5+9OjRM2bMULsQp7J27drHHnvMbDaHhoby++l2tHHjxr///e916tQJDQ19/vnnP/30U7Urqtm6d+8+Z86cN99803rl/czMzI4dOxYVFQUFBfXt2zc+Pt7WzZhhm9LSUsvjkydPurm56XQ6FetxMjqdrl27dnPnzm3atKnatTiP3NxcHx+f48ePq12IE4qNjX3vvfeUx5988kmvXr1ULafGU2ZktmrVatOmTZbG9957LzY2Vnk8Z86cQYMG2bgVjght5enpaXms0+k8PT3d3NxUrMfJzJgxY9iwYc2bN1e7EKdy5MiRsLAwPz+/JUuWfP3113b/URuZRUVFJSYmKl/fCQkJLVu2VLuimu2OK8gcPnzYcs65X79+hw8ftnUrNv49LMrKyqZOnfr6669Xfe0f3NvJkyf37ds3ZcoUtQtxNikpKfn5+WPHji0pKVm1atWjjz5aXl6udlFOYsaMGSaTqWHDhmFhYVeuXOEXH6tDZmZm3bp1lcdBQUEFBQXFxcW2DMhXtn0YjcbnnnsuJCRk+vTpatfiJMrKysaNG7dixQqOsO1Oq9XevHnz+++/f/vtt3ft2pWRkbFz5061i3ISCxYsyMzM3LZt2/bt211cXLi2XR1cXV0t/3RTlmC0cTU77hq1AyUFCwoKvv/+e1YXtJezZ88mJye/9NJLQojbt29fv349Ojp6z549LGRsu5CQkMDAwPr16wsh3NzcIiMjr127pnZRTmLFihWLFy/u2LGjEGLWrFmPP/74ggUL1C7K2YSGhmZkZCiPb9y4UadOHRuXzOWI0FYmk+mf//xndnb2li1bWL/Yjlq2bHnkyJHly5cvX758/PjxQUFBy5cv5zco7KJfv346ne73338XQhQXF58/fz4qKkrtopxEYGDglStXlMdXrlwJDAxUtx6nNGjQoM2bN5tMJiHEpk2bBg0aZOOALLptq+++++4f//hHVFSUZYHzbdu2hYaGqluVk9m2bdsbb7yRnJysdiHOY/HixYsWLRowYEB8fPxf//rXDRs2qF2Rk9i5c+fIkSMHDBjg5ua2bdu2Tz/9dMSIEWoXVYO9/PLLp06dunjxYnBwcEBAwMqVK9u1a1dUVNSzZ09/f/+6devGxcUdOXKkadOmtmyFILRVbm5uSkqKdUurVq04NLQv5dRoq1at1C7EqVy8eDExMTEiIkI5jwd7yc7OPnHihNFo7NSpE/8mtlFycnJhYaHlabNmzXx8fIQQer1+//79JSUlffv29ff3t3ErBCEAQGpcIwQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABIjSAEAEiNIAQASI0gBABI7f8BrV8mwFhKXEgAAAAASUVORK5CYII=",
"image/svg+xml": [
"\n",
"\n"
],
"text/html": [
"\n",
"\n"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"plot(β[2:end], line=:stem, label=\"true beta\")\n",
"plot!(β̂nonneg.value[2:end], line=:stem, title=\"Nonnegative LS\", label=\"fitted\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Example 3: lasso\n",
"\n",
"* Lasso regression [Tibshirani (1996)](https://www.jstor.org/stable/2346178), [Donoho (1994)](https://doi.org/10.1093/biomet/81.3.425) minimizes the least squares loss with $\\ell_1$ (lasso) penalty\n",
"$$\n",
"\\begin{array}{ll}\n",
" \\text{minimize}& \\frac 12 \\|\\mathbf{y} - \\beta_0 \\mathbf{1} - \\mathbf{X} \\beta\\|_2^2 + \\lambda \\|\\beta\\|_1,\n",
"\\end{array}\n",
"$$\n",
"where $\\lambda \\ge 0$ is a tuning parameter. Writing $\\beta = \\beta^+ - \\beta^-$, the equivalent QP is\n",
"$$\n",
"\\small\n",
"\\begin{array}{ll}\n",
" \\text{minimize}& \\frac 12 (\\beta^+ - \\beta^-)^T \\mathbf{X}^T \\left(\\mathbf{I} - \\frac{\\mathbf{1} \\mathbf{1}^T}{n} \\right) \\mathbf{X} (\\beta^+ - \\beta^-) + \\mathbf{y}^T \\left(\\mathbf{I} - \\frac{\\mathbf{1} \\mathbf{1}^T}{n} \\right) \\mathbf{X} (\\beta^+ - \\beta^-) + \\lambda \\mathbf{1}^T (\\beta^+ + \\beta^-) \\\\\n",
" \\text{subject to}& \\beta^+ \\geq \\mathbf{0}, \\, \\beta^- \\geq \\mathbf{0}\n",
"\\end{array}\n",
"$$\n",
"in $\\beta^+$ and $\\beta^-$; $\\mathbf{x} = (\\beta^+, \\beta^-)$."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" 17.769835 seconds (21.27 M allocations: 1.975 GiB, 4.78% gc time, 97.45% compilation time)\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAGQCAIAAAD9V4nPAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdZ2BUdb438HOm95ZkWpJJI5SEFooh1FANSBTQiLC4ruC9Lqu74rPYVt1lZXUteL1313J1Xb2rrpR1LRQFgkiHgPQMkAApk2RmMumZXs/z4uAQICiEZM5k5vt5NXNyMvObkvPN+Z9/ISmKIgAAAOIVi+kCAAAAmIQgBACAuIYgBIiQU6dO8Xi8nJwcpgsBgCsgCAEiJBQK+f1+n8/HdCEAcAUEIQAAxDUO0wUAwBVcLteBAwdqa2tbW1v1ev2UKVNSUlKu3e3s2bMnT55saGiQSCTJycnjx49XqVThn1IUdeTIkXPnztlsNpVKZTAYxo8fLxKJuj6C1+vdtWtXRUUFQRCDBw8uLCzk8Xh9/eoAohEFABFx/PhxgiCysrJ+ZJ9XX31VKBR2/QvlcDgrV64MhULhfTwez3333XfVHzKbzT5y5Ai9g9VqHTdu3FU7CAQCj8cTfpD9+/enp6d33SEjI+PAgQN99NoBohmaRgGiiM1m+9nPfvbll1+eOnXKaDR+/PHHGRkZa9asefvtt8P7/OUvf1m3bt20adN27txpMpmMRuPGjRt/9rOfsdlseocVK1YcOnTowQcfPHz4sMlkOnHixKeffjpjxozwI5w/f/722283mUzPPPNMeXn56dOnn3zyyZqamlmzZl24cCHSrxmAcUwnMUC8uJEzwmvV1taKxeLs7OzwlqKiIoIgqqurr/crGo1GpVJ1PYm8Cn1CuXLlyq4bH3/8cYIgFi1adFPlAcQAnBECRDWDwTBy5Mjz58+3tbXRW5RKJUEQBw8evN6vKBQKh8NRXl7e7U8DgcCmTZvoFteu25944gk2m71x48ZAINB75QP0AwhCgCgSCAT+93//l+4gw+VySZIkSXL//v0EQbS0tND7PPjggyRJLl68eNy4catWrdq1a9dV0bV06VKfzzdq1KiioqLXXnuNPhMNq6mpcTqdycnJGo2m63adTqfX651OZ21tbR+/SoDogl6jAFFkyZIl69evT05OvuOOO7RaLd3P88MPP6yoqAin3cyZM3fs2PHiiy/u2bOnrKzsj3/8o0qlevbZZx9//HGSJAmCeOKJJ9Rq9V//+tdt27Zt27aNIIj09PT/+q//mj9/PkEQDoeDIAi1Wn3ts2s0mrq6OrvdHrHXCxAVmG6bBYgXP3mN8PDhwwRB5OXlOZ3OrttHjx5NEMTZs2ev2r+9vX3jxo3Lly8Xi8UEQbz55ptX7WA2m//5z38uXLiQzWazWKy9e/dSFEV3hzEYDNcWQI/TuHjxYg9fIUD/hKZRgGhx8uRJgiDuvPPOrgP+nE4nPdTvWnK5vLi4+O233/78888Jgvjss8+u2kGn0y1evHjdunUvvvhiKBT64osvCIJIT0+XSqUNDQ0Wi6XrzvX19WazWSqVpqWl9e7rAohyCEKAaJGUlEQQhMlk6rrx1VdfpRszw666SxCETqcjCMLr9RIEQVGU0+n8kR3YbPa8efOCweCaNWu67vPKK6+EQqH58+eHh2EAxAlcIwSIqObm5kcfffTa7StXrhw3bpxAIPjoo48GDBhQXFzs8/k+/fTTN998MzU1ta6uLrznpEmTBg4ceNdddw0YMEAmk1VWVq5atYogiHvvvZcgCJ/Pl5ycvHjx4hkzZmRmZnK53KNHjz799NMkSZaUlNCP8MILL2zcuPGNN94gCGLRokUEQXz00UdvvfWWXC5/4YUX+v49AIgyTLfNAsSLq3pvXuXgwYMURa1fv75ru6hEIlm3bt0dd9xBdLlGWFxczGJd0ZbD4/GefvrpYDBIUZTP58vKyrrqwRUKxd///veuxXz//feDBw/uus+QIUOOHj0a+bcFgHEkhRXqASLC5/P9yMiE1NRUgUBAEERzc/PBgwfNZrNWq506dapMJrNarS6XKzU1lcvl0ju3tLQcPXrUYrEEAoHU1NTRo0cnJCR0fbS6urqTJ09arVYej2cwGPLz86+auY0giGAwePDgwfBco+PGjUOjKMQnBCEAAMQ1dJYBAIC4hiAEAIC4hiAEAIC4hiAEAIC4hiAEAIC4hiAEAIC4hiAEAIC4hiAEAIC4hiAEAIC4hiAEAIC4FtVBuHbt2srKyhvcmZ47tU/rgesJhUJMlxCn8M4zBe88U/riUB/VQfjFF1/QS5XeCJ/P5/f7+7QeuB6Xy8V0CXHq2qUHITJcLhf+82ZEIBDw+Xy9+5hRHYQAAAB9DUEIAABxDUEIAABxDUEIAABxDUEIAABxDUEIAABxjRP5p/T7/eXl5UePHnW5XL/5zW8iXwAAAEAYA0G4ZcuWxx9/PDk5uaKioheD0NPk87YEEodwWDyc5gIAwI1iIDPmzZtXXV398ssv9+7DBtwh677Wsj+cM/6t1nKg1duOwfUAAPDTGDgj7CMSg2Dwf6SyCU7HeWfrGXvdNhtHxFblyhQDxfIBYpJFMl0gAABEI5KpWYL27ds3f/78pqamH9ln3LhxIpEoIyODvisQCF555RUWq/uz2HMtvj1NHA7nUrSTFCFs9crqnLI6J9/ht+tEnanidoMoxGX37gsBgiC8Xi+fz2e6iniEd54pXq+Xx+ORJP7DjrRQMDgx0T84gXeD+/N4vHAuXE9UnxHKZLL09PRRo0bRd+VyuVAovN7OTUHW6U6Sze6ScxwOkSEmMgixy5fc6Eo578g50NSq4NdrJXU6aYf0Rt9H+El+P8X1RPV3KVbhnWeK309xOBwEYeSFQqwsOXsEn3uD+1/v3KmrqP4TUigUM2fOLCkpuZGdC3WBqXqSx+v2FXEIQkQQiSFfqP28s/WMvfWwicVlqXKkqlypPEtMsvFtviUOh0ciue7/KNB37Ha3VIp3ngEOh0csFiAII8/v94dCV57z3LKoDsJex+KxVLlSVa6UuEfvaHC3VzpM22xum1c+QEJv5wjRcAoAEF8YCMK6urr58+c7HI729vYxY8ZkZmZu2LAh0kWQhCRFKEkRpkxL8tsDbRWOVqO96nOLSMdX5UpVuTKRBtddAADiAgNBqFar33333fDdH7nsFxlcKUc9RqEeowj6Qu2VjrYz9vJ3qtkCdkKuVJkjlWWI0OMUACCGMRCEfD5/9OjRkX/en8TmsRKGyhKGygiKcNS7W8/Yq7+yetp8ykFSVa5UOViChlMAgNgTX9cIbxRJSFKFklSh4Xa13xFoO+doOdV58TOzSMtX5UoVAyWSFHRPAACIEQjCn8CVXGo4DflDndWuVqP97Acmkk0qBkpUOVLlYAl6nAIA9GsIwhvF4rIUAyWKgZLM+TqX1dt6prN+Z1PFJ3XyLLEqV5owVMaV4s0EAOh/cOzuCZGWL9ImpUxLohtOW432ms2NggQePQYDDacAAP0IgvCWXG44DVCdVc5Wo/3shyaSJBWD0HAKANA/IAh7B4tDhhtOnQ2e1jP2ulJb5af1lxJxiJQrRo9TAIBohCDsfeJkgThZkDozyW8PtJ6xt5zuvPi5WawTqHKlqhyZSIuh+gAAUQRB2Ie4Uo4mX6nJV1Ihyl7rphtOqSBFnyYqBklYHDScAgAwDEEYCSSLlGWIZBmi9LkaT4uv1Wg3722p+GedPFNMz+jGk+GDAABgBo6/kSZI4OknJ+gnJwScwfYL1/Q4TRYSOEsEAIggBCFjOGJ24gh54gh5uOG04pP6kD+kHCxFwykAQMQgCJl3vYZTqUGkypEmjpDx5De6BCUAANwsBGF0udxw6gq2n3e0Gu2mbTaejKPKlalypbJ0ERpOAQB6F4IwSnFEPzScBqmOKmer0V65tp4KUMocaUKuVJ6NhlMAgN6BIIx2JJtUZEsU2ZLMeTq3zdtitNfvbD73cZ1igESZI1XlSNHjFADgVuAY2p8I1fwUNT9lamK44bRmkxUNpwAAtwJB2C9dbjj9ocfphQ0NfmdQOViiypUqB0vZfBbTNQIA9A+xE4TuYDBIEAk8HtOFRNRVPU7bKx22I+3n1zfQPU4Thsv4CvQ4BQD4MbEThF9ZrL8+ZRyrUs7Wau7QaYfIpExXFGmCBJ62QKUtUAV9oY7zztYz9ro3LnJFbDScAgD8iNgJwvtSkufpdUc67Zss1tl7D5AkMUujnqFRz9ZqJJzYeZk3gs1j0fPUZN2tc5o9rUZ71RcWb7sfDacAANeKqYQQstkzNOoZGvX/jBxe5XRuMlvfq6p56Ptjt6lUc3Xaecm6NJGI6RojimSRkhShJEVouF3tafa1GDutB1rPr2uQZYoTcqXKHCkaTgEAYioIu8oUix/LznosO8sZCO60NW22WF6tqBSw2HP12mKddnJSIo8VX2dFgkRe8pTE5CmJAU+w/Zyj1Wiv+aaRr+CqcqSqXJk0FXOcAkCcitkgDBNz2MV6bbFeG6Ko4+0dOxptq86cO9XeMSUpsVivm6vT6oUCpmuMKI6AnThSnjhSTlCEo8HdarRf/Mx8ueF0kJQtiK9/EQAgzsV+EIaxSHK0UjFaqXhq8MAmr3dXU/Mms/WZ08YMsWiGRj1Xpx2foGKR8XRaRBKXG05bfe0VDtuR9vNrG8TJAlWuNGGYTJiENYQBIPbFURB2lcTnl6Qkl6QkBynqYEvrZot1xYlTJpe7MClxrl57p16n4MbXxTOB6lKP05Av1H7e2XrGfvqtahaXpcqRqnKl8gFikhVP/yIAQDyJ0yAMY5PkxMSEiYkJxLDcKqfza0vjWlP9o8dOjlUp79Tr7tTrMsTx1b+G9UOPU+IePd1wWrO50dvmkw+QqHKlqqFSjoDNdI0AAL0p3oOwq0yx+NEBmY8OyHQFg982Nm00W14+V6nm84v12rv0ujEqZXydE3VpOPW2+dvO2ZtPdFz8zHyp4XSoTKhGwykAxAIEYTdE7Ev9awiCMHZ2/quuYfmxEyaXu0irKdZr43BgIl/Jvbrh9O0uDadZYqYLBADoufg6oPdArkyWmytblTukxuna3mj7qMb0H98fn5SYUKzX3anXagXx1eP0csNpid5l9bae6TRts7lsXkm6QD0ioMqVcoRoOAWAfgZBeKPSxaL/zEz/z8z0Np9/h822yWx9+nR5plg8V6e9NzUlJ/5mdBNp+SJtUsq0JL8jYDnR3Gq0V31uEen4qlypKlcm0qDhFAD6BwThTVPyuHSP0wBFHWpp/Vd9Q9He/VySRQ/VL1QnceJqDAZBcCUc1UiJYaIk5A91VrtajXbjuzUkmww3nJLs+HpDAKB/QRD2HOeHHqf/M3K4sbNzs9m66sy5irIjs7XaYr22SKuRxtmlRBaXpRgoUQyUZM7XdW04VdA9TtFwCgBRKb6O1H0nVybLlcmeGjzQ5HJttV66lDhWpZyr096Tok8WCpkuMNK6Npy2nXPQE3+LtHxVrlQxUCJJibs3BACiFoKwlxlEly4l0mMw/lXf8MczZ3UCQUlKcrFeN1qpYLrASONKOOoxCvUYRbjh9OwHJpJNKgZKVDlS5WAJGk4BgFkIwr4SHoMRnrzmZ2VHvKHQLI16rk5XpFVz42zW72sbTut3NlV8UifPEqtypaqhMp4U30YAYAAOPX0uPHnNy8Ny6cWhXqmofPDI0WnqpLl67V16nTzOpnMjujSc+joDrWfsbWfs1ZuskmShMkeqypGixykARBKCMKLCi0M1eb3fWBs3m62/OX5qqFxWkpK8IFmfKoq7K2c8GUc7TqkdpwwFqI7zjlaj3fheDckilUOkqhypfICYxUHDKQD0LQQhM5L4/J+nGX6eZnAHgzsamzZbLC+fq1TxuPTKUBMSE+Lt8M/ikMohUuUQaRZBeFp8rUZ7w67mcx+ZpAaRKkeaMEzGV8bdeTMARAaCkGHCHy4lvjOKOt7esclsefjocUcgWKRVz9XpZmnV/Di7lEgQhCCBp5+coJ+cEPSFOs47W8/Y63de7DqjG/rXAEAvQhBGi/Byiatyh9CXEv9y4eL9h7+np3Obl6xT8+Puyhn7yqUw2isdpm02p9lzqX9NrownwxcYAG4VjiPRKHwp0eb1bjZbN1ksT54qH61UxOfKUARxeSkMemBix0Vnq9Fes6VRoOIpBkpUuVJZuojAWSIA9AiCMKqp+fylGWlLM9LoS4nxvjIUQRAEwZVwEkfIE0fIqRBlr3XTQ/W97f4fThMxfw0A3BwEYf8g/NGVoeZotWJO3B39SRYpyxDJMkQEofG0+torHPTE34JEHp2ImL8GAG4EgrD/wcpQ1xKoeJdXTLzgbDtjP/uhiSRJ5RCJcohUMUDM4sVdnyMAuEEIwn4svDJUq8/3ra0JK0MR9IqJOVJVjjSLIFwWT+s5R8Ou5opP6mTpIuVgqXKIVJjEY7pGAIguCMJYoOLxsDLUtUQ6gUgnSJmaGPKFOmtcrUZ7+f9WkyxSMVCiGChWDpay+ThNBAAEYWzBylDdYvEuT3NKj9a3Hmy7sN4sMQgVA8VYRhggzsXjYTFO/MjKUCUpyXphPF5KJLobrU8vI4zTRIC4hSCMfeGVoTr8/m+sjV81WF44c26QVEKPSozPS4lE19H6P0zqhtNEgPiEIIwjci73vtSU+1JT/KHQ7qbmjWbrHfsOcEiSTsSJiQnsuLyUSPzoaaIqR6oYJMHc3wAxDEEYj7gs1gyNeoZG/Ze8S5cSny0/c67THs8rQ9EunyZShKPB3XbOUb+zqeKfdfIssWqIVDFYIlCh0ylArEEQxrvwpUSsDHWFHyZ1S52RFHAF2yocbefstdtsHCFbOViiHIwlogBiB4IQLsHKUNfDEbGT8uRJeXKCIFxWb+uZzoZdzRUf1dFXExUDJZjCBqBfQxDC1bAy1I8QafkibVLKtKTw1cSzH5ouj00cJGUL4vfNAeinEIRwXT+yMlRJanKxTqfkxemlROI6nU7Pr20QJwtUuVLFQIkkWYgFMQD6BQQh3JDwylAtPt8Wi3Wz2frY8VO5clmxTjsvWZ8c30f8cKdTegqb9kpnxSf1QXdQMUhChyIWxACIZghCuDkJPF7XS4kbzZbCXXsTudy7UvRxuzJUWHgKm/S5Gk+Lr73S0Xyi88J68+UFMXCaCBB9EITQQ+FLiSGK2t1gLm1rf/DIsTa/r1inu1OvnaZOErDj+jRIkHB5QYyOi862c46Kj+uC3pByiFQ5WKLIlnBEcf3+AEQPBCHcKhZJjlXIp6YkvzQsl14Z6p2L1YvLvo/zlaHCWDyWcohUOURKEDpPi6/trMN2pP38+gaRRqAcJFEMlkgNQpKF80QAxiAIoTdhZagfJ0jg6SaqdBNVVIhymj30SsLuJq80TaTKkaqGSjFgHyDyEITQJ7Ay1I8jWSQ9YN9wu9rvDHZccLRXOht2NV+e/hsjMQAiBUEIfQsrQ/0krpidOEKeOEJOXDkSQ6jmo4sNQAQwcwwKBALl5eUSiWTAgAGMFACMwMpQP+nySAx/qLPa1V7pvLDB7G33y7PEioFiVY6UJ4/fsZsAfYSBIDSZTNOnT1coFI2NjZMmTfr4449ZcTxTSXy6amWojWbLC2fODZRK7tLrZms1g6SSOO9xShAEi3tpJAZBaHz2QHuFo9Vor9ncyBGxVTlSVa5UlonJTgF6BwNB+Mc//nHq1Knvvfee3W4fOXLkli1biouLI18GRIOuK0PtaW7ZaLbcd+hIjcsl5XBSRcIUoTBNJKJvGETCVJFQLxTG4cVFnpSjHqNQj1FQIcpucrdXOGq/sbkaPfJMscDA8WkproTNlXC4Yg5HHO//QAD0AANBuGHDhh07dhAEIZVKFy5cuGHDhl4JwoC7w9tWzReI2TwJyeayBRIWm8fiYjbk/oHLYk1XJ01XJ9F323x+s8dtcXuqnK4qp/N0R6fF46lyOE0ul5TLyRSLdQKBXijIFIszJZdup4tErFjPSJJFytJFsnSR4XZ1wB1sr3Q0n2u3NbT7HQG/M+i3B4K+EFfM5ko4XAmHK+XQt3lSDlfC4YrZXCmHK+Gw+WiAAbhCpIOwtbXV4XBkZmbSdzMzM/fv33+9nV0u1+nTp5VKJX1XLBYXFBRcb+cq49qzZz4hqFAoGOCECCroo6ggFQqwWFwOwWFx+ASLTbK5bDaPy+KxuAKSZLM4PDZXyGEJWDwBSbJZXAGHK22h+BWd1Hk3RfXuK49pFBUiyT48vCoJQkkQOQRBEUQwQPlclD9EOCnqGBUqCxH+EOUPUUGC4pIkh0XySZLDIjgsFo+MxmCkQkEiGOy1RyMokkUSMoKQEYTu0iaCIgmKIDwk4SKJJoII0VtIIkRe2oEkCBZFkARBhgjyh7sEhS45N46iiGj8esUDipidNfyewrm9+JCRDkKXy0UQBI93abCUQCBwOp3X27mhoaGmpmbfvn30XaFQuHbt2utdUJSmTDHwRBwOhyAIX8BJhS4da6igz+vroAKXctEZCNX5ydYgrzkkaAnyW928NkpsJ6RtpLyDLXOwJMKQU0jYxSF7L75q6H304ZsgiH7XFsgiCJySAdyCzgsnisYU3uDOAoGA81Nd0yMdhGq1miTJ1tZWuVxOEERLS4tWq73eztnZ2SUlJSUlJTfyyFzuEL0uh8fjtfmCZrfP4vGb3f4mr6/abq/2ORv83mZvqM3H9hMsUdCX4HXrvQGdO8hh+ShxsE3obea25nA7ZyeoZw4YUqBJ5aL/zs1wOBwSiYTpKqJFsKPFX3/eV3fBV3feb6qkgn6OJo2XOoCXms1NzeZq03rxuex2u1Qa79MUMMLhcIjF4qhsdIhxfr8/FArx+fxefMxIByGPxxs+fPiBAwcyMjIIgti3b9/YsWN75ZHfr7Y8d7bZHiDEbL+IcAuCDqnHofb60nzkJCcruyOU1uFXcamAVnw6RXQwKfRdqGOr35GfqLtTnTYhUZ+v0iH8oGe6Jp+vtoIIBejkE42cxC1e2rvJBwC9joHOMitWrPjd734nl8svXLiwY8eON954o1cedkRn9bu1Zwbb2aoOUtQWoGQCtlbJ0yez05ROjeSowPeZt2lHo6nS3pqfwJuhTvufxHyEH/TMVclHsljc1GxearZk/GzuwsfYMhXTBQLATWAgCH/xi1+w2ez3339foVDs3LkzJSWlVx52tGxAXpqIm5zA1ipYWoWTRR1qseyw1e5oPF15sS0/QTshIfnlYZMmJyXzWP3ushIwLGhv81WV++ou+OvO++rOk1weNyWblzpAMn4Okg+gv2NmZpn777///vvv793HZGWo3QbVQXvLDtvZfRcbyjtaxqo0CD+4FX6ryVN+0F1+MGBr4GfmclOzJVPmcVOy2TIl06UBQK+JnWkeP6mr+H/le8apdIXq1FeHT75NpeX0ZYd+iFmhkK/hosdY5jq+h/J7BIPHyGbexx88hmTHzh8LAHQVO3/b96Vk35cyUBzfS99Bj1E+r/f8cbfxsKf8IEskFeSOUy58jJ+Rg8FiADEvdoKQQ7LQlRluVsjR4Tl7xG0s8547xtGli0ZOks1axFYkMV0XAERO7AQhwI0LtFg85WWuE3sDtjp+9ghhbr7yvhUsgZjpugCAAQhCiBuhkLfmrMdY5j59gAj4+YNHy2aU4OIfAOAQADEufPHPfeoAWyITjpysuv8pXmo203UBQLRAEEJsCrY3ucvLPOUHfTVneZlDhUPHyYqWYMAfAFwLQQgxhKL8DRfd5Yfc5QeD7c3CnNvE4+ckLH2e5KEvMQBcF4IQ+r9Q0Ftzzn1ij/vUAZLNEeTeprjzIV7WMFz8A4AbgSMF9Fchl91becJtPOQxlnESdILc/MRfvsjVGpiuCwD6GQQh9DOBFqun/JDbWOY3VfKyhgpz8+XFD2HOMwDoMQQh9AcU5au/4DGWeYxlgbYmwZDR0il38QeNJjlcpisDgH4PQQjRi/L7vJXH3MbDnvJDLJFEkDtOPv+XmPYMAHoXghCizlXTngmH5if9+jVOUjLTdQFAbEIQQrTw1190G8s8xkOBFqtg8BjRiEnK+x5nCURM1wUAMQ5BCEyiAn7v+ZN05xeSwxUMHScvXsbPGkpg/UgAiBQEITAg5Oz0nj/ZdeRDwrLfY9ozAGAEghAih17zwW0s85sqeFnDhLn58uJlmPYMAJiFIIQ+Fl7zofwg5fMKhozByAcAiCoIQugTIZfDW3ncbTzkKS9jy1WC3HHKhSsw8gEAohCCEHpToMXqrTjmNpb5Lp7mGgYJc/Plc5ey5QlM1wUAcF0IQrhloVCg9mxH1Sl3+SHK4xLk5kvGz+H/4lmSy2O6MgCAn4YghB4KeVzec0fdxkOeM0dIeYJ4WIFqyRO8lGw0fgJA/4IghJsTbLV5zn3vNpZ5z5/kJmeJRk6Sz3nAzRVJJBKmSwMA6AkEIdwQv7XWfWKvx1gWaG3kZ48QjZykuv+py9O+OByMVgcA0HMIQrguyu/zVRvd5YfcJ/eRHJ4g9zZ58VL+gOGY9gUAYgmCEK7WzZzXv3qZo0llui4AgD6BIIRL/NZaj7HMXV4WsNXxs0cIc/OVCx9jCXHlDwBiHIIwrl1u/Dx1gGRzBLm3yWcv4WUNI9n4YgBAvMDxLh6FHO1uY5nHWOatPMFNzhLk5if96s8cdQrTdQEAMABBGEf8lhp6zs9AYz1/UJ5w+ETlwhUssYzpugAAmIQgjHXhOa9PH6ACPsHgMbKZ92HOawCAMARhbAq57N7KE27jIY/xMCdBK8jNV93/FC9lAKZ9AQC4CoIwpnRd8A9zXgMA3AgEYf8XCvkaLnqMZR5jWaCtSTBktGT8HMGy35N8IdOVAQD0AwjC/oryeb3nj7uNhz3lB1kiqSB3nHz+L7HgHwDAzeo+CP/0pz/df//9aWlpXTeaTKa333775Zdfjkhh0L3wnNfhBf9ksxaxFUlM1wUA0F91H4RvvvnmtGnTrgrChoaGV155BUHIAIry1Va4yw96jGVBe7swN1+cf3vCA8+QPAHTlQEA9Hs30TTa2toql8v7rhS4CuXzeM4d8xjL3MYytlQhyM1XLlzBSxuExk8AgF50RRDu29y77kMAACAASURBVLdv8+bNBEE4HI533nln48aN4R/5fL5NmzaNGDEi0gXGn5Cj3XP2+65zXqunr8G0LwAAfeSKIDx9+vR7771HEITb7f7qq684nMs/FQgEQ4cOff311yNdYNzAnNcAAIy4IgiXL1++fPlygiC0Wu3nn38+fvx4hqqKF5jzGgCAcd0fcK1Wa4TriCshZ6fnzGG3scxbcZyTqBPk5ics+z0vNZvpugAA4tFPnHkEAgG73d51i1Kp7Mt6YlnXaV94WcOEufmKBb9iy/B+AgAwqfsg9Pl8q1evXrt2bW1tbSAQ6PojiqIiUlisuGbOa+mUuzDnNQBA9Og+CJ988sm33377l7/85ZEjR1Qq1dixY7/++utz5849//zzEa6vn+oy53UZJ0GHOa8BAKJW90H4j3/846WXXlq5cuWDDz6o1+tXrVr1hz/8Yfny5d99990TTzwR4RL7Ecx5DQDQ73QThM3Nze3t7UVFRQRBcDgcl8tFEARJks8991xqampDQ0NycnKky4xmP8x57T6xJ+joxJzXAAD9SzdBKBAICIIIBoMEQWi12jNnztDbZTIZQRBWqxVBSBAE5fN4z59wGw97Th9kiaWC3HGKex/DnNcAAP1ON0EokUhSUlLOnDkzYsSI/Pz8N954Y+/evaNGjVq9ejWHw8nMzIx8ldEj0NroPXcUc14DAMSM7q8RLlu27NixY4sWLZozZ05eXt7kyZPp7b/73e/icfgERfnqL1xa8K+1UZAzVjx2hur+p1gCEdOVAQDAreo+CFetWkXfYLFY33777fbt26urq0eNGjVhwoTIlcY0yu/zVh5zGw97jGUkhyfIvU1evJQ/YDjBYjNdGgAA9JqfnsqLx+PNnTs3AqVEiZCz031qv/vUAV+1kZc2WJCbL52xkKPSMF0XAAD0iR8LQo/HY7VaQ6FQ142xeo0w5Ohwn9rvOrHXb6rkDxktvm2m6oFn0PgJABDzug/CCxcu/OY3v/nmm2+u/VGMzSxDT/vpOrHXd/E0L2uYeOx0IUY+AADEk+6DcOHChbW1tatXrx44cCAZi+MBrso/0chJCT9/GvkHABCHuglCu91+/PjxdevW3XvvvZEvqE91k38PPEPyBEzXBQAAjOkmCOmh9FlZWREvpq9czr+as/zsEcg/AAAI6yYIFQrF9OnTt27dOnr06MgX1IuC7c3uk/vcxjJ/w0VBzljJ+Nn8B5/Dsg8AANDV5SBsaWmpqamhb//6179evny5y+W6/fbbxWJx11+I/nQMtje5T+53ndgbaDQJcm+TTrmLP3gM1nwHAIBuXY6HjRs3Ll26tOvPXnrppZdeeumqX4jaXqOUy+4+urO9/ECgySwcWiCbtYg/cCTyDwAAftzlnJg6deqGDRsYLOUW+aqNQVu9rOh+fvYI5B8AANygy4GRnp6enp5+g79WX19/5MiR+fPn90lRPcLPHScYWsDj8ZguBAAA+hNWz37t5MmTzzzzTM9+12w2r1mzZtGiRffcc0/PHgEAAKC39DAIb0V1dfW5c+c0Gs3u3bsj/+wAAABdMRCEEyZMeP/993E6CAAA0YCBIAQAAIgefdK7srm5+ejRo9dunzx5slB4E/N5Go3GXbt2Pfnkk/RdsVi8f/9+Fqv78PZ6vSRJorMMI5xOZ9SOq4ltDoeD6RLilNPpDIVCMTkVc5Tz+/2hUMjn893g/gKBgMv9iXlU+iQIa2pq/vKXv1y7feTIkTcVhIMGDVq+fPmcOXPou1KpVC6XX29nHo+HIGQKSZISiYTpKuKUVCpluoR4RJKkWCxGEEYeHYR8Pr8XH7NPgnDMmDFbtmy59cfhcDgajSZWV0AEAIBowMA1wlAo1NbWZrfbKYpqa2vr7OyMfA0AAAC0Hp4Rzpw5c//+/T373ZqamjFjxtC3s7KyBg8efODAgZ49FAAAwC26bhCazeZNmzZVV1e3tbV13f7uu+8SBMHj8RISEnr2lJmZma2trT37XQAAgN7VfRDu2rWruLjY6/XyeDwul+v1et1ut1Ao1Ol0Ea4PAACgT3V/jfCxxx4bPXp0U1NTSUnJI4884nK5duzYodPpVq1aFdnyAAAA+lY3Z4Rer9doNO7YsYMeqxAIBAiCmD59+nvvvbdgwYKSkhKBAGu7AwBAjOjmjLC9vT0YDOr1eoIg5HJ5+BphQUFBZ2dnRUVFRAsEAADoS90EYVJSEp/PN5vNBEFkZGTs3bs3GAwSBHH69GmCIEQiUYRLBAAAIAgi4Hdaq7fbW3r5fKybIGSxWBMnTvzmm28Igrjvvvuqq6sLCwsfffTRu+66a+jQoRjeDgAAkeRor75w/G97P79vy3sja05/FPD38syC3fcafeedd+gWUY1G88UXX7z44otffvllfn7+66+/zmaze7cCAACAq1BUsMV81FpdajPtcdnNmrTCjKE/Gzf3bwQpCIVCvftc3QdhdnZ2+HZRUVFRUVHvPisAAMC1vO6WproDlqrt1upSsTxNbZg8fMrqxOSxBHFpWle/39/rT9p9EP7Hf/zHypUrBw0a1HVjZWXlc889t2HDhl4vAgAA4llnS4WlqtRStb2j+WxSSoEuc9awic8KJNrIPHv3Qbhp06YHH3zwqo0tLS2fffZZ35cEAACxL+B3NdXts1SVWqt3sNh8XebMnIKVickFLPZPrJrU625irtH6+voeT6sGAABAEISzo9Zm2mup2t5cf0ipzVMbJk26+19S1QAGS7oiCLds2fKPf/yDIIiOjo7nn3++a+w5HI4DBw5MmTIl0gUCAEA/R4UCLZZj1upS88VtPk+7Jm2KYUjJbbPf5vCiYh3TK4Kwo6OjqqqKIIhgMNjQ0NDR0RH+kUwme+CBB55++ulIFwgAAP2T19VsrfmO7vkplqdpM2aOLXpTqRkW7vkSJa4IwsWLFy9evJggiPz8/HfeeWfUqFEMVQUAAP0SRYXabeU20x5L1XZ76wW1YaLaMHlE4Z8EYjXTpV1X99cIy8rKIlwHAAD0X+GeL5aqUjZHcKnnS0oBixXpni89cN3OMpWVle+8847RaORwOF9//TVBEOvWrZNKpXfccUcEywMAgOjl7Ki1VG23VO1osx5XavN0mTMG568QSZOZruvmdB+Ee/funT17tlwuT09Pr62tpTeazeZ33nkHQQgAEM+CAW+Lucxm2mu+uDUY8GjSCjOGLSm48wMOV8x0aT3UfRD+6le/mjJlymeffVZWVrZkyRJ6Y1FR0W9/+1ur1arVRmiQIwAARAmvq8las8taXdpYu1uiSNdmzBxb9FYU9nzpgW6CsKWlpby8/P333xcKhSR5+RUaDAaCIMxmM4IQACAe0D1f6AnPXJ31SakTtBkz86a/xhPImS6tN3UThPRKvHw+/6rtTU1NBEHweLwIlAUAAEzxedoba7+zVO1orPlOJNVrM2eMmPqiSptHkrG56EI3QahWq3U63Zdffjly5MiuZ4QfffSRTCa7agJSAACIDR3NZ6zV31qrd3Q0nU1KnaDNmD5s0vPCSE34yaBugpAkyaeeemrlypUulystLS0YDB48eHD9+vVvvvnm888/z+X2g76wAABwI4IBT4v5sKWq1HzhG5LFVhsmDxzziNowhc25ulEwhnXfWeY3v/lNR0fHn//8Z4/HQxDE+PHj2Wz2I4888txzz0W2PAAA6H2uzvrG2l02057G2t2yhEG6zFnjij9QaoYzXRczug9CkiR///vfP/LII7t3725ubpZIJJMmTUpNTY1wcQAA0FsoKthuM17V82XUjDVcvozp0hj2Y6tPJCQkLFiwIGKlAABAr/N52mymfXT+CcQaXeasoROfTUoZT7JuYvWh2HbdN6Ktre3DDz88fvx4Q0ODVqsdOnTo0qVLMXACAKBfoJe6tZn2hud8GTrxd0KJjum6olH3QXjy5MnZs2dbLJasrCyNRnPs2LH169e/9tprX375JVZiAgCITgG/q8m011K9w1r9LZsj0GbMGDTmV4kpBSw2hr39mO6D8IEHHhCLxceOHcvLy6O3VFZWLlmyZPHixTU1Neg4CgAQPZwdtdbqby1Vpa2Wo0rtSG369IGjl0uUmUzX1W90E4Q2m+3kyZPbt28PpyBBEAMHDvzwww+HDh1qNBpHjhwZwQoBAOBqV/Z8aUhKHZ86eEH+He+i50sPdBOEfD6fJEm9Xn/V9uTkZIIgBAJBJOoCAIBreN2tTXX7LVXbLVXbhRKtLnPW8CmrE/RjSJLFdGn9WDdBKJfLZ82a9cEHH7z++utdt3/wwQe5ubmYWQYAIMK69nxJTBlH9/yMhzlfIuNyEJrN5jNnztC377///hUrVhiNxgULFmi12ubm5q+//rq0tPS1117rOukaAAD0kWDA3WI+YqkqbTi/hcXmqQ2TBuQt06QVoudLr7schNu2bVu6dGnXn23btm3btm1dtzz88MP/+Z//GaHSAADij7Oj1mbaa6na3lS3X56Uq8ucNenu9VJVNtN1xbLLQXjnnXd+//33DJYCABCfKCrYYj5qrS61mfa47GZNWqFhSMnY2W9xeVKmS4sLl4MwISEhISGBwVIAAOKKx2mz1nxrrf7WZtorVQ7QZc4cNWONQj00Bpa67V8wxQ4AQORQVKjddspStcNavcPZUaM2TNFl3p43/RW+EOchjEEQAgD0uYDf1VS3z1JVaq3ewWLzdZkzh078XWJyAYuN+UmYhyAEAOgr4Z4vzfWHlNo8tWHSpLv/JVUNYLouuAKCEACgN1GhQIvlmLW61Hxxm8/TRvd8uW322xyehOnSoHsIQgCAXuB1NVtrvqN7forladqMmWOL3lRqhqHnS/RDEAIA9BBFhdpt5TbTHkvVdnvrBbVhotoweUThnwRiNdOlwU1AEAIA3JyA39Vk2llp3mupKmVzBLrMmTkFKxNTClgs9HzplxCEAAA3xNlRa6nabqna0WY9Lksanjxg1uD8FSJpMtN1wa1CEAIAXFcw4G0xl9lMe80XtwYDHk1aYcawJQV3fuDxUmKxGHMvxwYEIQDA1byuJmvNLmt1aWPtbokiXZsxc2zRW1f0fPE6GC0QehOCEACAIH7o+fLDUrf1SakTtBkz86a/xhPImS4N+haCEADims/TbjPtpXt+8gQKeqm/pJTxJAuHx3iBTxoA4lHXni9KbZ4uc8aQ/MeFUj3TdQEDEIQAEC+CAU+L+bClqtR84RuSxVYbJg/IW6Y2TGFz+EyXBkxCEAJAjHN11lmqv7VWlbaYjyg0w7UZ0ycuWIcJPyEMQQgAMYiigu02Y9eeLymD5t025x0uX8Z0aRB1EIQAEDt8njabaR+dfwKxRpc5a/iU1Qn6MSTJYro0iF4IQgDo9zpbKixVpTbT3nDPl6ETfyeU6JiuC/oHBCEA9Evhni8NF75msbhqw6QBecs0aYUsNo/p0qCfQRACQH/i7DDRY/6a6vbLk3J1mbMKij9UaoYzXRf0YwhCAIh2FBVsMR+ll/pzdTYkpY5Pzi4eW/Qmer5Ar0AQAkCU8rqardXfWmu+tZn2SBSZ2owZedNfVWqGY6lb6F0IQgCIKlRb42lr9Q5r9Q5720W1YbIuY8bIqS/yRUlMFwYxC0EIAMwLBtwt5iOWqtKG81tYbJ7aMGlw/gpN2lQWG0vdQp9DEAIAY5wdtTbT3q49XybdvV6qyma6LogvCEIAiKgrer7YzZq0QsOQkrGz3+LypEyXBnEKQQgAkeB1tzTVHaDnfBHL09SGycOnrE5MHoueL8A4BCEA9CF6zhdL1faO5rNJKQW6zFnDJj0nEGuYrgvgMgaC0O/379q1q7y8XCgUzpkzx2AwRL4GAOg7fp/dVrvbWv2tpfpbvlCpzZiRO+HpRP1tWOoWohMD38vHHnvs2LFjEydObG5ufuKJJzZv3jxlypTIlwEAvcvedtFaVWqp/ra98aRKP0aXOXNw/uNiOf7ThWjHQBCuXr06ISGBvp2UlPTGG28gCAH6KSoUaLEcs1aXmi9u83naNGmFmcN/rk2fyuFJmC4N4EYxEIThFCQIQiAQsNnsyNcAALfC62q21nxH9/wUy9O0GTPHFr2p1AxDzxfoj0iKoph67rq6ury8vPXr10+fPr3bHSZNmqRUKgcNGkTflcvlK1euJMnu/9K8Xi9JkjweJp5ngMPhkEhwBsAAu90ulUZo1AFFhTqajM31+xprdjjaLialTkhMmahJnxGfc744HA6xWHy9wxH0Hb/fHwqF+Hz+De7P5XJZrJ9YjbJPzghPnTo1a9asa7dv2bJl9OjR9O3m5uY5c+Y8/vjj10tBgiDYbLZEIlEqlfRdoVDIYrGu982jf/STLxj6AovFwjvPiAi88wG/q6XhgLV6R2PNtyw2X5sxY3D+/0tIHsdixfWcL/Q7jyCMPPoL37tf+z4Jwtzc3PLy8mu3KxQK+kZbW9usWbOKi4ufffbZH3kctVo9f/78kpKSG3nSUChEkiSXG9d/nEzhcrl45xnRd++8s6PWUrXdUrUjvNRtzrjHRbKUvniu/oh+5xGEjAiFQr37te+TIGSz2YmJidf7aUdHR1FR0ZQpU1566aW+eHYA6JlgwNtiLrOZ9povbg0GPJq0woxhSwru/IDDFTNdGkAfYqCzzMqVK0+fPp2WlnbvvfcSBJGamvr6669HvgwAoLk66601O63V3zbXH5Qn5eoyZ4y742+yxMFM1wUQIQwE4bJly7peQZTJsLQmQKSFgv4Wc5m1eqe15luvu1WbPs0weMGY2//CE8iZLg0g0hgIwnHjxo0bNy7yzwsAXYY97BXLDWrD5LzpryXox5Ak+jpB/MKMRwAxjqKC7TYjPdu1q7M+KXWC2jB5ROFqTPgJQEMQAsSmrqs9CMQaXeasoROfTUwpiPNhDwDXQhACxA6KCrXbym2mPV1Xexg68VmhRMt0aQDRC0EI0O/5PG020z6baY+lqpTNEegyZ+YUrExMLmCxcfIH8NMQhAD9EkUF2xpPmiq2tZn32dsuJKVO0KZPHzLutzj5A7hZCEKA/sTtsDTWfNdYu9tm2iuU6JS6gqETn0nQ5+PkD6DHEIQA0S4Y8LSYD9tMe22mPa7OhqTU8WrD5OGT/yCU6iM56TZArEIQAkQpZ0etzbTXUrW9qW6/VDVAbZg8fMpqjPkD6HUIQoAo4nW3NtXtt5n2WGt2slhctWGSYUjJ2KI3uXxMwATQVxCEAAyjB7yHxzyotKPUhkmTFqyTqrKZLg0gLiAIAZhBt3zaTHvCs51hzAMAIxCEAJHj99mb6vbT3T5DQZ8mrTA5e27e9Fd4AiXTpQHELwQhQN+iqFC77XRj7a7Gml3tTeUJujGa9KlZI5fKEgYxXRoAEASCEKCP0FN92kx7LNU72Gy+2jApa+RSbcY0LHILEG0QhAC9hgoF2pvO0PNc29uq6Kk+B+evEEmTmS4NAK4LQQhwq7p0e9kjlqepDZOxzgNAP4IgBOiJgN/VavneZtprvrjN52lPShmnzZiZN/1VnkDBdGkAcHMQhAA3obOlwlJVajPtbWs8odSMVBsmjS16U6kZRhAk06UBQA8hCAF+gtfV1Fi7u7Hmu0bTHr5QpUkrHDhmeWLyODZHwHRpANALEIQA3QiF/C3mI401uxprv3N11ielTtSkFw6d+KxQqme6NADoZQhCgMucHSa6z0t4tpdhk55HtxeA2IYghHgXDLhbzEcuLXJkNyelFKgNk0cUrhaINUyXBgCRgCCEOOXsqLVUbbdU7Wi1fC9PytVlzsqb/hq6vQDEIQQhxBGfp81m2td1kaOMYUvGFb/P5WFtW4D4hSCEGBde5KjrmAcscgQAYQhCiE1uu9la+11jza6mur0imUGTPnVw/ooE/Rh0ewGAqyAIIXYEA96WhkPW2l2NNd95Xc2atCn6rKK8aS/xRUlMlwYA0QtBCP1eeKrPxtrdEkW62jA5b/qrCfoxJMliujQA6AcQhNAvhcc8XDnV52s8gZzp0gCgn0EQQn9y7ZgHTPUJALcIQQjRLuB3NtXtt1SVYswDAPQFBCFEI4oKtdvKrxrzMHH+WlnCQKZLA4BYgyCEKOJx2hprvmus3dVYu1so0WnSCwff9usEfT6LjTEPANBXEITAMCoUaLEctdbsbKzZ6eqsVxsma9ILh0/+g0CiZbo0AIgLCEJghtfV3FR/0FK13VpdKhBrdJmzsM4DADACQQiRQ892Roefq7M+KXWC2jB56MRnhTj5AwDmIAihz3mctsba3dbq0vAif0MnPpuUMp5k4esHAMzDkQj6BBUKtFiOWatLbaY9rs6GpNTxWOQPAKITghB6U5cV3veI5Wlqw+ThU1ZjtjMAiGYIQrhVwYCnuW7PxcZD1ppvAz6nNn1qcvZczHYGAP0FghB6yNFWZa3Zaa3Z2WI+IkvI0WfNyL/jPUVSDmY7A4D+BUEINyEY8LSYD4enutakTUnPXZQ/53+9fpZEImG6OgCAnkAQwk+j1zmyVG1vqtvf7VTXXr+D2QoBAHoMQQjdo9c5slSVmi9uJQhSkzbFMKRkbNGbXL6M6dIAAHoTghCucO06R+Pm/l2pGc50XQAAfQVBCETA72qq23fVOkcFxX/n8HDZDwBiH4IwfnW2VFiqSruuc1RQ/CFO/gAg3iAI44vP026r3W2t2dlYu4vLk2oypg0c86vE5HFsDp/p0gAAmIEgjAvXnvwNGvtrqWoA03UBADAPQRizfJ42m2mfzbTHUr2DzearDZMG5C3TpBWy2DymSwMAiCIIwphCUaF2W7nNtCd88qfLnDH4tsdEshSmSwMAiFIIwljgdbc21e23mfZYqkrZHIEuc+agsY8kJhew2FjkFgDgJyAI+yt6kVubaY+lantH89mklAJd5qwh+Y8LpXqmSwMA6E8QhP2M19XcVH+QXuRdINboMmflFKzEyR8AQI8hCPuBUMjfYv6+sea7xpqdLrtZnTZZmz5t2KTnBWI106UBAPR7CMLo5babrTXfNdbstNXtkygytelTR05/WaXNI0k206UBAMQOBGF0oUKBFssxa3WpzbTH1VmflDpBkz515NQXBRIt06UBAMQmBGFUcHXWNdbutpn22Ex7xPI0tWHy0InPJqWMJ1n4gAAA+haOs4y58uSvISl1vNoweUThn3DlDwAgkhCEkUYvcmsz7Wms3S1RpKsNk4dPWZ2gH0OSLKZLAwCIRwjCSAgGPC3mw3T+uezmpJQCbcbMvOmv8QRypksDAIh3CMI+dNXJHx1+Ss0wgiCZLg0AAC5hJgh9Pl9TU1NCQoJAIGCkgL4TDLhbzEdspr3mi9t8nnZN2pTk7OJRM9Zw+TKmSwMAgG4wEIQvvPDCmjVrlEplU1PT7bff/vHHH0sk/X4ldPrkz1K1valuvzwpV5c5a2zRmzj5AwCIfgwE4QMPPPDUU0/x+Xy73T59+vQ33njj+eefj3wZty7gd7VavrdUlZovbiVJltow2TCkZOzst7g8KdOlAQDAjWIgCNPS0ugbUql0+PDhbW1tka/hVjg7ai1V2y1VO8KL3I6b+3elZjjTdQEAQE8wc43w/Pnz27dvr6ioOH369IYNG663m9/vb2xsrKqqou/y+fzk5ORI1XiFgN/VVLfPUlVqrdnJYnHVhkkZw5YUFP+dw+v3jboAAHGuT4Jw//79X3311VUb2Wz2n//8Z/q2w+EwmUwmkykQCLjd7us9zvnz5/fv3//666/TdxUKxa5du1is7sfbeb1ekiR5vN5cft3Rdr7J9F2r+WBH0yl50nCVviBv1vsS5QD6p24vRXjtvfh0/ZfT6aQoiukq4pHD4WC6hDjldDpDoRBJohNApPn9/lAo5PP5bnB/gUDA5f7E4jx9EoQikUirvXpuzK4BlpeXl5eXRxDEM88889RTT12bmrScnJw//OEPJSUlN/KkPB6vV4IwFPRZa3aaL3xjrfmOJ5Br06cNyf91YvI4Nod/i48cw0iSjIEeT/2UVIpr0gwgSVIsFiMII48OQj6/Nw/IfRKE4Zz7SQaDYf/+/X1Rw80Khfw20976iq8sVdvlSbkp2cU5BStFslSm6wIAgL7FwDXCl19+eezYsUlJSWfPnn3ppZeefvrpyNcQRlGhFvP3Dec31VV8KZLqDUPuGTrxdwKxhsGSAAAgkhgIQi6X++c//7mtrU2v17/88suLFy+OfA0EQbQ1njKd/Vd95SaeQJ6cXTx14SaxIp2RSgAAgEEMBOFvf/vb3/72t5F/XlpnS0V95SbT2c/YHH5ydvGUks8lykymigEAAMbFy1yjdP7VVXwRCvpTB901Yd7HUlU200UBAADzYjwInR2m+sqNtWc2BP0u/YDZo2e+kZh8G9NFAQBAFInNIHTZG8wXvq6v3OTqrE/OvmPUjDWJyWMx7ScAAFwrpoLQ47SajNvqKzc52i5q0qcNGvuoNmM6SbKZrgsAAKJX7ARhfcW/zx54UZ81O6dgZVLqBOQfAADciNgJwuTsu1IGzuMLREwXAgAA/UnsBCHJ4mC6IwAAuFndT2ANAAAQJ2InCCsqKi5cuMB0FfGos7Nz3759TFcRp7Zu3cp0CXHqwIEDHR0dTFcRj6qrq8+ePdu7jxk7Qbhu3brPPvuM6Sri0dGjR8MLbEGELVq0KBgMMl1FPHr11VfLysqYriIeff7555988knvPmbsBCFBEFgSDwAgtvXFcT6mghAAAOBmIQgBACCukdHcnDhmzBi/369Wq29k54sXL7JYrIyMjL6uCq7S1tZWVVU1evRopguJRzt37pw6dSoGDkXesWPH0tPTVSoV04XEndraWp/Pl519o6smzJ8//1e/+tWP7xPVQbh792673S4QCG5k59bWVhaLpVAo+roquEogELBYLKmpqUwXEo+qq6vxzx8j6uvrNRoNl8tlupC409HREQwGb/xfkIyMjKysrB/fJ6qDEAAAoK/hGiEAAMQ1BCEAAMQ1BCEAAMQ1BCEAR9aQVQAADBNJREFUAMS1GFl9oqOjY+vWrSwWq6ioSCqVMl1OLPP7/bt37w7f7dojq62tbevWrVwut6ioSCKRMFRgDGpqajIajWlpaV07iAYCgR07djQ2NhYWFqalpYW319fX79y5MzExcebMmejTeIucTufp06d5PN6oUaPCG3fu3BkKhejber0+JyeHvu1wOLZu3er3+4uKipRKJQPlxpDa2tojR44Eg8GCggKDwRDeTh/qSZIsKiqSyWTh7ceOHTt58mROTk5+fn5Pno/q/+rr61NSUu6+++677rorIyOjsbGR6YpiWVNTE0mSM37w4Ycf0turq6u1Wu299957xx13DBw4sKWlhdEyY8e8efMEAoFEIvnTn/4U3hgMBmfNmjV27Nhly5apVKqdO3fS2/ft26dSqZYuXVpQUFBYWOj3+xmqOhb893//N4/HU6lU06ZN67qdx+NNnjyZ/v6vWbOG3tjS0pKdnT1nzpyFCxfqdLqamhomSo4Ra9euTUxMvPvuuxcuXCiVSj/55BN6e0NDQ2pq6oIFC+bNm5eenm61Wunta9asSU5Ofvjhh9PT059//vkePGMsBOHKlSuXLFlC37777rt///vfM1tPbGtqamKxWNduf+SRR5YtW0bfnjNnzksvvRTZumJWTU2N3+9fsGBB1yDcunVrenq6y+WiKOrtt9+eMGECvX3atGn0odnr9Q4aNOiLL75gpObYYLVa7Xb7W2+9dW0QWiyWq3Z+6aWXZs+eTd9+6KGHHn300QhVGYsaGhro7zZFUR9++KHBYKBvP/XUU4sWLaJv33vvvc8++yxFUXa7XSaTHT16lKKoixcvCoVCm812s88YC9cIN23adPfdd9O377777s2bNzNbTzzYvXs3Pd1BeMvmzZvvuece+jY+hV6UlpbG4Vx9CWPz5s133HGHUCgkCOKee+7Zv39/a2ur0+n87rvv6L8FHo9XXFyMT+FWaDSa67Xwl5WV7dy5s6WlJbyl6/f/nnvuwTt/K/R6Pf3dJghCp9P5fD76dreH+t27dycmJtJt15mZmTk5Odu3b7/ZZ4yFIGxoaEhOTqZvJycnNzQ0MFtPzNPr9WvWrHnyySfT09O3bdtGEARFURaLBZ9CxHT9ziclJfF4vIaGBvo0BZ9CX0tKSvrb3/62atWqjIyMf/7zn/TGq45CZrOZwlwltywYDL744ovLli2j73Z7qG9oaEhJSQn/Ss++9rHQWSYYDIbnWmSz2YFAgNl6YltCQkJtbS2LxSII4q233vrFL35B/82HQiF8ChETCAToj4BGv+H0HwI+hb5WW1vLZrMJgvjiiy+WLFlSXFwsk8mCwWD4E2Gz2aFQKBQK0btBz1AUtXz5cjab/fzzz9Nbuj3Ud91IEASHw+nB1z4Wzgh1Ol1TUxN9u7GxUa/XM1tPbCNJMvwHv2jRIqvVarFYWCyWWq3GpxAxer3eZrPRtzs7O91ut16v12q1FEU1NzfT2xsbG3U6HXM1xqxwvM2bN4+iqMrKSoIgdDpd+BNpbGzUaDRIwVu0YsWK8vLyjRs38vl8eku3h/qu7zzR04NPLATh1KlTw43C27dvLywsZLScOHLs2DGBQJCUlEQQRGFhIT6FiCksLCwtLaU78W/fvj0nJ0ej0cjl8ry8PPpToCiqtLR06tSpTFcay86ePevxeOjp5gsLC+nLBAS+/73hmWee2bt375YtW7oOh+v2UD9+/Pjq6ura2lqCIFpbW48ePTplypSbfbpYaBp9/PHHJ0yYIBaLA4HAhg0bDh8+zHRFsez999/ft29fTk5OU1PT3//+9xdeeIEerLZy5crCwkI+n+9wOLZs2XL06FGmK40Rn3766e7du48fP15XV2cymX7+859PmDBhwYIFq1evXrRo0W233bZmzZrXX3+d3vnZZ5/95S9/2djYeOrUqba2toULFzJbfL927Nixd99998yZM1VVVQ8//PDYsWMfeuihr776au3atSNHjrTb7R988MFjjz2m0WgIgnjkkUdGjRq1cuVKiUTy1ltvdR1rCzfr448/fvnll0tKSp5++ml6y1//+lcej7dixYqCggKpVEpR1KefflpWVkYQhFqtfuihh+bNm/eLX/xi/fr1JSUlPViPJUZWn6isrFy3bh1Jkj/72c8yMzOZLieWNTQ0bNq0yWQyyWSyadOm3XbbbeEfnTlz5l//+heHw1myZEnXId5wK/bt23fmzJnw3SlTpgwaNIggiPb29v/7v/9rbm6eOXNm13+B9+/f/8033yiVygcffBCr5d2K6urq0tLS8N2srKzp06c3Nzd/9dVXVVVVIpFo/PjxXc+5a2pq/vnPfwYCgZKSkvAoe+iB06dPHzx4sOuWpUuX0n2nz58/v3btWpIkFy9eHJ7KIxQKbdiw4cSJEzk5OYsXL762l/VPipEgBAAA6JlYuEYIAADQYwhCAACIawhCAACIawhCAACIawhCAACIawhCAACIawhCgEg4ceLEp59+2heP3Nra+t5775lMphv/lY8++ujQoUN9UQxAf4QgBIiEf//73w8//HBfPHJ9ff3DDz98+vTpG/+VlStX/vvf/+6LYgD6IwQhAADENQQhAGOcTmd7e/uP7ODxeMKrSdDsdrvL5fqRBwxPz38Vl8vV2Nh4vV+kKMpms2HZJohPCEIABjz++OPJyckSiUSpVGq12lWrVtFLSRAEYbfbVSrVW2+99dBDD8lksqSkpGHDhlVWVtbX18+YMUMmk8lksnvuucfhcHR9wKamprlz50qlUrVanZeXd/LkyfCPPB7P0qVLFQqFVqvNyMgIL5JAO3HixLRp0wQCgUajEYlE+fn533//fQTeAYDogSAE+P/t3U9IKlscB/CxP5oyYG4iqUVGRJKiREWbNgZhG4PcBBVJLSKIgvAuIlcZURpupF1Ui1oUYdgijTZBrVrkVIRS0iCVUNI/m1Lsz7zFwLzB+97i3bpefH4/qzPnd87MGTc/z5kD5w9gGMblclEUdXh4ODAwMDEx4Xa7uRDLsg8PD3a7nWXZ3d1dr9d7d3fX09PT0dHR0tJycHDgdru9Xu/s7KzwhlartbKykqIov9/PMIzRaLy/v+dCQ0NDy8vLMzMzp6enk5OT/f39z8/PfMdYLNbU1LS9vR0MBjc3N1mWbWtre3p6ythPAfDnsQDw+9lsNpIk/y3a2dnZ2NjIlbkk1NzczEcdDgdBEGNjY3xNe3u7Tqfjytzkz2Aw8FGKokQikdPpZFk2Go3m5+f/+PGDjy4tLREEYbVa/3EkV1dXBEGsrq7+yksCZKf/w3mEAFnn8/PT6/VSFHVzc8OybDgcpmla2KC1tZUvV1dX/1yzt7cnbG82m/myTqerqqoKBAIEQRwfH398fAijZrPZYrEI+0aj0bW1tUgkwn19FIvF4XD4O94SIDtgaRQg0xiGqa+v7+3tDQaDEolEoVBIpdK01UiFQsGXxWLxzzWpVErYvrS0VHipVCovLy8Jgri+vk6LkiQpPPV7fX1dpVLNz88zDFNcXKxQKPLy8uLx+Le8KUBWwIwQINM8Hk8gEDg5OdFoNFzN8PBw2kmk/1Xa5tJYLKbVagmCKCkp4aL8UcnJZFK40cZutxsMhq2tLZFIxEWdTudXRgKQdTAjBMg0mqaLiopqa2u5y/f397SdnL/A7/fz5YuLi7OzMy7LajQakUgkjPr9flZwHDdN03V1dVwWJAjC5/Px+1cBcgQSIUCm6fX6ZDI5PT2dSCQikYjFYolGo1+8587OjsvlisfjoVCou7tbJpP19fURBFFRUWEymRwOh8fjeXl52d/ft1qt3ForP5iVlZWjo6NkMunz+UZHRwsLC784GIDsgkQIkGkmk2lwcHB8fFwmk6lUqkQiMTIy8sV7Tk1NLSwsyOVytVpN0/TGxkZZWRkXWlxcbGhoMJvNJEkajUabzSaXy/mOc3NzBQUFer1eKpV2dXU5nU7hF0SAXCASLpIAwG/C7dLOy/v7r+ft7W0kEikvL1cqld/1iFAo9Pr6qtVqhXM+Tjgcfnx8rKmpIUkyLfT29nZ+fp5KpdRqtUQi+ZbBAGQRJEIAAMhpWBoFAICchkQIAAA5DYkQAAByGhIhAADkNCRCAADIaUiEAACQ0/4CLNDEGmW1wL8AAAAASUVORK5CYII=",
"image/svg+xml": [
"\n",
"\n"
],
"text/html": [
"\n",
"\n"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"using Mosek, MosekTools\n",
"\n",
"# solve at a grid of λ\n",
"λgrid = 0:10:200\n",
"β̂path = zeros(length(λgrid), size(X, 2)) # each row is β̂ at a λ\n",
"β̂lasso = Variable(size(X, 2))\n",
"@time for i in 1:length(λgrid)\n",
" λ = λgrid[i]\n",
" # objective\n",
" problem = minimize(0.5sumsquares(y - X * β̂lasso) + λ * norm(β̂lasso[2:end], 1))\n",
" solver = Mosek.Optimizer() # MOSEK this time!\n",
" MOI.set(solver, MOI.RawOptimizerAttribute(\"LOG\"), 0) # keep silent \n",
" solve!(problem, solver)\n",
" β̂path[i, :] = β̂lasso.value\n",
"end\n",
"\n",
"plt = plot(collect(λgrid), β̂path, legend=:none)\n",
"xlabel!(plt, \"lambda\")\n",
"ylabel!(plt, \"beta_hat\")\n",
"title!(plt, \"Lasso\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Example 4: elastic net\n",
"\n",
"* Elastic net [Zou and Hastie (2005)](https://www.jstor.org/stable/3647580)\n",
"$$\n",
"\\begin{array}{ll}\n",
"\t&\\text{minimize}& \\frac 12 \\|\\mathbf{y} - \\beta_0 \\mathbf{1} - \\mathbf{X} \\beta\\|_2^2 + \\lambda (\\alpha \\|\\beta\\|_1 + (1-\\alpha) \\|\\beta\\|_2^2),\n",
"\\end{array}\n",
"$$\n",
"where $\\lambda \\ge 0$ and $\\alpha \\in [0,1]$ are tuning parameters."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" 0.366815 seconds (491.97 k allocations: 42.703 MiB, 31.14% compilation time)\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAGQCAIAAAD9V4nPAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdd3gc1aE28DNbZnsvWvUuWZZsWe69Q2wHU4xtwOBwKRdCCXYIH70lxIQShyQ3FwffxHQIxsGAsXHHBvcuS3JR71qV1fY6uzvfH+sIY8td2llJ7+/h4ZFmR7uvF7Gvz8ycMxTLsgQAAGCg4nEdAAAAgEsoQoAesH///vvuu++rr76K2is++uijS5cujdrLAfRjKEKAi/nggw/oCystLY3sVllZuWrVqsOHD/fsq7vd7pUrV3799dfdBvv444979uWuUW1t7cqVK/fu3ct1EIArI+A6AEBMC4VCDMMYDIb8/PzzH5VKpb366lar9cEHH5w4ceKNN954zkOTJ08WCoW9+upX6tixYw8++ODSpUvHjRvHdRaAK4AiBLi0KVOmfP7551yn+IlvvvmG6wgA/QSKEKC3BAKBffv21dTUmM1mk8k0fvz47Ozs83erra09dOhQY2MjTdOR3UwmEyHEbDaXlJQQQlwuV9dBV71en5qaSgg5duwYj8cbOnToOc9WX1///fffm81mrVabnZ09fvx4Pp9/oYROp7O8vDzynE1NTZs2bbJarVlZWbNnz6Zp+vz9W1tbt23b1tLSIpfLJ0yYUFBQ0PVQVVVVVVVVZJ+utKmpqXq9/kreMwAusABwYatWrSKEzJ8//+K7ffTRR4SQF1544ewtCoXi7P/XKIr6xS9+4fP5uvYJh8NLly6lKOqc3dauXcuy7LJly87/H/aBBx6I/KxCodDr9WdncDqdv/jFL3i8n5z4T0hIuEjsbdu2EULuvffev/71r2cfaB00aFBDQ8PZewaDwSeffPKcdpw/f77L5YrsMG/evPPTvvPOO5d+iwG4hhEhQK9obW39+c9/fuutt2ZmZkokkpMnTy5btuyDDz6Ii4t74403Ivt8/vnnf/7zn4uKin7/+98XFBR4vd66urqvvvoq0jeLFi3KyMi44447CgoK3nrrrciPJCYmdvtywWBw7ty5O3bsGDNmzDPPPJOfn2+1Wo8dO/bBBx9cMur27dtXr169bNmyKVOmOByOZcuW7dix41e/+tXatWu79nn00Uf//ve/FxUVPf/884MHDzabzX/84x/XrFnDsuyaNWsIIS+99FJhYeFLL700b968hx56KPJTeXl51/AWAkQL100MENMiI0KtVjv2PLfeemvXbuePCM9ns9kSExPlcrnf749sefDBBwkhO3fuvNCPNDQ0EEImTpx4/kPnjAj/+c9/EkJGjx7t9Xov/08XGRESQrZs2dK10W63q9VqPp/fNXjdv38/IWTw4MEej6drt3A4PH78eELIgQMHIlsixbl06dLLDwAQCzB9AuDSHA5H2XkqKiqu6ElUKtWUKVNcLtfJkycjWzQaDSGkR+YbfPLJJ4SQF198USwWX+nPFhUVzZw5s+tbpVI5duzYUChUX18f2fLhhx8SQp544gmJRNK1G0VRv/zlLwkhGzZsuMbwANzCoVGAS7v55puv9KpRlmU/++yzVatWlZeXt7S0BAKBroc6OjoiX9x5551/+tOfnn766Y8//nj27NnTpk2bOnXqVTQZIaS4uJgQUlRUdBU/m5ube86WuLg4Qkhra2vk6p6jR48SQnbs2HH69Omzd4sMWGtra6/iRQFiB4oQoFc8/fTTb7zxhk6nu+GGG5KTk+VyOSHkyy+/3LdvXzAYjOxTUFCwb9++F198cevWrSUlJW+88YZMJluyZMmLL74oEomu6OWcTidFUZECu1Lnz4aMXHETDocj39psNkLIV199dc6VOIQQjUZz/kaAvgVFCNDzzGbz8uXLk5OTDx8+bDAYurYfOHDgnD2LiorWrVvndrv37NmzefPmVatWvfrqq8Fg8PXXX7+iV1SpVG1tbS0tLUlJST3wB/ipSItv3rx59OjRPf7kAJzDX+UAel5JSUkoFJoxY8bZLRgOh48dO9bt/jKZ7LrrrnvzzTd3795NCIlch0kIiUxp6BpBXkTkoOiRI0euPfyFnnzPnj0X3+3y0wLEFBQhQM+L9F/XxSYR7733XnV19dlbXC7XOT9oMpl4PJ7f7+96HpqmGxsb2UvdLm3x4sWEkN/+9rcej+caw5/vnnvuIYT86U9/amtrO+ehYDDo9XojX0emdkROHAL0ITg0CnBpR48effTRR8/ffvfdd48aNer87Xl5eXFxcdu3b3/88cfvuusugUDw9ddf//73v8/IyDi7C2+//fZwOLxw4cKcnBy9Xl9bW/vaa69FtkR24PF4w4cP37dv3x133DFu3DiJRJKXlzdp0qTzX/GOO+746KOPNm7cOGHChKeffrqgoKBrHuH5x2Ov1OjRo3/961+/9dZbI0eOfOKJJ4YNG6ZSqWpqag4cOPD++++vWbMmsrhoTk6OUqncsGHDkiVLcnJyhELhpEmTMJUQ+gCu528AxLTIPMIL+eCDDyK7nT+P8LvvvtPpdF170jT95z//+eGHHyaEbNy4MbLPww8/LBD85C+jPB7v3nvvPXsuYElJyahRo7ouSLnIyjIej+eBBx44Z0G17Ozsi/zpulaWOWd7ZAh49gTHcDi8fPnyyHyPLhRFjRw5sqqqqmu3devWZWZmdu2AlWWgT6BY3KEe4MIcDkdra+uFHjWZTJF11JxOp9ls1ul0Wq327J/du3dvXV2dRqOZNm2aXq/v6OhwOBzx8fFdE/KcTufBgwebmpp8Pl9iYmJRUVF8fPz5L8QwjNlsZhhGqVRGVu+srq6mKCo9Pf2cPc1m865duywWi0qlys3NvfiECq/X29LSolAozj6XSQhpb293Op0JCQnnzOXweDx79+6tqanh8Xgmk6mwsLDblW68Xq/ZbGZZVq/XK5XKiwQAiAUoQgAAGNBwsQwAAAxoKEIAABjQUIQAADCgoQgBAGBAQxECAMCAhiIEAIABDUUIAAADGooQAAAGNBQhAAAMaChCAAAY0GK6CD/99NPy8vLL3Dmydmqv5oEL6bqVOUQZ3nmu4J3nSm981Md0Ea5du7a4uPgydw4EAgzD9GoeuJDeuAceXA632811hAHK4/Hgb96cCAaDgUCgZ58zposQAACgt6EIAQBgQEMRAgDAgIYiBACAAQ1FCAAAAxqKEAAABjRB9F+SYZjS0tLDhw97PJ7HHnss+gEAAAC6cFCE69ev//Wvf52YmHj69OkeLEJfe4ANETqV7qknBACAgYCDQ6M333xzTU3Na6+91rNPy7hCp//Z1Lrf2rNPCwAA/Vv/OUeoSJfk/TKpaael/JPGMIPVjwAA4LJQXK0StGvXrltuuaW9vf0i+4wdO1Yqlaanp0e+FYvFr7/+Oo/XfXmfsgS+bxfQLC9ld5vIydRMMwXkwp7PDd3x+/0ikYjrFAMR3nmu+P1+mqYpiuI6yIATDoUm6plBuss9C0bTtEBwiZOAHJwjvHxKpTItLW348OGRb1UqlUQiudDO7SFeiYPi8/nFw5IGVVsLvmnaNSK+xSiLVtgBjWFYoS+mf5f6K7zzXGEYViAQoAijLxzmZar4haLLHedcaOx0tpj+X0itVl933XULFiy4nJ2nxgenJVA0LSCEkMkGe5VU9VFjwiRt0jQDwe9qL3O5fHL5Bf+OAr3H6fQqFHjnOeBy+WQyMYow+hiGCYcpPp/fg8/Zf84RnkOVKStckmEpcZ5YVRf0hbiOAwAAMYqDImxoaBg5cuT9999vs9lGjhy5cOHCXnohkVo45JF0sZYu/nO1p9XfS68CAAB9GgeHRo1G4zvvvNP17UVO+107noDKuCW+7ZCt5H9rMm9N0Bcqe++1AACgL+KgCEUi0YgRI6L5isaRamm8+NS79a4Gb+ocI8XDYX0AADij354jPIc8UVy4NMPV4C1bWRdwBLmOAwAAsWKgFCEhRCgX5D+YqkyXHv1jpXmflXAzfxIAAGLLACpCQgjFo1J+ZhzyUFrrfuvxv1V723AFDQDAQDewijBCGi8ufCzDOFJ9/G819Zva2BDGhgAAA9dALEJCCKGIaZy2cGmGs85z7K0qV72X60AAAMCNgVqEhBBCxFo6/4G05OsMJ/5ZV722JRTAUt0AAAPOgC7CCH2havhT2eEge/TNSlu5i+s4AAAQVTG91mjUCKT8rAUJ1pPOis+aVJmyjJviBbKeXMgOAABiFkaEP9LkKYY/mc0X8478sbKj2M51HAAAiAaMCH+CL+JlzkswFHmqvmhu3mlJv9GkSJNyHQoAAHoRirAbynRp0eNZHcftpz9qFOvpjJvjpSbc+xQAoH9CEV4ARfSFKl2BsmV3Z8nbNbohytRZRqECbxcAQH+Dc4QXQ/GphMm6EU9nCyT8w29U1H7TGvJjigUAQL+CIrw0gZSfdkNc0eNZAQdz5PUK895ONozFaAAA+gkc67tcIo0wZ1GSq95bs87c/L0lZZZRX6jiOhQAAFwrjAivjDxFMuSR9NQ5cXXftpX+vdaJtdkAAPo4jAivhm6IUpuvaN1nPfV+vURPJ80wqHPkXIcCAICr0X+K0BsKURRFR+vlKB5lGq+NG6NpP2qv/rKF4lOJU/SG4SqKR0UrAgAA9ID+c2j0y5bWwu07N7e2RfNFKT5lHKke/v+yU2fFtezpPPyHiubvLeEgLqUBAOgz+s+I8I6khGSJ+KEjxcPUqhUjhunoqA0OCaGINl+hzVc4ajyN29obt7WbxmsTpugEYixYCgAQ6/rPiJAQMkmvO3b99Ay5bOjmbWsam6IfQJkuHXx/av6DaT5L4PAfKuo3tQXdoejHAACAy9evipAQIuXzXxuSv2bcmBfLTs7dtbfJy8FVnbIEcc6ipKGPpvttzKE/lNd8ZfZZAtGPAQAAl6O/FWHEOJ22+LrpE/W64Vu+W1ldy0kGiUGUfVti0RNZFJ8U/6W6bGWtpdSBmfgAALGm/5wjPIeQx3tqUM7s+Lj7Dh5d29T8zohhKVIO7iMhUgvTbjClzo6zlDpadnVWrWk2jtTET9CKNMLohwEAgPP1zxFhl6Eq1d4ZU6Ya9EVbvnv9VHmY5WZARvEpfaGq4JdpQx5OJ4Qc+1Nl6d9rO8ucBONDAACu9fMiJIQIKOqpQTn7Z0zdaG6dvOOHU04nh2EkRlHaDXGjXhpkHKmu39h6+LXyxu3tDC6oAQDgTv8vwogsuWzblIkLkxInfff9c6UnnMEgh2F4Aso4Uj3sN1k5dyR5zP7Dr5ZXfNrkqPVggAgAEH399hzh+XgU9Vh25q1JCb87cSprw+bHc7J+nZNF87j8q4AiTapIkwbdodaD1srVTWGGNY5QG0aoJAbcBxgAIEoGyoiwS6JE8s6Iou+mTvqhwzJ087bPuZhueA6BjJ84VT/8yezB96USipS9U3fk9YrG7e0BJ5fDVgCAAWIAjQjPNlip+GbiuK2tbU8cL32rvPKPhUPG67RchyJSkyjFZEy53uio9bQdsh15rUKeLDGOVOuGKPmiAfdXFgCA6BigRRgxM854ZOa0j+obFuzdP0Gne21ofoZMxnUoQiiiTJcq06UZt8TbTrvaDtmqvmjW5CqMI9WaPDkW9QYA6FkDuggJITyK+kVqyvykxP+pqBq9dcedqcm/zc9TC2Nikh9PQEWWMGVcwfaj9oatbZWfN+mGKnVDlKpMGRoRAKBHDPQijJDy+U8NyrkvPe2Vk6dyv93yeE7W0pwsEafX0ZxNKBckTNIlTNJ52wOW4/a69a1eS0A7WKEbotTkynnCWMkJANAXoQh/pBfRfxk29L/T0/7f8dKV1bW/yc26Jy1Vwo+hO0hIDHTSDEPSDAPjClpPuVr3Wcs/aVRlyPTDVNp8hUASQ1EBAPoKFOG5ClTKbyeNP2K1/bmi6pUTpx/MSPtVdmZUb+p0GYRygXGk2jhSHfSEOk84O8uc1V+0SONF+kKVfpiKVuI/KwDA5cInZveGa9QfjB5R5XL/tbJq0MYt8xITnsjNzpbLuc51LoGUH2nEkD9sPeW0lDjqN7VJTSJNnkIzSC5PlBCcSQQAuCgU4cVkymV/GTb0ubzctyurx23bOU6nfXHwoFFaDde5usEX8fSFKn2hig2x9kq39ZTz9EeNQW9IM0iuGaTQ5MoFUhw4BQDoBorw0owi0cv5eb/JzV5VUzdvz/5UmfSp3Jy5CSauc3WP4lPqXLk6V55+E/F1BmynXZbjjqo1zWIdrc6Rq3NkqixccQoA8CMU4eVSCARLsjMfykz/sK7hyeOlr5w89WRu9i2JCXwqdktFrKVN47SmcVo2xNqr3dZTruovzYwrqM6VawfJ1TlyoQK/AAAw0OFz8MrQPN596an3pKWsazG/ebri8eKSe9JS701PTeXiZoeXj+JT6my5OluePpf4bYz1lKujxFH1RQutEqqzZaosmSpLhotOAWBgQhFeDR5F3ZQQf1NC/Cmn873a+tFbdwxVqxanJi9ISoyp6RbdEqmFprEa01gNG2bdzT5Htaf9iL3iX020UqDMkKlzZOocOUoRAAYOFOE1GaRQvDYk/7f5eV83t6ysrv31sZL5SQmPZGUMVam4jnZpFI+SJ0nkSZKEybpIKdrKXea91opPmyRGUeSEojJdign7ANC/oQh7gIjHW5CUuCApsdHr/biu4cZd+/Qi+oGM9EUpSXJB33iHu0oxabohzIQdtR57hbt+Y5u7xSdLECszZJHlTzFSBID+p298TPcVSRLJU4NyfpObvaHF/I+a2mdKyhYkJd6XnhqbMy4uhCfkRU4oEkJC/rCzzuOo8TTvtJz+sEGspSONqMyQiTQxsSIrAMA1QhH2PAFF3ZgQf2NCfLPX915t3Z37D7GEXZCUuDA5aZi6DxwyPRtfxFPnyNU5ckIIG2a9bQFHjdt62lW7oZUNsfIkiTJDqs6Rkz72xwIA+BGKsBclSMTP5uU+m5db5nB83tB0+74D/nD4xoT4BUmJE/U6rtNdMYpHSU0iqUlkGqclhHjbA45at6Pa03qgMWBnFClSRapEniJRpEixxhsA9CH4wIqGfKUyP1/5cn5epBHvO3Qk0JcbMUJioCUGOm6UhhDisDiJg++ocbfus1aubqZ4RJ4kkSdLIkNGnFkEgFiGIoyqcxrx3oNHgmx4bh9vREIIT0TJ06XKdCmZTgghPkvAWedx1nsbtrV7PvSJ9bQiRSJPkcqTJTKTiOLH7hIEADAAoQi50dWIBzutqxub7tp/SMTnzTGZ5sTHTTboY+dWiFdHrKPFOtowXE0IYUOsu8XnrPM6atzN31t8loA0TiRPEssSJfJEsSxBzKP79h8WAPo6FCHHRmk1o7SaN4YWHLPZNrS0vlx2stThmG40zDaZ5sTHJUkkXAe8VhT/zMSM+AlaQggbYr3tAVej19XgbT9iczf7hHKBNE4UOY6qSJMKZTiOCgBRhSKMCRQhRWp1kVr9XF5uZyCwra19a2vby2UnJXz+DQmmufGmKQa9sI8PEyMo/pkrbowjz4wXPWa/q8nravQ1bm+P9KIsQSyLF0vjRbJ4sVhPY4lwAOhVKMKYo6XpyPT8t4ezBzqt61vMT5WU1bo918cZ58THzYwzxovFXGfsMRSfkiWKZYniuNGEEEJY4m33u1t87hZ/+xF7bXNrwBmUGkWRUpTFi6XxYlySCgA9C58psYtPUeN02nE67e8LBrf4fBtaWr9qbll6rCROLJpq0E8zGqYa9AaRiOuYPYoiEqNIYhTpC89sCAXCXrPf1ezzmH3Wky53i49lWVm8WGoSS00iqVEkNYmEcvwaA8DVwydI3xAvFt+XnnpfeiohpNrt3tra/nlD0y8PH1MLhTPjDDPjjNONBh1Ncx2z5/FpnjxFIk/58Vwp4wy6W3wes9/d5Os4aveYfSxLpHEiaZxIEieSxoklRlqsoQmOpwLA5UER9j0ZMtkDGbIHMtJCLHvEavuuvePdmrr7Dx3JlMmnGfXTjIZJep1K2G/XPxMqBGrFmcVuIhh3yGP2edv8nla/9ZTL2+YPukMSo0gSJ5IaRRIDLTaIJAaaj8tTAaA7KMI+jE9RkYtOn8zNDrLsgU7rjrb2v1RULdp3ME0mHa/TRY6s5ijkl36uvkwo46syZapMWdeWkD/sbfV7Wv2eNn/HMbu3I+Bt8wvlAomBFutFEiMtMYgkBpFYK8SkRgBAEfYTAooar9OO12mfzcsNsewpp3N3R+e2tvY3Tpc3eX2jtJoJOu0IjWayoT8PFrvwReceUCUs8dsYb7vf2xbwtvttp13e9kDAzog0tMRAi/W0RE+LdbRYj3YEGHBQhP0Qn6Lylcp8pfKBjDRCSLPXt9fSucdi+cOp03fstw9SKMbrdGN1mjFabaZcdqkn6y8oItIIRRqhOufHbeEg67MEvG1+nyXgMfstZU5fRyBgZ2ilUKynxXpaoqPFZwoSR1YB+i0UYf+XIBHfmpRwa1ICIcQfDh+22vZZOv/d2Px0SZmTCY7QqEdo1CM0mhEa9QDqRUIIITwBFbnK5uyNbIj1WxlvR8BnCfg6/PZqt88S8HUEBBK+SEeLtbRYKxRpabFWKNbRIjWGjwB9HopwYBHxeJEjqI/nZBFCbAxTanccttq+aTH/9sTJRq93iEr1n2pU5ykUPGrAfcpTfCoyHPzJVpYEnEGfJeDrDPg7GWedp/1IwN/JBByMUCEQa2mRVijW0mIdHRl3ilQoSIA+A0U4oKmFwol6Xdd6321+/2Gr7bDVtrap+fnSE5HxYqFaNVSlGqJS5quUfX0R1KtHEVopoJUCZbr07M1smPXbGH8n47MEfFbGVu7ydTJ+a4BxBgVSvlhLizRCkVoo0ghFmjMdiXtxAMQaFCH8yCgSzTbFzTbFRb5t8/uPWm3FdseW1rbl5RWVLneGTDZEpSxUq4aqlENUqmRpn18K9RpRPEqspcVaWpX1k6PKbJgNOIJ+K+O3Mn5bIDKvI/Ity7KRdqRVQpH6xy9otUAgRkcCcICbIgwGg6WlpXK5PCsri5MAcDmMItHPTHE/+08vBsLhkw7ncbu9xO74S0XVcbvDHw4NVamGqlQ5YlGRQZ+nVGj746T+q0DxqEjJkfRzHwr5wn4b47cG/PZgwM44aj0BO+O3Mn47Q1hyphdVQpFGSKsEtEpIKwW0SkjLBVgiAKCXcFCE9fX1M2bMUKvVra2tkyZN+vDDD3kxc8DNGwrXeQL17oDZx/jCLBNmXcEQIcQaCBFCXMEQE2Z9YdYbCodY1sGEIz8l5fNEZ50QoghRC3/8qz2fopRCvohHSQU8uYAv5FFqIV9AUUohT8TjSQU8uYAX2SgX8OkYXmCa5vEK1apCtaprS7vfX2yzH7c7DlosH7eYTzmcIj5vsFI5SCHPVyoHKRWDlYr+tDJqj+CLeZFlx89/KOQP+21MwMb47YzfyrgafYEyZ8DOBBxBxhOiFQJaLaQVApH6P+2oFDACRiII4XArwLXgoAh/+9vfTps2beXKlU6nc9iwYevXr587d26UM7T5g/XuQL0nUO8J1Lr9kS/qPQFnMJwqpVOktEkslPApAY9SCPiEEDXNpwhJltJCHiXiUVI+j09RSuGZ/nYHw4Ew2/XkYULsTKjr22CYdQZDvhBrDYQaPAEmzFqZUDDMOoNhXyjsDYVdwTATZm1MyBkM8QhRCvkKAV9D8xVCvkLAUwj4CiFfLeQrhXyFgKcS8tU0XyMUqGm+WshX0/xIQk4YRKKZccaZcUaXyySXywkhzV7fSafzpMNZ5nD8u6m5zO4IhMN5SsVgpWKQQpGrUOQo5OkyKR0zf/WJKXwR7/yrWCPYEBtwBgM2JuAI+u1MwBH0tLoC9qDP5q9ytYaZMK0U0kqBUCmglUJaIaBVAlohoJVCoVKA0STAxXFQhKtXr966dSshRKFQ3HbbbatXr+6RIgx67QF7Q1ihF8p0fNG5a6mwhBzsdK9ttG0w2yucfpmAlyKlU6R0qpROldGTDPLIt3Fijieb+0JhZzDsYEI2JuRgQs5gyMmEncGQnQnZmVCLl7EzISsTtAVCNiZkC4RsTNAXYrtKUS0UaGi+RsjX0AItfebfWlqg+c+/e7s1EyTiBIl4htHQtaUzEDjhcJ50OE86ndvb2itcrkavL1EizpbLs+XyHIU8Wy7LVshTpVL+wLtC9fJR/P8ca/0pp9OpUCjCTDjgDAbsQcYZ9NsZxhV0VHkCrmDAzjDOIOMJCWV8oVxAq4RCGZ9WCoQKAS0XRDpSqBAIZWhKGNCiXYSdnZ0ulysjIyPybUZGxu7duy+0s8fjKSkp0Wg0kW9lMtm4ceMutHN12acnyz4kwQDF+Fk2TPMlPKGEL1aeEBbsFRbsorLlVHC62PJQoD493CkKhEiAENuPP+4m5CQhJwkJEtLOsjaWvdALcUJJiPICD4UIz+sTevy0lxV6KKGHFTmJsJWiPSztJrSHoj2E9hDaTWgPoYOEJ2UDMhKQUgxNglIS4BFWzDJCEhKSsIgwfCosJQyPDYtJUEjCQhK8nHgsG6aoSw/ysgnJJoQlhAmwASvbGWa/D4e3hll/mA2yhOYREe/M/ep55Ewr8igq8hnN/3FLLH5osywhbIiwIRKO7usS9oLvB02IgRDDmf0IS5EwIV6KuCnSTAhLSJhHWIqwhLCEUITwCCEsoQgh4TNP2fWflCKEYs88UQy++1xgWYK/uXGDJbMzh86fekMPPmW0i9Dj8RBC6P9cUiEWi91u94V2bmpqqq2t3bVrV+RbiUTy6aefXuiEoiJpSgotpXhsMOj1htjdDv5un2lfIMFEdRYxxUtdXyR6GwkbDFOhpv/8+oYJ5efxfRTfz+P7eDw/xffx+EGKR7NhOhwmJLa68HJICJEQorvwDiEi8PGkPiL18WRBIvRR0hDFZyhRkAj9lNBJxGHC81HSMMUPUOIgJQyS3hwi8876qO3rKHKmIXC2DqCXOSqPzRo59TJ3FovFAsElmi7aRWg0GimK6uzsVKlUhBCLxWIymS60c3Z29oIFCxYsWHA5zywU5kl0uU0CXWkAACAASURBVN9bfeua7V+brcmS4BC66cWWH4qqA/lWtT1+Ljs0kR07ZKPXUuawVLpsVS57R8CbLlNmyzVZcnWmXJ0lV2fJ1SlSheAyBjdwNpfLFTlH2D+E3Y5gZ2vI2hbsbA11tgYtraFOc8jWwYYYvtrAV+n4agNfreer9AKNnq/S81Q6vkLDSdTIoVFOXnqAc7lcMpmMwqgw6hiGCYfDoh69FWu0i5Cm6aFDh+7Zsyc9PZ0QsmvXrlGjRvXIM/+juvn/nbCkCTtS7SdebXJMb9dqPEI2P1czv8A72LSho+7DupOlR7+9NTF7tNa0KGVQllyTJJEPwJVToEvY5w51tAQt5jOF959/U3w+XxMn0MbxtUaBLl6UPUygNfI1Rp4UlQPQD3FwsczSpUufffZZlUpVWVm5devWt956q0eettBZ+23lybxaHl9rlA4fL1yYTqUZvmtv/KDuxPrtW8brEh7OKrwpIZPm4dDVwMOyIXtHsMMctDSHOsxBS3Owwxy0tLBBRqCPF+hMfK1JYEgU5Q6PlB9PPLDWXAUY4Dgowv/6r//i8/n/+Mc/1Gr19u3bk5KSeuRpRyiz2GEa8X2ZPL2izGH5sO7Ee+vXJkkUi1PzlhdOMYgG+hooA0U4HLS2BdubzvzT0Ry0mEMWM0+mFOji+fp4gS5enD9GoEsQ6E08uZrruADAPW5Wllm8ePHixYt79jl56cameNnnLafePVTqC4VuT8ndNe32LHzS9WshR2ewrSnY3hhsbwq2NwfbG4MWM0+uFhgSBIZEoSFRnFvE18ULdPGUEEveAED3+s9aox81nH6mbPdtKbnvj541RhvPdRzoYWwoGGxvDrbWM20NwZY6pq0x2N5ECWmBIUloTBQYEqUjpwsMiQJDIjoPAK5I/ynC25Oy70jOkYqwoFd/wAZ8TGtDsK2BMdcHWxuY1vpQZytfYxSaUgTGZPHg0fKp8wSGBJ6k/1yqCgBc6T9FKKB4uJS5j2KDTNBcz7TUMi01TEttsLUh5LIJjElCY5LAlCYdMVUQlyIwJFL8/vPrCgCxA58sEHUsG+w0M821Z5qvuTbUaRboE4TxacKEdPnEuQJTikBrwrodABAdKELodazfyzRXBxqrmOYaprmGMdfxZAqhKU2YkC4ZMl55/SKBMQmjPQDgCj59oOeFPS6msTLQWMk0VgaaqkLWdqEpVZiUSSdlyUbPFMSnYaIeAMQOFCH0ANbr9rfVMQ0VgcYKpqEiZOsQxKcL41Pp1EGyiXPplBxKwPFtPQAALgRFCFeDDfgCDRWButOBulOBulPhgF+UnC1MyhIPHq28fpFAn4AzfADQV6AI4XIFLS2B6rJAQ0WgoZJpqhJojXRGgaRgrHL2Yr9MK8fSzwDQN6EI4YLCboe/5kSg7lSg9hTTUM5TaOi0QXRKrnTkDGFixtmXt/hdLg5zAgBcCxQh/ESwszVQVeqvKQtUl4ZsHXT6YDp1kGL6fDo1F/deAIAoC/hsfk+H32vxezp8nraAt9Prao9Lm5mYNbMHXwVFCGeOefprTvhPH2EDfmFKjigjXzpyBp2ai1kNANBL2HDwTMO523yejoDX4nWb/R6L32vxudv8XkvAY+ELJWKpgZZoRRKdWGYUSfVyTaZEmdyzSfAxNyCFw4GGcn9lib+6NFBTxpOpRBn5dEaBYtqtAkMi1+EAoD9g2ZDf0+HzdPhc5sh4zudp93ssPneb39Pu91oCPptIohNJ9WKZUSzViyR6icyk0ueLpDqx1CCS6kUSHY9/7tLBkRvz9mxUFOEAwrTW+8uP+cuP+itL+BqDKGuobPRMzW1L+Upu7q4OAH0a43f43K0+T7vPZfZ7LF53q9/T5nO3+9ytPk8H47PREq1YqhfLTSKpXiw1yhTJ2rhhYplRJDWIJDqRVEdITFxejiLs50JOa6Cq1Fd+1HfyEAkF6Yx8cd4o9a2P8NV6rqMBQKzzezr8ng6v2+xzt/lcZp+n3esy+z1tXnebz2Xm8YRiuUks1Yvl8SKpTiKLU+pyxVK9WB4nkuhFUj1F8bj+E1wWFGE/xPq9gbpTvtNH/eVHg52tdFqeKCNfd+8LdHI219EAILYwfofXZfa5zV5Xq8/V4nW3+VwtPk+7x9nk97QLaaVIapDITWKZUSyLk6lSdQmjxbI4icwolpv4gn5ytx8UYf/BmOt9Zft9ZfsDTVWi9MGinCLNbUuEiZmY2w4wkIXDjM/d5nU2e11mn8vscTX7XK1eV7PP3eZ1tfD4tERuEstMEnmcWGaSq9MMSePEMqNEkSCWGnn8AbEmFIqwjwuH/bUnfWX7vaV7WcYvHjRSPvkmcd5ISiThOhkARE8o6PM4m7zOZq+rxets8brNXleLz9XidZkDPqtIapAqEsWyOIk8XqKI1xiHSBSJYplBIo/nC/BZgSLsm8Juh7+i2Fu2z1e2X6CLF+eP0d71JJ2UhcEfQD8WCvq9riavy+xxNHmdzV53i9fZ7HE0eV0toaBXokiQyOOligSJPEGpyzGlThXLTRK5SSQ19JVzdVxBEfYlTEutr3Sft2xfsLVRlFMkzh+tvukBnlzFdS4A6EGsz93mcTZF2s7jbPQ4Gr2uZo+zmfE7pIpEidwkUSRKFAlK3SBT2nSJIlEijxdJtFzH7sNQhH1AsK3Rc3Sn9+hONuAXF4xVzbmbzhyCqe4AfVo4FPA4Gj2OBrez0eto8jgbPY4mj6vZ62ymxWqpIlGiSJAoEmSqFH3SOKkiUapIEEkNXKfun/BhGrtC1jbv8T2eYz+EOs2SwonqhUtE6YNx8BOgbwkFvW57g8fR4HE0RoZ3Hkej29EQ8FmlikSpMkmqSJIqEw3JE6WKBIkiQapIOn8WOfQqFGHMCdk6vMW7PMd+CLY1iAePUs5cIM4bRXh8rnMBwMWEQ4zH2eC2N7jt9R5Hvdte73E0uO31QcYtVSZJlckyZZJUkRSfcZ1UmSxVJotlRpy6ixEowlgRdjt8Jw64D25jGivF+aOVMxeIBo3E8U+AGORzma3m0x3+Vq+zwW2vd9vr3Y56v6dDokiQKZNlyhSpKjkha5ZMmSJTpeB4ZuzD5yzH2FDQV7LXvW9joL5cXDBWMW2eKKcI/QcQC8IhxuNocNlr3bZat73OZa+LfCGgFRJFslKbLlOm6JPGpebfJlOmSBTxFIUjN30SPnA5E2xtcO/b6Dm0XWBKlY2bpbv/ZUowIOauAsSgIONx22pctlqXrcZtq3XZa932Op+7TapIlKlSZOo0uSpNnzhWpk6VqdIEQqnL5ZLJZBTO2fcLKMJoY5mAr2y/a8+GoLlOOmqmYclygT6B61AAA0go6HVZqyOd57LVuG01LltNwGeVqdLl6jS5Ol0dV5iUe6NMlSZVJFA8fEj2f/hvHD2Mud5zcKt7/yZhQoZ8/BzxkHE4BArQq8JhxmNvcHZWOG3VLmuVy1p9TudpTUUpg26RqdOlioQYuRMCRB8+iHtd2OfxHtnhPrgt1GmWjpxhfPyvAm0c16EA+iGfy+y0Vbus1U5rlcta5eys8rqaxPJ4hTpDrs1UG4cm5dwk12Sg8+AcKMJeFOpsc+78wnNwmyh7mPJni8S5wzELEKBHhMOM21bn7KxwdlY4OiucnRVOaxWfL1JoM+WaLIUmQ584RqHJlKnSBsiy0XAtUIS9gmmqdu74t6/sgHTk9LgnV+DmfwDXIsi4nZ2Vzs7KrubzOBoiK2oqtFnG5ImZw+6VqzNoMZYbhKuBIuxh/uoy57bVTEOFbPwc0wvv8iRyrhMB9DHBgMvRWeHoOOnorHB0nHJ2Vvi9nQptlkKTpdTlJA+6RaHJkmsyMdSDnoIi7CEs6zux37HpU5bxyyfdqLvnecyFALgcQcbj7KxwdJxydJY7Ok45OisCXotCm63U5Sp1Ocai+5W6HKkyCWf1oPegCK8V6/e6921y7vyCrzIof3aHePAYnAgEuBA2HHRaqxwdp2wdJxwdJx2W0z53u0KbpdTlKHW5GYV3K3W5UmUy1h6DaEIRXr2Q0+ra+aV777fi3CLdPS/QydlcJwKIOT6X2W45bW8vs3ecdFhOOTsrpYpEpT5PZRicln+HUj9IpkrBgizALRTh1WD9Xuf2z127vpGOmGZ8/K8CnYnrRAAxIRxiHJZTtvZSe/tJe8dJe3sZRfFUhsEqfZ4heXxW0f1KXS5fIOY6JsBPdF+Ev//97xcvXpyamnr2xvr6+rfffvu1116LSrAYxYaCnv2bHRs/ojPyUYEAjN9hay+1t5XZ2ktt7WUua5Vcna4y5KsN+fHpM5T6PLHMyHVGgEvovgj/9re/TZ8+/ZwibGpqev311wduEbKst3iX/Zt3+do4/YOvCBMzuQ4EwAGPs8neVmprL7O1l9rby/zeTpU+T20o0CWOzhx2n0o/CPfSgz7nCg6NdnZ2qlQDdJpOoOaE7et/khCjuW2JKLuQ6zgA0eNxNFrbjttaj1tbi21txymeQG0oUBkLknNvHjLpeZkqDRe2QF/3kyLctWvXN998QwhxuVwrVqz4+uuvux4KBALr1q0rLBxwHcC01ju+/TBQf1o1+xfSkTNwRSj0ez6X2dp23Np63NZ2vNN8jA0HFNocTVxh8qB5Q6e8rNTlch0QoIf9pAhLSkpWrlxJCPF6vV999ZVA8OOjYrG4oKBg+fLl0Q7InZCtw7H5E+/xPYpp87R3PYl5gdBf+dxtVvPRztajVnNkzCfUxA1VG4emD7lr+Iw3xHKcCId+7idF+NBDDz300EOEEJPJ9MUXX4wfP56jVBwL+zzOLZ+6922ST7wh/oV3KZGE60QAPSnIuK2tx62txzpbjljNR4OMR2sq0piKMofdozEORfPBQNP9OUKz2RzlHLHDd+KAdfX/iAeNiHvq73yllus4AD2AZUPOzsrI0U5ra7GtrVSmStYljI7PuH7wuCeUuhys2wID2SUulgkGg06n8+wtGo2mN/NwKexx2b9Z5T99VLvoN6KcYVzHAbgmfm9nZ8shS/MhS/NBW1uJVJmoMRVp44pS8haoDHk8Hg71A5zRfREGAoFXXnnl008/raurCwaDZz/EsmxUgkWb99gPtn+/LR4yLu7Jt3EsFPoilg07LeWWlkOW5oOdLYd9nnatabguYWTemKWa+OFCWsF1QIAY1X0RPvnkk2+//fYvf/nLgwcParXaUaNGbdiw4dSpUy+88EKU80VByGG1/ft/g22NuvtfolMHcR0H4AqEgp6OppOW5gOW5gOW5kM8Pq1PHK1LGJU57D61sQATGwAuB9XtCE+j0Tz33HNPPPHEPffck5CQsGzZMpZlH3roofr6+g0bNkQt3MKFCxcsWLBgwYLL2dnv91MURdNXNpk3MhCUjr5OOXsxrgu9ai6XSy7HDaeixO/ttDQfaG/cY2na7+is0hgLdAmjdPEjtQkjRRId1+kGCpfLJZPJKMynijqGYcLhsEgk6sHn7GZE2NHRYbPZZs2aRQgRCAQej4cQQlHU888/n5yc3NTUlJiY2IMJuBLsbLV99peQy65/8BVhUhbXcQAuxudu7WiKDPsOuKw1GlORLmFUwcTnRMp8lRrlB3BNuilCsVhMCAmFQoQQk8l04sSJyHalUkkIMZvNfb4IWda991v7hvcV025VTJtPeDh8BLHI5zJ3NB9sq//e0nzA7+3Umop0CaOHTnlFG1/UdanLOdeyAcBV6KYI5XJ5UlLSiRMnCgsLx4wZ89Zbb/3www/Dhw9/5ZVXBAJBRkZG9FP2oGBHs/Vfb5Fw2PjYcoExies4AD/hcTS0Nexqr9/V1rCbxxPok8bpE8dmD39QocVBC4De0v3FMvfdd9+RI0fuuOOOOXPmFBUVTZ48ObL92Wef7dPTJ7zHd1tX/49q9mLZ+DlYLA1ihN/T0d64t2vkZ0gaZ0yZnDv6MSxmBhAd3Rfhyy+/HPmCx+Nt27Zt8+bNNTU1w4cPnzBhQvSi9TTXzi+d363RP/gK7qALnAsy7s6Ww231P7TVfx8552dMmZQ+ZLEmbgjmtgNE2aXvPkHT9A033BCFKL0oHLZ9scJfVWJc8ie+BndHA26Ew4yl+VBr3Y62up0ua7U2YaQxeWLRjDfVxnzcoh2AQxcrQp/PZzabw+Hw2Rv73DlC1u+1fPAaCQUNS/7EE0u5jgMDjttWa67b0Vq3o6Nxr0KTGZc6deiU3519wQsAcKv7IqysrHzssce+/fbb8x/qWyvLhOyWjv97kU7O0Sx4lPDwl26IklDQa2k+GDny6XE2G5LGxafPHD7jdbEsjutoAHCu7ovwtttuq6ure+WVV3JycvrujFGmuabj/16SjbleOesurrPAQMDa2kpb63a01u6wth3Xxo8wpU4dcf2fVfo8roMBwMV0U4ROp/Po0aP/+te/Fi5cGP1APcV38pD10+XqBY9JhozjOgv0Z6Ggt63+h5bqLeaarXyh1JQ+I3fUo/qksXwBVqwF6Bu6KcLIVPrMzMyoh+kx7j0bHJs/0d3/Wzolh+ss0D/5PR3m2u/MNVta63YqdbnxGddPnPeZUoffN4C+p5siVKvVM2bM2Lhx44gRI6If6FqFw7Yv3/FXFBsf+xNfiwtEoYc5LKdbqre0VG92dlYaUyaa0q8rmvEmLVZxnQsArt6PRWixWGprayNf/+pXv3rooYc8Hs/PfvYzmUx29g/EcjuyAZ/lgz+FA37DkuU8sezSPwBwGcKhQGvdzpbqTS3VW4UiRXzG9QUTn9UljMKcB4D+4cci/Prrr++9996zH3v11VdfffXVc34gZq8aDTs67e/9XpSSo8MFotATQkGvuWZbU8WG1rrtKv3g+MxZOSMfkavTuc4FAD3sxyKcNm3a6tWrOYxyjZiGcsnwqaoZl3XPJoALiVz80li+rqV6s1KXm5Qzd+iUlzDtAaAf+7EI09LS0tLSLvPHGhsbDx48eMstt/RKqKsiyh/bd2d6AOcCPltL9ZaminUdjfv0SWPjM64fOvklkVTPdS4A6HWXXmKtW8XFxc8888zVFWFzc/Mnn3xy+PBhhmHWrFlzdQEAeoTf095Y8U1TxQZb2/G41KkpefNHz1khEOIEM8AAcpVFeC1qampOnToVFxf38ccfR//VAQghjN/RXPltw+m11tZjpvTrsorui0udxhf05D2vAaCv4KAIJ0yYMGHChF27dqEIIcrCoUBr3Y7G8nXmmq26hFHJg24dO3eVQIgVaAEGNA6KECDKWDZsaT5Uf/Lzpop1Cm1OUs7coVN+K5Jouc4FADGhV4qwo6Pj8OHD52+fPHmyRHIF606VlZXt2LHjySefjHwrk8l2797N4/G63dnv91MURdP0VQSGa+R2u2NzXo2jo6y5Ym1rzbdCkSo+6+Zx89aLpEZCSCBIAk4n1+l6gMvl4jrCAOV2u8PhMC7Qiz6GYcLhcCAQuMz9xWKxUHiJO730ShHW1tb+9a9/PX/7sGHDrqgIc3NzH3rooTlz5kS+VSgUKtUFl/CgaRpFyBWKouRyOdcpfuRxNNadWF13YjVfIErOvWXw7etkqlSuQ/UWhULBdYSBiKIomUyGIoy+SBGKRD15Rr9XinDkyJHr16+/9ucRCARxcXF97g6IwJVQ0NdcuaG27DN7e2lS7s1jb/g/tXEI16EAINZxcI4wHA7b7Xan08myrNVq5fP5SqUy+jGgP7G2Hq8/+XnDqbUqQ35K3vykm97DzR8A4DJdZRFed911u3fvvrqfra2tHTlyZOTrzMzMQYMG7dmz5+qeCga4gM/WVPFNdfF7AZ8tedAt0xd9K1Umcx0KAPqYCxZhc3PzunXrampqrFbr2dvfeecdQghN0zqd7upeMiMjo7Oz8+p+FoAQEg4z5ppttWX/sjTtS8icXThtmT5xNCE4WwMAV6P7ItyxY8fcuXP9fj9N00Kh0O/3e71eiUQSHx8f5XwAZ/O5zDWlH9eUfCRTpabm3z569v9iFRgAuEbdT0VYsmTJiBEj2tvbFyxY8Mgjj3g8nq1bt8bHx7/88svRjQdwhrX1+MGNv9ry4TSvyzzhlo+nLPwyLf92tCAAXLtuRoR+v7+srGzr1q2RuQrBYJAQMmPGjJUrV86bN2/BggVisTjaMWGgYgLOxtNfVR1bFQ4FUvNvHzZtmVCES6sAoCd1U4Q2my0UCiUkJBBCVCpV1znCcePGORyO06dPFxYWRjUjDEjOzsrq4+/Xn1xjTJk0dMrLxpRJOAsIAL2hmyI0GAwikai5uTknJyc9Pf2dd94JhUJ8Pr+kpIQQIpViYUboReEQ01y1sabkI0dneWre/JmLt0vkODMNAL2omyLk8XgTJ0789ttvp06devvttz/99NNTp04tLCxcs2ZNQUEBprdDLwn4rFXF71UXv6fU52UOuyc+4zqK4nMdCgD6v+6vGl2xYkXkiGhcXNzatWuXLVv25ZdfjhkzZvny5Xw+Ppugh7lttRVHVjacXpuY/fPJ8/+t0GZxnQgABpDuizA7O7vr61mzZs2aNStaeWBgsbWVVBxZaa7ZmpI3f+bi7yRyE9eJAGDA6X76xH//93+fPn36nI3l5eULFy7s/UgwELBt9d/v+eoX+765XxM3dPb9hwunvoIWBABOdD8iXLdu3T333HPORovFsmbNmt6PBP1ZOMQ0nP6y/NDfeHw6q+iBcXNXUTzcFBMAuHQFn0GNjY1XvawaQMBnqy5+v6r4XU3c0GHTXzMkjeM6EQAAIecU4fr1699//31CiN1uf+GFF86uPZfLtWfPnilTpkQ7IPR9AZ+t4sjfq4s/SMiaPXn+5wpt9qV/BgAgWn5ShHa7vbq6mhASCoWamprsdnvXQ0ql8u6773766aejHRD6soDPVnn0H9XF78WlTZt+xwaZOo3rRAAA5/pJES5atGjRokWEkDFjxqxYsWL48OEcpYI+jwk4q4vfrzj897i0adNu/wYVCAAxq/tzhPv3749yDug3UIEA0Ldc8GKZ8vLyFStWlJWVCQSCDRs2EEL+9a9/KRSKn//851GMB31JMOCqKn4PFQgAfUv3RfjDDz/Mnj1bpVKlpaXV1dVFNjY3N69YsQJFCOcLMe7TB1GBANAndT+h/uGHH54yZUplZeWyZcu6Ns6aNauystJsNkcrG/QBoaD31P4/f/+vaW573fRFG0fN+h+0IAD0Ld2MCC0WS2lp6T/+8Q+JREJRP974JiUlhRDS3NxsMmEFECAsG647sfrEnjf0iWPG3vyFIX4Q14kAAK5GN0UYuROvSCQ6Z3t7ezshhKbpKMSCGNdW/33J97/jCcSj5/xdnzja5XJxnQgA4Cp1U4RGozE+Pv7LL78cNmzY2SPCDz74QKlU5ubmRjEexBxnZ8WJvW/a2svyxz+dlHMDbpYLAH1dN0VIUdRTTz31xBNPeDye1NTUUCi0d+/ezz777G9/+9sLL7wgFAqjnxJigc9lPrFveXPVxuzhD46a9TceH8cGAKA/6P6q0ccee8xut//hD3/w+XyEkPHjx/P5/EceeeT555+PbjyICUHGU3VsVfmht1Pybv3Zf+0WipRcJwIA6DHdFyFFUS+++OIjjzyyc+fOjo4OuVw+adKk5OTkKIcDzrFsuP7kv0t3v6pPGDV90UaZKoXrRAAAPexid5/Q6XTz5s2LWhSINe2Ne45tf0Ysi5tw04dqYwHXcQAAesUFi9Bqtb777rtHjx5tamoymUwFBQX33nsvJk4MEH5P+/Hvf9fRtK9w6isJmbO4jgMA0Iu6n1BfXFycn5//m9/8Zu/evX6//8iRIy+88EJeXt7OnTujnA+ijGXDdSc+3/rRTFqsvm7xd2hBAOj3uh8R3n333TKZ7MiRI0VFRZEt5eXld91116JFi2pra3HhaH9lays9uv1pHk8wad5nSj0myAPAgNBNEba1tRUXF2/evLmrBQkhOTk57777bkFBQVlZ2bBhw6KYEKKB8TtO7H2z4dTa/AlPpw+5E7MDAWDg6KYIRSIRRVEJCQnnbE9MTCSEiMXiaOSCKGqp3nx0+zOGpPHX3f29SKLlOg4AQFR1U4Qqler6669ftWrV8uXLz96+atWq/Px8rCzTn7hs1ce2P+v3dIz9+Upt/Aiu4wAAcODHImxubj5x4kTk68WLFy9durSsrGzevHkmk6mjo2PDhg1btmx58803z150DfquUNB7cv+fa0s+zhv7eEbh3RTF5zoRAAA3fizCTZs23XvvvWc/tmnTpk2bNp295cEHH3zggQeiFA16jaX54KFNSzSmopmLt4tlRq7jAABw6ccivPHGGw8dOsRhFIiCUNB/ct8f6058XjT9DwlZs7mOAwDAvR+LUKfT6XQ6DqNAb7N3nDi08TGZOn3m4m0iCf5bAwAQcvEl1qDfYMPB8sN/rzjyTv74p9KH3MV1HACAGIIi7P8clvJDmx6jxZoZd26WyOO5jgMAEFtQhP0Zy4arjv3z1P6/YJo8AMCFoAj7Lbe9/vDmpeFwcOrt6+TqdK7jAADEKBRhv8TWlHxctvu17BG/zBn5MEV1v7Q6AAAQFGH/43OZD25aEmTcU2/7Wq7J4DoOAECsQxH2K+0Nuw9sfDRj6C8GjX4Mi8UAAFwOFGG/wVYe/cfpg/878md/iUudwnUYAIA+A0XYHzB+x6HNS73Olmm3fyNVJnEdBwCgL8FlFH2era1k28fXSxWJU2//Gi0IAHClMCLs2+pOfF7yw++GTXs1KWcu11kAAPokFGFfFWTcR7b8P2dn+dTbvsY0QQCAq4ZDo32S01r13ac3UDw+JssDAFwjjAj7noZTXxTveCF/wrPpQ+7kOgsAQJ+HIuxLwqFAyQ+vmGu2T7p1tcqQz3UcAID+AEXYZ/g97Xu++i+JIn76nRuFtILrOAAA/QSKsG9w2Wp2f3lXcu4tg8f9BjeRAADoQSjCPsDSfHDf+gcKJjybOngB11kAAPobFGGsayxfd2z7M6Nm/U9c2jSuswAAozJVgwAAGURJREFU9EMowphWefT/yg//fdKtn+HSGACAXoIijFFsOHjsu2ctLUem3b5eIjdxHQcAoN9CEcaiYMC1f/2DLGGnLvxSQMu5jgMA0J9hZZmY43WZd66+WaJImHDTB2hBAIDehiKMLfaOEzv+dUN85qzhM9+keBivAwD0OnzUxpDWup0Hv31k2LRlSbk3cZ0FAGCgQBHGiprSj8t2vz527ip94miuswAADCAowphwYu+bjeVfT7t9nUyVynUWAICBBUXIvRN732wsXzd5/r/FMiPXWQAABhwOipBhmB07dpSWlkokkjlz5qSkpEQ/Q+w4seeN5urNUxZ+KZJouc4CADAQcXDV6JIlS1544YWmpqZ9+/bl5+fv3Lkz+hliRNme15urN0+6dTVaEACAKxyMCF955RWdThf52mAwvPXWW1OmTIl+DM6V7XnNXL0VLQgAwC0OirCrBQkhYrGYz+dHPwPnyna/Zq7ZOmn+57RYw3UWAIABjcuLZRoaGlasWPHZZ59daIeWlpYPP/zwwIEDkW9VKtUTTzxBUd3fjc/v91MUxbJsr2TtUaf3/7Gt/ruxN37MUlK/3891nB7g9/uFQiHXKQYiv99P0zTXKQYiv98vEAgu9HEEvYdhmHA4fPn7C4VCHu8SJwF7pQiPHz9+/fXXn799/fr1I0aMiHzd0dExZ86cX//61zNmzLjQ8/D5fLlcrtGcGTNJJBIej3eh37zIQ5f8A3Pu5L432uq/G3fTJ/1pLMjj8WL/ne+X8M5zJfLOowijL/IL37O/9r1ShPn5+aWlpedvV6vVkS+sVuv1118/d+7c55577iLPYzQab7nllgULLututOFwmKKoGB+XlO56tb1+5+T5a2ixmussPUkoFMb4O99f4Z3nSuSdRxFyIhwO9+yvfa8UIZ/P1+v1F3rUbrfPmjVrypQpr776am+8eqxii3e81NG0b9Ktq/tZCwIA9GkcnCN84oknSkpKUlNTFy5cSAhJTk5evnx59GNEF1u840VL8wG0IABArOGgCO+7776zzyAqlcroZ4iuMy04cd5qWqziOgwAAPwEB0U4duzYsWPHRv91OcIe2/6cra1k0vw1QlrBdRgAADgXrjfrXSf3vdVpPjph3idoQQCA2IRFt3tRw+kv6058Nu32b9CCAAAxC0XYW9ob9xZ/9/yk+Z+LpAauswAAwAXh0GivcFjK969/YPScFSp9HtdZAADgYlCEPc/nbt395V1DJ79kTJnEdRYAALgEFGEPCwW9e7++N73gzpS8+VxnAQCAS0MR9iSWDR3Y8LBSlztozBKuswAAwGXBxTI9qXjHi0zAOebnK7kOAgAAlwtF2GPKD73d3rB76m1f8/hYBBkAoM9AEfaMpopvKo/+Y+rt64Sifr9iHABAv4JzhD2g03z06PZnx9/8oVSRyHUWAAC4MijCa+W21+39+p4R1/1RbcjnOgsAAFwxFOE1Cfisu9YuGjRmaXzG9ZfeGwAAYg+K8OqFgv49X92dlD03s/C/uM4CAABXCUV49Q5veVyqSMqf8BTXQQAA4OrhqtGrVFPykb39xPRF3xJCcZ0FAACuHorwajgsp8t2vzZl4Vq+QMx1FgAAuCY4NHrFgoxn//oHhkx+SaHN5joLAABcKxThFTu2/RmtaUTq4AVcBwEAgB6AQ6NXpuH0l5aWQzMWbeI6CAAA9AwU4RVw2aqLv3t+0q2fCWg511kAAKBn4NDo5QoF/fu/eaBg4rMqrCADANCPoAgvV/GO5+WazLSCRVwHAQCAnoRDo5elsXxde8Pu6Xfi1CAAQH+DIrw0t6322HfPTrzlEyGt4DoLAAD0MBwavYRwKLBv/QN5Y3+jNg7hOgsAAPQ8FOElHP/+dzJVCpbVBgDor3Bo9GJaqre0VG+acecWroMAAEBvwYjwgrzO5sNbfjN6zgparOY6CwAA9BYUYffYcHD/hl/mjnpUFz+S6ywAANCLUITdK9m1TChSZQ//b66DAABA78I5wm601e1sqvhmxp1bcK9BAIB+DyPCc4VDgWPfPTds2v9v707Dmjj7NYA/CUsAQzAEBYwsUSyKGyBqD74qWmqxVkVRW4pV6vJSl6u2CselVq30QotKj7W2vbS21mqrHrUXlVbAHUXBYwNKNWwGQQJvSFnDErLN+TA9eXNQWkVgIHP/Pk2emTxzZybhzyzJk4BLgwAAbIBC2JYsO8mp33D3QS8zHQQAALoDTo3+P4218pK7R19aeIHpIAAA0E1wRGiOkl5cP+w/1tnz3ZhOAgAA3QSF8N8e3juh16oHjVrEdBAAAOg+ODX6J62m9l7mjgnhRzkcK6azAABA98ER4Z/uZnzk4RuOX9YGAGAbFEJCCPlDkV1VenXYi+uYDgIAAN0NhZAYDbqci//pP3WHDU/AdBYAAOhuKISk8PZ+B8eBAwaHMR0EAAAYwPabZRrrSoqkB6a+mcp0EAAAYAbbjwjvXPlw6Lh3+zh5Mh0EAACYwepC+KjgpxZ1xeCApUwHAQAAxrD31KhOq87LiB//2gEu14bpLAAAwBj2HhH+fu1j98HTMO4uAADLsbQQ1ipzK+Tpw4M3Mh0EAAAYxsZCSBn10gtxoyZttbVzYjoLAAAwjI2FsCjnoK29yMM3nOkgAADAPNbdLNOsVhTc2hfyxlmmgwAAQI/AuiPCO5c3Dwn8p6NwMNNBAACgR2BXIaz5l7RO9fsLQauYDgIAAD0FuwqhLCvJN2g11wpfHAQAgD+xqBDWVeXV/yHzHhHJdBAAAOhBWFQI72ft8Q1axbWyZToIAAD0IGwphHWqe7X/yvUe8SbTQQAAoGdhSyHMz0ryDVplZW3HdBAAAOhZWFEIG6oLqit/k4xayHQQAADocVhRCGVZSUPGxFhZ2zMdBAAAehzLL4TqmiJV+Y1BoxYxHQQAAHoiyy+Esuz/GhL4T2ubPkwHAQCAnoiZQqjVahUKhUaj6eoVNdaVVJVlDB79dlevCAAAeikGCuH27dtdXFyCg4OdnZ3nzJnT2NjYdevKz/7Ux3+ptS2/61YBAAC9GgOFcPHixSqVqrS0VKlUKhSKTz/9tItW1FT3sFJ+YbD/ki7qHwAALAADwzB5eXnRE46OjqNGjaqtre2iFeXf+myw/xIbnqCL+gcAAAvAzHiERUVF6enpBQUFeXl5J0+ebG8xnU6nVCrlcjn9kMfjicXip1xFc0N5pTxtWvSNTogLAACWq0sKYWZmZnJycptGKyurHTt20NONjY1lZWVlZWV6vb6lpaW9foqKijIzM/fs2UM/7Nu375UrV7jcJ5/ObW1t5XA4trZ//pTo/cxd4qFvtOq4rTr1874e+DtNTU0URTGdgo269BI7/IWmpiaj0cjhcJgOwjo6nc5oNGq12qdc3s7Ozsbmb0Yc6pJC6ODg4Obm1qbRvIAFBAQEBAQQQjZu3Lh+/frHqybNz89v69at8+fPf5qV2tramgphi7qi6mH6tOjrPHvHDr4GeBYcDofPxx1JzHB0xJucARwOp0+fPiiE3Y8uhDwerxP77JJCaKpzf8vT0zMzM7PTA+T/z2eSUYt49qJO7xkAACwMA9cId+7cOXbs2H79+slksoSEhA0bNnRu/5omZXnBz9MWZ3RutwAAYJEYKIQ2NjY7duyora0dMGDAzp0733yzk4dGyr+1z3tEJM/BpXO7BQAAi8RAIVy3bt26deu6qHNNU9Wj/FMvL7raRf0DAICFsbTfGi28vd/L73W7Pq5MBwEAgN7Bogpha0tN6f3/HjJmBdNBAACg17CoQvgg5yuPoXPt+W2/uQEAANAeyymEuta6R7ITL4x5h+kgAADQm1hOIVQUJg8YMstBMJDpIAAA0Jsw81ujXcFreBSnnV9fAwAAaI/lFEIO1xo/dwQAAM8Kh1AAAMBqllMICwoKiouLmU7BRg0NDdevX2c6BUulpqYyHYGlbty4UV9fz3QKNiopKZHJZJ3bp+UUwuPHj586dYrpFGz022+/mQbYgm4WGRlpMBiYTsFGiYmJ2dnZTKdgozNnzhw9erRz+7ScQkgIwZB4AACWrSv+zltUIQQAAHhWKIQAAMBqnJ58OjEoKEin0/Xv3/9pFn7w4AGXy5VIJF2dCtqora2Vy+VjxoxhOggbXbp0acqUKfjiUPeTSqXe3t7Ozs5MB2Gd0tJSrVY7ZMiQp1x+zpw5K1eu/OtlenQhvHr1qlqttrOze5qFa2pquFxu3759uzoVtKHX6ysrKz08PJgOwkYlJSX4548R5eXlrq6uNjY2TAdhnfr6eoPB8PT/gkgkksGDB//1Mj26EAIAAHQ1XCMEAABWQyEEAABWQyEEAABWQyEEAABWs9q2bRvTGTpBfX19cnKyTCbz9PTk8XhMx7FMDx48kEqlIpHI/D7e9ra8VCpNTU3V6XRisZiJsJZDq9VmZGRcu3atpqbGy8vL/JsS2dnZ58+fJ4S4u7ubGhsaGpKTk3///XcPD4+nvOManqilpSUrK+vq1avFxcUikYjP55tmyeXy5ORklUolkUhMe8RgMKSnp2dmZgqFQty+3il0Ot3ly5fNvw7Q2tr6yy+/3L5929XV1XyPFBUVnT17tqamxtvbuyPfJqJ6v/Ly8oEDB0ZERMyePVsikSiVSqYTWRqdTicUCoVCoZWVVVZWlqm9vS2fmJgoFotjYmK8vLy2bt3KTGhLMWjQoODg4OjoaD8/v+Dg4JaWFrp98+bN3t7e77zzjlgsTkpKohsrKys9PT3Dw8Pnzp3r6elZUVHBXPBeb//+/f/4xz/efvvt2bNnCwSC9PR0uj0lJUUkEi1btiwgICA8PJxuNBgMr7zySlBQ0NKlS52dnS9evMhccMsRHx9vbW2dkJBAP2xqagoMDAwJCVm0aJFIJLp79y7dfvr0aZFItHz58pEjR0ZGRnZgRZZQCGNjYxcuXEhPR0REbNmyhdk8lsdoND548ICiKCcnJ/NCGBcXFxUVRU9HRER8+OGHFEU1NDQ4Ojrm5ORQFFVUVGRvb69SqZhIbSGKi4vpCY1G4+3tfezYMYqilEqlvb09vVOkUqlAIFCr1RRFbdq0af78+fTykZGRGzZsYCi1pdmyZUtYWBg9PXr06MOHD1MUpVar3d3dr127RlFUWlqal5dXc3MzRVFffvllcHAwg2ktg0wmCwoKmj59uqkQHjx4cNy4cQaDgaKoTZs2LViwgKIoo9Ho6+t7/PhxiqLq6upEItHt27efdV2WcI3w7NmzERER9HRERERKSgqzeSwPh8MZNGjQ4+3mW37evHn0lr9y5Yqrq6u/vz8hxMfHZ+jQofTpO+gY03eBeTyeUCjUarWEkPT0dD8/P3qnBAQEuLi4ZGRkEEJSUlLmzZtHL4/PQidqampycXEhhJSVld29e5d+2/P5/LCwMHojp6SkzJgxw97enhAyb968GzduVFdXM5u5VzMajcuXL9+7d6+tra2pMSUlZc6cOVwul5j9wSkqKpLL5bNnzyaEODk5vfzyyx1421vCCPUKhcJ0IUosFisUCmbzsMcTt7xCoRg4cKBpGeyRznL27FmFQjFz5kzS/kbGZ6FzFRUVrVy5sqampk+fPqdPnyaEVFRUODk5ma5OicXihw8fEkIUCoXpVwZdXFx4PJ5CoRCJRAwF7/X27NkTFBQUHBxs3tjm7d3c3FxbW6tQKFxcXEyXwzv2treEI0KDwWC6OmplZaXX65nNwx5P3PLmjYQQa2tr7JHnJ5VKly1b9v3339N/W9vbyPgsdC5XV9f169e/9957VVVVhw8fJo9tedNG1uv19JFKm3bogMLCwkOHDsXHx7dpNxgMpo1sZWVFCNHr9e3tkWdiCUeE7u7uKpWKnlYqlQMGDGA2D3s8ccu7u7tXVVWZlsEeeX55eXkzZsz46quvpk2bRre0t5HxWehcAoEgNDSUECIWi+fPnx8XF+fm5lZfX9/a2krfI61UKulbdgcMGGDaI2q1urm5GRu/wz7//HOBQLBu3TpCSG5ubmVlpaOj4+rVq83f9kql0tbW1sXFxd3dvaamxmAw0KVRqVR24E51SzginDJlSnp6Oj2dnp4eEhLCaBwWeeKWnzBhQnFx8aNHjwgh1dXVOTk5kyZNYjBkb1dQUDB9+vSkpKQ5c+aYGidNmiSVSmtqagghZWVlcrmcPokUEhKSlpZGL4PPQidSqVQCgYAQIpFIPD09L1y4QAgxGAwXL16cMmUKISQkJOT8+fNGo5EQkp6ePmzYMFdXV2Yz917R0dFxcXGhoaGhoaH9+vXz8fGh7zkICQkx/4MzefJkDofj6+vr7Ox89epV8n9ft6D3yDOxhB/dvnfv3oQJE1avXq3X6w8cOHDr1i0fHx+mQ1maLVu2KJXKw4cPv/rqq/37909ISBCJRPfv3w8ODjZt+ezsbHpslFWrVt28eXPx4sXHjx/39fWlzylBx3h4eNjb25s+2zNmzJg1axYhZOHChXK5/PXXX//uu+8mTpy4d+9eQkhBQcH48eNXrFjB5XK/+OKLmzdvDh06lMn0vdmKFStsbW09PDzKy8uPHDmSmJi4bNkyQsjBgwc/+uijtWvXXr9+vbS09NatW1ZWVlqtNjAwcPjw4ePGjdu9e/fu3bujoqKYfgWWIDw8fPz48Rs3biSEVFdXjxo1aubMmV5eXomJiWfOnKE/F5999llSUtKaNWsuXbpUXV2dmZn5rF8ltIRCSAgpLCw8fvw4h8OJiop64v2N8JxOnDhRX19vehgZGeno6Eja2fJGo/HEiRN37tzx8/OLioqiT1lAxxw6dMhgMJgeBgYGBgUFEUL0ev2xY8dkMllAQMCCBQtMn/zi4uIff/yRoqjIyMinH7MNHpefn3/+/Hn6XoywsLARI0aYZl24cOHy5ctubm7R0dH0B4EQUldXd/jwYZVKNW3atMmTJzOU2tKkpaWZ7kInhFRWVh45cqSpqSk8PDwwMNC0WGpqakZGhlgsjo6O7tOnz7OuxUIKIQAAQMdYwjVCAACADkMhBAAAVkMhBAAAVkMhBAAAVkMhBAAAVkMhBAAAVkMhBOgOubm5P/zwQ1f0XFNTc+DAgbKysqd/ypEjR7KysroiDEBvhEII0B1Onz4dExPTFT2Xl5fHxMTk5eU9/VNiY2PpsRQAgKAQAgAAy6EQAjCmqamprq7uLxbQaDR//PGHeQs9ssFfdGgafaKN5uZmpVLZ3hMpiqqqqsLIQcBOKIQADHj//ffFYjGfzxcKhW5ubtu2baMHLiCEqNVqZ2fn/fv3L1u2TCAQ9OvXb+TIkYWFheXl5aGhoQKBQCAQzJs3r7Gx0bxDlUr12muvOTo69u/fPyAg4M6dO6ZZGo1myZIlffv2dXNzk0gkpuEpaLm5uVOnTrWzs3N1dXVwcBg/fvzt27e7YQsA9BwohAAMaGxsTEpKys3NlUqlMTEx27dv37dvHz2Loqja2tr4+HiKoq5cuZKcnFxdXf3WW2/NnTv3pZdeunXr1r59+5KTk3fv3m3eYWxs7KBBg3Jzc1NTUxsbG8PCwuhBmgghq1evPnr06CeffHLv3r2PP/546dKlarXa9ESVSvXiiy+mpaXJZLKff/6Zoqjp06eb/8A6gOWjAKDrbd68mc/ntzf3jTfeGDduHD1NF6GJEyea5iYmJhJCNm7caGqZPXv26NGj6Wn64G/q1Kmmubm5uRwOZ9euXRRFVVRUWFlZxcXFmebSo2LFxsY+MUl5eTkh5MSJEx15kQC9kyWMUA/Q6xiNxuTk5NzcXKVSSVFUcXFxSUmJ+QKmwegJIS+88MLjLdeuXTNfPiIiwjQ9evRoHx+fnJwcQsjdu3cNBoP53IiIiOjoaPPnVlRUnDx5srS0lL76aGtrW1xc3BmvEqB3wKlRgO7W2NgYFBS0ePFimUzG4/GEQqG9vX2bs5FCodA0bWtr+3iLVqs1X97Nzc38obu7+6NHjwghCoWizVw+n28aQo8QcurUKYlE8vXXXzc2Nvbt21coFHK53IaGhk55pQC9Ao4IAbrbmTNncnJy8vLyTGO9vvvuuzdv3nyePtvcXKpSqUaOHEkI6d+/Pz3Xy8uLnqXRaMxvtImPj586deqvv/5KD+2r0Wh27dr1PEkAeh0cEQJ0t5KSEjs7u+HDh9MP9Xp9mzs5OyA1NdU0LZfLCwsL6So7YsQIDodjPjc1NZUyG467pKQkMDDQNMD9uXPnTPevArAECiFAd/P399doNDt37mxpaSktLY2Ojq6oqHjOPs+fP5+UlNTQ0JCfn79w4UIHB4clS5YQQry9vWfNmpWYmHjmzJmmpqbr16/HxsbS51pNYY4dO3bnzh2NRnPu3Lm1a9fa2Ng8ZxiA3gWFEKC7zZo1a8WKFR988IGDg4NEImlpaVmzZs1z9pmQkPDNN984OTkNGzaspKTkp59+EovF9Kxvv/127NixERERfD4/LCxs8+bNTk5Opifu37/f2tra39/f3t4+Kipq165d5lcQAdiAY36SBAC6CH2XNpf77389q6qqSktLBw4c6O7u3lmryM/Pb25uHjlypPkxH624uLiurm7o0KF8Pr/NLJ1OV1RUpNVqhw0bxuPxOiUMQC+CQggAAKyGU6MAAMBqKIQAAMBqKIQAAMBqKIQAAMBqKIQAAMBqKIQAAMBq/wtUxueLDLHC3AAAAABJRU5ErkJggg==",
"image/svg+xml": [
"\n",
"\n"
],
"text/html": [
"\n",
"\n"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# solve at a grid of λ\n",
"λgrid = 0:10:400\n",
"alpha = 0.5\n",
"elastipath = zeros(length(λgrid), size(X, 2)) # each row is β̂ at a λ\n",
"β̂elastic = Variable(size(X, 2))\n",
"@time for i in 1:length(λgrid)\n",
" λ = λgrid[i]\n",
" # objective\n",
" problem = minimize(0.5*sumsquares(y - X * β̂elastic) + \n",
" λ * (alpha*norm(β̂elastic[2:end],1) + (1-alpha)*sumsquares(β̂elastic[2:end])))\n",
" solver = Mosek.Optimizer() # MOSEK this time!\n",
" MOI.set(solver, MOI.RawOptimizerAttribute(\"LOG\"), 0) # keep silent \n",
" solve!(problem, solver)\n",
" elastipath[i, :] = β̂elastic.value\n",
"end\n",
"\n",
"plt = plot(collect(λgrid), elastipath, legend=:none)\n",
"xlabel!(plt, \"lambda\")\n",
"ylabel!(plt, \"beta_hat\")\n",
"title!(plt, \"Elastic net\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## QP example 5: constrained lasso\n",
"\n",
"* (Linearly) constrained lasso\n",
"\\begin{eqnarray*}\n",
"\t&\\text{minimize}& \\frac 12 \\|\\mathbf{y} - \\beta_0 \\mathbf{1} - \\mathbf{X} \\beta\\|_2^2 + \\lambda \\|\\beta\\|_1 \\\\\n",
"\t&\\text{subject to}& \\mathbf{G} \\beta \\leq \\mathbf{h} \\\\\n",
"\t& & \\mathbf{A} \\beta = \\mathbf{b},\n",
"\\end{eqnarray*}\n",
"where $\\lambda \\ge 0$ is a tuning parameter. \n",
"\n",
"* We have seen this example (zero-sum lasso) before."
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"tags": []
},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" 0.115292 seconds (120.25 k allocations: 11.904 MiB)\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAGQCAIAAAD9V4nPAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdZ0BT198H8JM9gbDCnmEPmYqgKIKiVeuoWitWfdS6qnV0WNtKtdVql9ZVqdZRR1tHq60D6xZxsUH23nsTQkjIeF6k/xTZI+EG8vu8kpuTc38hhm/uOOfgpFIpAgAAANQVHusCAAAAACxBEIJhj8/np6WlPX36NDExkcfjYV2OUly5cmXlypUxMTFDudMnT56sWbPmn3/+GcqdAjD0IAjBMBYbGzt79mxdXV1nZ+fx48d7eHiwWKzAwMC//voLq5LOnj17+vRphXcbGxt76tSpvLw8hffcg4yMjOPHjyckJPTQRiwWk8lkMpksFAqHrDAAFIuIdQEADNDhw4e3bNkiFotHjRo1efJkNpvd0tISFxd3//79hw8fnj9/fvHixUNf1QcffCAQCJYvX67YbjkcTkBAgIGBgWK7HTypVNrW1ib7B9a1ADBAEIRgWLpy5cqmTZvIZPLp06eXLFnS/qGSkpIPP/ywtbUVq9qUYeXKlStXrsS6CgBGJghCMPwIBIINGzZIpdJDhw51SEGEkKmp6YULF2pra9tvbG5uvnfvXn5+PolEcnd39/Pzw+NfuS6QlJQkkUg8PDwEAkF4eHheXp6urm5wcLCxsXGH/mtqap49e1ZYWIgQ0tfX9/LysrW1RQg1Njbm5OSIRCKxWBwXFydrzGAwHBwcEEJpaWl8Pt/Dw0MsFt+9ezc7O9vAwOCtt96SNcvIyHj58mVZWRmRSHRxcfH39ycQCO13WlZWVl5ebm1tra2tLdvSvsM7d+5kZWVpaGhMnjzZ0tKy829MJBJFRkampqa2tbXZ2toGBQXRaLTOzeLi4p49e4bD4Xx8fEaPHt39O9BvIpHoxYsXeXl5FRUVenp6fn5+sl9LB8XFxdHR0cXFxSQSycDAYOzYsaampu0bpKWlJSUllZWVaWhoGBsbjxs3Tv4LkZFIJC9evEhISBAKhZaWlpMnT9bQ0FDgCwEjkxSA4ebixYsIISsrK7FY3Jf2f/75p56eXvv/9h4eHtnZ2e3bsNlsBoORkJBgYWEhb0aj0S5dutS+WVhYGJVK7fAh2rlzp1QqvXnzZufP15gxY2RPdHZ2RghFRUXZ2NjIa5BKpdXV1dbW1h2e5eDgkJKS0n6/n3zyCULowoUL8i2yDhMTEx0dHeVPJJFIx44d6/DyHz9+zOFw2vdvYmLy6NGj9m0EAoE8lWXmzJlz5MgRhNCePXt6+N3KzosihFpbW7trc/XqVRaL1eE1LliwgMfjtW/22WefdYh/hNDZs2dlj/L5/AULFnR4lEAgJCUlyXvIy8vz8vJq30BXV/fixYs91A+AVCqFIATDz9q1axFC69ev70vjR48eEQgEGo32/fffZ2RkxMXFLVu2DCFkZmZWV1cnb8Zms8lksomJyeLFi+/cuRMdHf3JJ5/g8Xgmk1lTUyNrk5qaisfjjYyMfv/996ysrLy8vIiIiO3btx8+fFgqlVZXV9+9e1dLS4tGo939n6ioKNlzZbllbm4+c+bMixcvPnv27MqVK1KptKioyNvb+/Dhw5GRkTk5OZGRkWvWrMHhcFZWVu2jpbsgtLKyev3118PDw2NiYr766isSiUQmk3Nzc+XNYmNjqVQqnU7/8ssvo6Ojk5KS9u3bR6fTGQxGVlaWvNm6desQQqNHj3706FFRUdG1a9esrKxMTEwUEoTHjh2bN2/exYsXY2NjMzIyrl275uvrixBau3atvE14eDhCyMnJ6e+//87Pz8/Kyrp79+6mTZsuX74sa7B7926E0JQpU2QVpqSk/P3330uWLElOTpY1aGxstLKyQgi9/fbbsbGxmZmZP/zwA41Gw+Pxd+7c6eElAABBCIafKVOmIIR+/PHHvjT29vZGCP3000/tN77++usIodDQUPkWNpuNEHrnnXfaNwsJCUEInT9/Xvbj0aNHEUIHDx7sYXd6enoaGhqdt8tyKyAgQCKR9FqzLOnbH4x2F4SzZ89u/8SNGzcihH744Qf5Fh8fHxwOd/369fbNfv31V4TQkiVLZD/m5ubi8XgWi9X+m0FaWprs+GzwQdgZj8fjcDgUCqWpqUm25cMPP0QI3bhxo7unBAYGIoRKS0u7ayBLymnTprX/DZ84cQIhNGrUqL7XBtQQDJ8Aw09TUxNCqC/XfoqLi2NjYw0MDDrcxrlt2zaE0JUrVzq0/+ijj9r/KEvc/Px82Y+yy1ExMTEikWhglX/00Uc4HK7XZrNmzUIIRUdH99pSlh9ywcHBqF3BWVlZUVFR7u7uM2fObN9s0aJFLBbr1q1bsh+vXbsmkUhWrFjR/nqbo6Pj9OnTey1gYOh0+uTJkwUCQVJSkmyLbNcvXryQdnP3qazB8+fPu+tT9m5u27at/W946dKlxsbGL1++zM7OVmD9YISBm2XA8MNgMBBCLS0tvbZMTU1FCLm4uJDJ5PbbPT09cThcRkaGRCKR3zVDIpE6XK6TDVeoqKiQ/Tht2jQjI6Pz589HRkbOmDEjICBgypQpna9+9UB2GNdBWlrat99+GxMTU1xczOVy5dtramp67dDe3r6HguPj4xFCbW1tsuBvj0wmV1VVNTc3M5nMtLQ0hJCbm1uHNh4eHtevX++1hr64evXq8ePHMzMzy8rKBAKBfLv8NS5cuPCrr77avXv3n3/+OX369ICAgMDAQDqdLm+5fPnyP//8c/78+b6+vlOnTp00adK4cePaX1OUvQpPT8/2+yWRSG5ubmVlZWlpabJ7mgDoDIIQDD+yOwlzc3N7bdnc3IwQkp32bI9KpWppaTU0NPB4PPmRJZlMJhJf+UTIMlJ+jMJisaKiorZv337t2rWjR48ePXqURCItWrRo3759HW7G6U7nZlFRUYGBgUKhMCAgYObMmdra2jgcrrCwMCwsTCwW99ph+6joXHBDQwNCKCcn5/jx452fq62t3draymQyZb8lfX39Dg06bxmY3bt3h4aGamtrz5gxw8LCQvYLv3XrVkREhPzYmsPhREdHh4aG3r59e9++ffv27aPRaOvWrdu1a5fsNc6YMeP27dt79+6NjIx8/vz5zp07dXV1Q0NDN27ciMPhRCJRa2srjUbrfJ5A9uWg/TcMADqAIATDz/jx48+ePXvv3r1eW8r+LFZWVnbYzufzGxsbCQSC7OCy78zMzM6cOSMSiWJiYu7fv3/q1KmzZ89WVFTcvn27X/3Iff755y0tLZcvX54/f75849WrV8PCwgbWYXuylz9//vxz58712qyqqqrD9s5bBqCxsXH37t1sNjs+Pl52941Menp6RERE+5bOzs5Xrlzh8/kvXry4e/fuyZMn9+/f39LSIv9VBAcHBwcHNzQ0PH78ODw8/Ny5c5s3b6bT6atWrSISiTQajc/nc7ncDlkoOz7W1NQc/GsBIxVcIwTDz/z58zU0NBISEnqIH9mhhouLC0IoOTm5/ek4hFBsbKxUKnVycuowmrCPiESir6/v9u3bk5KS9PX17969K7tsiRAikUj9uoKYlJREpVLnzp3bfqN8GOIgeXh4oB4vvMnITth2nkpNIWWkp6cLBAJ/f//2KSiVSmWnbTuj0WiTJk3as2dPbGwsgUD4448/OjRgsVizZs366aefZKNo5A1k73VsbGz7xkKhMDExUf4oAF2CIATDj7a2dmhoKEJo2bJlnf98C4XCPXv2yG6MNDEx8fPzq66uPnnypLyBVCrdu3cvQqjzuLSeyU4htsdkMmX3iMqD1sTEhM/ndxjO3wM9PT2BQND+mLWqqurHH3/sV2HdcXJy8vHxycnJkd082YH85cyePZtAIJw+fbp92SkpKQqZblt2frW4uLj9xj/++CM5ObnLYto/kUQiyacw7dzAyMgIIST/zcsOqb/55pv2qX/69OmKigovL6/OgzUBkINTo2BY+vDDDzMyMk6dOuXj4/PWW29NnjzZ2NiYy+XGxsb+/vvv+fn58j/933333cSJE99///2GhoaZM2c2NTUdPnz41q1bVlZWmzZt6tdO9+zZc//+/SVLljg7O5uamlZWVh4/fjwvL2/SpEnyy2ne3t6xsbHz5s17/fXXNTQ02Gz2nDlzeuhz0qRJqampc+fO3bVrl6WlZVJS0vbt23V0dGSX9wbv+PHj48aNW7t2bWxs7IwZM6ysrKqrq7Ozsy9evGhsbHz+/HmEkIWFxYYNGw4ePBgUFLRnzx5bW9v4+PiPPvrI1NS0qKiojzvasmVL58PrxYsX+/j4WFhYREdHr127dtWqVRQKJTw8/IsvvrC2tm4/h/g777xTW1u7cOFCBwcHAwODoqKiffv2tba2yieM9fX1dXFxmTVrFofD0dTUzMjI+PzzzxFCb775pqzB+vXrT5w4cfv27bfeemvDhg0sFuvGjRtffPEFgUDYt2/fIH+NYITDbOAGAIN2+vTpzjOKGRkZ7d69u6WlRd7s5s2bHWZK8/PzKygoaN+VbGaZDv3LDoneffdd2Y9HjhzpPDPZlClTKioq5E+pqqqaMWMGiUSSPdphZpnm5uYOu2hoaPD392/f4WuvvSYbXS4f5yftfhxhhw5lJzMXLFjQfmNSUpKPj0+HsvX19Q8cOCBv09bWtmLFivYN3nrrLdm4yT6OI+zSzz//LJVKX7x40X66cCKR+NVXX3388ccIIfl4+a1bt8p/aTI4HC4kJET+AqdNm9Zh5AmFQtm+fXv7UYNFRUUdfplGRkbXrl3roX4ApFIpTgpzxoPhTCqVJicnp6SkNDU10Wg0JycnDw+PDjd/IoRaW1ufPn2ak5NDoVDc3Nzc3d07/FUtLCyUSCSyqUnk+Hx+eXm5pqam/G5PgUAQHx9fUFDQ1NRkaGjo5OTU5U35EomkoqKitbWVQqHIro2VlJQIhUIrK6vO4wilUmlsbGxqaioOh3N3d3dzc2ttbS0rK2MymfL7XWtra+vq6oyMjJhMpmxLlx0KBILS0lIGg9F5nYq0tLT4+Pjm5mY9PT0LCwtPT8/O85mlpaW9ePECj8d7eXm5urpyudzq6modHZ2eh4j0METPwMBAdpcKj8d79uxZfn6+lpaWbBmNurq6hoYGAwMD+f1KPB4vNja2uLi4paXF2NjY3d29w0SjNTU1cXFxFRUVYrHYzMzMy8tLR0en804TExMTExMFAgGHwxk/fnznKfEA6ACCEAAAgFqDm2UAAACoNQhCAAAAag2CEAAAgFqDIAQAAKDWIAgBAACoNQhCAAAAag2CEAAAgFqDIAQAAKDWIAgBAACoNQhCAAAAam24BiGfz//222/73l4ikSivGKBq4O1WK7J5k7GuAgwdhX/Ah2sQ1tTUHDlypO/teTye8ooBqgbebrUiFAr7tRgyGO4U/gEfrkEIAAAAKAQEIQAAALUGQQgAAECtQRACAABQaxCEAAAA1BoEIQAAALUGQQgAAECtqUUQigWS5txWrKsAAACgitQiCCUiafG12orndVgXAgAAQOWoRRCSGASbFYYl92sgCwEAAHSgFkGIECJpEVzWWhbfra54UY91LQAAAFSIugQhQoiqR3ZZZ1V8p6oqrgHrWgAAAKgKNQpChBBNn+yy1rLgRmV1PGQhAAAAhNQtCBFCNDbFdZ1l/vXK6oRGrGsBAACAPbULQoQQjU1xXmWR/3d5bXIT1rUAAADAmDoGIUKIYUx1XmWZ+0dZXSoX61oAAABgSU2DECHEMKE6rbbMuVRalwZZCAAA6kt9gxAhxDShOq2yyL5YWp8OWQgAAGpKrYMQIcQ0pTktN8/6vbQhm4d1LQAAADCg7kGIENKwpDsuN888V9yYA1kIAABqB4IQIYQ0reiOy80zzhY35kIWAgCAeoEg/JemFd3+bdOMX4qa8iALAQBAjUAQ/odlx7RfYpZ+pphbxMe6FgAAAEMEgvAVLDum3SLTtJOFzZCFAACgHiAIO9J2YNq9ZZJ6srC5BLIQAABGPgjCLmg7atjMN049DlkIAAAjHwRh13RdNTnzjdNOFLaUt2JdCwAAACWCIOyW3ihN67lGKccKWioFWNcCAABAWSAIe6LnpmU9xyglLB+yEAAARioIwl7ouWtZzDBIPV7QWivEuhYAAACKB0HYO4PR2ubB7OSj+ZCFAAAw8kAQ9omBj7bZFHby0fzWOshCAAAYUSAI+8pwrLZJgF7yj5CFAAAwoigrCM+fP//uu+8GBwc/efKkywZisXjHjh0uLi5jx47966+/5NtTU1NnzJhhZ2e3dOnSmpoaJZU3MMb+uiYT9VKPFQgb27CuBQAAgGIoKwgfPXpkamqalpZWVVXVZYMjR45cvXr18uXLO3bsWLZsWVpaGkJIJBJNnz7d39//7t27CKF33nlHSeUNmPEEXUM/neSjBcImEda1AAAAUABlBeGJEyc+/fRTLS2t7hqEhYXt2LHD0dHxtddeW7BgwYkTJxBCN2/eJBKJ27Zts7Cw2LdvX3h4eElJiZIqHDCTiXqGY7WTj+ZDFgIAwAiAzTVCoVCYlZXl5eUl+9Hb2zs5ORkhlJqaKt+or68vO6bEpMKemUzS0/fQSj6a38aFLAQAgOGNiMlea2pqpFKp/HiRxWJVV1cjhKqrq9sfRGpra3d3ZpXP55eXl1tZWcm3rF69esOGDd3tsbm5WTGly2vzowkEwpdheZxlBgQ63HOkWhT+dgNVJhAI8Hg8iUTCuhAwRPr1AadSqb3+38AmCFksFkKIx+Npa2sjhLhcruwfLBarsrJS3qypqUm2vTMajcZms+/fvy/fYmJiQqFQumxcwJXa3aK2SRT4CmSoq0U1Yw9WbbYwayIQFN47GAQq1gWAoQRvt3qZY0a+OrXrv/YDg00Q0ul0Q0PDzMxMU1NThFBWVpbs2M7KyuratWuyNi0tLSUlJdbW1t11QiAQeni0PUsNXO2CVg0NDUXU3oFRwc3KiIwSl3VWRDpkoargcrnKebuBKoIjQnXD5XIRUmQQKuucHpfLra+vF4vFPB6vvr5eJBIhhJ4/f75//35ZgyVLlhw4cEAsFpeXl//2229LlixBCM2ZMycnJ+fBgwcIoR9//NHFxcXR0VFJFSqK5QwDlj0z9edCcaviDzkBAAAom7KCcPHixRwOp6qqatOmTRwOJy4uDiGUmZl5/fp1WYPPPvtMJBLp6+s7OjquXLly0qRJCCFNTc0zZ84sWrTIwMDg1KlTp06dUlJ5imU5w5BpRk05BveRAgDA8IOTSqUY7p7H45FIJDKZ3H6jRCLhcrk9DL1ACBUXF48bN66oqKiPO1L6uTIpKr5XXf6szmGJqaY1Q4k7An0Ap0bVCpwaVTcK/4Bjc41QjsHoIjPweHzPKaiKcMhsir6mFT3jbLHxBF3TSfoIh3VJAAAA+gDu+1ckLRuG22ZObXJT+ukiUasY63IAAAD0DoJQwSgskut6a4o2KelAXksFLOcLAACqDoJQ8fBEnPVcI7PJ+slH82uSGrEuBwAAQE8wvkY4grG9WXQjasYvRU15LVazDHEEuGYIAACqCI4IlYhpQnXfzOFXC1J+KhDCrKQAAKCSIAiVi8ggOK+y1HZkJv2Q21TQgnU5AAAAOoIgVD4cMg3Ut1lgnH6qqORBNdbVAAAAeAUE4RDRdtRw38KpedmUeb5ELITJ2AAAQFVAEA4dijZp1AZrPAmX9ENuSyWMrAAAAJUAQTik8ESc7UIT4wm6yUfyal82YV0OAAAAGD6BBUNfHaYZLeOXYm4R32I6G4eHkRUAAIAZOCLEBtOU5raF01zCT/mpoA1GVgAAAHYgCDFDYhCcV1tocRiJB3K5hXysywEAADUFQYglHB5nPpXNecM47VRh2eNarMsBAAB1BEGIPR1njVEbrCuj6rN+K5HAyAoAABhaEIQqgaZPdttsjcPhXh7Oa60VYl0OAACoEQhCVYEn4W0XmRj66bw8lFef0Yx1OQAAoC5g+IRqMfTVoRtSM84Ws71YljMMYJl7AABQNjgiVDmaVnSPDzjNxS2pJwpFfFjmHgAAlAuCUBWRmETnNZYMI2rC9znNRTCyAgAAlAiCUEXh8DjLmQbWc4xSTxRWRtVjXQ4AAIxYcI1Qpem6atINKOmni7hFfM4bRrDMPQAAKBwcEao6GpvitpkjahEnHcxrrYORFQAAoGAQhMMAgYJ3WGqm56718lBeXRoX63IAAGBEgVOjwwQOmQbqaXEYmeeL61K51rMN8WT4EgMAAAoAf0yHEw0LmscHNmKBOPFALq+8FetyAABgJIAgHGYIVLz922amgfopR/Nhnm4AABg8ODU6LLG9WRrmtMxzxY25PNuFJkQ6AeuKAABguIIjwuGKxqaM2sSh6VMS9uU05fGwLgcAAIYrCMJhDE/EWc40sF1oknGupOh2lVQixboiAAAYfiAIhz2WHdPjAw63iP/ycD4MNAQAgP6CIBwJSEyi8zsW+h5aSQfyqhMasS4HAACGE7hZZqTAIeMJuppW9IxzxfXpXM58YwIMNAQAgD6Av5UjCtOM5vGBDUIocX8urxQGGgIAQO8gCEcaAgVvF2Jq8Ro79XhB2eNaBDfQAABAj+DU6Mik56bFNKNlnS+pS+fahZiSNeCNBgCArsER4YhF1SG7brDStKQn7supz2jGuhwAAFBRcKAwkuHwOPOpbE1rRtZvJXqjNK1mGcKKhgAA0AEcEY58LFuGxwec1lph0uG81hoYaAgAAK+AIFQLJCbRaaUF25OVdCivKq4B63IAAECFwKlRtYFDxhN0WXbMjLPFDZnNnHnGBAp8DQIAADgiVDN0Q4r7FmsijZDwXQ63oAXrcgAAAHsQhGoHT8JbzzWymmWYdqqo6HYVDDQEAKg5CEI1pTtK030LpyGbl/JTgbBJhHU5AACAGQhC9UXRJrm+a6lpTU/cl1OXxsW6HAAAwAbcLKPWZAMNtR2YmedLGjKbLV83xBNhoCEAQL3AESFAGhZ09w84Qq7o5aE8frUA63IAAGBIQRAChBAiUgkOS80Mxmq/PJRX9rgWFrsHAKgPODUK/mPkp6Ntz8y5XFYVU2+z0IRpSsO6IgAAUDo4IgSvoOqSXdZYGk/US/25sOBGpUQEh4YAgBEOghB0gkNsb5bnhzbCprb4b7IbsmHlCgDASAanRkHXSBpEuxDTujRu9oVSLQ7Deo4RkU7AuigAAFA8OCIEPdFx0vD8yJZII8R/l1OT1IR1OQAAoHgQhKAXBCreeq6Rw1Kzon8q004UChrasK4IAAAUCYIQ9ImmFd3jQxtNa3ri/tyyx7UwQykAYMSAa4Sgr3AEnGmgvo6jRvalspqkRtuFJjQ2BeuiAABgsOCIEPQP3YjqttGa7c16eSS/6HaVVAzHhgCA4Q2CEPQfDhn66nh8aMMra03cn8st4mNdEAAADJwST42Gh4ffvHmTxWKtW7fO1NS0w6NXrlzJzs6W/8hisdasWYMQOnHiRG1trWyjubn5okWLlFchGAyyJtFxuXlNUmPayUJdF02rWYaw5D0AYDhS1l+u3377beXKlW5ublwu19fXl8vtuMpPc3Nz/f+cO3fun3/+kW3fv39/SkqKbHvnZwFVo+em5bXNFiGU8F1OfQYMvQcADD/KOiL85ptvvv/++8WLFyOEkpKSzp8/v27duvYNli5dKvuHSCQ6d+7cypUr5Q+tXbt23LhxSioMKByRRrBZYNyYy8u5VMowpnHmGZGYcBMWAGDYUMoRIZfLffnyZVBQkOzHoKCgJ0+edNc4PDxcIpFMmzZNvuXXX3/duXPntWvXpFK4EWPY0OIwPD60oeqS47/NqXheh3U5AADQV0r55l5eXo4Q0tPTk/2or6//6NGj7hqfOnVq+fLlROK/lfj5+enr60skkvfff//MmTN//vlnl89qbW2tra2dN2+efMuCBQtmzZrV3V74fD6BADOEKR07UEPDkVJ4tboqqcF8lh6Zhc2hIbzdakUgEODxeBKJhHUhYIj06wNOJpPl+dIdpfydolAoCKG2tjbZ7oVCIZVK7bJlZWVleHj4119/Ld9y4sQJ2T82bNjA4XBevHgxduzYzk8kk8l0Ov3NN9+Ub3F1dZXtt0tCobCHR4ECUSwpWpuYFU/rM38qM5qgbTxRF4cf6lXv4e1WNxCEaqVfH3A8vvcTn0oJQkNDQwKBUFJSYmtrixAqKSkxNjbusuWZM2d8fX0dHBw6P8Rmsy0sLAoLC7sMQjweT6PRFi5c2Jd6qgSCs6VlHzo79edFgEEgINMAfT1XrZxLZXVJBUO/tCGBQIAjQvVBIBDweDy84+pD4R9wpVwjpFAor7322u+//44Q4vP5f/3115w5cxBCzc3N169fF4lE8pa//PLLihUr5D/y+fy2tn+nskxKSsrJyRk1atTg66ETCEcLiq6Ulg2+K9B3VF2yy1pY2hAAoOqUdQnnyy+/nDp1amJiYk5Ojr29/WuvvYYQKi4unjVrVn19PYvFQgg9ffq0uLi4/XW+1NTU6dOnjxkzRiwWP3369PPPP3d0dBx8MUwi8Rd31wVxid7aLHM6ffAdgr7CIbY3i2XPLLheEf9tts0CE5YtA+uaAADgFTjl3ZlZX1//5MkTXV3dsWPHys7SCgSC7OxsJycn2Y+VlZUNDQ329vbtn5Wbm5uRkUEkEl1dXbs7oYoQKi4uHjduXFFRUR+L4XK5R0rLb5VXPAzwJ+CG+pIVQAjVpXJzr5QNzdKGXC5XQ0NDqbsAqgNullE3Cv+AKzEIlWoAQchgMoMfP53E1v/M0b73JwAlELWKC65X1qdzrd8w0nXRVN6OIAjVCgShulH4B1yN5sTC43Bnxngdycl7Xguj3LBBpBJsFhjbLTYtuFGZcaZIyBX1/hwAAFAyNQpChJAJjXbS2+PtqNjGNlhdFjNaHIbnRzZMM1rCdzmwtCEAAHPqFYQIoelGhlMN2atiE7AuRK3JljZ0WWdZndDw8sd8fpUA64oAAOpL7YIQIbTfzTWT23y2sK/XF4GSMIyobhs5bLpDTskAACAASURBVC8tWNoQAIAhdQxCKoHw21jvD5NSsriwWgLWcMjQV8dtszW3sCXxB1jaEACAAXUMQoSQs6ZmqJN9SFSMUCLBuhaAqDpk59WWZlP0004W5lwuEwvgTQEADB01DUKE0AYbjgmNtiM1HetCwL/aL23YkAkH6wCAIaK+QYhD6KS35/nC4nuVVVjXAv4lW9qQM98453JZxpniNp4Y64oAACOf+gYhQkiPQj7v470iNr5GIMS6FvAfbQem51Ybqi45/ptsWNoQAKBsah2ECKGJ+npvmZkujY6FGxZVCp6Mt5xp4LzGsuJ5ferPhYJ6GPcJAFAWdQ9ChNBXLk51QmFYbh7WhYCOmCZUt83WWhx6wr6ckgfVUgl8XQEAKB4EISLh8b/6jN6ZmpHU0Ih1LaAjHB5nGqjvvoXTkNn88kh+SwUMvQcAKBgEIUIIcZiMb0e5hETF8MVwd4YqouqSXdZaGfnpJIflw9KGAADF6joId+/eXVhY2GFjUVHRtm3blF8SNv7P0tydxfroZQrWhYBu4BDbm+XxoU1rrSD+2+zGHB7WBQEARoiug/DIkSOlpaUdNpaWln7zzTfKLwkzRz3dwssr/i4rx7oQ0C2yBtFhmbn1bKOs30tyLpeJW2HoPQBgsPpxarSurk5LS0t5pWBOi0S6MHbM6tiE4haY6Eul6ThreHxkgyfi4r7JrnnZhHU5AIDhjdj+hydPnty4cQMh1NzcHBYWdu3aNflDQqHw+vXrbm5uQ13g0Bqjo73BxnpJdOz9ieNhIXtVRqQSrOca6blrZV8srYqut1lgTNaCdVkBAAPxShAmJycfP34cIcTn8//++28i8b9HqVSqi4vLvn37hrrAIfeZo/3D6prvM7M/drDDuhbQC00ruseHNiX3qxP25Vq8xjYcq4Pg2wsAoJ9wUmkXN+AZGhpeuXLFz89v6Avqo+Li4nHjxhUV9XUpJS6Xq6Gh0cfGJXy+972Hf/mNHaurM9ACwZDilbfmXCrFEfC2bxrT2JR+vd1guBMIBHg8nkSCUwLqQuEf8K6vEVZUVKhyCiqbKY121NP97ejYpjYR1rWAPoGlDQEAA9bLzTIikaj+VUNTFubeMDEOZOtvTEzCuhDQZ/9b2rCpoCX7eDmseg8A6KOug1AoFIaGhtrY2NBoNJ1XDXF9GDrgPiq6rv7XomKsCwH9QNUhu6yx1B2j+fJIfnUCTBUEAOgdscutW7duPXr06Nq1a2NiYnR0dEaPHh0eHp6RkREaGjrE9WGITiBcGjsmMOLJGB1tWyYT63JAP+h6MfVttdPPFDXm8DhvGOEIcAsNAKBbXR8RnjlzZs+ePYcOHXJwcHB3d9+5c2dUVFRISMjDhw+HuD5suWhpfuJgtzgqtg0Wsh9uGCZUj/dtRC3ipIN5rXWwzBYAoFtdBGFNTU1DQ8O0adMQQkQisaWlBSGEw+G2b99+69atzjPOjGyb7WwMKJQv0zKwLgT0G4GKd1hqxvZmvTyYV5/OxbocAICK6iIIqVQqQkgsFiOEDA0N5UMUNDU1EUIVFRVDWB72cAidGu15uqDoQVU11rWA/sMh4wm6Tistcv8sL7hRCQs5AQA66yIImUymqalpWloaQsjHx+f27duRkZE8Hm/Xrl1EItHa2nrIi8SYPoVyarTn/8XE1QrhDNuwxDSnuW3hNJfyU8IKhFwYEgMAeEXX1whXrlwZHx+PEJo+fbqHh8eECROYTOb333+/detWbW3toa1QJQQbsOebmqyKTcC6EDBAJAbBZbWltiMzcV9OQ3Yz1uUAAFRI1zPLtCcUCu/cuZOfn+/p6Tlu3LihKatXSp1ZpksCicT3fsRajtVqa8vB9AOGQA9vd0M2L+vXYkNfHfNgNszHNjLAzDLqRuEzy3Q9fKI9Mpk8c+ZMBe5ymKLg8Zd8x/g9iPDV1XHV0sS6HDBALFuG22ZOxtni5hK+XYgpkUbAuiIAAMZ6mlmmtbW1oKAg71VDVpkKsmEy9rg6wUL2wx2FRRq1wYpuQE34PodbBEtuAaDuuj4izMnJ2bhx461btzo/1Oup1JHtHSvL+5XVnySnHnAfhXUtYOBweJzlTAMNC1rayUKzIH3jCbpYVwQAwEzXQbhw4cLCwsJdu3bZ2dnhYFm+V4V5uXvcfRDEZr9ubIh1LWBQdF016YaUjF+Km0v4nPnGBHI/1qkGAIwYXQQhl8tNSEi4cOHCm2++OfQFqT4WiXRujPebz6O9dSYZUalYlwMGhaZPcdtsnXulPOmHXIf/M6cbULCuCAAw1Lr4CiwbSs/hcIa8mGFjvJ7uKmvL/4uOk6j3ieKRAU/C2y40MQ3ST/4xvyYR5ukGQO10EYQsFisoKOiff/4Z+mqGkVAnB55Y/EN2DtaFAMVge7Oc11gW3KzMuVwGyxkCoFb+OzVaW1tbUFAg+/d77723bt26lpaWqVOnMhiM9k/w8vIayvpUFhGH+91n9Oj7D/319MboqOMkAyMP04Tq8YFN9oWSpEN5DsvMqDpkrCsCAAyF/wbUnz59esWKFb0+QUXuGh36AfVd+qOk9JPk1PgpgRrE3kdkgiEzqLdbisoia0vuV9suMtV2gOW3hgEYUK9ulDigftKkSZcuXVJg1yqlVSxRfAwiNN/U5GZ5xZbElye8PZXQPcACDhlP0GWYUDPPl7C9WBbT2Tg83DgNwEjW+xRrXSopKYmJiZk7d67CC+qjfh0RFvCEY+6mHfA0DzHXUXglPJHY696DHU6Oi8xNFd45GBiFfGFsaxZlni+RSqT2S8zIGnDEr7rgiFDdKPyIcIADp5KSkj755BMF1qFUpjTCCRfqNxkVbz7PqxMqePEBBpHwm8/ozYkvC3gtiu0ZYIvEJDqvttDiMJJ+yG3KhzcXgBFLLUYQ17fWn0jYfcaDYEQled5Jf1Sl4DVaPbVZH9nbhkTFtMDUayMLDo8zn8rmzDdOP11U8qAaqcT1cQCAgqlFEOrT9d73XPtF5J6tNsTj3hZvR+VvSigWKnSN1g/sbe00mPOeRQklEgV2C1SBjpOG+/uc2mRu+ukiER++6wAw0qhFECKEvNhuK0aFfBKx21cHnxDsVNgi9L6bntyosAmXcQid9PbUIpEWvogWqcaNtUCBKCyS63orijYp6UAur6wV63IAAIqkLkGIEHrdZupoI/ftEXtYJPTXOM6H9gaBj7IOZlcpKrUIONy5MV5tEunyGJhxZgTCE3HWc43Mp7JTwvIrXtRjXQ4AQGHUKAgRQus939GgMPdFH0UILbXUjZrscKm4ftrj7DJ+m0L6J+Hxl33HlLTw30t4qZAOgarR92S5brAui6jJvlgqEcHXHQBGAvUKQjwOFzruw8LGovMplxFC1gxKxCQ7X12G1930a2UNCtkFjUC4Nt43pr5+S2KyQjoEqoZuQHHbzJEIJS8P5QnqFfMVCgCAIfUKQoQQhUDePeGzazm37+Q/QggRcbidzsZXx3HeTyxZGlXAEyngVhcNIvHOhHER1TVfpWcOvjeggggUvP0SM0Nf7aQDuQ2ZzViXAwAYlAEG4ZQpU54+farYUoaMLk3720k7jsafSq5Ol20Zq8uIn+JII+C876bH1StgxBiLRPpngt/5wuLvM7MH3xtQTYa+Og7/Z551obTodhWMrABg+Op2ZpmysrLr16/n5+fX179yX8CxY8eGpLBeDH6u0ejy+D3PDhwJ/tpUw1i+8Y+S+vVxRets9EOdjAiDXpG4hM+f8PDxJw72q6wtB9kV6BclTS3bJWGTKP2XIhKDYBdiSqQRhmanoD2YWUbdKPwD3nUQPnr06PXXXxcIBGQymUQiCQQCPp9Po9GMjIxyc3MVuPsBU8ik2zdz755PuRw27TsWReu/nluEy6ILWsWScz5WHOZgl2nNaeZNehT5nZvLW2YwAdvQGcogRAhJJdLC8Kra5CbH5eZ0Q1jad6hBEKqbIZpibdOmTV5eXtXV1QsWLFi/fn1LS8u9e/eMjIx27typwH1jbgZnykRzv9DHX7eJ/7vlwYxOvh9gt9Bcx/d+xvG8mkHuwobJuOXvtyUx+XpZxSC7AioLh8dZzjQwC9ZPPppfkwRL+wIwzHQRhAKBIDU1defOnVpaWgghkUiEEAoKCjp+/PiGDRtaW0fUaOI1Hsv06bp7nh+QtrvIg0Noky37QYDdjzlVg5+e1EVL85a/36q4+NsVlYOuF6guthfLeY1lwY3KvKvlUoXOWwQAUKougrChoUEsFhsbGyOEtLS05NcIfX19m5qaMjNH1J2QOIT7eOzGSl71Ly8vdHjIRYsWFeRgzaAMfnpSd5bWFb+xS6PjntTUDqYfoOKYJlT3zZyWKkFKWEEbV8HTuwMAlKSLINTX16dQKGVlZQghKyuryMhIsViMEEpOTkYI0en0IS5R2SgE8t6A7fcKIm7nPejwEJWA/3qUyc+jLZZEFQxyelI/XZ3zPt7znkXF1ytmwCJQTUQGwXmVhZYNI/FALrdQYXP4AQCUp4sgxOPx48ePv3XrFkLorbfeys/PDwgI2LBhw+zZs11cXKytrYe8SKXTomjuDdgelnA6riKp86NTDDTjgx0HPz3pFAP2MS+PmU+epzUpePkLoFL+XbPiDeO0U4UwGRsAqq/rm2XCwsLmzZuHEDIwMLh69SpC6K+//vLx8bl69SqBMDJvEDfXNP3Cf9uXT77Pbyjs/Kg+haiQ6UnnmBh9O8plWuTTfFi8cKTTcdYYtcGq7HFNzuUyqRguGQKguga4Qj3mFDJ8orNbeffPJF8Im/q9NlWrywb5PMHbUQVMIv70aEtj2gBv1z6am7c/K+dxwARjGnVgPYCeDfHwiR6IBZLsCyWttW2Oy80p2nB/v1LA8Al1M0TDJ1atWtX5ppisrKw333yz712npqZeuHAhLi6uy0cLCgri/ichIaH9Qy9evLhw4UJWVlbf96Uor1kHTbEM2Pboy1aRoMsGVoqYnvRdjvUGG+tJEZGVrV3vBYwYBAreYak525uVdDC3IRsmYwNAFXUdhNevX6+t7Xh/Y21t7R9//NHHfo8ePRoUFHT79u033nijy9GHO3fuDAkJ2bZt27Zt20JDQ+Xbt2zZsnjx4tu3b/v7+587d66Pu1OgFW4h5pome58f6G4pJYVMT7rZ1maBqcnUyKf1Qpi1eaTDIeMJuvZLzLJ+LSl5UI11NQCATqRdMTAwePr0aYeNly5d0tPT67J9By0tLdra2rIecnJyaDRaRUVFhzbLli3bv39/h415eXk0Gq20tFQqlf7zzz/GxsZtbW1d7qKoqMjMzKwvxcg0NTX1vbFQ3Lbp7qfHE8703KxRKFodU+AQnhJbx+t75+19kJg89v4jbjevEQxYv97uIdNaL0z8ISfjbJFIIMa6lhGltbVVKBRiXQUYOgr/gBPbh+LNmzfPnDmDEGpsbAwNDdXV1ZU/1Nzc/OzZs4kTJ/YlXCMjI5lMpp+fH0KIw+G4urr+888/y5Yt69CsrKzswYMH1tbWlpaW8gL8/PxkQxinTJnC5/NjY2PHjh078JwfEBKe+KX/tnfvbGUz9GfbvtZdM00S4Zi3xR8l9dMfZw9setLv3FzWxCXMefrixnhf6gi9CwnIUVgk1w3WeVfKEvfnOi43pxvAZGwAqIRXgrCxsTEvLw8hJBaLS0tLGxv/myxKU1Nz2bJl27Zt60unZWVlpqb/Ta1pampaWlraoQ2BQHj8+HFCQkJsbOzcuXNPnjyJx+NLS0vlT8Tj8cbGxp2fKCORSHg83k8//STfMnbsWFdX1+5KEovFstGQfcQg0vdO2L75/mdGdAMvQ7ceWs410vQOtFseW/Swkvu7j4UBtX9X7I+4uS6NiX/zefQlH28SXu1WxVKS/r7dQweHrOYZVr6oTz6SZz3fSMdFJe7oGe7EYrFUKsXDx0dt9OsDjsfjcb0dorwShCEhISEhIQghHx+fsLAwT0/PgVXZ1tbWfpQFkUhsa+t4Jeynn36S3eVVUVHh5eV18eLFRYsWtbW1tf/f3OUT5bsQCoWxsbHyLfr6+g4ODj2U1F1X3dGn6H7qs/mLZ99/PSHUWsuih5aGJNzNsRZ7sqp8HmT95m02WpvWrx397ObyVmxCSFTsOS/3wS95AdCA3u6hpOPFpBqRcn+taMznmU7VRfCeD06HvxtgxOvXB5xEIvU66o/Y5daoqKj+1fUqIyOjqqoq+Y9VVVWTJ0/uXJzsH4aGhsHBwXFxcYsWLTIyMmp/t2pVVZWRkVGXu6BQKNra2idOnOhjSW1tbVRqv8cqeJm4bxmzdsezb8KmfqdH1+258W43cz+21oKYwl0uxqus9fq+FypCV8f7znr64r2U9JOjPeGv4uAN7O0eSlRrqsb7zMyzxbnnK+3fhvWbBgWHw8HwCbWi8A94t1+jsrKytmzZEhwcPH36dNmWCxcu3Lx5sy+d+vj4FBUVFRYWIoSampqio6P9/f0RQm1tbS0tHQeSSySS5ORkExMThNCECROePHkim9c7JSWlubl5wEelijLJYvxMm+BPI75qFfU+2/h0I60ngfaHsqvWxBb2az42Mh5/xc8nu7l5c+LLQRQLhhMSg+C8xoJhRE38IZdXNqLmsgdgeOk6CCMjIz09PS9dusTj8V6+/PdPc1lZ2ebNm/vSqb6+/jvvvPPGG2+EhYXNnj17+vTpspOWJ06ckN1BgxCaOHHi7t27Dx48GBwcXF9fv3z5coTQ6NGjR48ePWfOnLCwsEWLFr333nuqMCx6qetCK5b5ziffSqS9j5SwYVKeBznUCcV+9zOKWoR93wudQLgx3vdJTe0XaRmDKBYMJ7L1myxeM0gOy6+MgcnYAMBG1zPLuLq6mpub//HHH1FRUW+//XZJSQlCKC0tzdnZuby83NDQsNd+JRLJ+fPnExMT7e3tly9fTiaTEUIpKSmpqakLFy5ECF2+fDkpKYnP5zs6Oi5atIjBYMieyOfzT548mZeX5+Pj8+abb3Z3kVNJM8t0RyQRb324k6Nttd5zRV/aSxH6NqPiQFbVb2OtJrH7sd9qgSDgUeT/WVp8ZG870GKBCs0s00f8KkHaqSItDoPzhhGOAGfH+wdmllE3Q7FCfW1trZ6e3osXL3x8fCIiIhYvXiwLwubmZg0Njbi4OMxPV6IhD0KEEK+tZf2dj2fZTHvDfkYfn/JPRdOyqPz37Q0+duj9q4NcKZ8/4WHkVgfbNdZWA6oUDL8gRAiJWyVZv5e0NYsclpmTNbu+eA+6BEGoboZiijXZSrwUSsdBTtXV1Qgh2bGdGmKQ6Hsnbj+feulpSXQfnzLNUDNmiuPl4volUfl8cV8noDGh0e5MGLc7LfO3ouKBFguGHwIV7/h/5jrOGkkHcpsKYE52AIZOF0HIZrONjIz++usvhFD7M5Nnz57V1NS0t7cfuupUjBHTYO/E0O+ijuTU5/XxKeZ08pNAezwON+5BZgGvr5cMOUzG/YnjP3qZcrmk62GUYGTCIdNAfZsFxumnimAyNgCGTBdBiMPhPv7446+++mrr1q3Jyclisfj58+ebN2/etWvX+++/r+bnH+x1bTaPXrPt0e6qlpo+PoVKwJ8ZY7mWo+93P+N+ZV9XIrTTYIaP93svIelWReVAiwXDkrajhttG6+q4xuzfSyWiYbk4DADDS9c3y0il0l27du3du1c2kgEhRCAQ1q9fv3//fhVZj3DorxG291van3fyH/0Y/A2DRO/7sx5XNy96kbfRlt33S4bPa+vmPH1x2XfMBP1+DEwEw/EaYQdioST7Qqmgvs1phTlJAy4Z9gSuEaqbobhZRq62tjYiIqKmpobJZPr7+5uZmSlwx4OEbRAihPZHh1XwqvYGbCfg+vHNoIQvfONpng2TcmK0BZ3Qp7kw7ldVh7yIuenv662tPdBi1c4ICEKEEJKikofV5U/qHFdaME1Uen4AbEEQqpshDUJVhnkQiqXijx9+acDQ/8hnQ7+e2CqWvBtfFF/fcnUcx4rRp2mXr5WVr41LvDdxvJPm8P/jPiRGSBAihBCqSWrM/bPcdqGJjvMIeUUKB0GoboZoYV6EUH19/f79+5csWRIYGBgSErJnz56KigoF7ni4I+AIX/pvS6/JupxxrV9PpBLwp0ZbvmvD9rufebeyqS9PmWVsdMhj1JTHTzK5sLKr2tFz03JaaZ5zuRRunwFASboOwqSkJGdn5w8++OD58+cCgSA+Pj40NNTR0TEiImKI61NldBLt20k7Lmf8HVn8vL/PXW2td9nPenl0wTcZFX05JJ9varLDyfG1yKeFneaoAyOehgXdbTOnOqEx53KZVDwsT+EAoMq6DsJly5YxGIz4+PicnJynT59mZGSkp6fb2tqGhISo8qT+Q0+Prrt7wqffRf2YVpPV3+eO12NGTXa8Wtqw6HleX5a5X21tuc3BPuBRZD4PslDtUFikURushY1tqScKRa0qucIUAMNWF0FYVVWVlJR09OhRDw8P+UY7O7vTp0+XlZWlpqYOYXnDgJ0O5xPfzZ9Hfl3Jq+q99atMaKSISfYaJIL33fQMbu/TLq+2tvzEwT4wIjKPxxtQsWAYI1DwjivM6WzKy0N5rXX9mMYWANCzLoKQQqHgcDjZMvHtyRaIUPHVbTDha+K90HHO1odf1PH7PW8yBY/72dtii72B/4PMv0sbem3/bxY+egJZqIZweJz1XCNjf92XB/Mac+E/AACK0UUQamlpBQcHnzp1qsP2U6dOOTs7q/PMMj1Y4DBrqnXgu3e2ljcPZPz7amu9v8dx1scXbXtZ2uvyTautLT9zhCxUX4a+OnYhphlniqviev/mBADo1X8DdcvKytLS0mT/XrJkyebNm1NTU9944w1DQ8Oamprw8PC7d+9+9913va55r7ZCnObRibTN9z7bF/SlqUbH4+le+ekxoyc7zn+WO/tpznkfKy1ST8MTV1lbIoQCHz15EDDe+n8LdwD1wbJnuq6zTDtZ1FIusJxhAGvcAzAY/40jPH369IoVva8xpCLjDjEfR9idO/mPjiWe+W7STmuWxQCeLpJKtyeXXS1tuDqO46TZy1non/MKvkrPhCzsbCSNI+xBG0+cfrqIrEG0CzHBk/o0P8OIBOMI1Y3CP+D/HRHOmjUrNjZWgV2rp2CrAAKe8P797XsDQh117fr7dCIO9/UoE2ct6qSHmT95W8w1YfXQGI4L1RyJQXBZa5lzsTT5aAHMxAbAgP33ydHV1dXV1cWwlBEjyMKfRqR+GvHVLv9PXPQdBtDDEgtdZ03avGe5UbW8Pa4m+O5PfEEWqjk8EWcXYlrysDrxh1ynlRYMmIkNgP5T39MpSuVnMvrzcR9uf7wntjxxYD14atNjJjvG1PFef5LT0NbTuLFVcO+MmsMh00B9q1mGKccK6tL6urwJAEAOglBZPAxcv/T/eNfTfZHFLwbWgx6FeHuirasWbczd9JRGfg8tIQuBnruW0wrznEulZZG1WNcCwDADQahEo9jO3wbu+CEm7FHR04H1ILtkuNPFOOhR1h8lPQ1ShCwEGpb0URutK57V5Vwuk/Y6CgcA8D8QhMplr2OzP2j3kbiTt3LvDbiTEHOd8Am2HyaWfJZcKu7+rl3IQkDVIbtt5ggb21J/hpnYAOgrtQhCaZsYFWF2vshSy2x/0K5TL3/7M/P6gDvx0qbHTHGMrmsJjsiubO12ulfIQtBuJrZ8mIkNgL5QjyDktYp/ui/KLseqAHNNkyPBX/+ZeeNC+tUBd6JPId6ZaBtsqOl+J/1+Zbf3RMizMLcZslBN/W8mNp2XB/Oa8uC/AQC9UIsgxLMY+KX+vMO3JNV9Wv9PGQwY7CNTvv4n78GxhDMD7gSH0McOhr+OtVoanb8ztay7y0CyLAyKgCxUa4a+OraLTNN/gZnYAOiFWgQhQgjnYESZ4dl8KFwqwGwZKR2a9g9Bu6PLEw7GHpOigd/LEMjWiJrscLeSO/tpTp1Q1GWbVdaW250gC9WdtgPTdZ1l0a2qghuVg/gfB8AIpy5BiBCiTnUnWhvwjt9D2M0Sp03VOjj5q8za3P3RYZJBlGFKI0dMsnPWpHncSY+q7Trq3rGCLASIbkR128JpyudlnCuWtPW+7CUAakiNghAhRF86Udrcyv8rBsMamGTG/qAvS7nlXz3bL5YO/L4+2ciKQx5ms5/mHszueilEyEKAZDOxrbPCEXDJRwvauF2fQgBAnalXECICnrnhNeHTDOGLbAyroBKpeyZubxJyQx/vbRMP6lTtbBNWZKD96fyat6Pym7ta5h6yECCE8EScfYiptgMz6VBeS4UA63IAUC1qFoQI4TSozI3TW85HiPL7vaC8AlGJlD0TtxPxxM8e7xGIB3WPuy2T8iLIQZdM9O5mAhrIQoAQQjhkPpVtOcMg+Wg+zMQGQHtqF4QIIYK5HmN5IO9QuKQBy2Ag4Yk7x2/VpmptfbCzpa2nGdR6RSXgD3qYfepoGPAw61R+TecGkIVABmZiA6AzdQxChBDJy5oy0Zl3+BYSYTn7Bh6H3+a7yVrbYsv97U3CwX5JX2qpGxlovy+zcmlUAV/c8TQpZCGQ0bCkj3oPZmID4D9qGoQIIers0Xg9Dd6ph9iWgUO4jd6rR7Gd378X2iBoHGRvjprUF5MdhBLJuAeZuc0dLwVBFgIZqi551Ebr1lph2skisQBuJQXqTn2DEOEQfWWQpKyu9VYC1oXg1nuuCLT0f+/OJzUtgz1hpUEkXPC1XsvRH3s/489O83S/Y2UZ6uQAWQiINILzaguKNunloTxBPWaDawFQBWochAjhyETG5hmCO0ltiQVY14JCnOa9xgnacHdbWXPF4Htbba13y9/2o6TSTQnFba+e/lppZQFZCBBCODzOZr6x0XidpAO5MBMbUGdqHYQIITyLwVg/jXfivri0DutaUIjTvEVOb2y+t72EWzb43rx16DFTHLKbBUERWWX8V77yQxYCOZiJDQB1D0KEHAKnfQAAIABJREFUENHGkL5ofPMPN6TcVqxrQbNtX3vH7e1N9z7LbSgYfG+6ZOJNf5sZRlped9PvVLwyzypkIZDTdmC6wExsQI1BECKEEHmcPdmbwwu7jTrdbDn0gq0C1nuu+OB+aFpN1uB7k83T/ftYqxUxBR3m6ZZlYSBkIUCIATOxATUGQfgv2kI/RCK0/P4E60IQQijQwv/jsZs+jdidXJ2ukA4D2BrxwU5PapqDH2e1X85wpZXF55CFACEEM7EBNQZB+D84HGNdsCi9VPAgBetSEELI18R75/iPQh/vjS1PVEiHbArxnwm24/WYY+5lPG83TzdkIZCTz8SW+EMurxT7KwUADA0Iwv/gqGTm5hmtf8eI0kuwrgUhhNwNXHdN2Lbr6b7I4hcK6ZCIw+10Nj7iaT7nSc43Gf/dmyrPwhzIQoBD5lPZVrMMU44VwExsQE1AEL4Cr6/JeHcqL+yOpGqwY9sVwlXf6dvAHftjwu7kK2zg/+vGWjFTHK+WNsx9mtvQ9u/EOiutLHY4OQRBFgKEEMzEBtQMBGFHRHtj6izv5h9uSPmDmgtbUex1bA4E7f458Vx47j1F9WlOJ0dMsjenk33uZbxs+Hea0xVWFtsd7YMinmRxmxW1IzB8aVjSR23830xsYriXFIxkEIRdoEweRXQw4R29jVRjJkYLLbN9Qbt+Sf79r6xwRfVJweMOeph97mQUFJH1c96/83Svsrbc4+o0KSIyqUElDogBtqg6ZLfNHGFjW+qJQlErlrPyAqBUEIRdo789Qdom5l+JwrqQf5lrmhyasvdSxt/nU/9QYLeLLXQiA+0PZVctjSpoEUsQQovNzcI83adFPntei/0MAwBzBArecYU5nU15eSi/tU4lzpEAoHAQhN0g4JnvTRNGZwufZ2Jdyr8MGewjU75+WBh5KPZnqeKGPTtoUJ8HOYik0nH3M3OaBQihWcZGp0Z7znr6/F4llks2AhWBw+Os5xoZ++u8PJjXmAuXkMEIBEHYLRyDytw8k//7U1FuJda1/EuHpn1g8leZddl7nx8USxV2qopJxP821mqVtd64+xlXShoQQq8ZGvzp6xMSFft3Wbmi9gKGNUNfHbsQ04wzxVWxMBMbGGkgCHtCMNamvxPEO3JLUqcq949okJn7Ar+s49fviPymTazIRQPetdG/6W+79WXJ+vgivlgyQV8v3N93XVzi5ZJSBe4FDF8se6brOsuif2AmNjDSQBD2gjTKghLk2nzwplSoKnNtUInUvQGhBBzh40df8kWKHPXsrUOPn+LY1Cb2upue1MD31ta+N3H8B0nJp/ILFbgXMHzRZTOxFbRknIWZ2MDIAUHYO+oML4KRTsvP91TnWzAJT9wx/iMDhv77iljavj1NEuGcj9U2B8PJEVkHs6scNTXuThj/RVrGgewcBe4FDF8kBsFlrSWeCDOxgZEDgrAPcIi+MlBSy229GYd1Kf/B4/Bbx77nou+08e6ntfyOC/AO0lJL3ejJDheK6uY+zdWjUCMn+R/Nyf8iLUOxewHDFJ6Iswsx1XXVSPwhtxlmYgPDHwRhn+BIBMamGYIHKW0J+VjX8h/Z0vbBVgGb731ayatWbOdWDEpkoL07i+Z5Jz2fJ348yf/PktJtyamK3QsYrnDINFDfarZh6rGCulSYiQ0MbxCEfYXXojM3Tm85/VBcolqTToU4zXvDfubGu58UNSn4rhbZ3KQnR1ssfpG/N7363gT/R1XV78YnSqQqc44YYErPTctppXnO5dKSBwr+HgbAUIIg7AeCpT5tyYTmAzelXD7Wtbxirt2MlW5vb7n3WXZ9nsI7n2ygmRDslNsseP1J3s/eYzK5zcti4kSQhQAhhJCGBd1tM6c6oRFmYgPDFwRh/5BH25B9bJsPhSORas04FWwV8IHPu1sffpFcnabwzvUpxOv+NiEWOkGPspdYOFQLhPOfRQkkcNMgQAghCos0aoO1sLEt9edCEV+1PhcA9AUEYb/R5o/FMam8Xx5hXUhHfiZjdoz7MPTx11Flir+pB4fQJlv2PxNs96ZX6pGMEA7/xrMovhj+6gGE/jcTG9OUlnQgl18twLocAPoHgrD/cDjGmmBxQZXgXjLWpXTkbuD6dUDoNy8OPSx8ooz+PbXpicGOuhRiRiODgMjTIp9xRXADPUAIIRweZznTwCRAL/lIPszEBoYXCMKBwFFJzM0zW6/HitJUYgnf9hx0bfcH7Q5LOH0j57Yy+qcR8Ac9zL5yNY6qJbaKmYGPntQKYS5m8C9DXx27xTATGxhmIAgHCK+nwVg/jffTHUmlyn3gLbXMDk7e81vald/TrihpF/NMtaOnOJJw9GqB1rgHT8tbYTAZ+BfLjun6rhXMxAaGEQjCgSPaGdEW+Db/cFPaonIXRYyYBoenfH0n/+GxhDNK2oUFnfxokt3/WRqUtWiNufeiuEW17qQFGKIbUty3cLiFLRlniyRCuKkKqDrlBmFFRUVLS0sPDSorKwUClUuRviP7OxKdzVRnCd/2dGnah6bsTaxK2R8dpqSRf7KBhtfG2/JEWi6349KbVGVqcoA5IoPgvMYSR8Anh+ULYSY2oNqUFYTl5eU+Pj7e3t7GxsY7d+7s3ODLL7/U0dHx8PDQ0dFZsmSJPA6dnJxw/zNp0iQlladA9MX+CIfjX3qGdSFd0CAz9wftKuGW7X62TyRR1h2eAWyNvBmj7DS03O6k3CxXrdkGAIbwRJz9YlNtR42kA7m8Mjh5DlSXsoJw+/btjo6OJSUlqampP/74Y3R0dIcG9vb2KSkpZWVlRUVFcXFxhw8flj/05MkTqVQqlUofPnyopPIUCY9jrJ0ijM8XPlHFqThpROo3AZ+3idu2P/5KIFbWXS0sEiFmsus71rqznuR9mlykpL2A4QeHzIPZVrOMUn6CmdiA6lJKEIpEogsXLmzcuBEhZGJiMm/evF9//bVDm4ULFxobGyOEdHV1AwICcnNz2z8qHlYD1HAMKnPLDP6lZ22pxVjX0gUSgfSF/8csitZHD3bw2no6Uz1IRz1tj3gafZdZHvgwo6FtOL2DQKn03DSdVlrATGxAZSklCCsrK1taWuzt7WU/2tnZ5ed3O1d1U1PTzZs3p0yZIt8ybdo0JpPp6en5+PHjHvYiFovz2mluxvICFcFIm7FpRsuxu+KCKgzL6A4eh//Yd6OdDmfLve0Ngkbl7Wgdx/Smv3VUfbXDreSnNXDJEPxLw4LmvoVTk9CYc6kUZmIDqgYnVcJtFOnp6c7OziKRCI/HI4SOHTv222+/RUREdG4pEokWLFhAJpMvXrwo25KYmOji4iKRSA4fPrx79+7MzEw2m935iVlZWS4uLiYmJvItGzZsWL16dXclNTc3M5nMwb6w3kiTiyW/P8dvmooz0FL2vgbmj5zr94oe7/bdpkfTVd5e4hqb5sZkSJHOu5baH3N08Tjl7aprQ/N2g/6SCKWFf1RLRcjiTT0CVWHfwgUCAR6PJ5FIiuoQqLh+fcCpVGqv/zeUEoS1tbV6enq1tbU6OjoIoa+//jouLu7y5csdmkkkkqVLl9bW1v71118UCqVzPzY2Nt9///2cOXM6P1RcXDxu3Liior5ejuJyuRoaGv18HQMheJTaeiNOI3Q+Xos+BLsbgD8zb1zO+Pv7wC9MNYyVt5f4+oZpkVG6JGMjGuWcj5UJbUj/SA3Z2w36SyqRFoZX1SY3Oq20oLG7+NQPAAShulH4B1wpp0Z1dHRMTU1jYmJkP8bExLi5uXVoI5VK/7+9+45r6mofAH5u9oQEwgiELUsRRFBRNgKOiqNorVpHf26tWts6Wn1b9a21rds6qq3a2qpt3YgLkI2IiqIs2TvITsiA7N8f8aWIqIyEJOR8//ATbk7uffQanuSM56xcufLFixeXL1/uNgvKZDKhUEggENQRofrgg4bh/V35P17TwsWFSpHOUz4ePmdd3JbiFjXurTiSTksOHieQs+l46ajY/OtsNfbHQjrkZSW2YJNnR8q4xbASG6QV1JIIEQRZuXLlV199lZWVdfbs2bi4uEWLFgEAysvLvby8lIN5n3322YULF+bPnx8dHX3hwoW0tDQAQElJyaFDhzIyMjIyMhYuXEgikfz9/dURoVoRpo3CuLL4h25p2w4VHSbYh6wftWJDwracBjXOdHWhUpOD/J9xqmdZ4z59UrUgo1wog2urIQAAMPehO89jPT9TVf9Q6wozQXoIo6bzbty4USqVLlu2jE6nR0VFsVgsAAAWi7W3t1cOHBIIhNDQ0Bs3bijbe3t7+/r6kkikR48enTt3Do1Gjxw5MjExkUwmqylCtSLN8xMcvSM4EUdeGQ6QAR8i6wE/1hgShrg1+buvxn06mjlSTVexJZOSgwPCktKmWprVtStGxz4/P9ZuuCFRTZeDdAjNieK+xj7v1wp+dZv9dCbQxncJpC/UMkY4ALR2jPBfUhl/fzTK1JC0MGhAr9sb+U2FW5J2rvVeFmTtq76r1ItEE5LT/BjG7oaWW7Jrdg63XGrPUN/lABwj1B1SgSz/t0osGe00l4XC9bGDCo4R6hvdGCOEAAAAgyZ/MklaUtcerfrdAVXF1dhp7/j/Hs48eaMkVn1XMcXjE4L8M1s4j1qqEoOdjhY3zLpX2iLW0n5jaCApK7GhcKinP5WKOBJNhwPpKZgI1Qgh4iifTREl5oqSVL9rvKrYGVofCvvuz5wLf+dfVd9VaFhsbIBfqUD4bX5OSoiTAwXvGZOX3AAXGkIAhUGc5rLMRtGfHSrlV8PS7ZAGwESoXigambpxWvvlDElmqaZjeSMLivlP4d/fKr2rvq0qAABkDDrK16dVIp3/4OF2N+Yvo2zm3S/b/KxGon31yqGBZxFgbDedmXuioimnVdOxQHoHJkK1Q5kaUj59T3A6QVpYq+lY3ohBNNo//tuHtU8OPfpFTVtVAACIaPTlcWOwCGpqarovg5wZ7vqM2+afUFAq0NKlJtBAYrgbuC23Lb1SW3lHG8szQYMYTIQDAW1nSlk1QXDktozdoulY3ohOMDwY9l0pp3xL0rdCibp6qHAo1HmfUSwScUJyGh6luOE/ZI61kU/c878qm9V0RUiHkC0JHmvtm3N5RX/DSmzQwIGJcIBghrKIc3z5u6/Jm7S3Bj8ZS9o7foc5xWzZ7c+qWmvUdBU0gvzqPdKbTg9JSm0Sidc5mt70d/w6h70go1wghQsN9R3OEDt8tZ1UIMs+WiYRwBlV0ECAiXDg4HycCO+N5O+9rhBo795saAS9znvZbNfpa+O+elKXraarIADsHzH8A5ZlQGJyTVubtxEpM9yViEa8Y/OzOGrcHwPSCWg8yvVja5oT5enBkrZ62G0OqR1MhAMKH+qOdbfh772uEGn1TPGIIRO2+W3ckbbncsEN9V1lk4vTQhtr/4TkEr6AikEf97bZ4WYxKbn4YFE97BTTdwiwnmDKCoGV2KCBABPhQCPO9kVZGAmO3gHaXW/Mw3TY4bDvrxbd3JNxRH27229ycdro7BSYmJLb2goAmGVFTx/v8k9Vy8TkohftWv1ZARoA5j50549Yz89UvbgHh5AhNdKXRCgWa83IHALIHwcDuUJw8i7Q7i8+llTmzxP2NLU1b0rczhOra83fCge7H9yHhSenPeVwAQC2ZFxSsNNYY/LImPzbL+BMen1Hc6R4rLWvSW4qvVKr5e8XSHfpRSLk8qou3ZgmEGrN7thoFHnNJHkdt+1KhqZDeQcSlrgzcIsT3WH57c8ruFVquso8a6tjI0dMTLmX3tQMAMAgyLZhFn/62C19WLHuSZUYLjTUbwQGzmOtvaC2Pf+3SplYq/tRIB2lF4nQkGrl7BB56+4ahUJbJqEhOAzl0ymSB8WimKeajuUdUAhquefCuUMj18R+mV7zSE1XmWrBPDVq5NS09Li6l2vIQkypT8JdywQiv/iCYj6cMaHXMCS023JbDAH97FCpqAX2mUMqpheJEADgOXwlAMjDrJ81Hci/ECqBsnFa+50s8YNiTcfyblOGhO8M3LIn4/C5vEtqusQkc7NLY8fMzXh0jf2y8gADj7nmN2SJPcMvvuDPCjhKpNcQNOI4x9JsNP3ZT6X8KliJDVIlfUmECIKaGLI/K/s0+4W6vtP0AcqIQlk/RfhHkiRHXb2OKjTcxPXYxD3x5Sm7Mw5L5FJ1XCLAhHHTf+zKzKwL1S9XMSIALLNn3A10/PH5iw/SSzkSbflOD2mERYCx/XRm7i8VTdlw/BhSGX1JhAAACtk8PGjPzbg17e1aVN4FzTKmrJ4oOB4jq9CaIcw3MyUxfgr/nitqXR+3taVdLZvOe9PpcYF+nz/NPlVW0XFwmCExI9SFScCOjMlPb4KT6fWasbuB2wrb0quwEhukMnqUCAEAttZBjvaT7iR8DrRp/hnGxZL8cQh/X7S8Tgd26yZiCP8N+NLL3GPVnQ1l3J7uB9krQw2osQF+2/OeHyj6t9OYiEYd9LTaO4I1PbV4Wy4bTqDRZ2QLgsc6B1iJDVIV/UqEAAB/n6/a2puf5vyh6UBegR1pR5g+irf3upyrA3VVEIB87D5nyYj56+O2pFU/UMclnKmUlGD/o8Vl2/Oedz4+w5L2KMw1vp4XnlzIboOTJvQXzgAz/BM7qVCWfbRMCiuxQf2jd4kQhcJMHH/wfuaB+sZcTcfyCnywG26ME39/tEJHFpKPt/H/LnDr3gdHTj87r47zW5NIKcEBl6vZm7NfuVNWJFxCkJMfg+IVmx/NVkv3LKQT0DiU6yJrA1tS/s9V7Y1iTYcD6TC9S4QAAJqBTbDfjhuxq8US7RptIkaOwTiY8w/eAFLd+IQ7lOF8YuK+dPbDHWl7RDLV/yYyI+ATg/wT6xtWPc7qvDkUGkG2DbO44uuw7knVuidVIthPqrcQYBthzgw0yvu5ClZig/pMHxMhAMDJYQqLOToh9WtNB9IVaZ4/QsILfrkL1LYpoGoxSMaHw75HALI+bmtLu+rHOOk4bFygXwGPv/BhpvTVfxMfY/LjcNd6kdQrJj+HC+fT6y+Gt4HjRxYFf1TVwkpsUJ/oaSIEAAT7/7e+MTevUF2r4voIhZCXh8lb+MI/kzUdSk/h0Litvp/5WY1ZduuzgmbVr4mkYDDX/cY2iMQz72WI5K8UFjHEos/72G10MQtOLDxYBOcQ6i+qLdF9jT0bVmKD+kR/EyEGjZ88/lBy+s5mTommY3kFgsNQ1k+RFta233qi6Vh6CgHI3KGRq70Wb0rYnlKVrvLzk9DoKF8fDAp5/15Gm6xrv/ECW+OUEOffypoi75W0iHWjVxlSOViJDeozvUiElULe6pxkoazrGnBjI6dxoz6/EbNKKtWuDQIRIo7yeYQoLluckq/pWHohyNp3T8j2nzJPHn/yu0LVH8txKNTfPqNN8biJKfdaJV1vpQuVkD7emUXEecbkpTaqqz44pOVeqcTG0Y1JZ5A20ItEaEmkyBTyySmXedKuEzrch85jGDun3N+lkcDeAkUjUzdObbt4X/Ks4t2ttcYQuv2xCbuf1GVvT93dLlVxgVA0gpwa5TWCZjg+KbVJ3PVWEtCog55Wh0ZazbpX+mNxk0xHBlkh1fq3EtuhUn41HDmGekQvEiEaQY4ND/KgmYQkXmgSd/3yF+K/s6wyvrjsjkZiewuUGY28brLgRJy0+IWmY+kFYyL9p7BdODTuk5hN9cJG1Z4cAeDACPcgU0ZAQjK7rZvv8VMtaI/CXFOa28YnFlUJ4ZR6PfWyEtsJWIkN6hG9SITg5S/Q4EATVmjShQbRK58T8Tjqe6GH7yZ/2cqr1lR4b4KxNyMvHS84dFNWq0Vl4d4Ji8Z+NfbTifbjV93Z+LypSLUnRwDY7e62wMY6JCmlStjNR35LIjZqNGsGi+YVm/9XJZxGqKeM3Q3clsNKbFCP6EsiBAAgAOzxCPzQyiUg4e+atleGkcxMPbw8lt+IXS1XTy3p/sB62BI/9OXvjpI369jQ10yXiC/GrNqcuCOmLFHlJ9/k4rRmiENAYnIxv5vVYwgA6xxNb/o7fp3DXpBRLpDCqRP6iGwJK7FBPaJHiVBpk8uoRbbD/BP+LhO8UpTEe8QyEpGR/mi/pgJ7C9w4Z3yIG3/vdYVAx7bl87Hw3jf+21PPzh5/8rtc1YN2q4fYf+XiHJiYnM3tvvvL24iUGe5KRCPesflPWnSgdh2kcjgDzPDVdlKBLPdEhbQNziiGuqd3iRAAsMll1AZn75CkC8X8zgvAkfCgH/MKLlVUp2gssjcjTPHCuFryD91UiLXuO+vb2dNsjk/cm9dY8FXSfwUSFWejpfa2ez2GhyalZjR333VMxaCPe9vscLOYnFL8w/MXsASNHkLjUS6LrChWxKz9JW31OvZREhoY+pgIAQArHTy+chkTmPhPDvff2RxEovGk8QfuxH8mEGrjoAJpnj+KThEcvq0rBdg6GOIN9ozfQSPQ1sRsrhOo+N/2QyvWyVEjp6WlpzU2vanNLCv6/VCX62zuxOSiWh0p5QqpEIJCbKeYsUJMnh0pg5XYoNfpaSIEACy1H77HPWBCyuWnnH83AmRZ+Li5fnjr7lqFQvtGlRCEvCwUwWP4R+8AmfaF91ZYFGazz9oZzu+tuLPhab2Ky51PYZr/5TM6Mj0jpu6NWdaGhEsMdhrHII+Myb9ZC0t16yNzH7rzPNbzM1X1D3VgvzNoIOlvIgQAzLF2+Xlk6MSUy/ebajsOjvVeDwDyKOu4BgN7IxRCXh4GpDLByXhdKUbaWcSQCZt81nyT8v2d0njVnjnIhHHdd+zCB5mXa9hvaoNBkG3DLM772K3IrISluvUTzYnivsa+Kq4BVmKDOtPrRAgAiLCwP+kdHpF2NaG+SnkEQVATQ/Y9yT7JfvFIs7F1D4MmfzJJ3sQT/qEzxUg787Hw/ins+z9zLxx69ItcpV+7RxnRb/qPXf346e/lb9suOMiUmj1haF27xDsWlurWR0QTnMdaewG7/fnvlXJYiQ0CAOhJIhRxqgtORbQ1dl9TdDLT7uLYiDkZN2LrXtZwoZCZYUG7b8ataRdpYxcKgsNQPn1PVlonPKeN83reycrA8siEH8u4FZsSd6h2+ownjZYQ5Pef3LzjlVVvaWaIRf811n6DMyzVracwZPSw5bYoHOrpT7ASGwSAniRCPI1lPm5V7pn5vOrH3TYINGFdHBsxL+PmNfbLZGlnHTzEbmJs4qYBDLMXECKOsnGa9Dm7PUorv7a+iwGOuidkO4tqsTpmUy2/ToVndqFSEwL9j5RVfptf8PaWnUt1N+vaXFyon1AYxGkOy8TTEFZig4CeJEIAgKHLJMcZewr++YRT3H2Poh/D8pb/+ysz4y5WFyqPBIzdwhfUPc09M4Bh9gJCwlO/mCpOL9ChTSo6QyPodd7LIp2nrIrZ+KQuW4VndqCQE8aOvlX74uOHmRL52/q+XKiE+6EujhTCyJj85AYdq1cA9RcCWCEmdtOYuScqmnN5mo4G0iT0tm3bNB1DX7S2tp46dWr9+vU9bC8Wiw3MhhjYjCq8vB5DNCSbD329jQWRMpFpNz/jtjGOOIJmgiAoK8uxd+LXW7P8yCRTlYavGggeix1pLzyThGDRGDttjPCdnI2GONLtt6fuJmDwrsZOqjotWiZdMMThz8qq3yoqp1kwCWj0m1piECTUzGCYIXFhRnm9SBpoQkUjiKrCgAaGTCZDEAT95rv8FiRzvKE9ufBslUKuMLAjqzw2SB3EYjEej1fhCfXlG6ESxcJ92Pwz1cmH2em/dttgmIFxbGDk1py0I8VZAACagU2w344bsavFEi1de4QyolA3TmuPeiROL9R0LH3kZe5xdMKPVwpv7sk4IpWrbIkkGYO+PG6MC5Xil5BcKXzHSGSYmUFmuOszbpt/QkGprpXvgfqJakP0+NSh8Qm3+B9YiU1P6VciBAAQGQ5uH//T8Oxq+Z1vQXcTqF2oRinBs/cXPT5Q9BgA4OQwxZI5KjH1mwGPtKdQpoaUjdPa/koTP1D97vADw5LK/HnCnqa25s2JO/hilX3mQCPIT54eS+xsfeOTn3DeMe/JFI+54T9kjrWRT9zzc7BUt57B07DDP7EX86S5v8BKbPpI7xIhAABHNR228CyfnVN8bZOiu68gtmSD+MBZR4uf7si7DwAI9tteW/8kv/DygEfaU2gmnfJ5RNufybq1eWFnJCxxZ+AWR7r98tufV7aqchuQdY4OBz3dJyTfu1n7jt2slKW6bwU4bs+tXZBRzoeluvUJGo9y/diawiI+PVDS1gA38NIv+pgIAQAYguHQj05LhM0FF1bLu9ue3ppETQmefbG6cHN2ChZDei/saFL6t82c7hdgaAO0NYO8brLglzhpwRtXlGs5FIJa7rlwztD318Z+lfniqQrP/L6lRZSvz5JHT46Xlr2zsRedlBXuSsehvWPzH8NS3fpEWYnNMoiRfbiUW6KloyGQOuhpIgQAoLBEl9nHsSSjvDMLpG3d9JuZEUiJQR/E11dteJZsbOQ8btRnN2JWSbvLmloC42BOWTmBf/iWrEyH18ZNGRL+34Avd97bdy7vkgpP62NslBIcsK+weF3Ws3eOAhHRqIOeVjuHW0xKLtqWy4YlaPSK+Vgjp7ms579X1T/SxmXEkDrobyIEACAotEPETqq1d+7v88St3fSbGeEIMQGRqY01KzPj3FznGRs5p2Z8P/Bx9hxmKIu8LIy/P1pWqeKt4QfScBPXw+E/xJQl7s44rMLpMw4U8r2QwMctnEUPMsVvXVahFMmiPwxzja/nhScXstvgsms9QnOmDF9pW3m7vjy6DlZi0wd6nQgBAAAgNqEbTUd+kPP73LambvrNaFh8XMDMIj5nwYPbgX47SivulpTdGfgoew473Jq4IJC/97pubWrfhQXF/NiE3Zx27qdxW1raVVYj2xiHiwv0E8nlk1LucSTvzm3WJFx8kJMfg+IVmx/NhqW69QiJSfA7wZ78AAAaz0lEQVRY79BaLnx+pkougaPFgxxMhAAAwBy90Cpwbe7v83g1Wa8/S8Zgr/tNbxAJF2elTQg5FJf8ZSuvZuCD7DmctwNx1lj+7ih5Y/c71uoEIobwbeBXXuYeq+5sKOO+rXxor+BRqPM+o/wZxn7xyRXvWlYB/leq+4qvw6dZVcsfVbTp2r4fUJ9hyWi3FbYoDJJ9tFzCg7WHBjOYCF8ycZ/uMOXbgr9WcEq6KeBJQmOu+00XyWUriis8hi+5EbtaLtfqNwbOz4Uw2ZP34zU5R4fH/BGAfOw+Z8mI+evjtqRVP1DdacG2Ya7L7G1945MzW3o0DuRjTM4Mc+VJ5aNin2fDUt16A4VBnOayaM6UpwdhJbbBTI8qy7yzEgHR2I5q4118+TOcgTnJtGuVEzSCirR0vMYuiRMRRsurONxSK8txfQl9oGDszUCbpO2fdNwYRwSH0XQ4fWdPs3E3HfZd+gGZQjbcpJuSQK/rye0eY2xkTybPuf/Q1YDqRKW885wENCqSRadgUPMyyvBolI8xrEKiLfpTWebdEEAbQsYZYArPVvPZ7RRLAoakngtBPQYry6gX1XKE60e/VcT9WHv/9OvPYlGov3zeMyNQfsb55zy/UFGt7Zs/EKZ6Y0fY8n64qtDxailDGc7HJ+xJrEj7b9pesUxla7ymWzKv+41dnvnkaElpD1/SUap75r3SKiFcbaYvGCMMvbY4kZn4pwdLiy+wxa1a3SEE9RZMhF2RTBzdFp6re/xX5d09rz+LRpBToyY4G9tG0SNux38mEDa83karEGeNxQ6z4u+7rmjX7XmPDJLxobDvxDLx+rtbW9pVNq99tBE9NTjwp6LSdVnP5D3b61hZqtuFSvCMyV/0oDy/VXtX1EAqhMahWCEmXpsdMUT0491FpVdqJXyYDgcJmAi7gadZun38T2vFg+Koza+XnkEAOOQZ4mTln4Efev3upwqV7i6rDsTZvmiWseDQTYVEt2tHETCEHQGbfVljlt36rKBZZfXk7MiktJCALA539v2HbbIe/RPhUci3wy3Kprh50klhSYURqcXpTTo8Fgv1HIaEtp1iNvKLIQCAzB+KyqPrZO3a/hsAeieYCLuHIRoOnf+7hN9QeHHN66VnEAD2eQTauC3LbK5JeHRYIxH2AgJIC4MQKkFw5DbQ8UmPCEDmDo1c7bV4U8L2lKp0VZ3WCIeLCfDFoVDjk1IbRT3t8KRi0OscTUveGz6LRV/0oNwvvuA6mwtXnekDnCHWfgZzxHoHaZvs0a7C6vgGuRTeeR2GKHrWHaRtqqqqfH19Kyt7Oquex+NRqdTeXkUhkxRf2yjmN7rMPorGd/PyH5/elj74fHLYzyNs/Xt78oEmk/MP3USwGPKqCQCl8zsNFbeUfpX03ST78YvcP0RA179OH283ANtz889WVt30H+dIeff0mc7kCnCjlrsjjy2WKz53MptrY4SB2zkNFJFIhEKhsFisRq4ufCGqvFPPqxRahZqYjaEjuv/m0n59e4O/BUyE76Qoj/2eW5rmOvcUjtrNnn8/3f+1JefQrPevuxrZ9On8A0gq4x+4gdDI5MXjX8sduqexrXlr0ndMitkmn7UEzCtTyPrzPjlZVrE1J+/SuDHjjI368PLURv4Pz1/kctvXOZkus2cQ0bDTRe00mwiVeOXC8pt1Ep7UeqIpw91wELy/tJnKEyF8l74TYhv2JWP41JzfPmxvLn/96TU+S4wsAg5dX5bb2jTQofUWBk1eO1lezxWeTdZ0KCrAIBr9FLYLg8J8ErOpXqiyknKL7Wx+H+0VeS8jil3bh5f7MSjX/YacGWMbV9dqdyNnWy6bo+NDs1BPUG1Jw1fZ2c9gVt9tfHqohFPE13REUC/AdYQ9YmDlhSFQi69tNLT1wVFMujzrZR9alv/HsYL7ZznSelGbARZviif1+VpqhaBROG+H9qhH8hY+1pWl6XD6C41C+1v5CCTCPRlH3U2HMkjGyuP9vN0OFHK4uen8B5lYFGq0Eb0PZ7Am4eZaG000N4hmt37yuIrdJvGgkahYuP5MLdS7jrA3CMY4cx8jDBFddu1Fcy6PZIbHGWrye+pgpfJ1hLBrtBeaC+JKo7c6zthraO/b5SkOtyw6bm0Tp7id4vAUMStAWwxheo83t4uwcGAStG7ltYLXztt1GTfOmTDFS9OxqEZ6zaMf7h9cNXJxuF0QUNHtrmlrey81PdCEsd9jOKofA37lAvH+wro/KpqmMGlbh5o7UQn9DAzqQhu6RrtQyBR1D1oqYxpIZni7qeZkC3jTVQmOEb6kkUQIAGityCi8uM5u0jfGQye9/qxEIqite1JZk1pUmcTllHFINk+BsYTq7GYbEGrmEGBiiUNp/kOrkpwr5O+6ggseRpgwQtOxqEYJp3xL0s5ga7+lIxYI+HyV3O4WseT9e/fpONzZMd7E/n3haBBJjxTXHylu8DEmfzPUwttIS/sMdJEWJkKll+nwdr2BPdl2ihnBGKfpiAYJmAhf0lQiBAAI6wvzzy9l+a80G/nhW5qJJYIXdU8qalILKhJ43PJ6gtVThEkxcfey9o+wdBxqYKySYPpD3szn7bxMmOqND+xR3TLtxxW1fp38vSHBYO3wJQwaQyXnFMvlix89LuYLonx9TPrdG8OXyk+WNe55XmdDxm1yMY+wMFRJkHpOaxOhkkwkr01rqkloNHY3tJ5oiqPqcL1DLQET4UsaTIQAABGnOu/s/xm7hFuP/6In7ZVJsaAysbAisZ1fVY1lVhPsLSx8/ex8w8ztaFhVdnb3iryOw9t1hTjHDzfGUVMxqJZEJtnz4EhhU0mIrT+LamFJZVpSmWRsv75+KZdV/FlZdcNvnHMPqpK+k1iu+KuyeVf+C2M8ZpOL+RQLQzjHsD+0PBEqSQSymoTGFxnN5mOMWKEMDEFbOod0kc4kQjabvX79+sePHw8dOnTv3r1Dhgzp0kAmk23fvv3SpUsUCmXz5s0zZsxQHs/JydmwYUNJScmYMWP279/PYHT/uV6ziRAAIBE05p9bQmV52k38D0B6Mfm2ra2ptv5JXlVaaWWyWFBdgTHjUlysWX4ThgT7Mlj9GYvqG1l1E//Ha6T/C8GOsB3gS6vPrYK4qjZ2Da+2mseu5tUSMXhLKvN/edGCRWFaUpkUXO/Gbk+XV3yZnXtx7Bg/hmq+zSuXHn6bVyuUyTc4m82xNsLCJWh9ohOJUEnEkVTFNjRlt1oGGVv4G6OwcN5+X+hMIgwODnZxcfnyyy9//fXXixcv5ubmIq/+ij9w4MCpU6f++eef8vLy2bNnp6enDx06VCKRDBkyZNWqVR9++OHXX3/N4XCuXbvW7fk1nggBADKxsODCajSO5DhjHwrTl291wram0uq0J+VJL+oypW0N1VhzLH2Ys03QFKdQK5KBaqN9C2lpHX9/NGXVBIzuzyNV6nK7eWI+m/+ill/H5r8o41RWcKuqeWwUgrKgmDMpZhYUcwuquQXF3NbQ2pj4tjmicXX18zIeHfR0/9BKlf9QyqWHOdy2T53MltozSHDpYS/pUCJUaqsXVdyq51UIrcLgGvy+0I1E+Pz5c09Pz4aGBgqFIpfLLSwszp8/Hxwc3LmNi4vLzp07IyMjAQBLliwxMDDYt2/ftWvXPvvss5KSEgBAQ0ODpaVlaWkpi9XNLx1tSIQAALlMXHx1g1TY4vzBUTS+X51mwrbG3IrkJ2VxLfVPpCJOPd6SZOzhZhcyzTmMglH7GLu0qJZ/6CZlzWSME1Pd1xoAPbndnbMjm/einFtZyql4PTsqH3e8KpvbOiU1/WNb623DXFUb85MW4f7C+ti61uUOjLWOpka6vHPWANO5RKjEr2wrv1nX3ihmjWeY+xjBNfg9p/Lf52p5s+Xm5jo7O1MoFAAACoXy9PTMycnpnAjFYnFhYaGX18u5+97e3pcuXVK+sOOgiYmJlZVVXl5et4lQS6DQOMf395Xd3J73xwKXOb9gyX3vNCMRGaNc3h/l8j4AgCeoTyuJyymPL32w/UDaFzyKo4npSG+74ECHgNfLiakExpFJXh7O/+km9fMItG03BXQGHyqO4mw0xNnolU77ztmxoKn4Tml8GbcSAUjn7HjMjfF5fmWTWDzd0oL22i9fMgaNQ73ylQ6DIFRM12YGWAz61T4STzrpzBjbEr7oUFG9863cudZGG1zMWEQ4z3DQolgT3VbYtpYJy2/UsVOarSeYMDzg5CnNUEsibGhoMDT8947S6fT6+vrODRobGxUKRUcbGo3W0NDw+gtpNFqXF3Zoa2tjs9l0+r8dWV988cXatWu7bSxkP8uInqNA1FhvmiLB8vaPBar7dm0AgHLbXw4GmyOqa+LdTi79JyNOLFdrMSA3AO78oMbz6xpLACyVj5pf/tEMAABgAQCgAjxQz0XNAViMMk5tnexYGIio8L8UpLWUJTqe14DnGg5EV9hJnt+fNreHjQkEwjt7C9SSCGk0mkDw7640ra2tnTOWsgEAQCAQKI/zeDzlAxqNVldX95YXdiASiebm5tnZ2R1HKBTKm/62VGdfP0o8iaTelVsysQCoZ0umwP89aGznytW865NCLEVkOv/LV9jeTiKodwmzUKbivY6lQCHstIPBNADEinaRHI4XvptCKkYQFEDreGeyQgFgMb6eMaZ46UDXqJ2dXUlJiUQiUWamwsLCJUuWdG5AIpHMzc0LCgqU3Z4FBQV2dnbKF0ZFRSnbCIXC6upqe3v7N10FhUK9KU2+DmtgRlDDGOEA07oSNdpKTUPCkHbS0TFCqM94PJ5qT6iWz5ujR482MzM7efIkACAqKorD4UyaNAkAkJ6evm/fPmWb+fPnHzhwQCqV1tbWnj9/fv78+QCA6dOnFxcXx8fHAwCOHDni5ubm6qriKQkQBEEQ1JlaEiGCIGfOnNmzZ4+5ufnKlSvPnj2rLJBaUFAQHR2tbLN161aZTGZqaurq6rp48WLlVBoDA4MzZ87MnTvXzMzs9OnTp06dUkk8fD4/JSVFJaeCdEJsbKysZ3vNQ4NAfn5+WVmZpqOABkh7e3tCQoJqz6neyjIcDkc5HPgmQqEQi8V26dNQKBStra2dZ828rlfLJ1JTUzdu3Hjv3r2eNIYGASsrq3v37llZWWk6EGggfPHFFwwGY/PmzZoOBBoIT548WbRo0dOnT1V4TvUOL789CwIAup3AgiDI27MgBEFQZzpaKhLSEnBOGgRBEKTXYCKEIAiC9Jqu7j5RUlIyfPhwX9+uG+R2i8vlFhUVeXt7qzsqSEukpqaOGjVKtXtYQ1qrqKgIh8PZ2NhoOhBoIPB4vLy8vDFjxvSw/YwZM1atWvX2NrqaCOVy+e+//97D2RDKRRpw6oT+KC8vt7GxQQZ8Kw9II5qbm9FoNJxYoCfkcnlVVVXPP/fY2dk5ODi8vY2uJkIIgiAIUgk4RghBEATpNZgIIQiCIL0GEyEEQRCk12AihCAIgvSajm9c0gMymSwuLu7FixeBgYG2traaDgdSvaysrMbGRuVjAoHg5+enfKxQKOLj46uqqvz8/IYMGfLmE0A6QCKR5OTktLa2BgYGdj5eXFycmprKYrFCQkJQ/9sSWSqVxsbG1tfXBwcHW1tbayJeqL8qKiqKi4tHjBhhbPxyw/PKysrCwsKOBuPGjeuoTZadnf3o0SNHR8eOt3/vKAY1uVw+efJkLy+vJUuWGBsbx8TEaDoiSPUmT57s5uYWGhoaGho6b968juMffPCBu7v70qVLGQzG1atXNRgh1E9xcXF4PJ7BYBgZGXU+HhUVZWxsvHTpUg8Pj8jISOVBmUwWGho6evToxYsXGxkZJSQkaCBiqH+srKwMDQ2xWOzt27c7Du7bt4/JZIb+T1VVlfL48ePHzczMVqxY4ejouG7duj5cbpAnwri4OCsrK4FAoFAoTpw4MXr0aE1HBKne5MmT//jjjy4HMzIyGAwGl8tVKBTnz593dXWVy+WaiA5SgZaWloaGhrS0tC6J0M3NTXnrW1tbzczM7t27p1Aobty4YW9v39bWplAoDh8+7Ofnp5GYof4oKSmRy+XW1tZdEuHChQu7tGxvbzc1NU1MTFQoFGw2m0gklpeX9/Zyg3yMMDo6etKkScqvzzNnznzw4EFdXZ2mg4JUr7i4+Pbt2+Xl5R1HoqOjw8LCDAwMAADTpk0rKioqKSnRWHxQ/9BoNAaD0eVgWVlZXl7ejBkzAABUKjU8PFy5y1t0dPR7771HIBAAADNnzkxNTeVwOAMfM9Qf9vb23RbEaG5uvnXrVlZWllwuVx558OCBQqEICAgAADCZTB8fn5s3b/b2coM8EdbU1FhaWiof0+l0IpFYU1Oj2ZAglSMQCMnJyQcPHvTw8FizZo3yYE1NDYvFUj4mEolGRkbw1g8ybDabTqeTyWTlj5aWlspb3Pldb2pqisVi4a0fHFAoFJvNPnbs2NSpU319fVtaWsD/bndH1rS0tGSz2b0+s4oj1TIymaxj/BwAgMFgpFKpBuOB1OGff/6Jj4+/devWs2fPzp49GxMTAwCQyWSdP1HCWz/4dLnFaDRaeYs7v+sRBEGhUPDWDw6ffPLJo0ePoqKilNVld+7cCd7836BXBnkiZDKZ9fX1ysdCoZDH41lYWGg2JEjl0Gi08oGNjc24ceMeP34MXr31Uqm0qakJ3vpBxtzcnMPhSCQS5Y91dXVMJhO8eus5HI5IJIK3fnDoeKfj8fjIyMisrCzw6u0Gnf4b9MogT4RBQUFxcXHK3uSYmBgnJ6eOPhNo8BGJRHl5ecrp8kFBQfHx8crPhomJiQwGw9HRUdMBQqrk4OBgYWFx9+5dAIBUKo2Pjw8ODgYABAUFxcbGKhQKAEBMTIybm5uJiYmGY4VULTMzU7mPgre3N5fLzc3NBQAIBIK0tDTlf4NeGeRFtyUSiZeXl7Oz89ixY/fs2bNr166FCxdqOihIlZqbm2fMmBEcHIzH4y9fvgwASE1NxePxcrl83LhxpqamwcHBBw4c2Lhx4+rVqzUdLNRHDQ0NW7durauru3379sKFC5lM5rZt2wAAx44d27Vr16effpqcnFxTU5ORkYFCoUQikaenp7u7+6hRo3bv3r1///45c+Zo+m8A9c6uXbvKy8vPnTsXEBDAYrG2bt1qZWU1e/ZsGxsbExOTjIyMu3fvpqenu7i4AAC2bNly9erVpUuXXrt2zdDQ8OrVq7293CBPhAAALpf722+/1dfXh4aG9uGTAqTlpFLp1atXc3Jy5HK5q6vrzJkzsVis8imBQHD69Gk2mx0YGDhhwgTNxgn1R2tr619//dXxI41G++CDD5SPY2JikpKSmEzmokWLKBSK8iCHwzl9+nRTU1N4eLhyPiGkW65evdq5wzMyMtLY2DgtLS0lJYXL5VpZWX3wwQcdE4kVCsWVK1cePnzo4OCwYMECHA7X28sN/kQIQRAEQW8xyMcIIQiCIOjtYCKEIAiC9BpMhBAEQZBeg4kQgiAI0mswEUIQBEF6DSZCCIIgSK/BRAhBGhATExMbG6uOM+fn5584cUIkEvWwfWtr64kTJ8rKytQRDATpBJgIIUgDfvjhh927d6vjzMnJycuXLxcKhT1sX19fv3z5cmWBVgjSTzARQhAEQXoNJkII0hZNTU1tbW1vadDa2srlcru85C2bzjQ1NbW2tnb7VEtLy1u2q5VIJPX19bDsFKQnYCKEIA3jcrkzZ840NDRkMBhkMtnJyen8+fMdz8bHxxsZGd26dSs0NNTQ0JBGo0VERPB4vHv37rm6ujIYDCqVunXr1i7nzMrK8vDwYDAYdDo9IiKisbGx46mqqqrx48cbGRnR6fSxY8cWFRV1fuGlS5dGjhyJx+PNzMwoFMr06dNra2vV+teHII2DiRCCNKytrc3Q0PD8+fN5eXmpqakjR4786KOP7t+/r3xWIpG0tLQsW7YsJCTkwYMHx44du3PnzpIlSxYtWrR58+aHDx+uWLFi586dt2/f7nzOjz76aPHixbm5uadPn05KSpo1a5by651EIomIiMjOzr5w4UJOTk5ERMSiRYs6v7C2tnbBggWpqan5+fm//PLLw4cPZ8+ePVD/EhCkIQoIggZcSEhIWFhYt09JpVJ7e/t169Ypf1RmuA0bNnQ0mDFjBgDg2rVrHe2ZTOaKFSuUP/78888AgK1bt3a0Vx5JTU1VKBRXrlwBAFy+fLnj2eXLlwMALl682G0wFy9eBABUVlb2/a8KQVoPo8EcDEGQkkAg+PvvvwsLC1taWgAAEomkpKSkc4PO20g5OTmhUKiwsDDlj2g02sHBoaqqqnP7yMjIzo9XrFjx+PFjX1/frKwsLBY7derUzs8eP36882ufPXsWHR1dW1srFouV8RQXFys3QYWgQQl2jUKQhuXn59vb2//nP/+pra2lUCh0Oh2Hw3WZFEOn0zse4/F4AoFAJBI7juBwOLFY3Lm9ubl5x2MGg4HD4aqrqwEANTU1DAYDjUZ3PGthYdH5hZs3b/b09IyJiZHJZHQ63dDQEADQJRgIGmTgN0II0rBDhw5hsdi8vDwDAwPlkbt37/bznI2NjR25kMPhiMViJpMJADAzM2tublYoFAiCKJ/tvP0pj8fbvXv3119//c033yiPJCUl/frrr/0MBoK0HPxGCEEaVlZW5ujo2JEFKysrs7Oz+3nOznNnlI/d3NyUf4pEooSEhG5bVlZWyuVyLy+vjiM3b97sZyQQpP1gIoQgDRsxYkR6enp0dLRIJMrKypo1axYWi+3nOffv3x8VFSUUCpOSkr744gt3d/fg4GAAwLRp02xtbVesWPHw4UOBQHDu3LlTp051vMrOzs7AwODAgQM1NTVcLvfnn38+efJkPyOBIO0HEyEEadiXX345duzYiIgIAoEwZsyY8PDwoKCgfp7zwIEDixcvJpPJQUFBDAbj6tWrynFBIpF4/fp1AMDo0aMpFMrGjRsPHDjQ8SoSiXTq1KnMzEwWi0Wj0fbt23f06NF+RgJB2g9RwOIREDTg5HI5AACFevlJVKFQlJeXNzY2Ojk5Keen9J9IJMrNzSUQCK6urh0jgh1Xz8nJUSgUw4YNw2C6ThQQCoWFhYV4PN7FxaXLCyFoUIKJEIIgCNJrsGsUgiAI0mswEUIQBEF6DSZCCIIgSK/BRAhBEATpNZgIIQiCIL0GEyEEQRCk1/4fP0DS54KdRjMAAAAASUVORK5CYII=",
"image/svg+xml": [
"\n",
"\n"
],
"text/html": [
"\n",
"\n"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"using Mosek, MosekTools\n",
"\n",
"# solve at a grid of λ\n",
"λgrid = 0:10:150\n",
"β̂conpath = zeros(length(λgrid), size(X, 2)) # each row is β̂ at a λ\n",
"β̂conlasso = Variable(size(X, 2))\n",
"@time for i in 1:length(λgrid)\n",
" λ = λgrid[i]\n",
" # objective\n",
" problem = minimize(0.5sumsquares(y - X * β̂conlasso) + λ * norm(β̂conlasso[2:end], 1))\n",
" problem.constraints += β̂conlasso[2:end] >= 0\n",
" solver = Mosek.Optimizer() # MOSEK this time!\n",
" MOI.set(solver, MOI.RawOptimizerAttribute(\"LOG\"), 0) # keep silent \n",
" solve!(problem, solver)\n",
" β̂conpath[i, :] = β̂conlasso.value\n",
"end\n",
"\n",
"plt = plot(collect(λgrid), β̂conpath, legend=:none)\n",
"xlabel!(plt, \"lambda\")\n",
"ylabel!(plt, \"beta_hat\")\n",
"title!(plt, \"Constrained Lasso\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## QP example 7: robust regression\n",
"\n",
"* The Huber loss function \n",
"\\begin{eqnarray*}\n",
"\t\\phi(r) = \\begin{cases}\n",
"\tr^2 & |r| \\le M \\\\\n",
"\tM(2|r| - M) & |r| > M\n",
"\t\\end{cases}\n",
"\\end{eqnarray*}\n",
"is commonly used in robust statistics. The robust regression problem\n",
"\\begin{eqnarray*}\n",
"\t&\\text{minimize}& \\sum_{i=1}^n \\phi(y_i - \\beta_0 - \\mathbf{x}_i^T \\beta)\n",
"\\end{eqnarray*}\n",
"can be transformed to a QP\n",
"\\begin{eqnarray*}\n",
"\t&\\text{minimize}& \\mathbf{u}^T \\mathbf{u} + 2 M \\mathbf{1}^T \\mathbf{v} \\\\\n",
"\t&\\text{subject to}& - \\mathbf{u} - \\mathbf{v} \\leq \\mathbf{y} - \\mathbf{X} \\beta \\leq \\mathbf{u} + \\mathbf{v} \\\\\n",
"\t& & \\mathbf{0} \\leq \\mathbf{u} \\leq M \\mathbf{1}, \\mathbf{v} \\geq \\mathbf{0}\n",
"\\end{eqnarray*}\n",
"in $\\mathbf{u}, \\mathbf{v} \\in \\mathbb{R}^n$ and $\\beta \\in \\mathbb{R}^p$. Hint: write $|r_i| = (|r_i| \\wedge M) + (|r_i| - M)_+ = u_i + v_i$.\n",
"\n",
"\n",
"\n",
"## QP example 8: SVM\n",
"\n",
"* In two-class classification problems, we are given training data $(\\mathbf{x}_i, y_i)$, $i=1,\\ldots,n$, where $\\mathbf{x}_i \\in \\mathbb{R}^n$ are feature vector and $y_i \\in \\{-1, 1\\}$ are class labels. Support vector machine (SVM) solves the optimization problem\n",
"$$\n",
"\\begin{array}{ll}\n",
"\t&\\text{minimize}& \\sum_{i=1}^n \\left[ 1 - y_i \\left( \\beta_0 + \\sum_{j=1}^p x_{ij} \\beta_j \\right) \\right]_+ + \\lambda \\|\\beta\\|_2^2,\n",
"\\end{array}\n",
"$$\n",
"where $\\lambda \\ge 0$ is a tuning parameters. This is a QP (why?)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"---"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
" 1.881291 seconds (2.51 M allocations: 167.199 MiB, 4.83% gc time, 93.43% compilation time)\n"
]
},
{
"data": {
"image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAGQCAIAAAD9V4nPAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdeZRcV30v+t/e+8w1d/XcrW6pbcm2JI+ysbGFjUEOeHxOghmMR5xAyEAuKwncxECcGPKAtQL3ER55ISEYbDCXIRcjYSfGBjvI4MSWMbIkbNmW1KN6qq75zHvv98epLpVas9yq7lb/PktL69Sp01WnanXXt36/vc85REoJCCGE0HJFF3oHEEIIoYWkLPQOIIQOLwiC3bt3FwqFVCrV0dHR1ta20HuE0OkJK0KEFp1SqfSRj3ykpaVl/fr1GzduPPfcc9vb21euXHn33Xf7vg8AH/vYxwghH/7wh4/0CDt27CCEtLa2ep4HAO9973sJIYSQj370o4dunMvlDMOINtixY8epe10ILU4YhAgtLr7vb9q06R/+4R8Mw/ijP/qjL3zhC/fdd9/73/9+13UfeOCBKNjuueceQsjDDz/sOM5hH+TrX/86ALz//e/Xdb2+khDyrW99K4rSRt/61rc8zyOEnLLXhNCihq1RhBaXb3zjG88999w555zzi1/8Ip1O19cHQfDYY49pmgYAZ5111qWXXvrss88+8sgj733ve+c8QhiG3/72twHg7rvvblx/xRVXbN269dFHH7355psb13/961/PZrOtra2vvPLKqXpVCC1iWBEitLhs3boVAO6+++7GFAQAVVVvuummeoUXhdw3vvGNQx/hxz/+8fj4+EUXXXTBBRc0rr/99ts1TZvzIy+99NKLL754++23N9aOCC0rGIQILS6qqgLAoQ3MOd773vdalvX4448PDw/PueuBBx6AQ8pBAMhms9ddd92WLVsmJibqK7/2ta8BwJ133vmGdxyhpQqDEKHFJSrjvvjFL27evJlzfqTNksnku971LiHEQw891Lg+l8s9+uijmqYd2jIFgDvvvDMMw4cffji6GQTBt7/97XPPPXdO7YjQsoJBiNDictddd11wwQW5XO6mm27KZrPXX3/93/7t327btu3Qc19ENd8DDzzQeNeDDz7o+/7NN9/c2tp66IPfcMMNHR0d//qv/xrd3LJly9TU1D333HPKXg1CSwAGIUKLSzwef/bZZz/96U+vXr26WCw++uijf/3Xf33xxRevXr368ccfb9zyqquuOuOMM3bv3v3ss8/WVx6pLxpRFOV973tfNC4Ybayq6q233noKXw9Cix4GIUKLjq7r99577+7du3fv3v3ggw9+4AMfaGlpef3116+//vqf//zn9c0IIdHYXhR+APDCCy/8+te/7unpueaaa4704FH9941vfGNycvKxxx674YYb8FB9tMxhECK0eK1evfq222772te+9uqrr1522WVhGH7mM59p3ODuu+9mjH3nO9+xbRtmE/Guu+5ijB3pMdevX3/hhRc++OCDX/va14IguOuuu07ta0Bo0cMgRGgJaGlp+eQnPwkA27Zta1zf29v79re/vVQq/fCHP/R9/+GHHyaEHDPb7rrrrlwud//997e3t1977bWnbrcRWhIwCBFaGgzDAABK5/7N1qfMbN68eXp6+sorrzzzzDOP/lDRGWccx7ntttuiozUQWs7wzDIILS4//vGPL7jggp6enjnrowP+NmzYMGf9zTff3NLS8uSTT+bzeQA4nlZnNpt96qmnKpXKeeedNz87jdBShkGI0OLygx/84Ld/+7ff+c53XnPNNWvWrFEUZd++fQ899NBTTz2ladonPvGJOdsbhnHrrbd++ctffv755+Px+Lve9a7jeZbLLrvsFOw7QksSBiFCi8umTZu2b9++ZcuWzZs3N65fu3btl770pcsvv/zQH7n77ru//OUvA8B73vOeeDzepB1F6HRB8Ar1CC1CExMTO3fuHBkZsW07nU6vX79+/fr1R9l+7969Usq2trZEInHovZOTk5VKpaOjIxaLHekRRkZGfN/v7e2NzuuN0PKBQYgQQmhZw1mjCCGEljUMQoQQQssaBiFCCKFlDYMQIYTQsoZBiBBCaFnDIEQIIbSsYRAihBBa1jAIEUIILWsYhAghhJY1DEKEEELL2tIOwlwuF51r+Djh+eSaTAix0Luw7OB73mT4qdJ88/5LvrSDcN++fQ888MBxbsw5dxznVO4OmqtarS70Liw7tm0v9C4sL7ZtYxY22bx/sCztIEQIIYTeIAxChBBCyxoGIUIIoWUNgxAhhNCyhkGIEEJoWcMgRAghtKxhECKEEFrWlIXeAYQQQuiAagg+h6IvuYSCD6GAcgCeADuUdggehzSh7z5rPp8RgxAhhNDJK/jAJRR9GQioBOBycEKohtIXUPSBSyh4EEooB9LjYIcQhVk5kKGEghf9LAQCKqGMfjamgMYgqRJGIKMDI5DUQKMQU4ipgMFgtTXPvUwMQoQQWo7yHjhc2iEUfaiG4IRQCmQlADuESgClQDohVMPZMPNlrTKrhZn0BJQDCAWkNFAIpDSiUoirYDAwFYgpRKOQ1KAeZhmNaAxiCkRhFleoSiGlgUIhpYFKIa6Q6GePqVwO5/etwCBECKGlp+iDw8EOZcGLFqDgSzsEpxZs0g6hHEA5ACeESihLfq0aK/jS4eCEkNbAUoipRAtgMkhpJK6CqUBChYRKsjrEVUiqwChkNMooJFXQKMRUMBkxGMRVUE+LeSYYhAghtACEhKIPeV8WfSj5UPRlKQA7rBVqTgiF2eiKCjWnFmzSDqEaQlKtFV6pKMYUyGjEVMBkkNYhppBOC9aoEFfBZJBQaUIFS4GYCimNmAws/OxvgG8GQgi9UYemWjGA+nLBh0IUdT4UfSgGUPJlJYC0DmmNpDRIqpDSapGW1sBUoEUnZyTBVMBikNJoTAWTQVKDuEIsBeLqQr/g0wsGIUIIzeWEkPdl3oO8B3m/NokjWq6vzHvgcqj6ajEMJx1QKWR0yGgko9eGwaLljA7dVrRAoqLNYJDRocMkjCz060QAgEGIEFoOfAHTrsy5MO3BlCOnXcj7UDpcrVb0ZfVwtVp9uTsDKQ1SGk2qkNJADb2OlJXSMNOWMAxChNASJiTkvHrIyWkXJh3IeXLahZwrpz2YdmHKkQ6HrA6tBska0GqQdgMyeq392JhqKQ1SGkmcSOOxWgULG5VLHAYhQmiRqvcn99swZsvGzuR+R45VIe/LKRcUUms/dlkHupF9MdIdI11m7Sb2IdFRYBAihJptzgjcgZzzDiTfSFVScmDUrZZzOgwkSKYVui0a3WwzyOkxgx8tIAxChNA88wWMVeVIFYarctSGkaqccmDalZMuRA1MAGjVSdaAduNAu3JNCtoMaDNpqw7RGg0TDjUFBiFC6GS4HEarctSGoYocrcKoLYcq0RqZc6HTIitisCJOui1YFSeXtELWoG0GtBnQahA8iA0tKvj7iBA6Io/DqC3HqrDfkXtKMGbX2pjR/1HHciBBuizoschbOqHLpN0x6I/jgBxaSjAIEVruXA5jh6TdnrKMxu0a067bIhtn025lnFBMO3RawCBEaFlwwsNUdfW0i6aidFtkIAkDCbKxEwYStMuCThPTDp3+MAgROn3YIewokFxejlTlWFUOVmDMliNVGKpIRmBFnPTGoMciK+Jwbgt5Zy9ZEYfeGElrC73fCC0oDEKElqpyAC8X5M683FWQu/JyVwEmHLkypq5K8u4Y6Y2Rq7qgJ0Z7Y7AidmIHiSO0rGAQIrQ0FH14rVSLvZ15uSsPY7YcSJB1GbI2A3euoWvT5Ow0caqVeDy+0DuL0FKCQYjQYlTw4fUjx94dq+naNDknjQN4CM0DDEKEFl7egz3lg2Iv58kzk2RtmqzLkA+eTdZlCM7SROgUwSBEqNnyHhzIvILcmZdOCGccHHurEgRTD6HmwCBE6NSaE3s7ZqQnYCBRi71NPRh7CC0wDEKE5tOYLXflD/Q5t8/IQECUeWvT5MY+ui5DuqyF3kuEUAMMQoROXhR79YJv+4xU6EGxt76FdJoLvZcIoaPCIEToxAxV5OOj8vER+eSYSKjk7DSsz5A3tZG719Bz0iSFB6cjtNRgECJ0bHYIv5iQT4yJJ0blUEW+tYtu6iFfuEzpjeHQHkJLHgYhQke0pyw3D8otw+KXE/LsNLmhj/zTRnZhFg9jQOi0gkGI0EEmHXh6XDwxKn88LHUKm3rIB8+mP9hEk3iKMoROUxiECIHLYet4rfP5alFe2UVu7KN/dQHtj2Pph9DpD4MQLV97yvKJUfnEqPzJqDgjSTb1kM9ewq7qIipd6D1DCDURBiFaXqZd+Nl+8cSofGxYqhQ29ZBbBshX36LipYgQWrYwCNHpLxTw6xn5xKjcPCS2z8hL28mmbvrv19K1aex8IoQwCNHpq975fGJMDCTIph5y30Xsyi6iYecTIdQAgxCdVioB/Gy/2DIkHx+Vbijf0klv6CP/30a1RV/oPUMILVYYhGjJ4xJezNUqv+em5CVtZFM3/f7b6YZW7HwihI4NgxAtVY2dzy6T3NhPPn4ee0sn0dlC7xlCaEnBIERLSTWEX86e6my4Kq/qpJt6yP96s9JtYfGHEDpJGIRoCXgxJ388LP9jRLyYk29uJ7/VS79+JT23BcMPITQPMAjRIuULeHq//NGg2DwkNQY39ZFPXMje0kFM/J1FCM0r/FBBi0vegyfGxOZB+eNhsSpBbugjP9jEcNoLQujUwSBEi8K+snx8VG4eEj8fl2/pJDf20c+9ScUruSOEmgCDEC0YIeFXObl5SGwZkkMV+c5eesdq+vDVNI7XeUAINREGIWo2J4RnJuTmIfH9vdJgcEMf+V+Xscs78CJ/CKGFgUGImmTahUeHxZYh+ZNRsS5DbuyjP72enpXC9EMILTAMQnRq1S/y/tyUfEsnuWUVxUs9IIQWFQxCNP+4hF9OyC3D4v/s1T3B39FLPrKOvqOX4tmuEUKLEAYhmjd2CE+OiS1D8pFBkdXJLQPkXy4LNvbFsPuJEFrMMAjRGzVUkf8+Ujvy4ZI2csMK+qkLlZ4YAYByWWAKIoQWOQxCdJJ25uWWIbl5SLxSlFd30VtW0W9dTZN45ANCaKnBIEQnIBTw7KT83l7xf/ZJlcINfeS+i9hbu4iCg38IoSULgxAd24wHT46JaPLnQILc0Ef+/Vq2No1dT4TQ6WAZBWHeg2fH6WXd0Gos9K4sEa8W5SNDcvOg+PWM3NRDb+ojX3yzmsVLvSOETi/LKAhHbPj8DrJ9a5DSyIZWclGWXNRKLmolneZC79liMmbLl2bgp2PiR0OyEsANfeQvL2BXd+HVbhFCp61lFITnZuDfN3HT0l8vyRem5Qs5+cUd4oVpaTByUSvUo7F3Oc32rwSwMy+3z8iX8nLHjNw+IxmFczPkig7y0FvZRa1kGb0XCKHlahkFYYQAnJkkZybJuwdqa8ZsuW1abpuW//yKeH6rdDmsy5ANrbV/azOnTxhwCYMVuTMvd+VhZ15um5Z7ynIgQTa0knUZck0PWZchA4nT5uUihNBxWXZBeKhui3T3kRv7ajfrubhlSH7u12LUlusbcvGc9FI6N3TeqwXeroLcmZe/ysmUBlHsbeohHz+fnp0mbOm8HIQQOhUwCOeak4v1LHliVH7u12KkKs9tOZCLiypIfAGvFg/E3rZp6YSwLkPWZcjaNLn9THpBluAVjhBCaA4MwmPI6LCxk2zsrMVd0YeXZg7k4nBVrkmRtelaLl7S1tRJJVHxOqfPuS5D1mbgg2dT7HMihNDxwCA8MSntoFwsBbA9V2ulfvVlUR9yi/5d3EaM+cvFqDatV3sv5mRSg6ja29RD/nQ9XZ/BuZ0IIXTCMAjfkKR6UC5WAngxV2tOfm+veDEn++MHcnFDKzGP+/2e0+fclYcZT9b7nLesoudnSQL7nAgh9IZhEM6n+MG5GAjYXazVi4fm4oWtJNbw9h+zz7kqcfrMX0UIocVjYYLw2Wefve+++6anp3/rt37rvvvu07S512m95557yuVytPzmN7/5ox/9aNP3cR6otDZX5Y7VAAAeh+0z8oWcfGFaPvia+E1BrkqQc9JksCJ35WW7Sc5tIeszcH0f+Z8X0DVJPIEnQgg1wwIEYS6Xu/baaz//+c9fdtllf/iHf3jffff93d/93ZxtHnnkkU996lNdXV0AsGLFiubv5KmgM7ikjVzSdqBe3JGXr5Vkb4yuyxC8bgNCCC2IBQjCBx98cMOGDb//+78PAJ///OdvvPHGv/mbv1HVuTlw7bXXrl69uvm71zQqhQuz5MIs9jsRQmghLUD3bfv27W9605ui5UsuuWRmZmZ0dPTQzf74j//4pptu+vSnP12pVJq7gwghhJaRBagIJycnzz777NrTK0oikZiYmFi5cmXjNn/xF39x3nnneZ73hS984fHHH3/qqacoPUxmT09P79ixY9WqVfU1f/mXf3nrrbce9nk5557nCSHm7ZWgY6lWqwSn+DRXtVpd6F1YXmzbFkLg73kzndAHi2EYinKMpFuAIEylUrZtR8tSStu20+n0nG0+/vGPRwtve9vb2tvbd+zYcd555x36UK2trWvWrPnhD39YX9PV1WWah7+cBOdcVVXLsubhNaDjI6WMx+MLvRfLDr7nzUQIsSwLg7CZ5v2DZQGCcOXKlbt3746W9+zZQwjp6ek50sbJZDIWixWLxSNtoGnawMDAke5FCCGEjm4Bxghvu+22Rx99dM+ePQDwpS996eabb46y/aGHHnryyScBYHR0dGhoCACklF/84hcJIeeff37z9xMhhNBysAAV4TnnnHPvvfdu2LAhmUy2tLTUG5s//OEPzzvvvLe//e2Dg4PXX3+9ruue53V1dX3/+99PJpPN30+EEELLAZFSLsgTu66bz+ejIwUPJYSYmpoyDCOVSh3lQbZt2/ahD33o+eefP55njCbL4BhhM5XL5UQisdB7sbxUKhUcI2ymarWKY4RNNu8fLAt2ijXDMI6UggBAKe3o6Gjm/iCEEFqe8CxeCCGEljUMQoQQQssaBiFCCKFlDYMQIYTQsoZBiBBCaFnDIEQIIbSsLaMgfHbGvuW54e8N5z2xMIdOIoQQWoSWURBuSJu/25366p7p3s3b//iFoedn7IXeI4QQQgtvwQ6obz6Vkvf2JD+wunPUCR4azN367B6FkjtXZu9cme008PLwCCG0TC2jirCux1Q/fnbn7uvWP3jpqj0V75zHdl7z9Kvf3JdzOF6qECGElp3lGIR1GzLWP13cv/+m8z440Prg4Ezv5pc+9Pzg1unKQu8XQgih5llGrdEjMRi9ZUXmlhWZEcf/1uDMXf+9T6fkjpXZu1ZmO7BlihBCp7tlXRHO0WtqHz+787Xr1n/z0lV7Kt5Zj+285ulXvzec93GWKUIInb4wCA+jsWX61T3TXT/a/qHnB1/I4yxThBA6DWFr9IjM2ZbpsO1/e2jmll/sMRi5Y2X27lWt7Tq+bwghdJrAivDYVljax8/ufPW69f90cX80y/TGra99bzgfYMsUIYSWPgzC40UJbGyN/9PF/XuvX39Lb6beMn2xgC1ThBBawrDFd8KSKrtjZfaOldndZffbQzO//czrFqN3rMzes6q1FVumCCG01GBFePLWJIz71nW/ft25Uct0zWM7opZpKLFlihBCSwYG4RvV0DI9N2qZ9m956U9/NfzrgrPQu4YQQujYsJU3b1KzLdNXyu7DQzP/19bXWnXl9pXZ2/pbshq+zwghtEhhRTj/zkoY963r3nP9uZ89r2fbjL360R3v/uWezWNFbJkihNAihJXKqUIJbOpIbupIFgL+3eH8514e//C2wdv6sx9YlV2TMBZ67xBCCNVgRXjKpVX2wYHWrW876ydXrQGAq362++Kf/Ob/eXVyxg8XetcQQghhEDbROUnjs+f1DN947l+v6/7PqfIZP95xx3/te3y8lMNERAihhbOMWqOVIXfoJxOplfFEn5VYaSoGW5DdUAi5sTt1Y3dq2gu/PTTzN7v27yo6lJAz4/rqhL4mYayO62fG9dUJI60uzB4ihNCysoyCUOvStAtNMSlHfjpVGXb0tJpYaSX6rWS/aXUaQJq9P6268pHV7R9Z3Q4AeZ/vqXp7Kt6eqvfUZPn/fW3qpaLDCAzE9IG4PhDT1yaNdSlzdVxPYjoihNC8WkZB+HypcOPI81d3tN1xQd+1HX1iOqyMOKW99tjT014xiHUZ8V4zOWClzoip8Wa/LRmNbdCsDRmrcWU9HXeWnC37i196dXJ32VNoLR3XJo11SXMgrp+VMOIKtrgRQugkLaMgvDzb8vqmt/60WPrXvYO///yvruvqvKN/xdsv7iYA3BXlIbu01558rvDa98aoQpKrYslVVnyFmegzCWt6tQgADel4C2TqK6N03Fl0dpXc743k91S8l8uuRslATF+bNNeljCgmz04YMUxHhBA6DssoCAHAZOyW3p5bentGHef7I2Mff2nn1PMv3Nq34u6V/WetiafXxKPN3Jxf2mtXhp3J5wvOlBfrNuK9ZnyFmTojpmcW+Jr1R6odd5acXSV3T8X7Xj6/p+LtKrkGI2uT5rqkETVXoyLSZJiOCCF0ECKX8lHe27Zt+9CHPvT8888fz8acc8/zLOugCNlZKj04OPzNfUMZTb2jv++ulf0dhn7QT3miOuaW9lZLe+zykEMoRB3U5KpYfIVJlYUpFo/HmBPsKjl7qn409Liz6Oyp+t2GOhDXB2JafejxrKShkFP1KsrlciKROEUPjg6rUqnE4/GF3otlpFqtWpZFTtkfETrUvH+wLK+K8FDrksnPnrvu79av/UVu5sHBobP+/SeXtGRu71/xuz09MYUBANNpcpWVXGXB2wAA/FIYheK+LeOVUddq16MOarzXtDr1YzxZc3Wbard5UP0aCLm36r1a8XaX3VfL3hMT5dcq7rgb9lva6oS+Om6sTuir4/rqhLHC1Cj+XSOEloflXhHO4XC+Zf/4N/cN/TI3c11X5y29Pdd1dbAjfNfjvqiOupVhp7TXLu6pEgJRBzUKTqoujSakJ+TrFW932X2t4r1adl+teK+WvUrIL2+NX94a29gav6QlZp1sQxUrwubDirDJsCJsvnn/YMEgPLz9rvvd4dHvjYwOVu3f6e2+e2X/BenU0X/EL4VRKJb2Vqujrp5RkwOx5Cor3mtaHXrzD894Iybc4L9n7G356jPT1V/mKv2WtrE1fkVr/Mq2xMqYdvyPg0HYfBiETYZB2HwYhAc5dUFYt6tU/u7wyDcHh0zG7ujvu3NlX6dx7DOFSi6r+93SHrsy4pT22DwQiRVm1EFNDliKuZSOBQyE3F50tk5XnpmuPDVZVgjZ2Ba/ojW+IWNd2hJTj9pCxSBsPgzCJsMgbD4MwoM0IQgjQspoEPG7w6MXt2Ru71/xOz3dceV4R1gPKhbHXD2tznZQY0uuWNxT9bZOVbbl7WemK7vL3nlpMyoW39IWP/RUOBiEzYdB2GQYhM2HQXiQpgVhncv5TyamHhwc+o+JiXd0dNze33dtV8cJzbqUXFZGnPKgUx60S/ts7gmzXTezmtGqGa1atND8I/pPTing/z1T3TpdeWa6+myu0jfbQd3YFh+I6YBBuBAwCJsMg7D5MAgP0vwgrJvx/e+PjH1zcGhf1f7d3u47+/suyqRP4nGCSuhM+e607+R8d9p3pz0n58tQGq2a2aoZWd1o1YysZrZqelpdzLVjKOWvC7UO6tOTZUbIhhbrkoT6tp7sm1piOAm1aTAImwyDsPkwCA+ygEFY93K5/J2hkYeGhnVKb+ntuWtl/8rYG30KEUp32rcnXDfnu7nAzfluzveLgZZSjaxmZDUjW1sw23WmLcbpqWNO8Mx05adjM8+V/MYO6hWtsRZtadS7SxQGYZNhEDYfBuFBFkMQRqJBxO+NjH5naGR1In5Hf9/7+noTxz2IeDwkl16hFor1dLQnPKqQKBStTt3q1I2sZrRqC3VtjTmi39dyyP8rd1AHdUMmtrEtfkU2ti5lLvQ+nm4wCJsMg7D58ID6RYoSsrE1u7E1+/nz1j8+Pvng4NBfbH8pGkR8Z2e7SuehaCOsFnhz1vul0B6v1Y7TL5bcnO9MeYSS2drxQPlotGgL0lxNKGxTR3JTRxIAQilfKbnP5Kpbpyp/t2u/L+SGFisqFi9pienYQUUINR0G4TzTKb2xu/PG7s68H3xvZPRzr+z+wPPbfqen+/b+vo2t2VPxjFpS0ZJzK4DQ4bO1o18Zcad/XWpsrlods7VjVtMzKmli/CiErEuZ61LmBwdaYbaDunW68j9+NdzYQb28NZbFDipCqCmwNXrKDdr2d4ZG/mXvPo3SW3p77lzZv+oNDyKenFo6TvvOtB8tuDk/qIR6SzQxR5udoaMZWW1errlxQh2Mcsh/XXCiXNw6VWnRlGgC6hXZ2NqUiaXiccLWaJNha7T5cIzwIEsiCOu25QvfHBz638MjZ8bjd/T3vac9m6BE+g5LtcLC/RVFE3PcnO9Me/WY9AuBllbNNt1q18w23WjTzTZNT5/wlTdO+vc1lPLFvPNMrvKL6coz0xUh4bJsbF3KXJs0zk4YZ+NlNI4Mg7DJMAibD8cIF5hwKsC58GzpezIMpFuVnEvXloEnA1+4NvBQuLYMfRn40rUlD4VThTCQgdfjOX/Jw496zhOxzu/v7v+LZM/bymO/XRp9U24w09KudvQqHX1qR5/SsUJp6yFKk673RBUSzbIBOPCLJYX08rPzcca96e21zqqR1axOIxp3tDqNWJfBjFMSSAohF7dYF7dYf7q6HQAGbf+/ctXflNxHRgufK7u7y16noZ6dNKJcXJs0zk4aOBkVIXRyltNnh+B8/15fVaVblYEvA194NoShcKsyDKTvSc+RPBROBXgoPFf6LvBQ2BXJQ+k70nMlD6kRA0WhukVUjagaMSzCFGJEN3VqWEAZteJEUYmqE8MijFEzDkwhmkF1ExijZuIeRfk9zZjx/e8Ojz4wMvrHM/k+TbmYiQ1u8cJfP3vG/gdFbpyl29SOFUpHn9KxQu3sU9p7qexMmxsAACAASURBVBFr2ltVn2tz0PvXcFBHaa89/st8fVbOgQmrWc3qNOb96lT9ltZvHbQz0UWmdpbc52aq/3s4v71gB1LWr04c/b8qpuO3dITQMR2+NfrpT3/69ttv7+/vb1w5NDT0la985bOf/Wyz9u3YTqg16r6+I/+DrzBFIUaMKCrRdKqbwBRqxomiEM0gukkoo1YCGKO6STSj4V6TaPopqtK4lC+Xy9vyhWemc1uncyOOc24qeaGhXibcy6rTmanBcHw4mBymhqV09KkdK5TOPrW9V+noZ8nMsR/9FAsdbo979rgbHdERzV+tH+/I0pDpTzZnwmrj1Yl3lpxdRXfMDQZiWtRQXZc0o8LxSNcSOW1ga7TJsDXafE0aI+zs7Py3f/u3yy+/vHHlL3/5y8svv3xRjSkurTHC41QKwu3FYhSKz87MqIRuyKQ3tmbfHLfOd/PK5HAwPhiMD/HcuLDLSmsXy3aqnf1qZ7/S2ae2r4D5OFTjjagf72iPe8WRCi/Jw7dVuw2mn9pdLQb8tYpXy8WSW7808dpULRfXpcyzE0ZMOa2GGzEImwyDsPkWcoxwZmYmlTrGpYjQG5dUleiQxI8DAMCY4z6Ty22dzv3V/vEXC4V+y9rYf9kVG67fkEmfo9Jwen84PhiMD9kv/mc4Phjmxlkyq3T2qZ39amef0tmvdvQRralXDK4f75heE0+Utej3tfFwjsa2qtWpW51GdKSj1WGY7do8HsuRUtmGjLUhY90CtdLZF3LE8XcW3V0l54mJ8pdenfxNyU2prFY1psyBmHZuyuwwmjRAixBaDA4Kwq1bt27ZsgUAKpXKP/7jP/7oRz+q3+X7/ubNm88///xm7+Cy120at/T23NLbAwDVkP+qUNiWLzwxMfXXO3/jcnFxJr0h07Nhw/q3tGWzqip5GE6NhuNDYW6/u/tX4X8+EowPUjOudParnX1qZ7/S2qV2r6Lxkzkt6huhmCzea8Z7DzqPTGNbdXJPwZ7wDnuk46HnEDhpGiUDMX0gpt/YfeArXX24cdtM9Zv7ci8VHUZgbdJclzQG4joONyJ02jsoCF966aWvfvWrAOA4ziOPPKI0nCHMMIz169f//d//fbN3EDWIKSwqFqObY467LV/Yls9/6bXXb/2v5/osc0MmsyGT3th/7oXnb6RRr0bwMD/Jc+PB/kF/5LXqc0+G+/cCpUpHv9rVr2Q7o8JRaels/iEcismSq6zkqgPNahFKZ9Jzpnxnyivtsyf+u+BMeSDBjI7iaNH0FlXPaEaLqqfVeTnSEQC6TbXbVKMT30Tqw407i84TE+U9FS/nh2fG9YG4vqyGGxFaJk5gjHAROi3HCE9OKOUr5fIz0zNbp3Pb8vkRxzk3ldrYmr0im70sm2nTD+qO8sJUMDEcTgwHE0PhxHAwPgQ8VDp61Y5+pWOF2rGCtXTQWIrFk0Df0DlL56WVH1S5M+m5074747szvpcPvJnALwVqXNFbNKNF1Vs0I6PWlucvIBvN+OHLJXdXyX257O4quS+X3HE3WJPQ+yyty1C7TLXDULsNtd1Qe0213VAX8FxxOEbYZDhG2Hx4QP1BMAiPpBgEz83kt07ntuULv8jN6JRubM1e0ZrdkElf0pLRD5lQI+xyODFUS8fxIZ6fFNUSrxSolaDxNIslaSLNEmkaT9NYksXTNJGh8SSLp2gsdZRS8pRej7Bx3LF+FnIvHzCdNp5nVUuqWkqx2nU6r5fpcLh4ueSOOMF+N9jvBBNuMOoEk14w6gSTbhBXWKehdppKl6F2GGqPqbbrao+pdhhql6keegXjeYRB2GQYhM3X7MkyYRiWy+XGNZnMwk/ZR8eUUtVNHe2bOtqjm3uq1SgUHxwc2lkqr0smolB8S2trdL43aiW0Veu0VevmPI4MfF7K8eKMdMrCrvDSTDg16g+9IuyKKM3wYo5XCoQpLJmlyRZqxakVZ8ksS7ZQK85SWc50LjpYInMq5rIedtwRDjnPqpsr+cXAnQnql+loDMiTvo6VyeiFGevCI/w1OFzsd4MxJ9jvBGNusN8JfpV38kG43wnGnCAf8IzKuk21y1QzqhItdBtql6l2m+oKU1Px5OMINdHhg9D3/fvvv//hhx8eHBwMw7DxriVdQS5bA7HYQCx2R38fAFTC8MVCcVu+sGVs/GPbd1AgGzLp6AiNy1uzFjuoWCGqpmS7lGzXUR5c2BVeygm7Ip0yL87w0kyY2y9GKrw4ExamHNfm5Tw1YyzZQsxEFJAs2ULNeLRMky3UjLNky3wNUp5YQOYDyuYzICMmo9GsnMPe6wmZ88IoKfN+uN8N9lS8Z6YrUXAOO35coV2GmtGUblPtMtTG/3ss7ZQWlAgtQ4cPwo997GNf+cpX/uAP/uC5555raWm55JJLHn300ZdffvmTn/xkk/cPzbu4Ujs8409XnwEAe6rVZ3Mz/zWTv3fHrh3F0tpk8tJs5uxEol3XOw29TdfbDT2rHW3eZlQIHvauqIMheSgqRVEp8nJeVIqiWuSVYjC2N1qI1gMPaSxJ42mWzNQW4imaSLN4isbSNJFiiQzRjDfywt9IQGpJRUspRlYz2/Q3fvijTkk0Q2fDEQrKvM/HHD8f8HpBubPo7neDMccfcwJXyHpBOVtHavWkjOFXVYRO0OHHCDOZzL333vvnf/7nd999d3d392c+8xkp5Yc//OGhoaFHH320+Xt5JCc0Rlga3T72y69l11ydPvNK1Wo51fu2FLmc/6pQfDY383q1Ou66k6435fnjrlvlvE3X2nW9yzDadL1N17tMvbZgGNFdh73m4vG38mXgi2qJl/OiUhCVIq+WRDkvygVeLYlKkZdnRKUIhLJUlsXTLJWlyQxLtrBkC01kWCobjV/O53shwSsG0QlXvXzgzvjeTODmfb8QKBYzWjS9RdMzqpZUFIupFlNiihpjakw5RSdfbVQJxajjT7rhqONPeOF+J9jvBhNuMOYEE24w5YVpjWVUJa2xtMoympJWWVpjGY2lVSWtRgssrdWW8fTlbxCOETZfM8YIp6enC4XCO9/5TgBQFMW2bQAghHziE59YsWLF6OhoT0/PPO5B05jZgXjfpfndT+77j08b2VWZ1Vdnzrwq1rUWFuRitYuSwdibsy1vzs79luAJkfP8fODn/WC/6445bt73d5crY46bD4L9jjvsOIyQbsPoMo2MqmY0rds0ugzDFHwg7XSZRkbVukzjKG80UTWWbmXp1qPsngx8YZd5aUaUclEP1tu7ixdz9dFKalgs2UKTWZZqiYYqa61XK65k2ok+txY8GgJ6WtXTauPRHQANARlF47QfVHloh0GVh1UeVEMRysZoVCymxpgSU1SLKTGmxurByU761AFxhZ6VMM46wudApVJhppX3ed4P8wFvWAh3l928z/NBWFvp83zAXS4MRjMqy2gsoykZjWVUJaPN3lTrK1lGU1p1RcPxS3TaOUwQGoYBAJxzAOjs7Ny1a1e0PplMAsD4+PgSDUKqmi3rb+59061S8srIi/ndP3v9x5/0i2PpM6/MrL46fcZGpp+q+Y1LnU5pt2l0m0frTDqcz2ZkFJbOnmp1uFyZHBvP+37eD/JBkFHVjKZ2m2aXYWQ0dTY4a6nZbujKUb9WE1VjqSxLZQFWH3aD2mhllIulmTC339u7M5rpE+anQPBoUg9LtUQZGU3qoVFkHucg5ZECskHocL8Yhg4PbV7/v1oIQof7pdAvBqHDgyonBBSLKSaL/tdSqpZU6jfr/2vJEz4zvsmoadJu83jPj+Nwcdjg3FPx5gTntB8yQg6bmiajs4F6IDg7DZz3g5aAw/yNxePx3t7eXbt2nX/++ZdeeukXv/jFn//85xdddNH999+vKMrAwEDz93J+EcISKzYkVmzoA/AKI4U9z0xt/+FrP/qfsc5zMmvell51eaxr/ULv49JjMhZNyWlc2djBsDmfcL1x153yvEnPG3e9Mdd9sVDc77pTnjfl+TO+36brbbrWYRgdem14sjZOqeudhp7V9JhytHkitdHKzv7D3lufAVtPymB8sD79VTgVatYm79Sn88zebFEy7cd/SKViMsU89sYiEKEjopj0S0GUnW7OD0cOJKhfCkUg5kRjPSC1pFq/qcZPvsQ8oeAsBrwQ8LwfFvxogReCsODzUSco+LwQhHmfFwIeLbtcpjWWUllaZTGFmowmVRZXqMVoXGFJlZmMxBSWUpnFqMlIRlNMRk1G0qoSUyhWn6g5Dv9l85577nnhhRfe9773XXfddRdeeOGVV14Zrf+rv/qr0+zwCT3d23HRezoueo8I3fLwC/ndP335u39EmZJadXlm9dWpMzZSNm/n92rkhu54dWq8OjFtz2hMi2tWTI3FVDOmxuJaLKbG6Gk35GAxtipmRUdrHFYo5ZTnTXneuOtNutGC+0q5POX5E6476XnTni8BWjUtq2tZTWvT9VZdy2padnZNq6616XpW0+LKYX6xjz4DVgY+L+ZEeYaX8ryYE+VCMPq6+8o2XpwR5RlRLdNYkiYzLJlliTRLZaOxSWrGiBGjZpyaseiaXMf/hlCVaio9ZsEnfBHYPKzywA7DKg+qPKyGgc3daT+wZ7uyFR56POq7Eh00U1VMxnRKNco0qliUaZTqlGlUMdmBZYsxjZ7o+QdSKkupbM5VsY4kELIQ8GLAiwGvhsLmohzwcsgdLishLwZ8yhN2KAoBd7iwuSj43ObC4SLanktIqjSuMIvRuEKTKjMZjSl0NjhpVImajB4haCme/Qcdj2MfUO/7/uOPP753796LLrroiiuuaM5uHadTdEC9mx/Ov/rT/O6fVca2x7vPy6y5Onv2b2nJox1CcCQVvzpenZyoTo1XJ8arUxPVyfHK5Hh10g3dznhHR6ytzcz6PKgG1Upg24Fd8avVwK74VVM16tEYU62YZsVrGTkbmVosrlqWasW12npGFt2s+nkf03Y4z/tBNFoZLex33DHXzftB3vfzQRC1YcddN62pXYaR0bRozLKxE5vR1IymZlSt09CP9wuHELxSEKU8L+V4Oc8LOVEp8OK0cKrStYVdEW5VujZQSo0YMSxqxIgZo2asdtOMVlrUiFErXtvAsKhhvcGpsHVSyLDKA5uXZyo600OHc09wTwhfhI7gPj+w7AnhC+6J0OHcFwBQD0WqUabXw5IwjSkmrQWqThWTRclaX573C0/OEUpZDkQ55A4XlVAUo7w8QnBWQuHUglY4XERBqxBycHAqJiMWoym11su1FKpTElOoRmlcoSohCZUpBJIqY4SkVEYJpFVGgGS0w/994WSZ5sMzyxzkVJ9ZJnSKxb2/KO79xczunypmqmX121IDlyf730To3G/xZb8y7czMOPmxyvhYeXysMr6/MjFWGfe432q2dMU7uuOd3YnOrJFptbJd8Y7OWMfRP4I97pf9is99j/sVv1L2K2WvUvarZb8yd71fKfuVklcmhOpM05ia0OLRv/jsgq5oGtUSWiyhH1if1OLaqal3D7wtp/LMMkfhcj5zcF42TvM5el5m1AOjmCecl7MzeoRTiQ6sFHZF2BUZ+o3r63fJ0BdOlShqdFRldJxldM1nasaplYjWUytOVB0UjVpxasZZIn2UPu2JnllGclkLxSgg/YOXbcF9IbzZ9fVluyFEo4DUCdOZYtLZsGxYNijTGaGgGAwoKCYDAorJCCWn+lJcnpB2KApBaIfC4bIQhDYXDpfFgNuh8ISohsIXshLyQMhyKEIpSwHnEooBF1IWAi4l5INQSigEnACkNUaBpFTGCCRVplBiETBUJa5QjZIYozqjFqMGI1GpajBqsXrWkrjCVEoSClUoSSoHshadkKYGoeu64+PjQojGlYtqjLBpp1iTktvjv8nt/un0K0/6xdGw86x8S+++VNtIYI9XJ8arkxrTOmPtHbH2zlhbZ6yjM97eGWvriLUntOad7MoJ3WpgV/1qNbCjf2W/Ug3sqm9Xgqo9u7Lq22W/Eq1hhEU1ZUy1YqqV0OK1BT2eNVsyRiqtp7JmJm2k9ZOKzIUKwuMUzYad9r2c50/7/rTnT3tezvdzvp/z/JzvT3netOe7QtQbsK2a1qprrboedWLjipJUlISqmozGFSWpqJbC5pyU4OgkD6VrC7cq7Ip0beHa0q0K145KTOFUpVMVblW4tnSq9XupGa8VlEaMmvWFODVjPmGGaR24vrSqgaJRwwLK6tegnq83UHJZD8XZKlNwnwtPcp83Vp/c41JA6HIQEDpcCsldIYXkniCMMI1SlVCVRq1aZlBCiWJQIESxGBBQDEYoMIPObkypSmob64wwUAwGBOobz9cLnENIKAacS1kKeShkORSBkNOVKlE1m0tPSJsLjwubC5dLhwuHC5cLmwtPyLmJK2Qp5FzKYsCFhLTKCIG0qhCAKDIpISmVQf0mwOxNplFCCTTcSxmBpHJgY0ZIUj34ZsO9p+jNaaYmnWLttdde+8hHPvLYY48deteSriCPk5Biys5NVCfHq5NRY3N/ZWKiOjVRnUq0ps/o6j3HcXpGd254aWhDS3/yjI09F30o033eQu81mIphKkareQKHSLqhZwd2JbCrTq46M2Tnh/yZUVEeDwK3Qsh+EEXg09yb5p7HVCvWasQycas1a2bSeipjRBkZhWWLpZ7IwQmLw/HMhgUAX4goGqd9P+f5U56X8/1h23mxUKyEYTkMy0Fgc14NeTEIHM5tztOqajJmMpbRVIsxk7GUqsYUZjKWVNWEopiMxRUlqSoWYxZjac0wUzGzhWVUzVKYdawz0s2m5oFolK4tnIpwqqJc8CkVTgU4F54jAw/CQLg28FC4VRkG0veIZhBFoUYMmEINCxSNqBrVTWCMWglCKdGtqDAlhkUoo2YcmEJ1g6g6KCo1Y4QqxLCIoiqmfjwzg472WkIpAiECKQLBfSF5LSNDV4CQocOlBO5yySG0hQhrW4pA1jb2uOQQOhwkhA6XUnJXRLUmUWYjVqFMj1KTEkoUkwIh0W5TlVCVABDFpAAw2++dcxNqG2s0oRAgpLXhnEFVA954a7ReegJAFJlcylLAD7oZCgCohtHrhrzPAWDEDnwhuIRSyAGgEoqg4WcP3Gy4lxGSVCkAxBWmEhKVtrWbtPEmVQlRKEkoDADq0QsAGa0WHCm1dhRq1E+G2ZwGAIMSU6EAoFESYwwAFAqJo052W0CHD8L3vOc9g4OD999//5o1a07j3ncoeMEr5pyZqI1Zb2lO2lOmYnbHO6OW5qpU/yVdF3bFO1Ykeho/7kXolQb/O//qU/v+7aN7BU+feVXmzKtSq95MF3EkCO77hTG3OOIVRrzCqFsY9gqjXmGEexU93due7tXTvXrf5VTRQqcYuqXQLYZOKXQKYbUY5F7mTllKIXSLK7rH1KKijBFSBF4GWQIhNYvocd1qsWKtupVNp7oslmxLtWbNlqzZ0mKkKVmSx25rlHYZRpdxAoVU3g8czh3OC0Fgc26HYSkMK2HocF4OwlIY5ny/GobFIIyCM+/7B7YPeSBlSlViTLEUllCUg4NTsRTWELRpy8zWg9ZijHq+mYgfdrpQnfRdGYbCqYDgwrUh6ty6Dggu7LIUXHqO9D1hV+TMpOShcKsQBsL3pO8CD4VTkYJL15aBLwOf6CZhjJoJoJQa1mwNagJTqBknTIlyN6pEqREDQoiqEVUHAtSMAwDRDGAKo1QxLACgaQsoI4p20sVrVKqKUApfiECKUHBvNmKlDB0BUoYOBwDhy9AWANLNCQAQvhChBClD9yg3IXQ5RDHJiCRSNRUAoCqlCgFaK0mjAI6awADANEJYdJMCQG2aEiGNN1tmNwYgTFeIogIAsyhRCABQRublxPFcylIgAKAS8kDKqLSt3RQylFCuh2jtXg4AYjZ6AWBPxYsWigGPGoblgIcSYDanAcDhwuUSADwhbC4AIBCyEgqYk6nqbKZqjZlKoKF+NRk1KAGAqO0MAH2q/MNzTnFrtFwup1Kp73znO+9+97vn8ZlOhRNqjeadwk/3/nwmKExUpsarE/urk0W31Gq1HNzPbO+ItbVZreoho4BH5+aHi3t/kd/909LQc/Hu81IDl7es2WS2LmQbOXSLXn7YzQ+7heFowSsM++VJLdGuxtu0RIeRXqFnVhiZFXp6hZHugeNOKRG6oVMK3SJ3S/W89Oy8XZmyS/uDyiR3y+BVqO8QEXLKPMZsyqqEhKoehSU1kpqZMq2sGWuNp3syLasyyd6WeNsSTcp5x6UsBbPBGYalIHA4r/LZijPk+WA2aP3A5tzmvBQElTB0uCgFfiihEoaUkJSqAEBaVQmQhKoohFiM6YzplFqMKZQkFIUASWsqAKRUlQLEFUWl0eAW0yiNMcYISaoqAGQ0FQCSisoIiSlMmy1bpedIzoVTnq1BfQh94dqSh9KpSh5GuSt9FwCEUwEAGXgy8EHO3vRd4KEUXLoOAAi3ClJExSsAUDMOBIiqE0Wbk52Eseg8CVHvlygqUXUAoGYUtzpR1GgGEwAQ3SBMAaoQ3Zj9EQoARDWiyDnRvrHwheDSrtgq0QghtUpN1GKydlNClLjRTSmAu6Lhpmy8Wd8YAEKXg4TaXaGE2dIZABrHVhWrlijMoFHRUq9iqUKoSgGAUGCz7eJ6+c6M2l9bLb8BogIaAAgh9RMkRQO6c1bCgcCGk+hF15MYoFYEA0DRb8xUCbOlMESZKiQARG1nAGin4u413Sf0pEd3mI/76FD6M844Yx6fZjGYcfKvF/Z1Jtvf1H1hV7yjI9bearbM1yevkVlhZN7TcdF7uFcp7NlaeO3pnd+8TTFTmdVvTZ95VbLv4kPn18wT6ZenvMJIvbbzCiNuYSQoTyqxrJHu1dM9enpFovei1nNv0lM9WrKDvOHJpVQxtIShJdqPuWUxP2UqMopMz86XyhPVykS1Mu3ZM0Flhk/t9b1qwbfLoTcRBgLAU1jA9FA1QI9RPa4YKd3KGLFsLNYaS3RYekLR4pYeZ0aSEMaMOGXaYq6/TxojJJqqcxI/W58sI6QsBiEAFIJAgiwHYSilzbnHuSuEw3kgRCXkQspiEMxuBkO2E0phc+5x4Qlhcx4KUQ5DCVDwAwAoBoGAaKxLGIyZjKqExhWFEZJUFQDIaBoAJBWFETWmxjWdGglmMgYABqXRgs6oxRQA0CiNjg1VCY0rUQONJhQlehOiBwTXSSmKDDzgQZpR4VQBQHouiFDyUHouAAjXBsFlGMjAAwBhR3Gbk2FQq3oBpOdKHoKY/RGnClJCVB/zAADq0QuEULN2RCw1YtF3RKLpUV5GnWQAiGI4DMNAUahVK1CobhLKAEBVa5vpTKGaASqARqIUBwBiWKQWwzrUYlglWq3jShQtSnQ4XDxHY6sA0BicUbELAFHtCw3BKTlwbzZfG7cXAAAiCEUoAUCGMpoAFbWXD2wvYc7Ken7X9sGtPeZBodgYlnBwcB5uvTkb6qk52wMoVv1TmiimBgDC4DCvDj9Z5pprrnnrW9967733zu+TzbvFfT1CWdm/s/Dqz/KvPu3ODKZWvTmz+q3pM69UY0c7i9hRhHbeLYx4hWGvMOoWRrzCqFcY9opjzEjOBl6vnuo1Mr16qkdPdRN2Mh+j8+uExrR5YM8Uxwql0WJpvFIZtyszrj3tO0XuFqVXZYEjw0CTXAoeE4IC6FwwEKoQnDBOWcgUYCxUdEoZV3TKVKnoVDUI06hmMqZTPa4yjRoJXTGoZllGkqqmZaRV1bCsLKGMGQlCFabFjr2vi1gzr0focO5y4QtR5WFUxQJA3vcBoBSGXMpqyH0hXM4dzgEgCmCofbUPAcAXohpyAAikqIQcAKLchdmyGAAE1KJaSigEQfTU9W8JUb0LUUVLAACSqhodPphQlKjJFleU6L6YokSFrMVY/aqc9e1rP0IJSCl9LxpCE54DADHG1DCQIgQAS3BNhAAgOTdCj/iBpqrSreoETALCc0BwAIj6xgAAPBS+CwAgZVQBA4B0bSkEAESDuNAYwwAy9KNEB4B6MQ0AUfN59u0/EKsAQMxYfRiLGFa9wUN1A2YPbyWqHmUzRLlbz1rKiGHWH7XhKaAe8LM3D/rVanyi6PE5n/3GL4BzhSi1pwtdDpRSzWgMb2gI5iOuBwhtMWc9TciBd87nCc4OlCm5XG7fvn3R8p/8yZ98+MMftm37He94R+zgc4Vs2LBhHp/+tEbiXevjXet7r/yToJorvPZ0/rWn9z3+fxstfZnVb02f+dZ417rDdiO5V3Zna7uovIsWCFOjHqae7rXa12TWvM1I9eiZXqrM2yTAhcVUq631zLbWM4+5JZfcDpxQhE7oetz3vZLtVbhXtf2q6lWqQQV8u+rbMnBC3xGhJwMn9PK0vJ/zgIWeED7jIQQ+k5yFIQGh85CBNLhkIFXBBaEhVUJFkYQKVZdAhWoQqoBmEKoRVaeqSZmmGAlCGVCmGykAIEAMKwMAhDIrOqs7oXGrlRCQhMViWUYYoZTpcQAghJ4Gp/SLxikBAODwF5w6RerlKQDko7ABKPhB9KW+GARCSpgNYwCIqmEAqIRhIAQAVDm3ZzN10Lb5bD1QDsNQ1JYrYRjMrq+GoT87fz4ql6Nlh3OHh4S4AODyKOYpANUptZRanGiMxuK1T1o1RerDtwohCbUW54yQZMOwblRq128qhCai4lhKGQZQi38pZ197nFElDOo1jcH9+ol9ZeArQsSj7whhIMMQANKMyDCEsPbjMcFZ1Z79AaFPT+hQezBhVyiRSVkrQOtBXtvWtUEeSCnhu9Bw2b7GOAcAybn0nMYfnxurukVmjwtSGkrwWpayA5WxlD0Afwzz50BF+PWvf/0DH/jAMX9gUc0aXdwV4WFIEZaHt+VffSr/2tOhXcisvirZfyn3q7OZN+oVhiUP9HSvnu41oqkrs8vRB+jSssgPn2gUHZrphm4gwqpb5IFdtfMiDGwnJwT3nXwQetyrhoEjQk/4thCB9KogAvHXyQAAIABJREFUBQguAxcAiJQkjBYECTwAoAAs9AGAgtR4KEFSKQ0BAEBA6lHdAMRnCgEiiQwUjQCRhIRMi77dh4rOCAVCuGoolElJhKarVAUAoZoa04AQoRkaVQmhoMU0qoZhqFkJoIrGVEooEEr0GCXEYDpE42FUVZmqUUXOhrGpmJQQphr17++nR043x2EPqPeEsMNaTRNVzNFyIGRlNipCKcuzYcylLDVEiMu5wxvKICnKwUHXhc3P/mCkHIRhQyA5XLj8QEVVr7YjUsrCnB9viH+ICn1xYHshoXjw9hCV1Adft4QASatzu1AGo+YhRxOlVTV6u+TsTuqUWIwB5wAHdiNJgDa8KJBClSJGKQD06Mb/uOxymD8HvoNcffXV3/3ud+fxoY9ESvmpT33qX/7lX6SUv/d7v3f//fcfOjH10Ucf/bM/+7Px8fErr7zyn//5n9vbjz0ctSQQqiT7L032X9q/6eNeYST/2tOF1/+T6Ukj3RPvPi8KPMU6rU5it1RoTNOYVjvuM9556p7ICd1QhAAQha6U3HUKAfdBgucUAh6AlKFX9HkgpRR+xef/f3t3Hh5FlfYN+NRevSfp7IGQBAgQwo6yiCwiiDgoCm6AoqiDy7jrh/qqgzguM27jqC8D8yLj6ODGKDsoCgICA7JDWBIIhJA9nfTetdf3RyWdJulAAklXSD/35eVVfbq6+yGE+vU5deqUoKoKyft5WUBIxQTOL/MYUnGRq5X9mKqS/hpRERWkUhIvKbKiyKSqEKoiK4qKVKQqtCypKpJVGSFEKwquKgpSVVXFkMpqYawqKkK0ohKqihDCEIbqn9KIOKFo58kQpmKqQFAIIQzDMIRJOKHg2vR6HENIrItSjMBwhaAUgkAIERiBY5iCk2r9mXIMw0icRDiuEA1dSQonMQxXaDa4+jmOcIqgEMmgkFPsNE7hJI3IhhfiGE6TNHb+N0WWYEjGhELOiBM4wTKWRiMoGIaZaDPBhB8PJxjzpZ1TZ3CcOW+GZ0R7zBEQ2i3WqKhxvqL6wfNGjU5RbNSh0s5GN9rNJYrK+d0uQVF8koQQir3Y9UWt1fDrlZGRkZGR0cKXnTt37rfffrv11lsv4SO/+eabZcuW7d69G8OwcePG9e3b9+677w7dweFw3HXXXcuWLRs/fvyjjz76xBNPfPXVV5fwQR0cE9MleejM5KEz9S4ERJSh/kDcsNiCpS3PdlzmOUJJkQNSw/hVXVqLnKqICCGv4EUqUngfQkhURFEWkcQjRUQIBSROUVWD4EcIKarCyzwhCUg7ESgLoqpgsoQpdQdKRVV8sogpMi4Lwc8SFUFRFNLtxOqPkgpSJUXEZQlXQ84YKTKmyKQSMgSnqrIi0/J5R2FZkSlFxkMOuCpSCUUm1cbHZaQojNKkESEMYYwiYyjMGJiMYVIz098EMvy5eRUn5WaWolWoZs9uKDiOXXA6mIqTOHmhG5zJBE1RF9pBoViSpC+wdpJKGXGcoPGLzTnASZWkSZygLranlWYxjKQIirjwQvYYhtNGhJCBZDGEI4QQhhCFEEIY0cbDFZc4lfHgwYMvvvjipQXhkiVL/vCHP3Tt2hUh9Pjjjy9ZsqRREH755ZeDBw/+3e9+hxCaP39+dnZ2bW3t5S/2XehzfXHmaHaMPcsUk2WyJbF6jpEC0DGROBG6HFIkl0bqaHhZEEJyOpQn4FTqvy74/X6DwVA3lK1IAV9N2PNHnOBWRK5pu6KqnL+m2SJkieNcF6pSFnnBe4HzVYQscEJADRfndTsI3oBfuMA5L1LiFFWVVKm5HTS4ohCKpKiK0vSrRuM3lDBVUVTlAlUhhHAVkbKEEFJUBZ2/J29N7vb7tRf+lFZppzn9F3L8+PF58+Zp2/3793/nnXca7XDixIkBAwZo2926dWNZtrCw8PIn6dA4HpClNaWFhT7XaZ/LL0uZJluW9p/ZFtxmW3MDAQBAZ8UQdHMrC4Z+P/Cx558jTIhAadHO4/G07RvqcNCvqakJTqCwWq3V1dVNdwgdpLXZbA6HI+xbVVdX79u3L/QU44cfftjclJ8YGXsuLSc4WcYri6d97jMBzxm/+0hN5Zpzp077XWcD3jia7WYwZxptGQZLhtGSYbRmGCwprKnTrq/Tnnw+XydemUh3PlkSlcZnVvx+vzFkql4jCkJuKXwvpyVUVXU2/+YtxCky1+SEUJsQVNkvX6Tj0iZCfwiCIFD1sz8iQFCUyPwZLwevyH658fnCNpRKMA/3GNjCnVmWJS+40BLSJQjtdrvb7da2XS5XQkLjb1BxcXGhge9yueLjw197Fx8fP3jw4JbPGqUoKhiEZoSSbbEjzt9HRagk4Cn0urRe4zZXxWcl+ad9rlqRzzRZQ3qQMZkma5YpxtzM+QCgUVX1ouereEX2S6KgKD5ZlBTFIwmyqrpFQUV1x1ynwKtIdYmC0mT0RlaV5g7rHlGQwo32CIrsk8L/E3WJvBJurIaT5UAzhx6XyDetKiy3JMgt29MjCtLFBpc0ZpKimpxlUVX1AgdlHCEbdekTNzAMi7mMl2sMBMm2Zl3ylmNw0ngJIzqqiuQW/dUEZznGhvwQJEkiCRKF/shlBSkXf0NVRahVXwgkBamIwjBTi+8RjcTWfuFQVbFFv3thXxv8OAZhcWrr/4plBTWZWROW3WgyD2zLQXsdgjA7O/vIkSPXX389QujIkSM9e/ZsusN3332nbRcVFQUCgZbP4rlMGEJdDJYuBsvohC6h7QFZKvS5Cr2uQp/ztM+9uapYC0sLSYcOq2rbXQyWTnY7ULcoyKrilURRVXySKChyQJY4WeYUKSBLWrSIiuKVRC2ZVLXuK3OV30uSpBYwblGQVdUrCaKi+GWJr3sTicEJI0lRGG4mKRLHLSSNY5iNohFCsRSLEIqhGQxhVopu+lMlMDy2mYkG6UYrGe5vgcKJ5r6+2CgGR2FewhKEoZnDq41q6R2arGSY+sOyUDQZ7gJT1c83xLSiqlzDNwBVkFD99Hq/329gDKHP1u+kqv4wXxpUf5genuoL1+jnUZMsVzmx6cFL5cM0IoRUv9D0HYKUcB8afGnYkuqfRGqg+WcVVQ1csBNMEhjToiMhZgzmX8M39abfPDCGQi1YXRrDMczQmpu6sCxGtG62ZEjBLXwBhrXslsvNvPayviRhFIHoFv1F8MY2Ti4dgnDOnDmvvPLK7bffjmHYxx9//Oqrr2rt999//zPPPNOvX7+77777f/7nf9avXz9u3LjXX3/91ltvjYtrxR0V2oOBIPta7X2t9kbt5ZyvPiBd26pLPis6Wuh1VfH+dKM102TNMtu0iTmZJluW2Xb5X6VbqFbgRFXxSqKWNB5JkBS1VuS0/pbWv/FKgqgqToEXVcUjClqkeSVRVBSnyKuqWlvXG+NUhCwkTeK4maQoDDeSFIMT2pd67Qs4jRMmkiIxzELROMJiKRbDUHezDSHEWBKMRoMWMBaKIjHcRFI0ThgJksEJliCbC5gIUetjQFFUTkQIqaKMBBEhpAqyqkULJ6qygBBSAwJS1YY4UZSGw6t6XpyosoL4hk6nyotIakgF1c83ZEHomyCEJFkVpOAVy40yBjMyDTGNYxjbcMDCaBLVL16sKEqAJMIcYZs5xoU9eJ33WcFGA42aTFvH7RbU5OiMMSQK1+fDjDS6QG813IcGX4qZmv/ngyHM0Pyzrc2bVoIb80ae0AnOEd511115eXmDBw9GCD3wwAMzZszQ2o8dO+bz+RBC8fHxX3755dNPP11WVjZ69OglS5ZEvsgWSmZNyaxppP285V95RT7jcxf6nIVe12mfa6ejVAtLCseDoZhlsmWZYjJN1nSjVUGqXxJ9sigoikvkJUVxiYKgyD5Z9EsSr0huUZBUxSnygqL4JNEvi7wsuyVBUhSnyIuK4pWEgCxxiqyNqsXSLIlhFpLWksZMUhSOx1AsiWFWitH6N2aSNmN4pslGYriVohmcMBKUiaRoHNd6OdrgT8t7PGFd9IL6YEionKCNJqk+Xlu2oS5a5LqIQmJdMp33EllBwX6GWvfFX5UVpKWaJKu8hBBCgqiKMtKiRQl5CVYfAziOsdp6jwRiKIQQRhOYFi0spX0Nxwx1B/G6IzKOY6b6ziiG8CRb8A+FEbj2JnUPGQqRIctQGZmGn2ijL9EkgYV8I8YYsiUdi0YiucQaABEmSQFJFpRwl7tcjku8Q/3atWufffbZ48ePt201rXVlrSxTzQcKfa5Cn6vQ6zztcxf6XIU+Z7HfQ2C4iaS0TpKVokkMj6EZCiPMJGUgSJYg6xophsLxYKOFpEkMj6UZEsMtFM3ihIEgmxtVayFVkFQfr/p5JEoqL6naahScqNbljdZ5qh9okmW1Pm8QL2kvR5KMtM6ToiKERI+fwAmkqg2pxtenmiChuq4MgRDCWBrhwZjBsGBEEXURhaj6ZGJIjKx/iRZR9f0M7SUYgaP6VKsb8qIpTPsUhmr0ks4HgjDC9OoRqqrMCw0LnsmyIIVcACpKATnk2g9B8Koh12JyfMMlGaoqCYKv4X0UQRRD38evKOedUxcEn3L+pRSi6FOU81qEJi2i2Ph9RNEvN2lpvM/5fwqEEEkaSIJOiB8wfcrnqO1cYo9wwoQJ27dvb8M6okE8Y4hnDFfHteOqJU2pAUH181q8af9XfNoGp/oE1c+pvoZ2DMcwI4OZGESRWH3eNO4SBUfYiLouEU4SiCGRlmpaRBnqUk1FstFkbOj3BAf0qPO6PgDoTpI4Sa4b3w7GhopUnq+b2acooijWLcgpSn65flakx1NNUSSGYYoqC/XJJCsNsRSaCjzv1i6JUxRJEOviR5Z4SeaalqG9Vg1JlNAAwzCCCbmKA8cpimr4lq8FRvAhRZnwkBUAWCZkAAMj6JD3IXCaCrmEnyQNFHle58FiTsWx8/7xNnpzhBDdpIWijPj5F9pTlJE4v4UkDcT516tQTVo0kbt8orS0dPXq1adPn66trQ1tX7RoEUKIpmm7vfEJMxAZqiDVpZqPV32cth0Sbw2xp3g5DEOYicVMjJZwmInBjCxmYoikmPpGtq7dxGJUG8/l4zwe4gpZaxToSxC8iiojhHjehUJyItjRCWaPIHpVRQ5GlKJKouhDCEn1cRLsWPCCR1UVpKq84EYIyUrdnrLMSxIX2oIQIkmWrF/vjabrVlbDEMYwVq0xNGlCD9A4bqC0uzIhnKlfoJXAaYauCxuzKSV4xGcYK0IYQgjHSZqqW9eNIBmSYJuWgRCiKRN2XoDVvRy0rfBB+Msvv0yZMoXneZqmKYrieT4QCBgMhpSUlAjXFyVakW2eAIZj4bMt0YqZEkKzDTezl3CSCYDzqRzvRkjleXd9/DS/LbhVLXjUkO1gvNX3mULirW7sK9iHYBgbCunuEARNkgYUkj00ZdbuUaB1a3CM1CLHaKApbc/6zgej5RmGMbQVhWQPQTAkySKECJykqMu96xZMlukEwgfhk08+OWTIkJUrVz711FNpaWl/+tOffv7559///vfz58+PbHltqaRs16oND8TF9kiIz0mw5yTY+yTYc2g9VpBSOUE6VSEXlEkF5dLpCjUgaGGGn5dtDGZk8FgTZmJwE4sZaczE1o1bAhCO1ovy+Molhb2UuGq6Xbc/pnVEGMZa30Nqfpu2YRgWY+2mxQ9WH0Las3j9KBxBUCRpRM2PfQEQSWGCkOf5vLy8n376yWazIYQkSUIIjR8/fvHixbfddtvtt9/OslfkDfDSUobdc/tWP1dc5ThaVX30eMH31TX5RoNdS8QEe5/4+BybpWs7fbri8Eha8hWUKRVOolsC2SOZub6fqccEzNIJ77EOLo0WZrzgFkWfKAZE0ccLbkH0SWKgoV0KiKKP591CyLYo+lSk0pSJIs04QWi9Ja13FWabtiGEGsdV0+2QgUEAOrEwQeh0OmVZTk1NRQjZbLbgOcIRI0a43e7QhUCvOBRlSrENTkkaHGzx+ioqqw9XVB0+fnKlY/c7Xl+5Pa6XPS7bHtszMaFfUnyuNixzKRRVLquVCsqk/FIpvwwJEpGZSGQkGu4cSWantvnZONChSBLH8S7tXBQnuHjepZ3B4nlXQzvv4nmXJPOS1NDO8S6SYBjGpp0rYhgby9hIkiUIhmVsDGNjaCtJ1rcTLEEyLG3T9tf6bTBrFIDWChOECQkJDMOUlpZmZ2dnZmYuWrRIlmWCIA4fPowQ0vfGtm3ObEoym5Kyul2vPeS42krH0SrH0cqqI3nHv6l1nbZZ0+u7jDkJ8TlGQ/jF3jQqJ8pnq4Phh5kYMjuFzE5lbxpCpMbBSe4rlyTzHFfjDzj8gepAoCZQv13XG5P8gugPbquqQlMmmrZQlIkijTRtYkK2adpiYGMp0khR9e2Uka7fhnFCACIvTBDiOD5q1Kj169ePHTv2rrvueuGFF8aOHTtgwIDly5fn5uZmZWVFvsqIYdnY9LRr0tOu0R7KilhTU1DlOFrlOHameGFV9VGCoOLtOYnxOQn2PvH2nFhbJnLzUn6ZlF8mFZQp5U6iq53ISKSv6W166HrMfEWOIUeb+pCr8QeqtJALBGp8DdsOf6BalgWDwW5gY43GRAMbazTYWTY21TaUpswMbaEoI0WZIMwAuEKFnyyzcOFCbUQ0KSnp+++/f+ONN1asWDFs2LD33nuPaJ/VcjsmAqcS4nMS4nOCLV5fRY0jv+rswZO/rdjpetujVlqE2HgyMzG+X8KNA5NzbzWY4S4sHYgsCwGuluddnODy+Sp9/gptQNLnr/T6KrQBSZ+/imWsJmOiNt5oMiWZjIkJ9j4MYzMbk7R2kzEBu4yVCgAAHVn4IAxdCHvSpEmTJk2KVD0dlDbPU8ovQwVl1lPlMfa43hkzyN6pWI94j9FVUX24surw7vLPqvJeoClTUkI/e1x2XGzPpPh+cbE94ADaTi455GzW9MSEfhByAABN+CB86KGHnnvuuV69eoU25ufnv/zyy998801ECtOf4vQ1GvMks1OYCf1Nj93QsMgkQnaUZI/LzsmehhBSVcXpOlPlOFblOJp/as32Xe+Iojdeu1QjPifBnmOPyw69WhaEJck8xzk53snzTo5zcryL450c5/QFqjiu1h9wBLgav79KlgWDIc7AxhmNCQY2zsDGGQxx8XG9u6aN1B4ajQkMDZfzA9BJKIooCX5FCgicgNp0pY7wQbh69er777+/UaPD4Vi+fHkbfnaHIytycbWWfOLxEgzHiYwEMjvVOPNaMiuxRfdVwfDYmKzYmKzs7jdpLcHZNyWluw4cXtra2TediSQFuPODjeOdAa42uM3xLm1DVWWWiWHZGJaJYRkby8SwTAzD2lKtQ7Tkg5AD4IogiX5FFiXBrSiSyHsURZBFvywGZFmQBI+iSCLvVhRRFv2SGFBkXhK8WqOqiJLokyVOlrRGUeTdGE5StAkn2JikIdfc0pY3Y2jFeo/nzp3rfMuqqQFBKqyQ8svkgjLpVDlut5A9U6iBGYY7RuIJbXAF1UVn3+A4QVEmlrFhGEnTJpJgSJLV1u5jaCuOkzRlJgiaIg3aehkMbcFxkqYtBEFRpFFbna/pyn4RI4i++hhzck1Szed3iJKH45w870IYpkUay9bHGxvDMjExtozz2tmYRmsbAgAiSRQ8shiQpYDAuWQpoMi8FleSoCUTJ4k+VZFE3qUosiR4ZFmQxYAs+hVFEHmPooiS4JElQZYCBGkgSJqkLRhO0owVx2mCMhKUgSBokrbgOEExNgwnKcbGmpIIkiVpE4aRNGvDcIqkTATJEiRD0matMVhh+641unbt2s8++wwh5HK5XnnlldDY83q9O3bsGDNmTNt+vC6UKrd2eYNUUKbUeIku9WOef5h0mTeWvKiws28kmeN5t6JIouiTZE6SeG01d453KarEC3WXoGkrKPKCR1EkQfDIsihKfkkKyLLAC15VlVnGhuEkTYVLU9qsraVLkUYcpxjGgmON05ShzRhOMrSVwElFkeuDzVUfcsENZ+iGdn1bfe8thmVjDEysyZgYF9uTZWIUmYqLS9We0ha1AgC0N5F3y6JfkgIi75FEnywFJMEnCm4t3kTeLYl+WQxIolfkPbLkl8T6RikgCV6SNhOkgaSMFGMlKQNOsBRjxXGSpM04wZCUgaCMOGUyWrtiOEHRVoKgCa0RpynGguMkSVtxgiKpK+Yb7XlB6HK5CgsLEUKyLJeUlLhcDSudW63W2bNnv/DCC5EusO3IpbXi19udp6swA032TCF7pjAT+hNpcfrejsdsSmqjd1K5YJpKnCQ3pKl2mxXtwm1R8omS3+0pVhRJEL2N01SReMGtqDKGMIaxGdjYkL5ajNmcHG/vHTJoGcOyMY3Wj2/kovcjBAA0pQ0JKlJA4F0C51JkTpY4gXMpMq911ETepXXORN4lS7wsBcT6PUXeixMkQbI4wdKsjSBZnGBoNoZibATJEiRLMTYDYyNIhmJi6neo25MgGMYYj+k0vKSj8/7AM2bM0G6TO2zYsIULF2r3zu00cDOLj+hpevB6POZyl9ntkLC6W6sYOtvwNQBXNJF3i7xL5N0C75a0/wtukfcIvEviPaLg1oJN5D0i79K6ZRRjJSgjSRooxkJSJoIyav0zgjQSlIGiLYzRTpAGkjJRjJUgDSRloBgrSRkJykhe9jLiUSh88u/atSvCdUQAZjUQgzLwzrUyDgAgkhRZFHmXKLhF3i3yboFzeT1VmMpJgkdrqftP8GjhJ/JuirFSjJWirXUbjIVibBRjYU2JVGwPirHSjJVibdoOWubp/aeMOs12gfPz8xcuXJiXl0eS5Lp16xBCX331lcViuemmmyJYHgAAtCNZ4kTOKdQNMHIi7xQ4V/3/XQLnDPm/S5E5gXMTJK0NKlKMjWZjMMJkNMdTjM0ck0GQBpxgaNam7UAzNtpoxy94+gB0BOGDcNu2bTfeeKPNZsvIyCgqKtIaS0tLFy5cCEEIAOj4JNEvBGo4f5UQqOEDNUKghg9U834HH3AIXC3vdwicQ+Q9BGmo76jV99vYut6bwZxM0ZbznmWsRJNV+OF+hJ1A+CB89NFHx4wZs3z58l27ds2aNUtrnDRp0rPPPlteXp6cnBzBCgEA4DwC5+QDDiFQwwccfMDB+6p5TntYw/urtacQwhhDHGOMZwx22hDHGOJoQ5w5JpM2xAVbKMaq3YkeRLkwQehwOI4cOfJ///d/BoMh9GtOeno6Qqi0tBSCEADQTgTOxfnKg+OTnK+C81Vog5OcryLgrRACNRiOU0yMwZzEmpK08UmKsVnjsllTYnBMkjXDYQq0VJgg1O7EyzCNr6irqqpCCNE0rKwPAGg1WeJFrla7HoDzVdSnnUvgnJyvUnvI+6txgmRNyawpUYs3gznZZEuPSeyvnXgzmJMM5lScgLNuoC2FCcLExMSUlJQVK1YMHDgwtEf4r3/9y2q1NlqAFAAAggSuNuAp9XtK/J7SgKc04C31u88FvOWcrxIhhWbjGKOdMSZoA5UMa7faezNGO83WDV0yhji9/wQgGoUJQgzD5s2b99xzz/n9/m7dusmyvHPnzq+//vrjjz9+5ZVXKAq+iwEQ1STR53eXBLylAU+p31Ma8JT4vWUBT4nfXUKQjMGcarSmGSypBnNKUrexBkuawZzMmhLh+jbQYYWfLPPEE0+4XK633nqL4ziE0MiRIwmCeOyxx15++eXIlgcA0Icii0LAEfBV+lxFPleRdq7O5yoKeCtE3qmNXhrMySZbt7iUIammRNaUbI7JoJg2WKEXgAgLH4QYhr366quPPfbYli1bqqurzWbztdde27Vr1wgXBwBobwLn8rmKOF85V5952kgm5yvXzsmZbN1YUxJrSrKnXm2ydTOYEllzEkJwtQDoPC60ppzdbr/tttsiVgoAoJ1oUzE5X4XPdTYYdT5Xkd9TQtGmYNSZbN3Sek5hTUkGc5LR2gUuLQBRotkgrK2tXbp06f79+0tKSpKTk3Nzc+fMmQMXTgDQYamq4nMVVZflVYo1AW+p310S8JT6vaUBTynFWI2WVIM51WjtYjCn2BJyjJYuRksaa0qMwhWWAWgk/L+BgwcP3njjjWVlZd27d09KStq3b9/XX3/9zjvvrFixonPciQmATsDvPueuyXdXH3c7Trgdx92OAtaUYLRmWmLTDebUxPTRRkuqwZJiMKcRZPveXwyAK1r4IJw9e7bJZNq3b9+gQYO0lvz8/FmzZs2YMePMmTMwcRSAyBM4l9txwlOT73accDvyXVV5qipZ4rKt9l4xibkZuTNjEnJI2uz1es1ms97FAnAlCROElZWVBw8e/PHHH4MpiBDKzs5eunRpbm5uXl7ewIEDI1ghANFI5N1e5xm344Sz8pDbke92HJclzmrvZbX3stqzU7Im2OL7MMYEvcsEoDMIE4QMw2AYlpqa2qg9LS0NIcSycJ9xANqYIgte5+naikPBDp/A1ZpjMrQOX2L6tVZ7L5Otm95lAtA5hQlCm802ceLETz/99L333gtt//TTT/v27QsrywBwmRRF9NYWuh35waFOn6vIZOtmtfeyxGVn9ptltfcy2dLhEgUAIqMhCEtLS48ePapt33PPPU899VReXt5tt92WnJxcXV29bt26jRs3vvPOO3C3EQBaRVUkv6dE6+dpQ50+VxFrSrbas2MS+6f1nNJn+LOWuB5wrQIAesFUVdW2li5dOmfOnIu+ILh/R7B37965c+fu2bOnJTvLsszzvBHuUB9BHo/HYrHoXUWkcd7y2roTeyc8NfmemlOMMd5qz9Y6fNp5vvabxgmTZSIM7kcYeW1+YGnoEd58880tTBQAQBDnLXfXn9hzO064qvJI2qJNaYlPG5bZ757YpH5N7+YKAOg4GoLQbrfb7XYdSwHgCqE6q45WnNlUcWazs/IISZut9mxrfJ+4lCGZ/WZa4rJJCkYdALiSwKISALSIwLkqz26tOLOp/MxmkjYlZ1zX66rR8DpmAAAdC0lEQVTH41KGwDLTAFzpIAgBuBC340RZ4cbKs9tqKw7EJg1MTL929NBHLXE99a4LANBmIAgBaEzgnJVnt1We3Vp++iecYBLTr+0x6IHE9DGwUBkAnRIEIQB1mnb+rp32rSWuh951AQDaFwQhiGoCV1t59tfKs1vLTv9E1Hf+krqNxQla79IAABECQQiijqoqzsojlWe3Bjt/KVnX9776SaO1i96lAQB0AEEIogUfqKkq3l55dmtZ4UaCZFOyJvS66rH4tOHQ+QMgykEQgs4s2PkrK/zRVX0sLnlwStb1vYc9ZbSk6V0aAKCjgCAEnVBI5+9HgjSkZE3IGfFcfNoInIBbaQIAGoMgBJ2EqsrOyrxg5y+hy4iUrIl9hj1tsDS+oRgAAISCIARXNj7gqCreUVb4Y/npjawpKSVrInT+AACtAkEIrjxa508LP09todb5yx31ksGcondpAIArDwQhuGLw/uqqcztDO3+5o/4nvssIHIfOHwDg0kEQgo7O7z539vh/zuWvDnhKk7qNScq4rv/oVxljgt51AQA6CQhC0EEJnKukYPXZY8s9NSe7ZN886Lq34lIGw23cAQBtDoIQdCyKLFQU/XIuf3X56Y321Ku7D3wgtfskmPkCAGg/EISgQ1BVxVG6p6RgdfGJFeaYzG45dwy87k2KtuhdFwCg84MgBDpzO/LP5a86e2w5QTJpPadcd/c6o7Wr3kUBAKIIBCHQR8BbXlKw+lz+ar/7XFrPm4b/7v9iEnP1LgoAEI0gCEFEiby79NQPJQWra8r2JmVclzPiucT0axHC9K4LABC9IAhBJKiqXFW8vejot+Wnf7KnXpXWc8qwmxYRpEHvugAAAIIQtLPaikNnj31bfGKlOSajS/aU/mNeYwxxehcFAAANIAhBu/DUnCw+saLo6HKSYtJ6Thl312qTrZveRQEAQBgQhKAtCVxtScHaoqPf+N3FaT1/13f02+k9x+pdFAAAXAgEIWgDssSVFW48e+xbR+me5Mzxva76Q3LGdRhOejwevUsDAICLgCAEly44Baas8MfYpIHpfaZfPfnvJGXUuy4AAGgFCEJwKdyOE2ePLT+T97XRkpLeZ3r/0X9kjPF6FwUAAJcCghC0gt99rvjEijN5X2IY3iX7lrF3rjTHZOpdFAAAXBYIQnBxAucsKVhTdPQbb+2p1B6Th0z4ID7tar2LAgCAtgFBCJolS3zl2S1njy2vOPNLUsbY4BQYvesCAIC2BAc10Jh2I4izx749l79KmwIzZOIHJGXSuy4AAGgXEITgPKUn1x/+9U80E5PeZ1rfa15gDHa9KwIAgPYFQQjq1JTvP7x1gcDV9B89PyVrgt7lAABAhEAQAuR3n8vb8eeq4u19hj+TkXs3hhF6VwQAAJGjWxByHOd2uxMTE/UqACCEBK42f8//nj787+4D5wy+/h2CZPWuCAAAIg3X5VPfe++95OTkwYMHDx48+OzZs0136NGjR1y9OXPmRL7CTk+WuBO/ffzjP0cJnHPi7C05I56DFAQARCcdeoTHjh1bsGDB3r17e/To8fjjjz///PNff/11o32cTueuXbt69uwZ+fI6PVVVSgrWHt72ui2+99i71sAV8QCAKKdDEP773/+ePHlyjx49EEJPPvlkTk6O1+s1m81N91RVFcPg3uVtqfLs1sNbF+AEc9Wkj+LThuldDgAA6E+HICwsLOzVq5e23b17d1VVS0pKgi1BQ4YMEUVxyJAhH3300aBBg5p7N0EQCgsLgw9TUlIMBrjveRhuR/6RX//kqT3Vd+QLXbJ/hxB8wwAAAITaKQjLyso+/PDDpu2PPPJIt27d3G630Vh3gwIMw4xGo9PpbLTnhg0bBg4cKAjCggULpkyZcuLECZMpzAXd1dXV+fn548ePD7a8+OKLM2bMCFuVLMs8zyuKcol/qisW5ys/te/jyqKfMvrNyR37IU5QXq8vMh/t8/mgTx9hPl+E/nKBxu/3K4oCv+eR1KoDC8uyJHmRpGuXICRJMj4+zL0ItGoSExODySdJksfjSUpKarTn0KFDtf3feuutv//97wcPHhw5cmTTN4yPj8/Nzd2zZ09LqpJlmaKoYAZHA0n05e9ZeOrAp5n9Zk6as5OiLREuQFXVsIPeoF3BzzyStG/zEISR1OYHlnYJwoSEhOeee665Z/v167d+/Xpte8+ePXFxcWlpac3tzPO8IAgsCxMaW0dRxKK8r4/ufCc+bdj4mT8arV30rggAADooHc4R3nPPPa+99tqSJUuGDx8+b968Bx98kKIohNAzzzyTnZ398MMPHzx4cPfu3UOHDuU47u233+7du3e/fv0iX+eVq6zwx0Nb5hutXUfdusyW0FfvcgAAoEPTIQjj4+PXr1//2muvffLJJzfccMP8+fO19oSEBJvNhhAyGAw///zzwoULDQbD8OHD//GPf2hJCS6qpmzv4W0LZIkbfP1fErqO0rscAAC4AmCqqupdw6Xbu3fv3LlzW36OkOf5znqO0FtbmLfjbUfZ3j7Dns7InYFh+iyV0IjH47FYIn1iMso1dzESaCc+nw/OEUZYmx9YYK3RKx4fqCnYu/BM3tc9B/9+6A0fESSjd0UAAHAlgSC8gslS4OT+JQV7F6b2mDzh3s1wyyQAALgEEIRXJFVVzh77T972N+2pV427a60pJkPvigAA4EoFQXjlqTy79dDWBSRpGHbTYnvqVXqXAwAAVzYIwitJbcWhw9sWcL6KnBH/r0v2FL3LAQCAzgCC8MoQ8JQe2/VB6akfel/9ePcB92M4/MUBAEDbgONpRydwrvw9H58+/O/MfjNvuH975JdJAwCAzg2CsOPSlknL2/GXpG5jJty7mTU1XpEVAADA5YMg7JjUc/lrjmx/02RNv3ba17b4PnrXAwAAnRYEYYfjKNtzeOsCRRaHTHgvoUuYe24AAABoQxCEHYin5uTRnX+pKduXM3Jet5zpcO9cAACIAAjCDoEPOI7v+mvxiRU9B8+9atLHOEHrXREAAEQLCEKdSaL/1IFP8/f8b3qfaTfct51irHpXBAAA0QWCUE9+97kt30yN7zJi/KyNRkuzdycGAADQfiAIdSMJ3h0rZ2cPfaT7wAf0rgUAAKJXh7hrXRRSVXn3+kfjUgZBCgIAgL4gCPVxcPMriiINuu5tvQsBAIBoB0OjOijYt7jq3Paxd66GJUMBAEB3cCCOtPIzmwr2/n3sXathgigAAHQEEIQR5azK27PhiZFT/wVzRAEAoIOAc4SRw/kqdq6cPWDcn+KSB+tdCwAAgDoQhBEiS9zOVfd3H3h/115T9a4FAABAAwjCSFBVZfe6R6z2XtlDH9O7FgAAAOeBc4SRcHjb6wLnHHbTYr0LAQAA0BgEYbs7c2RZ+emfxt65BicovWsBAADQGARh+6o6tyNvx1/G3PEdzdr0rgUAAEAYEITtyFNTsGvt3GE3LTLHZOldCwAAgPBgskx74QM1O1bO7j/6j3CXeQAA6MggCNuFLPE7V85O7zMtvc90vWsBAABwIRCE7UHdt/EZgyW1z/Bn9K4EAADARcA5wraXt+MvPnfx6OnLEcL0rgUAAMBFQBC2seITK4qPfz/urjU4QetdCwAAgIuDIGxL1SW7D25+efT05YwxXu9aAAAAtAicI2wzPlfRrnVzr578v9b43nrXAgAAoKUgCNuGwDm3fz+zz7CnE9NH610LAACAVoAgbAOKIu5a+/vUHjdm9b9X71oAAAC0DgRhGziw6SWKtvS95kW9CwEAANBqMFnmch3f9WFtxcExd6zAMPhWAQAAVx4IwstSUrCm8NBn4+5eS1JGvWsBAABwKaATc+lqKw7u3/TSNVM/N5hT9K4FAADAJYIgvER+97kdq+4bMuFdW0JfvWsBAABw6SAIL4UkeHesnN37qsdTsibqXQsAAIDLAkHYaqoq717/aELXkd0HztG7FgAAAJcLgrDVDm5+RVGk/qP/qHchAAAA2gDMGm2dgn2Lq87tGHvnKgyHHx0AAHQGcDRvhfLTPxfs/fvYu1ZTjFXvWgAAALQNCMKWclYe2fPDkyOn/stoSdO7FgAAAG0GzhG2COer2LnqvgHj/hSXPFjvWgAAALQlCMKLkyVu56r7uw+8v2uvqXrXAgAAoI1BEF6Eqiq71z1itffKHvqY3rUAAABoe3CO8CIOb1sg8q5hNy3WuxAAAADtAoLwQs4cWVZ++uexd67BCUrvWgAAALQLCMJmVRRtObL97bF3rqRZm961AAAAaC8QhOF5agp+2/CHYTctMsdk6l0LAACAdgSTZcLgAzU7Vs7uP3p+QpeRetcCAACgfUEQNiZL/M6Vs9P7TEvvM03vWgAAALQ7CMJG1H0bnzFYUvsMf0bvSgAAAEQCnCM8T96OP/vcxaOnL0cI07sWAAAAkQBB2KD4xIri4yvG3bUGJ2i9awEAABAhEIR1qkt2Hdz88ujpyxljvN61AAAAiBw4R4gQQj5X0a51D189+X+t8b31rgUAAEBEQRAigXP++v2MPsOeSUwfrXctAAAAIi3ag1BRxP+ueSitx01Z/e/RuxYAAAA6iPYgPLDpJZqx9r3mBb0LAQAAoI+onixzfNeHzspDo2//HsOi/QsBAABEregNwpKCNYWHPht39zqSMupdCwAAAN1EaU+otuLg/k0vXTP1c4M5We9aAAAA6Ckag9DvLt6x6r4hE961JfTVuxYAAAA6i7oglATvjpWze1/1eErWRL1rAQAAoL/oCkJVlXevfzSh6zXdB87RuxYAAAAdQnQF4dHtCxRF6j/6j3oXAgAAoKPQLQirq6tLS0svsMOZM2d+/fVXj8fTZp94bntt+d7hNy3G8OidKwsAAKARHYLwm2++6datW0JCwvTp05vb54UXXhg+fPirr77ao0ePnTt3tsnnxncZOfLW70na3CbvBgAAoHPQIQgHDBiwatWqxYsXN7fDsWPHFi5cuHfv3k2bNr388svPPNNW98jFcIJqo7cCAADQSegQhL169RowYABJNjs+uXz58uuvvz4tLQ0hdO+99/7222/FxcURLBAAAEAU6Yhny4qLizMzM7Vtm80WGxtbXFzctWvXpnsqiuLxeDZu3Bhs6du3b3Jy+GvkS0pKioqKrrnmmvaoGYS1adOmCRMmsCyrdyHRwu1279y584YbbtC7kCiyc+fOAQMG2O12vQuJFqIobty4cerUqS3cH8cv3t9rlyBct27dp59+2qiRIIivv/66JS8PBALx8Q13x2VZ1u/3h93T6XSWl5e/9dZb2kMMwx555JFJkyaF3Xn9+vWbN28eNGhQS2oAbeLll19OSEjo37+/3oVEix07drz55pvXXnut3oVEkXfffXfOnDmTJ0/Wu5Bocfr06WeeeWbixJZeCM6y7AUGIDXtEoS9e/eeNWtWo8aWxLImKSnJ4XBo26qqOhyO5jp5cXFxPXv23LRpU0velqZpkiTNZpgsEzk4jhuNRviZR4zBYCAIAn7gkUSSJMMw8DOPGKPRiON42/7A2yUIs7KysrKyLvnlV1999WuvvaZt79mzx2Aw9OjRo41KAwAAAM6jw2SZwsLCP//5z+vWrSspKfnzn//83Xffae1du3b95ZdfEEJTp07lOO7JJ59cvXr1ww8//Nhjj8FJJgAAAO1Eh8kyoijW1tZ27969e/futbW1Xq9Xa7///vu7dOmCEKJpesuWLe++++7nn38+Z86chx9+uLm3crlcJ0+enDBhQks+t6ysrKampoU7gzZRWFj4yCOPwKhRxNTW1hYWFsIveSQdOHCgpKTk448/1ruQaBEIBMrLy1v+S37rrbc++uijF94HU1X1sgvTDcdxX375ZdgJpU35/X6Px5OUlNTeVYGgoqKirl27tvz0MLhMkiSVlZW18F8EaBOlpaV2u51hGL0LiRaqqhYVFWVkZLRw/8zMzO7du194nys7CAEAAIDLBF/VAQAARDUIQgAAAFENghAAAEBUgyAEAAAQ1TriWqPtgeO4DRs2+Hy+CRMmJCYm6l1O5yRJ0s6dO0+fPp2SkjJu3LjgskabN2+WZVnbTklJ6du3r341diqCIGzdujX4MHR2XG1t7YYNGyiKmjRpEly+0ob27dtXU1MTfGg2m4cPH44Q+vXXXzmO0xrj4+MHDhyoT32dSHFxcUFBQd++fUOn+gcCgQ0bNgQCgYkTJ4auxHns2LH//ve/WVlZo0ePxjCstZ8VFbNGfT7fqFGjYmNju3Tpsn79+l9++QWOxe1h6NChGIbl5uYeOHBAVdUtW7bYbDaEkNVqHTBggLYqwvXXXz9v3jy9K+0kqqqqkpKSxo8frz2cNWvW7NmzEUJnzpwZMWLE6NGjfT5fQUHBzp074+LidK2085g3b96+ffu07by8vAEDBqxfvx4hlJ6enpaWpn3nGD58+Ouvv65nlVe+3Nzcs2fPCoLw+eef33777Vqjx+MZOXJkSkpKYmLijz/+uG3btl69eiGEli1b9tRTT912223btm0bPnz4kiVLWv15ahRYtGjR8OHDZVlWVXXevHl333233hV1TidPntQ2JEnq37//3/72N+2hxWI5c+aMfnV1WpWVlTiON21/7LHHHnzwQW178uTJb731VmTrigqKonTv3v2rr77SHnbt2lX7/gfaRGFhoSzLubm533zzTbDxb3/72+jRoxVFUVX1qaeeuu+++1RVlSSpW7duq1atUlXV4XBYrdYjR4609uOi4hzhmjVrbrvtNu2y7unTp69Zs0bvijqn4LgcQRAJCQmCIASf+u23337++efq6mqdSuvMtmzZsnXrVo/HE2xZs2bNtGnTtO1p06bBL3x72LRpU21t7S233BJsOXDgwMaNGysqKnSsqtPIzMxsuhCH9outjXwGj+RHjhyprq6+8cYbEUJxcXHjxo1bu3Ztaz8uKoKwpKREu80vQigtLc3j8bjdbn1L6ty2bt26b9++O+64Q3tot9uXLl26YMGCzMzMzz77TN/aOpnU1NR33333+eefz8jI+OGHHxBCqqqWlZVpqxUihNLS0kpKSnStsXNasmTJzJkzg8sgx8TEfPXVV2+//Xb37t0/+ugjfWvrrBodyR0OB8dxJSUlSUlJwRkJl/YLHxWTZWRZDn65IAgCISRJkq4VdWYnTpy4++67Fy1aFFzo6+TJk9qPfd26ddOmTZsyZQqcsmoT8fHxRUVF2u/2xx9/fN9995WWliqKoihKcL4AQRDw297mXC7XihUrduzYEWzZv3+/9kv+66+/jh8//pZbbklPT9evwM6p0ZFcVVVZlmVZDp0dc2m/8FHRI0xJSamsrNS2KyoqWJaNjY3Vt6TO6tSpUxMmTHjjjTeC57dR/ZcPhNDkyZMZhjl+/LhO1XU2GIYFjwszZswoLy8vLy8nCCIxMbGqqkprr6ioSE1N1a/GzumLL77IyckJnRoa/CUfNWpUSkrK4cOHdSqtM2t0JLfZbCaTKSUlpbq6Wq2f9VlRUZGSktLad46KIBw7dqw2aoQQ+vHHH8eOHXsJ82vBRZ09e3bixIkvvvjifffdF3aHkydPut1uWBK6Pezdu5dl2YSEBBTuF17PyjqjJUuWzJkzJ+xTpaWl5eXl8EveHsL+Yufm5pIk+d///hchxPP8li1bxo0b19p3jorLJ6qrq/v37z916tQuXbq88847K1asGDNmjN5FdUIDBgxwOp2TJk3SHo4ZM2bGjBnr1q375z//OWjQIL/fv3Tp0ltuueWTTz7Rt85O4x//+Mf27dtzcnKqqqqWLFny4osvPv/88wih/fv3jx079umnn/Z6vf/85z/37dsHw3Rt6NChQ8OGDSstLQ0OLG3fvv3dd98dOnSoKIqfffbZiBEjli1bpm+RV7oPPvjg+PHjy5cvHzx4cFZW1nPPPdezZ8/y8vIBAwbceeediYmJ77777rp160aOHIkQevvtt5csWfLoo4/+8MMPgiBs2rSptR8XFUGIECotLf388899Pt+tt946aNAgvcvpnL744gu/3x98mJOTM2rUKIfDsXLlysLCQpZlR4wYEbzoDVy+kpKS1atXnz171mq1XnfddVdffXXwqaNHj3777bckSc6aNatbt246Ftn5HDx4sKio6Oabbw62uN3uFStWFBQUUBQ1dOjQG2+8EcacLtPatWtD57xMmTJFG/A8d+7c559/znHctGnT+vfvH9xh9erVO3fuTE9Pv++++y7hRu7REoQAAABAWFFxjhAAAABoDgQhAACAqAZBCAAAIKpBEAIAAIhqEIQAAACiGgQhAACAqAZBCIA+tFtjtsc7Hz58ePHixcGbIV+Uw+FYvHhxcXFxexQDQMcHQQiAPhYsWPDhhx+2xztv3Lhx7ty5LV96+Ny5c3Pnzs3Ly2uPYgDo+CAIAQAARDUIQgA6kOrqao7jLrCDy+UKvQcvQsjhcFyg81ddXe31esM+VVNT43K5mnuhIAjBlf4B6NwgCAHQX1VV1dSpUy0WS0JCgslk6tOnz3fffRd8ds2aNXFxcT/99NPo0aNjYmJsNtv06dP9fv/mzZt79uwZHx9vs9neeOONRu+5e/fuvn37JiQk2Gy2adOmOZ3O4FOFhYXXXnut3W6PiYkZPXr0mTNnQl/473//e8CAASzLJiUlmc3m22+/vbq6uj3/9ADoDIIQAP15vd6kpKRvv/326NGjW7Zs6d2795133nnw4EHtWUEQamtrH3jggZtvvvm3337761//umLFioceemju3Lnz58/fvXv3vffe+/LLL2/bti30PWfMmPH444/n5eUtWrRow4YNM2fO1No5jps8efKpU6e+//77w4cPX3fddQ899FDoCysqKh588MHt27cfPXr0k08+2bp16z333BOZnwMA+lABAHoYPnz41KlTwz4lCEJKSspLL72kPfzPf/6DEJo/f35wh4kTJyKEfvrpJ+0hz/OxsbHPPvus9vC9995DCL355pvB/bWW/fv3q6r6xRdfIIR++OGH4LOzZs1CCK1fvz5sMZ999hlCSLv3KQCdEqljBgMAgjwez1dffVVQUBA8b3fq1KnQHW644YbgdnZ29i+//BK84y5N05mZmY2uf5g2bVro9rPPPrtv376BAwceOHDAYrFMmDAh9FktHYP27t27YcOGsrIyURS1m92fOnXKbre3yZ8UgI4GhkYB0N+BAwcyMjJef/31qqoqi8USGxtLUVSjmSzB28AihBiGsVqtBEEEW2iaFgQhdP/k5OTgdvBebgihkpKSpKSk0Bvmpaamhr7wiSeeuPrqqzdt2qQoSmxsrNVqRQhdYFoNAFc66BECoL/3338/Jibm0KFDJpNJa1m1atVlvmd1dbWWYQghrVenxWFSUlKjyS+hs0MrKys/+uijv/zlL9rN7hFC69at00ZHAeisoEcIgP5Onz7dp0+fYAoWFBQUFBRc5ntu2LCh0XZubq72f6fTuWvXrrB7ajNIhwwZEmxZt27dZVYCQAcHQQiA/gYOHLh58+aNGzfyPL9nz5477riDZdnLfM+33357/fr1gUBg48aNL7300rBhw4YPH44Quv3221NSUh544IH9+/d7vd6lS5cuW7Ys+Krs7Gyj0fj++++Xl5c7nc4PP/ww9FkAOiUIQgD098c//nHgwIETJ05kWXbUqFF33HHHVVdddZnv+cEHH8ycOdNoNE6cODE9PX358uXaeUGr1bp69Wq/3z948GCLxbJgwYL3338/+KqYmJjFixdv3bo1JSUlNjZ28eLFH3300WVWAkAHh6mqqncNAEQjRVEQQjhe92VUVdXTp0/X1NT06tXLYrG0yUdwHHf06FGDwdCnT59GT8myfOjQIZIkc3JyQifdaLxeb0FBgdFozM7ODp1WA0CnBEEIAAAgqsHQKAAAgKgGQQgAACCqQRACAACIahCEAAAAohoEIQAAgKgGQQgAACCq/X/RVjtM8jDDlgAAAABJRU5ErkJggg==",
"image/svg+xml": [
"\n",
"\n"
],
"text/html": [
"\n",
"\n"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# convert to classification problem\n",
"Y = sign.(X * β + 5 * randn(n))\n",
"\n",
"# solve at a grid of λ\n",
"λgrid = 0:10:100\n",
"β̂svmpath = zeros(length(λgrid), size(X, 2)) # each row is β̂ at a λ\n",
"β̂svm = Variable(size(X, 2))\n",
"@time for i in 1:length(λgrid)\n",
" λ = λgrid[i]\n",
" # objective\n",
" problem = minimize(sum(pos(1 - Y .* (X * β̂svm))) + λ * sumsquares(β̂svm[2:end]))\n",
" solver = Mosek.Optimizer() # MOSEK this time!\n",
" MOI.set(solver, MOI.RawOptimizerAttribute(\"LOG\"), 0) # keep silent \n",
" solve!(problem, solver)\n",
" β̂svmpath[i, :] = β̂svm.value\n",
"end\n",
"\n",
"plt = plot(collect(λgrid), β̂svmpath, legend=:none)\n",
"xlabel!(plt, \"lambda\")\n",
"ylabel!(plt, \"beta_hat\")\n",
"title!(plt, \"SVM\")\n",
" "
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"@webio": {
"lastCommId": null,
"lastKernelId": null
},
"kernelspec": {
"display_name": "Julia 1.9.3",
"language": "julia",
"name": "julia-1.9"
},
"language_info": {
"file_extension": ".jl",
"mimetype": "application/julia",
"name": "julia",
"version": "1.9.3"
},
"toc": {
"colors": {
"hover_highlight": "#DAA520",
"running_highlight": "#FF0000",
"selected_highlight": "#FFD700"
},
"moveMenuLeft": true,
"nav_menu": {
"height": "64.4000015258789px",
"width": "251.60000610351562px"
},
"navigate_menu": true,
"number_sections": true,
"sideBar": true,
"skip_h1_title": true,
"threshold": 4,
"toc_cell": true,
"toc_position": {
"height": "399.3333435058594px",
"left": "0px",
"right": "899px",
"top": "140.6666717529297px",
"width": "151px"
},
"toc_section_display": "block",
"toc_window_display": true,
"widenNotebook": false
}
},
"nbformat": 4,
"nbformat_minor": 4
}