{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# IAP 2022: Advanced Optimization Session (Session 1)\n", "\n", "**Shuvomoy Das Gupta**" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Response for the advanced optimization session\n", "\n", "* Convex programming (32%)\n", "* Integer programming (28%)\n", "* Nonconvex programming (28%)\n", "* Semidefinite programming (5%)\n", "* Other" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Based on the responses, I will cover the following topic today.\n", "\n", "# Topics\n", "\n", "---\n", "$\\textsf{Session 1: Convex Optimization and Extension}$\n", "---\n", "\n", "* How to model an optimization problem in a tractable manner\n", "\n", "\n", "* Tractable optimization models\n", "\n", "\n", "* Convex optimization through `Convex.jl` and `JuMP`\n", "\n", "\n", "* Modeling examples of common convex programs that shows up in OR\n", "\n", "\n", "* What to do when the problem is nonconvex\n", "\n", "\n", "---\n", "$\\textsf{Session 2: Discrete Optimization: Basic and Advanced}$\n", "---\n", "\n", "* Mixed Integer Linear Programs (MILP): basic\n", "\n", "\n", "* How MILPs are solved: Branch-and-Bound algorithm\n", "\n", "\n", "* How do we exploit structures to speed up MILPs\n", "\n", "\n", "* Solving Mixed Integer Quadratic Problems" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Modeling an optimization problem \n", "\n", "* We have our research problem, that usually come from a practical problem rooted in reality. \n", "\n", "* While modeling the problem, we want to stay as close to reality as possible, but without making the model intractable. \n", "\n", "* This is a balance. Safety-critical constraints have to be modeled exactly, where soft constraints can be relaxed.\n", "\n", "* It is a good practice to be considerate during the modeling phase so we end up with a model that is practically tractable.\n", "\n", "### Some good resources on how to model optimization problems\n", "\n", "* Model building in mathematical programming by H Williams ([Link](https://www.amazon.com/Model-Building-Mathematical-Programming-Williams/dp/1118443330))\n", " * A very practical book for modeling LP and MILP\n", " \n", "\n", "* Optimization models by G Calafiore and L El Ghaoui ([Link](https://www.amazon.com/Optimization-Models-Giuseppe-C-Calafiore/dp/1107050871))\n", " * Good book for modeling convex optimization problems, especially Chapter 9, 10, 11 \n", " \n", " \n", " \n", "* Applications of optimization with Xpress-MP by C Guéret, C Prins, and M Sevaux ([Link](https://www2.isye.gatech.edu/~sahmed/isye3133b/XpressBook.pdf))\n", " * Excellent for modeling integer optimization models, especially Chapter 3, with lot of shortcuts and hacks " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Modeling an optimization problems\n", "\n", "At present the following types of problems are *practically tractable*. Whenever we are modeling an optmization problem, we should try to ensure that our model is one of the following:\n", "\n", "1. Linear programs (LP)\n", "\n", "\n", "\n", "2. Quadratic convex programs (QCP)\n", "\n", "\n", "\n", "3. Second order cone programs (SOCP)\n", "\n", "\n", "\n", "4. Semidefinite programs (SDP)\n", "\n", "\n", "\n", "5. Mixed-integer linear programs (MILP)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## My modeling strategies\n", "\n", "* **Solve the smaller optimization problem first**\n", "\n", "> Thomson's Rule for First-Time Telescope Makers: \"It is faster to *make a four-inch mirror then a six-inch mirror* than to *make a six-inch mirror*.\"\n", "\n", "I follow a modified version of Thomson's rule\n", "\n", "It is faster to **solve a smaller simpler optimization model and then the larger complicated optimization model** than to *solve a larger complicated optimization model*.\n", "\n", "* **Do not optimize prematurely**\n", "\n", "> \"The real problem is that programmers have spent far too much time worrying about efficiency in the wrong places and at the wrong times; premature optimization is the root of all evil (or at least most of it) in programming.” \n", "> - Donald Knuth, The Art of Computer Programming" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Milestones in optimization\n", "\n", "\n", "* 1947: G. Dantzig, who works for US air-forces, presents the Simplex method for solving LP-problems\n", "\n", "\n", "\n", "* 1948: J. Von Neumann establishes the theory of duality for LP-problems\n", "\n", "\n", "\n", "* 1951: H.W. Kuhn and A.W. Tucker reinvent Karush's optimality conditions (known as KKT conditions) \n", "\n", "\n", "\n", "* 1951: H. Markowitz presents his portfolio optimization theory => (1990 Nobel prize)\n", "\n", "\n", "\n", "* 1954: L.R. Ford's and D.R. Fulkerson's research on network problems => start of combinatorial optimization\n", "\n", "\n", "\n", "* 1960-1970: Many of the early works on first-order optimization algorithms are done (mostly developed in Soviet Union)\n", "\n", "\n", "\n", "* 1983: Nesterov comes up with accelerated gradient descent\n", "\n", "\n", "\n", "* 1984: N. Karmarkar's polynomial time algorithm for LP-problems begins a boom period for interior point methods\n", "\n", "\n", "\n", "* 1990s: Semidefinite optimization theory\n", "\n", "\n", "\n", "* 2015s: First-order methods become very hot again due to machine learning" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "## General convex optimization problem\n", "\n", "Most authoritative reference is Convex Optimization by Boyd and Vandenberghe ([pdf](https://web.stanford.edu/~boyd/cvxbook/)).\n", "\n", "Convex optimization problems are defined through the notion of convex set.\n", "\n", "**Convex set**\n", "\n", "\"Drawing1\"\n", "\n", "**Convex function**\n", "\n", "A function is convex if and only if the region above its graph is a convex set.\n", "\n", "\n", "\"Drawing1\"\n", "\n", "## Standard form of a convex optimization problem\n", "\n", "A general convex optimization problem has\n", "the following form:\n", "$$\n", "\\begin{array}{ll}\n", "\\underset{x\\in\\mathbf{R}^{d}}{\\mbox{minimize}} & f_{0}(x)\\\\\n", "\\mbox{subject to} & a_{i}^{\\top}x=b_{i},\\quad i=1,\\ldots,p,\\\\\n", " & f_{i}(x)\\leq0,\\quad i=1,\\ldots,m.\n", "\\end{array}\n", "$$\n", " where the equality constraints are linear and the functions $f_{0},f_{1},\\ldots,f_{m}$\n", "are convex. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Convex optimization\n", "\n", "* a subclass of optimization problems that includes LP as special case\n", "\n", "\n", "* convex problems can look very difficult (nonlinear, even\n", "nondifferentiable), but like LP can be solved very efficiently\n", "\n", "\n", "* convex problems come up more often than was once thought\n", "\n", "\n", "* many applications recently discovered in control, combinatorial\n", "optimization, signal processing, communications, circuit design,\n", "machine learning, statistics, finance, . . ." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## General approaches to using convex optimization\n", "\n", "* Pretend/assume/hope $f_i$ are convex (around minimum) and proceed $\\Rightarrow \\textsf{Machine Learning in Practice}$ \n", " - easy on user (problem specifier)\n", " - but lose many benefits of convex optimization\n", " - risky to use in saftey-critical application domain\n", " - an important example `Adam`: one of the most heavily used solver for deep learning. (also see [Link](https://arxiv.org/pdf/1804.10587.pdf))\n", " \n", " \n", "* Verify problem is convex (around minimum) before attempting solution $\\Rightarrow \\textsf{Machine Learning Theory}$ \n", " - but verification for general problem description is hard \n", " \n", " \n", "* Model the problem in a way that results in a convex formulation $\\Rightarrow \\textsf{Operations Research}$ \n", " - user needs to follow a restricted set of rules and methods\n", " - convexity verification is automatic\n", "\n", "Each has its advantages, but we focus on 3rd approach." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## How can you tell if a problem is convex?\n", "\n", "* Need to check convexity of a function\n", "\n", "Approaches:\n", "* use basic definition, first or second order conditions, e.g., $\\nabla^2 f(x) \\succeq 0$\n", "* via convex calculus: construct $f$ using\n", " - library of basic examples or atoms that are convex\n", " - calculus rules or transformations that preserve convexity" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Basic convex functions (convex atoms)\n", "\n", "* $x^{p}$ for $p\\geq1$ or $p\\leq0$; $-x^{p}$ for $0\\leq p\\leq1$\n", "\n", "* $e^{x},$$-\\log x$, $x\\log x$\n", "\n", "* $a^{\\top}x+b$\n", "\n", "* $x^{\\top}x$; $x^{\\top}x/y$ (for $y>0$); $\\sqrt{x^{\\top}x}$\n", "\n", "* $\\|x\\|$ (any norm)\n", "\n", "* $\\max(x_{1},\\ldots,x_{n})$\n", "\n", "* $\\log(e^{x_{1}}+\\ldots+e^{x_{n}})$\n", "\n", "* $\\log\\det X^{-1}$ (for $X\\succ0$)\n", "\n", "These are also called *atom*s because they are building block of much more complex convex functions. There are many such atoms, most convex programs in practice can be built from these atoms. A more complete list can be found [here](https://jump.dev/Convex.jl/stable/operations/). \n", "\n", "When modeling a problem, we aim to construct a model in such a way so that the building blocks of the functions are the atoms above. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Convex calculus rules\n", "\n", "* nonnegative scaling: if $f$ is convex then $\\alpha f$ is convex\n", "if $\\alpha\\geq0$\n", "\n", "\n", "\n", "* sum: if $f$ and $g$ are convex, then so is $f+g$\n", "\n", "\n", "\n", "* affine composition: if $f$ is convex, then so is $f(Ax+b)$\n", "\n", "\n", "\n", "* pointwise maximum: if $f_{1},f_{2},\\ldots,f_{m}$ are convex, then\n", "so is $f(x)=\\max_{i\\in\\{1,\\ldots,m\\}}f_{i}(x)$ \n", "\n", "* pointwise supremum: if $f(x,y)$ is convex in $x$ for all $y \\in S$, then $g(x) = \\textrm{sup}_{y \\in S}{f(x,y)}$ is convex\n", "\n", "\n", "\n", "* partial minimization: if $f(x,y)$ is convex in $(x,y)$ and $C$\n", "is convex, then $g(x)=\\min_{y\\in C}f(x,y)$ is convex\n", "\n", "\n", "\n", "* composition: if $h$ is convex and increasing and $f$ is convex,\n", "then $g(x)=h(f(x))$ is convex" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## The function that may show up in your model can be much more complicated\n", "\n", "Many complicated convex functions that we can use in the modeling phase can build from the atoms through convex calculus.\n", "\n", "* piecewise-linear function: $f(x)=\\max_{i=1,\\ldots,k}(a_{i}^{\\top}x+b_{i})$\n", "\n", "\n", "\n", "* $\\ell_{1}$-regularized least-squares cost: $\\|Ax-b\\|_{2}^{2}+\\lambda\\|x\\|_{1}$\n", "with $\\lambda\\geq0$\n", "\n", "\n", "* support-function of a set: $S_C(x)= \\max_{y \\in C}{x^\\top y}$ where $C$ is any set\n", "\n", "\n", "\n", "* distance to convex set: $f(x)=\\min_{y\\in C}\\|x-y\\|_{2}$" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Basic modeling steps\n", "\n", "* When modeling a problem we face many choices how to mathematically model a certain reality\n", "\n", "\n", "* During the modeling phase try to select your functions with their building blocks as convex atoms\n", "\n", "\n", "* When modeling some operation on a mathematical object try to use convex calculus rules\n", "\n", "\n", "* If you can follow the last two steps, you will have a convex problem\n", "\n", "\n", "* If you have a nonconvex problem (that is not mixed integer linear program), you can try one of the two things:\n", " - Work with tightest convex approximation of the nonconvex problem (will see one example)\n", " - *Sequential convex programming*, a local optimization method for nonconvex problems that leverages convex optimization (works suprisingly well in my experience)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Linear programs\n", "\n", "$$\n", "\\begin{array}{ll}\n", "\\underset{x\\in\\mathbf{R}^{d}}{\\mbox{minimize}} & c^{\\top}x\\\\\n", "\\mbox{subject to} & Ax=b,\\\\\n", " & Cx\\geq d.\n", "\\end{array}\n", "$$\n", "\n", "Shows up when we are modeling system that can be completely described by linear equalities and inequalities." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Quadratic convex programs (QCP)\n", "\n", "$$\n", "\\begin{array}{ll}\n", "\\underset{x\\in\\mathbf{E}}{\\mbox{minimize}} & c^{\\top}x+x^{\\top}\\underbrace{Q_{0}}_{\\succeq0}x\\\\\n", "\\mbox{subject to} & Ax=b,\\\\\n", " & x^{\\top}\\underbrace{Q_{i}}_{\\succeq0}x+q_{i}^{\\top}x+r_{i}\\leq0,\\quad i=1,\\ldots,p.\n", "\\end{array}\n", "$$\n", "\n", "#### Where does it show up?\n", "\n", "QCP mostly show up in data fitting problems such as least-squares and its variants." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Second order cone programs (SOCP)\n", "\n", "$$\n", "\\begin{array}{ll}\n", "\\underset{x\\in\\mathbf{R}^{d}}{\\mbox{minimize}} & c^{\\top}x\\\\\n", "\\mbox{subject to} & \\|A_{i}x+b_{i}\\|_{2}\\leq q_{i}^{\\top}x+d_{i},\\quad i=1,\\ldots,p,\\\\\n", " & Cx\\leq f.\n", "\\end{array}\n", "$$\n", "\n", "* SOCP is a generalization of LP and QCP\n", "* Allows for linear combinations of variables to be constrained inside a special convex set, called a second-order cone\n", "\n", "\"image-20220117142714899\"\n", "\n", "* Shows up in: geometry problems, approximation problems, chance-constrained linear optimization problems" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Semidefinite programs (SDP)\n", "\n", "Recall that LP in standard form is:\n", "$$\n", "\\begin{array}{ll}\n", "\\underset{x\\in\\mathbf{R}^{d}}{\\mbox{minimize}} & c^{\\top}x\\\\\n", "\\mbox{subject to} & a_{i}^{\\top}x=b_{i},\\quad i=1,\\ldots,m,\\\\\n", " & x\\succeq0.\n", "\\end{array}\n", "$$\n", "An SDP is a generalization of LPs. Standard form of an SDP is:\n", "$$\n", "\\begin{array}{ll}\n", "\\underset{X\\in\\mathbf{R}^{d\\times d}}{\\mbox{minimize}} & \\mathbf{tr}(CX)\\\\\n", "\\mbox{subject to} & \\mathbf{tr}(A_{i}X)=b_{i},\\quad i=1,\\ldots,m,\\\\\n", " & X\\succeq0.\n", "\\end{array}\n", "$$\n", "\n", "* Among all convex models, SDP is the most useful in my opinion.\n", "* Shows up in:\n", " * sophisticated relaxations (approximations) of non-convex problems,\n", " * in control design for linear dynamical systems,\n", " * in system identification,\n", " * in algebraic geometry, and\n", " * in matrix completion problems." ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "## Mixed-integer linear programs (MILP) (Session 2)\n", "\n", "$$\n", "\\begin{array}{ll}\n", "\\underset{x,y}{\\mbox{minimize}} & c^{\\top}x+d^{\\top}y\\\\\n", "\\mbox{subject to} & Ax+By=b,\\\\\n", " & Cx+Dy\\geq f,\\\\\n", " & x\\succeq0,\\\\\n", " & x\\in\\mathbf{R}^{d},\\\\\n", " & y\\in \\mathbf{Z}^{n}.\n", "\\end{array}\n", "$$\n", "\n", "* Nonconvex, but \"practically tractable\" in many situations\n", "* MILP solvers have gained a speed-up factor of 2 trillion in the last 40 years!\n", "* Wide modeling capability\n", "* Usually shows up in situations where\n", " * modeling do/don't do decisions\n", " * logical conditions\n", " * implications (if something happens do this)\n", " * either/or" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Modeling and solving common convex models in `Julia`\n", "\n", "### Example of QCP: portfolio optimization\n", "\n", "\n", "Assume that we have a portfolio with $n$ assets at the beginning of time period $t$.\n", "\n", "Given some forecasts on risks and expected returns we try to find the optimal portfolio that rebalances the portfolio to achieve a good balance between expected risk (variance) $x^\\top \\Sigma x$ and returns $\\mu^\\top x$​​.\n", "\n", "In it's simplest form we want to solve:\n", "$$\n", "\\begin{array}{ll} \\text{minimize} & \t \\gamma (x^\\top \\Sigma x) -\\mu^\\top x \\\\\n", "\\text{subject to} & 1^\\top x = d + 1^\\top x^0 \\\\\n", " & x \\geq 0,\n", " \\end{array}\n", "$$\n", "with variable $x \\in \\mathbf{R}^n$, $\\mu$ forecasted (expected) returns, $\\gamma > 0 $ risk aversion parameter.\n", "\n", "* $x^0_i$ represents the initial investment in asset $i$ and $d$ represents the cash reserve.\n", "* The equality constraint tells us that the sum of the new allocation vector $x$​ has to equal the initial allocation plus the cash reserve\n", "* the covariance matrix of our risk model is given by $\\Sigma \\in \\mathbf{S}_+^n$​." ] }, { "cell_type": "code", "execution_count": 49, "metadata": {}, "outputs": [], "source": [ "# Load the packages \n", "using MosekTools, Mosek, Random, Convex, JuMP, Suppressor, Images, DelimitedFiles\n", "# using COSMO, Random, Convex, JuMP, Suppressor, Images, DelimitedFiles" ] }, { "cell_type": "code", "execution_count": 50, "metadata": {}, "outputs": [], "source": [ "# Load the data\n", "include(\"img//portfolio_data.jl\");" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Solving the problem using `Convex.jl`\n", "\n", "To solve this problem, we will use `Convex.jl`, which is a specialized `Julia` package for solving convex optimization problem." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Declare the variable $x$." ] }, { "cell_type": "code", "execution_count": 52, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Variable\n", "size: (50, 1)\n", "sign: real\n", "vexity: affine\n", "id: 163…310" ] }, "execution_count": 52, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x = Variable(n)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let us now the define the objective $ \\gamma (x^\\top \\Sigma x) -\\mu^\\top x $." ] }, { "cell_type": "code", "execution_count": 55, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "+ (convex; real)\n", "├─ * (convex; positive)\n", "│ ├─ 1.0\n", "│ └─ * (convex; positive)\n", "│ ├─ 1\n", "│ └─ qol_elem (convex; positive)\n", "│ ├─ …\n", "│ └─ …\n", "└─ - (affine; real)\n", " └─ sum (affine; real)\n", " └─ .* (affine; real)\n", " ├─ …\n", " └─ …" ] }, "execution_count": 55, "metadata": {}, "output_type": "execute_result" } ], "source": [ "objective = γ*quadform(x,Σ) - dot(x,μ) " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note, the list of all the functions `Convex.jl` can model directly is available at [https://jump.dev/Convex.jl/stable/operations/#Linear-Program-Representable-Functions](https://jump.dev/Convex.jl/stable/operations/#Linear-Program-Representable-Functions).\n", "\n", "Add the constraint $1^\\top x = d + 1^\\top x^0$." ] }, { "cell_type": "code", "execution_count": 56, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "== constraint (affine)\n", "├─ sum (affine; real)\n", "│ └─ 50-element real variable (id: 163…310)\n", "└─ 1" ] }, "execution_count": 56, "metadata": {}, "output_type": "execute_result" } ], "source": [ "constraint_linear = ( sum(x) == d + sum(x0) )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now add the positivity constraint." ] }, { "cell_type": "code", "execution_count": 57, "metadata": {}, "outputs": [ { "data": { "text/plain": [ ">= constraint (affine)\n", "├─ 50-element real variable (id: 163…310)\n", "└─ 0" ] }, "execution_count": 57, "metadata": {}, "output_type": "execute_result" } ], "source": [ "constraint_pos = ( x >= 0 )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, combine everything in a `problem`." ] }, { "cell_type": "code", "execution_count": 58, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "minimize\n", "└─ + (convex; real)\n", " ├─ * (convex; positive)\n", " │ ├─ 1.0\n", " │ └─ * (convex; positive)\n", " │ ├─ …\n", " │ └─ …\n", " └─ - (affine; real)\n", " └─ sum (affine; real)\n", " └─ …\n", "subject to\n", "├─ == constraint (affine)\n", "│ ├─ sum (affine; real)\n", "│ │ └─ 50-element real variable (id: 163…310)\n", "│ └─ 1\n", "└─ >= constraint (affine)\n", " ├─ 50-element real variable (id: 163…310)\n", " └─ 0\n", "\n", "status: `solve!` not called yet" ] }, "execution_count": 58, "metadata": {}, "output_type": "execute_result" } ], "source": [ "problem = minimize(objective, [constraint_linear, constraint_pos])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now time to solve the problem!" ] }, { "cell_type": "code", "execution_count": 60, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Problem\n", " Name : \n", " Objective sense : min \n", " Type : CONIC (conic optimization problem)\n", " Constraints : 107 \n", " Cones : 2 \n", " Scalar variables : 107 \n", " Matrix variables : 0 \n", " Integer variables : 0 \n", "\n", "Optimizer started.\n", "Presolve started.\n", "Linear dependency checker started.\n", "Linear dependency checker terminated.\n", "Eliminator started.\n", "Freed constraints in eliminator : 0\n", "Eliminator terminated.\n", "Eliminator - tries : 1 time : 0.00 \n", "Lin. dep. - tries : 1 time : 0.00 \n", "Lin. dep. - number : 0 \n", "Presolve terminated. Time: 0.00 \n", "Problem\n", " Name : \n", " Objective sense : min \n", " Type : CONIC (conic optimization problem)\n", " Constraints : 107 \n", " Cones : 2 \n", " Scalar variables : 107 \n", " Matrix variables : 0 \n", " Integer variables : 0 \n", "\n", "Optimizer - threads : 48 \n", "Optimizer - solved problem : the primal \n", "Optimizer - Constraints : 53\n", "Optimizer - Cones : 2\n", "Optimizer - Scalar variables : 104 conic : 54 \n", "Optimizer - Semi-definite variables: 0 scalarized : 0 \n", "Factor - setup time : 0.00 dense det. time : 0.00 \n", "Factor - ML order time : 0.00 GP order time : 0.00 \n", "Factor - nonzeros before factor : 1379 after factor : 1380 \n", "Factor - dense dim. : 0 flops : 1.84e+05 \n", "ITE PFEAS DFEAS GFEAS PRSTATUS POBJ DOBJ MU TIME \n", "0 1.0e+00 1.0e+00 2.0e+00 0.00e+00 0.000000000e+00 -1.000000000e+00 1.0e+00 0.00 \n", "1 2.9e-01 2.9e-01 1.3e-01 9.55e-01 -1.052337346e-01 -3.529023776e-01 2.9e-01 0.00 \n", "2 7.0e-02 7.0e-02 1.9e-02 3.28e+00 -3.462222178e-02 -4.832572741e-02 7.0e-02 0.00 \n", "3 1.3e-02 1.3e-02 1.3e-03 7.22e-01 2.043785564e-02 1.598816495e-02 1.3e-02 0.00 \n", "4 8.7e-04 8.7e-04 1.4e-05 9.90e-01 3.921134167e-02 3.878335008e-02 8.7e-04 0.00 \n", "5 1.4e-04 1.4e-04 9.0e-07 1.02e+00 4.058692192e-02 4.052022577e-02 1.4e-04 0.00 \n", "6 1.3e-07 1.3e-07 1.5e-11 1.00e+00 4.085428624e-02 4.085421395e-02 1.3e-07 0.00 \n", "7 1.3e-10 1.1e-10 1.4e-16 1.00e+00 4.085438091e-02 4.085438088e-02 7.2e-11 0.00 \n", "Optimizer terminated. Time: 0.00 \n", "\n" ] } ], "source": [ "solve!(problem, Mosek.Optimizer)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Get the solution now." ] }, { "cell_type": "code", "execution_count": 61, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "50-element Vector{Float64}:\n", " 0.027228140397089203\n", " 0.03226496898357304\n", " 0.040521154749669115\n", " 5.045416303584319e-12\n", " 0.021044660297784817\n", " -1.904956297667087e-11\n", " 0.050557306144882554\n", " 0.05394663344893841\n", " -1.408664747030976e-11\n", " 0.04398393358377368\n", " 0.019242404142878675\n", " 0.03770907944017934\n", " 0.03056528224454937\n", " ⋮\n", " 0.02068750776254753\n", " 0.06466324439231913\n", " 0.02198422135523888\n", " 0.004512499567966097\n", " -1.9802434781250454e-11\n", " 0.024926436958510417\n", " 0.0301004341433809\n", " 0.01383265358539451\n", " 5.481331048553427e-11\n", " 0.02848213497329782\n", " 0.043080956668940386\n", " -1.835093072354754e-11" ] }, "execution_count": 61, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x_sol = Convex.evaluate(x)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Get the optimal value." ] }, { "cell_type": "code", "execution_count": 62, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "0.040854380927766734" ] }, "execution_count": 62, "metadata": {}, "output_type": "execute_result" } ], "source": [ "p_star = Convex.evaluate(objective)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "So, combining everything together, the code to solve the portfolio optimization problem using `Convex.jl` would be:" ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Problem\n", " Name : \n", " Objective sense : min \n", " Type : CONIC (conic optimization problem)\n", " Constraints : 107 \n", " Cones : 2 \n", " Scalar variables : 107 \n", " Matrix variables : 0 \n", " Integer variables : 0 \n", "\n", "Optimizer started.\n", "Presolve started.\n", "Linear dependency checker started.\n", "Linear dependency checker terminated.\n", "Eliminator started.\n", "Freed constraints in eliminator : 0\n", "Eliminator terminated.\n", "Eliminator - tries : 1 time : 0.00 \n", "Lin. dep. - tries : 1 time : 0.00 \n", "Lin. dep. - number : 0 \n", "Presolve terminated. Time: 0.00 \n", "Problem\n", " Name : \n", " Objective sense : min \n", " Type : CONIC (conic optimization problem)\n", " Constraints : 107 \n", " Cones : 2 \n", " Scalar variables : 107 \n", " Matrix variables : 0 \n", " Integer variables : 0 \n", "\n", "Optimizer - threads : 48 \n", "Optimizer - solved problem : the primal \n", "Optimizer - Constraints : 53\n", "Optimizer - Cones : 2\n", "Optimizer - Scalar variables : 104 conic : 54 \n", "Optimizer - Semi-definite variables: 0 scalarized : 0 \n", "Factor - setup time : 0.00 dense det. time : 0.00 \n", "Factor - ML order time : 0.00 GP order time : 0.00 \n", "Factor - nonzeros before factor : 1379 after factor : 1380 \n", "Factor - dense dim. : 0 flops : 1.84e+05 \n", "ITE PFEAS DFEAS GFEAS PRSTATUS POBJ DOBJ MU TIME \n", "0 1.0e+00 1.0e+00 2.0e+00 0.00e+00 0.000000000e+00 -1.000000000e+00 1.0e+00 0.00 \n", "1 2.9e-01 2.9e-01 1.3e-01 9.55e-01 -1.052337346e-01 -3.529023776e-01 2.9e-01 0.00 \n", "2 7.0e-02 7.0e-02 1.9e-02 3.28e+00 -3.462222178e-02 -4.832572741e-02 7.0e-02 0.00 \n", "3 1.3e-02 1.3e-02 1.3e-03 7.22e-01 2.043785564e-02 1.598816495e-02 1.3e-02 0.00 \n", "4 8.7e-04 8.7e-04 1.4e-05 9.90e-01 3.921134167e-02 3.878335008e-02 8.7e-04 0.00 \n", "5 1.4e-04 1.4e-04 9.0e-07 1.02e+00 4.058692192e-02 4.052022577e-02 1.4e-04 0.00 \n", "6 1.3e-07 1.3e-07 1.5e-11 1.00e+00 4.085428624e-02 4.085421395e-02 1.3e-07 0.00 \n", "7 1.3e-10 1.1e-10 1.4e-16 1.00e+00 4.085438091e-02 4.085438088e-02 7.2e-11 0.01 \n", "Optimizer terminated. Time: 0.01 \n", "\n" ] } ], "source": [ "## Define the problem\n", "x = Variable(n);\n", "objective = γ*quadform(x,Σ) - dot(x,μ);\n", "constraint_linear = ( sum(x) == d + sum(x0) );\n", "constraint_pos = ( x >= 0 );\n", "problem = minimize(objective, [constraint_linear, constraint_pos]);\n", "\n", "## Solve the problem\n", "solve!(problem, Mosek.Optimizer);\n", "\n", "## Extract the solution\n", "x_sol = Convex.evaluate(x);\n", "p_star = Convex.evaluate(objective);" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Solving the problem using `JuMP`\n", "\n", "There is nothing special solving a problem via `Convex.jl` over `JuMP`. We could solve the same problem using `JuMP` with the following code." ] }, { "cell_type": "code", "execution_count": 63, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Problem\n", " Name : \n", " Objective sense : min \n", " Type : CONIC (conic optimization problem)\n", " Constraints : 53 \n", " Cones : 1 \n", " Scalar variables : 103 \n", " Matrix variables : 0 \n", " Integer variables : 0 \n", "\n", "Optimizer started.\n", "Presolve started.\n", "Linear dependency checker started.\n", "Linear dependency checker terminated.\n", "Eliminator started.\n", "Freed constraints in eliminator : 0\n", "Eliminator terminated.\n", "Eliminator - tries : 1 time : 0.00 \n", "Lin. dep. - tries : 1 time : 0.00 \n", "Lin. dep. - number : 0 \n", "Presolve terminated. Time: 0.00 \n", "Problem\n", " Name : \n", " Objective sense : min \n", " Type : CONIC (conic optimization problem)\n", " Constraints : 53 \n", " Cones : 1 \n", " Scalar variables : 103 \n", " Matrix variables : 0 \n", " Integer variables : 0 \n", "\n", "Optimizer - threads : 48 \n", "Optimizer - solved problem : the primal \n", "Optimizer - Constraints : 50\n", "Optimizer - Cones : 1\n", "Optimizer - Scalar variables : 101 conic : 52 \n", "Optimizer - Semi-definite variables: 0 scalarized : 0 \n", "Factor - setup time : 0.00 dense det. time : 0.00 \n", "Factor - ML order time : 0.00 GP order time : 0.00 \n", "Factor - nonzeros before factor : 1275 after factor : 1275 \n", "Factor - dense dim. : 0 flops : 9.49e+04 \n", "ITE PFEAS DFEAS GFEAS PRSTATUS POBJ DOBJ MU TIME \n", "0 1.0e+00 2.9e-01 2.4e+00 0.00e+00 7.071067812e-01 -7.071067812e-01 1.0e+00 0.00 \n", "1 2.9e-01 8.6e-02 1.5e-01 1.20e+00 -2.906081789e-02 -4.249358754e-01 2.9e-01 0.00 \n", "2 1.4e-01 4.0e-02 5.5e-02 4.49e+00 -2.640651336e-02 -6.905590568e-02 1.4e-01 0.00 \n", "3 3.0e-02 8.9e-03 5.9e-03 8.51e-01 7.377461763e-03 -4.116575695e-03 3.0e-02 0.00 \n", "4 4.7e-03 1.4e-03 2.2e-04 8.52e-01 3.494661566e-02 3.234045354e-02 4.7e-03 0.00 \n", "5 4.3e-04 1.3e-04 5.5e-06 1.03e+00 4.026643434e-02 4.002824506e-02 4.3e-04 0.00 \n", "6 5.1e-05 1.5e-05 2.3e-07 1.01e+00 4.077948136e-02 4.075136364e-02 5.1e-05 0.00 \n", "7 9.0e-10 2.1e-10 1.4e-14 1.00e+00 4.085438030e-02 4.085437979e-02 9.0e-10 0.00 \n", "Optimizer terminated. Time: 0.00 \n", "\n" ] }, { "data": { "text/plain": [ "0.040854380289201415" ] }, "execution_count": 63, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Define the problem\n", "using JuMP\n", "model = Model(Mosek.Optimizer)\n", "@variable(model, xs[1:n] >= 0)\n", "@objective(model, Min, γ * xs'*Σ*xs - μ' * xs);\n", "@constraint(model, sum(xs) .== d + sum(x0))\n", "\n", "## Solve the problem\n", "optimize!(model)\n", "\n", "## Get the optimal solution\n", "x_sol_JuMP = value.(xs)\n", "p_star_JuMP = objective_value(model)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Because we used the same solver, we hope to get the same optimal solution." ] }, { "cell_type": "code", "execution_count": 64, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "2.184720444565889e-8" ] }, "execution_count": 64, "metadata": {}, "output_type": "execute_result" } ], "source": [ "norm(x_sol - x_sol_JuMP)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### When to use `JuMP` vs `Convex.jl`?\n", "\n", "* Usually `JuMP`'s model creation time is somewhat faster than `Convex.jl`, so if you are working on a problem, where performance is key, then using `JuMP` is a good idea.\n", "* If you are in the prototyping phase of your model and you feel that you are working with a lot of convex-ish functions, then working with `Convex.jl` might be a good idea.\n", " * For example, suppose, in the last problem you are interested in finding a sparse portfolio with the risk being less than some bound. Within convex optimization framework, one way doing this is by minimizing the $\\ell_1$​ norm of $x$​, i.e., consider the modified objective $\\mu^\\top x + \\gamma \\| x\\|_1$​ and the additional constraint $x^\\top \\Sigma x \\leq \\delta$. If we want to do this modification via `JuMP`, then the right approach is reformulate the $\\ell_1$​ norm via epigraph etc, but `Convex.jl` will take this term as it is.\n", "\n", "\n", "$\\|x\\|_1$ = `norm(x,1)`\n", "\n", "$x^\\top \\Sigma x$ = `quadform(x_sparse,Σ)`" ] }, { "cell_type": "code", "execution_count": 65, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Problem\n", " Name : \n", " Objective sense : min \n", " Type : CONIC (conic optimization problem)\n", " Constraints : 208 \n", " Cones : 2 \n", " Scalar variables : 157 \n", " Matrix variables : 0 \n", " Integer variables : 0 \n", "\n", "Optimizer started.\n", "Presolve started.\n", "Linear dependency checker started.\n", "Linear dependency checker terminated.\n", "Eliminator started.\n", "Freed constraints in eliminator : 100\n", "Eliminator terminated.\n", "Eliminator started.\n", "Freed constraints in eliminator : 0\n", "Eliminator terminated.\n", "Eliminator - tries : 2 time : 0.00 \n", "Lin. dep. - tries : 1 time : 0.00 \n", "Lin. dep. - number : 0 \n", "Presolve terminated. Time: 0.00 \n", "Problem\n", " Name : \n", " Objective sense : min \n", " Type : CONIC (conic optimization problem)\n", " Constraints : 208 \n", " Cones : 2 \n", " Scalar variables : 157 \n", " Matrix variables : 0 \n", " Integer variables : 0 \n", "\n", "Optimizer - threads : 48 \n", "Optimizer - solved problem : the primal \n", "Optimizer - Constraints : 53\n", "Optimizer - Cones : 2\n", "Optimizer - Scalar variables : 104 conic : 54 \n", "Optimizer - Semi-definite variables: 0 scalarized : 0 \n", "Factor - setup time : 0.00 dense det. time : 0.00 \n", "Factor - ML order time : 0.00 GP order time : 0.00 \n", "Factor - nonzeros before factor : 1379 after factor : 1380 \n", "Factor - dense dim. : 0 flops : 1.84e+05 \n", "ITE PFEAS DFEAS GFEAS PRSTATUS POBJ DOBJ MU TIME \n", "0 1.0e+00 1.0e+00 1.0e+00 0.00e+00 0.000000000e+00 0.000000000e+00 1.0e+00 0.00 \n", "1 3.2e-01 3.2e-01 7.7e-02 1.05e+00 7.366400887e-01 5.964125089e-01 3.2e-01 0.00 \n", "2 1.5e-01 1.5e-01 1.2e-02 2.97e+00 8.880375655e-01 8.573569888e-01 1.5e-01 0.00 \n", "3 6.2e-02 6.2e-02 3.0e-03 2.76e+00 9.122172687e-01 9.063434088e-01 6.2e-02 0.00 \n", "4 2.8e-02 2.8e-02 9.3e-04 1.29e+00 9.079463544e-01 9.054055890e-01 2.8e-02 0.00 \n", "5 6.1e-03 6.1e-03 8.9e-05 1.14e+00 9.062916966e-01 9.057747609e-01 6.1e-03 0.00 \n", "6 6.9e-04 6.9e-04 3.3e-06 1.05e+00 9.059876688e-01 9.059299167e-01 6.9e-04 0.01 \n", "7 5.5e-05 5.5e-05 7.0e-08 1.01e+00 9.059030996e-01 9.058983290e-01 5.5e-05 0.01 \n", "8 1.7e-06 1.7e-06 3.3e-10 1.00e+00 9.058978757e-01 9.058977217e-01 1.7e-06 0.01 \n", "9 7.2e-08 7.2e-08 2.8e-12 1.00e+00 9.058977525e-01 9.058977456e-01 7.2e-08 0.01 \n", "Optimizer terminated. Time: 0.01 \n", "\n" ] }, { "data": { "text/plain": [ "0.9058977602024589" ] }, "execution_count": 65, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Define the problem (student)\n", "δ = 0.5\n", "x_sparse = Variable(n)\n", "objective_sparse = - dot(x_sparse,μ) + γ*norm(x_sparse,1)\n", "constraint_linear = ( sum(x_sparse) == d + sum(x0) )\n", "constraint_pos = ( x_sparse >= 0 )\n", "constraint_additional = ( quadform(x_sparse,Σ) <= δ)\n", "problem_sparse = minimize(objective_sparse, [constraint_linear, constraint_pos, constraint_additional])\n", "\n", "## Solve the problem\n", "solve!(problem_sparse, Mosek.Optimizer)\n", "\n", "## Extract the solution\n", "x_sol_sparse = Convex.evaluate(x_sparse)\n", "p_star_sparse = Convex.evaluate(objective_sparse)" ] }, { "cell_type": "code", "execution_count": 66, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "Gnuplot\n", "Produced by GNUPLOT 5.2 patchlevel 2 \n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t \n", "\t \n", "\t\n", "\t\n", "\t \n", "\t \n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t0.00\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t0.02\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t0.04\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t0.06\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t0.08\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t0\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t10\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t20\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t30\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t40\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t50\n", "\t\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\t\n", "\t\tx[k]\n", "\t\n", "\n", "\n", "\t\n", "\t\tk\n", "\t\n", "\n", "\n", "\t\n", "\t\t\n", "\t\n", "\n", "\n", "\n", "\n", "\t\n", "\tx original\n", "\n", "\t\n", "\t\n", "\tx sparse\n", "\n", "\t\n", "\t\n", "\t\n", "\t\t\n", "\t\n", "\n", "\t\n", "\tx original\n", "\n", "\t\n", "\t\tx original\n", "\t\n", "\n", "\n", "\t\n", "\t\n", "\tx sparse\n", "\n", "\t\n", "\t\tx sparse\n", "\t\n", "\n", "\n", "\t\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n" ] }, "execution_count": 66, "metadata": {}, "output_type": "execute_result" }, { "name": "stderr", "output_type": "stream", "text": [ "┌ Warning: Gnuplot returned an error message:\n", "│ \n", "│ gnuplot> set term svg size 600,400 font 'sans-serif,16' background rgb '#00ffffff' fontscale 1.0 lw 1.0 dl 1.0 ps 1.0\n", "│ ^\n", "│ line 0: unrecognized terminal option\n", "│ \n", "└ @ Gaston /home/gridsan/sdgupta/.julia/packages/Gaston/ctAQy/src/gaston_llplot.jl:182\n" ] } ], "source": [ "using Plots, Suppressor\n", "gaston()\n", "# inspectdr()\n", "plot([1:n],[x_sol_JuMP x_sol_sparse], xlabel = \"k\", ylabel = \"x[k]\", label = [ \"x original\" \"x sparse\"])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Example of SOCP: time series analysis\n", "\n", "A time series is a sequence of data points, each associated with a time. In our example, we will work with a time series of daily temperatures in the city of Melbourne, Australia over a period of a few years. Let $\\tau$ be the vector of the time series, and $\\tau_i$ denote the temperature in Melbourne on day $i$. Here is a picture of the time series:" ] }, { "cell_type": "code", "execution_count": 67, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "Gnuplot\n", "Produced by GNUPLOT 5.2 patchlevel 2 \n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t \n", "\t \n", "\t\n", "\t\n", "\t \n", "\t \n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t0\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t5\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t10\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t15\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t20\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t25\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t0\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t365\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t730\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t1095\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t1460\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t1825\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t2190\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t2555\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t2920\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t3285\n", "\t\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\t\n", "\t\tTemperature (°C)\n", "\t\n", "\n", "\n", "\t\n", "\t\tTime (days)\n", "\t\n", "\n", "\n", "\t\n", "\t\t\n", "\t\n", "\n", "\n", "\n", "\n", "\t\n", "\tdata\n", "\n", "\t\n", "\t\n", "\t\n", "\t\t\n", "\t\n", "\n", "\t\n", "\tdata\n", "\n", "\t\n", "\t\tdata\n", "\t\n", "\n", "\n", "\t\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n" ] }, "execution_count": 67, "metadata": {}, "output_type": "execute_result" }, { "name": "stderr", "output_type": "stream", "text": [ "┌ Warning: Gnuplot returned an error message:\n", "│ \n", "│ gnuplot> set term svg size 600,400 font 'sans-serif,16' background rgb '#00ffffff' fontscale 1.0 lw 1.0 dl 1.0 ps 1.0\n", "│ ^\n", "│ line 0: unrecognized terminal option\n", "│ \n", "└ @ Gaston /home/gridsan/sdgupta/.julia/packages/Gaston/ctAQy/src/gaston_llplot.jl:182\n" ] } ], "source": [ "using DelimitedFiles\n", "τ = readdlm(\"img//melbourne_temps.txt\", ',')\n", "n = size(τ, 1)\n", "plot(1:n, τ[1:n], ylabel=\"Temperature (°C)\", label=\"data\", xlabel = \"Time (days)\", xticks=0:365:n)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A simple way to model this time series would be to find a smooth curve that approximates the yearly ups and downs.\n", "We can represent this model as a vector $x$ where $x_i$ denotes the predicted temperature on the $i$-th day.\n", "To force this trend to repeat yearly, we simply want to impose the constraint\n", "\n", "$$\n", " x_i = x_{i + 365}\n", "$$\n", "\n", "for each applicable $i$.\n", "\n", "We also want our model to have two more properties:\n", "\n", "- The first is that the temperature on each day in our model should be relatively close to the actual temperature of that day and equal if possible.\n", "- The second is that our model needs to be smooth, so the change in temperature from day to day should be relatively small. The following objective would capture both properties:\n", "\n", "$$\n", "\\sum_{i=1}^{n}|x_{i}-\\tau_{i}|+\\lambda\\sqrt{\\sum_{i=2}^{n}(x_{i}-x_{i-1})^{2}}=\\|x-\\tau\\|_{1}+\\lambda\\|Ax\\|_{2},\n", "$$\n", " where $A$ is a matrix of size $(n-1)\\times n$ with $A_{i,i}=-1,A_{i,i+1}=1$\n", "for $i=1,\\ldots,n-1$ and the rest of the elements being zero. So,\n", "the optimization problem we want to solve is: \n", "\n", "$$\n", "\\begin{array}{ll}\n", "\\underset{x\\in\\mathbf{R}^{d}}{\\mbox{minimize}} & \\|x-\\tau\\|_{1}+\\lambda\\|Ax\\|_{2}\\\\\n", "\\mbox{subject to} & x_{i}=x_{i+365},\\quad i=1,\\ldots,n.\n", "\\end{array}\n", "$$\n", "This is an SOCP, because it can be written as: \n", "$$\n", "\\begin{array}{ll}\n", "\\underset{x,u,t}{\\mbox{minimize}} & \\sum_{i=1}^{n}u_{i}+\\lambda t\\\\\n", "\\mbox{subject to} & x_{i}=x_{i+365},\\quad i=1,\\ldots,n, \\\\\n", "& \\|Ax\\|_{2}\\leq t,\\\\\n", " & \\vert x_{i}-\\tau_{i}\\vert\\leq u_{i},\\quad i=1,\\ldots,n.\n", "\\end{array}\n", "$$\n", "\n", "Here, $\\lambda$ is the smoothing parameter. The larger $\\lambda$ is, the smoother our model will be.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "We will solve this problem using `Convex.jl`, because it would allow us to input the problem directly without manually converting into the SOCP format. " ] }, { "cell_type": "code", "execution_count": 75, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Variable\n", "size: (3648, 1)\n", "sign: real\n", "vexity: affine\n", "id: 121…269" ] }, "execution_count": 75, "metadata": {}, "output_type": "execute_result" } ], "source": [ "## Solution using Convex.jl\n", "x = Variable(n)" ] }, { "cell_type": "code", "execution_count": 76, "metadata": {}, "outputs": [], "source": [ "eq_constraints = [ x[i] == x[i - 365] for i in 365 + 1 : n ];" ] }, { "cell_type": "code", "execution_count": 77, "metadata": {}, "outputs": [], "source": [ "λ = 100 # smoothing parameter\n", "# Define the matrix A\n", "A = zeros(n-1,n)\n", "for i in 1:n-1\n", " A[i,i] = -1\n", " A[i,i+1] = 1\n", "end" ] }, { "cell_type": "code", "execution_count": 78, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "+ (convex; positive)\n", "├─ sum (convex; positive)\n", "│ └─ abs (convex; positive)\n", "│ └─ + (affine; real)\n", "│ ├─ …\n", "│ └─ …\n", "└─ * (convex; positive)\n", " ├─ 100\n", " └─ norm2 (convex; positive)\n", " └─ * (affine; real)\n", " ├─ …\n", " └─ …" ] }, "execution_count": 78, "metadata": {}, "output_type": "execute_result" } ], "source": [ "smooth_objective = norm(x-τ,1) + λ*norm(A*x,2)" ] }, { "cell_type": "code", "execution_count": 79, "metadata": {}, "outputs": [], "source": [ "smooth_problem = minimize(smooth_objective, eq_constraints);" ] }, { "cell_type": "code", "execution_count": 80, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Problem\n", " Name : \n", " Objective sense : min \n", " Type : CONIC (conic optimization problem)\n", " Constraints : 14228 \n", " Cones : 1 \n", " Scalar variables : 10946 \n", " Matrix variables : 0 \n", " Integer variables : 0 \n", "\n", "Optimizer started.\n", "Presolve started.\n", "Linear dependency checker started.\n", "Linear dependency checker terminated.\n", "Eliminator started.\n", "Freed constraints in eliminator : 6931\n", "Eliminator terminated.\n", "Eliminator started.\n", "Freed constraints in eliminator : 0\n", "Eliminator terminated.\n", "Eliminator - tries : 2 time : 0.00 \n", "Lin. dep. - tries : 1 time : 0.00 \n", "Lin. dep. - number : 0 \n", "Presolve terminated. Time: 0.02 \n", "Problem\n", " Name : \n", " Objective sense : min \n", " Type : CONIC (conic optimization problem)\n", " Constraints : 14228 \n", " Cones : 1 \n", " Scalar variables : 10946 \n", " Matrix variables : 0 \n", " Integer variables : 0 \n", "\n", "Optimizer - threads : 48 \n", "Optimizer - solved problem : the dual \n", "Optimizer - Constraints : 365\n", "Optimizer - Cones : 1\n", "Optimizer - Scalar variables : 7129 conic : 3648 \n", "Optimizer - Semi-definite variables: 0 scalarized : 0 \n", "Factor - setup time : 0.00 dense det. time : 0.00 \n", "Factor - ML order time : 0.00 GP order time : 0.00 \n", "Factor - nonzeros before factor : 1095 after factor : 1458 \n", "Factor - dense dim. : 1 flops : 2.15e+04 \n", "ITE PFEAS DFEAS GFEAS PRSTATUS POBJ DOBJ MU TIME \n", "0 1.1e+02 9.9e+01 1.0e+02 0.00e+00 1.560800000e+03 1.460800000e+03 1.0e+00 0.03 \n", "1 6.2e+01 5.9e+01 7.7e+01 -9.82e-01 2.730199769e+03 2.632087909e+03 5.9e-01 0.03 \n", "2 2.5e+01 2.3e+01 4.5e+01 -9.26e-01 6.736436307e+03 6.649247616e+03 2.3e-01 0.03 \n", "3 1.9e+01 1.8e+01 3.5e+01 -5.31e-01 7.737208735e+03 7.659354991e+03 1.8e-01 0.03 \n", "4 7.6e+00 7.1e+00 1.0e+01 -2.09e-01 1.012319530e+04 1.008440511e+04 7.2e-02 0.04 \n", "5 5.0e+00 4.7e+00 4.9e+00 9.67e-01 9.213769945e+03 9.189816268e+03 4.7e-02 0.04 \n", "6 1.6e+00 1.5e+00 7.9e-01 1.07e+00 8.455635415e+03 8.448656902e+03 1.5e-02 0.04 \n", "7 3.5e-01 3.3e-01 8.1e-02 1.05e+00 8.217712464e+03 8.216216265e+03 3.3e-03 0.04 \n", "8 8.6e-02 8.1e-02 1.0e-02 1.01e+00 8.174719566e+03 8.174348780e+03 8.2e-04 0.04 \n", "9 2.8e-02 2.7e-02 1.9e-03 1.00e+00 8.164215640e+03 8.164093597e+03 2.7e-04 0.04 \n", "10 6.5e-03 6.2e-03 2.2e-04 1.00e+00 8.159975775e+03 8.159947703e+03 6.2e-05 0.04 \n", "11 1.3e-03 1.2e-03 2.0e-05 1.00e+00 8.158895032e+03 8.158889346e+03 1.3e-05 0.05 \n", "12 1.6e-04 1.5e-04 8.4e-07 1.00e+00 8.158651808e+03 8.158651115e+03 1.5e-06 0.05 \n", "13 8.5e-06 8.0e-06 1.0e-08 1.00e+00 8.158619699e+03 8.158619662e+03 8.1e-08 0.05 \n", "14 7.6e-07 7.1e-07 2.7e-10 1.00e+00 8.158618093e+03 8.158618090e+03 7.2e-09 0.05 \n", "15 6.4e-08 6.1e-08 6.6e-12 1.00e+00 8.158617948e+03 8.158617948e+03 6.1e-10 0.05 \n", "Optimizer terminated. Time: 0.06 \n", "\n" ] } ], "source": [ "solve!(smooth_problem, Mosek.Optimizer)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's plot our smoothed time estimate vs the original data." ] }, { "cell_type": "code", "execution_count": 81, "metadata": {}, "outputs": [ { "data": { "image/svg+xml": [ "\n", "\n", "\n", "Gnuplot\n", "Produced by GNUPLOT 5.2 patchlevel 2 \n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t\n", "\t \n", "\t \n", "\t\n", "\t\n", "\t \n", "\t \n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t0\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t5\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t10\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t15\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t20\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t25\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t0\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t365\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t730\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t1095\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t1460\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t1825\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t2190\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t2555\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t2920\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\n", "\n", "\t\t\n", "\t\t3285\n", "\t\n", "\n", "\n", "\n", "\n", "\t\n", "\n", "\t\n", "\t\tTemperature (°C)\n", "\t\n", "\n", "\n", "\t\n", "\t\tTime (days)\n", "\t\n", "\n", "\n", "\t\n", "\t\t\n", "\t\n", "\n", "\n", "\n", "\n", "\t\n", "\tdata\n", "\n", "\t\n", "\t\n", "\tsmooth fit\n", "\n", "\n", "\n", "\t\n", "\t\n", "\t\n", "\t\t\n", "\t\n", "\n", "\n", "\n", "\t\n", "\tdata\n", "\n", "\t\n", "\t\tdata\n", "\t\n", "\n", "\n", "\t\n", "\t\n", "\tsmooth fit\n", "\n", "\n", "\n", "\t\n", "\t\tsmooth fit\n", "\t\n", "\n", "\n", "\t\n", "\t\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n" ] }, "execution_count": 81, "metadata": {}, "output_type": "execute_result" }, { "name": "stderr", "output_type": "stream", "text": [ "┌ Warning: Gnuplot returned an error message:\n", "│ \n", "│ gnuplot> set term svg size 600,400 font 'sans-serif,16' background rgb '#00ffffff' fontscale 1.0 lw 1.0 dl 1.0 ps 1.0\n", "│ ^\n", "│ line 0: unrecognized terminal option\n", "│ \n", "└ @ Gaston /home/gridsan/sdgupta/.julia/packages/Gaston/ctAQy/src/gaston_llplot.jl:182\n" ] } ], "source": [ "# Plot smooth fit\n", "plot(1:n, τ[1:n], label=\"data\")\n", "plot!(1:n, Convex.evaluate(x)[1:n], linewidth=2, label=\"smooth fit\", ylabel=\"Temperature (°C)\", xticks=0:365:n, xlabel=\"Time (days)\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Matrix completion problem: how to reconstruct a distorted image\n", "\n", "Suppose we are given a noisy image, and we want to clean the image up. In countless movies and tv shows, we have seen our hero figuring out who the killer is by polishing a very noisy image in seconds. In reality, this takes a while, and usually is done by solving an SDP. \n", "\n", "Let's take a look at an example. Suppose we are given this very noisy image. This image is the noisy version of a test image widely used in the field of image processing since 1973. See the wikipedia entry [here](https://en.wikipedia.org/wiki/Lenna) for more information." ] }, { "cell_type": "code", "execution_count": 82, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAIAAAACACAAAAADmVT4XAAAABGdBTUEAALGPC/xhBQAAAAFzUkdCAK7OHOkAAAAgY0hSTQAAeiYAAICEAAD6AAAAgOgAAHUwAADqYAAAOpgAABdwnLpRPAAAIABJREFUeAEdwQVgVefBgOH3nHtuEryydtZu67rStYQQJTgUh+LBEyDu7kRvlLi7Awnu7sUJREmAtrB2f9euq6yCRu495/vDnkeBHZt5aadi5kRZ4OYdPjqpHIJKgXhdappIStOSIV2Og3RZxLONraQmpUsigUxNiic9ISUZUoXAmIF/Bf9TJUteUO1T7VOnM5m/P3Lcg9ZWs2G5bNWpEpmJaZBEalQuTUqT2FznWeMNm/ayv39T5A+btWo/8Ff9KyCDJAEimZRkCTKFiEvPQMSRrqYmQFpiXGpSpkmFlOQUkcyQsEIMBqDKt9qbGuEDeNZ7ccXxoXitxj83RtoGcZmJIiE9ITWJXLimmMAT72rJu3kdNOVGuQjNp9K3ShPVIYo+C5KANCFQdYh4SEgXEmkJGNISUxLTEiEujZRk0PifwiDTs6xYmow1cj0edTXegEe1z7eoyl+KQtFyt4IgLT6DdEhN8rlbr3hS51mvad7bXfai9kXRjAf+FX5aNeTHkCBhMCSlJaWkaQIJ0hNEIkhgSE0T6DLjSCUpBUjBIAkD4QUExBZbCEWg24JnHVKNBvyK6YN/jI9+lss2YOu2DEhKTU2n57mNUucp1YPUuIV1Lc41ud9/vxMhqMRHksiOS8dgIC0xXSSmQ1xmnEBkxgHpCUnpiSBDEiQDBpkkQI4YNBJSoXnQJO/UVKnOs9a3SlSqTX2f/e3THF+GxGVKDEkl8zNugeKJR4M7NEGL83ZjVCwuoibAt8qn2g8QIEgVaEmpCWlpiSoijbg0ElJUUCFuW2YcpCWSLAwYDAbi1awgKfQNIbbLm7ZvgTpPvKp8q5AfT+ThWKogSs5EgEE2I5YhdkqtV6Nbg3uD6+4NatOWOgZNu918yqHaW9LCzLeRJCUnQVIKaMkkgpSQphoSVQmSSU4xSqRJanIKqUmyJhkMZIUV+uvigR07N9EouUKtVi3wLwaNR70Xy3Pj4qQM0MzEfax6xisdihcgNSlsQHVDlx8BEv6Iyhq/knARlykMvJQMyakkkZROogFVkkgSaYIkSExLxkCy0JBIMsVkI+dDtX7zdhrdaHBHkmWPKkIK3xp4qHtvlV+lSUccmYq43zK+h14clQbccOMltwZ3IcK/lxq8K/wF3lBAXFxaUqrBkJxCKklJgEEDkSC0bYmkoZGSDCJFMySLVEhOAcLV4shRyT61DaLBjQZBPZ7gW6UNrrk0+4uRjPTPgXgZKR7MsBcdmuLODjbzP+7bjZGsk9zwqcBbqgS0LFKTDAYNNCBFSjIYUpJTEtlGGrFSOsmkIZIBAaRAXKZSEJhnAK9a4VGPO3WaTJ1ntSpC82fffZeH192BDHhg1WPVMVlVbYzKjs2bGdKEa5MrnrlRe9c1+gqfaqnapzpcD/FSKgYS0wwky2gpwgDoDGzVskVWZB5JiamqwZAsC4OWSrKqReWEmzBAmeaFRy21mvACfIAI/vu5/q+38bYoIfVRT4+VnlsOmtqjCIY0urmyYzNbkMMKcVNr/XyqvWXIhgwS00jUgcGQlggk65IwZMa98hrv8Rfi+hjQpZCSmpqcmoxgSAEkjDILLKdWEnhVy+BJqRZCoTqnczSv77kiosyTrNXxPY4gukDZ0iRbGJ9RHCLxkrR+z5ZaKkGSGLJVSUtLFqkpKcmSQSVNNaRQGMboJtcDx+HdU8b/btsaFZdJUqqalEISmkyoqUwQVqSWqzovwKvGu0IzCQjLxxb+Rs43uTAo93Ib2i3vTVCQ9MIZQtjU7EKDOyA89FU+1dUB6GLlTCCFFEFKcqJMosEwKsICFFc0sYznq8+889ejX4RH55CUnAqpxGRTBLJEKFBdIzQNb0wCNS8yPyJvuM1kiGbzDh5gp9y26+AedxUUZaDSb+srMUgg8VK9h6fmVVtObnRWnIzISNJSkg0oyQXhahoFfaDW6NdySJJYeE6//DpmYcMyU0hMI0YNKwxSK9Jyi0MolVUZv1IgGMjNi8jHfw9DSuuwvOcg3XZow97UjYLZGmBbbr2H1IwRWGncTJ1HPX6KIsiMzYJUSEmVE+jDlJQaTr4se7Nz06qjy0/L4iO0246EhRahQnYEaBXBJSKkVNWCgKqBQuQQQJKQw/gP/PuPQX9FtmlzoI2Jd6ys9creNTs3sX5PFLvBBUL/7zCbTK71eFcGmAohCxKkNHg9gPhnbCMjngiKYJCDy1nE+QtzceT6tCj/YYMMKQwuQUuILgsEikWoL+RTooYVhuXnh0FY40QZzJD1E2Us791B7v5QWbd3XU70HrJj2AC1XuLI0pHo6/AUlAcTppPlLNKAb4NL1KyE9BSRkpwdM0iFbp98VNYtluZ+MuMGU9scDP9qCColP6gEnzLy5UrhTwi5UWUmTQouKhT5clhRKKVubOfveiv5zsRBhtkP9nRbPVYO9BG9Sn07pmHDHtXZi2KOb9gJ1Hl6mZUg50WDlpAOGTCIlCgSyFaJxr/GlcPLT5+f+4l8fSrXB5g7jScQXEJgGURU+dZWDIQRlRNYGspLklQklwbpoPKDT7GX7QbuOmrtzBz8bpIysAkOrSrLcd8tOVPnGfSVJNwa8aiv8ykPVvLDc2KRVLJjgjVdQaSqmFK1mOwsYku92buSRQyZxkWjWYcdbF+/p8RbLvM0e/UN3zrP6rDUXwqjCSoOgTAoCi0JqjCDJzrLe+1g6XDbhhlX4J+Kc3GI08FDQVL9hj0gcv51bNmeLR719Z511ZSEI8dmkbE1+TklgJwZh1EiJjOOAVh3Wl5wno/O8clHDPl81B/2OA+vgTrfDNCQyn8txL8CmRJRKIWGEozsuf7uA2zA1tjLBI2r0y3O71T4eeMglOZ4INH8IhpkRD3UedYxJI+tyBngXxGZR05sJqRmo0FkhYW7Ks7NO4bU5gDdwub9h/BCc5PrXbXYLLyrCYDpFbFZQQSXhBUVhhVJel96P8XGbOKdwXsObXenMv8cFU+VrC8HD6zfExhNy7pdmndW7ArBDtcmvGsIlERkHmyLB3SI2KzorGizdDmGHEB1bzYp83dr/E7rssG6HcZ+Kf+F9WZbmryz8iJrhVbxPMo48QeKQwgmtAhkM6b1gZmQ0E2+ZWVxY9Fw8EeJZYOLulYDwUuBZWzSN7hLNaBWhkvEEJcBoUXossIHMdel6HN1MhBUrYgl2zfAeLgzEegd/1d6K/55BjSkarypjApsu/MHQsoIhFBCKzV+tMBe4w537emB0XspC0Rxad7NRirAZbcAZJbvdPOsc8dbrgQpMjt2G2GFEEFEPqSRRhSQrYv0afl3y9ttQrG9bZoIatsAPTqLypwzrDUhBDWC/C9v8+OW7YFAQTj4RfC55cQ7E02WFvIdHPUj/lIcMgjK4MZd7IJtr/mIjdT+xKpDiDpkjxr8KxB5ZBH1PLQIjfwYKQvMo3MlEWWSKHSGqwOoNzUJUOy6QH2XNSP990FktVeNDznlcMOCjNEECUrNf40e+/DeJMfb1vp2QLse8LsQRoLCLnBuCdxavWNjizooC0C46urwkCuCCojOSUvMBcIKw8KzISM+mijIiE9jFFf1g9O4NPu6Be2A1Ckm9Fi9w5CoN3zwrtF8g0rH3b9IfI5Znokg1vEQ+1YcBi2s0atiSzmUeTej7HORRDNqlrS5RcgypSwfth0PL4QcIIGJRF4qDC+Iic9I0ecOyHFkDEYm4smAcQ5DptFhB50mRIfp/jiG5AK1gpxo7oPtuz++E1pMpd9erHpkh7Y2YFIrtJc/jx6JukXZpAkBlVTjvFMimlWH1rNZq8dTVx4UKhdE5/A/BWQnpei1WMiN6uvLh7ML5DnHdYtnc32aXZsDttx27LGCL0Zsjwn540hJ84Hge5fG93b+Ppcum/Yv+cBcBnmyegeMzLhKQGGjK1saFXUXGxjis1N2hpxvStjDDtgiKvw0ijzfJnCwJrgkTFKkVP4nioG/HNGbzTsygNmCy8q0adenOTBEh0q39btfx1CchSwB2qVxgxB58pbM7+0RQscd7Dpgyk3rq5V+jNE3CTc3ZRcuzawbU8MmaO73DFyinnZp9hDUg5CR/3zokxpKIvIjc4B4dWQC8HcPOHBMHTipuyBMXJvG1Rm3dao9Rqw77N4m7dPYSp86qn1U7r9H+wsmf8HgT6+bzNocbyOw6TJO6C7RCka6Q2O9hwIy6CSqfPcIzZOyZYJmdxrAmwrCCmvwxE9vDMwjdTA9I82U9Hj4tjcO9ivaMk6qczk3nz5uCvpncttxYqeQ6LZekIio9azzzHwEwUH2PHrvXd75HqnHTrPVOunCdDczuDAcaBQeKFu2K+4Nu8C3RcK58eeIYwyRvGq9agKgMG6lQx2VQFxm37a4kSKRxDSWtThz7NRiadmZhfOPLZvPlBvXuDZdA9vWSWhMPHXX6F+r8awRJp9ZyECHyRFRtw29ydSDja7dyBuENbmCRq2Xsh2dADLixbo9yBHBXx9mSwMvySG6vtcc8K4hoJzB6G2YxUFiGuDcYibEmYWIE0tMp81N84WYwW3H/W/OlG9Lg61mtoBXRXYMHXb2d++P61cde6y0X2UbU7st6O9wr1gtDnFtFEIgUDypxaM+UMJlz3q2ULJi+VE8pVovQSlQD+aedZp3TR7QT9joFMoDwGzNkX4NBgdZdXr22bN9825OecwamHhLmnTdBFSo/nvA7vq0CW1ocrcVfwh9ZK/aaVjfseqJDaES3HipRqljiHsZNcPXH2BwY2gRLtRBLV61AeL562e6Y/2o9CGolOgsCisoFfkRrDm0Ypeew7+8ymGNgWVc+C/zObuA65O5PI2Lc1BfYT0wDRzaHOD+OP5Nu307dDu0kQV+NLlCnSdCwQutnpBi710HJKfdxf9cdagZ8Kh3Fz6iwv316UQ/8a+oxiwm2yLylVf984Moa9Brq/ZZLNuxEo6sOHRIgrlwcY4CAzALBs9Y9L96dcadiZ2mieAACNhvea/drgP0KxxeHwjZLgQgqPFWUBtwl4phIxxaVSQJNpjJMu4IAcOmn140IgcCy1QzUqnwLwinROcOrN2DAqw4JBnh5McwB3Fh7hzg+NJFF2Zd6J/ZJWHLzSl02tJuCe8La8UEdjfD4ktD2NLEELlOoCB5aKgBf4jn0Cqx68dCVuzeItUzxEdD32lLcloiZQFFkDrSWBgGweQrw714vHcYu81XrdqzHkxwxrRkLpz8mKNLDzrJzG3vsIFWMblNtt0j2/NwrPo5L2kLCwmq99i+ZXujwB1QqAc3qRzQDjhREvaV5iI1MMSzGjfJlu1bzBN/eSqichOTsmK3UWTuVxpU2VfJa2t2sYE9qsruDc85tPAsHFY/PujUx2vMBvsrV2aChCwd+dkdfpj500QxqPVad0Ut96vSbdcahRs0SG4obNlOI4Fvx7B6//5B53BEs5tHPbiDZ5++9btfiU4X2/2fG8ZkxqZsJZSCIPzK/WhEAQb1Gxhk4wGVF6AO4nRoxCmtVW93VUO6NVlybLe78vNo4NG//4H9XdsJ3eRS6gs0CBo194Y6SWE7rk1uZdlwQKzZVRbISho93Glo8ARTTg7NVb4J0QGmaob0Q2FYf15koQZuPKfGQreBZhdYvZuV7Fpd1+yyiiGXPplxfVrrpDu023c/MHta64Ub2LbbyoKU33sHMcSdBjdwp17BXQj3hjDYI61mI3AYNDTcNbS9wDANhv/8ana/2Va25UUOoo1AjGLIQJ43iewyATsH2LnpMW+YHR8xm4tzZnNtGrp2+zb7e99vJCWZ24FPPpdtO2FTcnnZcPcGIfB0rxeedZ4eCkiNHr5STJ3kdFCYfv4in/UWktjOEA16f/nv5lrxa11CTPrW6Bw0nmXEb2OEa8SoF898qfFOS91YTmX/pmy/B5X646q2FC7P4qoicUeyxwFLUr5Izoxr/eErm8Ee7DreqfYBJOFJnacHtZ7VKB71HtT7Knme+5DAP3K5SWvCw0MSDDFIM9j9XVhh+CAJ4a/kqdHx+nh0Jc+TUuNzNoMOknxv+vmtWxAz3Rgw4umri25MRQfylNbb2NNtfbH30x6zuDhC3qcLmy4TqZSNlFU36vCs9arxqvFGgXqP+ipeEtL6in8eZd0WuR7cJZrW/Wp5S5uaShhDxsRt20oGaYnmwVG5CensAA8866o2fcXDrrFaq79pI9fVK/J0rkq3JjHEmubGNaOaTsLYz+06rLvG36ViMJDtbjS6gcC7xhuUeqj3qiX39bUHnA7uWR+5Qt0LHvVeWr1H/V4XJrcyCIQPvpLMwNZtkEhIzAj6YrIZEvIY75pZdPHwIZsvX5OmXZ4FzLiu3Xa8O4ENA888xhSgW/r5Q1S6J7UunWvS79i8pcnVrUF4eNcKoFrxFyZV+I+QNDgg1lf6AZt3CLdadw/Y8pTOQYbnRaYkR6djGNiWEW8wZP+Yne6bv5rJtz4++fQf6wa5zEstzrPOLGQWXJqNUOSjZ5JTdhP/+8BzSvlxsG+H1kljQ3jJlUagVta8ajWhSBW+Uo2viGSPWMOe9bDkhACJBncJ5RC2XPjSm2ReyYo1JKcMosuIiePufn7mFidpmL13FePfPVLhf2I0JxeeXQAXJbBwYPMOCsOaXPcynyET74Dj7dfyKXlVknDGrcEdrzrwqvZVyqHGqyqcemnNAUzEZoGEJnuq1FO/YUwVc/dD4G90MZCSqJJIdObi/Ux/wgfvHWPW2Peq6e0t7j+1+DTy6QUcN5e4xidOA7/DEMawfWsZYtcB2Km8XR4QvEs479qBUKHeg1ohqhSQvGpRtyGxGqp8lx9Fowm3RjcPvWSq2jHx33MO38gjKawwedA0LCPeOWfJiZW/XF7b/umncHnkCXBpNlnNQ0Lj0CpNmn1petm5Yb8ZxADrgL9Z3OuYfAs6bGe/GcBOiRZNYnMTeIAkfFACyyTZQ1O31q3loLTKN/ZT1gBujTR6IdVydfPtc/OTDHLS1rTEhPSU+OASTqz76pf5+8b+9hpwYvaliHxszDg3//jHR1ehaGdmEwh4JqfApdmTWv8B9tgZpbudGy0Mb+iF5sJOdmwGGt08a6qFAhX8z951Tof3rc0C0QyNblJDrV9F8ODPON7ALAGGDUSlMxhWuPDMpp0TP3+Xhw9tO+efY9hmPRhnn5t/WD2yHD7mEpDTvq+OzLhdp8Y9tuoBnejAzq7jt/E0sqGZZlx20OQK1GvCVykDr1oXizp13UEnJIYc2LjLHQFUhhQHmqUkW5BA8PA+LZeIgcIFzxyfcsfxBaByDl7fEc2BwVPzUZUVcGyMMvWGxBf7Zg0bHle8ca36mZXjbW6DpaSteo1tOm3XRlxg8/YtOzQBkg8KfpW17g0esEccWrmPuE8Po9AAHkAxUjOP3sv713DJZCqEJ/9AenoX0JkWv1esA8tJdeSwmjO7NwilRd4wZiY3pnJzU1ihVY9HCE/142UB018M9tp0TUlkKzs3MWTH5i3bNze5NbpR460g+aq1m6RU1h8Uh1bVeK9afGoHeNbVe9X6mSn5PqPeu7nXNFAIRKkF4xedxlr/G05DoNP3MG94SXBRKCzcsdN8SfOG8zK3Hem2ZszHX1BPRTNyN5PNrjCeLn67TRnj3QfNLmwGCdcGAd4oVOCNJJIAScI78bNTq4ftRPapFowcYQg3Pbk6JXhkDiSm/fMpI09/+MCi1dF8xhu/lM3+NxRyBN3e0eKZWLcdC3QyQ2QwYMXsuf6A4235CuN77ds3jegb5V1lqpBd6j2aXHnJvc6zxlvBTxUe9by06jDVX+/nAFCDZx1mBuSSwLuUhJGQrgSUT7K6tfhbNPvbTLS45DT8M7Jij0kE7Rs2a8/6WiPyhdkg0211z7LZxZwXv7D0247bU3UffdJrp/LHcHagN/lUMIgrdZvBHU80FN9KL2ECw+98nWD/GlhxxLXJqxbZp3qAaJkyQMYsPWGOY6slp5g+iKP585bZX334baaMNpKitbAexe3AqrNXZwDWvZaMoo2+XMyNE5UbYNchD/BG9ojNzS7UeyBD8YvygGqfanxA0aDOHTcDHJIkarxXD9KEBBIBpihNDSuMz4jKRXs6x/iCe8Dok9bD6HC8RDvij00rOLvgmGlVo5sbMgsuc9MEGoxmxZG7a/cdsLnDdN3lDntVCSamvkUFc/Cp9QopNRW9wKeyRkWp8dbcG1wEIKQV1MRnLFm311PgQ3VAETEIMsiNzTJ4GEf94HJf/G7U/nHdOCx+geW9iL/qNFjAMlB2K2tWgfHqlOswgd7nWVc/fLCPgM6ZV65h1yEP9qw05nqwY/sWF8Cr1qSGFAH9YVQo1OBJs6vh96Odjhwx/gonoM6Hai8fEaZkA/PPkQVfXIUpfSOe/ooe9NLlv9+LzWLIFflTb++ajiKO6hcz75Or02hV7McDH7z/een34gpzBp/Z3bEiGWhS3aDJdecm/KrRU2oU4K/gXSOxqcnAhsNIa2oznA4CEkjV/oURUOV7buzqN8IWnF19wKlhrF6np5v5w4+M+6PzmMMrzy5gJhlYUgTLdxxWFfWji3MmQUI6fPohQUyBi9h2ON4OLMv4nbnRDTChImpFjTeVQs2PyFCQ/FQ0DJWsBCRkNpg3VXnVql4VDKYm+ZYFPsz0nfuMA8sPrvy32VWcDvKfP0xqXTL6beX6tINj5hK/NTg3KqB844Ms9s29OoNDq/iXt+6nfQ9ISr05+dbMZx0Otz96lfgmlwbqNSEQdcILqFf9KATFqxrcGp3/5gdHMa5mP6rmKWseqG6N71LjHVj4a0oV693+c9T+MHOseg7yYa/RRPaBlVemnTHuX6z2X+Yw5bynUbv2/IybU1bBDqZfm3KTVJYen30JW/hkUXqCK6LWo86rtk7zrIZqH02jNCgvUql1bdrc6NRC5WvrQJCcsngfQ9xEE2cWMiSM9x6t2rPkTFApjLo4dlTHgwnP3vgHuuP669OY0teFJTeBlJQcL+Z1GiOu3QHr/sk3x90PK5RnXoJOh/ETlegGdzyoVhGIGu8azQcvioKIRKGJHev7wI+jLKfG22kAPASSac3+hRSEw5Sbf3/0xOGp73NrYTrCQ5jUatnK0eUMWXwKGA0ndu6lXw/YMln7yLqwG7g/abDaZ8GsJ6KrbeofQKLBvRqfauFdDd6UBZYEh2aKeBRgs1Gw9U9+0rKTg/9NfKSCXlQH9P9EWmJ4hT8/YYIRv9E1wAS7DsbrfuVesOVyhoQVOn1n8989dFsTKG+DXe/rrG/B9J2HD43vpbW1kbNTO+0m3nllVDhudVq1TxU+NdU+1ZV+ZVpxcO5gHDQq4FHPsmUClp34GJL3rN68QymjfOq7JF75784pNz/fsp0LKw/4xNx+Yhyw65C7/0ZY4f4TS0h/Mgzb+BtkWlqfXvRmEiAPexJjlpYVm/1kUiv2o/9g5txyg46JnDyZ+6qn8KpCahj0pkr1Lw0sUImiPAA3xbdKBnH8Q2nvOonjP7lyAL+y0CImVMBMcLnJj8wfMcAr2W73+3WmCcqHzxm+bw3XzySQTTzwdBn/4tUEXUqmUbXILiSWaxcA5dJqZ5Z+12bLrMtZimetqPZBuFdV+vmWlRhzjVp2DH/mwlxFo5bVL8iCYx+fVl1xbtlSGVREfkRW117G9zbDqYnPzjmTjXTHqgcb9YHd5jTOjjn+DblRVIiAPK3QzAc5PTMlrnE8h34k9dlPwHiNAwdW92sTOlmEGtZgwgd8yzX/Si2A/ChyomEEc1Fq2DignvrIpoBlLDre5NriBKUwaPiHaf1Xt8CtkTsTl7dE57gLerDs4n37tN2vz8/4/E8GqPPcuVOVN+/dsz6AuNQUNxi1quaZpAN6mXF1dVSurbDrOJ0tajTJC6j0o1wNLNJUcv/MdW0GFyUF5xaYJWnlAZxYIkj7rGWTZ13s8K0lhqhecGlWPB7cuoNzDg2uS34c6J5w94+/5dX5lX6eegPlUovzYQ3W8VISpUEPx+JdmGjgJY39vYgebGfGFHjXeVHlix9ogtAcU04USOKTj+ag0LJKHObShACOilOLSdNWaOZJw+S04AW5U24h8WqONdCy5ckXTcx4YXWXS5dYyPZ5dXFka6ozkhNHVhx0qvTLi8TI2E/+s1GXGQfT+9uuT+sbhdFG7VyBOZ41wqdS+IMILho0xcF1xMxPuDpDYblRcBmt/PXlx7WjP7ttlFoI7suFs8sew6jIT+kGG/PtU3oCy66O1b33iKLQa2U9reEyMUWmhpFwYvCgsaGvJBiDODdft7HpuQKYNGb2bS7l3qRWLPKGgTf4UYbQikOy4jg3fNrVGVdnYN6mcBTmnae4XDu09OjyRsRhV72xJHDELzVjXr/iWBGmTh7soMt+6iADW4cf7oStod1nuz9HpKfqhWZcC0sOOVEzGJYTbcifzwxcK3TAoDZDu0PQ4i9brSdGl7jXa14MCaQotDC3Pyd6/rUrAt2diY5tCksk9dSkVgJaOAAq2sdNnpX5EXN/y054c0vhll9G8AHyjcm8/dM2mDgvgx9iYGlhXBIvtTgfWLVnvTeZRhKMe8f32dV65cKyY45mg1B17TP0b6DfYfSi2qdcExqEkfrm5VnTr8xkane3USjLjy49hebo8D5Iyw86sRfe4jEXooDNO4jLBNunj2AYbyQBd+7sW1v79hudgkyDorcIwXnH6j3rgTgStFi67NBK+9J+zUe6AKm+Hoxva0OvelT5+iAFUSzIHYzn+tUZQLd1t+SgyMu0RS+eCk04cwQnhmx4Gpnq+kPulsdHlNUHMoHOCePfMvvbhKvOLe+9NrVgLfvf61yu+rwiEGp5QPXm7UZqvQymYUYDdQP/d/WZFJ3y3MbUCjzmVXo37Sz0rK/xrsTPH0KKiAKmXbk6A6wR9l2K7jBg2zUpeLf5Co7+4gp/yg6iaRnbWdCAe4NLs53ZLXpZu2/RaV7d2A/89dFUi2fVDCkwVfhg0jVjYEUBAAAT1ElEQVT1Y0gyj88YVCcBRag/dDHlJn/LZwzvv0lYo0lUoFXgXxYYSnoCMJM2B9olm25r5cDKw6DHHLGKo8vJ+GfdD5SCFFZof9blSy0yD/2tWd8+nLSP03DHwBCFG6z0G5OVkhxe5E+V0NwBs29c4+H0oipjEE8PYXtz3vObtPzfpFYXcKvx4aUyAbn9wNUZOHRIttwVKKjLjs01LS5CcGg5xLt83O9Z51ulf0o7T0c3Bad9Uz3h8lxja/Dn3yvS4scb+OuXDwFznSAZpDLJt9YdygITQpuCSnly5AeN9F6rnk4kaXyvM7NIBLwr8IcyLbgoVEvcs54ZrWYSEj2ILmXFkaUfn2QDOLPq6HJoXrrbnaqpA99MHP8dZyhJ+P2SE8y7wNfP/r4n6+uvmTDAS3JZbKLOQAgleNXq3PWlQb/JlgveG/aTCOH51z1MfF3pVyDnMlAvPP0rywPKRTAif5Dh16fdnARd1j1WdMkKHx+HeRbQ4nxw+bHHmziOFj0mfv453yoWrnrdIj11LCv/NeXm4fgMfipn7T6HydJN2EUWyenmUQUaeDXgA8PD4YZRM8E2a2vMFU3XNb43el3MWzoPqPQDISgJhmIzrit0azZ3Bdj0KoOCRafPA9I+7Zi0iU0/n2yC2ef4iomm58Y31z4fw+G5z+GdU4v/lf6fchuTbAG+A6OLk+UECIcKf/cab8oDAGkWkJLcbS1dYyH27bA3U8h1ngggsJjgAi0yhBvyRITUNQHoHa9oZzgNpP924/41R5e1OO9ctH5PRL7s3vDd/HMsOQ5e2UkPOx/6jR622Kv2b+WTb+F4Baq2FMdKpOtj8iLz/EsYBLneA3TXpsO3lmaii2UDnzAkKA5qazQVygJDikPCswBpMiBPoMcK7ilngFmXnSUZ6dByhpwODcnnwurV3xhnXX5l1aE1/0fqooeJaZzHlB/BLdvO2zi0YXLVDClaDFJ2JE/jocCvtDTo4H130vUxOLTBsSXzzr/7ReH3FA9D+JZTHlCmhhSFQoO7Qo9mzd0J8n1heU+ZLOl1l+bLssTqoxx/wlqLoo3p37f+orXOvbD09495cR6XZtKuzGzHsvv9z8FqxK021rUAkpqWGAFhDwktCqenNsbJA0aEjde12XSBfJ4v1odlluJdJwBRSkhRaEG/wf3GRLDulsCS+1gqt6Zc+Wimxc5sd1h+8mPSE2BXbilDJI4fd/Y2QfMmq6iZJw6mR1iNhE6mzpAv9686BEmJiQx5o5AioLYgnLEQRi90TZdHHZs28syz0iAq6jyrqnwJBEKLQ2CvniHW9HIf7lkq3OSTqceIgaPLOfbse1Zi892Gx6P32J6ff46WDbuBnSWwJDuGHrA2uyOuwtFleI3OVzGYWYSPhKRUSl4zPyA5AR/qlQ67a7P0XF9sc+IESBKyN5XCn+KQENLVdXDPsleylB982CusUBiiQdrvvJYfR9oIjy+Nz5//hvrhK8u0qBGG3RBaRHDuWxvGLvtaaaPbAZhzcdGxebWEGuMMacZ8jXRjkvbzC/2ARNmP55/I2Om4vHTlE6EAVYhaL6o1/3KhAgmceEf3917J8tMP7mPS6FDs27G/tV6fWMExBDs3rdnPj5zzquUvxzgRBasOvQ5EHT6+FHtpyk37tsk6TJzm/PKjA0VRCJMkkQAGQ7wuldK+uzfHdTm0TQRFvghzL9R6MaQGpdqH4sKw7OepSz4HWfpcMK7TBvQKdh3t7PFONqLJSyEnep5tttOvtVMeA3mR8Jb7tyHFYYVvHR1noSHAwXhrigbMeOFUiYqmiPDcqFzJnIyaOs+ggO9sumyVaQPWfzi8av45jChlZt61Gt5UVwhVyh9MBUn6lA8eaD1Wtj1WPSimbttOqDFIiGXHePxv+Cng64vctDwFkQS9PvCikqh/cy/dWtbkWzaSaEdliGr8admrBRAHOkb7NIB3E3zeOoFOhnSvFefgSq0kZPABhD9DMuMzLsvig/tolvfAqtuqUxFoNr/7vV4Kbll5RCDFItVteTL3jS/uLPzvSPu8UkgIKM+FC3TD5BlP2uzgNjjqJMHXx9bvMQBKmc8ejV0b9ZV+P3GXSWYvlFb2rVminZqBKzS4M0Sj1BiOyGDWl8Z747DkJa2zX7lr22n9vAHKQMhI2R3fuTUCkzBqly8Hdv9+v3iLIS0ObRNNt2Z3IznetulCvgkLr0z4Bq/auMzgIp5bsJF+PwRMQteOw992719xhKtXKQ7pp0IS/v5lqgAt+rcRf/3Mkvuyatllc8sWHwUNUx8Q2MJKjjoDV2GxsW9WnzZufuGa/Tx7EvNn/7z/nkGTZ5sx/Rp0gQ6maNyFWpfMiOGhDHoAHrlRqn27hAAdmwb2sfAM1T78lOVfqFCuhRSASX3twYdqL9q4uzzj5IKz1Ypdx3hj27JjMdlwUFoOySlg8+LyTHET7vOnedqz+pBfMiMzu2m3kc9wjZfsVKZdn8LafTHZzTnRgJ4GneL8d3rh1sTLdh2ouwGx4ogPwb8UxP2UTr+Mlm3sk1//8IFxAnRNOD99VTo2Y5RR06+N5wXZRdoBp4MHf/VI4eOTz4ev+PEG2HaSz0f1ud3FiS5xdf/MkHXzzs+5aNdhr3VMnKY53GQUwzFEb9tKWT/upSMZAZ/cvfUlHTatrQxRzMmMG51OZkiwHEHS8+TkDHL5sBO6rZnHQ3j7mHJ5/dynkwZw7hNinxM0nDh0cs6/noy6MeHN869AzI8NuY9MHmnwfYZnHbO4iGbTbovKzcmgbE3GwNbsmEAgaCezb0wdNctxMjydeIeVhxlUgJEkpeoK8KpN9SOFgCV/tLLtBhpPHrg367LZWmVb/x4WPrdtgX0S7F23KTXp4gfqi807pvpXzPgpO8T93zX4QbI2v27Rs8tgLzrotJGvz7nIPG2QBMWQKUGurLfQnxg+9ao93H866R8sN9u/+JTWUvOkKDQ/gjFxmbVexice9XzfLzr0mmbL6QPz9SfXSJJCCqE//3CNtEQEB53YCX//FK5y48bSX+7z7X4Wn6pMSlW/OMdpYNHpiTje1qtTLy48I9VGImspcRSERxUGM+SCemUmA5M+vepzlHkqF/EuDCWCnGiIGxgsBa/aA716KzphHyMPMXw7ytai0CKw7gZlFTIN7q5N5ox/RbvB9OPAb2dfUvFIzY4hoDz0i+OcnqRxmzsO5gvOOA4iJw5mkR0TnosMu8ydTELHLVs+YPsWXtz4+GRwSVi5KoRGlhhUzMMK077jtsz9cbaMu79pJyjrdUroC8C7Y0niThlWNuqq76Mx3OI8DtfA0VgGMlqwDOX8+BY2Xcp1+3Zo42P/CkcfXZ8ZORJEMUSYDi88Ne+yRqeimJcH3OAkI0uEFExheL4WlSExLCExBke4f9uR++x0aQ4t2mCuFFES3OAOGZJ8VOIX0y+f0msp92ElgNtz2bjr9GJV/GowRP7rd/l0ITDZt8N0s4rl3wwMFJAenUduVGFYtY/qDCqqrlWzuqcu5P3Pp9wcHgwUhRaEkx2TnpqsMOzo2A8YB4teUaQdYYWbZb3y/ufqXmPGqRvxLQKW7XyiO4/lvbk/26kW3vdNt81n7Fp5+NQKnVZe/c29MY0VbTOv0M3cp8ZrGzEzmhQy4olEJgwfNsPBpcy5IuzRxn51M+rzm7Z6yoQmCIeY7Bi0RAzw4MOeR07DduMT0r9xByjGaWHsNb8BzseWcfxZf9SKI/ccL1j1MKeGWWO//RO/EtD/U2ORD8x0A4mJg92Dt+2mDgsqtZV1KfHbUKSIgvCywBq9vMWJk3pN0C3x56/n/PBIZ0EgUKRGkKWmDabBldEWH9D/z3V7oRo2eMo1ypdvJqXKKlMW/0XiuLyhhCPYSAjbzouBZTzEHHXFoBCbQ8HqBXAZWcdVxI0b0Dl+WxpbIQsNFe86t10bD4n5F6cgxD3l7Ssn/tzWBpQEi4gckxaXrHL6dfTyp6avvtrrYj7cLH+jmWJE4dkAUoz9zYXyUpae2L0hJdlW3zq+9wObRd/yZKqplavzLH77xdF8XWhPDy7N8wf7FaCTiHxQEhO3bYVYIqFU8UTbJZacAWzoFY/0nW8uOl1GYDBheRqawQCL4J5E3z9LafYpBcmsChTU7F2ri0LtknYdXwob+MlB/4xe9H//93Y6Zz8HzsN6v4gssmJp5tlNhkwcMzYfqE9I20puVJYuqiAEaFAlcVQI6NCJD/i6NGgEklYsRFgkYIjXtn3+/l1L7k1M8BzcoeBd0wKsV9DYePDb+ecSWcpBY/MvQbMuA463zd/AWn/Jiv/Zw7ZYBgrDWP5kxlVAOW8FzPm7Tsowi8rRTITnR1Dt3qStPqjMvTJTWN+9P+7tTzAvCigLKQgnT5hUY0bK2ffRoK/WK6DOXQotAl9jvVH5s8YOJ7BLA5yAuEzAxuR4e+6SE5PowUG5Ba5/es6SY6P5+JXfK7855NB2c1N+cgrDzBU5jrxIIEcCUau54nQccUPtnND74MOPsKBEKgREFBkCRf/pBwJ+/g7JXRJF4FcJBxXdIzZT69UR98EmjklLK/+JXcf4LmZwYdakVib/IgTTX2sK0gf9JpkPvzkJTL828Q8741NYeCIuMQsiYZsUnUuJ6lu/XVGMp2fRKhh//8GHKJKshRUVhkWB0BEPd2wf9n2aSBlu5WvMRlayZburomeItGF35o4j0rJjSDl0IEPf+F5T66RW5TPmzbk4kxFZDHmAffukwX7urA/G8LjbPZNYCsJBjiGqAKo8YJdOvTBX2HFX8Kh7TUEIhJYUaeHZJh3Z5qE/MZbasEJoZD9sMW53Myqfv/sFuzcwY9JmDh9dhi9Lv79zd9bjNmCKftI12+G68xt+4gFLnn9S423dTitWLNwTmYGz0pCujyG8UI2MyZLkcMp8q3We2mpAgwn3xn36V8L9KxLTgikmJkcqITjz+Wfy97cFLoqp2bmF7aA9Ud5T51yUQc7h0EqOLSP+/p1xFpexGX7jOmDfDpN3M/UfnFjs5U03Q3oYHiwlmu5cTEogV5alyILw2Oyo/AjwAZe962Bqq94OsGNtBV9B4pMsObo0CPSrH71Hwgua3X5d1eJRD2yfoTx652LOOiLzQiy1wyuX7fshkKk3gOHatOuzjU/7Yfo1n+ob05n7Wu2i05b34MMHHIoW2jZI3bYVKSy/QJAtsiIKA8tkf1jHEM0OwQf/fOfZqkM/s/rhXoKSqX8eHNUmw08/wrMTHp51LopZzaLTylijreCg02zNCzgxEOj77B8T7n74QHdjGpfAyq7jGd/Ch3/dvnDcae5ZmdDNuBqM8tj299YZZEXlEUE+sioIK9ICqVXMnOHiFNDuySoj9jPSqa8PSpf9qQz2OHwOlTNWyEb0T2n21NbvQXn19l9jsMi9QI33EbHkMFUMeeBwlevA5FvQZfcv+MtWN9MZLO/1gP0oT30OQyZnEZsjgAiiICc6tKgsUGim3RYr53Br8oRe8TfewlerWX9y1mU+e0Gbw0j6a7ynWRyBgOfG1cP0A89xUN75i5I2/mNWlXJQWnF6ETh9Zt7p0GbfDtOuy1Y92L7ylHVq+lfD7NvFhw8Y/8rIkQoLv7XcJceSp0l5siwIA0GuGlrmDezn7ILJbYomQeGaqkCGbXgBDx/iwCgmODte/5hVh8ph9c6Q58cxU/YAh2A4VU5Hjy0iMe3ghE6MyA5Grk+9AY63Ydbevf61TEDX896j3l5is2afoQdtmw4ic8OLtfB8IiSi8hDlAexcc3ABaALuWYKH5F/BSx+f5Jpp34Ua5p/kEEMOULyRGYqy5E8jclcd+94LDiyHwyvhLnYydxg/fNnPYHmbKTfF4t9VLDh716qHR/btuGXZ9W16c7giSSIyl6iCEIrQ8kWe0AqERr22y4kzCx2BezB6oN4PWGs6dHLvuukwfcqoswxZdHrdG2WoXJ2tnNhsomUZoApOLJHYtBM6sG+feWXysVkLz7wKNxl9HMwZJyxBZvJ93jlwiyQpHojKhcIwmVBeyg8uxaNRAxO0TsIS6p1Q/SoZpniKdRfuhS4+9fFJPEb83HyaUWVBpXthUGHHBpyP/OwepK05tgTYCbaiq50rE8Tyowu4xpAxuL75YubPveO1+7ZTb+BbBclSUqpOiiMKKAqmiND8iAIIosYNDkvn9JPaHO5ZPnrvIDX+QabKUMnEXBdOoYd618d89FYdL03vVya17oYV+yktOSCd1PUlXO8f6GS6enPRd6pieRZYfvT9X9f8Z/SvsmrTBW+9Yq0vIiFdMpCUKWcJRUbSigmhWCK8sDgEje36lXABByxRL5Sb7a2AiHzC8arF92kLHv1NMHpMbFYpcM1OaZ2k/HnNijUQfGApZ9V0XlI1TjPx4Pj550K/+T+P+9+N3z/8MEMm/eZYkHletJmcJFJkKS4zLlsT0fkRFIEkURIMiFo2ckQ3F9p0ur93vfqza1OghT5GzQspDiiv2oT74GPgzVHb4KO+VjoUW1VraQHCCwTH1dVxmQy5yZBXNu/oXVcU/lnfb35o5DDM//LV10/4lkYiC0mTkiEtLismS5cXUSBLxSHBpRX+ZSIIv9otLZIqnV1w3QGwaADKGBIpeC0+Y6db325YfvSNzLD/fvdrG/NeV0Sb7XLv1ydRgECS9p2Z/quitMHCM5xb5tA2yM/37mE556JNF+eATVVhcpwkk2YwGNITkIjNikWEFocUyUEV5QFU+NeLJoQqLbioAXc/wKXZpTmwDPLYxla3xl9MbNx1lMyk1PnnsX/lzZb/B/PxyWYNOw99AAAAAElFTkSuQmCC", "text/plain": [ "128×128 Array{Gray{N0f8},2} with eltype Gray{N0f8}:\n", " Gray{N0f8}(0.0) Gray{N0f8}(0.0) … Gray{N0f8}(0.627)\n", " Gray{N0f8}(0.627) Gray{N0f8}(0.624) Gray{N0f8}(0.388)\n", " Gray{N0f8}(0.612) Gray{N0f8}(0.612) Gray{N0f8}(0.0)\n", " Gray{N0f8}(0.0) Gray{N0f8}(0.0) Gray{N0f8}(0.192)\n", " Gray{N0f8}(0.612) Gray{N0f8}(0.0) Gray{N0f8}(0.0)\n", " Gray{N0f8}(0.0) Gray{N0f8}(0.0) … Gray{N0f8}(0.2)\n", " Gray{N0f8}(0.608) Gray{N0f8}(0.0) Gray{N0f8}(0.0)\n", " Gray{N0f8}(0.0) Gray{N0f8}(0.0) Gray{N0f8}(0.216)\n", " Gray{N0f8}(0.62) Gray{N0f8}(0.62) Gray{N0f8}(0.208)\n", " Gray{N0f8}(0.0) Gray{N0f8}(0.0) Gray{N0f8}(0.188)\n", " Gray{N0f8}(0.635) Gray{N0f8}(0.0) … Gray{N0f8}(0.0)\n", " Gray{N0f8}(0.631) Gray{N0f8}(0.0) Gray{N0f8}(0.0)\n", " Gray{N0f8}(0.0) Gray{N0f8}(0.627) Gray{N0f8}(0.184)\n", " ⋮ ⋱ \n", " Gray{N0f8}(0.0) Gray{N0f8}(0.129) Gray{N0f8}(0.0)\n", " Gray{N0f8}(0.149) Gray{N0f8}(0.129) Gray{N0f8}(0.0)\n", " Gray{N0f8}(0.216) Gray{N0f8}(0.0) Gray{N0f8}(0.208)\n", " Gray{N0f8}(0.345) Gray{N0f8}(0.341) Gray{N0f8}(0.231)\n", " Gray{N0f8}(0.0) Gray{N0f8}(0.0) … Gray{N0f8}(0.259)\n", " Gray{N0f8}(0.298) Gray{N0f8}(0.416) Gray{N0f8}(0.259)\n", " Gray{N0f8}(0.0) Gray{N0f8}(0.369) Gray{N0f8}(0.235)\n", " Gray{N0f8}(0.0) Gray{N0f8}(0.0) Gray{N0f8}(0.208)\n", " Gray{N0f8}(0.22) Gray{N0f8}(0.0) Gray{N0f8}(0.2)\n", " Gray{N0f8}(0.0) Gray{N0f8}(0.22) … Gray{N0f8}(0.0)\n", " Gray{N0f8}(0.196) Gray{N0f8}(0.208) Gray{N0f8}(0.345)\n", " Gray{N0f8}(0.192) Gray{N0f8}(0.0) Gray{N0f8}(0.0)" ] }, "execution_count": 82, "metadata": {}, "output_type": "execute_result" } ], "source": [ "using Images\n", "lenna = load(\"img//lena128missing.png\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "A noisy image is basically a matrix, where lot of the pixels are not reliable. In a way, these unreliable pixels can be treated as missing entries of a matrix, because any $n\\times n$ image is nothing but a $n \\times n$ matrix. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "First, we convert the $128\\times128$ image into a matrix. Here, through some precalculation, I have already filled in the missing entries with zero (for the sake of illustration)." ] }, { "cell_type": "code", "execution_count": 83, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "128×128 Matrix{Float64}:\n", " 0.0 0.0 0.635294 0.0 … 0.0 0.0 0.627451\n", " 0.627451 0.623529 0.0 0.611765 0.0 0.0 0.388235\n", " 0.611765 0.611765 0.0 0.0 0.403922 0.219608 0.0\n", " 0.0 0.0 0.611765 0.0 0.223529 0.176471 0.192157\n", " 0.611765 0.0 0.615686 0.615686 0.0 0.0 0.0\n", " 0.0 0.0 0.0 0.619608 … 0.0 0.0 0.2\n", " 0.607843 0.0 0.623529 0.0 0.176471 0.192157 0.0\n", " 0.0 0.0 0.623529 0.0 0.0 0.0 0.215686\n", " 0.619608 0.619608 0.0 0.0 0.2 0.0 0.207843\n", " 0.0 0.0 0.635294 0.635294 0.2 0.192157 0.188235\n", " 0.635294 0.0 0.0 0.0 … 0.192157 0.180392 0.0\n", " 0.631373 0.0 0.0 0.0 0.0 0.0 0.0\n", " 0.0 0.627451 0.635294 0.666667 0.172549 0.0 0.184314\n", " ⋮ ⋱ ⋮ \n", " 0.0 0.129412 0.0 0.541176 0.0 0.286275 0.0\n", " 0.14902 0.129412 0.196078 0.537255 0.345098 0.0 0.0\n", " 0.215686 0.0 0.262745 0.0 0.301961 0.0 0.207843\n", " 0.345098 0.341176 0.356863 0.513725 0.0 0.0 0.231373\n", " 0.0 0.0 0.0 0.0 … 0.0 0.243137 0.258824\n", " 0.298039 0.415686 0.458824 0.0 0.0 0.0 0.258824\n", " 0.0 0.368627 0.4 0.0 0.0 0.0 0.235294\n", " 0.0 0.0 0.34902 0.0 0.0 0.239216 0.207843\n", " 0.219608 0.0 0.0 0.0 0.0 0.0 0.2\n", " 0.0 0.219608 0.235294 0.356863 … 0.0 0.0 0.0\n", " 0.196078 0.207843 0.211765 0.0 0.0 0.270588 0.345098\n", " 0.192157 0.0 0.196078 0.309804 0.266667 0.356863 0.0" ] }, "execution_count": 83, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# convert to real matrices\n", "Y = Float64.(lenna)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Goal \n", "Our goal is to find a the missing entries of this matrix $Y$ and thus reconstruct the image and hopefully figure out who this person is. \n", "\n", "#### Constraint to impose\n", "* Ofcourse, we want to ensure that in the reconstructed image, call it $X$, for the available pixels, both images agree, i.e., for any $(i,j)$ in the index set of observed entries, we have $X_{i,j} = Y_{i,j}$. \n", "\n", "#### But how to fill in the rest of the entries? \n", "\n", "One reasonable of way of doing it finding the simplest image that fits the observed entries. Simplicity of an image when traslated to a matrix can correspond to a matrix with low rank. In other words, we want to minimize the rank of the decision matrix $X$, but subject to $X_{i,j} = Y_{i,j}$ for the observed pixels.\n", "\n", "\\begin{array}{ll}\n", "\\underset{X}{\\mbox{minimize}} & \\text{rank}(X)\\\\\n", "\\mbox{subject to} & X_{i,j}=Y_{i,j},\\quad(i,j)\\in\\text{observed pixels of }Y\n", "\\end{array}\n", "\n", "But this is a very hard problem and solving to certifiable global optimality is an active research area. As of now, problems beyond matrix size of $50\\times 50$ cannot be solved in a tractable fashion. You can see Ryan's paper on how to solve low-rank problems to certifiable global optimality [here](http://www.optimization-online.org/DB_FILE/2020/09/8031.pdf).\n", "\n", "#### The best convex approximation of the problem\n", "\n", "To work around this issue, rather than minimizing $\\textrm{rank}(X)$ we minimize the best convex approximation of the rank function, which is the nuclear norm of $X$, denoted by $\\| X \\|_\\star$, which is equal to the sum of singular values of the matrix $X$. So, we solve:\n", "\n", "\\begin{array}{ll}\n", "\\underset{X}{\\mbox{minimize}} & \\| X \\|_\\star \\\\\n", "\\mbox{subject to} & X_{i,j}=Y_{i,j},\\quad(i,j)\\in\\text{observed pixels of }Y\n", "\\end{array}\n", "\n", "The problem above can be formulated as an SDP." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Find the index set of the observed entries." ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "8128-element Vector{CartesianIndex{2}}:\n", " CartesianIndex(2, 1)\n", " CartesianIndex(3, 1)\n", " CartesianIndex(5, 1)\n", " CartesianIndex(7, 1)\n", " CartesianIndex(9, 1)\n", " CartesianIndex(11, 1)\n", " CartesianIndex(12, 1)\n", " CartesianIndex(17, 1)\n", " CartesianIndex(19, 1)\n", " CartesianIndex(20, 1)\n", " CartesianIndex(23, 1)\n", " CartesianIndex(25, 1)\n", " CartesianIndex(26, 1)\n", " ⋮\n", " CartesianIndex(113, 128)\n", " CartesianIndex(114, 128)\n", " CartesianIndex(115, 128)\n", " CartesianIndex(116, 128)\n", " CartesianIndex(119, 128)\n", " CartesianIndex(120, 128)\n", " CartesianIndex(121, 128)\n", " CartesianIndex(122, 128)\n", " CartesianIndex(123, 128)\n", " CartesianIndex(124, 128)\n", " CartesianIndex(125, 128)\n", " CartesianIndex(127, 128)" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "observed_entries_Y = findall(x->x!=0.0, Y)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Find size of the matrix." ] }, { "cell_type": "code", "execution_count": 27, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "(128, 128)" ] }, "execution_count": 27, "metadata": {}, "output_type": "execute_result" } ], "source": [ "N, N = size(Y)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Declare the variable $X$." ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Variable\n", "size: (128, 128)\n", "sign: real\n", "vexity: affine\n", "id: 867…105" ] }, "execution_count": 28, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X = Variable(N,N)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Add the objective $\\| X \\|_\\star$." ] }, { "cell_type": "code", "execution_count": 29, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "nuclearnorm (convex; positive)\n", "└─ 128×128 real variable (id: 867…105)" ] }, "execution_count": 29, "metadata": {}, "output_type": "execute_result" } ], "source": [ "obj_SDP = nuclearnorm(X)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now add the constraints $X_{i,j} = Y_{i,j}$ for all $(i,j) \\in \\texttt{observed_entries_Y}$." ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "Constraint[]" ] }, "execution_count": 30, "metadata": {}, "output_type": "execute_result" } ], "source": [ "constraints_SDP = Convex.Constraint[]" ] }, { "cell_type": "code", "execution_count": 31, "metadata": {}, "outputs": [], "source": [ "for index_i_j in observed_entries_Y\n", " i = index_i_j[1]\n", " j = index_i_j[2]\n", " push!(constraints_SDP, X[i,j] == Y[i,j])\n", "end" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Create the problem now!" ] }, { "cell_type": "code", "execution_count": 32, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "minimize\n", "└─ nuclearnorm (convex; positive)\n", " └─ 128×128 real variable (id: 867…105)\n", "subject to\n", "├─ == constraint (affine)\n", "│ ├─ index (affine; real)\n", "│ │ └─ 128×128 real variable (id: 867…105)\n", "│ └─ 0.627451\n", "├─ == constraint (affine)\n", "│ ├─ index (affine; real)\n", "│ │ └─ 128×128 real variable (id: 867…105)\n", "│ └─ 0.611765\n", "├─ == constraint (affine)\n", "│ ├─ index (affine; real)\n", "│ │ └─ 128×128 real variable (id: 867…105)\n", "│ └─ 0.611765\n", "├─ == constraint (affine)\n", "│ ├─ index (affine; real)\n", "│ │ └─ 128×128 real variable (id: 867…105)\n", "│ └─ 0.607843\n", "├─ == constraint (affine)\n", "│ ├─ index (affine; real)\n", "│ │ └─ 128×128 real variable (id: 867…105)\n", "│ └─ 0.619608\n", "├─ == constraint (affine)\n", "│ ├─ index (affine; real)\n", "│ │ └─ 128×128 real variable (id: 867…105)\n", "│ └─ 0.635294\n", "├─ == constraint (affine)\n", "│ ├─ index (affine; real)\n", "│ │ └─ 128×128 real variable (id: 867…105)\n", "│ └─ 0.631373\n", "├─ == constraint (affine)\n", "│ ├─ index (affine; real)\n", "│ │ └─ 128×128 real variable (id: 867…105)\n", "│ └─ 0.662745\n", "├─ == constraint (affine)\n", "│ ├─ index (affine; real)\n", "│ │ └─ 128×128 real variable (id: 867…105)\n", "│ └─ 0.67451\n", "├─ == constraint (affine)\n", "│ ├─ index (affine; real)\n", "│ │ └─ 128×128 real variable (id: 867…105)\n", "│ └─ 0.647059\n", "├─ == constraint (affine)\n", "│ ├─ index (affine; real)\n", "│ │ └─ 128×128 real variable (id: 867…105)\n", "│ └─ 0.533333\n", "├─ == constraint (affine)\n", "│ ├─ index (affine; real)\n", "│ │ └─ 128×128 real variable (id: 867…105)\n", "│ └─ 0.360784\n", "├─ == constraint (affine)\n", "│ ├─ index (affine; real)\n", "│ │ └─ 128×128 real variable (id: 867…105)\n", "│ └─ 0.337255\n", "├─ == constraint (affine)\n", "│ ├─ index (affine; real)\n", "│ │ └─ 128×128 real variable (id: 867…105)\n", "│ └─ 0.337255\n", "├─ == constraint (affine)\n", "│ ├─ index (affine; real)\n", "│ │ └─ 128×128 real variable (id: 867…105)\n", "│ └─ 0.364706\n", "⋮\n", "\n", "status: `solve!` not called yet" ] }, "execution_count": 32, "metadata": {}, "output_type": "execute_result" } ], "source": [ "problem_SDP = minimize(obj_SDP, constraints_SDP)" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Problem\n", " Name : \n", " Objective sense : min \n", " Type : CONIC (conic optimization problem)\n", " Constraints : 73665 \n", " Cones : 0 \n", " Scalar variables : 49153 \n", " Matrix variables : 1 \n", " Integer variables : 0 \n", "\n", "Optimizer started.\n", "Presolve started.\n", "Linear dependency checker started.\n", "Linear dependency checker terminated.\n", "Eliminator started.\n", "Freed constraints in eliminator : 0\n", "Eliminator terminated.\n", "Eliminator started.\n", "Freed constraints in eliminator : 0\n", "Eliminator terminated.\n", "Eliminator - tries : 2 time : 0.00 \n", "Lin. dep. - tries : 1 time : 0.01 \n", "Lin. dep. - number : 0 \n", "Presolve terminated. Time: 0.05 \n", "Problem\n", " Name : \n", " Objective sense : min \n", " Type : CONIC (conic optimization problem)\n", " Constraints : 73665 \n", " Cones : 0 \n", " Scalar variables : 49153 \n", " Matrix variables : 1 \n", " Integer variables : 0 \n", "\n", "Optimizer - threads : 48 \n", "Optimizer - solved problem : the primal \n", "Optimizer - Constraints : 32896\n", "Optimizer - Cones : 1\n", "Optimizer - Scalar variables : 24769 conic : 24769 \n", "Optimizer - Semi-definite variables: 1 scalarized : 32896 \n", "Factor - setup time : 207.52 dense det. time : 0.00 \n", "Factor - ML order time : 177.58 GP order time : 0.00 \n", "Factor - nonzeros before factor : 5.41e+08 after factor : 5.41e+08 \n", "Factor - dense dim. : 2 flops : 1.19e+13 \n", "ITE PFEAS DFEAS GFEAS PRSTATUS POBJ DOBJ MU TIME \n", "0 1.9e+00 1.0e+00 1.0e+00 0.00e+00 0.000000000e+00 0.000000000e+00 1.0e+00 207.63\n", "1 6.8e-01 3.6e-01 5.9e-01 -9.71e-01 1.415377135e+02 1.431598397e+02 3.6e-01 257.51\n", "2 5.7e-01 3.1e-01 3.4e-01 -3.34e-01 1.116403593e+02 1.121811844e+02 3.1e-01 303.22\n", "3 9.9e-02 5.3e-02 1.1e-02 2.56e-01 1.532518635e+02 1.531753559e+02 5.3e-02 356.38\n", "4 8.4e-03 4.5e-03 5.5e-04 1.07e+00 1.479916370e+02 1.479971145e+02 4.5e-03 408.76\n", "5 1.9e-03 1.0e-03 5.8e-05 1.00e+00 1.481214349e+02 1.481225967e+02 1.0e-03 457.44\n", "6 4.0e-05 2.2e-05 5.0e-08 1.00e+00 1.479726003e+02 1.479725623e+02 2.2e-05 513.78\n", "7 6.6e-06 3.5e-06 1.2e-08 1.00e+00 1.479718423e+02 1.479718468e+02 3.5e-06 561.63\n", "8 6.9e-07 3.7e-07 4.2e-10 1.00e+00 1.479711247e+02 1.479711253e+02 3.7e-07 615.21\n", "9 1.5e-08 8.1e-09 5.1e-13 1.00e+00 1.479710833e+02 1.479710833e+02 8.1e-09 668.48\n", "Optimizer terminated. Time: 668.58 \n", "\n" ] } ], "source": [ "# [💀] Do not run on a laptop, I am solving this on MIT Supercloud\n", "# This is a problem requiring lot of memory\n", "# Constraints : 73665 \n", "# Scalar variables : 49153 \n", "solve!(problem_SDP, Mosek.Optimizer)" ] }, { "cell_type": "code", "execution_count": 84, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQAAAACACAAAAADB3ujWAAAABGdBTUEAALGPC/xhBQAAAAFzUkdCAK7OHOkAAAAgY0hSTQAAeiYAAICEAAD6AAAAgOgAAHUwAADqYAAAOpgAABdwnLpRPAAAIABJREFUeAEcwQWAVlXCBuD3nHtufD09MIiCimsgShuou+qvrrtrYKLUNA1DC0zT4dAwRYmKhbFrrLvr2pIqYCsiMcT0lzfPOT/u8zBg11j87jmmPYKNU8buKlXIJmDqBgALlZpaWVErKoHFdAGwmMqFWIZnUFOxmMhFWCrIQixeVF0J1EgJdwkmbcb/bKWkGKgvrS9tUjz9D8Hrvtu3T/OtwjMKJ1haXgtUoGbOKuw4ZfQ4fd6VffIo0+yE81urG1aIK6VPCgVhJRWRpl+cQbaftmRLlUoHBuIwLjlLJJgQJB7hkc5cYcelhJuevOCIQEQaNJOSn2hmpDUHiuLpfwhG0/bt03yKqymcAIrjGCr8HUpiCNshxzYVNZQAY17CK9aY2a1jRf1EYBKftBlYggoJyEpUVxJgqZQLFi+BXIDFvGYRUFu+oKZiqceB6spqWYmLyupQVQVg64T6EjTIUgBFzcX4aPhPMqNh0qp5ZBmwYGm5XLR4UU0FVgGf3Ca9XH+gK8wIhepzAFVwFVThkE4gzoISQtJcNRoANCkFNEjiBz+jUOJ6wZgv7MokLkRiIEHNcFp6pnSDBw0JaVxBJXoIVfE0af14k8wY8J1HVb+rmwFLqinFVWMxdHzCPKAIJfWkZPcTwI5Vc0ZLUbplwlYh66czdTlQAaBWSnAFciGwaLEkqF2Eqtry6vLacmBBLaorAYH/qZvqJZbPxw63gTajsKmhBEBhfelZcNZn7QyIVc8AErULl2AxUFNReqT5ZqiujKYRKVVwbgohBWOSCMkDTE03oaRCAprsYSm2tM5cInySCCJVh6imIk0ETA47kEwKMM/VZajVCxgKdFdSV410C0uDTGnuWYWzPr9p4RRNwUVKEqEzJgzmnaGsCE1FzUKU7Bz9Erg5B7tRiEmbJ4p6YM08LCKoqqqoraiuFRIEWLxIlgMEqKqplVCWLkANKqoBVKOKyCrMfBaT568zJJNQxqGoCaRBAOiGd80v189NrMIyAM8sWwJU1NQsxtHkQFvCiwhBuCKE4IS4ngFuG5ITdOaYlET9MU3XLVtKnajQYwYUQRQqpU0IldRNElvr4qpLiCVtkejKMeM5hmQSioh6zM89ocLXHfGs0PVn2/whbrncUB3H9Lfn0NYzyZ9ZUxFpBsj2cXji+acbVl248BykxBaUEoIVCxajqgq15Ytl+WJgwdIFEnLpAgCLF1UsLgcoUAFUAqiiqABAZzkupm8WhdhBnxOcNBU1Ttgqt/Ad5g9Xfr9yAi5asJTgohos/QFfALpFDIAwHfCMhCSahJRBxQEhTEFnml8lunRpKgAOpsugZI6CpE8SwlxGNcunqDqLawmKsGGnaK7DpasK10qjVBDC42FXlQ53zLDIy41Jln2OtPniAeEqTPvBnxkGK0LhtgJgB/D80zvdOfMxWjZMnrC1tH4iAAlI1EiIippFtbXlHLIWC2qxqJoDHFiwbOkCoLYclbIKVVVVWMiXTyUzsqXcScfsHAc0FaF464StoNFh+OkqbAXm0KWQQBXVMB8XDY6ZPNQdy9S46rIEVRg0qQvXooRARsMZikwGpMc6Mxy1lxRCEjAuHY0TKSIx1XMy7JQnbWl5Vrjb6pVgAiQV8ElfQApb8RRJTc1joNGj6df/GvhDq/cL1bOkKvxBLe+0X2O/JFhj8fb8bQXbxr84iu8Y1wTHezG/dBNQX0JEmb4MFaSyAqioBkQlygGyqJZXlXMCVKKy2iWoJbyyGjUVVJCqKiwvq5ukLASw67kx2E7GA42iXmLSOkDg52P/2bRqwQKyBBCa/BYDjl7PDk/Q/ak0vyapx12/Sx3PU51kppvUmQjpzKFUVT3NS0cq1K4aERBi81iAE2EocDhIzCbC0u1gIiyIcVLv8kuiiHhCzyAJneqqprpClXCIroheVx37JRC0PaonO3I1+a08FulmhBUDIDsYRoHnQ1kzCyCYBLmlYeL6mXLBUlmF31UClTWoQMVilFeBE4IKWStRAZTXVqIKlVKAoMKbtwJ0DVCvjt2J7fnYVgBCaeFWTK+7xP5J6Tdy4hZPwQIsZfLb568/imMYTkKcEoMxLjhxFC4JIwi70lGUaKbkriaiWUpXj6gvPZGN5AXVb/l4VoIQmFLKoK1FoinNcUgSAgR+L024VJFqUDUNH/TWMCeE0phPiQvD9iv9Tv+Q6ymwFMLkt+RohhdOqWwb8pGP3+VvK5By5gWyrWTzJIkS4FksWFBbUVNVVVmNGlRUAKgSgFwkxbJy1EKguhKQ1aKqUtYAldUAZvJ1s0OVpY3b5LZ8bJNoRhEwYatwHvvgzuNBBCetBBZSkIWAhiHysKCeYwmLkZCieEJKQqjNLENKBuZJnyEUm9t+Ey4U005GNKG7MkWE34FKheUinpKOAKTDiIyGGEsoXEjenqNKaSfDsjNEvKjtRNKc9kz3yqAItvJs1aGcxEhCcx2ZYqwAuzAW/1Ow052NJ0g+SjejhGwBIJajpqKqSgACQDWpqKqqrqwuxzLUYj5ZjErUQlYCkACqgQVL2bNTVlcBxY2ysBkFaBIUTUX1XM5Yc+eRK/DTpwUAlgDfDTg64PDNnA90pR30n+WpdKbbQVN4GS3EBzXqDxPHTYSE2YHLXVWoXCAtZaR1xkOyLVcg6PqslCn8jpcAjFQ4JrlI45FAt8f0uB3s0elC9jiPnyLZgtK09jQuOyPRy88l//Jp5Dv9bE63/d232W1BSbnfZbvGjsVFOzB+x3gUrZrz0hPbJ8jSelJfWj9TBRaSGlShvLYKlRSiWlYBUKrwjFghl89ejYryGl5VVUlllahBJRdzVs70UAVsFMUobESjkMUASgHMQvuP6uX7UWKsR83PR48OUPHFUMGPqi43goJncY8LQDlvGaYug5aRCKp+FeZlxOexVEIYSpxCpMt4MCcWMKNBU5NGgnepAZubbozKgGAxPQbJbNrbH/e5At8bejhLyPi5P5xocf0sLdp+Sd4n1506bSht6fK77DbVzugWXGcSF23PH49dYzEOtKwO+bxxYml9CQVWAEtQXotyBaiqqi0HUKlUoGrpgrQM9EMfLDBhK9WorqmprKmExEXPAotC2pRNaCQSxfUUKMIGMR11/K4vw8jc85Gco1fcyK8/OhyQXwFSemqYd/zWgzJPkZQEuI9TasgABTTLdoyU44PCOLeCJskwgwpIe16um+GzlCQnpqMocS+TSBlK6BKADqc7GtJCthohMtwdpka4/dKwEIK0ZyXDduYJRTL9t2O8NShUIZ1wio3bQQ03gXXTCX5HntwzrhFbAEJw0TOstrZS1lRXV5IqjlpeVY26MoR3jH/178AV77jty56Zs2ApKmp4RTUqIChmeBslytbyTVwpBlDcULJZeBIoW4NBwJVYeWYV4NBj2A8c6v/NDUk/FYqpRFQCT1FAhGlwkgoHO4mEkHlMBKjwOzD9Pp6VjLfm2FmeGg7G0vWI7CH7tLeyVoN0XprkcZ/DIajeDcsASVAoisICiAcjTHgyFBJnu3/I/pb7+p5pQbtDadgJx0EIGIgqnwamY8zu0dhWAEAWqltL6+snQ5lPlwKoRrVEdWU5RXlVVWiWAbDxEPIBJB99r+/lbx6fOXclKiprgBrMW4G1ACWYAaC+QQqBEngSfPXsNbNW+wfeDMzF2F34DoPZ/sGH8Q2OEKLK7ijhknHqENsTTBJKHBtEUbjmI955L9uR8PviNGWznJAwjBQ7HxBSTTlh2vfymBL5MTMpoGlSRlrTLRYwHJIW9rqJkISIsKbFJHh3B3PO+6/67hw7obEkV4nS7T/T28mI5TAwZm+Z+EzaPBCA4HfNhUWiuHETVs1dvoBCLqkQ1ZVVYJXPzuS1eNYEeIP6OPYSgvveVx/8FFqZb2k1ymsxj5fVTeWba1etm44NlFNM3ABgGoBVq2etwaQ9uGhDE/p/M5TsH3oQQ7yvIRmzaUJqChWuYIJIAQdMxMOsCz4bFCGNiwu5msOcyIU0lTPT380z1KTQKEmpnmoOa9e6fMTO65KOyS8odipMSZRrNiUUxE2YajRMiKJkdKLjZHuf6+weTVkXNIVkX+iVshLpDoP2GIBlq5oLyW64AB52x6KpsBkTGZNYOn85UANU19BFMOFV1MzEGkpL8NyYkW8++C6Vf4LYPxxlM9aCAytmAWLztPVy+gYupgLYateBTgdACGgZzgEtvaZeDjrw4FAcxLADA25UDS3mJl1OBSc6XKmAuCpMw5eCzmRMpZ0snAgiqVCt2888klKlrye1/NLTqNSpNFXRM+RvkwkfZ5GEB5/ZxdKklooT3czoMAQL0SRVPUJs2vvUuct6DT/4czyXSiq6kqmYEetS2UuPPTcGT+6ZgxeB0cCM317HGG98M0q2TPbqgOXAIlILZE7GwgSWYclCzMJawMFrD+LP+Ne/78ZwfDpiziSfg4vqpq2HWDR34xQA6+SMCcAarOdldWVr1pQBZduHUUADVYdR9P/mAOjX1wZaMoPnTqd1hvyA7TnCFZrwVC+hgXo0Pe4QwtJdzc08L/wJi/eQCdNnXWanlJTvu3QirlQoeE6/oye0FEnp4U4R5YoQBndTvRSzo6MzmAPkcIXaalJTLxXJYTSR0lSqSsWg0qcn1CR74qUnVs7dgxXzMApoLJZv/C0ItQlFEpumoUyhdDlqAZydtp4vX7S4WlZXrpjnYLPyMn2TKveTu/97+2e49eDQqlPbpm7AmqnrUboRa+gWOQnTsWrORk+QaWvr5BpatnYGNuRjJ65WB9ADwxz4hjhHvx4QtXqeFjKSHRRgDqG+mKM4TCWeIi03ZGmBEOVwouFzZ+MGRabpKkYaHMfnabn0WqokSCrUmX1V21Ardurq0ybxRFpcsmzX5zMMV5BwT+EJEohzqVLCvXP44fRlp39QTzOFO7ov6mToZpK9amLuSN573rZRe/jTxViHv496DkBTUbG2HnT1XEAsWgwsARyQcrkIKzjmYlLDeLz+4Lv/uvu/9NNb8amNu0cgBkxbjykbgVlbJzRutsswZ+WUDTPwO0LW0g1TFWDLNd9jCB1sHxkuDuEO5/xNAdNtPcnVDMMlzJKuRxRwEGiOoamcCgpLVSMI86TPgeqXxDS4NG2NnbxGlSzdCdEs082yh8vj3rdpkZRqeZK39KSG22VbSlv2r9dR95dcgBDQs1ag5Zd3olQj1PGkBSVT0PhNzB4D7B25cWXBi+RpNBVNPUlk/nYUNjeVbprG1sxcOR+EY8W8aUJ5djZnXo2Yt2I55m8owUsP48+4aAT+42qHBwM7n9yzvoRuLNLSsyc0FdWX1XTVzcXUddOBMmDtjPVTN2tATOn/zSGg/9D9A3H7R8CJB1wPWe2ey33Ek4xRRUqDEmq7PuHqupUMqYSLrFTKZzHCmUcNy6AIIBmSJo/paUCKaa5mnLXTM8/qcZWmtXOVdOQy4bqt/iTpSJfp8AsjGk2nAc1zYwoiHQGX+9SkxoXvUsKeXjf9kdf2TiXNo/YAcuWptx7YM66wubmoqR7rZ4LOX44lz1QmsR4AXboALsG8pQtgA0+8S+/9F/70Pv77J1z0Yyhvz9P+BqBpwhJAgGzqrsOkzaBYL+vIjBmYBlr05JHvMBAY5B7DDQIf32b867njkc5TvXpSQh0dAUuVqssFt3Sqp3whL+oLKaYb9CdD5/K0M+ndaXAJfIloV7bf8jPqs5HUAk4LyfLpd4Sc6//tz4pzIZy2DKG7AVc9FWKMRkyZIsFu/UwvvS14/BIrQcJJL41G4VdN5fohDJ1POcCGlYUg2J2aC1DIZqCpqAkXrcYzoEuASZtnr8bK+UuBmhUQwOzNRgGX7//fWyAHhwJfy4F/+AlIiXzaPF7MX46SekwGbts8f/lUTFtftraubC1RJ+DY9xioDTvgfDP04JFbcc/72Bzv4hdIiurS0Qgh0uekdL/kKnd1kjKYlwyRNKpeMP3UTmiJcELIXN0kkiT8aV0Z0orqhGjeZb/0kIkzvSlNCY8mhVBMy8wmce7+dmVue9IIqo7e3n6lXz13+lgeElqcOQE7dHlreiLX94c4W/6r8+qTe6bMxfNPvCBKls9/SGLX+B0oacAUImevBpYtBKBAzl8+d/lcbTGdh5UAeMFuj93zokAP8dVA3HgIuOpX2gdPauN2lCxfPbtRis3JOe6wVqybjmmYsRagGkaYgCYJlJu/GGB89mc/MAnLf3W0BLL8CpVM8CTRpev5PR5ySJqUoThwrpc/L5UISZ+rxAOaReMyIlN+3p7nsaSnZqYJ1pJ3NseIiOE064MWjXTkxfuGiRThpOV+S3J6qMJhVERAtV9PGMFATBJdobQ9ETBDmep3KpuPUaP54wKQ+N2UjRijbisgDQDfMpNgHhYsAWashbJ8pgNdqVZXKRTA1Hom/7pzFHA9cGAYgGPXX45jm0+8BwiQepRgy5wpBw/kYfpGTAFmYMYWgTYDQwQO4MgQHAXCL2HjFLQbKpVcCFVyAgI1IBUj5SfRAOFcJQGCPDWqmgklg0onx1NCqq5Qt8vPsphMmaqbiPt5TrsKn94aiF+a6h7hOUIjkhDd8mIy1KmFw8gwAykWzRHdbUZ7C0EgYLqX2SyvQ8/M+1my0btfxFPYDIx+UQKgePC5/KKmApTQLQCZvWL+MpTVAbMwaw1Qi1rMAbBCmV36fMvzvQ9KNmi/NwzgB20cVYwtK9/D4x6kRIPEml/3o23czikAnp0JTJyFH/sPOzDM62/QAxiuBvqsm+4A520Si8CLalST8HXTBGRmMGUEFRo3QCiVnYrOcEkKdsz/a4DmwjR9GT0V3Qy2pPf1SRbzed2O5EEbarZxZd/uHO4lpYR0qCbT0BahKekZ7XnC/FWc6dHh6q7XFqfC7FDNG/NOOZEQc556AS8AyzJK5VNo7MDIvZBNoIUNmLQZcjWWY05yxloIrJlHlgP63FVEzvEI6p4GPrbBPxcEABv8FcCvwGPBSS8Ds+uLG0qxchPwmYElYUyV2KB3z73qp29uGr7/RvUQAPHp5B7TEQQc1dAUEWJESAkHUBQpNc2RvlSYQkqqW2pSdgsF2YwYVh4lPr90PGKqamYebHBNEEMKFk0Pg5/Pu+ndyLDM7ogbDdsa90dC7WcyssKg0uIhfipppLsec9SA1cvfNoj3yLGDvhDDC8DTz095pn7XU89zh0oAcrzShEK6eeqzmLuytnwVgLK6spkrgCUL52IOsGRhLUL4WHVG4IM7PzVwCAD5Ut5wdEBfXDQnuxQlDWLC1A3XffsfLFyprfYwFU/gJwzZh6GOcSNULsdtAjaW7MZbcHwpaTtQAEGohBmUKYWTBPV0lXKFSJ5lS8VntiZ1r9fxICFeugo7U1N6qI4FGrYMNxNgRBCE7A+JGZetbjgkDCGt1Ld9L1VbVS3G26OJs7k9afwKeUIaUZznNBqMEj/zg708msjd4MvJ2OclpdiAB307UVgMSScTwEM5flc389l5C5dUq6tsugBLnNnlKILt3oWLRuDwYOBLD/Kw9+11uGgVgEaJlXPxLTDoira+M9Zhy8SXMOAoHXrwIICb9gGHNiXnBsHHjRXE1GyuEyYlpTBUDs79KiRlxLSCucTXRs5faL2mPS/LylSvsmko5TlJPXrJhbDbpdsZ7UY0rStIgh4kyFArpaX5I5ZDDAkSvPqKs21HrtD6hhJW3jl+pEtVfs3KArWSV0SC2f5kru2zVDZGSAlsQT2efo5gLkbufRJjRTOKlE1TZ9Bn567E/zyLFRXVqpgPrJpjmmuAf95L7/q7cv+d+HTE4INDMQj7hx8dABwP7Jw3vVeQiFJg2jcfXH/sy56r8NXAQ7/iGp0C9GZ+AHBx+8eYXLd9PMZt/0BKmSKnfOnCZQTSdV2f5ReWCCiJsBJMnlQFiXsBX4JQJayKk4ZPPRMx00RaDqOmrvideFBJRNp7SCmcCG8N5QU+zPm2V5AI1t1ufZP9w8Crv7/K6H/w1l9//FZnErSbZ6Wfd8+fvtSKZFiupDbjL2AULip9jj4NrDyzHnuwCxgnN08UWFvUG1OchmnrywgjNfifObD7vKFq//eGDe3eD9mIEZ+OGIqLFHB8feMVp+dh3XJQAkB8cJ0DzH77C4qeQyClggMYfBi45fMbP94yERF1h8zPf8oRusEhPKkRyYhgzPL5Wv2aVP0CSdk7ecYJZjo+kiZM/WyWwlpaidGnt6pFfbZCVcpEym8Kp0vTFYWL9MQPA9SoI0CJlolT2d941w77y6kraVpPHJRSyU6ybnREUlrHfZe10YhKpOTsBYzejSciDRgD7LaKpvyVvzt6d6FEMyAp6GV7/9uA9bPWzF4JYCEPLgJwdSHw6lvcflv5t/TwyQh8fPt+hQ+BixsPD+6N2u/nbyltQn0px7f9cCiFm4/D6cj0tIPD90Ni4FfuDV+vF88GC4DtzYUhx006LD2NcpoyHC5SPKiQDAkrCEkTocu8FhGw1VTgQkZmUPIe0u59/mrS4WRr3NZUQTz4iKbwgMKtBEu5GR0nPTt6LkKozS2e3v9LEk6ls2DA6dvT07qoEgcc5jpX5Ilg0CcCCa4ygAIKwdYJe6QowsYHJHYXYBtQgs0oq2tAESaq7pTVqHEWL6n1KqL+ZdmvWUw8gLf53Xj/Hpj4XMK6A/uHD/tSEnx9473lkI1FTUVLfwamTR2Cn/tdgb4XQI4OFoPEl/gK3pGl0+pmAtguC6GCR+CQANFJQPg0RqUUDuF+y1RUs4cmLph6uwqFRdItyQOeIJ16tmsFRVzzc666qmGmEemzUmoonoxIkhG8JvdSJAlXVGb1FyPUH+3bzp1RzvkZCajCFelxv+ENvKo7pCetgCOtLDZuJyvY9gIw4XmCp7d3znoLF5HixuKGyUDdgoeHNmELgAVLzWULgrIc5bV44Pmn8dY795MH3rvvnrceuAe3fPYJPrlNAIP23QSBYe8ccSc1CiS2Aze/dx/sw95wyKZlUD3vKAYqh1xko2zHeECgsThuMHhKSIFg8ASI52ZSEI0HCPVMnturO3IurzOtI+w3I7GwDLkypztKhZUePBuwtE47RPwp1x9EUIqofjo3ZQfSnOwjfqjCTfBcZ0B7Z9AOeK6UfqGonpeZ1Z2ibjC7RfcLorrQAmwnFAlgyUL5xB7QWdNOv45x2/A7Ol0xM4aipAGTN8GZuwzaAqC8FsDTz2tSvncf5D/+6r2re/dIeTv2D38l5w66nzj7tEEAijevmIfDg4cc+fY6iw8/OkB004HeoUGAegDfrOPrpo/fLqWERLoSk8wwzYBkrm5LrhuuRn3CASjNNjNSXjzg+Jx0h+pCiwXO+ILUlqbPkPISS0pD9XzEMX3SY3pU0xgNBB1Va81TozaNZ6nXnOvo2fsXixs+KrppUgkIxVLTe9/Wj6crMihdCZliRWhEYfMUgtF7nsQ4rH/owTdRRBqLJTYAaAb0oiZR0rAagIWycDU2TQa0x96wBOA4GPnunf/8p/l/n98SxWPAsC/ITZ96ADbzSXuAwZ+OuOEgBP16APJm/DyEDxa48cCAo/OnYwuQj981HEDCzqAgrudSheueFLYGy9aljDHhpUe7dNMFFynVbzmsM5wRQzApI/EMS8QMx4kI2R0wTASSFoHr2BFCXI/oipTZGV3mqexzds+goBLdv5zJ4Dwhu9qD52ikAz2JS4nr+aNgTbioYCMa/E++CuepGWsxGk1AI4obJ8tk5ntfz5+ILaWYugFzl6NuMzbINbPw2N6HXlDxelc6XhewH8C/23EP/nkvPr0ZH47Af+4CT8OTAEYAQw8OBb69Di04NOQQ8PXQg1gOTMSO8UBTEaSipoJIqYrGdC6J0CjxQRJiBbimyWSmxj01RTXH5xMW13IiGpVqh3EhL66oEaUrPaZosIWftrsZsbCtBBPxDE8Jy5M8LSXSz2o/GGjL/i3Ly7NPtfhZl0EksXLbGdGNRCiuREzFLxmKIZoxfV3JC6+SR15cd2Lk3t0ACpsLZKncXJB5G+bGJm2uhzZvhTE7LX3SmqnYuE0VI182Htj1MPDGQ3v3EuBu4D93McAG/gg47xlW+se3Hxj2pTcMGApAAq/0/+bQ4MOA+tDQTHv6TikBSDSUFIiAiCiOlJQTzQWQIpmmnxGHKtJVMmVmZyygxdJBfEhp4ajm9/R0VcR9hpPihqUTGyTJUz5XUKY5toiYBhxyhZVuR3OTPbPbgrJVM1XZfbajLRyK5QhfayAz047TkDRUS0pFY+DbUEDWAU8Be0euJRKjNEpRACkB323v/jmwEpiykWuoweZJz87EeqUAwON7wAA8tJe4wNt/Ae6C/PfddwH4+9/+/O8//tu64yuCQfj8Fnw5CIf6A3+QNzIPGPx52cIN0zFuBy6iTRKBpA1J0omicOISSaSiWooLASnhBDs1NZARS/kdReOZaoaq+02FtfWCK4SpJ+2AHSQuUmnJcJxqkhMSszMSrKNHW7olLiGyBZnEHz4f0VPGmRaWcniXpeT1Dbtc8xRQT1KPKAykUIBPzluIvSPlC211eOjFcaQZF5UKqF8OQmVtOTZOXgvUBN26MmAa1jB/MaIv+fCiPnLknicBD3jP++vdwNt/wZt/e+0RirsPHR4I7JM3H6SD9tAh+Okq/iN+J+6rw9Tmwp3jdm6XKACQIoYANz2iUCkFkUwCRHOEz2RCqL1bkt0ZnuqJgODRoGsnGTMF6/A70knzcaZLm8CGF+CBpEtjYeYnruf4qLDSSKvuy5Y/Xuedjyci6Md4CDkuJ5YbdZVIBwOBhCO6ejA0A/lkEwDx6iNYX3ZSjCbbcFFRPfLJIOwcp5d3xeWcVeUVy+cvw1p94oapW8wtyHjsBYzCHs7x4qgk9t73T+B1/pfXHjGRgTuBIR99dAdAQMkbnQVA6x0dw6Qjjt341ZwHJ25VdortMh/YRvJh+4kjCacKEaCEEA6FEJcRwiAhRroiAAAgAElEQVQZSTrBDN3pigd4dpw4nFohXVESmaaKjPZIlwwGBHy2lZTEzujKDrrMSmmBoEsdvc1vWEmQKylxU1oio/WEQWWLCJn9+lzu6EquACFGUiPBEMO4ndiOKb3n4dFXXnGengm5O7+wGSgAikx13/luzF0sd05KVkWWzq9+BjPw7FRM3DQR28EAOOooOHjqVY4UwB08sjfwjtinDv5YgHxxMxl+aPBHnWEAP7f8giFHBt3wNVZhwwQA2yS2i4JtTeS/NlF8JqOccMKFQiCtEIgjBCWeo6qK9Md1CSQDGabwDI+oJBCnSowYNlNoQlWp63AzwhOGn3NHlcEAF2E1xydAwoTE0k3eGU7EZLxFBITPSebx2K+XJn0iGpbSIq7qdDLsxPgd+RtXAK/Kx17YOAUPY3thAbZtKwK8lSuxe+uERXMne/W4yALqyqzVs+sEkI8kGgxlFHaPBh59EQ/jhUebdo8eiYs++O/tn47Yd9MBHBry9XdavLEY+cCgQ4OoRHXPkqm4qADb8oECNDNpWIpQIDwqVVeyYJwkQ5wQT3OJZ3fEWUBLBuD1TZ1Mzz4thWhFp5li0mg3I1RJV2xKXScmA2pXIKqnRDTnfNCJhzXLH7CVG5DuBL/T4p4qzg8VPxOHyvC1Pc8nxWkGI5mebPdrnZcxFEhZsK0M2EMexVMAXgcEBAoExEsAfALwd6avsLRnsGz1bAciABnCRfbqEpTjBQ/AczaeGxNFtvb3wJ34z1134pMRUA4NOTjkmwtPoboS+6fEfqSDvgTGVG7a6C/YJiWKCpplUVNR4UTbRpxRqELRbH/CiIHn2BChlFC4GzPSHHFK+DvTiZKrnWy/xnYsLXAuPdQqrVhOkqY6srI909B1O54mopFsLRJQ0z3LF/fFE5nkRN/OvAttWuslJ+9bFcugIhgJKSk3wx9ISokw8V1i68RlANleOIHMayKPvCa9zuNr8KRB5E5cJIBjXe1jG2V306J5i5+ZuxICiSULlyEwflYolZiAhpLamqc2YYs1ZsXE77aof+fib8CHf8THjOAAGYKh6I/q45VLF+xrPTnQOYrBh/vWlwIgsghNRYVoLKrHfhCVK0TziA1ieUmPRywqhQdIYYRzutppruxW9KB71mekCydhnFd6+86kFNPupBlxJer4hZZKeL+lXaqGuLA8R9qqGyTRy0knfoklL3Qf73fht32tmsV7uGev7durNT3qqTKhaa7matwAK2wuRPMEtrroZRBg0uwHPbEDhYVE4qIqcjtePF9WN9PBoplpq/nchepCKOuTFTULV44FFKBiwucTJz5x77zb3MmBePqfP7sVCkBv2bcfQ/D1jf859v1RbcECTP8DvsLArzzUYGOQ8nw0oaixuKG4oQSF3WmGEWPcDLsG4BBFCgVE8wgErHTSIZRENBJwzrpqmhZoCarIUGjSoa6dEGa251pn4+Hc77t7/OpODsTTSV5CUwgoRV+cTLvq+JXHvm/XzC7nhJHHfKchsszTPYKUSycIkBSLhb+VDGgubN6K30ny5OYTb+KJcbQZKCDY8UR3/y/ErTUow0WRBcuewRLUluvT5qxatBi7gEIUNW0dcxI/fXWV2DfJewqf8o/obfiYfHETLroRu7c/FtrxNnDVj4MP3/jV9Uew2ZmCnfnYng9IlDSUAMhyktJEHw+aJEIjHdnEo64wQLw0s/vwVeKXeLwtEZSBSGfmKUcg5+TAE6wj7xwLaord3RZO0JPJn4Z8lH2t9IiZww0aFwEiW39WWrNk+4V+nSFy6ws/HOzTET6lp3JDPZyEDKfSSIpEM2FAZp1jzUBzcSNWZT7+6iOv7Xly9kP8JaCwuVg0Fza/NBo374MDYKaTVgn7mWVAOabPC8CctwIXTY+ipOGP+Ao//YSxH35CRnz4RwC3fyr2Dz9yA0bZicLIs1D+9uNP4Pj6pn1/u9tTd40dt2N8/jZZWNIoAdS/aV0aTKndyUyVMo8SmeVZHom0BYQOLarGT13W6mSbUd3RaVcwrcPqSpw1IoF2M8tOdnXkpjzT76Quy7lCz87KYwoJRE2uk1RQ9OBXHP7lhK0c753epcTiN6I1HCU9jExPzRC6pB0ZskuNWHAzctgk6XE5KUAE8Kp8cstEAGN3yfzGgkJgXBxfOvCvnl1dOXcxquxlSxZWVa1oW7F4wppHcfMXf3k7/ssTDj7E755/+o/v3Yc/Ah/cCcnom+9VVr+IhT2nvM82/R0YcgjYd9NV0/G78dgOoJGK4kYh+0gvTrXeRHSFXehQmOsPEJmZ0m2cyI3HbnHyXGb0U7vFD/1s9bJUL/Pqts6Qflqovw1q8cd/6eVLd7v0c4O4vwW5CX9LW6/ebUqAUeW9VE/n+p966pexTQPyerydawcRH/BVltm3t+4GMoFIhApHF5KRzRNIwwQ5G3vkY9jzJPDXf0iAYFsBAduLQfj3ryWoRNry+VWV1Q6UJfMW4Mgr6MQXeBvb7nxpJK6/4o3Nk/4Rxtv3/fNe4D8EMIZi7C7Ule0Y/xLuwUXDDgDD92eswfp0QvA08rcVoLgJKK6fMLkzjUghwQwphechqmvSCpKgliR5qVx6lPVI/mDnxNNodlcadVQl3BL8sduV8cvTrWssRzNd7egV1/aI9gv/kquEaNqljGSH5b9iZ/Wenk962UcOBK/4wfwqL6bbvSzn0qx04nm6Hum+oMTDSIQUl20CGoq3zkQzeexVeJi/HCAQtIijGc2jIltx9yvAlCxlHlBdzlGOuUvvfwW3xXBNv7fwx6v61ePYsXXWO/e/C/ruvfi7TvAJ/vuI3QNVZfC9/DguGnwYwGCO3psmT3tBPv3CLkgONBeiUcqt79tpsZArUpx4KvX8JOA5aqef8gSJdvlS8kJeTnqMtWUpXGm3crh23sqMBuk5zUycYHEzfrVE7m3BXOtqIxmkXpT4CYFqfnHYvtamF3Tf4C/PXkinv/WFHSOk66qrz8WypCIsSyIzFQr9liGlygBS3Ai+DASPAlsnPPgmBHYgf3t+oUq8rbuGtdz1+merUVFWV+l4viULn17513883PXh44e+/x74MPgPYPRub8D/gUBg70hB7vzgto3v+7IcVAFPALjS+ObwzV8AhwfdmTMZzxE8LwjG7gAKASJL8SQloXYSVEGoTaRrSJlh+jLaFa5EVOIFBwxrif3ls3Pnr43bfRzPpyTdXy3X8fHL+6GHkWo5kxPksT7egF8zg7qAdsk50p51PPG+L8vRT2gy+m3P+HFjeFbLJXafzMMkp9Pfm4BLAkWGiZEIE5lhsykbCS0U/Jmmx/EaGTlh/vd4DED+dmwvBmnEx2P3v39PRRWteKa2fNHi6oXT1uMfT5zsuuflq3I/AfCPOz+YtQYDNbx/z9//8uZIMPHenZgCoKiyGvjgzpv2/QIMwWCXHPnyKaMqW5ViNJ7DrrEAtucXNdTLb34jNEPweLbQGUmpFli3zrsUf8IKgqSzj9tv7X3A3492GxktWTGjO9yWhvjJrstSL+VG86zzl/12uSoD12hX8pie0vBrSIQ7fX0u/fJsotfVmeGc+Lc9rnE7O9STLjmRm2G4GaoUZ3oo4JYZ85FoJjolAzbjf1564pHXX358OSB3A9vzybbGiZunOZ0Y/hm0RYDPnrMYTlndfe+NeW7Yj1fgp58GfXnP+/CNVQH3zvfveZ2/8SDwF3wAYOWhl5uwdMEL71wXHXAUUORhDB58OHchtmPUbuzG6F3YMR5As5ATpihdEqrCQDxKNKEzIoUwjZQiYhnCcmJxEbD17oSSZofOJEPecd7eIyb8LNq7M9nndDLlc1SNumdz2vxcdX1uW1q4Q1gXvkvYam+Id/aJVCaHInWFo2cac/y4kJXB4DCftACbEHmebQSKG0cbTfyJ1x4BwUWvPvVCASSALdPXTdGqKw0swjS/KVZhll13b2J4HAeGpwBwvA9k7pqLV5137gFnDwFvRditnxEcf/mPPv+CdU89zn8YMHw/9gP9iRiZgWWKeOEpjAbG7hy3S0iAlKIjoKmmTIa46xOUcI9bWULzQ0JodogyDYmMeKfQ/Kawcamd9JJuQOgkZfmDscykL51ni8xsJ8NydGZxxdeDWf0J6CVf+lq1QNcp3vtCKux3rCQ3RPRyaSgiJ2U7GcJR7UwPDCQtzjBxS2PBtkJgj9z78MtY8P3rYNgGFAJYB7IbP/dbfcpPPK8OiP0CEj8CQPHu77dOAfrf1ISVeBTvvThKsufpqMgd+OxWfD6mrG7A0cLpiKvXUwnclnKODfzqlnI8g+fG4KJdY8ftHLsjf3s+GkpyHDWlJXUFTCpCQpKgY+gO9xGf5fekEqR99l9yKkQ8j9sy9kseiQeSXency+53UDnnHM9OS0RilpmdNCTj1O6Z1HzdQfU3mfHL8MRpNa6eV2RMd20RSV2aZzHZldGZFc1RNNMJnMkjPUVMRBnIBN44htTgydfk3pENJSPvf2cXUNTUXNw4UWNrSkP9Pn/Js+sAzOHPXv/nd3GjmoV3gSmPXAD+z79+2toZwH27ntP/unvUvyj2D8fXNyLyl+NoxubdoF/jZu0jXI+vkLuMRUpMYPdojAUIxm+TQAlAHCI0osBTJdG6PZIKd2f6HPgsjzHWpUaPXH7Qs5WsTicc+LXdOBNMudkIGzbveeEyt6c/Nx7IMPGbnwSi0u+nWbJNRn7WIrZzPNgz7yNQEkrFqeGk3UQijEUsM+DmtoccrpMgJ2C6BYbNKAGRFQAIQUn5D+886nsOtLReIhiomunFPr5lWnAlUF57Io7gu9d+Z+wbrt+e3bXxzhagDm9AeSksE/KJnTCgUFxEgSoMwJ13TwIwfD/9CNcfG3JoTMAMlWz1NtPRzYU7xuN3BU1FDSVS4UQRklIVhHhCVzQvciaYTaIagoFzaV4s53sZbM+O6V0X4vEgEdJQO3UE7PNdLS26eUNaV6JvWCak15WTVBR6IUdQmtEmVM2X8KPXLypJMMlj3gUtmGeGUoYHD6FIgksQV2vNMrjLMJHLwmb8buTrqD/9Cl4F0ICiJmhVoOunHMH6MixazCZvumnAF/efhRiyH8OMDx7x/4Dl898imPqy7497nmx0Qf99J0Dx9YBv+u8erSPVhb+dPbz/VuVP/z02mKPXTOyC6pVuhoPxaBoLFKAIAgldSpCOzJCfCcUj0t9JKcCgEUNL2iGpHWntzozldfTQrGhKpmuqY2YkPCPP7PVDOLMzgzCd+BJCpohCDZf0ds0/hOiJ3sdbZKrLs6jKFUXxPD/PzFJCfk/1VMuWNrVYtB2a2xZiWWzClmLpAVU9JjwCvPIY8NAb43cUN4KW1tuYS7ERAIW2eNFdw/f1xzu4zcFwPfn8nSevPbuUQgSx9nHgSbD8V0f+8+PbAdx4rD9COAhzFXR3GPsMGHyY2sheERi7ezSaC0GBdalNk+tL61EKaClNek7Ar0swqRDHUgzSwzCpQmAjh9rp0aRCe2u9Th1PXPKjZ/sJQVS3W2js2pvTaYYI2h2KKV0wM0a5r5MEEh3KGX8gnG6ZLZfprsJMXbN1amvZViCR1q4y3aKOYzuKlpXyQ7MNJoCmAuRXAXsJQUPJow52gAAEk705gpfVLVwyZxVE/C43hW8AhN++0YfDwz/AIcheOx7CP+99yxu5PT8fFPd+iM89QABhPPTGkcdffnXgAdymfHh4CGfTMK/5eQ7oQGlj8fQN3toUSrc0cAhX0wXRUh4jkgCUCw+UmCwAzVMFZ8Q0s0QyEj/upkTaBeHXzwkfFDOQm9Mhe7mchYxsL8q4neghFRHQPE8RrhcO+onhkU5F4wFFZLdzxhBRDe6netxLhE1P9wIp1dJcyVlDiSjYNloCkOQhNCxc8tcnXiqSKEX95LWYB4klWDV/eVWhG2od/a3sEXrluq8x9P4U+n8z63JFAPfiAYC9yB4bCbgf3/IpcAOOJZd/fO13L2Pyl3d89AkGH6bO0YfdVYXYtXPcaADFjR6fvhaAVYbNJyFMj0pIRyWgUIggVoZU/XCYz1MgaWcqfolIdLmh1sBvAT2UtLO4lZ0SJ6/LuFwRZ7tpW0pXAgajVsLN1EPREFMDfZN+57dIT/2aLxUWZKczqHN5phtXEun6GTuzu5cV8HjKB8+fFDzI0IAi7B5f1TP8yBtvuN3AP4CmUtQXl8oytgLAPe9jOXD8Y+AWMxDvhgqo5MOrv5m/HBd9RL8vKWk4vBZvqvfj//778QjsY0OuB3DNH37ccEF+hLucxOADA1AJYAfPB3aMf24MJtZDxQZXApMmWk4wiJRfU0EgCHzCVlMh6lFNWuy3vnH7p2DgbLf3W24gqZmBePclqjAEifzYj8pux5dxCXV9n9xw+FrRUw0nTdYVlCx1+oQ4Q2WnmuHKzKjpiM50y3eeDbT8vFsLRtQQMRWaY0EkwKRgKGkgGLOjCqNeB3mscckjrwEgAKmfVDcL2Drh/asezS6795+PvvrItqtURcXXuMf/xnW9no68/vA/78UdWIL+WAs8uOt1zvif/nPXTcCixcD312IqbgH+g0GHh++fsnFJD93NB+CBQzbKhhJskXzNrCWdnJM4GEApiARLMhkECXu8PTcV9Gf1ceR7j2Z/eC5bnsxMCVVRSR68nM6OS4dFetvpHe2dwTa1f9vlbZqVEVIE73bb46LL9Idl3rlTKUK7dV88ec3P12VqmTLstls5HQGXW5SAM8Il9zLPMpCJHAJVW/AwAAKKUfqOrcWNvHgznJqKCRun/LR0wt0JvPrgaw+3aB/jkddwLu+mfX8N92afjngtcjcWPjNt1ZzJm576bjlevvvj27F3JE6VKB0vf4eKms9v/uKOxOGh+/+UjoU7Rm9Ds5ASskkWA2jmE1EHJEmYU09NCV0SBQTU9knFSKaMDDfd7OvfPzjW99fOpNnTDp/LTdN6RkVMxPOc4dnh3iy7Tb/2N6dTyw2Qs+GTl14WvVxmejKnhZ0ylEC6HujjyJ+tVCLyo3b+yhSUFMuUIkvGw0x2hMAUcNVqAyuuB/K3P33lROBNuI/iFXBRREUheP72K9BQMqWuu3ornsw/9+aQ13HXgKOv4dpjrocVrz780Yj33Ffu59aHeB2b0E+g8fF/3f75LSP/vyO4AJCqatgA/J5zz4250zNbwBIqoUi3oMhnowgqNiYp3aIoioBId6MYiK1Iifl9CNKdCiiILMvW7PTMzXN+/J8H+Ai37eq6B9Pw4JY7/ot2wP96znj9BYi1A94dtPZdPnA1sHoI51g6ct6EQQ4V1JOtw3LU41LiJj0uZzmhwRWBumDBmtYNT/uLOIlrltdHfq9DSs54LjnVvnplrfIe+8vLrmF1PnBvvk4TrkSz4YpSc2fddEHsgmzIhZ7KeC5q2UkpVuJX2V+kIc/4854SLlmMUofyvJbWOVv7wgfPvd93A1ZGngAE3nzr/i9wzYviA3x/H64ZiybnH/ms1/cjlwL+X5r6D59pnSn8E9IW+bdb0TV/FC2wB8Bbb80ZhLuP2ON3HQDaGLfsufn02IX09v8CRzq27MReXtcfA7DahYBYM3gNH4JBWDQSEyDbOTkLj+FIKiBUybHtiMOrouLvuh22efQr/v3yccmuCVzH8v/kSslFv1c4dmHdQqmRLB3Insw3JDFyq55MUj2WC5Y2tqv2dM4aZ2Qn6+oRi4ZqCGlwRb6B2YVKFo7ucgHhpgOc51RbljXqYfgAHz2ZB4ZiE/pgzeC+JjBAgDiPfXkfFowDuu658XyqY/qlbBvhfItzQJd9LfZhUx9cc/93AALA1vWfw5ABtMMt/D9tFh4DcLqLtXrIvT1S4ujBbnUBgnX9V2PIajF4NTAYy0YsGTVmpngN1TmvK+ctx8hqsBIiHTZDEKKAOIW5veFQ1bnQP41uzOXzRGQrhXM2UqrDil7yNmq+w8NY5cU6yXAmEPY1PWSZVK4y8kbiID/Vxi2ucW5IxK0uu4xISiR5oC4q6zauVh14IaoKYce5/6p6KV8qarwMwHO2wKsNhpLe26yaKeddQBarhxsxTJ8ybsUwxOAA3gJpHdC6/WG0lBI4NapFH1wzdmHfirY1n+FYG4yg7wCfNJPa7AVuW7/xm5YnsW/f+/ih25H2nQ6E/OPw4rt89ZBVGLJm9ZDVK4cu44tHzbUmA+//6ppBWwhFA+E6q/HKbt6MinTYiKmstrb59YnYdY4Eb4HEC0klM9VLknF9y8LoRRdCV5r9k/fm6yt3HKtvKIIUN2km7S9MRfLtNmoXPOb1QcUNlQXS2brcf1U/2RyZK3UrJG+hK4VCMd7QLrECuaIoAwa8h969BdB76wPAm589+txHbBmWd7sBU36tWd91z9nnP8TPD381ZNL+lG22P0yPNcbYhV9u7YUZKQ/avbYbM1u02d6z6A0A1JOapEyf9crsVJd96BCoq/TbsBuHO2HbtrnhgWLQKpB11mCscoctHbHAxUQsH44XxzDhSjAFPJQaIkEdpoPFiuw/bpBiD9esr7qxo1TsXAkXOMV/p7iputSfESVauG4NV6sCRrtU2BtsHiv+K5D0azb1pC4o6ahRnsqwipRh3kzjpYZeoxcYReG84KKhE7EqWMSlNbGAm/MRYkXS7KVVFBBbmpPPnyDYEnsBX2HosjGL0HoFcDvwzB5U4x6vidDsF08bktOaNc9C/+Ix/Pb965iN1wCke+MfhF+X3pppu9rshXgFu34GwP77aD88WHGwHXrsmMUGrhWrh0D0X7Vy6EvLlthzbT57Ehri57sMR4YlZBBqqQoKXVck6/qY6W3NLkc/jUd8dZIX5HzI5OHK604bkpNgMTtxfetLnvu+V0tsagaFJ/2Pbsthy/Iw29U6XrbM3Bm78ApjqnHQlC9oWVbpU1jGK5DyyC4KzZTuOF6ba9dxTWp8nnGsxaM5zAI2P7DdfQH9Njy/cuQizB8/6+jnaHnyY+C7Tpkf+2E2yIFWJ9DWPdP+uen4IbilDHMnYoUYPo8vVIaAzpj51uT3W+KbakzLxAC05Pjqq0cN3voIesIdu87BEOCl5XzYSj4c8ydizsuAF3eB5/Nw3SyxbY9grkZ002swKtWpjFRatb+3li/njRomimKU8Xp/IZh2byjvkL/UKrilzCgmcqEQsm7nTUGFmmZOeYTqQUcvjqTzVz2ewlt3neGpcNYTdOONHMSoC5dAlY1UQJWygTiNWlUuW4OnTfe7/7RdgN7oueWDFzb0BZYC1tQ/nScv7QVefB8HOvXZ8PKc/gIn0OIomnWY/mn0nrfPNpgKvDtw/XqXPvf5Z08Ox+Rpb70I+B9ZkyESgJPovvPRiXPbifaHt88WazgZBGDlUCx3RyziLuY2xG+8O34hayCZPA8TglMhDNcW4MGMbOl/OlWXi4kn7wmEynR6Q4YLT0F1LSs518NqH91PzzbwwePhHvioGRCXKdFTTi5UlHBIhkh5tY5Oa8hZpdkekW6lF4tSTi4bGX+jhOzhcX/QtUSjeDNuhotLGPptAHoQvnw4tvYSmP7HhmcHvvuK/uqSqRNPAs98zAac2XsA/eZg3Qu9qs1jrY/XK0b4npVDB8pTsZxs6LeRA0/gX29g6chzTTF44ZSp+BfHlychTqDd7ZMWDH53EFa9hKEAFxgzx5kzESDif/+5E8OSlMlxyqmkORBwZcJVqum6t+b6k+CCMFoZLvW5ppTypbJSUaW/XrEc/rvADsl+hxIv1TgorRZpySEOT4XLfCaJukIyRITnmp6sFgX68dtzwaoS3sg678/6bS6shO6wgCUTwdK5PMOGR8RG/Lf1cGwS392P6fwhrr7hodNH3Tu3614QhOe0AbDh+dRfH6B7rtVx/Pe/uA8f3v3uZMzmbj+Qvvj2oa/7rhw6bwJsNP3f1aelmZOB24yDv92a98Nu6x55CCoGrhFDVophgBi1yHImA79B3P4/7OxuJhVhI0AENylMDm4yFUGaypwvSJUYxOstzzfO1los9Ve7UCpXUcLqxL0nT57zN5c8ae6mnPrlEZ9RrOcdcGFXyo2uKiKRNmuljOSm835hx2siYaLmCbF5Y1GFXCgbtZxCuSAEkdHDQc7QxxbYAb482mcL31T74tNkA0bl5wI/9E4C/gm/4xjQVv2w64kRy3Y2lZqcx6Ixu5ad2DeOYtIiZ50P2Gp9ba/LLxmFqeLHe6SnP8gyAA7H7fnnluJUl33Q5nmAwcBQLIPgi0fPmowf9Vt3dt/ZHerBOZItZCfPCaPgCgEhVkbKS2mQtJxhfn7WK6R0jVpQ2cSXKKyQLhZkOhY2OJQMSTRjqY4WRSBoyTbnPlvPiiJXtlWHOXnb4Y3O5y9Gon6h1tOoxypnLC/0AqhcJ1z3RXU9Y/h5SK3LsAm4+ycsXs6/eXBTn/chNr4g20tGeONrgtFfO68Y695iHcbRDt0smK/qG48Ar4459sOxsxAzpsmC248Dvb7pizXW2DkvT51/D7rjhRUSAIt35wcw8v4L+9p0enlJ//f4IFwzAovGLJxrzHn5nl2/CkgHOnU+KFziSdBsANyGlIYl4JXtcMwXl4LRVNnNrnuJxePcuWTrZn39YiiuN7za44djLRoIxZAFt2u9aUmnCWoZElOQLs1mLUUieoXFz4kiFX/E/clm6UI74LP1C9K5ZqY3FnFsGsx40qi8XjNyPoZexP2uyz4M34CvABf8gQ8Grpw//q5irAeKnl/4fNyLm0B334L6sXeATne/japJwIMLJ7+Bf23o99Ujnz05GDNtvG5/3jLffu2guUDvzZ0VC1i16w/IhZA/sgdh9ZDlXHBgLKYV7ehx26+3o9uxY7ZwgSy11IwjwCXZkLhlMk+g5nBxQS2jRdUFhXGvmYVafFWuH7vMXLk4y6sSJeafzQx/nrh5H43K3FEMCFvOUcfPVNniBFK9SsXKuXK40BMvdGSPDW6yaEQwxKSifJEu8qrFDW+JLVifTQ9+B965Y8hZZdcAACAASURBVDOA9Pm6Lz4HSpHEzxMBPPcRJs8E2qXPAx4UvgHgwIEvHl9bv/CIwMypTNZGo99Hj372JIDJeJ2/gqPtwZfmpyfmg/wMTHtpAFoePAjZHbDqpSEgI7FYYK71Gn7b2R3AsTbHSMc+6YBkcsOmAHEYiGkUl6aSpla3Ml3oiddRWWm6yglGDI9eWJEvuimY7vrt2n2FpUmSVRQip0muMFYQkyxvnsS4YphUy3ARSeRV4pbYblKTr7a72FR2Y4GsJ1OArC783rhW1tifUakvamgkyWhv3jOXFlz0w7foi2ueSk+Y9kLV3OeT37JHv5oJ4EjrlqVK49Y7+21oEum24HF82eRIH3dISEC4y4evfu5DG2sHTXU89lS8a/69M0Nefivb1tkHIIkwTj67fuHA99YMXomhw4DRizARwK2/7uwOtIHocHRakLuOI7gwhY9nueNR82macvhVKURZJPB3LoOIaxL5Jr6T1pZVd7tcWRnSohmkQgLCDZveZMD2uPmE42FC2GaJ6xLOszWOR/ozkmThnJcXBQ1PKBOAx7qqZwqjoiExTObjtnGyoCQRZdJGAO2Odhn1qfoQNsVfABrMHokPeuND3LsO/dc983F7ZS9O4vEvem5H+GkDwPXnu2mZ1bhmgbNiCBzpAwNT31Bfe9tyuwBYBLfqKLruQeP5CKJZEca+74gV4CswbNmIMZjxOoDbcbAjDpG2x9o8mtGgwCu4R5VIwPWy+M3VXp5l3qs8r7q1ASpfUMJR44bLpqVHs08b5wU76s9FMjmfpCSjjiaEI1FLU5IqkPIlfrdMIueqCt181v07Wjdy7HIvA15HaODF8IVzwqCON6FmapsYkUISLzDZVw9vBGSoEI9gUx+8ffHdKiwFyNiFHX545gKfMA/y3h7l57p8ge3Agam4hmE3Hh4anPXWm+MWDcMqwfsDUMpeeA3Y3nOVPRLpb9Buz93ZPdjwd5d9zwAvrhmCfy0TwFwDwM7u6HiYtMNxAdenpVzZqzJhOiq3XdWt8lPfVaJL5RFeGfRJSZnipDdWWJrNs45NcFaXGIlGLZ+cEW7Wnxc+wSlXXaUs4S1WRaCaZfPpnJlWfYR1rPN3pl7oouUIJ23TVGFYSKpjJDw299uikBhKVKgMbu/Ndzn3L4LAN32A1555wBj47kur5DQOIR34YNT0stWtd9xl7xt1tpKR+5NP4foL5wCoksCbAFlGXlrbH1g24vUxH4xcitS3VRwzTrY6cQSEtDzZDz0wBcDgFRgGLOOjFo3hUz57Et33KQQEJyCOTpazJvVIXAiFUqHDoIZk+Lic1pvQdMBnlTjRsibsrzu8ZysZuT+59+rZG8y6iqJKgoWr/EQleZ9gLpFkFOfCgZRKOZPLy4NelxCr5oKsqUmzLBb1B8rjmsaFlOWkhIeNQD0CkqoLIdhD3z74wDY8BfTDI5v6AB8/+Gl/rOpmlnVqWYHvseT1Or224u6fcTlz42ezLl9GaxP/ostemSJNxWgswaC1Un956ciC2XRBE09MjEb28gl0ijKDAXN2AHhPDBy2cvnw5WIUxHwL+m+37ukCHG1zohWOUpUL1ZGCquMopiSEwqnC/bQ6UlbbsoL54nXNOlc6Ju4mNxzM3OhpdvlyRbmSvK7u1doSTqTaiFupBxJUtyBbboFcYMuBmLDzucvBYFWUGexQnZbno+ymOiJVWVIneb2oKCDUkfSIK8d1j5yuayYKKMMDW4C7NWBDv6/7bE4+iy3gLwdfu+fHl1bhvkei2oxpTfHwP133bHztbcSW4/EvOt5C9gCfYBbenKFOXMCBQeswBNDHAbtt7gDvtGkDlXHpaMuTLz8xqVQaAKwcCgiBJaOAxQp+YzjG2x4XQNuTcL0xQ9dKQowoii3MnO0Sf/BKrl6jampKuiHTpmn3n2ymSXPpSiyZRSgnWelSgUA8SlUVOvfmqZdxn2QwjxyxuZOpqs0h7WdcUipJob9uPalSN0okfp0QOksIElQgsXqc5wJC8RLKLIGe238CQL7gm8mzeLZ22wfAHT/iEjo5Wbvo8WwQG+/KAtd9d/8/M64ub+tQDXjJDCx+k74OjANWDOu/ZjCWDwdAegB4681jbcgu3IcOh4DPZwr67kAIACMWY9QCPmE0dtNOEORoawAnW94vcko2d6qOR+XEJpLit3OKXZ7SMxVuIpC1iyqvC8ZwNisiBXW/+6fl1eoUqJYL+sw8l2naNsygToktWSlG0naEaEV+k6Ru/INIxIVboIZoUNAiWeOxv5nSiKV8XKofEJTUBoleScJcijH+PbYDmFH89JePbeq9od/6nk9+Nn4+7b+u4p4f0WsLMGj2G+eOnBsa8Nw/aG3j5bfsRedfgVXPL36FYIY8ad6EecOWwALoewMAaddtQHkLRRxFb/N/uGbkZGDtGu4Cy0aMXjx63CwA5BYAtDVOtAJOTVA101PX0GVFEMEFXK4U0zwtKq8sLLruopB8tVcbnTt9Q4LXP29q+bQh+7Jh03AVxUPgl+2wZvhNUAgacPSsZF2oLnOPKYT5IXz+uChTK3S5gQnhby5d0uC4ru0mg3FFgOYCetwOFrDvAfTY0Y9QkG/64JrtY0bPx8+PPlpm99gReuSbx/7GtJ7npkzHT3Dmj8fedkf2o+NBOC/wqW/xSSCzJyD9GrBg6NKlI78+3R8z5EnoeBDY3Ovun274a2ElFnsgXlqO5cOXuaMXjQHW9Wc4wdvgeGt6WrQ4FS+VVME1QgkoIRDhZImwdTflS9hWVShn5BNNLydb3yjliw46rX6re8MZJenWWGFaXZTXuUlJVs7mAwk/pzmNFZ0u0HjDmrZ/1E2rMjeU8uKbKu06HoiMRyJWQyMZqIjGijUW9vJ4gV7rKxAXrme3EFn67z2UEjy6CVtSeFxb9PSMyn1xvu+unx+sk0TuJzzzMab/evshtDjW7CzQyrv3IJ7YAIC406eMB8aew5hF43Bi7aS+AwDv2JbSwbZHAfoT/npy7MylGPyuACCWYvSiMQuMqf13dwLaHCNAC5xGi1uuypITK6bElCVbAamOZnLNKzOUK1KZ/8ZkhRvUkqEbfx969JBocewMd1Du5WpAMhKZFsRldoWsx2pLzBhDIoJGh1pK3h/LpDr/UC1AWagRqwr4IaiwPJbE4AbydQwfInHNa4Q5jCthb0Ut29v11//crq2f3R/os+0BzHgd+GTuUlxDsGVLv8EO8PGzrSbevvXrGeNb+YAj6Nad7jAe+QZ4Y8oUXFO4EIsArF0wDk2BsTgJHL2N+jff6vs+s3QkVrw7cNWqlzACwJjFo4HPZVzTBidxGjjVIpjymJJGqKnJXhOKlc0VFqq5ppwkrbosni9xdNdSdb288dc3nDjuM4jqc9MskJCCdlxTLMnvtQtDmktktx730aaRmri/haGTdJYGLwkpU7dG9koGi8RDatqf1nWT+L2yJGAYufpKCdA1zbAH/+u2GZOATX2wOVOJh9G24qlk4LN2P93zIzY89SmA9UuAXrMn4QTQRjkgdgKbemNQYL6LqYo2zge8MQ1LIupXpC+A5jI73H5XDxm/3d9261aAENDBWCmGYfHo0ZjhPgGcanGStKBnmp8UrUCUbNrLqer1EBOu7Q1nE9dV5JKBmliDXDRop42QrRephfF/bvwz1tg1KhRH9vI4F3aIpcKOToWsmSwegRJRCUlXKJlTMrMUHpQTjFZ6/9DruYTQlAeGrOiVIZNkEqKIZqOkltapCYurBsM1HJheMqjPFpCngeR/W86/p9BtHurNJ3qnfgqMWYRRc0ufatr7MjuIYx0B3PlLz813r8UYe/LU6fZ8jhn2G7w2J5sEy6p/SlG0l7DjwYdTggFYBbF2EFbzYcuFC+B1bL1OuvEkafH7TafhcByeVChd5nI87dUlTonL3eT1TjJW6F4p4UrYG3Pkmkguf7R5WCrafYX50pIgoFYg468pzKm2DNdmxDRJLWc52SR2pHo/pVCkTH1HMQSLhupxIpSkl3LVSECjkOV6PinuI1V10t64n1ulrMMhdNj7pDxlBTZDYP2zj32Javw4aC0abcbWicAj30QBTNy45UF0IF33dDh4iwQH2/FTn03mookQDiF4HZg69TVpGpbmj++5+WjHg50ARn8B7vp57SBcswZs9RAsXjh2dnZar7MAJWcFbj7SFpATZsBUVRm2oUAAacf1VXuDeqy+BENkZclCNO2oZxr79nskYhZVyUJKE15ApCt6yIHwOIRIWt51aVrSJV/+eCBJqTdHDS+1PeGsLpvSpVKPBOaE4/ALJQuZQaGkntCy9d2kLjO0P3wInw1+0wanDwJzXr673ey+ibVdkwDmTQBK+5ePXjx2YemmmzUOAXS093blALrn+q6EC87EuLkT5xIVb695d+DI4RVtj7Zjt5pt6m585J4fYYMtUwav5RiM1SuES+Zb0wBCfsdNZ/iJVu1OtDoB5GSTZOo6NjjlktCpohplCbki2RC8pFYUkky5TzdQummjJiBs2CJWIFVEgjmXpQKuzJmgzJUVNckb0pr8pQozkmWOUBkVmg3bZqZkUwaDygkQwrkGEKLDAUcm1wCFYM6xdkeANVMJRO/NSF4BYsMv/4I9Lb4DJmBk1MytxMQrODWjDeV0b1siDsHFNa4d6x1eAEwGJASGrAMGfwCc3dcaR3DNscfFj8Cva4mgwBAAYhiumfna2zuouOk0eItTQKtjrY44PAEjnyHEEZLLCWeIqTYxZGKZrCLjiZo5K+BUSqcsmXJKapiQiQv7nzquHQsw6ios52pW2NXAXCV65eyNR0sSjPmqNUl4Mnm9HhFUxaXGVawklnNsKZqtp1JhMYfkdJQHo7EgE+BtS+rIZNSGh78VIK+AvPt86q7Cvw7cV+PrMG8p8Prw5XOBn3EMuKV76mB7YD/QWSIClzc/+dlUAGzZkM84PnlaXjk0huPoouTYPnzxWC/+XXe8AKzrj2s4ltrjIN5Gjwv2qZvRAv/iRwzBOSupIxMh24SAUI0YdoqpeYXV+OCcK6LXx92MZP5cr2VltcJEXrV9iiNpKSpwmZiRNEziUzwKl6OOlFdibRrn00qO2VY9x1Vh4TyRG1Z7Yhrj0ZD9ex1hS5UNbMmyFFNnCYsHDXa83ZE22XXAMkBQkNmHK158H0AX2HzHjhHH6nwpSnHNho4HOzl77zgG0nl/26Oge4D7fm1dhkFrJ88ctQhZDU/DGAoBdIF0CB0bf/rlQ99i504sHm1gBRHDhi1zBcBfLh5//R8tcJq6LY623dsOQxqWhWPZZKkrgQsKItKGbEjeCirZ3MhrlusRpX/rXqfo28YJUpCtX+VKOa5IwqOk/Mbf/lxxJpr206SuxasKfd4aEZOvQPLAreeDTTKR45GoMESUeO2shwtd4cjoMX+NFKFh7nf0w/G/GDicPIARG/AwNvUDsBO43873yPOb71n42JfIpCY1HDav5ntweoeC23YBRwEJ6MpxHFj7zMzx+hhYAwAMmDvR7XCIQAASnjW/wH3fY/UQxGYNW8iwnI9eADhu5Exz9yT4zceRwbZ7f1j9MIeTt+LCH7YNxZtx4cDIqoxpeZ5WsnX0VCAVanjq+ho7y2lWsRXuEDvowL2qc7c0RCy3xIxXeZtU2DTpusQ9GSEQ4TIFhmk5uvc6n2vE5Iow4zrnhMQCbqR9mR4Fv1Tv17aV2w42MVn7wy3tg703T5oNfE36AG++BbTN7bhd7AFOo8HdPPPe6PjMCTOP4VBb+j124V/tXdz6W1c8/sWk2R/PeRmAjHUS63cjTgJ7O+1ofxjupwDEQ98Owaj4gsmxGTAo+Gw7T6PNz9itgaOtf7rtkRloG/xHrnKKiObR4HcJ1flNbpyaOU0SrDBzxdfgBA/dWBYvPdLi1lwiTcsL4n5HpVUeJeK4mSZurqlMvUK9wZEjqsTyxfub3nn5bFnDU37TjVZLLY9lz0eygmmIWSFOU946dp5Gzzazy1M31jl308HLN3X9Nsj8t+1qiRxmL+Jf9f3668SAt/DAtqz+UPVuoN0RzMd/3pt7bPGUZya/e/FtKt39052/tD/cgR/udCvvuAd+6Jj68juvYpmB/kt98AL/O773Ag633bcP1zAVMycHZmDm6FF0PN7Ivvnm25iL5keAY21wN84B9Tf3cE05J1UylzuObFmimhk8q4fSRiE38w4VvAg1bpjKbS7GslJxrtixPDktlK7lTjhX4NGTAepRVBg0JfmoV4Rusi4Iu9EFGdBIBVOLVc4CaZ4WVRYvysaDeW+1r1Y3uPgjczlxrrR+fYXtePKudBcT/fJCfNEXWLf1m213/pPy725d9FMImFS9bu55Z8B0oPLtge+iB34Bb3uoHVzsuQVgr76JqXh19qQRAEauxx27u/l7dL4FSHc6gIc3wmIAfHhjmrQAg9ZOG4q3MLxXvVbtjgF4f9tXp3rsUB6XhJrOy5YvLwQ1vESr8FhcpPzUcaUQsk14OnLeyds0bSULTduVJZOn45G8izrlSZ35M3nTMEsyHtWwgixn3h/z9+h8kl04HhFlxB+3WCRrlKRoTqI0m06GJP3PUNd6Rv464Gp4z3UNTiQVyt4xPsN92XYbgC8I8PkTz05745eb3NxzH3UbtqJ7bPbo/lfWYCjwJr/n3Z6ZHUAHcRhH2tLf7vwFd3MLr7OpMwkwl8qavFXvtrMDcDrd5U/0Ub68/zu+YU1q0Zj54xGcPHPtIDs14D1UGuKwzHk7bP/qHnnbY4S0MhRUZTPMzGhEznurvU1qq5mbu64s1zp9KVYYxZUYVaJxws9pxLKF4VBSypJMTRTkstxClNEAUSSL+mS5npoOOTGpW/oGH/Yq1abgqsdulAvX8oiZDMBOaRVupSEkjfNfj1je43JxjhKGtzCmtmoXpk+BwNd9sR648XdgJ3bvfjB+GuVf4v7vVr4xzf3rR2wH0HN7J3TeL7vdfrnve7J2Aih/azIWjJu4cBSu+dn99XaYXX7fOWQT7nbxCwYvHIPxmPMyMNm0lgKD1n51Um6FI8AX8H0D/UO0SUeAqpwpeSAcNeWwv1mmiJrlNJRI8+J8eZrnlXhhzv3Lq/nKy6NyxuQ2l12D2YW1DnEoJ3mzJBGgV30JtaDYNVXbTJeGRd2M7HoKSmwpKUy3XloqyQesggDNBCUqxxAkxZKTLiCGzdiri8YsAtocA9gjoFjX/4UPVLQM8d24bQuA4jv+62LAtNmTMHz5mL+2YHsXjv040FG99/vOFugUaxZmTxo3FxT4RO3rCAl72+EmfPg8crsf2DZqydjlrhAcs4TF1LELp1dgP8Xpm9vh5tPPrgfYk9Llpql0iVLhwmWmLkSg4d9cTfJQPh9IKdF//PVjsutTr95oFZmGl6rJsI204pGFKhee86jUb8lMc3J+QYuVkModIWUKE9lyj4iGL9TNnQ+naIErBOd+oTI1AapEiymKLv11fkdbEkhxv+myMTkAgw/3mrKeAg+/L60+DQ5d+wkddwGd7WUABR9FgeWoLkXbo+y3DoeAg3hg2IrOQ6S8gjkEmIhrhLPxvu/u3sFxhDF1+fDd2AbfEkFGYeG4+Xzi2wSe16dMQmfg9P7OOI31z3w8ZtFTqpIzQiQh5blKJQIi1amt5rKmEWSbVRjK5UA4Txn30tqcVmuU1po2S/lL9SSljr/hJSHlFcikiEtUSKpwFC0nBXg4xOqoGf1CJK35vIJkhYdGeJ4ReOygz46bTnH0ktoKJVf1QIqobBGWjFrXH3ib0E0EcSf+O062oHm0EgD234WnP9l+vysSU6dO+KdkPo5CwOlwCLhNWdGnzDQXYMbL8zB34sKxq4e4/QAXrrSPtzrl3odmZ7vu0UcBWDRmwTjMnjRj2psMnk1Nb8LNQM8QIx+NXfgclU82u9EJVVKVEaoSQp3478KbUfNIK7Zj2hGoPFdVIBLBQFYusVld14rGy3TXlQuE6ZimxwOQWDAVoorfMFywRFAymUnd+w4ebXHVq+ccHkBtqEyOCpHzMtvTumEmdqmi4vcQqQyoVxVVZs3Oup/bb3+3+7UNAui9PiX9hBan7qpt72qDTzv71e6fPLzxu4ckvnx12ang+ysO3v4rjuGutL3raSi2w/D2a5gAirEYgueArx/Enb+KDuBNL+2ZeHZPOxnLBBcYB0yaPQl8CqYCZ5qfON/X8ymGjDae/ghoVsetZxcpnBIBWDQlBTMZeGsJ1YJljqxe0j2+cEridQvKTgW77qVcTdgeJW1rHluxHaYSK6NwygzJMYKWl9uePBOuqcZLGhSkvTFZ8gkunIxVVFOgMq4GJF5VmCzKW56QlpMNWnA1zOxbx+JzdTfQb3NvbMkYEx/69lTnn1udwJ1r0KNpeQMkMNyIvb9oCHD7iwBBJ+uYtb99N8/Ipe2o9NZr74CR8QvGLRuxRqbP98U2mQscI2h4+c6q85KGEQAWueMxy51uTQd+DWg3wbj4xOfAauCpgXSN/XcpK1KNtCEIiGobtmNImuHIISmdipZbDZyEJ6jGFG/l6SK2i1KhMS4s1bE9niSjkhCaxojjdT1EFjpsxhUurjYizpX6d36qerSsbDoFRW7UibqWXNcn1QtoWd24eEGTLC+3RcqfYReK3phGXXS9vxHBFvrUEnyLtgSi3ZFfRizDOahwH7KEeG4M0CoHYAeohJ0Qu3cDR1q+Mx2vArPA4WLwuy9+8vQ34p5fukKIU6z+r1sbHjwIYMkoMX6Owye/6WJ7FDL93bl06fNnVF2Z/7TCbGiO4qWukpNBFIuqatYhlBhCWC1lH0rjlQ3TiiVE3rA9tTk7KguXSrJeheuJEYwSiVKS9YJwuNzyO15HEdlAxhIiyOJ3bo2FYl6H+iwh+xyuplylIAqZetVLlyjN6ySXU5ntZciYIJM67LmPPogHt3761FtvtpP3tTx5U9ue5Uh1c/Zh591a8V+b5ktjTpzAMx/fYxkMwBGMnw+wKVPeeRV4BROApWwg+Cei1/cA2uKkOC8fKeq5fRlGjMLYeRx86lSgJ3CKIH9xKT4eshQgyirAypjpQN5mnNquo5kSSaeVLLxJES3Pmh4lrBSkteK/iuRobRPtn3qZvAUWjHLZ77nqEEaCWREQyMMLxlSDUwEPMnYi10B4Ai23+zUvbCkSzwE8XaWk5GxUJ8hfJB7CqBnJKiwEBnf2J48uGtP+jU+2PAg8hVhHOYOTkG+88iGO3JEF8BPw5NDxszDrFXyMzB5c0ynYdD6A916f/irmTpwlTVwwGsA6l4hNQgCHJXETLi8d6QXhi4UYOwHA1Nf4O2ebHW+BU51eH2h9xDB4zQYAT15I00KEGlzKQ3Y5aCKW0zJBieRuvJIIXAlleW24bjJWhxlSTccaPWtniqWM6iaCRKaWPyiSDDBlHZRxK+VxiXAlUVQbCIuMVNjBV2kTTu0Kw7Soo0V4/ZZlkXim/snKnCOxLP3LXyubaQaOp78uv+fHKXgQX9sfx0f22AGg8361EG3k/7bC//sM77wCc+FY9El13wmA/dQKwJ03SuRtZeIc7mDc/PFY3f8D/ujX7K5fbxdtjp++uf7/oC4avmz0gnGYJxzXfvutH5qBA/m1g4a/25+MWQS8ZL9nC24QImX8DK6QbDuQ9rrUkp3gn4XIFJicWk6MOEaEXomY9a4kFDUWrC5NsBItl887JX7CFUfljuERVGiccoOwnJwVjqxrZS2SiWA8ZMgqZOG4tt/0/65wF3leGPcnic8KiQJb2Kwhx0d9gfbTAfQFMHkmgLZO5/139draBSfQke0FXmiQRa/NATwQqsMKvul4cM+z8998Cx6V0cmYNwHAHAKItfwF9N0Csds90vrkmeb/gYYlZCEAMRFvCzD595sEUFsB0p+IRcDQlcDXDRXYPn/wgkOEJJhiUQpv1qZokNT8Nimwk5mgq0cKsrh8IOChoTqsXDru9SFbbCtplVEl7ecpr+Elao0uOLUsJRdo4Mp5r5k4ldMQIAoue3TTnxEOkxtcEvlyWqGRJBF23CfXEJ5n0nk8h7WDDk++6VlsJg+uvIj2h1seRXf83KPLPtwSFwK3RT4YKY8seBPNy7YBuG1Xp7rrX3sL922dPGUWMAF4h7w8F0vcl977kDF7ew/sE2h5+kxzMEL52EULx04EhITXgAPtzuV/n4JleHH5Y4pvJZ7/8IWdPnrFa7sacWQGyHolSVM3G1arg2nbpPm4T3BDv+yX0XFnnSNyLGjYaqA6pEfLnLhPI7TWww1CwzmvFXKZEBqzg7VyochnSxpelmKE8oy/+KpB8h7JpWcvtrzSrO7vkTzX8jVcoUmmZIqYjGvIU5/O/Ohb0nszyBwcBgXyLU86+7rsY3/g7jt/uR3eWbjmDDoc6mIZOPDkKExNHus/E69gwTiATsLEBcCqAcAnkvvzXaI9jgucP/bYgtHAmCWL+LjZjoTZ6pgYmmLt2IXA+/gSeN7+8EVbZjmdENXOCccFbOK4EqfCzhum7IOrRF2/i4TuFZmLqlQv6eRsS2rsIyrqJ/+pK4WoArfQsCzNligcKwBXcgN5oeiawH+OHXJpnkqiLneEIwnFzMWSh9Pkz3CeCBFKctnmAZudveEvfPoUund5Dhs39cZLeLDywPEeyYMAuspddrXTpZ+eiuEMemX/t2Zwm0PYh1a477MJb6MfWzdDnoRxC90Jk2YROg7LXlotDeSPAuBA61M3/349xg1bMWX6KCzGpDlkCUbNzP5BK/cLPMOcj/ttwIcAT/lrE6aZlTTFBqe2nZP8ObMonETEk8mppqdGl3xaTDvTqDzbrN2XVkDPS2k5GVVoZYoxNevPhkjWCKeJnJLCMnFML0nXmBE7ea6tQa9vlKqKkKTE68Y1magguWwHWrlf2DZzfLkoobLLU6yJe+cvFKBz8M3D2Nwbr50+cLO2A2313b8B6HAIuOVTdPsTW+8fNBjHcM0J6KPIFOfA6dWsGQAACqFJREFUL2+8jrmUkgkLxr0ye+L88cAQ4JnPnwC67ZPbA2iPx1fgEjAlNYu+vHQkID96vglez+HjFxOPbBjwHoAPuydFFaOqSi2bCy67qiJp2YSQHEdieTfpFEpWxQ1uVbxO8VV3fzYayjMPMnqUIHTuuhLhZHWHcD2tWvADBtFqIedytKY4KmDpf58nLRPVYTec8lIj7HJHzkq7axO5xNUGCSVgxf0CdoKdv+6XOU9gwrzRLfjGh3t/UTUC3XYD0Pmtv91hpw3gtl1DVu++DXdF1vbc3uIU0PwMvnlZ8HeAae+8CjJ2/gKB2WLW+IUjltFhwBO4hreHwE0Xr8s88k0tHj33OUa+ifeyoyYepECsGshsHTDw3WeYsqbn9qZCcwwWVoXEJQgIW7iS4HaAJ4nX1oyKqO33UilqlpVclI1U1pKyiierCV7utTmzQTiY8GWFmw1aCvHFzRhJc6ewIi8SUrsvU0e1lMGrA5IcgT8bcJtTb2Wsuq6W8euE6UTNJsCa2u0Evu57Bx8EYKs54qXMn62PNz8j7b4V/wVatT+cQTnQ/PoP77t5O061ciB13zkKLNmuTpu3MWviPIzHfFBXYOwiPgJrmdIP+KUrwE9RF94v4eubzwNLezdYBnzW8SywsvtD1IacxscD+ZOfAbgqVJ/H5g4Rji5Vp/yVhBpMUi1VtvxJRjO+csdX3wjWXK2f9uQcSAGtIVjyZJ1SBkV2mMOpoK4gXOJq0ssUq6FbzyNSvD51G/0lM59t5JOa71KDcL1c0fn2FeXZmFBtJqc1iXFXt1l4//WToM39GWsGfyt6bcQqXHOm4078BuCWvcDR9v8AjV590fkeLU6dADr4B8pzcM0ts/DKHAFgPCYCc14es2jZCMGdT7WH78TeW1qfFI1Ripf4mie39diBP3I42NEHY83gW7VvgeFZ+1GPbGbRkVLdUJWADIkSGdRMZN2Iq9hUzrnMkkqYRKuccLjRhYBeHlfdYsPhfjOY4HZ71cOQpoRbDrEtgwYc1THTUe5oWsC2qZz1iz9FqbdG81RaBqv4J9ejfZlca6hXy/WoqIpkbdMjm1lPkl3XiE1v+QAeWYqvyUPbewJ9/1CPdDzY4RBw62+01Qm0C6XxhDvjkqfDIdH8DFqGfD6G+8pbfEJfwTxO5lEqMBYQmOuOWTYYwJf44d5bDjJOgIWPrRoBz1M54Nw5dIQfrft1/u0BPPLNcuDR9aOzW6DUF4wSS/LKDmdCMoij5FyPjUzGge3IFYVMhNN+x214yaMEMqLWpsU+H0smy1sQmpbCnHCNCmTDrqB5qvi1ZCTFLtbP6Q0ZJ7mrDf/w+pinIKf7S3ngfK7Jhf8dvFQoYlRJcJfYSpbXKOwzAN8AOlb13bS5J6ZM/7r1EdigHW381m030Hk/0OPzz4etRWtIJ5qcP3kSr8y643ucAH9HAibMHbeYj5uP8QQT50EsH471j319L8AFcKoFMIAMW4F/PbANu5wvfl6De7bhG1zzFRY/je6sOqKmOCM6ow6TcmC0mknclqmAUV8UlFQ4TiiTVivdi/VtKrlZX5Bw7UymqDCscJ8E5DyJKJdC4EQjRiyU1lxdhPO+WEQATSMBSCQvGzr1JPISbeaU/OwpyRdapuspvOK9lPeqaZX1auCd+8jmykHAV32AjQ8Dx9Ge4gBa6r1rgRb70XWPuL9kxb0/HG91Auc7HMKLs9rnny3SGSFiwlxMXDAai8Dni3mCLxAc7/FP+uL7+zoDOAUEzPeGAnjc+Wbb50/cBtzW1f8Drum5/YnCZXCx8w5PoddORn2yA8mUcr58TtaoJQtdrtQJccxkRLvqiFw2KQX4Vd2N5D3F5J8rilykM0JE2udFKKtQiTNhCuEVPEuZYye1eJrFCppInoBJ5LRMc/kCVpD9Tb/4u+sH9bneWon5QwFTY2Bbn3OwoTcAV2BrL4Jn1wOH0eHQ7b/esrnHfd+HgT0IbAFU3CxaABS3nMZ1X+3FG+Q1ABPnAgvHUozBv+aPWooB73PAAfZ1QQvgvb5wh66Ehw0UT/x8asz93z2wDQO8tR9vh3/ZyKWfAxbNOJpSVZYPu8LRQVg45lCZGSb8IslchP1pv+JXI8Ku5jSDvH414RZcF3KSXmJZsuzNQvIEspBS8DBTEDgW5dAcj9GgovLqJZnmo/mQ63oYN80GqepzjavqpBSvnSWmP1e/zBOLVDB89BT6fVvbfyR/bHMvAOuBduLoIfzaWvTZdC924ZogXijK3V57siU/3a7bbry0CniTvDFNIpMxEcCiUViEMfPHLwBGYs2LwEbyo9zlYMdTLc43+Rprho10Vo4hDu56Bt9BBt57IYn/lL6Lf91msJqgpnvpHzanPq64bsJ2JCcd8eeynlCZWqAT2akutWuKvCdpYTX3Ot4bFOalqRKS8fkIzKxiSMQRsjcjsipM6pFydSwv8YZ8HR1/oPOvZziD6gQ5cc4er6xXLMmqV0olWT5qBsDStNJkXfZ9Cjz0JZYu+Ypsk/Kv/2aYR3Cbu6dnhcta/ACgz6ZmiceuBhLUbXsUKA21kRfh9RlkKt6YSWcJRkH4YozGYoJxCxePBseH8sPAz+iIFnB/Xq58vgIYPx/jMGgtXkpvwADjAyAQfGXWUgC72huQ43E74pWFIFRy1LRBPdmISz1Jl5mu7HHiWjbhuRpIUDceonWzIb9cq6sS8WQpoTC8FITLsCU/gezIpsxJWr5KtdI/TYm6bqNtSoHXkkguQtyUpKhpb4ZlYRbyoK8mAkkjObavC2v42EOPAaO+ehA/uDPwL5djOzp93fKeH8eU/T3gdEXLL/WNuKZLweaR6ryXFfqGeIuSyTMnz+bi5fnjsQggBEtGARBr8TS+le4CDkrSjUfDtS98MEKTJ7nzRi8evnzVs+hvJQEU+d8B/pPfh8NdDccXqYkZFpDVmWsGLMkgLjeDhHgIjfCyK63DFS2dy6AFngRJ+4MqShRaIIwAZNtjcUGR0VTYGvE4tpwIchkhjUq2HvJKkoiGa7NUaHLYybuykq70SdRKMiKK/Cnzr9q8P0lYO5dv2ABg3AKBLe6jk2fimj24JvTcRyefWDTuj3xB1fvYCNxzIRzd+tLSCaCCcPImMH3yrEmzpHnjF1CyePSopSuGLRMjMXTt8xuIS36497eOALR1AJbhmgkCkdfeXv9i/lOgz6bCmWNrKhIHcXf0nMTFP66hKlSAuJasKR43TYJWqojTQnGlUYM/8gVVlcwTVOJ/hYv0IJD3C8KJmalrcJr1SqB+SmRT4pB0oWbTdbOMqIQKdikntFYNWQkXTvAK9wof5eHCqrxWaAk96ChIZSIRRJk42K7P4GgXLIAAIV98f1uCsYPAfd/jx94dD1qoPXUKLe78pe1R/Ajg2VVj6WRCMX3q1KkzXgfBK7NegRizePQiOnLF8uFYMew98QGES+79hQM4fhOe+fiZj0csA+bhHbz64vtxB09/sgkz35h2z0/oECra0Fa42Ruih03DFg4hrOJqgjGvbKveVEpLy75a21OdypnsXGNuFotM5EpjGiQUgiiKYwa5bMMSvmRIFq7HxqUmhWap0CFc4qrcQ9D44Cm13GC+LLcydXyg5bVxJx5xc3/Vr1NTWT/4t1uc/j//s8Hpk6nUlwAAAABJRU5ErkJggg==", "text/plain": [ "128×256 Array{Gray{Float64},2} with eltype Gray{Float64}:\n", " Gray{Float64}(0.0) Gray{Float64}(0.0) … Gray{Float64}(0.627451)\n", " Gray{Float64}(0.627451) Gray{Float64}(0.623529) Gray{Float64}(0.388235)\n", " Gray{Float64}(0.611765) Gray{Float64}(0.611765) Gray{Float64}(0.337435)\n", " Gray{Float64}(0.0) Gray{Float64}(0.0) Gray{Float64}(0.192157)\n", " Gray{Float64}(0.611765) Gray{Float64}(0.0) Gray{Float64}(0.277469)\n", " Gray{Float64}(0.0) Gray{Float64}(0.0) … Gray{Float64}(0.2)\n", " Gray{Float64}(0.607843) Gray{Float64}(0.0) Gray{Float64}(0.196999)\n", " Gray{Float64}(0.0) Gray{Float64}(0.0) Gray{Float64}(0.215686)\n", " Gray{Float64}(0.619608) Gray{Float64}(0.619608) Gray{Float64}(0.207843)\n", " Gray{Float64}(0.0) Gray{Float64}(0.0) Gray{Float64}(0.188235)\n", " Gray{Float64}(0.635294) Gray{Float64}(0.0) … Gray{Float64}(0.206473)\n", " Gray{Float64}(0.631373) Gray{Float64}(0.0) Gray{Float64}(0.184312)\n", " Gray{Float64}(0.0) Gray{Float64}(0.627451) Gray{Float64}(0.184314)\n", " ⋮ ⋱ ⋮\n", " Gray{Float64}(0.0) Gray{Float64}(0.129412) Gray{Float64}(0.241805)\n", " Gray{Float64}(0.14902) Gray{Float64}(0.129412) Gray{Float64}(0.265213)\n", " Gray{Float64}(0.215686) Gray{Float64}(0.0) Gray{Float64}(0.207843)\n", " Gray{Float64}(0.345098) Gray{Float64}(0.341176) Gray{Float64}(0.231373)\n", " Gray{Float64}(0.0) Gray{Float64}(0.0) … Gray{Float64}(0.258824)\n", " Gray{Float64}(0.298039) Gray{Float64}(0.415686) Gray{Float64}(0.258824)\n", " Gray{Float64}(0.0) Gray{Float64}(0.368627) Gray{Float64}(0.235294)\n", " Gray{Float64}(0.0) Gray{Float64}(0.0) Gray{Float64}(0.207843)\n", " Gray{Float64}(0.219608) Gray{Float64}(0.0) Gray{Float64}(0.2)\n", " Gray{Float64}(0.0) Gray{Float64}(0.219608) … Gray{Float64}(0.203575)\n", " Gray{Float64}(0.196078) Gray{Float64}(0.207843) Gray{Float64}(0.345098)\n", " Gray{Float64}(0.192157) Gray{Float64}(0.0) Gray{Float64}(0.30479)" ] }, "execution_count": 84, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X_sdp_sol = Convex.evaluate(X);\n", "lenna_original = load(\"img//Lenna_(test_image).png\")\n", "# [lenna colorview(Gray, X_sdp_sol) lenna_original]\n", "[lenna colorview(Gray, X_sdp_sol)]" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAIAAAACACAAAAADmVT4XAAAABGdBTUEAALGPC/xhBQAAAAFzUkdCAK7OHOkAAAAgY0hSTQAAeiYAAICEAAD6AAAAgOgAAHUwAADqYAAAOpgAABdwnLpRPAAAIABJREFUeAEcwWmsbNl1H/b/WmvvfU6dqrp15/vmoeduNvs1u5uDqIESKVJUYsu2YGUS7CAIDAeOAQNGPiRfJAOxkBgZYFmf4iCIYg2WYksySQ20BooURVJkiyLZ8+t+/ebxvneHqnur6pyz91ordH6/8H8aKxUHVTF5e3DtAVfMBKJcJQoRYlXvoknZwRqUyUHFwXAXVk9LgXFhh3HnfZs2x8OIeSCSYgQEDzldGoXy9T+rGgaFnkHiJcG8oxdKMLfMyu5GcOvdPA+UTEAGZoUgg0BGZGwoTkYmAIyMXB1Kym4OOKkDXoppdHFjcgCkpOjaocuGmIFhDCYHqRC4unklqAZxdkDIXQvcUs9gJYGyMRd2wOFG5ADBYO4wCzA2MneCGaEwnELvko+qGIJYAMSdjM3drzVDufjM230SNxgIzABivPftPpC4E5yqQEXVjMhZyL8vwgQOB8ONoURGcGcQgQQOEji7E9hJigAuOfqyHZQQ3YXUjBjuRPv7SYYfvtlnYheCk3sAh+tvdDEYiACPAcWgTEZemEwMDjZ2YSNjNgIQ1Mm5BEPIEAWRgUBFqHiAsxgR9YuxaqXOEItqIIPj2mqSEy99JUULCCogZqHL3/WUAxkRHMwKV1BiILCiiMONHEakrARn4wwTUi9wZaibsmRyEApcyWCoF5Cub8xEjT1kInfAeXq3quMH356WWMRRarCE712OVGIwsINShPRGjNZKcCVSi8YCFymBshhDAXcmKQqYGJzNOACc3UEOJwfIgFhK7SbZncgJRE4uN3aSrP3g59grJ0XmIK9d78AiQQCCiKCoJXECM8jBcGcTuJIbgY0JBlcygikhKLIICgMgIocREeeSenEypVDgcDIiZjV4e6Ph9ORj10MXCMyC777OAUwc4MTgSAYid3YEd2dzg5gyOQTkiugm5mJKSs7qxOrmwhqVnJRAJnC2QoaSMyBKbC4GOMOD3tjZDsNP/2qfSmUi8Z03aVDE3QNDLARXFA1uSgXRkRnBAHiJxibmXEC9ELsVVwIxSiGwKxuMQDBiN7hXHRRupmB1GJsAShnSv79WY+OlP554CRxefwMiobh3gWEslt2YndyLi4sxeQmO4GxUWJ1A7uzuBIBCEWsJbsQEL+QaumTu0gMZgdWV4OJwFCgbMkxpd69O6dLrMwHVl99yCehJzYIDgZbZeEDGJExwYynEYHYiEJGTGKHUykVRt0rDEk/EYotHR54IAvJYCGRs7ihsbgVM5AYGCgoyQP7eWuDVj/++cHz7DRKJHIo7BzIOpes8kYHhQu7G5qLEKuQBBDDEwL0AXLAZYpmENPDoz8iNm4filAjsIIVKJjKATFECnJTI3AvcyA/vVhyffeteuvI9hiQ36ixQMA6+bI9iDXYQq7KBjAuTRyI4uwcyJwe0EHa8OfKqUoa0ZagXnz567cagCy6GwpmyuIkQwYgMKKwwdcpORPb2VuTVT/3O/TcgAsoolakGkmqxnM43hNgI8EKVmsDZzNjJ3Nng7MVXKjtoVOZV6qeR+5FHq4pMPnHvy05SVKBeFEqkZEHdCpMalOAGNiVevPtyxNmzX3D2aBqywEwCVaVd7toGiZExsVimqEwmbl6SUjACTOKJ7rjzpRRrrLbQDxYjD07K5fR/8o2rEkHKcMreC8BOjpK0WpoLAWziDLl59nRM13NgIecFNa1LHeqw7PcebJA4Ocy9mMQsDnLrhbIPzI08v/SnGcqoNFfaISypiXnspdYcM37o7J92TWZzVy5sgJCa+jKAlEoAORyE4G/s+Dc+J5DKwhKpNaqmQbp2dm++oUYKWHaDIJEbnJi6inuP5Nx8pyrEobfATgtkGbpgXpknJYRy8dxvP5Qo5FCCGzOTFvV+wEEZZCAGINzfvfc/RTD1sefkRuuH8+Bdu38gAiJRB7sxk7PDyEgrJYax5xPvppZiijQfSl8T50764cCHgIgVgv3MH9yYDRIKyAzCwkZtMYzZSAqcCWKC+iv/toATa9+taEmj7vCHQ99Nb0cDuTErkxNrLZmJCUIAHFTi2k3MY21t3TZdSiWU1KZhZOk5OqzAxs3f8//t9gpnJicCwMHzASElgzvYyY2Rln+cnQRebITMdXXniU+F/ujhQgKzu4FNlU0sMyuIXcHBxYiOFkjJRXpqyTtZjo5XFdEdgO0NJkMLeT78Oz/HNZkb3IxBFOfTLolkIgcRAMKXHzqlYGaJFxRXp+c+G8P88K4ENwWxwRCY4ORW9TAJTkrKwo88VlqZOsWQzNkE5kpUzw9Wz8Zc0IZWtn/2l8cVyIprgLIz7narqRFnJbgxp7ffdAyi8bySJU9SnnysmYfZnRKgYHVzdjcDkVZQgpiJC2BpHxxylUObooo5RW/6sKxJ3p88Pug6i64WWvqhq1+1KO4OF2igutzbjFXoDQCRyMGXN464csoymKOiVL2wWbrw8CBJYTNYKhbcmJipgIg1wEmZwnFJ8OSlTVVARQlxsQyadHF8qenmtffHiYYGDT/72v4gUGSGuxQhLKeT4VCcjJ0ofGXRUVTu80rH8DR8ctPb7XCPAC4weIE7BYO7mBPBgZCZiLXpmZBTIiawGnXk7HL5wqXUp7LseNTrEUXL6Z98/g/qqsDd2QiGeH88Sqm4MUm6fE1lyLkrA+saDPzxnbDYCkEjGdjICcQOSFFxEiMLgAbY6KBLYpbZZFSohsB6HfR0/aWV4qQegi0GxkRmzH9n9f+eJAXDPCrC8vBmUwdWcgrHXyqBoYxQz2onfX4HeVJr0ACxEAAFKUQBQhHyIu4mbs34BlNtRKUST7XMB06wvHji8QF7ySmHESgHCzm60fyzX7vauJZAXliC0+54JEM3JnxtmVAZuTUzMaKLT3s7GjhCghdXAxEZBwhYJcGCQUjcu5ff9iDFPURirWkWzDo/Tpe2Q5+jeV+HEiz1op7JBuC///NtAVygbkLINxsdrO8RX3uHvWZDqTXGfPzkh8LxYIhCQTLDipLD4Q4BSHJw98qhrPbM15Iycw4epbISiBZ6vPniSDOwl1aZcgJ6yi4h9MXk4qd/XwEFCYI7yfH9v/uB5Xz0zv9FycQ5uOW6L1sfHhzHiRRHKKQkRQsI5I7cQ7vommNmkLbP7xcLFDWRBoqdYbC/sKfPCxY+0ElRJ+5LbGVoKBDzNnzqtzvzwDAqhlye/YVwb+1E/GiVtYJYxwixs1e2e1lLPZyCcYSH4A5xhbqbDpiJYWzk9txR6EhZ2WrppstNn2Z6eTM8HIyWi7oEs6i2rAeuaoC4h/bEp68YE5nDu6x/47+9lU6T8ujTv+0aS5HisfdLj1te5QInD4HADgUEJMbEBA9gB3lCmp9+9Thq1Wdp0kLj5rFM08dG/eEAxyPvh2jJ+zDJpR2ZOETJlv6zP6fOxFa6Y/rvP/m9c1udm4cnSrIC78tA8soLaTmM5uxAYAb7IAAu0BgoWCQnhycHdU0qgeM8Dhubg2tv7o8+5FOTbp2Wg/EScTnbJKdcLZscjkMK7u3g7/2ZqXiZP3juHw/feTotcx9HxmUw6Hrymub6sY02DM3JXUMIRCimDFcWYjgTEyP2pQbhbL7LnQ+iHXPk2Ot09bmZLDe7lS6uHjgwjSMsEIhCzu8+KZzJezv3N74Uiy27//Tj9/ef9CWKRPL9Cm2CcsDRk0+rTtxdiUgDk7tQhJuEIggOcgL1rn2g5dZuP7BQloNhDmHBi+1zyw443Ka4XAyVjupRaaUNItX+owvKmZx00TenD0s++1PHN0bbObtzDIT3OeRi6qSTVwbLOiiBWI0DChgQEhGKxsYucAc7w7vu7LyW5Ag+m/SL4fGFel4G0416MZmXZHT7XHtcLwP60b0omwuqoyriUvqd+5Nnml0/QQsDi7tLuFpR8F6CDM6f7qg2MmcFWXCBMRzMxCAiOMzEFErk3ekbTchJE6LpeLpdhcPR/R0ie7CyOn8Uz8wHuXMd+3FMaT5iWrrGhVluPnDyYDqsVM1FrBs47GpkIQo2WH9euiHD8P+TQOZQhpsIuTvMNbk5OTssjEIc89wsakzzyWqejx9uaVbf4KPFIC2lYythQUl7H+XQd3nUFT88zZhVYzsOZELWHddmb85XwJmZusfWcohkAIFACEUcXBTQYICLGINYCRrca5TUlKxeQj46PSi5unmG6t3BahcfnYxzG3Jp67nVR2mwjDyTWObVoTyxPFivqOsFysz2vqx06Y8aE7BT2bgIHzE5mzGRI5AbFSKWYGxOBhCzEyBK7r3Xy7H306aZb4+W4NvnQzlat4ebe2dxWNGsOarnaboclfnwiKwb+UF1opqNVj13JAAz063FztefnfxBqkspIZ96vuoGQCE3BnHmwD05iAglOiBuDne4w8XKuFNU3eBhSAdnxsfV8uBMRh+PVkeHO/u8OieajucoA85U3Lo6H/bra0e8nrpchJjBJPP7J99b8denVeV9k0ebF1UqBhmxG4goGBsBQmAYiYuYExxkQaWsLysWl8nx/OxKF2fdWXC/tMlBtXEstt8O6+YRrfbcJdjUmmlXn10sN2I5DhQRmUCRL6/ePDE78W/rlCmRNE+FxQAwBQMQJQ8mrEHIAYYrs6m5wSI5SmmqwYjnYfFgZ3RYL/REX/WzsHpIq/28DNPycBqeo5bbsAjB5SE9Jgdbg3JkgcEiAIR2/WD22sdXv5SiY7II21t9iKRO5nAycg/BLYBBTk4USKEgAjtU1DUEt5GtTOJxWHQbbP20Ge8OmvtXTh9NKb14+q13h5sdLcb7b57b4mf3B+et10Ac6fucSHD31mv0wZd/v8Ti3nr9FMpIQEYEh4OUAhAseGQCkTvIuBgzsbJDO6bVshsHriV3G9LmWU2zRP3+c/e6o0X5CFbah1/fsaMHy7Wv/0K6dq7RXgTGBICV4uyb33u0sv1D8ReHETl6fnarl0hOIbM7HEQeEJQ8KTEDBIg7EeBwiFd3I40ezsIx9enwROgtNzCP+er53ba/sN9/88SVm3L2Rrye6+l/2eVL1hcCSiSnkrx+99X33VZP/ciJfxZZPKQOj6FMGA6IuxG7OwUi7oMFd3I4kTIxgRzEBXTXBjm1482HOl0faFnWoXST5cOT02Vz9MYM2zE+rofHp93oH94arrdMruRkZJGr+79xCB+uPP1M/V8/rLinyHp+S1NiAzk5sxqBLUR3UVG3EhwOko6MzPEfJLm2AR/WceX6eKj9PMU+r/V3TtreXeODbu10++aZOmY7euoffOfFrZkMDAoQ4Dj8CrdS7Zxbn4x/blZV6DCU4QdCNw4GchDB2MHqgZzJChFg5EAwKpHBSk6M+lt/c7p+8pHvD7a1m0647Tb6MMC3H567NzN9YfGXaTK7sf/CKz/z2qfi3ENHxsbEUr7wnVMXPhRXhePJ/+52XWdPxZZbJ9VrZy5sDncCO1lgJxc3JWKAnByRjIkNIE7v1EsZVQ+PTsEfrXouK0cPR/QV2b6+mNx/4s6t6kN2+8T+xY++94nhQU6lwGtylm/84fyTu7OnRGjl7P9wJ7Cy5Wi4WHUTsDsROYGdzZiCg9w4K7ETgcmLcjAnNhIazL59adfWvnYSy91ta7kmenCbq/rg6MN/3qxcoVNvra7JJ8LDT8aleKFeiMXa1z6fntwbt0cradX+/m4MLLJwlPHJ1mt2ODsBIFaCUSApUnKEaSQH3EWjBwdciGX0Wx+st8u6VtOJRHThqN2YHZx8++gUo+n76sbawxeB+U+69lpiLo35woaPnerscKtdrp/93r/wULNbZxTo1GC2k1QdICdWdgK7BWcHLDvYIQXOBU4KQiBw4OWf/Fh3Z/V+a4N2WTeL9rLVJ44lDe9PmmuDdYmz4zPxhYWFXqmlYI5Kqsmz350N3/6g0q99MTEJAwUhr5zMGDtBjJTIxY0c8OCFFKamKsYMMSoVGA4LYJJqdyVvfne4t7os63bIS13sn25rPT4rV0L7oB19YHfnVFmmeRuJ1ft1di7VufeuV+ngxP27XMLQGJ272OpW28AhBoiRk5ETiAM5q5sD5MGNQF6zuJjDiclEffzF48Fqbk9ZN70+37mPo3jqbX74oWuunb58/AMbT5ZaFxnjR2E8CVJIdFTLjWcfbA4PKocGgjMKzlfdpAQAymAjg5OzaShSnIUBJ3cwQi7JBGwslGtKj2T6fujN1x/WHd354GWS/XOPxs2N588vb5SfbV+Y3ELkae/jI1QVaZQQbbxxy3fbj8t+EYaJ9Z5s82S/EttcC4u7sboDZKDgyq7kIHdiMQa7M5OLa73zUHx2f69q5r6xPHi4SFXSiXX3hjrYuXt4L/81eXq7rQ7Wlv2A7vl4IMzOgeLKdhP2wu7FAbFAwpxiPDo1mG5F8zlxEIKC4OYOBEEGaXITIgcgIkHAaqCVXSGe/MknuW2qMmho1pelhCzWd9WZ3Reu//D50clIFddL9DNe4SgONiaKG6k69IdnRmpagxSUq7NWD80LPBs0BYHB3dWCu1OPbHAiONyoUhYlAfUQMN2aV92Kyp13jp9+VWRIyl4dHPTvP3Xx5W7HBnJ0mvf7Ng4jaQWQOFE1Xh0u9vdG1bKOho44LE9vL1fEk5gK1LtWRAz0fcGcS1oyGbuzs8NyLR6okCxTcfPtf+Ob0Q7eHDy6sDN74/zw7WbtaHTrhXde/6n9F+eNyaS5NxNJAbGYeBcQHavj9Uf9YeRIlnihseAZo0aNUCt1jCylGAcWIMCNjJyDMbkYyBwONzbAAznF85OrYbFcnqCdN+NT4yuEtH9p+mbzw2/9rWUsoZxbXBvXIsKuHROICttg2CkfLDceDRaGHLhf3+wlOBspc8oghZH1HkiCETmzm7mQK8EF5lFh4szwJ/hfPfzoJ1+bf/fJmxvwdH93VvPR5hztj+7/9W7rRsXN9d8bV2n7XB169ClkwFlvP9xlpcOt1yMPWmP37XqxScUJygwgSihkPTI0sLM7GYjcWViMiBwmruwqw/5wUR5/cLMdhEcvaF9XbP3sRLU73bm4pmshnwyvHn5Kp2+/FZ79SGVkplLat79327dvSL+/Yx7zIljiU0YDcoebG7s7RFBMvZSgEkopYuZwFxAbcSzsxoV149zsxCu73fR47wPXn98dNvKgefzV+tq5/vC/wGke31lf/3fLur/10tZoee+bl4IOI/WH39pNm6bre3S8NkRXGcU8Xs8DKfDvUyYjI4AkeZcRGL27ZwIJkJm45KRk8EKTC2pNfOrcby8unE+DyXu+Obp/VG9N0/v0yYNPHZ+gwfgXZ+3Myhvziy8aPZo4oV6+efvaUUhV8bDcWH8YWmNbnBzPG7g5wRkGJRjYMigiZKfiHXqQiZPClHtKCrSrP7G4mv3wj8qjT95fvVWdKmF+MuVvfeadmxSfuTAdpnbln9+fEXTtJ754ef7jbWkTD/k3vypbp96O7ZJtXnn0HmDe5jRyNScPORYCFyfNruYIUgzFjJ3ZWcwsSs1K3CEMr4FvNouDj1xZ+av6Q7tykO9eeJ+/7JP5TxytW5Lm/2hndKG71/zkD/zv63ucOoTJH36V+emXHkyFQmpT3R0bh8XonFWubOQolMmhrlxUsxkFMBUxU3IiKYxkzNFB8dqVE0996I2vz+Yf2JPDtdVus8S0dyoWWvabhz9298TEPj++MHjv4pXtFwbyo1ePx+0gDvW7H3+YKxvvqZcqj/vm0Jl5ZVxWHOYOEECGLJ6L5UwrG4HYHASCETkzcZIQEF8/uPTDl9bnv0L5Y/Fu3EyT6eNbU/d3Bwty/eQobKzb75Rwcmd09Mrkuek8TMJCWhrNvXmmefbyfKXXAZY1wz1HOw3UhQzGzdJDcUdWRUfbk9EgCLzPbk6siV0YCCS+eucnL+1f//qXFx954m4+uDRtm/OzD3yrJ+0Gx/kjDz++HMVfKRvBB0/rwwfTxWMnw83G0Of4iQfHe3/wbli/1SVkr7si8O1TfU3k7vDsrqRF1YxWt1YqouCm5lrUwACIvZTklv+b8d639tam9bf/1oWHR7ebx/eOT91PfSqxWPwwDtcnv/zwVBqCi53uq810GFLVtlqGdb1o5HSYi5XMMtzzMjxembTrZmTO6Aimbu5GZzYbcmjQ1jpvs7kCTiCQO+ho/fKD2ZnLfGr1zvW1/dH25c9efnP9iNUi6Cfe++zK5uevng0moTFSNKRxfyAg61fGC5r5dnYOyTqqjXLWLY9iXMBOgMEM1oentsXMiUNLSqWEbICJwcmpSLW++MD6Y3+etlbP3Hv2z4jW1l7befmrF3ZV1dZOVIPmzc8/FxwaKDRmjtR56rk/LoLmmGPNiUA+18mhCfgsqpAd5Owwc7Vi8cnt5OomHIp01HMJDAe5Q71Mtgaef/9+0/IWuud+D6l9/cz4zud/avBnR8HLD9//sVH8H89Vwm5mA0KCF5JoOZTscVktDMIWxfu83St3w4kNwMpKRczgXBQXNsXALpFDbhVkrkaAwiAkZbfm6+lMu//itUvyRVFh3H9UuQ8nBxqGm2m7+vlUJVIr0teIYUGlOBPIexFzEcZSUaplEbWV4/XKBl7Y3SkTK2XLJ7aSgl0kUtACLYWUtRibsDoV6mcv796JH317fuXC+IBSLqgWtT389L8s/advf7b++pUVd49w7ec6KLBcetVe6NAiKQvKIoNKz3LyWpCzSKMZw8mVzKDWV2drBzxEgQcHMSmxO6lYdu9euHLywvwLJ168++2TYX106sEod6ID3d66d+vHf3+w3Z/VX4qV9cldE2spymq2NHjuSm6KFyMc5xUmP2h3bvZ20poFseH7zCmz2+kxF6KQInoPxPCC4sYGBUN9+ky8cdBcendJg9WdODDVgs0nX33u5OP/y9nlj+x9NP1j44FClFQVrIHVC2d30+54oy+AL4+4rGTa26qxWB9ZdO8DQGAzWB7uRCOEmuHsgV3FclF3EgZD+K42k2l4e7R/bny1fOTVwYyabvzeJ6a/tbV4b+s0P/av30qh0b4ksixuYuJtXnbj7MhtlMxu3T4Lmx1sbXF7NpZQjNW5MDlgenoERxyIOokHcc8OB4gcTPD47PxY7zfR081zD5+Y0pkrgyNdvvzOC+//JYV/9KC/9laQMNoTVgJltmCA+eEkri+6bJa1z3pfA4B479KadGeF1WBwBblJ6ZsNMaJEZgT2QEQsVYhwYwdotX8/Nay+fnjrIzdI7j0ZXqPt9w7/aHh71/OPbp1Iv/tXIo4Eh7kS0IcARkq3H52SR0c3O8C79yW45WqvKjWtWgN3Z+fijuw2qeBUs5KACCErkVTVqAIDLjSfrTCOR95Ou5Pv7RR/4j0ozt+NP/aqDMKFfuW13xoxY9SJaTCIQciY48oibnat7y/recENdlTOcReRzjfG7krGSuoE8CSBYiBAnAzBqbDVKzUjEwdjJrc6Xzt76+an/3SzpTwbnbs3/Nv/dHLt+IMnz7x29Olfas7sLwfNw8gEzcGZjIjiWn0cYngp3VNovgaSoS8xV51sgkLB9ykZuaHEEVwiGwiAS+Cu6iWPAsMDO7MTU3n8L6x85p1pI7rx3lMvfan79im69deHH/xC9Zl/tsGLSgMSAmsKri4gAsKAx8/eHGXLarcNXMcWg+kh16dczAEymIPMdRDJJcCJnCAUeltyJ2ACGYGMS07nbjTtx/44sOU74WB2cvzYewdrnzl96t7BE//mao28uphpNGU2c/cekkMBNWvrh5UVdXqLDEQWjv3+2trQo8LcnJSMQJQSQkXKgBMIQU2Zqi4bIBqIDPb4lTU/e7fdSovN6cHx8uHHrt94fv0ojf51WPnVnVK4QmmhDhA4u2skUwpxkCKzFru5qCm4UxaabzQB4k4gdzGGdyUJJXYmc2EHhZwTSI2YxAMIsHP3c5sPvFKfNDg+/+qlJ+9uP/ljtvEr/OP/czj/NkRC1/p8QGzeh+yYiwQnu/Jg51v8aDl+k8QQSOoWB2fWkhPc3EJhYzOx4FHIQQwiwMOS+qiJhYQF7mZrza0XH3X7oQoH4fze7NYL/sbmDxwNtn9j8aHf5eqecpyno6Km5loC1fzOQTMiLHNYuXplW5+4NWX1qCXuroV9WxW4FIYZnB1SPEmKxmbEBAIFIzJqKiImsKk3T3/9RD2btN0gbdjW+8OnjsYPV7xbv/P+TvVm3S9ro76W3j1nOEse+pnNRYnNWoprzQoCf4Nj4aqdWBjItGYGFQCxOIOgTlUVAdB/ADgCgVHIzTQyPMrTf1o9dq2jF/68nO50cOPJob315MVQ9n/z2Wd+oWYsmjYFlsPV2SSAs8fCI7i7rm8lG/iAvnMk1A/mSZiYy7AwiEkZwuqipJ4qVgIRCO7moUjHBlU3hrMNv1KeWMrDE/vV+Hj8+Lsvdm+9vCXbXdh98uV/KhaGRRtxlrovy+QpkXcSQtVD9+dVX1bl0ZuJbCAsbTUycRYhcw1mIJCqk6VE7BDAe1fyIAoNw67vjQDYDOXcm83+s2/o2oOjH317uLr/7g/d+c7alrz0L6fEaGOJQCttXWapKECFjDi6tEtptlu8mkuQ0dxHRMUrJyoRJEbBVdRhOYuAwOy5OIQpqJs3K0rZlQ0cGQt+NDmY9ixH1druczO9MpitrFz83FuxBBaWPrFVR9ny0TiUWiMyheTDmDJnv35gIkmRiSiAeqmOa0DZyA1wqLowZ3ZTEzaQhQJBYyCCO6Aa5M5wb6NlS7PR+/fXp8P9ex8evpua32EKHtpgvGhUcvRyT9XD0JwiD5hXuC158FfkLOJJ6yIu1QLRCc4GE3Mzh1NNhgIguJlDg8cuSCYyIidiKPd2qM9J1QXJh2+9ciGEWPud21v7UhJcLPbdoARLfbqxseUBHAOHSbZ+0B8vH2zXTvGg4eBc9fU0duRkWdwANzejaiilgBmlmJN3gZT8aG1VhJzgzFpwZzEpOm5tb3z+0QPffOxwZ3N++8jNkVxrN6d7AAALoUlEQVSII/rIqmVl/rA9QfVYKbVcB2173SVLbqHuKWUNlKm4zaOzAebZza2sR5gLQYt5v3/l20Gj6yIvcnADkfbdxiu3P3/u5kCmzssT67PNwxnVF9eufy451VED9aw9VTH0/Yl9mc7HzWQ4bGC+0OP+UdVHuBpSqZVSN6R80A9jlAhWNqjmbVEld7PS3v/LP78TQ+LOrcvbCqCwdo9/9hcvVdNgTu6L5uLlN55ZOzqi3/hr//mvBnHOMAhI1V0X9amd0ZCbLDSMVgayPX7m7Ddm5J2K5RCX7o0uj/o2Jq9CHYtDi+6EvoDMyqNrn7vqzWoYwrPOQ9+bgV3Pjf/57EHfnrpPa3F2f2199dHaxS91Zx//k5/6r345Ba+TsLFLqZT6MB4NujAnSF/UeNSkMxd/+ptfvK0oo+SlpAie18NF7uiIY5LAZjzU3tnRLg++fl1GWwgv39k7pm52f8vhFramvxY/80c83eb9eObu9cFa/MTFr6U0bX9k+dYnv9IGC+Y6sMysUjSgp4LaKCsCJyY4PvqD3/71G8ZtKm7dqNiK131eaqucVELvJRcGUG7+5tV6OA4hnNi4fa8/Km02Y4uzW83T7QHh1g98Pr7U0jf+43NXmt1nSxMOJ1c+uWqlx/dx1YV+2NLS2wEHN6bCxClE9BpCeeWjv/4bvJQ+M5quDoWkqruubeddsz5/0BqC+fGNf3ftNEtNCDR6fPgQh0oED33J/9Hlr0rpX5h86Pzdsnjsev7A14f3n8CZ2dde+ZNP/XurJAI6sNKUcU8eNBpzZZbrEMXUqKdg9Hef/V/3qmxYri4ii0aXEOtlmQ5XZ3UjHIq+8yv3n4A1VdeH3AxjmLE7Rwf76mNfFIrx/uvPXFn6heYrP/u769OL9y4uTn8znJd0aD1cnAsl2nyQ3ZpOVkohCpSkZBaIOnH56M//k+V84F3tLhnRgUErKsnDUytiKNPv3Hlq2GNs/SDsxzQ8M73u5GIgpCsv/cVorGl0cHDq4HuroxubpT4ePbh4/WPvfvz//ZEvJKk7F06WbGV1GdJxNTIBQSIDHTGUowHl0j/8JUvFJ20NzuxMmpQGoPETFRj9zeunJiWOhjMbhFc/1mxVm6JM7AyEgy80mDfH/MjPHVV+4s3nrzZHqzYdrcx/52NPHta9FdNBaUfDZjwtxxvRhNyIrJc+mZegzIXIPvvaHy11Me5IIaSs7IYYYnUimXv3xvKEhjRc5kEIu++fDZjElkmZjKbV6btDbN0YBB62Ye/svf0zy+07Fx+sbq9ce3LCasGU6qNqvoPY9hsCKDHBCSV5lwxuDoPaP/iS5b5qRd3Y3AGS0o/XVwHowXsjSD1YTKsKId/PQnVZwMScA92djcpkl0o/5uXO7PIzdOPE4Nzd4cuL107SDhlGh8TzHNHMIwdG15gxPLmZksDVFE7sOvqbv2UyPGIFASjsnvP2Y+tjkHWXfahVZQtER6AyV9ESnMEQwjoHoqJRzvXzVMvt5aTr4NYtnv7ewR1tag4wUZk4RkQCYUYO1AuRqVkJDnOYKP9nv0ZrXTIpUrgYtFjzwQ2PBH/4bixNitNSSx4EnfZ9I6RG5HC43ZbRkVHXP3er2h31Le2lo/PvKU1ufXrnuz/9h1mZ+6MhxqvVQWSwuINLYFdF0H5Ygjlc3MrGh7+xtkxuDspg78XPnA0L4rJ8b06g5D0SSRfcjwPLYOmAC7HcC/WUwd1n3lxQ2XhwtJ3t7tObp7rltj7567srMxGqOl8/XdliRCgVESy4CpWgJEpGrkEB7j/11bWcCrkXNgd3ZSu6sOmjW6nzSDMbxOzzQEo58cpSYZq5kxQX7DGc+epjVeRSBsyjvT/+2LbePH5274Xdq4/C2gFrnSqmVoIEldBxFvWAEt0KOTS6MZn+oFXsXNzIwUoiA7QQ765ZKAJTsGnbBceMqAIpXEiCV1xEye5VswEND+q+lwnv7dX18Pq/WhV88Hm+DOdmkhDaURJ2GMMA0QglxGJZrARTsZUnxlxSIRixkTIPvAj3e3cV5DRfSuxVmxB41g05CdRJiNEv6gXc1x/VpZ5Wg+Om6+RDpc3tj/8/a3fo9azneF6NGghm5xqmQIi9gFWyV05GfXAVwKHlwx7EACZzMmHbX3FC3jvW0Fq3X48sS0ghlEVxrDxUJwdcNiaHRqPNG4qch10UOnnr2dd/+ljeWn72ysbRIshV8WGsYctOYojOVghQYoaRmksp4spuVD75Z2UZSdzIiInp3XZ8zvUg913fHHfblguHHJzmfaEVVjcioK3GU97AdRMKCNTWubqbNz+3+Up39fTP/AtbrrReOK1G0ftNE2HC0ouaAEbqlAO8RCMnkD/7vb2DelgNApm5BM+Xn6l8ceB5mZbzCeeOQIfBjQpFj2oWnEF7iP0cKiNb7jyaN4vTN9Nw+69ulUl+/+jhCrfa1aOmIsfNjXUBGbuQuzFAzhpJWzEVB0DhxB92PB42oxhDlMFxTuvS7x8sDoQP48rSfNwtD4NhyonW21LMBJKOQ+p4ycPF2OdVNx/snqj8xNk0bz/y6rQ+brJSWl0bWLm/3EhwCPXByMnISihRKaCv1NzY2deOB/NuGrlq6rXNahGrTVpc239UVo98U1Vr9lkbQt8ZZF2cnJzJZ1VOCxtLu2KQ0QE191XGl66X5i8Oj2rti4fUjMHdty9sE9jNGaRiRiUpZbZCobCrmCg/bc2wy/M2wzcu7rx4EJPuXr+9tz2j7TAVl5wXFFYa7yXViYzIiBfDa8sojS6lY+p0Mm0n/bv82M7t/ee/1BwFA+J4xBb/onosMQCjABcDsRUWMSWmzMpGJL5yvhvUXeT5rNs7/EcnFyNpb14+HPPx9nCheVC1s2zhyRI6S9XEwMSCXgalmsw1xDxaNnlKabmOt+LLz3x7J3Wk6mGrGUn6q9lzq2KUlGDEUDFjJbXa1HsEc2Mj0ue/MQjMJG5FN2VxNt/+4v0UFuOVPDOp03GoEDRNRo66gYPIZc6acs/alLy31tvKMVfLZuXBly+8+FtRg5bBKK7U+o37z593C1DAXUokNiI2oIRobbBM4vCCS388aEC1juSoebTyVHf5C1e5bsdr5aCV0XBZkJtwdKHJ1nuCuwmZ0xwixrDSbDycyCEevzmcr2zH39u5SpYpbKzLe5dXPrYmQk4gArl4sOheSEWlUOrAUpiY6UxdOg5l6NV6pEa/+Y3baUXr9Wo2l3rVep9XJRxs9VVfkbiByHx0bWEJIS+3diW4n9qf2cF5TFdeeOu4b0rg+viOjV9YHVZgA0wM7CzOShrgTspWaipaRIzdn7gqQTwMEneHD777Za1ql5OjvaOuXkmZpjm1obu3Mc+FVrI6gWn9KwuYjVEqcixWRuff4J13X7zaXL30rQTWsvAzg/FKEqHCbuIaFWxs7MHEzAFHTuymyjB69g1GqMuoRnrr8k2KG227lQ73l7Y9DH3O2nahWlQzG5ThIUDkdKgrx2QmJBmxm99/7NXjF37vwfbdD95IWfo02hqvSYxCIJiLl+hEGgyioYAFrsTSkZP0HIyeXQzgXAXp0zzHJsywMjiccaxWasuHS1UPJU+H6xurt+dETvAbXV/1YjbY3ZieiNH10vWzW/c+ciWcvM06kO31YWIwkbGYEURJ3GGBGMHZmJy+D245ah90dV3NrIrGHj00HTdNP+u53hnKwcGxwptgnH3kduoy3AzYiktNyUO9HOc5Nf2VS6/PJ0d3dg42X/n23bjRVGwRLspkUoTMKwO7GBgCIwicLEeFF3Ynf/o101wZp46bgc8xXHSDlDe34+Hx3pEPmo1QxKeqlolMiOSWKautdGX4/xUExzgIw0AQAPcujrFlkpZPIFHQ8P8P0NIgUVFAQRFkHMV3y0y7XAu63c4RS47f1/0UcwqqowxCgSnFRtnWoBQXgqBCLLkHbLDALgo/PljLby4tccKn+rtPw7qfd1afS4v5kP4fZ3MFyyfH+QAAAABJRU5ErkJggg==", "text/plain": [ "128×128 Array{Gray{N0f8},2} with eltype Gray{N0f8}:\n", " Gray{N0f8}(0.608) Gray{N0f8}(0.6) … Gray{N0f8}(0.592)\n", " Gray{N0f8}(0.6) Gray{N0f8}(0.596) Gray{N0f8}(0.349)\n", " Gray{N0f8}(0.588) Gray{N0f8}(0.576) Gray{N0f8}(0.153)\n", " Gray{N0f8}(0.592) Gray{N0f8}(0.588) Gray{N0f8}(0.169)\n", " Gray{N0f8}(0.584) Gray{N0f8}(0.584) Gray{N0f8}(0.149)\n", " Gray{N0f8}(0.588) Gray{N0f8}(0.58) … Gray{N0f8}(0.169)\n", " Gray{N0f8}(0.58) Gray{N0f8}(0.58) Gray{N0f8}(0.18)\n", " Gray{N0f8}(0.592) Gray{N0f8}(0.592) Gray{N0f8}(0.188)\n", " Gray{N0f8}(0.592) Gray{N0f8}(0.588) Gray{N0f8}(0.184)\n", " Gray{N0f8}(0.6) Gray{N0f8}(0.6) Gray{N0f8}(0.157)\n", " Gray{N0f8}(0.608) Gray{N0f8}(0.608) … Gray{N0f8}(0.145)\n", " Gray{N0f8}(0.604) Gray{N0f8}(0.604) Gray{N0f8}(0.153)\n", " Gray{N0f8}(0.608) Gray{N0f8}(0.6) Gray{N0f8}(0.157)\n", " ⋮ ⋱ \n", " Gray{N0f8}(0.11) Gray{N0f8}(0.106) Gray{N0f8}(0.176)\n", " Gray{N0f8}(0.125) Gray{N0f8}(0.102) Gray{N0f8}(0.18)\n", " Gray{N0f8}(0.18) Gray{N0f8}(0.22) Gray{N0f8}(0.173)\n", " Gray{N0f8}(0.325) Gray{N0f8}(0.31) Gray{N0f8}(0.2)\n", " Gray{N0f8}(0.314) Gray{N0f8}(0.333) … Gray{N0f8}(0.239)\n", " Gray{N0f8}(0.263) Gray{N0f8}(0.388) Gray{N0f8}(0.224)\n", " Gray{N0f8}(0.196) Gray{N0f8}(0.345) Gray{N0f8}(0.204)\n", " Gray{N0f8}(0.169) Gray{N0f8}(0.247) Gray{N0f8}(0.176)\n", " Gray{N0f8}(0.196) Gray{N0f8}(0.208) Gray{N0f8}(0.173)\n", " Gray{N0f8}(0.18) Gray{N0f8}(0.176) … Gray{N0f8}(0.212)\n", " Gray{N0f8}(0.173) Gray{N0f8}(0.18) Gray{N0f8}(0.31)\n", " Gray{N0f8}(0.157) Gray{N0f8}(0.169) Gray{N0f8}(0.365)" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "lenna_original" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Solving the same problem using JuMP\n", "\n", "To model the same problem in `JuMP`, we need to convert it into a traditional SDP. To that goal, write \n", "\n", "\n", "\\begin{align*}\n", " & \\left(\\begin{array}{ll}\n", "\\underset{X}{\\mbox{minimize}} & \\|X\\|_{\\star}\\\\\n", "\\mbox{subject to} & X_{i,j}=Y_{i,j},\\quad(i,j)\\in\\textrm{observed pixels of }Y.\n", "\\end{array}\\right)\\\\\n", "= & \\left(\\begin{array}{ll}\n", "\\underset{X,t}{\\mbox{minimize}} & t\\\\\n", "\\mbox{subject to} & X_{i,j}=Y_{i,j},\\quad(i,j)\\in\\textrm{observed pixels of }Y,\\\\\n", " & \\|X\\|_{\\star}\\leq t.\n", "\\end{array}\\right)\n", "\\end{align*}\n", "\n", "\n", "Use the result [Lemma 1 Fazel et. al. (2001)] ([paper here](https://web.stanford.edu/~boyd/papers/pdf/rank_min_heur_sys_approx.pdf)) we have: \n", "$$\n", "\\left(\\|X\\|_{\\star}\\leq t\\right)\\Leftrightarrow\\begin{bmatrix}U & X\\\\\n", "X^{\\top} & V\n", "\\end{bmatrix}\\succeq0,\\mathbf{tr}(U)+\\mathbf{tr}(V)\\leq2t,\n", "$$\n", "where the proof is nontrivial (to me). \n", "\n", "$$\n", "\\left(\\begin{array}{ll}\n", "\\underset{X,t,U,V}{\\mbox{minimize}} & t\\\\\n", "\\mbox{subject to} & X_{i,j}=Y_{i,j},\\quad(i,j)\\in\\textrm{observed pixels of }Y,\\\\\n", " & \\begin{bmatrix}U & X\\\\\n", "X^{\\top} & V\n", "\\end{bmatrix}\\succeq0,\\\\\n", " & \\mathbf{tr}(U)+\\mathbf{tr}(V)\\leq2t.\n", "\\end{array}\\right)\n", "$$" ] }, { "cell_type": "code", "execution_count": 36, "metadata": {}, "outputs": [], "source": [ "using JuMP, Mosek, MosekTools, LinearAlgebra" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "A JuMP Model\n", "Feasibility problem with:\n", "Variables: 0\n", "Model mode: AUTOMATIC\n", "CachingOptimizer state: EMPTY_OPTIMIZER\n", "Solver name: Mosek" ] }, "execution_count": 37, "metadata": {}, "output_type": "execute_result" } ], "source": [ "model_SDP_JuMP = Model(Mosek.Optimizer)" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "128×128 Matrix{VariableRef}:\n", " X2[1,1] X2[1,2] X2[1,3] X2[1,4] … X2[1,127] X2[1,128]\n", " X2[2,1] X2[2,2] X2[2,3] X2[2,4] X2[2,127] X2[2,128]\n", " X2[3,1] X2[3,2] X2[3,3] X2[3,4] X2[3,127] X2[3,128]\n", " X2[4,1] X2[4,2] X2[4,3] X2[4,4] X2[4,127] X2[4,128]\n", " X2[5,1] X2[5,2] X2[5,3] X2[5,4] X2[5,127] X2[5,128]\n", " X2[6,1] X2[6,2] X2[6,3] X2[6,4] … X2[6,127] X2[6,128]\n", " X2[7,1] X2[7,2] X2[7,3] X2[7,4] X2[7,127] X2[7,128]\n", " X2[8,1] X2[8,2] X2[8,3] X2[8,4] X2[8,127] X2[8,128]\n", " X2[9,1] X2[9,2] X2[9,3] X2[9,4] X2[9,127] X2[9,128]\n", " X2[10,1] X2[10,2] X2[10,3] X2[10,4] X2[10,127] X2[10,128]\n", " X2[11,1] X2[11,2] X2[11,3] X2[11,4] … X2[11,127] X2[11,128]\n", " X2[12,1] X2[12,2] X2[12,3] X2[12,4] X2[12,127] X2[12,128]\n", " X2[13,1] X2[13,2] X2[13,3] X2[13,4] X2[13,127] X2[13,128]\n", " ⋮ ⋱ \n", " X2[117,1] X2[117,2] X2[117,3] X2[117,4] X2[117,127] X2[117,128]\n", " X2[118,1] X2[118,2] X2[118,3] X2[118,4] X2[118,127] X2[118,128]\n", " X2[119,1] X2[119,2] X2[119,3] X2[119,4] X2[119,127] X2[119,128]\n", " X2[120,1] X2[120,2] X2[120,3] X2[120,4] X2[120,127] X2[120,128]\n", " X2[121,1] X2[121,2] X2[121,3] X2[121,4] … X2[121,127] X2[121,128]\n", " X2[122,1] X2[122,2] X2[122,3] X2[122,4] X2[122,127] X2[122,128]\n", " X2[123,1] X2[123,2] X2[123,3] X2[123,4] X2[123,127] X2[123,128]\n", " X2[124,1] X2[124,2] X2[124,3] X2[124,4] X2[124,127] X2[124,128]\n", " X2[125,1] X2[125,2] X2[125,3] X2[125,4] X2[125,127] X2[125,128]\n", " X2[126,1] X2[126,2] X2[126,3] X2[126,4] … X2[126,127] X2[126,128]\n", " X2[127,1] X2[127,2] X2[127,3] X2[127,4] X2[127,127] X2[127,128]\n", " X2[128,1] X2[128,2] X2[128,3] X2[128,4] X2[128,127] X2[128,128]" ] }, "execution_count": 38, "metadata": {}, "output_type": "execute_result" } ], "source": [ "@variable(model_SDP_JuMP, X2[1:N, 1:N])" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [], "source": [ "@variable(model_SDP_JuMP, U[1:N, 1:N]);" ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [], "source": [ "@variable(model_SDP_JuMP, V[1:N, 1:N]);" ] }, { "cell_type": "code", "execution_count": 41, "metadata": {}, "outputs": [], "source": [ "@variable(model_SDP_JuMP, t);" ] }, { "cell_type": "code", "execution_count": 42, "metadata": {}, "outputs": [], "source": [ "for index_i_j in observed_entries_Y\n", " i = index_i_j[1]\n", " j = index_i_j[2]\n", " @constraint(model_SDP_JuMP, X2[i,j] == Y[i,j])\n", "end" ] }, { "cell_type": "code", "execution_count": 43, "metadata": {}, "outputs": [], "source": [ "@constraint(model_SDP_JuMP, Symmetric([U X2; X2' V]) in PSDCone());" ] }, { "cell_type": "code", "execution_count": 44, "metadata": {}, "outputs": [], "source": [ "@constraint(model_SDP_JuMP, tr(U)+tr(V) <= 2*t);" ] }, { "cell_type": "code", "execution_count": 45, "metadata": {}, "outputs": [ { "data": { "text/latex": [ "$$ t $$" ], "text/plain": [ "t" ] }, "execution_count": 45, "metadata": {}, "output_type": "execute_result" } ], "source": [ "@objective(model_SDP_JuMP, Min, t)" ] }, { "cell_type": "code", "execution_count": 46, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Problem\n", " Name : \n", " Objective sense : min \n", " Type : CONIC (conic optimization problem)\n", " Constraints : 8129 \n", " Cones : 0 \n", " Scalar variables : 16257 \n", " Matrix variables : 1 \n", " Integer variables : 0 \n", "\n", "Optimizer started.\n", "Presolve started.\n", "Linear dependency checker started.\n", "Linear dependency checker terminated.\n", "Eliminator started.\n", "Freed constraints in eliminator : 0\n", "Eliminator terminated.\n", "Eliminator started.\n", "Freed constraints in eliminator : 0\n", "Eliminator terminated.\n", "Eliminator - tries : 2 time : 0.00 \n", "Lin. dep. - tries : 1 time : 0.00 \n", "Lin. dep. - number : 0 \n", "Presolve terminated. Time: 0.01 \n", "Problem\n", " Name : \n", " Objective sense : min \n", " Type : CONIC (conic optimization problem)\n", " Constraints : 8129 \n", " Cones : 0 \n", " Scalar variables : 16257 \n", " Matrix variables : 1 \n", " Integer variables : 0 \n", "\n", "Optimizer - threads : 48 \n", "Optimizer - solved problem : the primal \n", "Optimizer - Constraints : 8129\n", "Optimizer - Cones : 1\n", "Optimizer - Scalar variables : 2 conic : 2 \n", "Optimizer - Semi-definite variables: 1 scalarized : 32896 \n", "Factor - setup time : 2.49 dense det. time : 0.00 \n", "Factor - ML order time : 1.40 GP order time : 0.00 \n", "Factor - nonzeros before factor : 3.30e+07 after factor : 3.30e+07 \n", "Factor - dense dim. : 0 flops : 1.79e+11 \n", "ITE PFEAS DFEAS GFEAS PRSTATUS POBJ DOBJ MU TIME \n", "0 6.4e+01 1.0e+00 1.0e+00 0.00e+00 0.000000000e+00 0.000000000e+00 1.0e+00 2.52 \n", "1 2.1e+01 3.3e-01 5.0e-01 -9.99e-01 1.470310185e+02 1.484363063e+02 3.3e-01 4.47 \n", "2 1.7e+01 2.6e-01 3.0e-01 -4.85e-01 1.350828267e+02 1.356788512e+02 2.6e-01 6.02 \n", "3 2.7e+00 4.3e-02 8.6e-03 3.14e-01 1.541097696e+02 1.540464927e+02 4.3e-02 7.69 \n", "4 2.5e-01 3.8e-03 4.6e-04 1.14e+00 1.483267313e+02 1.483322905e+02 3.8e-03 9.59 \n", "5 3.2e-02 5.0e-04 2.2e-05 1.01e+00 1.481427694e+02 1.481435481e+02 5.0e-04 11.43 \n", "6 4.3e-03 6.7e-05 1.2e-06 1.00e+00 1.479864515e+02 1.479866107e+02 6.7e-05 13.24 \n", "7 1.1e-03 1.8e-05 1.6e-07 1.00e+00 1.479754287e+02 1.479754705e+02 1.8e-05 14.96 \n", "8 1.5e-05 2.3e-07 8.9e-11 1.00e+00 1.479711267e+02 1.479711263e+02 2.3e-07 17.23 \n", "9 4.8e-07 7.7e-09 3.5e-13 1.00e+00 1.479710840e+02 1.479710840e+02 7.6e-09 19.47 \n", "10 4.0e-08 2.7e-09 1.2e-14 1.00e+00 1.479710825e+02 1.479710825e+02 6.3e-10 21.45 \n", "11 3.1e-09 2.0e-08 1.8e-16 1.00e+00 1.479710823e+02 1.479710823e+02 4.8e-11 23.68 \n", "Optimizer terminated. Time: 23.68 \n", "\n" ] } ], "source": [ "optimize!(model_SDP_JuMP) " ] }, { "cell_type": "code", "execution_count": 47, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "128×128 Matrix{Float64}:\n", " 0.510133 0.541838 0.635294 0.522442 … 0.470371 0.422462 0.627451\n", " 0.627451 0.623529 0.621339 0.611765 0.404712 0.329829 0.388235\n", " 0.611765 0.611765 0.632581 0.598682 0.403922 0.219608 0.337453\n", " 0.585981 0.581618 0.611765 0.58344 0.223529 0.176471 0.192157\n", " 0.611765 0.581893 0.615686 0.615686 0.297408 0.27872 0.277459\n", " 0.557052 0.520124 0.570136 0.619608 … 0.387982 0.245395 0.2\n", " 0.607843 0.582899 0.623529 0.586153 0.176471 0.192157 0.197013\n", " 0.611249 0.637073 0.623529 0.591021 0.229532 0.192887 0.215686\n", " 0.619608 0.619608 0.596553 0.559789 0.2 0.151996 0.207843\n", " 0.617212 0.58893 0.635294 0.635294 0.2 0.192157 0.188235\n", " 0.635294 0.594272 0.62413 0.653188 … 0.192157 0.180392 0.206486\n", " 0.631373 0.592408 0.607137 0.675214 0.169149 0.224812 0.184314\n", " 0.588056 0.627451 0.635294 0.666667 0.172549 0.22933 0.184314\n", " ⋮ ⋱ ⋮ \n", " 0.149178 0.129412 0.293731 0.541176 0.356051 0.286275 0.241796\n", " 0.14902 0.129412 0.196078 0.537255 0.345098 0.324363 0.265208\n", " 0.215686 0.222665 0.262745 0.507814 0.301961 0.337263 0.207843\n", " 0.345098 0.341176 0.356863 0.513725 0.294473 0.259783 0.231373\n", " 0.263513 0.334628 0.38577 0.46014 … 0.320677 0.243137 0.258824\n", " 0.298039 0.415686 0.458824 0.438061 0.354361 0.25763 0.258824\n", " 0.268498 0.368627 0.4 0.430556 0.279037 0.328952 0.235294\n", " 0.249179 0.289784 0.34902 0.431445 0.284626 0.239216 0.207843\n", " 0.219608 0.251989 0.311455 0.363335 0.335256 0.291726 0.2\n", " 0.229011 0.219608 0.235294 0.356863 … 0.268871 0.212628 0.203565\n", " 0.196078 0.207843 0.211765 0.351703 0.348219 0.270588 0.345098\n", " 0.192157 0.202599 0.196078 0.309804 0.266667 0.356863 0.304792" ] }, "execution_count": 47, "metadata": {}, "output_type": "execute_result" } ], "source": [ "X_Lenna_JuMP = value.(X2)" ] }, { "cell_type": "code", "execution_count": 48, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAQAAAACACAAAAADB3ujWAAAABGdBTUEAALGPC/xhBQAAAAFzUkdCAK7OHOkAAAAgY0hSTQAAeiYAAICEAAD6AAAAgOgAAHUwAADqYAAAOpgAABdwnLpRPAAAIABJREFUeAEcwQWAVlXCBuD3nHtufD3zTcEgCiqugShtu6v+6rq7BiZKTdMwtMA0HQ4NU5So2LFrrLvr2pIqYCsiDdNf3jzn/LjPw4CdY/C7Z5n2MDZMHrOzTCEbgSnrASxQautkZZ2oAhbR+cAiKhdgKZ5GbeUiIhdiiSALsGhhTRVQKyXcxZi4Cf+zhZISoKGsoaxZ8fQ/BK/5bs8ezbcSTyucYElFHVCJ2tkrsf2E0ePkOVf2yadMs5POb61uWCGulD4pFISVdESafnEKOX56OkeqVDowkIBx0RkiwYQgiQiPdOYJOyEl3MzUeUcEItKgWZT8RLMirblQFE//QzCWsWeP5lNcTeEEUBzHUOHvUJJD2HY5prm4sRQY/SJetkbPah0jGiYAE/nETcBiVEpAVqGmigBLpJy/aDHkfCzitQuBuor5tZVLPA7UVNXIKlxQXo/qagBbxjeUolGWAShuKcFHw3+S0caJK+eSpcD8JRVy4aKFtZVYCXxyq/Ty/IGuMCMUqs8BVMFVUIVDOoEEC0oISfPUWADQpBTQIIkf/JRCiesF476wK1M4H4mDBDXDOd0zrRs8aEhI4zIq0UOoiqdJ68cbZHTAdx5V/a5uBiypphVXjcfR8QnzgGKUNpDSXY8D21fOHiVF2ebxW4RsmMbUZUAlgDopwRXIBcDCRZKgbiGq6ypqKuoqgPl1qKkCBP6nfoqXXDYP291G2oKi5sZSAEUNZWfAWZ810yFWPg1I1C1YjEVAbWXZoZYboboylkGkVMG5KYQUjEkiJA8wNdOEkg4JaLKHpdjSOnWR8EkiiFQdopqKNBEwOexAKiXAPFeXoVYvYCjQXUldNdItLA0yrblnFM76/KaF0zQNF2lJhM6YMJh3irJiNBe3CFG6Y9SL4OZs7EIRJm6aIBqA1XOxkKC6urKusqZOSBBg0UJZARCgurZOQlkyH7WorAFQg2oiqzHjGUyat9aQTEIZi+JmkEYBoBveVb9cOye5EksBPL10MVBZW7sIh1MDbQkvIgThihCCE+J6BrhtSE7QmWtSEvPHNV23bCl1okKPG1AEUaiUNiFUUjdFbK2Lqy4hlrRFsivXTOQakkkoIuYxP/eECl93xLNC155p84e45XJDdRzT355LW0+lfmbNxaQFINvG4vHnnmpcef78s5ASm1FGCJbPX4TqatRVLJIVi4D5S+ZLyCXzASxaWLmoAqBAJVAFoJqiEgCd6biYtkkUYTt9VnDSXNw0fovczLebP1z+/YrxuGD+EoILarHkB3wB6BYxAMJ0wDOSkmgSUgYVB4QwBZ0ZfpXo0qXpADiYLoOSOQpSPkkIcxnVLJ+i6iyhJSnChp2meQ6XripcK4NSQQhPhF1VOtwxwyI/Ly5ZzlnS5ksEhKsw7Qd/VhisGEVbC4HtwHNP7XBnz8Mo2Thp/JayhgkAJCBRKyEqaxfW1VVwyDrMr8PCGg5wYP7SJfOBugpUyWpUV1djAV82hUzPkXIHHb1jLNBcjJIt47eAxobhpyuwBZhNl0AC1VTDPFwwOG7yUHc8S+Oqy5JUYdCkLlyLEgIZC0cVmQpIj3VGHbWXFEISMC4djRMpInHVc6J22pO2tDwr3G31SjIBkg74pC8gha14iqSm5jHQ2OHMa38N/KHV+4Xq2VIV/qCWf9KvsV+SrKlkW8HWwq3jXhjJt49thuO9UFC2EWgoJaJcX4pKUlUJVNYAogoVAFlYx6srOAGqUFXjEtQRXlWD2koqSHU1lpXXT1QWANj57GhsI+OAJtEgMXEtIPDzkf9sXDl/PlkMCE1+iwGHr2UHx+v+dIZfk9Tjrt+ljuepTirLTelMhHTmUKqqnuZlIh1qV40ICLF5PMCJMBQ4HCRuE2HpdjAZFsQ4rnf5JVFEIqlHSVKnuqqprlAlHKIrotcVR34JBG2P6qmOPE1+K49EuhlhJQDIdoaR4AVQVs8ECCZCbm6csG6GnL9EVuN3VUBVLSpRuQgV1eCEoFLWSVQCFXVVqEaVFCCo9OYuB10NNKhjdmBbAbYWglBatAXT6i+yf1L6jZiw2VMwH0uY/Pa5aw/jCIaTEKfEYIwLThyFS8IIwq50FCWWJbmriVi20tUj5stM5iB1XvVbPp6dJASmlDJoa5FYWnMckoIAgd/LEC5VpBpUTcMHvTXMCaE07lMSwrD9Sr+TP+R5CiyFMPktORz1wmmVbUUBCvC7gq2FUs44T7aWbpooUQo8g/nz6yprq6uralCLykoA1QKQC6VYWoE6CNRUAbJGVFfJWqCqBsAMvnZWqKqsaavcWoCtEi0oBsZvEc6jH9xxNIjgxBXAAgqyANAwRB4U1HMsYTESUhRPSEkItZllSMnAPOkzhGJz22/ChWLaqYgmdFemifA7UKmwXCTS0hGAdBiRsRBjSYULydtzVSntVFh2hogXs51IhtOe5V4eFMFWnqM6lJM4SWquI9OMFWInxuB/Cne4s/A4KUDZJpSSzQDEMtRWVlcLQACoIZXV1TVVNRVYijrMI4tQhTrIKgASQA0wfwl7ZvKqaqCkSRa1oBDNgqK5uIHL6avvOHQZfvq0EMBi4LsBhwccvJHzga60g/4zPJ3JdDtoCi96mvigxvxh4rjJkDA7cKmrCpULZKSNjM5ESLblCQRdn5U2hd/xkoCRDsclFxk8Euj2mJ6wgz06Xcge5/BTJEdQmtGewWVnJHbp2dRfPo18p5/J7ba/+zanLSgp97ts55gxuGA7xm0fh+KVs198fNt4WdZAGsoaZqjAAlKLalTUVaOKQtTIagBKNZ4Wy+WyWatQWVHLq6urqKwWtajiYvaKGR6qgQ2iBEVNaBKyBEAZgJlo/1G9dC9KjXWo/fnw4QEqvhgq+GHV5UZQ8GzucQEo5yzD1GXQMpJB1a/CvIT4PJZOCkNJUIhMmQjmxgNmLGhq0kjyLjVgc9OJUxkQLK7HIZlNe/sTPlfge0MPZwuZOPuHY6ddP8uItV+U/8k1J04aSnuG/C6nTbWj3YLrTOKCbQXjsHMMxoKW16OAN00oayilwHJgMSrqUKEA1dV1FQCqlEpUL5mfEUU/9MF8E7ZSg5ra2qraKkhc8AywMKRN3ogmIlHSQIFirBfTUM/v/DKMrN0fydl65fX82sPDAfkVIKWnhnnHbz0o8xRJSYD7OKWGDFBAs2zHSDs+KIxzK2iSqBlUQNrz89yoz1JSnJiOoiQ8g0gZSuoSgA6nOxbSQrYaITLcHaZGuP3isBCCtGenwnbWMUUy/bcjvDUoVCGdcJqN3U4NN4m10wh+R57YPbYJmwFCcMHTrK6uStbW1FSRao46Xl2D+nKEt4975e/AZe+47Uufnj1/CSpreWUNKiEopnsbJMrX8I1cKQFQ0li6SXgSKF+NQcDlWHFqJeDQI9gLHOj/zXUpPxWKqURUAk9RQIRpcJIOBzuJhJD5TASo8Dsw/T6enUq05trZnhoOxjP1iOwh+7S3slaDdF6c4gmfwyGo3g3LAElSKIrCAkgEI0x4MhQSZ7p/yPmW+/qeOo12h9KwE06AEDAQVT4FTMPoXaOwtRCALFK3lDU0TIIyjy4BUIMaiZqqCoqK6urQTANg4yDk/Ug98l7fS988OmPOClRW1QK1mLscawBKMB1AQ6MUAqXwJPiqWatnrvIPvBGYgzE78R0Gs72DD+IbHCJEld0xwiXj1CG2J5gklDg2iKJwzUe8826OI+H3JWjaZrkhYRhpdi4gpJp2wrTvpXEl8mNWSkDTpIy0ZlosYDgkI+x1EyEJEWFNi0vw7g7mnPNf8d1ZdkxjKa4Spdt/qrcTjecyMGZvnvB0xlwQgOB3LUXFoqRpI1bOWTafQi6uFDVV1WBVz8zgdXjGBHij+hheIwT3vq8+8Cm0ct+SGlTUYS4vr5/CN9WtXDsN6ymnmLAewFQAK1fNXI2Ju3HB+mb0/2Yo2Tt0P4Z4X0MyZtOk1BQqXMEEkQIOmEiEWRd8NiiCGhfn8zSHOZHzGSpnpr+bR9WU0ChJq55qDmvXunzEzu+SjsnPK3Y6TEmMazYlFMRNmmosTIiiRDvRcby9zzV2j+bs85pCcs73SlvJTIdBexTA0pUtRWQXXAAPuWPQXNSCCYxJLJm3DKgFamrpQpjwKmtnYDWlpXh29Ig3H3iXyj9B7B2O8ulrwIHlMwGxaeo6OW09F1MAbLHrQacBIAS0HGeB072mXAo6cP9Q7MewfQOuVw0t7qZcTgUnOlypgLgqTMOXhs5kXKWdLJwMIqVQrdvPPJJWpa8ntfzS06jUqTRV0TPkb5NJH2eRpAef2cUypJZOEN2MdhiChWiKqh4hNu194uwlvYbv/zmRRyUVXal03Ih3qezFR58djSd2z8YLwChg+m+vY7Q3rgWlmyd59cAyYCGpA7ImYUESS7F4AWZiDeDg1QfwZ/zr33dhOD69ZfZEn4ML6qeug1g4Z8NkAGvl9PHAaqzj5fXlq1eXA+XbhlFAA1WHUfT/Zh/o11cHTmcFz57M6Az5AdtzhCs04aleUgP1aGbCIYRlupqbdU74kxbvIZOmz7rETitp33eZRFyuUPDcfoePaWmS1sOdIsYVIQzupnspZkdHZzAXyOUKtdWUpl4sUsNoMq2pVJWKQaVPT6op9viLj6+YsxvL52Ik0FQi3/hbEGoziiU2TkW5Quky1AE4M3UdX7ZwUY2sqVo+18Em5SX6JlXuI3f997bPcPP+odUntk5Zj9VT1qFsA1bTzXIipmHl7A2eIFPX1MvVtHzNdKwvwA5cqQ6g+4Y58A1xDn89IGb1PClkJCcowBxCfXFHcZhKPEVabsjSAiHK4cTCZ88kDIos01WMDDiOz9Py6NVUSZJ0qDPnirahVvzElSdN4omMhGQ5rs9nGK4g4Z7CEySQ4FKlhHtn8cPJS07+oJ5kCnd0X8yJ6maKvWJizgjee+7Wkbv5UyVYi7+PfBZAc3GJtg501RxALFwELAYckAq5EMs55mBi4zi8/sC7/7rrv/TTm/GpjbtuQRyYug6TNwAzt4xv2mSXY/aKyeun43eErKHrpyjA5qu+xxA62D40XBzA7c65GwKm23qcq1HDJcySrkcUcBBojqGpnAoKS1UjCPOUz4Hql8Q0uDRtjR2/SpUs0wnRbNPNtofLo963GZG0anmSn+5JDbfLtpS2nF+voe4veQAhoGeswOlf3olRjVDHkxaULEETNzB7NPDaiA0rCl8gT6G5eMpxIgu2oailuWzjVLZ6xop5IBzL504VyjOzOPNqxdzlyzBvfSlefAh/xgW34D+udnAwsOOJ3etK6YZiLTNnfHNxQ3ltV/0cTFk7DSgH1kxfN2WTBsSV/t8cAPoP3TsQt30EHLvf9ZDd7rncRzzJGFWkNCihtusTrq5bqZBKuMhOp30WI5x51LAMigBSIWnyuJ4BpJnmasYZOzPrjJ5QaUY7V0lHHhOu2+pPkY5MmQm/MGKxTBrQPDeuINIRcLlPTWlc+C4m7Km10x5+9bUppGXkbkCuOPHW/bvHFrW0FDc3YN0M0HnLsPjpqhTWAaBL5sMlmLtkPmzg8XfpPf/Cn97Hf/+EC34M5e9+yt8INI9fDAiQjd31mLgJFOtkPZk+HVNBi5849B0GAoPcI7hO4ONbjX89ezTSeaJXT0qooyNgqVJ1ueCWTvW0L+TFfCHFdIP+VOhsvnYqszsDLoEvGevK8Vt+Rn02UlrAOU2yffrtIefaf/uzE1wIpy0qdDfgqidCjNGIKdMk2K2f6qW3BY9eZCVJOOVl0Bj8qqlcO4Sh80kHWL+iCAS70nMACtkCNBc344JVeBp0MTBx06xVWDFvCVC7HAKYtcko5PL9/3sLZP9Q4Gs58A8/AWlRQFvGiXnLUNqAScCtm+Ytm4Kp68rX1JevIep4HPkeA7Vh+5xvhu4/dDPufh+bEl38PElTXToaIUT6nLTul1zlrk7SBvNSIZJB1fOmn9pJLRlOCpmnm0SSpD+jKyqtmE6I5l3ySw+ZPNWb0rTwaEoIxbTMHJLg7m+X57WnjKDq6O3tl/vVsyeP5COpJZgTsEOXtmYm83x/SLBlvzqvPLF78hw89/jzonTZvAcldo7bjtJGTCZy1ipg6QIACuS8ZXOWzdEW0blYAYAX7vLY3S8I9BBfDcT1B4ArfqV98IQ2dnvpslWzmqTYlJrtDmvF2mmYiulrAKrhFhPQJIFy4xcDjM/+7AcmYtmvjpZEtl+hkgmeIrp0Pb/HQw7JkDKUAM728uenkyHpc5VEQLNoQkZk2s/b8z2W8tSsDMFO55/JNSJiOM3+4LRGOvITfcNEinDKcr8luT1U4TAqIqDar8eMYCAuia5Q2p4MmKEs9TuVzcPIUfwxAUj8bvIGjFa3FpJGgG+eQTAX8xcD09dAWTbDga7UqCsVCmBKA5N/3TESuBbYNwzAkWsvxZFNx94DBEgDSrF59uT9+/IxbQMmA9MxfbNAm4EhAvtwaAgOA+EXsWEy2g2VSi6EKjkBgRqQipH2k1iAcK6SAEG+GlPNpBKl0sn1lJCqK9Tt8rNsJtOm6iYTfp7brsKntwYSF6e7b/EcoRFJiG55cRnq1MJhRM1AmsVyRXeb0X6aIBAw3Utslt+hZ+X/LNmoXS/gSWwCRr0gAVA88GxBcXMhSulmgMxaPm8pyuuBmZi5GqhDHWYDWK7MKnvu9HO990s2aK83DOD7bRxWjM0r3sNjHqREo8TqX/eibeyOyQCemQFMmIkf+w/bN8zrb9B9GK4G+qyd5gDnbBKPwItpVJPwddMkZFYwbQQVmjBAKJWdis5wURp23P9rgObBNH3RnopuBk9n9vVJFvd53Y7kQRtqjnF53+5c7qWkhHSoJjPQFqFp6Rnt+cL8VZzq0eHqrteWoMLsUM3r8084kRBznnwezwNLo2XySTR1YMRrkM2gRY2YuAlyFZZhdmr6GgisnkuWAfqclUTO9gjqnwI+tsE/FwQAG/wVwC/Do8GJLwGzGkoay7BiI/CZgcVhTJFYr3fPueKnb24Yvvd69QAA8emkHtMQBBzV0BQRYkRICQdQFCk1zZG+dJhCSqpbakp2CwU5jBhWPiU+v3Q8YqpqVj5scE0QQwoWywyDn8u/4d3IsKzuiBsL2xr3R0Ltp6LZYVBp8RA/kTIyXY85asDq5W8bxHvk2kFfiOF54KnnJj/dsPPJ57hDJQA5TmlGEd005RnMWVFXsRJAeX35jOXA4gVzMBtYvKAOIXysOrfggzs+NXAAAPlSXnd4QF9cMDunDKWNYvyU9dd8+x8sWKGt8jAFj+MnDNmDoY5xPVQux24ENpTuwltwfGlpO1AAQaiEGZRphZMk9XSVcoVInm1LxWe2pnSv19EgIV6mCjtLU3qojgUatgw3C2BEEITsD4mZkK1uOCQMIa30t30vVltVLc7bY8kzeT1p4jJ5TBoxnOM0FowRP/ODvTSKyF3gy8iY5ySlWI8HfDtQVAJJJxHAQwV+Vz/jmbkLFteoK206H4udWRUohu3eiQtuwcHBwJce5EHv22twwUoATRIr5uBbYNBlbX2nr8XmCS9iwGE6dP9+ADfsAQ5sTM0Jgo8dI4ip2VwnTEpKYagcnPtVSMqIaQXziK+NnDvfelV7fraVpV5h01Dac1J67KLzYbdLt6PtRiyjK0iCHiTIUCutZfgjlkMMCRK88rIzbYcu0/qGklb+WX6oS1V+zc4GtVKXRYI5/lSe7bNUNlpICWxGA556lmAORrz2BMaIFhQrG6dMp8/MWYH/eQbLK2tUMQ9YOds0VwP/vIfe+Xflvjvw6S2D9w/FIOwdfngAcDSwY+60XkEiyoCp33xw7ZEve67EVwMP/IqrdArQG/k+wMVtH2NS/bZxGLvtAyllmpzwZQqXEUjXdX2WX1gioCTDSjB1XBUk4QV8SUKVsCqO+wz1VMTMEBm5jJq64ncSQSUZae8hpXAivDWUH/gw99teQSJYd7v1Tc4PA6/8/gqj//6bf/3xW51J0G6enXnOPXfyYisStVxJbcafx0hcUPYsfQpYcWoddmMnMFZumiCwprg3JjuNU9eVE0Zq8T+zYfd5Q9X+7w0b2j0fsltu+fSWobhAAcfX1192ci7WLgMlAMQH1zjArLe/oOg5BFIq2IfBB4GbPr/+480TEFG3y4KCJx2hGxzCkxqRjAjGLJ+v1a9J1S+Qkr1Tp5xgluMjGcLUz2QryulWYvTprWoxn61QlTKR9pvC6dJ0ReEiM/nDADXmCFCiZeFEzjfe1cP+cuJymtET+6VUclKsGx2RtNZx7yVtSkQlUnL2PEbtwuORRowGdlnFk//K3x21q0iiBZAU9JLX/tuIdTNXz1oBYAEPLgRwZRHwylvcflv5t/TwyS34+La9Ch8CF9cfHNwbdd/P21zWjIYyjm/74UAaNx6F05HlafuH74XEwK/c675eJ54JFgLbWopCjptyWGYG5TRtOFykeVAhUQkrCEmToUu80yJgq+nA+WhWUPIe0u597krS4eRo3NZUQTz4iKbwgMKtJEt7mR3HPTt2NkKozS2e2f9LEk5nsmDA6dvT07qokgAc5jqX5Ytg0CcCSa4ygAIKwZbxu6Uoxob7JXYVYitQik0or29EMSao7uRVqHUWLa7zKmP+pTmvWkzcj7f5XXj/bpj4XMK6HXuHD/tSEnx9/T0VkE3FzcVLfgamThmCn/tdhr7nQQ4PFoPEl/gK3qElU+tnANgmi6CCR+CQANFJQPg0RqUUDuF+y1RUs4cmzpt6uwqFRTItyQOeIJ16jmsFRULzc666qmFmEOmz0mookYpIEg1elXcxUoQrKrP6i1vUH+1bz55SzvoZCajCFZkJv+ENvKI7pKesgCOtbDZ2Byvc+jww/jmCp7Z1znwLF5CSppLGSUD9/IeGNmMzgPlLzKXzg7ICFXW4/7mn8NY795H737v37rfuvxs3ffYJPrlVAIP23ACBYe8ccic2CSS3ATe+dy/sg95wyOalUD3vMAYqB1zkoHz7OECgqSRhMHhKSIFg8ASI52ZREI0HCPVMnterO3I2vzOjI+w3I/GwDLkytztGhZUZPBOwtE47RPxp1x9EUIqYfjIvbQcynJxDfqjCTfI8Z0B7Z9AOeK6UfqGonpeV3Z2mbjDntO4XRHWhBdgOKBLA4gXy8d2gM6eefB1jt+J3dJpiRoeitBGTNsKZsxTafKCiDsBTz2lSvncv5D/+6r2re3dLeRv2Dn8593a6lzh7tEEASjYtn4uDg4cc+vYaiw8/PEB004HegUGAug/frOVrp43bJqWERKYSl8wwzYBkrm5LrhuuRn3CASjNMaNpLxFwfE6mQ3WhxQOnfEFqS9NnSHmRJaWhej7imD7pMT2maYwGgo6qtearMZsmstWrznb07P2LxQ0fFd00pQSEYqmZvW/txzMVGZSuhEyzYjShqGUywajdT2As1j34wJsoJk0lEusBtAB6cbMobVwFwEJ5uAYbJwHao29YAnAcjHj3jn/+0/y/z2+K4VFg2Bfkhk89AJv4xN3A4E9vuW4/BP16APKn/zyEDxa4ft+Aw/OmYTNQgN817kPSjlIQ13OpwnVPCluDZetSxpnwMmNduumCi7TqtxzWGY7GEUzJSCJqibjhOBEhuwOGiUDKInAdO0KI6xFdkTIn2mWeyDlr9wwKKtH9y6ko50nZ1R48SyMd6ElcSlzPHwNrxgWFG9Dof+IVOE9OX4NRaAaaUNI0Saay3vt63gRsLsOU9ZizDPWbsF6unolHX3vweRWvd2XidQH7fvy7HXfjn/fg0xvx4S34z53gGXgCwC3A0P1DgW+vwWkcGHIA+HrofiwDJmD7OKC5GFJR00GkVUVjOpdEaJT4IAmxAlzTZCpL456apprj8wmLa7kRjUq1wzifn1DUiNKVGVc02MJP291oPGwrwWQi6ilheZxnpEXmGe0HA205v2V7+faJ037WZRBJrLx2RnQjGUooEVPxS4YSiBZMW1v6/Cvk4RfWHhvx2i4ARS2FskxuKsy6FXPiEzc1QJu73JiVkTlx9RRs2KqKES8Z9+98CHjjwddeI8BdwH/uZIAN/BFw3jOszI9v2zfsS28YMBSABF7u/82BwQcB9cGhWfa0HVICkGgsLRQBEVEcKSknmgsgTbJMPyMOVaSrZMmsznhAi2eC+JDWwjHN7+mZqkj4DCfNDUsnNkiKp32uoExzbBExDTjkMivTjuWleua0BWWrZqqy+0x7WzgUzxW+1kBWlp2gIWmolpSKxsC3opCsBZ4EXhuxhkiM1ChFIaQEfLe+++fACmDyBq6hFpsmPjMD65RCAI/tBgPw4GvEBd7+C3An5L/vuhPA3//253//8d/W7V8RDMLnN+HLQTjQH/iDvJ55wODPyxesn4ax23EBbZYIpGxIkkkUhROXSCIV1VJcCEgJJ9ipqYFoPO13FI1nqVFV95sKa+sFVwhTT9kBO0hcpDNS4QTVJCckbkeTrKNHW6YlLiLyNLKIP3wuoqeNU2dY2uFdlpLfN+xyzVNAPVCPKAykSIBPyl+A10bI59vq8eALY0kLLigTUL8chKq6CmyYtAaoDbr15cBUrGb+EsRe9OEFfcSI3U8AHvCe99e7gLf/gjf/9urDFHcdODgQ2CNv3E8H7aZD8NMV/Ef8TtxbjyktRTvG7tgmUQggTQwBbnpEoVIKIpkEiOYIn8mEUHufTnVHPdUTAcFjQddOMWYK1uF3pJPh40yXNoENL8ADKZfGw8xPXM/xUWFlkFbdlyN/vMY7l0hG0I/xEHJdTiw35iqRDgYCCUd09WBoAQrIRgDilYexrvy4GEW24oLiBhSQQdgxVq/oSsjZKysql81bijX6hPVTNpubEX30eYzEbs7xwsgUXrv3n8Dr/C+vPmwiijuAIR99dDtAQMkbnYVA6+0dw6Qjjlz/1ewHJmxRdohtsgDYSgpg+4kjCacKEaCEEA6FEJcRwiAc8kMkAAAgAElEQVQZSTnBqO50JQI8J0EcTq2QrijJLFNFtD3SJYMBAZ9tpSSxo105QZdZaS0QdKmjt/kNKwVyOSVuWktGW48ZVJ4WIbNfn0sdXckTIMRIaSQYYhi7A9swufdcPPLyy85TMyB3FRS1AIVAsanuOdeNOYvkjomp6siSeTVPYzqemYIJGydgGxgARx0JB0++wpEGuIOHXwu8I/aogz8WIF/cSIYfGPxRZxjAz6d/wZBDg677GiuxfjyArRLbROHWZvJfmyg+k1FOOOFCIZBWCMQRghLPUVVF+hO6BFKBqCk8wyMqCSSoEieGzRSaVFXqOtyM8KTh59xRZTDARVjN9QmQMCHxTJN3hpNxmTgtAsLnpPJ5/NeLUz4RC0tpEVd1Ohl2YNz2gg3LgVfko89vmIyHsK2oEFu3FgPeihXYtWX8wjmTvAZcYAH15daqWfUCKEAKjYYyErtGAY+8gIfw/CPNu0aNwAUf/Pe2T2/Zc8M+HBjy9XdaoqkEBcCgA4OoRE3P0im4oBBbC4BCtDBpWIpQIDwqVVeyYIKkQpwQT3OJZ3ckWEBLBeD1TR/PzDkphWhFp5lm0mg3I1TJVGxKXScuA2pXIKanRSz3XNBJhDXLH7CV65DpBL/TEp4qzg0VPxOHyvDVPc+lxEkGI5WZavdrnZcwFEpZuLUc2E0ewZMAXgcEBAoFxIsAfALwd2Yut7SnsXTVLAciABnCBfaqUlTgeQ/AszaeHR1Djvb3wB34z5134JNboBwYsn/IN+efRE0V9k6O/0gHfQmMrtq4wV+4VUoUF7bI4ubiogm2jQSjUIWi2f6kEQfPtSFCaaFwN25kOOKE8HdmEiVPO95+le1YWuBsZqhVWvHcFE13ZOd4pqHrdiJDxCI5WiSgZnqWL+FLJLPIsb6d+efbtNaLjt+7Mh6lIhgJKWk36g+kpESY+C6ydeIygGwrGk/mNpOHX5Ve59HVeMIgcgcuEMCRrvYxTbK7eeHcRU/PWQGB5OIFSxEYNzOUTo5HY2ld7ZMbsdkavXzCd5vVv3PxN+DDP+JjRrCPDMFQ9EfN0aol8/e0Hh/oHMbgg30bygAQWYzm4iI0FTdgL4jKFaJ5xAaxvJTHIxaVwgOkMMK5Xe00T3YretA94zMyhZM0zim9fafSiml30mhCiTl+oaWT3m8ZF6shLizPkbbqBknsUtKJX+Kp891H+53/bU+rZvEe7pmr+/ZqzYx5qkxqmqu5GjfAilqK0DKerSp+CQSYOOsBT2xHURGRuKCa3IYXzpXXz3CwcEbGKj5ngboAyrpUZe2CFWMABagc//mECY/fM/dWd1Igkfnnz26GAtCb9uzFEHx9/X+OfH9Ymz8f0/6ArzDwKw+12BCkvADNKG4qaSxpLEVRd4ZhxBk3w64BOESRQgHRPAIBK5N0CCUZiwScM66aoQVOB1VEFZpyqGsnhZnjudaZRDjv++4ev7qTAolMkp/UFAJK0RfHM644evmR79s1s8s5ZuQz30mIbPNkjyDl0gkCJM3i4W8lA1qKWrbgd5I8senYm3h8LG0BCgm2P97d/wtxcy3KcUFk/tKnsRh1FfrU2SsXLsJOoAjFzVtGH8dPX10h9kz0nsSn/CN6Kz4mX9yAC67Hrm2Phra/DVzx4+CD13917SFsciZjRwG2FQASpY2lALKdlDTRx4MmidBIRw7xqCsMEC/D7D54hfglkWhLBmUg0pl1whHIPT7wGOvIP8uCmmJ3t4WT9HjqpyEf5VwtPWLmcoMmRIDI1p+V1mzZfr5fZ4jc/PwP+/t0hE/o6bxQDycpw+kMkiaxLBiQ2WdZC9BS0oSVWY+98vCru5+Y9SB/EShqKREtRS0vjsKNe+AAmOFkVMF+eilQgWlzAzDnLscF02IobfwjvsJPP2HMh5+QWz78I4DbPhV7hx+6DiPtZFHkGSh/+/EncHx9w56/3eWpO8eM3T6uYKssKm2SABretC4OpdTuVJZKmUeJzPYsj0TaAkKHFlMTJy5pdXLMmO7otCuY0WF1Jc8YkUC7mW2nujry0p7pd9KX5F6m52TnM4UEYibXSTooevDLDv5yzFaO9s7sUuKJ69EajpEeRpanRoUuaUdUdqkRC240l02UHpcTA0QAr8gnNk8AMGanLGgqLALGJvClA/+qWTVVcxah2l66eEF19fK25YvGr34EN37xl7cTvzzu4EP87rmn/vjevfgj8MEdkIy++V5VzQtY0HPy+2zj34EhB4A9N1wxDb8bh20AmqgoaRKyj/TiVOtNRFfYhQ6Fuf4AkVlp3caxvET8JiffZUY/tVv80M9WL0n3Mq9s6wzpJ4X626DT/sQvvXyZbpd+dhD3n0Ze0n+6rVfvNiXAqPJeuqdz7U899UvYxgH5Pd7Os4NIDPgq2+rTW3cDWUAkQoWjC8nIpvGkcbychd3yUex+AvjrPyRAsLWQgL2GQfj3r6WoQsayedVVNQ6UxXPn49DL6MQXeBtb73hxBK697I1NE/8Rxtv3/vMe4D8EMIZizE7Ul28f9yLuxgXD9gHD90ZXY10mIXgKBVsLUdIMlDSMn9SZQaSQYIYE9zzEdE1aQRLUUiQ/nUcPsx6pH+zcRAbN6cqgjqqETwd/7HZl4tJM6yrL0UxXO3zZ1T1i/cK/5CkhmnExIzlh+a/4Gb2n55NezqF9wct+ML/Kj+t2L8u5ODuTeJ6uR7rPK4kwkiHFZRuBxpItM9BCHn0FHuYtAwgELeZoQcvIyBbc9TIwOVuZC9RUcFRgzpL7XsatcVzV7y388Yp+DThyZK31zn3vgr57D/6uE3yC/z5s90B1OXwvPYYLBh8EMJij98ZJU5+XTz2/E5IDLUVoknLL+3ZGPOSKNCeeyjw/CXiO2umnPEliXb60PJ+fmxlnbdkKV9qtXK6ds7JiQXpWM5PHWMJMXCmRd2swz7rSSAWpFyN+QqCaXxy0r7bped03+Msz5zPpb31hxwnpuuLKs/FsqQjLkshKh0K/RaVUGUBKmsCXguARYMv4B96EwHYUbCsoUom3Zeew03e+/tkqVJbXVzmeb/GCp1b89R8PdX342IHvvwc+DP4DGLXLG/B/IBB4bYQgd3xw64b3fdkOqoHHAVxufHPwxi+Ag4PuyJ2EZwmeEwRjtgNFAJFleIKSUDsJqiDUJtI1pIyavmi7wpWISrzggGGn43/57Oy5qxN2H8fzKSn3V8t1fPzSfuhhpE+fyg3yeB9vwK9ZQV1Au+gsac8+mnzfl+3oxzQZ+7Zn4qgxPPv0RXafrIMkt9Pfm4BLAkWGiZEMExm12eQNhBYJ/nTzY3iVjBg/73s8CqBgG7aVgDTh4zF737+7sppWPl1XsXBRzYKp6/CPx4933f3SFXmfAPjHHR/MXI2BGt6/++9/eXMEmHjvDkwGUFxVA3xwxw17fgGGYLBLDn35pFGdo0oxCs9i5xgA2wqKGxvkN78RGhU8kSN0RtKqBdat8y7Fn7SCIJns4/abe+/z96PdRvR0dtzoDrdlIHG865L0i3mxfOvcJb9dqsrAVdrlPK6nNfwaEuFOX5+LvzyT7HVlVjg38W2Pq9zODvW4S47lRQ03qkpxqocCbplxH4lloVMyYBP+58XHH379pceWAXIXsK2AbG2asGmq04nhn0FbCPjs2YvglNff+97oZ4f9eBl++mnQl3e/D98YFXDveP/u1/kbDwB/wQcAVhx4qRlL5j//zjWxAYcBRR7E4MEH8xZgG0buwi6M2ont4wC0CDl+stIloSoMxKNEEzojUgjTSCsiHhWWE0+IgK13J5UMO3QqFfKO8vYeceFnsd6dqT4nU2mfo2rUPZPb5ueq63PbMsIdwjr/XdJWe0O8s0ekszgUqSscPTOY48f57CiDw3zSAmxC5Dm2AShpGmU088dffRgEF7zy5POFkAA2T1s7WaupMrAQU/2mWImZdv09yeEJ7BueBsDxPpC1cw5ecd65G5w9CLwVYTd/RnD0pT/6/PPXPvkY/2HA8L3YC/QnYkQUSxXx/JMYBYzZMXankAApQ0dAU02ZCnHXJyjhHreyheaHhNDsEGUaktFEp9D8prBxsZ3yUm5A6CRt+YPxrJQvk+eIrBwnajk6s7ji68Gs/gT0oi99rVqg6wTvfT4d9jtWihsidqk0FJGbtp2ocFQ7ywMDyUgwTNjcVLi1CNgtX3voJcz//nUwbAWKAKwF2YWf+6064SeeVw/EfwFJHAKgePf1W6sA/W9oxgo8gvdeGCnZc3Rk5HZ8djM+H11eP+Bw0TQk1GupBG5NO0cGfnVTBZ7Gs6Nxwc4xY3eM2V6wrQCNpbmOmtZSugImFSEhSdAxdIf7iM/ye1IJ0j57LzoRIp7HbRn/JZ8kAqmuTO7l9NuvnHWO5mQkI3HLzEkZknFq90xpvu6g+puM/jI8eVJNqOcUGdddW0TSF+dbTHZFO7NjuYpmOoFT+aSniIsYAxnPm0aTWjzxqnxtRGPpiPve2QkUN7eUNE3Q2OqyUL/PX/TsegCz+TPX/vldXK9m411g8sPngf/zr5u6Zjpw785n9b/uGvkvir3D8fX1iPzlKFqwaRfo17hR+wjX4ivkLWWRUhPYNQpjAIJxWyVQChCHCI0o8FRJtG6PpMPdWT4HPstjjHWpsUOX7vdsJbvTCQd+bTdOBdNuDsKGzXuev8Tt6c9LBKImfvOTQEz6/TRbtsnIz1rEdo4Ge+Z/BEpC6QQ1nIwbSISxiGUG3Lz2kMN1EuQETLfAsAmlILISACEorfjhnUd8z4KWNUgEA9UzvPjHN00NrgAq6o4lEHz36u+MPcP123K6NtxxGqjHG1BeDMukfHwHDCgUF1CgGgNwx10TAQzfSz/CtUeGHBgdMEOlW7xNdFRL0fZx+F1hc3FjqVQ4UYSkVAUhntAVzYucCuaQmIZg4GyGF8/9Xgbbc+J61/lEIkiENNROHQH7XNfp07p5XUZXsm9YJqXXlZtSFHo+V1AabROq5kv60esXlSSZ5HHvvB7IN0Om7sFDKJLkEsTVWrMN7jJM4LKoBb8b8ToaTr6MVwA0orgZWjXousmHsK4cCxexSRtvGPDFfWcghuzFMOODh/0/YNm8twimvOT74+4nmlzQf98BUHw94Jv+u0bpSHfhb2cO7r1Z+dN/jwzm6DUDO6F6ZZvgYByaxwCFKIZAUpcSpCMr5GdC8Yj0d1IKMGjE0FJ2SGqHWruz4vkdPTQrlpaZmuqY0aRn5Ju9fghndUYJ04kvKWSaKNRwSW/X/EOIHut99LRMd3kWVbmiKJ7n59FsJeT3VE+1bGlTi8XaobltIZbNxm8ukR5Q3WP8w8DLjwIPvjFue0kTaFmDjTkUGwBQaIsW3jl8T3+8g1sdDNdTz91x/OozSyhEEGseA54AK3hlxD8/vg3A9Uf6I4T9MFdCd4exz4DBB6mNnOWBMbtGoaUIFFib3jipoawBZYCW1qTnBPy6BJMKcSzFID0MkyoENnKpnRlLKbS31uvE0eRFP3q2nxDEdPs0jV99YyaNiqDdoZjSBTPjlPs6SSDZoZzyB8KZlnn6Et1VmKlrtk5tLccKJDPaVaZb1HFsR9Gy035otsEE0FyIgmrgNULQWPqIg+0gAMEkb7bg5fULFs9eCZG4003jGwDht6/34eDwD3AAstf2B/HPe97yRmwrKADFPR/icw8QQBgPvnHosZdeGbgPtyofHhzC2VTMbXmOAzpQ1lQybb23Jo2yzY0cwtV0QbS0x4gkAOXCAyUmC0DzVMEZMc1skYokjrppkXFe+PWzwgfFDOTldsheLmchI8eLMW4ne0hFBDTPU4TrhYN+YnikU9F4QBE57ZwxRFSD+6me8JJh09O9QFq1NFdy1lgqCreOkgAkeRCNCxb/9fEXiyXK0DBpDeZCYjFWzltWXeSGWkd9K3uEXr7mawy9L43+38y8VBHAPbgfYC+wR0cA7sc3fQpchyOpZR9f/d1LmPTl7R99gsEHqXP4IXdlEXbuGDsKQEmTx6etAWCVY9NxCNOjEtJRCSgUIogVlaofDvN5CiTtTCcuEskuN9Qa+C2gh1J2Nrdy0uL4NdFLFXGmm7aldSVgMGol3Sw9FAsxNdA35Xd+i/TUr/pSYUF2MkqdS7PchJLM1E/ZWd29rIDH0z54/pTgQYZGFGPXuOqe4YffeMPtBv4BNJehoaRMlrPlAO5+H8uAox8DN5mBRDdUQCUfXvnNvGW44CP6fWlp48E1eFO9D//3349vwR425FoAV/3hx/Xn5Ue400kO3jcAVQC28wJg+7hnR2NCA1SsdyUwcYLlBINI+zUVBILAJ2w1HaIe1aTFfuubsH8KBs50e7/lBVKaGUh0X6QKQ5DIj/2o7HZ80Yuo6/vkuoNXi55qOGWyrqBk6ZPHxCkqO9WoK7NipiM6My3fOTbQ8vNuLRhRQ8RUaK4FkQSTgqG0kWD09mqMfB3k0abFD78KgACkYWL9TGDL+PeveCSn/J5/PvLKw1uvUBUVX+Nu/xvX9Hoq8vpD/7wHt2Mx+mMN8MDO1znjf/rPnTcACxcB31+NKbgJ+A8GHRy+d/KGxT10twCABw7ZJBtLsVny1TMXd3JOEmAApSASLMVkECTs8fa8dNCf3ceR7z2S8+HZHHk8Ky1URSX58HI7Oy4eFultZ3a0dwbb1P5tl7ZpVjSkCN7ttidEl+kPy/yzJ9KEduu+ROqqn6/J0rJk2G23cjsCLrcoAWeES+5lnWEgEzgEqjfjIQAEFCP17VtKmnjJJji1leM3TP5pyfi7knjlgVcfOq19jIdfxdn8G/b8NdybfXrLq5G7sODpqStnT9r45HfL8NJdH9+G10bgRKnS8dJ3qKz9/MYvbk8eHLr3T5lYsH3UVrQIKSGbZQmAFj4B9UCKhDn11LTQJVFAQG2fVIxU2oi6mWZf/97B8b6/dqbMnnb4bF6G1jMm4iKR7wzPCfdmOW361b85nVpegJwJH7/4ktilMsuTuafZCUMJZOqBPo782UonIz9q5y5PQ0mzLCmyZSLMZEcITAFXrTawkgagYNtTl08A3oT7CF4GF8VUFIEXbLsMjaWT67trtuCJgrNvDnkddw44/CquPuJ6WP7KQx/d8p778n3c+hCvYyP6CTQ99q/bPr9pxP93BBcAUlUNG4Dfc+65MXd6ZgtYQiUU6RYU+WwUQcXGJKVbFEUREOluFAOxFSkxvw9BulMBBZFl2Zqdnrl5zo//8wAf4bZdXfdgGh7ccsd/0Q74X88Zr78AsXbAu4PWvssHrgZWD+EcS0fOmzDIoYJ6snVYjnpcStykx+UsJzS4IlAXLFjTuuFpfxEncc3y+sjvdUjJGc8lp9pXr6xV3mN/edk1rM4H7s3XacKVaDZcUWrurJsuiF2QDbnQUxnPRS07KcVK/Cr7izTkGX/eU8Ili1HqUJ7X0jpna1/44Ln3+27AysgTgMCbb93/Ba55UXyA7+/DNWPR5Pwjn/X6fuRSwP9LU//hM60zhX9C2iL/diu65o+iBfYAeOutOYNw9xF7/K4DQBvjlj03nx67kN7+X+BIx5ad2Mvr+mMAVrsQEGsGr+FDMAiLRmICZDsnZ+ExHEkFhCo5th1xeFVU/F23wzaPfsW/Xz4u2TWB61j+n1wpuej3CscurFsoNZKlA9mT+YYkRm7Vk0mqx3LB0sZ21Z7OWeOM7GRdPWLRUA0hDa7INzC7UMnC0V0uINx0gPOcasuyRj0MH+CjJ/PAUGxCH6wZ3NcEBggQ57Ev78OCcUDXPTeeT3VMv5RtI5xvcQ7osq/FPmzqg2vu/w5AANi6/nMYMoB2uIX/p83CYwBOd7FWD7m3R0ocPditLkCwrv9qDFktBq8GBmPZiCWjxswUr6E653XlvOUYWQ1WQqTDZghCFBCnMLc3HKo6F/qn0Y25fJ6IbKVwzkZKdVjRS95GzXd4GKu8WCcZzgTCvqaHLJPKVUbeSBzkp9q4RTHnhkTc6rLLiKREkgfqorJu42rVgReiqhB2nPuvqpfypaLGywA8Zwu82mAo6b3Nqply3gVksXq4EcP0KeNWDEMMDuAtkNYBrdsfRkspgVOjWvTBNWMX9q1oW/MZjrXBCPoO8Ekzqc1e4Lb1G79peRL79r2PH7odad/pQMg/Di++y1cPWYUha1YPWb1y6DK+eNRcazLw/q+uGbSFUDQQrrMar+zmzahIh42Yymprm1+fiF3nSPAWSLyQVDJTvSQZ17csjF50IXSl2T95b76+csex+oYiSHGTZtL+wlQk326j54LHvD6ouKGyQDpbl/uv6iebI3OlboXkLXSlUCjGG9olViBXFGXAgPfQu7cAem99AHjzs0ef+4gtw/JuN2DKrzXru+45+/yH+Pnhr4ZM2p+yzfaH6bHGGLvwy629MCPlQbvXdmNmizbbexa9AYB6UpOU6bNemZ3qsg8dAnWVfht243AnbNs2NzxQDFoFss4ajFXusKUjFriYiOXD8eIYJlwJpoCHUkMkqMN0sFiR/ccNUuzhmvVVN3aUip0r4QKn+O8UN1WX+jOiRAvXreFqVcBolwp7g81jxX8Fkn7Npp7UBSUdNcpTGVaRMsybabzU0Gv0AqMonBdcNHQiVgWLuLQmFnBzPkKsSJq9tIoCYktz8vkTBFtiL+ArDF02ZhFarwBuB57Zg2rc4zURmv3iaUNyWrPmWehfPIbfvn8ds/EagHRv/IPw69JbM21Xm70Qr2DXzwDYfx/thwcrDrZDjx2z2MC1YvUQiP6rVg59adkSe67NZ09CQ/x8l+HIsIQMQi1VQaHrimRdHzO9rdnl6KfxiK9O8oKcD5k8XHndaUNyEixmJ65vfclz3/dqiU3NoPCk/9FtOWxZHma7WsfLlpk7YxdeYUw1DpryBS3LKn0Ky3gFUh7ZRaGZ9DqO1+badVyTGp9nHGvxaA6zgM0PbHdfQL8Nz68cuQjzx886+jlanvwY+K5T5sd+mA1yoNUJtHXPtH9uOn4IbinD3IlYIYbP4wuVIaAzZr41+f2W+KYa0zIxAC05vvrqUYO3PoKecMeuczAEeGk5H7aSD8f8iZjzMuDFXeD5PFw3S2zbI5irEd30GoxKdSojlVbt763ly3mjhomiGGW83l8Ipt0byjvkL7UKbikziolcKISs23lTUKGmmVMeoXrQ0Ysj6fxVj6fw1l1neCqc9QTdeCMHMerCJVBlIxVQpWwgTqNWlcvW4GnT/e4/bRegN3pu+eCFDX2BpYA19U/nyUt7gRffx4FOfTa8PKe/wAm0OIpmHaZ/Gr3n7bMNpgLvDly/3qXPff7Zk8MxedpbLwL+R9ZkiATgJLrvfHTi3Hai/eHts8UaTgYBWDkUy90Ri7iLuQ3xG++OX8gaSCbPw4TgVAjDtQV4MCNb+p9O1eVi4sl7AqEynd6Q4cJTUF3LSs71sNpH99OzDXzweLgHPmoGxGVK9JSTCxUlHJIhUl6to9MaclZptkekW+nFopSTy0bG3yghe3jcH3Qt0SjejJvh4hKGfhuAHoQvH46tvQSm/7Hh2YHvvqK/umTqxJPAMx+zAWf2HkC/OVj3Qq9q81jr4/WKEb5n5dCB8lQsJxv6beTAE/jXG1g68lxTDF44ZSr+xfHlSYgTaHf7pAWD3x2EVS9hKMAFxsxx5kwEiPjff+7EsCRlcpxyKmkOBFyZcJVquu6tuf4kuCCMVoZLfa4ppXyprFRU6a9XLIf/LrBDst+hxEs1DkqrRVpyiMNT4TKfSaKukAwR4bmmJ6tFgX789lywqoQ3ss77s36bCyuhOyxgyUSwdC7PsOERsRH/bT0cm8R392M6f4irb3jo9FH3zu26FwThOW0AbHg+9dcH6J5rdRz//S/uw4d3vzsZs7nbD6Qvvn3o674rh86bABtN/3f1aWnmZOA24+Bvt+b9sNu6Rx6CioFrxJCVYhggRi2ynMnAbxC3/w87u5tJRdgIEMFNCpODm0xFkKYy5wtSJQbxesvzjbO1Fkv91S6UylWUsDpx78mT5/zNJU+auymnfnnEZxTreQdc2JVyo6uKSKTNWikjuem8X9jxmkiYqHlCbN5YVCEXykYtp1AuCEFk9HCQM/SxBXaAL4/22cI31b74NNmAUfm5wA+9k4B/wu84BrRVP+x6YsSynU2lJuexaMyuZSf2jaOYtMhZ5wO2Wl/b6/JLRmGq+PEe6ekPsgyAw3F7/rmlONVlH7R5HmAwMBTLIPji0bMm40f91p3dd3aHenCOZAvZyXPCKLhCQIiVkfJSGiQtZ5ifn/UKKV2jFlQ28SUKK6SLBZmOhQ0OJUMSzViqo0URCFqyzbnP1rOiyJVt1WFO3nZ4o/P5i5GoX6j1NOqxyhnLC70AKtcJ131RXc8Yfh5S6zJsAu7+CYuX828e3NTnfYiNL8j2khHe+Jpg9NfOK8a6t1iHcbRDNwvmq/rGI8CrY479cOwsxIxpsuD240Cvb/pijTV2zstT59+D7nhhhQTA4t35AYy8/8K+Np1eXtL/PT4I14zAojEL5xpzXr5n168C0oFOnQ8Kl3gSNBsAtyGlYQl4ZTsc88WlYDRVdrPrXmLxOHcu2bpZX78YiusNr/b44ViLBkIxZMHtWm9a0mmCWobEFKRLs1lLkYheYfFzokjFH3F/slm60A74bP2CdK6Z6Y1FHJsGM540Kq/XjJyPoRdxv+uyD8M34CvABX/gg4Er54+/qxjrgaLnFz4f9+Im0N23oH7sHaDT3W+jahLw4MLJb+BfG/p99chnTw7GTBuv25+3zLdfO2gu0HtzZ8UCVu36A3Ih5I/sQVg9ZDkXHBiLaUU7etz26+3oduyYLVwgSy014whwSTYkbpnME6g5XFxQy2hRdUFh3GtmoRZflevHLjNXLs7yqkSJ+Wczw58nbt5HozJ3FAPClnPU8TNVtjiBVK9SsXKuHC70xAsd2WODmywaEQwxqShfpIu8anHDW2IL1mfTg9+Bd+7YDNFIdTEAACAASURBVCB9vu6Lz4FSJPHzRADPfYTJM4F26fOAB4VvADhw4IvH19YvPCIwcyqTtdHo99Gjnz0JYDJe56/gaHvwpfnpifkgPwPTXhqAlgcPQnYHrHppCMhILBaYa72G33Z2B3CszTHSsU86IJncsClAHAZiGsWlqaSp1a1MF3ridVRWmq5yghHDoxdW5ItuCqa7frt2X2FpkmQVhchpkiuMFcQky5snMa4YJtUyXEQSeZW4Jbab1OSr7S42ld1YIOvJFCCrC783rpU19mdU6osaGkky2pv3zKUFF/3wLfrimqfSE6a9UDX3+eS37NGvZgI40rplqdK49c5+G5pEui14HF82OdLHHRISEO7y4auf+9DG2kFTHY89Fe+af+/MkJffyrZ19gFIIoyTz65fOPC9NYNXYugwYPQiTARw6687uwNtIDocnRbkruMILkzh41nueNR8mqYcflUKURYJ/J3LIOKaRL6J76S1ZdXdLldWhrRoBqmQgHDDpjcZsD1uPuF4mBC2WeK6hPNsjeOR/owkWTjn5UVBwxPKBOCxruqZwqhoSAyT+bhtnCwoSUSZtBFAu6NdRn2qPoRN8ReABrNH4oPe+BD3rkP/dc983F7Zi5N4/Iue2xF+2gBw/fluWmY1rlngrBgCR/rAwNQ31NfettwuABbBrTqKrnvQeD6CaFaEse87YgX4CgxbNmIMZrwO4HYc7IhDpO2xNo9mNCjwCu5RJRJwvSx+c7WXZ5n3Ks+rbm2AyheUcNS44bJp6dHs08Z5wY76c5FMzicpyaijCeFI1NKUpAqkfInfLZPIuapCN591/47WjRy73MuA1xEaeDF84ZwwqONNqJnaJkakkMQLTPbVwxsBGSrEI9jUB29ffLcKSwEydmGHH565wCfMg7y3R/m5Ll9gO3BgKq5h2I2HhwZnvfXmuEXDsErw/gCUshdeA7b3XGWPRPobtNtzd3YPNvzdZd8zwItrhuBfywQw1wCwszs6HibtcFzA9WkpV/aqTJiOym1Xdav81HeV6FJ5hFcGfVJSpjjpjRWWZvOsYxOc1SVGolHLJ2eEm/XnhU9wylVXKUt4i1URqGbZfDpnplUfYR3r/J2pF7poOcJJ2zRVGBaS6hgJj839tigkhhIVKoPbe/Ndzv2LIPBNH+C1Zx4wBr770io5jUNIBz4YNb1sdesdd9n7Rp2tZOT+5FO4/sI5AKok8CZAlpGX1vYHlo14fcwHI5ci9W0Vx4yTrU4cASEtT/ZDD0wBMHgFhgHL+KhFY/iUz55E930KAcEJiKOT5axJPRIXQqFU6DCoIRk+Lqf1JjQd8FklTrSsCfvrDu/ZSkbuT+69evYGs66iqJJg4So/UUneJ5hLJBnFuXAgpVLO5PLyoNclxKq5IGtq0iyLRf2B8rimcSFlOSnhYSNQj4Ck6kII9tC3Dz6wDU8B/fDIpj7Axw9+2h+rupllnVpW4Hsseb1Or624+2dcztz42azLl9HaxL/oslemSFMxGkswaK3UX146smA2XdDEExOjkb18Ap2izGDAnB0A3hMDh61cPny5GAUx34L+2617ugBH25xohaNU5UJ1pKDqOIopCaFwqnA/rY6U1basYL54XbPOlY6Ju8kNBzM3eppdvlxRriSvq3u1toQTqTbiVuqBBNUtyJZbIBfYciAm7HzucjBYFWUGO1Sn5fkou6mOSFWW1EleLyoKCHUkPeLKcd0jp+uaiQLK8MAW4G4N2NDv6z6bk89iC/jLwdfu+fGlVbjvkag2Y1pTPPxP1z0bX3sbseV4/IuOt5A9wCeYhTdnqBMXcGDQOgwB9HHAbps7wDtt2kBlXDra8uTLT0wqlQYAK4cCQmDJKGCxgt8YjvG2xwXQ9iRcb8zQtZIQI4piCzNnu8QfvJKr16iampJuyLRp2v0nm2nSXLoSS2YRyklWulQgEI9SVYXOvXnqZdwnGcwjR2zuZKpqc0j7GZeUSlLor1tPqtSNEolfJ4TOEoIEFUisHue5gFC8hDJLoOf2nwCQL/hm8iyerd32AXDHj7iETk7WLno8G8TGu7LAdd/d/8+Mq8vbOlQDXjIDi9+krwPjgBXD+q8ZjOXDAZAeAN5681gbsgv3ocMh4POZgr47EALAiMUYtYBPGI3dtBMEOdoawMmW94ucks2dquNRObGJpPjtnGKXp/RMhZsIZO2iyuuCMZzNikhB3e/+aXm1OgWq5YI+M89lmrYNM6hTYktWipG0HSFakd8kqRv/IBJx4RaoIRoUtEjWeOxvpjRiKR+X6gcEJbVBoleSMJdijH+P7QBmFD/95WObem/ot77nk5+Nn0/7r6u450f02gIMmv3GuSPnhgY89w9a23j5LXvR+Vdg1fOLXyGYIU+aN2HesCWwAPreAEDadRtQ3kIRR9Hb/B+uGTkZWLuGu8CyEaMXjx43CwC5BQBtjROtgFMTVM301DV0WRFEcAGXK8U0T4vKKwuLrrsoJF/t1UbnTt+Q4PXPm1o+bci+bNg0XEXxEPhlO6wZfhMUggYcPStZF6rL3GMKYX4Inz8uytQKXW5gQvibS5c0OK5ru8lgXBGguYAet4MF7HsAPXb0IxTkmz64ZvuY0fPx86OPltk9doQe+eaxvzGt57kp0/ETnPnjsbfdkf3oeBDOC3zqW3wSyOwJSL8GLBi6dOnIr0/3xwx5EjoeBDb3uvunG/5aWInFHoiXlmP58GXu6EVjgHX9GU7wNjjemp4WLU7FSyVVcI1QAkoIRDhZImzdTfkStlUVyhn5RNPLydY3Svmig06r3+recEZJujVWmFYX5XVuUpKVs/lAws9pTmNFpws03rCm7R9106rMDaW8+KZKu44HIuORiNXQSAYqorFijYW9PF6g1/oKxIXr2S1Elv57D6UEj27ClhQe1xY9PaNyX5zvu+vnB+skkfsJz3yM6b/efggtjjU7C7Ty7j2IJzYAIO70KeOBsecwZtE4nFg7qe8AwDu2pXSw7VGA/oS/nhw7cykGvysAiKUYvWjMAmNq/92dgDbHCNACp9Hilquy5MSKKTFlyVZAqqOZXPPKDOWKVOa/MVnhBrVk6Mbfhx49JFocO8MdlHu5GpCMRKYFcZldIeux2hIzxpCIoNGhlpL3xzKpzj9UC1AWasSqAn4IKiyPJTG4gXwdw4dIXPMaYQ7jSthbUcv2dv31P7dr62f3B/psewAzXgc+mbsU1xBs2dJvsAN8/Gyribdv/XrG+FY+4Ai6dac7jEe+Ad6YMgXXFC7EIgBrF4xDU2AsTgJHb6P+zbf6vs8sHYkV7w5cteoljAAwZvFo4HMZ17TBSZwGTrUIpjympBFqarLXhGJlc4WFaq4pJ0mrLovnSxzdtVRdL2/89Q0njvsMovrcNAskpKAd1xRL8nvtwpDmEtmtx320aaQm7m9h6CSdpcFLQsrUrZG9ksEi8ZCa9qd13SR+rywJGEauvlICdE0z7MH/um3GJGBTH2zOVOJhtK14Khn4rN1P9/yIDU99CmD9EqDX7Ek4AbRRDoidwKbeGBSY72Kqoo3zAW9Mw5KI+hXpC6C5zA6339VDxm/3t926FSAEdDBWimFYPHo0ZrhPAKdanCQt6JnmJ0UrECWb9nKqej3EhGt7Q7nEdRW5ZKAm1iAXDdppI2TrRWph/J8b/4w1do0KxZG9PM6FHWKpsKNTIWsmi0egRFRC0hVK5pTMLIUH5QSjld4/9HouITTlgSEremXIJJmEKKLZKKmldWrC4qrBcA0HppcM6rMF5Gkg+d+W8+8pdJuHevOJ3qmfAmMWYdTc0qea9r7MDuJYRwB3/tJz891rMcaePHW6PZ9jhv0Gr83JJsGy6p9SFO0l7Hjw4ZRgAFZBrB2E1XzYcuECeB1br5NuPEla/H7TaTgchycVSpe5HE97dYlT4nKevN5JxgrdKyVcCXtjjlwTyeWPNg9LRbuvMF9aEgTUCmT8NYU51Zbh2oyYJqnlLCebxI5U76cUipSp7yiGYNFQPU6EkvRSrhoJaBSyXM8nxX2kqk7aG/dzq5R1OIQOe5+Up6zAZgisf/axL1GNHwetRaPN2DoReOSbKICJG7c8iA6k654OB2+R4GA7fuqzyVw0EcIhBK8DU6e+Jk3D0vzxPTcf7XiwE8DoL8BdP68dhGvWgK0egsULx87OTut1FqDkrMDNR9oCcsIMmKoqwzYUCCDtuL5qb1CP1ZdgiKwsWYimHfVMY99+j0TMoipZSGnCC4h0RQ85EB6HEEnLuy5NS7rkyx8PJCn15qjhpbYnnNVlU7pU6pHAnHAcfqFkITMolNQTWra+m9RlhvaHD+GzwW/a4PRBYM7Ld7eb3TextmsSwLwJQGn/8tGLxy4s3XSzxiGAjvberhxA91zflXDBmRg3d+JcouLtNe8OHDm8ou3RduxWs03djY/c8yNssGXK4LUcg7F6hXDJfGsaQMjvuOkMP9Gq3YlWJ4CcbJJMXccGp1wSOlVUoywhVyQbgpfUikKSKffpBko3bdQEhA1bxAqkikgw57JUwJU5E5S5sqImeUNak79UYUayzBEqo0KzYdvMlGzKYFA5AUI41wBCdDjgyOQaoBDMOdbuCLBmKoHovRnJK0Bs+OVfsKfFd8AEjIyauZWYeAWnZrShnO5tS8QhuLjGtWO9wwuAyYCEwJB1wOAPgLP7WuMIrjn2uPgR+HUtERQYAkAMwzUzX3t7BxU3nQZvcQpodazVEYcnYOQzhDhCcjnhDDHVJoZMLJNVZDxRM2cFnErplCVTTkkNEzJxYf9Tx7VjAUZdheVczQq7GpirRK+cvfFoSYIxX7UmCU8mr9cjgqq41LiKlcRyji1Fs/VUKizmkJyO8mA0FmQCvG1JHZmM2vDwtwLkFZB3n0/dVfjXgftqfB3mLQVeH758LvAzjgG3dE8dbA/sBzpLRODy5ic/mwqALRvyGccnT8srh8ZwHF2UHNuHLx7rxb/rjheAdf1xDcdSexzE2+hxwT51M1rgX/yIIThnJXVkImSbEBCqEcNOMTWvsBofnHNF9Pq4m5HMn+u1rKxWmMirtk9xJC1FBS4TM5KGSXyKR+Fy1JHySqxN43xayTHbque4KiycJ3LDak9MYzwasn+vI2ypsoEtWZZi6ixh8aDBjrc70ia7DlgGCAoy+3DFi+8D6AKb79gx4lidL0UprtnQ8WAnZ+8dx0A67297FHQPcN+vrcswaO3kmaMWIavhaRhDIYAukA6hY+NPv3zoW+zcicWjDawgYtiwZa4A+MvF46//owVOU7fF0bZ722FIw7JwLJssdSVwQUFE2pANyVtBJZsbec1yPaL0b93rFH3bOEEKsvWrXCnHFUl4lJTf+NufK85E036a1LV4VaHPWyNi8hVIHrj1fLBJJnI8EhWGiBKvnfVwoSscGT3mr5EiNMz9jn44/hcDh5MHMGIDHsamfgB2Avfb+R55fvM9Cx/7EpnUpIbD5tV8D07vUHDbLuAoIAFdOY4Da5+ZOV4fA2sAgAFzJ7odDhEIQMKz5he473usHoLYrGELGZbz0QsAx42cae6eBL/5ODLYdu8Pqx/mcPJWXPjDtqHqGRcOjKzKmJbnaSVbR08FUqGGp66vsbOcZhVb4Q6xgw7cqzp3S0PEckvMeJW3SYVNk65L3JMRAhEuU2CYlqN7r/O5RkyuCDOuc05ILOBG2pfpUfBL9X5tW7ntYBOTtT/c0j7Ye/Ok2cDXpA/w5ltA29yO28Ue4DQa3M0z742Oz5ww8xgOtaXfYxf+1d7Frb91xeNfTJr98ZyXAchYJ7F+N+IksLfTjvaH4X4KQDz07RCMii+YHJsBg4LPtvM02vyM3Ro42vqn2x6ZgbbBf+Qqp4hoHg1+l1Cd3+TGqZnTJMEKM1d8DU7w0I1l8dIjLW7NJdK0vCDud1Ra5VEijptp4uaaytQr1BscOaJKLF+8v+mdl8+WNTzlN91otdTyWPZ8JCuYhpgV4jTlrWPnafRsM7s8dWOdczcdvHxT12+DzH/brpbIYfYi/lXfr79ODHgLD2zL6g9V7wbaHcF8/Oe9uccWT3lm8rsX36bS3T/d+Uv7wx344U638o574IeOqS+/8yqWGei/1Acv8L/jey/gcNt9+3ANUzFzcmAGZo4eRcfjjeybb76NuWh+BDjWBnfjHFB/cw/XlHNSJXO548iWJaqZwbN6KG0UcjPvUMGLUOOGqdzmYiwrFeeKHcuT00LpWu6EcwUePRmgHkWFQVOSj3pF6CbrgrAbXZABjVQwtVjlLJDmaVFl8aJsPJj3VvtqdYOLPzKXE+dK69dX2I4n70p3MdEvL8QXfYF1W7/Zduc/Kf/u1kU/hYBJ1evmnncGTAcq3x74LnrgF/C2h9rBxZ5bAPbqm5iKV2dPGgFg5Hrcsbubv0fnW4B0pwN4eCMsBsCHN6ZJCzBo7bSheAvDe9Vr1e4YgPe3fXWqxw7lcUmo6bxs+fJCUMNLtAqPxUXKTx1XCiHbhKcj5528TdNWstC0XVkyeToeybuoU57UmT+TNw2zJONRDSvIcub9MX+PzifZheMRUUb8cYtFskZJiuYkSrPpZEjS/wx1rWfkrwOuhvdc1+BEUqHsHeMz3JdttwH4ggCfP/HstDd+ucnNPfdRt2Erusdmj+5/ZQ2GAm/ye97tmdkBdBCHcaQt/e3OX3A3t/A6mzqTAHOprMlb9W47OwCn013+RB/ly/u/4xvWpBaNmT8ewckz1w6yUwPeQ6UhDsuct8P2r+6Rtz1GSCtDQVU2w8yMRuS8t9rbpLaaubnrynKt05dihVFciVElGif8nEYsWxgOJaUsydREQS7LLUQZDRBFsqhPluup6ZATk7qlb/Bhr1JtCq567Ea5cC2PmMkA7JRW4VYaQtI4//WI5T0uF+coYXgLY2qrdmH6FAh83RfrgRt/B3Zi9+4H46dR/iXu/27lG9Pcv37EdgA9t3dC5/2y2+2X+74nayeA8rcmY8G4iQtH4Zqf3V9vh9nl951DNuFuF79g8MIxGI85LwOTTWspMGjtVyflVjgCfAHfN9A/RJt0BKjKmZIHwlHTDvubZYqoWU5DiTQvzpeneV6JF+bcv7yar7w8KmdMbnPZNZhdWOsQh3KSN0sSAXrVl1ALil1Ttc10aVjUzciup6DElpLCdOulpZJ8wCoI0ExQonIMQVIsOekCYtiMvbpozCKgzTGAPQKKdf1f+EBFyxDfjdu2ACi+478uBkybPQnDl4/5awu2d+HYjwMd1Xu/72yBTrFmYfakcXNBgU/Uvo6QsLcdbsKHzyO3+4Fto5aMXe4KwTFLWEwdu3B6BfZTnL65HW4+/ex6gD0pXW6aSpcoFS5cZuqCBxr+zdUkD+XzgZQS/cdfPya7PvXqjVaRaXipmgzbSCseWahy4TmPSv2WzDQn5xe0WAmp3BFSpjCRLfeIaPhC3dz5cIoWuEJw7hcqUxOgSrSYoujSX+d3tCWBFPebLhuTAzD4cK8p6ynw8PvS6tPg0LWf0HEX0NleBlDwURRYjupStD3KfutwCDiIB4at6DxEyiuYQ4CJuEY4G+/77u4dHEcYU5cP341t8C0RZBQWjpvPJ75N4Hl9yiR0Bk7v74zTWP/Mx2MWPaUqOSNEElKeq1QiIFKd2mouaxpBtlmFoVwOhPOUcS+tzWm1RmmtabOUv1RPUur4G14SUl6BTIq4RIWkCkfRclKAh0OsjprRL0TSms8rSFZ4aITnGYHHDvrsuOkURy+prVByVQ+kiMoWYcmodf2BtwndRBB34r/jZAuaRysBYP9dePqT7fe7IjF16oR/SubjKAScDoeA25QVfcpMcwFmvDwPcycuHLt6iNsPcOFK+3irU+59aHa26x59FIBFYxaMw+xJM6a9yeDZ1PQm3Az0DDHy0diFz1H5ZLMbnVAlVRmhKiHUif8uvBk1j7RiO6YdgcpzVQUiEQxk5RKb1XWtaLxMd125QJiOaXo8AIkFUyGqBPKGC5YISiYzqXvfwaMtrnr1nMMDqA2VyVEhcl5me1o3zMQuVVT8HiKVAfWqosqs2Vn3c/vt73a/tkEAvdenpJ/Q4tRdte1dbfBpZ7/a/ZOHN373kMSXry47FXx/xcHbf8Ux3JW2dz0NxXYY3n4NE0AxFkPwHPD1g7jzV9EBvOmlPRPP7mknY5ngAuOASbMngU/BVOBM8xPn+3o+xZDRxtMfAc3quPXsIoVTIgCLpqRgJgNvLaFasMyR1Uu6xxdOSbxuQdmpYNe9lKsJ26Okbc1jK7bDVGJlFE6ZITlG0PJy25NnwjXVeEmDgrQ3Jks+wYWTsYpqClTG1YDEqwqTRXnLE9JyskELroaZfetYfK7uBvpt7o0tGWPiQ9+e6vxzqxO4cw16NC1vgASGG7H3Fw0Bbn8RIOhkHbP2t+/mGbm0HZXeeu0dMDJ+wbhlI9bI9Pm+2CZzgWMEDS/fWXVe0jACwCJ3PGa5063pwK8B7SYYF5/4HFgNPDWQrrH/LmVFqpE2BAFRbcN2DEkzHDkkpVPRcquBk/AE1ZjirTxdxHZRKjTGhaU6tseTZFQSQtMYcbyuh8hCh824wsXVRsS5Uv/OT1WPlpVNp6DIjTpR15Lr+qR6AS2rGxcvaJLl5bZI+TPsQtEb06iLrvc3IthCn1qCb9GWQLQ78suIZTgHFe5DlhDPjQFa5QDsAJWwE2L3buBIy3em41VgFjhcDH73xU+e/kbc80tXCHGK1f91a8ODBwEsGSXGz3H45DddbI9Cpr87ly59/oyqK/OfVpgNzVG81FVyMohiUVXNOoQSQwirpexDabyyYVqxhMgbtqc2Z0dl4VJJ1qtwPTGCUSJRSrJeEA6XW37H6ygiG8hYQgRZ/M6tsVDM61CfJWSfw9WUqxREIVOveukSpXmd5HIqs70MGRNkUoc999EH8eDWT59668128r6WJ29q27McqW7OPuy8Wyv+a9N8acyJE3jm43ssgwE4gvHzATZlyjuvAq9gArCUDQT/RPT6HkBbnBTn5SNFPbcvw4hRGDuPg0+dCvQEThHkLy7Fx0OWAkRZBVgZMx3I24xT23U0UyLptJKFNymi5VnTo4SVgrRW/FeRHK1tov1TL5O3wIJRLvs9Vx3CSDArAgJ5eMGYanAq4EHGTuQaCE+g5Xa/5oUtReI5gKerlJScjeoE+YvEQxg1I1mFhcDgzv7k0UVj2r/xyZYHgacQ6yhncBLyjVc+xJE7sgB+Ap4cOn4WZr2Cj5HZg2s6BZvOB/De69NfxdyJs6SJC0YDWOcSsUkI4LAkbsLlpSO9IHyxEGMnAJj6Gn/nbLPjLXCq0+sDrY8YBq/ZAODJC2laiFCDS3nILgdNxHJaJiiR3I1XEoEroSyvDddNxuowQ6rpWKNn7UyxlFHdRJDI1PIHRZIBpqyDMm6lPC4RriSKagNhkZEKO/gqbcKpXWGYFnW0CK/fsiwSz9Q/WZlzJJalf/lrZTPNwPH01+X3/DgFD+Jr++P4yB47AHTerxaijfzfVvh/n+GdV2AuHIs+qe47AbCfWgG480aJvK1MnMMdjJs/Hqv7f8Af/Zrd9evtos3x0zfX/x/URcOXjV4wDvOE49pvv/VDM3Agv3bQ8Hf7kzGLgJfs92zBDUKkjJ/BFZJtB9Jel1qyE/yzEJkCk1PLiRHHiNArEbPelYSixoLVpQlWouXyeafET7jiqNwxPIIKjVNuEJaTs8KRda2sRTIRjIcMWYUsHNf2m/7fFe4izwvj/iTxWSFRYAubNeT4qC/QfjqAvgAmzwTQ1um8/65eW7vgBDqyvcALDbLotTmAB0J1WME3HQ/ueXb+m2/BozI6GfMmAJhDALGWv4C+WyB2u0danzzT/D/QsIQsBCAm4m0BJv9+kwBqK0D6E7EIGLoS+LqhAtvnD15wiJAEUyxK4c3aFA2Smt8mBXYyE3T1SEEWlw8EPDRUh5VLx70+ZIttJa0yqqT9POU1vESt0QWnlqXkAg1cOe81E6dyGgJEwWWPbvozwmFyg0siX04rNJIkwo775BrC80w6j+ewdtDhyTc9i83kwZUX0f5wy6Pojp97dNmHW+JC4LbIByPlkQVvonnZNgC37epUd/1rb+G+rZOnzAImAO+Ql+diifvSex8yZm/vgX0CLU+faQ5GKB+7aOHYiYCQ8BpwoN25/O9TsAwvLn9M8a3E8x++sNNHr3htVyOOzABZryRp6mbDanUwbVs0H/cJbuiX/TI67qxzRI4FDVsNVIf0aJkT92mE1nq4QWg457VCLhNCY3awVi4U+WxJw8tSjFCe8RdfNUjeI7n07MWWV5rV/T2S51q+his0yZRMEZNxDXnq05kffUt6bwaZg8OgQL7lSWdfl33sD9x95y+3wzsL15xBh0NdLAMHnhyFqclj/WfiFSwYB9BJmLgAWDUA+ERyf75LtMdxgfPHHlswGhizZBEfN9uRMFsdE0NTrB27EHgfXwLP2x++aMsspxOi2jnhuIBNHFfiVNh5w5R9GVeJun4XCd0rMhdVqV7SydmW1NhHVNRP/lNXClEFbqFhWZotUThWAK7kBvJC0TWB/xw75NI8lURd7ghHEoqZiyUPp8mf4TwRIpTkss0DNjt7w1/49Cl07/IcNm7qjZfwYOWB4z2SBwF0lbvsaqdLPz0Vwxn0yv5vzeA2h7APrXDfZxPeRj+2boY8CeMWuhMmzSJ0HJa9tFoayB8FwIHWp27+/XqMG7ZiyvRRWIxJc8gSjJqZ/YNW7hd4hjkf99uADwGe8tcmTDMraYoNTm07J/lzZlE4iYgnk1NNT40u+bSYdqZRebZZuy+tgJ6X0nIyqtDKFGNq1p8NkawRThM5JYVl4phekq4xI3byXFuDXt8oVRUhSYnXjWsyUUFy2Q60cr+wbeb4clFCZZenWBP3zl8oQOfgm4exuTdeO33gZm0H2uq7fwPQ4RBwy6fo9ie23j9oMI7hmhPQR5EpzoFfaUxfmgAACqFJREFU3ngdcyklExaMe2X2xPnjgSHAM58/AXTbJ7cH0B6Pr8AlYEpqFn156UhAfvR8E7yew8cvJh7ZMOA9AB92T4oqRlWVWjYXXHZVRdKyCSE5jsTybtIplKyKG9yqeJ3iq+7+bDSUZx5k9ChB6Nx1JcLJ6g7helq14AcMotVCzuVoTXFUwNL/Pk9aJqrDbjjlpUbY5Y6clXbXJnKJqw0SSsCK+wXsBDt/3S9znsCEeaNb8I0P9/6iagS67Qag81t/u8NOG8Btu4as3n0b7oqs7bm9xSmg+Rl887Lg7wDT3nkVZOz8BQKzxazxC0cso8OAJ3ANbw+Bmy5el3nkm1o8eu5zjHwT72VHTTxIgVg1kNk6YOC7zzBlTc/tTYXmGCysColLEBC2cCXB7QBPEq+tGRVR2++lUtQsK7koG6msJWUVT1YTvNxrc2aDcDDhywo3G7QU4oubMZLmTmFFXiSkdl+mjmopg1cHJDkCfzbgNqfeylh1XS3j1wnTiZpNgDW12wl83fcOPgjAVnPES5k/Wx9vfkbafSv+C7RqfziDcqD59R/ed/N2nGrlQOq+cxRYsl2dNm9j1sR5GI/5oK7A2EV8BNYypR/wS1eAn6IuvF/C1zefB5b2brAM+KzjWWBl94eoDTmNjwfyJz8DcFWoPo/NHSIcXapO+SsJNZikWqps+ZOMZnzljq++Eay5Wj/tyTmQAlpDsOTJOqUMiuwwh1NBXUG4xNWklylWQ7eeR6R4feo2+ktmPtvIJzXfpQbhermi8+0ryrMxodpMTmsS465us/D+6ydBm/sz1gz+VvTaiFW45kzHnfgNwC17gaPt/wEavfqi8z1anDoBdPAPlOfgmltm4ZU5AsB4TATmvDxm0bIRgjufag/fib23tD4pGqMUL/E1T27rsQN/5HCwow/GmsG3at8Cw7P2ox7ZzKIjpbqhKgEZEiUyqJnIuhFXsamcc5kllTCJVjnhcKMLAb08rrrFhsP9ZjDB7faqhyFNCbccYlsGDTiqY6aj3NG0gG1TOesXf4pSb43mqbQMVvFPrkf7MrnWUK+W61FRFcnapkc2s54ku64Rm97yATyyFF+Th7b3BPr+oR7peLDDIeDW32irE2gXSuMJd8YlT4dDovkZtAz5fAz3lbf4hL6CeZzMo1RgLCAw1x2zbDCAL/HDvbccZJwACx9bNQKep3LAuXPoCD9a9+v82wN45JvlwKPrR2e3QKkvGCWW5JUdzoRkEEfJuR4bmYwD25ErCpkIp/2O2/CSRwlkRK1Ni30+lkyWtyA0LYU54RoVyIZdQfNU8WvJSIpdrJ/TGzJOclcb/uH1MU9Bzusv5YHzuSYX/nfwUqGIUSXhusRWsrxGYZ8B+AbQsarvps09MWX6162PwAbtaOO3bruBzvuBHp9/PmwtWkM60eT8yZN4ZdYd3+ME+DsSMGHuuMV83HyMJ5g4D2L5cKx/7Ot7AS6AUy2AAWTYCvzrgW3Y5Xzx8xrcsw3f4JqvsPhpdGfVETXFGdEZdZiUA6PVTOK2TAWM+qKgpMJxQpm0WulerG9Tyc36goRrZzJFhWGF+yQg50lEuRQCJxoxYqG05uoinPfFIgJoGglAInnZ0ImeyEu0mVPys6ckX2iZrqfoivdS3qumVdargXfuI5srBwFf9QE2PgwcR3uKA2ip964FWuxH1z3i/pIV9/5wvNUJnO9wCC/Oap9/tkhnhIgJczFxwWgsAp8v5gm+QHC8xz/pi+/v6wzgFBAw3xsK4HHnm22fP3EbcFtX/w+4puf2JwqXwcXOOzyFXjsZ9ckOJFPK+fI5WaOWLHS5UifEMZMR7aojctmkFOBXdTeS9xSTf64ocpHOCBFpnxehrEIlzoQphFfwLGWOndTiaRYraCJ5AiaR0zLN5QtYQfY3/eLvrh/U53prJeYPBUyNgW19zsGG3gBcga29CJ5dDxxGh0O3/3rL5h73fR8G9iCwBVBxs2gBUNxyGtd9tRdvkNcATJwLLBxLMQb/mj9qKQa8zwEH2NcFLYD3+sIduhIeNlA88fOpMfd/98A2DPDWfrwd/mUjl34OWDTjaEpVWT7sCkcHYeGYQ2VmmPCLJHMR9qf9il+NCLua0wzy+tWEW3BdyEl6iWXJsjcLyRPIQkrBw0xB4FiUQ3M8RoOKyquXZJqP5kOu62HcNBukqs81rqqTUrx2lpj+XP0yTyxSwfDRU+j3bW3/kfyxzb0ArAfaiaOH8Gtr0WfTvdiFa4J4oSh3e+3Jlvx0u2678dIq4E3yxjSJTMZEAItGYRHGzB+/ABiJNS8CG8mPcpeDHU+1ON/ka6wZNtJZOYY4uOsZfAcZeO+FJP5T+i7+dZvBaoKa7qV/2Jz6uOK6CduRnHTEn8t6QmVqgU5kp7rUrinynqSF1dzreG9QmJemSkjG5yMws4ohEUfI3ozIqjCpR8rVsbzEG/J1dPyBzr+e4QyqE+TEOXu8sl6xJKteKZVk+agZAEvTSpN12fcp8NCXWLrkK7JNyr/+m2EewW3unp4VLmvxA4A+m5olHrsaSFC37VGgNNRGXoTXZ5CpeGMmnSUYBeGLMRqLCcYtXDwaHB/KDwM/oyNawP15ufL5CmD8fIzDoLV4Kb0BA4wPgEDwlVlLAexqb0COx+2IVxaCUMlR0wb1ZCMu9SRdZrqyx4lr2YTnaiBB3XiI1s2G/HKtrkrEk6WEwvBSEC7DlvwEsiObMidp+SrVSv80Jeq6jbYpBV5LIrkIcVOSoqa9GZaFWciDvpoIJI3k2L4urOFjDz0GjPrqQfzgzsC/XI7t6PR1y3t+HFP294DTFS2/1Dfimi4Fm0eq815W6BviLUomz5w8m4uX54/HIoAQLBkFQKzF0/hWugs4KEk3Hg3XvvDBCE2e5M4bvXj48lXPor+VBFDkfwf4T34fDnc1HF+kJmZYQFZnrhmwJIO43AwS4iE0wsuutA5XtHQugxZ4EiTtD6ooUWiBMAKQbY/FBUVGU2FrxOPYciLIZYQ0Ktl6yCtJIhquzVKhyWEn78pKutInUSvJiCjyp8y/avP+JGHtXL5hA4BxCwS2uI9Onolr9uCa0HMfnXxi0bg/8gVV72MjcM+FcHTrS0sngArCyZvA9MmzJs2S5o1fQMni0aOWrhi2TIzE0LXPbyAu+eHe3zoC0NYBWIZrJghEXnt7/Yv5T4E+mwpnjq2pSBzE3dFzEhf/uIaqUAHiWrKmeNw0CVqpIk4LxZVGDf7IF1RVMk9Qif8VLtKDQN4vCCdmpq7BadYrgfopkU2JQ9KFmk3XzTKiEirYpZzQWjVkJVw4wSvcK3yUhwur8lqhJfSgoyCViUQQZeJguz6Do12wAAKEfPH9bQnGDgL3fY8fe3c8aKH21Cm0uPOXtkfxI4BnV42lkwnF9KlTp854HQSvzHoFYszi0YvoyBXLh2PFsPfEBxAuufcXDuD4TXjm42c+HrEMmId38OqL78cdPP3JJsx8Y9o9P6FDqGhDW+Fmb4geNg1bOISwiqsJxryyrXpTKS0t+2ptT3UqZ7JzjblZLDKRK41pkFAIoiiOGeSyDUv4kiFZuB4bl5oUmqVCh3CJq3IPQeODp9Ryg/my3MrU8YGW18adeMTN/VW/Tk1l/eDfbnH6/wCF3sDVl8uhSQAAAABJRU5ErkJggg==", "text/plain": [ "128×256 Array{Gray{Float64},2} with eltype Gray{Float64}:\n", " Gray{Float64}(0.0) Gray{Float64}(0.0) … Gray{Float64}(0.627451)\n", " Gray{Float64}(0.627451) Gray{Float64}(0.623529) Gray{Float64}(0.388235)\n", " Gray{Float64}(0.611765) Gray{Float64}(0.611765) Gray{Float64}(0.337453)\n", " Gray{Float64}(0.0) Gray{Float64}(0.0) Gray{Float64}(0.192157)\n", " Gray{Float64}(0.611765) Gray{Float64}(0.0) Gray{Float64}(0.277459)\n", " Gray{Float64}(0.0) Gray{Float64}(0.0) … Gray{Float64}(0.2)\n", " Gray{Float64}(0.607843) Gray{Float64}(0.0) Gray{Float64}(0.197013)\n", " Gray{Float64}(0.0) Gray{Float64}(0.0) Gray{Float64}(0.215686)\n", " Gray{Float64}(0.619608) Gray{Float64}(0.619608) Gray{Float64}(0.207843)\n", " Gray{Float64}(0.0) Gray{Float64}(0.0) Gray{Float64}(0.188235)\n", " Gray{Float64}(0.635294) Gray{Float64}(0.0) … Gray{Float64}(0.206486)\n", " Gray{Float64}(0.631373) Gray{Float64}(0.0) Gray{Float64}(0.184314)\n", " Gray{Float64}(0.0) Gray{Float64}(0.627451) Gray{Float64}(0.184314)\n", " ⋮ ⋱ ⋮\n", " Gray{Float64}(0.0) Gray{Float64}(0.129412) Gray{Float64}(0.241796)\n", " Gray{Float64}(0.14902) Gray{Float64}(0.129412) Gray{Float64}(0.265208)\n", " Gray{Float64}(0.215686) Gray{Float64}(0.0) Gray{Float64}(0.207843)\n", " Gray{Float64}(0.345098) Gray{Float64}(0.341176) Gray{Float64}(0.231373)\n", " Gray{Float64}(0.0) Gray{Float64}(0.0) … Gray{Float64}(0.258824)\n", " Gray{Float64}(0.298039) Gray{Float64}(0.415686) Gray{Float64}(0.258824)\n", " Gray{Float64}(0.0) Gray{Float64}(0.368627) Gray{Float64}(0.235294)\n", " Gray{Float64}(0.0) Gray{Float64}(0.0) Gray{Float64}(0.207843)\n", " Gray{Float64}(0.219608) Gray{Float64}(0.0) Gray{Float64}(0.2)\n", " Gray{Float64}(0.0) Gray{Float64}(0.219608) … Gray{Float64}(0.203565)\n", " Gray{Float64}(0.196078) Gray{Float64}(0.207843) Gray{Float64}(0.345098)\n", " Gray{Float64}(0.192157) Gray{Float64}(0.0) Gray{Float64}(0.304792)" ] }, "execution_count": 48, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# [lenna colorview(Gray, X_Lenna_JuMP) lenna_original]\n", "[lenna colorview(Gray, X_Lenna_JuMP)]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Time taken by `JuMP` is 25 s whereas time taken by `Convex.jl` is 415 s. The main reason behind the time difference is that, to use `Convex.jl` we could feed it the problem formulation rather than converting into an SDP form, so internally `Convex.jl` converts the model into an SDP programmatically. The converted SDP has the follwoing size:\n", "\n", "```\n", "The SDP size in Convex.jl (constructed internally Convex.jl) \n", "#------------------------------------------\n", " Constraints : 73665 \n", " Scalar variables : 49153\n", "```\n", "\n", "Whereas, we converted the problem ourselves into an SDP to feed it into `JuMP`. The final formulation was much tighter than what `Convex.jl` does automatically. Note that we invested some time here by researching into google scholar and finding the right paper. The SDP in `JuMP` has the following size:\n", " \n", "```\n", "The SDP size in JuMP \n", "#-------------------------\n", " Constraints : 8129 \n", " Scalar variables : 16257 \n", "``` " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Sequential convex programming\n", "\n", "* Solving a nonconvex problem using a local convex optimization method\n", "\n", "\n", "\n", "* Convex portions of a problem are handled \"exactly\" and efficiently\n", "\n", "\n", "\n", "* Sequential convex programming is a heuristic, it can fail\n", "\n", "\n", "\n", "* Success often depend on a good starting point\n", "\n", "\n", "\n", "* We consider the nonconvex problem: \n", "\n", "\n", "$$\n", "\\begin{array}{ll}\n", "\\underset{x\\in\\mathbf{R}^{d}}{\\mbox{minimize}} & f_{0}(x)\\\\\n", "\\mbox{subject to} & f_{i}(x)\\leq0,\\quad i=1,\\ldots,m,\\\\\n", " & h_{i}(x)=0,\\quad j=1,\\ldots,p.\n", "\\end{array}\n", "$$\n", "\n", "\n", "where $f_{0}$ and $f_{i}$ are possibly nonconvex, $h_{i}$ are\n", "possibly non-affine. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Basic idea of Sequential convex programming\n", "\n", "* Maintain the current iterate $x^{(k)}$ and convex trust region $\\mathcal{T}^{(k)}$\n", "\n", "* Form convex approximation $f_{i}^{\\textrm{cvx}}$ of $f_{i}$ over\n", "$\\mathcal{T}^{(k)}$\n", "\n", "* Form affine approximation $h_{i}^{\\textrm{afn}}$ of $h_{i}$ over\n", "$\\mathcal{T}^{(k)}$\n", "\n", "* Then update the iterate $x^{(k+1)}$ is the optimal point found by\n", "solving the convex problem \n", "$$\n", "\\begin{array}{ll}\n", "\\underset{x\\in\\mathbf{R}^{d}}{\\mbox{minimize}} & f_{0}^{\\textrm{cvx}}(x)\\\\\n", "\\mbox{subject to} & f_{i}^{\\textrm{cvx}}(x)\\leq0,\\quad i=1,\\ldots,m,\\\\\n", " & h_{i}^{\\textrm{afn}}(x)=0,\\quad j=1,\\ldots,p,\\\\\n", " & x\\in\\mathcal{T}^{(k)},\n", "\\end{array}\n", "$$\n", "which is a convex approximation of the original nonconvex problem." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## How to compute the approximations\n", "\n", "Trust region is computed using $\\mathcal{T}^{(k)}=\\{x\\mid\\|x-x^{(k)}\\|\\leq\\rho\\}$\n", "\n", "* $h_{i}^{\\textrm{afn}}=h_{i}(x^{(k)})+\\nabla h_{i}(x^{(k)})^{\\top}(x-x^{(k)})$\n", "\n", "* $f_{i}^{\\textrm{cvx}}=f_{i}(x^{(k)})+\\nabla f_{i}(x^{(k)})^{\\top}(x-x^{(k)})+\\frac{1}{2}(x-x^{(k)})^{\\top}P(x-x^{(k)})$\n", "where $P=\\left[\\nabla^{2}f(x^{(k)})\\right]_{+}$ which is the PSD\n", "part of Hessian" ] } ], "metadata": { "kernelspec": { "display_name": "Julia 1.6.1", "language": "julia", "name": "julia-1.6" }, "language_info": { "file_extension": ".jl", "mimetype": "application/julia", "name": "julia", "version": "1.6.1" }, "latex_envs": { "LaTeX_envs_menu_present": true, "autoclose": false, "autocomplete": true, "bibliofile": "biblio.bib", "cite_by": "apalike", "current_citInitial": 1, "eqLabelWithNumbers": true, "eqNumInitial": 1, "hotkeys": { "equation": "Ctrl-E", "itemize": "Ctrl-I" }, "labels_anchors": false, "latex_user_defs": false, "report_style_numbering": false, "user_envs_cfg": false }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": {}, "toc_section_display": true, "toc_window_display": false }, "varInspector": { "cols": { "lenName": 16, "lenType": 16, "lenVar": 40 }, "kernels_config": { "python": { "delete_cmd_postfix": "", "delete_cmd_prefix": "del ", "library": "var_list.py", "varRefreshCmd": "print(var_dic_list())" }, "r": { "delete_cmd_postfix": ") ", "delete_cmd_prefix": "rm(", "library": "var_list.r", "varRefreshCmd": "cat(var_dic_list()) " } }, "types_to_exclude": [ "module", "function", "builtin_function_or_method", "instance", "_Feature" ], "window_display": false } }, "nbformat": 4, "nbformat_minor": 4 }