{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "# Probability Theory Review"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Preliminaries\n",
    "\n",
    "- Goal \n",
    "  - Review of probability theory as a theory for rational/logical reasoning with uncertainties (i.e., a Bayesian interpretation)\n",
    "- Materials        \n",
    "  - Mandatory\n",
    "    - These lecture notes\n",
    "    - [Ariel Caticha - 2012 - Entropic Inference and the Foundations of Physics](https://github.com/bertdv/BMLIP/blob/master/lessons/notebooks/files/Caticha-2012-Entropic-Inference-and-the-Foundations-of-Physics.pdf), pp.7-26 (sections 2.1 through 2.5), on deriving probability theory. You may skip section 2.3.4: Cox's proof (pp.15-18). \n",
    "      - The assignment is only meant to appreciate how this line of \"axiomatic derivation\" of the rules of PT goes. I will not ask questions about any details of the derivations at the exam. \n",
    "    \n",
    "  - Optional\n",
    "    - the pre-recorded video guide and live class of 2020\n",
    "    - [Ariel Caticha - 2012 - Entropic Inference and the Foundations of Physics](https://github.com/bertdv/BMLIP/blob/master/lessons/notebooks/files/Caticha-2012-Entropic-Inference-and-the-Foundations-of-Physics.pdf), pp.7-56 (ch.2: probability)\n",
    "      - Great introduction to probability theory, in particular w.r.t. its correct interpretation as a state-of-knowledge.\n",
    "      - Absolutely worth your time to read the whole chapter!\n",
    "    - [Edwin Jaynes - 2003 - Probability Theory -- The Logic of Science](https://archive.org/details/ProbabilityTheoryTheLogicOfScience). \n",
    "      - Brilliant book on Bayesian view on probability theory. Just for fun, scan the annotated bibliography and references.\n",
    "    - Bishop pp. 12-24"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Example Problem: Disease Diagnosis\n",
    "\n",
    "- **Problem**: Given a disease with prevalence of  1%  and a test procedure  with sensitivity ('true positive' rate) of  95%  and specificity ('true negative' rate) of  85% , what is the chance that somebody who tests positive actually has the disease?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- **Solution**: Use probabilistic inference, to be discussed in this lecture. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### The Design of Probability Theory\n",
    "\n",
    "- Define an **event** (or \"proposition\") $A$ as a statement, whose truth can be contemplated by a person, e.g., \n",
    "\n",
    "$$𝐴= \\texttt{'there is life on Mars'}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- If we assume the fact $$I = \\texttt{'All known life forms require water'}$$ and a new piece of information $$x = \\texttt{'There is water on Mars'}$$ becomes available, how _should_ our degree of belief in event $A$ be affected (if we were rational)? "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "- [Richard T. Cox, 1946](https://aapt.scitation.org/doi/10.1119/1.1990764) developed a **calculus for rational reasoning** about how to represent and update the degree of _beliefs_ about the truth value of events when faced with new information.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- In developing this calculus, only some very agreeable assumptions were made, e.g.,\n",
    "  - (Transitivity). If the belief in $A$ is greater than the belief in $B$, and the belief in $B$ is greater than the belief in $C$, then the belief in $A$ must be greater than the belief in $C$.\n",
    "  - (Consistency). If the belief in an event can be inferred in two different ways, then the two ways must agree on the resulting belief."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "- This effort resulted in confirming that the [sum and product rules of Probability Theory](#PT-calculus) are the **only** proper rational way to process belief intensities. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- $\\Rightarrow$ Probability theory (PT) provides _the_ **theory of optimal processing of incomplete information** (see [Cox theorem](https://en.wikipedia.org/wiki/Cox%27s_theorem), and [Caticha](https://github.com/bertdv/BMLIP/blob/master/lessons/notebooks/files/Caticha-2012-Entropic-Inference-and-the-Foundations-of-Physics.pdf), pp.7-24), and as such provides the (only) proper quantitative framework for drawing conclusions from a finite (read: incomplete) data set."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Why Probability Theory for Machine Learning?\n",
    "\n",
    "- Machine learning concerns drawing conclusions about model parameter settings from (a finite set of) data and therefore PT provides the _optimal calculus for machine learning_. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- In general, nearly all interesting questions in machine learning can be stated in the following form (a conditional probability):\n",
    "\n",
    "$$p(\\texttt{whatever-we-want-to-know}\\, | \\,\\texttt{whatever-we-do-know})$$\n",
    "\n",
    "where $p(a|b)$ means the probability that $a$ is true, given that $b$ is true."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Examples\n",
    "  - Predictions\n",
    "  $$p(\\,\\texttt{future-observations}\\,|\\,\\texttt{past-observations}\\,)$$\n",
    "  - Classify a received data point $x$ \n",
    "  $$p(\\,x\\texttt{-belongs-to-class-}k \\,|\\,x\\,)$$\n",
    "  - Update a model based on a new observation\n",
    "   $$p(\\,\\texttt{model-parameters} \\,|\\,\\texttt{new-observation},\\,\\texttt{past-observations}\\,)$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Frequentist vs. Bayesian Interpretation of Probabilities\n",
    "\n",
    "- The interpretation of a probability as a **degree-of-belief** about the truth value of an event is also called the **Bayesian** interpretation.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- In the **Bayesian** interpretation, the probability is associated with a **state-of-knowledge** (usually held by a person). \n",
    "  - For instance, in a coin tossing experiment, $p(\\texttt{tail}) = 0.4$ should be interpreted as the belief that there is a 40% chance that $\\texttt{tail}$ comes up if the coin were tossed.\n",
    "  - Under the Bayesian interpretation, PT calculus (sum and product rules) **extends boolean logic to rational reasoning with uncertainty**. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "- The Bayesian interpretation contrasts with the **frequentist** interpretation of a probability as the relative frequency that an event would occur under repeated execution of an experiment.\n",
    "\n",
    "  - For instance, if the experiment is tossing a coin, then $p(\\texttt{tail}) = 0.4$ means that in the limit of a large number of coin tosses, 40% of outcomes turn up as $\\texttt{tail}$.     "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- The Bayesian viewpoint is more generally applicable than the frequentist viewpoint, e.g., it is hard to apply the frequentist viewpoint to events like '$\\texttt{it will rain tomorrow}$'. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- The Bayesian viewpoint is clearly favored in the machine learning community. (In this class, we also strongly favor the Bayesian interpretation). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Probability Theory Notation\n",
    "\n",
    "##### events\n",
    "-  Define an **event** $A$ as a statement, whose truth can be contemplated by a person, e.g.,\n",
    "\n",
    "$$A = \\text{'it will rain tomorrow'}$$\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- We write the denial of $A$, i.e. the event **not**-A, as $\\bar{A}$. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Given two events $A$ and $B$, we write the **conjunction** \"$A \\wedge B$\" as \"$A,B$\" or \"$AB$\". The conjunction $AB$ is true only if both $A$ and $B$ are true. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- We will write the **disjunction** \"$A \\lor B$\" as \"$A + B$\", which is true if either $A$ or $B$ is true or both $A$ and $B$ are true. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Note that, if $X$ is a variable, then an assignment $X=x$ (with $x$ a value, e.g., $X=5$) can be interpreted as an event. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### probabilities\n",
    "\n",
    "- For any event $A$, with background knowledge $I$, the **conditional probability of $A$ given $I$**, is written as \n",
    "$$p(A|I)\\,.$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- All probabilities are in principle conditional probabilities of the type $p(A|I)$, since there is always some background knowledge. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "##### Unfortunately, PT notation is usually rather sloppy :(\n",
    "\n",
    "- We often write $p(A)$ rather than $p(A|I)$ if the background knowledge $I$ is assumed to be obviously present. E.g., $p(A)$ rather than $p(\\,A\\,|\\,\\text{the-sun-comes-up-tomorrow}\\,)$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- (In the context of random variable assignments) we often write $p(x)$ rather than $p(X=x)$, assuming that the reader understands the context.  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- In an apparent effort to further abuse notational conventions, $p(X)$ denotes the full distribution over random variable $X$, i.e., the distribution for all assignments for $X$. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- If $X$ is a *discretely* valued variable, then $p(X=x)$ is a probability *mass* function (PMF) with $0\\le p(X=x)\\le 1$ and normalization $\\sum_x p(x) =1$. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- If $X$ is *continuously* valued, then $p(X=x)$ is a probability *density* function (PDF) with $p(X=x)\\ge 0$  and normalization $\\int_x p(x)\\mathrm{d}x=1$. \n",
    "  - Note that if $X$ is continuously valued, then the value of the PDF $p(x)$ is not necessarily $\\le 1$. E.g., a uniform distribution on the continuous domain $[0,.5]$ has value $p(x) = 2$.\n",
    "  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Often, we do not bother to specify if $p(x)$ refers to a continuous or discrete variable.   "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### <a id=\"PT-calculus\">Probability Theory Calculus</a>\n",
    " \n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Let $p(A|I)$ indicate the belief in event $A$, given that $I$ is true. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- The following product and sum rules are also known as the **axioms of probability theory**, but as discussed above, under some mild assumptions, they can be derived as the unique rules for *rational reasoning under uncertainty* ([Cox theorem, 1946](https://en.wikipedia.org/wiki/Cox%27s_theorem), and [Caticha, 2012](https://github.com/bertdv/BMLIP/blob/master/lessons/notebooks/files/Caticha-2012-Entropic-Inference-and-the-Foundations-of-Physics.pdf), pp.7-26)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- **Sum rule**. The disjunction for two events $A$ and $B$ given background $I$ is given by\n",
    "$$ \\boxed{p(A+B|I) = p(A|I) + p(B|I) - p(A,B|I)}$$\n",
    "  "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- **Product rule**. The conjuction of two events $A$ and $B$ with given background $I$ is given by \n",
    "$$ \\boxed{p(A,B|I) = p(A|B,I)\\,p(B|I)}$$\n",
    "  \n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- **All legitimate probabilistic relations can be derived from the sum and product rules!**"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Independent and Mutually Exclusive Events\n",
    "\n",
    "- Two events $A$ and $B$ are said to be **independent** if the probability of one is not altered by information about the truth of the other, i.e., $p(A|B) = p(A)$\n",
    "  - $\\Rightarrow$ If $A$ and $B$ are independent, given $I$, then the product rule simplifies to $$p(A,B|I) = p(A|I) p(B|I)$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Two events $A_1$ and $A_2$ are said to be **mutually exclusive** if they cannot be true simultanously, i.e., if $p(A_1,A_2)=0$.\n",
    "  - $\\Rightarrow$ For mutually exclusive events, the sum rule simplifies to\n",
    "  $$p(A_1+A_2) = p(A_1) + p(A_2)$$\n",
    "        "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- A set of events $A_1, A_2, \\ldots, A_N$ is said to be **collectively exhaustive** if one of the statements is necessarily true, i.e., $A_1+A_2+\\cdots +A_N=\\mathrm{TRUE}$, or equivalently \n",
    "$$p(A_1+A_2+\\cdots +A_N)=1$$\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Note that, if $A_1, A_2, \\ldots, A_n$ are both **mutually exclusive** and **collectively exhausitive** (MECE) events, then\n",
    "    $$\\sum_{n=1}^N p(A_n) = p(A_1 + \\ldots + A_N) = 1$$\n",
    "    - More generally, if $\\{A_n\\}$ are MECE events, then $\\sum_{n=1}^N p(A_n,B) = p(B)$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### The Sum Rule and Marginalization\n",
    "\n",
    "- We mentioned that every inference problem in PT can be evaluated through the sum and product rules. Next, we present two useful corollaries: (1) _Marginalization_ and (2) _Bayes rule_ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- If $X \\in \\mathcal{X}$ and $Y \\in \\mathcal{Y}$ are random variables over finite domains, than it follows from the above considerations about MECE events that \n",
    "$$\n",
    "\\sum_{Y\\in \\mathcal{Y}} p(X,Y) = p(X) \\,.\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Summing $Y$ out of a joint distribution $p(X,Y)$ is called **marginalization** and the result $p(X)$ is sometimes referred to as the **marginal probability**. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Note that this is just a **generalized sum rule**. In fact, Bishop (p.14) (and some other authors as well) calls this the sum rule.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Of course, in the continuous domain, the (generalized) sum rule becomes\n",
    "$$p(X)=\\int p(X,Y) \\,\\mathrm{d}Y$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### <a id=\"Bayes-rule\">The Product Rule and Bayes Rule</a>\n",
    "\n",
    "- Consider two variables $D$ and $\\theta$; it follows from symmetry arguments that \n",
    "$$p(D,\\theta)=p(D|\\theta)p(\\theta)=p(\\theta|D)p(D)$$ \n",
    "and hence that\n",
    "$$ p(\\theta|D) = \\frac{p(D|\\theta) }{p(D)}p(\\theta)\\,.$$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- This formula is called **Bayes rule** (or Bayes theorem). While Bayes rule is always true, a particularly useful application occurs when $D$ refers to an observed data set and $\\theta$ is set of model parameters. In that case,\n",
    "\n",
    "  - the **prior** probability $p(\\theta)$ represents our **state-of-knowledge** about proper values for $\\theta$, before seeing the data $D$.\n",
    "  - the **posterior** probability $p(\\theta|D)$ represents our state-of-knowledge about $\\theta$ after we have seen the data."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "$\\Rightarrow$ Bayes rule tells us how to update our knowledge about model parameters when facing new data. Hence, \n",
    "\n",
    "<center>\n",
    "<div style=\"font-size:large; color:red\">\n",
    "Bayes rule is the fundamental rule for learning from data!\n",
    "</div>\n",
    "</center>"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Bayes Rule Nomenclature\n",
    "- Some nomenclature associated with Bayes rule:\n",
    "$$\n",
    "\\underbrace{p(\\theta | D)}_{\\text{posterior}} = \\frac{\\overbrace{p(D|\\theta)}^{\\text{likelihood}} \\times \\overbrace{p(\\theta)}^{\\text{prior}}}{\\underbrace{p(D)}_{\\text{evidence}}}\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Note that the evidence (a.k.a. _marginal likelihood_ ) can be computed from the numerator through marginalization since\n",
    "$$ p(D) = \\int p(D,\\theta) \\,\\mathrm{d}\\theta = \\int p(D|\\theta)\\,p(\\theta) \\,\\mathrm{d}\\theta$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "source": [
    "- Hence, having access to likelihood and prior is in principle sufficient to compute both the evidence and the posterior. To emphasize that point, Bayes rule is sometimes written as a transformation:\n",
    "\n",
    "$$ \\underbrace{\\underbrace{p(\\theta|D)}_{\\text{posterior}}\\cdot \\underbrace{p(D)}_{\\text{evidence}}}_{\\text{this is what we want to compute}} = \\underbrace{\\underbrace{p(D|\\theta)}_{\\text{likelihood}}\\cdot \\underbrace{p(\\theta)}_{\\text{prior}}}_{\\text{this is available}}$$ \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- For a given data set $D$, the posterior probabilities of the parameters scale relatively against each other as\n",
    "\n",
    "$$\n",
    "p(\\theta|D) \\propto p(D|\\theta) p(\\theta)\n",
    "$$\n",
    "\n",
    "- $\\Rightarrow$ All that we can learn from the observed data is contained in the likelihood function $p(D|\\theta)$. This is called the **likelihood principle**."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### The Likelihood Function vs the Sampling Distribution\n",
    "\n",
    "- Consider a distribution $p(D|\\theta)$, where $D$ relates to variables that are observed (i.e., a \"data set\") and $\\theta$ are model parameters."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- In general, $p(D|\\theta)$ is just a function of the two variables $D$ and $\\theta$. We distinguish two interpretations of this function, depending on which variable is observed (or given by other means). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "-  The **sampling distribution** (a.k.a. the **data-generating** distribution) $$p(D|\\theta=\\theta_0)$$ (which is a function of $D$ only) describes a probability distribution for data $D$, assuming that it is generated by the given model with parameters fixed at $\\theta = \\theta_0$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- In a machine learning context, often the data is observed, and $\\theta$ is the free variable. In that case, for given observations $D=D_0$, the **likelihood function** (which is a function only of the model parameters $\\theta$) is defined as $$\\mathrm{L}(\\theta) \\triangleq p(D=D_0|\\theta)$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Note that $\\mathrm{L}(\\theta)$ is not a probability distribution for $\\theta$ since in general $\\sum_\\theta \\mathrm{L}(\\theta) \\neq 1$."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Code Example: Sampling Distribution and Likelihood Function for the Coin Toss\n",
    "\n",
    "Consider the following simple model for the outcome (head or tail) $y \\in \\{0,1\\}$ of a biased coin toss with parameter $\\theta \\in [0,1]$:\n",
    "\n",
    "$$\\begin{align*}\n",
    "p(y|\\theta) &\\triangleq \\theta^y (1-\\theta)^{1-y}\\\\\n",
    "\\end{align*}$$\n",
    "\n",
    "We can plot both the sampling distribution $p(y|\\theta=0.8)$ and the likelihood function $L(\\theta) = p(y=0|\\theta)$."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "slideshow": {
     "slide_type": "subslide"
    }
   },
   "outputs": [],
   "source": [
    "using Pkg; Pkg.activate(\"probprog/workspace\");Pkg.instantiate();\n",
    "IJulia.clear_output();"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAjwAAAD7CAYAAABufU6/AAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAPYQAAD2EBqD+naQAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdd1QU59fA8e/Si4oCioCAqLGClRhFiR0DBqOJib38LIktFlIUS4xGxaixRbFHo7EQS9REopLYMGAXldgrGMUCCohK23n/8LCvK6CAwFLu55w9h519ZubOluHO00alKIqCEEIIIUQxpqfrAIQQQggh8pskPEIIIYQo9iThEUIIIUSxJwmPEEIIIYo9SXiEEEIIUexJwiOEEEKIYk8SHiGEEEIUe5LwCCGEEKLYk4RHCCGEEMWeJDz56MiRI3Tu3BlHR0eMjY2xsbGhadOmfPHFF7oO7bX69etH5cqVtZZVrlyZfv366SSerKhUKr799lvN89WrV6NSqbhx40aOtjN9+nS2bduWo3Uy21fLli1xcXHJ0XZeJygoSOsYX1QYPxNRMNK/f8ePH8+yzI0bN1CpVKxevVqz7Ntvv0WlUvHgwYM8jePF30Fm5w+VSsXw4cPzZJ95Yf/+/ahUKvbv3//asoGBgdSpUwdTU1NUKhXh4eH5H2AWQkND+fbbb3n06FGG11q2bEnLli0LPqgiQhKefLJz507c3d2Jj49n5syZ7Nmzh/nz59OsWTMCAwN1HV6u/Pbbb0ycOFHXYbxShw4dCAsLw9bWNkfr5Sbhye2+ciooKIjJkydn+lpR+EyE7tja2hIWFkaHDh0KdL8TJ07kt99+K9B95pf79+/Tu3dvqlatyq5duwgLC6N69eo6iyc0NJTJkydnmvAEBAQQEBCgg6iKBgNdB1BczZw5E2dnZ3bv3o2Bwf+/zd26dWPmzJk6jCz3GjRooOsQXqt8+fKUL18+X/fx9OlTTExMCmRfr1MUPhOhO8bGxjRp0qTA91u1atUC32d+uXTpEikpKfTq1YsWLVroOpxXql27tq5DKNSkhiefxMTEYG1trZXspNPT037bAwMD8fT0xNbWFlNTU2rVqsXYsWNJTEzUKtevXz9KlSrFhQsXaN++Pebm5tja2jJjxgwADh8+TPPmzTE3N6d69er8/PPPWuunVz0HBwfzv//9D0tLS8zNzfHx8eHatWuvPaaXm0/Sq4Q3bNjA+PHjsbOzo0yZMrRt25aLFy9qrasoCtOnT8fJyQkTExPc3NwIDg7OdhVsfHw8gwYNwsrKilKlSvHee+9x6dKlDOUyq14/deoU77//PhUqVMDY2Bg7Ozs6dOjArVu3gOdV7YmJifz888+oVCpUKpUmpvTt7dmzh/79+1O+fHnMzMxISkp6ZfNZSEgITZo0wdTUFHt7eyZOnEhaWlqG9+7l6vSXmyD69evHokWLNHGmP9L3mVmTVmRkJL169dIcb61atfjhhx9Qq9UZ9jN79mzmzJmDs7MzpUqVomnTphw+fPi1n4coGjJr0srMhQsXqFKlCu+88w737t3TLP/rr79o06YNZcqUwczMjGbNmvH333+/dr+ZNWmlW7t2LbVq1cLMzIx69erxxx9/ZChz6NAh2rRpQ+nSpTEzM8Pd3Z2dO3dmKBcREcEHH3xAuXLlMDExoX79+hnOe+nH995772FmZoa1tTWDBw8mISEhW8fRvHlzALp27ap1bsjq3PXysef0t3bkyBF8fHywsrLCxMSEqlWrMmrUKOB5c+RXX30FgLOzs+Z8kH4eySym2NhYhg4dir29PUZGRlSpUoXx48eTlJSkVS69yTE7n09RJQlPPmnatClHjhxhxIgRHDlyhJSUlCzLXr58GW9vb1auXMmuXbsYNWoUv/76Kz4+PhnKpqSk8OGHH9KhQwe2b9+Ol5cXfn5+jBs3jr59+9K/f39+++03atSoQb9+/Thx4kSGbQwYMAA9PT3Wr1/PvHnzOHr0KC1btsy0ijQ7xo0bx82bN1mxYgXLli3j8uXL+Pj4aP2DHz9+POPHj+e9995j+/btDB48mIEDB2aatLxMURQ6derE2rVr+eKLL/jtt99o0qQJXl5er103MTGRdu3acffuXRYtWkRwcDDz5s3D0dFRc8ILCwvD1NQUb29vwsLCCAsLy1At3L9/fwwNDVm7di2bN2/G0NAwy31GR0fTrVs3evbsyfbt2+nSpQtTp05l5MiRr433ZRMnTqRLly6aONMfWTWj3b9/H3d3d/bs2cN3333Hjh07aNu2LV9++WWm/SdefE/WrVtHYmIi3t7exMXF5ThWUTQdOHAAd3d36taty759+6hQoQIAv/zyC56enpQpU4aff/6ZX3/9FUtLS9q3b5+tpCczO3fuZOHChUyZMoUtW7ZgaWlJ586dtS64Dhw4QOvWrYmLi2PlypVs2LCB0qVL4+Pjo9Ud4OLFi7i7u/Pvv/+yYMECtm7dSu3atenXr59WLfrdu3dp0aIFERERBAQEsHbtWh4/fpyt/kQTJ07UXHBMnz4903NDdmXnt7Z79248PDyIjIxkzpw5/Pnnn0yYMIG7d+8CMHDgQD7//HMAtm7dqjkfNGzYMNN9Pnv2jFatWrFmzRp8fX3ZuXMnvXr1YubMmXz44YcZymfn8ynSFJEvHjx4oDRv3lwBFEAxNDRU3N3dFX9/fyUhISHL9dRqtZKSkqIcOHBAAZTTp09rXuvbt68CKFu2bNEsS0lJUcqXL68AysmTJzXLY2JiFH19fcXX11ezbNWqVQqgdO7cWWuf//zzjwIoU6dO1dqXk5OTVjknJyelb9++muf79u1TAMXb21ur3K+//qoASlhYmKIoihIbG6sYGxsrXbt21SoXFhamAEqLFi2yfD8URVH+/PNPBVDmz5+vtXzatGkKoEyaNCnDMV6/fl1RFEU5fvy4Aijbtm175T7Mzc21ju3l7fXp0yfL19L3pSiK0qJFCwVQtm/frlV20KBBip6ennLz5k1FUf7/vdu3b59WuevXryuAsmrVKs2yYcOGKVn9VF/+TMaOHasAypEjR7TKDRkyRFGpVMrFixe19uPq6qqkpqZqyh09elQBlA0bNmS6P1F4pH//jh07lmWZzL5PkyZNUgDl/v37ytq1axUjIyNlxIgRSlpamqZMYmKiYmlpqfj4+GhtLy0tTalXr57SuHHjDHG8+DvI7PwBKDY2Nkp8fLxmWXR0tKKnp6f4+/trljVp0kSpUKGC1nkyNTVVcXFxUSpVqqSo1WpFURSlW7duirGxsRIZGam1Hy8vL8XMzEx59OiRoiiKMmbMGEWlUinh4eFa5dq1a5fpb/Bl6b/VTZs2aS1v0aJFpueul489J7+1qlWrKlWrVlWePn2aZTyzZs3K8H5nFdOSJUsUQPn111+1yn3//fcKoOzZs0ezLLufT1EmNTz5xMrKipCQEI4dO8aMGTP44IMPuHTpEn5+fri6umqNkLh27Ro9evSgYsWK6OvrY2hoqGkrPn/+vNZ2VSoV3t7emucGBgZUq1YNW1tbrf4clpaWVKhQgZs3b2aIrWfPnlrP3d3dcXJyYt++fbk61o4dO2o9r1u3LoBm34cPHyYpKYlPPvlEq1yTJk2yrPZ+UXpcL8fdo0eP165brVo1ypUrx5gxY1iyZAnnzp177TqZ+eijj7JdtnTp0hnekx49eqBWqzl48GCu9p9de/fupXbt2jRu3Fhreb9+/VAUhb1792ot79ChA/r6+prnL392oviaNm0a/fr1Y8aMGcyfP1+rqT00NJTY2Fj69u1Lamqq5qFWq3nvvfc4duxYhib37GjVqhWlS5fWPLexsdE6TyUmJnLkyBG6dOlCqVKlNOX09fXp3bs3t27d0jSX7927lzZt2uDg4KC1j379+vHkyRPCwsKA5+ePOnXqUK9ePa1y2Tl/5KXX/dYuXbrE1atXGTBgACYmJnmyz71792Jubq6pJU6X3gz+ck3d6z6fok46LeczNzc33NzcgOfNUWPGjGHu3LnMnDmTmTNn8vjxYzw8PDAxMWHq1KlUr14dMzMzoqKi+PDDD3n69KnW9szMzDL8GIyMjLC0tMywbyMjI549e5ZhecWKFTNdFhMTk6tjtLKy0npubGwMoIk9fbs2NjYZ1s1s2ctiYmIwMDDIsJ/MjuNlFhYWHDhwgGnTpjFu3DgePnyIra0tgwYNYsKECa9smnpRTkZiZXZM6bHm9j3OrpiYmEyTSDs7u0z3/7rPThRfv/zyC/b29nTr1i3Da+lNKC//o3xRbGws5ubmOdrny983eP6dS/++PXz4EEVRMv29vfwdjomJyXY5Z2fnDOWyc/7IS6/7rd2/fx+ASpUq5dk+Y2JiqFixIiqVSmt5hQoVMDAweO35ID3O4nI+kISnABkaGjJp0iTmzp1LREQE8DwDv337Nvv379caAZDb/jTZER0dnemyatWq5cv+0n9E6SfRl/f7uloeKysrUlNTiYmJ0fpBZnYcmXF1dWXjxo0oisKZM2dYvXo1U6ZMwdTUlLFjx2ZrGy+fMF4lq+OE/38v0pPWlzsOvuncKFZWVty5cyfD8tu3bwNgbW39RtsXxceuXbvo2rUrHh4e/P333zg5OWleS/+e/Pjjj1mO8srOxUpOlStXDj09vWx9h7P7XbeyssrynPcmTExMMu3rltvfcPqIz/TBFHnBysqKI0eOoCiK1jns3r17pKamlrjzgTRp5ZPMfojw/01U6Vch6V/C9Gw/3dKlS/MttnXr1mk9Dw0N5ebNm/k2YdU777yDsbFxhvmHDh8+nK2q0latWgEZ416/fn2O4lCpVNSrV4+5c+dStmxZTp48qXktL69iEhIS2LFjR4ZY9fT0ePfddwE0Sd6ZM2e0yr28XnpskL1alzZt2nDu3DmtYwNYs2YNKpVK814K4eTkREhICMbGxnh4eHD58mXNa82aNaNs2bKcO3dOU0v98sPIyCjPYzI3N+edd95h69atWt93tVrNL7/8QqVKlTRz4LRp00ZzwfiiNWvWYGZmpknUWrVqxb///svp06e1yuX0/PGyypUrc+nSJa2LlpiYGEJDQ3O1verVq1O1alV++umnDBdCL8rp+eDx48cZ5hhbs2aN5vWSRGp48kn79u2pVKkSPj4+1KxZE7VaTXh4OD/88AOlSpXSjNhxd3enXLlyDB48mEmTJmFoaMi6desy/Djz0vHjxxk4cCAff/wxUVFRjB8/Hnt7e4YOHZov+7O0tMTX1xd/f3/KlStH586duXXrFpMnT8bW1jbDMP2XeXp68u677/L111+TmJiIm5sb//zzD2vXrn3tvv/44w8CAgLo1KkTVapUQVEUtm7dyqNHj2jXrp2mnKurK/v37+f333/H1taW0qVLU6NGjVwdr5WVFUOGDCEyMpLq1asTFBTE8uXLGTJkCI6OjsDz6vS2bdtq3hMnJyf+/vtvtm7dmmF7rq6uAHz//fd4eXmhr69P3bp1M/2HM3r0aNasWUOHDh2YMmUKTk5O7Ny5k4CAAIYMGaLTCdNE/ti7d2+mUyO82NcvK7a2thw4cID27dvz7rvvEhwcjIuLC6VKleLHH3+kb9++xMbG0qVLFypUqMD9+/c5ffo09+/fZ/HixflwNODv70+7du1o1aoVX375JUZGRgQEBBAREcGGDRs0F4mTJk3ijz/+oFWrVnzzzTdYWlqybt06du7cycyZM7GwsABg1KhR/PTTT3To0IGpU6diY2PDunXruHDhwhvF2bt3b5YuXUqvXr0YNGgQMTExzJw5kzJlyuR6m4sWLcLHx4cmTZowevRoHB0diYyMZPfu3ZoLvvTzwfz58+nbty+GhobUqFFDq+9Nuj59+rBo0SL69u3LjRs3cHV15dChQ0yfPh1vb2/atm2b61iLJJ12mS7GAgMDlR49eihvvfWWUqpUKcXQ0FBxdHRUevfurZw7d06rbGhoqNK0aVPFzMxMKV++vDJw4EDl5MmTGUZX9O3bVzE3N8+wrxYtWih16tTJsNzJyUnp0KGD5nn6aIo9e/YovXv3VsqWLauYmpoq3t7eyuXLl7XWzckorZdHL2Q2MkStVitTp05VKlWqpBgZGSl169ZV/vjjD6VevXoZRo1l5tGjR0r//v2VsmXLKmZmZkq7du2UCxcuvHaU1oULF5Tu3bsrVatWVUxNTRULCwulcePGyurVq7W2Hx4erjRr1kwxMzPTGjn2qpEwWY3SqlOnjrJ//37Fzc1NMTY2VmxtbZVx48YpKSkpWuvfuXNH6dKli2JpaalYWFgovXr10owqe/G9S0pKUgYOHKiUL19eUalUWvt8+TNRFEW5efOm0qNHD8XKykoxNDRUatSoocyaNUtrFE76ZzRr1qwMx/XyeyoKp/TvX1aP69evv3aUVrpHjx4pzZo1UywtLbW+6wcOHFA6dOigWFpaKoaGhoq9vb3SoUMHrd98TkZpDRs2LMNxZPYdDgkJUVq3bq2Ym5srpqamSpMmTZTff/89w7pnz55VfHx8FAsLC8XIyEipV6+e1rGmO3funNKuXTvFxMREsbS0VAYMGKBs3779jUZpKYqi/Pzzz0qtWrUUExMTpXbt2kpgYGCWo7Sy+1sLCwtTvLy8FAsLC8XY2FipWrWqMnr0aK0yfn5+ip2dnaKnp6d1DJmNHIuJiVEGDx6s2NraKgYGBoqTk5Pi5+enPHv2LEMs2f18iiqVoihKQSRWQvdWr17N//73P44dO6bpSK1L169fp2bNmkyaNIlx48bpOhwhhBDFmDRpiQJx+vRpNmzYgLu7O2XKlOHixYua6t8BAwboOjwhhBDFnCQ8okCYm5tz/PhxVq5cyaNHj7CwsKBly5ZMmzYtX0Z7CCGEEC+SJi0hhBBCFHs6H5Z+8OBBfHx8sLOzQ6VSZRg+l5kDBw7QqFEjTExMqFKlCkuWLCmASIUQQghRVOk84UlMTKRevXosXLgwW+WvX7+Ot7c3Hh4enDp1inHjxjFixAi2bNmSz5EKIYQQoqgqVE1aKpWK3377jU6dOmVZZsyYMezYsUPrHlODBw/m9OnTmnunCCGEEEK8qMh1Wg4LC8PT01NrWfv27Vm5ciUpKSmZ3hspKSlJa+ZKtVpNbGwsVlZWObplgBAibyiKQkJCAnZ2dq+deLKwUKvV3L59m9KlS8t5QwgdeZNzR5FLeKKjozOM6rGxsSE1NZUHDx5kejM5f39/Jk+eXFAhCiGyKSoqKk9vlpifbt++neHO3EII3cjNuaPIJTyQ8UaO6a1yWV11+fn54evrq3keFxeHo6MjUVFRbzQNuBAid+Lj43FwcMh0OvzCKj1WOW8IoTtvcu4ocglPxYoVM9zl9t69exgYGGR6a3t4frO1l2/OCVCmTBk5cQmhQ0WpaSg9VjlvCKF7uTl3FI3G8xc0bdqU4OBgrWV79uzBzc0t0/47QojiR6azEELklM4TnsePHxMeHk54eDjwfNh5eHg4kZGRwPPmqD59+mjKDx48mJs3b+Lr68v58+f56aefWLlyJV9++aVO4hdCFDyZzkIIkVM6b9I6fvw4rVq10jxP72vTt29fVq9ezZ07dzTJD4CzszNBQUGMHj2aRYsWYWdnx4IFC/joo48KPHYhhG54eXnh5eWV7fJLlizB0dGRefPmAVCrVi2OHz/O7Nmz5dwhRCGQplbQ18vfJm6dJzwtW7bkVVMBrV69OsOyFi1acPLkyXyMSghRnOTFdBbx8fH5HqcQJdGZW48YHRjOD5/Up75D2Xzbj86btIQQIr+9bjqLzPj7+2NhYaF5yJB0IfKWoij8dOg6Hy0O5er9RL7/80K+7k8SHiFEiZCb6Szi4uI0j6ioqHyPUYiSIu5JCp+tPcGUP86RkqbQvo4NS3o3ytd96rxJSwgh8lteTmchhHgzpyIfMnz9Kf579BQjfT3Gedekr3vlfJ+mQhIeIUSx17RpU37//XetZTKdhRAFS61WWHnoOt/vukCqWsHJyoyF3RviWsmiQPYvCY8Qosh5/PgxV65c0TxPn87C0tISR0dH/Pz8+O+//1izZg3wfDqLhQsX4uvry6BBgwgLC2PlypVs2LBBV4cgRInyMDGZLzadZu+FewB0qGuL/4eulDEpuAsOSXiEEEWOTGchRNFx/EYsn284xZ24ZxgZ6PHN+7Xp+Y5jgc+0rlJeNSa8mIqPj8fCwoK4uDiZIl4IHSiKv8GiGLMQuqRWKyw5eJUf9lwiTa1QxdqcH3s0oI5d7puw3uR3KDU8QgghhMhTDx4n4fvraQ5eug/AB/XtmNbZlVLGuks7JOERQgghRJ45fC2GERtOcS8hCRNDPaZ0dOFjt0o6v1mwJDxCCCGEeGNpaoWFe68w/+9LqBWoVqEUi3o0pEbF0roODZCERwghhBBv6F7CM0YHhvPPlRgAPmpYie861cHMqPCkGYUnEiGEEEIUOf9cecDIjeE8eJyEqaE+Uzu58FGjSroOKwNJeIQQQgiRY6lpahb8fZkf911BUaBmxdIs7NGQahVK6Tq0TEnCI4QQQogciY57xoiNpzh6PRaA7o0dmORTBxNDfR1HljVJeIQQQgiRbfsv3sP319PEJiZjbqTP9A9d+aC+va7Dei1JeIQQ+U5RFA4cOEBISAg3btwgLi4OgF9++QUfHx8cHBx0HKEQ4nVS09T8EHyJxfuvAlDbtgwLezSgSvnC2YT1Mj1dByCEKL6ePn3K9OnTcXBwwMvLi507d/Lo0SP09Z9Xe/v7++Ps7Iy3tzeHDx/WcbRCiKzcfvSUbssOa5Kd3k2c2DrUvcgkOyAJjxAiH1WvXp2TJ0+yZMkS4uPjOXz4MFu2bGH58uUA/Pvvv1y9ehUPDw+6du2qWS6EKDz+Pn8X7wUhHL/5kNLGBgT0bMh3nVwKdX+dzMi9tOSeOELkm4iICFxcXDIsz+w3mJyczM2bN3nrrbcKOsxskfOGKGmSU9XM3HWBFYeuA1C3kgULuzfE0cpMZzHJvbSEEIVSZslOVoyMjAptsiNESRMV+4TPN5wiPOoRAP9rVpmxXjUxNihatTovkoRHCJHvwsLCCA4O5tq1azx58gRDQ0MAjh07Rps2bXQcnRDiRbv/jearTaeJf5ZKGRMDZn9cD886FXUd1huTPjxCiHwTExND69at8fDwYOvWrSQmJrJ9+3ZiYp5PP9+xY0fatm2reS6E0J2k1DS+3fEvn609QfyzVBo4liVopEexSHagkCQ8AQEBODs7Y2JiQqNGjQgJCXll+Xnz5lGjRg1MTU1xcHBg9OjRPHv2rICiFUJk12effYaBgQFRUVGEh4ezadMmjIyM+P777wG4fPkylpaWDB48WMeRClGy3YxJ5OMlYawOvQHAp+9W4dfPmlKpnO766+Q1nTdpBQYGMmrUKAICAmjWrBlLly7Fy8uLc+fO4ejomKH8unXrGDt2LD/99BPu7u5cunSJfv36ATB37twCjl4I8Sq7du3ixIkT2NraZvp6qVKlmDZtGg0aNCjgyIQQ6XaeucPYLWdISEqlnJkhP3xSj9Y1bXQdVp7TeQ3PnDlzGDBgAAMHDqRWrVrMmzcPBwcHFi9enGn5sLAwmjVrRo8ePahcuTKenp50796d48ePF3DkQojXsbKy4vLly68sc+PGDSwtLQsoIiFEumcpaUzYdpZh60+SkJSKm1M5gkZ6FMtkB3Sc8CQnJ3PixAk8PT21lnt6ehIaGprpOs2bN+fEiRMcPXoUgGvXrhEUFESHDh2y3E9SUhLx8fFaDyFE/vPz86NHjx6MHz+ef/75h7t376JSqTS/wV9++YW+ffsyYcIEHUcqRMly7f5jOgeE8svhSACGtqzKxk+bYGthquPI8o9Om7QePHhAWloaNjba2aSNjQ3R0dGZrtOtWzfu379P8+bNURSF1NRUhgwZwtixY7Pcj7+/P5MnT87T2IUQrzd48GAqVqzItGnT8Pf3R6VSoSgKLVu2BGDlypUsXryYDz74QLeBClGCbA//j3Fbz5KYnIaVuRFzutanRfXyug4r3+m8SQtApVJpPVcUJcOydPv372fatGkEBARw8uRJtm7dyh9//MF3332X5fb9/PyIi4vTPKKiovI0fiFE1jp16sSxY8eIjY3l2LFjHDx4kH379gGwb9++XCc7MthBiJx5mpzG2C1nGLkxnMTkNJpUsSRopEeJSHZAxzU81tbW6OvrZ6jNuXfvXoZan3QTJ06kd+/eDBw4EABXV1cSExP59NNPGT9+PHp6GXM4Y2NjjI2N8/4AhBDZVrZsWRo2bAjwxs3KMthBiJy5ci+BYetOcfFuAioVfN76LUa2eQt9vcwrF4ojnSY8RkZGNGrUiODgYDp37qxZHhwcnOVV35MnTzIkNfr6+iiKQgm8S4YQhdrBgwczXZ6YmAjAP//8g7m5eYbXK1eunGniku7FwQ7wvPZm9+7dLF68GH9//wzlXxzskL797t27a/oCClGcbT5xi4nbIniakoZ1KWMWdKuPezVrXYdV4HQ+LN3X15fevXvj5uZG06ZNWbZsGZGRkZp5Ofr06YO9vb3mJObj48OcOXNo0KAB77zzDleuXGHixIl07NhRcwdmIUTh0Ldv30yXq9Vq4Hkfn5cvYFQqFaNGjWLEiBGZrps+2OHlfnuvG+zwyy+/cPToURo3bqwZ7JBVfPB8sENSUpLmuQx2EEXNk+RUJm77ly0nbwHQrJoVc7vWp0JpEx1Hphs6T3i6du1KTEwMU6ZM4c6dO7i4uBAUFISTkxMAkZGRWifECRMmoFKpmDBhAv/99x/ly5fHx8eHadOm6eoQhBBZuH79eqbL028AePbs2RzfAFAGOwjxehejExi2/iRX7j1GTwWj21ZnaKtqJaoJ62Vyt3S567EQBe5NfoO3b9/G3t6e0NBQmjZtqlk+bdo01q5dy4ULFzKss3//frp168bUqVM1NcMjR45k0KBBTJw4MdP9ZFbD4+DgIOcNUagpikLgsSgm7fiXpFQ1NmWMmd+tAU2qWOk6tDwhd0sXQhQJKSkpREdHc+/evVxvQwY7CJG5x0mpTPjtLNvCbwPQonp55nxSD6tS8j2GQjIsXQhRfD1+/JilS5fSsmVLLCwsqFy5Mo0bNwbAxcWFQRNxkqQAACAASURBVIMGcezYsWxv78XBDi8KDg7G3d0903VksIMo7s7djqfjj4fYFn4bfT0VY96ryap+b0uy8wJJeIQQ+Wbu3LlUrlyZ5cuX07p1a7Zu3Up4eLjmVjBjxowhNTWVdu3a8d577732NhTpfH19WbFiBT/99BPnz59n9OjRGQY7+Pn5acr7+PiwePFiNm7cyPXr1wkODpbBDqJYUBSFtYdv0ingH649SMTWwoTAT5swpGVV9Epwf53MSJOWECLfhIaGsm/fPlxdXbWWp4946t27N8OGDWPJkiWsXLmSAwcO8NZbb712uzLYQQhIeJbC2K1n2XnmDgBtalZg9sf1KGdupOPICifptCydD4UocEXxN1gUYxbF19lbcQzfcJKbMU8w0FMx1qsmA5o7Z3mXguJCOi0LIYqEtLQ0Hjx4wOPHj3UdihBFkqIorA69wfSg86SkKdiXNWVhjwY0cCyn69AKPenDI4TId7/99hvNmjXDzMwMOzs7qlevDjyfLHDbtm06jk6IoiHuSQqfrT3B5N/PkZKm4FnbhqARHpLsZJMkPEKIfLV06VK6detG3bp1CQwM5NChQ+zatQuAOnXq0K1bN5YvX67jKIUo3E5FPsR7QQh7zt3FUF/Ftz61Wdq7ERZmhroOrciQPjzSFi9EvqpWrRp+fn4MGDBAs+zF3+DmzZuZNm0aV69e1WGUryfnDaELiqKwIuQ63++6QKpawdHSjEU9GuJayULXoemE9OERQhRa//33H82bN8/ydXd3d27fvl2AEQlRNDxMTObLTaf5+8LziTo7uNri/5ErZUykVic3pElLCJGv6tSpw7Jly7J8ffny5dSpU6cAIxKi8Dt+IxbvBSH8feEeRgZ6TO3kwsIeDSTZeQO5quFRFIUDBw4QEhLCjRs3ePLkCeXLl6dBgwa0bdsWBweHvI5TCFFE/fDDD3To0IFdu3bh6emJjY2N5h5VTZo0ISoqiqCgIB1HKUThoFYrLDl4lR/2XCJNreBsbc7CHg2oY1cym7DyUo4SnqdPnzJ37lwCAgKIiYmhXr162NvbY2pqypUrV9i2bRuDBg3C09OTb775hiZNmuRX3EKIIqJFixZERESwePFiDh8+THR0NGq1GoD27dszcuRIKleurNsghSgEYh4n4fvraQ5cug/AB/XtmNbZlVLG0vskL+So07KDgwPvvPMO/fr1o3379hgaZqxau3nzJuvXr2fJkiVMmDCBQYMG5WnAeUE6HwqhW0XxN1gUYxZFx5FrMYzYeIq78UkYG+gx5YM6fOLmUOwnEsypN/kd5ijhiYiIwMXFJVtlk5OTuXnzZramiS9ocuISQreK4m+wKMYsCr80tcKifVeY99cl1ApULW/Oop4NqVlRvmOZKbBRWtlNduD5HY0LY7IjhChc+vbtS1RUFHv37tV1KEIUqHsJzxgdGM4/V2IA+LChPd994IK5NGHlizx5Vx8+fMiVK1ewtbWlUqVKebFJIUQJYW9vr3WjTyFKgn+uPGDkxnAePE7C1FCf7zq50KWR/P/MTzk+y4wbN44nT54AkJKSwqeffoq1tTXvvPMOTk5OfPjhhzx79izPAxVCFE/Tp09n1apVug5DiAKRplaYs+civVYe4cHjJGrYlOb3z5tJslMAcpzwfP/995ob/82aNYtt27axadMmbt26xfbt2zl69CizZs3K80CFEEKIouxu/DN6LD/Mgr1XUBTo9rYD24Y1o1qF0roOrUTIccLzYh/nTZs2MWPGDD788EPs7Ox4//33mTNnDhs2bMjTIIUQxVdUVBT9+/fXdRhC5KsDl+7jNT+EI9djMTfSZ363+sz4qC6mRvq6Dq3EyFXDefowuaioKBo3bqz1WuPGjbl58+abRyaEKBFiY2P5+eefdR2GEPkiNU3N97su0Peno8QmJlPLtgy/f96cD+rb6zq0EidXnZaXL19OqVKlMDY25uHDh1qvxcXFYWxsnCfBCSGKvh07dmRYlt4PMCgoiOjo6IIOSYgCcfvRU0ZsOMXxm8//T/Zq4siEDrUxMZRaHV3IccLj6OjI8uXLgedDz0+ePImHh4fm9X379lGjRo28i1AIUaR16tQJlUpFZlN+9ejRA0AmVxPFzt/n7/LFptM8epJCaWMD/D9y5f26droOq0TLcZPWjRs3uH79uuYxcuRIrdebNGlCQEBAjrYZEBCAs7MzJiYmNGrUiJCQkFeWf/ToEcOGDcPW1hYTExNq1aol9+IRopCytbVly5YtqNVqzePRo0fA89/yyZMndRyhEHknJU3N9KDzDPj5OI+epOBqb8EfI5pLslMI5PnsRjm9f1ZgYCCjRo0iICCAZs2asXTpUry8vDh37hyOjo4ZyicnJ9OuXTsqVKjA5s2bqVSpElFRUZQuLb3chSiMGjVqxMmTJ+nUqVOmr2dV+yNEURMV+4QRG09xKvJ5Qt/PvTJ+3jUxNpAmrMIgz2f7unPnDpGRkdkuP2fOHAYMGMDAgQOpVasW8+bNw8HBgcWLF2da/qeffiI2NpZt27bRrFkznJycaN68OfXq1curQxBC5KGvvvoKd3f3LF+vVq0a+/bty/F2pWZYFCa7/42mw4IQTkU+ooyJAUt6NeLbjnUk2SlE8ryGp3Xr1ly6dIm0tLTXlk1OTubEiROMHTtWa7mnpyehoaGZrrNjxw6aNm3KsGHD2L59O+XLl6dHjx6MGTMGff3Mv1hJSUkkJSVpnsfHx+fgiIQQb+LFPn6ZMTc3p0WLFjnaptQMi8IiKTUN/6ALrA69AUA9h7Is7N4AB0sz3QYmMsjzhGfNmjWaERiv8+DBA9LS0rCxsdFabmNjk+XIjWvXrrF371569uxJUFAQly9fZtiwYaSmpvLNN99kuo6/vz+TJ0/O2YEIIQqtF2uGAebNm8fu3btZvHgx/v7+Gcqn1wyHhoZiaGgIgJOT0yv3IRdK4nUiY54wbP1Jzv4XB8AgD2e+al8TIwO5VUphlOefyttvv53jq7WXR2goipLlqA21Wk2FChVYtmwZjRo1olu3bowfPz7LJjAAPz8/4uLiNI+oqKgcxSeEyBszZszQdFgGtP7OrvSaYU9PT63l2a0ZtrGxwcXFhenTp7+yJtrf3x8LCwvNw8HBIcexiuIr6OwdOiwI4ex/cZQ1M2RlXzfGd6gtyU4hlusansePH3PixAmio6NRqVTY2NjQqFEjSpUqle1tWFtbo6+vn6E25969exlqfdLZ2tpiaGio1XxVq1YtoqOjSU5OxsjIKMM6xsbGMjeQEIXA9OnT+eSTT7C2tgae33g4syaoVymommE/Pz98fX01z+Pj4yXpETxLSWPazvOsPfx8gl03p3Is6N4Au7KmOo5MvE6OE57U1FS++OILli9fzrNnzzAyMkJRFFJSUjAxMeHTTz9l1qxZmmrjVzEyMqJRo0YEBwfTuXNnzfLg4GA++OCDTNdp1qwZ69evR61Wa+6wfOnSJWxtbTNNdoQQhUdejsbKbc2wvr4+jRo14vbt28yaNSvLhEculMTLrt1/zPD1pzh353nz5uAWVfnCszqG+lKrUxTk+FP64osv2LJlC6tWrSI2NpZnz56RlJREbGwsq1atYuvWrXz11VfZ3p6vry8rVqzgp59+4vz584wePZrIyEgGDx4MQJ8+ffDz89OUHzJkCDExMYwcOZJLly6xc+dOpk+fzrBhw3J6KEKIIii3NcPVq1fPsmZYiNfZHv4fPj8e4tydeCzNjVj9v7cZ61VTkp0iJMc1POvXrycwMJDWrVtrLS9btixdu3bF2tqabt26MW/evGxtr2vXrsTExDBlyhTu3LmDi4sLQUFBmg6FkZGRmpocAAcHB/bs2cPo0aOpW7cu9vb2jBw5kjFjxuT0UIQQRZDUDIuC9DQ5jcm//8vGY8/7fjZ2tmRBtwZUtDDRcWQip3Kc8Dx9+lTT/p4ZKysrnj59mqNtDh06lKFDh2b62v79+zMsa9q0KYcPH87RPoQQxYevry+9e/fGzc2Npk2bsmzZsgw1w/b29poRW0OGDOHHH39k5MiRfP7551y+fJnp06czYsQIXR6GKOSu3HvMsHUnuXg3AZUKhreqxsg2b2EgtTpFUo4TnlatWuHr68u6desyVB/fvXuXr7/+OkPtjxBC5CWpGRb5bfOJW0zcFsHTlDSsSxkzr2t9mr+V9cW+KPxUSg57EUZFReHt7c2FCxdwcXHBxsYGlUpFdHQ0ERER1K5dm507d1KpUqX8ivmNxcfHY2FhQVxcHGXKlNF1OEKUGKVLl+b06dNYW1tjYWFBeHh4kZklXc4bJcOT5FS+2f4vm0/cAsC9qhXzutWnQmlpwioM3uR3mOMaHgcHB06fPs3u3bs5fPiwpuNg48aN8ff3x9PTU+vKSgghhCgKLkYnMGz9Sa7ce4yeCka1rc6wVtXQ18t89J8oWnI1D4+enh5eXl54eXnldTxCiGLszz//xN7eXjODsZ2d3EFa6J6iKPx6PIpvtv9LUqqaCqWNmd+tAU2rWuk6NJGH8vzWEkIIkZXmzZsDaBIemedG6NrjpFQm/HaWbeG3AXi3ennmfFIP61Ly3Sxu3qjtqUyZMly7di3D30J30tQKYVdj2B7+H2FXY0hT591Eb6JgFNfPsF+/fhw8eFDXYQihce52PB1/PMS28Nvo66n4+r0arO73tiQ7xdQb1fC82N85L2dQFbmzK+IOk38/x524Z5plthYmTPKpzXsutjqMTGRXcf4MExIS8PT0xMHBge7du+s6HFGCKYrCuiORTPnjHMmpamwtTPixewPcKlvqOjSRj6R3cTGxK+IOQ345qfWPEiA67hlDfjnJrog7OopMZFdx/wy3bNnCf//9x/Dhw9m2bRsAH330EZs3byYlJUXH0YmSIv5ZCsM3nGLCtgiSU9W0rlmBoBEekuyUAJLwFANpaoXJv58jszq29GWTfz9XbJpGiqOS8hlaWVkxcuRIDh06BECVKlXo3bs3dnZ2jB49msuXL+s4QlGcnb0Vh8+Ph9h55g4GeirGe9diRR83ypnLbNslgSQ8xcDR67EZagVepAB34p5x9HpswQUlcqSkfYbp01ns3bsXfX19vL29+ffff6lduzZz587VcXSiuFEUhdX/XOejxaHcjHmCfVlTfh3clEHvVkFPhpyXGDJKqxi4l5D1P8rclBMFryR8hikpKezYsYNVq1axZ88e4PltZQYOHEjp0qUB2LhxI0OGDGH06NG6DFUUI3FPUxiz+Qy7/n2eZLerbcPsLvWwMDPUcWSioEnCUwxkdwZQmSm08CoJn6GtrS1qtZru3buzd+9ePDw8GDBggCbZAWjfvj1ly5bVYZSiODkV+ZDPN5zi1sOnGOqrGOddi37ulVGppFanJJKEpxho7GyJrYUJ0XHPMu0DogIqWpjQ2Fk65RVWJeEznDt3Lh9//DEmJibEx8dnWqZcuXJcv369gCMTxY2iKKw8dJ0Zf14gVa3gaGnGwh4NqFtJkumS7I368Hh4eGBqaprhb1Gw9PVUTPKpDTz/x/ii9OeTfGrL9OiFWEn4DHv37o2JSdGtoRJFw6MnyQxac5ypO8+Tqlbwdq3IHyOaS7Ij3izhCQoKwtbWNsPfouC952LL4l4NqVBGe8KsihYmLO7VsMjP4VISFMfPcPDgwURFRWWrbGBgIOvWrcvniERxduJmLN7zQ/jr/D2MDPT4rpMLi3o0pIyJ9NcR0qRVrLznYkuzata4fvu8Q+jq/72Nx1vli3StQElT3D7D8uXL4+Ligru7Ox07dsTNzQ07OzvNvDtBQUGcPHmSjRs3Ym9vz7Jly3QcsSiK1GqFpQevMXvPRdLUCs7W5vzYvQEu9ha6Dk0UIjlOeGJjY1m6dCnBwcFcu3aNJ0+eYGZmRpUqVWjXrh2fffYZlpZFt59BUffiP8bGzpZF9h9lSVacPsPvvvuOzz//nJUrV7JkyRIiIiK0Xh88eDBt27ZlxYoVeHp66ihKUZTFPE7C99fTHLh0H4CO9eyY/qErpYzlel5oy1GT1qlTp6hZsybr1q2jbt26fPnllyQkJDBgwABat27Nrl27qFWrFuHh4fkVrxCiiKlQoQJ+fn6cPn2amJgYTp48qRmWfvPmTTZv3izJjsiVI9di8F4QwoFL9zE20MP/Q1fmd6svyY7IVI6+FUOGDKFv377MmjVLs8zPz4+ePXtSpUoVJkyYgL+/P5999hlHjhzJ82CFEEVb2bJlKVu2rGaUlgwPFrmRplYI2HeFuX9dQq1A1fLmLOzRkFq2ZXQdmijEclTDc+bMGYYPH/7KMj179uTs2bNvFJQQonh6+PAhs2fPZtiwYQAsWLCA2NjiMXu0KBj3E5Lo+9NRfgh+nux82NCeHcObS7IjXitHCU+NGjXYuHHjK8v8+eef1KhR442CEkIUPwcOHMDZ2ZkFCxbw6NEjAJYtW4azszMHDhzQcXSiKAi98gCv+SEcuvIAU0N9Zn9cjzmf1MdcmrBENuQo4VmwYAHTp0/H3d2dqVOnEhgYiFqtZs+ePaxYsYI+ffowbtw4Fi5cmF/xCiGKqGHDhvHJJ59w/fp1zfDz06dP061bN02NT04FBATg7OyMiYkJjRo1IiQkJFvrbdy4EZVKRadOnXK1X1Gw0tQKc4Iv0XPlER48TqK6TSl2DG9Gl0aVdB2aKEJylBZ7eHgQERHB/Pnz2bx5M9euXSMpKYmvv/5aM0orPDwcBweH/IpXCFFEXb16lS1btqCvr69Zpq+vj6+vL2vWrMnx9gIDAxk1ahQBAQE0a9aMpUuX4uXlxblz53B0dMxyvZs3b/Lll1/i4eGRq+MQBetu/DNGbjzF4WvPmz67ujnwbcc6mBrpv2ZNIbTleOJBBwcHZs+eTXh4OPHx8aSmphIfH094eDizZs3KVbIjV2lCFH8NGzbk/PnzGZafP3+e+vXr53h7c+bMYcCAAQwcOJBatWoxb948HBwcWLx4cZbrpKWl0bNnTyZPnkyVKlVyvE9RsA5euo/3/BAOX4vFzEifeV3r832XupLsiFzRecOnXKUJUTKMGDGCkSNHcuXKFVxdXYHnzeQrV65kxowZnDlzRlO2bt26r9xWcnIyJ06cYOzYsVrLPT09CQ0NzXK9KVOmUL58eQYMGPDaC6ukpCSSkpI0z7O6/5fIe6lpaub+dYmA/VdRFKhlW4ZFPRpQpXwpXYcmirAcJTxTpkzJ1U5atmzJu+++m+lrL16lAcybN4/du3ezePFi/P39M13nxau0kJAQTQdIIUTh1b17dwC+/vprzbJvvvlG85pKpUJRFFQqFWlpaa/c1oMHD0hLS8PGxkZruY2NDdHR0Zmu888//7By5cpszxPm7+/P5MmTs1VW5J07cU8ZseEUx248BKDnO45MfL82JoZSqyPeTI4SntzexTir6uqCuEoDuVITojB48fyRkJCAq6srZ86coXTp0rne5svz+KQnTC9LSEigV69eLF++HGtr62xt28/PD19fX83z+Ph46Z+Yz/ZeuMsXv57m4ZMUShkbMOMjV96va6frsEQxkaOEZ9WqVXm684K4SgO5UhOiMHByctL8nX7R4ejoSJkyOZ8/xdraGn19/QzniXv37mU4n8DzDtM3btzAx8dHs0ytVgNgYGDAxYsXqVq1qtY6xsbGGBtr38hV5I+UNDWzdl9k2cFrALjYl2Fh94ZUtjbXcWSiOMn13dITExPzLIj8vEqD51dqcXFxmkd2794shCicjIyMaNSoEcHBwVrLg4ODcXd3z1C+Zs2anD17lvDwcM2jY8eOtGrVSkaW6tith0/4eEmYJtnp516ZLUPcJdkReS7XnZZtbGz45JNP6N+/P82bN8/VNgriKg3kSk2I4sjX15fevXvj5uZG06ZNWbZsGZGRkQwePBiAPn36YG9vj7+/PyYmJri4uGitX7ZsWYAMy0XB2fNvNF9uOk38s1TKmBgws0s93nOpqOuwRDGV64Rnw4YNrF69mjZt2uDk5ET//v3p06cPdnbZb2998Sqtc+fOmuXBwcF88MEHGcqnX6W9aMKECSQkJDB//ny5ShOiBOnatSsxMTFMmTKFO3fu4OLiQlBQkKbpLDIyEj29XFdii3yUnKrG/8/zrPrnBgD1HMqysHsDHCzNdBuYKNZynfD4+Pjg4+NDTEwMa9asYfXq1UycOJH27dvTv39/OnbsiIHB6zcvV2lCiNwaOnQoQ4cOzfS1/fv3v3Ld1atX531A4rUiY54wfMNJztyKA2Bgc2e+fq8mRgaSnIr89cbz8FhZWTF69GhGjx7Njz/+yFdffUVQUBDW1tYMHjyYsWPHYmaWddYuV2lCCFEyBJ29w5jNZ0hISqWsmSGzu9Sjbe2M3ReEyA9vnPBER0ezZs0aVq1aRWRkJF26dGHAgAHcvn2bGTNmcPjwYfbs2fPKbchVmhAlQ4cOHVixYgXm5s87pEZHR+dqlJYoWp6lpDFt53nWHr4JQCOncvzYvQF2ZU11HJkoSXKd8GzdupVVq1axe/duateuzbBhw+jVq5emiQmez7/ToEGDPAlUCFH0HTx4kKdPn2oSnqdPn+o4IpHfrj9IZPj6k/x7+/lUBINbVOULz+oY6kvNvShYuU54/ve//9GtWzf++ecf3n777UzLVKlShfHjx+c6OCGEEEXXjtO38dtyhsTkNCzNjZjzST1a1qig67BECZXrhOfOnTuv7JsDYGpqyqRJk3K7CyGEEEXQs5Q0Jv9+jg1HIwFo7GzJgm4NqGhhouPIREmWo4QnMTFRUxX9umTn5fJCCCGKvyv3HjN8/UkuRCegUsHwVtUY2eYtDKQJS+hYjr6B1apVY/r06dy+fTvLMoqiEBwcjJeXFwsWLHjjAIUQQhQNW0/eouPCQ1yITsC6lDFr+7/DF541JNkRhUKOanj279/PhAkTmDx5MvXr18fNzQ07OztMTEx4+PAh586dIywsDENDQ/z8/Pj000/zK24hhBCFxJPkVCZt/5dNJ24B4F7Vinnd6lOhtDRhicIjRwlPjRo12LRpE7du3WLTpk0cPHiQ0NBQnj59irW1NQ0aNGD58uV4e3vL3DlCiAwyu0eeKNou3U1g2LqTXL73GD0VjGxTneGtq6GvJ5+1KFxy1Wm5UqVKmskG4XkzFsjJTAjxaunnClH0KYrCpuO3+GZHBM9S1FQobcz8bg1oWtVK16EJkak3qoZZuXIlLi4umJiYaG77sGLFiryKTQhRzCQkJFClShXNc2dnZx1GI3IrMSkV319P8/WWMzxLUePxljVBIz0k2RGFWq6HpU+cOJG5c+fy+eef07RpUwDCwsIYPXo0N27cYOrUqXkWpBBCiMLh3O14hq8/ybUHiejrqfBtV50hLaqiJ01YopDLdcKzePFili9fTvfu3TXLOnbsSN26dfn8888l4RFCiGJEURTWH41k8u/nSE5VY2thwoLuDXi7sqWuQxMiW3Kd8KSlpeHm5pZheaNGjUhNTX2joIQQQhQeCc9SGLv1LDvP3AGgdc0KzP64HpbmRjqOTIjsy3Ufnl69erF48eIMy5ctW0bPnj3fKCghhBCFQ8R/cbz/4yF2nrmDgZ6Kcd41WdHHTZIdUeS80d3SV65cyZ49e2jSpAkAhw8fJioqij59+uDr66spN2fOnDeLUgghRIFSFIU1YTeZtvM8yWlq7MuasqB7Axo5ldN1aELkSq4TnoiICBo2bAjA1atXAShfvjzly5cnIiJCU06GqgshRNES9zSFsVvO8GdENADtatswu0s9LMwMdRyZELmX64Rn3759eRmHEKIYu3jxIhs2bCAkJIQbN26QmJgIwODBg3n//ff56KOPMDY21nGUAiA86hHD15/k1sOnGOqr8POqxf+aVZaLV1HkyXTIQoh8c+rUKdq1a0e9evU4ePAgb7/9NqNGjWL8+PHA82aT8ePHY2dnx/fff09SUpKOIy65FEVhRcg1Pl4Syq2HT3GwNGXzYHf6N3eWZEcUC2/Uh0cIIV6lU6dOfPXVVwQGBmJp+f/Dl+Pj4xkxYgRLly6lTJkyhIWFMXfuXH744QfGjRunw4hLpkdPkvly0xn+On8XAG/Xisz4qC5lTKQJSxQfUsMjhMg3ly9fZvjw4VrJTmaaNm3Kr7/+ypdffpmj7QcEBODs7IyJiQmNGjUiJCQky7LLly/Hw8ODcuXKUa5cOdq2bcvRo0dztL/i6MTNh3RYcIi/zt/FSF+P7z6ow6IeDSXZEcWOJDxCiHxjZJSzocs5KR8YGKhpHjt16hQeHh54eXkRGRmZafn9+/fTvXt39u3bR1hYGI6Ojnh6evLff//lKMbiQq1WWHLgKp8sDeO/R0+pbGXG1qHu9G4q/XVE8SQJjxAiX8XGxuLv70/r1q2pXLkyFSpUwNXVFYAffviB2NjYXG13zpw5DBgwgIEDB1KrVi3mzZuHg4NDpvODAaxbt46hQ4dSv359atasyfLly1Gr1fz999+5PraiKjYxmf4/H2PGnxdIUyv41LPj98+b42JvoevQhMg3kvAIIfLNqVOnqFmzJuvWraNu3bp8+eWXJCQk0Lt3bwD+/vtvatWqRXh4eI62m5yczIkTJ/D09NRa7unpSWhoaLa28eTJE1JSUrJsbktKSiI+Pl7rURwcvR6L9/wQ9l+8j7GBHv4furKgW31KSxOWKOYKRcIj7fBCFE9Dhgyhb9++REREMG/ePIYPH46BgQEff/wxAEFBQYwaNYrPPvssR9t98OABaWlp2NjYaC23sbEhOjo6W9sYO3Ys9vb2tG3bNtPX/f39sbCw0DwcHBxyFGNhk6ZW+PHvy3RbFkZ0/DOqljdn+/BmdG/sKE1YokTQecIj7fBCFF9nzpxh+PDhryzTs2dPzp49m6vtv/yPWlGUbP3znjlzJhs2bGDr1q2YmJhkWsbPz4+4uDjNIyoqKlcxFgb3E5Lo+9NRfgi+hFqBDxvas2N4c2pW7F3kuAAAF+xJREFULKPr0IQoMDpPeKQdXojiq0aNGmzcuPGVZf78809q1KiRo+1aW1ujr6+foTbn3r17GWp9XjZ79mymT5/Onj17qFu3bpbljI2NKVOmjNajKAq98gDvBSEcuvIAU0N9ZnWpy5xP6mNuLLOSiJJFp9/49Hb4sWPHai3Py3Z4eN4W/+KEZsWlLV6Iwm7BggW8//77bN++HW9vb9566y3UajV79+4F4LPPPmPPnj3s2LEjR9s1MjKiUaNGBAcH07lzZ83y4OBgPvjggyzXmzVrFlOnTmX37t24ubnl7qCKiDS1woK/L7Ng72UUBarblGJRj4a8ZVNa16EJoRM6TXgKoh0enrfFT548+Y1iFULknIeHBxEREcyfP5/Nmzdz7do1kpKSmDRpEvD8/nvh4eG56h/j6+tL7969cXNzo2nTpixbtozIyEgGDx4MQJ8+fbC3t8ff3x943ow1ceJE1q9fT+XKlTXnmFKlSlGqVKk8OuLC4V78M0ZsPMXha89HwH3iVonJHV0wNdLXcWRC6E6hqNN803b4/fv3Z9kOD8/b4l+8e3t8fHyR74AoRFHh4ODA7NmztZbFx8djYWHB1KlTc91U1LVrV2JiYpgyZQp37tzBxcWFoKAgnJycAIiMjERP7/9b7QMCAkhOTqZLly5a25k0aRLffvttrmIojEIu32d0YDgPHidjZqTP9M6udGpgr+uwhNA5nSY8edEO/9dff72yHR6et8XLjQmFKPyye7GTbujQoQwdOjTT1/bv36/1/MaNG28QWeGXmqZm7l+XCNh/FUWBWrZlWNSjAVXKF6/aKyFyS6edll9sh39RcHAw7u7uWa43a9YsvvvuO3bt2lXs2+GFKMpq1arF+vXrSU5OfmW5y5cvM2TIEL7//vsCiqx4uRP3lB7Lj7Bo3/Nkp8c7jvw21F2SHSFeoPMmLWmHF6L4WrRoEWPGjGHYsGF4enri5uaGnZ0darUagHHjxnH06FHOnTvH8OHDs6ytEVnbd+Eevr+G8/BJCqWMDfD/0BWfena6DkuIQkfnCY+0wwtRfLVu3Zpjx44RGhpKYGAg69ev58aNGzx9+hSAq1ev0qdPH3r16kXZsmV1HG3RkpKmZvbuiyw9eA0AF/syLOzekMrW5jqOTIjCSecJD0g7vBDFnbu7u1YzdXqn5cDAwCI7v40u3Xr4hM83nOJU5CMA+rlXxs+7JsYGMgpLiKzofOJBIUTJFhUVRf/+/XUdRpGx599oOiw4xKnIR5Q2MWBJr4Z827GOJDtCvIYkPEIInYqNjeXnn3/WdRiFXnKqmim/n+PTtSeIe5pCvUoWBI3w4D0XW12HJkSRUCiatIQQQmQtKvYJw9ef5PStOAAGNnfm6/dqYmQg16xCZJckPEIIUYjtirjDV5vPkPAsFQtTQ2Z/XI92tV89T5kQIiNJeIQQohB6lpKGf9B5fg67CUBDx7L82KMh9mVNdRyZEEWTJDz/1969R0Vd530Af49clWBCUbmoiMaCLOsFUMEbbHJzqxNHT2KxSJ70yefUE2rtHnh82tB9HoLW1PJ4ObmKuZFaIVrhdsCzoBZkptSCYLlJoSUprHKR5Obn+cN1zo7DZQZn5jfz6/06Z47yne/M7/Phy/fLh/nN7ztEZFELFy40aOvq6gIApKSk4MaNG9YOyebVNd7As2+fwdkfbn/Q8dPRE/BCfBCcHHgKi2iwWPAQkUVptVqDtjsFj1arhZeXF5YuXWrtsGzWB1/+gMyDVWjr6MZwN2e8ungKfh00SumwiOweCx4isqi8vDyDtpaWFuTn52Pbtm3ch+dfbnb1YP2HNXj7ZD0AYMb44Xjt8anw0fIUFpE5sOAhIlLYN1fb8Ez+GZxraIVGAzz76weQPj8QjjyFRWQ2LHiIiBRUWHkJawur0d7ZA6/7nLEpeSrmBo5UOiwi1WHBQ0SkgJ86e/CHw9V49/QlAEDUhBF4bclUjPJwVTgyInViwUNEZGVf/9iKZ/LP4PyVNmg0QPr8QPzXg4FwGKJROjQi1WLBQ0RkJSKCd09fwh8OV+Nm1y2MdHfBa0umYtZEL6VDI1I9FjxERFZwo6MbLx6qxsHK7wEAcwO9sHHxVIx0d1E4MqKfBxY8REQWVnu5Bc+8fQYXrt7AEA3wfHwQ/jN6IobwFBaR1fCaRyKyW9u2bUNAQABcXV0RHh6OEydO9Nu/oKAAISEhcHFxQUhICAoLCy0an4jg7ZP1SNr6CS5cvQFvD1fs/48oPPPrB1jsEFkZCx4isksHDhzAqlWrsHbtWlRWVmLu3LlYsGAB6uvre+1fUVGB5ORkpKam4ssvv0RqaioWL16MkydPWiS+1ptdeG7/F/jvwip0dN9CTNBIHEmfixkBwy1yPCLqHwseIrJLGzduxFNPPYXly5dj0qRJ2Lx5M8aOHYvt27f32n/z5s2Ii4tDZmYmgoODkZmZifnz52Pz5s1mj636+2Y8suVjfPDlD3AcokHmgmDsTpuO4W7OZj8WERmHBQ8R2Z3Ozk6cPn0a8fHxeu3x8fEoLy/v9TEVFRUG/RMSEvrs39HRgZaWFr2bMd769Dss3FaOb5va4Xf/UBx4OgpP8/06RIpjwUNEdqexsRE9PT0YPXq0Xvvo0aPR0NDQ62MaGhpM6v/yyy9Dq9XqbmPHjjUqtp5bgs6eW4idNBpFz81BuL+nUY8jIsviVVpEZLc0Gv1XTUTEoG2w/TMzM7FmzRrd1y0tLUYVPUuj/OF7/1DEThrVbyxEZF0seIjI7nh5ecHBwcHg1ZkrV64YvIpzh7e3t0n9XVxc4OJi+h45Go0GcSG9PycRKYentIjI7jg7OyM8PBwlJSV67SUlJZg1a1avj4mKijLoX1xc3Gd/IlIXmyh4bH0vDSKyPWvWrMGf//xn7N69G7W1tVi9ejXq6+uxcuVKAMDSpUuRmZmp65+eno7i4mLk5ubi3LlzyM3NxdGjR7Fq1SqlUiAiK1K84LH1vTSIyDYlJydj8+bNWL9+PaZOnYrjx4/jyJEj8Pf3BwDU19fj8uXLuv6zZs3C/v37kZeXh8mTJ2PPnj04cOAAZs6cqVQKRGRFGhERJQOYOXMmwsLC9PbOmDRpEpKSkvDyyy8b9E9OTkZLSwv++te/6toSExPh6emJffv2GXXMlpYWaLVaNDc3w8PD496TsCE3OroQtvYDAMDp/4nFMGe+TcvetHd2I/x/jwIAzvzfI3BzcVI4IvOzxzlojzETqc29zENFfxve2UsjIyNDr32gvTRWr16t15aQkNDv5mEdHR3o6OjQfd3c3AwARu+rYU/arrfgrcO3v5+1hxUOhgbtrX/92/zcDPTcr75frnfmnsJ/b5nkTqxqXDeI7MW9rB2KFjzW2EsDuL2fxrp16wzajd1Xg0gx/ur+GW1qaoJWq1U6DKO0trYC4LpBZAtaW1tNXjts4nyHJffSAAz307h+/Tr8/f1RX19vN4utKe7sF3Lx4kXVvvSu9hzVnl9zczPGjRuH4cPt53OlfH19cfHiRbi7uw+4v46axo+52CY15QIYn4+IoLW1Fb6+viYfQ9GCxxp7aQB976eh1WpV8YPSFw8PD1XnB6g/R7XnN2SI4tdNGG3IkCEYM2aMSY9R0/gxF9ukplwA4/IZ7AsViq423EuDiIiIrEHxU1pr1qxBamoqIiIiEBUVhTfeeMNgLw0/Pz/dFVvp6emYN28ecnNz8eijj+Lw4cM4evQoPv74YyXTICIiIhvmkJWVlaVkAKGhoRgxYgSys7OxYcMG/PTTT/jLX/6CKVOmAABee+01ODo6IikpCcDtNwyGhIRg48aNyM7ORn19PbZv3464uDiTjuvg4ICYmBg4Oipe81mE2vMD1J8j87NvasqPudgmNeUCWD4fxffhISIiIrI0+3nHIBEREdEgseAhIiIi1WPBQ0RERKrHgoeIiIhU72dR8Fy7dg2pqanQarXQarVITU3F9evX+31MTEwMNBqN3m3JkiVWinhg27ZtQ0BAAFxdXREeHo4TJ07027+goAAhISFwcXFBSEgICgsLrRTp4JiS3549ewzGSqPR4ObNm1aM2HjHjx/HI488Al9fX2g0Ghw6dGjAxxw7dgzh4eFwdXXFhAkTsGPHDitEOnim5lhWVtbrGJ47d85KEffP3PNNRJCVlQVfX18MHToUMTExOHv2rCVT0DEll507d2Lu3Lnw9PSEp6cnYmNj8dlnn+n1efLJJw3GLTIy0tJp6FhirTB1vM3FlOP29jtKo9HgoYce0vVRamwstcbd87jIz0BiYqKEhoZKeXm5lJeXS2hoqDz88MP9PiY6OlpWrFghly9f1t2uX79upYj7t3//fnFycpKdO3dKTU2NpKeni5ubm3z33Xe99i8vLxcHBwfJzs6W2tpayc7OFkdHR/n000+tHLlxTM0vLy9PPDw89Mbq8uXLVo7aeEeOHJG1a9dKQUGBAJDCwsJ++1+4cEGGDRsm6enpUlNTIzt37hQnJyd57733rBSx6UzNsbS0VADIV199pTeG3d3dVoq4b5aYbzk5OeLu7i4FBQVSVVUlycnJ4uPjIy0tLTaVyxNPPCFbt26VyspKqa2tlWXLlolWq5VLly7p+qSlpUliYqLeuDU1NVk0j8HmY8xaYepzKpVLU1OTXg7V1dXi4OAgeXl5uj5KjY0l1jhzjIvqC56amhoBoLfYVFRUCAA5d+5cn4+Ljo6W9PR0a4RoshkzZsjKlSv12oKDgyUjI6PX/osXL5bExES9toSEBFmyZInFYrwXpuaXl5cnWq3WGqGZnTGLwe9//3sJDg7Wa3v66aclMjLSkqGZjSkFz7Vr16wUlfHMPd9u3bol3t7ekpOTo7v/5s2botVqZceOHWaOXp+pudytu7tb3N3d5c0339S1paWlyaOPPmrWOI1libXiXr9Hg3Wvx920aZO4u7tLW1ubrk3JsbnDXGucOcZF9ae0KioqoNVqMXPmTF1bZGQktFotysvL+31sfn4+vLy88Mtf/hIvvPCC7tOSldTZ2YnTp08jPj5erz0+Pr7PfCoqKgz6JyQkDJi/EgaTHwC0tbXB398fY8aMwcMPP4zKykpLh2o1fY3f559/jq6uLoWisoxp06bBx8cH8+fPR2lpqdLhWGS+1dXVoaGhQa+Pi4sLoqOjLTonBzu3/l17ezu6uroMPvS1rKwMo0aNwi9+8QusWLECV65cMVvcfbHEWmGO79FgmOO4u3btwpIlS+Dm5qbXrsTYmGqgNc5c46L6gqehoQGjRo0yaB81apTBh5D+u5SUFOzbtw9lZWV48cUXUVBQgIULF1oyVKM0Njaip6fH4MNSR48e3Wc+DQ0NJvVX0mDyCw4Oxp49e/D+++9j3759cHV1xezZs3H+/HlrhGxxfY1fd3c3GhsbFYrKvHx8fPDGG2+goKAABw8eRFBQEObPn4/jx48rGpcl5tudf609JweTy90yMjLg5+eH2NhYXduCBQuQn5+Pv/3tb3j11Vdx6tQpPPjgg+jo6DBr/HezxFphju/RYNzrcT/77DNUV1dj+fLleu1KjY2pBlrjzDUudrsfdVZWFtatW9dvn1OnTgEANBqNwX0i0mv7HStWrND9PzQ0FIGBgYiIiMCZM2cQFhY2yKjN5+7YB8rH1P5KMyXeyMhIvTfizZ49G2FhYdiyZQtef/11i8ZpLb19P3prt1dBQUEICgrSfR0VFYWLFy9iw4YNmDdvnoKR3WaJ+abUnBzscV955RXdH4Gurq669uTkZN3/Q0NDERERAX9/fxQVFVnlj0RLrBX2Nja7du1CaGgoZsyYodeu9NiYor81rq/1ztRxsduC59lnnx3wqqnx48fj73//O3788UeD+65evWpQLfYnLCwMTk5OOH/+vKIFj5eXFxwcHAyq2itXrvSZj7e3t0n9lTSY/O42ZMgQTJ8+XTWv8PQ1fo6OjhgxYoRCUVleZGQk3nrrLUVjsMR88/b2BnD7r1ofHx+jntMc7mVubdiwAdnZ2Th69CgmT57cb18fHx/4+/tbfP5ZYq0wx3MOxr0ct729Hfv378f69esHPI61xsZUA61xImKWcbHbU1peXl4IDg7u9+bq6oqoqCg0NzfrXUp58uRJNDc3Y9asWUYf7+zZs+jq6tJboJTg7OyM8PBwlJSU6LWXlJT0mU9UVJRB/+LiYpPyt5bB5Hc3EcEXX3yh+FiZS1/jFxERAScnJ4WisrzKykrFx9AS8y0gIADe3t56fTo7O3Hs2DGLzsnBzq0//elP+OMf/4iPPvoIERERAx6nqakJFy9etPjYWWKtMMdzDsa9HPedd95BR0cHfvvb3w54HGuNjakGWuPMNi5Gv73ZjiUmJsrkyZOloqJCKioq5Fe/+pXeZemXLl2SoKAgOXnypIiI/OMf/5B169bJqVOnpK6uToqKiiQ4OFimTZtmU5fJ7tq1S2pqamTVqlXi5uYm3377rYiIpKam6r1z/ZNPPhEHBwfJycmR2tpaycnJsYvL0o3NLysrSz766CP55ptvpLKyUpYtWyaOjo668bQ1ra2tUllZKZWVlQJANm7cKJWVlbrLKzMyMiQ1NVXX/84lm6tXr5aamhrZtWuXzV+WbmqOmzZtksLCQvn666+lurpaMjIyBIAUFBQolYKOJeZbTk6OaLVaOXjwoFRVVcnjjz9u1cvSjc0lNzdXnJ2d5b333tO7tLm1tVVEbo/z888/L+Xl5VJXVyelpaUSFRUlfn5+Fs9lMPkYs1YM9Jy2kssdc+bMkeTkZIN2JcfGEmucOcblZ1HwNDU1SUpKiri7u4u7u7ukpKToXf5aV1cnAKS0tFREROrr62XevHkyfPhwcXZ2lokTJ8pzzz1ntb0ljLF161bx9/cXZ2dnCQsLk2PHjunui46OlrS0NL3+7777rgQFBYmTk5MEBwfbxC+S/piS36pVq2TcuHHi7OwsI0eOlPj4eCkvL1cgauPcuQT77tudnNLS0iQ6OlrvMWVlZTJt2jRxdnaW8ePHy/bt260fuAlMzTE3N1cmTpworq6u4unpKXPmzJGioiJlgu+FuefbrVu35KWXXhJvb29xcXGRefPmSVVVlTVSMSkXf3//XsfxpZdeEhGR9vZ2iY+Pl5EjR4qTk5OMGzdO0tLSpL6+3iq5mJqPsWtFf89pSab+nH311VcCQIqLiw2eS8mxsdQad6/johH517uBiIiIiFTKbt/DQ0RERGQsFjxERESkeix4iIiISPVY8BAREZHqseAhIiIi1WPBQ0RERKrHgoeIiIhUjwUPERERqR4LHiIi+lnYsWMHxo0bBzc3NyxatAiNjY1Kh0RWxIKHiIhU79ChQ/jd736HLVu24PPPP0dLSwsee+wxpcMiK+JHSxARkepFREQgNjYWOTk5AIAffvgBY8eOxfHjxzF79myFoyNr4Cs8ZDf27t2LESNGoKOjQ6990aJFWLp0qUJREZGtu3btGk6fPo3f/OY3ujZfX1+EhoaipKREwcjImljwkN147LHH0NPTg/fff1/X1tjYiA8//BDLli1TMDIismUXLlwAAAQGBuq1BwYG6u4j9WPBQ3Zj6NCheOKJJ5CXl6dry8/Px5gxYxATE6NcYERk09rb2wHcLnDuu+8+3e3w4cO6+0j9HJUOgMgUK1aswPTp0/H999/Dz88PeXl5ePLJJ6HRaJQOjYhs1LBhwwAAZWVluP/++3Xt6enpuvtI/VjwkF2ZNm0apkyZgr179yIhIQFVVVX44IMPlA6LiGzYhAkTAAAeHh544IEHdO03b97U3Ufqx1NaZHeWL1+OvLw87N69G7GxsRg7dqzSIRGRDfP09ER4eDhOnDiha2tra0NFRQXi4uIUjIysiZelk91paWmBj48Puru7sXfvXiQnJysdEhHZuEOHDmHlypXYu3cvAgICsHbtWly9ehWlpaVKh0ZWwlNaZHc8PDywaNEiFBUVISkpSelwiMgOJCUloaGhAU899RSampqQmJiId955R+mwyIr4Cg/Zpbi4OEyaNAmvv/660qEQEZEdYMFDduWf//wniouLkZKSgpqaGgQFBSkdEhER2QGe0iK7EhYWhmvXriE3N5fFDhERGY2v8BAREZHq8bJ0IiIiUj0WPERERKR6LHiIiIhI9VjwEBERkeqx4CEiIiLVY8FDREREqseCh4iIiFSPBQ8RERGp3v8DYWVb2E8b2hMAAAAASUVORK5CYII=",
      "text/plain": [
       "Figure(PyObject <Figure size 640x480 with 2 Axes>)"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "using PyPlot\n",
    "#using Plots\n",
    "p(y,θ) = θ.^y .* (1 .- θ).^(1 .- y)\n",
    "f = figure()\n",
    "\n",
    "θ = 0.5 # Set parameter\n",
    "# Plot the sampling distribution\n",
    "subplot(221); stem([0,1], p([0,1],θ)); \n",
    "title(\"Sampling distribution\");\n",
    "xlim([-0.5,1.5]); ylim([0,1]); xlabel(\"y\"); ylabel(\"p(y|θ=$(θ))\");\n",
    "\n",
    "subplot(222);\n",
    "_θ = 0:0.01:1\n",
    "y = 1.0 # Plot p(y=1 | θ)\n",
    "plot(_θ,p(y,_θ))\n",
    "title(\"Likelihood function\"); \n",
    "xlabel(\"θ\"); \n",
    "ylabel(\"L(θ) = p(y=$y)|θ)\");\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "The (discrete) sampling distribution is a valid probability distribution. \n",
    "However, the likelihood function $L(\\theta)$ clearly isn't, since $\\int_0^1 L(\\theta) \\mathrm{d}\\theta \\neq 1$. \n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Probabilistic Inference\n",
    "\n",
    "- **Probabilistic inference** refers to computing\n",
    "$$\n",
    "p(\\,\\text{whatever-we-want-to-know}\\, | \\,\\text{whatever-we-already-know}\\,)\n",
    "$$\n",
    "  - For example: \n",
    "  $$\\begin{align*}\n",
    " p(\\,\\text{Mr.S.-killed-Mrs.S.} \\;&|\\; \\text{he-has-her-blood-on-his-shirt}\\,) \\\\\n",
    " p(\\,\\text{transmitted-codeword} \\;&|\\;\\text{received-codeword}\\,) \n",
    "  \\end{align*}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- This can be accomplished by repeated application of sum and product rules."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- In particular, consider a joint distribution $p(X,Y,Z)$. Assume we are interested in $p(X|Z)$:\n",
    "$$\\begin{align*}\n",
    "p(X|Z) \\stackrel{p}{=} \\frac{p(X,Z)}{p(Z)} \\stackrel{s}{=} \\frac{\\sum_Y p(X,Y,Z)}{\\sum_{X,Y} p(X,Y,Z)} \\,,\n",
    "\\end{align*}$$\n",
    "where the 's' and 'p' above the equality sign indicate whether the sum or product rule was used. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- In the rest of this course, we'll encounter many long probabilistic derivations. For each manipulation, you should be able to associate an 's' (for sum rule), a 'p' (for product or Bayes rule) or an 'm' (for a simplifying model assumption) above any equality sign."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Working out the example problem: Disease Diagnosis\n",
    "\n",
    "- **Problem**: Given a disease $D$ with prevalence of $1\\%$ and a test procedure $T$ with sensitivity ('true positive' rate) of $95\\%$ and specificity ('true negative' rate) of $85\\%$, what is the chance that somebody who tests positive actually has the disease?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- **Solution**: The given data are $p(D=1)=0.01$, $p(T=1|D=1)=0.95$ and $p(T=0|D=0)=0.85$. Then according to Bayes rule,\n",
    "\n",
    "$$\\begin{align*}\n",
    "p( D=1 &| T=1) \\\\\n",
    "&\\stackrel{p}{=} \\frac{p(T=1|D=1)p(D=1)}{p(T=1)} \\\\\n",
    "&\\stackrel{s}{=} \\frac{p(T=1|D=1)p(D=1)}{p(T=1|D=1)p(D=1)+p(T=1|D=0)p(D=0)} \\\\\n",
    "&= \\frac{0.95\\times0.01}{0.95\\times0.01 + 0.15\\times0.99} = 0.0601\n",
    "\\end{align*}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Inference Exercise: Bag Counter\n",
    "\n",
    "- **Problem**:  A bag contains one ball, known to be either white or black. A white ball is put in, the bag is shaken,\n",
    " and a ball is drawn out, which proves to be white. What is now the\n",
    " chance of drawing a white ball?\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- **Solution**: Again, use Bayes and marginalization to arrive at $p(\\text{white}|\\text{data})=2/3$, see the [Exercises](https://nbviewer.org/github/bertdv/BMLIP/blob/master/lessons/exercises/Exercises-Probability-Theory-Review.ipynb) notebook.\n",
    "\n",
    "- $\\Rightarrow$ Note that probabilities describe **a person's state of knowledge** rather than a 'property of nature'."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Inference Exercise: Causality?\n",
    "\n",
    "- **Problem**: A dark bag contains five red balls and seven green ones. (a) What is the probability of drawing a red ball on the first draw? Balls are not returned to the bag after each draw. (b) If you know that on the second draw the ball was a green one, what is now the probability of drawing a red ball on the first draw?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- **Solution**: (a) $5/12$. (b) $5/11$, see the [Exercises](https://nbviewer.org/github/bertdv/BMLIP/blob/master/lessons/exercises/Exercises-Probability-Theory-Review.ipynb) notebook.\n",
    "\n",
    "- $\\Rightarrow$ Again, we conclude that conditional probabilities reflect **implications for a state of knowledge** rather than temporal causality."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Moments of the PDF\n",
    "\n",
    "- Consider a distribution $p(x)$. The **expected value** or **mean** is defined as \n",
    "$$\\mu_x = \\mathbb{E}[x] \\triangleq  \\int x \\,p(x) \\,\\mathrm{d}{x}$$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- The **variance** of $x$ is defined as \n",
    "$$\\Sigma_x \\triangleq \\mathbb{E} \\left[(x-\\mu_x)(x-\\mu_x)^T \\right]$$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- The **covariance** matrix between _vectors_ $x$ and $y$ is defined as\n",
    "$$\\begin{align*}\n",
    "    \\Sigma_{xy} &\\triangleq \\mathbb{E}\\left[ (x-\\mu_x) (y-\\mu_y)^T \\right]\\\\\n",
    "    &= \\mathbb{E}\\left[ (x-\\mu_x) (y^T-\\mu_y^T) \\right]\\\\\n",
    "    &= \\mathbb{E}[x y^T] - \\mu_x \\mu_y^T\n",
    "\\end{align*}$$\n",
    "  - Clearly, if $x$ and $y$ are independent, then $\\Sigma_{xy} = 0$, since $\\mathbb{E}[x y^T] = \\mathbb{E}[x] \\mathbb{E}[y^T] = \\mu_x \\mu_y^T$.\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "\n",
    "### <a id=\"linear-transformation\">Linear Transformations</a> \n",
    "\n",
    "- Consider an arbitrary distribution $p(X)$ with mean $\\mu_x$ and variance $\\Sigma_x$ and the linear transformation $$Z = A X + b \\,.$$ \n",
    "\n",
    "- No matter the specification of $p(X)$, we can derive that (see [Exercises](https://nbviewer.org/github/bertdv/BMLIP/blob/master/lessons/exercises/Exercises-Probability-Theory-Review.ipynb) notebook)\n",
    "$$\\begin{align}\n",
    "\\mu_z &= A\\mu_x + b \\tag{SRG-3a}\\\\\n",
    "\\Sigma_z &= A\\,\\Sigma_x\\,A^T \\tag{SRG-3b}\n",
    "\\end{align}$$\n",
    "  -  (The tag (SRG-3a) refers to the corresponding eqn number in Sam Roweis [Gaussian identities](https://github.com/bertdv/BMLIP/blob/master/lessons/notebooks/files/Roweis-1999-gaussian-identities.pdf) notes.)\n",
    "\n",
    "\n"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### PDF for the Sum of Two Variables\n",
    "\n",
    "\n",
    "- Given eqs SRG-3a and SRG-3b (previous cell), you should now be able to derive the following: for any distribution of variable $X$ and $Y$ and sum $Z = X+Y$ (proof by [Exercise](https://nbviewer.org/github/bertdv/BMLIP/blob/master/lessons/exercises/Exercises-Probability-Theory-Review.ipynb))\n",
    "\n",
    "$$\\begin{align*}\n",
    "    \\mu_z &= \\mu_x + \\mu_y \\\\\n",
    "    \\Sigma_z &= \\Sigma_x + \\Sigma_y + 2\\Sigma_{xy} \n",
    "\\end{align*}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Clearly, it follows that if $X$ and $Y$ are **independent**, then\n",
    "\n",
    "$$\\Sigma_z = \\Sigma_x + \\Sigma_y $$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- More generally, given two **independent** variables\n",
    "$X$ and $Y$, with PDF's $p_x(x)$ and $p_y(y)$. The PDF $p_z(z)$for $Z=X+Y$ is given by the **convolution**\n",
    "\n",
    "$$\n",
    "p_z (z) = \\int_{ - \\infty }^\\infty  {p_x (x)p_y (z - x)\\,\\mathrm{d}{x}}\n",
    "$$ "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- **Proof**: Let $p_z(z)$ be the probability that $Z$ has value $z$. This occurs if $X$ has some value $x$ and at the same time $Y=z-x$, with joint probability $p_x(x)p_y(z-x)$. Since $x$ can be any value, we sum over all possible values for $x$ to get\n",
    "$\n",
    "p_z (z) = \\int_{ - \\infty }^\\infty  {p_x (x)p_y (z - x)\\,\\mathrm{d}{x}}\n",
    "$    \n",
    "  \n",
    "  - Note that $p_z(z) \\neq p_x(x) + p_y(y)\\,$ !!\n",
    " "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- [https://en.wikipedia.org/wiki/List_of_convolutions_of_probability_distributions](https://en.wikipedia.org/wiki/List_of_convolutions_of_probability_distributions) shows how these convolutions work out for a few common probability distributions. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- In linear stochastic systems theory, the Fourier Transform of a PDF (i.e., the characteristic function) plays an important computational role. Why?"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Code Example: Sum of Two Gaussian Distributed Variables\n",
    "\n",
    "- Consider the PDF of the sum of two independent Gaussian distributed $X$ and $Y$:\n",
    "\n",
    "$$\\begin{align*}\n",
    "p_X(x) &= \\mathcal{N}(\\,x\\,|\\,\\mu_X,\\sigma_X^2\\,) \\\\ \n",
    "p_Y(y) &= \\mathcal{N}(\\,y\\,|\\,\\mu_Y,\\sigma_Y^2\\,) \n",
    "\\end{align*}$$\n",
    "\n",
    "- Let $Z = X + Y$. Performing the convolution (nice exercise) yields a Gaussian PDF for $Z$: \n",
    "\n",
    "$$\n",
    "p_Z(z) = \\mathcal{N}(\\,z\\,|\\,\\mu_X+\\mu_Y,\\sigma_X^2+\\sigma_Y^2\\,).\n",
    "$$"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {},
   "outputs": [
    {
     "data": {
      "image/png": "iVBORw0KGgoAAAANSUhEUgAAAiMAAAGdCAYAAADAAnMpAAAABHNCSVQICAgIfAhkiAAAAAlwSFlzAAAPYQAAD2EBqD+naQAAADh0RVh0U29mdHdhcmUAbWF0cGxvdGxpYiB2ZXJzaW9uMy4xLjEsIGh0dHA6Ly9tYXRwbG90bGliLm9yZy8QZhcZAAAgAElEQVR4nOzdeVyU1f7A8c8Aw6YYmoqKaKjlBoaCuaV5S8mlRcsyszTT0qx+KddberVMy/T++l3TFryVS4stVlezBRXMzI3SUBDNBVdMUQQXBAQGeH5/nAZE1oEZnpnh+369eM0w88zzfDmMzpdzvuccg6ZpGkIIIYQQOnHROwAhhBBC1G2SjAghhBBCV5KMCCGEEEJXkowIIYQQQleSjAghhBBCV5KMCCGEEEJXkowIIYQQQleSjAghhBBCV256B1AVhYWFnDlzBh8fHwwGg97hCCGEEKIKNE3jypUrtGjRAheX8vs/HCIZOXPmDAEBAXqHIYQQQohqOHXqFC1btiz3eYdIRnx8fAD1wzRo0MBq5zWZTERHRxMeHo7RaLTaeZ2JtFHFpH0qJu1TMWmfykkbVcze2ycjI4OAgICiz/HyOEQyYh6aadCggdWTEW9vbxo0aGCXv0R7IG1UMWmfikn7VEzap3LSRhVzlPaprMRCCliFEEIIoStJRoQQQgihK0lGhBBCCKErh6gZEUIIIeyNpmnk5+dTUFCgWwwmkwk3NzdycnJ0icPV1RU3N7caL7shyYgQQghhoby8PFJSUsjOztY1Dk3TaNasGadOndJtHS5vb2+aN2+Ou7t7tc8hyYgQQghhgcLCQo4fP46rqystWrTA3d1dt0SgsLCQzMxM6tevX+GiYragaRp5eXmcP3+e48ePc/PNN1c7BklGhBBCCAvk5eVRWFhIQEAA3t7eusZSWFhIXl4enp6etZ6MAHh5eWE0Gjl58mRRHNUhBaxCCCFENejx4W+PrNEO1TpDZGQkgYGBeHp6EhoaytatWys8ftGiRbRv3x4vLy8CAgKYOnUqOTk51QpYCCGEEM7F4mRk1apVTJkyhZkzZ7Jnzx769u3L4MGDSU5OLvP4zz77jOnTpzN79mwOHDjAsmXLWLVqFTNmzKhx8EIIIYRwfBYnIwsXLmT8+PFMmDCBjh07smjRIgICAliyZEmZx8fGxtKnTx8effRRbrrpJsLDwxk1ahS///57jYMXQgghhOOzqIA1Ly+PuLg4pk+fXuLx8PBwduzYUeZrbr/9dlauXMnOnTu57bbbOHbsGFFRUYwdO7bc6+Tm5pKbm1v0fUZGBqDmU5tMJktCrpD5XNY8p7ORNqqYtE/FpH0qJu1TOXtsI5PJhKZpFBYWUlhYqGssmqYV3eoVS2FhIZqmYTKZcHV1LfFcVX9vFiUjaWlpFBQU4OfnV+JxPz8/zp49W+ZrHnnkEc6fP8/tt99etEDMM888Uyqhudb8+fOZM2dOqcejo6NtUrkcExNj9XM6G2mj0jQNTp3yYffutvz00yHuvDMZnWb32T15/1RM2qdy9tRGbm5uNGvWjMzMTPLy8vQOB4ArV65U+djU1FTat2/P/Pnz+fzzzzl8+DCBgYEsXLiQXr16WXztvLw8rl69ypYtW8jPzy/xXFXXYanW1N7r51NrmlbuHOvNmzczb948IiMj6dGjB0eOHOGFF16gefPmvPzyy2W+ZsaMGURERBR9b96CODw83Oq79sbExDBw4EC73u1QT9JGJWkaREcb+PFHA+vWuXDyZPH7PjPzVt59twBppmLy/qmYtE/l7LGNcnJyOHXqFPXr1y+ayqppmi4LoHl5eZGZmYmPj0+V1zoxj2SsWLGCJUuW0Lx5c/7+978zadIkjh49avHsmJycHLy8vOjXr1+pqb3mkY3KWJSMNG7cGFdX11K9IKmpqaV6S8xefvllHn/8cSZMmABAcHAwWVlZPP3008ycObPMH9rDwwMPD49SjxuNRpu8GW11XmcibaQ8/zy8+27x9x4eGu3apXPgwI2sWOHC6dMufP01WDFndgry/qmYtE/l7KmNCgoKMBgMuLi4FH2GZWVlWfWP5aoyf9ib46mKxMREjEYj69evJzAwEIDXXnuNsLAwUlJS+OKLL8jKyioaoRg5ciRhYWH84x//KPN8Li4uGAyGMn9HVf2dWZT+uLu7ExoaWqq7LCYmht69e5f5muzs7FIN5OrqiqZpRWNdQjiCqKjiROSpp+C77+Ds2XzmzdvOf/9bgLc3REdD377w55/6xiqEEOWJj4/ngQceKEpEgBIdAOPHj+fTTz/FZDLxzjvvkJOTw7Rp02wak8XDNBERETz++OOEhYXRq1cvPvjgA5KTk5k0aRIAY8aMwd/fn/nz5wNw7733snDhQrp27Vo0TPPyyy9z3333lSp0EcJepaXB+PHq/pQp8NZb6r65NmvoUI1ffoF77oG9e6FnT4iJgY4d9YlXCFG7vL29yczMrPXrenp6WlQvAioZuX4Sye7du2ncuDH+/v64uLjQv39/pk+fzo8//sivv/5q8+XuLU5GRo4cSXp6OnPnziUlJYWgoCCioqJo3bo1AMnJySV6QmbNmoXBYGDWrFmcPn2aJk2acO+99zJv3jzr/RRC2JCmwaRJcPasSi7eeKPs48LC4NdfYcgQOHAAnn4atmxBilqFqAMMBgP16tWr9etaOoPm6tWrJCUlldjht7CwkMWLFzN27Niiz++xY8dy5513snPnTnx9fa0ac1mqVcA6efJkJk+eXOZzmzdvLnkBNzdmz57N7Nmzq3MpIXS3ciX897/g5qbue3mVf+xNN6mhmnbtYNs22LQJ7rqr1kIVQogKJSYmYjAYWLlyJXfeeSe+vr688sorXLp0iVmzZgGqGHfBggXceOONtRaXLKwvRAWSk+G559T92bOhW7fKX9OypeoVAXj1VdWzIoQQ9iA+Pp4OHTowa9YsRowYQVhYGC4uLsTGxhb1gMydOxd/f38WL17Mu9dW7NuQJCNClEPTYNw4yMhQNSAVLI1TyksvgYeH6h356SfbxSiEEJZISEggODiY0aNHc/r0abKysli1ahVNmzYF1Hpea9eu5d1332XEiBH88ssvpKen2zwuSUaEKMemTerLyws++UQN01SVv7/0jggh7E98fDxdunQp87lTp07xzDPP8NVXX+Hp6YnRaGTs2LEsW7bM5nFJMiJEOT78UN2OHQs332z566dPV70j27dL74gQQn+appGYmFhuMhIQEMDRo0dp165d0WOzZ8/mxRdftHlskowIUYbz52HNGnXf3MNhqRYtYOJEdV96R4QQejMYDGRkZDBkyBC9QylFkhEhyvDJJ5CXB6Gh0LVr9c8zfTp4eqrekY0brRefEEI4E0lGhLiOphUP0Tz1VM3O1by59I4IIURlJBkR4jrbtsGhQ1CvHowaVfPzvfSS6h3ZsQN+/73m5xNCCGcjyYgQ1zH3ijzyiHU2vGveHO6/X93/5puan08IIZyNJCNCXOPiRfj6a3W/pkM01xoxQt1+/bUM1QghxPUkGRHiGitXQk4OBAfDbbdZ77xDhoC3Nxw/Drt3W++8QgjhDCQZEeIv1xauPv20dTe48/ZWCQnIUI0QQlxPkhEh/vLbb5CYqIpNR4+2/vkfekjdylCNEEKUJMmIEH9ZvlzdPvQQNGxo/fMPGaKWlj96FBISrH9+IYRwVJKMCAEUFsL336v7jz9um2vUrw+DB6v75iJZIYQQkowIAaieirNn1doi/frZ7joyVCOEEKVJMiIEsG6dur3rLrW5na0MHarOn5Sk6lOEEKK2nTt3DoPBwOLFi+natSuenp507tyZbdu26RaTBZuiC+G8zMmIeRjFVnx81DW+/Vb1jpSzeaYQwsFoGmRn1/51PT0tf82ePXsAiIyM5P3336d58+ZEREQwevRojh8/jotL7fdTSM+IqPMuXlRLtYPtkxGQBdCEcEbZ2aourLa/qpMAJSQkYDQaWb9+Pf3796d9+/bMnTuX5ORktm7dSqdOnYqOzcrKIiQkhN9++82KrVWaJCOizouJUQWsnTpB69a2v96994K7u9r/Zv9+219PCCGuFR8fzwMPPEBgYGDRYx5/jU+3adOGJk2asGXLFgCeeuopJkyYQI8ePWwakwzTiDqvtoZozBo0gLvvVrN3vv4agoJq57pCCNvx9obMzNq/rqcnXLli2Wvi4+MZO3Zsicd2795N48aN8ff35/nnn+f9999n7969aJrGc889Z8WIyybJiKjTCgth/Xp1v7aSEVBDNd9/Dz/8AHPm1N51hRC2YTCo2Xi1rbDQsuOvXr1KUlISBQUF15yjkMWLFzN27FhcXFwYNmwY06dPZ//+/bVW1CrJiKjT4uOLp/TefnvtXXfAAHW7Z4+qWbHFImtCCHG9xMREDAYDK1eu5M4778TX15dXXnmFS5cuMWvWLADc3NwICQnhscceo379+rUSl9SMiDrNPEQzYIBtp/Rer0ULaN9eFbD+NTQrhBA2Fx8fT4cOHZg1axYjRowgLCwMFxcXYmNj8fX1LTru4MGDdKnF6X6SjIg6rbbrRa71t7+p259/rv1rCyHqpoSEBIKDgxk9ejSnT58mKyuLVatW0bRp06JjTCYTZ86cKVHgamuSjIg66+JFiI1V9yUZEULUBfHx8ZX2eCQlJdGuXTsM1ty6vBKSjIg6Kzq6eEpvq1a1f/3+/dXt3r1w/nztX18IUbdomkZiYmKlyUinTp3YuXNnLUWlSDIi6izzEM2QIfpcv2nT4mm9v/yiTwxCiLrDYDCQkZHBEL3+06uAJCOiTtJrSu/1ZKhGCCEkGRF1VEICnDunllOuzSm91zMnI5s26ReDEELoTZIRUSeZp9P27auWZtfLHXeoxZIOHoSUFP3iEEIIPVUrGYmMjCQwMBBPT09CQ0PZunVrucf2798fg8FQ6mvo0KHVDlqImtq+Xd3q2SsC0KgRhISo+5s36xqKEELoxuJkZNWqVUyZMoWZM2eyZ88e+vbty+DBg0lOTi7z+NWrV5OSklL0tW/fPlxdXXnooYdqHLwQ1aFp9pOMgAzVCCGExcnIwoULGT9+PBMmTKBjx44sWrSIgIAAlixZUubxjRo1olmzZkVfMTExeHt7SzIidHPiBJw5A0YjdO+udzRSxCqEEBYlI3l5ecTFxREeHl7i8fDwcHbs2FGlcyxbtoxHHnmEenrsKCQExb0i3bqBl5e+sQD06weurnD0KJTTwSiEEE7Noo3y0tLSKCgowM/Pr8Tjfn5+nD17ttLX79y5k3379rFs2bIKj8vNzSU3N7fo+4yMDEAtUWsymSwJuULmc1nznM7GGdtoyxYXwJXevQswmSzc8vI61mgfLy/o1s2VXbtciInJZ8wYrUYx2RNnfP9Yk7RP5eyxjUwmE5qmUVhYSKGl2+ZamaZpRbd6xVJYWIimaZhMJlxdXUs8V9XfW7V27b1+iVhN06q0bOyyZcsICgritttuq/C4+fPnM6eMfdWjo6Px9va2LNgqiImJsfo5nY0ztdGGDX8DGuDhEUdUlHWmsNS0fQICOrFr18189tkZGjfeY5WY7IkzvX9sQdqncvbURm5ubjRr1ozMzEzy8vL0DgeAK1eu6HbtvLw8rl69ypYtW8jPzy/xXHZ2dpXOYVEy0rhxY1xdXUv1gqSmppbqLblednY2X375JXPnzq30OjNmzCAiIqLo+4yMDAICAggPD6dBgwaWhFwhk8lETEwMAwcOxGg0Wu28zsTZ2ujiRUhOVj/Hc891pWnTrjU6n7Xax83NwOrVcPRoAIMHN6cWt4SwKWd7/1ibtE/l7LGNcnJyOHXqFPXr18fT01PXWDRN48qVK/j4+NTqXjLXysnJwcvLi379+pVqD/PIRmUsSkbc3d0JDQ0lJiaG4cOHFz0eExPD/fffX+Frv/rqK3Jzc3nssccqvY6HhwceZeznbjQabfJmtNV5nYmztNHvv6vbm28Gf3/r/Tw1bZ/+/cHNDZKTDZw6ZaRtW6uFZhec5f1jK9I+lbOnNiooKMBgMODi4oKLi77LdZmHZszx6MHFxQWDwVDm76iqvzOLI4+IiGDp0qUsX76cAwcOMHXqVJKTk5k0aRIAY8aMYcaMGaVet2zZMoYNG8aNN95o6SWFsBp7mtJ7rXr1imf2mGMUQghbOHfuHAaDgcWLF9O1a1c8PT3p3Lkz27Zt0y0mi2tGRo4cSXp6OnPnziUlJYWgoCCioqJo3bo1AMnJyaWys8OHD7Nt2zaio6OtE7UQ1WT+t9anj75xlKVnT4iNhd9+gzFj9I5GCGERTYMq1kdYVTWGifbsUXVpkZGRvP/++zRv3pyIiAhGjx7N8ePHdelhqVYB6+TJk5k8eXKZz20uYxnJW265pajiVwi95OWBeVdse0xGevRQt7/9pm8cQohqyM5Wm13VtirWZFwrISEBo9HI+vXrCQwMBGDu3LmEhYVx+vRpXnzxRQ4cOFB0/MmTJwkJCeFnGy6GVK1kRAhHtHs35OTAjTdC+/Z6R1Naz57qNiEBrl61jzVQhBDOJz4+ngceeKAoEQFK1Gl+8cUXRfd37tzJ/fffz9tvv23TmCQZEXWGuRajTx/scrZKq1bg56d2E9692z57b4QQ5fD2hszM2r+upydYOK03Pj6esWPHlnhs9+7dNG7cGH9//6LHjh8/zgMPPMDHH39McHCwVcItjyQjos4w14vYW/GqmcGghmq++04N1UgyIoQDMRhUJXpts3Chs6tXr5KUlERBQcE1pyhk8eLFjB07tqhe5OLFiwwdOpRXX3211KrrtqDvnCQhasm1m+PZ84e81I0IIWwpMTERg8HAypUriY2N5cCBA4wcOZJLly4xa9YsQC1iNnz4cIYNG8aECRNqJS5JRkSdcOQInD8PHh4QGqp3NOWTZEQIYUvx8fF06NCBWbNmMWLECMLCwnBxcSE2NhZfX18Axo8fT7NmzZg3b16txSXDNKJOMA/RdO+uEhJ71b276u09eVLVjlSysLEQQlgkISGB4OBgRo8ezejRo0s9//LLL3P8+HF++umnWl3RVXpGRJ3gCEM0AA0aQKdO6r70jgghrC0+Pp4uXbqU+dyJEyd4/fXXSUlJoUePHoSEhBASEsI///lPm8clPSOiToiNVbf2noyAGqrZv18lI/fdp3c0QghnoWkaiYmJzJw5s8znb7rpJt3WBJNkRDi9zEwwr99jXnLdnvXoAcuXS8+IEMK6DAZDlTeuq20yTCOc3p49ajZNy5bQrJne0VTOXMS6cydcM/tOCCGcliQjwumZd+oNC9M3jqrq3Fmtn3TlChw8qHc0Qghhe5KMCKfnaMmIm1txrDJUI4SoCyQZEU7P0ZIRKN6nRpIRIURdIMmIcGqXL8Phw+q+PS92dj1Z/EwI+ye70SvWaAdJRoRT271b3bZuDY0b6xuLJczJSGIiZGXpG4sQoiSj0QhAdna2zpHYB3M7mNulOmRqr3BqjjhEA+Dvr75On4a4OOjXT++IhBBmrq6u+Pr6kpqaCoC3t3etrlZ6rcLCQvLy8sjJySna5K62aJpGdnY2qamp+Pr64urqWu1zSTIinJqjJiOgekdWr1ZDNZKMCGFfmv21ToA5IdGLpmlcvXoVLy8v3RIiX1/fovaoLklGhFNzlmRECGFfDAYDzZs3p2nTpphMJt3iMJlMbNmyhX79+tVomKS6jEZjjXpEzCQZEU7r4kU4dkzdd6TiVbNrFz8TQtgnV1dXq3wY1+T6+fn5eHp66pKMWIsUsAqnFRenbtu2hYYN9Y2lOrp2VbenTkF6ur6xCCGELUkyIpyWIw/RgNrBt107dX/PHn1jEUIIW5JkRDgtR09GoLh3xDxFWQghnJEkI8JpOUMy0q2bupWeESGEM5NkRDil8+fh5El139y74IikZ0QIURdIMiKckrl49ZZb4IYb9I2lJszJSFKS2sVXCCGckSQjwik5wxANQNOmaiVWTYOEBL2jEUII25BkRDglZ0lGQOpGhBDOT5IR4ZScKRmRuhEhhLOTZEQ4nZQUtcGcweDYxatm0jMihHB2kowIp2MuXu3YEerX1zcWazAnVPv3Q26uvrEIIYQtSDIinE58vLp1hl4RgIAAuPFGyM+Hffv0jkYIIaxPkhHhdJwtGbl2uEnqRoQQzqhayUhkZCSBgYF4enoSGhrK1q1bKzz+0qVLPPvsszRv3hxPT086duxIVFRUtQIWojLmZOTWW/WNw5qkbkQI4czcLH3BqlWrmDJlCpGRkfTp04f333+fwYMH88cff9CqVatSx+fl5TFw4ECaNm3KN998Q8uWLTl16hQ+Pj5W+QGEuNaVK3D0qLrvTMmI9IwIIZyZxcnIwoULGT9+PBMmTABg0aJFbNiwgSVLljB//vxSxy9fvpwLFy6wY8cOjEYjAK1bt65h2EKUbe9edevvD02a6BuLNZl7RvbuVbUjbhb/yxVCCPtl0X9peXl5xMXFMX369BKPh4eHs2PHjjJf891339GrVy+effZZ1q5dS5MmTXj00Ud56aWXcHV1LfM1ubm55F4zbSAjIwMAk8mEyWSyJOQKmc9lzXM6G0dro7g4F8CVW28txGQqsPn1aqt9WreG+vXdyMw0sG+fic6dbXo5q3G0909tk/apnLRRxey9faoal0XJSFpaGgUFBfj5+ZV43M/Pj7Nnz5b5mmPHjrFp0yZGjx5NVFQUSUlJPPvss+Tn5/PKK6+U+Zr58+czZ86cUo9HR0fj7e1tSchVEhMTY/VzOhtHaaMffrgVuAlv7yNERR2otevWRvsEBNzOgQM38vHHe+nf/0+bX8+aHOX9oxdpn8pJG1XMXtsnOzu7SsdVq7PXYDCU+F7TtFKPmRUWFtK0aVM++OADXF1dCQ0N5cyZM7z55pvlJiMzZswgIiKi6PuMjAwCAgIIDw+nQYMG1Qm5TCaTiZiYGAYOHFg0hCRKcrQ2ev111dv2wANtGDIk0ObXq832iYlx4cAB0LQQhgzpYtNrWYujvX9qm7RP5aSNKmbv7WMe2aiMRclI48aNcXV1LdULkpqaWqq3xKx58+YYjcYSQzIdO3bk7Nmz5OXl4e7uXuo1Hh4eeHh4lHrcaDTapLFtdV5n4ghtdO06HGFhbtRmuLXRPual7RMSXDEayx7itFeO8P7Rk7RP5aSNKmav7VPVmCya2uvu7k5oaGip7qCYmBh69+5d5mv69OnDkSNHKCwsLHrs8OHDNG/evMxERIjqOnwYcnKgXj1o21bvaKzPPKNmzx61i68QQjgLi9cZiYiIYOnSpSxfvpwDBw4wdepUkpOTmTRpEgBjxoxhxowZRcc/88wzpKen88ILL3D48GF+/PFH3njjDZ599lnr/RRCULy+SJcu4OKEy/l16gTu7nD5Mhw/rnc0QghhPRbXjIwcOZL09HTmzp1LSkoKQUFBREVFFU3XTU5OxuWaT4KAgACio6OZOnUqXbp0wd/fnxdeeIGXXnrJej+FEEBCgroNCdE3DlsxGiE4WO29s3s3tGmjd0RCCGEd1SpgnTx5MpMnTy7zuc2bN5d6rFevXvz666/VuZQQVWbuGXHWZATUzxYXpxKvESP0jkYIIazDCTuzRV1VF5IR86qy5sXdhBDCGUgyIpzC2bOQmqpqRYKC9I7GdszJiHlISgghnIEkI8IpmHtFbrkFbLAunt0IDla3J0+qQlYhhHAGkowIp1AXhmgAGjYE836UMlQjhHAWkowIp2BORpxpp97ydPlr8VUZqhFCOAtJRoRTcPZpvdeSIlYhhLORZEQ4vKwsOHRI3a8LyYj0jAghnI0kI8Lh7dunlkf384NmzfSOxvbMPSP79kFBgb6xCCGENUgyIhxeXaoXAWjXDry8IDsbjh7VOxohhKg5SUaEw6tL9SIArq7Fa6lI3YgQwhlIMiIcXl2Z1nstWfxMCOFMJBkRDq2wsLh3oK4M00BxEav0jAghnIEkI8KhHT+uZtN4eKjVV+sK6RkRQjgTSUaEQzP3DHTqBG7V2oPaMcmy8EIIZyLJiHBoiYnq1jxsUVfIsvBCCGciyYhwaOYP4rqWjIDUjQghnIckI8Kh1eVkROpGhBDOQpIR4bCysuDIEXVfkhEhhHBckowIh7V/v1oGvmlT9VXXmBMwWRZeCOHoJBkRDquuFq+aybLwQghnIcmIcFh1uV4EZFl4IYTzkGREOKy6noyA1I0IIZyDJCPCIWmaJCMg03uFEM5BkhHhkM6cgQsX1FBFx456R6Mf6RkRQjgDSUaEQzIXr95yC3h66huLnsw9I7IsvBDCkUkyIhySDNEovr4QEKDu79unbyxCCFFdkowIhyTJSDHzpnlSNyKEcFSSjAiHJMlIMXMyYh66EkIIRyPJiHA4eXlw4IC6L8mIzKgRQjg+SUaEwzl0CPLz4YYbiusl6rJre0Y0Td9YhBCiOiQZEQ7H3AMQHAwGg76x2IP27cFohIwMSE7WOxohhLBctZKRyMhIAgMD8fT0JDQ0lK1bt5Z77EcffYTBYCj1lZOTU+2gRd0m9SIlubtDhw7qvtSNCCEckcXJyKpVq5gyZQozZ85kz5499O3bl8GDB5NcwZ9kDRo0ICUlpcSXZ11eHELUiCQjpUndiBDCkVmcjCxcuJDx48czYcIEOnbsyKJFiwgICGDJkiXlvsZgMNCsWbMSX0JUlyQjpcmMGiGEI3Oz5OC8vDzi4uKYPn16icfDw8PZsWNHua/LzMykdevWFBQUEBISwmuvvUbXrl3LPT43N5fc3Nyi7zMyMgAwmUyYTCZLQq6Q+VzWPKezsbc2Sk+HM2eMALRvb0LvsOylfTp1MgBuJCRomEz5usZyLXtpH3sl7VM5aaOK2Xv7VDUui5KRtLQ0CgoK8PPzK/G4n58fZ8+eLfM1HTp04KOPPiI4OJiMjAwWL15Mnz59SEhI4Oabby7zNfPnz2fOnDmlHo+Ojsbb29uSkKskJibG6ud0NvbSRomJNwK34+eXxdatG/UOp4je7ZOW5gnczaFDGmvXrsdoLNQ1nuvp3T72TtqnctJGFbPX9snOzq7ScRYlI2aG66YwaJpW6jGznj170rNnz6Lv+/TpQ7du3XjnnXd4++23y3zNjBkziIiIKPo+IyODgK71/ugAACAASURBVIAAwsPDadCgQXVCLpPJZCImJoaBAwdiNBqtdl5nYm9tdOyYGlns3t2LIUOG6ByN/bSPpsG0aRqXLrnQuvUgQkJ0C6UEe2kfeyXtUzlpo4rZe/uYRzYqY1Ey0rhxY1xdXUv1gqSmppbqLSmPi4sL3bt3JykpqdxjPDw88PDwKPW40Wi0SWPb6rzOxF7aaP9+ddu1qwtGo/3MTLeH9unSBbZsgYMHjXTvrmsopdhD+9gzaZ/KSRtVzF7bp6oxWfS/ubu7O6GhoaW6g2JiYujdu3eVzqFpGvHx8TRv3tySSwsBFBdomgs2RTHZo0YI4agsHqaJiIjg8ccfJywsjF69evHBBx+QnJzMpEmTABgzZgz+/v7Mnz8fgDlz5tCzZ09uvvlmMjIyePvtt4mPj+e9996z7k8inF5hYfHOtDKTpjRzm8iMGiGEo7E4GRk5ciTp6enMnTuXlJQUgoKCiIqKonXr1gAkJyfj4lLc4XLp0iWefvppzp49yw033EDXrl3ZsmULt912m/V+ClEnHDsG2dng6Qnt2ukdjf2RnhEhhKOqVgHr5MmTmTx5cpnPbd68ucT3b731Fm+99VZ1LiNECeYP2c6dwdVV31jsUVCQuk1JgbQ0aNxY33iEEKKq7KcCUIhKSL1IxXx8IDBQ3ZehGiGEI5FkRDgMWXm1crISqxDCEUkyIhyG9IxUTvaoEUI4IklGhEPIyoIjR9R96Rkpn/SMCCEckSQjwiH88YdaZbRpU/UlymZO1PbtU1OhhRDCEUgyIhyC1ItUTbt24OGhpkAfO6Z3NEIIUTWSjAiHIPUiVePmpqY+g9SNCCEchyQjwiFIz0jVyeJnQghHI8mIsHuaVvzBKj0jlZNl4YUQjkaSEWH3zp6F9HRwcYFOnfSOxv7J9F4hhKORZETYPfNf+DffDF5e+sbiCMzJyNGjkJmpbyxCCFEVkowIuyf1IpZp2hT8/NTw1v79ekcjhBCVk2RE2D2pF7GcDNUIIRyJJCPC7pmHaaRnpOqkiFUI4UgkGRF2zWRSq6+C9IxYQnpGhBCORJIRYdeSkiAvD+rXh5tu0jsax3FtMqJp+sYihBCVkWRE2DXzX/ZBQWpqr6iajh3B1RUuXoTTp/WORgghKib/vQu7JvUi1ePhAR06qPsyVCOEsHeSjAi7JjNpqk/qRoQQjkKSEWHXpGek+iQZEUI4CklGhN26fBlOnlT3pWfEcrJhnhDCUUgyIuyW+UM0IAAaNtQ3Fkdk7hk5dAhyc/WNRQghKiLJiLBb5mTk1lv1jcNRtWwJvr6Qnw8HD+odjRBClE+SEWG3EhLUrdSLVI/BIHUjQgjHIMmIsFvmZER6RqpPkhEhhCOQZETYpYIC2LdP3ZdkpPokGRFCOAJJRoRdOnoUsrPBywvatdM7GsclyYgQwhFIMiLsknmIJihILWsuqqdzZ1U7cvYspKbqHY0QQpRNkhFhl2QmjXXUrw9t26r75gXkhBDC3kgyIuySzKSxHnMbSjIihLBXkowIuyQzaaxH6kaEEPauWslIZGQkgYGBeHp6EhoaytatW6v0ui+//BKDwcCwYcOqc1lRR1y6BMnJ6r70jNScJCNCCHtncTKyatUqpkyZwsyZM9mzZw99+/Zl8ODBJJs/Pcpx8uRJpk2bRt++fasdrKgbzB+arVqpFURFzZiTkf371WqsQghhbyxORhYuXMj48eOZMGECHTt2ZNGiRQQEBLBkyZJyX1NQUMDo0aOZM2cObdq0qVHAwvnJEI11BQaqQtacHEhK0jsaIYQozc2Sg/Py8oiLi2P69OklHg8PD2fHjh3lvm7u3Lk0adKE8ePHV2lIJzc3l9xrdvbKyMgAwGQyYTKZLAm5QuZzWfOczkaPNoqPdwVcCAoqwGQqrLXrVkTTNJKTk9m7dy9JSUlkZmZy9epVsrKyOHLkCFFRUQQEBODv70/Lli1p2bIlbdq0wcXFPsqygoNdiY114fff82nXTqu168q/sYpJ+1RO2qhi9t4+VY3LomQkLS2NgoIC/Pz8Sjzu5+fH2bNny3zN9u3bWbZsGfHx8VW+zvz585kzZ06px6Ojo/H29rYk5CqJiYmx+jmdTW220dat/YCG5OfvJirqTK1d91qFhYUcPnyYX3/9laSkJE6cOEFWVpZF5/Dx8SEoKIjg4GCCg4Np2bIlBoPBRhFXzNe3CxDImjXHadDgj1q/vvwbq5i0T+WkjSpmr+2TnZ1dpeMsSkbMrv8PVdO0Mv+TvXLlCo899hgffvghjRs3rvL5Z8yYQURERNH3GRkZBAQEEB4eToMGDaoTcplMJhMxMTEMHDgQo9FotfM6k9puo4ICGDVKvS3Hjg3hlltCbH5Ns8LCQrZt28bq1av59ttvOXOmZCJkNBrp0KEDnTp1wtfXFy8vL9zd3Tl9+jT+/v6kpKRw+vRp/vzzT5KTk7ly5QqxsbHExsYCEBAQwJgxYxg3bhytWrWqtZ8L4MwZA+vWQWZmW4YMuanWriv/xiom7VM5aaOK2Xv7mEc2KmNRMtK4cWNcXV1L9YKkpqaW6i0BOHr0KCdOnODee+8teqywUHW7u7m5cejQIdqaV2S6hoeHBx4eHqUeNxqNNmlsW53XmdRWGx07Blevgrc3dOhgrJXVV3Nzc/n000958803OXz4cNHjPj4+3HfffYSHhxMSEkKHDh1wd3cv8VqTyURUVBRDhgwp0T4mk4ldu3bx888/8/PPP7N9+3ZOnTrFvHnzeOONNxg0aBBPPfUU9957L25u1fqbwCKhoep2714XjMbaHzqSf2MVk/apnLRRxey1faoak0X/C7q7uxMaGkpMTAzDhw8vejwmJob777+/1PEdOnQg8bqVlmbNmsWVK1dYvHgxAQEBllxe1AG1uQx8RkYG77//Pm+99RYpKSkA3HDDDQwfPpwRI0YwYMCAMpPiqjAajfTu3ZvevXszc+ZMcnJyWLt2LR988AGbNm1i3bp1rFu3jltuuYXXX3+dESNG2HQIJygIXFzg3Dm1NHyzZja7lBBCWMziP8kiIiJ4/PHHCQsLo1evXnzwwQckJyczadIkAMaMGYO/vz/z58/H09OToKCgEq/3/Wuu5vWPCwG1M5OmoKCA//znP7z88stcvHgRAH9/fyIiInjqqafw8fGx+jU9PT0ZOXIkI0eO5MiRIyxdupRly5Zx+PBhHn74YUJDQ1mwYAEDBgyw+rVB9TTdcgscPAjx8TBokE0uI4QQ1WJxf+3IkSNZtGgRc+fOJSQkhC1bthAVFUXr1q0BSE5OLvorUwhL2XpPmtjYWLp3785zzz3HxYsX6dixIytWrODYsWNERETYJBG5Xrt27ViwYAHHjh3j1VdfpX79+sTFxTFw4EAGDhxIko3m34b8VX5jTviEEMJeVGvwePLkyZw4cYLc3Fzi4uLo169f0XObN2/mo48+Kve1H330Ed9++211LivqAFvtSXP+/HmefPJJevfuzZ49e/D19SUyMpLExESeeOKJUrUgtcHHx4fZs2dz9OhRXnjhBYxGIxs3buTWW29l0aJFFBQUWPV65mTEgoltQghRK+xjEQQhgAsX4NQpdd+ayciGDRsICgpixYoVADz55JMcPnyYZ555BtfaqJCtRNOmTVm0aBEHDx7krrvu4urVq0ydOpU77rijREFtTZl7myQZEULYG0lGhN0w1zrfdBPccEPNz5eXl8c//vEPBg0aRGpqKp07d2bHjh0sW7aMJk2a1PwCVtamTRtiYmL4z3/+Q/369dm+fTu33nor7777LppW84XKzD0jhw9DFaf+CyFErZBkRNgNaw7RHDlyhD59+vB///d/gBpa3LVrF7169ar5yW3IYDAwceJE9u3bx4ABA8jJyeH555/nscceq/LiQeVp1gz8/KCwEPbts1LAQghhBZKMCLthrZk03333HV27duX333+nYcOGrF69mvfeew8vL6+aB1lLWrduTXR0NG+99Raurq58/vnn9OrVi6NHj9bovDJUI4SwR5KMCLuxZ4+67dq1eq/XNI1///vfDBs2jMzMTPr27UtCQkKJNXEcicFgYMqUKfz00080bdqUvXv3EhYWRlRUVLXPKUWsQgh7JMmIsAt5ecVDB9VJRkwmExMnTmTatGlomsbEiRP56aefnGJhvTvuuIPdu3fTs2dPLl26xD333MOiRYuqdS5JRoQQ9kiSEWEX9u8HkwkaNoS/lqypsosXLzJo0CA+/PBDDAYDb731FkuWLLHLpZGry9/fn82bNzNx4kQ0TWPq1KlMnz7d4sJWczKyd6+qHRFCCHsgyYiwC+YhmpAQsGRV9DNnznD77bezadMm6tWrx3fffceUKVN02x3Xljw8PFiyZAlvvPEGAP/617944oknLNo6/OabwdMTsrKghuUnQghhNZKMCLtQnXqRkydP0q9fP/744w/8/f3Zvn0799xzj20CtBMGg4EZM2awYsUKXF1d+eSTT7j//vvJysqq0uvd3CA4WN2XoRohhL2QZETYBUuTkaSkJPr27cvRo0cJDAxk69at3GrLDW3szBNPPMHatWvx8vJi3bp1DBgwoMpbdUvdiBDC3th+73IhKlFYWDyttyrJiHkNjnPnztGhQwc2btyIv7+/bYO0Q0OHDmXTpk0MHTqUX3/9lbvvvpsNGzbQoEGDCl9nztlkj5paUFgISUkq205OhtOni78uXsRN07gzKwu3evXU+GSjRuDvr75atFAFVN26Qbt2lo1fCuFgJBkRujtyBDIzVS1D+/YVHxsfH8+AAQNIT0+nS5cuxMTE0LRp09oJ1A717NmTjRs3ctddd/Hrr78yaNAgNmzYUOGGf9IzYkO5ufDzz/DLL7BrF/z+O1y+XO7hBqBKWzP6+kJYGHTvDnfcAf37g4eHlYIWQn+SjAjdmYdobr1V1TSU5+DBg4SHh5Oenk737t1Zv349jRo1qp0g7VjXrl3ZuHEjAwYMIDY2lsGDB7Nu3bpyExLzCrenT0NaGjRuXIvBOqNLlyAqCr79FtatU5n1tTw9VQbYrl1xr4e/P9x4I/kFBfz666/07NkTNxcXSE9Xv5gzZ9RtUpLKGi9dgo0b1df8+eDjA4MHw7Bh6tbXV5+fXQgrkWRE6K4q9SLHjx9nwIABnD9/nm7duhETE8MN1tjAxkmY22TAgAFs376dIUOGsG7dOurXr1/qWB8faNtWzaZJSIC77tIhYEenaaoH5D//gTVrID+/+LnmzVWC0KMH3HYbdO4M5Uwz10wm0q9cQevbt9xjMJnUxk27dsFvv6mE5+xZ+Oor9WU0wvDh8MwzqtdEhnOEA5ICVqG73bvVbXnJyOnTp7nrrrs4ffo0nTp1YsOGDZKIlCE0NLQoSdu2bRvDhw8nLy+vzGNlqKaaLl6ERYugY0eVxX39tUpEOnWCGTNUsvDnn7BsGTz9tGromq53YzSqupGJE2H5ctVj8uuv6nqdOqlk5auv4G9/U98vXqziFMKBSDIidKVpFfeMnD9/noEDB3L8+HHatm1LTEwMjWVcoVxhYWFER0dTr149Nm7cyNixYyksY3UzczJibntRifR0+Oc/oVUrmDoVDh2C+vVh0iSV0e3fD2+8oXpCXGz836qLi+p1eeMNdd09e1SiUq8eHDwIU6aowtdZs+DCBdvGIoSVSDIidGWuW3B1LV7/wuzKlSsMGjSIAwcO0LJlSzZu3EiLFi30CdSB3HbbbaxevRqj0ciXX37JlClTSq3U2q2buo2L0yFAR3LhgvpQDwxUtRqZmeqNumSJqutYsqTmOzvWVEiIGi46cwYiIyEoCK5cgXnzVNyvvCI9JcLuSTIidGX+y7xjR1XnZ5afn8/DDz/M7t27adKkCRs3buSmm27SJUZHFB4ezscffwzAO++8w/z580s8Hxqqbg8dUp9b4jq5ufCvf6kP83nzVCPdequqD0lIUD0iFcxY0kWDBqpuJCEBVq9WlcoZGfDaa+rnePNNtQmUEHZIkhGhq7KGaDRNY/Lkyaxfvx4vLy9+/PFH2lc251eUMmrUqKIN9WbOnMnSpUuLnvPzUxM6NE3qRkrZsEF9kE+frj7Mu3RRH+67d6vZK/ZeIOriogpa9+yBb75RPSWXL8OLL6qfJTpa7wiFKEWSEaGrspKRBQsW8OGHH+Li4sKXX35J9+7d9QnOCbzwwgvMmDEDgEmTJrFhw4ai58y9IzJU85fjx9WH+KBBcPiwytg+/li9SYcPt30tiLW5uMCDD6qekhUroGlT1RV2993q8ZMn9Y5QiCIO9q9LOJvrk5HPPvuMf/7znwC8/fbb3HfffTpF5jzmzZvHmDFjKCgo4OGHH2b//v1AcTJins1UZxUWwttvqym4336rFruJiFAJyZgxjpeEXM/FBZ54Qv08U6aoAq3Vq9XMm/fek+2bhV1w8H9lwpFduFD8x1lICPzyyy+MGzcOgGnTpvHss8/qGJ3zMBgMfPDBB/Tt25eMjAzuueceUlNTpWcE4MQJNUX3hRfg6lU1PTYhAf79b1WD4UxuuAHeekuNy91xB2Rnw3PPQXi4WqpeCB1JMiJ0Y65VaNMGLlw4xoMPPojJZOKhhx7iX//6l77BORkPDw9Wr15N27ZtOXHiBMOGDaNTpxxAzQat4qa/zkPT4IMP1MyYzZvB21vNRPnpJ9Vj4MyCgmDTJnjnHfDyUj9zUJBaw+S6WVdC1BZJRoRuzMMDwcEm7r///qJl3j/++GNcHL1r3A41btyYH374AV9fX2JjY5k1azzNm2sUFtaxItYLF+D++9XaHJmZ0Lcv7N2rZqLYe3Gqtbi4qF6RhATo3VvNFho/Hh54QC09L0Qtk//xhW7M9SKHDq1i3759NG/enDVr1uDl5aVvYE6sQ4cOfPPNN7i5ufH555/ToEESUIeGanbuVIusfP+92mju3/9Wy7q3bat3ZPq4+WbYskVN+3V3VzUz3brVoTeEsBeSjAjdmJORgwc/x8PDgzVr1uDv769vUHXAXXfdRWRkJACHDn0B1IHPHk1TRaq3364Kldq2hdhYVajq6qp3dPpydYVp02DHDrUeyfHjqrfkvfdk2EbUGklGhC6ys+HgQXMV/x6WLl1Kjx49dI2pLnnqqad45plngN8BiI3N1TcgW7pyBR5+WBWpmkxqWmtcXMU7M9ZFoaHFa6nk5alhnFGjSu9CLIQNSDIidPHVV4fRNBcghRdfHMNjjz2md0h1zqJFiwgLU/8FJCW5cfasEy7FeuIE9OmjFv8yGlXvyNdfq5klojRfXzXt96231BTnVauKe5OEsCFJRkStu3jxItOmrQKgSZOTvPHGGzpHVDe5u7vz/ffv4+JyDnBl5Mg3ytxUz2Ft3Qrdu0NiIjRrpmojnn++7hSpVpfBoNYj+eUXtVBaQoLaAHDHDr0jE05MkhFRqwoLC3n88cdJTw8EYMKELrjW9TF7HTVr1oyePd0B2LLlSqk9bBzW8uVq/ZC0NFWQuWsX9Oypd1SOpXdv1W4hIZCaqtZg+egjvaMSTkqSEVGr5s2bx48//ojBcBsA/fp56xyRGDCg4V/3Qnn55ZeJduS9SwoL4R//UNNUTSYYMUL1iLRsqXdkjqlVK9i2TS2Hn5cH48bBSy/Jqq3C6qqVjERGRhIYGIinpyehoaFs3bq13GNXr15NWFgYvr6+1KtXj5CQED799NNqBywc14YNG5g9ezZwA5p2CwBhYfrGJIqXhW/UaCCapvHoo49y6tQpfYOqjtxceOwx+L//U9/Pnq1qHurV0zcuR1evnqq5efll9f3//q9aJl92ABZWZHEysmrVKqZMmcLMmTPZs2cPffv2ZfDgwSSXs5xwo0aNmDlzJrGxsezdu5dx48Yxbty4Eht2Ced34sQJHn30UTRNY+jQVwE1i7BxY33jEmoUA+DyZX9CQnqRnp7OyJEjyXOkD5vLl2HIEPjiC1V4+ckn8Oqrjr+vjL1wcYG5c9UwjZsbfPYZDB2qZioJYQUW/0tduHAh48ePZ8KECXTs2JFFixYREBDAkiVLyjy+f//+DB8+nI4dO9K2bVteeOEFunTpwrZt22ocvHAMeXl5PPzww1y4cIHu3bvTo8dzgKotFPrz91d1igUFBmbN+pobbriB2NhYXnrpJb1Dq5ozZ6BfP7XEef368OOP8PjjekflnMaOVQvG1asHGzeqPW7OntU7KuEE3Cw5OC8vj7i4OKZPn17i8fDwcHZUodJa0zQ2bdrEoUOHKtx7JDc3l9zc4nUPMjIyADCZTJhMJktCrpD5XNY8p7OxRhv9/e9/Z9euXTRq1IgvvviCv/9d5cDduhVgMjn22LOzvIe6dXNl/XoXTp9uxvLly3nwwQdZtGgRPXr04MEHH6z2eW3ePocP4zZ0KIaTJ9H8/Mj/7ju1foiD/D4c8v1z110YNm7E9f77MezZg9arF/lRUdCunU0u55BtVIvsvX2qGpdFyUhaWhoFBQX4+fmVeNzPz4+zFWTHly9fxt/fn9zcXFxdXYmMjGTgwIHlHj9//nzmzJlT6vHo6Gi8va1f8BgTE2P1czqb6rZRbGws7777LgDPPPMM+/btY9u2FoAX+fmxREWlWzFK/Tj6e8jHpwPQnu+++5Pnn3dl+PDhrFmzhieffJKLFy/SokWLGp3fFu3jc+IEvV99FeOlS2S2aEHsK6+QnZICKSlWv5atOeL7x3vOHHrNnUv9Eyco6NOHHXPmcKVVK5tdzxHbqDbZa/tkZ2dX6TiDplV9vd8zZ87g7+/Pjh076NWrV9Hj8+bN49NPP+XgwYNlvq6wsJBjx46RmZnJTz/9xGuvvca3335L//79yzy+rJ6RgIAA0tLSaGDFbb1NJhMxMTEMHDgQo9FotfM6k5q00bFjx+jRoweXL18mIiKCBQsWcPYstGplxGDQSEvLx8fHRoHXEmd5D61da+Chh9wIDtaIi8snPz+f8PBwtm3bVjSs6unpafF5bdU+hl27cL3nHgwXL6J16aL+Mm/a1Grnry0O//45dw63IUMwJCai3Xgj+T/+WFyEZCUO30Y2Zu/tk5GRQePGjbl8+XKFn98W9Yw0btwYV1fXUr0gqamppXpLruXi4kK7v7rwQkJCOHDgAPPnzy83GfHw8MDDw6PU40aj0SaNbavzOhNL2yg3N5fRo0dz+fJlevfuzYIFCzAajUW7w3bsaKBRI+dpc0d/D5lX4v/jDwP5+Ua8vIysWrWKkJAQ9u7dy/Tp03nvvfeqfX6rts+WLap4MjMTevbEEBWFsWHDyl9nxxz2/dOyJWzeDIMHY9i5E2N4uKrZuf12q1/KYduolthr+1Q1JosKWN3d3QkNDS3VHRQTE0Pv3r2rfB5N00r0fAjnM23aNOLi4mjUqBFffvll0Rty1y71/G236RicKKVlS3MRK0UJY4sWLfjkk08ANZ3/v//9r44R/mXDBhg0SCUif/sbxMSAgyciDq9RI1XM2q8fZGTA3Xer34sQFrB4Nk1ERARLly5l+fLlHDhwgKlTp5KcnMykSZMAGDNmDDNmzCg6fv78+cTExHDs2DEOHjzIwoUL+eSTT2QvEie2Zs2aojqRTz75hICAgKLnzMmIzKSxLwZDce/Ib78VPz5o0KCiWTXjx4/n+PHjOkT3l3Xr4L774OpV1TPy449q9ozQn4+P+v0MGqR2wbz3XpU4ClFFFg3TAIwcOZL09HTmzp1LSkoKQUFBREVF0bp1awCSk5NxuWZuf1ZWFpMnT+bPP//Ey8uLDh06sHLlSkaOHGm9n0LYjeTkZJ588klA9Y4MHTq06DlNk2TEnvXooWZtXpuMALz22mts2bKF2NhYRo0axdatW2u/O3jduuLdZB94QK0n4u5euzGIinl7w7ffwsiRsHYt3H+/ur37br0jEw6gWisCTZ48mRMnTpCbm0tcXBz9+vUrem7z5s18dM3+Ba+//jpJSUlcvXqVCxcusGPHDklEnFR+fj6PPvooly5donv37sybN6/E8ydOQHq62jy1Sxd9YhTlM2/d8uuvJR83Go188cUX+Pr68ttvvzFz5szaDez6ROTLLyURsVceHvDVVyoRyc1Vt9JDIqpAlicUVjN37ly2b9+Oj48PX375Je7XfWCYe0VuvVX9nyXsS/fuarjmxAk4d67kc61bt2b58uUAvPnmm6xfv752giorEbHDIj1xDXd3SUiExSQZEVaxefNmXn/9dQDef/992rRpU+oYGaKxbw0aQKdO6v71QzUAw4cP57nn1Oq5Y8eO5dz1GYu1rV8viYijKishkaJWUQFJRkSNpaWlMXr0aDRNY9y4cYwaNarM43buVLeSjNiv8oZqzN58802Cg4NJTU1l7NixFNpq99affy7eKVYSEcdkTkiGDStOSLZs0TsqYackGRE1omkaTz75JGfOnKF9+/a88847ZR5XUABxceq+JCP2q7JkxNPTky+//BJPT082bNjAokWLrB/Ejh1qNkZOjrr94gtJRByVu7vaOXnIkOJZUGV1u4k6T5IRUSNLlizh+++/x93dnVWrVlGvnO3aDx6ErCy1v1bHjrUcpKgy8/TeXbtUAlmWTp068dZbbwEwffp0du/ebb0A4uJg8GD1Zhk4UP1lLcWqjs3dHb75Bu68U60PM2hQ8WI2QvxFkhFRbfv37+fvf/87AP/7v//LrbfeWu6x5nqRbt3A1bU2ohPV0amTWrojMxP++KP84yZOnMiwYcMwmUw88sgjZGZm1vziiYkQHq4WzurbV00TrcYS9MIOeXmpab59+sClSyrRrOgNJuocSUZEteTk5DBq1ChycnIYPHgw//M//1Ph8VK86hhcXYt/R+UN1QAYDAaWLl2Kv78/SUlJlf7+K5WUpD6gLlxQy/P+8INat0I4j/r11UJ1YWGQlgYDBsCxY3pHJeyEJCOiWl566SUSExNp2rQpK1aswGAwVHi8+YNNloG3f+a6kcqG9m+88UZWrlyJwWBgxYoV0vubdQAAIABJREFUfP3119W74OnTKhE5d04tQLN+vZraI5zPDTeoab7BwWp35YEDHXKXZWF9kowIi0VFRfH2228DsGLFigo3SQTV5Z+QoO736WPr6ERNVVbEeq3+/fszffp0AJ5++mlOnTpl2cXS0tQH0smT0K4dREfLXjPOrlEjlZC0bat6RsLDVY+YqNMkGREWOXfuHOPGjQPgf/7nfxgyZEilr/ntN1UM2aqV2pBN2LfiHXxV+UZl5syZQ1hYGJcuXeLxxx+noLzK1+tduaJmWRw4AP7+arO1ShJb4SSaN1frjjRvDvv2Fe/CLOosSUZElZnXEUlNTSU4OJh//etfVXrd9u3qVnpFHIOfH9x0U8m9hCpiNBr5/PPPqVevHr/88gtvvvlm5S/KyVHrT+zaBTfeqD6Y/trfStQRgYHq996okeqGe+ABtR6JqJMkGRFV9t5777Fu3To8PT354osv8KziTIcdO9Rt7942DE5YlSVDNQA333xz0dDdyy+/zO+//17+wQUF8OijsGmTKmpcv17me9dVnTtDVJSa8x8TA489Vv6ccuHUJBkRVbJ//36mTZsGqFU4O3fuXKXXFRRAbKy6Lz0jjsM8VFPVZARg3LhxjBgxomjDxDKn+2oaPPMMrFmj1p/47js1u0LUXT16qGnc5vVInn9evU9EnSLJiKiUyWRizJgx5ObmMmTIEJ599tkqv3b/flV3UL++KqAXjuHaGTVV/VwwGAy8//77tGzZkqSkJKZOnVr6oNmz4cMPwcVFraz6t79ZL2jhuAYMgJUr1U6NS5bAa6/pHZGoZZKMiEp9+umnJCYm0qRJE5YvX17pNN5rmetFevYENzcbBSisrmtX9Yfq+fNw/HjVX9eoUSM+/fTTonVIvv322+In3323+ENmyRJVIyCE2UMPqfcIqKT1P//RNx5RqyQZERXauHEj3333HQDLly+vdBrv9aRexDF5eEBIiLpvyVANqOm+L774IgCTJk3iwoULGL7+GswLo82dC08/bcVohdOYPBleeaX4/n//q288otZIMiLKlZaWxvjx4wH1oXLPPfdYfA6ZSeO4qrr4WVnmzp1Lt27duHDhAjtefx3XJ55Q4z3PPguzZlkzTOFsXn0VJk5U75dHH8Xwyy96RyRqgSQjokyapjFhwgRSUlJo2bIlCxYssPgcKSmqi9/FpfiDTTgO8+9s2zbLX+vu7s5nn31GDw8P/n3sGAaTSXXDL16s6gKEKI/BAO+9p4bx8vJwHTECnxMn9I5K2JgkI6JMS5cuZe3atRiNRiIiIvCuxj4h5l6R4GBZ3dsR9e2rbuPjq7b42fU6eHryk4cHDYDNBgOJ//iH7JIoqsbVFT77DPr2xXD5Mr3mzoXkZL2jEjYkyYgo5fDhw0yZMgWA1157jTZt2lTrPFIv4thatoQ2baCwsPh3WWVpaXD33dTLyCDJy4v7NY1R48Zx9epVm8QqnJCnJ6xdi9apE14XLuB2zz2ybLwTk2RElJCXl8ejjz5KdnY2d955Z1FSUh1SL+L4+vVTtxYN22dlwT33wOHDaK1aEf/GG3j5+bF//35eeuklm8QpnFTDhuT/8ANXb7wRw8GDcO+9IAmtU5JkRJTw6quvEhcXR8OGDfnkk09wcaneWyQ7G3bvVvclGXFcd9yhbrdsqeIL8vPhkUdU1WvDhuR//z3ugYEsXboUgHfeeYeoqCjbBCucU8uWxM6ejebrq7roRo2SVVqdkCQjosgvv/xSVKj64Ycf4u/vX+1z7dqlPpdatJAtRxyZuWdk1y6VYFZI09R0zB9+UF3s339ftMz73XffzQsvvAColVrPnTtnw6iFs7nSqhUFa9aoOedr16pp4rJKq1ORZEQAcPHiRR577LGizfAefPDBGp3v2noRmTzhuAID1Ya6JlMV1ht5/fWSq6te1yW2YMECgoODSU1N5cknn0STDxNhAa1PH1XUajBAZCRUY4afsF+SjAg0TWPixIn8+eeftGvXrmjDs5qQehHnYDBUcahmxYrixarefVftyHsdT09PPv/8czw8PIiKiiIyMtL6AQvn9uCDano4wD//CZ98om88wmokGRF89NFHfP3117i5ufH5559Tv379Gp3v2tkXkow4PvNQTbnJyLp18NRT6v6MGWojvHIEBQXx5ptvAjBt2jT2799vxUhFnfD88/CPf6j748dDdLS+8QirkGSkjktKSuL5558H1KqZ3bt3r/E5Dx6EixfBy6t4SXHhuMzJSGws5OVd92RcnFrMrKAAxoyBefMqPd9zzz3H4MGDycnJYdSoUeTk5Fg/aOHcFixQhaz5+aq3xFwtLxyWJCN1mMlkYvTo0WRlZXHHHXcU7SdSU5s3q9uePcFotMophY46dIAmTSAnRxWyFjl2DIYMUVN5Bw5U9SJVKBAyGAysWLGCpk2bkpiYKNN9heVcXNTQ4J13QmYmDB0KskqrQ5NkpA6bPXs2u3btomHDhnz66ae4Wml1zJ9+Urd33WWV0wmdGQxlDNWkpcGgQZCaqrq//vtftc1vFfn5+bFixQoA3n77bX788UcrRy2cnocHrF4NXbrA2bPq/ZierndUopokGamjfv7556JpvB988AEBAQFWOW9hIfz8s7ovyYjzKJGMZGerxaeSktS87ago8PGx+JxDhgwpMd337NmzVoxY1Ak33KDefwEBcOgQ3HefLIrmoCQZqYPS0tKKpvGOHz+eESNGWO3c8fGqXsTHB8LCrHZaobOiupFtBRSOelTN823YUBWvNm9e7fMuWLCALl26cP78ecaOHUthYaGVIhZ1hr+/eh+aF0UbPVoWRXNA1UpGIiMjCQwMxNPTk9DQULZu3VrusR9++CF9+/alYcOGNGzYkAEDBrBz585qByxqxpyAnDlzhvbt27PYPE3OSjZtUrf9+oGbm1VPLXQUHAy+N2i8kfk8Lt+tVV3k1yxqVl2enp588cUXeHp6Eh0dbfX3o6gjOneGb79VQ4Vr1sCUKbIomoOxOBlZtWoVU6ZMYebMmezZs+f/27vz8Biv9oHj35nJZkkkErtYi8RSIql9qyUIWnuo4qUbWmsXS6uWt2qp1vLrS4uqXdLaiiJCBbELaW0VNEQJaksi6yTz/P44klBEEpM8z0zO57qea5aMmTvHzOR+znIfmjdvTseOHYl6xo6KISEh9O3blz179nDo0CEqVKiAr68v165de+HgpZxbsGABmzdvxs7OjoCAAIoUKWLW509PRuQQjXUxGGBe6ekMYyGKTieKT5lp3XbNmjWZM2cOAGPHjiUsLMwszysVMC1bwsqV4vq338LDJeSSZchxMvLNN9/w1ltv8fbbb+Pp6cncuXNxd3dn4cKFT3386tWrGTZsGPXq1cPDw4PFixdjMpnYnT7LUco3f/zxBx9++CEAs2bNop6Z192mpGROcGzd2qxPLalt2TIGnP8UgMW154vllGb03nvv0bVrV4xGI3369CEuLs6szy8VEL17w8PElrFjM5MTSfNy1JGekpJCWFgY48aNe+x+X19fDmZzj/GEhASMRiPFixd/5mOSk5NJTk7OuB0bGwuIpahGozEnIWcp/bnM+ZxalZCQQJ8+fUhOTsbPz4+hQ4dm6/fOSRsdOqQjPt4GNzcFD49UCkCzFoj3kC4oCMPbb6MDZjCWr669z8BkI9nZQzEn7fPdd98RFhbGxYsXGTp0aMZqG2tWEN4/LyrHbfT+++ijojDMmYMyeDBpbm4obdvmYYTq0vp7KLtx6ZQcbBBx/fp1ypUrx4EDB2jSpEnG/V9++SXLly/n/Pnzz32O999/n6CgIE6fPo2Dg8NTHzN58mSmTJnyxP1r1qyhcOHC2Q1XesSCBQvYuXMnLi4uzJ07l2LFipn9NQIDq7N2rSdNmlzjk0+Om/35pfznfPEiTT/7DJukJK60aIXnkSASk+2YO3cPlSrFmv31zp49y2effYbJZGLkyJG8+uqrZn8NqQAwmfCeM4fy+/eT6uBA6JdfElOlitpRFUgJCQm88cYbxMTE4OTk9MzH5WqKoe5fhY0URXnivqeZNWsWa9euJSQk5JmJCMD48eMZM2ZMxu3Y2Fjc3d3x9fXN8pfJKaPRSHBwMO3atcPWiqtz/fTTT+zcuROdTseaNWtok4MJHTlpo2++EXVK+vUrjZ+f3wvFbCms+j106RI2776LLikJU5s2lP3lF5p3t2HnTkhJaYGf3/NXvuS0ffz8/EhOTmbKlCksWbKEd955h+rVq5vjt9Ekq37/mEmu26hdO0yvvYbNnj20nDmT1H37xM6PVkbr76H0kY3nyVEy4ubmhsFgeKIewK1btyhVqlSW/3b27Nl8+eWX7Nq1i5dffjnLx9rb22Nvb//E/ba2tnnS2Hn1vFpw6dIlhj7cK2TChAl06NAhV8/zvDZKSMjc1bVdOwO2tuYpoGYprO49dPMmdO6cUdRMv2ED+iJF6NBBbAWya5eBjz/O/v9xTtpn4sSJ7N27l5CQEPr378+hQ4ee+n1gTazu/ZMHctxGtrZiZU2LFuj++APbzp3FDp4lSuRdkCrS6nsouzHlaAKrnZ0d3t7eBAcHP3Z/cHDwY8M2//bVV1/x3//+lx07duAji0/km+TkZPz9/YmLi6NZs2ZMnjw5z17rwAExgbV8eXjppTx7GSk/xMWJ8tqXLokzye3b4WGPZPv24iF794oENC8YDAZWrVqFq6srJ0+eNNs2BVIBVKyYeP9WrCiK9HXqJMrHS5qT49U0Y8aMYcmSJSxdupRz584xevRooqKiGDJkCAADBgxg/PjxGY+fNWsWn332GUuXLqVSpUrcuHGDGzdu8EC+IfLcuHHjCAsLo3jx4qxZswabPCz88WgJ+GyM2ElalZICPXuKDfDc3GDHDihdOuPHnp6i2GVycha7+JpBuXLlWL58OSDKxW/cuDHvXkyybmXLQlAQuLqKzZV696ZAzK63MDlORvz9/Zk7dy5Tp06lXr167Nu3j23btlGxYkUAoqKiiI6Oznj8ggULSElJoWfPnpQpUybjmD17tvl+C+kJmzdvZu7cuQAsW7bMbOXenyW9vohc0mvBTCYYPFiMwxQpIsps/2u+hk6X2TsSFJS34XTq1ImPH24VP2jQICIjI/P2BSXrVaMGbN0qthLfvh3eflsWRdOYXJ0qDxs2jGHDhj31ZyHpW7Y+dFnupJjvoqKiGDRoEACjRo2iS5cuefp69++LE2mQyYhFGztWFDOzsYF16+CVV576sPbtYcmSvE9GAKZNm0ZoaCiHDh3C39+f0NBQ7HKwIZ8kZWjUCH7+GV5/HVasENsYPNyfS1Kf3JvGyqSkpNC7d2/u3r2Lj49PxmZ4eWnvXnFSXb26mDMiWaCvvoL03sqlS8UOqM/Qpo3Ywf3cOXhG4WWzsbW1JSAggOLFi3Ps2DE5f0R6MZ06weLF4vrMmfDNN+rGI2WQyYiV+eSTTzhy5AjOzs789NNP+bIKQZaAt3A//gjpf+Rnz4b+/bN8uIuLOMmE/OkdqVChQsb8kXnz5sn5I9KLGTQos0fkww9FL4mkOpmMWJF169ZlbDS2YsUKKufTmvqdO8WlHKKxQJs3wzvviOuffCK+nLMhv+aNpOvcufNj80cuXbqUPy8sWadPPoH0WlaDB8Ovv6objySTEWtx8eJFBg8eDIjekbyeJ5L5uvDnn2KaQbt2+fKSkrns2wf+/mK79UfPFrMhPRnZtQtSU/Movn+ZNm0aTZo0ISYmhp49e5KYmJg/LyxZH51ODE327y/e/716ifoEkmpkMmIFEhMT6dmzJ3FxcTRv3pxp06bl22tv3SouW7QQS/olC/H77/Daa5CUJC4XLcrRmmwfHyheHGJi4OjRPIzzEba2tgQGBlKiRAnCw8P54IMP8ueFJeuk18MPP4h5JImJosjfqVNqR1VgyWTEwimKwgcffMDvv/9OyZIlCQgIyNN6Iv+2ZYu47Nw5315SelEREeDrKzKJ5s0hIEB0beWAwZDZE7ZjRx7E+Azly5dn7dq16PV6li5dyg8//JB/Ly5ZH1tb+OknaNJELAv09RXdvVK+k8mIhVu8eDFLly5Fr9ezZs0aypYtm2+vHROTWfgqn0aFpBd19arIIh6WeWfzZlF7IRfye95IujZt2vDFF18AYuPNEydO5G8AknUpXFh08b78Mty4IT4f166pHVWBI5MRC3bkyBGGDx8OiJ2Tc7IBnjns2CHmC3h4yBLwFuHWLfFFGxUl1mEHBYGzc66fztdXXB47BnfumCnGbBo7diydO3cmOTmZHj16cPfu3fwNQLIuLi5iJv5LL8Hly+Jzcvu22lEVKDIZsVC3bt2iR48epKSk0KNHD1XqL6QP0cheEQsQEyNqh5w/L+q5BwdDyZIv9JTlykGdOqKQ5a5dZoozm/R6fcaKscuXL/Pmm2+SlpaWv0FI1qVUKfFGLl9eFNHp2BGyueOs9OJkMmKBUlNT8ff359q1a3h4ePDjjz+iy+cNYVJTRVVlkMmI5sXHi0k9J0+KHUt37YIKFczy1OlDNfk5bySdi4sL69evx8HBge3btzNp0qT8D0KyLhUrikTdzQ2OHxeTu/NqR0jpMTIZsUDjxo0jJCQER0dHNm7ciKOjY77HcOgQ3L0rejcbN873l5eyKykJunaF0FCx3Cko6In9Zl5EeqHWbdvECsn85uXlxZIlSwCx9Hf9+vX5H4RkXTw8xOfEyUmUl+7eXewMKeUpmYxYmDVr1vD1118DYgM8Dw8PVeJIH6Lx88vxQgwpv6TvwLtrV+bGd15eZn2JFi1EQnrrlnplGvr168eYhwWsBg4cyOnTp9UJRLIe9euLQmiFC4vExN9f7vSbx2QyYkGOHTvGW2+9BcD48ePp3r27arHI+SIal5oK/fqJL1QHB7FaoEkTs7+Mra3YdwxAzU6JmTNn0qZNG+Lj4+natSv37t1TLxjJOjRrJlab2dvDL79kFkiT8oRMRixEdHQ0Xbt2JSkpic6dO2csbVTDo1VXs9hPTVKLySQqqq5bB3Z2sGkTtGqVZy/Xo4e4XL9evLQabGxsCAwMpFKlSly6dIm+ffvKCa3Si2vTBjZsEFl3YCC8/bZ6b3IrJ5MRC5CUlES3bt24fv06np6erF69Gr1evf86WXVVw0wmePddWLVKVCb76afMWaZ5pF07cHQUpRnyqxrr07i6urJp0yYKFSpEUFBQxl42kvRC/PxEYUCDAZYtg6FDZUKSB2QyonGKovDuu+9y5MgRXFxc2Lx5M05OTqrGJKuuapTJBO+9J0pc6/UiIUkfQ8lD9vaZ7wW154/WrVs3Y4ffOXPmsDh9u3hJehHdu4vdffV6sXXC+++LNe2S2chkROO+/vprVq5cicFg4Oeff+YllauLyaqrGmUyiTO2JUvEF+bKldCnT769fM+e4nLdOvW/o3v16sXUqVMBGDZsGHv27FE3IMk6vPGG6BnR6eC77+CDD9R/s1sRmYxo2KZNmzKKmX3zzTf5XmH1aX79VVZd1RxFEWdq6ZvdLV8uvjjzUYcOYuHB5cuinInaPvvsM/r27Utqaio9evTgwoULaockWYP+/eHHH8XnbMECGDFCJiRmIpMRjTp+/DhvvPEGiqIwdOjQjLLvalu7VlymT1qUVKYo4gztu+/EF+SyZfDmm/keRuHComAlqD9UA6DT6fjhhx9o2LAh9+7do3PnznKFjWQeAweKoVCdDr79FkaNkgmJGchkRIOuXLlCly5dSExMpEOHDsyfPz/fK6w+ze3bmZU2+/VTNxYJMTQzZIg4Q9PpxBnbgAGqhZOeoGphqAagUKFCbNq0CXd3dyIiIujZsycpKSlqhyVZg0GDIH0+0vz5omdSTmp9ITIZ0ZiYmBg6d+7MjRs3qFOnDoGBgdhopKrYzz+LIRovL/D0VDuaAi4tDQYPzhyaWbpUnLGpqFMnMZk1IgLOnFE1lAylS5dmy5YtFC1alN9++423334bRQuZkmT53npLzNHS6WDhQrGKTS4nzzWZjGiI0Wikd+/enD59mjJlyvDrr7+qvnLmUWvWiEvZK6Ky1FQxdr18uVhuuGoV/Oc/akeFk1PmTr5aGKpJV7duXdatW4fBYGDlypVMnDhR7ZAka/HWW+JzqNeLoZtBg8TnU8oxmYxohKIovPPOO+zcuZPChQuzZcsW3N3d1Q4rw5UrYnsTnS5fF2lI/5aSIv4D1q4VVecCA/N9smpWHi2ApiXt27fPWOY7bdo0Fi1apHJEktXo3198Hg0GsYqtXz9ZOj4XZDKiERMmTGD58uUYDAYCAgLw9vZWO6THBASIt0qrVmLreEkFiYmi3sH69aKy6oYNmptJ/NprIkc6dUoM12jJoEGDMnb2HTp0KFvTq/dJ0ovq3VtMlrK1FYUGe/YUm1RK2SaTEQ2YN28eM2bMAGDRokV00WABj/RkRA7RqCQmRqyfTd9r5pdfNFnoxcVFVNCGzGE9LZk0aRKDBg3CZDLh7+/PkSNH1A5JshZdu4qtFxwcxJ42fn4QF6d2VBZDJiMqCwgIYNSoUYDoPh48eLDKET3p8mUnzpzRYWenuRPxguGff6B1a1FtzskJdu7U9KZA6fNof/xRe/P5dDod33//Pb6+viQkJODn58cZrcy2lSyfn59YcujoCHv2iM/t7dtqR2URZDKiouDgYAY8XIo5fPhwxo8fr3JET7d3b3lArJZwdlY5mILm6lVo3hxOnIASJSAkRNzWsG7dRA9JVBTs3q3+kvR/s7W1Zf369TRs2JC7d+/i6+tLZGSk2mFJ1qJlS5GIuLrC8eNiE69r19SOSvNkMqKSgwcP0q1bt4wVNHPnztVELZF/M5lg/34xSUQO0eSzc+fENubnz4O7u5hB7OWldlTP5eCQWXftxx+1+RVTtGhRtm3bRu3atbl+/Trt2rXjxo0baoclWQtvb9i/H8qXF5/jpk3F51h6Jm1+U1i5sLAwOnbsSHx8PO3atWPFihWq7sKblQMHdNy+XZhixRQ6dVI7mgLkwAHxBRYVBTVqiNvVq6sdVba99Za43LxZR2ysnbrBPEPx4sUJCgqicuXKXLp0ifbt28sqrZL5eHqKE4hq1cRyxKZN4dAhtaPSLG3+BbRip0+fxtfXl9jYWFq0aMGmTZuwt7dXO6xnWrtW9NZ066bg4KByMAXFxo3Qti3cuweNGokvNA0t886OunXFyaHRqCMkpLza4TxT2bJlCQ4OpnTp0vzxxx/4+fkRJycdSuZSsaI4kWjQAO7cEXNIfvlF7ag0SSYj+SgiIoK2bdty9+5dGjRowNatWylcuLDaYT1TfDz8/LN4i/TpI0sd54tvvxWzhJOSxDrZ3bvBzU3tqHLl7bfF5a5dFTVRHv5Zqlatys6dO3FxceHw4cN07NhRJiSS+ZQoAb/9JibdJSWJ5fnffad2VJqTq2RkwYIFVK5cGQcHB7y9vdm/f/8zH3vmzBl69OhBpUqV0Ol0zJ07N9fBWrLIyEjatGnDzZs3qVu3Ljt27MDR0VHtsLK0ahXExOgoXfoBrVpp+K+JNTCZYOxYGD5cbOzy3nuinoiGk9Xn6dsXChVSiIpy4tgx7c2HelSdOnUIDg7G2dmZAwcO0KlTJx48eKB2WJK1KFJELPt95x3xWR86FMaPl/vZPCLHyUhgYCCjRo3i008/5eTJkzRv3pyOHTsSFRX11McnJCRQpUoVZsyYQenSpV84YEt08eJFWrZsyd9//42np2fGWZiWKQr83/+J635+kWh0Sot1iI8XRZJmzRK3//tfsdeFRvYkyq1ixaB7d5HEanUi66O8vb3ZuXMnxYoVY//+/XTu3Jn4+Hi1w5KshY0NfP89TJkibs+YIYqlJSSoG5dG5Pgb4ptvvuGtt97i7bffxtPTk7lz5+Lu7s7ChQuf+vhXXnmFr776ij59+mh6bkReOX/+PC1btuTq1at4eHiwe/duSpYsqXZYz7Vnj9jsrEgRhTZtnp5oSmZw7ZpY+rdxo6iqunIlfPaZqLtvBQYNEmd+gYE6LKGj4ZVXXiEoKAgnJyf27t1Lly5dSJB/LCRz0eng88/Ffja2tqL3s2VLiI5WOzLV5ejUKyUlhbCwMMaNG/fY/b6+vhw8eNBsQSUnJ5OcnJxxOzY2FhAbyRnNWPM//bnM+ZyPOnv2LB06dODGjRvUrFmToKAg3Nzc8uz1zGnePAOgp1+/VIoUSbWImNXwQu+hkyex6dYN3fXrKG5upK1bh9KkiVXta9GokZEyZZKIji5KQEAqAwdqf7ivfv36bN26FT8/P/bs2UP79u3ZtGlTnmxamdffQdbAKtuob1907u4YevVCd/w4SoMGpG7YAPXq5fiptN4+2Y0rR8nI7du3SUtLo1SpUo/dX6pUKbOu0Z8+fTpT0ruyHpG+iZy5BQcHm/05L1++zKRJk4iJiaFSpUqMHTuWsLAws79OXrh5sxBbt7YDoE6dfUDetJE1yWn7lD1wAK/589ElJxPr7s6RTz8l4f592LYtjyJUT9u21Vi5siazZ8dSosSz55dpzWeffcbUqVMJDQ2lUaNGfP7553m2i7b8fD2fNbZR4S++oNG0aTj+/Te6Fi04MWIE0U2a5Oq5tNo+2e1ZzNWg9L+LcymKYtaCXePHj2fMmDEZt2NjY3F3d8fX19esXwZGo5Hg4GDatWuHra2t2Z736NGjDB48mJiYGOrVq8f27dtxdXU12/PntXHj9JhMOtq0MTF4cOM8aSNrkeP3UFoa+s8/x/DVVwCYfH0ptHo1rYoVy+NI1WE0Grl7dx8BAZ6cP1+c4sU70aiR9ntHAPz8/Hj11Vfx8/Pj4sWLzJgxg+3bt1OmTBmzvUZefQdZE6tvox49MPXti83u3TSYNYu0sWMxTZ4sdgHOBq23T/rIxvPkKBlxc3PDYDA80Qty69atJ3pLXoS9vf1T55fY2trmSWOb83mDgoLo3r07CQkJNGjQgB07dmh+surE4E9CAAAgAElEQVSjEhLEniIAI0fqM9olr9reWmSrfe7ehTfegKAgcfvjj9F/+SV6C5+o+jzFiyfTr5/CsmU6Zs+2sagyCw0aNGDfvn20a9eOs2fP0qZNG3bt2kXFihXN+jry8/V8VttGJUqI/Ww++QTmzMEwcyaGP/4QO03mYP8NrbZPdmPK0QRWOzs7vL29n+gOCg4Opkkuu5asydq1azMmvPn6+rJ7926LSkQAVq8WtbYqVxZ7PklmcuoUvPKKSEQKFYK1a8XqGStPRNJ9+GEaOp3YzPT0abWjyZmaNWuyf/9+KlWqxMWLF2natCmnLe2XkLTNxga++UbUU3BwgO3bxfdFAXqf5Xg1zZgxY1iyZAlLly7l3LlzjB49mqioKIYMGQLAgAEDHtvwLSUlhfDwcMLDw0lJSeHatWuEh4dz8eJF8/0WGvB///d/9OvXD6PRSJ8+fdiyZQtFixZVO6wceXQ57/vvZ7uXUHqeZcugYUP46y+oVEmUhO7TR+2o8lWNGpk7Ps+cqW4suVGlShVCQ0Px9PTk2rVrNGvWjD179qgdlmRt+vWDgwdF5daLF8X3xooVakeVL3KcjPj7+zN37lymTp1KvXr12LdvH9u2bcvotoyKiiL6kWVK169fx8vLCy8vL6Kjo5k9ezZeXl68nV6e0cKZTCbGjRvHiBEjUBSFDz74gNWrV2Nnp839OLKyd684gS9cGAYPVjsaKxAfD//5DwwaBImJ4OsrdvGsW1ftyFSRvghv7VqwxE1yy5UrR2hoKM2aNSMmJob27duzZs0atcOSrI2Xl/ieaNtWjJsPHCg2e7LyJea5qkQ0bNgwLl++THJyMmFhYbRo0SLjZyEhISxbtizjdqVKlVAU5YkjJCTkRWNXXUJCAr169WLmw1O9qVOnMn/+fM1uepcVRcmsxTNggNgCXnoBZ8+K/SiWLwe9Hr74QnS9WtBEZnPz9oZ27SAtDWbPVjua3ClevDjBwcH07NkTo9FIv379mDlzJoqW691LlsfNTcwjmTJF1CZZulT0kvz5p9qR5RnL+6upEdHR0bRs2ZINGzZgZ2fHihUrmDhxollXFeWnXbsgJETU3ZowQe1oLJiiiC+OV14RCUnp0mJ/mU8/RZaxFRWwQTTRzZvqxpJbDg4OBAYGMnr0aADGjRvHe++9R0pKisqRSVbFYBAF0nbtglKlxPwRHx8x7GuFya/8dsyF33//nQYNGnD8+HFcXV3ZvXs3/fv3VzusXFOUzARk2DCL2yBWO+7cEWXd07tU27SB8HBo1UrtyDSjVStxgpeUBPPmqR1N7un1er755hvmzJmDTqdj8eLFtGnThlu3bqkdmmRtWrcW3yOvviqGfgcNEmXk79xROzKzkslIDgUGBtK0aVP+/vtvPDw8OHLkCM2aNVM7rBeyaZMYoixSJPPMVcoZ3a5dUKcObNggyjzPnClWzphxybs10Oky32P/+x/ExKgbz4saNWoUW7duxcnJidDQUF555RVOnjypdliStSldGoKD4csvxcqbdevg5ZdFr4mVkMlINhmNRsaMGUOfPn2Ij4+nbdu2HDp0iKpVq6od2gtJSxNboQCMHg0WsG2OtiQkUHvJEmz8/MT+Eh4ecPiwqBkglyM9VZcuULMmxMbCnDlqR/Pi/Pz8OHLkCNWrVycqKoqmTZsSGBiodliStTEYRCZ/6BBUrw7Xr0O7dug//hjDI9unWCqZjGTDzZs3adeuHXMefnOOHTuW7du345yDgjRatXq1mNrg4gIffqh2NBZm3z5sfHyounWruD1sGISFQf366salcXo9TJokrs+aBc/Y8NuipPeStm/fnsTERPr06cPIkSPlPBLJ/Hx84MQJeFhOwzBvHq1Gj0Z34IDKgb0YmYw8x4EDB6hfvz579+6laNGirF+/nhkzZmBjBcWqUlIy/yiMHZujYn8F24MHMHw4tGyJ7uJFEl1dSd28WYw75MHeSdaoVy+xWXFiInz8sdrRmIezszO//vorn3zyCQDz58+nWbNmXL58Wd3AJOtTpAgsXAhbt6KULUvR69cxtG4NI0eKeSUWSCYjz5CamsrUqVNp0aIF169fx8PDg2PHjtG9e3e1QzObJUvg8mUxHDl8uNrRWIidO8XckG+/BcD01lv8Nn8+SocOKgdmWXQ6mD9f9JL89JNYyWUNDAYDM2fOZPPmzbi4uHDs2DG8vLzYtGmT2qFJ1qhTJ1LDw7nSpg06RREfqjp1xPwSCyOTkaeIiori1VdfZdKkSZhMJvr168fRo0fx8PBQOzSzuX8fpk4V1ydOlCf0z3X9Ovj7Q/v2IoOrWBGCg0lbuJDUIkXUjs4i1a2b0dPMiBGQmqpuPObUpUsXTp48ScOGDbl//z7dunVj+PDh2d7BVJKyzdmZ8OHDSd26VSyFjIwUBRb79hXz2CyETEb+Zd26ddStW5fQ0FCKFi3KypUrWbVqFY6OjmqHZlYffijqPFSvDlZSDDdvpKaKNageHuIUXq+HUaNEqdq2bdWOzuJNnQrFi4vm/P57taMxr4oVK7Jv3z4+fDgZ69tvv6V+/focO3ZM5cgka6T4+opaJCNGiO+pgADxvfXtt2KlgsbJZOSh27dv07dvX3r16sX9+/dp0KAB4eHhvPnmm2qHZnY7d4qiU+mF/Sywcn3+2LtXFC8bNQri4kSBjLAwsQTEypJTtbi6iuK0IHrobt9WNx5zs7OzY/bs2ezYsYMyZcpw/vx5GjduzOTJkzEajWqHJ1kbJydx8nTsmPjuio0VY/ANGsD+/WpHl6UCn4woisLPP/9MzZo1CQgIQK/XM2HCBEJDQy1+2e7TxMXBO++I68OHQ9Om6sajSX/9JXZ1a9VKFBtydhan7QcPQr16akdndd59V5RMuHdPJCTWqH379pw+fRp/f3/S0tKYMmUKTZo04Y8//lA7NMka1a8vlgAvWADFionVNy1aiGJpGt0YqkAnIzdv3mTmzJn069ePf/75h9q1a3PkyBGmTZuGra2t2uHliXHjxFLKypVF/RzpETExYlmRp6coXqbXw9ChEBEh/mLKcu55wmAQ8+5A5HxWVMfpMcWLFycgIIC1a9fi7OzM8ePHadiwIcuXL5dzSSTzMxgyv7/ee098f/38s/h+Gz9e9JpoSIH9dk1MTKRBgwYcPnwYGxsbPv/8c8LCwvDx8VE7tDyzd69IlEGspJHzLh9KSBAFLypXFpcpKWJHt99/Fw1WooTaEVq9li1Fvqco8OablrtvTXb06dOHM2fO0KNHD9LS0ti4cSNeXl4EBQWpHZpkjUqWhO++g5MnxRYVyckwYwZUqQJffy3W12tAgU1GChUqxNChQ6lSpQoHDx5kypQp2Fnx5ImEBLFlCogkuXVrdePRhJQUsVb/pZdEj8i9e2LC19atopR77dpqR1igzJkjmvzmTejfH0wmtSPKO2XLlmXdunWsX78eV1dXIiMj6dChA/7+/ly5ckXt8CRr9PLLYsnvL79AjRpib5uPPoJq1WDRIlB5DlOBTUYAPvroI2bNmkU9K58HoChigvWlS1C+vDj5L9CSk8WHz8NDVE2NjhZLdZctE7PRO3USs3ulfFW4MAQGQqFC4jtz5ky1I8p7Xbp04dtvv2X48OHo9Xp++uknPDw8mDhxIg8ePFA7PMna6HTw2mvie27pUqhQAa5dE2eonp6i+1wlBToZsbGxsYpKqs8zdy788IMYMly6VEy4LpASEsTkhKpVxYcvMlJsZPd//wfnz8PAgXI/GZXVrJlRT46JE8HCK1xnS6FChfj6668JCwujZcuWJCUl8cUXX1C9enWWL19OmgUsy5QsjI2N2P03IkKsvilZUkzcd3VVLaQCnYwUBL/+mrnnzNdfi6kQBc7t2zBtGlSqJMolX7sGZcuKcYFLl+CDD8DeXu0opYcGDYI33hClEfr2hbt31Y4of9SrV489e/awYcMGqlSpQnR0NP/5z3+oW7cuGzZsQFEUtUOUrI29fWa3+caNqg5Ny2TEip06BX36iGGad94Rf4cLlHPnRA+Iu7vYmviff0RC8t134ixg1Cg5i1eDdDrxX/TSS3D1KnTrJjq1CgKdTke3bt04e/YsM2fOxNnZOWOyq4+PD9u2bZNJiWR+RYvC66+rGoJMRqzUrVtiq/YHD0S5jP/9r4BMg0hNhS1bwM9P9PkvWgRJSWLd/cqVmcvcZE+Ipjk6wrp1Ykhx3z5R9sUKdknPNnt7ez755BMiIyOZOHEiRYsW5cSJE3Tq1IlGjRqxYcMGOXwjWRWZjFihuDjo2hWuXBFnl+vWgZWWTcn0998wZYpYnvvaa7B9u8i+unYVk7KOHxdrRq2+IaxH3bqwbZuY2Lpjhxiysab9a7LD2dmZqVOnEhkZyccff0yhQoU4evQoPXr0oGbNmixevJjkgpSlSVZLJiNW5p9/4NVXRfG9YsVEJ4GKc5LyVlKS2C+mSxexGmbyZJGUuLqKJWsXLohx0BYtCki3kPVp2hQ2bxYdWRs3wn/+Y91Lfp/Fzc2NWbNmERkZyaeffoqzszMRERG8++67VKpUiSlTpnD9+nW1w5SkXJPJiBW5cgWaNRPbp7i5ieWRVrTRsGAyiX77d96B0qXFTrpbt4r7W7aENWvEBNWvvhKrZiSL16aNKBxpYwOrV4udfgvqCEWpUqX44osviIqK4uuvv6ZcuXLcuHGDyZMnU6FCBXr16kVISIicVyJZHJmMWIkzZ8RZZESEWDoeGir2SbIKaWlik6cRI8Rk1JYtRQnZmBhxe8IEMVk1JET05cv5IFanSxdYtUosT1+8WJSCKSirbJ7G0dGRMWPG8Ndff7F69WqaNWtGWloa69at49VXX8XT05Pp06dz9epVtUOVpGyRyYgV2LcPmjcXHQI1a4raDDVqqB3VC0pKEvM+hg0TldpatBD1QK5fF7MaBw2CPXvg8mWxbNfquoCkf/P3h7VrRVG0oCCRbJ86pXZU6rKzs+ONN95g//79hIeH895771GkSBHOnz/PhAkTqFixIm3btmXFihXEamwvEkl6lExGLFhKitjvqFUrUcm8USORmJQvr3ZkuXT1qujx6NpVzPvw8xPl2m/cEBNgBgwQk2Bu3RLV21q1kpvXFTC9e4vNkytVEquzGzUS04YkqFu3Lt999x3Xr1/nhx9+oGXLliiKwu7duxk4cCAlS5bktddeY8WKFdy/f1/tcCXpMdZfftRKnT4tFof8/ru4PXCgWL5rUWUzYmLE0EpwsNiq9fz5x39evjx07iz66Nu2BSveO0jKvnr1xOIof3/YvVtchoSIDjIXF7WjU5+TkxODBw9m8ODBREZGsmrVKlatWkVERARbtmxhy5Yt2Nra0rp1azp16oSfnx9V5fwqSWXytNLCpKTA7Nng7S0SETc3sdv9smUWkIjcvCnWGY8cCV5eULy46AX53/9EIqLXi1PdqVPFDpNRUaJnxM9PJiLSY1xdxXLfjz8WtxcuFEOTS5cWzNU2z1K5cmUmTpzIn3/+yalTp5g0aRK1a9fGaDQSFBTEiBEjeOmll6hRowajRo3i119/lcM5kipkz4iFSE0VNbumThXTJEB0GixeLBaVaE5iokgojh6FI0fE5V9/Pfm4atVEjfp27cSwi7NzvocqWSYbG7HpY4cOoqL/uXNiZ+rFi8X+Nt7eakeoHTqdjtq1a1O7dm0mT57M+fPn2bJlC9u2bWP//v1EREQQERHBvHnzMBgMeHt706pVK1q1akWjRo1wkV1OUh6TyYjGpaWJnUwnTxZlM0AkH198AYMHa6R8xj//iJmE4eEiAQkPF38Z/r3+UqeDOnXEbNsWLcQ65LJl1YlZshqtW4tewvnzxefk8GHw8YGOHUXF/3btNPI50ZAaNWpQo0YNPvroI2JjY9m1axfbt29nz549XLp0iaNHj3L06FFmPdzi28PDg8aNG9O4cWN8fHyoVasWdrK3UjIjmYxo1MWLsHw5rFghRitADMmMGwdDh4qqlPnKZBIFxSIixJDKmTOZxz//PP3flCwJDRtmHj4+sudDyhO2tmJDyD594JNPxKqb7dvFUbOmSEr69VPhc2MBnJyc6N69O927dwfg6tWrhISEsGfPHkJDQ7lw4QJ//vknf/75Jz/++CMAtra21K5dGy8vL7y8vKhVqxa1atWiRIkS6GTmJ+WCTEY05No1Uf56xQpRJySdi4v4oh0xQuzZkWcePBArWiIjMw7DpUu0Cg/H5tatrHcrq1JFzCysV0/MB6lXD8qVk6ekUr4qV04URps6VfSULF0KZ8/Cu++KqUodOoiN9zp3lpNdn8Xd3Z3+/fvTv39/AG7fvs3hw4c5fPgwhw4d4sSJE9y/f5+TJ09y8uTJx/6tq6srtWrVwsPDg5deeolq1apRrVo1qlatioODgxq/jmQhcpWMLFiwgK+++oro6Ghq1arF3Llzad68+TMfv379eiZOnMilS5eoWrUq06ZNo1u3brkO2lrcuyeSjvTFJOfOZf5MrwdfX1H++rXXRG2FXEtOFr0XN29CdLQ4rl8Xl3//LRKQq1dFQP+iB4ql37CxEUlHtWridLNWLXF4elrA7FmpIKlaFebNE0nJDz+IOdJ//SVKym/cKN7KLVuK0cImTUTHXZ4m+hbMzc2Nzp0707lzZwAUReHKlSucOHGCkydP8vvvv3PmzBkiIyO5c+cO+/btY9++fU88T5kyZahUqVLG4e7uTrly5TKOkiVLopdL9QusHCcjgYGBjBo1igULFtC0aVO+//57OnbsyNmzZ6lQocITjz906BD+/v7897//pVu3bmzcuJHevXsTGhpKw4YNzfJLaN2DB6Kj4dIlMbadPq3iypXHH6fXi0l33btD//7iLC+DoohJobGx4rh/XyQP6Zf37sGdO48f//wjanLExGQ/WCcnUcShcmWoUoW0ChU4ducO3n37YlutmtxoTrIoxYrBmDEwerT4zG3cKFafnTkjlgXv3i0ep9eL6Uz164vc2sNDHJUri8RFyqTT6TISivShHYCEhAT+/PNPzpw5Q0REBBcuXODixYtcuHCB2NhYoqOjiY6O5tChQ099XoPBQMmSJSlVqlTGUbJkSZydnYmOjsZoNFKqVClcXFxwcXHB2dmZwoULy2EhK6FTcriJQcOGDalfvz4LFy7MuM/T05OuXbsyffr0Jx7v7+9PbGws27dvz7ivQ4cOuLi4sHbt2my9ZmxsLMWKFSMmJgYnJ6echJslo9HItm3b8PPzwzabf2QVBVIS04i7ayTmtpG4Oyk8uGck7q6RezdTuHcjmZh/Uoi9La7fvZ5EYkwyDiRhTzKFSHzsKOucSPXyCVQulUDZYvHYpSaI7OXRIy5OJCAvsiGHjQ2UKCEmjJYpI46yZcVRoYIoq+7uLpKRF2yjgkS2T9a02j4XLogqrocOiSJq6SvU/s3GRpwUlCsnyt6UKycmkBcvLpYXu7qK646O4ihaNGer0LXaPuakKAq3b9/mypUrXL58OeP4+++/uXbtGteuXePGjRu52k/HxsYGZ2dnnJyccHJywtHREScnJ4oWLUrRokUpUqRIxmXhwoUpVKhQxmWhQoVwcHDIOOzt7bG3t8fOzi7jMv2wtbXVbNKj9fdQdv9+5yjnT0lJISwsjHHjxj12v6+vLwcPHnzqvzl06BCjR49+7L727dszd+7cZ75OcnLyY9tip697NxqNGI3GnIScpd/rDaJW5AmuGyagV9LQmdLQK2noTWnolVQMplT0ShoGJRUbxYgNRmxIxR4Fe8DNHEHcf3iczt7DFZ1OfOu5uECxYijOzmJSqLMzysNvxvRL3NxQSpaEUqXEY7LzYfpX+6a3tznb3ZrI9smaVtunUiV47z1xgBi1PHxYx9mzOv78U8f58zoiIiAxUceVK0/2YmbF1lahSBFwcBCHvb24tLVVsLMTnYvph06n486dBixbpsPGxoTBIHpp0g+dLvNSXFcyPsbp96Vff9rlo7J73/Pk7m9y8YeHFyB+pwoVxAFgMinEx8eTmJhAQkICCQmJJCQkkJiYSGJiIvfu3UOn05GUlPTw70MSiqKQmgq3b4sjexIfHjmn0+nR6/UYDPqM6+mHTqdDr9c9vF9c6nS6x+4X/1+Z92ceALqHP9cBuoev9+jPMu8Tl5n/RlEgPj6VokV3o9PpH3vc4/HzyHNnXn/UvHmVad7cPVft8yzZ/eznKBm5ffs2aWlplCpV6rH7S5UqxY0bN576b27cuJGjxwNMnz6dKVOmPHH/zp07KWzG6fAV/orgpaRzz39gNqTo7DDqbEk12JFmsMFkYwu2BhQHW/SFxKXJ1pY0OztMdnakPTxMdnakOjiI2/b2pNnbk1qokLgv/dLBAWPhwhgLFybNwSH73wbpPStPq++RQ8HBwS/8HNZMtk/WLKF9HBzEME39+uK2yQR37jhw504h7t4Vl3fuOBATY09cnN0jhy1JSTYYjQYAjEYdT6+2/rTPrR4ok0e/kaUpxiMz1DRHUUTntDXvGL1+/Uri4sw3+gBi+C47cjUa+u+sS1GULLuwcvr48ePHM2bMmIzbsbGxuLu74+vra9Zhmt8muDLv2GkqVamMjb0dOls9ehsDNg42GOxtsHUQ120dDNgXtcHB0RaHojY4FLWhiLMtNoUfnuYYDOh0OuwAa1t5bzQaCQ4Opl27dprsAlSbbJ+sFZz2MWE0mjLy//h4MW88OVlHUpLY99FoFBWUjUZxpKZCSkoap0+fo1q1moABk4knDkXJPNKry6bfTr/+tMtn3fcsuRglyRcmk4nIyEgqV66crxNcTSYTaWlpmExppKWlXzc99VAUEyaT8sht5YnLxw/Tw/9D0ejpj0u/DumP4+H1Z10qmEwK9+7dxdnZBb1e/9hw17+Hvp42FPbofX37tqRhQ/Mmx9mt6JujZMTNzQ2DwfBEr8atW7ee6P1IV7p06Rw9HsgYu/s3W1tbs36htR7XkKRtd/Dza23lX5Qvztxtb21k+2StILSPra2oY1KyZPb/jdGosG3bFfz8amFra8i74CyY0Whi27Zz+PlVzuc2MgDaf89mzhlppMnPWHZjylGaaWdnh7e39xNdrsHBwTRp0uSp/6Zx48ZPPH7nzp3PfLwkSZIkSQVLjodpxowZQ//+/fHx8aFx48YsWrSIqKgohgwZAsCAAQMoV65cxsqakSNH0qJFC2bOnMnrr7/OL7/8wq5duwh9tKqXJEmSJEkFVo6TEX9/f+7cucPUqVOJjo6mdu3abNu2jYoVKwIQFRX12LhekyZNCAgI4LPPPmPixIlUrVqVwMDAAlNjRJIkSZKkrOVqAuuwYcMYNmzYU38WEhLyxH09e/akZ8+euXkpSZIkSZKsnKy9K0mSJEmSqmQyIkmSJEmSqmQyIkmSJEmSqmQyIkmSJEmSqmQyIkmSJEmSqmQyIkmSJEmSqmQyIkmSJEmSqmQyIkmSJEmSqmQyIkmSJEmSqnJVgTW/pW9xnN2tiLPLaDSSkJBAbGysJnc71ALZRlmT7ZM12T5Zk+3zfLKNsqb19kn/u53+d/xZLCIZiYuLA8Dd3V3lSCRJkiRJyqm4uDiKFSv2zJ/rlOelKxpgMpm4fv06jo6O6HQ6sz1vbGws7u7uXL16FScnJ7M9rzWRbZQ12T5Zk+2TNdk+zyfbKGtabx9FUYiLi6Ns2bKPbaL7bxbRM6LX6ylfvnyePb+Tk5Mm/xO1RLZR1mT7ZE22T9Zk+zyfbKOsabl9suoRSScnsEqSJEmSpCqZjEiSJEmSpCrD5MmTJ6sdhJoMBgOtWrXCxsYiRqxUIdsoa7J9sibbJ2uyfZ5PtlHWrKF9LGICqyRJkiRJ1ksO00iSJEmSpCqZjEiSJEmSpCqZjEiSJEmSpCqZjEiSJEmSpKoCnYwsWLCAypUr4+DggLe3N/v371c7JM3Yt28fXbp0oWzZsuh0OjZt2qR2SJoyffp0XnnlFRwdHSlZsiRdu3bl/PnzaoelGQsXLuTll1/OKMTUuHFjtm/frnZYmjV9+nR0Oh2jRo1SOxRNmDx5Mjqd7rGjdOnSaoelOdeuXePNN9/E1dWVwoULU69ePcLCwtQOK1cKbDISGBjIqFGj+PTTTzl58iTNmzenY8eOREVFqR2aJsTHx1O3bl2+/fZbtUPRpL179/L+++9z+PBhgoODSU1NxdfXl/j4eLVD04Ty5cszY8YMjh8/zvHjx2ndujWvv/46Z86cUTs0zTl27BiLFi3i5ZdfVjsUTalVqxbR0dEZx6lTp9QOSVPu3btH06ZNsbW1Zfv27Zw9e5avv/4aZ2dntUPLHaWAatCggTJkyJDH7vPw8FDGjRunUkTaBSgbN25UOwxNu3XrlgIoe/fuVTsUzXJxcVGWLFmidhiaEhcXp1SrVk0JDg5WWrZsqYwcOVLtkDRh0qRJSt26ddUOQ9PGjh2rNGvWTO0wzKZA9oykpKQQFhaGr6/vY/f7+vpy8OBBlaKSLFlMTAwAxYsXVzkS7UlLSyMgIID4+HgaN26sdjia8v7779OpUyfatm2rdiiac+HCBcqWLUvlypXp06cPf/31l9ohacrmzZvx8fGhV69elCxZEi8vLxYvXqx2WLlWIJOR27dvk5aWRqlSpR67v1SpUty4cUOlqCRLpSgKY8aMoVmzZtSuXVvtcDTj1KlTFC1aFHt7e4YMGcLGjRupWbOm2mFpRkBAACdOnGD69Olqh6I5DRs2ZMWKFQQFBbF48WJu3LhBkyZNuHPnjtqhacZff/3FwoULqVatGkFBQQwZMoQRI0awYsUKtUPLFcutHWsGOp3usduKojxxnyQ9zwcffMAff/xBaGio2qFoSo0aNQgPD+f+/fusX7+egQMHsnfvXpmQAFevXmXkyJHs3LkTBwcHtcPRnI4dO2Zcr1OnDo0bN6Zq1aosX76cMWPGqBiZdphMJnx8fPjyyy8B8PLy4syZMyxcuJABAwaoHF3OFcEWJSMAAAKzSURBVMieETc3NwwGwxO9ILdu3Xqit0SSsjJ8+HA2b97Mnj17KF++vNrhaIqdnR0vvfQSPj4+TJ8+nbp16zJv3jy1w9KEsLAwbt26hbe3NzY2NtjY2LB3717mz5+PjY0NaWlpaoeoKUWKFKFOnTpcuHBB7VA0o0yZMk8k9p6enha7CKNAJiN2dnZ4e3sTHBz82P3BwcE0adJEpagkS6IoCh988AEbNmzgt99+o3LlymqHpHmKopCcnKx2GJrQpk0bTp06RXh4eMbh4+NDv379CA8Px2AwqB2ipiQnJ3Pu3DnKlCmjdiia0bRp0yfKCURERFCxYkWVInoxBXaYZsyYMfTv3x8fHx8aN27MokWLiIqKYsiQIWqHpgkPHjzg4sWLGbcjIyMJDw+nePHiVKhQQcXItOH9999nzZo1/PLLLzg6Omb0shUrVoxChQqpHJ36JkyYQMeOHXF3dycuLo6AgABCQkLYsWOH2qFpgqOj4xPzi4oUKYKrq6ucdwR89NFHdOnShQoVKnDr1i2++OILYmNjGThwoNqhacbo0aNp0qQJX375Jb179+bo0aMsWrSIRYsWqR1a7qi7mEdd//vf/5SKFSsqdnZ2Sv369eWyzEfs2bNHAZ44Bg4cqHZomvC0tgGUH3/8Ue3QNGHw4MEZn60SJUoobdq0UXbu3Kl2WJoml/Zm8vf3V8qUKaPY2toqZcuWVbp3766cOXNG7bA0Z8uWLUrt2rUVe3t7xcPDQ1m0aJHaIeWaTlEURaU8SJIkSZIkqWDOGZEkSZIkSTtkMiJJkiRJkqpkMiJJkiRJkqpkMiJJkiRJkqpkMiJJkiRJkqpkMiJJkiRJkqpkMiJJkiRJkqpkMiJJkiRJkqpkMiJJkiRJkqpkMiJJkiRJkqpkMiJJkiRJkqpkMiJJkiRJkqr+H15PyNPyPTndAAAAAElFTkSuQmCC",
      "text/plain": [
       "Figure(PyObject <Figure size 640x480 with 1 Axes>)"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "using PyPlot, Distributions\n",
    "μx = 2.\n",
    "σx = 1.\n",
    "μy = 2.\n",
    "σy = 0.5\n",
    "μz = μx+μy; σz = sqrt(σx^2 + σy^2)\n",
    "x = Normal(μx, σx)\n",
    "y = Normal(μy, σy)\n",
    "z = Normal(μz, σz)\n",
    "range_min = minimum([μx-2*σx, μy-2*σy, μz-2*σz])\n",
    "range_max = maximum([μx+2*σx, μy+2*σy, μz+2*σz])\n",
    "range_grid = range(range_min, stop=range_max, length=100)\n",
    "plot(range_grid, pdf.(x,range_grid), \"k-\")\n",
    "plot(range_grid, pdf.(y,range_grid), \"b-\")\n",
    "plot(range_grid, pdf.(z,range_grid), \"r-\")\n",
    "legend([L\"p_X\", L\"p_Y\", L\"p_Z\"])\n",
    "grid()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### PDF for the Product of Two Variables\n",
    "\n",
    "- For two continuous random **independent** variables\n",
    "$X$ and $Y$, with PDF's $p_x(x)$ and $p_y(y)$, the PDF of \n",
    "$Z = X Y $ is given by \n",
    "$$\n",
    "p_z(z) = \\int_{-\\infty}^{\\infty} p_x(x) \\,p_y(z/x)\\, \\frac{1}{|x|}\\,\\mathrm{d}x\n",
    "$$\n",
    "\n",
    "- For proof, see [https://en.wikipedia.org/wiki/Product_distribution](https://en.wikipedia.org/wiki/Product_distribution)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Generally, this integral does not lead to an analytical expression for $p_z(z)$. For example, [**the product of two independent variables that are both normally (Gaussian) distributed does not lead to a normal distribution**](https://nbviewer.jupyter.org/github/bertdv/BMLIP/blob/master/lessons/notebooks/The-Gaussian-Distribution.ipynb#product-of-gaussians).\n",
    "  - Exception: the distribution of the product of two variables that both have [log-normal distributions](https://en.wikipedia.org/wiki/Log-normal_distribution) is again a lognormal distribution.\n",
    "    - (If $X$ has a normal distribution, then $Y=\\exp(X)$ has a log-normal distribution.)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Variable Transformations\n",
    "\n",
    "- Suppose $x$ is a **discrete** random variable with probability **mass** function $P_x(x)$, and $y = h(x)$ is a one-to-one function with $x = g(y) = h^{-1}(y)$. Then\n",
    "\n",
    "$$\n",
    "P_y(y) = P_x(g(y))\\,.\n",
    "$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- **Proof**: $P_y(\\hat{y}) = P(y=\\hat{y}) = P(h(x)=\\hat{y}) = P(x=g(\\hat{y})) = P_x(g(\\hat{y})). \\,\\square$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- If $x$ is defined on a **continuous** domain, and $p_x(x)$ is a probability **density** function, then probability mass is represented by the area under a (density) curve. Let $a=g(c)$ and $b=g(d)$. Then\n",
    "$$\\begin{align*}\n",
    "P(a ≤ x ≤ b) &= \\int_a^b p_x(x)\\mathrm{d}x \\\\\n",
    "  &= \\int_{g(c)}^{g(d)} p_x(x)\\mathrm{d}x \\\\\n",
    "  &= \\int_c^d p_x(g(y))\\mathrm{d}g(y) \\\\\n",
    "  &= \\int_c^d \\underbrace{p_x(g(y)) g^\\prime(y)}_{p_y(y)}\\mathrm{d}y \\\\  \n",
    "  &= P(c ≤ y ≤ d)\n",
    "\\end{align*}$$\n",
    "\n",
    "- Equating the two probability masses leads to identificaiton of the relation \n",
    "$$p_y(y) = p_x(g(y)) g^\\prime(y)\\,,$$ \n",
    "which is also known as the [Change-of-Variable theorem](https://en.wikipedia.org/wiki/Probability_density_function#Function_of_random_variables_and_change_of_variables_in_the_probability_density_function). \n",
    "\n",
    "- If the tranformation $y = h(x)$ is not invertible, then $x=g(y)$ does not exist. In that case, you can still work out the transformation by equating equivalent probability masses in the two domains. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Example: Transformation of a Gaussian Variable\n",
    "\n",
    "- Let $p_x(x) = \\mathcal{N}(x|\\mu,\\sigma^2)$ and $y = \\frac{x-\\mu}{\\sigma}$. \n",
    "\n",
    "- **Problem**: What is $p_y(y)$? \n",
    "\n",
    "- **Solution**: Note that $h(x)$ is invertible with $x = g(y) = \\sigma y + \\mu$. The change-of-variable formula leads to\n",
    "$$\\begin{align*}\n",
    "p_y(y) &= p_x(g(y)) \\cdot g^\\prime(y) \\\\\n",
    "  &= p_x(\\sigma y + \\mu) \\cdot \\sigma \\\\\n",
    "  &= \\frac{1}{\\sigma\\sqrt(2 \\pi)} \\exp\\left( - \\frac{(\\sigma y + \\mu - \\mu)^2}{2\\sigma^2}\\right) \\cdot \\sigma \\\\\n",
    "  &=  \\frac{1}{\\sqrt(2 \\pi)} \\exp\\left( - \\frac{y^2 }{2}\\right)\\\\\n",
    "  &= \\mathcal{N}(y|0,1) \n",
    "\\end{align*}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "slide"
    }
   },
   "source": [
    "### Summary\n",
    "\n",
    "- Probabilities should be interpretated as degrees of belief, i.e., a state-of-knowledge, rather than a property of nature."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- We can do everything with only the **sum rule** and the **product rule**. In practice, **Bayes rule** and **marginalization** are often very useful for inference, i.e., for computing\n",
    "\n",
    "$$p(\\,\\text{what-we-want-to-know}\\,|\\,\\text{what-we-already-know}\\,)\\,.$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- Bayes rule $$ p(\\theta|D) = \\frac{p(D|\\theta)p(\\theta)} {p(D)} $$ is the fundamental rule for learning from data!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- For a variable $X$ with distribution $p(X)$ with mean $\\mu_x$ and variance $\\Sigma_x$, the mean and variance of the **Linear Transformation** $Z = AX +b$ is given by \n",
    "$$\\begin{align}\n",
    "\\mu_z &= A\\mu_x + b \\tag{SRG-3a}\\\\\n",
    "\\Sigma_z &= A\\,\\Sigma_x\\,A^T \\tag{SRG-3b}\n",
    "\\end{align}$$"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "slideshow": {
     "slide_type": "fragment"
    }
   },
   "source": [
    "- That's really about all you need to know about probability theory, but you need to _really_ know it, so do the [Exercises](https://nbviewer.org/github/bertdv/BMLIP/blob/master/lessons/exercises/Exercises-Probability-Theory-Review.ipynb)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "slideshow": {
     "slide_type": "skip"
    }
   },
   "outputs": [
    {
     "data": {
      "text/html": [
       "<!--\n",
       "This HTML file contains custom styles and some javascript.\n",
       "Include it a Jupyter notebook for improved rendering.\n",
       "-->\n",
       "\n",
       "<!-- Fonts -->\n",
       "<link href='http://fonts.googleapis.com/css?family=Alegreya+Sans:100,300,400,500,700,800,900,100italic,300italic,400italic,500italic,700italic,800italic,900italic' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Arvo:400,700,400italic' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=PT+Mono' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Shadows+Into+Light' rel='stylesheet' type='text/css'>\n",
       "<link href='http://fonts.googleapis.com/css?family=Nixie+One' rel='stylesheet' type='text/css'>\n",
       "\n",
       "<!-- Custom style -->\n",
       "<style>\n",
       "\n",
       "@font-face {\n",
       "    font-family: \"Computer Modern\";\n",
       "    src: url('http://mirrors.ctan.org/fonts/cm-unicode/fonts/otf/cmunss.otf');\n",
       "}\n",
       "\n",
       "#notebook_panel { /* main background */\n",
       "    background: rgb(245,245,245);\n",
       "}\n",
       "\n",
       "div.container {\n",
       "    min-width: 960px;\n",
       "}\n",
       "\n",
       "div #notebook { /* centre the content */\n",
       "    background: #fff; /* white background for content */\n",
       "    margin: auto;\n",
       "    padding-left: 0em;\n",
       "}\n",
       "\n",
       "#notebook li { /* More space between bullet points */\n",
       "    margin-top:0.8em;\n",
       "}\n",
       "\n",
       "/* draw border around running cells */\n",
       "div.cell.border-box-sizing.code_cell.running {\n",
       "    border: 1px solid #111;\n",
       "}\n",
       "\n",
       "/* Put a solid color box around each cell and its output, visually linking them*/\n",
       "div.cell.code_cell {\n",
       "    background-color: rgb(256,256,256);\n",
       "    border-radius: 0px;\n",
       "    padding: 0.5em;\n",
       "    margin-left:1em;\n",
       "    margin-top: 1em;\n",
       "}\n",
       "\n",
       "div.text_cell_render{\n",
       "    font-family: 'Alegreya Sans' sans-serif;\n",
       "    line-height: 140%;\n",
       "    font-size: 125%;\n",
       "    font-weight: 400;\n",
       "    width:800px;\n",
       "    margin-left:auto;\n",
       "    margin-right:auto;\n",
       "}\n",
       "\n",
       "\n",
       "/* Formatting for header cells */\n",
       ".text_cell_render h1 {\n",
       "    font-family: 'Nixie One', serif;\n",
       "    font-style:regular;\n",
       "    font-weight: 400;\n",
       "    font-size: 45pt;\n",
       "    line-height: 100%;\n",
       "    color: rgb(0,51,102);\n",
       "    margin-bottom: 0.5em;\n",
       "    margin-top: 0.5em;\n",
       "    display: block;\n",
       "}\n",
       "\n",
       ".text_cell_render h2 {\n",
       "    font-family: 'Nixie One', serif;\n",
       "    font-weight: 400;\n",
       "    font-size: 30pt;\n",
       "    line-height: 100%;\n",
       "    color: rgb(0,51,102);\n",
       "    margin-bottom: 0.1em;\n",
       "    margin-top: 0.3em;\n",
       "    display: block;\n",
       "}\n",
       "\n",
       ".text_cell_render h3 {\n",
       "    font-family: 'Nixie One', serif;\n",
       "    margin-top:16px;\n",
       "    font-size: 22pt;\n",
       "    font-weight: 600;\n",
       "    margin-bottom: 3px;\n",
       "    font-style: regular;\n",
       "    color: rgb(102,102,0);\n",
       "}\n",
       "\n",
       ".text_cell_render h4 {    /*Use this for captions*/\n",
       "    font-family: 'Nixie One', serif;\n",
       "    font-size: 14pt;\n",
       "    text-align: center;\n",
       "    margin-top: 0em;\n",
       "    margin-bottom: 2em;\n",
       "    font-style: regular;\n",
       "}\n",
       "\n",
       ".text_cell_render h5 {  /*Use this for small titles*/\n",
       "    font-family: 'Nixie One', sans-serif;\n",
       "    font-weight: 400;\n",
       "    font-size: 16pt;\n",
       "    color: rgb(163,0,0);\n",
       "    font-style: italic;\n",
       "    margin-bottom: .1em;\n",
       "    margin-top: 0.8em;\n",
       "    display: block;\n",
       "}\n",
       "\n",
       ".text_cell_render h6 { /*use this for copyright note*/\n",
       "    font-family: 'PT Mono', sans-serif;\n",
       "    font-weight: 300;\n",
       "    font-size: 9pt;\n",
       "    line-height: 100%;\n",
       "    color: grey;\n",
       "    margin-bottom: 1px;\n",
       "    margin-top: 1px;\n",
       "}\n",
       "\n",
       ".CodeMirror{\n",
       "    font-family: \"PT Mono\";\n",
       "    font-size: 90%;\n",
       "}\n",
       "\n",
       ".boxed { /* draw a border around a piece of text */\n",
       "  border: 1px solid blue ;\n",
       "}\n",
       "\n",
       "h4#CODE-EXAMPLE,\n",
       "h4#END-OF-CODE-EXAMPLE {\n",
       "    margin: 10px 0;\n",
       "    padding: 10px;\n",
       "    background-color: #d0f9ca !important;\n",
       "    border-top: #849f81 1px solid;\n",
       "    border-bottom: #849f81 1px solid;\n",
       "}\n",
       "\n",
       ".emphasis {\n",
       "    color: red;\n",
       "}\n",
       "\n",
       ".exercise {\n",
       "    color: green;\n",
       "}\n",
       "\n",
       ".proof {\n",
       "    color: blue;\n",
       "}\n",
       "\n",
       "code {\n",
       "  padding: 2px 4px !important;\n",
       "  font-size: 90% !important;\n",
       "  color: #222 !important;\n",
       "  background-color: #efefef !important;\n",
       "  border-radius: 2px !important;\n",
       "}\n",
       "\n",
       "/* This removes the actual style cells from the notebooks, but no in print mode\n",
       "   as they will be removed through some other method */\n",
       "@media not print {\n",
       "  .cell:nth-last-child(-n+2) {\n",
       "    display: none;\n",
       "  }\n",
       "}\n",
       "\n",
       "footer.hidden-print {\n",
       "    display: none !important;\n",
       "}\n",
       "    \n",
       "</style>\n",
       "\n",
       "<!-- MathJax styling -->\n",
       "<script>\n",
       "    MathJax.Hub.Config({\n",
       "                        TeX: {\n",
       "                           extensions: [\"AMSmath.js\"],\n",
       "                           equationNumbers: { autoNumber: \"AMS\", useLabelIds: true}\n",
       "                           },\n",
       "                tex2jax: {\n",
       "                    inlineMath: [ ['$','$'], [\"\\\\(\",\"\\\\)\"] ],\n",
       "                    displayMath: [ ['$$','$$'], [\"\\\\[\",\"\\\\]\"] ]\n",
       "                },\n",
       "                displayAlign: 'center', // Change this to 'center' to center equations.\n",
       "                \"HTML-CSS\": {\n",
       "                    styles: {'.MathJax_Display': {\"margin\": 4}}\n",
       "                }\n",
       "        });\n",
       "</script>\n"
      ]
     },
     "metadata": {},
     "output_type": "display_data"
    }
   ],
   "source": [
    "open(\"../../styles/aipstyle.html\") do f\n",
    "    display(\"text/html\", read(f,String))\n",
    "end"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": []
  }
 ],
 "metadata": {
  "@webio": {
   "lastCommId": null,
   "lastKernelId": null
  },
  "anaconda-cloud": {},
  "celltoolbar": "Slideshow",
  "kernelspec": {
   "display_name": "Julia 1.6.3",
   "language": "julia",
   "name": "julia-1.6"
  },
  "language_info": {
   "file_extension": ".jl",
   "mimetype": "application/julia",
   "name": "julia",
   "version": "1.6.3"
  },
  "widgets": {
   "state": {
    "0c9c7079-a918-4ed4-ad45-8de3d7874019": {
     "views": [
      {
       "cell_index": 12
      }
     ]
    },
    "261fa34c-df0a-4604-a2ff-35a9cb4d9fb2": {
     "views": [
      {
       "cell_index": 20
      }
     ]
    },
    "42bb27af-6a2c-4a24-bcd3-1d8b8aec035c": {
     "views": [
      {
       "cell_index": 20
      }
     ]
    },
    "50e83b63-be9e-4e83-987a-8b3ee9bbabd6": {
     "views": [
      {
       "cell_index": 20
      }
     ]
    },
    "8053293d-ebc1-46b4-ac1c-afeb3a53d190": {
     "views": [
      {
       "cell_index": 20
      }
     ]
    },
    "f496660b-92e4-41a4-a67b-b3ca19795f03": {
     "views": [
      {
       "cell_index": 12
      }
     ]
    }
   },
   "version": "1.2.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}