{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Intelligent Agents and Active Inference" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Preliminaries\n", "\n", "- Goal \n", " - Introduction to Active Inference and application to the design of synthetic intelligent agents \n", "- Materials \n", " - Mandatory\n", " - These lecture notes\n", " - Bert de Vries - 2021 - Presentation on [Beyond deep learning: natural AI systems](https://youtu.be/QYbcm6G_wsk?si=G9mkjmnDrQH9qk5k) (video)\n", " - Optional\n", " - Bert de Vries, Tim Scarfe and Keith Duggar - 2023 - Podcast on [Active Inference](https://youtu.be/2wnJ6E6rQsU?si=I4_k40j42_8E4igP). Machine Learning Street Talk podcast\n", " - Quite extensive discussion on many aspect regarding the Free Energy Principle and Active Inference, in particular relating to its implementation.\n", " - Raviv (2018), [The Genius Neuroscientist Who Might Hold the Key to True AI](./files/WIRED-Friston.pdf).\n", " - Interesting article on Karl Friston, who is a leading theoretical neuroscientist working on a theory that relates life and intelligent behavior to physics (and Free Energy minimization). (**highly recommended**) \n", " - Friston et al. (2022), [Designing Ecosystems of Intelligence from First Principles](https://arxiv.org/abs/2212.01354)\n", " - Friston's vision on the future of AI. \n", " - Van de Laar and De Vries (2019), [Simulating Active Inference Processes by Message Passing](https://www.frontiersin.org/articles/10.3389/frobt.2019.00020/full)\n", " - How to implement active inference by message passing in a Forney-style factor graph.\n", "\n", " " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Agents\n", "\n", "- In the previous lessons we assumed that a data set was given. \n", "- In this lesson we consider _agents_. An agent is a system that _interacts_ with its environment through both sensors and actuators.\n", "- Crucially, by acting onto the environment, the agent is able to affect the data that it will sense in the future.\n", " - As an example, by changing the direction where I look, I can affect the (visual) data that will be sensed by my retina.\n", "- With this definition of an agent, (biological) organisms are agents, and so are robots, self-driving cars, etc.\n", "- In an engineering context, we are particularly interesting in agents that behave with a *purpose* (with a goal in mind), e.g., to drive a car or to design a speech recognition algorithm.\n", "- In this lesson, we will describe how __goal-directed behavior__ by biological (and synthetic) agents can also be interpreted as minimization of a free energy functional. " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Illustrative Example: the Mountain Car Problem\n", "\n", "- In this example, we consider [the mountain car problem](https://en.wikipedia.org/wiki/Mountain_car_problem) which is a classical benchmark problem in the reinforcement learning literature.\n", "- The car aims to drive up a steep hill and park at a designated location. However, its engine is too weak to climb the hill directly. Therefore, a successful agent should first climb a neighboring hill, and subsequently use its momentum to overcome the steep incline towards the goal position. \n", "- We will assume that the agent's knowledge about the car's process dynamics (i.e., its equations of motion) are known up to some additive Gaussian noise.\n", "- Your challenge is to design an agent that guides the car to the goal position. (The agent should be specified as a probabilistic model and the control signal should be formulated as a Bayesian inference task). \n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "

\n", "\n", "- Solution at the end of this lesson." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Karl Friston and the Free Energy Principle\n", "\n", "- We begin with a motivating example that requires \"intelligent\" goal-directed decision making: assume that you are an owl and that you're hungry. What are you going to do?\n", "\n", "- Have a look at [Prof. Karl Friston](https://www.wired.com/story/karl-friston-free-energy-principle-artificial-intelligence/)'s answer in this [video segment on the cost function for intelligent behavior](https://youtu.be/L0pVHbEg4Yw). (**Do watch the video!**)\n", "\n", "- Friston argues that intelligent decision making (behavior, action making) by an agent requires *minimization of a functional of beliefs*. \n", "\n", "- Friston further argues (later in the lecture and his papers) that this functional is a (variational) free energy (to be defined below), thus linking decision-making and acting to Bayesian inference. \n", "\n", "- In fact, Friston's **Free Energy Principle** (FEP) claims that all [biological self-organizing processes (including brain processes) can be described as Free Energy minimization in a probabilistic model](https://arxiv.org/abs/2201.06387).\n", " - This includes perception, learning, attention mechanisms, recall, acting and decision making, etc.\n", " \n", "- Taking inspiration from FEP, if we want to develop synthetic \"intelligent\" agents, we have (only) two issues to consider:\n", " 1. Specification of the FE functional.\n", " 2. *How* to minimize the FE functional (often in real-time under situated conditions). \n", "\n", "- Agents that follow the FEP are said to be involved in **Active Inference** (AIF). An AIF agent updates its states and parameters (and ultimately its model structure) solely by FE minimization, and selects its actions through (expected) FE minimization (to be explained below). \n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Execution of an AIF Agent\n", "\n", "- Consider an AIF agent with observations (sensory states) $x_t$, latent internal states $s_t$ and latent control states $u_t$ for $t=1,2,\\ldots$. \n", "\n", "

\n", "\n", "- The agent is embedded in an environment with \"external states\" $\\tilde{s}_t$. The dynamics of the environment are driven by actions. \n", "\n", "- Actions $a_t$ are selected by the agent. Actions affect the environment and consequently affect future observations. \n", "\n", "- In pseudo-code, an AIF agent executes the following algorithm:\n", "\n", "> **ACTIVE INFERENCE (AIF) AGENT ALGORITHM** \n", ">\n", "> SPECIFY generative model $p(x,s,u)$ \n", "> ASSUME/SPECIFY environmental process $R$\n", ">\n", "> FORALL t DO \n", "> \n", "> 1. $(x_t, \\tilde{s}_t) = R(a_t, \\tilde{s}_{t-1})$ % environment generates new observation \n", "> 2. $q(s_t) = \\arg\\min_q F[q]$ % update agent's internal states (\"perception\") \n", "> 3. $q(u_{t+1}) = \\arg\\min_q F_>[q]$ % update agent's control states (\"actions\") \n", "> 4. $a_{t+1} \\sim q(u_{t+1})$ % sample next action and push to environment\n", "> \n", "> END \n", "\n", "- In the above algorithm, $F[q]$ and $F_>[q]$ are appropriately defined Free Energy functionals, to be discussed below. Next, we discuss these steps in more details." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The Generative Model in an AIF agent\n", "\n", "- What should the agent's model $p(x,s,u)$ be modeling? This question was (already) answered by [Conant and Ashby (1970)](https://www.tandfonline.com/doi/abs/10.1080/00207727008920220) as the [*good regulator theorem*](https://en.wikipedia.org/wiki/Good_regulator ): **every good regulator of a system must be a model of that system**. See the [OPTIONAL SLIDE for more information](#good-regulator-theorem). \n", "\n", "- Conant and Ashley state: \"The theorem has the interesting corollary that the living brain, so far as it is to be successful and efficient as a regulator for survival, __must__ proceed, in learning, by the formation of a model (or models) of its environment.\"\n", "\n", "- Indeed, perception in brains is clearly affected by predictions about sensory inputs by the brain's own generative model.\n", "\n", "

\n", "\n", "- In the above picture (The Gardener, by Giuseppe Arcimboldo, ca 1590\n", "), on the left you will likely see a bowl of vegetables, while the same picture upside down elicits with most people the perception of a gardener's face rather than an upside-down vegetable bowl. \n", "\n", "- The reason is that the brain's model predicts to see straight-up faces with much higher probability than upside-down vegetable bowls. \n", "\n", "- So the agent's model $p$ will be a model that aims to explain how environmental causes (latent states) lead to sensory observations." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Specification of AIF Agent's model and Environmental Dynamics\n", "\n", "- In this notebook, for illustrative purposes, we specify the **generative model** at time step $t$ of an AIF agent as \n", "$$\n", "p(x_t,s_t,u_t|s_{t-1}) = \\underbrace{p(x_t|s_t)}_{\\text{observations}} \\cdot \\underbrace{p(s_t|s_{t-1},u_t)}_{\\substack{\\text{state} \\\\ \\text{transition}}} \\cdot \\underbrace{p(u_t)}_{\\substack{\\text{action} \\\\ \\text{prior}}}\n", "$$\n", "\n", "- We will assume that the agent interacts with an environment, which we represent by a dynamic model $R$ as\n", "$$\n", "(x_t,\\tilde{s}_t) = R\\left( a_t,\\tilde{s}_{t-1}\\right)$$\n", "where $a_t$ are _actions_ (by the agent), $x_t$ are _outcomes_ (the agent's observations) and $\\tilde{s}_t$ holds the environmental latent _states_. \n", "\n", "- Note that $R$ only needs to be specified for simulated environments. If we were to deploy the agent in a real-world environment, we would not need to specify $R$. \n", "\n", "- The agent's knowledge about environmental process $R$ is expressed by its generative model $p(x_t,s_t,u_t|s_{t-1})$. \n", "\n", "- Note that we distinguish between _control states_ and _actions_. Control states $u_t$ are latent variables in the agent's generative model. An action $a_t$ is a realization of a control state as observed by the environment. \n", "\n", "- Observations $x_t$ are generated by the environment and observed by the agent. Vice versa, actions $a_t$ are generated by the agent and observed by the environment. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### State Updating in the AIF Agent\n", "\n", "- After the agent makes a new observation $x_t$, it will update beliefs over its latent variables. First the internal state variables $s$. \n", "\n", "- Assume the following at time step $t$:\n", " - the state of the agent's model has already been updated to $q(s_{t-1}| x_{1:t-1})$. \n", " - the agent has selected a new action $a_t$.\n", " - the agent has recorded a new observation $x_t$. \n", "\n", "- The **state updating** task is to infer $q(s_{t}|x_{1:t})$, based on the previous estimate $q(s_{t-1}| x_{1:t-1})$, the new data $\\{a_t,x_t\\}$, and the agent's generative model. \n", "\n", "- Technically, this is a Bayesian filtering task. In a real brain, this process is called **perception**. \n", "\n", "- We specify the following FE functional\n", "$$\n", "F[q] = \\sum_{s_t} q(s_t| x_{1:t}) \\log \\frac{\\overbrace{q(s_t| x_{1:t})}^{\\text{state posterior}}}{\\underbrace{p( x_t|s_t) p(s_t|s_{t-1},a_t)}_{\\text{generative model w new data}} \\underbrace{q(s_{t-1}|x_{1:t-1})}_{\\text{state prior}}}\n", "$$\n", "\n", "- The state updating task can be formulated as minimization of the above FE (see also [AIF Algorithm](#AIF-algorithm), step 2):\n", "$$\n", "q(s_t|x_{1:t}) = \\arg\\min_q F[q]\n", "$$\n", "\n", "- In case the generative model is a _Linear Gaussian Dynamical System_, minimization of the FE can be solved analytically in closed-form and [leads to the standard Kalman filter](https://nbviewer.jupyter.org/github/bertdv/BMLIP/blob/master/lessons/notebooks/Dynamic-Models.ipynb#kalman-filter). \n", "\n", "- In case these (linear Gaussian) conditions are not met, we can still minimize the FE by other means and arrive at some approximation of the Kalman filter, see for example [Baltieri and Isomura (2021)](https://arxiv.org/abs/2111.10530) for a Laplace approximation to variational Kalman filtering. \n", "\n", "- Our toolbox [RxInfer](http://rxinfer.ml) specializes in automated execution of this minimization task. \n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Policy Updating in an AIF Agent\n", "\n", "- Once the agent has updated its internal states, it will turn to inferring the next action. \n", "\n", "- In order to select a __good__ next action, we need to investigate and compare consequences of a _sequence_ of future actions. \n", "\n", "- A sequence of future actions $a= (a_{t+1}, a_{t+2}, \\ldots, a_{t+T})$ is called a **policy**. Since relevant consequences are usually the result of an future action sequence rather than a single action, we will be interested in updating beliefs over policies. \n", "\n", "- In order to assess the consequences of a selected policy, we will, as a function of that policy, run the generative model forward-in-time to make predictions about future observations $x_{t+1:t+T}$. \n", "\n", "- Note that perception (state updating) preceeds policy updating. In order to accurately predict the future, the agent first needs to understand the current state of the world. \n", "\n", "- Consider an AIF agent at time step $t$ with (future) observations $x = (x_{t+1}, x_{t+2}, \\ldots, x_{t+T})$, latent future internal states $s= (s_t, s_{t+1}, \\ldots, s_{t+T})$, and latent future control variables $u= (u_{t+1}, u_{t+2}, \\ldots, u_{t+T})$. \n", "\n", "- From the agent's viewpoint, the evolution of these future variables are constrained by its generative model, rolled out into the future:\n", "$$\\begin{align*}\n", "p(x,s,u) &= \\underbrace{q(s_{t})}_{\\substack{\\text{current}\\\\ \\text{state}}} \\cdot \\underbrace{\\prod_{k=t+1}^{t+T} p(x_k|s_k) \\cdot p(s_k | s_{k-1}, u_k) p(u_k)}_{\\text{GM roll-out to future}}\n", "\\end{align*}$$\n", "\n", "- Consider the Free Energy functional for estimating posterior beliefs $q(s,u)$ over latent _future_ states and latent _future_ control signals: \n", "$$\\begin{align*}\n", "F_>[q] &= \\overbrace{\\sum_{x,s} q(x|s)}^{\\text{marginalize }x} \\bigg( \\overbrace{\\sum_u q(s,u) \\log \\frac{q(s,u)}{p(x,s,u)} }^{\\text{\"regular\" variational Free Energy}}\\bigg) \\\\\n", "&= \\sum_{x,s,u} q(x,s,u) \\log \\frac{q(s,u)}{p(x,s,u)}\n", "\\end{align*}$$\n", "\n", "- In principle, this is a regular FE functional, with one difference to previous versions: since future observations $x$ have not yet occurred, $F_>[q]$ marginalizes not only over latent states $s$ and policies $u$, but also over future observations $x$.\n", "\n", "- We will update the beliefs over policies by minimization of Free Energy functional $F_>[q]$. In the [optional slides below, we prove that the solution to this optimization task](#q-star) is given by (see [AIF Algorithm](#AIF-algorithm), step 3, above)\n", "$$\\begin{aligned}\n", "q^*(u) &= \\arg\\min_q F_>[q] \\\\\n", "&\\propto p(u)\\exp(-G(u))\\,,\n", "\\end{aligned}$$\n", " where the factor $p(u)$ is a prior over admissible policies, and the factor $\\exp(-G(u))$ updates the prior with information about future consequences of a selected policy $u$. \n", "\n", "- The function \n", "$$G(u) = \\sum_{x,s} q(x,s|u) \\log \\frac{q(s|u)}{p(x,s|u)}$$ \n", "is called the **Expected Free Energy** (EFE) for policy $u$. \n", "\n", "- The FEP takes the following stance: if FE minimization is all that an agent does, then the only consistent and appropriate behavior for an agent is to select actions that minimize the **expected** Free Energy in the future (where expectation is taken over current beliefs about future observations). \n", "\n", "- Note that, since $q^*(u) \\propto p(u)\\exp(-G(u))$, the probability $q^*(u)$ for selecting a policy $u$ increases when EFE $G(u)$ gets smaller. \n", "\n", "- Once the policy (control) variables have been updated, in simulated environments, it is common to assume that the next action $a_{t+1}$ (an action is the _observed_ control variable by the environment) gets selected in proportion to the probability of the related control variable (see [AIF Agent Algorithm](#AIF-algorithm), step 4, above), i.e., the environment samples the action from the control posterior:\n", "$$\n", "a_{t+1} \\sim q(u_{t+1}) \n", "$$\n", "\n", "- Next, we analyze some properties of the EFE." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Active Inference Analysis: exploitation-exploration dilemma \n", "\n", "- Consider the following decomposition of EFE:\n", "$$\\begin{aligned}\n", "G(u) &= \\sum_{x,s} q(x,s|u) \\log \\frac{q(s|u)}{p(x,s|u)} \\\\\n", "&= \\sum_{x,s} q(x,s|u) \\log \\frac{1}{p(x)} + \\sum_{x,s} q(x,s|u) \\log \\frac{q(s|u)}{p(s|x,u)}\\frac{q(s|x)}{q(s|x)} \\\\\n", "&= \\sum_x q(x|u) \\log \\frac{1}{p(x)} + \\sum_{x,s} q(x,s|u) \\log \\frac{q(s|u)}{q(s|x)} + \\underbrace{\\sum_{x,s} q(x,s|u) \\log \\frac{q(s|x)}{p(s|x,u)}}_{E\\left[ D_{\\text{KL}}[q(s|x),p(s|x,u)] \\right]\\geq 0} \\\\\n", "&\\geq \\underbrace{\\sum_x q(x|u) \\log \\frac{1}{p(x)}}_{\\substack{\\text{goal-seeking behavior} \\\\ \\text{(exploitation)}}} - \\underbrace{\\sum_{x,s} q(x,s|u) \\log \\frac{q(s|x)}{q(s|u)}}_{\\substack{\\text{information-seeking behavior}\\\\ \\text{(exploration)}}} \n", "\\end{aligned}$$ \n", "\n", "- Apparently, minimization of EFE leads to selection of policies that balances the following two imperatives: \n", "\n", " 1. minimization of the first term of $G(u)$, i.e. minimizing $\\sum_x q(x|u) \\log \\frac{1}{p(x)}$, leads to policies ($u$) that align the inferred observations $q(x|u)$ under policy $u$ (i.e., predicted future observations under policy $u$) with a prior $p(x)$ on future observations. We are in control to choose any prior $p(x)$ and usually we choose a prior that aligns with desired (goal) observations. Hence, policies with low EFE leads to **goal-seeking behavior** (a.k.a. pragmatic behavior or exploitation). [In the OPTIONAL SLIDES](#ambiguity-plus-risk), we derive an alternative (perhaps clearer) expression to support this interpretation]. \n", " \n", " 1. minimization of $G(u)$ maximizes the second term \n", " $$\\begin{aligned}\n", " \\sum_{x,s} q(x,s|u) \\log \\frac{q(s|x)}{q(s|u)} &= \\sum_{x,s} q(x,s|u) \\log \\frac{q(s|x)}{q(s|u)}\\frac{q(x|u)}{q(x|u)} \\\\\n", " &= \\underbrace{\\sum_{x,s} q(x,s|u) \\log \\frac{q(x,s|u)}{q(x|u)q(s|u)}}_{\\text{(conditional) mutual information }I[x,s|u]}\n", " \\end{aligned}$$ \n", " which is the (conditional) [__mutual information__](https://en.wikipedia.org/wiki/Mutual_information) between (posteriors on) future observations and states, for a given policy $u$. Thus, maximizing this term leads to actions that maximize statistical dependency between future observations and states. In other words, a policy with low EFE also leads to **information-seeking behavior** (a.k.a. epistemic behavior or exploration). \n", "\n", "- (The third term $\\sum_{x,s} q(x,s|u) \\log \\frac{q(s|x)}{p(s|x)}$ is an (expected) KL divergence between posterior and prior on the states. This can be interpreted as a complexity/regularization term and $G(u)$ minimization will drive this term to zero.) \n", "\n", "- Seeking actions that balance goal-seeking behavior (exploitation) and information-seeking behavior (exploration) is a [fundamental problem in the Reinforcement Learning literature](https://en.wikipedia.org/wiki/Exploration-exploitation_dilemma). \n", "\n", "- **Active Inference solves the exploration-exploitation dilemma**. Both objectives are served by EFE minimization without any need for tuning parameters. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### AIF Agents learn both the Problem and Solution\n", "\n", "- We highlight another great feature of FE minimizing agents. Consider an AIF agent ($m$) with generative model $p(x,s,u|m)$.\n", "\n", "- Consider the Divergence-Evidence decomposition of the FE again:\n", "\n", "$$\\begin{aligned}\n", "F[q] &= \\sum_{s,u} q(s,u) \\log \\frac{q(s,u)}{p(x,s,u|m)} \\\\\n", "&= \\underbrace{-\\log p(x|m)}_{\\substack{\\text{problem} \\\\ \\text{representation costs}}} + \\underbrace{\\sum_{s,u} q(s,u) \\log \\frac{q(s,u)}{p(s,u|x,m)}}_{\\text{solution costs}}\n", "\\end{aligned}$$\n", "\n", "- The first term, $-\\log p(x|m)$, is the (negative log-) evidence for model $m$, given recorded data $x$. \n", "\n", "- Minimization of FE maximizes the evidence for the given model. The model captures the **problem representation**. A model with high evidence predicts the data well and therefore \"understands the world\". \n", "\n", "- The second term scores the cost of inference. In almost all cases, the solution to a problem can be phrased as an inference task on the generative model. Hence, the second term **scores the accuracy of the inferred solution**, for the given model. \n", "\n", "- FE minimization optimizes a balanced trade-off between a good-enough problem representation and a good-enough solution proposal for that model. Since FE comprises both a cost for solution _and_ problem representation, it is a neutral criterion that applies across a very wide set of problems. \n", "\n", "- A good solution to the wrong problem is not good enough. A poor solution to a great problem statement is not sufficient either. In order to solve a problem well, we need both to represent the problem correctly (high model evidence) and we need to solve it well (low inference costs). \n", "\n", "" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The Brain's Action-Perception Loop by FE Minimization\n", "\n", "- The above derivations are not trivial, but we have just shown that FE-minimizing agents accomplish variational Bayesian perception (a la Kalman filtering), and a balanced exploration-exploitation trade-off for policy selection. \n", "\n", "- Moreover, the FE by itself serves as a proper objective across a very wide range of problems, since it scores both the cost of the problem statement and the cost of inferring the solution. \n", "\n", "- The current FEP theory claims that minimization of FE (and EFE) is all that brains do, i.e., FE minimization leads to perception, policy selection, learning, structure adaptation, attention, learning of problems and solutions, etc.\n", "\n", "

" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### The Engineering Challenge: Synthetic AIF Agents \n", "\n", "- We have here a framework (the FEP) for emergent intelligent behavior in self-organizing biological systems that\n", " - leads to optimal (Bayesian) information processing, including balancing accuracy vs complexity.\n", " - leads to balanced and continual learning of both problem representation and solution proposal\n", " - actively selects data in-the-field under situated conditions (no dependency on large data base)\n", " - pursues a optimal trade-off between exploration (information-seeking) and exploitation (goal-seeking) behavior\n", " - needs no external tuning parameters (such as step sizes, thresholds, etc.)\n", "\n", "- Clearly, the FEP, and synthetic AIF agents as a realization of FEP, comprise a very attractive framework for all things relating to AI and AI agents. \n", "\n", "- A current big AI challenge is to design synthetic AIF agents based solely on FE/EFE minimization.\n", "\n", "

\n", "\n", "- Executing a synthetic AIF agent often poses a large computational problem because of the following reasons: \n", " 1. For interesting problems (e.g. speech recognition, scene analysis), generative models may contain thousands of latent variables. \n", " 2. The FE function is a time-varying function, since it is also a function of observable variables. \n", " 3. An AIF agent must execute inference in real-time if it is engaged and embedded in a real world environment.\n", " \n", "- So, in practice, executing a synthetic AIF agent may lead to a **task of minimizing a time-varying FE function of thousands of variables in real-time**!!\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Factor Graph Approach to Modeling of an Active Inference Agent\n", "\n", "- How to specify and execute a synthetic AIF agent is an active area of research. \n", "\n", "- There is no definitive solution approach to AIF agent modeling yet; we ([BIASlab](http://biaslab.org)) think that (reactive) message passing in a factor graph representation provides a promising path. \n", "\n", "- After selecting an action $a_t$ and making an observation $x_t$, the FFG for the rolled-out generative model is given by the following FFG:\n", "\n", "

\n", "\n", "- The open red nodes for $p(x_{t+k})$ specify __desired future observations__, whereas the open black boxes for $p(s_k|s_{k-1},u_k)$ and $p(x_k|s_k)$ reflect the agent's beliefs about how the world actually evolves (ie, the __veridical model__). \n", "\n", "- The (brown) dashed box is the agent's Markov blanket. Given the states on the Markov blanket, the internal states of the agent are independent of the state of the world. \n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### How to minimize FE: Online Active Inference\n", "\n", "- [Online active inference proceeds by iteratively executing three stages](https://www.frontiersin.org/articles/10.3389/frobt.2019.00020/full): \n", " 1. act-execute-observe \n", " 2. infer: update the latent variables and select an action \n", " 3. slide forward\n", "\n", "

" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### The Mountain car Problem Revisited\n", "\n", "Here we solve the mountain car problem as stated at the beginning of this lesson. Before implementing the active inference agent, let's first perform a naive approach that executes the engine's maximum power to reach the goal. As can be seen in the results, this approach fails since the car's engine is not strong enough to reach the goal directly. " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "using Pkg; Pkg.activate(\"../.\"); Pkg.instantiate();\n", "using IJulia; try IJulia.clear_output(); catch _ end" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "using LinearAlgebra, Plots, RxInfer\n", "import .ReactiveMP: getrecent, messageout\n", "include(\"./scripts/mountaincar_helper.jl\")\n", "\n", "# Environment variables\n", "initial_position = -0.5\n", "initial_velocity = 0.0\n", "engine_force_limit = 0.04\n", "friction_coefficient = 0.1\n", "\n", "Fa, Ff, Fg, height = create_physics(\n", " engine_force_limit = engine_force_limit,\n", " friction_coefficient = friction_coefficient\n", ")\n", "\n", "# Target position and velocity\n", "target = [0.5, 0.0];" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "" ], "text/plain": [ "Plots.AnimatedGif(\"/home/wmkouw/syndr/Wouter/Onderwijs/Vakken/tueindhoven/5SSD0 - Bayesian Machine Learning & Information Processing/2024-2025 Q2/BMLIP/lessons/notebooks/ai_agent/ Mountain Car Problem: naive policy.gif\")" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Simulation of a naive policy, going full power toward the parking place \n", "\n", "# Let there be a world\n", "(execute, observe) = create_world(\n", " Fg = Fg, Ff = Ff, Fa = Fa, \n", " initial_position = initial_position, \n", " initial_velocity = initial_velocity\n", ")\n", "\n", "# Total simulation time\n", "N = 40 \n", "\n", "y = Vector{Vector{Float64}}(undef, N)\n", "for n in 1:N\n", " execute(100.0) # Act with the maximum power \n", " y[n] = observe() # Observe the current environmental outcome\n", "end\n", "\n", "plot_car(y, target, title_plot=\"Mountain Car Problem: naive policy\", fps=5)" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAGQCAIAAAD9V4nPAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdd1zU9eMH8Ndxiy17C6KA4MKtOBH3CvVrrtI0SzMtW5b98pt8zawsKxvOcqRWznLkwIUTF7nCDaIyRUSZxzju98en7ntfhqIcfA4+r+ejP87Pvfl8Xnd09+KzZTqdDkRERFJlJnYAIiIiMbEIqUalp6efO3fu2LFjV69eLSkpETuOcSxYsGDy5MmZmZmV/5F79+6dP3/+2LFjFy5cKCgoqL5sgkmTJr333nvVvZRKio6Onjhx4vbt2x87cufOnRMnTjxy5EgNpCIpYxFSTSguLv7hhx+Cg4NdXFxatWrVpUuXwMBAJyenSZMmxcXFiZ2uqnbs2LFs2bKcnJzHjiwsLFy0aFGLFi2cnZ1btmzZpUuX4OBgGxubHj16rFmzRqvVVlPCFStW/Prrr48eM2PGDJkBW1vbgICAyZMn37hxw7hhrl+/vmLFij///FM/5eDBg8uWLUtNTS018vz58ytWrLh69apxAxCVohA7ANV92dnZw4cPj4yMVCqVgwcPbtOmjbm5eWpq6sGDB5cvX/7TTz9lZ2crlUqxY1a79PT0oUOHHjt2TKVSDRo0qHXr1paWlllZWadOnTp8+HBUVFRsbOynn34qbsiGDRs2bNgQQH5+/sWLF5ctW7Zu3brIyMhOnToZaxFubm6hoaG+vr76KatWrfrpp5+CgoLc3NwMR/r4+ISGhnp4eBhr0UTlYhFS9dLpdOPGjYuMjGzWrNlvv/3m5+dn+Oy+ffsmT54shSO2iouLhw8ffuzYsU6dOv3yyy/e3t6GzyYmJs6ePVuj0YgVT2/cuHGzZ88WHmdmZo4ePXrPnj1Tpkw5f/68sRbRp0+fPn36VGbkmDFjxowZY6zlElWERUjVa+fOnb///ruDg8OePXvK/mnfq1evM2fOKBT//f8wIyPjxIkTt2/fzs/P9/b27tmzp729veGPZGdnX7t2zcnJycfHJykpad++fXfv3g0PDw8ICKgow507d86cOXPnzh2dTufn5xcWFmZhYWE44O7du3fu3PHy8nJ1db18+fKhQ4c0Gk2LFi169Oghk8lKzU2j0ezevTsuLs7Z2blPnz6lVmIqsmbNmsOHD/v4+OzcubNevXqlnvXy8vrxxx/LbhssLCyMioq6cuUKgMaNG4eGhqrV6lJj8vPzjx8/fuvWrYyMDHd3927dupVq2admb2//3Xff+fv7X7hwITEx0cvLS5h++/btgwcPpqWlOTs7d+/eXViDNKTT6WJiYi5fvpyWlubg4ODl5dW5c2crKyvh2fv379+8edPDw8Pd3R1ATExMRkYGgKtXr1paWgpjgoODFQpFampqUlJSgwYNHB0dDeefnp6+b9++pKQkGxubkJCQFi1aGD5bVFR04cIFKyurwMDABw8e7Ny5Mzk5uX79+v369Sv7zickJJw5cyYxMVGlUrm5uYWEhAipSFp0RNVp4MCBAGbNmlWZwRMmTJDL5Yb/f1pbWy9btsxwzP79+wFMnDjxk08+0Q9euXJlRfMMCQkp9f+8q6trZGSk4ZiFCxcC+OKLL1577TXDkWFhYbm5uYYjL168aPi9r1arf/zxx9DQUAC3b99+xEtr3749gG+//bYy74Pg+PHjhtsPATRo0ODYsWOGY7766qtSpS6Xy9944w2tVms4TC6Xe3t7P3px77zzDoCIiAjDiSUlJcJW65iYGJ1OV1xc/Prrrxv+jszMzF5++eWCggL9j9y9e7dLly6l3nO1Wp2VlSUMWL16teGCyv1eSk9P1+l08+bNA7B8+XLDSJ9//nmplzxw4MD79+/rBwh/T3To0GH79u2Gzefs7Hz69GnDl/bWW2+ZmZU+TmLz5s2P+cVQncMipGpUXFxsa2sL4MSJE5UZP2TIkHfffXf37t2XLl06e/bswoULnZyczMzMDh06pB8jFKG3t7e1tfVHH320f//+yMjI8+fPVzTP1q1bz5s378CBA1evXj158uSHH35obm5ubW1t2FtCEfr6+rq7u//www9nzpzZunVrYGAg/rfCHzx4IKwVTZ8+/dKlS3FxcXPnzlWr1a6uro8uwqysLKE84uPjK/M+6HS669ev29jYyGSy99577+LFi3/99dfMmTNlMpm1tfW1a9f0w95///0XX3zx999/P3/+fGxs7Lp16xo3bgzgq6++MpzbUxfh7du3hXpISEjQ6XTTp08H0Lhx461bt8bFxe3atat58+YAxo8fr/+RF154AcDzzz9/8uTJ27dvnz9//tdff33mmWcqKsK9e/f27t0bwJdffrn3H4WFhbryinDRokUAXF1df/rpp+vXrx8+fDgsLAxAly5diouLhTFCEbq5uVlbW8+YMePQoUNHjx4dO3askLykpEQYtnHjRgDBwcE7duxISEi4cuXKnj17pk6dun379kr+jqjOYBFSNUpJSRG+Rh8+fPh0cxBqb/jw4aWmANi0adPTzVOovdmzZ5eaYmFhYVhUf/31l0wmCwwM1E+ZO3cugDFjxhjO7d///reQ5xFF+NdffwlrRZUPKewbe+uttwwnvv322wBGjhz5iB9MTEy0sbFp0KCB4cSnK8L8/PwRI0YA8Pf31+l0cXFxZmZm5ubmwkZmwd27d4W/dfR/i9SvX9/GxqbUKqmhUkWo0+nGjRsH4PDhw6VGlirC3NxcYTv50aNH9WM0Go3Q/Rs2bBCm6Lcwf/311/phJSUlwhbUc+fOCVOmTJkC4MCBA49+W0gKePoEVaOHDx8CkMlkNjY2TzeHHj162Nranjp1qtR0f3//YcOGPd08w8PDAZSd5/Dhww03RTZt2tTT01NYExKm/P777wCENtKbPn36Yw95Fd4Ha2vrUtN37NjR28CgQYOE6Vqtdtu2bXK5fMaMGYbjZ8yYoVAotm/fXlRUVNGyPD0927Rpk5CQcPfu3UenKtf27dsnT548efLk4cOH+/v7b9iwQaFQfPPNNwC2bt1aUlIyduxY/c5CAM7OzhMnTgSwZcsWYYq9vX1eXp4RD67Ri4qKyszM7NatW+fOnfUT1Wr1m2++aRhAYGlpOXXqVP0/ZTKZsN558+ZNfU4A0dHRRs9JtQ6LkKqRcHyETqfLz8+vzPicnJw5c+a0a9fO3d3dzMxMJpOZmZllZWUJB1MYCgoKKnsYS7lSUlLeeOON4OBgJycn4Qy5Bg0aALh3716pkWUPt3F1ddVoNA8ePBD+eenSJTMzM2FjoJ6jo2P9+vUfnUF4H8q+Cenp6TExMTExMWfOnNm3b9/BgweF6bdu3crJyfHw8Ch1JI6rq6unp2deXp7+21yr1S5fvjw0NNTLy0upVAovMCoqCkDZN60yYmJili1btmzZss2bN+fk5AwZMuT48eP9+vUTXj6AVq1alfqRNm3aAIiNjRX++eKLL2q12nbt2vXp0+ezzz4Tdi4+RZKyhACtW7d+dABBo0aNDA/CAiBswU5LSxP++dxzz6nV6g8++KB58+bvvfferl27TOGoXRIFjxqlauTi4qJWqwsKCuLi4kr1R1kajaZr167nzp0LCgoaMWKEo6OjcITkvHnzyl57xcnJqTIBEhMT27dvn5qa2r59+/Hjx9vb2ysUCo1GExERUfbsdf0hi3rCkRTC93hJSUleXp6Dg0PZ9T8XF5f4+PhHxPD09JTJZHl5eWlpacLXsWDChAkTJkwAkJmZ6eDgoJ8unJvv4uJSdlaurq63bt3Kzs7Wz2HNmjUeHh4DBgzw8PAQXsJPP/106dKl4uLiR0SqyMyZM999910AarW61BtSUSrhFekjTZ8+3dHRceHChfv27du7dy8Ab2/vzz//XNjKWhWVDCAo+9sU/nLSX8+oSZMmJ06c+PDDD/fu3Tt//vz58+dbWVm99tprERERZQ/NpbqNRUjVSKVStW/f/siRI/v27XtsEa5bt+7cuXMjR4785Zdf9Gt7Wq1Wf1qboUquDi5YsCAlJWXOnDn6PXkArl69GhERUdnX8A8zMzNLS8sHDx4UFhaqVCrDp/QrGRVxcnJq3LjxlStX9u/fX5kT44QtyeXOVtgBJuyWO3fu3Jo1a1q0aHHs2DHD7a7CJtynY25uXup8lcemMowkeP75559//vnU1NSoqKht27Zt3Lhx1KhRTk5OwoEtT63yASqpZcuW27Zty8vLO378eGRk5IoVKz799NOioqIvvviiKjmp1uGmUapewhrPt99+m5ubW+4A4agKAMJepZEjRxqWXGxsbFUuxSnMc9SoUYYTY2Jinm5uzZo1KykpOXfunOFE4RzEx/7s+PHjASxYsKAyK2re3t62trYpKSnJycmG05OTk5OSkqysrIR9mcKrGzRokGEL5ufnX758uVKv5wk1a9YM5b17p0+f1j9ryM3NbdSoUT///PPnn3+u0+lK7cMzJKxkP/YKc8Iizpw5U8kAlWRpadmrV6/58+cL+ws3bdr0dPOh2otFSNXrueeea9Omzc2bN8eMGZOXl1fq2Tt37vTt21c49EPY2qk/Xl9Q7upg5ZWdZ0FBwccff/x0cxMOz1mwYIHhxIULF1am21599VU/P78///xz4sSJj612uVw+dOhQrVY7f/58w+nz58/XarVDhw4V9n45OzujzDu2YMEC4dgcoxsyZIhCofj5558Nl5iamrpixQqZTDZ8+HBhStlrrgp7Oh/xqoUrLTz274nu3bs7OzsfO3bM8DLc+fn5X331FYBnn332iV5O2ZwuLi5yubwGroFOpoabRql6qVSqTZs29e7de9u2bQEBARMmTGjbtq2lpWVKSsr+/fs3bNigP0JBOC09IiLCysqqS5cu9+7d++abb44ePWpra/vU3009evTYuHHjpEmTPv/882bNmsXHx8+dO/epj4mYOnXqkiVLNmzYYGdnN2nSJJVKtX79+gULFri7u+tPFKmIjY3Nb7/91r9//59++ikqKmrcuHGtWrWytbXNyMi4cePG2rVrARie/f2f//xn69at33zzjUwme+655wCsXbv2m2++sbW1nTNnjjCmffv2lpaWP//8c0BAQHh4eHFx8a+//rpw4UJvb+9S7WgU3t7eb7311vz588PCwj755JMmTZpcu3bt/fffz8nJmTx5cpMmTYRhHh4eo0eP7tOnT8OGDVUq1dmzZ99//30Aj9hH2K5dOwCzZs26efOm0Jrjxo0zNzcvNczc3PzTTz+dOHHisGHD5s2b16lTp6SkpDlz5ty4cSMsLOyZZ555opfz3HPPFRYWjhgxIiAgwNnZ+datW59++qlWq636vkyqfcQ8d4MkIyMjY9q0aWW/2kJCQnbv3q0f9p///MfwSh9eXl7R0dFeXl6GZ+DpryxTmeUWFxc///zzhkts2bLl2bNnAbRu3Vo/TDiPcMGCBaV+XPiCzsjI0E+5cuVKUFCQfm4WFhZr166tzJVlBCkpKZMmTSp7LIajo+Pbb78tXE5FLyYmxnBZAAIDA8+cOWM4ZvPmzfpLlwGwsrJas2bNkCFDAFy4cEE/7KlPqC9Fq9W+//77hrtIFQrF9OnTi4qK9GP8/f1LvTpbW9slS5boB5Q9j7CkpOS9994z/DvgEVeWWbRoUamzcZ599lnDE1X1V5YpFf7zzz8HsGjRIuGf06ZNK3VYqZmZ2YQJE/Lz8x/9RlHdI9NJ4HrHZCKEoxISExM1Go2zs3Pbtm19fHxKjbl169bp06fv37/v6+vbvXt3lUp1+/ZtrVarP8MvPz8/JSXFxsZG2DBYGbGxsefPn8/Pzw8MDBSuuJaQkKBWqz09PYUBDx8+zMjIcHR0LHUtyqSkpIKCggYNGhjWc1FR0aFDh+Li4uzt7Xv16uXg4JCcnKzRaLy9vUt9sT7ifYiOjk5MTMzPz7e1tQ0KCmrevHm5P6vVak+cOHHlyhWdTieEL3UJOgAZGRnR0dFJSUmurq49evSoV69eWlpabm6ul5eXvrFu3LihUCiE80YqkpGR8fDhQ3t7+4oOltG7e/fuoUOHMjIy7O3tu3XrVvbinElJSWfPnk1LS1MoFN7e3u3btzds6+zs7NTUVEdHR8MDZQX3798XTlbx8fGRy+WZmZn37t1zdXUtdSBMVlbWoUOHhN2lISEhpa7krtVqb926Zfj7FQi/ZScnJ/3csrOzhQuNajQaDw+PVq1a8U4X0sQiJCIiSePBMkREJGksQiIikjQWIRERSRqLkIiIJI1FSEREksYiJCIiSWMREhGRpLEIiYhI0liEREQkaSxCIiKSNJMrws8++0yj0Tz2zmRUM57uLudkXPpbNpK4+HEwBdXxcTC5Ivz2228zMjI0Gk1JSYnYWQj5+fliRyAUFBTw42AK+HEwBUVFRUZfUzK5IiQiIqpJLEIiIpI0FiEREUkai5CIiCSNRUhERJLGIiQiIkljERIRkaSxCImISNp0JsbT0zMxMTEnJ0e4fMCjzZ49u6LXdfDgQf2wwYMHlzvG3Nz87t27+mEeHh7lDgsICNCPuXXrlkKhKHfYmDFj9MO2bdtWUbAvvviC+Zmf+Zmf+Z8uf35+flFRkc6oZDoTu3STl5fXyZMn7ezsLCwszMy4wiqy7OxsGxsbsVNIXV5enlqtlsvlYgeROn4cTIFGo1EoFBUV6tNh0xARkaSxCImISNJYhEREJGksQiIikjQWIRERSRqLkIiIJI1FSEREksYiJCIiSWMREhGRpLEIiYhI0liEREQkaSxCIiKSNBYhERFJGouQiIgkjUVIRESSxiIkIiJJYxESEZGksQiJiEjSWIRERCRpLEIiIpI0FiEREUkai5CIiCSNRUhERJLGIiQiIkljERIRkaSxCImISNJYhEREJGnGKcKcnJxVq1Z99dVXV65cecSwzMzMjRs3JiQkGGWhREREVWeEItRoNJ06ddq0aVNKSkpISMjBgwcrGvn666+PGzfu0KFDVV8oERGRUSiqPov169crlcpt27aZmZl5eXl99NFHPXr0KDts586dmZmZLVq0qPoSiYiIjMUIa4R79uwZOHCgmZkZgMGDB0dFRRUUFJQak5WVNWPGjMWLF1d9cUREREZkhDXC5OTkrl27Co89PDx0Ol1KSkqDBg0Mx7z99ttTp06tX7/+Y+eWm5s7d+5ctVqtUChkMlnTpk1Hjx5d9ZD0dAoKClQqldgppE74y1Iul4sdROr4cTAFBQUFWq1Wq9VWcrxSqRTW0x7BCEUok8l0Op3wWHggk8kMBxw4cCA2Nnbp0qWVnJutra25ublSqZTJZPzfjoiIqpURitDd3T0tLU14nJqaKpPJ3NzcDAd89913JSUlo0aNAnDjxo3vv/9eo9FMnjy53LlZWlq+/vrrdnZ2FhYWj61xqm6FhYVqtVrsFFKn1WrVajXXCEXHj4Mp0Ol0CoVCoTBCeekZYV59+vT57rvvZs+ebWZmtn379u7duwv/r9y+fdvGxsbe3n727Nnp6enC4AsXLvTo0aNbt25VXy4REVHVGaEIR44cuWDBgvDw8KCgoB9++GHz5s3C9NGjRw8dOvSdd94JDg7WD65Xr16TJk2CgoKqvlwiIqKqM0IRWlhYREdHb9q0KTMz8/jx44GBgcL0Tz75xMPDo9TgefPmNWrUqOoLJSIiMor/HudiIry8vE6ePMl9hCYiOzvbxsZG7BRSl5eXx32EpoAfB1Og0WiMvo+QTUNERJLGIiQiIkljERIRkaSxCImISNJYhEREJGksQiIikjQWIRERSRqLkIiIJI1FSEREksYiJCIiSWMREhGRpLEIiYhI0liEREQkaSxCIiKSNBYhERFJGouQiIgkjUVIRESSxiIkIiJJYxESEZGksQiJiEjSWIRERCRpLEIiIpI0FiEREUkai5CIiCSNRUhERJLGIiQiIkljERIRkaSxCImISNJYhEREJGksQiIikjQWIRERSRqLkIiIJI1FSEREksYiJCIiSWMREhGRpLEIiYhI0liEREQkaSxCIiKSNBYhERFJGouQiIgkjUVIRESSxiIkIiJJYxESEZGksQiJiEjSWIRERCRpLEIiIpI0FiEREUkai5CIiCSNRUhERJLGIiQiIklTiB2ApERXgsL7KMhA4f2/HxTc/+/jv6dkoCgL5i4wd4WlJ8xdYeEBCzeYu8PCDRbuUDmI/TKIqE5hEVI1KynEvRNI3Y+0/cg4BaUtVA5QO0Ll8PcDtQNsA/87Ue0IlR00adDcRV4iNHeRn4TM89CkID8F+akozoWFGyw8YOEBpxC4dod9K8jkYr9OIqqtWIRUDXQlyDyHtP1I3Y970bANhGsYmkfAuTPkFpWag8oBtkHlP6XVQJOG/CTk3kH6UZxYhbxEOHeFayhcQmHfEjJu8CeiJ8AiJOPJiUfqPqTuQ9oBKOvBrRcaTUTnn428MVNuDisfWPnACfAZCQAF6bh3AunHcGoSsq/DsT3cesGtF+xbsRSJ6LFYhFRl98/g2iKk7gMAt57wegZtFsLCveYCqJ3hORiegwFAk4a7h5AWheNjoUmDSze4hcH7WZi71VweIqpVWIRUBXePIPZjZF1G4zfQ9H3Y+IsdCDB3hfcIeI8AAE0q0qKQEokLs+EaBr+X4dab64hEVIrRijA6OjoqKsrNzW306NHm5ualnn3w4EFkZGR8fLyrq+uwYcPq1atnrOWSONKP4uJ/kBOHxtPR7XfIS//GTYK5G3xGwWcUihYi8Tdc/gInJ6LB8/B/FVbeYocjIlNhnL+O16xZM3To0MLCwl9++aVXr14lJSWlBvTq1WvdunW5ublbtmwJCgpKSkoyynKpxumQtB17OuL0FPiOxeBraDzdRFvQkNIGvuMQthc9IgFgdxsc6I3bG6ErFjsZEYlPptPpqjgLnU4XEBAwf/78oUOHFhUVBQQELFmypG/fvoZjMjMz7e3thcddu3YdOHDgzJkzy52bl5fXyZMn7ezsLCwszMy4FUtk2dnZNjY2AKArQfIfuBgBnRaBb6HB87V4G6NWg6TtuLEMWZfR4Hn4T4GVj9iZHiUvL0+tVsvlPEVEZP/9OJB4NBqNQqFQKIy5X88I32UJCQnx8fH9+/cHoFQqe/fuvW/fvlJj9C0IQKVSKZXKqi+XakhJIeJ+xI7GuPwFgueh/zn4jqvFLQhAbg7vZxG2F913oCgbu1rjcDiS/4Cu9JYMIpICI5RqcnKynZ2dfr+gm5tbXFxcRYN37tx54cKFX375paIBubm5c+fOVavVCoVCJpMFBASMGzeu6iHpaeiKZde/191epLMJ1LZeWuLUGQAKCsSOZTyWQWjxJZrMlSdtkp//EOf/XdxsbolLT7FjlVZQUACAa4SiKygoUKlUYqeQuoKCAq1Wq9VqKzleoVA89rNjhCKUy+WG21d1Ol1FSz1z5syECRN+/vlnFxeXiuYmk8nq1atnbm6uVCqFx9xAKgpZ3h35iefMYK7ttEFn3wZ1+Lq0Kmud7/hi3xfMEn9TnH0DVj7a5h/r7FuJHeu/zP4hdhCp42/BFDzpx0Emkz12jBGK0M3N7eHDh3l5eZaWlgBSUlLc3Mo5Z+v8+fODBg1atmxZ7969HzE3S0vL1157jfsIRZa4FacmwX+KxudNG1vJHOLrOxIN/oW4FYpjQ+HcGS0/gXUjsTMBgFKpVCqVXCMUnfCLEDuF1Gm1WlPcR+jj49O4cePt27cD0Gg0kZGRwv7C3NzcixcvCmOuXLkyYMCAr7/+Ojw8vOpLpGpUUoCY6YiZjm6/oXlE7d4X+BRkCvhNwuDrcGiDPR1xajI0aWJnIqLqZYSjRgFs2rRpypQp48aNO3nypJWV1e7du2UyWVRUVN++fYXdG02bNs3Ozu7YsaMwvmfPnpMnTy53VjxqVEy5CTg6Chbu6LgCKntI/DC5ggxc/hxxP6DRS2j6AZSivQ88atRESPrjYDKq46hR48xr+PDhgYGBUVFRXbp0GTx4sLBNtkWLFtu2bRMGLF26VKPR6Md7enoaZblkTHc24/RUNH0fjaeLHcU0qB3R8lP4v4rYj7HdH03fR8BUyHgxJqK6xjhrhEbENUIRFOch5nWkH0PnX2EfbPgM/wT+2/0zODcTeUkI/hj1h9XwwrlGaCL4cTAFJnoeIdVuD2Oxpx1KitDvdKkWpP9yaIuwfWjzNS7+B4cGQ5MqdiAiMhoWobTd/An7QuE3CSGrobAWO43Jc++L/jFw7oKdwUhYK3YaIjIO7vCQqqJsnH4FmefQ8yDsmomdpvaQKdDkPbiGIXocEreh3WKoHcXORERVwjVCScq+jt2tobRFvxi24NNwbIf+f8LSE7uCkbxT7DREVCUsQunJvYUDvdHkPbRbXAtuHGGy5BZo/RVC1uLMNJyahKJssQMR0VNiEUqMJg0H+yLgNTR6SewodYJrKPqfB2TY2QJpB8VOQ0RPg0UoJQX3sL8nGo5H0NtiR6lDlDZovxTtlyB6HE5NRnGe2IGI6MmwCCWj6CEO9kP9YWhS/p0gqUrc+2LARWjzsbst7p8ROw0RPQEWoTQU5+Bgf7h0Q4s5Ykepu1R2CPkJzWcjahAuRkBXLHYgIqoUFqEEaPNx6BnYNUfrBWJHkQCfkeh/FvdO4EBfFN4XOw0RPR6LsK4rKcKRZ2HhiXaLgcffl4uMwMIdPXbBpSt2t8fDWLHTENFjsAjrNJ0W0WNhpkTISsndUElkMjSPQPBH2B+GxG1ihyGiR+GVZeowHU5NRsF9hG7nPRPE4TMaVg1w9FlkXUaT98ROQ0Tl41pCXaXD6VeRdQXdfoOZWuwwEuYUgj4ncXsTjo2BNl/sNERUDhZhHXVuJu6fReguKKzEjiJ5lp7ofQQyGfb35G0riEwQi7AuuvAhknejx04Rb6pO/0Nujk5r4RWO3e1xP0bsNET0P1iEdc6Vr3FrPcIioXIQOwoZkqHJe2i9AFEDcGeL2GGI6L94DEXdcutXXPsOvQ/D3FXsKFQe72dh3RCHh+LhJTT7gCe0EJkCrhHWIXmJiJmOzr/AwkPsKKDHGlcAACAASURBVFQxhzboexJJO3B0JC9MSmQKWIR1ha4Ex59H0DtwbCd2FHocC3f0ioKZGvu6Iz9F7DREUscirCsufw6ZGQJ5W4laQm6OTmvgFY593ZF3R+w0RJLGfYR1wv0/ceVL9D3Ny8fUMs1mQe2Ivd3Qcx+sG4mdhkiiWIS1X3Eejj+HNgth5S12FHpy/lMgk2N/T4Ttg42f2GmIpIhFWPudfRtOHeAzSuwc9LT8JkEmx77uCItEvaZipyGSHBZhLZeyGymR6H9W7BxUNY0mQmGJA33QYzfsmoudhkhaWIS1WUE6TkxE55+htBU7ClWZz2jI5DjQC6F/wKGt2GmIJITHVtReOpx4EY1egkt3sZOQkXiPQIcfETUIGSfFjkIkISzCWuva98hPRbNZYucgo/IchI4rcCgc96LFjkIkFSzC2inrMi7OQed1MFOKHYWMzWMAQlbj0DNIOyB2FCJJYBHWQiUFODYGLT+FTYDYUah6uPdF1804Ngap+8SOQlT3sQhrofOzYNUAjV4UOwdVJ5du6LoJx8YgabvYUYjqOBZhbXP3MG6tR4cfxM5B1c+5C3rsxMlJ8hR2IVE1YhHWKoWZiB6Ljj9C7Sh2FKoRDm0Rul117nVZ8h9iRyGqs1iEtcqpyaj/L7j1FjsH1SCHtgUhm81Ov4x7J8SOQlQ3sQhrj/jVeHgJwfPEzkE1rcSudUnIOhwZhqwrYmchqoNYhLVEbgLOvYvOv0JuLnYUEoHOpQeCP0HUQN6/kMjoWIS1xLmZCHgNds3EzkHiafgCGr2EqIEoyhI7ClGdwiKsDTJOI/0oAt8UOweJren7cOmGw0NQUiB2FKK6g0VYG5x/H80joLASOweZgNZfQu2I6PHQlYgdhaiOYBGavOSdyEtEw/Fi5yDTIDNDp3UoSMe5mWJHIaojWISmTVeC8x+g5XzIeMMs+oeZCl03I2UPrnwldhSiuoBFaNpu/gS5ObwGi52DTIyyHkJ34upC3FwjdhSiWo/rGSZMq8HF2ej0MyATOwqZHktPhP6B/T1h4Q63XmKnIarFuEZowq5+A4e2cO4sdg4yVfWaossGHH8OmefEjkJUi7EITVXhA1z+Ai0+EjsHmTaXbmi/DIeeQe4tsaMQ1VYsQlMV+zHqD0O9JmLnIJPnFY6m7yNqAArvix2FqFZiEZqkvCTEr0SzWWLnoFrCfwo8B+HgABTniR2FqPZhEZqkCx/AfwosvcTOQbVHy09h0winXhY7B1HtwyI0PQ8uInk3gmaInYNqFxk6/Ijs6zy5kOhJsQhNz9l30ewDKG3FzkG1jdwcXTbh0nykRIodhag2YRGamLuHkH0VfpPEzkG1k5U3Oq9D9Djk3BQ7ClGtwSI0KTqcm4ngT2CmFjsJ1VquYQiagSPDoM0XOwpR7cAiNCW3NqCkGD4jxM5BtVzQ27ANxKnJYucgqh1YhCajpAgXZqHlp7ygGhlBxxV4GIvri8XOQVQLsAhNxo0lsPGHW0+xc1CdILdA1824OAd3j4gdhcjUsQhNQ3EOYj9B8Mdi56A6xKoBOv2E46ORlyR2FCKTZpy7T+Tl5a1ZsyYxMbFHjx5hYWHljvnjjz+io6P9/PzGjBmjUqmMsty649JncOsN+1Zi56C6xa03/F7B0WfR6yCPwCKqiBHWCHU6Xa9evbZu3WptbT1u3Ljly5eXHfPJJ59Mnz69Xr16q1evHjZsWNUXWqdo7uL6UrSYI3YOqouafQBLT8S8KXYOItNlhDXCvXv3JiUlHTp0SKlUBgcHv/LKKy+++KJcLtcPyM/P/+KLL3bt2tW+ffupU6d6eXnFxMS0adOm6ouuIy58iIYvwMpH7BxUJ8nQcSX2dEDcj2g0UewwRKbICGuEBw4c6Nmzp1KpBNCzZ8/k5OS4uDjDAWfPnpXJZO3atQNgaWnZrVu3AwcOVH25dUT2NdzZgqbvi52D6i6FNbpuxvn/w/0zYkchMkVGWCNMSUnx8PAQHiuVSkdHx+Tk5ICAAMMBLi4uMtnfZwW4ubklJydXNLecnJyPPvpIrVYrFAqZTObn5zd+/PiqhzRZyr/m6xq9UlxiCY1G7Czl0Gg0wp84JCKNRqPT6Qy3sjwxVQN5q+8Uh/9VGHZMp3IyXjRp4cfBFGg0GoVCoVBUtryUSuVjPztGKEK5XF5SUqL/p1arLbXUsgMe8RrkcrmdnZ25ublSqZTJZG5ublX6/Ju4wkx50paivhdM9jXK5XKTzSYd8n9UaS5eQ3QPz6lOjCruHgkzfps/DX4cTMGTfhz062CPYIQi9PDwuHXr77tjazSazMxM/QqifkBqampJSYmZmRmA5OTkJk0qvN+shYXFa6+9ZmdnZ2FhIYyvy66vgtczShvTvd2SUqnkn8CiE34LRvgKDv4Ih55Rxv4brRcYI5fk8ONgCoRVqcqvEVaGEZpmwIABe/fuzcnJAbB9+3Z/f/+GDRsCiIuLEwqyVatWlpaWBw8eBHD//v0jR47079+/6sut9XRaXF+MgGli5yDJkJmh0xokbcOtX8WOQmRCjFCqnTp16tixY2hoaMeOHdevX7906VJhVfTf//63o6Pjt99+q1QqP/roo+eff3748OFRUVEjRowIDAys+nJrvaTtsHCHQ1uxc5CUqOzRZSMO9oVjO1g3EjsNkUmQ6XS6qs+lpKRkz549ycnJnTt31pfcpUuXlEqlv7+/8M8LFy6cOnXK19c3LCzsERttvby8Tp48KYlNo/vD4DcJPqPEzvEo2dnZNjY2YqeQury8PLVabcy9U9e+RfxP6HMMZry0xRPgx8EUPOnBMpVhnCI0IqkU4cNYHOiD8AQTP2yBn3xTYPwihA6Hh8ImAK3mG2+edR8/DqagOoqw7jaNibu6EP5TTLwFqe6SoeNK3N6ApB1iJyESH4tQDIWZuL0Zfi+LnYMkTGWPkDU4NQn5KWJHIRKZMdcuqbLifoTnQJi7ip2DpM2lKxq9jBPjEboLMv5NTE9Lm4+CDBTcQ0EGCtJRkIHCDBTo/7uHwvtQ2sDKB1YNYOXz93+W3jB3ETv631iENU6nxbXv0XWj2DmIgOYfYn8PXP4CTd4VOwrVHiUFuHsYybuQvBt5t6ArgdoRKkeoHaF2htoJakdYN4Rju38mOqIoC7m3kHsLuQlIP4rc28i9heJcWPvC0vu/7ejYHjZ+Nf+CWIQ1jmdNkOmQydFpHXa3g2t3OHYQOw2ZttwEJO9C8i7cPQS75vAYgM7rYBMAhVWlfrzsbeaKc5Gb8E9B3kLSdpx7F3JLePSHxwC4dIfc3Ogvolwswhp39Rs0fl3sEET/sKyPjj/i2HPoHwNlPbHTkInRFePeCSTtQOo+5N6Gayi8hyNkNVT2Rpi5wgr1mqJe0/+ZmBOPpO24/AWOjoRjO7j1glc4bKv31HMWYc168BeyrqL+v8TOQWTAYyA8duHUK+j8i9hRyDTkJyNpB5J3Ie0g6gXBYwDaL4F965rYl2zdEI2no/F0FD5A6l4k78K+HlDZwWMAPPrDpWt1LJNFWLOufQf/V3jWBJmcVl/wnoUEANk3EDsPSdvh3g/ez6LDD1A7ipNEZQfvZ+H9LKDD/T+RvAsXZuNhrMq5m67FXDi0MOKiWIQ1qDATtzdg0GWxcxCVITdHl1+xtzucO8E2SOw0JIasK4idh+TdaDwNg69DZSd2ID0ZHNrAoQ2azUJBhjZxj0xeub2SlcYirEFxP8BzMM+aIBNlG4SW83B0BPqegtxC7DRUgx7G4tJnSImE/ysYfM2UKrAMtaPWc5hxLysDnlBfc3ivCTJ9jV6CXQuc5akUkvHgIqLHYX8vWDfE4GtoHmHSLVhtWIQ1JXEbzN3h2E7sHESP1G4Rknci8Xexc1A1yzyPoyNwoDfqNcUzcWgeAaWt2JlEw02jNeXat2j8mtghiB5HWQ+df8WhQbBvBSsfsdNQNbgXjdh5eHABgW8hZDU3g4NFWEMexvKsCao1HNsh8C1Ej0PPA5AZ8ZYXJLacOJx+Fdk30GQmum7mTbj0uGm0Rlz9Fv6TedYE1RpBM2CmxsU5YucgI9GV4OpC7OkI974YfBV+L7MFDXGNsPoVPuBZE1TLyMzQaS12t4FLF7j1FjsNVU1OPE5OhLYQvY/CtrHYaUwR1wirX9xynjVBtY+5CzqtRfQLyE8WOwo9LV0JbizDno5w74feR9iCFeEaYTUTzprovF7sHERPzqU7/F/BsTHouZ87C2uf7Bs4ORG6EvQ5CpsAsdOYNK4RVrOk7TB341kTVFs1nQW5Ghf/I3YOehK6Ylz6DHs6wGMAeh1iCz4W1wir2VWeNUG1mcwMIWuwuw2cO8O9r9hpqBIexuLEBMgt0PekKPf2q424RlidHsYi6wrPmqDazdwFndbhxATkJYkdhR5JWBHc1wONXkKvKLZg5bEIq9PfZ03wMGWq5Vy6IWAajo+BrljsKFSBzPPY3R53D6F/DPwmATKxA9UmLMJqI9xrwm+y2DmIjKHJTMgtcGG22DmoPNeX4GAfNH4doTthWV/sNLUP9xFWm/hV8BjIsyaojpCZodMa7GoDl65w7yd2GvpHcR5OT0HmOfQ+Cht/sdPUVlwjrDbxq+D3ktghiIxH7YwuGxA9Hrm3xI5CAIDsG4gMgU6LPsfZglXBIqweGadRnA2XbmLnIDIqp44IfAPHRqGkSOwokpe0A5Gd0GAMOq2Fwsg3qpUaFmH1iF+Jhi9yfzXVQU3eg7kLLnwodg4J02lxMQJnpqL7VjR5T+w0dQH3EVYDrQa3N6Dfn2LnIKoOMnRciV1t4NQRXuFih5Gegns4/hxKCtH3FA9BMBauEVaDxN/g0AZW3mLnIKoeKgd0+RWnJnNnYU27H4M97WHfCmH72IJGxCKsBnEr0XCC2CGIqpNjBwS+haMjUVIodhTJuLEMB/uh1edo+Skv/WpcLEJjy0vE/RhuMqK6r8kMWLjh/Cyxc0iAVoOTE3F9Efqe4JWqqgOL0NjiV8JnFOQWYucgqm4ydFyB2xuQtF3sJHVa9nXs6YCSYvQ+DutGYqepm1iExqVD/Co04nZRkgaVA7qsx8mXkHle7Ch1VMpu7O0C/ykIWQ2Fpdhp6iwWoVGlRUFhCYe2YucgqimOHdD2exwahLw7Ykepc24sw4kX0WUD/F8RO0odx9MnjCp+JRpOFDsEUc3yHo6cGzgUjt6HobAWO02dUFKAU1OQ+Sf6RMPKR+w0dR/XCI2nKAtJ2+H7nNg5iGpck5lw6oijI3h7CiMoSMeBPijOQe/jbMGawSI0nlvr4RoGtbPYOYjE0PYb6LQ4PVXsHLVc5jnsbg+nEHRZz52CNYZFaDzxPH2QJEymQJdNuHcCV74WO0qtdWs9DvZF6y/R8lNeoLEmcR+hkWRfQ85NuPcVOweReJQ2CN2JvZ1g5YP6Q8VOU7vocPE/iF+J0F1waC12GMlhERpJ3I9oOA5mSrFzEInK0hNdf8PBfrD0gGMHsdPUEsU5OD4WBffQ9zTMXcROI0XcNGoMumLcXIuG48XOQWQCHFojZCWODOeVSCslJw57OsLcBT0PsAXFwiI0huTdsPKBbZDYOYhMg8dABL6NqAEofCB2FNN29wj2doXfJLRfyu1JImIRGgMPkyEqJfANuIbh6AjewrdC177D0RHo/Asavy52FKljEVZZQTrSDsBnpNg5iExMm68hV+P0q2LnMD3FeYgeh7gf0Oc4XLqLnYZYhFV3cy08n4HSVuwcRCZGJkfn9XhwAZc+EzuKKcm+gcgQ6ErQ+xisfcVOQwCL0AhuruZ2UaLyKSzR7XdcX4yEn8WOYhqSdmBvFzR6EZ3WQmEldhr6G0+fqJr7MSjKgis3bhBVwMId3bbiYB9YN4BTJ7HTiEenxYV/I+FndN8Gx/Zip6H/wTXCqolfiYbjeQ0IokexD0bITzg8FKl7xY4iEs1dHOiDjDPod4YtaIJYhFVQUohbG+D7gtg5iEyee19034bj4xD3o9hRatz9M4jsCMd26LELaiex01A5uGm0Cu5sgX1LXh6eqFIcO6DXARwcgLw7aB4hdpqacmMZzn+A9ktRf5jYUahCLMIqiF/Jm9ETPQHbIPQ5jqiBKLiPNl9DVqe3SGk1OP0qMs+i7wlYNxI7DT1Knf4fsVrlJSLjDDzDxc5BVKtYuKP3YWRfw9Hh0OaLnabaZF/HnvYoKULvY2xB08cifFo3V8NnBG8YRvTEFNbovhVmahzsXzevwXbnN+ztgoBp6LSGXxG1Aovw6egQv4qnDxI9JTM1Oq2DQ2vs64a8JLHTGE/hfZyYgLNvo/sO+E0SOw1VFovwqdw9DDM1D4MmenoyM7T+Eo0mIrIjMs+LncYYkrZjZzDMVOh/Ho7txE5DT4AHyzyV+JVo9KLYIYhqv8bTYemFqP7o/Ctcuomd5mnlp+DMNGRfQ5eNcOoodhp6YkYrwj179hw4cMDNzW3ixIm2tqUvvJmUlPTHH3/Ex8e7ubmNGTPGxaU233arOAeJW9HyU7FzENUJ9f8FlQOOPos239TCi9frcHMNzr6LhuPR+WeYqcXOQ0/DOJtGly5d+tJLL7m7ux89erRHjx5arbbUgGHDhh09etTR0fHMmTNBQUE3b940ynLFcWs9XEJh7iZ2DqK6wrUHekTi7Du49r3YUZ5E1hXs647rS9HzAFp+yhasvWQ6na6Ks9BqtY0aNVq0aNGAAQO0Wm1gYOCXX345ePBgwzF5eXmWln8fPdWzZ8+ePXv+3//9X7lz8/LyOnnypJ2dnYWFhZmZSe7C3NcNge/A6xmxc9SE7OxsGxsbsVNIXV5enlqtlsvlYgepZrkJONgfbr0R/BGU9cROU47/fhxKinB5Pq4sRPMP4f9qHT8h0sRoNBqFQqFQGHO/nhF+fwkJCYmJib169QIgl8t79eoVFRVVaoy+BQEUFRVZWdXay65n30DWNXj0FzsHUZ1j1QB9jkGbh+2NcX0xdMViB6pAxinsbov0aPSPQcA0tmAdYIRSTU1NrVevnkqlEv7p4uJy/fr1igZv2rTp+vXrY8eOrWhAdnb2O++8o1KpFAqFTCYLCgqaOnVq1UMai+L6j/AaWVyoBUpv/q2TNBqNUqkUO4XUaTQanU5X99cIAcASwd/JfKcq//o/2ZWvi5vM1nqa0JXJCvIeWFz9jzxhdXHTCK3vRADQaMQOJTlPukaoVCof+9mp7Lz69++/b9++UhPDw8M3bdqkUCgMdwoWFxdX9NV5/PjxV199ddOmTQ4ODo8IHRwcbGlpqVQqZTJZgwYNTOiLWFciv/NzSdetJhSpmimVSum8WJMl/BakUYQAAMcWuu47kLxDef49xa1VupZf6Oo1FTlSUbbs1lqHS5/LXLuVDIg1UztyNVAsWq32iYqwMrvYKjuvXbt2ld2bKJPJAHh4eGRlZeXk5FhbWwNITk728PAoO4fo6OihQ4f+8ssv3bo96iBpc3PzsWPHmug+wtSDUDuZObQUO0fNkcvlEvr+NVXyf4gdpGbVD4fnANmNJbKo3vAaghZzYO4qQoysq7j2PRLWwS0sr/VyS58+Evs1mJzq+Dg8QdPIyhCm169fv0WLFps3bwaQk5Oze/fuQYMGAXj48OHRo0eFMWfPnh06dOiKFSt69uxpxPQ1LX4VGo4XOwSRZJgpEfAaBl2Fyh5/NMXFiJq7PKmuBKn7cGgw9oVCYYn+f6LLRq1DSA0tnWqWEY4aBbBz585x48aFh4fHxMT4+flt2rQJQFRUVN++fQsKCgA0btw4Ozu7adO/t28MGDDgzTffLHdWpnvUaFEWtvpg8HVJ3VGMR42aAqkcNfpoubdxYRbSohA8F75jq/Fu2EUPEb8KVxdC5QC/SfAdC7mF8Aw/DqagOo4aNU4RAkhISDhy5Iinp2doaKhQYFlZWZcuXerYsSOAixcvFhYW6gc7OTn5+JR/Gz/TLcIby5GyB103iZ2jRvGTbwpYhP919zD+fAtmSjSPgHNnKKyNOfP7Mbj2PRJ/g9cQBEyDQ5tSz/PjYApMugiNxXSLMLIzmr4Pz0Fi56hR/OSbAhbh/9CVIGEtri/FgwuwbgCHdnBsD8d2sGsBsyc/sKukELm3kXES175HfjL8p6DRxIq2+vDjYAqqowh5rdHKyb6GnHh49BM7B5HkyczgOw6+46ArxoO/kHEKGadwfTFy4mDXAo7t4NgOju1h4/+/m091yE9BTjxybiL3JnJu/v0gPw2WnqjXFE3eg+cgyPjXhhSxCCsnfjV8n4eMbxeRyZApYN8S9i3/vuFRcS7u/4mMU0jagQsfojATDu1g3QB5iX/XntoBVr6w9oW1L5y7wnccrH1h6cXPNfH/gErQaXHzJ4TuEjsHEVVMYQWXrnDp+vc/C9KRcRq5t+AV/nf//XPMC1EpLMJKSN0HC3fYNRM7BxFVmtoZHgPEDkG1gykdjWKy4lfz9EEiorqKRfg4RQ+Rsgvete42aUREVCkswsdJ+AVufaB2FDsHERFVCxbh49xcjYYviB2CiIiqC4vwkbKvIScB7n3EzkFERNWFRfhIcSvQ8AWeZkREVIexCCumK0HCz9wuSkRUt7EIK5ayBxYesA0SOwcREVUjFmHFePdBIiIJYBFWoOghUvfCh6cPEhHVcSzCCiSsg3tfqOzFzkFERNWLRVgBXlaNiEgaWITleXgJ+Ulw6yV2DiIiqnYswvLEr4LvON6ik4hICliEZeiKkbAWvuPEzkFERDWBRVhGyh5YNYBtoNg5iIioJrAIy+Dpg0REUsIi/F+F95G6D94jxM5BREQ1hEX4vxJ+hscAqOzEzkFERDWERfi/uF2UiEhiWIQGHsZCkwrXMLFzEBFRzWERGoj7Eb7jefogEZGk8Jaz/9Dm4+Za9Dsldg4iIqpRXCP8R8I6OHeCVQOxcxARUY1iEf7j+hL4vyp2CCIiqmksQgDAvWgUPeRVtomIJIhFCAC4tgj+r0LGd4OISHL41Q8U3EPyH2j4gtg5iIhIBCxC4MZy1P8XVA5i5yAiIhFIvgh1JYhbDv9XxM5BRETikHwRJu2AuRsc2oidg4iIxCH5Iry+CAE8a4KISLqkXYQ5ccg8i/rDxc5BRESikXYRXluERhMhNxc7BxERiUbC1xrV5uPmGvQ9KXYOIiISk4SLMOFnOIfA2lfsHERU7QoLCxMTE6s4k9zcXCsrK6Pkoafg5ORka2tbHXOWcBFeX4LguWKHIKKa8M0333z88ccODlU6XVin08lkMmNFoieSl5fXqVOnzZs3V8fMpVqE906gMBNuvcXOQUQ1oaCgYOrUqXPn8m/f2mrHjh1Lly6tpplL9WAZ4awJXlyUiEjyJNkEBfeQtAMNx4udg4iIxCfJIoz7AfWH8eKiREQEKRahrgQ3lvHiokREJJBeESb/AXNXOLQVOwcREZkE6RWhcA9eIiIiAJIrwpw4ZP4J72fFzkFEJJozZ87ExsbWzLKuXr164sQJANnZ2Vu2bKmZhT4piRXh9cVo+CIvLkpEUrZ69erffvutZpa1a9euJUuWAEhLS3vlFRM9OENKJ9Rr8xH/Ey8uSkQS9+2339b8Qv38/O7evVvzy60MKa0RJvwCp468uCgRmYKioqKIiAg/Pz83N7fOnTtnZmYCGD16tI+Pj7u7e58+feLi4oSRo0ePXrRoUYsWLezt7QEsX7784sWLpeb2+++/v/HGG5MmTXJycgoKCjp27Jgwfc6cOY0aNXJ3d2/Xrt3hw4eFifPmzVu6dKlOpxs4cOC5c+eEibm5uZ06dUpPTwewdevWVq1aeXh4dO/evexG1DNnzowePXrGjBkeHh6BgYHbtm0Tpqenp48aNcrT09PPz++rr77S6XSGP3X79u1u3boJj5OTk0eNGuXl5eXp6Tlt2jStVtunT5/Lly8Lz967d69Tp045OTlVfIcrT0prhNcXI/gjsUMQkWnQ5uPhpZpbXL2mpXbKfPzxx5GRkUePHnVzc/vrr79UKhWAyZMnr1271szMbMGCBZMmTdq/fz+Ay5cvx8XF7dq1S7hW6ldfffXuu+82b97ccG7p6emLFy/esmXLsmXLFi9e/Oqrr54/fx5Az549Z8yYYWFhERkZOXr06Pj4eLVanZycrNVqZTJZYGDgypUrFy5cCGDz5s1KpdLZ2fn06dPTpk3bu3dvYGDgxo0bhw0b9tdffymVSv2ysrOzN2zYMH/+/KSkpOjo6H79+sXGxtavX//ll192cHC4efNmUlJSaGiot7f3v/71L/1PFRQUCFVXUlIydOjQrl27xsfHy+Xys2fPyuXyZs2aLV++/MsvvwSwatUqT09Pa2vravtllCaZIsw4icJMuPUROwcRmYbkXYidV3OLa/oB6g81nLBq1aqVK1e6ubkBaNasmTCxTZs2W7ZsSUpKys7Ojo6O1l/me/r06Z6ensKYS5fK7++QkJCBAwcCGD169GuvvVZcXKxQKDp06LB37964uLjc3Nz8/PwbN240bdpU/yMTJkwIDQ2dP3++Wq1evXr1Cy+8AGDFihXDhg1TqVTx8fFt2rQpKCi4cuVKqd61s7N74403ZDJZp06dwsLCtm7dOnHixG3btiUmJqpUKl9f3ylTpqxfv96wCPViY2OvXbt29OhRoVzbtm0L4NVXXw0JCZk3b565ufmKFSu++eabp3ubn45kivDaIvhP4cVFiehv9Yeh/jCxFq7T6VJSUurXr284MSUlpUOHDuHh4YGBgY6Ojvn5+QUFBebm5gDc3d0fO09HR0fhgbm5uVarLSoq0ul0Xbt29fb27ty5s729vUqlevDggeGPNGvWzMfHZ+fOne3atTt9+vTvv/8OIDk5OTExMSUlRRjTvn37Uhs5Abi4uMjlcuGxu7t7Wlpaenq6TCYTeh2Ap6fnzp07y82ZnJzs5uZmuIoJwM/Pr2XLlps3b/b09CwoKAgLC3vs6zUiaRRhwT0kbUfrL8XOQUQEADKZ+HL3OQAAEStJREFUzNfX99KlS35+fvqJu3btateunXAkS0xMTKnxT7GU2NjY1NRU4eyFvLy8N998s+yY8ePHr169+tKlS0OHDrWxsQHg6+vr7e396ANqEhMT8/PzLSwsAFy/fr1Nmzbu7u5yuTwuLs7f3x/AtWvXfHx8yv3Zhg0b3r59Ozs7W1ic3pQpUxYuXOjp6Tlp0iQzsxpdaTHOwpKTk+fMmfPmm28ePHjwEcMSExM/++yzv/76yygLfQJxP8BrCNSONb1cIqIKvPPOO2+//fbBgwdv3bq1cePGhw8fenh4xMTEXLhw4eLFizNnzqzoB4cOHbp169bKLMLFxSUjI2Pv3r03b96cMmVK2RU7AGPGjNm/f/+yZcuE7aIAXnvttXXr1v34448JCQnnz5+fN29ecXFxqZ/SarVvv/32zZs3f/jhh3Pnzg0fPlypVE6cOHHatGmXL1/euXPn4sWLJ0+eXG4qf3//3r17jx8/PjY29urVqxs3bhSmP/PMM3Fxcdu2bZswYUJlXp0RGWGNMCsrq0OHDoMGDWratOnIkSOXLFkybFj5GxwmT54s7BnWbxCvCYUPcOUr9HxUQxMR1bCXXnrJ2tr6yy+/vHfvXrNmzfr379+vX78pU6ZMmTLFzs5u1qxZ9vb2wubH3r17u7i46H+wUaNGdnZ2pebWsGHDkJAQ4bFcLn/22WflcrmHh8fKlSvnzZsH4JVXXnF0dHRycgLQtm3bevXqCYMdHR3ffffd+Pj40NBQ/fyPHz8+f/78H374wcbGpmvXrmXXzwICAtq2bTt27FhnZ+f9+/cLeRYuXPjZZ59NmTKlXr16a9eu7dKlC4DGjRsLW3dtbGz01fDrr79++umn06ZNEw6cESYqFIohQ4akp6cbvtgaoquyb7/9tlu3bsLjlStXtmvXrtxhK1eufPHFF9u3b79q1apHzM3T0zMxMTEnJ0er1VY9m06n0519T3fiJePMSnqysrLEjkC63Nzc4uJisVPUbnPnzv3ggw/ETlFHHDhwIDg42OizzcvLq1+//uHDh8t9dvv27YMGDdLpdPn5+cLuTyMywqbRI0eO9OrVS3jcq1ev06dP5+fnlxqTkpIyf/78zz//vOqLezL5yYj7Ac0/rOnlEhFRpS1evNjf379r165du3at+aVXdtOoVqstO1FYbU9NTe3Ro4cwRVilTU5ObtSokeHIqVOnRkRECCfBPFp2dvY777yjUqnkcrlMJmvcuPG0adMqGbIs1dn3dT4TiuCIvLynnomU5efn648NI7Hk5eVptVr+IqqiqKhI7Ah1R8eOHXfs2GHceY4cOXLo0KH6g07LVVJSkpeXp9FoFAqFQlHZ8lKpVI8dXKl5HTp0qE+fcs7AO3fuXFBQkFKp1O9KFR6o1WrDYb/88ktBQcGIESMqsyylUhkcHGxpaalUKoUiLDW3J5B1RZ66Szvgklr1tHOQvMLCwqd//8lItFqtWq1mEVaFXC4ve8RHbRcTE6PVatu3b1/Dy7WwsPDy8jLuPCuzmiSTydRqtU6ne6IirMwBqJWaV/fu3QsKCip61sPDIykpSXicmJgol8tdXV0NB6xfv/7q1avCWZNXrlyJiIi4c+fOrFmzyp2bubn52LFj7ezsLCwsqnoE7YX30fR9uYVTlWYibXK5nN+/opP/Q+wgtVgNH45fM7Zs2VJQUFDzRSgWmUwmN2DEORvhqNHw8PBZs2ZFRESo1eoNGzYMGDBAOFPy9OnTLi4uPj4+S5Ysyftny+SwYcOef/55/XG61ejuETy4iK4bq31BRERUmxnhr6Tw8HAPD4/OnTuPHDly4cKFERERwvQ33nhDOEHEzc2t4T/UarWzs7Ozs3PVl/sY52Yi+COYcbMeEZmi+/fvh4eHu7i4NG3adN68ec8++/d9Uvfu3du0aVM3N7fg4GD9ZbI//vhjX19fDw+Pxo0bV/IkQqo8I6wRKhSKyMjIw4cPP3jw4PvvvxfOUwGwcuXKsie7rF69uiZa8M4WFOfAZ0y1L4iIaqfs7Oxr166V+5S/v7+tra3wOC0tLTExsdxhwcHB+j1VN2/evH//ftkxKpWq1FU69d59911bW9uUlJScnJyePXsKO+NTUlKGDx++ZcuWnj17bt26dciQITdu3HBwcBgwYMCMGTNUKtWff/7Zv3//7t27l/12padmnEusyeVy/YGjegEBAWVHBgYGGmWJj6LT4vwstPmaVxYloors379/7ty55T41c+bM4cOHC49Xrly5adOmcoetWrVKf22QiIiIcu/5rlar9+7da2lpWfap7du3R0ZGyuXyevXqTZo0afXq1QD27NnTunXrnj17AggPD/f19T1w4MDw4cObNWv2xx9/xMfHFxUVKRSK2NjYzp07P/mLpvLVxWuN3lgOC1e480YTRFShIUOGDBky5LHDZs6c+YirnekJNVZ5Op3uwYMHwv0FYXC97PT0dMNTCNzc3O7evVtSUtKjRw9PT8+wsDAHBwcLC4uHDx8+0eLo0ercOpM2H7Hz0OoLsXMQEVVIJpP5+fkJtwwEoL87ro+Pj/7+tDqd7urVq76+vgkJCZcuXVq/fv3kyZOfeeaZUneQoKqrc2uEl7+Acxc4tBE7BxHRo7z33nuvv/76gwcP7t27t2HDBuHoisGDB7/77rsRERHDhw9fvXq1Uqns1atXfn6+Vqv99ddfmzVrtmDBgsLCwv9v715jmujyMICfSosWCqRc2oCgCJLXWN1uFC1EYsDKi4QWjSYkonhBjX5yNUZN8INv1hA1mpig4CUQjcp6wQsCRjSIXCKgQJEAigGtF2irSBYoVAptZz9M0rDurrpvpad0nt+nmWEyPOEw8+/MnJ5DO7u7ca9CaP5CXueQpAbaOQAAfmDTpk0SiaS8vDwsLGz//v3s7H1CobCuri47O3vPnj0ymay6ulogEAgEgjt37pw6dcpisWzZsmXevHmzZs0ihERHR7vfKAFUuFchbPs7CU8nosgf7wkAQNXg4OCyZctWrVo1MjKiUqlWr17Nbg8JCcnNzf1mZ6VSyfagmcg+bwM4yI3eEQ5ryft/EFkW7RwAAD/27t07mUwmk8nmzp0bGRm5a9cu2om4y43uCFuzyG97yAzpj/cEAKBNLpezE7WLRKI/NwE9/Cruckf4zxfkUxX57W+0cwAA/B98fHxQBalzl0LYsp/85Q8i8KGdAwAAphi3KIT6h2TkA4nIpJ0DAACmnqlfCBkbac0ifz1KpgloRwEAgKln6neWeVdIeHwShm7EAPA9Wq22oqKCdgr4k+yD70yGKV4IbWOk7Q+iyCcEb5sB4H+Sy+VVVVXHjx935CBWqxXTI1OUmJg4SUee4oWw+zzxkxHptxNfAABMpFKpVCqVgwcxGo0+PuiR54ameCEULyIzU2mHAACAKWyKF8IgzMgFAAAOmfq9RgEAAByAQggAAJzmooWwqKjozZs3tFNwndFoPHv2LO0UQO7du/fy5UvaKbjObDbn5OTQTgGkvLz8l3+VwkULYXFxsX3uZqBFp9Pl5+fTTgHk/v37TU1NtFNw3ZcvX/Ly8minAPLw4cP6+vpfe0wXLYQAAADOgUIIAACchkIIAACcxmMYhnaGfyMSiQIDA/v7+0UikZeXF+04nGaxWAwGQ2hoKO0gXNfX1ycUCkUiEe0gnGa1WnU6XVhYGO0gXNff3+/p6fnzQ/ykp6cfOXLk+/u4XCHU6XSjo6NjY2MCgQDzVVJnNpunT59OOwXXjY+Pe3h4TJuG5zeU4XRwBRaLhcfj/fygr8HBwUKh8Pv7uFwhBAAAcCZ8xgQAAE5DIQQAAE5DIQQAAE5DIQQAAE5zxWmYamtri4uL/f39MzMzg4ODacfhCoZhuru7NRrN4OBgRkbGxH5WdXV1d+/e9fX13bp1K75NMdl6enrKysq6urokEsmGDRvsf3CGYa5du9bU1BQREbF9+/YZM2bQzenebDbbrVu32traRkZGFixYkJ6ebv+Dv3///uLFiyaTKS0tLTo6mm5O7mhubm5ubl6/fr39ixOVlZX3798PDAzctm2bRCJx5OAud0dYUlKyZs2asLCwjx8/KhSKoaEh2om4oqurKzY29ty5czt37jQajfbtjx49SklJCQ4O7uvrW7p0aX9/P8WQXJCRkVFfXx8aGtrd3T1//vzOzk52+6FDh44ePRoVFVVaWrp27Vq6Id3e2NjY9evXvb29w8PDCwoKlEol28Fer9dHR0cbjUaJRLJy5cra2lraSTlhYGBg48aNO3futF9/bt68uX79+tmzZ3d3d8fGxppMJod+AeNiYmJiLly4wC7Hx8efOXOGbh7usFqtDMOw/2efPn2yb09ISMjJyWGXk5OTT5w4QScfZ3z9+tW+rFars7KyGIYZHBwUiUTt7e0Mw5hMJj8/v+bmZmoROWZgYIDH4719+5ZhmMOHD69bt47dfuzYMZVKRTUaV2RmZubm5hJCtFotu0Uul1+5coVdVigUBQUFjhzfte4IzWbzs2fPEhMT2dXExMTq6mq6kbjjv35f22az1dbWrly5kl1FizjBxGeeZrOZfRCk0Wh8fX1lMhkhRCgUxsXFoSGc5unTp0FBQVKplBBSU1ODC5STVVZWarXazZs327cMDg62trb+woZwrXeEBoOBYRj7016pVKrT6ehG4ri+vj6LxWJvEYlEotfr6UbijpKSkpaWlqtXrxJCDAZDUFCQ/Uc4NZxDrVY/f/7cbDaXlJSwIz7q9Xp7Q0gkEqPRODw8jNHvJs/IyMju3buLi4snDjSm1+unTZsWGBjIrkql0oaGBkd+i2sVQoFAQAixWCzs6vj4OAY0ouubFrFYLJ6enlQTcUVDQ8OOHTtu3rzJXnb5fL7VarX/FKeGcxQWFg4NDd24cSMtLa2trS0oKIjP5088HXg8Hp/vWldRN3PgwIHNmzfPnTt34ltAgUBgs9msVis70Jrjp4NrPRqVSCR8Pr+3t5dd7e3tRa9RusRisVAoRIs4WWNj4+rVqy9fvhwfH89uCQkJ0ev1NpuNXUVDOIevr29oaOi+ffsCAgIqKysJITNnzrTfi/f29gYEBKD77qQqLCwsLCyMjo6Oi4sjhKSmpj548CA4OJjH401sCAdPB9cqhHw+Pzk5uaioiBAyPj5eXFycmppKOxSn8Xg8tVp969YtQojVar179y5aZLK9ePEiNTU1Pz8/KSnJvnHJkiUCgYC9Fut0uoaGBpVKRS+j+2O7LLHLBoOhp6dn9uzZhBC1Wn379m32E0lRUZFaraaZkgOqqqoKCgrOnz9/+vRpQkh2drZCofDy8lIqlWylMJvNpaWlDl6XXG7QbY1Gk5SU9Pvvv799+9bT07OiooJ9OgdOEBMTMzo62traKpfLBQJBY2MjIaS9vX3FihVKpbKnp8disTx58gQfgSeVXC7/8OFDZGQku5qcnMxOInPp0qWDBw+mpKTU1NSsWbPm5MmTVGO6ubKysr179y5atIhhmMrKyrS0tLy8PELI8PDw8uXLfX19pVJpTU1NTU1NVFQU7bCcYDKZvL29tVpteHg4IaS+vl6tVicnJ3d2dorF4gcPHvz8fBT/yeUKISHk8+fPVVVVYrE4ISEBz9+dSaPRTPx/WLx4MbvQ39//+PFjPz+/hIQEvCOcbB0dHaOjo/ZVf3//OXPmsMuvX7/WaDQREREKhYJSOg7p6Oh49eqVh4fHggULJlY7s9n8+PFjk8mkVCrFYjHFhJxis9laWloWLlxovwQZDIbq6uqAgICEhARHqiBxzUIIAADgNK71jhAAAMDJUAgBAIDTUAgBAIDTUAgBAIDTUAgBAIDTUAgBAIDTUAgBAIDTUAgBAIDTUAgBAIDTUAgBAIDTUAgBAIDT/gUGrSq9XPQq0AAAAABJRU5ErkJggg==", "image/svg+xml": [ "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n" ], "text/html": [ "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Let's also plot the goal and car positions over time \n", "trajectories = reduce(hcat,y)'\n", "plot(trajectories[:,1], label=\"car: naive policy\", title = \"Car and Goal Positions\", color = \"orange\")\n", "plot!(0.5 * ones(N), color = \"black\", linestyle=:dash, label = \"goal\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Next, we try a more sophisticated active inference agent. Above, we specified a probabilistic generative model for the agent's environment and then constrained future observations by a prior distribution that is located on the goal position. We then execute the (1) Act-execute-observe --> (2) infer --> (3) slide procedures as discussed above to infer future actions. " ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "create_agent (generic function with 1 method)" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "@model function mountain_car(m_u, V_u, m_x, V_x, m_s_t_min, V_s_t_min, T, Fg, Fa, Ff, engine_force_limit)\n", " \n", " # Transition function modeling transition due to gravity and friction\n", " g = (s_t_min::AbstractVector) -> begin \n", " \n", " s_t = similar(s_t_min) # Next state\n", " s_t[2] = s_t_min[2] + Fg(s_t_min[1]) + Ff(s_t_min[2]) # Update velocity\n", " s_t[1] = s_t_min[1] + s_t[2] # Update position\n", " \n", " return s_t\n", " end\n", " \n", " # Function for modeling engine control\n", " h = (u::AbstractVector) -> [0.0, Fa(u[1])] \n", " \n", " # Inverse engine force, from change in state to corresponding engine force\n", " h_inv = (delta_s_dot::AbstractVector) -> [atanh(clamp(delta_s_dot[2], -engine_force_limit+1e-3, engine_force_limit-1e-3)/engine_force_limit)] \n", " \n", " # Internal model perameters\n", " Gamma = 1e4*diageye(2) # Transition precision\n", " Theta = 1e-4*diageye(2) # Observation variance\n", "\n", " s_t_min ~ MvNormal(mean = m_s_t_min, cov = V_s_t_min)\n", " s_k_min = s_t_min\n", "\n", " local s\n", " \n", " for k in 1:T\n", " u[k] ~ MvNormal(mean = m_u[k], cov = V_u[k])\n", " u_h_k[k] ~ h(u[k]) where { meta = DeltaMeta(method = Linearization(), inverse = h_inv) }\n", " s_g_k[k] ~ g(s_k_min) where { meta = DeltaMeta(method = Linearization()) }\n", " u_s_sum[k] ~ s_g_k[k] + u_h_k[k]\n", " s[k] ~ MvNormal(mean = u_s_sum[k], precision = Gamma)\n", " x[k] ~ MvNormal(mean = s[k], cov = Theta)\n", " x[k] ~ MvNormal(mean = m_x[k], cov = V_x[k]) # goal\n", "\n", " s_k_min = s[k]\n", " end\n", " \n", " return (s, )\n", "end\n", "\n", "@meta function car_meta()\n", " dzdt() -> DeltaMeta(method = Linearization())\n", "end\n", "\n", "function create_agent(;T = 20, Fg, Fa, Ff, engine_force_limit, target, initial_position, initial_velocity)\n", " \n", " Epsilon = fill(huge, 1, 1) # Control prior variance\n", " m_u = Vector{Float64}[ [ 0.0] for k=1:T ] # Set control priors\n", " V_u = Matrix{Float64}[ Epsilon for k=1:T ]\n", "\n", " Sigma = 1e-4*diageye(2) # Goal prior variance\n", " m_x = [zeros(2) for k=1:T]\n", " V_x = [huge*diageye(2) for k=1:T]\n", " V_x[end] = Sigma # Set prior to reach goal at t=T\n", "\n", " # Set initial brain state prior\n", " m_s_t_min = [initial_position, initial_velocity] \n", " V_s_t_min = tiny * diageye(2)\n", " \n", " # Set current inference results\n", " result = nothing\n", "\n", " # The `compute` function performs Bayesian inference by message passing\n", " compute = (upsilon_t::Float64, y_hat_t::Vector{Float64}) -> begin\n", " \n", " m_u[1] = [ upsilon_t ] # Register action with the generative model\n", " V_u[1] = fill(tiny, 1, 1) # Clamp control prior to performed action\n", "\n", " m_x[1] = y_hat_t # Register observation with the generative model\n", " V_x[1] = tiny*diageye(2) # Clamp goal prior to observation\n", "\n", " data = Dict(:m_u => m_u, \n", " :V_u => V_u, \n", " :m_x => m_x, \n", " :V_x => V_x,\n", " :m_s_t_min => m_s_t_min,\n", " :V_s_t_min => V_s_t_min)\n", " \n", " model = mountain_car(T=T, Fg=Fg, Fa=Fa, Ff=Ff, engine_force_limit=engine_force_limit) \n", " result = infer(model = model, data = data)\n", " end\n", " \n", " # The `act` function returns the inferred best possible action\n", " act = () -> begin\n", " if result !== nothing\n", " return mode(result.posteriors[:u][2])[1]\n", " else\n", " # Without inference result we return some 'random' action\n", " return 0.0 \n", " end\n", " end\n", " \n", " # The `future` function returns the inferred future states\n", " future = () -> begin \n", " if result !== nothing \n", " return getindex.(mode.(result.posteriors[:s]), 1)\n", " else\n", " return zeros(T)\n", " end\n", " end\n", "\n", " # The `slide` function modifies the `(m_s_t_min, V_s_t_min)` for the next step\n", " # and shifts (or slides) the array of future goals `(m_x, V_x)` and inferred actions `(m_u, V_u)`\n", " slide = () -> begin\n", "\n", " model = RxInfer.getmodel(result.model)\n", " (s, ) = RxInfer.getreturnval(model)\n", " varref = RxInfer.getvarref(model, s) \n", " var = RxInfer.getvariable(varref)\n", " \n", " slide_msg_idx = 3 # This index is model dependent\n", " (m_s_t_min, V_s_t_min) = mean_cov(getrecent(messageout(var[2], slide_msg_idx)))\n", "\n", " m_u = circshift(m_u, -1)\n", " m_u[end] = [0.0]\n", " V_u = circshift(V_u, -1)\n", " V_u[end] = Epsilon\n", "\n", " m_x = circshift(m_x, -1)\n", " m_x[end] = target\n", " V_x = circshift(V_x, -1)\n", " V_x[end] = Sigma\n", " end\n", "\n", " return (compute, act, slide, future) \n", "end" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "" ], "text/plain": [ "Plots.AnimatedGif(\"/home/wmkouw/syndr/Wouter/Onderwijs/Vakken/tueindhoven/5SSD0 - Bayesian Machine Learning & Information Processing/2024-2025 Q2/BMLIP/lessons/notebooks/ai_agent/ Mountain Car Problem: active inference agent.gif\")" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Create another world\n", "(execute_ai, observe_ai) = create_world(\n", " Fg = Fg, \n", " Ff = Ff, \n", " Fa = Fa, \n", " initial_position = initial_position, \n", " initial_velocity = initial_velocity\n", ")\n", "\n", "# Planning horizon\n", "T_ai = 50\n", "\n", "# Let there be an agent\n", "(compute_ai, act_ai, slide_ai, future_ai) = create_agent(; \n", " \n", " T = T_ai, \n", " Fa = Fa,\n", " Fg = Fg, \n", " Ff = Ff, \n", " engine_force_limit = engine_force_limit,\n", " target = target,\n", " initial_position = initial_position,\n", " initial_velocity = initial_velocity\n", ") \n", "\n", "# Length of trial\n", "N_ai = 100\n", "\n", "# Step through experimental protocol\n", "agent_a = Vector{Float64}(undef, N_ai) # Actions\n", "agent_f = Vector{Vector{Float64}}(undef, N_ai) # Predicted future\n", "agent_x = Vector{Vector{Float64}}(undef, N_ai) # Observations\n", "\n", "for t=1:N_ai\n", " agent_a[t] = act_ai() # Invoke an action from the agent\n", " agent_f[t] = future_ai() # Fetch the predicted future states\n", " execute_ai(agent_a[t]) # The action influences hidden external states\n", " agent_x[t] = observe_ai() # Observe the current environmental outcome (update p)\n", " compute_ai(agent_a[t], agent_x[t]) # Infer beliefs from current model state (update q)\n", " slide_ai() # Prepare for next iteration\n", "end\n", "\n", "plot_car(agent_x, target, title_plot=\"Mountain Car Problem: active inference agent\", fps=5)" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "image/png": "iVBORw0KGgoAAAANSUhEUgAAAlgAAAGQCAIAAAD9V4nPAAAABmJLR0QA/wD/AP+gvaeTAAAgAElEQVR4nOzdd1xT19sA8CchQNh7BQTZguBgqDhBRXGPumpdiKutv2pbrbW1tdpatWptrdpqq6ivddZZJyhoRRQBAREUBGTvnUnWff+4bZqyZCS5QJ7vH36Sk5NzHi6SJ/feM2gEQQBCCCGkruhUB4AQQghRCRMhUqmysrKnT5/Gxsa+evWqx1yN+Oabb1auXCkQCNr+lvLy8uTk5IcPH6ampopEIuXFBgBCoTAsLOyrr75Sai9tFxUVFRYWdufOnTfWvHDhQlhYWEJCggqiQuoMEyFSBaFQ+NNPP3l6elpbW/v6+g4bNszNzc3S0nL16tUFBQVUR9dZFy9ePHz4sFAofGNNPp+/d+9eDw8PKyurgQMHDh8+vF+/fvr6+uPHjz9//rySvhmIRKKjR49evHix9WrLly+nyTEyMvLw8Fi9enV+fr5i40lLSzt69Ojz589lJbdv3z58+HB1dXWjmvHx8UePHn39+rViA0CoEQbVAaCer6qqatq0aQ8fPmQymTNmzBgwYIC2tnZRUdHdu3cPHDhw+vTpqqoqqmNUhaKioilTpiQlJeno6EyfPn3AgAFMJrOmpiYuLi4qKioiImL79u2ffvoptUG6ubnZ29sDAJfLTU1NPXDgwO+//37v3r3+/fsrqgs7O7vAwMBevXrJSg4ePHj16tUhQ4aYmprK13R2dg4MDLS0tFRU1wg1j0BImcRi8ZgxYwDA398/Ly+v0atXr151dXWlJDAFGjhwIADU1dW1UofP5/v4+ADA2LFji4uLG72anZ09b968zZs3KyM8DocDAN7e3q1XW7ZsGQDs3btXVlJWVjZ8+HAAGDlypDICk5k6dSoApKSkKLUXhFqCZ4RIuc6ePXv37l0bG5ubN2+amZk1enXKlClDhw6VLykvL4+Li8vPzxcKhY6OjqNHjzY0NJSvUFtbm52dbWVlZWdnl5ubGxUVVVVVNXv27N69e7cUw+vXr58+fVpYWEij0dzd3YOCgrS0tOQrlJSUFBcXOzg4mJubP3v2LCYmRiQS+fj4jBgxomlrPB7vxo0beXl5VlZWISEh5ubmbTkOv/zyy9OnTz08PK5evaqjo9PoVScnp9OnT5eWljYqFwgEUVFRr169otPpnp6eo0aNYjAa/81yudyHDx/m5+fX1tayWKzAwEAWi9WWkN7I0tLyhx9+8PPzi4mJqa+vl/0isrOz//rrr4qKCmtr68DAQPIMUp5UKn3y5ElmZmZ5ebmZmZmDg0NAQIDsp66oqMjPz+/Vq5elpaVEIklOTq6trQWA9PR02e1SHx8fGo1WVFRUWlrq7OxsbGws335paendu3eLi4uNjY2HDx/u4eHR6KClpaUZGhq6urpWVVXduHGjrKysd+/eISEh+vr6jULNysp6+vRpUVERk8lksVgBAQF4AqqOqM7EqIcjTyl27drVlspvvfUWnf6f+9ZGRkanTp2Sr3PhwgUA+OijjzZu3Eij0chqFy9ebLZBqVTq6enZ6P+8nZ1dTEyMfLUtW7YAwKFDh5YsWSJfc8qUKQ0NDfI1nzx5YmdnJ6ugp6d39uzZtpwRuru7A8D//d//teU4kO7evWtraysfj7u7+9OnTxtFrq2tLV9HU1Nz06ZN8nU6fEYoey8AZGdnEwQhEAjCwsJkhx0ANDQ01q5dKxaLZW8pLCz09fVtdMx1dHRkFfbt2yfrqKamptnPJZFIRBDEhg0bAODcuXOy90ql0s2bNzf6HjN79mw2my2r8/LlSwAYP378mTNn9PT0ZNVsbW2fP38uqyYWi1esWCH/swAAjUaLjIx88+8G9SyYCJEScblcTU1NAHj58mVb6gcHB3/xxRcREREvXrxITEzctWuXkZERg8GQ//QnE6G9vb2JicnOnTujo6Nv3rz54sWLZhuUSCQeHh67du26d+9eZmbmo0eP1q9fr6mpaWZmVlFRIatGJkJHR0cHB4fjx48nJib+8ccfjo6O8N8UXlZWZm5uTqPRPvvss4yMjMzMzE2bNuno6JAnha0kwuLiYvJDtrKysi3HgSCIlJQUJpPJYDC2bNmSlpaWkpLywQcfAICpqWlBQYGs2urVq1etWnX16tXU1NTnz5+Hh4eTYR89elRWpzOJMDU1lYy8pqaGIIjFixcDQP/+/W/evJmdnX3lyhVXV1cAWLNmjewt06dPB4CVK1cmJCTk5+cnJSWdPHly4sSJsgryiVAkEkVGRgYEBADAoUOHIv8hlUqJ5hLht99+CwC9evU6c+ZMVlZWVFQU+d4JEyaQbyH+SYR2dnZ6enpffPFFTEzM/fv3Z86cCQD+/v6ypn777TcAGDJkyO3bt/Pz89PT02/cuLF8+fLo6Og2/o5Qj4GJECkR+ZHEYDBkH1LtRaa9sLCwRiUA0OFv7mTak//EJ0uMjIxKS0tlhbGxseQHpaxk3bp1APDee+/Jt0bmp9YT4cOHDwHAwsKi7UFOnDgRALZs2SJfSOaqFStWtPLGV69eaWlp9e/fX1bS4UTI4XAmTZokyx/JycnkUZL/DpGXl6etra2hoZGTk0OWGBkZ2drattKRfCIktXSPsFEirKqq0tXVpdPpz549kw+SPG++ffs2WUL+rwOAY8eOyaqJRCLyK0Jubi5ZMn/+fABodIaN1BNOn0BKVFdXBwB6enqNLkC13aRJkxgMxpMnTxqV+/r6jh07tmNtTps2DQCatrlo0SIrKyvZ04CAAH19ffmx+5cvXwaAjz/+WP5dZHZsHXkcDAwMGpWfOXMmWM68efPIci6XGxERoauru2bNGvn6ZGK4dOkS0fJECxcXl759+6ampvL5/DcG1tS5c+dWrly5cuXKmTNnOjs7X79+nclk7t27l+wXAJYvXy5/W9Te3v6dd96RSCTkwQEAY2PjmpoaWTZSoFu3bvF4vEmTJnl7e8sK9fT0yO8ijeaHWFlZLVq0SPaUwWCMHj0aAGS/UBMTEwB49OiRwuNE3Q4mQqRE5B0aHo/Xxvq1tbWbNm3y8fGxtrYmZ7MxmUyxWNx0fkXTO38tyc/Pf++997y9vU1NTck2BwwYAACVlZWNarq5uTUqsbKyKi8vJ7NOQ0NDdna2gYGBk5OTfJ1evXo1HQTUCHkcmmam0tLSxMTExMTE+Pj4O3fuPHjwgCzPzMwUi8VOTk5GRkby9V1cXIyNjSsqKioqKsgSkUi0f//+4cOH29raMhgM8gdMSkqSSqUt3X5r3aNHjw4fPnz48OFLly6JxeI5c+bExcUNGzYMANLT0wGAvCEqj7wjmJaWRj4NDQ3l8Xj9+vWbOHHi999/n5KS0oEwmvXixQsAIAffthIAyc3NrdHXL/JbTllZGfl00aJFGhoa77//vq+v7+effx4ZGdmWmaCoR8JRo0iJbG1t6XS6SCQqKCiQnzfWrPr6+iFDhmRkZPTr1+/tt982NTUlx0Rs3rxZLBY3qtzGsZqvXr0KCAioqakZOnRoSEiIiYmJhoZGTU3Nzp07JRJJo8q6urqNSuh0OnnlhEajcblcgiAsLCya9mJpadn6VEhyfE15eTmXy5UfvrF27dq1a9cCQG5uLnnhjkRezGx2+KKVlVVtbW19fb2lpSVBELNmzbp69aq9vf3UqVOtrKzIkZmHDx/OyclpetDa4ttvv121ahUAMJnMRqNbyajkT5pJZJxsNpt8+uWXX7JYrIMHD968efPmzZsA4OLisnfv3smTJ3cgnqYBND0sjQIgNfvbBACpVEo+HTRoUExMzJYtW6Kiop4+ffrtt98aGhquW7du48aNTYfmop4Nf99IiYyNjb28vJ49exYZGbl06dLWK//6668ZGRkrVqw4dOiQrJDD4WzcuLFp5TZea/3222+rqqp+/PFH2Z08AIiLi9u5c2fbfoJ/kRd4y8vLm74kO8loiZOTk42NTUlJyb1798i7bq0jL6I22xc5xYI8U/zrr7+uXr0aEBAQHR0tP3b0+PHjb+yiJTo6OuQ1w5aiavrDkiWyyRU0Gm358uXLly8vKiqKjo6+fPnypUuXZsyY8ejRIz8/vw4HBi0flkYBtN2QIUNu3rzJ4XBiYmJu374dHh7+5Zdf0mi0TZs2dSZO1O3gpVGkXKGhoQDw/ffft3TdSXbiQl5Dk90nI5FjGTrcO9nm22+/3ajNDjSlra3t6urK4XAyMzPly3Nzc5uuDdYUOTFj165dsjOSVri7u2tqamZnZze6vJmRkVFXV2dlZUWemJKjV2bMmCGfBWtra3Nyctr0I7VT3759ASAxMbFReXx8PADI37cj2draLliw4I8//vjiiy/EYrHsJmJT5NDiN57CkgE0XXq0pQDaSF9fPyQkZO/evZGRkQDwxx9/dKwd1H1hIkTKtXz58j59+qSlpYWGhjY0NDR6NTs7e8KECeRj8mqn/MqWUqm0k0tFkwkjLy9PVsLlcjtwOkgih+Dv2bNHvnD37t1tee9HH31ka2t7//79NWvWvPETX0dHZ+LEiXw+/8cff5Qv3759OwDMmjWLfNr0pwOAbdu2NT3OCvHWW2/RaLTffvtNdocSAF6/fn369GkGgzFjxgwAkEgkTW+F2tjYAEArUZErABQWFrYeADkj/ubNm/L3Hdls9k8//QQAs2fPbtePI5si2SgMJR091JXhpVGkXHp6ehcvXhw/fvypU6diY2NDQ0PJNTaLiooiIiIuXLhAng0AQFBQ0N69ez/55BM6nT5o0KDi4uI9e/ZkZmY2mjDeLkFBQREREYsWLdq5c6e7u3tmZuZXX32loaHRsdbWrVt37Nixw4cP6+rqLl68mEajnThxIjw83MLCQj43NMvc3PzSpUuTJk3av3//rVu3FixY0L9/f319/YqKioyMjJMnT8I/FzxJ27dvj4yM/Oabb4RC4ezZs4VC4ZEjR44fP25hYSG7cDd8+HBNTc3Dhw+Ty6bw+fwTJ0789ttvtra2RUVFHfsZW9G3b9/ly5cfPnw4MDBw27Ztrq6uaWlpGzZsEAqFn3zyCXkPmM1mOzo6Lly4MCgoyNnZmU6nx8fHf/HFF3Q6XZa/m/L39weAjz/+ODU1lczuy5Yta7S0Anl8tmzZ8vHHH0+YMGHHjh2+vr55eXlffvllSUnJ9OnTAwMD2/XjTJ482cjIaNasWS4uLqampjk5Od988w0AzJkzp50HBnV/lE3cQOqkpKRk2bJljRYEodFoo0aNunfvnqzaxx9/LH/zz8nJKTk52cDAgBwYQpKtLNOWfhsaGsjJEjJDhgwhR8yPHj1aVo2cR3jkyJFGbydni0skEllJcnKy/KhRAwODy5cvt2VlGVJ+fv6iRYtkuV/G2tr6888/r62tla8cExPj7OwsX61///7ya6MQBHH8+HEmkymrYGRkdOnSpaCgIACQrezamQn1jQiFwv/973/yY0m0tLQ+++wz2SFis9lN17ozNTWVX1Kn6TxCsVi8evVq+WFELa0sQxDEzp075QfC0Gi0xYsX83g8WQXZyjKNgie/QMgWKiJHjcrHyWAw3n//faFQ2PqBQj0Pjegpe8Khro/D4Tx8+LC4uLihocHKymrQoEGNlhADgOzs7MTExLq6Omdn55EjRzIYjNzcXBqN5uDgQFbgcrllZWVGRkZvnLQgk5KS8uzZM7FY7OnpOWjQIIlEkp+fr6OjQ16yA4CampqamhoLC4tGU/0KCgpEIlGj+RINDQ3R0dF5eXnm5uZjx441MjIqKipqaGjo3bt305OYVo5DSUmJQCAwNjb29PTs27dvs+epYrH44cOHmZmZGhoaffv29ff3b9pFeXn5o0ePSktLWSxWUFCQvr5+SUkJn8+3t7cnMxZBEFlZWdra2k0XBZVXUVHBZrPNzMwazdloqri4+MGDBzU1NWZmZqNGjWo6jDM3Nzc1NbWsrExLS6t3796DBg2Sz9Z1dXXl5eUWFhaNVhAFgMrKyvr6egBwdHSk0WhVVVXV1dU2NjaN1gitqam5f/9+aWmpoaHh0KFDG6VecpSy/O+XVF1dXVtba2lpKWutrq4uPj6+uLhYKBTa2tr6+Pg0HROL1AEmQoQQQmoNB8sghBBSa5gIEUIIqTVMhAghhNQaJkKEEEJqDRMhQgghtYaJECGEkFrDRIgQQkitYSJECCGk1jARIoQQUmuYCBFCCKm1rpUIiX+WP25JxzbdRgqEvwJq4fGnHP4KqEWu8K7YNrvWWqNisVhHR0ckErVUgc1mN1oWGakSQRA8Hk9+lwCkYhwOp9EK1EjF8FOIWgKBgMFgyG+B0nld64wQIYQQUjFMhAghhNQa7lCPkPoRc4BfDIJyEFSAqB4kPBDVg5gDUhGI6oCQ/l1NIgAJv5m3M3SBrt1MOY0Omk32MtQyBqC9oYSuBYwm19tb6kXLpPkfqtlGZDSYoKHT4qt/19EBDeYb6qCeCBMhQj0dNx9qnkL1U6hJgrp04JcATQN0WMC0BKYFaBqChg5oGgFDDxj6oOcAtH+2CG4peYh5IG1oppyQgqiucaGwFuC/AxE4rxuXSIUg5ra1F2FN8z9ms43ItJTU/1OHDxLBG+oAGAAA0ECr8a7CbdJSdu8wGgM0u8ANS00DoKkom2hJpYTXl2A9SoFtYiJEqCcixFASCbmnoOQW0Bhg6gOmPuC0FIy9Qdf2zedGqGVsNtvAQB+EtR15c0vZvcMIMYjYimywY0RsIFQ0mFYsFNKNvBXbJiZChHoSAiofQ+4pyD8H+k7Qez4M2A66dlRH1fPQWrxC27qOvQvJkQoEdIUOGQVMhAj1HNx8eLwE+MXQ+x0YFwv6zlQHhFD3gIkQoR4h/zwkrAbXd8Hri39v8iGE2gATIULdnKgOEv4HVQkQeBNMfaiOBqHuB+cRItSdVTyEG/1AywQmJGIWRKhj8IwQoW6LnQUP3oIhR4E1kepQEOrG8IwQoe5JzIEHM6DfVsyCCHUSJkKEuiMCHoeBqS+4rKA6EoS6Pbw0ilA3lPYt8PJh7D2q40CoJ1BMIhQKhUeOHMnKyho4cOA777xDo9EaVbh27VpxcTH52NjYeM6cOQrpFyF1VBoJr36GcXEKXqwLIXWlmEQ4d+7c2traWbNm7d69Oykpac+ePY0q7N2719DQ0N3dHQAsLCwU0ilCaojOy4PYRTD8HOjaUh0LQj2EAhJhenp6ZGRkSUmJgYFBSEiIt7f3559/bmpq2qjaO++8M2vWrM53h5A60366HLy/AMsRVAeCUM+hgMEy9+/fHzx4MLlls7OzM4vFevLkSdNqkZGRe/bsuX79ulQqbfoqQujNap/R+QXgspLqOBDqURRwRlhaWip/tdPKykp2O1DG3d1dU1OzrKzsl19+2bdv340bNzQ0ml8FSiqVLlu2TPZ06NCh8+fPlz0VCASampqyp8ePH4+Pj2/aCI1G+/zzz62trcmn33zzTWlpadNqOjo6O3bsICPh8Xiff/65SCRqWs3FxWXt2rXk46ysrB9++KHZyEePHj1z5kzycXR09IULF5qttmTJEj8/v24aP0EQ8+bNGzFiRDeNH7r58adXxS6YOnpQgwhA1B3jh25+/Mn4yU+z7ht/dz/+Xl5eDAaD0eZ1tzU1NVtKNzIKSIQMBkMikcieikQi+VxFOnjwIPlg06ZN7u7u169fnzp1arOt0Wg02e8JAPr06SPfmqampvxTDw8Pgvjvxmb/NGJiYiKr6ePjU15e3rSarq4uk/n3Ppx6enp+fn7N/iLt7e1lTVlaWsqHJ8/V1VVWzd7evqVqLBZLVg3jx/jbEb+0gf7sjHm//d01fgDo1sdfLn6yZveNn3zcfeNvVyKk09984ZPW7IFolyNHjhw5ciQ2NpZ8amdnd+LEidGjR7dUf9y4cRMnTpR9R5AnFot1dHSaPZokNptNXoNFlCAIgsfj6em1vA84Up6sw1ByizPwhL6+PtWhqDX8FKKWQCBoVyJsCwXcI5wwYUJSUlJeXh4AxMbGCgSCoUOHAsDr169TU1MBQCwWy3JbSUlJQkJC3759O98vQuol6zDeHURIGRSQVFks1tq1awMDA8eMGXP9+vVt27aR58u//fbbs2fP/vzzz7KyMj8/v6FDh2ppaUVGRs6cOTM4OLjz/SKkRqqegLAGrIOBy6M6FIR6GgVcGiXFx8e/evVqwIABnp6eZElhYSGPx3NzcwOAFy9epKenSyQSb29vDw+PlhrBS6NdHF4apUxcGBi4gecGDoeDl0aphZ9C1FLGpVGFJUKFwETYxWEipIaoDq44wuQXwLTCREg5/BSiVhe9R4gQUq7XJ8BmPDCtqI4DoZ4JEyFCXR4Ok0FImTARItS1VcSAVARWo6iOA6EeCxMhQl3bq0PgshKg8Y4uCCFFwUSIUBcm5kDRn+C0mOo4EOrJMBEi1IUV/QmWI0Cr8V4uCCEFwkSIUBeWdw7sZ1MdBEI9HCZChLoqERvK74HdNKrjQKiHw0SIUFdVdAUsR4KmEdVxINTDYSJEqKvKPw/2c6gOAqGeDxMhQl2SqB7K/wLbKVTHgVDPh4kQoS6p8DJYBYGmIdVxINTzYSJEqEvC8aIIqQomQoS6HmEtVDwA1iSq40BILShyJwvUmFQEnGyoS4P6l8DJBVEdEGIQsUHaABIBaJmBtjlom4G2OTAtwbAPmPTHIYIIAKDwMliPxeuiCKkGJkJFkzZAWTQUXoHyv4DzGnR7gZEnGHmAmT9omQCNAZoGQNcCDSY0VEJD1d//Vj2BnGNQmwra5mDSH4z7geVIsBwBdG2qfx5Ehfzz4LiI6iAQUheYCBVEVA9Ff0LhFSiNBGNvsJ0Kru+CYR+ga7WjEUIKnByoTYGaZHi2Geqeg8UIYIWATQgYuCgtdNTFCKuh4iEMP0d1HAipC0yEndZQCRk/wqtfwGIY2E0D/wOgbdHBpmh0MHABAxfo9Rb0+xqENVB6B0puQ/oO0DIFpyXQ+x3cnbXnK7gMNuOAoUd1HAipC0yEncArhJd7IOcE2M+G8XGg76Tg9rVMwH422M8GIKD8AeSEwzUPsBgOTovBdkr7zjVRN5J/DpzDqA4CITWCo0Y7RMyDxLVwoz8AHSalwqBfFJ8F/4MGliNhSDhMLwD7WZB5AK44wotdIKpXZqeICsIaqHyM40URUiVMhO1XEQM3+4OwGqZkgs8e0GGprmuGHjgugjFREHQTalLgqjOkfAaCMtUFgJStJAIsRwBDl+o4EFIjmAjbQyKApHUQMxcG7oaAE6BtRlkkxv1g6EkY/wRE9XDNExI+gIYKyoJBClR8A1gTqQ4CIfWisET46tWrx48fNzQ0tFShvLw8Nja2oqLbfl5XxcNNH+AVwsRnXWVnHH1H8NsPU14CXQOueUL6DpAIqI4JdQIhhZJbmAgRUjEFJEKpVLpgwYLg4OB169a5u7tnZWU1rfN///d/np6eX3zxhYeHx++//975TlWt4BLcnwz9tsCwM1SeCDZL2wJ89sK4R1CVANf6QO7vAATVMaEOqU4AbQvQc6A6DoTUiwISYWRk5IMHD549exYTEzNz5swvv/yyUQU+n79mzZqLFy/evXv3woULa9as4fP5ne9XdbIOQ8JqCLrVpdd+NHCBEX/A0JOQ8RPcHgI1KVQHhNqv+AbY4jAZhFRNAYnw/Pnzs2bNMjQ0BIDQ0NCLFy9KJBL5CpGRkaampiNHjgSAUaNGGRkZRUVFdb5fFXm+FV7sguC/wGQg1aG0gcVwGP8IXFdB9DhI+QyvlHYzxTfAZgLVQSCkdhQwjzA/P3/KlL93TXN0dGxoaCgrK2Ox/h1LWVBQ4OjoKHvq6OiYn5/fUmsEQdy5c0e+srOzc+eD7AhCAgn/g4oYGHtfpUNDO4sGTqHAmghJ6+G6Fww6BNZjqA4JtYGgHNivwGIY1XEgpHYUkAj5fL629t9LYjKZTADgcrnyFXg8npbWv7O/mUwmj8drqTWpVPrtt9/Knk6aNGnlypWyp1wul0ajdT7mNyMkzMRQmqhGMDSCkOgDh6OKThVJD7wPMkpvaD9aIrYcJ+y7jWDod75RgiD4fD5B4D1IxdMsuKRhHiTgNQC0OOIMmvxxIdVT3acQao5AIGAwGAxGW5MXk8l8Y2UFJEIrK6uqqirycWVlJQDY2NjIV7C2tq6urpY9raystLa2bqk1DQ2NVi6cEgShr6+AD/Q3S1gNUg6MuaXXrZe9dpkDDiGaTz/S/Gs4DD0J5kM62R5BEHQ6XU8PV/9SgqoosJ/alv/eKvoTQC1Q3acQag7jHwpsUwH3CP39/WNiYsjHMTExnp6ejf6X+Pv7JycnczgcAGCz2SkpKf7+/p3vV4le7IayezD8XE/Y/EHTEAb/BgN3wV/TIflTkIqoDgg1hxBDaSTYjKc6DoTUkQISYWho6OPHj7/99tsrV66sX7/+ww8/JMsnTpx44MABAOjTp8/o0aMXLlx469atRYsWBQcHu7m5db5fZcn/AzJ+gMAboGVMdSiK02sGTEyB2ucQORzYr6iOBjVR8RD0nUHH5s01EUKKpoBEaGlp+ddff+Xm5p48efLrr79etmwZWT558mRvb2/y8ZkzZ7y8vH799Vdvb+9Tp051vlNlqYyFhPdh1DXQs6c6FEVjWkHgn9B7PkQMg+wjVEeD/gsXlEGIOrQuNfBBLBbr6OiIRC1evmOz2QYGBsrqnpMDkcNh0CGwnaKsLrqCunR4tAi0LWDIkfaOhiUIgsfj4T1CxbvhDYN/A7PBb6zI4XDwBhW1lPsphN6kvYNl2gK3YfqHsBaiQ8B7Sw/PggBg5AnjHkHqFrjlB/4HwW461QE1h/0Ksg5DTRLoO4GBK+i7gIELGLiCBpPqyJSAmw+CcjDt2jfOEeq5cNFtEgGPF4PtJHBZTnUkKkHXhP7fwIgLkLQeHi8FUR3VAf1DKoKCCxAVDJEjgEaHPh+DiQ8IyuD1MYiZA5d7QdavPXABueIbYDMeaPjHiBA18IwQAADSdkBDFQz/g+o4VMs8ACYkQ9IncBlOG6wAACAASURBVM0TBn4Hvd+hMhh+Mbw6BNm/goEruK6CXjObGbJb+xzilkHu7zD4MBh04fFW7VV8A3rPpzoIhNQXfgkFKIuCzP0w7CzQNakOReUYeuB/AEZegpffw90xUP+SghgqYuDhPLjuDcIqGH0Hxt4Hh7ebn7hi7AXjYqHXTIgYBmnf9pCpIBIBlP8FNuOojgMh9aX2iZBXBLELYehJ0LWlOhTqmA2C8U+g13SIHAkpn4NYJWuXiDmQ/RvcHAhxy8BiGEx7DX77wcjzDe+i0cH9AwhJgIqHEDFERaEqVUUMGHuBlinVcSCkvtQ7EUpF8HAuuP8PrIKoDoVqNA1w+x9MTAFuHlx1hrRtIKxVVl/VifBkFVx2gOIbMGAnTH4Bbv8DTcN2tKDnAIHXwdgbkjcqK0iVKb0DVrgYLEJUUu9EmPwJaJuB5waq4+gydGxg6EkYGw3sV/CnCyRvBEG5whrnl0DmAbjlCzFzQM8eJj2HERfBZhxAR5dt9NkLhRehIkZhEVKi9A5Yj6U6CITUmhoPlsk/B4V/QkhCxz+IeypDDxhyDLi5kL4LrnmA3TSwmwbWwcDQ7Uhr3FwouAgFF6HuBbAmwoAdYDVGMSMktUzA7wA8DoOJyaCho4AGVU9YDexXnV8DFiHUGeo6oZ6dCRHDIegWmPoooLUeTFAG+eeg8ApUxYNVEMGawjcaqmvu3lomI8RQlw5VCVAdD5WPgVcEvaZDr5lgNRroWi2+q8Mevg26vWDgd4pvWQXy/4CccAi83vZ34IR6yuGEemopY0K9WiZCMQ8ihoDbanBZ0dmm1IewFopvQOEVoiKG1lAFunag7wh6vUHbDKRCEHNAWAtiDghroPY56NmDqR+Y+YGpP5j5A01DiYE1VMD1fjDqKph1wwnpT1aBoTv0+bDt78BESDlMhNTClWUUJOF9MBmAWbB9tIyh93zC4W0ej6enwwBuHnBygZsLwhqgawJDH7RMgKEHmkZg3A80VfgxoW0Bvj9A3FIISVTKGadSld0Ft/epDgIhdad+iTD7N6iKh/FxVMfRndG1wcCtC01pd5gL+efg+Tbot4XqUNqDmwciNhh7UR0HQupOzUaN1qRA8mcw4g9g4LLRPYvffsj6BdhZVMfRHqWRYD0Gx2ohRDl1SoSiOoiZDX4/gmEfqkNBiqZjAy6r4MVuquNoj9K7OHECoa5AbRIhIYaYeWATAg5vUx0KUg731ZB/HgRlVMfRRgSUReFUeoS6ArVJhIkfAkjB93uq40BKo20BDvMgcz/VcbRNTQpomfTA/Z8R6obUIxFm/Qqld2HYWaCp3+AgteKxDl4dAjGH6jjaABeUQajLUINEWHIbnn0Bo66CljHVoSAl03cEqyDI+o3qONqg9C5eF0Woi+jpibD+BTxaBMPPgoEL1aEglei7EV5+39V3aJIKoTIWrAKpjgMhBNDDE6GgHO5NAp/vwXIU1aEgVTEZAAaukH+W6jhaVfkIDPuAlgnVcSCEAHpyIuQVwN0gcAqleON1pHqen0D6LoAutHZgY6V3wBqviyLUVfTQRFj/AiJHgONi8PqC6lCQytmMB5oGlNymOo6W4UgZhLqSnpgIq+Lhzmjw+hI8P6E6FEQRj48gfRfVQbRAVA91aWAeQHUcCKG/KSYRpqamBgYGslisKVOmFBQUNK3w7rvv+v3jrbfeUkinzSu+Cfcnw5Cj4LxUib2gLs5+LnCyofop1XE0pywazAO66waKCPVECkiEEolk6tSpU6ZMef78uYuLy4IFC5rWyczMnDNnzqFDhw4dOrR169bOd9oMqRAyfoTHS2HkFWBNUEoXqLuga4LLSsg6RHUczSmNxOuiCHUpCphgHhkZKRaLP/roIxqN9s0335ibm798+bJPn8breTo5Ofn6+na+u2ZIRfD6ODzfBkYeMPYeGLorpRfUvTiHwrW+4LMHGF1s977SOzDsDNVBIIT+pYAzwvT09IEDB9JoNADQ09NzdXV98eJF02pfffWVv7//woULMzIyOt/p3wgp5J+H656QcwwCjkHgDcyC6G9Ma7AcCXldbB4FNx8aqsC4H9VxIIT+1dYzwnPnzjUt9PX1dXZ2rqqqkt+v2djYuKKiolHNd999187OTktL6/jx48OHD3/+/LmVlVWzHUkkEhOTf+dXLViwYPv27bKnXC6XzLgk7eT36fx8Yf+fJaZDAAA43WFtre6MIAg+n08QXXhmghwN24VaGd/yreZSHci/NPOva5gHCbi8DrfA5XIVGA/qgEafQkjF2rtDPZPJfGPltrZ1+fLlpoWmpqbOzs4mJibPnz+XFdbV1ZmamjaqOWvWLPKBj4/P/fv3b9y4ERoa2mxHGhoaOTk5sqdGRkZ0+r+nrQRB6OvLXekafhwAcNSByhAEQafT9fS6yW6OetPh+Uf6oiwwGUB1KP+ouQ+9Jvzn/3D7dfLtqJMafwoh1WL8Q5FttrHeqVOnWnrJxcXl0KG/RyU0NDTk5OS4uLS2npmhoaFAIGilgvwZIUIdR6ODUyhkHwG/n6gOBQAACCmURsHArjqvAyF1pYB7hBMmTKirqzt79ixBEPv27XN2dh4wYAAAXLly5eDBgwDAZrPv3LnT0NAgEomOHz+ekJAwZgwuq4FUwjkM8k6DuOOXIhWpJhm0zUC3F9VxIIT+QwFnl9ra2ufPnw8LCwsLC3Nzc/v999/J8qysrJcvXwKASCRat27dixcvGAyGq6vr+fPn3dzcOt8vQm+mawdmg6HgAjgupDoUcuJEMNVBIIQaoylw4INUKpW/n9eBCmKxWEdHRyRqcesANpstPzAHqRhBEDwer9vcIyQVXoaX38PYv6iOAyAqGNw/ANspnWmDw+HgDSpq4acQtdo7WKYtFLnEWutJri0VEFI828nAzoa6dIrDkAigKg43QkGoC8LMhHo6GgOclkDOUYrDqHgAxv1A05DiMBBCTWAiRGrAeRnknABpA5UxlOANQoS6KEyESA3oO4JJfyhoZi6s6uDWSwh1VYq834hQ1+UcBjlHwYGiVWYaKoGTA2aDqOkdtdPRo0ebLo8l09DQoK2trcp4EMnPz09JU+8UOWq083DUaBfXLUeNkqRCuNwLxj8BPQcKes87DXlnYaQCTklx1KgK6OnprVq1SlNTk+pA0L+ysrKEQuHVq1eVMWoUzwiReqBrgf1ceH0cvL6koHe8QdjdfP3117q6ulRHgf71559//vrrr0pqHO8RIrXhsgyyw4GQUtB16R2wwUSIUBeFiRCpDeN+oGUCZdGq7rc+A4AAA1xNCaEuChMhUifkkBkVK40Am3Gq7hQh1GaYCJE66f0OFN8EYY1KOy28CrZTVdojQqg9MBEidaJlDDbjIe+06noU1UFVPM4gRKgrw0SI1IzzUsgOV113xTfAahQwuuGEE4TUBk6fQGrGeiwIV0FNsoq2rS+8ArbTVNERQu2UmJhYWVk5fvx4WcmTJ08MDAw8PDzq6+vv3LkjKzcxMQkKCqIiRhXBM0KkbmjguBByjqmiK6kISiLBdqIq+kKoPQiCmDdv3tSpU4uLi2WFu3btOnPmDADk5+e/9dZbZ/4RGRmpmqhqa2tpNJpUquo5TnhGiNSPUyjc8oOBO4Gu5IWyyqLAyAOY1srtBamHsrKy1NRUXV1dPz8/LS0tAMjJycnIyDA3N/fx8dHQ0AAAHo9XU1NjZmb28OHDXr16tbIF+oMHDyQSyaxZs06cOPHpp582W+fcuXOtxJOenp6Xl8disfr160ej0chCHo8XHx+vp6fXv3//wsJCR0dHsryuri4pKcnAwGDAgAFkqJWVlQwGQyAQJCcne3h4ODg4AEBRURH5c9HpdFtbW5UtZYeJEKkfPQcwGQCFV8B+jnI7KvoTx4sihdi3b9+OHTuCgoK4XK6Njc3PP/986NChEydOuLu7Z2Vl8fn8v/76S0dH5/79+2vXrjUxMbGxsZk2bVpJScnXX38tf5FTJjw8fPHixUOHDl29evWGDRtkmayNPv/888ePH/fu3Ts5OZnFYl29epVGo5WVlQUGBtrb2xsYGAgEgujoaC6XCwA3b95ctWrV8OHDS0pKRCJRRESEjo7Opk2bcnNz6+rqHBwcbt++ffr06ZCQkIMHDwLAxo0baTTa9u3bnZ2dFXL03ggTIVJLzssg67CSEyEBhVdhtIquKSHlyjqsulk3DD1wWy1f8OLFi02bNiUnJzs5OQEAeeUwLCxs5cqVZIUZM2acOnUqLCwMAHJycp49e+bh4QEACQkJzeYSDodz4cKFpKQkR0dHPp8fGxs7bNiwptUGDfp7mfj169fPnj1b/qUtW7aQq31KpVJ/f/+oqKgxY8Z89913Q4YMCQ8PB4CdO3dGR0cDAJfLXbp06Y0bNwYOHAgACxYs+OWXXz788EMAqK+vf/DgAYPB+Pnnn3/66aeQkJBt27YdPHjw7NmzKt7FHRMhUku9ZsLTj6AuDYz6KquL6kRg6IGhu7LaR6okEaguEdIa54AHDx6MGDGCzIIAQCYJNpu9e/fupKSk8vLy4uJi2VVQNzc3MgsCgJ+fn5+fX9Mezp49O3DgQDJHzp8/Pzw8vNlE+Msvv5APevXq1eilsrKyXbt2paenV1dX5+XlZWZmjhkzJi4u7uOPPyYrTJ48eevWrQDw/PlzPp8fEREREREBADweLyEhgawzbtw4Mpt6enrK+qIEJkKkluia4LwUsn4F3x+U1UXhFbDD8aI9hfsHFHYuFAqbbrawYsUKKyurffv2mZmZbd68WSgUkuVt2Z8nPDz8+fPnZCLkcrk8Hu/HH39suquMj49PSy1Mnz59+vTpv/76q6GhYWhoaEPD37teN93OSCAQaGtry7K4k5MTi8UiH8tuAWpoaEgkkjeGrTw4ahSpK5cVkHsSxFxltY+JECnIoEGDYmNj6+rq5AsTExOXLFni4uJibGz8+PHjZt/I5XJzcnIaFWZkZCQmJsbExERGRkZGRsbGxrq7u58/f77t8Uil0uTk5BUrVjg4OOjo6CQmJpLlgwcPvnbtGvlY9sDb25vH43l5ec3+R7NnnyQdHR0ajSYQCNoejELgGSFSV7q9wHIk5J4Cl+WKb5ybC4JyMBus+JaR+hk0aNCcOXOGDRu2YMECDofDYDC++uqr4ODg1atXz507986dOy1ljjt37qxYsaKsrEy+MDw8fOLEiV5eXrKSd955Jzw8fMmSJW2Mh06njx49etGiRWPHjv3zzz9l56CffPJJYGBgSEiIgYEBj8cjz2JNTU337Nkzfvz4RYsWGRoaJicnDx8+/L333mu2ZW1t7WHDhr311lv29vafffYZOZRUBTARIjXm+i4krVdKIiy4DHZTm97sQahjDhw4EBMTk5CQYGZmFhISQpZcuHChoKBg8+bNenp65H7mAwcO3LZtm+xdgwYNIoeuyAsODl66dKl8yeLFi+3s7KRS6fr168msZm9vf/HixVbiuXz58tmzZ6urqw8ePMjlco2NjQHAysrq6dOncXFx+vr6PB4vLy+PrLxixYrAwMD79+9zudxFixaRc/OXL18uuxjbp0+fPXv2kI/v3Lnz6NGjiooKQ0PDjh+vdlLADvVisTg9PT0hIaG2tvajjz5qto5IJDp69GhaWpq3t3doaGhLmwvjDvVdXDfeob55BFzzgCFHwXyoghu+GwQe64A1ScHN4g71KqGnp1dRUYEb87aXQCA4d+5cQEBAdXX1Bx98MG3atM8++0xRjZMb83bdHeqjoqKWLVvm4ODw9OnTlhJhWFhYTk7OokWLjh07Fhsb2/RLCkJUoIHLSsj8WcGJUFgNNclgNUaRbSLU5dHp9MePH4eHh+vr6y9cuPDdd9+lOqK2UkAiHDduXH5+fmJi4siRI5utkJeXd+7cuYKCAgsLixkzZvTq1Wvr1q1Nx+MiRAGnUHj+NQjKgWmpsDaLroPVaNBgKqxBhLoDLS0tckZ8t6OKexgPHz709va2sLAAAAsLCy8vr4cPH6qgX4TeTMsYes1U8G69eaeh10xFNohQp02cOLG+vp7qKLqotp4RNrsKahsn/5eUlJBZkGRpaVlSUtJKR+T6CKSAgID58+fLnvL5fHKdOkQJ8h5he1dj6uLo9su04+bwHd4HmgL+a9E42cyqBL7vSeDxOt9aUzweT8WLbqCe4fbt27K5ht2UVCrl8XjtvUeopaX1xsptauv+/fvBwcFNy1NSUmRLGLQeh1gslj0ViUTkirHNotFo/v7+sqf9+/eXX3dVKBSqbBlW1BRBEBKJpKf9Ciz9QYfFrI4mFDG2hZ7+Kzgv19Y16nxTzRKJRD3t+KM2IAjiyZMnpaWlo0aNKisrs7KyIgdqcjgcchLh4MGDZQMJCwsLk5KSNDU1/f39zczMqIxboWg0mra2NkEQ7UqEbfni2Ka2Ro0a1ZmvEiwWq7CwUPa0qKjI1ta2pco0Gm3VqlUtvaqhoYFnhBQiCKJn/grc3qe9+gl6dXqBbDEH8k7BhCTlHaKeefzRmyxdujQhIWHkyJE7duyorq7etWvX1KlTMzMzx48f7+vrS6PRli1bduvWrT59+qSlpS1YsGDw4MFcLjcsLOzPP/9sZYGY7oVGo2nIUWDLSpxHGB8fb2lp6eDgMGbMmMWLF6empnp7ez979qygoGD06NHK6xehdnOYB6lfQfkDsBzRqXZyjoNVEOjiQLCeZvfu3ZWVlU3L9fT0Nm3aRN4s4HK527dvl7/6JePm5iabupeTk3P48OFmewkKCpLfJlfm0aNHt27dys7O1tXVzc/Pd3V1Jcs3btw4Z86cnTt3ko8//fTTy5cve3h4JCUlkRV++eWXXbt2nT59ut0/sJpRQCIsLy+fOHEieenWz8+P3JIDANauXTtjxox169YZGxt/9dVXISEhwcHBERERW7duVeVMSYTejK4JfT+D51/D6IhOtZP1C/gdUFBMqAsxNTVtdjFMfX192S1zDQ0NExOTZhOhkdG/l8q1tbVNTEya7aWlGaIJCQkjRowg5zXa29vLbkjFxcWtX7+efDxjxoxjx44BAEEQe/bsuX37dnV1NZvNxg/btlBAIjQxMTl06JDsqewGxtGjR8mr2ACwbt26CRMmpKenf/LJJ56enp3vFCEFc1oC6Ts7dVJYGgkAnT2nRF1So6VYmsVkMmV7L7TC1tZ2w4YN7eqdTqfLD1eUpWSpVCqfhskcfODAgevXrx8+fNjW1vbevXstbbqL5CkgEWpqavr6+jYtd3f/zwY0ffv27dtXaVveINRJNA3w/LRTJ4WZ+8F9DUCPGlKLuoKAgIDNmzfX1NSYmJi8fPnyxYsXZPnAgQPv3LkzePBgAIiIiCA/h589ezZu3DgXFxcAiIzE7TDbBNcaRegfToshfSeU/wWWzS8N0RpuHlTEwtBTSggLqTsfH58lS5YMGDBgyJAhhYWFbm5umpqaALBt27aQkJDi4mIajfbHH39cv34dACZNmrRq1So+n5+XlydLmah1mAgR+gdNA/qSJ4Xt/x6duR+cQoHRY1ZhRV3L7t2733333ZKSEl9fX1dXV3t7ewAYMGBASkrK3bt3CYLYtGmTtbU1AMyYMcPOzu7JkycBAQGDBw/OyMggW7hx44b8rUokDxMhQnIcF0HajnafFEr4kHMcxscpLSyk7s6cOePg4KCvr79u3TobGxvZSAsrKyv5JUdI/v7+stnYQ4YMIR80Ox4VkTARIiSHpgF9N8LzrTD6Tjve9fokWAwFfUelhYXUnVQq/f777/l8vre39+3bt3vY6k6Uw0SI0H85LoC0be0YPsorhLRvIOCEksNCam3+/PlNz/yQouCihQj9F40BfTdB8gaQ8N9cWVgN0SHgvgYsRyk/MoSQUmAiRKgJxwVg4ALRISBqdbV+MQ/uTQHWROjT/DacCKFuARMhQk3QNCDgOJj6wN3R0FDRfB1CAo8WgL4TDNyp2uAQQgqGiRChZtHAZy/Yz4bIkcArbPIqAU9WgEQIAeE4gx6h7q6bDZb56aefPvzwQx0dHaoDUVOvX7++d+9eW5ab6iE8NwCNDndGwehI0HcEQRnwioBXBEVXoe4FjL4DNFX/BZ08eXLKlCmt7N+CFOJ///tfSxv9iMXitu8BhBQlLy+P3L/v5s2bvXr18vPzU2Dj3ezX+fPPPy9YsICcTIpU79mzZ5cuXVKjRAgAHutB0whu9AOpCLRMQZcFOrag7wSB14Chq/pwzp075+LigolQqY4fP15dXd3sSyUlJT///PPWrVtVHBLy9fX18vICgNu3b3t5eal1IkSIAi4rwH4WMPSB3uKG0qgnmTVrVksvvXjx4vTp0ytWrFBlPEjZMBEi1AZaplRHgBBSFhwsgxBCSK3RCIKgOoZ/SSQSbW1tBweHlioUFBSwWCwNDQ1VRoVkeDwem822srKiOhD1VVpaamxszGQyqQ5ETYlEorKyMjs7O6oDUV9VVVVaWloGBgZtrD9//vyvv/669Tpd69KohoZGTk5Os1s8kxoaGmQb/yLVIwhCJBKRY7cQJfBPgHL4K6CWWCym0WhtPx2ysbF5Y52udUaIEEIIqRjeI0QIIaTWMBEihBBSa5gIEUIIqTVMhAghhNRa1xo12gqCIM6dOxcXF9e7d+/ly5fjcqMqIBQK7969+/jxY6FQOHz48EmTJpHlqampjx49klWbN2+eoaEhRTH2cI8ePUpNTZU9DQsLIwfLiUSio0ePZmRk9OvXb+HChTihSEkqKysvXrwoXxIUFOTq6pqdnX337l1Z4dSpU62trVUeXY8lFovT0tJSUlIYDEaj7YgvXLjw8OFDe3v75cuX6+npkYWlpaVHjx6tqqqaOnXqqFEd2Rm025wRbt68eevWra6urjdv3pw6dSrV4aiFS5cuffnllzQazcTE5L333lu3bh1ZHhkZuX///px/iEQiauPswc6fP3/8+HHZoZaN8V64cOHvv//u6up64MCBDz74gNoge7CGhgbZwU9KSlq5ciWHwwGAJ0+e7NixQ/aSQCCgOtIe5dy5c1OnTt2/f//GjRvly3fs2LFx40YXF5eoqKjx48eTfw5sNnvIkCGvX7+2t7efNWvWpUuXOtIl0R2w2WwjI6OkpCSCIAQCgZmZWVxcHNVB9Xx8Pl/2ODo6Wk9PTyKREASxZ8+eJUuWUBeXGvnwww+/+OKLRoWvXr1iMpnV1dUEQRQUFDCZzNLSUiqiUy8HDx7s168f+fjUqVPkBzFSBvJz5vbt2/b29rJCgUBgbm7+8OFDgiCEQiGLxYqOjiYI4ueffx42bBhZJzw83N/fvwM9do8zwuTkZC0trQEDBgCAtrb2iBEj7t+/T3VQPZ/88iUCgUBXV5dO//s/TFZW1nfffXfixIm6ujqKolMXSUlJ33333ZkzZ2SnHQ8ePPD19TUxMQEAOzs7Jyenx48fUxqjWjhy5Mjy5ctlT4uLi3fv3n3kyJGysjIKo+qRZJ8z8tLS0hoaGgICAgBAU1MzMDCQzAL3798fN24cWSc4ODg+Pp7H47W7x84FrCKlpaUWFhayp1ZWVsXFxRTGo274fP4nn3yyYcMG8qmZmZmrq2t9ff2JEyf69OmTm5tLaXQ9mY2Nja2tbV1d3Q8//NC/f/+amhrAPwcqpKampqWlye5XGRgY9OvXr7a29urVqx4eHklJSdSGpw5KS0vNzc1ptL/3wZb9ty8pKZH9OVhaWpIl7W28ewyWYTAYEolE9lQkEhkbG1MYj1oRCoVz5sxxc3Nbu3YtWbJ48eLFixeTj+fMmbN9+/ZDhw5RF2BPtn79evLB119/PWLEiAMHDmzatKnpnwMueqdsv/3224wZM0xN/96EZPLkyZMnTyYfr1mzZvPmzVevXqUuOrXAYDDkV98UiUTkQneampqycvJBB/4cuscZIYvFKi0tlf3xFxUVtWX5ONR5IpFo3rx5WlpaZ86caXZo4rBhw3JyclQfmLqh0+kBAQHkoba1tS0qKpK9VFRUxGKxqAut5xMKhadOnWppP+qhQ4fin4AKsFisiooKoVBIPpVlAVtbW9kVkcLCQg0NjQ7sCtA9EqGPj4+BgUFkZCQAlJWVxcTEyL6OIeWRSCSLFy/m8/mnTp1iMP69eMDn82UVrl+/Tm4bjZRBdqgFAkFkZGTfvn0BIDg4+MWLF69evQKAxMTE6urqkSNHUhllT3fp0iV9ff3Ro0fLSmS/F4Igrl27hn8CKuDp6Wlra3vt2jUAqKqqioqKmjJlCgBMmTLlypUrZIL8448/xo8f34Ezwm5zaXT79u2LFi2aPHnygwcPwsLCnJ2dqQ6q5zt58uTp06e9vLyGDRtGlkRERJiamgYFBRkZGVlaWsbHx2tra586dYraOHswT0/Pvn37GhsbP3jwwNHR8b333gMACwuLTz/9dMyYMcHBwTdv3vz6669lE6qQMhw5ciQ0NFR+BMfcuXM5HI6trW1qaiqHw4mIiKAwvJ7n1atXb7/9dn19fWlpqZ+fn6en54kTJ+h0+s6dO1euXPnnn38+evRo9uzZ5PcPcqLFiBEjXF1db9++ffPmzQ702J12n3j16lVCQoKjo+OQIUOojkUtVFZW5uXlyZf069dPU1OzvLw8Pj6+vr7ewcFhyJAhzQ7xQgpRWFj49OlTHo/n4uLi5+cn/9LTp08zMjK8vb3xdETZkpKSnJ2d5VeNqKmpefLkSXV1NYvFCggIwHu0isXn89PT02VP9fT0+vTpQz7OycmJi4uzt7eXfTsHALFYHB0dXV1dHRQURI6Xaa/ulAgRQgghhcPv8gghhNQaJkKEEEJqDRMhQgghtYaJECGEkFrDRIgQQkitYSJECCGk1jARIoQQUmuYCBFCCKk1TIQIIYTUGiZChBBCag0TIUIIIbWGiRAhhJBaw0SIEEJIrWEiRKib2bp1a1hYmGxvWIRQJ2EiREgpTpw4QaPRaDTa5cuXO9bCkSNHTp482bT8pGsxrAAAIABJREFU8uXLR48eJbfkRgh1HiZChJQiPDy80YP2Wr169fr165uW+/v7BwYGMhiMjgeHEJKDG/MipHg5OTkuLi4+Pj51dXW5ubkFBQXW1tbtbURHR8fY2LikpEQZESKEZPBLJUKKd+zYMYIgFi9eXFNTs3nz5v/7v/9r9twOAF68ePHo0aOqqiorKysvLy8fHx8AqKmpycnJkUqlIpEoMTGRrGlgYODm5ka+hcfjDRw4kE7/zxWd58+fx8bG1tfXs1issWPHWlpayr9aXl5eUFBgZ2dnZWX18uXLe/fuCQQCb2/v0aNH02g0+ZoEQTx58iQjI6O8vNzU1NTe3n7YsGE6OjoKPD4IdS0EQkihJBKJvb09g8EoLS3Nzc2l0+nu7u5NqxUVFQUHBzf6exwzZgxBEGfPnm36pzp69GjyjQMHDgSA2tpaWVO1tbVTp06Vr8xkMrdv3y7f3b59+wBg165dH3zwgXzNoKAgDocjq1ZSUjJo0KBGXTOZTKFQqJSDhVAXgPcIEVKwyMjI/Pz8SZMmWVlZOTg4jBgxIiMj4/Hjx/J1amtrR44cGRkZOWXKlMjIyOzs7IcPH+7evdvU1BQARo0aFRkZqaWlZWpqGvmPXbt2NdudVCqdOXPm1atXAwMD79+/n5WVdfLkSWNj440bN+7du7dR5YMHD54/f/7XX39NSEi4cuWKh4dHdHT09u3bZRXWrFnz5MmTpUuXxsfH5+fnJycn//7772PHjlX0QUKoK6E6EyPU08ydOxcALly4QD49evQoAKxYsUK+DnmldO7cuVKptKV2mEymtbV10/JGZ4RXrlwBACcnJz6fL6sTFxcHAIaGhvX19WQJeUbIZDKzs7Nl1dLS0mg0mvwJq4WFhYWFRbt/ZoS6MzwjREiRqqurr1y5YmpqOmnSJLJk9uzZ+vr6Z86c4fF4smqnTp0CgG3btjW6P9cBly5dAoAPP/yQyWTKCgcNGjRmzJj6+vrIyEj5yrNmzXJycpI99fT0tLOzy8vLI/4ZNGdsbFxfX5+WltbJqBDqRjARIqRIp06dEggE8+bN09bWJkv09fWnTZtWX19/4cIFsqSysrKoqMjQ0NDZ2bnzPaanpwMAeZooz9fXFwAapTRyuI08KysrgUBQU1NDPg0NDW1oaBgwYMCECRN2796dnJzc+QgR6uJw1ChCinTs2DEA4HK5O3fulBVKJBIACA8PX7hwIQDU19cDgI2NjUJ65HA4AGBlZdWonCxhs9nyhbq6uo2qkUNPZWeEn376qbW19f79+2/dunXr1i0AcHJy+v7776dNm6aQaBHqgjARIqQwqamp5GyH48ePN3313r172dnZzs7ORkZGAFBcXKyQTg0MDACgrKzMxcVFvry0tBQADA0N29UajUYLDQ0NDQ0tLi6+d+/elStXLly4MHPmzJiYmICAAIUEjFBXg5dGEVKYI0eOAMDKlSsjm1i6dClBEGSCNDMzs7e3Z7PZmZmZrbSmqakpFovf2Gnfvn0BICEhoVF5fHw8AHh7e3fsZ2GxWPPnzz979uzWrVulUil5JxKhHgkTIUKKIRQKySEwH3300dgm1q1bBwDh4eHkZdJFixYBwMaNG6VSaUsN2tra1tTUkFc+WzFr1iwA2Ldvn/wy3LGxsffu3TM2Nm7XzAepVCo/oodEXsJtaGhoezsIdS94aRQhxbhy5UpFRcXgwYObDkgBAA8Pj/79+6ekpERFRQUHB2/YsOHixYsXL14cP378mjVrHB0dy8rK4uLi4uLiZIt0+/n5vXz5csaMGRMnTtTT02OxWJMnT27ackhISEhIyK1bt8aOHfvll1/a2dk9fvx448aNALBt2zY9Pb22/wgCgYDFYr3zzjtjx451cnJiMBgJCQmfffYZjUabPXt2h44KQt0BxdM3EOopJkyYAAD79u1rqQI5fObtt98mn5aXlzdaDgYApk6dKqtPLj0jW1y7lZVl2Gz2vHnz5Gdi6Ovr//jjj/K9k/MId+/e3Sgqch2ZyspKgiAEAoH85AqSiYlJeHh4pw4NQl0bLrqNkGLk5uZKpVIWiyU/n08en88vKSlhMBj29vaywpycnMePH7PZbHNzc29v76ZnkxKJpLS0tKGhQUdHh7xKWVBQIBAInJ2dG601mpOTExsby+FwbGxsRo0aZWxsLP9qfX19ZWWlmZkZOVRHpqioqKGhwcHBQUNDgyzJz89/9uxZaWmplpaWg4PDoEGDcKFR1LNhIkQIIaTWcLAMQgghtYaJECGEkFrDRIgQQkitYSJECCGk1jARIoQQUmuYCBFCCKk1TIQIIYTUGiZChBBCag0TIUIIIbWGiRAhhJBa61qJkCCILVu2tFKhLduzIaXCXwG18PhTDn8F1JJIJApfGbRrrTUqFot1dHREIlFLFdhsNrkfN6IEQRA8Hq9dO/sgxeJwOPr6+lRHodbwU4haAoGAwWDIdmVRiK51RogQQgipGCZChBBCaq1n7VCf/wfkn6M6iJ6MBqAtFoNCL0q0xmYcOC9TUV8KVHAB8s4qrDUNJvj/DAy8HI2QsvSsRFhwEXRtwXwo1XH0WARBSIRChra2KjqrS4fcU90yEeZfUOT/w5TPof4FmPoppjWkZq5fv75w4UKqo1CMuXPn/vzzz8poWUWJ8MMPP7xy5UphYaGDg8OmTZsWL16slG4aKsBpCdiMU0rjCAAIQszjaatmsExNChRcUEVHCqfY/4d5p4Gbh4kQdUx5efmECRP2799PdSCddevWrdOnTyupcRUlQiMjoxs3bri4uERFRU2fPt3Ly8vX11fx3TRUgba54ptFlNA2h4ZKqoPokIYKYFoorDW93sDJVVhrSP1oa2ubmJhQHUVnKXWwtIoGy3z11Vd9+vRhMBjjxo3r27dvcnKyUrppqMRE2HMwyUTYhab3tFVDJWgrMBE6ADdPYa0hhJpQ9T3CgoKCly9fDh48uJU6NTU1sseGhoYaGhptbR0TYU9C1wYNJojqQdOI6lDaSaDQ/4d6DlAWpbDWEEJNqDQR8vn8efPmrVq1ysvLq6U6EonEyclJ9nT+/Pk7duyQPeVwOC29kSbh6QOw+RIAtqICRo0QBMHn86VSqWq609M041fnSXUdVdOdQtDEbD0ag8MTAbS4LkS70MFCpz6Hy/77fzWXy+1Si2CooVY+hboggUDQY/7DSCQSNpvd3gn1TCZTU1Oz9TqqS4QNDQ3Tp093cnLauXNnK9U0NDTkzwibanFNB241aJvjig9KRRCEhoaG6laW0bHUY/Che/1OORWgY6nI/4fansDPlzVIo9FwZRnKdaPPGSaTSaPRqI5CMTQ0NAwMDDQ1NbvryjJCoXDOnDkGBgbh4eF0unI6xZEyPU93HC/TUKHg/4daJgA0ENYqsk2EkBwVnRHOmTMnNzf38OHDKSkpAMBisWxsbBTcB94g7Hm0zUHQ7RKhQkfKkPQcgJsLWgMU3CxCCABUc0ZIEERhYaGmpub777+/cuXKlStX3rhxQ/HdYCLsebrjGaFA0WeEgANHkVr4/vvvZ82a5efnFxMTo+KuVXFGSKPREhISlN5NQyVomym9F6RK2mbdLxEqdhIhSa83cHMV3CZCXUxRUVFISMjmzZvr6upU3HUPWnS7oer/27vz8KjKs3/g95k5c2ayTkJCyELDGjaRRfAqlKLGEvQVzFUvakSrAlJqyw8X3le09merV2mF0tqrvwup1oICrwYEKlcpvsheRQSkEhahL1KjIIQA2ZdZzpmZ5/fHgSHGBLKc7Tnn+/lrzsnkOXcmmbnzbPdBIrQbHnuEeoxMoEcINvLcc8/t3bs3fvjkk0+qU2YvvfTSj370o4SEBONDslMixNCo7fCaCPWYI0QiBJvIzMz8wx/+oD4+evToW2+9NWTIEHNDslHRbSRC++ExEeoxR5iMoVHQzoXdtPN24y43egkNXdDyxMyZM1944YWKiorc3NzXXntt1qxZXmPq+LfPRolQxvYJ2+ExEeoyR4geIWinVyE9YOYW+7S0tGnTpq1cuXL+/Plr1qzZt2+ficGobJQI0SO0Hy4ToQ5/h96eFA2R0kgebvZxA1zDvHnzpk2blpWVNXr06EGDBpkdDuYIwcq8GSTXEYuaHUdnhC5pP0dI6BSCrYwePTozM/OZZ5758Y9/HD9ZUVFRXl6uKEplZWV5eXk4HDYsHjslQqwatR3BTZ5Ukq9Vcs9aYgpFAySlad9yUh8KIBGCfcyaNcvj8Xz/+9+Pn3niiSeKiopEUXzxxReLioo+++wzw4Kxy9Co0kiCm9wmrLsFfamjo7z09cNVJGUQ6VDaEXclBHvZv3//7NmzJUmKn1m/fr1ZwdglEWKljF35evI0TRi+RD59/g4xNAp2UVZWtnjx4g8++EDdPmgFdhka5ajTAJ3C13oZPTYRqpAIwS6ysrJ++MMfHj16NCsry+xYLrNLjxCJ0K74SoR6bCJUocoa2EVeXl5eXp7ZUXyNnXqEWCljR3wlwrA+S0YJPUIAHdklEYbQI7QpbwaFq80OosPCVdrvplclZJPSQJGALo0DOJtdEqGMvRM25c2k8CWzg+gwHYfoBUr8FgXO6NM4gKPZJRFijtCu+Lo3r35zhITRUQC9IBGCtXE2R6jbqlFC6W3oioyMjDfeeEPgX3Fxcc+eer25bLNqFPsIbYqzRIgeIVhLcXExY2aW2OYCeoRgbXwlwpAOt56IQyIE0IeNEqGExTJ2JKVRNEAx2ew4OoLpu2gLVdYA9GGbRFitV2krMJlAUg8+dlDI9eROJJdutxhF3W0AfRg0R1hVVXXw4METJ06MHDly0qRJGreu1JPbq+MHEJhLHR1NyDE7juvRdYKQiBJyKVRFsTBRso5XAXAegxLhU089dfLkydra2smTJ2ufCLFSxt54mSbUbze9SnBTYp4reI5SMQsAoCWDEuHKlSuJaN68ebq0jpUy9sZLItR1E6EqqY8QOEM0Qt+rADiMLbZPhKtJ6mF2EKAbXhKhAfcCS+rrCqK4DJHSSCxyzWcwkus62los3Knade5AgJREIiIWI6W+49/YCdf/Ac0m1xGZsyvDrSjU+25KydewTcslwlgsNm3atPhhUVHRww8/HD8MBoNut7vVt4iNFS53mhxAGUbdMcYCgYAg6HDj2fZ5XH5qOq9Y/vfraawgl1/XOD1SbqT+86jVXopIwBU8IwS+FEKVxGIUaRaYoiYJIRaiaFBQ6inSTDGZWFSINF79xpgsRK/8LG19sAqRRmLRb16QickkXPuzSyAprYPhM0EiMbGDTyYiKRaLuVxERIKLiakd/8ZO8KRc7wc0GfP4dbkBdQfEotFIyijZ3dF/OiVJEsXrvJiWe60FQSgpKYkfDho0yOu9ugpGluWWh5e/JVovJPb65nnQHGMsGo0a/FILib2EwJcuy/9+XdE6lpita5xC6gCpYptJLwUTqvdT/QkKVwtyNck1FK6m4HkhcJqUBkrqy5L6UkIuCW4SE8nlJcHFEjLJ7SN3Aknp8ZPk8V9t0uWJuZMuP5bS2vhg9aSS0PofX9M1NTampKSYHYXJzMmBREQUCYVEUbxubotzua6/OcKKifC+++5r76tut/ubPUJSasiX2cZ50BpjrO1fga4Ss6juEw5+v3KV4B9KusaZ0o8FvzL6paj7lE6X0pelJCZT5jiSepAvi/xDyZtBvl6U1Fdd0PvNT0YTPyt1ZcJbAFpwX6Fhm5ZLhF0RrqL0kWYHAbrhZY5Q71WjRJRs1Byh0kg1B+nSXjqzgeRa6ns/3fI3vMvArgxKhMuXL3/11VfPnDkjCMJHH33005/+dPbs2Zq1jlWj9sZLIjRg1WhibyF0kVhErwmkr/5K57dR1T5q+oLSR1HmOBq7lHp+lwS7VN4AaItBiXDKlCmjR4+OH+bm5mrZugGr9cBEvCRCXW89oRJE5ssSAmcpqa/WTTMqW0AV71HBT2jgHEobSS6P1pcAsCiDEmFOTk5Ojm6VQdAjtDduEqH+PUKiWMK33M2nNU6EMYUOzKamciraQ1K6li0D8MAWc4QGDEmBicRkYjGKBDq1xt1osTBFQyT5r//M7mGedJJrtWwxEqAPS0gQqHCbpV9hAN3YYOifkVyjY8l/sALrdwov/zem/0pJTyopDZq1JtfS7snk7UET30EWBMfiPxEqDeROIJdkdhygJ+snQqMK3jIxVbNqJlX7adsE6jmBxq/CjCA4Gf9Do5ggdAIOEqFB4/NMTCW524kweJ4O/4wu7KLRS6jP/VrEBcAx/nuEoUu6L9UD03GRCH1ZRlzIk0Itq5R1Vkyhk/+P/mcESWk05QSyIADZoUcoV5MXFbftjoNEWGXMRDUTUylwrovfHDxPOwspZRBN3kcpAzWNC4Bj/CfCEIZGHcD6iTCk/yZCIiJinm7MEZ5YTDl30pg/ahoRAPf4HxoNY2jUAbyZFK42O4hrCl/Uvb6aqsurRkOV9MV/07CntQ4IgHs2SITV2Dthf9bvERpQVoaIiJg7pYuJ8Phi6j+TEjQt6gRgC/wPjcrVlNzP7CBAZ9ZPhEZVdWBd6xGq3cEpx3SICIB7NugRYo7QAayfCI3qEVLX5ghPLKF+D6E7CNAm/nuESIRO4M2k8CWzg7im8CXyGbahvpM9wtAFKl9Fdx3VJyIA7vHfIwwZtGwdzORTF8sws+NoDyO5liRjtk+kkNLYuZfiX7+jvj+kxDzdggLgG3qEwAOXl9xeUhrIo3tV664I15CYbFCVMsFNbi9FmklM7tDzw1VUvpL+47DOYQFwjPMeIYuRXEsSNtQ7gJWnCQ2bIFR5/J0YHT2xhPpMp8TeegYEwDfOE6FSTx6j/hMHc1l5K6FRZWUu8/g7ul4mXEWfr6Bhz+gcEADfOE+EGBd1DiuvlwlXGbSbXuXpcN3tQ/9J/R+mxG/pHBAA3zifIzSq5D+Yz5tJIasOjYYuGjs02rGFo2c30aWP6C7MDgJcB+eJEIVGnQNzhHEdSYThKjr4E5rwdkfX1AA4mHFDozU1NZ988klNTY2WjWJo1Dm8mSRbdo7Q2JEJqQNzhAfnUr+HKWuiIQEB8M2gRFhaWlpQUDB//vyCgoK1a9dq1i4SoXNYvEdo8BzhtXuEX75FdZ/SjS8YFA8A54xIhMFg8LHHHtuwYcMHH3ywfv36xx57LBQKadO0XIO9E07hzaSQJRfLKI1U96nh2yfa7xEGK+jQf9F3/pvcPuNCAuCZEYlwx44daWlphYWFRHT77benpKTs2rVLm6bRI3QOa/YIq/bTltHU4ybKvt24i167R3jgx1TwE+oxxrh4ADhnxGKZM2fO9O/fP37Yr1+/06dPt/dkxtiOHTvih/3792/5va2Fq5EInSIpn2oP0f+MoLQbKW0kpY+g9JvIl2VaPCxCn/6GTr1KN/+JvnWPoZf2+Ek50faXzmyg0Hm64f8aGg8A54xIhIFAQJKk+GFCQkIgEGjvybFY7MUXX4wfTpo06fHHH48fNjc3C4IQP5RSRka8A2NNTVqHDG1jjAWDQcZMqfmZSXeecTX+r6v+U3fjcde591w1HzdPOU8kXP9bteZq/F/v4f9DnpTQLR8wXw4Z+BfY3NwsRiUxWB1q66LSxX9S1n/IgTBR2LCQnKbVpxAYLBQKiaIoih1NXj6f77pPNiIRZmdnt1wsWl1dnZ2d3d6T3W73NQZOGWPJyS2Wg49+QWrvqaADxpjL5UpKSjLp+smUOoHyJlw+Wp+WLEVIStf9sixC791MzWeIGMl1RIx8WTTsWRryRJIZadgX60Us8LU3QhxroOQbpDa/BBpp/SkExhKv0LJNDdtqz5gxY44cOdLc3JyUlNTU1HTkyJGxY8cacF2wObXomgGJMHSJAufo7s9IcJGUZkof9GuuUVkGkwUAnWfEYplhw4bdeuutM2fO3L59+8yZMwsLCwcPHmzAdcHmDFs+E75ECb3Im0FSuvlZkK65WAa1lgA6z6B9hGvXrh04cODSpUsHDx68Zs0aYy4KNmdcIrTY4mSPnyLtJUKLhQrAA4NKrPn9/kWLFhlzLXAKn5GJ0MBtgteFoVEATXF+9wlwMinDqT3CZIoGiUXb+FK42tAbQgHYAhIhcMubYdAdCkNWm3gTSEwmpbH1aaWRXB4UlAHoLCRC4JZxc4TWG29ss+62wfcHBrALJELglmMXy1A7C0ctGCcAD5AIgVtGbp+wWoJps+62bL2eKwAPkAiBW+gRtmLBOAF4gEQI3FIryxjA4NsNdkSbPcIQ5ggBugKJELjlzSC5llhM9wtZsKfVZo9Qxt4JgK5AIgRuCW7ypJJcq+9VIk0kuMmdoO9VOsvjb2NPvQVXtwLwAIkQeObVf0+95TYREhGRJ5Ui39hHaMGeKwAPkAiBZwasl7Fmdmmzylq4iiQMjQJ0GhIh8MygRGixlTLUTt1ta+ZsAMtDIgSeoUfYEuYIAboEiRB4ZkQivEQ+62UXyd/WPkKsGgXoCiRC4JkBdbet2c3ypLbeR4iK2wBdhUQIPPNmkqx3IrTq0GirHiEqbgN0FRIh8MyYoVErJsJvVJaxZsIG4AESIfDMm0khpy6WadUjRMVtgK5CIgSeGbGh3pKJ0J1ALEax8NUz1kzYADxAIgSeOXYfIX2jU4iK2wBdJRpzmY8++ui999777LPPJk+e/MgjjxhzUbA/qQcpDcQiJOjzl8xipNSRt4cujXeTmgjjSRoVtwG6yqAe4T/+8Y+mpqbz588fPnzYmCuCIwguktIoXKNX+3INiSl6ZdlualV325rbPAB4YNA7/Oc//zkRzZs3z5jLgYOoOyh8Wbo0buWJt1ZDo1YOFcDaMEcInNN14aiVs4v09R0UqLgN0FWa9QhPnDhRWVnZ6qTf7x8zZkyn2olGo/369YsfTps27YUXXogfNjc3C4LQjTChWxhjwWCQMWZ2IFf53GmR+rORxCY9Ghfrz4pieqhJl8a7prm5WX3go8RI48XIldgSgxdD0cSYlUK1K3wKmSsUComiKIodTV4+n++6T9YsEb777rs7duxodXLo0KGdTYRut3vnzp3xw8zMzOTk5PghY6zlIRiMMeZyuZKSkswOpIWkbFFoIr3+KhopKdtqf3KX40nMEN3y1R9cqU1Mz6dEa4VqS/gUMpd4hZZtatXQggULFixYoElT/fv316QdcARdtxJaeWi0VblRVNwG6CqDFsuEQqFgMBgKhVwuV21tbUJCgs+H6sCgBW8GhS7o1bhcTb5svRrvJk8qyXWXH6PiNkA3GLRYZunSpQMGDHjnnXc2bNgwYMCAZcuWGXNdsD9d99SHLFloVOVpcScmVNwG6AaDeoQaDpwCfI1jV422HBq1cpwAloftE8A5XXuEVk4wLfcRouI2QDcgEQLn9E6EPksWGqWv34kJQ6MA3YBECJzT9d68vPQIrXmLDABOIBEC56Q0ijRTTNa+5ViYokHypGrfsiZa1hpFxW2AbkAiBN4JJPUgWYe625e7WVatIdKyR4iK2wDdgEQI/PNm6LJw1MrjoqTWGm0gYkSWDxXA2pAIgX86rZex7C15VYJIbokiASJU3AboFiRC4J+OidDa3SzxyugohkYBugGJEPinVyK0cFkZVfxOTNbP2QAWhkQI/HNsj9DTskeIoVGALkIiBP55M3TZShiuJp/FE6GflAZU3AboJiRC4J83k8J6JELLD42q5UZRVgage5AIgX/OHRr1k9LAQZwA1oZECPxz5vYJUm9JWI+K2wDdhEQI/HNyjzDSgKFRgG5CIgT+eTN0SYTWr2TtSSGlgYM4AawNiRD45/FTNEzRkJZtKg0cLMVU626j4jZA9yARgi14MzReOGr9cVG6so8QZWUAugeJEGxB82lCK9+SN06tLMNFzgawMNHsAAC0oPntebnILmqPMCaj4jZAdxiUCOvr63ft2nX27Nn8/PwpU6aIIhIwaErzHmHI8rvpSd1HWE+RZg5CBbAwg4ZGBw8evHz58s8///zXv/71uHHjAoGAMdcFp9BjaNT62eXyHCEPoQJYmEE9s8OHD2dnZxORLMsFBQWbN28uKSkx5tLgCJonQi52qas9wqiMVaMA3WFQj1DNgkQkSVJCQoIxFwUH0WXVqOWziyeFIs3kEq2+zQPA2oyeq1u1alUwGLzrrrvae0IsFvvNb34TPxw1atSkSZPih+FwWJIkfUOE9jHGwuGwBad43W6/K3hRCYe1atATuBDNTItp16BWwuGwx+OJH3rFZOZJk60Xp43hU8hc4XA4Go1Go9EOPt/j8bhc1+nyafaJtmjRotdee63Vyd69e+/Zsyd+uHv37qeffnrTpk3JycnXaKquri7+OBKJaBUh2BiTtC4uY/1Co0RExMRUDoZwAaxNs0Q4d+7cBx98sHXrLboOH3744fTp099+++1vf/vb12jH5XL97ne/a++rsix7vd5uhgpdxhiLRqNW/BUk55BSo2VgSo0rJYes95MqivK1H1Pyk6+nFX8j9oVPIXMxxkRR1HZcSrO2/H6/3+9v76v79++/5557Vq1addttt2l1RYCrtF81ysP2CSLy+DmYywSwNiMWyzDG7rzzTr/fv3LlypKSkpKSknfeeceA64KDaHtvXhYluZ6kHpo1qB8PhkYBusuIRCgIwl/+8pdFixbde8XQoUO71tTSpUuDwaC24UHHffHFF2+//bbZUbRFTCYWpahGfxtyDUl+EtzatKapN99889y5c1ePJfQIDdXQ0PDqq6+aHYWjbdmy5fDhw9q2adDyv3vvvVeTdl555ZUHH3wwPz9fk9ags44ePbpx48ZHHnnE7EDakpBDm4dqk71YhBJyNGhHB+vWrRs4cGBeXt7lY28m+bJNjchZzp07t2LFigULFpgdiHNt3bp1+PDhY8eO1bBNy62DB+iiOz4mpUGz1jztTnhby6gl5MZSfoBuQSIEu/BmOnG2TEw0OwIA7uE2TAAA4GgCY8zsGK5F4if8AAAHXklEQVRS96j16dOnvSd89dVXubm5brcVVzE4QSAQaGxs7NWrl9mBOFdlZWVaWprPh5pq5lAU5cKFC7179zY7EOeqrq6WJCklJaWDz3/ggQcWLlx47edYa2jU7XaXl5dfo5pMOBzGVlYTMcYURUF9KRPhLWA6/ArMFYlEBEHoeHcoJ+f6C9+s1SMEAAAwGOYIAQDA0ZAIAQDA0ZAIAQDA0ZAIAQDA0ay1avQaGGPr1q07cOBA375958yZg9vcG0CW5Z07d+7fv1+W5e9+97tTpkxRzx87dmzfvn3xp02fPj01NdWkGG1u3759x44dix/Onj1bXSynKMrrr79+8uTJESNGPPTQQ9hQpJOqqqpWdwgoLCwsKCj4/PPPd+7cGT9ZXFycnY1Cd5qJRCLHjx8/cuSIKIoPPPBAyy/99a9/3bt3b35+/pw5c5KSktSTlZWVr7/+enV1dXFx8a233tqFK3LTI3z++ed/9atfFRQUbNmypbi42OxwHGHjxo2//OUvBUFIT0+fO3fuU089pZ7fvn37yy+/XH6Foijmxmlj69evX7VqVfyljq/xfuihh956662CgoJly5Y9/vjj5gZpY+FwOP7il5WVPfroo01NTUT08ccfL168OP6lUChkdqS2sm7duuLi4pdffvnZZ59teX7x4sXPPvvswIEDd+3adccdd6hvh8bGxnHjxn3xxRf5+fk/+MEPNm7c2JVLMh40Njb6/f6ysjLGWCgUysjIOHDggNlB2V8wGIw/3r17d1JSUjQaZYy99NJLM2fONC8uB5k/f/4vfvGLVidPnTrl8/lqamoYY1999ZXP56usrDQjOmf505/+NGLECPVxaWmp+kEMelA/Z7Zu3Zqfnx8/GQqFMjMz9+7dyxiTZTk3N3f37t2MsVdeeWXChAnqc954442bb765C1fko0d4+PBhSZJGjRpFRF6vd+LEie+//77ZQdlfy/IloVAoMTHR5br8B/Pvf/97yZIlq1evrq+vNyk6pygrK1uyZMnatWvj3Y49e/aMGTMmPT2diHr37t2/f//9+/ebGqMjrFixYs6cOfHDioqK3//+9ytWrLhw4YKJUdlS/HOmpePHj4fD4fHjxxORx+O57bbb1Czw/vvvT548WX1OUVHRwYMHA4FAp6/YvYANUllZ2bNnz/hhr169KioqTIzHaYLB4NNPP/3MM8+ohxkZGQUFBQ0NDatXrx4yZMiXX35panR2lpOTk5eXV19f/8c//nHkyJG1tbWEt4MZjh07dvz48fh8VUpKyogRI+rq6jZt2jR06NCysjJzw3OCysrKzMxMQRDUw/if/fnz5+Nvh6ysLPVMZxvnY7GMKIrRaDR+qChKWlqaifE4iizLJSUlgwYNevLJJ9UzM2bMmDFjhvq4pKRk0aJFf/7zn80L0M7i971buHDhxIkTly1b9txzz33z7YCid3pbvnz5Pffc06NHD/Vw6tSpU6dOVR8/8cQTzz///KZNm8yLzhFEUWxZfVNRFLXQncfjiZ9XH3Th7cBHjzA3N7eysjL+5j937lxHysdB9ymKMn36dEmS1q5d2+bSxAkTJpSXlxsfmNO4XK7x48erL3VeXl7Lm9SfO3cuNzfXvNDsT5bl0tLS9u5H/Z3vfAdvAQPk5uZeunRJlmX1MJ4F8vLy4iMiZ8+edbvdXbgrAB+J8KabbkpJSdm+fTsRXbhw4cMPP4z/Owb6iUajM2bMCAaDpaWlonh18CAYDMaf8O677w4fPtykAO0v/lKHQqHt27ffcMMNRFRUVPSvf/3r1KlTRPTJJ5/U1NTccsstZkZpdxs3bkxOTr799tvjZ+K/F8bY5s2b8RYwwLBhw/Ly8jZv3kxE1dXVu3btuvvuu4no7rvv/tvf/qYmyA0bNtxxxx1d6BFyMzS6aNGihx9+eOrUqXv27Jk9e/aAAQPMDsr+3nzzzTVr1gwfPnzChAnqmW3btvXo0aOwsNDv92dlZR08eNDr9ZaWlpobp40NGzbshhtuSEtL27NnT79+/ebOnUtEPXv2/NnPfva9732vqKhoy5YtCxcujG+oAj2sWLFi1qxZLVdw3HfffU1NTXl5eceOHWtqatq2bZuJ4dnPqVOn7r///oaGhsrKyrFjxw4bNmz16tUul+u3v/3to48++ve//33fvn333nuv+v+HutFi4sSJBQUFW7du3bJlSxeuyNPdJ06dOvXPf/6zX79+48aNMzsWR6iqqjp9+nTLMyNGjPB4PBcvXjx48GBDQ0OfPn3GjRvX5hIv0MTZs2cPHToUCAQGDhw4duzYll86dOjQyZMnb7zxRnRH9FZWVjZgwICWVSNqa2s//vjjmpqa3Nzc8ePHY45WW8Fg8MSJE/HDpKSkIUOGqI/Ly8sPHDiQn58f/++ciCKRyO7du2tqagoLC9X1Mp3FUyIEAADQHP6XBwAAR0MiBAAAR0MiBAAAR0MiBAAAR0MiBAAAR0MiBAAAR0MiBAAAR0MiBAAAR0MiBAAAR0MiBAAAR0MiBAAAR/v/MHqNfnNWdBAAAAAASUVORK5CYII=", "image/svg+xml": [ "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n" ], "text/html": [ "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", " \n", " \n", " \n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "# Again, let's plot the goal and car positions over time\n", "trajectories = reduce(hcat, agent_x)'\n", "p1 = plot(trajectories[:,1], label=\"car: AIF agent\", title = \"Car and Goal Positions\", color = \"orange\")\n", "plot!(0.5 * ones(N_ai), color = \"black\", linestyle=:dash, label = \"goal\")\n", "p2 = plot(agent_a, title = \"Actions\", color = \"orange\")\n", "plot(p1,p2, layout = @layout [a ; b])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that the AIF agent __explores__ other options, like going first in the opposite direction of the goal prior, to reach its goals. This agent is able to mix exploration (information-seeking behavior) with exploitation (goal-seeking behavior)." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Extensions and Comments\n", "\n", "\n", "- Just to be sure, you don't need to memorize all FE/EFE decompositions nor are you expected to derive them on-the-spot. We present these decompositions only to provide insight into the multitude of forces that underlie FEM-based action selection.\n", "\n", "- In a sense, the FEP is an umbrella for describing the mechanics and self-organization of intelligent behavior, in man and machines. Lots of sub-fields in AI, such as reinforcement learning, can be interpreted as a special case of active inference under the FEP, see e.g., [Friston et al., 2009](https://journals.plos.org/plosone/article?id=10.1371/journal.pone.0006421). \n", "\n", "- Is EFE minimization really different from \"regular\" FE minimization? Not really, it appears that [EFE minimization can be reformulated as a special case of FE minimization](https://link.springer.com/article/10.1007/s00422-019-00805-w). In other words, FE minimization is still the only game in town.\n", "\n", "- Active inference also completes the \"scientific loop\" picture. Under the FEP, experimental/trial design is driven by EFE minimization. Bayesian probability theory (and FEP) contains all the equations for running scientific inquiry.\n", "

\n", "\n", "- Essentially, AIF is an automated Scientific Inquiry Loop with an engineering twist. If there would be no goal prior, AIF would just lead to learning of a veridical (\"true\") generative model of the environment. This is what science is about. However, since we have goal prior constraints in the generative model, AIF leads to generating behavior (actions) with a purpose! For instance, when you want to cross a road, the goal prior \"I am not going to get hit by a car\", leads to inference of behavior that fulfills that prior. Similarly, through appropriate goal priors, the brain is able to design algorithms for object recognition, locomotion, speech generation, etc. In short, **AIF is an automated Bayes-optimal engineering design loop**!!\n", "\n", "- The big engineering challenge remains the computational load of AIF. The human brain consumes about 20 Watt and the neocortex only about 4 Watt (which is about the power consumption of a bicycle light). This is multiple orders of magnitude (at least 1 million times) cheaper than what we can engineer on silicon for similar tasks. \n", "\n", "\n", "\n", "\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Final Thoughts\n", "\n", "- In the end, all the state inference, parameter estimation, etc., in this lecture series could have been implemented by FE minimization in an appropriately specified generative probabilistic model. However, the Free Energy Principle extends beyond state and parameter estimation. Driven by FE minimization, brains change their structure as well over time. In fact, the FEP extends beyond brains to a general theory for biological self-organization, e.g., [Darwin's natural selection process](https://www.ncbi.nlm.nih.gov/pmc/articles/PMC5857288/) may be interpreted as a FE minimization-driven model optimization process, and here's an article on [FEP for predictive processing in plants](https://royalsocietypublishing.org/doi/10.1098/rsif.2017.0096). Moreover, Constrained-FE minimization (rephrased as the Principle of Maximum Relative Entropy) provides an elegant framework to derive most (if not all) physical laws, as Caticha exposes in his [brilliant monograph](./files/Caticha-2012-Entropic-Inference-and-the-Foundations-of-Physics.pdf) on Entropic Physics. Indeed, the framework of FE minimization is known in the physics community as the very fundamental [Principle of Least Action](https://en.wikipedia.org/wiki/Stationary-action_principle) that governs the equations-of-motion in nature. \n", "\n", "- So, the FEP is very fundamental and extends way beyond applications to machine learning. At [our research lab](http://biaslab.org) at TU/e, we work on developing FEP-based intelligent agents that go out into the world and autonomously learn to accomplish a pre-determined task, such as learning-to-walk or learning-to-process-noisy-speech-signals. Free free to approach us if you want to know more about that effort. " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "##
OPTIONAL SLIDES
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "### In an AIF Agent, Actions fulfill Desired Expectations about the Future\n", "\n", "- In the [derivations above](#goal-seeking), we decomposed the EFE into an upperbound on the sum of a goal-seeking and information-seeking term. Here, we derive an alternative (exact) decomposition that more clearly reveals the goal-seeking objective.\n", "\n", "- We consider again the EFE and factorize the generative model $p(x,s|u) = p^\\prime(x) p(s|x,u)$ as a product of a __goal prior__ $p^\\prime(x)$ on observations and a __veridical__ state model $p(s|x,u)$. \n", "\n", "- Through the __goal prior__ $p^\\prime(x)$, the agent declares which observations it **wants** to observe in the future. (The prime is just to distinguish the semantics of a desired future from the model for the actual future).\n", "\n", "- Through the __veridical__ state model $p(s|x,u)$ , the agent implicitly declares its beliefs about how the world will **actually** generate observations.\n", " - In particular, note that through the equality (by Bayes rule)\n", "$$p(s|x,u) = \\frac{p(x|s)p(s|u)}{p(x|u)} = \\frac{p(x|s)p(s|u)}{\\sum_s p(x|s)p(s|u)}\\,,$$ \n", "it follows that in practice the agent may specify $p(s|x,u)$ implicitly by explicitly specifying a state transition model $p(s|u)$ and observation model $p(x|s)$. \n", "\n", "- Hence, an AIF agent holds both a model for its beliefs about how the world will actually evolve AND a model for its beliefs about how it desires the world to evolve!! \n", "\n", "- To highlight the role of these two models in the EFE, consider the following alternative EFE decomposition:\n", "$$\\begin{aligned}\n", "G(u) &= \\sum_{x,s} q(x,s|u) \\log \\frac{q(s|u)}{p^\\prime(x)p(s|x,u)} \\\\\n", "&= \\sum_{x,s} q(x,s|u) \\log \\frac{q(s|u)}{p^\\prime(x)} \\frac{1}{p(s|x,u)}\\\\\n", "&= \\sum_{x,s} q(x,s|u) \\log \\frac{q(s|u)}{p^\\prime(x)} \\frac{p(x|u)}{p(x|s)p(s|u)} \\quad \\text{(use Bayes)}\\\\\n", "&= \\sum_{x,s} q(x,s|u) \\log \\frac{q(s|u)}{p(x|s)p(s|u)} \\frac{p(x|u)}{p^\\prime(x)} \\\\\n", "&= \\sum_{x,s} q(x,s|u) \\log \\frac{q(s|u)}{p(x|s)p(s|u)} + \\sum_{x,s} q(x,s|u) \\log \\frac{p(x|u)}{p^\\prime(x)} \\\\\n", "&= \\sum_{x,s} p(s|u) p(x|s) \\log \\frac{p(s|u)}{p(x|s)p(s|u)} + \\sum_{x,s} p(s|u) p(x|s) \\log \\frac{p(x|u)}{p^\\prime(x)} \\quad \\text{( assume }q(x,s|u)=p(x|s)p(s|u)\\text{ )}\\\\\n", "&= \\sum_{s} p(s|u) \\sum_x p(x|s) \\log \\frac{1}{p(x|s)} + \\sum_x p(x|u) \\log \\frac{p(x|u)}{p^\\prime(x)} \\\\\n", "&= \\underbrace{E_{p(s|u)}\\left[ H[p(x|s)]\\right]}_{\\text{ambiguity}} + \\underbrace{D_{\\text{KL}}\\left[ p(x|u), p^\\prime(x)\\right]}_{\\text{risk}}\n", "\\end{aligned}$$\n", "\n", "- In this derivation, we have assumed that we can use the generative model to make inferences in the \"forward\" direction. Hence, $q(s|u)=p(s|u)$ and $q(x|s)=p(x|s)$. \n", "\n", "- The terms \"ambiguity\" and \"risk\" have their origin in utility theory for behavioral ecocomics. Minimization of EFE leads to minimizing both ambiguity and risk.\n", "\n", "- Ambiguous (future) states are states that map to large uncertainties about (future) observations. We want to avoid those ambiguous states since it implies that the model is not capable to predict how the world evolves. Ambiguity can be resolved by selecting information-seeking (epistemic) actions. \n", "\n", "- Minimization of the second term (risk) leads to choosing actions ($u$) that align **predicted** future observations (represented by $p(x|u)$) with **desired** future observations (represented by $p^\\prime(x)$). Agents minimize risk by selecting pragmatic (goal-seeking) actions.\n", "\n", "- $\\Rightarrow$ **Actions fulfill desired expectations about the future!**\n", "\n", "- ([return to related cell in main text](#goal-seeking))." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Proof $q^*(u) = \\arg\\min_q F_>[q] \\propto p(u)\\exp(-G(u))$ \n", "\n", "- Consider the following decomposition:\n", "$$\\begin{aligned}\n", "F_>[q] &= \\sum_{x,s,u} q(x,s,u) \\log \\frac{q(s,u)}{p(x,s,u)} \\\\\n", "&= \\sum_{x,s,u} q(x,s|u) q(u) \\log \\frac{q(s|u) q(u)}{p(x,s|u) p(u)} \\\\\n", "&= \\sum_{u} q(u) \\bigg(\\sum_{x,s} q(x,s|u) \\log \\frac{q(s|u) q(u)}{p(x,s|u) p(u)}\\bigg) \\\\\n", "&= \\sum_{u} q(u) \\bigg( \\log q(u) + \\log \\frac{1}{p(u)}+ \\underbrace{\\sum_{x,s} q(x,s|u) \\log \\frac{q(s|u)}{p(x,s|u)}}_{G(u)}\\bigg) \\\\\n", "&= \\sum_{u} q(u) \\log \\frac{q(u)}{p(u)\\exp\\left(- G(u)\\right) }\n", "\\end{aligned}$$\n", "\n", "- This is a KL-divergence. Minimization of $F_>[q]$ leads to the following posterior for the policy:\n", "$$\\begin{aligned}\n", "q^*(u) &= \\arg\\min_q F_>[q] \\\\\n", "&= \\frac{1}{Z}p(u)\\exp(-G(u))\n", "\\end{aligned}$$\n", "\n", "- [(click to return to linked cell in the main text.)](#q-star-main-cell)" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### What Makes a Good Agent? [The Good Regulator Theorem](https://en.wikipedia.org/wiki/Good_regulator)\n", "\n", "- According to Friston, an ``intelligent'' agent like a brain minimizes a variational free energy functional, which, in general, is a functional of a probability distribution $p$ and a variational posterior $q$. \n", "\n", "- What should the agent's model $p$ be modeling? This question was (already) answered by [Conant and Ashby (1970)](https://www.tandfonline.com/doi/abs/10.1080/00207727008920220) as the Good Regulator Theorem: **every good regulator of a system must be a model of that system**. \n", " \n", "- A Quote from Conant and Ashby's paper (this statement was later finessed by [Friston (2013)](https://royalsocietypublishing.org/doi/full/10.1098/rsif.2013.0475)): \n", "> \"The theory has the interesting corollary that the living brain, insofar as it is successful and efficient as a regulator for survival, *must* proceed, in learning, by the formation of a model (or models) of its environment.\"\n", "\n", "

\n", "\n", "- ([Return to related cell in main text](#model-specification)).\n", "\n", "\n" ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "slideshow": { "slide_type": "skip" } }, "outputs": [ { "data": { "text/html": [ "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n", "\n" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "open(\"../../styles/aipstyle.html\") do f\n", " display(\"text/html\", read(f,String))\n", "end" ] } ], "metadata": { "@webio": { "lastCommId": null, "lastKernelId": null }, "anaconda-cloud": {}, "celltoolbar": "Slideshow", "kernelspec": { "display_name": "Julia 1.10.5", "language": "julia", "name": "julia-1.10" }, "language_info": { "file_extension": ".jl", "mimetype": "application/julia", "name": "julia", "version": "1.10.5" } }, "nbformat": 4, "nbformat_minor": 4 }