{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# 3 - Serial information processing\n",
"This notebook is part of the supplementary material for: \n",
"Genewein T., Leibfried F., Grau-Moya J., Braun D.A. (2015) *Bounded rationality, abstraction and hierarchical decision-making: an information-theoretic optimality principle*, Frontiers in Robotics and AI. \n",
"\n",
"More information on how to run the notebook on the accompanying [github repsitory](https://github.com/tgenewein/BoundedRationalityAbstractionAndHierarchicalDecisionMaking) where you can also find updated versions of the code and notebooks.\n",
"\n",
"This notebook corresponds to Section 3 and reproduces Figures 5, 6, 7 and 8 of the paper. \n",
"\n",
"Note that the notebook **is not limited to the serial case** as it computes the serial-case solutions through the general case by setting the inverse temperatures accordingly. The parallel or general case can thus be easily reproduced with different settings of the $\\beta$-values."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Two-stage serial information processing\n",
"Here, we design a two-stage serial information processing system that first processes a world-state $w$ into an internal percept $x$ and then uses this percept to compute an action $a$. We assume that the action is conditionally independent of the world-state given the percept. This simply means that there is no direct flow of information from $W$ to $A$ and corresponds to the following graphical model\n",
"$$ W \\rightarrow X \\rightarrow A$$\n",
"that leads to $p(w,x,a)=p(w)p(x|w)p(a|x)$.\n",
"\n",
"Each stage of the information processing system is characterized by an information-theoretic channel \n",
" * $W \\rightarrow X$ with the rate $I(W;X)$\n",
" * $X \\rightarrow A$ with the rate $I(X;A)$\n",
" \n",
"We want to maximize the utility function $U(w,a)$ while at the same time not exceeding some limits on the information processing rates on each stage. This can be formalized with the following variational problem\n",
"$$\\underset{p(x|w),p(a|x)}{\\operatorname{arg~max}}~\\mathbf{E}_{p(w,x,a)}[U(w,a)] - \\frac{1}{\\beta_1} I(W;X) - \\frac{1}{\\beta_2} I(O;A)$$\n",
"\n",
"The solution to the optimization problem is given by\n",
"$$\\begin{align}\n",
"p^*(x|w)&=\\frac{1}{Z(w)}p(x)\\exp\\left(\\beta_1 \\Delta F_{\\text{ser}}(w,x)\\right)\\\\\n",
"p(x)&=\\sum_w p(w)p^*(x|w)\\\\\n",
"p^*(a|x)&=\\frac{1}{Z(x)} p(a) \\exp \\left(\\beta_2 \\sum_w p(w|x) U(w,a) \\right)\\\\\n",
"p(a)&=\\sum_{w,x} p(w)p^*(x|w)p^*(a|x),\n",
"\\end{align}$$\n",
"where $Z(w)$ and $Z(x)$ denote the corresponding normalization constants or partition sums. The conditional probability $p(w|x)$ is given by Bayes' rule $p(w|x) = \\frac{p(w)p^*(x|w)}{p(x)}$ and \n",
"$$\\Delta F_{\\text{ser}}(w,x):=\\mathbf{E}_{p^*(a|x)}[U(w,a)] - \\frac{1}{\\beta_2} D_{\\mathrm{KL}}(p^*(a|x)||p(a))$$\n",
"is the free energy difference of the action stage.\n",
"\n",
"## Perception-action systems\n",
"Interestingly, the solutions above can be used for constructing bounded-optimal perception-action systems. Perception-action systems need to perform inference over a (hidden) world-state $w$ and then use the gathered information $x$ to drive a decision-maker. Classically, inference and decision-making are separated:\n",
"$$\\begin{align}\n",
"&\\text{Bayesian inference:} & p(w|x)&=\\frac{p(w)p(x|w)}{\\sum_w p(w)p(x|w)} \\\\\n",
"&\\text{Bounded rational decision:} & p^*(a|x)&=\\frac{1}{Z(x)}p(a)e^{\\beta_2 U(x,a)} \n",
"\\end{align}$$\n",
"where the last equation describes a bounded rational decision-maker as in Section 2 of the paper. The problem is that in such a case there is no principled way to specify the likelihood model $p(x|w)$. Often it is chosen to represent the world-state as faithfully as possible. \n",
"\n",
"Now compare the two equations for Bayesian inference and bounded rational decision making with the four equations that we got as the solution to our variational problem. Interestingly Bayesian inference and bounded rational decision making drop out naturally - additionally the bounded optimal likelihood model $p^*(x|w)$ is also well-defined. Interestingly it does not try to represent $w$ as faithfully as possible but rather it tries to optimize the downstream free-energy trade-off of the action-part of the system. This leads to a tight coupling between action and perception, which will be illustrated in the example below.\n",
"\n",
"## Comparison against hand-crafted perception $p_\\lambda(x|w)$\n",
"Below, we compare the bounded-optimal perception-action system against a system that uses a handcrafted likelihood $p_\\lambda(x|w)$. The model has a tunable precision-parameter $\\lambda$ that affects the information processing rate of the perceptual channel $I(X;W)$ - this parameter corresponds to the level of perceptual noise and influences the likelihood of \"confusing\" one particular $w$ with similar ones.\n",
"\n",
"The observation or percept $x$ is a discretized noisy version of $w$ with precision $\\lambda$:\n",
"\n",
"$$x|w,\\lambda \\sim \\text{round}(\\mathcal{N}(w,1/\\lambda))$$\n",
"\n",
"By using this construction it is already obvious that the action-part of the system or the utility function have no influence on the perceptual part of the system. We will highlight this further in the example below."
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {
"collapsed": false
},
"outputs": [
{
"name": "stderr",
"output_type": "stream",
"text": [
"WARNING: replacing module RateDistortionDecisionMaking\n"
]
},
{
"data": {
"text/plain": [
"RateDistortionDecisionMaking"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#only run this once\n",
"include(\"RateDistortionDecisionMaking.jl\")"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"using RateDistortionDecisionMaking, Distances, DataFrames, Colors, Gadfly, Distributions, Interact, Reactive\n",
"\n",
"#make the default plot size a bit larger\n",
"set_default_plot_size(15cm, 12cm)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Utility U(a,w)\n",
"\n",
"To illustrate perception-action systems we design the follwing example. The agent is an animal and it encounters nine other kinds of animals that come in three different size-groups - each group has three members.\n",
"* G1: Small: 2,3,4\n",
"* G2: Medium: 6,7,8\n",
"* G3: Large: 10,11,12\n",
"\n",
"The agent has sensors to observe $x$ which is a noisy version of $w$. The rate $I(W;X)$ governs how precise or noisy the percept is (for the hand-crafted model the precision is directly controlled by $\\lambda$). Medium precision mostly leads to within-group confusion, low precision $\\lambda$ leads to strong across-group confusion. \n",
"\n",
"The agent can hunt the animals of the small and medium-sized group for food but it must flee from the large animals as it could fall prey to them. Alternatively, you can change the utility function to a mating scenario, where the agent must try to find a animals of a specific size within the medium-sized group to mate with (see description below the plot of the utility function for more details).\n",
"\n",
"### [Interact] Change the utility function and see how the percept is affected \n",
"... change the utility function by commenting/uncommenting in the code-block below. If all other parameters are kept the same, we can observe how the bounded optimal percept $p^*(x|w)$ is coupled to the utility function."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"image/svg+xml": [
"\n",
"\n"
],
"text/html": [
"\n",
"\n"
],
"text/plain": [
"Plot(...)"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"#set up predator-prey example\n",
"include(\"PredatorPreyExample.jl\")\n",
"\n",
"###### Predator-prey scenario #############\n",
"#same utility as used in the main paper\n",
"w_values, w_strings, a_values, a_strings, p_w, U = setup_predator_prey_example()\n",
"\n",
"###### Mating scenario ####################\n",
"#alternative utility - mating scenario\n",
"#w_values, w_strings, a_values, a_strings, p_w, U = setup_predator_prey_example(mating_utility=true)\n",
"\n",
"\n",
"numa = length(a_strings)\n",
"a_vec = collect(1:numa)\n",
"\n",
"numw = length(w_strings)\n",
"w_vec = collect(1:numw)\n",
"\n",
"#pre-compute utility\n",
"U_pre, Umax = setuputilityarrays(a_values,w_values,U)\n",
"\n",
"#visualize utility\n",
"\n",
"plt_utility = visualizeMatrix(U_pre, w_values, a_values, w_strings, a_strings, xlabel=\"animal size w\",\n",
" ylabel=\"Action a\", legendlabel=\"U(a,w)\")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Predator-prey scenario\n",
"Possible kinds of **animals**:\n",
"* Small animals (w={2,3,4})\n",
" * Can't hear well - sneaking up on them is very likely succesful\n",
" * The generic sneak-up pattern works very well\n",
" * An ambush also works but bears the risk that the animal is not moving towards you\n",
" * For each w2, w3, w4 there is a specific sneak-up pattern that increases chances of success for that particular kind of animal - however, using it on the wrong animal, the success rate will be 20% lower \n",
"* Medium-sized animals (w={6,7,8})\n",
" * Can hear well - sneaking up on them has lower chances of success compared to ambush\n",
" * The generic ambush pattern works well but bears the risk that the animal is not moving towards you\n",
" * A sneak up might also work but chances are quite low\n",
" * There are specific ambush patterns for w6, w7, w8 but using them has no advantage over the generic ambush, however using them on the wrong animal, the sucess rate will be 20% lower \n",
"* Large animals (w={10,11,12})\n",
" * Can potentially kill you, your chances of survival are low if you sneak up or ambush them (no matter with which kind of pattern)\n",
" * If you flee, your chances of survival are quite good\n",
" \n",
" \n",
"Possible kinds of **actions**:\n",
"* Generic *Ambush*: wait for the animal to get close and then strike\n",
" * advantage: causes no noise\n",
" * disadvantage: animal might not come towards you\n",
"* Generic *Sneak up*: slowly move closer to the animal and then strike\n",
" * advantage: works also if animal is not moving towards you\n",
" * disadvantage: animal might hear or you and flee\n",
"* Generic *Flee*: run away from animal\n",
" * advantage: if animal could kill you, your chances of survival are significantly increased\n",
" * disadvantage: if animal was potential prey you missed out on food\n",
"* Specific *Sneak up* for w={2,3,4}: specific sneak up pattern that increases success when applied to exactly the right kind of animal, but decreases success when applied to another animal within the small animal group. When applied to an animal from the medium-sized animal group, the success-rate is equal to the generic sneak up pattern.\n",
"* Specific *Ambush* for w={6,7,8}: does not increase success compared to generic ambush pattern, but decreases success when applied to another animal within the medium-sized group. When applied to an animal from the small group, the success-rate is equal to the generic ambush pattern. As there is no advantage of the specific pattern over the genric pattern (but it might be disadvantegous when applied to the wrong animal), any bounded-rational decision-maker should assign no probability mass to these specific ambush patterns since using them would not increase the expected utility but would increase the informational cost.\n",
"\n",
"\n",
" \n",
" \n",
"Compared to the generic sneak-up and ambush patterns, the specific patterns require more computational resources, that is they require a larger rate on the action-channel, or in other words: they require good motor skills and a good motor hardware (which is more costly than a cheap motor system).\n",
"\n",
"If you have bad motoric hardware, you'll never be able to use the specific hunting patterns, therefore it doesn't make sense to waste resources on an expensive perceptual system that allows you to precisely distinguish individual animals. Rather a system that allows you to tell the different groups from each other suffices. Dually, if your perceptual system does not allow you to distinguish between the individual animals, it would be lavish to have an expensive motor system that allows you to accurately execute very specific hunting patterns.\n",
"\n",
"\n",
"## Mating scenario\n",
"You are a medium sized animal (w=6) and try to find a mate - potential mates are w=8.\n",
"\n",
"Possible actions\n",
"* Display\n",
" * Get the attention of the other animal which could be a potential mate\n",
"* Flee\n",
" * Quickly flee from the other animal\n",
"\n",
"Possible kinds of **animals**:\n",
"* Small animals (w={2,3,4})\n",
" * Uninteresting. Fleeing from them will cost resources, displaying to them will also waste a similar amount of resources. \n",
"* Medium-sized animals\n",
" * w=6 rival, same size as you: displaying to them might drive them away, but has a risk of confrontation. Fleeing significantly decreases your chances of mating.\n",
" * w=7 rival, bigger than you. Displaying to them will most probably get you injured, fleeing from them is the better option.\n",
" * w=8 potential mate. Displaying to them will significantly increase your chances to mate, fleeing from them has a low utility.\n",
"* Large animals (w={10,11,12})\n",
" * Can potentially kill you, fleeing from them is good, displaying to them will almost surely get you killed.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Hand-crafted perceptual model $p_\\lambda(x|w)$\n",
"\n",
"Below, the hand-crafted perceptual model can be inspected by changing the precision parameter $\\lambda$. Note that this will also directly influence the rate $I(W;X)$."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [],
"text/plain": [
"Interact.Slider{Float64}([Reactive.Input{Float64}] 2.5,\"Perceptual precision λ\",2.5,0.1:0.1:5.0)"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"image/svg+xml": [
"\n",
"\n"
],
"text/html": [
""
],
"text/plain": [
"Plot(...)"
]
},
"metadata": {
"comm_id": "eb104344-859a-48aa-89fc-1ba13cd3cf6e",
"reactive": true
},
"output_type": "display_data"
}
],
"source": [
"#set up hand-crafted likelihood model and p(x|w)\n",
"x_values = collect(1:13)\n",
"x_vec = collect(1:length(x_values))\n",
"x_strings = map((x)->string(x), x_values)\n",
"\n",
"\n",
"#slider for selecting λ\n",
"λ_vals = 0.1:0.1:5\n",
"slider_l = slider(λ_vals,label=\"Perceptual precision λ\")\n",
"\n",
"\n",
"\n",
"#use lift to connect the actual plotting-code to the slider\n",
"plt_pxgw_sl = lift(λ_vis->begin\n",
" p_xgw_hc = pogw_handcrafted(x_values, w_values, λ_vis)\n",
" visualizeBAconditional(p_xgw_hc, x_vec, w_vec, x_strings, w_strings,\n",
" wlabel=\"animal size w\", alabel=\"observed size x\", legendlabel=\"p(x|w,λ)\")\n",
" end, slider_l)\n",
"\n",
"display(slider_l)\n",
"display(plt_pxgw_sl)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"### [Interact] See how the precision $\\lambda$ governs perceptual noise\n",
"...drag the slider above to change the perceptual precision and see how observations get more and more \"blurry\" with decreasing precision. Note that low perceptual precision incurs a low mutual information $I(X;W)$ whereas high precision also incurs a large mutual information between the actual size and the observed size.\n",
"\n",
"If you see the slider but not the plot and ``\"Javascript error adding output!\"``, try moving the slider once, then the plot should appear.\n",
"\n",
"If this does not work (i.e. kernel is busy with this cell but nothing ever happens), try ckecking out the most current version of Interact with: `Pkg.checkout(\"Interact\")` in a Julia console. You can undo this and go back to the latest released version with `Pkg.free(\"Interact\")`"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Compare bounded-optimal and hand-crafted models\n",
"\n",
"## [Interact] Set parameters to compare the two cases here:\n",
"When changing the parameters make sure to re-run the corresponding cells below - in case of doubt simply re-run all cells below.\n",
"\n",
"### Cases that work nicely:\n",
"\n",
"* $\\lambda=1.65$, $\\beta_1 = 8$, $\\beta_2 = 10$ (same $I(X;W)$ for both models - almost unbounded actor)\n",
"* $\\lambda=1.65$, $\\beta_1 = 8$, $\\beta_2 = 1$ (same EU - bounded actor, optimal perception changes, given the limits on the action channel)\n",
"* $\\lambda=0.4$, $\\beta_1 = 1$, $\\beta_2 = 10$ (same $I(X;W)$ for both models - bounded actor, optimal action changes, given the limits on the perceptual channel)\n",
"\n",
"\n",
"\n",
"* $\\lambda=10$, $\\beta_1 = 10$, $\\beta_2 = 5$, mating-utility (optimal perception changes under a different utility function - compare against case above with identical parameters but different utility function)\n",
"\n",
"### Things to try:\n",
"\n",
"Change the temperature on the action-channel (with very large resources on the observation channel) and see how the perception is affected by the action.\n",
"\n",
"Change the temperature on the perception-channel (with very large resources on the action channel) and see how the action is affected by perception.\n",
"\n",
"Change the utility function (mating-utility) and see how the percept is affected by the new utility."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"#precision of the hand-crafted perceptual model\n",
"λ = 1.65 #0.4 #1.65\n",
"#the \"price\" for I(X;W) in the handcrafted case will be β1 (as given below)\n",
"#to make the two cases comparable. If you want I(X;W) to be the same in both\n",
"#cases, set β1 and run the sequential case and then tune λ until the mutual\n",
"#information terms are equal.\n",
"#\n",
"#Note that β2 will also be used for the bounded rational decision-maker that\n",
"#uses the hand-crafted perception. This should allow for easy comparison of\n",
"#the action-channels in both cases (hand-crafted vs. sequential case)\n",
"\n",
"#inverse temperatures for sequential case\n",
"#β1: perceptual channel -> price for I(X;W)\n",
"#β2: action channel -> price for I(A;X)\n",
"#β3=0 sequential case (otherwise the general three-variable case is specified)\n",
"β1 = 8 #1\n",
"β2 = 10\n",
"β3 = 0;"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Note that the simulation implements the soltuions of the general case. By setting $\\beta_3=0$, the serial case is recovered (which corresponds to the setting in the paper). However, any of the other cases (parallel or general) can be simulated by simply setting the inverse temperatures accordingly (in the code-cell above)**"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Bounded rational decision maker based on hand-crafted perception\n",
"\n",
"The code cells below simulate a bounded-rational decision-maker using the hand-crafted perceptual model and visualize the solution."
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"#compute hand-crafted perception, then feed it into bounded rational decision-maker\n",
"#with inv. temperature β2 and compute optimal p(a|x) using hand-crafted p(x|w)\n",
"p_xgw_hc = pogw_handcrafted(x_values, w_values, λ)\n",
"\n",
"\n",
"numx = length(x_vec)\n",
"numa = length(a_vec)\n",
"\n",
"#comptue expected utility under posterior over w given x\n",
"#\n",
"#posterior p(w|x)\n",
"p_wgx = zeros(numw, numx)\n",
"for k in 1:numx\n",
" p_wgx[:,k] = p_xgw_hc[k,:]' .* p_w\n",
" p_wgx[:,k] += eps() #add small epsilon to prevent numerical problems for λ>>\n",
" p_wgx[:,k] /= sum(p_wgx[:,k])\n",
"end\n",
"\n",
"#U(a,x)=∑_w p(w|x)U(a,w)\n",
"U_ax = zeros(numa, numx)\n",
"for k in 1:numx\n",
" U_ax[:,k] = U_pre * p_wgx[:,k]\n",
"end\n",
"\n",
"\n",
"#p(x)=∑_w p(w)p(x|w)\n",
"p_x_hc = p_xgw_hc * p_w\n",
"#make sure that p(x) has non-zero entries (otherwise numerical problems arise in I(X;W))\n",
"p_x_hc += eps()\n",
"p_x_hc /= sum(p_x_hc)\n",
"\n",
"\n",
"#solve the decision-making part with one-step Blahut-Arimoto\n",
"ε = 0.0001 #convergence critetion for BAiterations\n",
"maxiter = 10000 #maximum number of BA iterations\n",
"\n",
"#initialize p(a) uniformly\n",
"num_acts = length(a_vec)\n",
"pa_init = rand(num_acts)\n",
"pa_init /= sum(pa_init)\n",
"\n",
"#BA iterations\n",
"p_agx_hc, p_a_hc, perf_df_hc = BAiterations(pa_init, β2, U_ax, p_x_hc, ε, maxiter,\n",
" compute_performance=true, performance_as_dataframe=true)\n",
"\n",
"; #suppress output"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"#visualize solution p(x|w), p(a|x), p(a|w)\n",
"\n",
"#p(x|w)\n",
"plt_pxgw_hc = visualizeBAconditional(p_xgw_hc, x_vec, w_vec, x_strings, w_strings,\n",
" wlabel=\"animal size w\", alabel=\"observed size x\", legendlabel=\"p(x|w,λ)\")\n",
"\n",
"#p(a|x)\n",
"plt_pa_hc, plt_pagx_hc = visualizeBAsolution(p_a_hc, p_agx_hc, a_vec, x_vec, a_strings, x_strings,\n",
" wlabel=\"observed size x\", alabel=\"action a\",\n",
" legendlabel_marginal=\"p(a)\", legendlabel_conditional=\"p(a|x,λ)\",\n",
" suppress_vis=true)\n",
"\n",
"#compute p(a|w)\n",
"#p(a|w) = ∑_x p(x|w)p(a|x)\n",
"p_agw_hc = p_agx_hc * p_xgw_hc\n",
"plt_pagw_hc = visualizeBAconditional(p_agw_hc, a_vec, w_vec, a_strings, w_strings,\n",
" wlabel=\"animal size w\", alabel=\"action a\", legendlabel=\"p(a|w,λ)\")\n",
"\n",
"\n",
"#visualize p(x|w)\n",
"display(plt_pxgw_hc)\n",
"#visualize p(a|x)\n",
"display(plt_pagx_hc)\n",
"#visualize p(a|w)\n",
"display(plt_pagw_hc)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The plots above show $p_\\lambda(x|w)$, $p_\\lambda(a|x)$ and the overall behavior $p_\\lambda(a|w)$ respectively. \n",
"\n",
"Below we compute the information terms of the hand-crafted solution and visualize them.\n",
"\n",
"**! Note that in the legends below $O$ is used instead of $X$**"
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"#compute p(a|x,w) = p(a|x) ∀w\n",
"p_agxw_hc = zeros(numa, numx, numw)\n",
"for j in 1:numw\n",
" p_agxw_hc[:,:,j] = p_agx_hc\n",
"end\n",
"\n",
"#compute performance measures of solution\n",
"res_hc = analyze_three_var_BAsolution(p_w, p_x_hc, p_a_hc, p_xgw_hc, p_agx_hc, p_agxw_hc, p_agw_hc, U_pre, β1, β2, β3)\n",
"perf_df_hc = performancemeasures2DataFrame(res_hc...)\n",
" \n",
"#visualize\n",
"p_MI, p_composed, p_perf = plot_three_var_performancemeasures(perf_df_hc, maximum(U_pre), β1, β2, β3)\n",
"\n",
"display(hstack(p_MI, p_composed))\n",
"display(p_perf)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Two-step sequential bounded rational decision maker (perception + action)\n",
"\n",
"The code below simulates and visualizes a bounded optimal perception action system."
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"ε = 0.0001 #convergence critetion for BAiterations\n",
"maxiter = 10000 #maximum number of BA iterations\n",
"\n",
"\n",
"\n",
"#This function performs Blahut-Arimoto iterations for the three-variable general case\n",
"px, pa, pxgw, pagx, pagxw, pagw, performance_df = threevarBAiterations(numx, β1, β2, β3, U_pre, p_w, ε, maxiter,\n",
" compute_performance=true, performance_as_dataframe=true,\n",
" init_pogw_sparse = true, init_pogw_uniformly = false,\n",
" init_pagow_uniformly = true);"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"#call the routine that creates plots for all the probability distributions\n",
"plt_px, plt_pa, plt_pxgw, plt_pagx, plt_pagw, dpdown, plt_pagow_vis = visualize_three_var_BAsolution(px, pa,\n",
" pxgw, pagx, pagxw, pagw,\n",
" x_vec, a_vec, w_vec,\n",
" x_strings, a_strings, w_strings, \n",
" olabel_string=\"x\", alabel_string=\"a\", wlabel_string=\"w\")\n",
"\n",
"#visualize p(x|w)\n",
"display(plt_pxgw)\n",
"#visualize p(a|x)\n",
"display(plt_pagx)\n",
"#visualize p(a|w)\n",
"display(plt_pagw)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The plots above show the bounded optimal solutions $p^*(x|w)$, $p^*(a|x)$ and the overall behavior $p^*(a|w)$ respectively. \n",
"\n",
"Below we compute the information terms of the bounded-optimal solution and visualize them.\n",
"\n",
"**! Note that in the legends below $O$ is used instead of $X$**"
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"p_MI, p_composed, p_perf = plot_three_var_performancemeasures(performance_df, maximum(U_pre), β1, β2, β3)\n",
"\n",
"display(hstack(p_MI, p_composed))\n",
"display(p_perf)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## For the paper: compare hand-crafted $p_\\lambda(x|w)$ vs. bounded-optimal $p^*(x|w)$\n",
"\n",
"Rearange the plots and compare the most interesting quantities for the paper."
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"#compare both solutions\n",
"\n",
"#compare performance measures, EU, J, I(A;W)\n",
"\n",
"util_all = [perf_df_hc[end,:E_U], performance_df[end,:E_U],\n",
" perf_df_hc[end,:Objective_value], performance_df[end,:Objective_value]]\n",
"\n",
"color_label = [\"E[U]_hc\", \"E[U]\", \"J_hc\", \"J\"]\n",
"\n",
"colors = BAdiscretecolorscale(4).f(4)\n",
"colscale = Scale.color_discrete_manual(colors[1],colors[1],colors[end],colors[end])\n",
"\n",
"p_perf_comp = plot(x=color_label, y=util_all, color=color_label, Geom.bar(),\n",
" Guide.ylabel(\"[utils]\"), Guide.xlabel(\"\"), Guide.colorkey(\"\"),\n",
" Guide.title(\"λ=$λ vs. β1=$β1, β2=$β2, β3=$β3\"),\n",
" colscale, Scale.y_continuous(minvalue=0, maxvalue=maximum(U_pre)),\n",
" BAtheme(key_position = :none))\n",
"\n",
"\n",
"MI_comp = [perf_df_hc[end,:I_ow], performance_df[end,:I_ow],\n",
" perf_df_hc[end,:I_ao], performance_df[end,:I_ao]]\n",
"color_label = [\"I(X;W)_hc\", \"I(X;W)\", \"I(A;X)_hc\", \"I(A;X)\"]\n",
"\n",
"colors = Scale.color_discrete_hue().f(4)\n",
"colscale = Scale.color_discrete_manual(colors[1],colors[1],colors[2],colors[2])\n",
"\n",
"p_perf_comp_MI = plot(x=color_label, y=MI_comp, color=color_label, Geom.bar(),\n",
" Guide.ylabel(\"[bits]\"), Guide.xlabel(\"\"), Guide.colorkey(\"\"),\n",
" Guide.title(\"λ=$λ vs. β1=$β1, β2=$β2, β3=$β3\"),\n",
" colscale, Scale.y_continuous(minvalue=0), BAtheme(key_position = :none))\n",
"\n",
"display(hstack(p_perf_comp,p_perf_comp_MI))"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"#compare final behavior p(a|w) of both cases\n",
"display(plt_pagw_hc)\n",
"display(plt_pagw)"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
},
{
"data": {
"text/html": [
"\n",
"\n"
]
},
"metadata": {},
"output_type": "display_data"
}
],
"source": [
"#compare perceptual model p(x|w) of both cases\n",
"display(plt_pxgw_hc)\n",
"display(plt_pxgw)"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {
"collapsed": false
},
"outputs": [],
"source": [
"#generate plots for comparison\n",
"\n",
"#display(plt_utility)\n",
"#draw(SVG(\"Figures/SequentialComparison_Utility.svg\", 10cm, 8cm), plt_utility) #uncomment to store figure\n",
"\n",
"\n",
"#plot_final = vstack(hstack(plt_pogw_hc, plt_pagw_hc),hstack(plt_pogw, plt_pagw))\n",
"#plot_bars = (vstack(p_perf_comp,p_perf_comp_MI))\n",
"#display(plot_final)\n",
"#display(plot_bars)\n",
"#draw(SVG(\"Figures/SequentialComparison_3.svg\", 18cm, 12cm), plot_final) #uncomment to store figure#\n",
"#draw(SVG(\"Figures/SequentialComparisonBars_3.svg\", 8cm, 12cm), plot_bars) #uncomment to store figure\n"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Julia 0.4.0",
"language": "julia",
"name": "julia-0.4"
},
"language_info": {
"file_extension": ".jl",
"mimetype": "application/julia",
"name": "julia",
"version": "0.4.0"
}
},
"nbformat": 4,
"nbformat_minor": 0
}