{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# Probability Theory Review" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Preliminaries\n", "\n", "- Goal \n", " - Review of probability theory from a logical reasoning viewpoint (i.e., a Bayesian interpretation)\n", "- Materials \n", " - Mandatory\n", " - These lecture notes\n", " - Optional\n", " - Bishop pp. 12-20 \n", " - [Bruininkx - 2002 - Bayesian Probability](./files/Bruyninkx-2002-Bayesian-probability.pdf)\n", " - [Edwin Jaynes, Probability Theory -- The Logic of Science](https://www.amazon.com/Probability-Theory-Science-T-Jaynes/dp/0521592712). Cambridge University Press, 2003\n", " - brilliant book on Bayesian view on probability theory.\n" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Example Problem: Disease Diagnosis\n", "\n", "- **[Question]** Given a disease with prevalence of 1% and a test procedure with sensitivity ('true positive' rate) of 95% and specificity ('true negative' rate) of 85% , what is the chance that somebody who tests positive actually has the disease?" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- **[Solution]** Use probabilistic inference, to be discussed in this lecture. " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Why Probability Theory?\n", "\n", "- Probability theory (PT) is the **theory of optimal processing of incomplete information** (see [Cox theorem](https://en.wikipedia.org/wiki/Cox%27s_theorem)), and as such provides a quantitative framework for drawing conclusions from a finite (read: incomplete) data set. " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- Machine learning concerns drawing conclusions from (a finite set of) data and therefore PT is the _optimal calculus for machine learning_. " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- In general, nearly all interesting questions in machine learning can be stated in the following form (a conditional probability):\n", "$$p(\\text{whatever-we-want-to-know}\\, | \\,\\text{whatever-we-do-know})$$\n", " - For example:\n", " - Predictions\n", " $$p(\\,\\text{future-observations}\\,|\\,\\text{past-observations}\\,)$$\n", " - Classify a received data point \n", " $$p(\\,x\\text{-belongs-to-class-}k \\,|\\,x\\,)$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ " \n", "- **Information theory** (\"theory of log-probability\") provides a source coding view on machine learning that is consistent with probability theory (more in part-2). " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Probability Theory Notation\n", "\n", "- Define an **event** $A$ as a statement, whose truth is contemplated by a person, e.g.,\n", "\n", "$$A = \\text{'it will rain tomorrow'}$$\n", " " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- We write the denial of $A$, i.e. the event **not**-A, as $\\bar{A}$. " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "##### probabilities\n", "\n", "- For any event $A$, with background knowledge $I$, the **conditional probability of $A$ given $I$**, is written as \n", "$$p(A|I)$$" ] }, { "cell_type": "markdown", "metadata": { "collapsed": true, "slideshow": { "slide_type": "fragment" } }, "source": [ " \n", "- The value of a probability is limited to $0 \\le p(A|I) \\le 1$. " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- All probabilities are in principle conditional probabilities of the type $p(A|I)$, since there is always some background knowledge. " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ " - Still, we often write $p(A)$ rather than $p(A|I)$ if the background knowledge $I$ is assumed to be obviously present. E.g., $p(A)$ rather than $p(\\,A\\,|\\,\\text{the-sun-comes-up-tomorrow}\\,)$.\n", " " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "##### probabilities for random variable assignments\n", "\n", "- Note that, if $X$ is a random variable, then the assignment $X=x$ (with $x$ a value) can be interpreted as an event. " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- We often write $p(x)$ rather than $p(X=x)$ (hoping that the reader understands the context ;-) " ] }, { "cell_type": "markdown", "metadata": { "collapsed": true, "slideshow": { "slide_type": "fragment" } }, "source": [ "- In an apparent effort to further abuse notational conventions, $p(X)$ often denotes the full distribution over random variable $X$, i.e., the distribution for all assignments for $X$. " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "subslide" } }, "source": [ "##### compound events\n", "\n", "- The **joint** probability that both $A$ and $B$ are true, given $I$ (a.k.a. **conjunction**) is written as\n", "$$p(A,B|I)$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- $A$ and $B$ are said to be **independent**, given $I$, if (and only if) $$p(A,B|I) = p(A|I)\\,p(B|I)$$ " ] }, { "cell_type": "markdown", "metadata": { "collapsed": true, "slideshow": { "slide_type": "fragment" } }, "source": [ "- The probability that either $A$ or $B$, or both $A$ and $B$, are true, given $I$ (a.k.a. **disjunction**) is written as \n", "$$p(A+B|I)$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Probability Theory Calculus\n", " \n", "- **Normalization**. If you know that event $A$ given $I$ is true, then $p(A|I)=1$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- **Product rule**. The conjuction of two events $A$ and $B$ with given background $I$ is given by \n", "$$p(A,B|I) = p(A|B,I)\\,p(B|I) \\,.$$\n", " - If $A$ and $B$ are independent given $I$, then $p(A|B,I) = p(A|I)$. " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- **Sum rule**. The disjunction for two events $A$ and $B$ given background $I$ is given by\n", "$$p(A+B|I) = p(A|I) + p(B|I) - p(A,B|I)\\,.$$\n", " - As a special case, it follows from the sum rule that $p(A|I) + p(\\bar{A}|I) = 1$\n", " - Note that the background information may not change, e.g., if $I^\\prime \\neq I$, then \n", "$$p(A+B|I^\\prime) \\neq p(A|I) + p(B|I) - p(A,B|I)\\,.$$ " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- **All legitimate probabilistic relations can be derived from the sum and product rules!**" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- The product and sum rules are also known as the **axioms of probability theory**, but in fact, under some mild conditions, they can be derived as the unique rules for rational reasoning under uncertainty ([Cox theorem, 1946](https://en.wikipedia.org/wiki/Cox%27s_theorem))." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Frequentist vs. Bayesian Interpretation of Probabilities\n", "\n", "- In the **frequentist** interpretation, $p(A)$ relates to the relative frequency that $A$ would occur under repeated execution of an experiment. " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ " - For instance, if the experiment is tossing a coin, then $p(\\texttt{tail}) = 0.4$ means that in the limit of a large number of coin tosses, 40% of outcomes turn up as $\\texttt{tail}$. " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- In the **Bayesian** interpretation, $p(A)$ reflects the **degree of belief** that event $A$ is true. I.o.w., the probability is associated with a **state-of-knowledge** (usually held by a person). \n", " - For instance, for the coin tossing experiment, $p(\\texttt{tail}) = 0.4$ should be interpreted as the belief that there is a 40% chance that $\\texttt{tail}$ comes up if the coin were tossed.\n", " - Under the Bayesian interpretation, PT calculus (sum and product rules) **extends boolean logic to rational reasoning with uncertainty**. " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- The Bayesian viewpoint is more generally applicable than the frequentist viewpoint, e.g., it is hard to apply the frequentist viewpoint to the event '$\\texttt{it will rain tomorrow}$'. " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- The Bayesian viewpoint is clearly favored in the machine learning community. (In this class, we also strongly favor the Bayesian interpretation). " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### The Sum Rule and Marginalization\n", "\n", "- We discussed that every inference problem in PT can be evaluated through the sum and product rules. Next, we present two useful corollaries: (1) Marginalization and (2) Bayes rule " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- If $X$ and $Y$ are random variables over finite domains, than it follows from the sum rule that \n", "$$\n", "p(X) = \\sum_Y p(X,Y) = \\sum_Y p(X|Y) p(Y) \\,.\n", "$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- Note that this is just a **generalized sum rule**. In fact, Bishop (p.14) (and some other authors as well) calls this the sum rule.\n", " - EXERCISE: Proof the generalized sum rule." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- Of course, in the continuous domain, the (generalized) sum rule becomes\n", "$$p(X)=\\int p(X,Y) \\,\\mathrm{d}Y$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- Integrating $Y$ out of a joint distribution is called **marginalization** and the result $p(X)$ is sometimes referred to as the **marginal probability**. " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### The Product Rule and Bayes Rule\n", "\n", "- Consider 2 variables $D$ and $\\theta$; it follows symmetry arguments that \n", "$$p(D,\\theta)=p(D|\\theta)p(\\theta)=p(\\theta|D)p(D)$$ \n", "and hence that\n", "$$p(\\theta|D) = \\frac{p(D|\\theta) p(\\theta)}{p(D)}\\,.$$ " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- This formula is called **Bayes rule** (or Bayes theorem). While Bayes rule is always true, a particularly useful application occurs when $D$ refers to an observed data set and $\\theta$ is set of model parameters that relates to the data. In that case,\n", "\n", " - the **prior** probability $p(\\theta)$ represents our **state-of-knowledge** about proper values for $\\theta$, before seeing the data $D$.\n", " - the **posterior** probability $p(\\theta|D)$ represents our state-of-knowledge about $\\theta$ after we have seen the data." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "$\\Rightarrow$ Bayes rule tells us how to update our knowledge about model parameters when facing new data. Hence, \n", "\n", "
\n", "
\n", "Bayes rule is the fundamental rule for machine learning!\n", "
\n", "
" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### Bayes Rule Nomenclature\n", "- Some nomenclature associated with Bayes rule:\n", "$$\n", "\\underbrace{p(\\theta | D)}_{\\text{posterior}} = \\frac{\\overbrace{p(D|\\theta)}^{\\text{likelihood}} \\times \\overbrace{p(\\theta)}^{\\text{prior}}}{\\underbrace{p(D)}_{\\text{evidence}}}\n", "$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- Note that the evidence (a.k.a. _marginal likelihood_) can be computed from the numerator through marginalization since\n", "$$p(D) = \\int p(D,\\theta) \\,\\mathrm{d}\\theta = \\int p(D|\\theta)\\,p(\\theta) \\,\\mathrm{d}\\theta$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- Hence, having access to likelihood and prior is sufficient to compute both the evidence and the posterior. To emphasize that point, Bayes rule is sometimes written as \n", "$$p(\\theta|D)\\,p(D) = p(D|\\theta)\\, p(\\theta)$$ " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- For given $D$, the posterior probabilities of the parameters scale relatively against each other as\n", "$$\n", "p(\\theta|D) \\propto p(D|\\theta) p(\\theta)\n", "$$\n", "\n", "$\\Longrightarrow$ All that we can learn from the observed data is contained in the likelihood function $p(D|\\theta)$. This is called the **likelihood principle**." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "### The Likelihood Function vs the Sampling Distribution\n", "\n", "- Consider a model $p(D|\\theta)$, where $D$ relates to variables that are observed (i.e., a \"data set\") and $\\theta$ are model parameters." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- In general, $p(D|\\theta)$ is just a function of the two variables $D$ and $\\theta$. We distinguish two interpretations of this function, depending on which variable is observed (or given by other means). " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- The **sampling distribution** (a.k.a. the **data-generating** distribution) $$p(D|\\theta=\\theta_0)$$ (which is a function of $D$ only) describes the probability distribution for data $D$, assuming that it is generated by the given model with parameters fixed at $\\theta = \\theta_0$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- In a machine learning context, often the data is observed, and $\\theta$ is the free variable. For given observations $D=D_0$, the **likelihood function** (which is a function only of the model parameters $\\theta$) is defined as $$\\mathrm{L}(\\theta) \\triangleq p(D=D_0|\\theta)$$" ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- Note that $\\mathrm{L}(\\theta)$ is not a probability distribution for $\\theta$ since in general $\\sum_\\theta \\mathrm{L}(\\theta) \\neq 1$." ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "fragment" } }, "source": [ "- Technically, it is more correct to speak about the likelihood of a model (or model parameters) than about the likelihood of an observed data set. (Why?) " ] }, { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "#### CODE EXAMPLE\n", "\n", "Consider the following simple model for the outcome (head or tail) of a biased coin toss with parameter $\\theta \\in [0,1]$:\n", "\n", "\\begin{align*}\n", "y &\\in \\{0,1\\} \\\\\n", "p(y|\\theta) &\\triangleq \\theta^y (1-\\theta)^{1-y}\\\\\n", "\\end{align*}\n", "\n", "We can plot both the sampling distribution (i.e. $p(y|\\theta=0.8)$) and the likelihood function (i.e. $L(\\theta) = p(y=0|\\theta)$)." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "slideshow": { "slide_type": "subslide" } }, "outputs": [ { "data": { "text/html": [ "
\n", "
\n", "
" ], "text/plain": [ "Widget{:manipulate,Any}(OrderedDict{Symbol,Any}(:y=>Widget{:toggle,Bool}(OrderedDict{Symbol,Any}(:changes=>Observable{Int64} with 1 listeners. Value:\n", "0,:value=>Observable{Bool} with 2 listeners. Value:\n", "false), Observable{Bool} with 2 listeners. Value:\n", "false, Scope(\"knockout-component-70b08ae5-558a-45eb-9f83-50461148baa0\", Node{DOM}(DOM(:html, :div), Any[Node{DOM}(DOM(:html, :input), Any[], Dict{Symbol,Any}(:attributes=>Dict{Any,Any}(:type=>\"checkbox\",Symbol(\"data-bind\")=>\"checked: value, valueUpdate: 'change', event: {change : function () {this.changes(this.changes()+1)}}\"),:id=>\"##363\",:className=>\"switch \",:style=>Dict{Any,Any}()), 0), Node{DOM}(DOM(:html, :label), Any[\"y\"], Dict{Symbol,Any}(:attributes=>Dict(\"className\"=>\"\",\"for\"=>\"##363\")), 1)], Dict{Symbol,Any}(:className=>\"field\"), 3), Dict{String,Tuple{Observables.AbstractObservable,Union{Nothing, Bool}}}(\"changes\"=>(Observable{Int64} with 1 listeners. Value:\n", "0, nothing),\"value\"=>(Observable{Bool} with 2 listeners. Value:\n", "false, nothing)), Set(String[]), nothing, Any[\"knockout\"=>\"/Users/bert/.julia/packages/Knockout/JIqpG/src/../assets/knockout.js\", \"knockout_punches\"=>\"/Users/bert/.julia/packages/Knockout/JIqpG/src/../assets/knockout_punches.js\", \"/Users/bert/.julia/packages/InteractBase/3SqBl/src/../assets/all.js\", \"/Users/bert/.julia/packages/InteractBase/3SqBl/src/../assets/style.css\", \"/Users/bert/.julia/packages/InteractBulma/Ohu5Y/src/../assets/main.css\"], Dict{Any,Any}(\"_promises\"=>Dict{Any,Any}(\"importsLoaded\"=>Any[JSString(\"function (ko, koPunches) {\\n ko.punches.enableAll();\\n ko.bindingHandlers.numericValue = {\\n init : function(element, valueAccessor, allBindings, data, context) {\\n var stringified = ko.observable(ko.unwrap(valueAccessor()));\\n stringified.subscribe(function(value) {\\n var val = parseFloat(value);\\n if (!isNaN(val)) {\\n valueAccessor()(val);\\n }\\n })\\n valueAccessor().subscribe(function(value) {\\n var str = JSON.stringify(value);\\n if ((str == \\\"0\\\") && ([\\\"-0\\\", \\\"-0.\\\"].indexOf(stringified()) >= 0))\\n return;\\n if ([\\\"null\\\", \\\"\\\"].indexOf(str) >= 0)\\n return;\\n stringified(str);\\n })\\n ko.applyBindingsToNode(element, { value: stringified, valueUpdate: allBindings.get('valueUpdate')}, context);\\n }\\n };\\n var json_data = JSON.parse(\\\"{\\\\\\\"changes\\\\\\\":0,\\\\\\\"value\\\\\\\":false}\\\");\\n var self = this;\\n function AppViewModel() {\\n for (var key in json_data) {\\n var el = json_data[key];\\n this[key] = Array.isArray(el) ? ko.observableArray(el) : ko.observable(el);\\n }\\n \\n \\n [this[\\\"changes\\\"].subscribe((function (val){!(this.valueFromJulia[\\\"changes\\\"]) ? (WebIO.setval({\\\"name\\\":\\\"changes\\\",\\\"scope\\\":\\\"knockout-component-70b08ae5-558a-45eb-9f83-50461148baa0\\\",\\\"id\\\":\\\"ob_12\\\",\\\"type\\\":\\\"observable\\\"},val)) : undefined; return this.valueFromJulia[\\\"changes\\\"]=false}),self),this[\\\"value\\\"].subscribe((function (val){!(this.valueFromJulia[\\\"value\\\"]) ? (WebIO.setval({\\\"name\\\":\\\"value\\\",\\\"scope\\\":\\\"knockout-component-70b08ae5-558a-45eb-9f83-50461148baa0\\\",\\\"id\\\":\\\"ob_11\\\",\\\"type\\\":\\\"observable\\\"},val)) : undefined; return this.valueFromJulia[\\\"value\\\"]=false}),self)]\\n \\n }\\n self.model = new AppViewModel();\\n self.valueFromJulia = {};\\n for (var key in json_data) {\\n self.valueFromJulia[key] = false;\\n }\\n ko.applyBindings(self.model, self.dom);\\n}\\n\")]),\"changes\"=>Any[JSString(\"(function (val){return (val!=this.model[\\\"changes\\\"]()) ? (this.valueFromJulia[\\\"changes\\\"]=true, this.model[\\\"changes\\\"](val)) : undefined})\")],\"value\"=>Any[JSString(\"(function (val){return (val!=this.model[\\\"value\\\"]()) ? (this.valueFromJulia[\\\"value\\\"]=true, this.model[\\\"value\\\"](val)) : undefined})\")]), ConnectionPool(Channel{Any}(sz_max:9223372036854775807,sz_curr:3), Set(AbstractConnection[]), Channel{AbstractConnection}(sz_max:32,sz_curr:0))), ##52#53{#dom#15{##dom#13#14{Dict{Any,Any},DOM}},typeof(scope)}(#dom#15{##dom#13#14{Dict{Any,Any},DOM}}(##dom#13#14{Dict{Any,Any},DOM}(Dict{Any,Any}(:className=>\"field\"), DOM(:html, :div))), scope)),:θ=>Widget{:slider,Float64}(OrderedDict{Symbol,Any}(:changes=>Observable{Int64} with 1 listeners. Value:\n", "0,:index=>Observable{Any} with 2 listeners. Value:\n", "6,:formatted_vals=>Observable{Any} with 1 listeners. Value:\n", "[\"0.0\", \"0.1\", \"0.2\", \"0.3\", \"0.4\", \"0.5\", \"0.6\", \"0.7\", \"0.8\", \"0.9\", \"1.0\"],:formatted_value=>Observable{String} with 1 listeners. Value:\n", "\"0.5\",:value=>Observable{Float64} with 2 listeners. Value:\n", "0.5), Observable{Float64} with 2 listeners. Value:\n", "0.5, Scope(\"knockout-component-4c616732-3ed4-4a12-ba19-997bb9deb0b4\", Node{DOM}(DOM(:html, :div), Any[Node{DOM}(DOM(:html, :div), Any[Node{DOM}(DOM(:html, :label), Any[\"θ\"], Dict{Symbol,Any}(:className=>\"interact \",:style=>Dict{Any,Any}(:padding=>\"5px 10px 0px 10px\")), 1)], Dict{Symbol,Any}(:attributes=>Dict(\"class\"=>\"interact-flex-row-left\")), 2), Node{DOM}(DOM(:html, :div), Any[Node{DOM}(DOM(:html, :input), Any[], Dict{Symbol,Any}(:max=>11,:min=>1,:attributes=>Dict{Any,Any}(:type=>\"range\",Symbol(\"data-bind\")=>\"numericValue: index, valueUpdate: 'input', event: {change : function () {this.changes(this.changes()+1)}}\",\"orient\"=>\"horizontal\"),:step=>1,:className=>\"slider slider is-fullwidth\",:style=>Dict{Any,Any}()), 0)], Dict{Symbol,Any}(:attributes=>Dict(\"class\"=>\"interact-flex-row-center\")), 1), Node{DOM}(DOM(:html, :div), Any[Node{DOM}(DOM(:html, :p), Any[], Dict{Symbol,Any}(:attributes=>Dict(\"data-bind\"=>\"text: formatted_value\")), 0)], Dict{Symbol,Any}(:attributes=>Dict(\"class\"=>\"interact-flex-row-right\")), 1)], Dict{Symbol,Any}(:attributes=>Dict(\"class\"=>\"interact-flex-row\")), 7), Dict{String,Tuple{Observables.AbstractObservable,Union{Nothing, Bool}}}(\"formatted_vals\"=>(Observable{Any} with 1 listeners. Value:\n", "[\"0.0\", \"0.1\", \"0.2\", \"0.3\", \"0.4\", \"0.5\", \"0.6\", \"0.7\", \"0.8\", \"0.9\", \"1.0\"], nothing),\"changes\"=>(Observable{Int64} with 1 listeners. Value:\n", "0, nothing),\"formatted_value\"=>(Observable{String} with 1 listeners. Value:\n", "\"0.5\", nothing),\"index\"=>(Observable{Any} with 2 listeners. Value:\n", "6, nothing)), Set(String[]), nothing, Any[\"knockout\"=>\"/Users/bert/.julia/packages/Knockout/JIqpG/src/../assets/knockout.js\", \"knockout_punches\"=>\"/Users/bert/.julia/packages/Knockout/JIqpG/src/../assets/knockout_punches.js\", \"/Users/bert/.julia/packages/InteractBase/3SqBl/src/../assets/all.js\", \"/Users/bert/.julia/packages/InteractBase/3SqBl/src/../assets/style.css\", \"/Users/bert/.julia/packages/InteractBulma/Ohu5Y/src/../assets/main.css\"], Dict{Any,Any}(\"formatted_vals\"=>Any[JSString(\"(function (val){return (val!=this.model[\\\"formatted_vals\\\"]()) ? (this.valueFromJulia[\\\"formatted_vals\\\"]=true, this.model[\\\"formatted_vals\\\"](val)) : undefined})\")],\"_promises\"=>Dict{Any,Any}(\"importsLoaded\"=>Any[JSString(\"function (ko, koPunches) {\\n ko.punches.enableAll();\\n ko.bindingHandlers.numericValue = {\\n init : function(element, valueAccessor, allBindings, data, context) {\\n var stringified = ko.observable(ko.unwrap(valueAccessor()));\\n stringified.subscribe(function(value) {\\n var val = parseFloat(value);\\n if (!isNaN(val)) {\\n valueAccessor()(val);\\n }\\n })\\n valueAccessor().subscribe(function(value) {\\n var str = JSON.stringify(value);\\n if ((str == \\\"0\\\") && ([\\\"-0\\\", \\\"-0.\\\"].indexOf(stringified()) >= 0))\\n return;\\n if ([\\\"null\\\", \\\"\\\"].indexOf(str) >= 0)\\n return;\\n stringified(str);\\n })\\n ko.applyBindingsToNode(element, { value: stringified, valueUpdate: allBindings.get('valueUpdate')}, context);\\n }\\n };\\n var json_data = JSON.parse(\\\"{\\\\\\\"formatted_vals\\\\\\\":[\\\\\\\"0.0\\\\\\\",\\\\\\\"0.1\\\\\\\",\\\\\\\"0.2\\\\\\\",\\\\\\\"0.3\\\\\\\",\\\\\\\"0.4\\\\\\\",\\\\\\\"0.5\\\\\\\",\\\\\\\"0.6\\\\\\\",\\\\\\\"0.7\\\\\\\",\\\\\\\"0.8\\\\\\\",\\\\\\\"0.9\\\\\\\",\\\\\\\"1.0\\\\\\\"],\\\\\\\"changes\\\\\\\":0,\\\\\\\"formatted_value\\\\\\\":\\\\\\\"0.5\\\\\\\",\\\\\\\"index\\\\\\\":6}\\\");\\n var self = this;\\n function AppViewModel() {\\n for (var key in json_data) {\\n var el = json_data[key];\\n this[key] = Array.isArray(el) ? ko.observableArray(el) : ko.observable(el);\\n }\\n \\n \\n [this[\\\"formatted_vals\\\"].subscribe((function (val){!(this.valueFromJulia[\\\"formatted_vals\\\"]) ? (WebIO.setval({\\\"name\\\":\\\"formatted_vals\\\",\\\"scope\\\":\\\"knockout-component-4c616732-3ed4-4a12-ba19-997bb9deb0b4\\\",\\\"id\\\":\\\"ob_15\\\",\\\"type\\\":\\\"observable\\\"},val)) : undefined; return this.valueFromJulia[\\\"formatted_vals\\\"]=false}),self),this[\\\"changes\\\"].subscribe((function (val){!(this.valueFromJulia[\\\"changes\\\"]) ? (WebIO.setval({\\\"name\\\":\\\"changes\\\",\\\"scope\\\":\\\"knockout-component-4c616732-3ed4-4a12-ba19-997bb9deb0b4\\\",\\\"id\\\":\\\"ob_17\\\",\\\"type\\\":\\\"observable\\\"},val)) : undefined; return this.valueFromJulia[\\\"changes\\\"]=false}),self),this[\\\"formatted_value\\\"].subscribe((function (val){!(this.valueFromJulia[\\\"formatted_value\\\"]) ? (WebIO.setval({\\\"name\\\":\\\"formatted_value\\\",\\\"scope\\\":\\\"knockout-component-4c616732-3ed4-4a12-ba19-997bb9deb0b4\\\",\\\"id\\\":\\\"ob_16\\\",\\\"type\\\":\\\"observable\\\"},val)) : undefined; return this.valueFromJulia[\\\"formatted_value\\\"]=false}),self),this[\\\"index\\\"].subscribe((function (val){!(this.valueFromJulia[\\\"index\\\"]) ? (WebIO.setval({\\\"name\\\":\\\"index\\\",\\\"scope\\\":\\\"knockout-component-4c616732-3ed4-4a12-ba19-997bb9deb0b4\\\",\\\"id\\\":\\\"ob_14\\\",\\\"type\\\":\\\"observable\\\"},val)) : undefined; return this.valueFromJulia[\\\"index\\\"]=false}),self)]\\n \\n }\\n self.model = new AppViewModel();\\n self.valueFromJulia = {};\\n for (var key in json_data) {\\n self.valueFromJulia[key] = false;\\n }\\n ko.applyBindings(self.model, self.dom);\\n}\\n\")]),\"changes\"=>Any[JSString(\"(function (val){return (val!=this.model[\\\"changes\\\"]()) ? (this.valueFromJulia[\\\"changes\\\"]=true, this.model[\\\"changes\\\"](val)) : undefined})\")],\"formatted_value\"=>Any[JSString(\"(function (val){return (val!=this.model[\\\"formatted_value\\\"]()) ? (this.valueFromJulia[\\\"formatted_value\\\"]=true, this.model[\\\"formatted_value\\\"](val)) : undefined})\")],\"index\"=>Any[JSString(\"(function (val){return (val!=this.model[\\\"index\\\"]()) ? (this.valueFromJulia[\\\"index\\\"]=true, this.model[\\\"index\\\"](val)) : undefined})\"), JSString(\"(function (val){return WebIO.setval({\\\"name\\\":\\\"formatted_value\\\",\\\"scope\\\":\\\"knockout-component-4c616732-3ed4-4a12-ba19-997bb9deb0b4\\\",\\\"id\\\":\\\"ob_16\\\",\\\"type\\\":\\\"observable\\\"},WebIO.getval({\\\"name\\\":\\\"formatted_vals\\\",\\\"scope\\\":\\\"knockout-component-4c616732-3ed4-4a12-ba19-997bb9deb0b4\\\",\\\"id\\\":\\\"ob_15\\\",\\\"type\\\":\\\"observable\\\"})[(WebIO.getval({\\\"name\\\":\\\"index\\\",\\\"scope\\\":\\\"knockout-component-4c616732-3ed4-4a12-ba19-997bb9deb0b4\\\",\\\"id\\\":\\\"ob_14\\\",\\\"type\\\":\\\"observable\\\"})-1)])})\")]), ConnectionPool(Channel{Any}(sz_max:9223372036854775807,sz_curr:4), Set(AbstractConnection[]), Channel{AbstractConnection}(sz_max:32,sz_curr:0))), ##52#53{#dom#15{##dom#13#14{Dict{Any,Any},DOM}},typeof(scope)}(#dom#15{##dom#13#14{Dict{Any,Any},DOM}}(##dom#13#14{Dict{Any,Any},DOM}(Dict{Any,Any}(:className=>\"field\"), DOM(:html, :div))), scope))), Observable{Any} with 0 listeners. Value:\n", "Figure(PyObject