{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# ExprOptimization.jl\n",
"\n",
"ExprOptimization.jl is a Julia package for optimizing Julia expressions. The package implements algorithms to optimize expression trees derived from a grammar to optimize a user-defined objective function. The package depends on ExprRules.jl.\n",
"\n",
"## Installation\n",
"\n",
"To install the package:\n",
"\n",
" Pkg.add(\"ExprOptimization\")\n",
"\n",
"## Usage\n",
"\n",
"To start using the package:"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"using ExprOptimization"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Example -- Symbolic Regression\n",
"\n",
"We consider the example of finding an algebraic expression that approximates a given function.\n",
"\n",
"First, we define a grammar:"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"1: Real = x\n",
"2: Real = Real * Real\n",
"3: Real = Real + Real\n",
"4: Real = Real - Real\n",
"5: Real = 1\n",
"6: Real = 2\n",
"7: Real = 3\n",
"8: Real = 4\n",
"9: Real = 5\n"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"const grammar = @grammar begin\n",
" Real = x\n",
" Real = Real * Real\n",
" Real = Real + Real\n",
" Real = Real - Real\n",
" Real = |(1:5)\n",
"end"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Create a custom symbol table and automatically populate it from the grammar. This is not strictly necessary since ExprOptimization.jl can use the global symbol table, but it leads to much better performance."
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"Dict{Symbol,Any} with 3 entries:\n",
" :+ => +\n",
" :- => -\n",
" :* => *"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"const S = SymbolTable(grammar)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, we define the ground truth expression and loss function by overloading the `loss` function in ExprOptimization. The loss function returns the real-valued loss of a given expression tree. The loss is minimized."
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"loss (generic function with 1 method)"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"ground_truth(x) = x*x + 2x + 1\n",
"function loss(tree::RuleNode, grammar::Grammar)\n",
" ex = get_executable(tree, grammar)\n",
" los = 0.0\n",
" for x = -5.0:1.0:5.0\n",
" S[:x] = x\n",
" los += abs2(Core.eval(S,ex) - ground_truth(x))\n",
" end\n",
" los\n",
"end"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Once these are defined, we can use any of the implemented algorithms to perform the optimization.\n",
"\n",
"### Monte Carlo\n",
"\n",
"Monte Carlo (MC) draws a number of random expression trees from the grammar and returns the one with the best loss."
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"search: \u001b[0m\u001b[1mM\u001b[22m\u001b[0m\u001b[1mo\u001b[22m\u001b[0m\u001b[1mn\u001b[22m\u001b[0m\u001b[1mt\u001b[22m\u001b[0m\u001b[1me\u001b[22m\u001b[0m\u001b[1mC\u001b[22m\u001b[0m\u001b[1ma\u001b[22m\u001b[0m\u001b[1mr\u001b[22m\u001b[0m\u001b[1ml\u001b[22m\u001b[0m\u001b[1mo\u001b[22m \u001b[0m\u001b[1mM\u001b[22m\u001b[0m\u001b[1mo\u001b[22m\u001b[0m\u001b[1mn\u001b[22m\u001b[0m\u001b[1mt\u001b[22m\u001b[0m\u001b[1me\u001b[22m\u001b[0m\u001b[1mC\u001b[22m\u001b[0m\u001b[1ma\u001b[22m\u001b[0m\u001b[1mr\u001b[22m\u001b[0m\u001b[1ml\u001b[22m\u001b[0m\u001b[1mo\u001b[22ms\n",
"\n"
]
},
{
"data": {
"text/markdown": [
"```\n",
"MonteCarlo\n",
"```\n",
"\n",
"Monte Carlo.\n",
"\n",
"# Arguments:\n",
"\n",
" * `num_samples::Int`: number of samples\n",
" * `max_depth::Int`: maximum depth of derivation tree\n"
],
"text/plain": [
"\u001b[36m MonteCarlo\u001b[39m\n",
"\n",
" Monte Carlo.\n",
"\n",
"\u001b[1m Arguments:\u001b[22m\n",
"\u001b[1m ≡≡≡≡≡≡≡≡≡≡≡≡\u001b[22m\n",
"\n",
" • \u001b[36mnum_samples::Int\u001b[39m: number of samples\n",
"\n",
" • \u001b[36mmax_depth::Int\u001b[39m: maximum depth of derivation tree"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"?MonteCarlo"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(:((x + 3) * x), 121.0)"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"using Random\n",
"Random.seed!(10)\n",
"p = MonteCarlo(20000, 6)\n",
"results_mc = optimize(p, grammar, :Real, loss)\n",
"(results_mc.expr, results_mc.loss)"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n"
],
"text/plain": [
"TreeView.LabelledTree({5, 4} directed simple Int64 graph, Any[:*, :+, :x, 3, :x])"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"display(results_mc.tree, grammar)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Genetic Programming\n",
"\n",
"Genetic Programming (GP) is an evolutionary algorithm for trees.\n",
"\n",
"See: Koza, \"Genetic Programming: On the Programming of Computers by Means of Natural Selection\", MIT Press, 1992."
]
},
{
"cell_type": "code",
"execution_count": 8,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"search: \u001b[0m\u001b[1mG\u001b[22m\u001b[0m\u001b[1me\u001b[22m\u001b[0m\u001b[1mn\u001b[22m\u001b[0m\u001b[1me\u001b[22m\u001b[0m\u001b[1mt\u001b[22m\u001b[0m\u001b[1mi\u001b[22m\u001b[0m\u001b[1mc\u001b[22m\u001b[0m\u001b[1mP\u001b[22m\u001b[0m\u001b[1mr\u001b[22m\u001b[0m\u001b[1mo\u001b[22m\u001b[0m\u001b[1mg\u001b[22m\u001b[0m\u001b[1mr\u001b[22m\u001b[0m\u001b[1ma\u001b[22m\u001b[0m\u001b[1mm\u001b[22m \u001b[0m\u001b[1mG\u001b[22m\u001b[0m\u001b[1me\u001b[22m\u001b[0m\u001b[1mn\u001b[22m\u001b[0m\u001b[1me\u001b[22m\u001b[0m\u001b[1mt\u001b[22m\u001b[0m\u001b[1mi\u001b[22m\u001b[0m\u001b[1mc\u001b[22m\u001b[0m\u001b[1mP\u001b[22m\u001b[0m\u001b[1mr\u001b[22m\u001b[0m\u001b[1mo\u001b[22m\u001b[0m\u001b[1mg\u001b[22m\u001b[0m\u001b[1mr\u001b[22m\u001b[0m\u001b[1ma\u001b[22m\u001b[0m\u001b[1mm\u001b[22ms\n",
"\n"
]
},
{
"data": {
"text/markdown": [
"```\n",
"GeneticProgram\n",
"```\n",
"\n",
"Genetic Programming.\n",
"\n",
"# Arguments\n",
"\n",
" * `pop_size::Int`: population size\n",
" * `iterations::Int`: number of iterations\n",
" * `max_depth::Int`: maximum depth of derivation tree\n",
" * `p_reproduction::Float64`: probability of reproduction operator\n",
" * `p_crossover::Float64`: probability of crossover operator\n",
" * `p_mutation::Float64`: probability of mutation operator\n",
" * `init_method::InitializationMethod`: initialization method\n",
" * `select_method::SelectionMethod`: selection method\n"
],
"text/plain": [
"\u001b[36m GeneticProgram\u001b[39m\n",
"\n",
" Genetic Programming.\n",
"\n",
"\u001b[1m Arguments\u001b[22m\n",
"\u001b[1m ≡≡≡≡≡≡≡≡≡≡≡\u001b[22m\n",
"\n",
" • \u001b[36mpop_size::Int\u001b[39m: population size\n",
"\n",
" • \u001b[36miterations::Int\u001b[39m: number of iterations\n",
"\n",
" • \u001b[36mmax_depth::Int\u001b[39m: maximum depth of derivation tree\n",
"\n",
" • \u001b[36mp_reproduction::Float64\u001b[39m: probability of reproduction operator\n",
"\n",
" • \u001b[36mp_crossover::Float64\u001b[39m: probability of crossover operator\n",
"\n",
" • \u001b[36mp_mutation::Float64\u001b[39m: probability of mutation operator\n",
"\n",
" • \u001b[36minit_method::InitializationMethod\u001b[39m: initialization method\n",
"\n",
" • \u001b[36mselect_method::SelectionMethod\u001b[39m: selection method"
]
},
"execution_count": 8,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"?GeneticProgram"
]
},
{
"cell_type": "code",
"execution_count": 9,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(:((x * 2 + (x * x - 3)) + 4), 0.0)"
]
},
"execution_count": 9,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Random.seed!(1)\n",
"p = GeneticProgram(1000,20,6,0.3,0.3,0.4)\n",
"results_gp = optimize(p, grammar, :Real, loss)\n",
"(results_gp.expr, results_gp.loss)"
]
},
{
"cell_type": "code",
"execution_count": 10,
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n"
],
"text/plain": [
"TreeView.LabelledTree({11, 10} directed simple Int64 graph, Any[:+, :+, :*, :x, 2, :-, :*, :x, :x, 3, 4])"
]
},
"execution_count": 10,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"display(results_gp.tree, grammar)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Grammatical Evolution\n",
"\n",
"Grammatical Evolution (GE) is an evolutionary algorithm based on sequentializing the decisions in the derivation tree (e.g., using depth-first traversal order). Optimization is performed over integer arrays using genetic algorithms.\n",
"\n",
"See: C. Ryan, J.J. Collins, M. O'Neil, \"Grammatical Evolution: Evolving Programs for an Arbitrary Language\", in European Conference on Genetic Programming, Springer, 1998, pp. 83-96."
]
},
{
"cell_type": "code",
"execution_count": 11,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"search: \u001b[0m\u001b[1mG\u001b[22m\u001b[0m\u001b[1mr\u001b[22m\u001b[0m\u001b[1ma\u001b[22m\u001b[0m\u001b[1mm\u001b[22m\u001b[0m\u001b[1mm\u001b[22m\u001b[0m\u001b[1ma\u001b[22m\u001b[0m\u001b[1mt\u001b[22m\u001b[0m\u001b[1mi\u001b[22m\u001b[0m\u001b[1mc\u001b[22m\u001b[0m\u001b[1ma\u001b[22m\u001b[0m\u001b[1ml\u001b[22m\u001b[0m\u001b[1mE\u001b[22m\u001b[0m\u001b[1mv\u001b[22m\u001b[0m\u001b[1mo\u001b[22m\u001b[0m\u001b[1ml\u001b[22m\u001b[0m\u001b[1mu\u001b[22m\u001b[0m\u001b[1mt\u001b[22m\u001b[0m\u001b[1mi\u001b[22m\u001b[0m\u001b[1mo\u001b[22m\u001b[0m\u001b[1mn\u001b[22m \u001b[0m\u001b[1mG\u001b[22m\u001b[0m\u001b[1mr\u001b[22m\u001b[0m\u001b[1ma\u001b[22m\u001b[0m\u001b[1mm\u001b[22m\u001b[0m\u001b[1mm\u001b[22m\u001b[0m\u001b[1ma\u001b[22m\u001b[0m\u001b[1mt\u001b[22m\u001b[0m\u001b[1mi\u001b[22m\u001b[0m\u001b[1mc\u001b[22m\u001b[0m\u001b[1ma\u001b[22m\u001b[0m\u001b[1ml\u001b[22m\u001b[0m\u001b[1mE\u001b[22m\u001b[0m\u001b[1mv\u001b[22m\u001b[0m\u001b[1mo\u001b[22m\u001b[0m\u001b[1ml\u001b[22m\u001b[0m\u001b[1mu\u001b[22m\u001b[0m\u001b[1mt\u001b[22m\u001b[0m\u001b[1mi\u001b[22m\u001b[0m\u001b[1mo\u001b[22m\u001b[0m\u001b[1mn\u001b[22ms\n",
"\n"
]
},
{
"data": {
"text/markdown": [
"```\n",
"GrammaticalEvolution\n",
"```\n",
"\n",
"Grammatical Evolution.\n",
"\n",
"# Arguments\n",
"\n",
" * `grammar::Grammar`: grammar\n",
" * `typ::Symbol`: start symbol\n",
" * `pop_size::Int`: population size\n",
" * `iterations::Int`: number of iterations\n",
" * `init_gene_length::Int`: initial length of genotype integer array\n",
" * `max_gene_length::Int`: maximum length of genotype integer array\n",
" * `max_depth::Int`: maximum depth of derivation tree\n",
" * `p_reproduction::Float64`: probability of reproduction operator\n",
" * `p_crossover::Float64`: probability of crossover operator\n",
" * `p_mutation::Float64`: probability of mutation operator\n",
" * `select_method::SelectionMethod`: selection method (default: tournament selection)\n",
" * `mutate_method::InitializationMethod`: mutation method (default: multi-mutate)\n"
],
"text/plain": [
"\u001b[36m GrammaticalEvolution\u001b[39m\n",
"\n",
" Grammatical Evolution.\n",
"\n",
"\u001b[1m Arguments\u001b[22m\n",
"\u001b[1m ≡≡≡≡≡≡≡≡≡≡≡\u001b[22m\n",
"\n",
" • \u001b[36mgrammar::Grammar\u001b[39m: grammar\n",
"\n",
" • \u001b[36mtyp::Symbol\u001b[39m: start symbol\n",
"\n",
" • \u001b[36mpop_size::Int\u001b[39m: population size\n",
"\n",
" • \u001b[36miterations::Int\u001b[39m: number of iterations\n",
"\n",
" • \u001b[36minit_gene_length::Int\u001b[39m: initial length of genotype integer array\n",
"\n",
" • \u001b[36mmax_gene_length::Int\u001b[39m: maximum length of genotype integer array\n",
"\n",
" • \u001b[36mmax_depth::Int\u001b[39m: maximum depth of derivation tree\n",
"\n",
" • \u001b[36mp_reproduction::Float64\u001b[39m: probability of reproduction operator\n",
"\n",
" • \u001b[36mp_crossover::Float64\u001b[39m: probability of crossover operator\n",
"\n",
" • \u001b[36mp_mutation::Float64\u001b[39m: probability of mutation operator\n",
"\n",
" • \u001b[36mselect_method::SelectionMethod\u001b[39m: selection method (default:\n",
" tournament selection)\n",
"\n",
" • \u001b[36mmutate_method::InitializationMethod\u001b[39m: mutation method (default:\n",
" multi-mutate)"
]
},
"execution_count": 11,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"?GrammaticalEvolution"
]
},
{
"cell_type": "code",
"execution_count": 12,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(:(x * ((5 + x) - 3) + 1), 0.0)"
]
},
"execution_count": 12,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Random.seed!(1)\n",
"p = GrammaticalEvolution(grammar,:Real,1000,20,10,10,6,0.2,0.4,0.4; select_method=GrammaticalEvolutions.TruncationSelection(300))\n",
"results_ge = optimize(p, grammar, :Real, loss)\n",
"(results_ge.expr, results_ge.loss)"
]
},
{
"cell_type": "code",
"execution_count": 13,
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n"
],
"text/plain": [
"TreeView.LabelledTree({9, 8} directed simple Int64 graph, Any[:+, :*, :x, :-, :+, 5, :x, 3, 1])"
]
},
"execution_count": 13,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"display(results_ge.tree, grammar)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Cross-Entropy Method\n",
"\n",
"The Cross-Entropy (CE) Method is a population-based optimization algorithm based on repeatedly estimating the probability distribution of good solutions. This implementation uses a probabilistic grammar to represent the distributions.\n",
"\n",
"See: Rubinstein, \"Optimization of Computer Simulation Models with Rare Events\", European Journal of Operations Research, 99, 89-112, 1197"
]
},
{
"cell_type": "code",
"execution_count": 14,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"search: \u001b[0m\u001b[1mC\u001b[22m\u001b[0m\u001b[1mr\u001b[22m\u001b[0m\u001b[1mo\u001b[22m\u001b[0m\u001b[1ms\u001b[22m\u001b[0m\u001b[1ms\u001b[22m\u001b[0m\u001b[1mE\u001b[22m\u001b[0m\u001b[1mn\u001b[22m\u001b[0m\u001b[1mt\u001b[22m\u001b[0m\u001b[1mr\u001b[22m\u001b[0m\u001b[1mo\u001b[22m\u001b[0m\u001b[1mp\u001b[22m\u001b[0m\u001b[1my\u001b[22m \u001b[0m\u001b[1mC\u001b[22m\u001b[0m\u001b[1mr\u001b[22m\u001b[0m\u001b[1mo\u001b[22m\u001b[0m\u001b[1ms\u001b[22m\u001b[0m\u001b[1ms\u001b[22m\u001b[0m\u001b[1mE\u001b[22m\u001b[0m\u001b[1mn\u001b[22m\u001b[0m\u001b[1mt\u001b[22m\u001b[0m\u001b[1mr\u001b[22m\u001b[0m\u001b[1mo\u001b[22m\u001b[0m\u001b[1mp\u001b[22m\u001b[0m\u001b[1my\u001b[22ms\n",
"\n"
]
},
{
"data": {
"text/markdown": [
"```\n",
"CrossEntropy\n",
"```\n",
"\n",
"Cross Entropy method.\n",
"\n",
"# Arguments\n",
"\n",
" * `pop_size::Int`: population size\n",
" * `iterations::Int`: number of iterations\n",
" * `max_depth::Int`: maximum depth of derivation tree\n",
" * `top_k::Int`: top k elite samples used in selection\n",
" * `p_init::Float64`: initial value when fitting MLE\n",
" * `init_method::InitializationMethod`: Initialization method\n"
],
"text/plain": [
"\u001b[36m CrossEntropy\u001b[39m\n",
"\n",
" Cross Entropy method.\n",
"\n",
"\u001b[1m Arguments\u001b[22m\n",
"\u001b[1m ≡≡≡≡≡≡≡≡≡≡≡\u001b[22m\n",
"\n",
" • \u001b[36mpop_size::Int\u001b[39m: population size\n",
"\n",
" • \u001b[36miterations::Int\u001b[39m: number of iterations\n",
"\n",
" • \u001b[36mmax_depth::Int\u001b[39m: maximum depth of derivation tree\n",
"\n",
" • \u001b[36mtop_k::Int\u001b[39m: top k elite samples used in selection\n",
"\n",
" • \u001b[36mp_init::Float64\u001b[39m: initial value when fitting MLE \n",
"\n",
" • \u001b[36minit_method::InitializationMethod\u001b[39m: Initialization method"
]
},
"execution_count": 14,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"?CrossEntropy"
]
},
{
"cell_type": "code",
"execution_count": 15,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(:(x * (2 + x) + 1), 0.0)"
]
},
"execution_count": 15,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Random.seed!(1)\n",
"p = CrossEntropy(1000,20,6,500)\n",
"results_ce = optimize(p, grammar, :Real, loss)\n",
"(results_ce.expr, results_ce.loss)"
]
},
{
"cell_type": "code",
"execution_count": 16,
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n"
],
"text/plain": [
"TreeView.LabelledTree({7, 6} directed simple Int64 graph, Any[:+, :*, :x, :+, 2, :x, 1])"
]
},
"execution_count": 16,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"display(results_ce.tree, grammar)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## PIPE\n",
"\n",
"Probabilistic Incremental Program Evolution (PIPE) is an expression tree optimization algorithm based on the probabilistic prototype tree (PPT) model.\n",
"\n",
"See: Salustowicz and Schmidhuber, \"Probabilistic Incremental Program Evolution\", Evolutionary Computation, vol. 5, no. 2, pp. 123-141, 1997."
]
},
{
"cell_type": "code",
"execution_count": 17,
"metadata": {},
"outputs": [
{
"name": "stdout",
"output_type": "stream",
"text": [
"search: \u001b[0m\u001b[1mP\u001b[22m\u001b[0m\u001b[1mI\u001b[22m\u001b[0m\u001b[1mP\u001b[22m\u001b[0m\u001b[1mE\u001b[22m \u001b[0m\u001b[1mP\u001b[22m\u001b[0m\u001b[1mI\u001b[22m\u001b[0m\u001b[1mP\u001b[22m\u001b[0m\u001b[1mE\u001b[22ms \u001b[0m\u001b[1mP\u001b[22m\u001b[0m\u001b[1mi\u001b[22m\u001b[0m\u001b[1mp\u001b[22m\u001b[0m\u001b[1me\u001b[22m \u001b[0m\u001b[1mp\u001b[22m\u001b[0m\u001b[1mi\u001b[22m\u001b[0m\u001b[1mp\u001b[22m\u001b[0m\u001b[1me\u001b[22mline \u001b[0m\u001b[1mP\u001b[22m\u001b[0m\u001b[1mi\u001b[22m\u001b[0m\u001b[1mp\u001b[22m\u001b[0m\u001b[1me\u001b[22mBuffer \u001b[0m\u001b[1mp\u001b[22mart\u001b[0m\u001b[1mi\u001b[22malsort\u001b[0m\u001b[1mp\u001b[22m\u001b[0m\u001b[1me\u001b[22mrm \u001b[0m\u001b[1mp\u001b[22mart\u001b[0m\u001b[1mi\u001b[22malsort\u001b[0m\u001b[1mp\u001b[22m\u001b[0m\u001b[1me\u001b[22mrm!\n",
"\n"
]
},
{
"data": {
"text/markdown": [
"```\n",
"PIPE\n",
"```\n",
"\n",
"Probabilistic Incremental Program Evolution. Example parameters from paper are indicated in parentheses)\n",
"\n",
"# Arguments:\n",
"\n",
" * `ppt_params::PPT`: parameters for PPT (e.g., [0.8, 0.2])\n",
" * `pop_size::Int`: population size\n",
" * `iterations::Int`: number of iterations\n",
" * `p_elitist::Float64`: elitist update probability (e.g., 0.2)\n",
" * `c::Float64`: learning rate multiplier (e.g., 0.1)\n",
" * `α::Float64`: learning rate (e.g., 0.05)\n",
" * `ϵ::Float64`: fitness constant (e.g., 1)\n",
" * `p_mutation::Float64`: mutation probability (e.g., 0.2)\n",
" * `β::Float64`: mutation rate (e.g., 0.6)\n",
" * `p_threshold::Float64`: prune threshold (e.g., 0.999)\n",
" * `max_depth::Int`: maximum depth of derivation tree\n"
],
"text/plain": [
"\u001b[36m PIPE\u001b[39m\n",
"\n",
" Probabilistic Incremental Program Evolution. Example parameters from paper\n",
" are indicated in parentheses)\n",
"\n",
"\u001b[1m Arguments:\u001b[22m\n",
"\u001b[1m ≡≡≡≡≡≡≡≡≡≡≡≡\u001b[22m\n",
"\n",
" • \u001b[36mppt_params::PPT\u001b[39m: parameters for PPT (e.g., [0.8, 0.2])\n",
"\n",
" • \u001b[36mpop_size::Int\u001b[39m: population size \n",
"\n",
" • \u001b[36miterations::Int\u001b[39m: number of iterations\n",
"\n",
" • \u001b[36mp_elitist::Float64\u001b[39m: elitist update probability (e.g., 0.2)\n",
"\n",
" • \u001b[36mc::Float64\u001b[39m: learning rate multiplier (e.g., 0.1)\n",
"\n",
" • \u001b[36mα::Float64\u001b[39m: learning rate (e.g., 0.05) \n",
"\n",
" • \u001b[36mϵ::Float64\u001b[39m: fitness constant (e.g., 1)\n",
"\n",
" • \u001b[36mp_mutation::Float64\u001b[39m: mutation probability (e.g., 0.2)\n",
"\n",
" • \u001b[36mβ::Float64\u001b[39m: mutation rate (e.g., 0.6)\n",
"\n",
" • \u001b[36mp_threshold::Float64\u001b[39m: prune threshold (e.g., 0.999)\n",
"\n",
" • \u001b[36mmax_depth::Int\u001b[39m: maximum depth of derivation tree"
]
},
"execution_count": 17,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"?PIPE"
]
},
{
"cell_type": "code",
"execution_count": 18,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"(:(x * (1x - ((5 - 1) - 5)) + (1 + x)), 0.0)"
]
},
"execution_count": 18,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"Random.seed!(3)\n",
"p = PIPE(PPT(0.8),1000,20,0.2,0.1,0.05,1,0.2,0.6,0.999,6)\n",
"results_pipe = optimize(p, grammar, :Real, loss)\n",
"(results_pipe.expr, results_pipe.loss)"
]
},
{
"cell_type": "code",
"execution_count": 19,
"metadata": {},
"outputs": [
{
"data": {
"image/svg+xml": [
"\n",
"\n",
"\n"
],
"text/plain": [
"TreeView.LabelledTree({15, 14} directed simple Int64 graph, Any[:+, :*, :x, :-, :*, 1, :x, :-, :-, 5, 1, 5, :+, 1, :x])"
]
},
"execution_count": 19,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"display(results_pipe.tree, grammar)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Julia 0.7.0",
"language": "julia",
"name": "julia-0.7"
},
"language_info": {
"file_extension": ".jl",
"mimetype": "application/julia",
"name": "julia",
"version": "0.7.0"
}
},
"nbformat": 4,
"nbformat_minor": 2
}