{ "cells": [ { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "# Log output\n", "\n", "We use the same example as in the quickstart." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [], "source": [ "using MIPVerify\n", "using Gurobi" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/plain": [ "9" ] }, "execution_count": 2, "metadata": {}, "output_type": "execute_result" } ], "source": [ "mnist = MIPVerify.read_datasets(\"MNIST\")\n", "n1params = MIPVerify.get_example_network_params(\"MNIST.n1\")\n", "sample_image = MIPVerify.get_image(mnist.train.images, 1)\n", "target_label_index = 9" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "## Viewing Log Output\n", "\n", "Our package logs to STDOUT at the `debug`, `info`, `notice` and `warn` levels. \n", "The default output level is to show logs only at the `notice` level and above, which is also the recommended level." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[36m[notice | MIPVerify]: Rebuilding model from scratch. This may take some time as we determine upper and lower bounds for the input to each non-linear unit. The model built will be cached and re-used for future solves, unless you explicitly set rebuild=false.\n", "\u001b[39m\u001b[36m[notice | MIPVerify]: Attempting to find adversarial example. Neural net predicted label is 8, target labels are [9]\n", "\u001b[39mOptimize a model with 3385 rows, 3256 columns and 71132 nonzeros\n", "Variable types: 3196 continuous, 60 integer (60 binary)\n", "Coefficient statistics:\n", " Matrix range [2e-05, 7e+02]\n", " Objective range [1e+00, 1e+00]\n", " Bounds range [1e+00, 1e+02]\n", " RHS range [1e-02, 7e+02]\n", "Presolve removed 2860 rows and 2184 columns\n", "Presolve time: 0.11s\n", "Presolved: 525 rows, 1072 columns, 65472 nonzeros\n", "\n", "MIP start did not produce a new incumbent solution\n", "MIP start violates constraint R1024 by 1.000000000\n", "\n", "Variable types: 1012 continuous, 60 integer (60 binary)\n", "\n", "Root relaxation: objective 0.000000e+00, 242 iterations, 0.01 seconds\n", "\n", " Nodes | Current Node | Objective Bounds | Work\n", " Expl Unexpl | Obj Depth IntInf | Incumbent BestBd Gap | It/Node Time\n", "\n", " 0 0 0.00000 0 9 - 0.00000 - - 0s\n", "Another try with MIP start\n", "H 0 0 55.3332559 0.00000 100% - 0s\n", " 0 0 0.00000 0 10 55.33326 0.00000 100% - 0s\n", "H 0 0 27.1696380 0.00000 100% - 0s\n", " 0 0 0.00000 0 10 27.16964 0.00000 100% - 0s\n", " 0 0 0.00000 0 11 27.16964 0.00000 100% - 0s\n", "H 0 0 9.8928387 0.00000 100% - 0s\n", "H 0 0 9.8622769 0.00000 100% - 0s\n", " 0 0 0.00000 0 1 9.86228 0.00000 100% - 0s\n", " 0 0 0.00000 0 1 9.86228 0.00000 100% - 0s\n", " 0 0 0.00000 0 1 9.86228 0.00000 100% - 0s\n", " 0 2 0.00000 0 1 9.86228 0.00000 100% - 0s\n", "* 354 156 25 9.7569972 0.00000 100% 68.9 1s\n", " 4777 468 8.90557 19 10 9.75700 4.59017 53.0% 58.4 5s\n", "\n", "Cutting planes:\n", " Gomory: 3\n", " Cover: 3\n", " MIR: 7\n", " Flow cover: 10\n", "\n", "Explored 6902 nodes (388477 simplex iterations) in 6.66 seconds\n", "Thread count was 8 (of 8 available processors)\n", "\n", "Solution count 5: 9.757 9.86228 9.89284 ... 55.3333\n", "Pool objective bound 9.757\n", "\n", "Optimal solution found (tolerance 1.00e-04)\n", "Best objective 9.756997241475e+00, best bound 9.756997241475e+00, gap 0.0000%\n" ] } ], "source": [ "d = MIPVerify.find_adversarial_example(n1params, sample_image, target_label_index, GurobiSolver(), rebuild=true);" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "More information is available if we set the log level to `debug`." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[36m[notice | MIPVerify]: Rebuilding model from scratch. This may take some time as we determine upper and lower bounds for the input to each non-linear unit. The model built will be cached and re-used for future solves, unless you explicitly set rebuild=false.\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 0.0\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 236.19499039114618\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 329.4706219653086\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 347.3507384582369\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 490.3354319365959\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 111.66031695101941\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 230.85744098698652\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 110.09320929282381\n", "\u001b[39m\u001b[32m[info | MIPVerify]: Hit user limit during solve to determine bounds. Multiplicative gap was 0.5778275495135097.\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 250.1943900717896\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 263.9686250131348\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 443.7377089444059\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 170.58623081063135\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 546.9967283877565\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 342.81111856390635\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 561.8499789605797\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 322.1609214959067\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 252.888615674684\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 190.94181084213062\n", "\u001b[39m\u001b[32m[info | MIPVerify]: Hit user limit during solve to determine bounds. Multiplicative gap was 0.9188018145812635.\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 389.16818001562916\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 120.5688694681638\n", "\u001b[39m\u001b[32m[info | MIPVerify]: Hit user limit during solve to determine bounds. Multiplicative gap was 0.43244015627890753.\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 314.1279773027621\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 333.3978216299831\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 395.66929790133076\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 156.25265560651098\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 263.63110342096843\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 306.08282612869147\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 259.3597215144569\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 176.38206258926618\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 279.1469821915614\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 301.9881744933099\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 363.84051783382193\n", "\u001b[39m\u001b[32m[info | MIPVerify]: Hit user limit during solve to determine bounds. Multiplicative gap was 0.4156007693486492.\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 306.02212780906495\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 223.20192365427127\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 146.17861158898927\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 397.6660964044127\n", "\u001b[39m\u001b[32m[info | MIPVerify]: Hit user limit during solve to determine bounds. Multiplicative gap was 0.23529601270679312.\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 278.0674748427061\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 354.40977058244283\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 184.93657306973148\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 347.3454124031803\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δu = 256.17035614083665\n", "\u001b[39m\u001b[34m[debug | MIPVerify]: Δl = 540.5128944047548\n", "\u001b[39m\u001b[36m[notice | MIPVerify]: Attempting to find adversarial example. Neural net predicted label is 8, target labels are [9]\n", "\u001b[39mOptimize a model with 3385 rows, 3256 columns and 71132 nonzeros\n", "Variable types: 3196 continuous, 60 integer (60 binary)\n", "Coefficient statistics:\n", " Matrix range [2e-05, 7e+02]\n", " Objective range [1e+00, 1e+00]\n", " Bounds range [1e+00, 1e+02]\n", " RHS range [1e-02, 7e+02]\n", "Presolve removed 2860 rows and 2184 columns\n", "Presolve time: 0.13s\n", "Presolved: 525 rows, 1072 columns, 65472 nonzeros\n", "\n", "MIP start did not produce a new incumbent solution\n", "MIP start violates constraint R1024 by 1.000000000\n", "\n", "Variable types: 1012 continuous, 60 integer (60 binary)\n", "\n", "Root relaxation: objective 0.000000e+00, 242 iterations, 0.01 seconds\n", "\n", " Nodes | Current Node | Objective Bounds | Work\n", " Expl Unexpl | Obj Depth IntInf | Incumbent BestBd Gap | It/Node Time\n", "\n", " 0 0 0.00000 0 9 - 0.00000 - - 0s\n", "Another try with MIP start\n", "H 0 0 55.3332559 0.00000 100% - 0s\n", " 0 0 0.00000 0 10 55.33326 0.00000 100% - 0s\n", "H 0 0 27.1696380 0.00000 100% - 0s\n", " 0 0 0.00000 0 10 27.16964 0.00000 100% - 0s\n", " 0 0 0.00000 0 11 27.16964 0.00000 100% - 0s\n", "H 0 0 9.8928387 0.00000 100% - 0s\n", "H 0 0 9.8622769 0.00000 100% - 0s\n", " 0 0 0.00000 0 1 9.86228 0.00000 100% - 0s\n", " 0 0 0.00000 0 1 9.86228 0.00000 100% - 0s\n", " 0 0 0.00000 0 1 9.86228 0.00000 100% - 0s\n", " 0 2 0.00000 0 1 9.86228 0.00000 100% - 0s\n", "* 839 309 27 9.7569972 0.00000 100% 70.8 1s\n", " 4504 347 5.30680 13 13 9.75700 5.30680 45.6% 62.8 5s\n", "\n", "Cutting planes:\n", " Gomory: 3\n", " Cover: 2\n", " MIR: 7\n", " Flow cover: 11\n", "\n", "Explored 6030 nodes (364445 simplex iterations) in 6.24 seconds\n", "Thread count was 8 (of 8 available processors)\n", "\n", "Solution count 5: 9.757 9.86228 9.89284 ... 55.3333\n", "Pool objective bound 9.757\n", "\n", "Optimal solution found (tolerance 1.00e-04)\n", "Best objective 9.756997241475e+00, best bound 9.756997241475e+00, gap 0.0000%\n" ] }, { "data": { "text/plain": [ "Dict{Symbol,Any} with 7 entries:\n", " :PerturbationParameters => additive\n", " :TargetIndexes => [9]\n", " :SolveStatus => :Optimal\n", " :Output => JuMP.GenericAffExpr{Float64,JuMP.Variable}[-0.0120…\n", " :Model => Minimization problem with:…\n", " :Perturbation => JuMP.Variable[__anon__ __anon__ __anon__ __anon__ …\n", " :PerturbedInput => JuMP.Variable[__anon__ __anon__ __anon__ __anon__ …" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "MIPVerify.setloglevel!(\"debug\")\n", "d = MIPVerify.find_adversarial_example(n1params, sample_image, target_label_index, GurobiSolver(), rebuild=true)" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "When we select `rebuild=false`, we do not do any of the work in determining upper and lower bounds, and none of the `info` and `debug` logs are produced." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "\u001b[36m[notice | MIPVerify]: Loading model from cache.\n", "\u001b[39m\u001b[36m[notice | MIPVerify]: Attempting to find adversarial example. Neural net predicted label is 8, target labels are [9]\n", "\u001b[39mOptimize a model with 3385 rows, 3256 columns and 71132 nonzeros\n", "Variable types: 3196 continuous, 60 integer (60 binary)\n", "Coefficient statistics:\n", " Matrix range [2e-05, 7e+02]\n", " Objective range [1e+00, 1e+00]\n", " Bounds range [1e+00, 1e+02]\n", " RHS range [1e-02, 7e+02]\n", "Presolve removed 2860 rows and 2184 columns\n", "Presolve time: 0.13s\n", "Presolved: 525 rows, 1072 columns, 65472 nonzeros\n", "\n", "MIP start did not produce a new incumbent solution\n", "MIP start violates constraint R1024 by 1.000000000\n", "\n", "Variable types: 1012 continuous, 60 integer (60 binary)\n", "\n", "Root relaxation: objective 0.000000e+00, 242 iterations, 0.01 seconds\n", "\n", " Nodes | Current Node | Objective Bounds | Work\n", " Expl Unexpl | Obj Depth IntInf | Incumbent BestBd Gap | It/Node Time\n", "\n", " 0 0 0.00000 0 9 - 0.00000 - - 0s\n", "Another try with MIP start\n", "H 0 0 55.3332559 0.00000 100% - 0s\n", " 0 0 0.00000 0 10 55.33326 0.00000 100% - 0s\n", "H 0 0 27.1696380 0.00000 100% - 0s\n", " 0 0 0.00000 0 10 27.16964 0.00000 100% - 0s\n", " 0 0 0.00000 0 11 27.16964 0.00000 100% - 0s\n", "H 0 0 9.8928387 0.00000 100% - 0s\n", "H 0 0 9.8622769 0.00000 100% - 0s\n", " 0 0 0.00000 0 1 9.86228 0.00000 100% - 0s\n", " 0 0 0.00000 0 1 9.86228 0.00000 100% - 0s\n", " 0 0 0.00000 0 1 9.86228 0.00000 100% - 0s\n", " 0 2 0.00000 0 1 9.86228 0.00000 100% - 0s\n", "* 839 309 27 9.7569972 0.00000 100% 70.8 1s\n", " 4504 347 5.30680 13 13 9.75700 5.30680 45.6% 62.8 5s\n", "\n", "Cutting planes:\n", " Gomory: 3\n", " Cover: 2\n", " MIR: 7\n", " Flow cover: 11\n", "\n", "Explored 6030 nodes (364445 simplex iterations) in 6.24 seconds\n", "Thread count was 8 (of 8 available processors)\n", "\n", "Solution count 5: 9.757 9.86228 9.89284 ... 55.3333\n", "Pool objective bound 9.757\n", "\n", "Optimal solution found (tolerance 1.00e-04)\n", "Best objective 9.756997241475e+00, best bound 9.756997241475e+00, gap 0.0000%\n" ] } ], "source": [ "d = MIPVerify.find_adversarial_example(n1params, sample_image, target_label_index, GurobiSolver(), rebuild=false);" ] }, { "cell_type": "markdown", "metadata": { "deletable": true, "editable": true }, "source": [ "## Handling log output\n", "\n", "We use the logger provided by the [`Memento`](https://github.com/invenia/Memento.jl) package. Rather than just having logs written to STDOUT, you can choose to capture the output (for example, to write logs to file) by adding your own handlers to `MIPVerify.LOGGER`. See the Memento documentation for more." ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false, "deletable": true, "editable": true }, "outputs": [ { "data": { "text/plain": [ "Memento.Logger" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "typeof(MIPVerify.LOGGER)" ] } ], "metadata": { "kernelspec": { "display_name": "Julia 0.6.0", "language": "julia", "name": "julia-0.6" }, "language_info": { "file_extension": ".jl", "mimetype": "application/julia", "name": "julia", "version": "0.6.0" } }, "nbformat": 4, "nbformat_minor": 2 }