{ "cells": [ { "cell_type": "markdown", "metadata": { "slideshow": { "slide_type": "slide" } }, "source": [ "# MLRepo - A Quick Introduction\n", "In this notebook we give a quick introduction working with the repository and explain the basic priniples." ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "code_folding": [] }, "outputs": [ { "data": { "text/html": [ " \n", " " ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "#import all things you need to get startes\n", "import time\n", "import pandas as pd\n", "import logging as logging\n", "import pprint\n", "import plotly\n", "plotly.offline.init_notebook_mode(connected=True)\n", "\n", "# Here start the repository specific imports\n", "import pailab.ml_repo.memory_handler as memory_handler\n", "from pailab import RepoInfoKey, MeasureConfiguration, MLRepo, DataSet, MLObjectType, FIRST_VERSION, LAST_VERSION\n", "from pailab.job_runner.job_runner import SimpleJobRunner #, JobState\n", "\n", "#You may set the loglevel and log-format here. \n", "logging.basicConfig(level=logging.FATAL)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Read the data\n", "As an example machine learning task to ilustrate the way of working with the repository we use the Boston housing data from the [UCI Machine Learning Repository](http://archive.ics.uci.edu/ml/index.php) where we have applied some preprocessing. The data consists of house prices together with the house features `'RM'`, `'LSTAT'`, and `'PTRATIO'`:\n", "- `'RM'` is the average number of rooms among homes in the neighborhood.\n", "- `'LSTAT'` is the percentage of homeowners in the neighborhood considered \"lower class\" (working poor).\n", "- `'PTRATIO'` is the ratio of students to teachers in primary and secondary schools in the neighborhood.\n", "We just read the csv-file containing the data (also in the repository) into a pandas dataframe." ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": true }, "outputs": [], "source": [ "data = pd.read_csv('housing.csv')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Create a new repository\n", "We first create a new repository for our task. The repository is the central key around all functionality is built. Similar to a repository used for source control in classical software development it contains all data and algorithms needed for the machine learning task. The repository needs storages for \n", "- scripts containing the machine learning algorithms and interfaces,\n", "- numerical objects such as arrays and matrices representing data, e.g. input data, data from the valuation of the models,\n", "- json documents representing parameters, e.g. training parameter, model parameter.\n", "\n", "To keep things simple, we just start using in memory storages. Note that the used memory interfaces are except for testing and playing around not be the first choice, since when ending the session, everything will be lost...\n", "\n", "In addition to the storages the repository needs a reference to a JobRunner which the platform can use to execute machine learning jobs. For this example we use the most simple one, executing everything sequential in the same thread, the repository runs in." ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "code_folding": [], "collapsed": true }, "outputs": [], "source": [ "# setting up the repository\n", "config = {'user': 'test_user',\n", " 'workspace': 'c:/temp',\n", " 'repo_store': \n", " {\n", " 'type': 'memory_handler', \n", " 'config': {}\n", " },\n", " 'numpy_store':\n", " {\n", " 'type': 'memory_handler',\n", " 'config':{}\n", " },\n", " 'job_runner':\n", " {\n", " 'type': 'simple',\n", " 'config': {\n", " 'throw_job_error': True\n", " }\n", " }\n", " }\n", "ml_repo = MLRepo( user = 'test_user', config=config)\n", "job_runner = ml_repo._job_runner" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Add a tree\n", "To navigate in a simple way over all objects, one can add a so-called tree to the repository. The tree allows one to use auto completion to acces objcts and respectiv methods." ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from pailab.tools.tree import MLTree\n", "MLTree.add_tree(ml_repo)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Adding data\n", "The data in the repository is handled by two different data objects:\n", "- RawData is the object containing real data.\n", "- DataSet is the object conaining the logical data, i.e. a reference to a RawData object together with a specification, which data from the RawData will be used. Here, one can specify a fixed version of the underlying RawData object (then changes to the RawData will not affect the derived DataSet) or a fixed or floating subset of the RawData by defininga start and endindex cutting the derived data just out of the original data.\n", "\n", "Normally one will add RawData and then define DataSets which are used to train or test a model which is exactly the way shown in the following." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": true }, "outputs": [], "source": [ "# Add RawData. A convenient way to add RawData is to use the method add on the raw_data collection.\n", "# This method just takes a pandas dataframe and the specification, which columns belong to the input \n", "#and which to the targets.\n", "ml_repo.tree.raw_data.add('boston_housing', data, input_variables=['RM', 'LSTAT', 'PTRATIO'], target_variables = ['MEDV'])\n", "\n", "# based on the raw data we now define training and test sets\n", "ml_repo.tree.training_data.add('sample1', ml_repo.tree.raw_data.boston_housing(), 0, 300)\n", "ml_repo.tree.test_data.add('sample2', ml_repo.tree.raw_data.boston_housing(), 301, None)" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "{'83d7f948-7719-11e9-bad3-fc084a6691eb': {'RepoInfoKey.AUTHOR': 'test_user',\n", " 'RepoInfoKey.COMMIT_DATE': '2019-05-15 15:58:28.136788',\n", " 'RepoInfoKey.COMMIT_MESSAGE': ''}}" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ml_repo.tree.training_data.sample1.history()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "When creating the DataSet we have to set two important informations for the repository, given as a dictionary:\n", "- The object name. Each object in the repository needs to have a unique name in the repository.\n", "- The object type which gives. In our example here we say that we specify that the DataSet are training and test data. Note that on can have only one training data object pre repository while the repository can obtain many different test data sets.\n", "\n", "Some may wonder what is now stored in *version_list*.\n", "** Adding an object (independent if it is a data object or some other object such as a parameter), the object gets a version number and no object will be removed, adding just adds a new version.** The add method returns a dictionary of the object names together with their version number." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Adding a model\n", "The next step to do machine learning would be to define a model which will be used in the repository. A model consists of the following pieces\n", "- a skript where the code for the model valuation is defined together with the function name of the evaluation method\n", "- a skript where the code for the model training is defined together with th function nam of the training method\n", "- a model parameter object defining the model parameter and which must have implemented the correct interface so that it can be used within the repository (see the documentation on integrating new objects, normally there is not more to do then just simply add *@repo_object_init()* to the line above your *__init__* method)\n", "- a training parameter object defining training parameters (such as number of optimization steps etc.), if necessary for your algorithms (this oen is optional)\n", "\n", "** SKLearn models as an example**\n", "\n", "We do not have to define the pieces defined above, if we use the sklearn module. Instead we can use the pailab.externals.sklearn module interfacing \n", "the sklearn package so that this can be used within the repository. This interface provides a simple method (add_model) to add an arbitrary sklearn model as a model which can be handled by the repository. This method adds a bunch of repo objects to the repository (according to the pieces described above):\n", "- An object defining the function to be called to evaluate the model\n", "- An object defining the function to be called to train the model\n", "- An object defining the model\n", "- An object defining the model parameter\n", "For the following we just use a DecisionTree as our model." ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import pailab.externals.sklearn_interface as sklearn_interface\n", "from sklearn.tree import DecisionTreeRegressor\n", "\n", "sklearn_interface.add_model(ml_repo, DecisionTreeRegressor(), model_param={'max_depth': 5})" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Train the model\n", "Now, model taining is very simple, since you have defined training and testing data as well as methods to value and fit your model and the model parameter.\n", "So, you can just call *run_training* on the repository, and the training is perfomred automatically.\n", "The training job is executed via the JobRunner you specified setting up the repository. All method of the repository involving jobs return the job id when adding the job to the JobRunner so that you can control the status of the task and see if it sucessfully finished." ] }, { "cell_type": "code", "execution_count": 8, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "('DecisionTreeRegressor/jobs/training', '83f22f1a-7719-11e9-81b1-fc084a6691eb')\n", "test_user, successfully_finished, started 2019-05-15 15:58:28.308559, finished 2019-05-15 15:58:28.316548\n" ] } ], "source": [ "job_id = ml_repo.run_training() \n", "print(job_id)\n", "job_info = job_runner.get_info(job_id[0], job_id[1])\n", "print(job_info)" ] }, { "cell_type": "code", "execution_count": 9, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'No new training started: A model has already been trained on the latest data.'" ] }, "execution_count": 9, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# train a second time\n", "ml_repo.run_training()" ] }, { "cell_type": "code", "execution_count": 10, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'No new training started: A model has already been trained on the latest data.'" ] }, "execution_count": 10, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# starting training again\n", "ml_repo.run_training() " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Run evaluation\n", "To measure errors and to provide plots the model must be evaluated on all test and training datasets." ] }, { "cell_type": "code", "execution_count": 11, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "test_user, successfully_finished, started 2019-05-15 15:58:28.492314, finished 2019-05-15 15:58:28.496309\n" ] } ], "source": [ "job_id = ml_repo.run_evaluation()\n", "# print information about the job\n", "info = job_runner.get_info(job_id[0][0], job_id[0][1]) \n", "print(str(info))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Add and compute measures" ] }, { "cell_type": "code", "execution_count": 12, "metadata": { "collapsed": true }, "outputs": [], "source": [ "ml_repo.add_measure(MeasureConfiguration.MAX)\n", "ml_repo.add_measure(MeasureConfiguration.R2)\n", "ml_repo.add_measure(MeasureConfiguration.MSE)" ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": true }, "outputs": [], "source": [ "job_ids = ml_repo.run_measures()" ] }, { "cell_type": "code", "execution_count": 14, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "223200.0\n" ] } ], "source": [ "ml_repo.tree.models.DecisionTreeRegressor.measures.sample1.max.load()\n", "print(str(ml_repo.tree.models.DecisionTreeRegressor.measures.sample1.max.obj.value))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that the steps\n", "- *run_evaluation*\n", "- *run_measures*\n", "\n", "are not necessary if *run_training* is called with the keyword argument *run_descendants=True*. \n", "In This case the repository would have automatically triggered all evaluations and measurement calculations automatically." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Working with the repository\n", "This section shows how one can work with the audit and revision functionality of the repository." ] }, { "cell_type": "code", "execution_count": 15, "metadata": { "code_folding": [ 0 ] }, "outputs": [ { "data": { "text/html": [ " \n", " " ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "2a4a4e2955af40f4977f2587a25c59ab", "version_major": 2, "version_minor": 0 }, "text/plain": [ "HBox(children=(VBox(children=(SelectMultiple(options=('EVAL_DATA', 'RAW_DATA', 'TRAINING_DATA', 'TEST_DATA', '…" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from pailab.analysis.tools_jupyter import ObjectOverviewList\n", "\n", "repo_overview = ObjectOverviewList(ml_repo)\n", "display(repo_overview.get_widget())" ] }, { "cell_type": "code", "execution_count": 16, "metadata": { "code_folding": [ 0 ] }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "064ba133e0e047dc97023b9b61150262", "version_major": 2, "version_minor": 0 }, "text/plain": [ "HBox(children=(VBox(children=(SelectMultiple(index=(16,), options=('EVAL_DATA', 'RAW_DATA', 'TRAINING_DATA', '…" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from pailab.analysis.tools_jupyter import ObjectView\n", "\n", "obj_view = ObjectView(ml_repo)\n", "display(obj_view.get_widget())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Repository information such as version number, author, date of change are attached to the repo objects and can simply be retrieved:" ] }, { "cell_type": "code", "execution_count": 17, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'author': 'test_user',\n", " 'big_objects': [],\n", " 'category': 'MEASURE',\n", " 'classname': 'pailab.ml_repo.repo_objects.Measure',\n", " 'commit_date': '2019-05-15 15:58:28.628133',\n", " 'commit_message': 'computing measure r2 on data sample1',\n", " 'description': None,\n", " 'modification_info': {'DecisionTreeRegressor': '83eb7ac1-7719-11e9-8248-fc084a6691eb',\n", " 'DecisionTreeRegressor/eval/sample1': '840f712c-7719-11e9-92ea-fc084a6691eb',\n", " 'DecisionTreeRegressor/model': '83f2cb24-7719-11e9-abf4-fc084a6691eb',\n", " 'DecisionTreeRegressor/model_param': '83eb7ac0-7719-11e9-b37e-fc084a6691eb',\n", " 'sample1': '83d7f948-7719-11e9-bad3-fc084a6691eb'},\n", " 'modifiers': {},\n", " 'name': 'DecisionTreeRegressor/measure/sample1/r2',\n", " 'version': '8422f276-7719-11e9-8b9d-fc084a6691eb'}\n" ] } ], "source": [ "ml_repo.tree.models.DecisionTreeRegressor.measures.sample1.r2.load()\n", "pprint.pprint(ml_repo.tree.models.DecisionTreeRegressor.measures.sample1.r2.obj.repo_info.get_dictionary())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "The commits can also be queried and printed. " ] }, { "cell_type": "code", "execution_count": 18, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'author': 'test_user',\n", " 'message': 'data raw_data/boston_housing added to repository',\n", " 'objects': {'raw_data/boston_housing': '83d75d42-7719-11e9-b70e-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 136788)}\n", "{'author': 'test_user',\n", " 'message': '',\n", " 'objects': {'sample1': '83d7f948-7719-11e9-bad3-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 136788)}\n", "{'author': 'test_user',\n", " 'message': '',\n", " 'objects': {'sample2': '83d7f949-7719-11e9-b21a-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 136788)}\n", "{'author': 'test_user',\n", " 'message': 'add function eval_sklearn of category MODEL_EVAL_FUNCTION to the '\n", " 'repo',\n", " 'objects': {'eval_sklearn': '83eade94-7719-11e9-bf9b-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 260622)}\n", "{'author': 'test_user',\n", " 'message': 'add function train_sklearn of category TRAINING_FUNCTION to the '\n", " 'repo',\n", " 'objects': {'train_sklearn': '83eade95-7719-11e9-a1a7-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 260622)}\n", "{'author': 'test_user',\n", " 'message': 'adding model and training parameter',\n", " 'objects': {'DecisionTreeRegressor/model_param': '83eb7ac0-7719-11e9-b37e-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 264620)}\n", "{'author': 'test_user',\n", " 'message': 'add model DecisionTreeRegressor',\n", " 'objects': {'DecisionTreeRegressor': '83eb7ac1-7719-11e9-8248-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 264620)}\n", "{'author': 'test_user',\n", " 'message': '',\n", " 'objects': {'DecisionTreeRegressor/jobs/training': '83f22f1a-7719-11e9-81b1-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 308559)}\n", "{'author': 'test_user',\n", " 'message': 'training of model DecisionTreeRegressor',\n", " 'objects': {'DecisionTreeRegressor/model': '83f2cb24-7719-11e9-abf4-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 316548)}\n", "{'author': 'test_user',\n", " 'message': '',\n", " 'objects': {'DecisionTreeRegressor/model/jobs/eval_job/sample2': '840e390a-7719-11e9-8606-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 492314)}\n", "{'author': 'test_user',\n", " 'message': 'evaluate data sample2 with model DecisionTreeRegressor/model',\n", " 'objects': {'DecisionTreeRegressor/eval/sample2': '840ed512-7719-11e9-bed2-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 496309)}\n", "{'author': 'test_user',\n", " 'message': '',\n", " 'objects': {'DecisionTreeRegressor/model/jobs/eval_job/sample1': '840ed513-7719-11e9-9ce4-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 496309)}\n", "{'author': 'test_user',\n", " 'message': 'evaluate data sample1 with model DecisionTreeRegressor/model',\n", " 'objects': {'DecisionTreeRegressor/eval/sample1': '840f712c-7719-11e9-92ea-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 500305)}\n", "{'author': 'test_user',\n", " 'message': 'added measure max for coordinates all',\n", " 'objects': {'measure_config': '8416c194-7719-11e9-bb6e-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 548238)}\n", "{'author': 'test_user',\n", " 'message': 'added measure r2 for coordinates all',\n", " 'objects': {'measure_config': '8416c195-7719-11e9-ab4c-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 548238)}\n", "{'author': 'test_user',\n", " 'message': 'added measure mse for coordinates all',\n", " 'objects': {'measure_config': '8416c196-7719-11e9-8caf-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 548238)}\n", "{'author': 'test_user',\n", " 'message': '',\n", " 'objects': {'DecisionTreeRegressor/model/jobs/measure/sample2/max': '841fe638-7719-11e9-988f-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 608159)}\n", "{'author': 'test_user',\n", " 'message': 'computing measure max on data sample2',\n", " 'objects': {'DecisionTreeRegressor/measure/sample2/max': '84208246-7719-11e9-b996-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 612154)}\n", "{'author': 'test_user',\n", " 'message': '',\n", " 'objects': {'DecisionTreeRegressor/model/jobs/measure/sample2/r2': '84208247-7719-11e9-9a2a-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 616148)}\n", "{'author': 'test_user',\n", " 'message': 'computing measure r2 on data sample2',\n", " 'objects': {'DecisionTreeRegressor/measure/sample2/r2': '84211e50-7719-11e9-b5b5-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 616148)}\n", "{'author': 'test_user',\n", " 'message': '',\n", " 'objects': {'DecisionTreeRegressor/model/jobs/measure/sample2/mse': '8421ba5c-7719-11e9-8038-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 620143)}\n", "{'author': 'test_user',\n", " 'message': 'computing measure mse on data sample2',\n", " 'objects': {'DecisionTreeRegressor/measure/sample2/mse': '8421ba5d-7719-11e9-8d36-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 620143)}\n", "{'author': 'test_user',\n", " 'message': '',\n", " 'objects': {'DecisionTreeRegressor/model/jobs/measure/sample1/max': '8422566c-7719-11e9-ad3a-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 624138)}\n", "{'author': 'test_user',\n", " 'message': 'computing measure max on data sample1',\n", " 'objects': {'DecisionTreeRegressor/measure/sample1/max': '8422566d-7719-11e9-95cb-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 624138)}\n", "{'author': 'test_user',\n", " 'message': '',\n", " 'objects': {'DecisionTreeRegressor/model/jobs/measure/sample1/r2': '8422566e-7719-11e9-8430-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 624138)}\n", "{'author': 'test_user',\n", " 'message': 'computing measure r2 on data sample1',\n", " 'objects': {'DecisionTreeRegressor/measure/sample1/r2': '8422f276-7719-11e9-8b9d-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 628133)}\n", "{'author': 'test_user',\n", " 'message': '',\n", " 'objects': {'DecisionTreeRegressor/model/jobs/measure/sample1/mse': '8422f277-7719-11e9-9a6b-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 628133)}\n", "{'author': 'test_user',\n", " 'message': 'computing measure mse on data sample1',\n", " 'objects': {'DecisionTreeRegressor/measure/sample1/mse': '84238e76-7719-11e9-b020-fc084a6691eb'},\n", " 'time': datetime.datetime(2019, 5, 15, 15, 58, 28, 632127)}\n" ] } ], "source": [ "for k in ml_repo.get_commits():\n", " pprint.pprint(k.to_dict())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Labeling models\n", "There is the possibility to label a certain version of a model. The label can then be used to access the model instead of the version number. It is vry useful to\n", "compare e.g. the current productive model (labeld e.g. 'prod') against other model versions. abels are supported by many functions and tools and make life much easier. So the consistency checks only check for the latest and labeled models if there are changes make a rerun of training/evaluation/measures needed. Also som figures will automatically highlight the results belonging to labeled versions.\n", "\n", "Let us label the latest model version in the repo." ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "collapsed": true }, "outputs": [], "source": [ "ml_repo.tree.models.DecisionTreeRegressor.set_label('prod',version = LAST_VERSION, message='we found our first production model')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Tests\n", "It is possible to define model tests.\n", "### Regressiontests\n", "Regressiontests compare measuresments on the repositories dataset of a model to th measurements of labeled reference model." ] }, { "cell_type": "code", "execution_count": 20, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "'846b7b9a-7719-11e9-b0be-fc084a6691eb'" ] }, "execution_count": 20, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pailab.tools.tests\n", "reg_test = pailab.tools.tests.RegressionTestDefinition(reference='prod', models=None, data=None, labels=None, measures=None, tol=1e-3)\n", "\n", "reg_test.repo_info.name='reg_test'\n", "#reg_test.repo_info.category = MLObjectType.TEST_DEFINITION\n", "ml_repo.add(reg_test, message='regression test definition')\n", "#ml_repo.tree.models.DecisionTreeRegressor.measures.sample1." ] }, { "cell_type": "code", "execution_count": 21, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[('DecisionTreeRegressor/tests/reg_test/sample2',\n", " '847367f4-7719-11e9-992f-fc084a6691eb'),\n", " ('DecisionTreeRegressor/tests/reg_test/sample1',\n", " '847367f5-7719-11e9-8f48-fc084a6691eb')]" ] }, "execution_count": 21, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ml_repo.run_tests()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Consistency checks\n", "Pailab's *checks*-submodule provides functionality to check for consistency and quality issues as well as for outstanding tasks (such as rerunning a training after the training set has been changed).\n", "\n", "### Model consistency\n", "There are different checks to test model consistency such as if the tests of a model are up to date and succeeded or if the latest model is trained on the latest trainin data. All model tests are performed for **labeled** models and the latest model only.\n", "In our first example we change a model parameter but do not train for a new model version wih this parameter.\n", "\n", "The following checks are performed:\n", "- Is the latest model calibrated on the latest parameters and training data\n", "- Are all labeled models (including latest model) evaluated on the latest available training and test data\n", "- Are all measures of all labeled models computed on the latest data\n", "- Have all tests been run on the labeled models" ] }, { "cell_type": "code", "execution_count": 22, "metadata": { "code_folding": [] }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "{'criterion': 'mse', 'max_depth': 2, 'max_features': None, 'max_leaf_nodes': None, 'min_impurity_decrease': 0.0, 'min_impurity_split': None, 'min_samples_leaf': 1, 'min_samples_split': 2, 'min_weight_fraction_leaf': 0.0, 'presort': False, 'random_state': None, 'splitter': 'best'}\n" ] } ], "source": [ "param = ml_repo.get('DecisionTreeRegressor/model_param')\n", "param.sklearn_params['max_depth'] = 2\n", "version = ml_repo.add(param)\n", "print(param.sklearn_params)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "After we have changed the model parameter, we use the *tools* submodules *check_model* method for open tasks/inconsistencies. This method can be called for a certain model or also for a lbeled model. If nothing is specified, all labeled models will be checked.\n", "Applying the method to the latest model we see that the output shows that the models last version has been calibrated using a different model parameter version then the current version. " ] }, { "cell_type": "code", "execution_count": 23, "metadata": { "code_folding": [ 0 ] }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "72e081eef8df4df686b9a3ffade2c266", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(Button(description='update', style=ButtonStyle()), Output()))" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "from pailab.analysis.tools_jupyter import ConsistencyChecker\n", "\n", "consistency_checker = ConsistencyChecker(ml_repo)\n", "display(consistency_checker.get_widget())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that the labeled model would not show an issue. Only the latest model is affected by this change (it is the definition of latest model that it has been calibrated on latest inputs).\n", "\n", "We can resolve this issue by simply training the model again (now on the new training data set)." ] }, { "cell_type": "code", "execution_count": 24, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "('DecisionTreeRegressor/jobs/training', '848f71da-7719-11e9-afa6-fc084a6691eb')" ] }, "execution_count": 24, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ml_repo.run_training()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Note that the solution to simply retrain introduced new issues: The model has not yet been evaluated and no measures have been computed. (If we would have set **run_descendants=False** as argument, the preceding steps would have also been performed and the issues would not have been present)." ] }, { "cell_type": "code", "execution_count": 25, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[('DecisionTreeRegressor/model/jobs/eval_job/sample2',\n", " '8497fb9e-7719-11e9-b233-fc084a6691eb'),\n", " ('DecisionTreeRegressor/model/jobs/eval_job/sample1',\n", " '849b069e-7719-11e9-b3bb-fc084a6691eb')]" ] }, "execution_count": 25, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ml_repo.run_evaluation(run_descendants=True)# we use run_descendants so that the issues with th measures are resolved too" ] }, { "cell_type": "code", "execution_count": 26, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[('DecisionTreeRegressor/tests/reg_test/sample2',\n", " '84a69b80-7719-11e9-804a-fc084a6691eb'),\n", " ('DecisionTreeRegressor/tests/reg_test/sample1',\n", " '84a7378c-7719-11e9-a0b4-fc084a6691eb')]" ] }, "execution_count": 26, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ml_repo.run_tests()" ] }, { "cell_type": "code", "execution_count": 27, "metadata": { "collapsed": true }, "outputs": [], "source": [ "ml_repo.tree.test_data.add('sample3', ml_repo.tree.raw_data.boston_housing(), 0, 50)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Append RawData\n", "One can append data to the RawData object. The repository manages which objects are affected by appending data and directly updates these objects." ] }, { "cell_type": "code", "execution_count": 28, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "sample1: {'raw_data': 'raw_data/boston_housing', 'start_index': 0, 'end_index': 300, 'raw_data_version': 'last', 'x_coord_names': ['RM', 'LSTAT', 'PTRATIO'], 'y_coord_names': ['MEDV'], 'n_data': 489} Version: 83d7f948-7719-11e9-bad3-fc084a6691eb\n", "\n", "sample2: {'raw_data': 'raw_data/boston_housing', 'start_index': 301, 'end_index': None, 'raw_data_version': 'last', 'x_coord_names': ['RM', 'LSTAT', 'PTRATIO'], 'y_coord_names': ['MEDV'], 'n_data': 489} Version: 83d7f949-7719-11e9-b21a-fc084a6691eb\n", "\n", "sample3: {'raw_data': 'raw_data/boston_housing', 'start_index': 0, 'end_index': 50, 'raw_data_version': 'last', 'x_coord_names': ['RM', 'LSTAT', 'PTRATIO'], 'y_coord_names': ['MEDV'], 'n_data': 489} Version: 84afc00c-7719-11e9-87ab-fc084a6691eb\n" ] } ], "source": [ "train_data = ml_repo.get_training_data(full_object = False)\n", "print(train_data.repo_info[RepoInfoKey.NAME] +': ' +str(train_data) +' Version: '+str(train_data.repo_info.version))\n", "test_data = ml_repo.get_names(MLObjectType.TEST_DATA)\n", "for k in test_data:\n", " t = ml_repo.get(k)\n", " print('')\n", " print(str(k) + ': ' + str(t)+ ' Version: ' + str(t.repo_info.version))" ] }, { "cell_type": "code", "execution_count": 29, "metadata": { "collapsed": true }, "outputs": [], "source": [ "from numpy import array\n", "ml_repo.tree.raw_data.boston_housing.append(x_data = array([[ 6.575, 4.98, 15.3]]), y_data =array([[504000.0]]))" ] }, { "cell_type": "code", "execution_count": 30, "metadata": {}, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "sample1: {'raw_data': 'raw_data/boston_housing', 'start_index': 0, 'end_index': 300, 'raw_data_version': 'last', 'x_coord_names': ['RM', 'LSTAT', 'PTRATIO'], 'y_coord_names': ['MEDV'], 'n_data': 490} Version: 83d7f948-7719-11e9-bad3-fc084a6691eb\n", "\n", "sample2: {'raw_data': 'raw_data/boston_housing', 'start_index': 301, 'end_index': None, 'raw_data_version': '84c03540-7719-11e9-a263-fc084a6691eb', 'x_coord_names': ['RM', 'LSTAT', 'PTRATIO'], 'y_coord_names': ['MEDV'], 'n_data': 490} Version: 84c0d13a-7719-11e9-92a0-fc084a6691eb\n", "\n", "sample3: {'raw_data': 'raw_data/boston_housing', 'start_index': 0, 'end_index': 50, 'raw_data_version': 'last', 'x_coord_names': ['RM', 'LSTAT', 'PTRATIO'], 'y_coord_names': ['MEDV'], 'n_data': 490} Version: 84afc00c-7719-11e9-87ab-fc084a6691eb\n" ] } ], "source": [ "train_data = ml_repo.get_training_data(full_object = False)\n", "print(train_data.repo_info[RepoInfoKey.NAME] +': ' +str(train_data) +' Version: '+str(train_data.repo_info.version))\n", "test_data = ml_repo.get_names(MLObjectType.TEST_DATA)\n", "for k in test_data:\n", " t = ml_repo.get(k)\n", " print('')\n", " print(str(k) + ': ' + str(t)+ ' Version: ' + str(t.repo_info.version))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Repo-Analysis\n", "Having parameters, evaluations, measures in one place enables out of the box analysis- and plotting functionality. The submodule *plot* provides automated, standardized plots." ] }, { "cell_type": "code", "execution_count": 31, "metadata": { "collapsed": true }, "outputs": [], "source": [ "import pailab.analysis.plot as plot" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Plotting\n", "### Plot errors and measures" ] }, { "cell_type": "code", "execution_count": 32, "metadata": { "code_folding": [ 1 ], "collapsed": true }, "outputs": [], "source": [ "# create more different model params etc. to make the repo more interesting ;-)\n", "for j in range(2):\n", " training_data = ml_repo.get(ml_repo.tree.training_data.sample1())\n", " training_data.end_index += 50\n", " ml_repo.add(training_data, message='add 50 datapoints to end_index')\n", " for i in range(6,12):\n", " #print(i)\n", " param = ml_repo.get(ml_repo.tree.models.DecisionTreeRegressor.model_param())\n", " param.sklearn_params['max_depth'] = i\n", " version = ml_repo.add(param)\n", " ml_repo.add(param)\n", " ml_repo.run_training()\n", " ml_repo.run_evaluation()\n", " ml_repo.run_measures()\n", " if j == 1 and i==6:\n", " ml_repo.tree.models.DecisionTreeRegressor.set_label('prod', message='')\n", " if j == 1 and i==8:\n", " ml_repo.tree.models.DecisionTreeRegressor.set_label('candidate', message='')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Plot error measure vs parameter" ] }, { "cell_type": "code", "execution_count": 33, "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "linkText": "Export to plot.ly", "plotlyServerURL": "https://plot.ly", "responsive": true, "showLink": false }, "data": [ { "mode": "markers", "name": "DecisionTreeRegressor/measure/sample2/max: 84c0d13a-7719-11e9-92a0-fc084a6691eb", "text": [ "model version: 84d7facd-7719-11e9-9a34-fc084a6691eb
sample3: 84c0d13a-7719-11e9-92a0-fc084a6691eb
train_data: 84d75ec8-7719-11e9-b375-fc084a6691eb", "model version: 83f2cb24-7719-11e9-abf4-fc084a6691eb
sample3: 84c0d13a-7719-11e9-92a0-fc084a6691eb
train_data: 83d7f948-7719-11e9-bad3-fc084a6691eb", "model version: 84e563b8-7719-11e9-958b-fc084a6691eb
sample3: 84c0d13a-7719-11e9-92a0-fc084a6691eb
train_data: 84d75ec8-7719-11e9-b375-fc084a6691eb", "model version: 84edec62-7719-11e9-bc17-fc084a6691eb
sample3: 84c0d13a-7719-11e9-92a0-fc084a6691eb
train_data: 84d75ec8-7719-11e9-b375-fc084a6691eb", "model version: 84f4a0d8-7719-11e9-af7c-fc084a6691eb
sample3: 84c0d13a-7719-11e9-92a0-fc084a6691eb
train_data: 84d75ec8-7719-11e9-b375-fc084a6691eb", "model version: 84fdc568-7719-11e9-8f99-fc084a6691eb
sample3: 84c0d13a-7719-11e9-92a0-fc084a6691eb
train_data: 84d75ec8-7719-11e9-b375-fc084a6691eb", "model version: 85078610-7719-11e9-9303-fc084a6691eb
sample3: 84c0d13a-7719-11e9-92a0-fc084a6691eb
train_data: 84d75ec8-7719-11e9-b375-fc084a6691eb", "model version: 8511e2da-7719-11e9-a641-fc084a6691eb
sample3: 84c0d13a-7719-11e9-92a0-fc084a6691eb
train_data: 851146d0-7719-11e9-ae8c-fc084a6691eb", "model version: 851d77e9-7719-11e9-b60b-fc084a6691eb
sample3: 84c0d13a-7719-11e9-92a0-fc084a6691eb
train_data: 851146d0-7719-11e9-ae8c-fc084a6691eb", "model version: 85269c2e-7719-11e9-a96b-fc084a6691eb
sample3: 84c0d13a-7719-11e9-92a0-fc084a6691eb
train_data: 851146d0-7719-11e9-ae8c-fc084a6691eb", "model version: 85319506-7719-11e9-a547-fc084a6691eb
sample3: 84c0d13a-7719-11e9-92a0-fc084a6691eb
train_data: 851146d0-7719-11e9-ae8c-fc084a6691eb", "model version: 853b55c2-7719-11e9-8f00-fc084a6691eb
sample3: 84c0d13a-7719-11e9-92a0-fc084a6691eb
train_data: 851146d0-7719-11e9-ae8c-fc084a6691eb", "model version: 85451654-7719-11e9-b144-fc084a6691eb
sample3: 84c0d13a-7719-11e9-92a0-fc084a6691eb
train_data: 851146d0-7719-11e9-ae8c-fc084a6691eb" ], "type": "scatter", "uid": "3f588952-1cdb-490d-979f-b2f1abfaf419", "x": [ 6, 5, 7, 8, 9, 10, 11, 6, 7, 8, 9, 10, 11 ], "y": [ 400050, 415450, 342300, 336000, 342300, 342300, 342300, 407400, 407400, 401100, 407400, 407400, 401100 ] }, { "mode": "markers", "name": "DecisionTreeRegressor/measure/sample1/max: 851146d0-7719-11e9-ae8c-fc084a6691eb", "text": [ "model version: 8511e2da-7719-11e9-a641-fc084a6691eb
sample3: 851146d0-7719-11e9-ae8c-fc084a6691eb
train_data: 851146d0-7719-11e9-ae8c-fc084a6691eb", "model version: 83f2cb24-7719-11e9-abf4-fc084a6691eb
sample3: 851146d0-7719-11e9-ae8c-fc084a6691eb
train_data: 83d7f948-7719-11e9-bad3-fc084a6691eb", "model version: 851d77e9-7719-11e9-b60b-fc084a6691eb
sample3: 851146d0-7719-11e9-ae8c-fc084a6691eb
train_data: 851146d0-7719-11e9-ae8c-fc084a6691eb", "model version: 85269c2e-7719-11e9-a96b-fc084a6691eb
sample3: 851146d0-7719-11e9-ae8c-fc084a6691eb
train_data: 851146d0-7719-11e9-ae8c-fc084a6691eb", "model version: 85319506-7719-11e9-a547-fc084a6691eb
sample3: 851146d0-7719-11e9-ae8c-fc084a6691eb
train_data: 851146d0-7719-11e9-ae8c-fc084a6691eb", "model version: 853b55c2-7719-11e9-8f00-fc084a6691eb
sample3: 851146d0-7719-11e9-ae8c-fc084a6691eb
train_data: 851146d0-7719-11e9-ae8c-fc084a6691eb", "model version: 85451654-7719-11e9-b144-fc084a6691eb
sample3: 851146d0-7719-11e9-ae8c-fc084a6691eb
train_data: 851146d0-7719-11e9-ae8c-fc084a6691eb" ], "type": "scatter", "uid": "e92f6f1e-fd2d-4fc4-aaef-eeee3b27ef56", "x": [ 6, 5, 7, 8, 9, 10, 11 ], "y": [ 182100, 415450, 170730, 160883.3333333333, 165900, 159096, 116200 ] }, { "mode": "markers", "name": "DecisionTreeRegressor/measure/sample3/max: 84afc00c-7719-11e9-87ab-fc084a6691eb", "text": [ "model version: 84d7facd-7719-11e9-9a34-fc084a6691eb
sample3: 84afc00c-7719-11e9-87ab-fc084a6691eb
train_data: 84d75ec8-7719-11e9-b375-fc084a6691eb", "model version: 83f2cb24-7719-11e9-abf4-fc084a6691eb
sample3: 84afc00c-7719-11e9-87ab-fc084a6691eb
train_data: 83d7f948-7719-11e9-bad3-fc084a6691eb", "model version: 84e563b8-7719-11e9-958b-fc084a6691eb
sample3: 84afc00c-7719-11e9-87ab-fc084a6691eb
train_data: 84d75ec8-7719-11e9-b375-fc084a6691eb", "model version: 84edec62-7719-11e9-bc17-fc084a6691eb
sample3: 84afc00c-7719-11e9-87ab-fc084a6691eb
train_data: 84d75ec8-7719-11e9-b375-fc084a6691eb", "model version: 84f4a0d8-7719-11e9-af7c-fc084a6691eb
sample3: 84afc00c-7719-11e9-87ab-fc084a6691eb
train_data: 84d75ec8-7719-11e9-b375-fc084a6691eb", "model version: 84fdc568-7719-11e9-8f99-fc084a6691eb
sample3: 84afc00c-7719-11e9-87ab-fc084a6691eb
train_data: 84d75ec8-7719-11e9-b375-fc084a6691eb", "model version: 85078610-7719-11e9-9303-fc084a6691eb
sample3: 84afc00c-7719-11e9-87ab-fc084a6691eb
train_data: 84d75ec8-7719-11e9-b375-fc084a6691eb", "model version: 8511e2da-7719-11e9-a641-fc084a6691eb
sample3: 84afc00c-7719-11e9-87ab-fc084a6691eb
train_data: 851146d0-7719-11e9-ae8c-fc084a6691eb", "model version: 851d77e9-7719-11e9-b60b-fc084a6691eb
sample3: 84afc00c-7719-11e9-87ab-fc084a6691eb
train_data: 851146d0-7719-11e9-ae8c-fc084a6691eb", "model version: 85269c2e-7719-11e9-a96b-fc084a6691eb
sample3: 84afc00c-7719-11e9-87ab-fc084a6691eb
train_data: 851146d0-7719-11e9-ae8c-fc084a6691eb", "model version: 85319506-7719-11e9-a547-fc084a6691eb
sample3: 84afc00c-7719-11e9-87ab-fc084a6691eb
train_data: 851146d0-7719-11e9-ae8c-fc084a6691eb", "model version: 853b55c2-7719-11e9-8f00-fc084a6691eb
sample3: 84afc00c-7719-11e9-87ab-fc084a6691eb
train_data: 851146d0-7719-11e9-ae8c-fc084a6691eb", "model version: 85451654-7719-11e9-b144-fc084a6691eb
sample3: 84afc00c-7719-11e9-87ab-fc084a6691eb
train_data: 851146d0-7719-11e9-ae8c-fc084a6691eb" ], "type": "scatter", "uid": "cdb58463-6e57-46b6-adb2-56c52d923fe4", "x": [ 6, 5, 7, 8, 9, 10, 11, 6, 7, 8, 9, 10, 11 ], "y": [ 157956.52173913043, 162247.82608695654, 151600, 91190.90909090912, 84437.5, 88473.91304347827, 90872.7272727273, 150757.89473684208, 120197.36842105264, 113531.25, 120450, 103223.07692307694, 88473.91304347827 ] } ], "layout": { "annotations": [ { "arrowhead": 2, "showarrow": true, "text": "prod", "x": 6, "xref": "x", "y": 407400, "yref": "y" }, { "arrowhead": 2, "showarrow": true, "text": "candidate", "x": 8, "xref": "x", "y": 401100, "yref": "y" }, { "arrowhead": 2, "showarrow": true, "text": "prod", "x": 6, "xref": "x", "y": 182100, "yref": "y" }, { "arrowhead": 2, "showarrow": true, "text": "candidate", "x": 8, "xref": "x", "y": 160883.3333333333, "yref": "y" }, { "arrowhead": 2, "showarrow": true, "text": "prod", "x": 6, "xref": "x", "y": 150757.89473684208, "yref": "y" }, { "arrowhead": 2, "showarrow": true, "text": "candidate", "x": 8, "xref": "x", "y": 113531.25, "yref": "y" } ], "title": { "text": "measure by parameter" }, "xaxis": { "title": { "text": "max_depth" } }, "yaxis": { "title": { "text": "max" } } } }, "text/html": [ "
\n", " \n", " \n", "
\n", " \n", "
" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/plain": [ "['DecisionTreeRegressor/measure/sample2/r2',\n", " 'DecisionTreeRegressor/measure/sample1/r2',\n", " 'DecisionTreeRegressor/measure/sample3/r2']" ] }, "execution_count": 33, "metadata": {}, "output_type": "execute_result" } ], "source": [ "import pailab.analysis.plot_helper as plt_helper\n", "import pailab.analysis.plot as plot\n", "plot.measure_by_parameter(ml_repo, ml_repo.tree.models.DecisionTreeRegressor.measures('max'), 'max_depth', data_versions=LAST_VERSION)\n", "ml_repo.tree.models.DecisionTreeRegressor.measures('r2')" ] }, { "cell_type": "code", "execution_count": 34, "metadata": { "code_folding": [] }, "outputs": [ { "data": { "application/vnd.jupyter.widget-view+json": { "model_id": "6ca0fa4cfc784ae0b8742ede4961f1dc", "version_major": 2, "version_minor": 0 }, "text/plain": [ "VBox(children=(HBox(children=(Button(description='plot', style=ButtonStyle()), SelectMultiple(options=('Decisi…" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "#\n", "from pailab.analysis.tools_jupyter import Plotter\n", "plotter = Plotter(ml_repo)\n", "display(plotter.get_widget())" ] }, { "cell_type": "code", "execution_count": 35, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "['DecisionTreeRegressor/model_param']" ] }, "execution_count": 35, "metadata": {}, "output_type": "execute_result" } ], "source": [ "ml_repo.get_names(MLObjectType.MODEL_PARAM)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Plot error vs input variable" ] }, { "cell_type": "code", "execution_count": 36, "metadata": { "scrolled": true }, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "linkText": "Export to plot.ly", "plotlyServerURL": "https://plot.ly", "responsive": true, "showLink": false }, "data": [ { "mode": "markers", "name": "prod, DecisionTreeRegressor/eval/sample1: 85158afe-7719-11e9-89dd-fc084a6691eb", "text": "sample1:last
DecisionTreeRegressor/model:8511e2da-7719-11e9-a641-fc084a6691eb
", "type": "scatter", "uid": "7355b73a-34a0-4d29-b8c8-26b2dfcea49e", "x": [ 15.3, 17.8, 17.8, 18.7, 18.7, 18.7, 15.2, 15.2, 15.2, 15.2, 15.2, 15.2, 15.2, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 19.2, 19.2, 19.2, 19.2, 18.3, 18.3, 17.9, 17.9, 17.9, 17.9, 17.9, 17.9, 17.9, 17.9, 17.9, 16.8, 16.8, 16.8, 16.8, 21.1, 17.9, 17.3, 15.1, 19.7, 19.7, 19.7, 19.7, 19.7, 19.7, 18.6, 16.1, 16.1, 18.9, 18.9, 18.9, 19.2, 19.2, 19.2, 19.2, 18.7, 18.7, 18.7, 18.7, 18.7, 18.7, 19, 19, 19, 19, 18.5, 18.5, 18.5, 18.5, 17.8, 17.8, 17.8, 17.8, 18.2, 18.2, 18.2, 18, 18, 18, 18, 18, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 16.6, 16.6, 16.6, 16.6, 16.6, 16.6, 16.6, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 15.2, 15.2, 15.2, 15.2, 15.2, 15.2, 15.6, 15.6, 12.6, 12.6, 12.6, 17, 17, 14.7, 14.7, 14.7, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 16.4, 16.4, 16.4, 16.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 16.6, 16.6, 16.6, 16.6, 16.6, 16.6, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 16.4, 16.4, 15.9, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 18.6, 18.6, 18.6, 18.6, 18.6, 17.6, 17.6, 17.6, 17.6, 17.6, 14.9, 14.9, 14.9, 14.9, 15.3, 15.3, 18.2, 16.6, 16.6, 16.6, 19.2, 19.2, 19.2, 16, 16, 16, 16, 16, 14.8, 14.8, 14.8, 16.1, 16.1, 16.1, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 16.9, 16.9, 16.9, 16.9, 16.9, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 15.5, 15.9, 17.6, 17.6, 18.8, 18.8, 17.9, 17, 19.7, 19.7, 18.3, 18.3, 17, 22, 22, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2 ], "y": [ -60738.4615384615, -42329.4117647059, 7665, -19635, 39165, 106770.5882352941, 36554.23728813557, 150757.89473684208, 0, -21442.105263157922, 0, -47445.76271186443, 11354.23728813557, -15711.111111111124, -38107.317073170736, -26211.111111111124, 40988.888888888876, -2672.727272727294, 3892.682926829264, -38107.317073170736, 1500, -8707.317073170736, -25725, 0, -17325, -13020, -21572.727272727294, -34125, -33907.317073170736, 0, -17400, -115807.31707317074, -6900, -29820, -600, -23407.317073170736, -307.31707317073597, -3111.111111111124, 98392.68292682926, -16100, 11865, -6138.461538461503, 35370.5882352941, 22770.588235294097, 1088.888888888876, -39045.76271186443, -24345.76271186443, -69742.10526315792, 0, -10942.105263157922, -30645.76271186443, -13611.111111111124, 29070.588235294097, 47288.888888888876, 26727.272727272706, 22365, 22770.588235294097, 7350, -6629.411764705903, -32511.111111111124, -27607.317073170736, -34172.727272727294, -29729.411764705903, -38010, -8400, -2429.411764705903, -36945.76271186443, 17888.888888888876, -54907.317073170736, -5211.111111111124, 12270.588235294097, 35392.682926829264, 34688.888888888876, -4529.411764705903, 10170.588235294097, -46529.4117647059, -24345.76271186443, -7545.7627118644305, 854.2372881355695, -17811.111111111124, 23261.538461538497, -61110, 24870.588235294097, -15029.411764705903, -76440, -6138.461538461503, 28154.23728813557, 22088.888888888876, -80325, 26775, -21329.411764705903, -33929.4117647059, -15029.411764705903, 29070.588235294097, -11745.76271186443, 31661.538461538497, 5054.2372881355695, 0, 0, -4200, 14490, -6510, -29707.317073170736, -15007.317073170736, 1792.682926829264, 64575, 64575, 8092.682926829264, -4507.317073170736, 37227.272727272706, 35392.682926829264, 0, -23542.105263157922, -25642.105263157922, -55845.76271186443, -60045.76271186443, 854.2372881355695, -41145.76271186443, -15945.76271186443, -39045.76271186443, 41692.682926829264, 5992.682926829264, 12157.894736842078, -55042.10526315792, -23542.105263157922, 79227.2727272727, -88642.10526315792, 35280, 7827.272727272706, -4620, -17107.317073170736, -8707.317073170736, 62692.682926829264, 16227.272727272706, -17325, 35175, 20475, -11072.727272727294, -4800, 28875, 9900, 18300, -22750, 23450, 0, -14350, 23450, -33600, 33600, 19250, 7154.2372881355695, -32745.76271186443, -8400, -36945.76271186443, -87345.76271186443, -116745.76271186443, -29050, 0, 66188.88888888888, -6629.411764705903, 71070.5882352941, 32354.23728813557, 80654.23728813557, 55454.23728813557, 55454.23728813557, 23954.23728813557, -78945.76271186443, -43245.76271186443, 40754.23728813557, -329.4117647059029, 30488.888888888876, 52661.5384615385, 42854.23728813557, 20670.588235294097, 10500, 60165, 0, 181860, 74865, 117761.5384615385, 110054.23728813557, 177254.23728813557, 25900, -20300, 11865, 55965, -5600, 43365, -3150, -10500, -21735, -65100, 25200, 11865, -30135, 10170.588235294097, -20300, -3150, 30254.23728813557, 68054.23728813557, 54157.89473684208, 68054.23728813557, 1657.8947368420777, 37357.89473684208, -13042.105263157922, 26054.23728813557, 11760, 79357.89473684208, -53340, 44954.23728813557, 39690, 33157.89473684208, 38654.23728813557, -15225, 37357.89473684208, 47775, 69090, -31500, 0, -37800, 8400, -1400, 65954.23728813557, -55335, 0, 42000, 45990, 59654.23728813557, -35910, -59535, 1770.588235294097, -18900, -67725, -22245.76271186443, 21854.23728813557, 1770.588235294097, -50707.317073170736, -29842.105263157922, 66188.88888888888, 10192.682926829264, -63840, -14538.461538461503, 16470.588235294097, 24870.588235294097, -99435, 0, 15788.888888888876, -5211.111111111124, 15400, 54600, 14700, 8400, 0, 3150, -50400, 65100, 34454.23728813557, 68775, 4900, -9645.76271186443, -1245.7627118644305, 33270.5882352941, 16470.588235294097, 0, 17500, 15750, -4200, 38850, 115170.5882352941, 0, -18900, 22365, 0, -25200, -33929.4117647059, -22245.76271186443, -8729.411764705903, -27629.411764705903, 12600, -57750, 62265, 21161.538461538497, 5970.588235294097, 11354.23728813557, 35861.5384615385, 60900, -18045.76271186443, -23429.411764705903, -112035, -8925, -46200, 58470.5882352941, -25935, 56700, 33390, 0, -25200, -85938.4615384615, -18045.76271186443, 8400, 19988.888888888876, -36945.76271186443, -42329.4117647059, -8400, -104145.76271186443, -44542.10526315792, -28545.76271186443, 40754.23728813557, -3345.7627118644305, 3870.588235294097, -10829.411764705903, -15711.111111111124, -31807.317073170736, 29070.588235294097, 20670.588235294097, -12929.411764705903, 45892.682926829264, -39045.76271186443, -21329.411764705903, -80129.4117647059, -85245.76271186443, -36711.111111111124, -29729.411764705903, -61229.4117647059, -1011.111111111124, -10807.317073170736, -31807.317073170736, -11511.111111111124, -21307.317073170736, -51411.111111111124, -14700, -149429.4117647059, -61110, -1050, -52807.317073170736, -59107.317073170736, -10829.411764705903, -50238.4615384615, 28875, -15029.411764705903, -58638.4615384615, -53511.111111111124, 10500, -61911.111111111124, -11511.111111111124, 28875, 35392.682926829264, 56392.682926829264, 54292.682926829264, 29070.588235294097, -2407.317073170736, 16492.682926829264, -17372.727272727294, 0, 39592.682926829264, 64792.682926829264, 96000, 96000, 0, 50505, 37905, 33705, -27195, 0, -12495, -4095, 16905, -9000, -42600, 26700, -38400, 20400, 105, 12180, 0, -37695, -80372.7272727273, -78225, -69825, 21105, -62895, -88800, -61500, -76200, -90195, 12705, -67095, -15300, -136395, 8505, 165592.68292682926, 119805, 0, -105307.31707317074, 119805, 182100, 100905, -46800, -42600 ] }, { "mode": "markers", "name": "candidate, DecisionTreeRegressor/eval/sample1: 852a446c-7719-11e9-adf7-fc084a6691eb", "text": "sample1:last
DecisionTreeRegressor/model:85269c2e-7719-11e9-a96b-fc084a6691eb
", "type": "scatter", "uid": "3ac0cedc-41e8-432b-b569-97f9eff59b6c", "x": [ 15.3, 17.8, 17.8, 18.7, 18.7, 18.7, 15.2, 15.2, 15.2, 15.2, 15.2, 15.2, 15.2, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 19.2, 19.2, 19.2, 19.2, 18.3, 18.3, 17.9, 17.9, 17.9, 17.9, 17.9, 17.9, 17.9, 17.9, 17.9, 16.8, 16.8, 16.8, 16.8, 21.1, 17.9, 17.3, 15.1, 19.7, 19.7, 19.7, 19.7, 19.7, 19.7, 18.6, 16.1, 16.1, 18.9, 18.9, 18.9, 19.2, 19.2, 19.2, 19.2, 18.7, 18.7, 18.7, 18.7, 18.7, 18.7, 19, 19, 19, 19, 18.5, 18.5, 18.5, 18.5, 17.8, 17.8, 17.8, 17.8, 18.2, 18.2, 18.2, 18, 18, 18, 18, 18, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 16.6, 16.6, 16.6, 16.6, 16.6, 16.6, 16.6, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 15.2, 15.2, 15.2, 15.2, 15.2, 15.2, 15.6, 15.6, 12.6, 12.6, 12.6, 17, 17, 14.7, 14.7, 14.7, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 16.4, 16.4, 16.4, 16.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 16.6, 16.6, 16.6, 16.6, 16.6, 16.6, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 16.4, 16.4, 15.9, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 18.6, 18.6, 18.6, 18.6, 18.6, 17.6, 17.6, 17.6, 17.6, 17.6, 14.9, 14.9, 14.9, 14.9, 15.3, 15.3, 18.2, 16.6, 16.6, 16.6, 19.2, 19.2, 19.2, 16, 16, 16, 16, 16, 14.8, 14.8, 14.8, 16.1, 16.1, 16.1, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 16.9, 16.9, 16.9, 16.9, 16.9, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 15.5, 15.9, 17.6, 17.6, 18.8, 18.8, 17.9, 17, 19.7, 19.7, 18.3, 18.3, 17, 22, 22, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2 ], "y": [ -52290, -19472.727272727294, -7875, -35175, 23625, 96492.30769230769, 20183.333333333314, 0, 0, -7905.882352941204, 0, -63816.666666666686, 21700, 2100, -35831.25, -8400, 58800, -10200, -36050, -35831.25, 7140, -6431.25, -24300, 0, -15900, -4200, -29100, -32700, -31631.25, 0, -11760, -113531.25, -1260, 0, 5040, -21131.25, 1968.75, 14700, 100668.75, 0, -3675, 2310, 25092.307692307688, 12492.307692307688, 7350, -20250, -5550, -56205.882352941204, 0, 2594.117647058796, -47016.666666666686, -7350, 18792.307692307688, 23100, 19200, 6825, 12492.307692307688, -13300, -16907.692307692312, -14700, -25331.25, 9800, -40007.69230769231, -15750, 15400, -12707.692307692312, -18150, 35700, 2100, 12600, 1992.3076923076878, 37668.75, -2800, -14807.692307692312, -107.69230769231217, -23672.727272727294, -40716.666666666686, -23916.666666666686, -15516.666666666686, 0, 31710, 0, 14592.307692307688, -25307.692307692312, -11900, 2310, 11783.333333333314, -15400, 0, 0, 1527.272727272706, -11072.727272727294, 7827.272727272706, 18792.307692307688, -28116.666666666686, 40110, -11316.666666666686, 0, 0, -24966.666666666628, -13300, 15750, -27431.25, -12731.25, 4068.75, 0, 0, 10368.75, -2231.25, 29700, 37668.75, 0, -10005.882352941204, -12105.882352941204, -72216.66666666669, 28000, -15516.666666666686, -57516.666666666686, -5600, -20250, 43968.75, 8268.75, 25694.117647058796, -41505.882352941204, -10005.882352941204, 0, -75105.8823529412, 11550, 300, 4200, -14831.25, -6431.25, 64968.75, 8700, -15900, 36600, 21900, -18600, 840, 30300, 0, 0, 0, 1400, 0, 0, 1400, 0, 0, -2800, -9216.666666666686, -13950, 0, -26600, 700, -28700, 0, 0, 28700, -16907.692307692312, 60792.30769230769, 51150, 64283.333333333314, 39083.333333333314, 39083.333333333314, 7583.333333333314, -60150, -59616.666666666686, 59550, 22527.272727272706, 6300, 0, 26483.333333333314, 10392.307692307688, 0, 44625, 0, 0, 59325, 0, 128850, 160883.3333333333, 0, 0, -3675, 40425, 0, 27825, -1050, 0, 0, 0, 4433.333333333372, -3675, -45675, -107.69230769231217, 0, 0, 13883.333333333314, 51683.333333333314, 67694.1176470588, 51683.333333333314, 15194.117647058796, 50894.117647058796, 494.11764705879614, 36400, 0, 0, 11200, 28583.333333333314, 11900, 46694.117647058796, 22283.333333333314, 0, 50894.117647058796, 9450, 41300, 0, 0, -14000, 13650, 0, 49583.333333333314, 2100, 0, 0, 18200, 43283.333333333314, -63700, -2100, -8507.692307692312, -14000, 0, -38616.666666666686, 5483.333333333314, -8507.692307692312, -90650, -16305.882352941204, 28700, 12468.75, 700, -6090, 6192.307692307688, 14592.307692307688, -114975, 0, -21700, -29400, 0, 33833.33333333337, 0, -12366.666666666628, 0, 0, 0, 44333.33333333337, 53250, 0, 0, -26016.666666666686, 17550, 22992.307692307688, 6192.307692307688, 0, 0, -4900, -24966.666666666628, 18200, 104892.30769230769, 0, -13650, 6825, 0, -1400, -11072.727272727294, -38616.666666666686, -19007.692307692312, -37907.69230769231, 17500, 0, 46725, 29610, 28827.272727272706, -5016.666666666686, 44310, 0, -7700, -33707.69230769231, 0, 0, 0, 81327.2727272727, -41475, 35933.33333333337, 5600, -20766.666666666628, 0, 0, -34416.666666666686, 0, -17500, -53316.666666666686, -19472.727272727294, -3500, -85350, -31005.882352941204, -18200, 24383.333333333314, -19716.666666666686, -6407.692307692312, -21107.692307692312, 2100, -29531.25, 18792.307692307688, 10392.307692307688, -23207.692307692312, 48168.75, -20250, -31607.692307692312, -57272.727272727294, -66450, -18900, -40007.69230769231, -71507.69230769231, 16800, -8531.25, -29531.25, 6300, -19031.25, -33600, -35466.66666666663, 0, 0, 1050, 0, -2100, -21107.692307692312, -41790, -9450, -25307.692307692312, -50190, -35700, 0, -44100, 6300, 0, 37668.75, 58668.75, 56568.75, 18792.307692307688, -131.25, -23450, 26600, 0, -350, 24850, 14175, 14175, 0, 28087.5, 15487.5, 11287.5, 0, 0, -34912.5, 23100, 44100, -21700, 5600, 14000, 9800, 7700, 27300, -11550, 0, -10500, -36400, -4200, 4200, -1312.5, -35700, -40600, -13300, 0, -63000, -9712.5, -39900, 32900, -109200, 35700, 125650, 0, 0, -103031.25, 97387.5, 100275, 128100, -128625, 5600 ] }, { "mode": "markers", "name": "DecisionTreeRegressor/eval/sample1: 8547866f-7719-11e9-8333-fc084a6691eb", "text": "sample1:last
DecisionTreeRegressor/model:85451654-7719-11e9-b144-fc084a6691eb
", "type": "scatter", "uid": "4ad4ff5b-8bb6-4870-89c1-d2eb88741ff5", "x": [ 15.3, 17.8, 17.8, 18.7, 18.7, 18.7, 15.2, 15.2, 15.2, 15.2, 15.2, 15.2, 15.2, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 19.2, 19.2, 19.2, 19.2, 18.3, 18.3, 17.9, 17.9, 17.9, 17.9, 17.9, 17.9, 17.9, 17.9, 17.9, 16.8, 16.8, 16.8, 16.8, 21.1, 17.9, 17.3, 15.1, 19.7, 19.7, 19.7, 19.7, 19.7, 19.7, 18.6, 16.1, 16.1, 18.9, 18.9, 18.9, 19.2, 19.2, 19.2, 19.2, 18.7, 18.7, 18.7, 18.7, 18.7, 18.7, 19, 19, 19, 19, 18.5, 18.5, 18.5, 18.5, 17.8, 17.8, 17.8, 17.8, 18.2, 18.2, 18.2, 18, 18, 18, 18, 18, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 16.6, 16.6, 16.6, 16.6, 16.6, 16.6, 16.6, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 15.2, 15.2, 15.2, 15.2, 15.2, 15.2, 15.6, 15.6, 12.6, 12.6, 12.6, 17, 17, 14.7, 14.7, 14.7, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 16.4, 16.4, 16.4, 16.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 16.6, 16.6, 16.6, 16.6, 16.6, 16.6, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 16.4, 16.4, 15.9, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 18.6, 18.6, 18.6, 18.6, 18.6, 17.6, 17.6, 17.6, 17.6, 17.6, 14.9, 14.9, 14.9, 14.9, 15.3, 15.3, 18.2, 16.6, 16.6, 16.6, 19.2, 19.2, 19.2, 16, 16, 16, 16, 16, 14.8, 14.8, 14.8, 16.1, 16.1, 16.1, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 16.9, 16.9, 16.9, 16.9, 16.9, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 15.5, 15.9, 17.6, 17.6, 18.8, 18.8, 17.9, 17, 19.7, 19.7, 18.3, 18.3, 17, 22, 22, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2 ], "y": [ 0, 0, -2100, -14700, 0, 88473.91304347827, 24245.45454545453, 0, 0, -24966.666666666686, 0, -59754.54545454547, 0, -954.5454545454704, 16100, -11454.54545454547, 0, 4200, -6300, 0, 0, 45500, -4200, 0, 0, 0, 0, 0, -33810, 0, 0, -61600, 0, 0, 0, -4900, -29400, 11645.45454545453, 69300, 0, 16800, -14350, 17073.91304347827, 18060, 0, 0, -6300, -38640, 0, -14466.666666666686, -42954.54545454547, 0, 10773.913043478271, 0, 9800, 0, 4473.913043478271, 0, 10500, -17754.54545454547, 2100, 0, -12600, 0, 0, -20726.08695652173, -18900, 32645.45454545453, 0, 9545.45454545453, -6026.086956521729, 6300, 0, -9240, -8126.086956521729, -2100, -9975, -19854.54545454547, 15225, -3054.5454545454704, 15050, 0, 6573.913043478271, -19740, 0, -14350, 15845.45454545453, 2800, 0, 0, -4900, -17500, 1400, 10773.913043478271, 2625, 23450, -56000, 0, 0, 0, 0, 0, -29610, -14910, 1890, 0, 0, 8190, -4410, 0, 35490, 0, -27066.666666666686, -29166.666666666686, 0, 0, -60200, -53454.54545454547, 0, -21000, 12600, -23100, 8633.333333333314, 0, -27066.666666666686, 0, 0, 0, -9100, 0, -17010, -8610, 62790, -700, 4200, 0, 0, -4200, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, 0, -5154.54545454547, 0, 0, 0, 0, 0, 0, 0, 0, 4725, 52773.91304347827, 50400, 68345.45454545453, 43145.45454545453, 0, 1050, 0, -55554.54545454547, 3150, 16100, 0, 0, 30545.45454545453, 2373.913043478271, 0, 2100, 0, 0, 14700, 0, 0, 116200, 0, 0, 2100, -2100, 0, -16800, 0, 0, 0, 0, 0, 16800, -25200, 21525, 0, 0, 17945.45454545453, 0, 50633.333333333314, 55745.45454545453, 32760, 33833.333333333314, 18060, 0, 0, 0, 0, 32645.45454545453, 0, 29633.333333333314, -1050, 0, 0, 0, 0, 0, 0, 0, 0, 0, 53645.45454545453, 0, 0, 0, 0, 47345.45454545453, 0, 0, -16526.08695652173, 0, 0, -34554.54545454547, -1050, -16526.08695652173, 0, 1260, 0, -18900, 0, -22750, -1826.086956521729, 6573.913043478271, 0, 0, -3500, 0, 0, -4200, 0, 0, 0, 0, 0, 6300, -3150, 0, 0, -21954.54545454547, 16800, 14973.913043478271, 11760, 0, 0, 0, 0, 0, 0, 0, 0, 27300, 0, 0, -17500, -7875, -27026.08695652173, -16275, 0, 0, 2100, 12950, 22400, -954.5454545454704, 0, 0, 0, -41726.08695652173, 0, 0, 0, 0, -21000, -2100, 0, 0, 0, 0, -30354.54545454547, 0, 700, -49254.54545454547, 2100, 0, -9450, -13440, 0, 1050, -15654.54545454547, -840, -29126.08695652173, -954.5454545454704, -2100, 10773.913043478271, 2373.913043478271, -31226.08695652173, 16800, -21000, -9975, 0, 9450, 8400, 15750, -15750, 13745.45454545453, 7700, -60900, 3245.4545454545296, -2800, -36654.54545454547, 0, 0, 0, 0, 0, 0, -29126.08695652173, 0, 0, 2100, 0, -8400, 0, 0, 0, 0, 6300, 27300, 25200, 0, -31500, 6300, 0, 0, -12600, 12600, 0, 0, 0, 6300, -6300, 11200, 0, 0, 0, -840, 20160, 0, 0, 0, 0, 0, 3360, 0, 0, -34440, 0, 0, 0, -1400, 13650, 0, -9450, 0, -13650, -9800, 0, 0, 0, 11760, 0, 0, 0, 0, 0, 0, 0, 0, 9450 ] }, { "mode": "markers", "name": "prod, DecisionTreeRegressor/eval/sample2: 85131af6-7719-11e9-a648-fc084a6691eb", "text": "sample2:last
DecisionTreeRegressor/model:8511e2da-7719-11e9-a641-fc084a6691eb
", "type": "scatter", "uid": "56ddbaf9-e807-4f86-b66c-f9fe4b01a073", "x": [ 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 16.9, 16.9, 16.9, 16.9, 16.9, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 15.5, 15.9, 17.6, 17.6, 18.8, 18.8, 17.9, 17, 19.7, 19.7, 18.3, 18.3, 17, 22, 22, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.1, 20.1, 20.1, 20.1, 20.1, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 21, 21, 21, 21 ], "y": [ -36945.76271186443, -42329.4117647059, -8400, -104145.76271186443, -44542.10526315792, -28545.76271186443, 40754.23728813557, -3345.7627118644305, 3870.588235294097, -10829.411764705903, -15711.111111111124, -31807.317073170736, 29070.588235294097, 20670.588235294097, -12929.411764705903, 45892.682926829264, -39045.76271186443, -21329.411764705903, -80129.4117647059, -85245.76271186443, -36711.111111111124, -29729.411764705903, -61229.4117647059, -1011.111111111124, -10807.317073170736, -31807.317073170736, -11511.111111111124, -21307.317073170736, -51411.111111111124, -14700, -149429.4117647059, -61110, -1050, -52807.317073170736, -59107.317073170736, -10829.411764705903, -50238.4615384615, 28875, -15029.411764705903, -58638.4615384615, -53511.111111111124, 10500, -61911.111111111124, -11511.111111111124, 28875, 35392.682926829264, 56392.682926829264, 54292.682926829264, 29070.588235294097, -2407.317073170736, 16492.682926829264, -17372.727272727294, 0, 39592.682926829264, 64792.682926829264, 96000, 96000, 0, 50505, 37905, 33705, -27195, 0, -12495, -4095, 16905, -9000, -42600, 26700, -38400, 20400, 105, 12180, 0, -37695, -80372.7272727273, -78225, -69825, 21105, -62895, -88800, -61500, -76200, -90195, 12705, -67095, -15300, -136395, 8505, 165592.68292682926, 119805, 0, -105307.31707317074, 119805, 182100, 100905, -46800, -42600, -83895, 24600, -56595, -401100, -19472.727272727294, -71972.7272727273, 16492.682926829264, 40005, -59220, -67095, -155972.7272727273, -141272.7272727273, -10395, -41895, -40425, -281400, -82207.31707317074, -44625, -124472.7272727273, 40005, -143325, -58695, -17400, 27405, -20895, 117705, 41475, -163800, -14595, 6405, -32025, -80325, -48825, 31605, -63525, -25725, -6825, 58800, -32025, -48825, -220500, -61425, -32025, 49827.272727272706, -525, 1527.272727272706, -10807.317073170736, 3892.682926829264, 29092.682926829264, -2407.317073170736, 54075, 159705, 56175, 51927.272727272706, 72975, -8707.317073170736, 66892.68292682926, 407400, -15120, 37905, 5775, 10605, -38325, 29092.682926829264, 62692.682926829264, -65310, 306600, 37492.682926829264, 12292.682926829264, 24892.682926829264, 30927.272727272706, 12292.682926829264, 14280, -94395, -23700, -59325, 1792.682926829264, 37492.682926829264, 94192.68292682926, 66757.89473684208, -4642.105263157922, -36007.317073170736, 24892.682926829264, -2672.727272727294, -67507.31707317074, -37800, -11511.111111111124, 283500, -102738.4615384615 ] }, { "mode": "markers", "name": "candidate, DecisionTreeRegressor/eval/sample2: 85287054-7719-11e9-b73c-fc084a6691eb", "text": "sample2:last
DecisionTreeRegressor/model:85269c2e-7719-11e9-a96b-fc084a6691eb
", "type": "scatter", "uid": "aeb98a28-a02d-4192-9b52-5145405b6d7d", "x": [ 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 16.9, 16.9, 16.9, 16.9, 16.9, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 15.5, 15.9, 17.6, 17.6, 18.8, 18.8, 17.9, 17, 19.7, 19.7, 18.3, 18.3, 17, 22, 22, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.1, 20.1, 20.1, 20.1, 20.1, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 21, 21, 21, 21 ], "y": [ -53316.666666666686, -19472.727272727294, -3500, -85350, -31005.882352941204, -18200, 24383.333333333314, -19716.666666666686, -6407.692307692312, -21107.692307692312, 2100, -29531.25, 18792.307692307688, 10392.307692307688, -23207.692307692312, 48168.75, -20250, -31607.692307692312, -57272.727272727294, -66450, -18900, -40007.69230769231, -71507.69230769231, 16800, -8531.25, -29531.25, 6300, -19031.25, -33600, -35466.66666666663, 0, 0, 1050, 0, -2100, -21107.692307692312, -41790, -9450, -25307.692307692312, -50190, -35700, 0, -44100, 6300, 0, 37668.75, 58668.75, 56568.75, 18792.307692307688, -131.25, -23450, 26600, 0, -350, 24850, 14175, 14175, 0, 28087.5, 15487.5, 11287.5, 0, 0, -34912.5, 23100, 44100, -21700, 5600, 14000, 9800, 7700, 27300, -11550, 0, -10500, -36400, -4200, 4200, -1312.5, -35700, -40600, -13300, 0, -63000, -9712.5, -39900, 32900, -109200, 35700, 125650, 0, 0, -103031.25, 97387.5, 100275, 128100, -128625, 5600, -106312.5, 100800, -29400, -401100, 24500, -28000, -23450, 67200, -50400, -39900, -112000, -97300, 16800, -64312.5, -69300, -281400, -79931.25, 29400, -80500, 17587.5, -172200, -178500, 30800, 54600, 6300, 95287.5, 115500, -163800, 12600, -16012.5, -60900, -6300, -77700, 9187.5, -92400, -54600, 67200, 155400, -60900, -77700, -220500, 12600, 42000, 93800, 73500, 45500, -8531.25, 6168.75, 31368.75, -131.25, 128100, 186900, 27300, 95900, 147000, -6431.25, 69168.75, 310800, -6300, 65100, -23100, 37800, -67200, 31368.75, 64968.75, -43050, 210000, 39768.75, 14568.75, 27168.75, 74900, 14568.75, 23100, -67200, 24500, -88200, 4068.75, 39768.75, 96468.75, 80294.1176470588, 8894.117647058796, -33731.25, 27168.75, -81900, -65231.25, -32900, -49000, 186900, -94290 ] }, { "mode": "markers", "name": "DecisionTreeRegressor/eval/sample2: 85464e65-7719-11e9-9ada-fc084a6691eb", "text": "sample2:last
DecisionTreeRegressor/model:85451654-7719-11e9-b144-fc084a6691eb
", "type": "scatter", "uid": "a63a2e1e-939e-4076-8017-afa39209a886", "x": [ 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 16.9, 16.9, 16.9, 16.9, 16.9, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 15.5, 15.9, 17.6, 17.6, 18.8, 18.8, 17.9, 17, 19.7, 19.7, 18.3, 18.3, 17, 22, 22, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.1, 20.1, 20.1, 20.1, 20.1, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 21, 21, 21, 21 ], "y": [ -49254.54545454547, 2100, 0, -9450, -13440, 0, 1050, -15654.54545454547, -840, -29126.08695652173, -954.5454545454704, -2100, 10773.913043478271, 2373.913043478271, -31226.08695652173, 16800, -21000, -9975, 0, 9450, 8400, 15750, -15750, 13745.45454545453, 7700, -60900, 3245.4545454545296, -2800, -36654.54545454547, 0, 0, 0, 0, 0, 0, -29126.08695652173, 0, 0, 2100, 0, -8400, 0, 0, 0, 0, 6300, 27300, 25200, 0, -31500, 6300, 0, 0, -12600, 12600, 0, 0, 0, 6300, -6300, 11200, 0, 0, 0, -840, 20160, 0, 0, 0, 0, 0, 3360, 0, 0, -34440, 0, 0, 0, -1400, 13650, 0, -9450, 0, -13650, -9800, 0, 0, 0, 11760, 0, 0, 0, 0, 0, 0, 0, 0, 9450, 0, 100800, 19950, -401100, 60900, 8400, -149100, 176400, -46200, 69300, -75600, -107100, 16800, -64400, -69300, -281400, -111300, 33600, -44100, -4200, -172200, -178500, 21000, 163800, 6300, 95200, 119700, -163800, 121800, -16100, -60900, -2100, -77700, 9100, -92400, -54600, 63000, 155400, -60900, -77700, -220500, 8400, 46200, 67200, 77700, 18900, -39900, -25200, 0, 27300, 123900, 186900, 27300, 69300, 151200, -37800, 37800, 310800, -2100, 174300, -23100, 13860, -67200, 0, 33600, -27300, 210000, 142800, -16800, -4200, 111300, -16800, 27300, -91140, 65100, -88200, -27300, 67200, 65100, 63233.333333333314, 26460, -6300, -4200, -81900, -96600, -29400, -77700, 186900, -110950 ] } ], "layout": { "title": { "text": "pointwise error (MEDV)" }, "xaxis": { "title": { "text": "PTRATIO" } }, "yaxis": { "title": { "text": "model-target [MEDV]" } } } }, "text/html": [ "
\n", " \n", " \n", "
\n", " \n", "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "import pailab.analysis.plot as plot\n", "plot.scatter_model_error(ml_repo, ml_repo.tree.models.DecisionTreeRegressor.model(), ['sample1', ml_repo.tree.test_data.sample2()], 'PTRATIO')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Plot histogram of model error" ] }, { "cell_type": "code", "execution_count": 37, "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "linkText": "Export to plot.ly", "plotlyServerURL": "https://plot.ly", "responsive": true, "showLink": false }, "data": [ { "histnorm": "percent", "name": "prod, DecisionTreeRegressor/eval/sample2: 85131af6-7719-11e9-a648-fc084a6691eb", "opacity": 0.5, "text": "sample2:last
DecisionTreeRegressor/model:8511e2da-7719-11e9-a641-fc084a6691eb
", "type": "histogram", "uid": "c1e28e47-7a94-4858-8bb0-a6a9f0a84d75", "x": [ -36945.76271186443, -42329.4117647059, -8400, -104145.76271186443, -44542.10526315792, -28545.76271186443, 40754.23728813557, -3345.7627118644305, 3870.588235294097, -10829.411764705903, -15711.111111111124, -31807.317073170736, 29070.588235294097, 20670.588235294097, -12929.411764705903, 45892.682926829264, -39045.76271186443, -21329.411764705903, -80129.4117647059, -85245.76271186443, -36711.111111111124, -29729.411764705903, -61229.4117647059, -1011.111111111124, -10807.317073170736, -31807.317073170736, -11511.111111111124, -21307.317073170736, -51411.111111111124, -14700, -149429.4117647059, -61110, -1050, -52807.317073170736, -59107.317073170736, -10829.411764705903, -50238.4615384615, 28875, -15029.411764705903, -58638.4615384615, -53511.111111111124, 10500, -61911.111111111124, -11511.111111111124, 28875, 35392.682926829264, 56392.682926829264, 54292.682926829264, 29070.588235294097, -2407.317073170736, 16492.682926829264, -17372.727272727294, 0, 39592.682926829264, 64792.682926829264, 96000, 96000, 0, 50505, 37905, 33705, -27195, 0, -12495, -4095, 16905, -9000, -42600, 26700, -38400, 20400, 105, 12180, 0, -37695, -80372.7272727273, -78225, -69825, 21105, -62895, -88800, -61500, -76200, -90195, 12705, -67095, -15300, -136395, 8505, 165592.68292682926, 119805, 0, -105307.31707317074, 119805, 182100, 100905, -46800, -42600, -83895, 24600, -56595, -401100, -19472.727272727294, -71972.7272727273, 16492.682926829264, 40005, -59220, -67095, -155972.7272727273, -141272.7272727273, -10395, -41895, -40425, -281400, -82207.31707317074, -44625, -124472.7272727273, 40005, -143325, -58695, -17400, 27405, -20895, 117705, 41475, -163800, -14595, 6405, -32025, -80325, -48825, 31605, -63525, -25725, -6825, 58800, -32025, -48825, -220500, -61425, -32025, 49827.272727272706, -525, 1527.272727272706, -10807.317073170736, 3892.682926829264, 29092.682926829264, -2407.317073170736, 54075, 159705, 56175, 51927.272727272706, 72975, -8707.317073170736, 66892.68292682926, 407400, -15120, 37905, 5775, 10605, -38325, 29092.682926829264, 62692.682926829264, -65310, 306600, 37492.682926829264, 12292.682926829264, 24892.682926829264, 30927.272727272706, 12292.682926829264, 14280, -94395, -23700, -59325, 1792.682926829264, 37492.682926829264, 94192.68292682926, 66757.89473684208, -4642.105263157922, -36007.317073170736, 24892.682926829264, -2672.727272727294, -67507.31707317074, -37800, -11511.111111111124, 283500, -102738.4615384615 ] }, { "histnorm": "percent", "name": "candidate, DecisionTreeRegressor/eval/sample2: 85287054-7719-11e9-b73c-fc084a6691eb", "opacity": 0.5, "text": "sample2:last
DecisionTreeRegressor/model:85269c2e-7719-11e9-a96b-fc084a6691eb
", "type": "histogram", "uid": "51b45df6-a68d-406f-b897-6e7364907075", "x": [ -53316.666666666686, -19472.727272727294, -3500, -85350, -31005.882352941204, -18200, 24383.333333333314, -19716.666666666686, -6407.692307692312, -21107.692307692312, 2100, -29531.25, 18792.307692307688, 10392.307692307688, -23207.692307692312, 48168.75, -20250, -31607.692307692312, -57272.727272727294, -66450, -18900, -40007.69230769231, -71507.69230769231, 16800, -8531.25, -29531.25, 6300, -19031.25, -33600, -35466.66666666663, 0, 0, 1050, 0, -2100, -21107.692307692312, -41790, -9450, -25307.692307692312, -50190, -35700, 0, -44100, 6300, 0, 37668.75, 58668.75, 56568.75, 18792.307692307688, -131.25, -23450, 26600, 0, -350, 24850, 14175, 14175, 0, 28087.5, 15487.5, 11287.5, 0, 0, -34912.5, 23100, 44100, -21700, 5600, 14000, 9800, 7700, 27300, -11550, 0, -10500, -36400, -4200, 4200, -1312.5, -35700, -40600, -13300, 0, -63000, -9712.5, -39900, 32900, -109200, 35700, 125650, 0, 0, -103031.25, 97387.5, 100275, 128100, -128625, 5600, -106312.5, 100800, -29400, -401100, 24500, -28000, -23450, 67200, -50400, -39900, -112000, -97300, 16800, -64312.5, -69300, -281400, -79931.25, 29400, -80500, 17587.5, -172200, -178500, 30800, 54600, 6300, 95287.5, 115500, -163800, 12600, -16012.5, -60900, -6300, -77700, 9187.5, -92400, -54600, 67200, 155400, -60900, -77700, -220500, 12600, 42000, 93800, 73500, 45500, -8531.25, 6168.75, 31368.75, -131.25, 128100, 186900, 27300, 95900, 147000, -6431.25, 69168.75, 310800, -6300, 65100, -23100, 37800, -67200, 31368.75, 64968.75, -43050, 210000, 39768.75, 14568.75, 27168.75, 74900, 14568.75, 23100, -67200, 24500, -88200, 4068.75, 39768.75, 96468.75, 80294.1176470588, 8894.117647058796, -33731.25, 27168.75, -81900, -65231.25, -32900, -49000, 186900, -94290 ] }, { "histnorm": "percent", "name": "DecisionTreeRegressor/eval/sample2: 85464e65-7719-11e9-9ada-fc084a6691eb", "opacity": 0.5, "text": "sample2:last
DecisionTreeRegressor/model:85451654-7719-11e9-b144-fc084a6691eb
", "type": "histogram", "uid": "62e01220-58bb-458b-8bf1-a42b6d73cf2c", "x": [ -49254.54545454547, 2100, 0, -9450, -13440, 0, 1050, -15654.54545454547, -840, -29126.08695652173, -954.5454545454704, -2100, 10773.913043478271, 2373.913043478271, -31226.08695652173, 16800, -21000, -9975, 0, 9450, 8400, 15750, -15750, 13745.45454545453, 7700, -60900, 3245.4545454545296, -2800, -36654.54545454547, 0, 0, 0, 0, 0, 0, -29126.08695652173, 0, 0, 2100, 0, -8400, 0, 0, 0, 0, 6300, 27300, 25200, 0, -31500, 6300, 0, 0, -12600, 12600, 0, 0, 0, 6300, -6300, 11200, 0, 0, 0, -840, 20160, 0, 0, 0, 0, 0, 3360, 0, 0, -34440, 0, 0, 0, -1400, 13650, 0, -9450, 0, -13650, -9800, 0, 0, 0, 11760, 0, 0, 0, 0, 0, 0, 0, 0, 9450, 0, 100800, 19950, -401100, 60900, 8400, -149100, 176400, -46200, 69300, -75600, -107100, 16800, -64400, -69300, -281400, -111300, 33600, -44100, -4200, -172200, -178500, 21000, 163800, 6300, 95200, 119700, -163800, 121800, -16100, -60900, -2100, -77700, 9100, -92400, -54600, 63000, 155400, -60900, -77700, -220500, 8400, 46200, 67200, 77700, 18900, -39900, -25200, 0, 27300, 123900, 186900, 27300, 69300, 151200, -37800, 37800, 310800, -2100, 174300, -23100, 13860, -67200, 0, 33600, -27300, 210000, 142800, -16800, -4200, 111300, -16800, 27300, -91140, 65100, -88200, -27300, 67200, 65100, 63233.333333333314, 26460, -6300, -4200, -81900, -96600, -29400, -77700, 186900, -110950 ] } ], "layout": { "barmode": "overlay", "title": { "text": "pointwise error (MEDV)" }, "xaxis": { "title": { "text": "model-target [MEDV]" } } } }, "text/html": [ "
\n", " \n", " \n", "
\n", " \n", "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot.histogram_model_error(ml_repo, ml_repo.tree.models.DecisionTreeRegressor.model(), ml_repo.tree.test_data.sample2())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Plot data" ] }, { "cell_type": "code", "execution_count": 38, "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "linkText": "Export to plot.ly", "plotlyServerURL": "https://plot.ly", "responsive": true, "showLink": false }, "data": [ { "histnorm": "percent", "name": "sample2: 84c0d13a-7719-11e9-92a0-fc084a6691eb", "opacity": 0.5, "text": "", "type": "histogram", "uid": "664a4b78-7265-4cd9-8792-8d948a0b9289", "x": [ 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 16.9, 16.9, 16.9, 16.9, 16.9, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 15.5, 15.9, 17.6, 17.6, 18.8, 18.8, 17.9, 17, 19.7, 19.7, 18.3, 18.3, 17, 22, 22, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.1, 20.1, 20.1, 20.1, 20.1, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 21, 21, 21, 21 ] }, { "histnorm": "percent", "name": "sample1: 83d7f948-7719-11e9-bad3-fc084a6691eb", "opacity": 0.5, "text": "", "type": "histogram", "uid": "d11b03ef-0eb1-41f9-a226-f0303d04ecc6", "x": [ 15.3, 17.8, 17.8, 18.7, 18.7, 18.7, 15.2, 15.2, 15.2, 15.2, 15.2, 15.2, 15.2, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 19.2, 19.2, 19.2, 19.2, 18.3, 18.3, 17.9, 17.9, 17.9, 17.9, 17.9, 17.9, 17.9, 17.9, 17.9, 16.8, 16.8, 16.8, 16.8, 21.1, 17.9, 17.3, 15.1, 19.7, 19.7, 19.7, 19.7, 19.7, 19.7, 18.6, 16.1, 16.1, 18.9, 18.9, 18.9, 19.2, 19.2, 19.2, 19.2, 18.7, 18.7, 18.7, 18.7, 18.7, 18.7, 19, 19, 19, 19, 18.5, 18.5, 18.5, 18.5, 17.8, 17.8, 17.8, 17.8, 18.2, 18.2, 18.2, 18, 18, 18, 18, 18, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 16.6, 16.6, 16.6, 16.6, 16.6, 16.6, 16.6, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 15.2, 15.2, 15.2, 15.2, 15.2, 15.2, 15.6, 15.6, 12.6, 12.6, 12.6, 17, 17, 14.7, 14.7, 14.7, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 16.4, 16.4, 16.4, 16.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 16.6, 16.6, 16.6, 16.6, 16.6, 16.6, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 16.4, 16.4, 15.9, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 18.6, 18.6, 18.6, 18.6, 18.6, 17.6, 17.6, 17.6, 17.6, 17.6, 14.9, 14.9, 14.9, 14.9, 15.3, 15.3, 18.2, 16.6, 16.6, 16.6, 19.2, 19.2, 19.2, 16, 16, 16, 16, 16, 14.8, 14.8, 14.8, 16.1, 16.1, 16.1, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4 ] }, { "histnorm": "percent", "name": "sample1: 851146d0-7719-11e9-ae8c-fc084a6691eb", "opacity": 0.5, "text": "", "type": "histogram", "uid": "a1381846-fad7-43e0-a6ce-b1699b05697c", "x": [ 15.3, 17.8, 17.8, 18.7, 18.7, 18.7, 15.2, 15.2, 15.2, 15.2, 15.2, 15.2, 15.2, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 21, 19.2, 19.2, 19.2, 19.2, 18.3, 18.3, 17.9, 17.9, 17.9, 17.9, 17.9, 17.9, 17.9, 17.9, 17.9, 16.8, 16.8, 16.8, 16.8, 21.1, 17.9, 17.3, 15.1, 19.7, 19.7, 19.7, 19.7, 19.7, 19.7, 18.6, 16.1, 16.1, 18.9, 18.9, 18.9, 19.2, 19.2, 19.2, 19.2, 18.7, 18.7, 18.7, 18.7, 18.7, 18.7, 19, 19, 19, 19, 18.5, 18.5, 18.5, 18.5, 17.8, 17.8, 17.8, 17.8, 18.2, 18.2, 18.2, 18, 18, 18, 18, 18, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 20.9, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 21.2, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 14.7, 16.6, 16.6, 16.6, 16.6, 16.6, 16.6, 16.6, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 17.8, 15.2, 15.2, 15.2, 15.2, 15.2, 15.2, 15.6, 15.6, 12.6, 12.6, 12.6, 17, 17, 14.7, 14.7, 14.7, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 18.6, 16.4, 16.4, 16.4, 16.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 17.4, 16.6, 16.6, 16.6, 16.6, 16.6, 16.6, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 19.1, 16.4, 16.4, 15.9, 13, 13, 13, 13, 13, 13, 13, 13, 13, 13, 18.6, 18.6, 18.6, 18.6, 18.6, 17.6, 17.6, 17.6, 17.6, 17.6, 14.9, 14.9, 14.9, 14.9, 15.3, 15.3, 18.2, 16.6, 16.6, 16.6, 19.2, 19.2, 19.2, 16, 16, 16, 16, 16, 14.8, 14.8, 14.8, 16.1, 16.1, 16.1, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 16.9, 16.9, 16.9, 16.9, 16.9, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 15.5, 15.9, 17.6, 17.6, 18.8, 18.8, 17.9, 17, 19.7, 19.7, 18.3, 18.3, 17, 22, 22, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2 ] } ], "layout": { "barmode": "overlay", "title": { "text": "data distribution" }, "xaxis": { "title": { "text": "PTRATIO" } } } }, "text/html": [ "
\n", " \n", " \n", "
\n", " \n", "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot.histogram_data(ml_repo, {ml_repo.tree.test_data.sample2() :['last'], ml_repo.tree.training_data.sample1(): ['first','last']}, x_coordinate = 'PTRATIO') #, y_coordinate='MEDV')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Plot projections" ] }, { "cell_type": "code", "execution_count": 39, "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "linkText": "Export to plot.ly", "plotlyServerURL": "https://plot.ly", "responsive": true, "showLink": false }, "data": [ { "name": "prod", "type": "scatter", "uid": "1a06128b-9818-47a4-bd16-e448e98f532a", "x": [ 0, 0.010101010101010102, 0.020202020202020204, 0.030303030303030304, 0.04040404040404041, 0.050505050505050504, 0.06060606060606061, 0.0707070707070707, 0.08080808080808081, 0.09090909090909091, 0.10101010101010101, 0.1111111111111111, 0.12121212121212122, 0.13131313131313133, 0.1414141414141414, 0.15151515151515152, 0.16161616161616163, 0.1717171717171717, 0.18181818181818182, 0.1919191919191919, 0.20202020202020202, 0.21212121212121213, 0.2222222222222222, 0.23232323232323232, 0.24242424242424243, 0.25252525252525254, 0.26262626262626265, 0.2727272727272727, 0.2828282828282828, 0.29292929292929293, 0.30303030303030304, 0.31313131313131315, 0.32323232323232326, 0.3333333333333333, 0.3434343434343434, 0.35353535353535354, 0.36363636363636365, 0.37373737373737376, 0.3838383838383838, 0.3939393939393939, 0.40404040404040403, 0.41414141414141414, 0.42424242424242425, 0.43434343434343436, 0.4444444444444444, 0.45454545454545453, 0.46464646464646464, 0.47474747474747475, 0.48484848484848486, 0.494949494949495, 0.5050505050505051, 0.5151515151515151, 0.5252525252525253, 0.5353535353535354, 0.5454545454545454, 0.5555555555555556, 0.5656565656565656, 0.5757575757575758, 0.5858585858585859, 0.5959595959595959, 0.6060606060606061, 0.6161616161616161, 0.6262626262626263, 0.6363636363636364, 0.6464646464646465, 0.6565656565656566, 0.6666666666666666, 0.6767676767676768, 0.6868686868686869, 0.696969696969697, 0.7070707070707071, 0.7171717171717171, 0.7272727272727273, 0.7373737373737373, 0.7474747474747475, 0.7575757575757576, 0.7676767676767676, 0.7777777777777778, 0.7878787878787878, 0.797979797979798, 0.8080808080808081, 0.8181818181818182, 0.8282828282828283, 0.8383838383838383, 0.8484848484848485, 0.8585858585858586, 0.8686868686868687, 0.8787878787878788, 0.8888888888888888, 0.898989898989899, 0.9090909090909091, 0.9191919191919192, 0.9292929292929293, 0.9393939393939394, 0.9494949494949495, 0.9595959595959596, 0.9696969696969697, 0.9797979797979798, 0.98989898989899, 1 ], "y": [ 564738.4615384615, 564738.4615384615, 564738.4615384615, 564738.4615384615, 564738.4615384615, 564738.4615384615, 564738.4615384615, 564738.4615384615, 564738.4615384615, 564738.4615384615, 564738.4615384615, 564738.4615384615, 564738.4615384615, 508200, 508200, 508200, 508200, 495929.4117647059, 495929.4117647059, 495929.4117647059, 495929.4117647059, 495929.4117647059, 495929.4117647059, 495929.4117647059, 495929.4117647059, 495929.4117647059, 495929.4117647059, 495929.4117647059, 578340, 578340, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 444345.76271186443, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000 ] }, { "name": "candidate", "type": "scatter", "uid": "ca2e4abc-a5c2-4415-a31d-e3fc55d61ab2", "x": [ 0, 0.010101010101010102, 0.020202020202020204, 0.030303030303030304, 0.04040404040404041, 0.050505050505050504, 0.06060606060606061, 0.0707070707070707, 0.08080808080808081, 0.09090909090909091, 0.10101010101010101, 0.1111111111111111, 0.12121212121212122, 0.13131313131313133, 0.1414141414141414, 0.15151515151515152, 0.16161616161616163, 0.1717171717171717, 0.18181818181818182, 0.1919191919191919, 0.20202020202020202, 0.21212121212121213, 0.2222222222222222, 0.23232323232323232, 0.24242424242424243, 0.25252525252525254, 0.26262626262626265, 0.2727272727272727, 0.2828282828282828, 0.29292929292929293, 0.30303030303030304, 0.31313131313131315, 0.32323232323232326, 0.3333333333333333, 0.3434343434343434, 0.35353535353535354, 0.36363636363636365, 0.37373737373737376, 0.3838383838383838, 0.3939393939393939, 0.40404040404040403, 0.41414141414141414, 0.42424242424242425, 0.43434343434343436, 0.4444444444444444, 0.45454545454545453, 0.46464646464646464, 0.47474747474747475, 0.48484848484848486, 0.494949494949495, 0.5050505050505051, 0.5151515151515151, 0.5252525252525253, 0.5353535353535354, 0.5454545454545454, 0.5555555555555556, 0.5656565656565656, 0.5757575757575758, 0.5858585858585859, 0.5959595959595959, 0.6060606060606061, 0.6161616161616161, 0.6262626262626263, 0.6363636363636364, 0.6464646464646465, 0.6565656565656566, 0.6666666666666666, 0.6767676767676768, 0.6868686868686869, 0.696969696969697, 0.7070707070707071, 0.7171717171717171, 0.7272727272727273, 0.7373737373737373, 0.7474747474747475, 0.7575757575757576, 0.7676767676767676, 0.7777777777777778, 0.7878787878787878, 0.797979797979798, 0.8080808080808081, 0.8181818181818182, 0.8282828282828283, 0.8383838383838383, 0.8484848484848485, 0.8585858585858586, 0.8686868686868687, 0.8787878787878788, 0.8888888888888888, 0.898989898989899, 0.9090909090909091, 0.9191919191919192, 0.9292929292929293, 0.9393939393939394, 0.9494949494949495, 0.9595959595959596, 0.9696969696969697, 0.9797979797979798, 0.98989898989899, 1 ], "y": [ 556290, 556290, 556290, 682500, 682500, 682500, 682500, 682500, 682500, 682500, 682500, 617400, 617400, 569100, 569100, 569100, 569100, 346500, 506207.6923076923, 473072.7272727273, 473072.7272727273, 473072.7272727273, 473072.7272727273, 473072.7272727273, 473072.7272727273, 473072.7272727273, 473072.7272727273, 473072.7272727273, 760200, 760200, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 460716.6666666667, 356300, 356300, 356300, 356300, 356300, 356300, 356300, 356300, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000 ] }, { "name": "latest", "type": "scatter", "uid": "62f49190-e09e-48ba-a510-fa01d8206312", "x": [ 0, 0.010101010101010102, 0.020202020202020204, 0.030303030303030304, 0.04040404040404041, 0.050505050505050504, 0.06060606060606061, 0.0707070707070707, 0.08080808080808081, 0.09090909090909091, 0.10101010101010101, 0.1111111111111111, 0.12121212121212122, 0.13131313131313133, 0.1414141414141414, 0.15151515151515152, 0.16161616161616163, 0.1717171717171717, 0.18181818181818182, 0.1919191919191919, 0.20202020202020202, 0.21212121212121213, 0.2222222222222222, 0.23232323232323232, 0.24242424242424243, 0.25252525252525254, 0.26262626262626265, 0.2727272727272727, 0.2828282828282828, 0.29292929292929293, 0.30303030303030304, 0.31313131313131315, 0.32323232323232326, 0.3333333333333333, 0.3434343434343434, 0.35353535353535354, 0.36363636363636365, 0.37373737373737376, 0.3838383838383838, 0.3939393939393939, 0.40404040404040403, 0.41414141414141414, 0.42424242424242425, 0.43434343434343436, 0.4444444444444444, 0.45454545454545453, 0.46464646464646464, 0.47474747474747475, 0.48484848484848486, 0.494949494949495, 0.5050505050505051, 0.5151515151515151, 0.5252525252525253, 0.5353535353535354, 0.5454545454545454, 0.5555555555555556, 0.5656565656565656, 0.5757575757575758, 0.5858585858585859, 0.5959595959595959, 0.6060606060606061, 0.6161616161616161, 0.6262626262626263, 0.6363636363636364, 0.6464646464646465, 0.6565656565656566, 0.6666666666666666, 0.6767676767676768, 0.6868686868686869, 0.696969696969697, 0.7070707070707071, 0.7171717171717171, 0.7272727272727273, 0.7373737373737373, 0.7474747474747475, 0.7575757575757576, 0.7676767676767676, 0.7777777777777778, 0.7878787878787878, 0.797979797979798, 0.8080808080808081, 0.8181818181818182, 0.8282828282828283, 0.8383838383838383, 0.8484848484848485, 0.8585858585858586, 0.8686868686868687, 0.8787878787878788, 0.8888888888888888, 0.898989898989899, 0.9090909090909091, 0.9191919191919192, 0.9292929292929293, 0.9393939393939394, 0.9494949494949495, 0.9595959595959596, 0.9696969696969697, 0.9797979797979798, 0.98989898989899, 1 ], "y": [ 504000, 504000, 506100, 617400, 617400, 617400, 617400, 617400, 617400, 617400, 617400, 617400, 617400, 569100, 569100, 569100, 569100, 346500, 484575, 554400, 554400, 554400, 554400, 554400, 554400, 554400, 554400, 554400, 760200, 760200, 484050, 484050, 484050, 484050, 484050, 484050, 484050, 484050, 467250, 467250, 467250, 467250, 467250, 467250, 467250, 467250, 467250, 467250, 467250, 467250, 467250, 467250, 467250, 467250, 467250, 467250, 467250, 467250, 467250, 467250, 467250, 467250, 467250, 467250, 327600, 327600, 327600, 327600, 327600, 327600, 327600, 327600, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000, 315000 ] } ], "layout": { "title": { "text": "projection" }, "xaxis": { "title": { "text": "steps" } }, "yaxis": { "title": { "text": "MEDV" } } } }, "text/html": [ "
\n", " \n", " \n", "
\n", " \n", "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "data_1 = ml_repo.get('sample1', full_object = True)\n", "left = data_1.x_data[0,:]\n", "right = data_1.x_data[10,:]\n", "output_index = 0\n", "plot.projection(ml_repo, left = left, right=right, n_steps = 100, output_index = output_index)" ] }, { "cell_type": "code", "execution_count": 40, "metadata": {}, "outputs": [ { "data": { "application/vnd.plotly.v1+json": { "config": { "linkText": "Export to plot.ly", "plotlyServerURL": "https://plot.ly", "responsive": true, "showLink": false }, "data": [ { "histnorm": "percent", "name": "prod:0.1", "opacity": 0.5, "text": "sample2:last
DecisionTreeRegressor/model:8511e2da-7719-11e9-a641-fc084a6691eb
", "type": "histogram", "uid": "7084a87e-4086-4ec4-8fbe-e98e9c395684", "x": [ 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 15.9, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 21, 20.2, 20.2, 20.2 ] }, { "histnorm": "percent", "name": "prod", "opacity": 0.5, "text": "sample2:last
DecisionTreeRegressor/model:8511e2da-7719-11e9-a641-fc084a6691eb
", "type": "histogram", "uid": "1326c66b-d5da-46ae-ba85-3ba9682ab680", "x": [ 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 16.9, 16.9, 16.9, 16.9, 16.9, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 15.5, 15.9, 17.6, 17.6, 18.8, 18.8, 17.9, 17, 19.7, 19.7, 18.3, 18.3, 17, 22, 22, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.1, 20.1, 20.1, 20.1, 20.1, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 21, 21, 21 ] }, { "histnorm": "percent", "name": "candidate:0.1", "opacity": 0.5, "text": "sample2:last
DecisionTreeRegressor/model:85269c2e-7719-11e9-a96b-fc084a6691eb
", "type": "histogram", "uid": "e09d6188-4511-42f4-bf0c-da62c1c20e95", "x": [ 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 21, 20.2, 20.2, 20.2, 20.2, 20.2 ] }, { "histnorm": "percent", "name": "candidate", "opacity": 0.5, "text": "sample2:last
DecisionTreeRegressor/model:85269c2e-7719-11e9-a96b-fc084a6691eb
", "type": "histogram", "uid": "d01e5346-0440-4997-9f37-acf749507ac8", "x": [ 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 16.9, 16.9, 16.9, 16.9, 16.9, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 15.5, 15.9, 17.6, 17.6, 18.8, 18.8, 17.9, 17, 19.7, 19.7, 18.3, 18.3, 17, 22, 22, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.1, 20.1, 20.1, 20.1, 20.1, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 21, 21, 21 ] }, { "histnorm": "percent", "name": "DecisionTreeRegressor/eval/sample2: 85464e65-7719-11e9-9ada-fc084a6691eb:0.1", "opacity": 0.5, "text": "sample2:last
DecisionTreeRegressor/model:85451654-7719-11e9-b144-fc084a6691eb
", "type": "histogram", "uid": "9553fd6e-3083-443a-a8ac-25e4e1b699a2", "x": [ 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 21, 20.2, 20.2, 20.2, 20.2, 20.2 ] }, { "histnorm": "percent", "name": "DecisionTreeRegressor/eval/sample2: 85464e65-7719-11e9-9ada-fc084a6691eb", "opacity": 0.5, "text": "sample2:last
DecisionTreeRegressor/model:85451654-7719-11e9-b144-fc084a6691eb
", "type": "histogram", "uid": "575ddce2-4aae-42c5-8f68-715357e1ba6d", "x": [ 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 18.4, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 19.6, 16.9, 16.9, 16.9, 16.9, 16.9, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 15.5, 15.9, 17.6, 17.6, 18.8, 18.8, 17.9, 17, 19.7, 19.7, 18.3, 18.3, 17, 22, 22, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.2, 20.1, 20.1, 20.1, 20.1, 20.1, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 19.2, 21, 21, 21 ] } ], "layout": { "barmode": "overlay", "title": { "text": "" }, "xaxis": { "title": { "text": "PTRATIO" } } } }, "text/html": [ "
\n", " \n", " \n", "
\n", " \n", "
" ] }, "metadata": {}, "output_type": "display_data" } ], "source": [ "plot.histogram_data_conditional_error(ml_repo, ml_repo.tree.models.DecisionTreeRegressor.model(),ml_repo.tree.test_data.sample2(), x_coordinate = 'PTRATIO')" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "celltoolbar": "Slideshow", "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.6.1" }, "toc": { "base_numbering": 1, "nav_menu": { "height": "12px", "width": "252px" }, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": false, "toc_position": { "height": "786px", "left": "0px", "right": "1470.45px", "top": "65.9943px", "width": "260px" }, "toc_section_display": "block", "toc_window_display": true }, "varInspector": { "cols": { "lenName": 16, "lenType": 16, "lenVar": 40 }, "kernels_config": { "python": { "delete_cmd_postfix": "", "delete_cmd_prefix": "del ", "library": "var_list.py", "varRefreshCmd": "print(var_dic_list())" }, "r": { "delete_cmd_postfix": ") ", "delete_cmd_prefix": "rm(", "library": "var_list.r", "varRefreshCmd": "cat(var_dic_list()) " } }, "types_to_exclude": [ "module", "function", "builtin_function_or_method", "instance", "_Feature" ], "window_display": false } }, "nbformat": 4, "nbformat_minor": 2 }