{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "![SciUnit Logo](https://raw.githubusercontent.com/scidash/assets/master/logos/sciunit.png)\n",
    "\n",
    "# SciUnit is a framework for validating scientific models by creating experimental-data-driven unit tests.  \n",
    "\n",
    "<a href=\"https://colab.research.google.com/github/scidash/sciunit/blob/master/docs/chapter4.ipynb\" target=\"_parent\"><img src=\"https://colab.research.google.com/assets/colab-badge.svg\" alt=\"Open In Colab\"/></a>\n",
    "\n",
    "# Chapter 4. Example of RunnableModel and Backend\n",
    "(or [back to Chapter 3](chapter3.ipynb))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "#### If you are using this file in Google Colab, this block of code can help you install sciunit from PyPI in Colab environment."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "try:\n",
    "    import google.colab\n",
    "    IN_COLAB = True\n",
    "except:\n",
    "    IN_COLAB = False\n",
    "if IN_COLAB:\n",
    "    !pip install -q sciunit"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Beside the usual model in previous sections, let’s create a model that run a Backend instance to simulate and obtain results.\n",
    "\n",
    "Firstly, import necessary components from SciUnit package."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import sciunit, random\n",
    "from sciunit import Test\n",
    "from sciunit.capabilities import Runnable\n",
    "from sciunit.scores import BooleanScore\n",
    "from sciunit.models import RunnableModel\n",
    "from sciunit.models.backends import register_backends, Backend"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let’s define subclasses of SciUnit Backend, Test, and Model."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Note that:\n",
    "1. A SciUnit Backend subclass should implement _backend_run method.\n",
    "2. A SciUnit Backend subclass should implement run method."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "class RandomNumBackend(Backend):\n",
    "    '''generate a random integer between min and max'''\n",
    "\n",
    "    def set_run_params(self, **run_params):\n",
    "\n",
    "        # get min from run_params, if not exist, then 0.\n",
    "        self.min = run_params.get('min', 0)\n",
    "\n",
    "        # get max from run_params, if not exist, then self.min + 100.\n",
    "        self.max = run_params.get('max', self.min + 100)\n",
    "\n",
    "    def _backend_run(self):\n",
    "        # generate and return random integer between min and max.\n",
    "        return random.randint(self.min, self.max)\n",
    "\n",
    "class RandomNumModel(RunnableModel):\n",
    "    \"\"\"A model that always produces a constant number as output.\"\"\"\n",
    "\n",
    "    def run(self):\n",
    "        self.results = self._backend.backend_run()\n",
    "\n",
    "\n",
    "class RangeTest(Test):\n",
    "    \"\"\"Tests if the model predicts the same number as the observation.\"\"\"\n",
    "\n",
    "    # Default Runnable Capability for RunnableModel\n",
    "    required_capabilities = (Runnable,)\n",
    "\n",
    "    # This test's 'judge' method will return a BooleanScore.\n",
    "    score_type = BooleanScore\n",
    "\n",
    "    def generate_prediction(self, model):\n",
    "        model.run()\n",
    "        return model.results\n",
    "\n",
    "    def compute_score(self, observation, prediction):\n",
    "        score = BooleanScore(\n",
    "            observation['min'] <= prediction and observation['max'] >= prediction\n",
    "        )\n",
    "        return score"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let’s define the model instance named ``model 1``."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "model = RandomNumModel(\"model 1\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We must register any backend isntance in order to use it in model instances.\n",
    "\n",
    "``set_backend`` and ``set_run_params`` methods can help us to set the run-parameters in the model and its backend."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "register_backends({\"Random Number\": RandomNumBackend})\n",
    "model.set_backend(\"Random Number\")\n",
    "model.set_run_params(min=1, max=10)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, create an observation that requires the generated random integer between 1 and 10 and a test instance that use the observation and against the model\n",
    "\n",
    "Then we get a more quantitative summary of the results:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "observation = {'min': 1, 'max': 10}\n",
    "oneToTenTest = RangeTest(observation, \"test 1\")\n",
    "score = oneToTenTest.judge(model)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "print the score, and we can see the result."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "print(score)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.3"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}