{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Custom Adjustments\n", "\n", "The ParamTools adjustment format and logic can be augmented significantly. This is helpful for projects that need to support a pre-existing data format or require custom adjustment logic. Projects should customize their adjustments by writing their own `adjust` method and then calling the default `adjust` method from there:\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import paramtools\n", "\n", "\n", "class Params(paramtools.Parameters):\n", " def adjust(self, params_or_path, **kwargs):\n", " params = self.read_params(params_or_path)\n", "\n", " # ... custom logic here\n", "\n", " # call default adjust method.\n", " return super().adjust(params, **kwargs)\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Example\n", "\n", "Some projects may find it convenient to use CSVs for their adjustment format. That's no problem for ParamTools as long as the CSV is converted to a JSON file or Python dictionary that meets the ParamTools criteria.\n" ] }, { "cell_type": "code", "execution_count": 2, "metadata": {}, "outputs": [], "source": [ "import io\n", "import os\n", "\n", "import pandas as pd\n", "\n", "import paramtools\n", "\n", "\n", "class CSVParams(paramtools.Parameters):\n", " defaults = {\n", " \"schema\": {\n", " \"labels\": {\n", " \"year\": {\n", " \"type\": \"int\",\n", " \"validators\": {\"range\": {\"min\": 2000, \"max\": 2005}}\n", " }\n", " }\n", " },\n", " \"a\": {\n", " \"title\": \"A\",\n", " \"description\": \"a param\",\n", " \"type\": \"int\",\n", " \"value\": [\n", " {\"year\": 2000, \"value\": 1},\n", " {\"year\": 2001, \"value\": 2},\n", " ]\n", " },\n", " \"b\": {\n", " \"title\": \"B\",\n", " \"description\": \"b param\",\n", " \"type\": \"int\",\n", " \"value\": [\n", " {\"year\": 2000, \"value\": 3},\n", " {\"year\": 2001, \"value\": 4},\n", " ]\n", " }\n", " }\n", "\n", " def adjust(self, params_or_path, **kwargs):\n", " \"\"\"\n", " A custom adjust method that converts CSV files to\n", " ParamTools compliant Python dictionaries.\n", " \"\"\"\n", " if os.path.exists(params_or_path):\n", " paramsdf = pd.read_csv(params_or_path, index_col=\"year\")\n", " else:\n", " paramsdf = pd.read_csv(io.StringIO(params_or_path), index_col=\"year\")\n", "\n", " dfdict = paramsdf.to_dict()\n", " params = {\"a\": [], \"b\": []}\n", " for label in params:\n", " for year, value in dfdict[label].items():\n", " params[label] += [{\"year\": year, \"value\": value}]\n", "\n", " # call adjust method on paramtools.Parameters which will\n", " # call _adjust to actually do the update.\n", " return super().adjust(params, **kwargs)\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now we create an example CSV file. To keep the example self-contained, the CSV is just a string, but this example works with CSV files, too. The values of \"A\" are updated to 5 in 2000 and 6 in 2001, and the values of \"B\" are updated to 6 in 2000 and 7 in 2001.\n" ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "OrderedDict([('a',\n", " [OrderedDict([('year', 2000), ('value', 5)]),\n", " OrderedDict([('year', 2001), ('value', 6)])]),\n", " ('b',\n", " [OrderedDict([('year', 2000), ('value', 6)]),\n", " OrderedDict([('year', 2001), ('value', 7)])])])" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "# this could also be a path to a CSV file.\n", "csv_string = \"\"\"\n", "year,a,b\n", "2000,5,6\\n\n", "2001,6,7\\n\n", "\"\"\"\n", "\n", "params = CSVParams()\n", "params.adjust(csv_string)\n" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[OrderedDict([('year', 2000), ('value', 5)]),\n", " OrderedDict([('year', 2001), ('value', 6)])]" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "params.a" ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "[OrderedDict([('year', 2000), ('value', 6)]),\n", " OrderedDict([('year', 2001), ('value', 7)])]" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "params.b" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Now, if we use `array_first` and [`label_to_extend`](/api/extend/), the params instance can be loaded into a Pandas\n", "DataFrame like this:\n" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
ab
056
167
267
367
467
567
\n", "
" ], "text/plain": [ " a b\n", "0 5 6\n", "1 6 7\n", "2 6 7\n", "3 6 7\n", "4 6 7\n", "5 6 7" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "csv_string = \"\"\"\n", "year,a,b\n", "2000,5,6\\n\n", "2001,6,7\\n\n", "\"\"\"\n", "\n", "params = CSVParams(array_first=True, label_to_extend=\"year\")\n", "params.adjust(csv_string)\n", "\n", "params_df = pd.DataFrame.from_dict(params.to_dict())\n", "params_df\n" ] }, { "cell_type": "code", "execution_count": 7, "metadata": {}, "outputs": [ { "data": { "text/html": [ "
\n", "\n", "\n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", " \n", "
ab
year
200056
200167
200267
200367
200467
200567
\n", "
" ], "text/plain": [ " a b\n", "year \n", "2000 5 6\n", "2001 6 7\n", "2002 6 7\n", "2003 6 7\n", "2004 6 7\n", "2005 6 7" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "params_df[\"year\"] = params.label_grid[\"year\"]\n", "params_df.set_index(\"year\")" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.8.5" } }, "nbformat": 4, "nbformat_minor": 4 }