{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Custom Adjustments\n",
"\n",
"The ParamTools adjustment format and logic can be augmented significantly. This is helpful for projects that need to support a pre-existing data format or require custom adjustment logic. Projects should customize their adjustments by writing their own `adjust` method and then calling the default `adjust` method from there:\n"
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {},
"outputs": [],
"source": [
"import paramtools\n",
"\n",
"\n",
"class Params(paramtools.Parameters):\n",
" def adjust(self, params_or_path, **kwargs):\n",
" params = self.read_params(params_or_path)\n",
"\n",
" # ... custom logic here\n",
"\n",
" # call default adjust method.\n",
" return super().adjust(params, **kwargs)\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Example\n",
"\n",
"Some projects may find it convenient to use CSVs for their adjustment format. That's no problem for ParamTools as long as the CSV is converted to a JSON file or Python dictionary that meets the ParamTools criteria.\n"
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {},
"outputs": [],
"source": [
"import io\n",
"import os\n",
"\n",
"import pandas as pd\n",
"\n",
"import paramtools\n",
"\n",
"\n",
"class CSVParams(paramtools.Parameters):\n",
" defaults = {\n",
" \"schema\": {\n",
" \"labels\": {\n",
" \"year\": {\n",
" \"type\": \"int\",\n",
" \"validators\": {\"range\": {\"min\": 2000, \"max\": 2005}}\n",
" }\n",
" }\n",
" },\n",
" \"a\": {\n",
" \"title\": \"A\",\n",
" \"description\": \"a param\",\n",
" \"type\": \"int\",\n",
" \"value\": [\n",
" {\"year\": 2000, \"value\": 1},\n",
" {\"year\": 2001, \"value\": 2},\n",
" ]\n",
" },\n",
" \"b\": {\n",
" \"title\": \"B\",\n",
" \"description\": \"b param\",\n",
" \"type\": \"int\",\n",
" \"value\": [\n",
" {\"year\": 2000, \"value\": 3},\n",
" {\"year\": 2001, \"value\": 4},\n",
" ]\n",
" }\n",
" }\n",
"\n",
" def adjust(self, params_or_path, **kwargs):\n",
" \"\"\"\n",
" A custom adjust method that converts CSV files to\n",
" ParamTools compliant Python dictionaries.\n",
" \"\"\"\n",
" if os.path.exists(params_or_path):\n",
" paramsdf = pd.read_csv(params_or_path, index_col=\"year\")\n",
" else:\n",
" paramsdf = pd.read_csv(io.StringIO(params_or_path), index_col=\"year\")\n",
"\n",
" dfdict = paramsdf.to_dict()\n",
" params = {\"a\": [], \"b\": []}\n",
" for label in params:\n",
" for year, value in dfdict[label].items():\n",
" params[label] += [{\"year\": year, \"value\": value}]\n",
"\n",
" # call adjust method on paramtools.Parameters which will\n",
" # call _adjust to actually do the update.\n",
" return super().adjust(params, **kwargs)\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now we create an example CSV file. To keep the example self-contained, the CSV is just a string, but this example works with CSV files, too. The values of \"A\" are updated to 5 in 2000 and 6 in 2001, and the values of \"B\" are updated to 6 in 2000 and 7 in 2001.\n"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"OrderedDict([('a',\n",
" [OrderedDict([('year', 2000), ('value', 5)]),\n",
" OrderedDict([('year', 2001), ('value', 6)])]),\n",
" ('b',\n",
" [OrderedDict([('year', 2000), ('value', 6)]),\n",
" OrderedDict([('year', 2001), ('value', 7)])])])"
]
},
"execution_count": 3,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"# this could also be a path to a CSV file.\n",
"csv_string = \"\"\"\n",
"year,a,b\n",
"2000,5,6\\n\n",
"2001,6,7\\n\n",
"\"\"\"\n",
"\n",
"params = CSVParams()\n",
"params.adjust(csv_string)\n"
]
},
{
"cell_type": "code",
"execution_count": 4,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[OrderedDict([('year', 2000), ('value', 5)]),\n",
" OrderedDict([('year', 2001), ('value', 6)])]"
]
},
"execution_count": 4,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"params.a"
]
},
{
"cell_type": "code",
"execution_count": 5,
"metadata": {},
"outputs": [
{
"data": {
"text/plain": [
"[OrderedDict([('year', 2000), ('value', 6)]),\n",
" OrderedDict([('year', 2001), ('value', 7)])]"
]
},
"execution_count": 5,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"params.b"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now, if we use `array_first` and [`label_to_extend`](/api/extend/), the params instance can be loaded into a Pandas\n",
"DataFrame like this:\n"
]
},
{
"cell_type": "code",
"execution_count": 6,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"
\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" a | \n",
" b | \n",
"
\n",
" \n",
" \n",
" \n",
" 0 | \n",
" 5 | \n",
" 6 | \n",
"
\n",
" \n",
" 1 | \n",
" 6 | \n",
" 7 | \n",
"
\n",
" \n",
" 2 | \n",
" 6 | \n",
" 7 | \n",
"
\n",
" \n",
" 3 | \n",
" 6 | \n",
" 7 | \n",
"
\n",
" \n",
" 4 | \n",
" 6 | \n",
" 7 | \n",
"
\n",
" \n",
" 5 | \n",
" 6 | \n",
" 7 | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" a b\n",
"0 5 6\n",
"1 6 7\n",
"2 6 7\n",
"3 6 7\n",
"4 6 7\n",
"5 6 7"
]
},
"execution_count": 6,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"csv_string = \"\"\"\n",
"year,a,b\n",
"2000,5,6\\n\n",
"2001,6,7\\n\n",
"\"\"\"\n",
"\n",
"params = CSVParams(array_first=True, label_to_extend=\"year\")\n",
"params.adjust(csv_string)\n",
"\n",
"params_df = pd.DataFrame.from_dict(params.to_dict())\n",
"params_df\n"
]
},
{
"cell_type": "code",
"execution_count": 7,
"metadata": {},
"outputs": [
{
"data": {
"text/html": [
"\n",
"\n",
"
\n",
" \n",
" \n",
" | \n",
" a | \n",
" b | \n",
"
\n",
" \n",
" year | \n",
" | \n",
" | \n",
"
\n",
" \n",
" \n",
" \n",
" 2000 | \n",
" 5 | \n",
" 6 | \n",
"
\n",
" \n",
" 2001 | \n",
" 6 | \n",
" 7 | \n",
"
\n",
" \n",
" 2002 | \n",
" 6 | \n",
" 7 | \n",
"
\n",
" \n",
" 2003 | \n",
" 6 | \n",
" 7 | \n",
"
\n",
" \n",
" 2004 | \n",
" 6 | \n",
" 7 | \n",
"
\n",
" \n",
" 2005 | \n",
" 6 | \n",
" 7 | \n",
"
\n",
" \n",
"
\n",
"
"
],
"text/plain": [
" a b\n",
"year \n",
"2000 5 6\n",
"2001 6 7\n",
"2002 6 7\n",
"2003 6 7\n",
"2004 6 7\n",
"2005 6 7"
]
},
"execution_count": 7,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"params_df[\"year\"] = params.label_grid[\"year\"]\n",
"params_df.set_index(\"year\")"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": []
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3",
"version": "3.8.5"
}
},
"nbformat": 4,
"nbformat_minor": 4
}