{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "english-billy",
   "metadata": {},
   "source": [
    "# Test: collected errors\n",
    "\n",
    "* Project: c3s-cmip6\n",
    "* Site: DKRZ\n",
    "* From 2021-03-23 to 2021-03-25\n",
    "* rook v0.4.2, daops=0.5.0, clisops=0.6.2"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "olympic-donor",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "# os.environ['ROOK_URL'] = 'http://rook3.cloud.dkrz.de/wps'\n",
    "os.environ['ROOK_URL'] = 'http://rook.dkrz.de/wps'\n",
    "\n",
    "from rooki import operators as ops"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "owned-myrtle",
   "metadata": {},
   "source": [
    "## Day 7: Mon 29.03\n",
    "\n",
    "DKRZ: 15 requests, 0 failures"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "sacred-seller",
   "metadata": {},
   "source": [
    "## Day 6: Sun 28.03\n",
    "\n",
    "DKRZ: 145 requests, 0 failures"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "lovely-count",
   "metadata": {},
   "source": [
    "## Day 5: Sat 27.03\n",
    "\n",
    "DKRZ: 100 requests, 5 failures\n",
    "    \n",
    "    \n",
    "Duplicate errors:\n",
    "* Process error: invalid day number provided in cftime.Datetime3\n",
    "60Day 2100, 12, 31, 0, 0, 0, 0\n",
    "* Process error: The longitude of this dataset runs from 0.01 to 360.00, and rolling could not be completed successfully. Please re-run your request with longitudes between these bounds."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "distinguished-independence",
   "metadata": {},
   "source": [
    "## Day 4: Fri 26.03\n",
    "\n",
    "DKRZ: 85 requests, 27 failures\n",
    "\n",
    "Duplicate errors:\n",
    "* Process error: Cannot apply_along_axis when any iteration dimensions are 0 \n",
    "* Process error: invalid day number provided in cftime.Datetime3\n",
    "60Day 2300, 12, 31, 0, 0, 0, 0\n",
    "* Process error: float divmod  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "indoor-taiwan",
   "metadata": {},
   "source": [
    "### Error 4-1\n",
    "\n",
    "Issue: https://github.com/roocs/daops/issues/63\n",
    "\n",
    "Probably a temporary access failure.\n",
    "\n",
    "Process error: list index out of range\n",
    "\n",
    "```\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/daops/utils/consolidate.py\", line 42, in consolidate\n",
    "    ds = open_xr_dataset(dset)\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/roocs_utils/xarray_utils/xarray_utils.py\", line 33, in open_xr_dataset\n",
    "    return xr.open_dataset(dset[0], use_cftime=True)\n",
    "IndexError: list index out of range\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "latest-ownership",
   "metadata": {},
   "outputs": [],
   "source": [
    "wf = ops.Subset(\n",
    "        ops.Input(\n",
    "            'tas', ['c3s-cmip6.ScenarioMIP.NIMS-KMA.KACE-1-0-G.ssp245.r1i1p1f1.Amon.tas.gr.v20191217']\n",
    "        ),\n",
    "        time=\"2021-01-01/2100-12-30\",\n",
    "        area=\"-10,30,35,70\"\n",
    ")\n",
    "resp = wf.orchestrate()\n",
    "resp.status"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "declared-vegetable",
   "metadata": {},
   "source": [
    "## Day 3: Thu 25.03\n",
    "\n",
    "DKRZ: 53 requests, 9 failures\n",
    "\n",
    "Duplicate erros:\n",
    "* invalid day number provided in cftime.Datetime360Day 2050, 12, 31, 0, 0, 0, 0\n",
    "* float divmod "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "recent-approach",
   "metadata": {},
   "source": [
    "## Day 2: Wed 24.03\n",
    "\n",
    "DKRZ: 218 requests, 8 failures, 1 rejected"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "distinguished-paraguay",
   "metadata": {},
   "source": [
    "### Error 2-1\n",
    "\n",
    "Issue: https://github.com/roocs/daops/issues/64\n",
    "\n",
    "Process error: Cannot apply_along_axis when any iteration dime\n",
    "nsions are 0  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "consolidated-breath",
   "metadata": {},
   "source": [
    "```\n",
    "Traceback (most recent call last):\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/rook/director/director.py\", line 156, in process\n",
    "    file_uris = runner(self.inputs)\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/rook/utils/subset_utils.py\", line 5, in run_subset\n",
    "    result = subset(**args)\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/daops/ops/subset.py\", line 77, in subset\n",
    "    result_set = Subset(**locals()).calculate()\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/daops/ops/base.py\", line 88, in calculate\n",
    "    process(self.get_operation_callable(), norm_collection, **self.params),\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/daops/processor.py\", line 19, in process\n",
    "    result = operation(dset, **kwargs)\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/clisops/ops/subset.py\", line 165, in subset\n",
    "    return op.process()\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/clisops/ops/base_operation.py\", line 89, in process\n",
    "    processed_ds = self._calculate()\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/clisops/ops/subset.py\", line 63, in _calculate\n",
    "    result = subset_bbox(ds, **self.params)\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/clisops/core/subset.py\", line 251, in func_checker\n",
    "    return func(*args, **kwargs)\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/clisops/core/subset.py\", line 875, in subset_bbox\n",
    "    da[var] = da[var].where(lon_cond & lat_cond, drop=True)\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/xarray/core/common.py\", line 1273, in where\n",
    "    return ops.where_method(self, cond, other)\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/xarray/core/ops.py\", line 203, in where_method\n",
    "    keep_attrs=True,\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/xarray/core/computation.py\", line 1134, in apply_ufunc\n",
    "    keep_attrs=keep_attrs,\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/xarray/core/computation.py\", line 271, in apply_dataarray_vfunc\n",
    "    result_var = func(*data_vars)\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/xarray/core/computation.py\", line 632, in apply_variable_ufunc\n",
    "    for arg, core_dims in zip(args, signature.input_core_dims)\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/xarray/core/computation.py\", line 632, in <listcomp>\n",
    "    for arg, core_dims in zip(args, signature.input_core_dims)\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/xarray/core/computation.py\", line 542, in broadcast_compat_data\n",
    "    data = variable.data\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/xarray/core/variable.py\", line 374, in data\n",
    "    return self.values\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/xarray/core/variable.py\", line 554, in values\n",
    "    return _as_array_or_item(self._data)\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/xarray/core/variable.py\", line 287, in _as_array_or_item\n",
    "    data = np.asarray(data)\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/numpy/core/_asarray.py\", line 102, in asarray\n",
    "    return array(a, dtype, copy=False, order=order)\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/xarray/core/indexing.py\", line 693, in __array__\n",
    "    self._ensure_cached()\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/xarray/core/indexing.py\", line 690, in _ensure_cached\n",
    "    self.array = NumpyIndexingAdapter(np.asarray(self.array))\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/numpy/core/_asarray.py\", line 102, in asarray\n",
    "    return array(a, dtype, copy=False, order=order)\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/xarray/core/indexing.py\", line 663, in __array__\n",
    "    return np.asarray(self.array, dtype=dtype)\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/numpy/core/_asarray.py\", line 102, in asarray\n",
    "    return array(a, dtype, copy=False, order=order)\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/xarray/core/indexing.py\", line 568, in __array__\n",
    "    return np.asarray(array[self.key], dtype=None)\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/xarray/backends/netCDF4_.py\", line 86, in __getitem__\n",
    "    key, self.shape, indexing.IndexingSupport.OUTER, self._getitem\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/xarray/core/indexing.py\", line 853, in explicit_indexing_adapter\n",
    "    result = raw_indexing_method(raw_key.tuple)\n",
    "\n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "rubber-foundation",
   "metadata": {},
   "outputs": [],
   "source": [
    "wf = ops.Subset(\n",
    "        ops.Input(\n",
    "            'tos', ['c3s-cmip6.ScenarioMIP.CNRM-CERFACS.CNRM-CM6-1.ssp245.r1i1p1f2.Omon.tos.gn.v20190219']\n",
    "        ),\n",
    "        # time=\"2021-01-01/2050-12-31\",\n",
    "        area=\"1,40,2,4\"\n",
    ")\n",
    "resp = wf.orchestrate()\n",
    "resp.status"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "presidential-hughes",
   "metadata": {},
   "source": [
    "### Error 2-2\n",
    "\n",
    "* Issue: https://github.com/roocs/daops/issues/61\n",
    "* PR: https://github.com/roocs/clisops/pull/152\n",
    "\n",
    "Process error: float divmod"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "australian-belief",
   "metadata": {},
   "outputs": [],
   "source": [
    "wf = ops.Subset(\n",
    "        ops.Input(\n",
    "            'hur', ['c3s-cmip6.CMIP.BCC.BCC-ESM1.historical.r1i1p1f1.Amon.hur.gn.v20181217']\n",
    "        ),\n",
    "        # time=\"2021-01-01/2050-12-31\",\n",
    "        area=\"0,0,0.01,0.01\",\n",
    "        level=\"85000/85000\"\n",
    ")\n",
    "resp = wf.orchestrate()\n",
    "resp.status"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "streaming-freeze",
   "metadata": {},
   "source": [
    "### Error 2-3\n",
    "\n",
    "Request rejected due to exception\n",
    "\n",
    "Probably this one:\n",
    "https://github.com/roocs/rook/issues/161"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "olive-retro",
   "metadata": {},
   "source": [
    "## Day 1: Tue 23.03\n",
    "\n",
    "DKRZ: 147 requests, 11 failures"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "chronic-politics",
   "metadata": {},
   "source": [
    "### Error 1-1\n",
    "\n",
    "Issue: https://github.com/roocs/clisops/issues/151\n",
    "\n",
    "ValueError: invalid day number provided in cftime.Datetime360Day(2050, 12, 31, 0, 0, 0, 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "concerned-providence",
   "metadata": {},
   "outputs": [],
   "source": [
    "wf = ops.Subset(\n",
    "        ops.Input(\n",
    "            'tas', ['c3s-cmip6.ScenarioMIP.NIMS-KMA.KACE-1-0-G.ssp585.r1i1p1f1.Amon.tas.gr.v20190920']\n",
    "        ),\n",
    "        time=\"2021-01-01/2050-12-31\",\n",
    "        area=\"-10,30,35,70\"\n",
    ")\n",
    "resp = wf.orchestrate()\n",
    "resp.status"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "hybrid-pickup",
   "metadata": {},
   "source": [
    "### Error 1-2\n",
    "\n",
    "ValueError: invalid day number provided in cftime.Datetime360Day(2050, 12, 31, 0, 0, 0, 0)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "powerful-emperor",
   "metadata": {},
   "outputs": [],
   "source": [
    "wf = ops.Subset(\n",
    "        ops.Input(\n",
    "            'tas', ['c3s-cmip6.ScenarioMIP.MOHC.HadGEM3-GC31-MM.ssp585.r1i1p1f3.Amon.tas.gn.v20200515']\n",
    "        ),\n",
    "        time=\"2021-01-01/2050-12-31\",\n",
    "        area=\"-10,30,35,70\"\n",
    ")\n",
    "resp = wf.orchestrate()\n",
    "resp.status"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "photographic-registrar",
   "metadata": {},
   "source": [
    "### Error 1-3\n",
    "\n",
    "PR: https://github.com/roocs/clisops/issues/151\n",
    "\n",
    "```\n",
    "  File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/clisops/core/subset.py\", line 242, in func_checker\n",
    "    f\"Input longitude bounds ({kwargs[lon]}) cross the 0 degree meridian but\"\n",
    "NotImplementedError: Input longitude bounds ([ 160. -100.]) cross the 0 degree meridian but dataset longitudes are all positive.\n",
    "    \n",
    "\n",
    " File \"/usr/local/anaconda/envs/rook/lib/python3.7/site-packages/clisops/ops/subset.py\", line 66, in _calculate\n",
    "    f\"The input longitude bounds {self.params.get('lon_bnds')} are not within the longitude bounds \"\n",
    "Exception: The input longitude bounds (160.0, -100.0) are not within the longitude bounds of this dataset and rolling could not be completed successfully. Please re-run your request with longitudes within the bounds of the dataset.\n",
    " \n",
    "```"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "analyzed-statistics",
   "metadata": {},
   "outputs": [],
   "source": [
    "wf = ops.Subset(\n",
    "        ops.Input(\n",
    "            'snd', ['c3s-cmip6.ScenarioMIP.NOAA-GFDL.GFDL-ESM4.ssp245.r1i1p1f1.LImon.snd.gr1.v20180701']\n",
    "        ),\n",
    "        time=\"2080-01-01/2100-12-31\",\n",
    "        area=\"160,45,-100,90\"\n",
    ")\n",
    "resp = wf.orchestrate()\n",
    "resp.status"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "seven-advice",
   "metadata": {},
   "source": [
    "### Error 1-4\n",
    "\n",
    "Process error: The input longitude bounds 160.0, -100.0 are no\n",
    "t within the longitude bounds of this dataset and rolling could not be completed successfully. Please re-run your request with longitudes within the bounds of the dataset."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "plain-printing",
   "metadata": {},
   "source": [
    "### Error 1-5\n",
    "\n",
    "Process error: float divmod "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "nervous-tribune",
   "metadata": {},
   "source": [
    "### Error 1-6\n",
    "\n",
    "Process error: invalid day number provided in cftime.Datetime3\n",
    "60Day 2100, 12, 31, 0, 0, 0, 0   "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "plain-dressing",
   "metadata": {},
   "source": [
    "### Error 1-7\n",
    "\n",
    "Process error: Unable to parse the time values entered  "
   ]
  },
  {
   "cell_type": "markdown",
   "id": "democratic-jumping",
   "metadata": {},
   "source": [
    "### Error 1-8\n",
    "\n",
    "Process error: Start date 2020-02-01T12:00:00 is after end dat\n",
    "e 2020-01-30T12:00:00 "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.9.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}