{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "cUfEWS_3PWZr"
   },
   "source": [
    "One of the most commonly asked questions by Earth Engine users is - *How do I download all images in a collection*? The Earth Engine Python API comes with a `ee.batch` module that allows you to launch batch exports and manage tasks. The recommended way to do batch exports like this is to use the Python API's `ee.batch.Export` functions and use a Python for-loop to iterate and export each image. The `ee.batch` module also gives you ability to control *Tasks* - allowing you to automate exports.\n",
    "\n",
    "> You can also export images in a collection using Javascript API in the Code Editor but this requires you to manually start the tasks for each image. This approach is fine for small number of images. You can check out the [recommended script](https://code.earthengine.google.co.in/?scriptPath=users%2Fujavalgandhi%2FEnd-to-End-GEE%3ASupplement%2FImage_Collections%2FExporting_ImageCollections)."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "CpVwpjkjIAJQ"
   },
   "source": [
    "#### Initialization\n",
    "\n",
    "First of all, you need to run the following cells to initialize the API and authorize your account. You must have a Google Cloud Project associated with your GEE account. Replace the `cloud_project` with your own project from [Google Cloud Console](https://console.cloud.google.com/)."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "JuN5rEoyjmjK"
   },
   "outputs": [],
   "source": [
    "import ee"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "gjRJhBKF2iLf"
   },
   "outputs": [],
   "source": [
    "cloud_project = 'spatialthoughts'\n",
    "\n",
    "try:\n",
    "    ee.Initialize(project=cloud_project)\n",
    "except:\n",
    "    ee.Authenticate()\n",
    "    ee.Initialize(project=cloud_project)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "mW0ljMgeg72n"
   },
   "source": [
    "#### Create a Collection"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "S60c2-FOjx0h"
   },
   "outputs": [],
   "source": [
    "geometry = ee.Geometry.Point([107.61303468448624, 12.130969369851766])\n",
    "s2 = ee.ImageCollection('COPERNICUS/S2_HARMONIZED')\n",
    "rgbVis = {\n",
    "  'min': 0.0,\n",
    "  'max': 3000,\n",
    "  'bands': ['B4', 'B3', 'B2'],\n",
    "}\n",
    "\n",
    "filtered = s2 \\\n",
    "  .filter(ee.Filter.date('2019-01-01', '2020-01-01')) \\\n",
    "  .filter(ee.Filter.lt('CLOUDY_PIXEL_PERCENTAGE', 30)) \\\n",
    "  .filter(ee.Filter.bounds(geometry)) \\\n",
    "\n",
    "# Load the Cloud Score+ collection\n",
    "csPlus = ee.ImageCollection('GOOGLE/CLOUD_SCORE_PLUS/V1/S2_HARMONIZED')\n",
    "csPlusBands = csPlus.first().bandNames()\n",
    "\n",
    "# We need to add Cloud Score + bands to each Sentinel-2\n",
    "# image in the collection\n",
    "# This is done using the linkCollection() function\n",
    "filteredS2WithCs = filtered.linkCollection(csPlus, csPlusBands)\n",
    "\n",
    "# Function to mask pixels with low CS+ QA scores.\n",
    "def maskLowQA(image):\n",
    "  qaBand = 'cs'\n",
    "  clearThreshold = 0.5\n",
    "  mask = image.select(qaBand).gte(clearThreshold)\n",
    "  return image.updateMask(mask)\n",
    "\n",
    "filteredMasked = filteredS2WithCs \\\n",
    "  .map(maskLowQA)\n",
    "\n",
    "# Write a function that computes NDVI for an image and adds it as a band\n",
    "def addNDVI(image):\n",
    "  ndvi = image.normalizedDifference(['B5', 'B4']).rename('ndvi')\n",
    "  return image.addBands(ndvi)\n",
    "\n",
    "withNdvi = filteredMasked.map(addNDVI)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "s-zNDMS9g72r"
   },
   "source": [
    "#### Export All Images\n",
    "\n",
    "Exports are done via the ``ee.batch`` module. This module allows you to automatically start an export - making it suitable for batch exports."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "STvfd9ABg72s"
   },
   "outputs": [],
   "source": [
    "image_ids = withNdvi.aggregate_array('system:index').getInfo()\n",
    "print('Total images: ', len(image_ids))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "CoXn1_hgoj81"
   },
   "outputs": [],
   "source": [
    "# Export with 100m resolution for this demo\n",
    "for i, image_id in enumerate(image_ids):\n",
    "  image = ee.Image(withNdvi.filter(ee.Filter.eq('system:index', image_id)).first())\n",
    "  task = ee.batch.Export.image.toDrive(**{\n",
    "    'image': image.select('ndvi'),\n",
    "    'description': 'Image Export {}'.format(i+1),\n",
    "    'fileNamePrefix': image_id,\n",
    "    'folder':'earthengine',\n",
    "    'scale': 100,\n",
    "    'region': image.geometry(),\n",
    "    'maxPixels': 1e10\n",
    "  })\n",
    "  task.start()\n",
    "  print('Started Task: ', i+1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "J8BtAZftg720"
   },
   "source": [
    "#### Manage Running/Waiting Tasks\n",
    "\n",
    "You can manage tasks as well. Get a list of tasks and get state information on them"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "AhNgXPb1XGBX"
   },
   "outputs": [],
   "source": [
    "tasks = ee.batch.Task.list()\n",
    "for task in tasks:\n",
    "  task_id = task.status()['id']\n",
    "  task_state = task.status()['state']\n",
    "  print(task_id, task_state)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "Y4kf_z5Wg723"
   },
   "source": [
    "You can cancel tasks as well"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "-u2_GpPog724"
   },
   "outputs": [],
   "source": [
    "tasks = ee.batch.Task.list()\n",
    "for task in tasks:\n",
    "    task_id = task.status()['id']\n",
    "    task_state = task.status()['state']\n",
    "    if task_state == 'RUNNING' or task_state == 'READY':\n",
    "        task.cancel()\n",
    "        print('Task {} canceled'.format(task_id))\n",
    "    else:\n",
    "        print('Task {} state is {}'.format(task_id, task_state))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "nujfioGtPWZ2"
   },
   "source": [
    "### Exercise\n",
    "\n",
    "The code below uses the TerraClimate data and creates an ImageCollection with 12 monthly images of maximum temperature. It also extract the geometry for Australia from the LSIB collection. Add the code to start an export task for each image in the collection for australia.\n",
    "\n",
    "- **Hint1**: TerraClimate images have a scale of 4638.3m\n",
    "- **Hint2**: You need to export the image contained in the clippedImage variable"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "27Bwko2sPWZ2"
   },
   "outputs": [],
   "source": [
    "import ee\n",
    "\n",
    "lsib = ee.FeatureCollection('USDOS/LSIB_SIMPLE/2017')\n",
    "australia = lsib.filter(ee.Filter.eq('country_na', 'Australia'))\n",
    "geometry = australia.geometry()\n",
    "\n",
    "terraclimate = ee.ImageCollection('IDAHO_EPSCOR/TERRACLIMATE')\n",
    "tmax = terraclimate.select('tmmx')\n",
    "\n",
    "def scale(image):\n",
    "  return image.multiply(0.1) \\\n",
    "    .copyProperties(image,['system:time_start'])\n",
    "\n",
    "tmaxScaled = tmax.map(scale)\n",
    "\n",
    "filtered = tmaxScaled \\\n",
    "  .filter(ee.Filter.date('2020-01-01', '2021-01-01')) \\\n",
    "  .filter(ee.Filter.bounds(geometry))\n",
    "\n",
    "image_ids = filtered.aggregate_array('system:index').getInfo()\n",
    "print('Total images: ', len(image_ids))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {
    "id": "fwp4iHF6PWZ2"
   },
   "source": [
    "Replace the comments with your code."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "id": "mbR6LaBDPWZ2"
   },
   "outputs": [],
   "source": [
    "for i, image_id in enumerate(image_ids):\n",
    "    exportImage = ee.Image(filtered.filter(ee.Filter.eq('system:index', image_id)).first())\n",
    "    # Clip the image to the region geometry\n",
    "    clippedImage = exportImage.clip(geometry)\n",
    "\n",
    "    ## Create the export task using ee.batch.Export.image.toDrive()\n",
    "\n",
    "    ## Start the task"
   ]
  }
 ],
 "metadata": {
  "colab": {
   "name": "03_export_a_collection.ipynb",
   "provenance": [],
   "toc_visible": true
  },
  "kernelspec": {
   "display_name": "Python 3 (ipykernel)",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.8.13"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}