{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Tutorial 2: Bird migration analysis\n",
    "\n",
    "<img align=\"right\" src=\"https://anitagraser.github.io/movingpandas/pics/movingpandas.png\">\n",
    "\n",
    "This tutorial uses data published on Movebank, specifically: [Navigation experiments in lesser black-backed gulls (data from Wikelski et al. 2015)-gps.csv](https://www.datarepository.movebank.org/handle/10255/move.494)\n",
    "\n",
    "This tutorial covers: \n",
    "1. Trajectory data preprocessing\n",
    " 1. Loading movement data from common geospatial file formats\n",
    " 1. Exploring spatial & non-spatial data distributions\n",
    " 1. Converting GeoDataFrames into Trajectories describing continuous tracks of moving objects\n",
    "1. Trajectory data analysis\n",
    " 1. Investigating individual trajectories\n",
    " 1. Comparing different years\n",
    " 1. Investigating trajectories of multiple individuals"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%matplotlib inline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.core.display import display, HTML\n",
    "display(HTML(\"<style>.container { width:100% !important; }</style>\"))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "import urllib\n",
    "import os\n",
    "import pandas as pd\n",
    "from geopandas import GeoDataFrame, read_file\n",
    "from shapely.geometry import Point, LineString, Polygon\n",
    "from datetime import datetime, timedelta\n",
    "import matplotlib.pyplot as plt\n",
    "\n",
    "import sys\n",
    "sys.path.append(\"..\")\n",
    "import movingpandas as mpd\n",
    "\n",
    "import warnings\n",
    "warnings.simplefilter(\"ignore\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Loading the bird movement data \n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%%time\n",
    "df = read_file('data/demodata_gulls.gpkg')\n",
    "wgs84 = df.crs\n",
    "df['t'] = pd.to_datetime(df['timestamp'])\n",
    "df = df.set_index('t')\n",
    "print(\"Finished reading {}\".format(len(df)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This is what the data looks like:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df.plot()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see how many individuals we have in the dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['individual-local-identifier'].unique()"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The records per individual are not evenly distributed:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "df['individual-local-identifier'].value_counts().plot(kind='bar', figsize=(17,3))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Finally, let's create trajectories:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "MIN_LENGTH = 100 # meters\n",
    "\n",
    "traj_collection = mpd.TrajectoryCollection(df, 'individual-local-identifier', min_length=MIN_LENGTH)     \n",
    "all_trajectories = traj_collection.trajectories\n",
    "    \n",
    "print(\"Finished creating {} trajectories\".format(len(all_trajectories)))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Investigating individual trajectories"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's pick out a specific individual. For example, '91916A' is the individual with most records in our dataset:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "filtered = traj_collection.filter('individual-local-identifier', '91916A')\n",
    "my_traj = filtered.trajectories[0].copy()\n",
    "my_traj.df.head()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "my_traj.hvplot(line_width=2) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This individual has been travelling back and forth for quite a few years!\n",
    "\n",
    "One way to take a closer look at this individual's travels is to split the overall track into yearly trips:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "trips_by_year = mpd.TemporalSplitter(filtered).split(mode='year')\n",
    "for trip in trips_by_year:\n",
    "    print(trip.id)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we can explore individual years:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "one_year = trips_by_year.get_trajectory('91916A_2010-12-31 00:00:00')\n",
    "print(one_year)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "one_year.hvplot(line_width=5.0, c='speed', cmap='RdYlGn', colorbar=True, clim=(0,15)) "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Let's see where this individual was on a specific day:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_location_at_timestamp(traj, t, fig_size=300):\n",
    "    loc = GeoDataFrame([traj.get_row_at(t)])\n",
    "    return (loc.hvplot(geo=True, tiles='OSM', size=200, color='red', width=fig_size, height=fig_size) * \n",
    "            traj.hvplot(line_width=1.0, color='black', tiles=False, width=fig_size, height=fig_size))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "( plot_location_at_timestamp(one_year, datetime(2010,9,1)) + \n",
    "  plot_location_at_timestamp(one_year, datetime(2010,10,1)) +\n",
    "  plot_location_at_timestamp(one_year, datetime(2010,11,1)) )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Of course, it might also be of interest to see the different locations on a certain day each year:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def plot_location_at_day_of_year(traj, month, day, ax=None):\n",
    "    ts = [datetime(year, month, day) for year in traj.df.index.year.unique()]\n",
    "    return plot_locations_at_timestamps(traj, ts, ax=ax)\n",
    "\n",
    "def plot_locations_at_timestamps(traj, ts, ax=None):  \n",
    "    loc = GeoDataFrame([traj.get_row_at(t) for t in ts])\n",
    "    loc['date_label'] = loc.index.strftime('%Y-%m-%d')\n",
    "    return (loc.hvplot(c='date_label', size=200, geo=True, tiles='OSM') *\n",
    "            traj.hvplot(line_width=1.0, color='black', geo=True, tiles=False) )"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "plot_location_at_day_of_year(my_traj, month=10, day=1)"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It's pretty clear that this individual does not follow the same schedule and route every year. However, it seems to always be heading to the same area Red Sea coast to spend the winter there.\n",
    "\n",
    "Let's find its arrival times in this area:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "area_of_interest = Polygon([(30, 25), (50, 25), (50, 15), (30, 15), (30, 25)])\n",
    "plotted_area_of_interest = GeoDataFrame(pd.DataFrame([{'geometry': area_of_interest, 'id': 1}]), crs=wgs84).hvplot(geo=True, color='yellow', alpha=0.5)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "arrivals = [traj for traj in my_traj.clip(area_of_interest)]\n",
    "print(\"Found {} arrivals\".format(len(arrivals)))\n",
    "\n",
    "for traj in arrivals:\n",
    "    print(\"Individual '{}' arrived at {}\".format(traj.df['individual-local-identifier'].iloc[0], traj.get_start_time()))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "( plot_locations_at_timestamps(my_traj, [traj.get_start_time() for traj in arrivals]) * plotted_area_of_interest )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Investigating trajectories of multiple individuals\n",
    "\n",
    "Multiple individuals travel to this area every year. Let's have a closer look:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_trajectories_by_year(trajs, year):\n",
    "    result = []\n",
    "    for traj in trajs:\n",
    "        if traj.get_start_time().year <= year_of_interest and traj.get_end_time().year >= year_of_interest:\n",
    "            result.append(traj)\n",
    "    return result"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "year_of_interest = 2010\n",
    "\n",
    "arrivals = [ traj.clip(area_of_interest) for traj in all_trajectories ]  # list of lists\n",
    "arrivals = [ item for sublist in arrivals for item in sublist ]  # flat list\n",
    "\n",
    "relevant = get_trajectories_by_year(arrivals, year_of_interest)\n",
    "print(\"Found {} arrivals\".format(len(relevant)))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "for traj in relevant:\n",
    "    print(\"Individual '{}' arrived at {} (duration: {})\".format(\n",
    "        traj.df['individual-local-identifier'].iloc[0], traj.get_start_time().date(), \n",
    "        traj.get_end_time()-traj.get_start_time()))"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Based on the duration of the individuals' trajectory segments within our area of interest, it looks like some individuals spend the winter here while others only pass through.\n",
    "\n",
    "For example, Individual '91761A' passed through twice? What has it been up to?"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "def get_individual_traj_for_year(trajs, id, year):\n",
    "    individual = traj_collection.get_trajectory(id)\n",
    "    return individual.get_segment_between(datetime(year,1,1), datetime(year,12,31))\n",
    "\n",
    "plotted_trajectory = get_individual_traj_for_year(all_trajectories, '91761A', year_of_interest).hvplot(color='black', line_width=1.0)\n",
    "\n",
    "( plotted_trajectory * plotted_area_of_interest )"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Turns out that this individual does not stay at the Red Sea but continues its journey into Africa."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Continue exploring MovingPandas\n",
    "\n",
    "* [Tutorial 3: Horse collar data](3_horse_collar.ipynb)"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 4
}