{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "4b28bd57",
   "metadata": {},
   "source": [
    "# 2 HiSPARC-API\n",
    "\n",
    "Deze notebooks werken alleen met Python 3.\n",
    "\n",
    "## Inleiding\n",
    "\n",
    "Informatie over en van meetstations (configuraties, events, coincidenties) is op\n",
    "te halen via de publieke database op https://data.hisparc.nl\n",
    "\n",
    "Dit notebook gaat over het ophalen en verwerken van gegevens van meetstations\n",
    "GPS posities, hardwareserienummers, detector tijdoffsets en PMT spanning via de\n",
    "**HiSPARC API**.\n",
    "\n",
    "Via de API kan door middel van een URL informatie uit de publieke database over\n",
    "stations worden opgevraagd:\n",
    "- https://data.hisparc.nl/api/clusters/  (de clusters in het netwerk in JSON)\n",
    "- https://data.hisparc.nl/api/station/22/  (informatie over station 22 in JSON)\n",
    "- https://data.hisparc.nl/show/source/gps/501/  (De GPS posities van station 501, in TSV)\n",
    "\n",
    "Bovenstaande links zijn ook te vinden als links op https://data.hisparc.nl.\n",
    "Meer informatie over de HiSPARC API is te vinden via:\n",
    "https://docs.hisparc.nl/publicdb/api_tutorial.html\n",
    "\n",
    "In SAPPHiRE is de informatie uit de API op twee manieren beschikbaar:\n",
    "1. Direct uitlezen van API informatie via `Station()` en `Network()`\n",
    "2. Analyse van groepen van stations (clusters) via `HiSPARCStations()` en\n",
    "`HiSPARCNetwork()`\n",
    "\n",
    "## Uitlezen van de API via Network() en Station()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "b3b6d45b",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sapphire import Network, Station"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "d918fe3d",
   "metadata": {},
   "source": [
    "### Network()\n",
    "\n",
    "Network is voor het ophalen van informatie over het netwerk via de API:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c6c0dc0d",
   "metadata": {},
   "outputs": [],
   "source": [
    "network = Network()\n",
    "print(network.station_numbers())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "139e9e00",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(network.clusters())"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "3946b161",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(network.station_numbers(cluster=2000))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "2829ac74",
   "metadata": {},
   "source": [
    "### Station()\n",
    "\n",
    "`Station()` kan informatie over een meetstation uit de API lezen.\n",
    "\n",
    "De informatie uit de URL https://data.hisparc.nl/api/station/22/ (informatie over\n",
    "meetstation 22, St. Ignatius, Adam) kunnen we ook met Station() opvragen:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "ffcf4227",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(Station(22).info)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "6a8e2a21",
   "metadata": {},
   "source": [
    "Een belangrijke eigenschap is de plaats waar een station zich bevindt.\n",
    "\n",
    "Deze informatie is te vinden via de API met de URL:\n",
    "https://data.hisparc.nl/show/source/gps/22/\n",
    "\n",
    "Via Station():"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "53e1bbdd",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(Station(22).gps_locations)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "41c21032",
   "metadata": {},
   "source": [
    "Dit zijn *alle* GPS posities van meetstation 22.\n",
    "\n",
    "We zien hier een array met daarin een aantal lijsten. Het eerste item op iedere\n",
    "regel legt vast vanaf welke moment, een GPS-tijdstempel, een station volgens de\n",
    "HiSPARC database op een plaats stond. Na dit GPS-tijdstempel zijn\n",
    "achtereenvolgens de lengte, breedte en hoogte van het station volgens GPS-84\n",
    "gegeven.\n",
    "([https://nl.wikipedia.org/wiki/WGS_84](https://nl.wikipedia.org/wiki/WGS_84))\n",
    "\n",
    "We kunnen ook de *huidige* locatie ophalen:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "2962831a",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sapphire.transformations.clock import datetime_to_gps\n",
    "from datetime import datetime\n",
    "\n",
    "ts = datetime_to_gps(datetime.now())\n",
    "print(\"timestamp: \", ts)\n",
    "print(\"GPS: \", Station(22).gps_location(ts))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "5487009d",
   "metadata": {},
   "source": [
    "# Opgave\n",
    "\n",
    "Bepaal de GPS coördinaten van station 22 op 5 December 2012:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "4eec987a",
   "metadata": {},
   "outputs": [],
   "source": [
    "ts = datetime_to_gps(datetime(2012, 12, 5))\n",
    "print(\"timestamp: \", ts)\n",
    "print(\"GPS: \", Station(22).gps_location(ts))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "0b1382f5",
   "metadata": {},
   "source": [
    "## Clusters via HiSPARCStations() en HiSPARCNetwork()\n",
    "\n",
    "### HiSPARCStations()\n",
    "\n",
    "Met behulp van HiSPARCStations() kunnen we een cluster van een aantal\n",
    "meetstations aanmaken. HiSPARCNetwork() maakt een cluster van alle meetstations\n",
    "in het netwerk."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "a51c2d15",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sapphire import HiSPARCStations, HiSPARCNetwork"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "9caf4645",
   "metadata": {},
   "source": [
    "We maken een cluster van vier meetstations:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "cae58e20",
   "metadata": {},
   "outputs": [],
   "source": [
    "stations = [301, 303, 304, 305]\n",
    "cluster = HiSPARCStations(stations)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "01cf3156",
   "metadata": {},
   "source": [
    "Van het cluster kunnen we posities van meetstations, detectoren, maar ook\n",
    "onderlinge afstanden e.d. eenvoudig bepalen."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "d8e0fc71",
   "metadata": {},
   "outputs": [],
   "source": [
    "for station in cluster.stations:\n",
    "    print(station.number, station.get_lla_coordinates())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7359a5b8",
   "metadata": {},
   "source": [
    "### set_timestamp()\n",
    "Een essentiele eigenschap van een cluster is dat we een tijdstempel kunnen\n",
    "opgeveven. De informatie die het cluster ophaalt en berekent is geldig op het\n",
    "tijdstip dat overeenkomt met het tijdstempel.\n",
    "\n",
    "In het geval van een coincidentie op timestamp 1368403200 willen we weten waar\n",
    "de stations op dat gegeven tijdstip stonden:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "0cb4aa89",
   "metadata": {},
   "outputs": [],
   "source": [
    "ts = 1368403200\n",
    "cluster.set_timestamp(ts)\n",
    "print(\"LLA coordinaten op timestamp = %d\\n\" % ts)\n",
    "for station in cluster.stations:\n",
    "    print(station.number, station.get_lla_coordinates())"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "1afb132d",
   "metadata": {},
   "source": [
    "# Opgave:\n",
    "\n",
    "In Alphen a/d Rijn staat een driehoek van stations: [3301, 3302, 3303]\n",
    "\n",
    "Maak een plot van de onderlinge ligging van de stations.\n",
    "\n",
    "*hint:* `get_coordinates()` geeft de (x, y, z, alpha)-coordinaten van een\n",
    "station in een cluster zoals HiSPARCStations(). (x,y,z) zijn in meter, alpha is\n",
    "de orientatiehoek in graden."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "c46b9c47",
   "metadata": {},
   "outputs": [],
   "source": [
    "import matplotlib.pyplot as plt\n",
    "%matplotlib notebook\n",
    "\n",
    "stations = [3301, 3302, 3303]\n",
    "cluster = HiSPARCStations(stations)\n",
    "\n",
    "for station in cluster.stations:\n",
    "    x, y, z, alpha = station.get_coordinates()\n",
    "    plt.plot(x, y, 'or', markersize=10)\n",
    "    plt.text(x, y, station.number)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "7c17138c",
   "metadata": {},
   "source": [
    "### HiSPARCNetwork()\n",
    "\n",
    "Een cluster van het hele network kunnen we maken met HiSPARCNetwork(). Omdat\n",
    "daarbij informatie van *alle* stations uit het netwerk via de API worden\n",
    "opgehaald, is het aanmaken van HiSPARCNetwork() redelijk traag.\n",
    "\n",
    "#### force_stale\n",
    "\n",
    "Door gebruik te maken van de optie `force_stale=True` wordt die informatie\n",
    "*niet* uit de API opgehaald, maar gelezen uit SAPPHiRE. Deze informatie is\n",
    "mogelijk veroudert, maar wel veel sneller beschikbaar.\n",
    "\n",
    "De optie 'force_stale' is ook beschikbaar voor Station(), Network() en\n",
    "HiSPARCStations()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "6a065ad0",
   "metadata": {},
   "outputs": [],
   "source": [
    "network = HiSPARCNetwork(force_stale=True)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "834ac652",
   "metadata": {},
   "source": [
    "De *UserWarnings* kunnen hier genegeerd worden."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "f554c8d3",
   "metadata": {},
   "outputs": [],
   "source": [
    "print(\"De afstand tussen 505 en 509 is %.f m\" % network.calc_distance_between_stations(505, 509))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "f4bed032",
   "metadata": {},
   "source": [
    "Ook voor het hele netwerk kan een timestamp worden opgegeven:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "94b9b36b",
   "metadata": {},
   "outputs": [],
   "source": [
    "network.set_timestamp(datetime_to_gps(datetime(2015,3, 4)))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "23d3180c",
   "metadata": {},
   "source": [
    "In combinatie met clusters maken we vaak gebruik van de functie\n",
    "`itertools.combinations`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "e528fca0",
   "metadata": {},
   "outputs": [],
   "source": [
    "stations = [3, 22, 501, 509]\n",
    "from itertools import combinations\n",
    "for eerste, tweede in combinations(stations, 2):\n",
    "    print(eerste, tweede)"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "32113f0d",
   "metadata": {},
   "source": [
    "# Opgave:\n",
    "\n",
    "Maak een lijst van de onderlinge afstanden van meetstation in cluster Leiden,\n",
    "waarvoor de afstand kleiner is dan 1000 m.\n"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "21530b75",
   "metadata": {},
   "outputs": [],
   "source": [
    "from sapphire import Network, HiSPARCStations\n",
    "stations = Network().station_numbers(cluster=2000)\n",
    "cluster = HiSPARCStations(stations, force_stale=True)\n",
    "for sn1, sn2 in combinations(stations, 2):\n",
    "    d = cluster.calc_distance_between_stations(sn1, sn2)\n",
    "    if d < 1000:\n",
    "        print(\"De afstand tussen station %d en %d is %.f m.\" % (sn1, sn2, d))"
   ]
  }
 ],
 "metadata": {
  "jupytext": {
   "cell_metadata_filter": "-all",
   "main_language": "python",
   "notebook_metadata_filter": "-all"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}