{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# viresclient API"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"> Author: Luca Mariani\n",
">\n",
"> Abstract: Describe the main classes and methods defined in the viresclient package"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Introduction\n",
"\n",
"The `viresclient` Python package allows to connect to the VirES server to download [Swarm](https://earth.esa.int/web/guest/missions/esa-operational-eo-missions/swarm) data and data calculated using magnetic models.\n",
"\n",
"Documentation:\n",
"\n",
"- https://viresclient.readthedocs.io/"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"\n",
"\n",
"## Contents\n",
"- [Access token configuration](#access_token_configuration)\n",
"- [Send requests to the server - SwarmRequest](#SwarmRequest)\n",
" - [Get the available collections](#SwarmRequest.available_collections)\n",
" - [Get the available measurements](#SwarmRequest.available_measurements)\n",
" - [Get the available auxiliaries](#SwarmRequest.available_auxiliaries)\n",
" - [Get the available magnetic models](#SwarmRequest.available_models)\n",
" - [Get information about one or mode models](#SwarmRequest.get_model_info)\n",
" - [Get the orbit number](#SwarmRequest.get_orbit_number)\n",
" - [Get times for orbits](#SwarmRequest.get_times_for_orbits)\n",
" - [Set collections](#SwarmRequest.set_collection)\n",
" - [Set products](#SwarmRequest.set_products)\n",
" - [Set/clear filters](#SwarmRequest.set_range_filter)\n",
" - [Send request to the server](#SwarmRequest.get_between)\n",
"- [Handle downloaded data - ReturnedData](#ReturnedData)\n",
" - [Get the list of source data](#ReturnedData.sources)\n",
" - [ReturnedData contents](#ReturnedData.contents)\n",
" - [Get type of downloaded data files](#ReturnedData.filetype)\n",
" - [Get list of magnetic models used during calculations](#ReturnedData.magnetic_models)\n",
" - [Get list of filters applied to the request](#ReturnedData.data_filters)\n",
" - [Convert ReturnedData to Pandas DataFrame](#ReturnedData.as_dataframe)\n",
" - [Convert ReturnedData to xarray Dataset](#ReturnedData.as_xarray)\n",
" - [Save downloaded data to a file](#ReturnedData.to_file)\n",
" - [Save downloaded data to multiple files](#ReturnedData.to_files)\n",
"- [Handle downloaded temporary file - ReturnedDataFile](#ReturnedDataFile)\n",
" - [Get NamedTemporaryFile associated to a ReturnedDataFile](#ReturnedDataFile._file)\n",
" - [Get type of the downloaded data file](#ReturnedDataFile.filetype)\n",
" - [Convert ReturnedDataFile to Pandas DataFrame](#ReturnedDataFile.as_dataframe)\n",
" - [Convert ReturnedDataFile to xarray Dataset](#ReturnedDataFile.as_xarray)\n",
" - [Save ReturnedDataFile object to a file](#ReturnedDataFile.to_file)\n",
"- [Handle viresclient configuration - ClientConfig](#ClientConfig)\n",
" - [Get path of the configuration file](#ClientConfig.path)\n",
" - [Get or set the default URL](#ClientConfig.default_url)\n",
" - [Set site configuration](#ClientConfig.set_site_config)\n",
" - [Get site configuration](#ClientConfig.get_site_config)\n",
" - [Save configuration](#ClientConfig.save)\n",
"- [Upload data to the server - DataUpload](#DataUpload)\n",
" - [Upload a file to the server](#DataUpload.post)\n",
" - [Get the identifier(s) of the uploaded file(s)](#DataUpload.ids)\n",
" - [Get info about the uploaded file](#DataUpload.get)\n",
" - [Set constant parameters to the uploaded file](#DataUpload.set_constant_parameters)\n",
" - [Get constant parameters](#DataUpload.get_constant_parameters)\n",
" - [Delete a specific uploaded file](#DataUpload.delete)\n",
" - [Delete the uploaded files](#DataUpload.clear)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"## Access token configuration\n",
"\n",
"Before using the client, you need to set the access token. This can be done using the `set_token()` function:\n",
"\n",
"```python\n",
"set_token(url='https://vires.services/ows', token=None, set_default=False)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **url** (*str*, optional): server URL (default value: `https://vires.services/ows`).\n",
"- **token** (*str*, optional): token string obtained from the VirES access token management page (https://vires.services/accounts/tokens/). If this parameter is not set, the user is prompted to enter its value interactively.\n",
"- **set_default** (*bool*, optional): if `True`, the server identified by *url* is configured as default server (default `False`)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Since the *url* parameter is by default set to the correct value you don't need to specify it. You can avoid also to specify the *token* parameter because this value will be asked interactively. Moreover, setting `set_default=True` you can configure this URL as default in the configuration file and avoid to specify it while sending requests to the server."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"First, the function must be imported from the `viresclient` package:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from viresclient import set_token"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Now it can be executed:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"set_token(set_default=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"## Send requests to the server - SwarmRequest\n",
"\n",
"The `SwarmRequest` object allows to create a request to the server and to download data according to the input parameters.\n",
"\n",
"```python\n",
"class SwarmRequest(url=None, username=None, password=None, token=None, config=None, logging_level='NO_LOGGING')\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **url** (*str*, optional): if not provided, the request will be sent to the default URL set in the `~/.viresclient.ini` configuration file.\n",
"- **username** (*str*, optional): username. The usage of username and password is deprecated and will be removed in future releases.\n",
"- **password** (*str*, optional): password. The usage of username and password is deprecated and will be removed in future releases.\n",
"- **token** (*str*, optional): token string obtained from the VirES access token management page (https://vires.services/accounts/tokens/). If this parameter is not specified and it is not set in the configuration file, the user is prompted to enter its value interactively.\n",
"- **config** (*str* or *ClientConfig*, optional): viresclient configuration. By default, it is read from `~/.viresclient.ini`.\n",
"- **logging_level** (*str*, optional): set the logging level. Allowed values are: `DEBUG`, `INFO`, `WARNING`, `ERROR`, `NO_LOGGING` (default)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's import the class from the `viresclient` package:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"from viresclient import SwarmRequest"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"After configuring the access token using the `set_token()` function (see [Access token configuration](#access_token_configuration)) , the `SwarmRequest` object can be created as follows:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"request = SwarmRequest()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The access token for the `https://vires.services/ows` default URL configured in the `~/.viresclient.ini` file is automatically retrieved:\n",
"\n",
"```ini\n",
"[https://vires.services/ows]\n",
"token = \n",
"\n",
"[default]\n",
"url = https://vires.services/ows\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If the default server URL is not configured (i.e. the \"default\" section is not present), you must specify the server URL:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"request = SwarmRequest('https://vires.services/ows')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`SwarmRequest` object has the following methods:\n",
"\n",
"- `SwarmRequest.available_collections()`\n",
"- `SwarmRequest.available_measurements()`\n",
"- `SwarmRequest.available_auxiliaries()`\n",
"- `SwarmRequest.available_models()`\n",
"- `SwarmRequest.get_model_info()`\n",
"- `SwarmRequest.get_orbit_number()`\n",
"- `SwarmRequest.get_times_for_orbits()`\n",
"- `SwarmRequest.set_collection()`\n",
"- `SwarmRequest.set_products()`\n",
"- `SwarmRequest.set_range_filter()`\n",
"- `SwarmRequest.clear_range_filter()`\n",
"- `SwarmRequest.get_between()`\n",
"\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Get the available collections\n",
"\n",
"Swarm data are organized in *collections*. Each collection is related to a Swarm file type (e.g. collection *SW_OPER_MAGA_LR_1B* is related to file type *MAGA_LR_1B*). The list of the available collections are provided invoking the `SwarmRequest.available_collections()` method.\n",
"\n",
"```python\n",
"SwarmRequest.available_collections(details=True)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **details** (*bool*, optional): if `True` (default), the method prints the list of all the available collections and the related details. If `False`, it returns the available collections as a *list*."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: print the list of the available collections and their details:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmRequest object (this step can be skipped if SwarmRequest has been already imported)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Create the request object\n",
"request = SwarmRequest()\n",
"\n",
"# Get the available collections\n",
"request.available_collections()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: get the list of available collections without the details:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"request.available_collections(details=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Get the available measurements\n",
"\n",
"It is possible to get the available measurements using the `SwarmRequest.available_measurements()` method:\n",
"\n",
"```python\n",
"SwarmRequest.available_measurements(collection=None)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **collection** (*str*, optional). If indicated, returns the available measurements for this collection as a *list*. It can be set to: `MAG`, `EFI`, `IBI`, `TEC`, `FAC`, `EEF`, `IPD` or one of the collections returned by the *available_collections* method. If not indicated, it returns the available measurements for all the collections as a *dict*."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: get the list of measurements for the `MAG` collections:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmRequest object (this step can be skipped if SwarmRequest has been already imported)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Create the request object\n",
"request = SwarmRequest()\n",
"\n",
"# Get the available measurements\n",
"request.available_measurements('MAG')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Setting `collection=SW_OPER_MAGA_LR_1B` we obtain the same result:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"request.available_measurements('SW_OPER_MAGA_LR_1B')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: get all the available measurements:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"request.available_measurements()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Get the available auxiliaries\n",
"\n",
"It is possible to get the available auxiliaries using the `SwarmRequest.available_auxiliaries()` method:\n",
"\n",
"```python\n",
"SwarmRequest.available_auxiliaries()\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This method does not accept input parameters and provides the available auxiliaries as a list."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: get the list of the available auxiliaries:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmRequest object (this step can be skipped if SwarmRequest has been already imported)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Create the request object\n",
"request = SwarmRequest()\n",
"\n",
"# Get the list of available auxiliaries\n",
"request.available_auxiliaries()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Get the available magnetic models\n",
"\n",
"The available magnetic models can be obtained using the `SwarmRequest.available_models()` method:\n",
"\n",
"```python\n",
"SwarmRequest.available_models(param=None, details=True, nice_output=True)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **param** (*str*, optional): it can be set to one of `F`, `C`, `D`, `MCO`, `MLI`, `MMA`, `MIO` to filter all the available magnetic models.\n",
"- **details** (*bool*, optional): if it is set to `True` (default), it gets the models and their details. If `False`, it returns only the models as a *list*.\n",
"- **nice_output** (*bool*, optional): if it is set to `True` (default), it prints the models and their details. If `False`, it returns the models as a dictionary (if `details=True`) or a s a list (if `details=False`).\n",
"\n",
"**Example**: get the list of all the available models and their details:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmRequest object (this step can be skipped if SwarmRequest has been already imported)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Create the request object\n",
"request = SwarmRequest()\n",
"\n",
"# Get available models\n",
"request.available_models()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: print the list of the available `F` (Fast-Track) models and their details:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"request.available_models('F')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: get the If the list of models of type `F` as a dictionary:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"request.available_models(param='F', nice_output=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: get the list of all the available models without their details:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"request.available_models(details=False)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Get information about one or more models.\n",
"\n",
"It is possible to get information about one or more specific models using the `SwarmRequest.get_model_info()` method:\n",
"\n",
"```python\n",
"SwarmRequest.get_model_info(models=None, custom_model=None, original_response=False)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **models** (*list[str]*, optional): models as a list of strings. If the list is not provided, returns information about all the available magnetic models.\n",
"- **custom_model** (*str*, optional): name of the file containing the spherical harmonics coefficients (custom model).\n",
"- **original_response** (*bool*, optional): if set to `False` (default), returns the result as a dictionary. If set to `True`, returns the result as a list of dictionaries.\n",
"\n",
"**Example**: get info about all the available magnetic models:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmRequest object (this step can be skipped if SwarmRequest has been already imported)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Create the request object\n",
"request = SwarmRequest()\n",
"\n",
"# Get model info\n",
"request.get_model_info()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: get info about CHAOS-Core and CHAOS-Static:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"request.get_model_info(models=['CHAOS-Core', 'CHAOS-Static'])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: set `original_response=True` to get the result as a list of dictionaries:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"request.get_model_info(models=['CHAOS-Core', 'CHAOS-Static'], original_response=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: get info on a custom model by providing the name of the file containing its coefficients:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Download shc file with WGET (command line tool)\n",
"!wget \"http://www.spacecenter.dk/files/magnetic-models/LCS-1/LCS-1.shc\"\n",
"\n",
"# Upload a .shc file and update the file name\n",
"request.get_model_info(custom_model='LCS-1.shc')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Delete downloaded file(s)\n",
"!rm LCS-1.shc*"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Get the orbit number\n",
"\n",
"The `SwarmRequest.get_orbit_number()` allows to get the orbit number of a given spacecraft providing date and time:\n",
"\n",
"```python\n",
"SwarmRequest.get_orbit_number(spacecraft, input_time)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **spacecraft** (*str*): spacecraft identifier: `A`, `B` or `C`.\n",
"- **input_time** (*datetime.datetime* or *str*): date and time."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: get orbit numbers corresponding to date 2020-01-01 00:00:00 for the three spacecrafts:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Create the request object\n",
"request = SwarmRequest()\n",
"\n",
"# Get results\n",
"for sc in ('A', 'B', 'C'):\n",
" orbit = request.get_orbit_number(sc, '2020-01-01T00:00:00')\n",
" print(f's/c {sc}: {orbit}')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Get times for orbits\n",
"\n",
"Get the time interval corresponding to a pair of orbit numbers using the `SwarmRequest.get_times_for_orbits()` method:\n",
"\n",
"```python\n",
"SwarmRequest.get_times_for_orbits(spacecraft, start_orbit, end_orbit)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **spacecraft** (*str*): spacecraft identifier: `A`, `B` or `C`.\n",
"- **start_orbit** (*int*): start orbit number\n",
"- **end_orbit** (*int*): end orbit number\n",
"\n",
"**Example**: get time intervals corresponding to *start_orbit* 1000 and *end_orbit* 2000 for the three spacecrafts:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmRequest object (this step can be skipped if SwarmRequest has been already imported)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Set start/end orbits\n",
"start_orbit = 1000\n",
"end_orbit = 2000\n",
"\n",
"# Create the request object\n",
"request = SwarmRequest()\n",
"\n",
"# Get results\n",
"for sc in ('A', 'B', 'C'):\n",
" start_date, end_date = request.get_times_for_orbits(sc, start_orbit, end_orbit)\n",
" print(f's/c {sc}: {start_date} - {end_date}')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Set collections\n",
"\n",
"Before sending the request to the server, you need to set the Swarm collection including the measurement(s) of interest. This can be done using the `SwarmRequest.set_collection()` method:\n",
"\n",
"```python\n",
"SwarmRequest.set_collection(*args)\n",
"```\n",
"\n",
"**Parameters**:\n",
"\n",
"- ***args** (*str*): one or more collections (see [Get available collections](#SwarmRequest.available_collections)) as a string."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: to get data from SW_OPER_MAGA_LR_1B and SW_OPER_EFIA_LP_1B collections:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmRequest object (this step can be skipped if SwarmRequest has been already imported)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Create the request object\n",
"request = SwarmRequest()\n",
"\n",
"# Set collections\n",
"request.set_collection('SW_OPER_MAGA_LR_1B', 'SW_OPER_EFIA_LP_1B')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Set products\n",
"\n",
"After setting the collection, you must set the combination of measurements and/or auxiliaries and/or magnetic model(s) data to retrieve. This can be done using the `SwarmRequest.set_products()` method:\n",
"\n",
"```python\n",
"SwarmRequest.set_products(measurements=None, models=None, custom_model=None, auxiliaries=None, residuals=False, sampling_step=None)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **measurements** (*list[str]*, optional): list of measurements to be downloaded. To get the list of the available measurements see [Available measurements](#SwarmRequest.available_measurements) (e.g.: `['F', 'B_NEC', 'Ne']`.).\n",
"- **models** (*list[str]*, optional): list of magnetic models. To get the list of the available models see [Available models](#SwarmRequest.available_models) (e.g.: `['CHAOS-Core', 'CHAOS-Static']`). In addition to the list, this parameters accepts also expression for the definition of magnetic models (e.g.: `'CHAOS = \"CHAOS-Core\" + \"CHAOS-Static\"'`).\n",
"- **custom_model** (*str*, optional): path to the the file containing the spherical harmonics coefficients of the custom model.\n",
"- **auxiliaries** (*list[str]*, optional): list of auxiliaries to be downloaded. To get the list of the available auxiliaries see [Available auxiliaries](#SwarmRequest.available_auxiliaries). Please note that the following parameters are always retrieved (i.e. they don't need to be specified): `Spacecraft`, `Timestamp`, `Latitude`, `Longitude`, `Radius`.\n",
"- **residuals** (*bool*, optional): if it is set to `True`, returns the residuals between measurements (specified with *measurements*) and models (specified with *models*). If it is set to `False` (default), returns measurements and models.\n",
"- **sampling_step** (*str*, optional): set the sampling step as an [ISO 8601 time interval](https://en.wikipedia.org/wiki/ISO_8601#Time_intervals). If not provided, data is returned with the original sampling.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: get measurements: `F` and `B_NEC` from `SW_OPER_MAGA_LR_1B` and `Ne` from `SW_OPER_EFIA_LP_1B`, model `CHAOS = \"CHAOS-Core\" + \"Chaos-Static\"` and auxiliary `OrbitNumber` with a sampling step of 10 seconds:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmRequest object (this step can be skipped if SwarmRequest has been already imported)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Create the request object\n",
"request = SwarmRequest()\n",
"\n",
"# Set collections\n",
"request.set_collection('SW_OPER_MAGA_LR_1B', 'SW_OPER_EFIA_LP_1B')\n",
"\n",
"# Set products\n",
"request.set_products(\n",
" measurements=['F', 'B_NEC', 'Ne'],\n",
" models=['CHAOS = \"CHAOS-Core\" + \"CHAOS-Static\"'],\n",
" auxiliaries=['OrbitNumber'],\n",
" residuals=False,\n",
" sampling_step='PT10S'\n",
")"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Set/clear filters\n",
"\n",
"Filter(s) can be applied to the requested measurements using the `SwarmRequest.set_range_filter()` method:\n",
"\n",
"```python\n",
"SwarmRequest.set_range_filter(parameter=None, minimum=None, maximum=None)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This method allows to set filter(s) in the form: $minimum \\le parameter \\le maximum$.\n",
"\n",
"**Parameters**:\n",
"\n",
"- **parameter** (*str*, optional): parameter to be used as a filter (e.g. `Latitude`)\n",
"- **minimum** (*float*, optional): allowed minimum value\n",
"- **maximum** (*float*, optional): allowed maximum value\n",
"\n",
"It is possible to apply multiple filters with consecutive calls to this method."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example:** to set filter: -20 <= `Longitude` <= 50 and 30 <= `Latitude` <= 70:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Create the request object\n",
"request = SwarmRequest()\n",
"\n",
"# Set collection\n",
"request.set_collection('SW_OPER_MAGA_LR_1B')\n",
"\n",
"# Set product\n",
"request.set_products(measurements=['F', 'B_NEC'])\n",
"\n",
"# Set filters\n",
"request.set_range_filter('Longitude', -20.0, 50.0)\n",
"request.set_range_filter('Latitude', 30.0, 70.0)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Filters can be removed using the `SwarmRequest.clear_range_filter()` method:\n",
"\n",
"```python\n",
"SwarmRequest.clear_range_filter()\n",
"```"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"request.clear_range_filter()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Send request to the server\n",
"\n",
"After setting collection(s), measurements, auxiliaries and models, we are ready to send the request to the server using the `SwarmRequest.get_between()` method:\n",
"\n",
"```python\n",
"SwarmRequest.get_between(start_time=None, end_time=None, filetype='cdf', asynchronous=True, show_progress=True, nrecords_limit=None, tmpdir=None)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **start_time** (*datetime.datetime* or *str*, optional): lower bound of temporal interval. If provided as string, it must be compliant to [ISO-8601](https://en.wikipedia.org/wiki/ISO_8601).\n",
"- **end_time** (*datetime.datetime* or *str*, optional): upper bound of temporal interval. If provided as string, it must be compliant to [ISO-8601](https://en.wikipedia.org/wiki/ISO_8601).\n",
"- **filetype** (*str*, optional): file format. Allowed values: `csv` and `cdf` (default).\n",
"- **asynchronous** (*bool*, optional): if `True` (default), set the asynchronous processing.\n",
"- **show_progress** (*bool*, optional): if `True` (default), enable the progress bar while processing and downloading data.\n",
"- **nrecords_limit** (*int*): overrides the limit of 3456000 records.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: download data according to the following inut parameters and get the list of source data files:\n",
"- measurements: F, B_NEC, Ne\n",
"- magnetic models: CHAOS = \"CHAOS-Core\" + \"Chaos-Static\"\n",
"- auxiliaries: OrbitNumber\n",
"- sampling step: 10 seconds\n",
"- time interval: [2019-10-01T00:00:00, 2019-10-01T00:00:00]\n",
"- file format: CDF\n",
"- filters: none"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Create the request object\n",
"request = SwarmRequest()\n",
"\n",
"# Set collections\n",
"request.set_collection('SW_OPER_MAGA_LR_1B', 'SW_OPER_EFIA_LP_1B')\n",
"\n",
"# Set products\n",
"request.set_products(\n",
" measurements=['F', 'B_NEC', 'Ne'],\n",
" models=['CHAOS = \"CHAOS-Core\" + \"CHAOS-Static\"'],\n",
" auxiliaries=['OrbitNumber'],\n",
" residuals=False,\n",
" sampling_step='PT10S'\n",
")\n",
"\n",
"# Get data\n",
"data = request.get_between('2019-10-01T00:00:00', '2019-10-01T01:00:00')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Data is returned as a `ReturnedData` object:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"type(data)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"## Handle downloaded data - ReturnedData\n",
"\n",
"Once the server receives the user's request, it creates the product, than this product is automatically downloaded and returned to the client as a `ReturnedData` object. Thus you don't need to create it by yourself. It is now possible to convert this data to a `pandas.DataFrame` object or to a `xarray.Dataset` object or to save it to one or more files.\n",
"\n",
"`ReturnedData` object has the following attributes:\n",
"\n",
"- `ReturnedData.sources`\n",
"- `ReturnedData.contents`\n",
"- `ReturnedData.file_types`\n",
"- `ReturnedData.magnetic_models`\n",
"- `ReturnedData.data_filters`\n",
"\n",
"and the following methods:\n",
"\n",
"- `ReturnedData.as_dataframe()`\n",
"- `ReturnedData.as_xarray()`\n",
"- `ReturnedData.to_file()`\n",
"- `ReturnedData.to_files()`\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Get the list of source data\n",
"\n",
"This attribute contains the list of source data files from which the values have been extracted."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: download data according to the following inut parameters and get the list of source data files:\n",
"- measurements: F, B_NEC, Ne\n",
"- magnetic models: CHAOS = \"CHAOS-Core\" + \"Chaos-Static\"\n",
"- auxiliaries: OrbitNumber\n",
"- sampling step: 10 seconds\n",
"- time interval: [2019-10-01T00:00:00, 2019-10-01T00:00:00]\n",
"- file format: CDF\n",
"- filters: none"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_MAGA_LR_1B', 'SW_OPER_EFIA_LP_1B')\n",
"request.set_products(\n",
" measurements=['F', 'B_NEC', 'Ne'],\n",
" models=['CHAOS = \"CHAOS-Core\" + \"CHAOS-Static\"'],\n",
" auxiliaries=['OrbitNumber'],\n",
" residuals=False,\n",
" sampling_step='PT10S'\n",
")\n",
"data = request.get_between('2019-10-01T00:00:00', '2019-10-01T01:00:00')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get list of sources\n",
"data.sources"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This is the list of the files from which the measurement and auxiliaries values and the magnetic models values have been retrieved."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### ReturnedData contents\n",
"\n",
"Downloaded data is saved to one or more temporary files represente as `ReturnedDataFile` objects. The `ReturnedData.contents` attribute contains the list of these objects."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: download data according to the following inut parameters and get the `ReturnedData` contents:\n",
"- measurements: F, B_NEC, Ne\n",
"- magnetic models: CHAOS = \"CHAOS-Core\" + \"Chaos-Static\"\n",
"- auxiliaries: OrbitNumber\n",
"- sampling step: 10 seconds\n",
"- time interval: [2019-10-01T00:00:00, 2019-10-01T00:00:00]\n",
"- file format: CDF\n",
"- filters: none"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_MAGA_LR_1B', 'SW_OPER_EFIA_LP_1B')\n",
"request.set_products(\n",
" measurements=['F', 'B_NEC', 'Ne'],\n",
" models=['CHAOS = \"CHAOS-Core\" + \"CHAOS-Static\"'],\n",
" auxiliaries=['OrbitNumber'],\n",
" residuals=False,\n",
" sampling_step='PT10S'\n",
")\n",
"data = request.get_between('2019-10-01T00:00:00', '2019-10-01T01:00:00')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get the ReturnedData contents\n",
"data.contents"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Get type of downloaded data files\n",
"\n",
"This attribute contains the type of downloaded files (i.e. `cdf` or `csv`)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: download data according to the following inut parameters and get the `ReturnedData` file type:\n",
"- measurements: F, B_NEC, Ne\n",
"- magnetic models: CHAOS = \"CHAOS-Core\" + \"Chaos-Static\"\n",
"- auxiliaries: OrbitNumber\n",
"- sampling step: 10 seconds\n",
"- time interval: [2019-10-01T00:00:00, 2019-10-01T00:00:00]\n",
"- file format: CDF\n",
"- filters: none"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_MAGA_LR_1B', 'SW_OPER_EFIA_LP_1B')\n",
"request.set_products(\n",
" measurements=['F', 'B_NEC', 'Ne'],\n",
" models=['CHAOS = \"CHAOS-Core\" + \"CHAOS-Static\"'],\n",
" auxiliaries=['OrbitNumber'],\n",
" residuals=False,\n",
" sampling_step='PT10S'\n",
")\n",
"data = request.get_between('2019-10-01T00:00:00', '2019-10-01T01:00:00')\n",
"\n",
"# Get ReturnedData file type\n",
"data.filetype"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Get list of magnetic models used during calculations\n",
"\n",
"This attribute contains the list of the magnetic models used during calculations."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: get F and B_NEC from SW_OPER_MAGA_LR_1B, Ne from SW_OPER_EFIA_LP_1B, model CHAOS = \"CHAOS-Core\" + \"Chaos-Static\" and auxiliary OrbitNumber with a sampling step of 10 seconds, between 2019-10-01T00:00:00 and 2019-10-01T01:00:00 in CDF format and get the list of magnetic models used during calculations:"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: download data according to the following inut parameters and get the list of magnetic models:\n",
"- measurements: F, B_NEC, Ne\n",
"- magnetic models: CHAOS = \"CHAOS-Core\" + \"Chaos-Static\"\n",
"- auxiliaries: OrbitNumber\n",
"- sampling step: 10 seconds\n",
"- time interval: [2019-10-01T00:00:00, 2019-10-01T01:00:00]\n",
"- file format: CDF\n",
"- filters: none"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_MAGA_LR_1B', 'SW_OPER_EFIA_LP_1B')\n",
"request.set_products(\n",
" measurements=['F', 'B_NEC', 'Ne'],\n",
" models=['CHAOS = \"CHAOS-Core\" + \"CHAOS-Static\"'],\n",
" auxiliaries=['OrbitNumber'],\n",
" residuals=False,\n",
" sampling_step='PT10S'\n",
")\n",
"data = request.get_between('2019-10-01T00:00:00', '2019-10-01T01:00:00')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get magnetic models\n",
"data.magnetic_models"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Get list of filters applied to the request\n",
"\n",
"This attribute contains the list of applied filters"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: download data according to the following inut parameters and get the list of the applied filters:\n",
"- measurements: F, B_NEC, Ne\n",
"- magnetic models: CHAOS = \"CHAOS-Core\" + \"Chaos-Static\"\n",
"- auxiliaries: OrbitNumber\n",
"- sampling step: 10 seconds\n",
"- time interval: [2019-10-01T00:00:00, 2019-10-02T00:00:00]\n",
"- file format: CDF\n",
"- filters:\n",
" - Longitude: $[-20.0, 50.0]$\n",
" - Latitude: $[30.0, 70.0]$"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_MAGA_LR_1B', 'SW_OPER_EFIA_LP_1B')\n",
"request.set_products(\n",
" measurements=['F', 'B_NEC', 'Ne'],\n",
" models=['CHAOS = \"CHAOS-Core\" + \"CHAOS-Static\"'],\n",
" auxiliaries=['OrbitNumber'],\n",
" residuals=False,\n",
" sampling_step='PT10S'\n",
")\n",
"request.set_range_filter('Longitude', -20.0, 50.0)\n",
"request.set_range_filter('Latitude', 30.0, 70.0)\n",
"data = request.get_between('2019-10-01T00:00:00', '2019-10-02T00:00:00')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get list of applied filters\n",
"data.data_filters"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Convert ReturnedData to Pandas DataFrame\n",
"\n",
"Data downloaded from the server can be converted to a `pandas.DataFrame` object. This is a general 2D labeled, size-mutable tabular structure with potentially heterogeneously-typed column allowing the user to directly access data. For more information:\n",
"\n",
"- https://pandas.pydata.org/pandas-docs/stable/\n",
"- https://pandas.pydata.org/pandas-docs/stable/reference/frame.html\n",
"\n",
"This conversion can be obtained with the `ReturnedData.as_dataframe()` method:\n",
"\n",
"```python\n",
"ReturnedData.as_dataframe(expand=False)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **expand** (*bool*, optional): If set to `False` (default), the vector parameters are represented as arrays (i.e. all the vector components in the same column). If this parameter is stet to `True`, the vector parameters are expanded (i.e. each component in a separate column). "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: download data according to the following inut parameters and convert the `ReturnedData` object to a pandas `DataFrame`:\n",
"- measurements: F, B_NEC, Ne\n",
"- magnetic models: CHAOS = \"CHAOS-Core\" + \"Chaos-Static\"\n",
"- auxiliaries: OrbitNumber\n",
"- sampling step: 10 seconds\n",
"- time interval: [2019-10-01T00:00:00, 2019-10-01T01:00:00]\n",
"- file format: CDF\n",
"- filters: none"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_MAGA_LR_1B', 'SW_OPER_EFIA_LP_1B')\n",
"request.set_products(\n",
" measurements=['F', 'B_NEC', 'Ne'],\n",
" models=['CHAOS = \"CHAOS-Core\" + \"CHAOS-Static\"'],\n",
" auxiliaries=['OrbitNumber'],\n",
" residuals=False,\n",
" sampling_step='PT10S'\n",
")\n",
"data = request.get_between('2019-10-01T00:00:00', '2019-10-01T01:00:00')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Convert ReturnedData to pandas DataFrame\n",
"df = data.as_dataframe()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can visualize the first 5 records using the `DataFrame.head()` method:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Setting `expand=True`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"df = data.as_dataframe(expand=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"each vector component is in a separate column:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"df.head()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Convert ReturnedData to xarray Dataset\n",
"\n",
"Data downloaded from the server can be converted to a `xarray.Dataset` object. This is a multi-dimentional array allowing the user to directly access data. For more information:\n",
"\n",
"- http://xarray.pydata.org/en/stable/\n",
"- http://xarray.pydata.org/en/stable/data-structures.html#dataset\n",
"\n",
"This conversion can be obtained with the `ReturnedData.as_xarray()`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: download data according to the following inut parameters and convert the `ReturnedData` object to an xarray `Dataset`:\n",
"- measurements: F, B_NEC, Ne\n",
"- magnetic models: CHAOS = \"CHAOS-Core\" + \"Chaos-Static\"\n",
"- auxiliaries: OrbitNumber\n",
"- sampling step: 10 seconds\n",
"- time interval: [2019-10-01T00:00:00, 2019-10-01T01:00:00]\n",
"- file format: CDF\n",
"- filters: none"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_MAGA_LR_1B', 'SW_OPER_EFIA_LP_1B')\n",
"request.set_products(\n",
" measurements=['F', 'B_NEC', 'Ne'],\n",
" models=['CHAOS = \"CHAOS-Core\" + \"CHAOS-Static\"'],\n",
" auxiliaries=['OrbitNumber'],\n",
" residuals=False,\n",
" sampling_step='PT10S'\n",
")\n",
"data = request.get_between('2019-10-01T00:00:00', '2019-10-01T01:00:00')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Convert ReturnedData to pandas DataFrame\n",
"ds = data.as_xarray()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"You can se how the xarray `Dataset` is represented:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"ds"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Save downloaded data to a file\n",
"\n",
"Data downloaded from the server can be saved to a file using the `ReturnedData.to_file()` method:\n",
"\n",
"```python\n",
"ReturnedData.to_file(path, overwrite=False)\n",
"```\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **path** (*str*): output file path.\n",
"- **overwrite** (*bool*, optional):if `True` allows to overwrite the file if already present at *path*.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: download data according to the following inut parameters and save the `ReturnedData` object to a file:\n",
"- measurements: F, B_NEC, Ne\n",
"- magnetic models: CHAOS = \"CHAOS-Core\" + \"Chaos-Static\"\n",
"- auxiliaries: OrbitNumber\n",
"- sampling step: 10 seconds\n",
"- time interval: [2019-10-01T00:00:00, 2019-10-01T01:00:00]\n",
"- file format: CDF\n",
"- filters: none"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_MAGA_LR_1B', 'SW_OPER_EFIA_LP_1B')\n",
"request.set_products(\n",
" measurements=['F', 'B_NEC', 'Ne'],\n",
" models=['CHAOS = \"CHAOS-Core\" + \"CHAOS-Static\"'],\n",
" auxiliaries=['OrbitNumber'],\n",
" residuals=False,\n",
" sampling_step='PT10S'\n",
")\n",
"data = request.get_between('2019-10-01T00:00:00', '2019-10-01T01:00:00')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Save ReturnedData to a file\n",
"data.to_file('out.cdf', overwrite=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Note**: this method can be used only if the amount of downloaded data is small (i.e. if the request is not split between multiple requests)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: download data according to the following inut parameters and save the `ReturnedData` object to a file:\n",
"- measurements: U_orbit, Ne, Te, Vs\n",
"- magnetic models: none\n",
"- auxiliaries: none\n",
"- sampling step: default\n",
"- time interval: [2019-10-01T00:00:00, 2019-11-01T00:00:00]\n",
"- file format: CDF\n",
"- filters: none"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_EFIA_LP_1B')\n",
"request.set_products(\n",
" measurements=['U_orbit', 'Ne', 'Te', 'Vs'],\n",
")\n",
"data = request.get_between('2019-10-01T00:00:00', '2019-11-01T00:00:00')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"The request is split between multiple requests. Try to execute the cell below:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# data.to_file('huge.cdf', True)\n",
"## will return: \n",
"## NotImplementedError: Data is split into multiple files. Use .to_files instead"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Remove saved files (if any)\n",
"!rm *.cdf"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Save downloaded data to multiple files\n",
"\n",
"Data downloaded from the server can be saved to one or more files using the `ReturnedData.to_files()` method:\n",
"\n",
"```python\n",
"ReturnedData.to_files(paths, overwrite=False)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **paths** (*list[str]*): output files path as a list of strings.\n",
"- **overwrite** (*bool*, optional):if `True` allows to overwrite the file if already present at *path*.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: download data according to the following inut parameters and save the `ReturnedData` object a file using the `ReturnedData.to_files()` method:\n",
"- measurements: F, B_NEC, Ne\n",
"- magnetic models: CHAOS = \"CHAOS-Core\" + \"Chaos-Static\"\n",
"- auxiliaries: OrbitNumber\n",
"- sampling step: 10 seconds\n",
"- time interval: [2019-10-01T00:00:00, 2019-10-01T01:00:00]\n",
"- file format: CDF\n",
"- filters: none"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_MAGA_LR_1B', 'SW_OPER_EFIA_LP_1B')\n",
"request.set_products(\n",
" measurements=['F', 'B_NEC', 'Ne'],\n",
" models=['CHAOS = \"CHAOS-Core\" + \"CHAOS-Static\"'],\n",
" auxiliaries=['OrbitNumber'],\n",
" residuals=False,\n",
" sampling_step='PT10S'\n",
")\n",
"data = request.get_between('2019-10-01T00:00:00', '2019-10-01T01:00:00')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Save ReturnedData to a file\n",
"data.to_files(['out.cdf'], overwrite=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This method is very useful in case the request has been split between multiple requests"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: download data according to the following inut parameters and save the `ReturnedData` object to a files:\n",
"- measurements: U_orbit, Ne, Te, Vs\n",
"- magnetic models: none\n",
"- auxiliaries: none\n",
"- sampling step: default\n",
"- time interval: [2019-10-01T00:00:00, 2019-11-01T00:00:00]\n",
"- file format: CDF\n",
"- filters: none"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_EFIA_LP_1B')\n",
"request.set_products(\n",
" measurements=['U_orbit', 'Ne', 'Te', 'Vs'],\n",
")\n",
"data = request.get_between('2019-10-01T00:00:00', '2019-11-01T00:00:00')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Save ReturnedData to files\n",
"data.to_files(['first.cdf', 'second.cdf'], overwrite=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Note**: the number of files to be specified must be equal to the number of files indicated by the `ReturnedData.contents` attribute. In the above case:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"data.contents"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Remove saved files (if any)\n",
"!rm *.cdf"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"## Handle downloaded temporary data file - ReturnedDataFile\n",
"\n",
"This object holds the file downloaded from the server. Even if data has not ben saved to a file with the `ReturnedData.to_files()`, it is stored in a temporary file and automatically deleted when not needed anymore. As indicated in the \"[ReturnedData contents](#ReturnedData.contents)\" section, you can get the list of the returned data files using the `ReturnedData.contents` attribute. Thus, you don't need to create this object by yourself.\n",
"\n",
"**Note**: the description of this object has been included for completeness only. You won't need to use this object and its methods directly. To handle the downloaded data is preferable to use the [ReturnedData](#ReturnedData) object. "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`ReturnedDataFile` object has the following attributes:\n",
"\n",
"- `ReturnedDataFile._file`\n",
"- `ReturnedDataFile.filetype`\n",
"\n",
"and the following methods:\n",
"\n",
"- `ReturnedDataFile.as_dataframe()`\n",
"- `ReturnedDataFile.as_xarray()`\n",
"- `ReturnedDataFile.to_file()`"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Get the NamedTemporaryFile associated to a ReturnedDataFile\n",
"\n",
"The `NamedTemporaryFile` corresponding to the `ReturnedDataFile` is contained in the `ReturnedDataFile._file` attribute."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: download data according to the following inut parameters and get the NamedTemporaryFiles objects associated to ReturnedDataFiles:\n",
"- measurements: F, B_NEC, Ne\n",
"- magnetic models: CHAOS = \"CHAOS-Core\" + \"Chaos-Static\"\n",
"- auxiliaries: OrbitNumber\n",
"- sampling step: 10 seconds\n",
"- time interval: [2019-10-01T00:00:00, 2019-10-01T01:00:00]\n",
"- file format: CDF\n",
"- filters: none"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_MAGA_LR_1B', 'SW_OPER_EFIA_LP_1B')\n",
"request.set_products(\n",
" measurements=['F', 'B_NEC', 'Ne'],\n",
" models=['CHAOS = \"CHAOS-Core\" + \"CHAOS-Static\"'],\n",
" auxiliaries=['OrbitNumber'],\n",
" residuals=False,\n",
" sampling_step='PT10S'\n",
")\n",
"data = request.get_between('2019-10-01T00:00:00', '2019-10-01T01:00:00')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get ReturnedData contents\n",
"data.contents"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`ReturnedData` contains only one `ReturnedDataFile`. Let's get the associated `NamedTemporaryFile`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"data.contents[0]._file"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`tempfile.NamedTemporaryFile` is part of the Python standard library. For more information see: https://docs.python.org/3/library/tempfile.html#tempfile.NamedTemporaryFile"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Get type of the downloaded data file\n",
"\n",
"This attribute contains the type of downloaded files (i.e. `cdf`, `csv` or `nc`)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: download data according to the following inut parameters and get the file type of the `ReturnedDataFile`:\n",
"- measurements: F, B_NEC, Ne\n",
"- magnetic models: CHAOS = \"CHAOS-Core\" + \"Chaos-Static\"\n",
"- auxiliaries: OrbitNumber\n",
"- sampling step: 10 seconds\n",
"- time interval: [2019-10-01T00:00:00, 2019-10-01T01:00:00]\n",
"- file format: CDF\n",
"- filters: none"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_MAGA_LR_1B', 'SW_OPER_EFIA_LP_1B')\n",
"request.set_products(\n",
" measurements=['F', 'B_NEC', 'Ne'],\n",
" models=['CHAOS = \"CHAOS-Core\" + \"CHAOS-Static\"'],\n",
" auxiliaries=['OrbitNumber'],\n",
" residuals=False,\n",
" sampling_step='PT10S'\n",
")\n",
"data = request.get_between('2019-10-01T00:00:00', '2019-10-01T01:00:00')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get ReturnedData contents\n",
"data.contents"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Get file type\n",
"data.contents[0].filetype"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Convert ReturnedDataFile to Pandas Dataframe\n",
"\n",
"As for the `ReturnedData` object (see [Convert ReturnedData to Pandas DataFrame](#ReturnedData.as_dataframe)), the `ReturnedDataFile` object can be converted to a Pandas DataFrame using the `ReturnedDataFile.as_dataframe()` method:\n",
"\n",
"```python\n",
"ReturnedDataFile.as_dataframe(expand=False)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **expand** (*bool*, optional): If set to `False` (default), the vector parameters are represented as arrays (i.e. all the vector components in the same column). If this parameter is stet to `True`, the vector parameters are expanded (i.e. each component in a separate column). "
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Convert ReturnedDataFile to xarray Dataset\n",
"\n",
"As for the `ReturnedData` object (see [Convert ReturnedData to xarray Dataset](#ReturnedData.as_xarray)), the `ReturnedDataFile` object can be converted to an xarray Dataset using the `ReturnedDataFile.as_xarray()` method:\n",
"\n",
"```python\n",
"ReturnedDataFile.as_xarray()\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Save ReturnedDataFile object to a file\n",
"\n",
"Data stored in the `ReturnedDataFile` object can be saved to a file with the `ReturnedDataFile.to_file()` method:\n",
"\n",
"```python\n",
"ReturnedDataFile.to_file(path, overwrite=False)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **path** (*str*): output file path.\n",
"- **overwrite** (*bool*, optional):if `True` allows to overwrite the file if already present at *path*.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: download data according to the following inut parameters and get the file type of the `ReturnedDataFile`:\n",
"- measurements: F, B_NEC, Ne\n",
"- magnetic models: CHAOS = \"CHAOS-Core\" + \"Chaos-Static\"\n",
"- auxiliaries: OrbitNumber\n",
"- sampling step: 10 seconds\n",
"- time interval: [2019-10-01T00:00:00, 2019-10-01T01:00:00]\n",
"- file format: CDF\n",
"- filters: none"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_MAGA_LR_1B', 'SW_OPER_EFIA_LP_1B')\n",
"request.set_products(\n",
" measurements=['F', 'B_NEC', 'Ne'],\n",
" models=['CHAOS = \"CHAOS-Core\" + \"CHAOS-Static\"'],\n",
" auxiliaries=['OrbitNumber'],\n",
" residuals=False,\n",
" sampling_step='PT10S'\n",
")\n",
"data = request.get_between('2019-10-01T00:00:00', '2019-10-01T01:00:00')"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Save the ReturnedDataFile to a file\n",
"data.contents[0].to_file('out.cdf', overwrite=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Remove saved files (if any)\n",
"!rm out.cdf"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"## Handle viresclient configuration - ClientConfig\n",
"\n",
"You can acces the `viresclient` configuration using the `ClientConfig` class:\n",
"\n",
"```python\n",
"class viresclient.ClientConfig(path=None)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"- **path** (*str*, optional): path of the configuration file. If not specified, the default configuration file is assumed: `~/.viresclient.ini`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: create the `ClientConfig` object associated to the default configuration file:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import the ClientConfig\n",
"from viresclient import ClientConfig\n",
"\n",
"# Create the ClientConfig object\n",
"default = ClientConfig()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`ClientConfig` object has the following attributes:\n",
"\n",
"- `ClientConfig.path`\n",
"- `ClientConfig.default_url`\n",
"\n",
"and the following methods:\n",
"\n",
"- `ClientConfig.set_site_config()`\n",
"- `ClientConfig.get_site_config()`\n",
"- `ClientConfig.save()`"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Get path of the configuration file\n",
"\n",
"The `ClientConfig.path` read-only attribute contains the path of the configuration file."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: create the ClientConfig object associated to the default configuration file and check its path:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import the ClientConfig (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import ClientConfig\n",
"\n",
"# Create the ClientConfig object\n",
"default = ClientConfig()\n",
"\n",
"default.path"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Get or set the default URL\n",
"\n",
"The `ClientConfig.default_url` attribute contains server's default URL (i.e. the one used when `SwarmClient` class is invoked without URL)."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: create the ClientConfig object associated to the default configuration file and check the default URL:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import the ClientConfig (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import ClientConfig\n",
"\n",
"# Create the ClientConfig object\n",
"default = ClientConfig()\n",
"\n",
"default.default_url"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If the default URL is not set, the attribute returns `None`."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: create a new configuration:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import the ClientConfig (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import ClientConfig\n",
"\n",
"# Create new configuration\n",
"newcfg = ClientConfig('newfile.ini')\n",
"\n",
"# Print default URL \n",
"print(newcfg.default_url)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Set default URL to: `https://vires.services/ows`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"newcfg.default_url = 'https://vires.services/ows'"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Get the updated result:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Print default URL \n",
"print(newcfg.default_url)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Set site configuration\n",
"\n",
"It is possible to set the configuration for a server identified by an URL with the `Client.Config.set_site_config()` method:\n",
"\n",
"```python\n",
"ClientConfig.set_site_config(url, **options)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **url** (*str*): server URL\n",
"- ****options** (*str*): configuration options in the form: *key*=*value* (e.g.: token='...')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: create a new configuration, set default URL to: `https://vires.services/ows` and set the access token for this URL:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import the ClientConfig (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import ClientConfig\n",
"\n",
"# Create new configuration\n",
"newcfg = ClientConfig('newfile.ini')\n",
"\n",
"# Set default URL \n",
"newcfg.default_url = 'https://vires.services/ows'\n",
"\n",
"# Set the access token\n",
"newcfg.set_site_config(newcfg.default_url, token='mytokenstring')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Get site configuration\n",
"\n",
"It is possible to get the configuration for a server identified by an URL with the `ClientConfig.get_site_config()` method:\n",
"\n",
"```python\n",
"ClientConfig.get_site_config(url)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **url** (*str*): server URL"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: create a new configuration, set default URL to: `https://vires.services/ows`, set the access token for this URL and get the configuration for `https://vires.services/ows`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import the ClientConfig (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import ClientConfig\n",
"\n",
"# Create new configuration\n",
"newcfg = ClientConfig('newfile.ini')\n",
"\n",
"# Set default URL \n",
"newcfg.default_url = 'https://vires.services/ows'\n",
"\n",
"# Set the access token\n",
"newcfg.set_site_config(newcfg.default_url, token='mytokenstring')\n",
"\n",
"# Get the configuration\n",
"newcfg.get_site_config('https://vires.services/ows')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Save configuration\n",
"\n",
"The configuration stored in the `ClientConfig` object can be saved using the `ClientConfig.save()` method:\n",
"\n",
"```python\n",
"ClientConfig.save()\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"This method saves the configuration to the path specified during the `ClientConfig` creation. You can check this value via the `ClientConfig.path` attribute."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: create a new configuration, set default URL to: `https://vires.services/ows`, set the access token for this URL and save the configuration to file:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import the ClientConfig (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import ClientConfig\n",
"\n",
"# Create new configuration\n",
"newcfg = ClientConfig('newfile.ini')\n",
"\n",
"# Set default URL \n",
"newcfg.default_url = 'https://vires.services/ows'\n",
"\n",
"# Set the access token\n",
"newcfg.set_site_config(newcfg.default_url, token='mytokenstring')\n",
"\n",
"# Save the configuration\n",
"newcfg.save()"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Let's have a look to the new configuration file:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"!cat newfile.ini"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# delete newfile.ini\n",
"!rm newfile.ini"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"## Upload data to the server - DataUpload\n",
"\n",
"You can upload your data to the server to view it in the VirES web interface. File format can be CDF or CSV and must be compliant to: https://github.com/ESA-VirES/VirES-Server/blob/master/vires/custom_data_format_description.md.\n",
"\n",
"Data can be uploaded using the `DataUpload` object:\n",
"\n",
"```python\n",
"class DataUpload(url, token, **kwargs)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **url** (*str*): server URL\n",
"- **token** (*str*): access token\n",
"- ****kwargs** (*str*): additional parameters (currently not used)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"`DataUpload` object has the following attributes:\n",
"\n",
"- `DataUpload.ids`\n",
"\n",
"and the following methods:\n",
"\n",
"- `DataUpload.post()`\n",
"- `DataUpload.get()`\n",
"- `DataUpload.set_constant_parameters()`\n",
"- `DataUpload.get_constant_parameters()`"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: create a `DataUpload` object for data upload to the default server. You can retrieve the default URL and the access token from the configuration, using the `ClientConfig` object:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import DataUpload object\n",
"from viresclient import DataUpload\n",
"\n",
"# Import ClientConfig object (this step can be avoided if ClientConfig has been already imported)\n",
"from viresclient import ClientConfig\n",
"\n",
"# Create ClientConfig object associated to the default configuration file: ~/.viresclient.ini\n",
"default = ClientConfig()\n",
"\n",
"# Get default URL and access token from the configuration\n",
"url = default.default_url\n",
"token = default.get_site_config(url)['token']\n",
"\n",
"# Create the DataUpload object:\n",
"du = DataUpload(url, token)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Upload a file to the server\n",
"\n",
"You can upload a file to the server using the `DataUpload.post()` method:\n",
"\n",
"```python\n",
"DataUpload.post(file, filename=None)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **file** (*str*): file to be uploaded\n",
"\n",
"The method returns the info about the uploaded file as a dictionary."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: upload a product to the server:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download 1 hour of MAGA_LR_1B data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_MAGA_LR_1B')\n",
"request.set_products(measurements=['F', 'B_NEC'])\n",
"data = request.get_between('2020-01-01T00:00:00', '2020-01-01T01:00:00')\n",
"data.to_file('out.cdf', overwrite=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import ClientConfig and DataUpload (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import ClientConfig, DataUpload\n",
"\n",
"# Create ClientConfig object associated to the default configuration file: ~/.viresclient.ini\n",
"default = ClientConfig()\n",
"\n",
"# Get default URL and access token from the configuration\n",
"url = default.default_url\n",
"token = default.get_site_config(url)['token']\n",
"\n",
"# Create the DataUpload object:\n",
"du = DataUpload(url, token)\n",
"\n",
"# Upload the file to the server to be visualized in the web client\n",
"info = du.post('out.cdf')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Check info about the uploaded file:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"info"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Delete test file (if any)\n",
"!rm out.cdf"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Get the identifier(s) of the uploaded file(s)\n",
"\n",
"You can obtain the identifiers of the uploaded files via the `DataUpload.ids` attribute as a list. Please note that currently the server accepts only one file at a time, thus the returned list will have length 1."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: upload a product to the server and get its identifier:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download 1 hour of MAGA_LR_1B data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_MAGA_LR_1B')\n",
"request.set_products(measurements=['F', 'B_NEC'])\n",
"data = request.get_between('2020-01-01T00:00:00', '2020-01-01T01:00:00')\n",
"data.to_file('out.cdf', overwrite=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import ClientConfig and DataUpload (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import ClientConfig, DataUpload\n",
"\n",
"# Create ClientConfig object associated to the default configuration file: ~/.viresclient.ini\n",
"default = ClientConfig()\n",
"\n",
"# Get default URL and access token from the configuration\n",
"url = default.default_url\n",
"token = default.get_site_config(url)['token']\n",
"\n",
"# Create the DataUpload object:\n",
"du = DataUpload(url, token)\n",
"\n",
"# Upload the file to the server to be visualized in the web client\n",
"info = du.post('out.cdf')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Get id of the uploaded file:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"du.ids"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Delete test file (if any)\n",
"!rm out.cdf"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Get info about the uploaded file\n",
"\n",
"You can get the info of the uploaded file using the `DataUpload.get()` method:\n",
"\n",
"```python\n",
"DataUpload.get(identifier=None)\n",
"```\n",
"\n",
"**Parameters**:\n",
"\n",
"- **identifier** (*str*, optional): identifier of the uploaded file obtained via `DataUpload.ids` attribute (see [Get the identifier(s) of the uploaded file(s)](#DataUpload.ids)) or from the info returned by the `DataUpload.post()` method (see [Upload a file to the server](#DataUpload.post)). If not provided, returns the info af all the uploaded files as a list."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: upload a product to the server and get product's info with `DataUpload.get()`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download 1 hour of MAGA_LR_1B data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_MAGA_LR_1B')\n",
"request.set_products(measurements=['F', 'B_NEC'])\n",
"data = request.get_between('2020-01-01T00:00:00', '2020-01-01T01:00:00')\n",
"data.to_file('out.cdf', overwrite=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import ClientConfig and DataUpload (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import ClientConfig, DataUpload\n",
"\n",
"# Create ClientConfig object associated to the default configuration file: ~/.viresclient.ini\n",
"default = ClientConfig()\n",
"\n",
"# Get default URL and access token from the configuration\n",
"url = default.default_url\n",
"token = default.get_site_config(url)['token']\n",
"\n",
"# Create the DataUpload object:\n",
"du = DataUpload(url, token)\n",
"\n",
"# Upload the file to the server to be visualized in the web client\n",
"info = du.post('out.cdf')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Get info about the uploaded file:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"du.get(du.ids[0])"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If the identifier is not provided, you will get info about all the files as a list:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"du.get()"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Delete test file (if any)\n",
"!rm out.cdf"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Set constant parameters to the uploaded file\n",
"\n",
"It is possible to set constant parameters to the uploaded file using the `DataUpload.set_constant_parameters()` method:\n",
"\n",
"```python\n",
"DataUpload.set_constant_parameters(identifier, parameters, replace=False)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **identifier** (*str*): file identifier.\n",
"- **parameters** (*dict*): constant parameters provided as a dictionary\n",
"- **replace** (*bool*, optional): if set to `True`, all the parameters will be replaced by the new parameters, otherwise the new parameters will update the existing ones (default behaviour).\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: upload a product to the server and set constant parameters to the uploaded file:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download 1 hour of MAGA_LR_1B data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_MAGA_LR_1B')\n",
"request.set_products(measurements=['F', 'B_NEC'])\n",
"data = request.get_between('2020-01-01T00:00:00', '2020-01-01T01:00:00')\n",
"data.to_file('out.cdf', overwrite=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import ClientConfig and DataUpload (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import ClientConfig, DataUpload\n",
"\n",
"# Create ClientConfig object associated to the default configuration file: ~/.viresclient.ini\n",
"default = ClientConfig()\n",
"\n",
"# Get default URL and access token from the configuration\n",
"url = default.default_url\n",
"token = default.get_site_config(url)['token']\n",
"\n",
"# Create the DataUpload object:\n",
"du = DataUpload(url, token)\n",
"\n",
"# Upload the file to the server to be visualized in the web client\n",
"info = du.post('out.cdf')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Assign constant parameters: $param1 = 12345$ and $param2 = 34567$:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"du.set_constant_parameters(du.ids[0], {'param1': 12345, 'param2': 34567})"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"If you want to set `param1` to a new value you can update the existing set of parameters:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"du.set_constant_parameters(du.ids[0], {'param1': 1})"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"or replace the entire set of parameters:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"du.set_constant_parameters(du.ids[0], {'param1': 1}, replace=True)"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Note that `param2` has been removed."
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Delete test file (if any)\n",
"!rm out.cdf"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Get constant parameters applied to the uploaded file\n",
"\n",
"It is possible to get the list of constant parameters applied to the uploaded file using the `DataUpload.get_constant_parameters()` method:\n",
"\n",
"```python\n",
"DataUpload.get_constant_parameters(identifier)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **identifier** (*str*): file identifier.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: upload a product to the server and get constant parameters applied to the uploaded file:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download 1 hour of MAGA_LR_1B data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_MAGA_LR_1B')\n",
"request.set_products(measurements=['F', 'B_NEC'])\n",
"data = request.get_between('2020-01-01T00:00:00', '2020-01-01T01:00:00')\n",
"data.to_file('out.cdf', overwrite=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import ClientConfig and DataUpload (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import ClientConfig, DataUpload\n",
"\n",
"# Create ClientConfig object associated to the default configuration file: ~/.viresclient.ini\n",
"default = ClientConfig()\n",
"\n",
"# Get default URL and access token from the configuration\n",
"url = default.default_url\n",
"token = default.get_site_config(url)['token']\n",
"\n",
"# Create the DataUpload object:\n",
"du = DataUpload(url, token)\n",
"\n",
"# Upload the file to the server to be visualized in the web client\n",
"info = du.post('out.cdf')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Assign constant parameters: $param1 = 12345$ and $param2 = 34567$:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"du.set_constant_parameters(du.ids[0], {'param1': 12345, 'param2': 34567})"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Get the list of constant parameters:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"du.get_constant_parameters(du.ids[0])"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Delete test file (if any)\n",
"!rm out.cdf"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Delete a specific uploaded file\n",
"\n",
"You can delete a specific uploaded file using the `DataUpload.delete()` method:\n",
"\n",
"```python\n",
"DataUpload.delete(identifier)\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Parameters**:\n",
"\n",
"- **identifier** (*str*): identifier of the uploaded file obtained via `DataUpload.ids` attribute (see [Get the identifier(s) of the uploaded file(s)](#DataUpload.ids)) or from the info returned by the `DataUpload.post()` method (see [Upload a file to the server](#DataUpload.post)). If not provided, returns the info af all the uploaded files as a list.\n"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: upload a product to the server and delete it with `DataUpload.delete()`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download 1 hour of MAGA_LR_1B data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_MAGA_LR_1B')\n",
"request.set_products(measurements=['F', 'B_NEC'])\n",
"data = request.get_between('2020-01-01T00:00:00', '2020-01-01T01:00:00')\n",
"data.to_file('out.cdf', overwrite=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import ClientConfig and DataUpload (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import ClientConfig, DataUpload\n",
"\n",
"# Create ClientConfig object associated to the default configuration file: ~/.viresclient.ini\n",
"default = ClientConfig()\n",
"\n",
"# Get default URL and access token from the configuration\n",
"url = default.default_url\n",
"token = default.get_site_config(url)['token']\n",
"\n",
"# Create the DataUpload object:\n",
"du = DataUpload(url, token)\n",
"\n",
"# Upload the file to the server to be visualized in the web client\n",
"info = du.post('out.cdf')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Delete the uploaded product:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"du.delete(du.ids[0])\n",
"\n",
"du.ids"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Delete test file (if any)\n",
"!rm out.cdf"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"[TOP](#top)\n",
"\n",
"\n",
"\n",
"### Delete the uploaded files\n",
"\n",
"You can delete *all* the uploaded files using the `DataUpload.clear()` method:\n",
"\n",
"```python\n",
"DataUpload.clear()\n",
"```"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"**Example**: upload a product to the server and delete it with `DataUpload.clear()`:"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import SwarmClient (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import SwarmRequest\n",
"\n",
"# Download 1 hour of MAGA_LR_1B data\n",
"request = SwarmRequest()\n",
"request.set_collection('SW_OPER_MAGA_LR_1B')\n",
"request.set_products(measurements=['F', 'B_NEC'])\n",
"data = request.get_between('2020-01-01T00:00:00', '2020-01-01T01:00:00')\n",
"data.to_file('out.cdf', overwrite=True)"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"# Import ClientConfig and DataUpload (this step can be omitted if already executed in the previous examples)\n",
"from viresclient import ClientConfig, DataUpload\n",
"\n",
"# Create ClientConfig object associated to the default configuration file: ~/.viresclient.ini\n",
"default = ClientConfig()\n",
"\n",
"# Get default URL and access token from the configuration\n",
"url = default.default_url\n",
"token = default.get_site_config(url)['token']\n",
"\n",
"# Create the DataUpload object:\n",
"du = DataUpload(url, token)\n",
"\n",
"# Upload the file to the server to be visualized in the web client\n",
"info = du.post('out.cdf')"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Delete the uploaded product(s):"
]
},
{
"cell_type": "code",
"execution_count": null,
"metadata": {},
"outputs": [],
"source": [
"du.clear()\n",
"\n",
"du.ids"
]
}
],
"metadata": {
"kernelspec": {
"display_name": "Python 3 (ipykernel)",
"language": "python",
"name": "python3"
},
"language_info": {
"codemirror_mode": {
"name": "ipython",
"version": 3
},
"file_extension": ".py",
"mimetype": "text/x-python",
"name": "python",
"nbconvert_exporter": "python",
"pygments_lexer": "ipython3"
}
},
"nbformat": 4,
"nbformat_minor": 4
}