{
"cells": [
{
"cell_type": "markdown",
"metadata": {},
"source": [
"# Step 2 - Cleaning step and sleep data and looking for trends\n",
"\n",
"This notebook explains how Fitbit data may be cleaned by removing clear outliers. We will furthermore look for trends in the step and sleep data. Note that all analysis performed here is rather simple and can be seen as first steps in a more sophisticated workflow."
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"## Getting and cleaning step data\n",
"\n",
"Let's start with downloading and visualizing my daily step data - a process that was already discussed in Step 1 of this analysis. The following code snippet downloads my daily step data over the last three months and prints today's step count as an example."
]
},
{
"cell_type": "code",
"execution_count": 1,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"{'dateTime': '2016-03-28', 'value': '14241'}"
]
},
"execution_count": 1,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import fitbit\n",
"from bin.parse_credentials import parse_client_credentials, parse_tokens\n",
"\n",
"client_id, client_secret = parse_client_credentials('client_id_secret.txt')\n",
"\n",
"!./gather_keys_oauth2.py $client_id $client_secret >access_refresh_tokens.txt 2>/dev/null\n",
"\n",
"access_token, refresh_token = parse_tokens('access_refresh_tokens.txt')\n",
"\n",
"authd_client = fitbit.Fitbit(client_id, client_secret, oauth2=True,\n",
" access_token=access_token,\n",
" refresh_token=refresh_token)\n",
"\n",
"steps_ts = authd_client.time_series('activities/steps', period='3m')\n",
"steps_ts['activities-steps'][-1]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"Next, we convert all dates to Python [datetime](https://docs.python.org/3/library/datetime.html#module-datetime) objects."
]
},
{
"cell_type": "code",
"execution_count": 2,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/plain": [
"(datetime.datetime(2016, 3, 28, 0, 0), 14241)"
]
},
"execution_count": 2,
"metadata": {},
"output_type": "execute_result"
}
],
"source": [
"import dateutil.parser\n",
"date_steps = [(dateutil.parser.parse(date_steps_dict['dateTime']), int(date_steps_dict['value']))\n",
" for date_steps_dict in steps_ts['activities-steps']]\n",
"date_steps[-1]"
]
},
{
"cell_type": "markdown",
"metadata": {},
"source": [
"As a last step prior to taking an initial look at the daily step counts, the step data are stored as a [pandas](http://pandas.pydata.org/) DataFrame. This will simplify the downstream analyses and facilitates simple data visualizations. The last five rows of the resulting DataFrame looks like this:"
]
},
{
"cell_type": "code",
"execution_count": 3,
"metadata": {
"collapsed": false
},
"outputs": [
{
"data": {
"text/html": [
"\n",
"
\n",
" \n",
" \n",
" | \n",
" Date | \n",
" Steps | \n",
"
\n",
" \n",
" \n",
" \n",
" | 88 | \n",
" 2016-03-24 | \n",
" 5210 | \n",
"
\n",
" \n",
" | 89 | \n",
" 2016-03-25 | \n",
" 13451 | \n",
"
\n",
" \n",
" | 90 | \n",
" 2016-03-26 | \n",
" 17618 | \n",
"
\n",
" \n",
" | 91 | \n",
" 2016-03-27 | \n",
" 9304 | \n",
"
\n",
" \n",
" | 92 | \n",
" 2016-03-28 | \n",
" 14241 | \n",
"
\n",
" \n",
"
\n",
"