{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "#### New to Plotly?\n", "Plotly's Python library is free and open source! [Get started](https://plotly.com/python/getting-started/) by downloading the client and [reading the primer](https://plotly.com/python/getting-started/).\n", "
You can set up Plotly to work in [online](https://plotly.com/python/getting-started/#initialization-for-online-plotting) or [offline](https://plotly.com/python/getting-started/#initialization-for-offline-plotting) mode, or in [jupyter notebooks](https://plotly.com/python/getting-started/#start-plotting-online).\n", "
We also have a quick-reference [cheatsheet](https://images.plot.ly/plotly-documentation/images/python_cheat_sheet.pdf) (new!) to help you get started!" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Imports\n", "The tutorial below imports [NumPy](http://www.numpy.org/), [Pandas](https://plotly.com/pandas/intro-to-pandas-tutorial/), [SciPy](https://www.scipy.org/) and [Plotly](https://plotly.com/python/getting-started/)." ] }, { "cell_type": "code", "execution_count": 1, "metadata": {}, "outputs": [], "source": [ "import plotly.plotly as py\n", "import plotly.graph_objs as go\n", "\n", "import numpy as np\n", "import pandas as pd\n", "import scipy\n", "\n", "from scipy import signal" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Savitzky-Golay Filter\n", "`Smoothing` is a technique that is used to eliminate noise from a dataset. There are many algorithms and methods to accomplish this but all have the same general purpose of 'roughing out the edges' or 'smoothing' some data.\n", "\n", "There is reason to smooth data if there is little to no small-scale structure in the data. The danger to this thinking is that one may skew the representation of the data enough to change its percieved meaning, so for the sake of scientific honesty it is an imperative to at the very minimum explain one's reason's for using a smoothing algorithm to their dataset." ] }, { "cell_type": "code", "execution_count": 3, "metadata": {}, "outputs": [ { "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "execution_count": 3, "metadata": {}, "output_type": "execute_result" } ], "source": [ "x = np.linspace(0, 10, 100)\n", "y = np.sin(x)\n", "y_noise = [y_item + np.random.choice([-1, 1])*np.random.random() for y_item in y]\n", "\n", "trace1 = go.Scatter(\n", " x=x,\n", " y=y,\n", " mode='markers',\n", " marker=dict(\n", " size=2,\n", " color='rgb(0, 0, 0)',\n", " ),\n", " name='Sine'\n", ")\n", "\n", "trace2 = go.Scatter(\n", " x=x,\n", " y=y_noise,\n", " mode='markers',\n", " marker=dict(\n", " size=6,\n", " color='#5E88FC',\n", " symbol='circle-open'\n", " ),\n", " name='Noisy Sine'\n", ")\n", "\n", "trace3 = go.Scatter(\n", " x=x,\n", " y=signal.savgol_filter(y, 53, 3),\n", " mode='markers',\n", " marker=dict(\n", " size=6,\n", " color='#C190F0',\n", " symbol='triangle-up'\n", " ),\n", " name='Savitzky-Golay'\n", ")\n", "\n", "layout = go.Layout(\n", " showlegend=True\n", ")\n", "\n", "data = [trace1, trace2, trace3]\n", "fig = go.Figure(data=data, layout=layout)\n", "py.iplot(fig, filename='smoothing-savitzky-golay-filter')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "#### Triangular Moving Average\n", "\n", "Another method for smoothing is a moving average. There are various forms of this, but the idea is to take a window of points in your dataset, compute an average of the points, then shift the window over by one point and repeat. This will generate a bunch of points which will result in the `smoothed` data.\n", "\n", "Let us look at the common `Simple Moving Average` first. In the 1D case we have a data set of $N$ points with y-values $y_1, y_2, ..., y_N$. Setting our window size to $n < N$, the new $i^{th}$ y-value after smoothing is computed as:\n", "\n", "$$\n", "\\begin{align*}\n", "SMA_i = \\frac{y_i + ... + y_{i+n}}{n}\n", "\\end{align*}\n", "$$\n", "\n", "In the `Triangular Moving Average`, two simple moving averages are computed on top of each other. This means that our $SMA_i$ are computed then a Triangular Moving Average $TMA_i$ is computed as:\n", "\n", "$$\n", "\\begin{align*}\n", "TMA_i = \\frac{SMA_i + ... + SMA_{i+n}}{n}\n", "\\end{align*}\n", "$$" ] }, { "cell_type": "code", "execution_count": 4, "metadata": {}, "outputs": [ { "data": { "text/plain": [ "array([0, 1, 2, 3, 4, 5, 4, 3, 2, 1, 0])" ] }, "execution_count": 4, "metadata": {}, "output_type": "execute_result" } ], "source": [ "np.array(list(range(5)) + [5] + list(range(5)[::-1])) " ] }, { "cell_type": "code", "execution_count": 5, "metadata": {}, "outputs": [ { "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "execution_count": 5, "metadata": {}, "output_type": "execute_result" } ], "source": [ "def smoothTriangle(data, degree, dropVals=False):\n", " triangle=np.array(list(range(degree)) + [degree] + list(range(degree)[::-1])) + 1\n", " smoothed=[]\n", "\n", " for i in range(degree, len(data) - degree * 2):\n", " point=data[i:i + len(triangle)] * triangle\n", " smoothed.append(sum(point)/sum(triangle))\n", " if dropVals:\n", " return smoothed\n", " smoothed=[smoothed[0]]*int(degree + degree/2) + smoothed\n", " while len(smoothed) < len(data):\n", " smoothed.append(smoothed[-1])\n", " return smoothed\n", "\n", "trace1 = go.Scatter(\n", " x=x,\n", " y=y,\n", " mode='markers',\n", " marker=dict(\n", " size=2,\n", " color='rgb(0, 0, 0)',\n", " ),\n", " name='Sine'\n", ")\n", "\n", "trace2 = go.Scatter(\n", " x=x,\n", " y=y_noise,\n", " mode='markers',\n", " marker=dict(\n", " size=6,\n", " color='#5E88FC',\n", " symbol='circle-open'\n", " ),\n", " name='Noisy Sine'\n", ")\n", "\n", "trace3 = go.Scatter(\n", " x=x,\n", " y=smoothTriangle(y_noise, 10), # setting degree to 10\n", " mode='markers',\n", " marker=dict(\n", " size=6,\n", " color='#C190F0',\n", " symbol='triangle-up'\n", " ),\n", " name='Moving Triangle - Degree 10'\n", ")\n", "\n", "layout = go.Layout(\n", " showlegend=True\n", ")\n", "\n", "data = [trace1, trace2, trace3]\n", "fig = go.Figure(data=data, layout=layout)\n", "py.iplot(fig, filename='smoothing-triangular-moving-average-degree-10')" ] }, { "cell_type": "code", "execution_count": 6, "metadata": {}, "outputs": [ { "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "" ], "text/plain": [ "" ] }, "metadata": {}, "output_type": "display_data" }, { "name": "stdout", "output_type": "stream", "text": [ "Collecting git+https://github.com/plotly/publisher.git\n", " Cloning https://github.com/plotly/publisher.git to /private/var/folders/tc/bs9g6vrd36q74m5t8h9cgphh0000gn/T/pip-req-build-A8O70g\n", "Building wheels for collected packages: publisher\n", " Running setup.py bdist_wheel for publisher ... \u001b[?25ldone\n", "\u001b[?25h Stored in directory: /private/var/folders/tc/bs9g6vrd36q74m5t8h9cgphh0000gn/T/pip-ephem-wheel-cache-wsotGP/wheels/99/3e/a0/fbd22ba24cca72bdbaba53dbc23c1768755fb17b3af0f33966\n", "Successfully built publisher\n", "Installing collected packages: publisher\n", " Found existing installation: publisher 0.11\n", " Uninstalling publisher-0.11:\n", " Successfully uninstalled publisher-0.11\n", "Successfully installed publisher-0.11\n" ] }, { "name": "stderr", "output_type": "stream", "text": [ "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/IPython/nbconvert.py:13: ShimWarning:\n", "\n", "The `IPython.nbconvert` package has been deprecated since IPython 4.0. You should import from nbconvert instead.\n", "\n", "/Library/Frameworks/Python.framework/Versions/2.7/lib/python2.7/site-packages/publisher/publisher.py:53: UserWarning:\n", "\n", "Did you \"Save\" this notebook before running this command? Remember to save, always save.\n", "\n" ] } ], "source": [ "from IPython.display import display, HTML\n", "\n", "display(HTML(''))\n", "display(HTML(''))\n", "\n", "! pip install git+https://github.com/plotly/publisher.git --upgrade\n", "import publisher\n", "publisher.publish(\n", " 'python-Smoothing.ipynb', 'python/smoothing/', 'Smoothing | plotly',\n", " 'Learn how to perform smoothing using various methods in Python.',\n", " title='Smoothing in Python | plotly',\n", " name='Smoothing',\n", " language='python',\n", " page_type='example_index', has_thumbnail='false', display_as='signal-analysis', order=1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.12" } }, "nbformat": 4, "nbformat_minor": 1 }