{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "# Location Viz Sample Notebook - Scala\n", "\n", "In this notebook we will work with data from our Weighted Position model for Presence Insights to visualize location data before and after we apply our smoothing algorithms. \n", "\n", "### Load the data\n", "\n", "First, we will need to upload our location data file to the object store.\n", "- Create an object store in your Spark service instance\n", "- On the palette to the right, navigate to the Data Sources tab\n", "- Download [sample data](https://raw.githubusercontent.com/reinaldo422/bluemix-spark-viz/master/smoothed_data.json) (right-click and Save As...) and add as a source\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [], "source": [ "//Import SQLContext and data types\n", "import org.apache.spark.sql._\n", "import org.apache.spark.sql.types._\n", "\n", "// Spark Notebook has existing SparkContext (sc)\n", "val sqlContext = new SQLContext(sc)\n", "import sqlContext.implicits._\n", "\n", "// Read sample Presence Insights location data\n", "val json = sqlContext.read.json(\"swift://notebooks.spark/smoothed_data.json\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Separate each individual event" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import org.apache.spark.sql.functions.explode\n", "\n", "val devicesDF = json.select(\n", " $\"sites\",\n", " explode($\"data\").alias(\"data\")).select(\n", " $\"data.original.device_descriptor\".alias(\"device\"),\n", " $\"sites.floors.properties.z\".getItem(0).alias(\"z\"),\n", " $\"data.original.x\",\n", " $\"data.original.y\",\n", " $\"data.smooth_x\",\n", " $\"data.smooth_y\") \n", "\n", "devicesDF.registerTempTable(\"devices\")\n" ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [], "source": [ "devicesDF.show()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Select device and floor" ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [], "source": [ "val plot_data = sqlContext.sql(\"SELECT * FROM devices WHERE device='device1' AND z=0\")" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Plotting with Brunel\n", "[Brunel project](http://brunel.mybluemix.net/docs/) defines interactive data visualizations based on tabular data. It supports Spark dataframes and automatically chooses good transforms, mappings, and formatting for your data." ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": true }, "outputs": [], "source": [ "%AddJar -magic http://www.brunelvis.org/jar/spark-kernel-brunel-all-1.1.jar " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Raw data" ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false, "scrolled": true }, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", "
\n", "\n", "\n", "" ] }, "execution_count": 6, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%%brunel data('plot_data') x(x) y(y)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Smooth data" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "data": { "text/html": [ "\n", " \n", " \n", " \n", "
\n", "\n", "\n", "" ] }, "execution_count": 7, "metadata": {}, "output_type": "execute_result" } ], "source": [ "%%brunel data('plot_data') x(smooth_x) y(smooth_y)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Scala 2.10", "language": "scala", "name": "spark" }, "language_info": { "name": "scala" } }, "nbformat": 4, "nbformat_minor": 0 }