{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "

1) Load all of the needed python packages to run this notebook. These include ODM2API, ipython widgets for user interaction, bokeh for interactive plots. ui_utils

\n" ] }, { "cell_type": "code", "execution_count": 1, "metadata": { "collapsed": false }, "outputs": [], "source": [ "import sys\n", "import os\n", "import datetime\n", "import getpass\n", "import utilities.ui_utils as ui_utils\n", "from bokeh.plotting import figure, show\n", "from IPython.core.display import HTML\n", "from ipywidgets import Layout, HBox\n", "from IPython.display import display\n", "from odm2api.ODMconnection import dbconnection\n", "import odm2api.ODM2.services.readService as odm2\n", "from odm2api.ODM2.models import *\n", "from bokeh.io import output_notebook\n", "from bokeh.models import ColumnDataSource" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "2) The code below retreives the ODM2 login prompt from ui_utils python code which should be in your utilities folder. This will log directly into SQL. " ] }, { "cell_type": "code", "execution_count": 6, "metadata": { "collapsed": false }, "outputs": [], "source": [ "\n", "#print(\"Enter your ODM2 username\") \n", "container = ui_utils.ODM2LoginPrompt()\n", "container" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "3) Next we prompt a user for their password. " ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "enter your password: \n", "········\n" ] } ], "source": [ "print(\"enter your password: \")\n", "p = getpass.getpass()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

4) Here we look into the login prompt container. to retrieve the database connection information entered by the user.

\n", "

5) On the second line, I'm using the ODM2API read service for retrieving result values this should help make retrieving \n", "them agnostic to the result type.

" ] }, { "cell_type": "code", "execution_count": 2, "metadata": { "collapsed": false }, "outputs": [], "source": [ "session_factory = dbconnection.createConnection('postgresql', container.children[1].value, \n", " container.children[2].value, \n", " container.children[0].value, p) \n", "\n", "read = odm2.ReadODM2(session_factory)\n", "DBSession = session_factory.getSession()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "6) Next we retreive another UI element for selecting observation actions. Which we will use to filter results. Otherwise the list of results maybe too long. " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "7) Now we get a UI element for results, for which we would like to display values. " ] }, { "cell_type": "code", "execution_count": 3, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "34\n" ] } ], "source": [ "HTML('')\n", "actionWidget, awidget = ui_utils.actionWidgetNotPulse(DBSession)\n", "#box = HBox([awidget])\n", "#box.layout.display = 'flex'\n", "#box.layout.align_items = 'stretch'\n", "display(awidget) \n" ] }, { "cell_type": "code", "execution_count": 7, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "16647\n" ] } ], "source": [ "\n", "#featureaction = 1700\n", "actionid = actionWidget.value\n", "print(actionid)\n", "resultWidget, rwidget = ui_utils.resultWidget(read, actionid)\n", "display(rwidget)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "8) Now we create a UI element for entering the dates we would like to filter the time series on. " ] }, { "cell_type": "code", "execution_count": 8, "metadata": { "collapsed": false }, "outputs": [], "source": [ "startdate_enddate_container = ui_utils.startDateEndDateContainer()\n", "startdate_enddate_container" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "9) Next we read more information about the result, pull the results values for the time frame we specified and create a data dictionary our selection graph can display. " ] }, { "cell_type": "markdown", "metadata": { "collapsed": false }, "source": [ "10) Now we create a bokeh (graphing package) ColumnDataSource for our graph, create our plot (also using ui_utils) and display it. " ] }, { "cell_type": "code", "execution_count": 13, "metadata": { "collapsed": false }, "outputs": [], "source": [ "selectedResult = read.getDetailedResultInfo(resultTypeCV = 'Time series coverage',resultID=resultWidget.value)\n", "#resultValues = read.getResultValues(resultid=resultWidget.value, starttime='2016-8-1', \n", "# endtime= '2016-8-30')\n", "resultValues = read.getResultValues(resultid=resultWidget.value, starttime=startdate_enddate_container.children[0].value, \n", " endtime= startdate_enddate_container.children[1].value)\n", "#print(startdate_enddate_container.children[0].value)\n", "#print(startdate_enddate_container.children[1].value)\n", "data = dict(x=resultValues.ValueDateTime, y= resultValues.DataValue, z=resultValues.ValueID)\n", "#print(data)" ] }, { "cell_type": "code", "execution_count": 14, "metadata": { "collapsed": false, "scrolled": false }, "outputs": [ { "data": { "text/html": [ "\n", "
\n", " \n", " Loading BokehJS ...\n", "
" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "application/javascript": [ "\n", "(function(global) {\n", " function now() {\n", " return new Date();\n", " }\n", "\n", " if (typeof (window._bokeh_onload_callbacks) === \"undefined\") {\n", " window._bokeh_onload_callbacks = [];\n", " }\n", "\n", " function run_callbacks() {\n", " window._bokeh_onload_callbacks.forEach(function(callback) { callback() });\n", " delete window._bokeh_onload_callbacks\n", " console.info(\"Bokeh: all callbacks have finished\");\n", " }\n", "\n", " function load_libs(js_urls, callback) {\n", " window._bokeh_onload_callbacks.push(callback);\n", " if (window._bokeh_is_loading > 0) {\n", " console.log(\"Bokeh: BokehJS is being loaded, scheduling callback at\", now());\n", " return null;\n", " }\n", " if (js_urls == null || js_urls.length === 0) {\n", " run_callbacks();\n", " return null;\n", " }\n", " console.log(\"Bokeh: BokehJS not loaded, scheduling load and callback at\", now());\n", " window._bokeh_is_loading = js_urls.length;\n", " for (var i = 0; i < js_urls.length; i++) {\n", " var url = js_urls[i];\n", " var s = document.createElement('script');\n", " s.src = url;\n", " s.async = false;\n", " s.onreadystatechange = s.onload = function() {\n", " window._bokeh_is_loading--;\n", " if (window._bokeh_is_loading === 0) {\n", " console.log(\"Bokeh: all BokehJS libraries loaded\");\n", " run_callbacks()\n", " }\n", " };\n", " s.onerror = function() {\n", " console.warn(\"failed to load library \" + url);\n", " };\n", " console.log(\"Bokeh: injecting script tag for BokehJS library: \", url);\n", " document.getElementsByTagName(\"head\")[0].appendChild(s);\n", " }\n", " };\n", "\n", " var js_urls = ['https://cdn.pydata.org/bokeh/release/bokeh-0.11.1.min.js', 'https://cdn.pydata.org/bokeh/release/bokeh-widgets-0.11.1.min.js', 'https://cdn.pydata.org/bokeh/release/bokeh-compiler-0.11.1.min.js'];\n", "\n", " var inline_js = [\n", " function(Bokeh) {\n", " Bokeh.set_log_level(\"info\");\n", " },\n", " \n", " function(Bokeh) {\n", " Bokeh.$(\"#91d49101-6d6f-4c12-ba0f-4970800d08c1\").text(\"BokehJS successfully loaded\");\n", " },\n", " function(Bokeh) {\n", " console.log(\"Bokeh: injecting CSS: https://cdn.pydata.org/bokeh/release/bokeh-0.11.1.min.css\");\n", " Bokeh.embed.inject_css(\"https://cdn.pydata.org/bokeh/release/bokeh-0.11.1.min.css\");\n", " console.log(\"Bokeh: injecting CSS: https://cdn.pydata.org/bokeh/release/bokeh-widgets-0.11.1.min.css\");\n", " Bokeh.embed.inject_css(\"https://cdn.pydata.org/bokeh/release/bokeh-widgets-0.11.1.min.css\");\n", " }\n", " ];\n", "\n", " function run_inline_js() {\n", " for (var i = 0; i < inline_js.length; i++) {\n", " inline_js[i](window.Bokeh);\n", " }\n", " }\n", "\n", " if (window._bokeh_is_loading === 0) {\n", " console.log(\"Bokeh: BokehJS loaded, going straight to plotting\");\n", " run_inline_js();\n", " } else {\n", " load_libs(js_urls, function() {\n", " console.log(\"Bokeh: BokehJS plotting callback run at\", now());\n", " run_inline_js();\n", " });\n", " }\n", "}(this));" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "\n", "\n", "
\n", "" ] }, "metadata": {}, "output_type": "display_data" }, { "data": { "text/html": [ "

<Bokeh Notebook handle for In[14]>

" ], "text/plain": [ "" ] }, "execution_count": 14, "metadata": {}, "output_type": "execute_result" } ], "source": [ "s1 = ColumnDataSource(data=data)\n", "plot = ui_utils.selectPlot(s1,selectedResult)\n", "output_notebook()\n", "show(plot)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "11) The code below prints out the selected data points. This can be used for diagnostic purposes and is commented out here. " ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": false }, "outputs": [], "source": [ "# data = zip([s1.data['x'][i] for i in inds],\n", "# [s1.data['y'][i] for i in inds],\n", "# [s1.data['z'][i] for i in inds])\n", "# print(data)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "12) Next we prompt a user to enter an annotation and annotation code to prompt the user for. " ] }, { "cell_type": "code", "execution_count": 10, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Annoatation : d, Annotation Code: d\n" ] } ], "source": [ "\n", "annotationcontainer = ui_utils.annotationContainer()\n", "annotationcontainer" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "13) Using the below drop down list select the data quality code for these records. " ] }, { "cell_type": "code", "execution_count": 4, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Bad\n" ] } ], "source": [ "dataqualityWidget, qdwidget =ui_utils.dataqualityWidget(DBSession)\n", "qdwidget\n", "#print(qdwidget)\n", "#os.write(1, str(qdwidget))\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "14) This creates the actual annotation database record and the associated result value annotations. This links each result value to the annotation. " ] }, { "cell_type": "code", "execution_count": 5, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Bad\n" ] } ], "source": [ "print(dataqualityWidget.value)" ] }, { "cell_type": "code", "execution_count": 17, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "Time series result value annotation\n", "Miguel\n", "Lithological Influences on Contemporary and Long-Term Regolith Weathering at the Luquillo Critical Zone Observatory\n", "\n", "Buss, H.L., Lara Chapela M., Moore, O.W., Kurtz A.C., Schulz, M.S., White A.F.\n", "\n", "\n" ] } ], "source": [ "annotation_type = DBSession.query(CVAnnotationType).filter_by(Name=\"Time series result value annotation\").one() \n", "author = DBSession.query(People).filter_by(PersonFirstName=\"Miguel\").one()\n", "citation = DBSession.query(Citations).filter_by(CitationID=327).one() \n", "print(annotation_type.Name)\n", "print(author.PersonFirstName)\n", "print(citation.Title)\n", "tsrvavalueids = []\n", "annotation = Annotations(AnnotationTypeCV=annotation_type.Name,\n", " AnnotationCode = annotationcontainer.children[0].value,\n", " AnnotationText = annotationcontainer.children[1].value,\n", " AnnotationDateTime = datetime.datetime.now(),\n", " AnnotationUTCOffset = 0,\n", " AnnotationLink = 'test',\n", " AnnotatorID = 1,\n", " CitationID =327)\n", "print(annotation)\n", "DBSession.add(annotation)\n", "DBSession.commit()\n", "annotationid = annotation.AnnotationID\n", "print(annotation)\n", "for i in inds:\n", " valueid = s1.data['z'][i]\n", " tsrvAnnotation = TimeSeriesResultValueAnnotations(ValueID = valueid,\n", " AnnotationID =annotationid)\n", " DBSession.add(tsrvAnnotation)\n", " DBSession.commit()\n", " tsrvavalueids.append(tsrvAnnotation.BridgeID)\n", "for i in inds:\n", " valueid = s1.data['z'][i]\n", " tsrv = DBSession.query(TimeSeriesResultValues).filter_by(ValueID=valueid).one()\n", " tsrv.QualityCodeCV = dataqualityWidget.value\n", "DBSession.commit() " ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

1) This will last block of code will delete the values back out of the database.

" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [ "\n", "#print(tsrvavalueids)\n", "time_series_values_annotations = DBSession.query(TimeSeriesResultValueAnnotations).filter(\n", " TimeSeriesResultValueAnnotations.BridgeID.in_(tsrvavalueids))\n", "for time_series_value_annotation in time_series_values_annotations:\n", " DBSession.delete(time_series_value_annotation)\n", "DBSession.commit()\n", "\n", "DBSession.delete(annotation)\n", "DBSession.commit()" ] }, { "cell_type": "code", "execution_count": 19, "metadata": { "collapsed": false }, "outputs": [ { "name": "stdout", "output_type": "stream", "text": [ "[21, 22, 23, 24, 25, 26, 27]\n" ] } ], "source": [ "print(tsrvavalueids)" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "collapsed": true }, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 2", "language": "python", "name": "python2" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 2 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython2", "version": "2.7.12" }, "widgets": { "state": { "10bd72d513e547bd8b59bb4768f466e7": { "views": [ { "cell_index": 10 } ] }, "5ec2fbee59244763974e7f9c753f60b0": { "views": [ { "cell_index": 11 } ] }, "9121ce0d00664e5e9570b2d93f5017a8": { "views": [ { "cell_index": 23 } ] }, "e894fdd7875b42b5b4e5b08187554de4": { "views": [ { "cell_index": 3 } ] }, "fbb57071ef5c488da67705269ce0299a": { "views": [ { "cell_index": 13 } ] } }, "version": "1.2.0" } }, "nbformat": 4, "nbformat_minor": 2 }