#+TITLE: bokeh and Emacs org-mode #+DATE: 03 Nov 2017 #+OPTIONS: toc:t num:2 #+KEYWORDS: bokeh, ob-ipython, babel, org-mode, emacs #+HTML_HEAD_EXTRA: #+HTML_HEAD_EXTRA: #+HTML_HEAD_EXTRA: #+HTML_HEAD_EXTRA: #+HTML_HEAD_EXTRA: * Introduction I've been wanting to play around with interactive Javascript plots for a while; for example: https://demo.bokehplots.com/apps/selection_histogram. The selection functionality is really cool! This post is an experiment in embedding [[http://bokeh.pydata.org][bokeh]] plots in org-mode notebooks that rendered properly when exported as HTML pages. There are two approaches for embedding. * Using ~autoload_static~ (preferred) Based on https://necromuralist.github.io/data_science/posts/bokeh-test/ Some notes: 1. This method stores the data in a separate ~.js~ file which should make the org file easier to handle. Also, the figure can embedded freely in many places. 2. Figuring where to get the bokeh css and js files can be tricky ([[https://bokeh.pydata.org/en/latest/docs/reference/resources.html][docs]]): 1. You can use the CDN: ~from bokeh.resources import CDN~ to load the bokeh js file over the internet. Not good for future-proofing. 2. Using ~bokeh.resources.Resources(mode='absolute')~ or ~'mode=relative'~ loads bokeh from your local python installation; so that's good for /local/ notebooks but your figures/notebooks might break when bokeh updates. 3. The ~inline~ option inlines the necessary js and css in the exported ~.js~ file but then with multiple images; you end up with multiple copies so that's wasteful, but seems like the only future-proof option. Also, you can have two figures made with different bokeh versions embedded in the same document, so that's a plus. ** Prerequisites I am using 1. bokeh 0.12.10 2. org 9.1.2 3. emacs 25.3.50.2 4. [[https://github.com/gregsexton/ob-ipython/commit/1642a74d4402b77ce051879e7605bc7c6537f922][ob-ipython @ 1642a74]] I define a function ~export_bokeh~ to do the actual heavy lifting. This function could be moved to your ipython ~startup.py~ file so that it is defined for every ipython session. #+NAME: define-org-static #+BEGIN_SRC ipython :session :exports code :results none def export_bokeh(plot, outPNG, outJS, outHTML, bkjs='inline'): from bokeh.io import export_png from bokeh.embed import autoload_static, file_html import bokeh.resources if bkjs is 'local': # use local installed bokeh scripts wherebokeh = bokeh.resources.Resources(mode='absolute') if bkjs is 'remote' or bkjs is 'CDN': # use CDN bokeh scripts wherebokeh = bokeh.resources.CDN if bkjs is 'relative': # bkjs is a relative path to the locally installed bokeh files wherebokeh = bokeh.resources.Resources(mode='relative', root_dir='./') if bkjs is 'inline': wherebokeh = bokeh.resources.INLINE # save the png file export_png(plot, filename=outPNG) # save the html file html = file_html(plot, wherebokeh, None) with open(outHTML, 'w') as file: file.write(html) js, script = autoload_static(plot, wherebokeh, outJS) # save the .js file with open(outJS, "w") as writer: writer.write(js) # embed in the org-exported HTML file print('''#+BEGIN_EXPORT html\n{script}\n#+END_EXPORT'''.format(script=script.lstrip())) #+END_SRC Embedding the figure is then quite easy once you 1. tell ob-ipython to capture stdout and put it in a drawer so that it's replaced everytime to rerun the source block. 2. provide a custom filename for export (~fname~ below). I use this header line: #+BEGIN_EXAMPLE #+BEGIN_SRC ipython :session :results output drawer :exports both :var fname="my-image" :var titlestr="Demonstrating bokeh" #+END_EXAMPLE ** Example This will export a [[../static/my-image.png][png image]], an [[../static/html/my-image.html][HTML file]] and a [[../static/js/my-image.js][Javascript file]]. That way there are both static and dynamic versions that are easy to share as well as an emebeddable version. #+BEGIN_SRC ipython :session :results output drawer :exports both :var fname="my-image" :var titlestr="Demonstrating Bokeh" from bokeh.plotting import figure from bokeh.layouts import gridplot from bokeh.models import ColumnDataSource, HoverTool, WheelZoomTool tools='box_select, reset' # create "subplots" p1 = figure(tools=tools) p1.background_fill_alpha = 0.0 p1.border_fill_alpha = 0.0 p2 = figure(tools=tools, x_range=p1.x_range, y_range=p1.y_range) p2.background_fill_alpha = 0.0 p2.border_fill_alpha = 0.0 # generate data x = np.random.randn(200) y0 = x**3 - 100 y1 = x**2 - 100 # generate random letters import string import random label = [random.choice(string.ascii_letters) for aa in range(200)] # needed for linked selection source = ColumnDataSource(data={'x': x, 'y0': y0, 'y1': y1, 'label': label}) # plot the data p1.circle('x', 'y0', source=source) p2.scatter('x', 'y1', source=source) p1.title.text = titlestr wheel = WheelZoomTool() # add some tooltips hover = HoverTool() hover.tooltips = [ ("(x,y)", "($x, $y)"), # '$' for co-ordinates ("label", "@label") # note '@' for column lookup ] p1.add_tools(hover, wheel) p2.add_tools(hover, wheel) p1.toolbar.active_scroll = wheel p2.toolbar.active_scroll = wheel # layout the subplots pg = gridplot([[p1, p2]], plot_width=400, plot_height=400) export_bokeh(pg, '../static/'+fname+'.png', '../static/js/'+fname+'.js', '../static/html/'+fname+'.html') #+END_SRC #+RESULTS: :RESULTS: #+BEGIN_EXPORT html #+END_EXPORT :END: The above figure is a rendering of the ~RESULTS~ block: #+BEGIN_EXAMPLE #+RESULTS: :RESULTS: #+BEGIN_EXPORT html #+END_EXPORT :END: #+END_EXAMPLE ** Improvements Things I would like: - Automatically run the HTML export when running the code block. - Can I add-hook to ~org-babel-execute-maybe~? - The ability to auto-hide the javascript in the ~RESULTS~ drawer but show the exported png file. - Ideally, I would see the png in the org file and the javascript in the HTML file. The other should be hidden. Seems too complicated. - A nice solution would be to insert a link to the png file in the image caption. - Are _actual_ captions (~#+CAPTION~) possible with bokeh figures? - Avoid the explicit ~export_bokeh~ call if possible. - Ideally, ~ob-ipython~ would be able to tell that this is a bokeh block; use the ~fname~ var to pass the figure handle ~p~ and ~fname~ to ~export_bokeh~ and do everything. This would be close to jupyter notebook workflow. - This might be possible using IPython formatters as suggested by the ~ob-ipython~ README. - Could we then define ~export_bokeh~ in ~imports.py~ and call that when the returned object is a bokeh figure handle? - Can we get holoviews to work with this? Optionally tell it to use either matplotlib or bokeh backends * Using components (not-recommended) I made some slight modifications to http://kitchingroup.cheme.cmu.edu/blog/2016/02/07/Interactive-Bokeh-plots-in-HTML/ though I didn't need to embed anything in a frame. The principal drawback here is that all the data is returned in ~stdout~ and written to the org file. This would get unwieldy for big plots. 1. Add the following at the top of my org file to use locally downloaded versions of bokeh. Note that versions need to match/be compatible with what you have installed (~bokeh.__version__~). #+BEGIN_SRC org ,#+HTML_HEAD: ,#+HTML_HEAD: ,#+HTML_HEAD: ,#+HTML_HEAD: #+END_SRC 2. For this website I use the CDN urls they provide : ~http://cdn.pydata.org/bokeh/release/bokeh-x.y.z.min.css~ etc. #+BEGIN_SRC ipython :session :results output drawer :exports both def WriteToOrg(p): script, div = components(p) script = '\n'.join(['#+HTML_HEAD_EXTRA: ' + line for line in script.lstrip().split('\n')]) print('''{script}\n#+BEGIN_EXPORT html\n{div}\n#+END_EXPORT'''.format(script=script, div=div)) from bokeh.plotting import figure from bokeh.embed import components p = figure() p.line(np.random.randn(200), np.random.randn(200)) WriteToOrg(p) #+END_SRC #+RESULTS: :RESULTS: #+HTML_HEAD_EXTRA: #+BEGIN_EXPORT html
#+END_EXPORT :END: * Appendix ** yasnippet A simplified version of the one on ~ob-ipython~. This autogenerates a random filename. #+BEGIN_EXAMPLE #+BEGIN_SRC ipython :session :results output drawer :exports both :var fname=${1:`(make-temp-name "img")`} from bokeh.plotting import figure from bokeh.layouts import gridplot from bokeh.models import ColumnDataSource, HoverTool, WheelZoomTool tools='box_select, reset' # create "subplots" hf = figure(tools=tools) hf.background_fill_alpha = 0.0 hf.border_fill_alpha = 0.0 hf.$0 export_bokeh(hf, 'images/'+fname+'.png', 'images/js/'+fname+'.js', bkjs='inline') #+END_SRC #+END_EXAMPLE ** Dependencies Looks like exporting to PNG requires #+BEGIN_EXAMPLE conda install phantomjs selenium #+END_EXAMPLE ** Styling 1. Either include a css file or add a style tag to ~HTML_HEAD~ at the top of your org file. #+BEGIN_SRC org ,#+HTML_HEAD: #+END_SRC 2. Targeting the css class ~bk-plot-layout~ lets you center the image. ~bk-grid-column~ does it for ~gridplot~ #+BEGIN_SRC css .bk-plot-layout .bk-grid-column { max-width: 100%; margin: auto;} #+END_SRC ** Source Here is [[https://raw.githubusercontent.com/dcherian/dcherian.github.io/sources/org/posts/bokeh-org-mode.org][org-mode source]] for this page.