{ "cells": [ { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "# Good Night Bluesky: Running your overnight script\n", "\n", "In this notebook you will:\n", "\n", "* Perform final checks prior to executing the tested overnight plan\n", " * Additional modifications (transient metadata)\n", " * How to add ancillary signals that are important (baseline versus primary data streams)\n", "* Experience common mistakes that trip up experienced bluesky users\n", "\n", "\n", "**BONUS Material** \n", "Located after our objectives. It is in the form of Q & A.\n", "\n", "Recommend Prerequisites:\n", "* [Hello Python and Jupyter](./Hello%20Python%20and%20Jupyter.ipynb)\n", "* [Good Morning Bluesky](./Good%20Morning%20Bluesky.ipynb)\n", "* [Good Morning Afternoon](./Good%20Afternoon%20Bluesky.ipynb)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "#pip install -U --pre databroker[all]" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Hosted by Andi" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## For Bluesky Session Continuity\n", "Run the next cells to make the ipython kernel match the previous session\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "%run -i gm_user/user_profile.py" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "%run -i gm_user/user_startup.py \n", "%run -i gm_user/user_startup_night.py" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "md_info()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "temperature.readback.get()" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Let's collect a good dataset for our current temperature\n", "\n", "First, open the file [gm_user/user_startup_night.py](./gm_user/user_startup_night.py) and view it side by side with this notebook.\n", "\n", "It's pretty clear that we should run `RE(one_temperature())` and this is a good first \"real\" test of our functions.\n", "\n", "\n", "```python\n", "RE(one_temperature())\n", "```\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "But we can test how to put this together with our planned overnight script:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "RE(one_temperature())" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**But** what if we want to try to automate some processing or take better advantage of data access tools like `databroker` or its replacement `tiled`?\n", "\n", "Let's look at **line 22** in [gm_user/user_startup_night.py](./gm_user/user_startup_night.py)." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
\n", "Tip: Transient metadata is possible on a per scan basis for any built in bluesky plan.\n", "
\n", "\n", "
line 22\n", " \n", "Not the new agurment utilized in `count()`.\n", "\n", "```python\n", " md={'purpose':'analyze'}\n", "```\n", " \n", "* The python dicitionary can be as complex as you like.\n", "* Note that user defined `md` key names (`purpose`) are not enforced/checked\n", " \n", "
\n", "Caution: If you reply heavily on the keys then creating custom plans are the best way to avoid typographical errors.\n", "
\n", "\n", "[Read about more about bluesky metadata](https://blueskyproject.io/bluesky/metadata.html).\n", " \n", "
\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "db[-1].start" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Let's test how to put this together with our planned overnight script:" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "check_limits( bpp.pchain(one_temperature(), my_experiment([41, 42])) )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**NOW** edit the cell below to pass the pchained plans to the RE." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "( bpp.pchain(one_temperature(), my_experiment([41, 42])) )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Recording ancillary signals\n", "Is the tempeature being recorded? \n", "\n", "We know that it is not. The next tutorial deals with data access. However, two simple verifications utilize the start and stop documents." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "db[-1].start" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "db[-1].stop" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Available Data Streams for Ancillary Signals\n", "\n", "
\n", "Tip: Using the recommended data streams guarantees that the key names are always consistent.\n", "
\n", "\n", "\n", "It is recommended to record all data in data streams:\n", "* primary\n", "* [baseline](https://blueskyproject.io/bluesky/tutorial.html#baseline-readings-and-other-supplemental-data)\n", "* monitors\n", "* flyers\n", "\n", "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Try adding `temperature` to the `primary` datastream\n", "\n", "\n", "
Copy Paste Solution\n", "\n", "```python\n", "\n", "RE(count([noisy_det, temperature], 5) )\n", "\n", "```\n", "
\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "### Add temperature to the primary datastream\n", "\n", "RE(count([noisy_det, ], 5) )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Configuring and using the baseline stream\n", "* check [gm_user/user_profile.py](./gm_user/user_profile.py) for initialization\n", "* baseline is part of `SupplenmentalData` (`sd`)\n", "* like the primary detectors, baseline is a python list\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "sd" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "sd.baseline =[temperature]\n", "sd.baseline" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "RE(count([noisy_det], 5) )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "Tip: Baseline print to screen can be turned OFF - especially helpful when there are >10 devices.\n", "
\n", "\n", "`BestEffortCallbacks` (`bec`) is used to control some of the bluesky interface features" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# OPTIONAL, explore bec by uncommenting the line directly below\n", "# dir(bec)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "bec.disable_baseline()" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "RE(count([noisy_det], 5) )" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "db[-1].table('baseline')" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Hosted by Josh" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Common mistakes made by expert users\n", "* switching shifts with team members\n", "* a little tired\n", "\n", "Some mistakes are difficult to understand immediately. Let's have a look at a couple common ones and how troubleshoot." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### **#1** - adding to `sd.baseline`" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "sd.baseline.append([motor1, motor2])" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "RE(count([noisy_det], 2) )" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "sd.baseline" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "
Copy Paste Solution\n", "\n", "\n", "```python\n", "sd.baseline =[temperature]\n", "sd.baseline.extend([motor1, motor2])\n", "\n", "```\n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Try again after entering your **solution** above" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "RE(count([noisy_det], 2) )" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## **#2** - using a long list of detectors\n", "\n", "### **#2a**\n", "\n", "Code is easier to read when you make a long python list of detectors and just use that variable.\n", "\n", "```python\n", "dets = [det, noisy_det, temperature, motor1.readback, det2]\n", "RE(scan(dets, motor, -5, 5, 3))\n", "```\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "RE(scan(dets, motor, -5, 5, 3))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
Hint\n", "\n", "Let's look at **line 37** in [gm_user/user_startup_night.py](./gm_user/user_startup_night.py).\n", " \n", " \n", "```python\n", "my_dets\n", "```\n", " \n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "my_dets" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "tags": [] }, "outputs": [], "source": [ "dets = my_dets\n", "\n", "dets is my_dets" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### **#2b** \n", "
Copy Paste Solution\n", "\n", "\n", "```python\n", "RE(scan(dets, motor, -5, 5, 3))\n", "\n", "```\n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "RE(scan([dets], motor, -5, 5, 11))" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "dets" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "### **#2c**\n", "\n", "And this is also a **trap** if you use **another common DAQ**\n", "\n", "```\n", " ascan motor start stop steps time\n", "\n", "```\n", "\n", "
Copy Paste Solution\n", "\n", "\n", "```python\n", "RE(scan([det], motor, -5, 5, 3))\n", "\n", "```\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "RE(scan(det, motor, -5, 5, 3))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## **#3** **the other DAQ trap**\n", "\n", "### **#3a**\n", "\n", "
Copy Paste Solution\n", "\n", "\n", "```python\n", "\n", "RE(scan([det], motor, -5, 5, 3))\n", "\n", "```\n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "RE(scan( motor, -5, 5, 3))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### **#3b**\n", "\n", "
Copy Paste Solution\n", "\n", "\n", "```python\n", "\n", "RE(scan(my_dets, motor, -5, 5, 3))\n", "\n", "\n", "```\n", " \n", "Counting time is handled per detector which makes asynchronous collection possible (more flexibility).\n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "RE(scan( motor, -5, 5, 3, 1))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### **#3c**\n", "\n", "
Copy Paste Solution\n", "\n", "\n", "```python\n", "\n", "RE(scan(my_dets, motor, -5, 5, 3))\n", "\n", "\n", "```\n", " \n", "Counting time is handled per detector which makes asynchronous collection possible (more flexibility).\n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "RE(scan(my_dets, motor, -5, 5, 3, 1))" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Questions for the above?" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Questions in general about bluesky data collection?" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Self-guided Tour\n", "\n", "## BONUS / Q&A\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q1** What other pre-assembled plans are avaialbe with blueksy \"out-of-the-box\" installation? How do I make a 2D scan?\n", "\n", "**A1** Check the docs:\n", "* [pre-assembeled plans](https://blueskyproject.io/bluesky/plans.html#pre-assembled-plans)\n", "* [other essential stub plans](https://blueskyproject.io/bluesky/plans.html#stub-plans)\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q2** Why is the scan so noisy when I know the detector has a low noise level?\n", "\n", "**A2** Realy motors are not perfect and electrical ground loops are difficult to remove. Try:\n", "* plotting the detector signal as a function of `motor.setpoint`\n", "* switch to absolute scans\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q3** Why is `summarize_plan(rel_scan([det], motor, -1, 1, 21))` incorrect\n", "\n", "**A3** `bluesky.simulators` use the current live position of all motors. Currently, relative scans generate the motor trajectory as the scan proceeds in the RE. For best results, you can fake relative scans:\n", "\n", "```python\n", "my_motor_pos = motor.setpoint.get()\n", "yield from scan([det], motor, my_motor_pos-1, my_motor_pos+1, 21)\n", "\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q4** What if I want to control `.get()` like the rest of bluesky's plan generators inside my custom plan? **NoteL** `.get()` emmits a query to the network inside `summarize_plan()`.\n", "\n", "**A4** Use `bps.rd()`\n", "```python\n", "my_variable = yield from bps.rd(top_level_device.device_child_to_return_single_value)\n", "\n", "\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q5** How do perform a line scan on multiple motors (or through reciprocal space) on multiple \"motors\"?\n", "\n", "**A5** Using \n", "```python\n", "scan?\n", "```\n", "\n", "it is possible to see that you can scan N motors. Below N=2. \n", "\n", "```python\n", "\n", "RE(scan([det], motor, -5, 5, motor1, -5, 5, 3))\n", "\n", "```\n", "\n", "Not that only the first motor is plotted in the LivePlot." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q6** What about a theta-2theta scan?\n", "\n", "**A6** Alter the scan arguments \n", "\n", "\n", "```python\n", "twotheta = motor\n", "theta = motor1\n", "\n", "\n", "\n", "RE(scan([det], twotheta, -5, 5, theta, -5/2, 5/2, 3))\n", "\n", "```\n", "\n", "Not that only the first motor is plotted in the LivePlot." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q7** How do I control what detectors and motors are plotted and included in the table?\n", "\n", "**A7** Bluesky hints. Each signal has a \"kind\". The options are:\n", "\n", "- \"normal\" --> recorded but not displayed in the user interface during collection\n", "- \"omit\" --> not recorded\n", "- \"config\" --> record as metadata in descriptors (just once per scan)\n", "- \"hinted\" --> plot in LivePlot and LiveTable\n", "\n", "\n", "\n", "\n", "```python\n", "print(f'{temperature.setpoint.kind=}')\n", "print(f'{temperature.readback.kind=}')\n", "\n", "temperature.setpoint.kind='hinted'\n", "\n", "print(f'\\n{temperature.setpoint.kind=}')\n", "```\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q8** Bluesky doesn't have device I want to record or control. What is the fastest way to get going?\n", "\n", "**A8** ophyd EpicsSignal or EpicsSignalRO (**R**ead**O**nly) may be set up in 1 line. As long as you do not need bluesky to check many aspects of how the signal is used to ensure performance, then this should be a decent quick fix. \n", "\n", "```python\n", "from ophyd.signal import EpicsSignal\n", "my_signal = EpicsSignal('sting_representing_EPICS_PV', name=\"my_signal\")\n", "```\n", "\n", "In terms of EPICS or pyepics, the usage would be:\n", "```\n", "pyepics.caget('sting_representing_EPICS_PV')\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "**Q9** How do I make a 2D mesh-like scan for imaging?\n", "\n", "**A9** There are a few different pre-assembled plans. Check the [docs](https://blueskyproject.io/bluesky/plans.html) for the full list.\n", "\n", "```python\n", "RE(grid_scan([det], motor, -1, 1, 9,\n", " motor1, -1, 1, 9,\n", " True))\n", "```\n", "\n", "Last argument is for \"snaking\" (True or False).\n", "```python\n", "grid_scan??\n", "\n", "```\n", "\n" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "bec.disable_table()\n", "RE(grid_scan([det], motor, -1, 1, 9,\n", " motor1, -1, 1, 9,\n", " True))" ] }, { "cell_type": "markdown", "metadata": { "tags": [] }, "source": [ "**Q10** Are there additional inspection/simulation plans for 2D imaging scans?\n", "\n", "**A10** Yes.\n", "\n", "```python\n", "\n", "from ophyd.sim import motor, motor1, det\n", "from bluesky.simulators import summarize_plan, check_limits, plot_raster_path\n", "from bluesky.plans import *\n", "\n", "plan = grid_scan([det], motor, -1, 1, 9,\n", " motor1, -1, 1, 9,\n", " True)\n", "plot_raster_path(plan, 'motor', 'motor1', probe_size=.1)\n", "\n", "```" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "# from ophyd.sim import motor, motor1, det\n", "# from bluesky.simulators import summarize_plan, check_limits, plot_raster_path\n", "# from bluesky.plans import *\n", "\n", "plan = grid_scan([det], motor, -1, 1, 9,\n", " motor1, -1, 1, 9,\n", " True)\n", "plot_raster_path(plan, 'motor', 'motor1', probe_size=.1)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "Remember, plans are generators so the below will fail." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "plan = grid_scan([det], motor, -1, 1, 9,\n", " motor1, -1, 1, 9,\n", " False)\n", "plot_raster_path(plan, 'motor', 'motor1', probe_size=.1)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "Watch Out for a DAQ Trap: Relative scans producing image are labeled with relative positions. \n", "
" ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true, "tags": [] }, "outputs": [], "source": [ "def see_diff_with_relative():\n", " yield from mv(motor, -1, motor1, -1)\n", " yield from rel_grid_scan([det], motor, -1, 1, 9, \n", " motor1, -1, 1, 9,\n", " True)\n", "\n", "bec.disable_table()\n", "RE(see_diff_with_relative())" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "kernelspec": { "display_name": "Python 3 (ipykernel)", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.9.13" } }, "nbformat": 4, "nbformat_minor": 4 }