{
 "cells": [
  {
   "cell_type": "markdown",
   "id": "theoretical-airplane",
   "metadata": {},
   "source": [
    "# Array Detector (non-EPICS)\n",
    "\n",
    "In this tutorial we will take a first look at how we might add an array detector device to Ophyd.\n",
    "\n",
    "To allow us to focus purely on the Ophyd side of things, we've stripped out EPICS entirely here and kept all other complexity to a minimum."
   ]
  },
  {
   "cell_type": "markdown",
   "id": "authorized-pharmacy",
   "metadata": {},
   "source": [
    "## Acquiring the Image\n",
    "\n",
    "You will need to define a function that integrates directly the hardware to acquire an image and save it at a specified filepath. \n",
    "\n",
    "This function must return the array shape (i.e. dimensions) of the image. The name of the function does not matter."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "jewish-royal",
   "metadata": {},
   "outputs": [],
   "source": [
    "import numpy\n",
    "from pathlib import Path\n",
    "\n",
    "def acquire_image(filepath):\n",
    "    \"\"\"\n",
    "    This function should integrate directly with the hardware.\n",
    "    \n",
    "    No concepts particular to ophyd are involved here.\n",
    "    Just tell the hardware to take an image, however that works.\n",
    "    This function should block until acquisition is complete or\n",
    "    raise if acquisition fails.\n",
    "    \n",
    "    It will be run on a worker thread, so it will not block\n",
    "    ophyd / the RunEngine.\n",
    "    \"\"\"\n",
    "    # For this tutorail, just generate a random image.\n",
    "    from PIL import Image\n",
    "    \n",
    "    image = numpy.random.randint(0, 255, (512, 512)).astype('uint8')\n",
    "    # Ensure the directory exists.\n",
    "    Path(filepath).parent.mkdir(parents=True, exist_ok=True)\n",
    "    # Save the image.\n",
    "    Image.fromarray(image).save(filepath)\n",
    "    return image.shape"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "complimentary-bullet",
   "metadata": {},
   "outputs": [],
   "source": [
    "acquire_image('test.jpg')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "korean-exemption",
   "metadata": {},
   "source": [
    "This sample function simply generated a random image in the current directory.\n",
    "\n",
    "Let's have a quick look at it:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "noble-diameter",
   "metadata": {},
   "outputs": [],
   "source": [
    "from IPython.display import Image\n",
    "Image('test.jpg')"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "recent-female",
   "metadata": {},
   "source": [
    "## Integrating with Ophyd and Bluesky\n",
    "\n",
    "Let's get some imports out of the way before we move on:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "biblical-period",
   "metadata": {},
   "outputs": [],
   "source": [
    "import os\n",
    "import uuid\n",
    "import threading\n",
    "import itertools\n",
    "\n",
    "import requests\n",
    "from ophyd import Device, Component, Signal, DeviceStatus\n",
    "from ophyd.areadetector.filestore_mixins import resource_factory"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "caroline-fields",
   "metadata": {},
   "source": [
    "We will need to define a signal to help us reference the image file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "canadian-identification",
   "metadata": {},
   "outputs": [],
   "source": [
    "class ExternalFileReference(Signal):\n",
    "    \"\"\"\n",
    "    A pure software signal pointing to data in an external file\n",
    "    \n",
    "    The parent device is intended to set the value of this Signal to a datum_id.\n",
    "    \"\"\"\n",
    "    def __init__(self, *args, shape, **kwargs):\n",
    "        super().__init__(*args, **kwargs)\n",
    "        self.shape = shape\n",
    "\n",
    "    def describe(self):\n",
    "        res = super().describe()\n",
    "        # Tell consumers that readings from this Signal point to \"external\" data,\n",
    "        # data that is not in-line in the reading itself.\n",
    "        res[self.name].update(dict(external=\"FILESTORE:\", dtype=\"array\", shape=self.shape))\n",
    "        return res"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "different-verification",
   "metadata": {},
   "source": [
    "Our `Camera` device will use this ExternalFileReference, and implement the bulk of the staging and acquisition logic:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "eastern-recording",
   "metadata": {},
   "outputs": [],
   "source": [
    "class Camera(Device):\n",
    "    \"\"\"\n",
    "    An ophyd device for a camera that acquires images and saves them in files.\n",
    "    \"\"\"\n",
    "    # We initialize the shape to [] and update it below once we know the shape\n",
    "    # of the array.\n",
    "    image = Component(ExternalFileReference, value=\"\", kind=\"normal\", shape=[])\n",
    "\n",
    "    def __init__(self, *args, root_path, **kwargs):\n",
    "        super().__init__(*args, **kwargs)\n",
    "        self._root_path = root_path\n",
    "        # Use this lock to ensure that we only process one \"trigger\" at a time.\n",
    "        # Generally bluesky should care of this, so this is just an extra\n",
    "        # precaution.\n",
    "        self._acquiring_lock = threading.Lock()\n",
    "        self._counter = None  # set to an itertools.count object when staged\n",
    "        # Accumulate Resource and Datum documents in this cache.\n",
    "        self._asset_docs_cache = []\n",
    "        # This string is included in the Resource documents to indicate which\n",
    "        # can of reader (\"handler\") is needed to access the relevant data.\n",
    "        self._SPEC = \"MY_FORMAT_SPEC\"\n",
    "\n",
    "    def stage(self):\n",
    "        # Set the filepath where will be saving images.\n",
    "        self._rel_path_template = f\"images/{uuid.uuid4()}_%d.jpg\"\n",
    "        # Create a Resource document referring to this series of images that we\n",
    "        # are about to take, and stash it in _asset_docs_cache.\n",
    "        resource, self._datum_factory = resource_factory(\n",
    "            self._SPEC, self._root_path, self._rel_path_template, {}, \"posix\")\n",
    "        self._asset_docs_cache.append((\"resource\", resource))\n",
    "        self._counter = itertools.count()\n",
    "        return super().stage()\n",
    "\n",
    "    def unstage(self):\n",
    "        self._counter = None\n",
    "        self._asset_docs_cache.clear()\n",
    "        return super().unstage()\n",
    "\n",
    "    def trigger(self):\n",
    "        status = DeviceStatus(self)\n",
    "        if self._counter is None:\n",
    "            raise RuntimeError(\"Device must be staged before triggering.\")\n",
    "        i = next(self._counter)\n",
    "        # Start a background thread to capture an image and write it to disk.\n",
    "        thread = threading.Thread(target=self._capture, args=(status, i))\n",
    "        thread.start()\n",
    "        # Promptly return a status object, which will be marked \"done\" when the\n",
    "        # capture completes.\n",
    "        return status\n",
    "\n",
    "    def _capture(self, status, i):\n",
    "        \"This runs on a background thread.\"\n",
    "        try:\n",
    "            if not self._acquiring_lock.acquire(timeout=0):\n",
    "                raise RuntimeError(\"Cannot trigger, currently triggering!\")\n",
    "            filepath = os.path.join(self._root_path, self._rel_path_template % i)\n",
    "            # Kick off requests, or subprocess, or whatever with the result\n",
    "            # that a file is saved at `filepath`.\n",
    "            shape = acquire_image(filepath)\n",
    "            self.image.shape = shape\n",
    "            # Compose a Datum document referring to this specific image, and\n",
    "            # stash it in _asset_docs_cache.\n",
    "            datum = self._datum_factory({\"index\": i})\n",
    "            self._asset_docs_cache.append((\"datum\", datum))\n",
    "            self.image.set(datum[\"datum_id\"]).wait()\n",
    "            \n",
    "        except Exception as exc:\n",
    "            status.set_exception(exc)\n",
    "        else:\n",
    "            status.set_finished()\n",
    "        finally:\n",
    "            self._acquiring_lock.release()\n",
    "\n",
    "    def collect_asset_docs(self):\n",
    "        \"Yield the documents from our cache, and reset it.\"\n",
    "        yield from self._asset_docs_cache\n",
    "        self._asset_docs_cache.clear()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "demanding-closer",
   "metadata": {},
   "source": [
    "Finally, we will need a File Handler to allow us to load data from the file. Handlers are explained in more detail in the [event model documentation](https://blueskyproject.io/event-model/external.html#handlers).\n",
    "\n",
    "A simple one might look like this:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "critical-shakespeare",
   "metadata": {},
   "outputs": [],
   "source": [
    "class MyHandler:\n",
    "    def __init__(self, resource_path):\n",
    "        # resource_path is really a template string with a %d in it\n",
    "        self._template = resource_path\n",
    "\n",
    "    def __call__(self, index):\n",
    "        import PIL, numpy\n",
    "        filepath = str(self._template) % index\n",
    "        return numpy.asarray(PIL.Image.open(filepath))"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "palestinian-simpson",
   "metadata": {},
   "source": [
    "And, of course, we will want an instance of our `Camera` device to work with:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "earned-syria",
   "metadata": {},
   "outputs": [],
   "source": [
    "camera = Camera(root_path=\"external_data\", name=\"camera\")\n",
    "camera"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "internal-consultancy",
   "metadata": {},
   "source": [
    "## Manually walk through cycle\n",
    "\n",
    "As before, we'll manually walk through the individual steps such as staging and reading from the device. Typically this would be done as part of a plan executed by the RunEngine."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "violent-preliminary",
   "metadata": {},
   "outputs": [],
   "source": [
    "camera.stage()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "killing-paris",
   "metadata": {},
   "outputs": [],
   "source": [
    "status = camera.trigger()\n",
    "status"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "graphic-multiple",
   "metadata": {},
   "outputs": [],
   "source": [
    "status"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "paperback-knight",
   "metadata": {},
   "outputs": [],
   "source": [
    "camera.describe()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "genuine-penetration",
   "metadata": {},
   "outputs": [],
   "source": [
    "camera.read()"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "apart-basin",
   "metadata": {},
   "outputs": [],
   "source": [
    "documents = list(camera.collect_asset_docs())\n",
    "documents"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "brilliant-kennedy",
   "metadata": {},
   "outputs": [],
   "source": [
    "camera.unstage()"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "decimal-housing",
   "metadata": {},
   "source": [
    "## Manually inspect documents and access array data\n",
    "\n",
    "Let's take a closer look at what is going on inside the documents:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "effective-earth",
   "metadata": {},
   "outputs": [],
   "source": [
    "documents"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "vertical-spencer",
   "metadata": {},
   "source": [
    "We can pull out the interesting structures, and finally put our Handler to use:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "indoor-simple",
   "metadata": {},
   "outputs": [],
   "source": [
    "_, resource_document = documents[0]\n",
    "_, datum_document = documents[1]\n",
    "\n",
    "handler = MyHandler(\n",
    "    Path(resource_document[\"root\"], resource_document[\"resource_path\"]),\n",
    "    **resource_document[\"resource_kwargs\"]\n",
    ")"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "large-witness",
   "metadata": {},
   "source": [
    "When we invoke the handler and pass in the `datum_kwargs` with the `index`, we should get back an array with our data:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "incorrect-granny",
   "metadata": {},
   "outputs": [],
   "source": [
    "handler(**datum_document[\"datum_kwargs\"])"
   ]
  },
  {
   "cell_type": "markdown",
   "id": "timely-pharmacology",
   "metadata": {},
   "source": [
    "## Use with Bluesky RunEngine and Databroker"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "increasing-roberts",
   "metadata": {},
   "outputs": [],
   "source": [
    "from bluesky import RunEngine\n",
    "from databroker.v2 import temp\n",
    "\n",
    "RE = RunEngine()\n",
    "db = temp()\n",
    "RE.subscribe(db.v1.insert)\n",
    "\n",
    "db.register_handler(\"MY_FORMAT_SPEC\", MyHandler)"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "coated-humidity",
   "metadata": {},
   "outputs": [],
   "source": [
    "from bluesky.plans import count"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "demographic-tsunami",
   "metadata": {},
   "outputs": [],
   "source": [
    "RE(count([camera]))"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "persistent-omega",
   "metadata": {},
   "outputs": [],
   "source": [
    "run = db[-1]  # Acccess the most recent run.\n",
    "dataset = run.primary.read()  # Access the dataset of its 'primary' stream.\n",
    "dataset"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "id": "reverse-windows",
   "metadata": {},
   "outputs": [],
   "source": [
    "dataset[\"camera_image\"]"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.10"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 5
}