{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Nipype Showcase\n",
    "\n",
    "What's all the hype about Nipype? Is it really that good? Short answer: Yes!\n",
    "\n",
    "Long answer: ... well, let's consider a very simple fMRI preprocessing workflow that just performs:\n",
    "1. slice time correction\n",
    "2. motion correction\n",
    "3. smoothing"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Preparing the preprocessing workflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First, we need to import the main Nipype tools: `Node` and `Workflow`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from nipype import Node, Workflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we can import the interfaces that we want to use for the preprocessing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "from nipype.interfaces.fsl import SliceTimer, MCFLIRT, Smooth"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Next, we will put the three interfaces into a node and define the specific input parameters."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initiate a node to correct for slice wise acquisition\n",
    "slicetimer = Node(SliceTimer(index_dir=False,\n",
    "                             interleaved=True,\n",
    "                             time_repetition=2.5),\n",
    "                  name=\"slicetimer\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initiate a node to correct for motion\n",
    "mcflirt = Node(MCFLIRT(mean_vol=True,\n",
    "                       save_plots=True),\n",
    "               name=\"mcflirt\")"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Initiate a node to smooth functional images\n",
    "smooth = Node(Smooth(fwhm=4), name=\"smooth\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "After creating the nodes, we can now create the preprocessing workflow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "preproc01 = Workflow(name='preproc01', base_dir='.')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, we can put all the nodes into this preprocessing workflow. We specify the data flow / execution flow of the workflow by connecting the corresponding nodes to each other."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "preproc01.connect([(slicetimer, mcflirt, [('slice_time_corrected_file', 'in_file')]),\n",
    "                   (mcflirt, smooth, [('out_file', 'in_file')])])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To better understand what we did we can write out the workflow graph and visualize it directly in this notebook."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "preproc01.write_graph(graph2use='orig')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Visualize graph\n",
    "from IPython.display import Image\n",
    "Image(filename=\"preproc01/graph_detailed.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Run the workflow on one functional image\n",
    "\n",
    "Now, that we've created a workflow, let's run it on a functional image.\n",
    "\n",
    "For this, we first need to specify the input file of the very first node, i.e. the `slicetimer` node."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "slicetimer.inputs.in_file = '/data/ds000114/sub-01/ses-test/func/sub-01_ses-test_task-fingerfootlips_bold.nii.gz'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To show off Nipype's parallelization power, let's run the workflow in parallel, on 5 processors and let's show the execution time:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%time preproc01.run('MultiProc', plugin_args={'n_procs': 5})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "Nice, the whole execution took ~2min. But wait... The parallelization didn't really help.\n",
    "\n",
    "That's true, but because there was no possibility to run the workflow in parallel. Each node depends on the output of the previous node."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Results of `preproc01`\n",
    "\n",
    "So, what did we get? Let's look at the output folder `preproc01`:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!tree preproc01 -I '*js|*json|*pklz|_report|*.dot|*html'"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Rerunning of a workflow"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now, for fun. Let's run the workflow again, but let's change the `fwhm` value of the Gaussian smoothing kernel to `2`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "smooth.inputs.fwhm = 2"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "And let's run the workflow again."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "%time preproc01.run('MultiProc', plugin_args={'n_procs': 5})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Conclusion\n",
    "\n",
    "Interesting, now it only took ~15s to execute the whole workflow again. **What happened?**\n",
    "\n",
    "As you can see from the log above, Nipype didn't execute the two nodes `slicetimer` and `mclfirt` again. This, because their input values didn't change from the last execution. The `preproc01` workflow therefore only had to rerun the node `smooth`."
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Running a workflow in parallel"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ok, ok... Rerunning a workflow again is faster. That's nice and all, but I want more. **You spoke of parallel execution!**\n",
    "\n",
    "We saw that the `preproc01` workflow takes about ~2min to execute completely. So, if we would run the workflow on five functional images, it should take about ~10min total. This, of course, assuming the execution will be done sequentially. Now, let's see how long it takes if we run it in parallel."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# First, let's copy/clone 'preproc01'\n",
    "preproc02 = preproc01.clone('preproc02')\n",
    "preproc03 = preproc01.clone('preproc03')\n",
    "preproc04 = preproc01.clone('preproc04')\n",
    "preproc05 = preproc01.clone('preproc05')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We now have five different preprocessing workflows. If we want to run them in parallel, we can put them all in another workflow."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "metaflow = Workflow(name='metaflow', base_dir='.')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# Now we can add the five preproc workflows to the bigger metaflow\n",
    "metaflow.add_nodes([preproc01, preproc02, preproc03,\n",
    "                    preproc04, preproc05])"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "**Note:** We now have a workflow (`metaflow`), that contains five other workflows (`preproc0?`), each of them containing three nodes.\n",
    "\n",
    "To better understand this, let's visualize this `metaflow`."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# As before, let's write the graph of the workflow\n",
    "metaflow.write_graph(graph2use='flat')"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "# And visualize the graph\n",
    "from IPython.display import Image\n",
    "Image(filename=\"metaflow/graph_detailed.png\")"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Ah... so now we can see that the `metaflow` has potential for parallelization. So let's put it to test"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {
    "scrolled": false
   },
   "outputs": [],
   "source": [
    "%time metaflow.run('MultiProc', plugin_args={'n_procs': 5})"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "This time we can see that Nipype uses all available processors.\n",
    "\n",
    "And if all went well, the total execution time should still be around ~2min.\n",
    "\n",
    "That's why Nipype is so amazing. The days of opening multiple SPMs, FSLs, AFNIs etc. are past!"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Results of `metaflow`"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": null,
   "metadata": {},
   "outputs": [],
   "source": [
    "!tree metaflow -I '*js|*json|*pklz|_report|*.dot|*html'"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Python 3",
   "language": "python",
   "name": "python3"
  },
  "language_info": {
   "codemirror_mode": {
    "name": "ipython",
    "version": 3
   },
   "file_extension": ".py",
   "mimetype": "text/x-python",
   "name": "python",
   "nbconvert_exporter": "python",
   "pygments_lexer": "ipython3",
   "version": "3.7.8"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 2
}