{ "cells": [ { "cell_type": "markdown", "metadata": {}, "source": [ "
\n", "
\n", "
\n", "
\n", "
\n", "\n", "# Automating Neuroimaging Analysis Workflows with Nipype, Arcana and Banana – Sydney University, February 2020\n", "\n", "#### Presented by Tom Close, Monash Biomedical Imaging/National Imaging Facility\n", "\n", "## About\n", "\n", "Analysis of neuroimaging-research data involves the sequential application of algorithms implemented in a number of heterogeneous toolkits (e.g. FSL, SPM, MRTrix, ANTs, AFNI, DiPy). This makes constructing complete workflows challenging as it requires not only the relevant scientific knowledge but also familiarity with the syntax and options of each of the tools involved.\n", "\n", "The workshop will show how to wrap neuroimaging tools within consistent Python interfaces and link them together into robust workflows using *Nipype* (http://nipype.readthedocs.io). Participants will then be shown how common components of these analysis workflows can be consolidated within object-oriented base classes using the *Abstraction of Repository Centric ANAlysis (Arcana)* (http://arcana.readthedocs.io) framework, and how this is used in *Brain imAgiNg Analysis iN Arcana (Banana)* to capture the \"arcana\" (obscure knowledge) of neuroimaging analysis workflow design.\n", "\n", "In the last part of the course, participants will learn how to extend and customise the classes in Banana to the specific needs of their own analysis.\n", "\n", "## Course Info\n", "\n", "The full content of this course, all notebooks and slides can be found on the [github repository](https://github.com/MonashBI/nipype_arcana_workshop).\n", "\n", "Most of the materials for the Nipype section of this course have been adpated from [Michael Notter's](https://github.com/miykael) excellent [Nipype tutorial](https://github.com/miykael/nipype_tutorial) and a series of workshops he and [Peer Herholz](https://github.com/PeerHerholz) have run on \"Python in neuroimaging\" ([Cambridge 2018](https://nbviewer.jupyter.org/github/miykael/workshop_cambridge/blob/master/program.ipynb) and [Marburg 2019](https://nbviewer.jupyter.org/github/PeerHerholz/workshop_marburg/blob/master/program.ipynb)). The introductory notebook on Python classes was written by Steven C. Howell for the [Anaconda Cloud](https://anaconda.org/StevenCHowell/python-class-demonstration/notebook).\n", "\n", "Communication in the lead up to the workshop (and afterwards) will be held in the *nipype-arcana-workshop* channel of the https://melb-neuroinformatics.slack.com Slack workspace (an invite should have been sent to you before the course).\n", "\n", "**Note:** If you are viewing this course via the [nbview.jupyter.org link](https://nbviewer.jupyter.org/github/MonashBI/nipype_arcana_workshop/blob/master/program.ipynb) then you won't be able to actually run any of the notebooks. Please see the [Configuring Your Workstation](#configure) section for instructions on how to run them on your workstation.\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "\n", "\n", "## Using Your Own Workstation\n", "\n", "If you would like to run this tutorial from your own workstation please see the [GitHub README](http://github.com/MonashBI/nipype_arcana_workshop/blob/master/README.md) for instructions on how to use either Docker, Pip or Conda to configure a virtual environment on your workstation.\n", "\n", "For background on virtualisation technologies see\n", "\n", " - Slides: [Virtualisation using Docker, Pip or Conda](slides/background_docker_pip_conda.html)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Using the CVL\n", "\n", "During the workshop you can use the [Characterisation Virtual Laboratory (CVL)](https://www.cvl.org.au) to run this notebook (you can use the CVL to run it before/after too) using a guest account that will be provided to you. If you would like to use the CVL outside of the workshop you will need to [register for an account](https://www.cvl.org.au/cvl-desktop/cvl-accounts) and either [request access to an existing project](https://hpc.erc.monash.edu.au/karaage/) or apply for a new project.\n", "\n", "\n", "### Logging in to the CVL\n", "\n", "Follow these instructions to use the CVL for the first time.\n", "\n", "1. Open a new browser tab on Chrome \n", "1. https://desktop.cvl.org.au \n", "1. Select a remote system: M3 and CVL @ MASSIVE\n", "1. Click login\n", "1. Login with your university credentials\n", "1. It will request permissions to use your email address for login\n", "1. From the left pane select P4 - Light compute\n", "1. From the right pane select your project ('training' during the workshop)\n", "1. Click launch\n", "1. Click show desktop\n", "1. It will open a new tab with the CVL desktop (using an application call strudel)\n", "1. On the top menu click on the black box - terminal\n", "1. If you want the font to be bigger, click View on the top menu and then zoom in\n", "\n", "\n", "### Running the course materials\n", "\n", " \n", " 1. Create your own directory in the `~/` (project) directory\n", " ```bash\n", " mkdir ~/training/${USER}\n", " ```\n", " \n", " 1. In your directory, clone the workshop repository\n", " ```bash\n", " cd ~/training/${USER}\n", " git clone https://github.com/MonashBI/nipype_arcana_workshop\n", " ```\n", " \n", " 1. Create symlinks to shared reference data within your repository\n", " ```bash\n", " cd nipype_arcana_workshop\n", " ln -s ~/training/additional-data/* notebooks/data/\n", " ```\n", " \n", " 1. Make a directory for outputs\n", " ```bash\n", " mkdir notebooks/output\n", " ```\n", " \n", " 1. Create a work directory and link it into to the notebooks\n", " ```bash\n", " mkdir ~/training_scratch/${USER}\n", "ln -s ~/training_scratch/${USER} notebooks/work\n", " ```\n", "\n", " 1. Load the 'neuro-workshop' module\n", " ```bash\n", " module load neuro-workshop\n", " ```\n", " \n", " 1. Enable the \"Exercise 2\" notebook extension (to activate the \"hiding\" of solutions by default)\n", " ```bash\n", " jupyter nbextension enable exercise2/main\n", " ```\n", " \n", " 1. Run the notebook\n", " ```bash\n", " jupyter notebook program.ipynb\n", " ```\n", "\n", "This will open up a Firefox window with an interactive version of this program, which you can link to interactive versions of the course materials from.\n", "\n", "***Note:*** The visualisations won't work with Firefox but will with Chrome (but the slides won't visualise properly with Chrome). You can run this notebook with Chrome by copying the 'localhost+token' link displayed in your terminal after you have run the notebook into Chrome's navigation bar." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Background\n", "\n", "The section contains some background information required for the workshop and working with scientific Python packages in general. If you have any questions regarding any of the sections please write to the workshop Slack channel and *@Tom Close*.\n", "\n", "\n", "### Introduction to Python and Jupyter Notebooks (**Essential**)\n", "\n", "This section is meant as a quick introduction to Jupyter Notebooks, Python and Object-Orientated (OO) programming. It is ***STRONGLY RECOMMENDED*** that you go through this section **before** the workshop if you are not already familiar with these technologies/concepts. \n", "\n", "- Slides: [Introduction to Python, Jupyter Notebook and OO Design](slides/background_python_and_jupyter_notebook.html)\n", "- Notebook 1: [Jupyter Notebook](notebooks/introduction_jupyter-notebook.ipynb)\n", "- Notebook 2: [Python Intro](notebooks/introduction_python.ipynb)\n", "- Notebook 3: [Classes & Objects](notebooks/introduction_oo.ipynb)\n", "\n", "\n", "### How to handle your MRI data with Nibabel and Nilearn\n", "\n", "It's liberating to have direct access to your neuroimaging data. `Nibabel` and `Nilearn` allow exactly that. With those two neuroimaging packages, you can consider the brain a simple 3D/4D matrix of datapoints and do with it whatever you want.\n", "\n", "- Slides: [Data Manipulation](slides/background_data_manipulation.html)\n", "- Notebook 1: [Nibabel](notebooks/image_manipulation_nibabel.ipynb)\n", "- Notebook 2: [Nilearn](notebooks/image_manipulation_nilearn.ipynb)\n", "\n", "### Crash course in Python scientific toolboxes\n", "\n", "One advantage of Python is the vast availability of toolboxes. There's a toolbox for almost everything! In this section, we want to introduce you to the main scientific toolboxes that every researcher should know. While not essential for this workshop they will definitely come in handy in the future.\n", "\n", "- Slides: [Scientific Toolboxes](slides/background_scientific_toolboxes.html)\n", "- Notebook 1: [Numpy](notebooks/python_numpy.ipynb)\n", "- Notebook 2: [Statistics](notebooks/python_statistics.ipynb)\n", "- Notebook 3: [Visualization](notebooks/python_visualization.ipynb) (when running on CVL use Chrome for to avoid issue with WebGL for one of the visualisaions)\n", "- Notebook 4: [Scipy](notebooks/python_scipy.ipynb)\n", "- Notebook 5: [Scikit](notebooks/python_scikit.ipynb)\n" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Workshop Program (1-day)\n", "\n", "The workshop will be held over one day and is split into two parts: an introduction to Nipype concepts and applications in the morning, and how to use Arcana and Banana in the afternoon.\n", "\n", "\n", "### `9:00-9:30` Introduction\n", "\n", "In this short introduction, we will explain a little about what Nipype, Arcana and Banana are and how they relate to eacher other.\n", "\n", " - Slides: [Intro to the Workshop](slides/intro.html)\n", " - Slides: [Introduction to Nipype, Arcana & Banana](slides/intro_nipype_arcana_banana.html)\n", " - Notebook 1: [Nipype Showcase](notebooks/nipype_showcase.ipynb)\n", "\n", "

Nipype\n", "\n", "### `9:30-11:00` Nipype Basics: Interfaces, Nodes & Workflows\n", "\n", "Nipype can be learned very quickly, but it's nonetheless important that you know about some of the main building blocks.\n", "\n", " - Slides: [Nipype Basics](slides/nipype_basics.html)\n", " - Notebook 1: [Interfaces](notebooks/basic_interfaces.ipynb)\n", " - Notebook 2: [Nodes](notebooks/basic_nodes.ipynb)\n", " - Notebook 3: [Workflows](notebooks/basic_workflow.ipynb)\n", " - Notebook 4: [Visualisation](notebooks/basic_graph_visualization.ipynb)\n", " - Notebook 5: [Execution Plugins](notebooks/basic_plugins.ipynb) (stretch)\n", " - Notebook 6: [Execution Configuration](notebooks/basic_execution_configuration.ipynb) (stretch)\n", " - Notebook 7: [Errors & Crashes](notebooks/basic_error_and_crashes.ipynb) (stretch)\n", " - Notebook 8: [Debugging Workflows](notebooks/basic_debug.ipynb) (stretch)\n", " - Notebook 9: [Data Input](notebooks/basic_data_input.ipynb) (stretch)\n", " - Notebook 10: [Data Output](notebooks/basic_data_output.ipynb) (stretch)\n", " \n", "\n", "### `11:00-11:30` Coffee & Tea Break\n", "\n", "\n", "### `11:30-12:30` Advanced Nipype: Iteration & Custom Interfaces\n", "\n", "Once you have the building blocks in place, you can start iterating over your data and\n", "writing custom interfaces\n", "\n", " - Slides: [Advanced Nipype Concepts](slides/nipype_advanced.html)\n", " - Notebook 1: [Iteration/Iterables](notebooks/basic_iteration.ipynb)\n", " - Notebook 2: [Map Nodes](notebooks/basic_mapnodes.ipynb)\n", " - Notebook 3: [Join Nodes](notebooks/basic_joinnodes.ipynb)\n", " - Notebook 4: [Function Interfaces](notebooks/basic_function_interface.ipynb)\n", " - Notebook 5: [Writing Your Own Interfaces](notebooks/advanced_create_interfaces.ipynb) (stretch)\n", "\n", "### `12:30-13:30` Lunch" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "

Arcana/Banana

\n", "\n", "### `13:30-15:00` Abstraction of Repository-Centric ANAlysis (Arcana) \n", " \n", "Arcana is a framework for designing \"data-centric\" analysis suites for different types of dataset (e.g. DWI, fMRI or PET images). In this section you will learn how to apply existing analyses to a dataset and construct your own analyses.\n", "\n", " - Slides: [Introduction to Arcana](slides/arcana.html)\n", " - Notebook 1: [Applying Analysis Classes](notebooks/arcana_application.ipynb)\n", " - Notebook 2: [Designing Analysis Classes](notebooks/arcana_design.ipynb)\n", "\n", " \n", "### `15:00-15:30` Coffee & Tea Break\n", "\n", "\n", "### `15:30-17:00` Brain imAgiNg Analysis iN Arcana (Banana)\n", "\n", "Banana implements analysis methods for a a range of MR contrasts (e.g. DWI, T1w, BOLD, T2star) in Analysis classes. In this section you will learn how to use Banana to analyse datasets and extended it to meet the requirements of your analysis.\n", "\n", " - Slides: [Introduction to Banana](slides/banana.html)\n", " - Notebook 1: [Using the Banana Command Line Interface](notebooks/banana_cli.ipynb)\n", " - Notebook 2: [Customising Banana for Your Analysis](notebooks/banana_extending.ipynb)\n", " \n", "**Note:** If you would like to use Banana to analyse data from your own project, please ensure you have access to another project on MASSIVE (i.e. not just 'training' as the data is shared amongst all particpants), or bring along a laptop configured for this course (see [Configuring Your Workstation](#configure)) and your data, and have a go during this section :)" ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [] } ], "metadata": { "file_extension": ".py", "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.5" }, "mimetype": "text/x-python", "name": "python", "npconvert_exporter": "python", "pygments_lexer": "ipython3", "version": 3 }, "nbformat": 4, "nbformat_minor": 2 }