{
 "cells": [
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# fMRI preprocessing\n",
    "This tutorial shows how to run the NIAK fMRI preprocessing pipeline, using a limited set of options. \n",
    " * **More documentation** on the pipeline can be found [here](http://niak.simexp-lab.org/pipe_preprocessing.html)\n",
    " * **Download:** This .ipynb notebook can be found [here](https://raw.githubusercontent.com/SIMEXP/niak_tutorials/master/niak_tutorial_fmri_preprocessing.ipynb) and as a .m Octave script [here](https://raw.githubusercontent.com/SIMEXP/niak_tutorials/master/niak_tutorial_fmri_preprocessing.m). \n",
    " * **Time for completion**: about 30 minutes. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Downloading data\n",
    "First download a small fMRI dataset, with a structural scan. Be aware that all raw and derivatives data will be generated in the current folder. Note that you will need to manually remove the `data_test_niak_mnc1` and `fmri_preprocess` folders to restart this tutorial from scratch."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {},
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "--2017-04-29 11:24:32--  https://github.com/SIMEXP/niak/releases/download/data-test-v1.0/data_test_niak_nii.zip\n",
      "Resolving github.com (github.com)... 192.30.253.113, 192.30.253.112\n",
      "Connecting to github.com (github.com)|192.30.253.113|:443... connected.\n",
      "HTTP request sent, awaiting response... 302 Found\n",
      "Location: https://github-cloud.s3.amazonaws.com/releases/21707384/7e845d76-2c05-11e7-897f-59bcbb27c4a6.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAISTNZFOVBIJMK3TQ%2F20170429%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20170429T112432Z&X-Amz-Expires=300&X-Amz-Signature=186f851e26a81baef60b568a1d5148e2238d5531b0c3f9f904f7ef627931355a&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Ddata_test_niak_nii.zip&response-content-type=application%2Foctet-stream [following]\n",
      "--2017-04-29 11:24:32--  https://github-cloud.s3.amazonaws.com/releases/21707384/7e845d76-2c05-11e7-897f-59bcbb27c4a6.zip?X-Amz-Algorithm=AWS4-HMAC-SHA256&X-Amz-Credential=AKIAISTNZFOVBIJMK3TQ%2F20170429%2Fus-east-1%2Fs3%2Faws4_request&X-Amz-Date=20170429T112432Z&X-Amz-Expires=300&X-Amz-Signature=186f851e26a81baef60b568a1d5148e2238d5531b0c3f9f904f7ef627931355a&X-Amz-SignedHeaders=host&actor_id=0&response-content-disposition=attachment%3B%20filename%3Ddata_test_niak_nii.zip&response-content-type=application%2Foctet-stream\n",
      "Resolving github-cloud.s3.amazonaws.com (github-cloud.s3.amazonaws.com)... 54.231.49.120\n",
      "Connecting to github-cloud.s3.amazonaws.com (github-cloud.s3.amazonaws.com)|54.231.49.120|:443... connected.\n",
      "HTTP request sent, awaiting response... 200 OK\n",
      "Length: 2801759 (2.7M) [application/octet-stream]\n",
      "Saving to: `/sandbox/home/git/niak_tutorials/data_test_niak_nii/data_test_niak_nii.zip'\n",
      "\n",
      "100%[======================================>] 2,801,759   1.57M/s   in 1.7s    \n",
      "\n",
      "2017-04-29 11:24:34 (1.57 MB/s) - `/sandbox/home/git/niak_tutorials/data_test_niak_nii/data_test_niak_nii.zip' saved [2801759/2801759]\n",
      "\n",
      "Deleting file '/sandbox/home/git/niak_tutorials/data_test_niak_nii/data_test_niak_nii.zip' \n",
      "ans = 0\n"
     ]
    }
   ],
   "source": [
    "clear\n",
    "niak_wget('data_test_niak_nii');"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Preparing files\n",
    "Now, set up the names of the structural and functional files."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "path_data = [pwd filesep];\n",
    "% Structural scan subject 1\n",
    "files_in.subject1.anat = ...\n",
    "    [path_data 'data_test_niak_nii/anat_subject1.nii.gz'];       \n",
    "% fMRI run 1 subject 1\n",
    "files_in.subject1.fmri.session1.motor = ...\n",
    "    [path_data 'data_test_niak_nii/func_motor_subject1.nii.gz'];\n",
    "% Structural scan subject 2\n",
    "files_in.subject2.anat = ...\n",
    "    [path_data 'data_test_niak_nii/anat_subject2.nii.gz'];       \n",
    "% fMRI run 1 subject 2\n",
    "files_in.subject2.fmri.session1.motor = ...\n",
    "    [path_data 'data_test_niak_nii/func_motor_subject2.nii.gz'];"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We also specify where to write the results."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "% Where to store the results\n",
    "opt.folder_out  = [path_data 'fmri_preprocess/'];"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Parallel computing \n",
    "Next we specify how many threads to use. A value of `N` means that up to `N` jobs can be executed in parallel. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "% Use up to 2 threads\n",
    "opt.psom.max_queued = 2;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Slice timing\n",
    "We now set the options of the slice timing correction. Note that we specify the type of the scanner (in practice, only 'Siemens' has an impact), because the definition of the scanner impacts the definition of the slice timing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "opt.slice_timing.type_acquisition = 'interleaved ascending'; \n",
    "opt.slice_timing.type_scanner     = 'Bruker';                \n",
    "opt.slice_timing.delay_in_tr      = 0;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "other options are available as part of the slice timing step, which are not part of the slice timing per say. It is notably possible to center the functional images (if the header information is not accurate), or to suppress some volumes at the beginning of the time series: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "% Center the functional volumes on the brain center-of-mass (true/false)\n",
    "opt.slice_timing.flag_center = false;\n",
    "% Suppress some volumes at the beginning of the run\n",
    "opt.slice_timing.suppress_vol = 3;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Resampling\n",
    "The voxel size for resampling in stereotaxic space is specified by the user:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "% The voxel size to use in the stereotaxic space\n",
    "opt.resample_vol.voxel_size    = 10;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## T1 processing\n",
    "The options for non-uniformity correction of the T1 image is often useful to tweak:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "% Parameter for non-uniformity correction. \n",
    "% 200 is a suggested value for 1.5T images, \n",
    "% 75 for 3T images. \n",
    "opt.t1_preprocess.nu_correct.arg = '-distance 75';"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Regression of confounds\n",
    "The options for temporal filtering, motion parameter regression, white matter and ventricle signal regression, COMPCOR, global signal correction and scrubbing."
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "% Cut-off frequency for high-pass filtering, \n",
    "% or removal of low frequencies (in Hz). \n",
    "opt.time_filter.hp = 0.01; \n",
    "% Cut-off frequency for low-pass filtering, \n",
    "% or removal of high frequencies (in Hz). \n",
    "opt.time_filter.lp = 0.1;\n",
    "% Remove slow time drifts (true/false)\n",
    "opt.regress_confounds.flag_slow = true;\n",
    "% Remove high frequencies (true/false)\n",
    "opt.regress_confounds.flag_high = false;\n",
    "% Apply regression of motion parameters (true/false)\n",
    "opt.regress_confounds.flag_motion_params = true;\n",
    "% Reduce the dimensionality of motion parameters with PCA (true/false)\n",
    "opt.regress_confounds.flag_pca_motion = true;\n",
    "% How much variance of motion parameters (with squares) to retain\n",
    "opt.regress_confounds.pct_var_explained = 0.95;\n",
    "% Apply average white matter signal regression (true/false)         \n",
    "opt.regress_confounds.flag_wm = true;\n",
    "% Apply average ventricle signal regression (true/false)         \n",
    "opt.regress_confounds.flag_vent = true;\n",
    "% Apply anat COMPCOR (white matter+ventricles, true/false)\n",
    "% We recommend not using FLAG_WM and FLAG_VENT together with FLAG_COMPCOR\n",
    "opt.regress_confounds.flag_compcor = false;\n",
    "% Apply global signal regression (true/false)         \n",
    "opt.regress_confounds.flag_gsc = true; \n",
    "% Apply scrubbing (true/false)\n",
    "opt.regress_confounds.flag_scrubbing = true;     \n",
    "% The threshold on frame displacement for scrubbing \n",
    "opt.regress_confounds.thre_fd = 0.5;    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Spatial smoothing\n",
    "The size of the spatial Gaussian blurring kernel: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "% Full-width at maximum (FWHM) of the Gaussian blurring kernel, in mm.\n",
    "opt.smooth_vol.fwhm      = 6;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Run the pipeline\n",
    "This command can take up to 20-30 minutes to complete. It runs the full preprocessing pipeline on one subject/run. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "scrolled": true
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Generating pipeline for individual fMRI preprocessing :\n",
      "    Adding subject1 ; 0.22 sec\n",
      "    Adding subject2 ; 0.08 sec\n",
      "Adding group-level quality control of coregistration in anatomical space (linear stereotaxic space) ; 0.01 sec\n",
      "Adding group-level quality control of coregistration in anatomical space (non-linear stereotaxic space) ; 0.01 sec\n",
      "Adding group-level quality control of coregistration in functional space ; 0.01 sec\n",
      "Adding group-level quality control of motion correction (motion parameters) ; 0.02 sec\n",
      "Adding group-level quality control of scrubbing time frames with excessive motion ; 0.00 sec\n",
      "Adding the report on fMRI preprocessing ; 0.25 sec\n",
      "\n",
      "Logs will be stored in /sandbox/home/git/niak_tutorials/fmri_preprocess/logs/\n",
      "Generating dependencies ...\n",
      "   Percentage completed :  0 5 10 15 20 25 30 35 40 45 50 55 60 65 70 75 80 85 90 95 100- 0.07 sec\n",
      "Setting up the to-do list ...\n",
      "   I found 75 job(s) to do.\n",
      "I could not find any log file. This pipeline has not been started (yet?). Press CTRL-C to cancel.\n",
      "Deamon started on 29-Apr-2017 11:24:36\n",
      "29-Apr-2017 11:24:36 Starting the pipeline manager...\n",
      "29-Apr-2017 11:24:36 Starting the garbage collector...\n",
      "29-Apr-2017 11:24:37 Starting worker number 1...\n",
      "29-Apr-2017 11:24:37 Starting worker number 2...\n",
      "\n",
      "Pipeline started on 29-Apr-2017 11:24:37\n",
      "user: , host: 3f86dc5d68dc, system: unix\n",
      "****************************************\n",
      "29-Apr-2017 11:24:38 pipe_params                               submitted  (1 run | 0 fail | 0 done | 74 left)\n",
      "29-Apr-2017 11:24:38 resample_aal                              submitted  (2 run | 0 fail | 0 done | 73 left)\n",
      "29-Apr-2017 11:24:38 cp_confounds_keys                         submitted  (3 run | 0 fail | 0 done | 72 left)\n",
      "29-Apr-2017 11:24:38 cp_template                               submitted  (4 run | 0 fail | 0 done | 71 left)\n",
      "29-Apr-2017 11:24:39 pipe_params                               finished   (3 run | 0 fail | 1 done | 71 left)\n",
      "29-Apr-2017 11:24:39 cp_confounds_keys                         finished   (2 run | 0 fail | 2 done | 71 left)\n",
      "29-Apr-2017 11:24:39 resample_fmri_stereo                      submitted  (3 run | 0 fail | 2 done | 70 left)\n",
      "29-Apr-2017 11:24:39 t1_preprocess_subject1                    submitted  (4 run | 0 fail | 2 done | 69 left)\n",
      "29-Apr-2017 11:24:40 resample_fmri_stereo                      finished   (3 run | 0 fail | 3 done | 69 left)\n",
      "29-Apr-2017 11:24:40 slice_timing_subject1_session1_motor      submitted  (4 run | 0 fail | 3 done | 68 left)\n",
      "29-Apr-2017 11:24:46 cp_template                               finished   (3 run | 0 fail | 4 done | 68 left)\n",
      "29-Apr-2017 11:24:46 t1_preprocess_subject2                    submitted  (4 run | 0 fail | 4 done | 67 left)\n",
      "29-Apr-2017 11:24:47 resample_aal                              finished   (3 run | 0 fail | 5 done | 67 left)\n",
      "29-Apr-2017 11:24:47 slice_timing_subject2_session1_motor      submitted  (4 run | 0 fail | 5 done | 66 left)\n",
      ".........................\n",
      "29-Apr-2017 11:37:39 t1_preprocess_subject1                    finished   (3 run | 0 fail | 6 done | 66 left)\n",
      "29-Apr-2017 11:37:39 mask_anat2func_subject1                   submitted  (4 run | 0 fail | 6 done | 65 left)\n",
      "29-Apr-2017 11:37:40 slice_timing_subject1_session1_motor      finished   (3 run | 0 fail | 7 done | 65 left)\n",
      "29-Apr-2017 11:37:40 motion_target_subject1_session1_motor     submitted  (4 run | 0 fail | 7 done | 64 left)\n",
      ".\n",
      "29-Apr-2017 11:38:00 t1_preprocess_subject2                    finished   (3 run | 0 fail | 8 done | 64 left)\n",
      "29-Apr-2017 11:38:00 mask_anat2func_subject2                   submitted  (4 run | 0 fail | 8 done | 63 left)\n",
      "29-Apr-2017 11:38:00 slice_timing_subject2_session1_motor      finished   (3 run | 0 fail | 9 done | 63 left)\n",
      "29-Apr-2017 11:38:00 motion_target_subject2_session1_motor     submitted  (4 run | 0 fail | 9 done | 62 left)\n",
      "..\n",
      "29-Apr-2017 11:39:09 motion_target_subject1_session1_motor     finished   (3 run | 0 fail | 10 done | 62 left)\n",
      "29-Apr-2017 11:39:09 mask_anat2func_subject1                   finished   (2 run | 0 fail | 11 done | 62 left)\n",
      "29-Apr-2017 11:39:09 motion_Wrun_subject1_session1_motor       submitted  (3 run | 0 fail | 11 done | 61 left)\n",
      "29-Apr-2017 11:39:09 anat2func_subject1                        submitted  (4 run | 0 fail | 11 done | 60 left)\n",
      ".\n",
      "29-Apr-2017 11:39:26 mask_anat2func_subject2                   finished   (3 run | 0 fail | 12 done | 60 left)\n",
      "29-Apr-2017 11:39:26 qc_coregister_group_anat_stereolin        submitted  (4 run | 0 fail | 12 done | 59 left)\n",
      "29-Apr-2017 11:39:26 motion_target_subject2_session1_motor     finished   (3 run | 0 fail | 13 done | 59 left)\n",
      "29-Apr-2017 11:39:27 motion_Wrun_subject2_session1_motor       submitted  (4 run | 0 fail | 13 done | 58 left)\n",
      ".\n",
      "29-Apr-2017 11:40:04 qc_coregister_group_anat_stereolin        finished   (3 run | 0 fail | 14 done | 58 left)\n",
      "29-Apr-2017 11:40:04 anat2func_subject2                        submitted  (4 run | 0 fail | 14 done | 57 left)\n",
      "29-Apr-2017 11:40:11 motion_Wrun_subject2_session1_motor       finished   (3 run | 0 fail | 15 done | 57 left)\n",
      "29-Apr-2017 11:40:11 motion_parameters_subject2_session1_motor submitted  (4 run | 0 fail | 15 done | 56 left)\n",
      "..........\n",
      "29-Apr-2017 11:44:52 anat2func_subject1                        finished   (3 run | 0 fail | 16 done | 56 left)\n",
      "29-Apr-2017 11:44:52 concat_transf_nl_subject1                 submitted  (4 run | 0 fail | 16 done | 55 left)\n",
      "29-Apr-2017 11:44:59 motion_Wrun_subject1_session1_motor       finished   (3 run | 0 fail | 17 done | 55 left)\n",
      "29-Apr-2017 11:44:59 motion_parameters_subject1_session1_motor submitted  (4 run | 0 fail | 17 done | 54 left)\n",
      "29-Apr-2017 11:45:00 motion_parameters_subject1_session1_motor finished   (3 run | 0 fail | 18 done | 54 left)\n",
      "29-Apr-2017 11:45:00 concat_transf_nl_subject1                 finished   (2 run | 0 fail | 19 done | 54 left)\n",
      "29-Apr-2017 11:45:00 resample_subject1_session1_motor          submitted  (3 run | 0 fail | 19 done | 53 left)\n",
      "29-Apr-2017 11:45:00 qc_group_coregister_anat_stereonl         submitted  (4 run | 0 fail | 19 done | 52 left)\n",
      ".\n",
      "29-Apr-2017 11:45:38 qc_group_coregister_anat_stereonl         finished   (3 run | 0 fail | 20 done | 52 left)\n",
      "29-Apr-2017 11:45:38 rep_cp_report_templates                   submitted  (4 run | 0 fail | 20 done | 51 left)\n",
      "..\n",
      "29-Apr-2017 11:46:23 motion_parameters_subject2_session1_motor finished   (3 run | 0 fail | 21 done | 51 left)\n",
      "29-Apr-2017 11:46:23 anat2func_subject2                        finished   (2 run | 0 fail | 22 done | 51 left)\n",
      "29-Apr-2017 11:46:23 concat_transf_nl_subject2                 submitted  (3 run | 0 fail | 22 done | 50 left)\n",
      "29-Apr-2017 11:46:23 rep_params                                submitted  (4 run | 0 fail | 22 done | 49 left)\n",
      "29-Apr-2017 11:46:25 concat_transf_nl_subject2                 finished   (3 run | 0 fail | 23 done | 49 left)\n",
      "29-Apr-2017 11:46:25 resample_subject2_session1_motor          submitted  (4 run | 0 fail | 23 done | 48 left)\n",
      "29-Apr-2017 11:46:25 rep_params                                finished   (3 run | 0 fail | 24 done | 48 left)\n",
      "29-Apr-2017 11:46:25 rep_summary_anat                          submitted  (4 run | 0 fail | 24 done | 47 left)\n",
      "29-Apr-2017 11:46:32 resample_subject1_session1_motor          finished   (3 run | 0 fail | 25 done | 47 left)\n",
      "29-Apr-2017 11:46:33 rep_cp_report_templates                   finished   (2 run | 0 fail | 26 done | 47 left)\n",
      "29-Apr-2017 11:46:33 qc_motion_subject1                        submitted  (3 run | 0 fail | 26 done | 46 left)\n",
      "29-Apr-2017 11:46:33 time_filter_subject1_session1_motor       submitted  (4 run | 0 fail | 26 done | 45 left)\n",
      "29-Apr-2017 11:46:37 qc_motion_subject1                        finished   (3 run | 0 fail | 27 done | 45 left)\n",
      "29-Apr-2017 11:46:37 mask_confounds_subject1                   submitted  (4 run | 0 fail | 27 done | 44 left)\n",
      "29-Apr-2017 11:46:37 time_filter_subject1_session1_motor       finished   (3 run | 0 fail | 28 done | 44 left)\n",
      "29-Apr-2017 11:46:37 rep_template_stereo                       submitted  (4 run | 0 fail | 28 done | 43 left)\n",
      "29-Apr-2017 11:46:47 mask_confounds_subject1                   finished   (3 run | 0 fail | 29 done | 43 left)\n",
      "29-Apr-2017 11:46:47 build_confounds_subject1_session1_motor   submitted  (4 run | 0 fail | 29 done | 42 left)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      ".\n",
      "29-Apr-2017 11:47:00 rep_template_stereo                       finished   (3 run | 0 fail | 30 done | 42 left)\n",
      "29-Apr-2017 11:47:00 rep_average_t1_stereo                     submitted  (4 run | 0 fail | 30 done | 41 left)\n",
      "29-Apr-2017 11:47:01 build_confounds_subject1_session1_motor   finished   (3 run | 0 fail | 31 done | 41 left)\n",
      "29-Apr-2017 11:47:01 regress_confounds_subject1_session1_motor submitted  (4 run | 0 fail | 31 done | 40 left)\n",
      "29-Apr-2017 11:47:11 rep_average_t1_stereo                     finished   (3 run | 0 fail | 32 done | 40 left)\n",
      "29-Apr-2017 11:47:12 rep_t1_outline_registration               submitted  (4 run | 0 fail | 32 done | 39 left)\n",
      "29-Apr-2017 11:47:12 regress_confounds_subject1_session1_motor finished   (3 run | 0 fail | 33 done | 39 left)\n",
      "29-Apr-2017 11:47:12 smooth_subject1_session1_motor            submitted  (4 run | 0 fail | 33 done | 38 left)\n",
      "29-Apr-2017 11:47:19 rep_t1_outline_registration               finished   (3 run | 0 fail | 34 done | 38 left)\n",
      "29-Apr-2017 11:47:19 rep_t1_subject1                           submitted  (4 run | 0 fail | 34 done | 37 left)\n",
      ".\n",
      "29-Apr-2017 11:47:27 resample_subject2_session1_motor          finished   (3 run | 0 fail | 35 done | 37 left)\n",
      "29-Apr-2017 11:47:28 rep_summary_anat                          finished   (2 run | 0 fail | 36 done | 37 left)\n",
      "29-Apr-2017 11:47:28 qc_motion_subject2                        submitted  (3 run | 0 fail | 36 done | 36 left)\n",
      "29-Apr-2017 11:47:28 time_filter_subject2_session1_motor       submitted  (4 run | 0 fail | 36 done | 35 left)\n",
      "29-Apr-2017 11:47:30 smooth_subject1_session1_motor            finished   (3 run | 0 fail | 37 done | 35 left)\n",
      "29-Apr-2017 11:47:30 rep_t1_subject2                           submitted  (4 run | 0 fail | 37 done | 34 left)\n",
      "29-Apr-2017 11:47:32 qc_motion_subject2                        finished   (3 run | 0 fail | 38 done | 34 left)\n",
      "29-Apr-2017 11:47:32 time_filter_subject2_session1_motor       finished   (2 run | 0 fail | 39 done | 34 left)\n",
      "29-Apr-2017 11:47:32 mask_confounds_subject2                   submitted  (3 run | 0 fail | 39 done | 33 left)\n",
      "29-Apr-2017 11:47:32 qc_group_coregister_func                  submitted  (4 run | 0 fail | 39 done | 32 left)\n",
      "29-Apr-2017 11:47:40 rep_t1_subject1                           finished   (3 run | 0 fail | 40 done | 32 left)\n",
      "29-Apr-2017 11:47:40 qc_group_motion_estimation                submitted  (4 run | 0 fail | 40 done | 31 left)\n",
      "29-Apr-2017 11:47:41 mask_confounds_subject2                   finished   (3 run | 0 fail | 41 done | 31 left)\n",
      "29-Apr-2017 11:47:41 build_confounds_subject2_session1_motor   submitted  (4 run | 0 fail | 41 done | 30 left)\n",
      "29-Apr-2017 11:47:43 qc_group_coregister_func                  finished   (3 run | 0 fail | 42 done | 30 left)\n",
      "29-Apr-2017 11:47:43 rep_summary_func                          submitted  (4 run | 0 fail | 42 done | 29 left)\n",
      "29-Apr-2017 11:47:44 build_confounds_subject2_session1_motor   finished   (3 run | 0 fail | 43 done | 29 left)\n",
      "29-Apr-2017 11:47:44 rep_summary_func                          finished   (2 run | 0 fail | 44 done | 29 left)\n",
      "29-Apr-2017 11:47:44 regress_confounds_subject2_session1_motor submitted  (3 run | 0 fail | 44 done | 28 left)\n",
      "29-Apr-2017 11:47:44 rep_summary_intra                         submitted  (4 run | 0 fail | 44 done | 27 left)\n",
      "29-Apr-2017 11:47:45 regress_confounds_subject2_session1_motor finished   (3 run | 0 fail | 45 done | 27 left)\n",
      "29-Apr-2017 11:47:45 rep_summary_intra                         finished   (2 run | 0 fail | 46 done | 27 left)\n",
      "29-Apr-2017 11:47:45 smooth_subject2_session1_motor            submitted  (3 run | 0 fail | 46 done | 26 left)\n",
      "29-Apr-2017 11:47:45 qc_group_scrubbing                        submitted  (4 run | 0 fail | 46 done | 25 left)\n",
      "29-Apr-2017 11:47:45 qc_group_scrubbing                        finished   (3 run | 0 fail | 47 done | 25 left)\n",
      "29-Apr-2017 11:47:45 rep_summary_scrubbing                     submitted  (4 run | 0 fail | 47 done | 24 left)\n",
      "29-Apr-2017 11:47:50 rep_t1_subject2                           finished   (3 run | 0 fail | 48 done | 24 left)\n",
      "29-Apr-2017 11:47:50 rep_average_func_stereo                   submitted  (4 run | 0 fail | 48 done | 23 left)\n",
      "29-Apr-2017 11:47:52 qc_group_motion_estimation                finished   (3 run | 0 fail | 49 done | 23 left)\n",
      "29-Apr-2017 11:47:52 rep_mask_func_group_stereo                submitted  (4 run | 0 fail | 49 done | 22 left)\n",
      "29-Apr-2017 11:47:53 rep_average_func_stereo                   finished   (3 run | 0 fail | 50 done | 22 left)\n",
      "29-Apr-2017 11:47:53 rep_avg_mask_func_stereo                  submitted  (4 run | 0 fail | 50 done | 21 left)\n",
      ".\n",
      "29-Apr-2017 11:47:54 smooth_subject2_session1_motor            finished   (3 run | 0 fail | 51 done | 21 left)\n",
      "29-Apr-2017 11:47:54 rep_summary_scrubbing                     finished   (2 run | 0 fail | 52 done | 21 left)\n",
      "29-Apr-2017 11:47:54 rep_bold_subject1                         submitted  (3 run | 0 fail | 52 done | 20 left)\n",
      "29-Apr-2017 11:47:54 rep_bold_subject2                         submitted  (4 run | 0 fail | 52 done | 19 left)\n",
      "29-Apr-2017 11:47:55 rep_mask_func_group_stereo                finished   (3 run | 0 fail | 53 done | 19 left)\n",
      "29-Apr-2017 11:47:55 rep_t1_subject1_overlay                   submitted  (4 run | 0 fail | 53 done | 18 left)\n",
      "29-Apr-2017 11:47:56 rep_bold_subject1                         finished   (3 run | 0 fail | 54 done | 18 left)\n",
      "29-Apr-2017 11:47:56 rep_t1_subject2_overlay                   submitted  (4 run | 0 fail | 54 done | 17 left)\n",
      "29-Apr-2017 11:47:56 rep_avg_mask_func_stereo                  finished   (3 run | 0 fail | 55 done | 17 left)\n",
      "29-Apr-2017 11:47:56 rep_template_stereo_overlay               submitted  (4 run | 0 fail | 55 done | 16 left)\n",
      "29-Apr-2017 11:47:57 rep_t1_subject1_overlay                   finished   (3 run | 0 fail | 56 done | 16 left)\n",
      "29-Apr-2017 11:47:57 rep_init_report                           submitted  (4 run | 0 fail | 56 done | 15 left)\n",
      "29-Apr-2017 11:47:57 rep_bold_subject2                         finished   (3 run | 0 fail | 57 done | 15 left)\n",
      "29-Apr-2017 11:47:57 rep_template_stereo_overlay               finished   (2 run | 0 fail | 58 done | 15 left)\n",
      "29-Apr-2017 11:47:57 rep_init_report                           finished   (1 run | 0 fail | 59 done | 15 left)\n",
      "29-Apr-2017 11:47:57 rep_motion_native_subject1_session1_motor submitted  (2 run | 0 fail | 59 done | 14 left)\n",
      "29-Apr-2017 11:47:57 rep_target_native_subject1_session1_motor submitted  (3 run | 0 fail | 59 done | 13 left)\n",
      "29-Apr-2017 11:47:57 rep_motion_stereo_subject1_session1_motor submitted  (4 run | 0 fail | 59 done | 12 left)\n",
      "29-Apr-2017 11:47:58 rep_t1_subject2_overlay                   finished   (3 run | 0 fail | 60 done | 12 left)\n",
      "29-Apr-2017 11:47:58 rep_motion_native_subject1_session1_motor finished   (2 run | 0 fail | 61 done | 12 left)\n",
      "29-Apr-2017 11:47:58 rep_target_native_subject1_session1_motor finished   (1 run | 0 fail | 62 done | 12 left)\n",
      "29-Apr-2017 11:47:58 rep_motion_stereo_subject1_session1_motor finished   (0 run | 0 fail | 63 done | 12 left)\n",
      "29-Apr-2017 11:47:58 clean_subject1                            submitted  (1 run | 0 fail | 63 done | 11 left)\n",
      "29-Apr-2017 11:47:58 rep_target_stereo_subject1_session1_motor submitted  (2 run | 0 fail | 63 done | 10 left)\n",
      "29-Apr-2017 11:47:58 rep_motion_native_subject2_session1_motor submitted  (3 run | 0 fail | 63 done | 9 left)\n",
      "29-Apr-2017 11:47:58 rep_target_native_subject2_session1_motor submitted  (4 run | 0 fail | 63 done | 8 left)\n",
      "29-Apr-2017 11:47:59 clean_subject1                            finished   (3 run | 0 fail | 64 done | 8 left)\n",
      "29-Apr-2017 11:47:59 rep_target_stereo_subject1_session1_motor finished   (2 run | 0 fail | 65 done | 8 left)\n",
      "29-Apr-2017 11:47:59 rep_target_native_subject2_session1_motor finished   (1 run | 0 fail | 66 done | 8 left)\n",
      "29-Apr-2017 11:47:59 rep_motion_stereo_subject2_session1_motor submitted  (2 run | 0 fail | 66 done | 7 left)\n",
      "29-Apr-2017 11:47:59 rep_target_stereo_subject2_session1_motor submitted  (3 run | 0 fail | 66 done | 6 left)\n",
      "29-Apr-2017 11:47:59 rep_motion_ind_subject1_session1_motor    submitted  (4 run | 0 fail | 66 done | 5 left)\n"
     ]
    },
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "29-Apr-2017 11:47:59 rep_motion_native_subject2_session1_motor finished   (3 run | 0 fail | 67 done | 5 left)\n",
      "29-Apr-2017 11:47:59 rep_target_stereo_subject2_session1_motor finished   (2 run | 0 fail | 68 done | 5 left)\n",
      "29-Apr-2017 11:47:59 rep_motion_ind_subject1_session1_motor    finished   (1 run | 0 fail | 69 done | 5 left)\n",
      "29-Apr-2017 11:47:59 clean_subject2                            submitted  (2 run | 0 fail | 69 done | 4 left)\n",
      "29-Apr-2017 11:47:59 rep_motion_ind_subject2_session1_motor    submitted  (3 run | 0 fail | 69 done | 3 left)\n",
      "29-Apr-2017 11:47:59 rep_motion_report_subject1_session1_motor submitted  (4 run | 0 fail | 69 done | 2 left)\n",
      "29-Apr-2017 11:48:00 clean_subject2                            finished   (3 run | 0 fail | 70 done | 2 left)\n",
      "29-Apr-2017 11:48:00 rep_motion_stereo_subject2_session1_motor finished   (2 run | 0 fail | 71 done | 2 left)\n",
      "29-Apr-2017 11:48:00 rep_motion_ind_subject2_session1_motor    finished   (1 run | 0 fail | 72 done | 2 left)\n",
      "29-Apr-2017 11:48:00 rep_motion_report_subject1_session1_motor finished   (0 run | 0 fail | 73 done | 2 left)\n",
      "29-Apr-2017 11:48:00 rep_motion_report                         submitted  (1 run | 0 fail | 73 done | 1 left)\n",
      "29-Apr-2017 11:48:00 rep_motion_report_subject2_session1_motor submitted  (2 run | 0 fail | 73 done | 0 left)\n",
      "29-Apr-2017 11:48:01 rep_motion_report                         finished   (1 run | 0 fail | 74 done | 0 left)\n",
      "29-Apr-2017 11:48:01 rep_motion_report_subject2_session1_motor finished   (0 run | 0 fail | 75 done | 0 left)\n",
      "29-Apr-2017 11:48:01 Stopping idle worker 1 (not enough jobs left to do).\n",
      "29-Apr-2017 11:48:01 Stopping idle worker 2 (not enough jobs left to do).\n",
      "\n",
      "*******************************************\n",
      "Pipeline terminated on 29-Apr-2017 11:48:01\n",
      "All jobs have been successfully completed.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "niak_pipeline_fmri_preprocess(files_in,opt);\n",
    "% Check the content of fmri_preprocess/logs/PIPE_history.txt to monitor the progress of the pipeline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Once the pipeline has completed, an interactive report is built as part of the output. Just open the file [fmri_preprocess/report/index.html](./fmri_preprocess/report/index.html) in your browser, located inside the output folder. Note that the images of the test data have very low resolution. You can check an example report with a more typical resolution [here](https://simexp.github.io/qc_cobre/index.html).  "
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Octave",
   "language": "octave",
   "name": "octave"
  },
  "language_info": {
   "file_extension": ".m",
   "help_links": [
    {
     "text": "GNU Octave",
     "url": "https://www.gnu.org/software/octave/support.html"
    },
    {
     "text": "Octave Kernel",
     "url": "https://github.com/Calysto/octave_kernel"
    },
    {
     "text": "MetaKernel Magics",
     "url": "https://github.com/calysto/metakernel/blob/master/metakernel/magics/README.md"
    }
   ],
   "mimetype": "text/x-octave",
   "name": "octave",
   "version": "4.0.2"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 1
}