{
 "cells": [
  {
   "cell_type": "code",
   "execution_count": 1,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "Adding library niak to the search path.\n",
      "\n",
      "Adding library psom to the search path.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "cd\n",
    "build_path niak psom\n",
    "cd /home/pbellec/git/niak_tutorials/connectome"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Subtype pipeline\n",
    "This tutorial shows how to use the NIAK subtype pipeline to identify subgroups of subjects with homogeneous seed-based functional connectivity maps in the COBRE lightweight20 release, and check if some subtypes are associated with a diagnosis of schizophrenia. See the [documentation](http://niak.simexp-lab.org/pipe_connectome.html) of the pipeline for a more detailed description of the options. Download the tutorial as a notebook [here](https://raw.githubusercontent.com/SIMEXP/niak_tutorials/master/connectome/niak_tutorial_subtype_cobre.ipynb) and a matlab script [here](https://raw.githubusercontent.com/SIMEXP/niak_tutorials/master/connectome/niak_tutorial_subtype_cobre.m). To run this tutorial, we recommend to use [jupyter](http://jupyter.org/) from a niak docker container, as described in the [NIAK installation page](http://niak.simexp-lab.org/niak_installation.html). "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "To run that tutorial, you need to complete first the [correlation maps tutorial](http://niak.simexp-lab.org/niak_tutorial_rmap_connectome.html). You also need to run both tutorials in the same folder. "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Preparing the files"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Phenotypic data"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We first load the phenotypic data from the COBRE sample. This command will not work if you haven't downloaded the data yet as part of the rmap tutorial. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 2,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "path_cobre = [pwd filesep 'cobre_lightweight20'];\n",
    "file_pheno = [path_cobre filesep 'phenotypic_data.tsv.gz'];\n",
    "tab = niak_read_csv_cell(file_pheno);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we convert the values into a series of numerical covariates, that we save in a separate .csv file:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 3,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "list_subject = tab(2:end,1);\n",
    "patient = strcmp(tab(2:end,5),'Patient');\n",
    "age = str2double(tab(2:end,2)); \n",
    "FD = str2double(tab(2:end,9));\n",
    "opt_csv.labels_x = list_subject; % Labels for the rows\n",
    "opt_csv.labels_y = { 'age' , 'patient' , 'fd' };\n",
    "niak_write_csv('model_patient.csv', [age patient FD] , opt_csv);"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "We specify to the pipeline where to find the model:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 4,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "files_in.model = [pwd filesep 'model_patient.csv'];"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Connectivity maps"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we are going to get a list of the connectivity maps associated with each subject for one network, say the DMN. Labels for each network have been specified when running the `connectome` pipeline. We simply grab the outputs of the connectome pipeline. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 5,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "path_connectome = [pwd filesep 'connectome'];\n",
    "files_conn = niak_grab_connectome(path_connectome);\n",
    "files_in.data = files_conn.rmap;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "## Brain mask\n",
    "We specify the mask of brain networks to the pipeline, so that it can use it to mask the grey matter. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 6,
   "metadata": {
    "collapsed": false
   },
   "outputs": [],
   "source": [
    "files_in.mask = files_conn.network_rois;"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Set up the options of the pipeline"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "First specify where to save the outputs, and how many networks to use: "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 7,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%% General\n",
    "opt.folder_out = [pwd filesep 'subtype'];    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Then specify which covariates to use as confounds **before** the generation of subtypes. "
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 8,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "% a list of variable names to be regressed out\n",
    "% If unspecified or left empty, no confounds are regressed\n",
    "opt.stack.regress_conf = {'fd'};     "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "The options for the subtypes themselves:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 9,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "%% Subtyping\n",
    "opt.subtype.nb_subtype = 2;        % the number of subtypes to extract\n",
    "opt.subtype.sub_map_type = 'mean'; % the model for the subtype maps (options are 'mean' or 'median')"
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "Now we add an association test between subtypes and the patient label:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 10,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "% scalar number for the level of acceptable false-discovery rate (FDR) for the t-maps\n",
    "opt.association.patient.fdr = 0.05;                           \n",
    "% turn on/off normalization of covariates in model (true: apply / false: don't apply)\n",
    "opt.association.patient.normalize_x = false;                   \n",
    "% turn on/off normalization of all data (true: apply / false: don't apply)\n",
    "opt.association.patient.normalize_y = false;                  \n",
    "% turn on/off adding a constant covariate to the model\n",
    "opt.association.patient.flag_intercept = true;     \n",
    "% To test a main effect of a variable\n",
    "opt.association.patient.contrast.patient = 1; % scalar number for the weight of the variable in the contrast\n",
    "opt.association.patient.contrast.fd = 0;      % scalar number for the weight of the variable in the contrast\n",
    "opt.association.patient.contrast.age = 0;     % scalar number for the weight of the variable in the contrast\n",
    "% type of data for visulization (options are 'continuous' or 'categorical')\n",
    "opt.association.patient.type_visu = 'continuous'; "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "It is also possible to add a single chi-square test on the relationship between subtypes and a categorical variable:"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 11,
   "metadata": {
    "collapsed": true
   },
   "outputs": [],
   "source": [
    "% string name of the column in files_in.model on which the contigency table will be based\n",
    "opt.chi2 = 'patient';    "
   ]
  },
  {
   "cell_type": "markdown",
   "metadata": {},
   "source": [
    "# Run the pipeline"
   ]
  },
  {
   "cell_type": "code",
   "execution_count": 12,
   "metadata": {
    "collapsed": false
   },
   "outputs": [
    {
     "name": "stdout",
     "output_type": "stream",
     "text": [
      "\n",
      "Logs will be stored in /home/pbellec/git/niak_tutorials/connectome/subtype/logs/\n",
      "Generating dependencies ...\n",
      "   Percentage completed :  10 20 30 40 55 65 75 85 100- 0.01 sec\n",
      "Setting up the to-do list ...\n",
      "   I found 9 job(s) to do.\n",
      "I could not find any log file. This pipeline has not been started (yet?). Press CTRL-C to cancel.\n",
      "Deamon started on 29-Nov-2016 22:35:50\n",
      "29-Nov-2016 22:35:50 Starting the pipeline manager...\n",
      "29-Nov-2016 22:35:50 Starting the garbage collector...\n",
      "29-Nov-2016 22:35:50 Starting worker number 1...\n",
      "\n",
      "Pipeline started on 29-Nov-2016 22:35:51\n",
      "user: pbellec, host: acacia, system: unix\n",
      "*****************************************\n",
      "29-Nov-2016 22:35:52 stack_DMN                submitted  (1 run / 0 fail / 0 done / 8 left)\n",
      "29-Nov-2016 22:35:52 stack_MOTOR              submitted  (2 run / 0 fail / 0 done / 7 left)\n",
      "29-Nov-2016 22:35:53 stack_DMN                finished   (1 run / 0 fail / 1 done / 7 left)\n",
      "29-Nov-2016 22:35:53 subtype_DMN              submitted  (2 run / 0 fail / 1 done / 6 left)\n",
      "29-Nov-2016 22:35:54 stack_MOTOR              finished   (1 run / 0 fail / 2 done / 6 left)\n",
      "29-Nov-2016 22:35:54 subtype_MOTOR            submitted  (2 run / 0 fail / 2 done / 5 left)\n",
      "29-Nov-2016 22:35:57 subtype_DMN              finished   (1 run / 0 fail / 3 done / 5 left)\n",
      "29-Nov-2016 22:35:57 chi2_network_1           submitted  (2 run / 0 fail / 3 done / 4 left)\n",
      "29-Nov-2016 22:36:00 subtype_MOTOR            finished   (1 run / 0 fail / 4 done / 4 left)\n",
      "29-Nov-2016 22:36:00 weight_extraction        submitted  (2 run / 0 fail / 4 done / 3 left)\n",
      "29-Nov-2016 22:36:01 chi2_network_1           finished   (1 run / 0 fail / 5 done / 3 left)\n",
      "29-Nov-2016 22:36:01 chi2_network_2           submitted  (2 run / 0 fail / 5 done / 2 left)\n",
      "29-Nov-2016 22:36:02 weight_extraction        finished   (1 run / 0 fail / 6 done / 2 left)\n",
      "29-Nov-2016 22:36:02 association_test_patient submitted  (2 run / 0 fail / 6 done / 1 left)\n",
      "29-Nov-2016 22:36:03 chi2_network_2           finished   (1 run / 0 fail / 7 done / 1 left)\n",
      "29-Nov-2016 22:36:04 association_test_patient finished   (0 run / 0 fail / 8 done / 1 left)\n",
      "29-Nov-2016 22:36:04 visu_patient             submitted  (1 run / 0 fail / 8 done / 0 left)\n",
      "Deamon terminated on 29-Nov-2016 22:36:06\n",
      "\n",
      "29-Nov-2016 22:36:05 visu_patient             finished   (0 run / 0 fail / 9 done / 0 left)\n",
      "29-Nov-2016 22:36:06 Stopping idle worker 1 (not enough jobs left to do).\n",
      "\n",
      "*******************************************\n",
      "Pipeline terminated on 29-Nov-2016 22:36:06\n",
      "All jobs have been successfully completed.\n",
      "\n"
     ]
    }
   ],
   "source": [
    "opt.flag_test = false;  % Put this flag to true to just generate the pipeline without running it.\n",
    "pipeline = niak_pipeline_subtype(files_in,opt);"
   ]
  }
 ],
 "metadata": {
  "kernelspec": {
   "display_name": "Octave",
   "language": "octave",
   "name": "octave"
  },
  "language_info": {
   "file_extension": ".m",
   "help_links": [
    {
     "text": "MetaKernel Magics",
     "url": "https://github.com/calysto/metakernel/blob/master/metakernel/magics/README.md"
    }
   ],
   "mimetype": "text/x-octave",
   "name": "octave",
   "version": "0.16.0"
  }
 },
 "nbformat": 4,
 "nbformat_minor": 0
}