{ "cells": [ { "cell_type": "markdown", "metadata": { "toc": true }, "source": [ "

Table of Contents

\n", "
" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Introduction\n", "\n", "\n", "**Welcome to MoSeq2 Extract Modeling Notebook!**\n", "\n", "MoSeq2 is a software toolkit for unsupervised modeling and characterization of animal behavior. Moseq takes depth recordings of animals as input and outputs a rich description of behavior as a series of kinematic motifs called 'syllables'. __The model output can be analyzed using the [MoSeq2 Analysis Visualization Notebook](./MoSeq2-Analysis-Visualization-Notebook.ipynb).__\n", "\n", "In this notebook, the **Markdown** above each cell describes the purpose of the cell(s), cell output, and the instructions for running the cell(s) and/or interacting with the widget. The **inline code comments** in the code block provides contextual information about the function, code structure, and parameters.\n", "\n", "For more detailed documentation, please visit our [Wiki](https://github.com/dattalab/moseq2-app/wiki).\n", "For general feedback and feature requests, please fill out [this survey](https://forms.gle/FbtEN8E382y8jF3p6).\n", "\n", "***\n", "\n", "## Check MoSeq Versions\n", "- **Run the following cell** to check if `moseq2-app` is installed in your current environment.\n", "- Disregard any warning text with a red background in the output of this cell. They are generated from other packages. They do not mean there is a MoSeq2-related error and do not impact performance or usability for the MoSeq2 pipeline.\n", "\n", "This cell should print the version numbers of each MoSeq2 package installed on your system, as well as the path to the python executable that MoSeq2 is run on." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": false }, "outputs": [], "source": [ "!moseq2-extract --version\n", "!moseq2-pca --version\n", "!moseq2-model --version\n", "!moseq2-viz --version\n", "\n", "import sys\n", "import moseq2_app\n", "\n", "print('MoSeq2 app version:', moseq2_app.__version__)\n", "print('Python path:', sys.executable)" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "# Project setup\n", "\n", "## Files and Directory Structure\n", "\n", "The currently accepted depth data extensions are:\n", "- `.dat` (raw depth files from our kinect2 data acquisition software)\n", "- `.tar.gz` (compressed depth files from our kinect2 data acquisition software)\n", "- `.avi` (compressed depth files from the `moseq2-extract` CLI)\n", "\n", "\n", "**We recommend recording more than 10 hours of depth video (~1 million frames at 30 frames per second) to ensure quality MoSeq models.**\n", "\n", "Each MoSeq project should be organized within one project directory.\n", "To better organize the extraction, modeling, and analysis results, you should copy the MoSeq notebooks to the data directory that contains the depth recording to be analyzed (i.e. in `` in the directory structure below).\n", "\n", "At this stage, the data directory should contain **separate subfolders** for each depth recording session, as shown below:\n", "\n", "```\n", ". ** current working directory\n", "└── / ** the directory with all depth recordings\n", " ├── session_1/ ** the folder containing all of a single session's data\n", " ├ ├── depth.dat ** the depth data recording itself\n", " ├ ├── depth_ts.txt ** csv/txt file of the frame timestamps\n", " ├ └── metadata.json ** json file that contains metadata of the rodent (group, subjectName, etc.)\n", " ...\n", " ├── session_n/\n", " ├ ├── depth.dat\n", " ├ ├── depth_ts.txt\n", " └── └── metadata.json\n", "```\n", "You can find more information about the file structure [here](https://github.com/dattalab/moseq2-app/wiki/Directory-Structures-and-yaml-Files-in-MoSeq-Pipeline).\n", "\n", "## Set up or Restore Progress Variables\n", "\n", "**[IMPORTANT] ALWAYS run this cell when you open this notebook.** If you don't, the pipeline won't work properly.\n", "\n", "MoSeq uses a [progress.yaml file](https://github.com/dattalab/moseq2-app/wiki/Directory-Structures-and-yaml-Files-in-MoSeq-Pipeline#progressyaml) to keep track of your progression through the pipeline. \n", "\n", "The following cell generates a `progress.yaml` file if there isn't one present.\n", "If there is, it loads your progress from the last saved checkpoint (each step in the pipeline saves a checkpoint).\n", "It also displays your progress through the pipeline and prints session names that haven't been extracted.\n", "\n", "**Instructions:**\n", "- **Specify the data directory, the folder with all the depth recordings,** in the `base_dir` field. No change is needed if the notebooks are in the base directory.\n", "- **Run the following cell**." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from os.path import join\n", "from moseq2_app.gui.progress import check_progress, restore_progress_vars\n", "\n", "base_dir = './' # Add the path to your data folder here.\n", "progress_filepath = join(base_dir, 'progress.yaml')\n", "\n", "progress_paths = restore_progress_vars(progress_filepath, init=True, overwrite=False)\n", "check_progress(progress_filepath)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Generate Configuration Files\n", "\n", "MoSeq uses a [config.yaml file](https://github.com/dattalab/moseq2-app/wiki/Directory-Structures-and-yaml-Files-in-MoSeq-Pipeline#configyaml) to hold all the configurable parameters for each step in the MoSeq pipeline.\n", "Parameters are added to this file as you progress through the notebook.\n", "You can copy the config file to another project to run MoSeq with the same parameters as this project.\n", "\n", "The following cell generates the `config.yaml` file for the analysis pipeline.\n", "\n", "**Instructions:**\n", "- **Run the following cell**." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from os.path import join\n", "from moseq2_app.gui.progress import update_progress\n", "from moseq2_extract.gui import generate_config_command\n", "\n", "config_filepath = join(progress_paths['base_dir'], 'config.yaml')\n", "# specify the specify the camera type\n", "camera_type = 'k2' # 'k2' or 'azure' - set to \"azure\" if using the Orbbec camera\n", "\n", "print(f'generating file in path: {config_filepath}')\n", "generate_config_command(config_filepath, camera_type=camera_type)\n", "progress_paths = update_progress(progress_filepath, 'config_file', config_filepath)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Download a Pre-trained Flip Classifier Model File\n", "\n", "MoSeq2 uses a Random Forest flip classifier to guarantee that the mouse is always pointed to the right after cropping and rotationally aligning mice in the depth videos. The flip classifiers we provide __are trained for experiments run with C57BL/6 mice using Kinect v2 depth cameras__.\n", "\n", "If your dataset does not work with our pre-trained flip classifiers, we provide a [flip-classifier training notebook](./Flip-Classifier-Training-Notebook.ipynb) to train your own classifier.\n", "Instead of running the cell below, add the path of your custom classifier to the `flip_classifier` field in your `config.yaml` file.\n", "\n", "The following cell downloads a pretrained flip classifier.\n", "\n", "**Instructions:**\n", "- **Select a classifier** that is trained on a dataset that best matches your experiment set up by changing the `selection` parameter in the `download_flip_command` function.\n", "- **Run the following cell**." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from moseq2_extract.gui import download_flip_command\n", "# selection=0 - large mice with fibers (K2)\n", "# selection=1 (default) - adult male C57s (K2)\n", "# selection=2 - mice with Inscopix cables (K2)\n", "# selection=3 - adult male C57s (Azure) - use selection 3 if using the Orbbec camera\n", "selection = 1\n", "download_flip_command(progress_paths['base_dir'], config_filepath, selection=selection)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Raw Data Extraction" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Interactive Arena Detection Tool\n", "\n", "Many parameters are related to detecting the arena floor, which is the region the extraction step should look to find the mouse. Use this interactive tool to optimize parameters before extracting all of your data. This tool can also be used to catch possibly corrupted or inconsistent sessions, and to diagnose arena detection or extraction errors. You can find more detailed information on how to use this widget in the [wiki](https://github.com/dattalab/moseq2-app/wiki/MoSeq2-Extract-Modeling-Notebook-Instructions#interactive-arena-detection-tool).\n", "\n", "Run the following cell to start the interactive widget. After running this tool, a [session-config.yaml file](https://github.com/dattalab/moseq2-app/wiki/Directory-Structures-and-yaml-Files-in-MoSeq-Pipeline#session-configyaml) will be generated, containing session-specific parameters.\n", "\n", "If the previous steps are run using the Command Line Interface, please run the [Setup or Restore Progress Variables cell](#Set-up-or-Restore-Progress-Variables) to setup/update the `progress.yaml` file to record the progress of the analysis pipeline.\n", "\n", "**Instructions:**\n", "- **Run the following cell** to initialize the Arena detection widget. The cell renders a control panel to configure parameters for detecting the arena.\n", "- By default, the widget selects the first session in your dataset, sorted alphanumerically.\n", "- Adjust the depth range for detecting the floor of the arena.\n", "- Adjust the dilation iterations to include more of the wall of the arena.\n", "- Click the `Compute arena mask` button to compute and display the mask for the detected floor given the parameters. The displayed mask won't recompute and refresh when you change the parameters unless you click the button.\n", "- Check the \"Show advanced arena mask parameters\" checkbox to display more advanced arena mask parameters and you can find more information about the parameters by running the CLI `moseq2-extract extract --help`. You can find documentation for CLI [here](https://github.com/dattalab/moseq2-app/wiki/Command-Line-Interface-for-Extraction-and-Modeling#extract-data).\n", "- If you like the arena mask, click the `Compute extraction` button to extract a subset of the data.\n", "- Once you are satisfied with the extraction, click the `Save parameters...` button to move on to the next session and save this session's parameters." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": false }, "outputs": [], "source": [ "from os.path import join\n", "from moseq2_app.util import update_config\n", "from moseq2_app.gui.progress import update_progress\n", "from moseq2_app.roi.widget import ArenaMaskWidget\n", "import panel as pn\n", "pn.extension()\n", "\n", "session_config_path = join(progress_paths['base_dir'], 'session_config.yaml')\n", "progress_paths = update_progress(progress_filepath, 'session_config', session_config_path)\n", "\n", "with update_config(progress_paths['config_file']) as config_data:\n", "\n", " config_data['camera_type'] = 'auto' # 'kinect' or 'manual'\n", " config_data['output_dir'] = 'proc' # the subfolder extracted data is saved to\n", " # if using an azure or orbbec camera, uncomment the following line\n", " # config_data['crop_size'] = [120, 120]\n", "\n", " # OPTIONAL additional parameters\n", " # config_data['flip_classifier'] = './alternative-flip-classifier.pkl' # updated flip classifier path\n", "\n", "# Set to False to show and re-extract extracted recordings\n", "skip_extracted = True\n", "\n", "# Set to False if running this widget a second time\n", "# to preserve the configs for previous recordings\n", "overwrite_session_configs = True\n", "\n", "ArenaMaskWidget(progress_paths['base_dir'], progress_paths['config_file'], progress_paths['session_config'], \n", " skip_extracted=skip_extracted, overwrite_session_configs=overwrite_session_configs)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Extract Session(s)\n", "The following cell uses the parameters from the [config.yaml](https://github.com/dattalab/moseq2-app/wiki/Directory-Structures-and-yaml-Files-in-MoSeq-Pipeline#configyaml) file or [session-config.yaml](https://github.com/dattalab/moseq2-app/wiki/Directory-Structures-and-yaml-Files-in-MoSeq-Pipeline#session-configyaml) to extract the mouse and scalar values related to mouse movement (velocity, height, position, etc) from the depth videos. You can find the file structure after extracting session(s) [here](https://github.com/dattalab/moseq2-app/wiki/Directory-Structures-and-yaml-Files-in-MoSeq-Pipeline#after-extracting-the-data).\n", "\n", "This cell takes quite a while to run. Each ~30-minute depth recording can take around 10-15 minutes to extract assuming data is saved in `.dat` format. Each session is extracted serially.\n", "\n", "If you are on a computing cluster managed with Slurm, [check out our wiki](https://github.com/dattalab/moseq2-app/wiki/MoSeq2-Extract-Modeling-Notebook-Instructions#extracting-sessions-in-parallel-with-slurm) to learn about how to run extractions in parallel, rather than serially.\n", "\n", "**Note:** If sessions are not listed when running the cell, ensure the contents of the `extensions` variable match extension of your depth files.\n", "\n", "**Instructions:**\n", "- **Run the following cell**." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "from os.path import join\n", "from moseq2_app.util import update_config\n", "from moseq2_extract.gui import extract_found_sessions\n", "\n", "with update_config(progress_paths['config_file']) as config_data:\n", " \n", " config_data['cluster_type'] = 'local' # currently supported cluster_types = 'local' or 'slurm'\n", "\n", " ## SLURM PARAMETERS\n", " # config_data['prefix'] = 'source ~/.bashrc; conda activate moseq2-app; ' # prefix that activates the conda environment\n", " # config_data['memory'] = '6GB'\n", " # config_data['wall_time'] = '1:00:00' # allocated time to run an extraction\n", " # config_data['partition'] = 'short' # slurm partition\n", " # config_data['run_cmd'] = True # if False, extract_out_script must be run manually\n", " # config_data['extract_out_script'] = 'extract_out.sh'\n", "\n", "# include the file extensions for the depth files you would like to search for and extract.\n", "extensions = ['.avi', '.dat', '.tar.gz'] # .avi, .dat, and/or `.tar.gz`\n", "\n", "# Set to False to select specific recordings to extract\n", "extract_all = True\n", "\n", "# skip_extracted is set in the arena mask widget cell above\n", "# uncomment the line below if you are not running then interactive arena tool above\n", "# otherwise, the function below will produce an error\n", "# skip_extracted = True\n", "\n", "extract_found_sessions(progress_paths['base_dir'], \n", " progress_paths['config_file'], \n", " extensions, \n", " extract_all=extract_all, \n", " skip_extracted=skip_extracted)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## [OPTIONAL] Run Extraction Validation Tests\n", "\n", "The following few cells allow you to review the extraction quality, as it can affect the model results.\n", "The cell below indicates potential outlier sessions relative to the rest of your dataset by looking at the distribution of extracted features averaged over the session. \n", "To visually inspect the extraction, use the [Review Extraction Output](#[OPTIONAL]-Review-Extraction-Output) below.\n", "To visualize potential anomalies in the extracted features, use the [Further Extraction Diagnostics](#[OPTIONAL]-Further-Extraction-Diagnostics) below.\n", "\n", "The following cell runs data validation tests on the extracted data, displaying warnings if there are any potential outliers.\n", "You can find more information about what the warnings mean in the [wiki](https://github.com/dattalab/moseq2-app/wiki/MoSeq2-Extract-Modeling-Notebook-Instructions#run-extraction-validation-tests).\n", "\n", "**Instructions:**\n", "- **Run the following cell**." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from moseq2_app.main import validate_extractions\n", "\n", "validate_extractions(progress_paths['base_dir'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## [OPTIONAL] Review Extraction Output\n", "Use the following cell to visually inspect the extractions.\n", "You can find examples of a good and a bad extraction [here](https://github.com/dattalab/moseq2-app/wiki/MoSeq2-Extract-Modeling-Notebook-Instructions#examples-of-a-good-extraction-and-a-bad-extraction).\n", "\n", "**Note:** this cell takes a while (5-10 seconds) to load each video into the notebook, and although it might seem like nothing is happening, the widget is preparing the video\n", "Be patient as the data loads. This widget only works in Chrome.\n", "\n", "**Instructions:**\n", "- **Run the following cell**.\n", "- **Select a session** from the `Session` dropdown menu to preview its extraction output.\n", "- **Change the Playback Speed** slider to speed up or slow down the video." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from moseq2_app.main import preview_extractions\n", "\n", "preview_extractions(progress_paths['base_dir'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Aggregate the Extraction Results\n", "The extraction results are aggregated into a folder called `aggregate_results` to organize the extracted data in a new folder for the rest of the pipeline.\n", "\n", "The following cell aggregates extraction results into one folder and generates a [moseq2-index.yaml](https://github.com/dattalab/moseq2-app/wiki/Directory-Structures-and-yaml-Files-in-MoSeq-Pipeline#moseq2-indexyaml) file to store all the session-specific information.\n", "This step requires that all your sessions have a `metadata.json` file containing a session name for them to be properly aggregated. **Running this cell will generate a new `moseq2-index.yaml` file and overwrite the original one. You will need to reset the group labels and add back the PCA score path at the bottom of the new `moseq2-index.yaml`.**\n", "\n", "**Instructions:**\n", "- **run the following cell**." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "from os.path import join\n", "from moseq2_app.gui.progress import update_progress\n", "from moseq2_extract.gui import aggregate_extract_results_command\n", "\n", "recording_format = '{start_time}_{session_name}_{subject_name}' # filename formats for the copied extracted data files\n", "\n", "# directory name to save all metadata + extracted videos to with the format above\n", "aggregate_results_dirname = 'aggregate_results/'\n", "\n", "train_data_dir = join(progress_paths['base_dir'], aggregate_results_dirname)\n", "update_progress(progress_filepath, 'train_data_dir', train_data_dir)\n", "\n", "index_filepath = aggregate_extract_results_command(progress_paths['base_dir'], recording_format, aggregate_results_dirname)\n", "progress_paths = update_progress(progress_filepath, 'index_file', index_filepath)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Assign Groups\n", "Sessions can be given group labels for analyses comparing different cohorts or experimental conditions and the labels will be stored in the [moseq2-index.yaml file](https://github.com/dattalab/moseq2-app/wiki/Directory-Structures-and-yaml-Files-in-MoSeq-Pipeline#moseq2-indexyaml). You can find more information about this widget in the [wiki](https://github.com/dattalab/moseq2-app/wiki/MoSeq2-Extract-Modeling-Notebook-Instructions#assign-groups).\n", "\n", "If any of the previous steps are run using the Command Line Interface, please run the [Setup or Restore Progress Variables cell](#Set-up-or-Restore-Progress-Variables) to set up/update the `progress.yaml` file with results run from the CLI.\n", "\n", "The following cell runs the widget to assign groups and update the `moseq2-index.yaml` file.\n", "\n", "**Instructions:**\n", "- **Run the following cell**.\n", "- **Click the column header** to sort the column and use the filter icon to filter if needed.\n", "- **Click on the session** to select the session. **To select multiple sessions, click the sessions while holding down the [Ctrl]/[Command] key, or click the first and last entry while holding down the [Shift] key.**\n", "- **Enter the group name in the `Desired Group Name` field** and click `Set Group` to update the `group` column for the selected sessions.\n", "- Click the `Update Index File` button to save current group assignments." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "from moseq2_app.main import interactive_group_setting\n", "\n", "interactive_group_setting(progress_paths['index_file'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## [OPTIONAL] Further Extraction Diagnostics\n", "Using the following tool to visualize the distribution of scalar values can help with discovering possible outlier sessions.\n", "\n", "If outlier sessions do exist, review the extraction video using the Review Extractions tool and check for any irregularities.\n", "You may need to re-extract or discard sessions depending on the issues you catch.\n", "You can find more information in the wiki [here](https://github.com/dattalab/moseq2-app/wiki/MoSeq2-Extract-Modeling-Notebook-Instructions#scalar-summary-for-further-extraction-diagnostics).\n", "\n", "The following cell starts the interactive widget that visualizes the distribution of average scalar values for each session.\n", "\n", "**Instructions:**\n", "- **Run the following cell**.\n", "- **Select a scalar to plot from the window that appears.**\n", "- **Hold [CTRL]/[Command] and click on the scalars** to select multiple scalars to plot.\n", "- **Hover over the data points** to display the session information.\n", "- **Click on the legend items to show/hide groups** from the plot. Double click an item to show a single group.\n", "- **Click the camera icon in the toolbar on top of the figures** to save the figures as SVG in your default web image download folder. The toolbar will display when you hover your mouse over the figures." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from moseq2_app.main import interactive_scalar_summary\n", "\n", "viewer = interactive_scalar_summary(progress_paths['index_file'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Look Up Sessions Using Their UUID\n", "Use the widget below to look up additional information about individual sessions, such as where the sessions are located, by using their UUID. \n", "You can find more information about this tool in the wiki [here](https://github.com/dattalab/moseq2-app/wiki/MoSeq2-Extract-Modeling-Notebook-Instructions#uuid-lookup).\n", "\n", "The following cell outputs additional information about the session with the target UUID.\n", "\n", "**Instructions:**\n", "- **Hover over the data point of interest** in the widget above to display Session Name, Subject Name, and UUID.\n", "- To look up the file paths associated with the UUID, **input the UUID in the `target_uuid` variable** below, partial UUID is supported." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from moseq2_app.util import uuid_lookup\n", "\n", "# Input full/partial UUID to lookup, eg. 40 is a partial UUID\n", "target_uuid = '40'\n", "\n", "# viewer.sorted_index['files'] is the uuid info dictionary\n", "uuid_lookup(target_uuid, viewer.sorted_index['files'])" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# Principal Component Analysis (PCA)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Fitting PCA\n", "This step learns a low-dimensional representation of mouse pose that is much more manageable to learn syllables from. \n", "\n", "You can find more information in the wiki [here](https://github.com/dattalab/moseq2-app/wiki/MoSeq2-Extract-Modeling-Notebook-Instructions#pca).\n", "\n", "**Note:** PCA is a resource-intensive step and in some cases, the kernel in Jupyter may die before the PCA step finishes.\n", "If you continue to experience a dead kernel after adjusting your resources as described in the [wiki](https://github.com/dattalab/moseq2-app/wiki/MoSeq2-Extract-Modeling-Notebook-Instructions#pca), please run the Fitting PCA step and Computing Principal Component Scores step in the CLI.\n", "Find the CLI instructions in the wiki [here](https://github.com/dattalab/moseq2-app/wiki/Command-Line-Interface-for-Extraction-and-Modeling#pca).\n", "\n", "**Instructions:**\n", "- **Run the following cell** to fit PCA." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from os.path import join\n", "from moseq2_app.util import update_config\n", "from moseq2_pca.gui import train_pca_command\n", "from moseq2_app.gui.progress import update_progress\n", "\n", "pca_filename = 'pca' # Name of your PCA model h5 file to be saved\n", "pca_dirname = join(progress_paths['base_dir'], '_pca/') # Directory to save your computed PCA results\n", "progress_paths = update_progress(progress_filepath, 'pca_dirname', pca_dirname)\n", "\n", "with update_config(progress_paths['config_file']) as config_data:\n", "\n", " # PCA parameters you may need to configure\n", " config_data['camera_type'] = 'k2' # specify camera type, could be 'k2' or 'azure'\n", " config_data['overwrite_pca'] = False\n", "\n", " config_data['gaussfilter_space'] = (1.5, 1) # gaussian spatial filter on mouse depth images to remove noise\n", " config_data['medfilter_space'] = [0] # Median spatial filter\n", " config_data['medfilter_time'] = [0] # Median temporal filter\n", "\n", " # Change default parameters for data recorded with Azure Kinect\n", " if config_data['camera_type'] == 'azure':\n", " config_data['gaussfilter_space'] = (2.25, 1.5)\n", " config_data['tailfilter_size'] = (13, 13)\n", "\n", " # Dask Configuration\n", " config_data['nworkers'] = 1 # the number of workers for the job\n", " # config_data['dask_port'] = '8787' # port to access Dask Dashboard - don't change if using Docker\n", "\n", " # UNCOMMENT to use SLURM\n", " # config_data['cluster_type'] = 'slurm'\n", " # config_data['nworkers'] = 8 # number of spawned jobs\n", " # config_data['queue'] = 'short' # partition\n", " # config_data['memory'] = '40GB' # amount of memory per worker\n", " # config_data['cores'] = 1 # number of cores per worker\n", " # config_data['wall_time'] = '01:00:00' # worker time limit\n", "\n", " # UNCOMMENT if recordings contain occlusions (e.g. from overhead cables)\n", " # config_data['missing_data'] = True\n", " # config_data['recon_pcs'] = 10 # Number of PCs to use for missing data reconstruction\n", " # config_data['missing_data_iters'] = 10 # Number of iterations used to reconstruct data\n", "\n", "# will train on data in aggregate_results/\n", "train_pca_command(progress_paths, pca_dirname, pca_filename)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Visualize PCA Results\n", "\n", "Work from the Datta Lab has found that the top 10 PCs generally capture about 90% of the variance across mouse poses.\n", "By default, the AR-HMM is trained on the top 10 PCs, and most of the AR-HMM hyperparameters are set with this in mind.\n", "\n", "The PCA visualization can help you assess the quality of the PCs, and make sure you're capturing enough variance with 10 PCs.\n", "You can find more information about PCA visualization interpretation in the wiki [here](https://github.com/dattalab/moseq2-app/wiki/Analysis-Troubleshooting#possible-pca-problems). \n", "\n", "The figures are automatically saved as png and pdf in the same folder as the PCA results.\n", "\n", "**Instructions:**\n", "- **Run the following cell** to visualize PCA results. " ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from os.path import join\n", "from IPython.display import display, Image\n", "images = [join(progress_paths['pca_dirname'], 'pca_components.png'), \n", " join(progress_paths['pca_dirname'], 'pca_scree.png')]\n", "for im in images:\n", " display(Image(im))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Computing Principal Component Scores\n", "This step applies the learned PCs from the previous step to the extracted depth recordings. \n", "It produces a 10-dimensional time series of PC scores for each session in your project.\n", "\n", "**Instructions:**\n", "- **Run the following cell** to compute principal component scores." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": true }, "outputs": [], "source": [ "from os.path import join\n", "from moseq2_pca.gui import apply_pca_command\n", "from moseq2_app.gui.progress import update_progress\n", "\n", "scores_filename = 'pca_scores' # name of the scores file to compute and save\n", "\n", "scores_file = join(progress_paths['pca_dirname'], scores_filename + '.h5') # path to input PC scores file to model\n", "progress_paths = update_progress(progress_filepath, 'scores_path', scores_file)\n", "\n", "apply_pca_command(progress_paths, scores_filename)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Compute Model-free Changepoints\n", "Model-free changepoints are used to determine a target syllable duration for the modeling step.\n", "This step is optional if you don't plan to run [Get Best Model Fit](#Get-Best-Model-Fit), but highly recommended.\n", "You can find more information about model-free changepoints [here](https://github.com/dattalab/moseq2-app/wiki/MoSeq2-Extract-Modeling-Notebook-Instructions#computing-model-free-changepoints).\n", "\n", "**Note:** the parameters below are configured for C57 mouse data and have not been tested for other strains/species, and we strongly recommend not changing them. They are sensitive, and can drastically change interpretation if changed.\n", "\n", "**Instructions:**\n", "- **Run the following cell** to compute model-free syllable changepoints. The figure is automatically saved as png and pdf in the same folder as the PCA results." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from moseq2_app.util import update_config\n", "from moseq2_app.gui.progress import update_progress\n", "from moseq2_pca.gui import compute_changepoints_command\n", "\n", "changepoints_filename = 'changepoints' # name of the changepoints images to generate\n", "\n", "with update_config(progress_paths['config_file']) as config_data:\n", " # WARNING: don't change these parameters if you're using C57 mice\n", " # config_data['threshold'] = 0.5 # Peak threshold to use for changepoints\n", " # config_data['neighbors'] = 1 # Number of neighbors to use for peak identification\n", " # config_data['klags'] = 6 # Number of lags to use for derivative calculation\n", " # config_data['sigma'] = 3.5 # Standard deviation used for gaussian smoothing\n", " config_data['dims'] = 300 # Number of random projections to compare the computed principal components with\n", "\n", "progress_paths = update_progress(progress_filepath, 'changepoints_path', changepoints_filename)\n", "compute_changepoints_command(progress_paths['train_data_dir'], progress_paths, changepoints_filename)\n", "\n", "# plot model-free change point results\n", "from os.path import join\n", "from IPython.display import display, Image\n", "\n", "changepoints_filename = 'changepoints'\n", "display(Image(join(progress_paths['pca_dirname'], changepoints_filename + '_dist.png')))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "# AR-HMM Modeling" ] }, { "attachments": {}, "cell_type": "markdown", "metadata": {}, "source": [ "## Fitting the AR-HMM\n", "Fitting the AR-HMM typically requires adjusting the `kappa` hyperparameter to achieve a target syllable duration (higher values of `kappa` lead to longer syllable durations). The target mean duration can be determined using changepoints analysis or set heuristically to 0.5-0.6 seconds based on prior literature.\n", "\n", "In the code below, set `kappa` to `'scan'` to generate a bash script to run a series of models with different kappa values. You can specify minimum kappa value using `--min-kappa`(eg. 10,000), and maximum kappa value using `max-kappa` (eg. 10,000,000). The bash script will be generated in the specified model directory. You can read more about kappa scan in the wiki [here](https://github.com/dattalab/moseq2-app/wiki/Analysis-Tips#recommended-practice-for-modeling).\n", "\n", "\n", "You can use the \"Get Best Model Fit\" cell to pick an optimal value. We recommend fitting for 100-200 iterations to pick `kappa`. For final model fitting, set `kappa` to the chosen value and fit for ~1000 iterations.\n", "\n", "You can find more information about AR-HMM modeling parameters in the wiki [here](https://github.com/dattalab/moseq2-app/wiki/MoSeq2-Extract-Modeling-Notebook-Instructions#most-relevant-model-paramters) and learn more about different AR-HMM models we support in the [CLI wiki](https://github.com/dattalab/moseq2-app/wiki/Command-Line-Interface-for-Extraction-and-Modeling#ar-hmm-modeling).\n", "\n", "**Note:** if loading a model checkpoint, ensure the modeling parameters (especially the selected groups) are identical to that of the checkpoint. Otherwise, the model will fail." ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Set Model Parameters\n", "\n", "Read the inline code comments for more information about the model parameters." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from os.path import join\n", "from moseq2_app.util import update_config\n", "\n", "# set and save model paths\n", "base_model_path = 'models/' # specify the folder that keeps all the trained models\n", "model_name = 'model.p' # Specify the model name\n", "\n", "session_path = join(progress_paths['base_dir'], base_model_path)\n", "model_path = join(session_path, model_name) # path to save the model\n", "\n", "progress_paths = update_progress(progress_filepath, 'model_path', model_path) # record the path to the model\n", "progress_paths = update_progress(progress_filepath, 'base_model_path', session_path) # record the path to folder that keeps trained models\n", "\n", "# update configuration file\n", "with update_config(progress_paths['config_file']) as config_data:\n", " # model saving frequency (in iterations); will create a checkpoints/ directory containing checkpointed models\n", " config_data['checkpoint_freq'] = -1\n", " config_data['use_checkpoint'] = False # resume training from latest saved checkpoint. False = start anew\n", "\n", " config_data['npcs'] = 10 # number of PCs being used; npcs should explained ~90% of the variance in mouse pose\n", " config_data['max_states'] = 100 # number of maximum states (i.e., syllables) the AR-HMM will try to learn \n", "\n", " # use robust AR-HMM with Student's t-distribution -> able to tolerate more noise\n", " config_data['robust'] = True \n", "\n", " # save model object at the end of training\n", " # the saved model object can be used to applied to new data\n", " config_data['save_model'] = True\n", "\n", " # use separate transition matrix for each experimental group assigned above\n", " config_data['separate_trans'] = False\n", "\n", " # number of iterations to train model\n", " # Use 100-200 for finding the optimal kappa value. Use 1000 for fitting your final models\n", " config_data['num_iter'] = 100 \n", "\n", " # select specific groups to model; if False, will model all data as is in moseq2-index.yaml\n", " config_data['select_groups'] = False\n", "\n", " # syllable length probability distribution prior; (None, int or 'scan');\n", " # if None, kappa is set to the number of frames of your dataset\n", " config_data['kappa'] = None\n", " if config_data['kappa'] == 'scan':\n", " config_data['scan_scale'] = 'log' # log or linear\n", " config_data['min_kappa'] = None # optionally set lower bound for kappa scan\n", " config_data['max_kappa'] = None # optionally set upper bound for kappa\n", " config_data['n_models'] = 15 # total number of models to spool\n", " config_data['out_script'] = 'train_out.sh' # script name to save kappa-scanning commands\n", " \n", " # Runs the commands via os.system(...) and progress is displayed in terminal when set to True. Otherwise, script must be run manually\n", " config_data['run_cmd'] = True\n", "\n", "\n", " ### Advanced Modeling Parameters ###\n", "\n", " config_data['hold_out'] = False # boolean to hold out data subset during the training process\n", " config_data['nfolds'] = 2 # if hold_out is True, sets the number of folds to hold out during training\n", "\n", " # hyperparameters for hierarchical dirichlet process - leave these alone\n", " # config_data['alpha'] = 5.7\n", " # config_data['gamma'] = 1e3\n", "\n", " # hyperparameter for the autogregressive process - generally leave this alone too\n", " # config_data['nlags'] = 3\n", "\n", " # Select platform to run models on\n", " config_data['cluster_type'] = 'local' # currently supported cluster_types = 'local' or 'slurm'\n", " config_data['ncpus'] = 0 # 0 means all the CPU cores will be used\n", "\n", " ## SLURM PARAMETERS\n", " # if config_data['kappa'] == 'scan' and config_data['culuster_type'] = 'slurm'\n", " ## only edit these parameters if cluster_type == 'slurm'\n", " # config_data['prefix'] = 'source ~/.bashrc; conda activate moseq2-app; ' # prefix that activates the conda environment\n", " # config_data['memory'] = '16GB'\n", " # config_data['wall_time'] = '3:00:00'\n", " # config_data['partition'] = 'short'" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "### Train Model\n", "\n", "Training the model can take a considerable amount of time.\n", "Hold tight until this process finishes." ] }, { "cell_type": "code", "execution_count": null, "metadata": { "scrolled": false }, "outputs": [], "source": [ "from moseq2_model.gui import learn_model_command\n", "\n", "learn_model_command(progress_paths)" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "## Get Best Model Fit\n", "This feature compares the trained model(s) with the principal components' changepoints and it supports two objectives for the comparison. `duration` finds the model where **the syllable duration** best matches that of the principal components' changepoints. The best match could be done on `mean`, `median` or `mode` and the default is matching the mean syllable duration with that of the changepoints. `jsd` finds the model where **the distribution of syllable durations** best match that of the principal components' changepoints. You can find more information in the [wiki](https://github.com/dattalab/moseq2-app/wiki/MoSeq2-Extract-Modeling-Notebook-Instructions#finding-best-model-fit).\n", "\n", "A good rule of thumb is that models trained with `kappa` values within an order of magnitude of the number of frames in your dataset most closely match the model-free changepoint distribution.\n", "\n", "Once completed, if there are multiple models, this feature returns the best model from a list of models found in the `progress_paths['base_model_path']` and produces a `.csv` file with the summary statistics for each model saved in the same folder. The figure will be automatically saved as png and pdf in the base directory.\n", "\n", "**Instructions:**\n", "- **Run the following cell** to get the best model fit." ] }, { "cell_type": "code", "execution_count": null, "metadata": {}, "outputs": [], "source": [ "from os.path import join\n", "from IPython.display import display, Image\n", "from moseq2_viz.gui import get_best_fit_model\n", "from moseq2_app.gui.progress import update_progress\n", "\n", "output_file = join(progress_paths['base_dir'], 'model_vs_pc_changepoints')\n", "# objective could be either duration, jsd or median_loglikelihood\n", "objective = 'duration (mean match)'\n", "# change the objective to median_loglikelihood when finding the final model\n", "# objective = 'median_loglikelihood'\n", "\n", "best_model_fit = get_best_fit_model(progress_paths, output_file, plot_all=True, objective=objective)\n", "\n", "progress_paths = update_progress(progress_filepath, 'model_path', best_model_fit[f'best model - {objective}'])\n", "# uncomment the line below if no image is displayed\n", "# display(Image(output_file + '.png'))" ] }, { "cell_type": "markdown", "metadata": {}, "source": [ "***\n", "\n", "# Notebook End \n", "\n", "# User Survey\n", "\n", "Please take some time to tell us your thoughts about this notebook:\n", "**[user feedback survey](https://forms.gle/S88jptAEs41mQjff7)**" ] } ], "metadata": { "finalized": { "timestamp": 1623072349278, "trusted": true }, "kernelspec": { "display_name": "Python 3", "language": "python", "name": "python3" }, "language_info": { "codemirror_mode": { "name": "ipython", "version": 3 }, "file_extension": ".py", "mimetype": "text/x-python", "name": "python", "nbconvert_exporter": "python", "pygments_lexer": "ipython3", "version": "3.7.12" }, "moseq2": { "version": "1.4" }, "pycharm": { "stem_cell": { "cell_type": "raw", "metadata": { "collapsed": false }, "source": [] } }, "toc": { "base_numbering": 1, "nav_menu": {}, "number_sections": true, "sideBar": true, "skip_h1_title": false, "title_cell": "Table of Contents", "title_sidebar": "Contents", "toc_cell": true, "toc_position": { "height": "calc(100% - 180px)", "left": "10px", "top": "150px", "width": "303.965px" }, "toc_section_display": true, "toc_window_display": true }, "varInspector": { "cols": { "lenName": 16, "lenType": 16, "lenVar": 40 }, "kernels_config": { "python": { "delete_cmd_postfix": "", "delete_cmd_prefix": "del ", "library": "var_list.py", "varRefreshCmd": "print(var_dic_list())" }, "r": { "delete_cmd_postfix": ") ", "delete_cmd_prefix": "rm(", "library": "var_list.r", "varRefreshCmd": "cat(var_dic_list()) " } }, "types_to_exclude": [ "module", "function", "builtin_function_or_method", "instance", "_Feature" ], "window_display": false }, "vscode": { "interpreter": { "hash": "82eecf3e690e1a91ab50e0913e9887c9714287148abb8904ea9640df75dffa19" } } }, "nbformat": 4, "nbformat_minor": 4 }