##################################################
## Submit workflow from command-line to cluster ##
##################################################
## Last update: 26-May-2024

## These are step-wise instructions for running the RNA-Seq or other workflows
## entirely from the command-line. After logging in to your HPCC cluster
## account, do the following:

## (1) Start a `tmux` session (optional but very helpful). Make a note of the
## name of the head node (`skylark` or `bluejay`) from where you started `tmux`
## since it can only be restarted from the head node where it is running.

tmux a

## (2) Then `cd` into your bigdata space of GEN242

cd /bigdata/gen242/<user_name>

## (3) Log into a computer node with `srun`

srun --partition=gen242 --account=gen242 --mem=10gb --cpus-per-task 8 --ntasks 1 --time 20:00:00 --pty bash -l

## (4) Generate `rnaseq` workflow directory/environment and `cd` in to it

Rscript -e "systemPipeRdata::genWorkenvir(workflow='rnaseq')"
cd rnaseq

## (5) Create submission subdirectory and `cd` in to it

mkdir cl_sbatch_run
cd cl_sbatch_run

## (6) Download submission files. Instructions for the following steps are from
## here: rebrand.ly/4tvgnyr

wget https://raw.githubusercontent.com/tgirke/GEN242/main/static/custom/spWFtemplates/cl_sbatch_run/wf_run_script.R
wget https://raw.githubusercontent.com/tgirke/GEN242/main/static/custom/spWFtemplates/cl_sbatch_run/wf_run_script.sh

## In user accounts where `sbatch` has a problem with the `gen242` partition,
## replace the default batchtools.slurm.tmpl file in the workflow directory
## with an updated version by running:

wget https://raw.githubusercontent.com/tgirke/GEN242/main/static/custom/spWFtemplates/cl_sbatch_run/batchtools.slurm.tmpl -O ../batchtools.slurm.tmpl

## Alterntively, changing in the wf_run_script.R script “gen242” to “epyc” under the 
## partition argument will also work.


## (7) Submit workflow via sbatch from `cl_sbatch_run` directory

sbatch wf_run_script.sh

## (8) Now monitor the run with the following commands (there are many other
## useful commands, but these are the most important ones)

squeue -u <your_username> # shows your current processes in Slurm
ll ../result/ # shows output files generated by workflow
less slurm-<idno>.out # show logs of run. Type capital G to get to end of file and keep refreshing to see latest output

## The run will be finished when the analysis report is generated (here
## SPRrnaseq.Rmd and/or SPRrnaseq.html). You can also check the
## `slurm-<idno>.out` log file with `less` as shown above. You can also request
## to receive an email when the run is finished by providing your email address
## under `--mail-user` in `wf_run_script.sh`.

## When running the workflow on the real data, I recommend to make sure that
## the maximum run time parameter (walltime) in the Slurm settings is set to at
## least 2 hours and the memory to >=10Gb.