.out` file generated by the scheduler in the
same directory. This file contains most of the `STDOUT` and `STDERROR`
generated by a cluster job. Once everything is working on the toy data, users
can run the workflow on real data the same way.
### Visualize workflow
_`systemPipeR`_ workflows instances can be visualized with the `plotWF` function.
The resulting plot includes the following information.
- Workflow topology graph (dependency graphs between different steps)
- Workflow step status, *e.g.* `Success`, `Error`, `Pending`, `Warnings`
- Sample status and statistics
- Workflow timing: run duration time
If no argument is provided, the basic plot will automatically detect width,
height, layout, plot method, branches, _etc_.
```{r, eval=TRUE}
plotWF(sal)
```
### Technical reports
_`systemPipeR`_ compiles all workflow execution logs in one central location,
making it easy to check any standard output (`stdout`) or standard error
(`stderr`) for any command-line tool used in a workflow.
Also, the information is appended to the workflow plot making it easy to click on
respective steps.
```{r, eval=FALSE}
sal <- renderLogs(sal)
```
### Scientific reports
_`systemPipeR`_ auto-generates scientific analysis reports in HTML format. These reports
compile the results of all workflow steps including text, code, plots and tables.
```{r, eval=FALSE}
sal <- renderReport(sal)
```
Alternatively, scientific reports can be rendered with the `render` function from `rmarkdown`.
```{r, eval=FALSE}
rmarkdown::render("systemPipeRNAseq.Rmd", clean=TRUE, output_format="BiocStyle::html_document")
```
## How to modify workflows?
### Modifying steps
If needed one can modify existing workflow steps in a pre-populated
`SYSargsList` object, and potentially already executed workflows, with the
`replaceStep(sal) <-` replacement function. The following gives an example
where step number 3 in a `SYSargsList` (sal) object will be updated with
modified or new code. Note, this is a generalized example where the user needs
to insert the code lines between `{` and `}`, and also adjust the values assigned
to the arguments: `step_name` and `dependency`.
```{r modify_WF, eval=FALSE}
replaceStep(sal, step=3) <- LineWise(
code = {
...
},
step_name = "my_step_name",
dependency = "my_dependency")
```
Since `step_names` need to be unique, one should avoid using the same
`step_name` as before. If the previous name is used, a default name will be
assigned. Rerunning the assignment will then allow to assign the previous name. This
behavior is enforced for version tracking. Subsequently, one can view and check
the code changes with `codeLine()`, and then rerun the corresponding step (here
3) as follows:
```{r run_WF_steps, eval=FALSE}
codeLine(stepsWF(sal)$my_step_name)
runWF(sal, steps=3)
```
As mentioned above, any step in a workflow can only be run in isolation if its
expected input exists (see `dependency`).
### Adding steps
New steps can be added to the Rmd file of a workflow by inserting new R
Markdown code blocks (chunks) starting and ending with the usual `appendStep<-`
syntax and then creating a new `SYSargsList` instance with `importWF` that
contain the new step(s). To add steps to a pre-populated `SYSargsList` object,
one can use the `after` argument of the `appendStep<-` function. The following
example will add a new step after position 3 to the corresponding `sal` object.
This can be useful if a workflow with a long runtime has already been completed
and one only wants to make some refinements without re-running the entire
workflow.
```{r adding_WF_steps, eval=FALSE}
appendStep(sal, after=3) <- ...
```
## Workflow templates
Workflow templates are provided via the affiliated Bioconductor data package
named `systemPipeRdata`, as well as by a dedicated GitHub repository. Instances
of these workflows can be created with a single command. The following gives
several examples.
### RNA-Seq WF template
[ [Tutorial 1](https://bit.ly/3fMB5hm) ] [ [Tutorial 2](https://systempipe.org/SPrnaseq/index.html) ]
Load the RNA-Seq sample workflow into your current working directory.
```{r genRna_workflow_single, eval=FALSE}
library(systemPipeRdata)
genWorkenvir(workflow="rnaseq")
setwd("rnaseq")
```
#### Create the workflow
This template includes the most common analysis steps of `RNAseq` workflows. One can add, remove, modify
workflow steps by modifying the `sal` object.
```{r project_rnaseq, eval=FALSE}
sal <- SPRproject()
sal <- importWF(sal, file_path = "systemPipeRNAseq.Rmd", verbose = FALSE)
```
**Workflow steps:**
1. Read preprocessing
+ Quality filtering (trimming)
+ FASTQ quality report
2. Alignments: _`HISAT2`_ (or any other RNA-Seq aligner)
3. Alignment stats
4. Read counting
5. Sample-wise correlation analysis
6. Analysis of differentially expressed genes (DEGs)
7. GO term enrichment analysis
8. Gene-wise clustering
#### Run workflow
```{r run_rnaseq, eval=FALSE}
sal <- runWF(sal)
```
Workflow visualization
```{r plot_rnaseq, eval=FALSE}
plotWF(sal)
```
Report generation
```{r report_rnaseq, eval=FALSE}
sal <- renderReport(sal)
sal <- renderLogs(sal)
```
### ChIP-Seq WF template
[ [Tutorial 1](https://bit.ly/3oQ14Mf) ] [ [Tutorial 2](https://systempipe.org/SPchipseq/index.html) ]
Load the ChIP-Seq sample workflow into your current working directory.
```{r genChip_workflow_single, eval=FALSE}
library(systemPipeRdata)
genWorkenvir(workflow="chipseq")
setwd("chipseq")
```
**Workflow steps:**
1. Read preprocessing
+ Quality filtering (trimming)
+ FASTQ quality report
2. Alignments: _`Bowtie2`_ or _`rsubread`_
3. Alignment stats
4. Peak calling: _`MACS2`_
5. Peak annotation with genomic context
6. Differential binding analysis
7. GO term enrichment analysis
8. Motif analysis
#### Create the workflow
This template provides some common steps for a `ChIPseq` workflow. One can add, remove, modify
workflow steps by operating on the `sal` object.
```{r project_chipseq, eval=FALSE}
sal <- SPRproject()
sal <- importWF(sal, file_path = "systemPipeChIPseq.Rmd", verbose = FALSE)
```
#### Run workflow
```{r run_chipseq, eval=FALSE}
sal <- runWF(sal)
```
Workflow visualization
```{r plot_chipseq, eval=FALSE}
plotWF(sal)
```
Report generation
```{r report_chipseq, eval=FALSE}
sal <- renderReport(sal)
sal <- renderLogs(sal)
```
### VAR-Seq WF template
[ [Tutorial](https://systempipe.org/SPvarseq/index.html) ]
Load the VAR-Seq sample workflow into your current working directory.
```{r genVar_workflow_single, eval=FALSE}
library(systemPipeRdata)
genWorkenvir(workflow="varseq")
setwd("varseq")
```
**Workflow steps:**
1. Read preprocessing
+ Quality filtering (trimming)
+ FASTQ quality report
2. Alignments: _`gsnap`_, _`bwa`_
3. Variant calling: _`VariantTools`_, _`GATK`_, _`BCFtools`_
4. Variant filtering: _`VariantTools`_ and _`VariantAnnotation`_
5. Variant annotation: _`VariantAnnotation`_
6. Combine results from many samples
7. Summary statistics of samples
#### Create the workflow
This template provides some common steps for a `VARseq` workflow. One can add, remove, modify
workflow steps by operating on the `sal` object.
```{r project_varseq, eval=FALSE}
sal <- SPRproject()
sal <- importWF(sal, file_path = "systemPipeVARseq.Rmd", verbose = FALSE)
```
#### Run workflow
```{r run_varseq, eval=FALSE}
sal <- runWF(sal)
```
Workflow visualization
```{r plot_varseq, eval=FALSE}
plotWF(sal)
```
Report generation
```{r report_varseq, eval=FALSE}
sal <- renderReport(sal)
sal <- renderLogs(sal)
```
### Ribo-Seq WF template
[ [Tutorial](https://systempipe.org/SPriboseq/index.html) ]
Load the Ribo-Seq sample workflow into your current working directory.
```{r genRibo_workflow_single, eval=FALSE}
library(systemPipeRdata)
genWorkenvir(workflow="riboseq")
setwd("riboseq")
```
**Workflow steps:**
1. Read preprocessing
+ Adaptor trimming and quality filtering
+ FASTQ quality report
2. Alignments: _`HISAT2`_ (or any other RNA-Seq aligner)
3. Alignment stats
4. Compute read distribution across genomic features
5. Adding custom features to workflow (e.g. uORFs)
6. Genomic read coverage along transcripts
7. Read counting
8. Sample-wise correlation analysis
9. Analysis of differentially expressed genes (DEGs)
10. GO term enrichment analysis
11. Gene-wise clustering
12. Differential ribosome binding (translational efficiency)
#### Create the workflow
This template provides some common steps for a `RIBOseq` workflow. One can add, remove, modify
workflow steps by operating on the `sal` object.
```{r project_riboseq, eval=FALSE}
sal <- SPRproject()
sal <- importWF(sal, file_path = "systemPipeRIBOseq.Rmd", verbose = FALSE)
```
#### Run workflow
```{r run_riboseq, eval=FALSE}
sal <- runWF(sal)
```
Workflow visualization
```{r plot_riboseq, eval=FALSE}
plotWF(sal, rstudio = TRUE)
```
Report generation
```{r report_riboseq, eval=FALSE}
sal <- renderReport(sal)
sal <- renderLogs(sal)
```
```{r quiting, eval=TRUE, echo=FALSE}
knitr::opts_knit$set(root.dir = '../')
unlink("rnaseq", recursive = TRUE)
```
## Funding
This project is funded by NSF award [ABI-1661152](https://www.nsf.gov/awardsearch/showAward?AWD_ID=1661152).
## Version information
**Note:** the most recent version of this tutorial can be found here.
```{r session_info, eval=TRUE}
sessionInfo()
```
## References
