# nf-core: agents

This is the main AI context file for nf-core pipelines. All AI agents and coding assistants **MUST** follow the rules contained in this document.

## Natural language
All comments and documentation **MUST** be written in English with British spelling. Documentation files **SHOULD** additionally follow the style guide at https://nf-co.re/docs/developing/documentation/style-guide.
Never use emdashes in prose text, be succinct and to the point. Avoid telltale LLM phrasing such as "Not X, but Y", and excessive use of bold formatting.

## Key nf-core terms
- Module: a single process that achieves a single, well defined task (e.g. aligning reads to a genome)
- Subworkflow: a sequence of chained modules that achieve a specific objective (e.g. FASTQ cleanup and quality check)
- Workflow: a complete sequence of modules and subworkflows that performs a specific analysis (e.g. bulk RNA-seq analysis)
- Pipeline: a complete, executable Nextflow project that defines workflow logic, input handling, and output publishing

## Nextflow pitfalls
- Nextflow supports 2 ways to publish files to the output directory: workflow outputs (modern) and `publishDir` configuration directives in modules.config (legacy). You **SHOULD** publish output consistently with the existing code.
- nf-core tools commands may fail. If that happens, ask the user for help. You **MUST NOT** generate any file that is supposed to be generated by nf-core tools.
- Certain very old pipelines might be using Nextflow DSL1 syntax (with the entire workflow in a single file and channel from/to keywords). This syntax is now deprecated. You **MUST NOT** attempt to work on those pipelines.

## nf-core template structure
The directory you are working on was created with the nf-core pipeline template. Key features of the template are demonstrated below:
```
.

├── conf                   // directory containing Nextflow configurations for the pipeline (see "Configuration files" below)
├── main.nf                // core Nextflow script, may need editing if input structure changes
├── modules                // Nextflow DSL2 modules
│   ├── local              // local modules (see "Modules" below)
|   |   └── mymodule       // each module must be in a separate directory 
│   └── nf-core            // nf-core modules (see "Modules" below)
├── nextflow_schema.json   // JSON schema describing pipeline parameters
├── subworkflows           // Nextflow subworkflows (see "Subworkflows" below)
│   ├── local              // local subworkflows
|   |   └── myswf          // each subworkflow must be in a separate directory 
│   └── nf-core            // nf-core subworkflows
├── tests                  // nf-test end-to-end tests for the pipeline
│   └── default.nf.test    // main test script, must exist
└── workflows              // do not add files 
    └── {pipeline-name}.nf // Nextflow file containing main pipeline logic
```

The pipeline also contains other files and directories. If a file does not follow the treemap above, you **MUST** verify with the user before editing it.

## Modules
- You **SHOULD** use existing nf-core modules for the tools you need, where available.
- You can find available modules and install modules with nf-core tools (see "nf-core tools" section below).
- You **SHOULD NOT** edit nf-core modules in the pipeline modules directory. If unavoidable, you **MAY** edit their `main.nf` if necessary, and if target pipeline logic cannot be achieved with the existing module code. If you do it, you **MUST** run `nf-core modules patch {name}` afterwards, and flag that a PR will be needed to upstream the change.
- The pipeline has a local modules directory. If a script is only useful within this pipeline, you **MAY** create a local module for it.
- Use nf-core tools (see below) to create local module boilerplate and then edit the files.
- Use `ext.args` to pass any command-line arguments (except input files) to the underlying tool.
- Use `ext.prefix` to customize the name of the output files. To include runtime variables in those arguments, use Groovy-style closures, for example: `ext.prefix = { "${meta.id}_filtered" }`, usually through the modules.config file.

## Subworkflows
- You **SHOULD** use nf-core subworkflows that are relevant to the pipeline tasks.
- If none is applicable, create a local subworkflow when it thematically makes sense.

## Pipeline structure
An nf-core pipeline contains 3 main parts called by the root `workflow` block in `main.nf`:
- initialisation workflow (defined in `subworkflows/local/utils_nfcore_{name}_pipeline/main.nf`): handles input processing and validation
- main workflow (defined in `workflows/{name}.nf`): contains the main analysis, including generation of all output files (some nf-core pipelines contain more than one workflow)
- completion workflow (defined in `subworkflows/local/utils_nfcore_{name}_pipeline/main.nf`): handles sending completion notifications

## Configuration files
- You **MUST NOT** edit `base.config`, `igenomes.config`, and `igenomes_ignored.config`
- Set `ext.args` and `ext.prefix` for modules in `modules.config`, using `withName` blocks
- The test in `test.config` **SHOULD** take a few minutes and only test the basic functionality with minimal input
- The test in `test_full.config` **SHOULD** use input and parameters that trigger all pipeline functionality

## Meta map
The meta map is a Nextflow map passed along with each file that contains sample-specific information. The map is created during input processing and passed through modules.

- The meta map **MUST** contain an `id` field with a unique identifier.
- nf-core modules and subworkflows may **only** access `id` and `single_end` fields.
- Local modules, subworkflows, and workflows may create and access any meta fields that are useful for the pipeline.
- Channel operations **MUST** preserve the meta map when present; they **MAY** add, remove, or modify specific keys as required.

## nf-core tools
nf-core provides a CLI toolkit for working with the nf-core template. The core command is `nf-core`. You **SHOULD** always use the tools instead of creating files manually.

Use `nf-core --help` to obtain information about nf-core commands. You can also use `--help` for subcommands.

Write the names of subtool modules in commands with a slash, like `samtools/sort`.

## nf-test and testing
- Each pipeline **MUST** have at least 1 test case.
- Tests have a standardized syntax, with setup (optional), input ("when"), and assertion ("then") sections.
- Tests at a path can be executed with `nf-test test {path}`.
- Most tests create at least 1 snapshot file. You **MUST NOT** edit snapshots manually.
- If you expect the output to change (e.g. after a tool update), update the snapshot with `nf-test test --profile +{docker|singularity|conda} --update-snapshot`. Only regenerate snapshots on the same CPU architecture as CI.
- If a new output file has unstable content, add it to `.nftignore`. 

## git and branch policy
This repository has at least 3 git branches: `main` (or `master`), `dev`, and `TEMPLATE`.
- You **MUST NOT** switch or write to the TEMPLATE branch.
- You **MUST NOT** write any code to `main`; use a pull request instead.
- Always create a new branch with a meaningful name for each feature, then open a pull request to `dev`.
- You **MUST NOT** commit feature work directly to `dev`, even on a fork.
- If you work on multiple features in parallel, you **SHOULD** use a separate worktree for each task to prevent clobber.

## Commit rules and routine
- Each commit **SHOULD** contain one logical change.
- Commit title **SHOULD** be concise and written in imperative mood.
- If the commit consists only of installing or updating an nf-core module or subworkflow, limit the commit title to `Install/update nf-core module/subworkflow {name}`.
- Before each commit, you **MUST** stage changes and then run `prek`. Resolve all errors and all possible warnings. Repeat until there are no solvable outstanding issues.

## Push routine
- You should only push to GitHub after implementing some meaningful changes and if the code is working.
- Before pushing, you **MUST** run `nf-core pipelines lint`, resolve all errors and all possible warnings. Repeat until there are no solvable outstanding issues.
- If you are preparing a release (PR to main), use `nf-core pipelines lint --release` instead.
- You **MUST** also run `nf-test test tests/`. If the pipeline fails, resolve the underlying issues. If the test fails due to mismatching snapshots, update them if permitted (see "nf-test and testing" above). Otherwise, fix the issue that caused the unexpected change.
- If you know the code will cause issues or you intend to push more changes, you **SHOULD** add `[skip ci]` at the end of the commit title. You **SHOULD** omit this tag for final review-ready commits.

## PR procedure
- A PR **SHOULD** contain a single feature.
- You **SHOULD** add a line in the relevant section in CHANGELOG.md, listing contributors and the expected PR number.
- The PR **MUST** use and follow the nf-core PR template, including the checklist.
- The PR message **SHOULD** start with a brief explanation of the changes made and the motivation.
- Each PR requires reviews (1 for dev, 2 for main) and passing CI before merging.
- A human can request PR reviews on Slack.
 
## Agent self-disclosure
- If you generated a majority of the code in a commit, you **MUST** add "Generated by {your name}" at the end of the commit message body.
- If you open a PR autonomously, you **MUST** add "Generated by {your name}" at the end of the PR message (above the checklist).

## References
- Nextflow documentation: https://docs.seqera.io/nextflow
- nf-core tools documentation: https://nf-co.re/docs/nf-core-tools/
- nf-test documentation: https://www.nf-test.com/docs/getting-started/