---
name: bio-clip-seq-clip-preprocessing
description: Preprocess CLIP-seq reads (eCLIP, iCLIP, iCLIP2, iCLIP3, irCLIP, PAR-CLIP, FLASH) with protocol-specific UMI extraction, adapter trimming, length filtering, and post-alignment PCR-duplicate collapse. Use when raw CLIP FASTQ must be turned into deduplicated, crosslink-preserving BAM input for peak calling; choosing between two-pass and single-pass adapter trimming; deciding minimum read length; or mapping UMI patterns to specific eCLIP/iCLIP/iCLIP2/iCLIP3 library preps.
tool_type: cli
primary_tool: umi_tools
---

## Version Compatibility

Reference examples tested with: umi_tools 1.1.5+, cutadapt 4.6+, fastp 0.23.4+, samtools 1.19+, pysam 0.22+, picard 3.1+, preseq 3.2+.

Before using code patterns, verify installed versions match. If versions differ:
- Python: `pip show <package>` then `help(module.function)` to check signatures
- CLI: `<tool> --version` then `<tool> --help` to confirm flags

If code throws ImportError, AttributeError, or TypeError, introspect the installed package and adapt the example to match the actual API rather than retrying.

# CLIP-seq Preprocessing

**"Preprocess raw CLIP reads into UMI-deduplicated, alignable FASTQ"** -> Extract random barcodes, trim adapters without disturbing the 5' truncation site, length-filter to remove unmappable shorts, and (post-alignment) collapse PCR duplicates by UMI + position. The 5' end of the read carries the iCLIP/eCLIP truncation signature one base downstream of the protein-RNA crosslink; preserving this base is the single most important constraint of CLIP preprocessing.

- CLI (eCLIP, paired-end): `umi_tools extract --bc-pattern=NNNNNNNNNN --stdin R1.fq.gz --read2-in R2.fq.gz --stdout R1_umi.fq.gz --read2-out R2_umi.fq.gz`
- CLI (iCLIP/iCLIP2, single-end): `umi_tools extract --bc-pattern=NNNXXXXNN --extract-method=string --stdin R1.fq.gz --stdout R1_umi.fq.gz` (3+2 random Ns flanking a 4 nt library barcode; demultiplex by the X positions first if multiplexed)
- CLI (PAR-CLIP): `umi_tools extract --bc-pattern=NNNN ...` (most protocols use 4 nt random barcodes; verify the lab's exact prep)
- CLI (3' trim only, eCLIP convention): `cutadapt -a AGATCGGAAGAGCACACGTCT -A AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGT --quality-base 33 --quality-cutoff 6 -m 18 -o R1.trim.fq.gz -p R2.trim.fq.gz R1_umi.fq.gz R2_umi.fq.gz`

The eCLIP convention is: trim quality and adapter from the 3' end ONLY. The 5' end of read 2 (in paired-end eCLIP) is the truncation site of the RT enzyme at the protein-RNA adduct, located one nucleotide downstream of the crosslink. Trimming the 5' end discards that exact base. `cutadapt -g`, `fastp --trim_front1`, and aggressive quality trimming of the 5' end are all banned for CLIP unless a documented protocol-specific reason exists.

## Read Structure by Protocol

| Protocol | UMI length and location | Truncation-site read | Adapter set | Notes |
|----------|-------------------------|----------------------|-------------|-------|
| eCLIP (Van Nostrand 2016 / ENCODE) | 10 nt at R1 5' end (random nucleotides preceding insert) | R2 5' end (after R1 UMI is stripped) = crosslink -1 | Illumina TruSeq R1 + R2 + inline X1A/X1B inverted adapter for two-pass | ENCODE pipeline normative |
| seCLIP (single-end eCLIP) | 10 nt at R1 5' end | R1 5' end | TruSeq R1 only | ENCODE accepts both eCLIP and seCLIP |
| iCLIP (Konig 2010) | 5 nt random (NNNXXXXNN: 3 N + 4 X library barcode + 2 N), single-end | R1 5' end after barcode strip | L3 adapter at 3' | Multiplexed - demultiplex by the 4 X bases |
| iCLIP2 (Buchbender 2020) | 5 or 9 nt random (NNNXXXXNN or longer), single-end | R1 5' end | L3 adapter at 3' | Increased complexity vs iCLIP; same UMI pattern |
| iCLIP3 (Despic et al, bioRxiv 2026.03.01.708747) | 10 nt random + dual sample index, single-end | R1 5' end | TruSeq | Silica-column RNA isolation; non-radioactive; streamlined low-input protocol. Preprint - verify final published version before pinning a pipeline. |
| irCLIP (Zarnegar 2016) | 5 nt random + barcode (similar to iCLIP) | R1 5' end | IR700 adapter + standard TruSeq sequencing adapter | Infrared replaces 32P; otherwise iCLIP-like |
| PAR-CLIP (Hafner 2010) | 0-4 nt depending on prep | T->C transitions within reads (NOT a truncation method) | TruSeq R1 | UMI optional; rely on T->C signature for CL |
| FLASH (Ilik 2020) | Sample barcode + UMI in custom adapter | R1 5' | Custom L3 design | 1.5 day protocol; adapter design proprietary to MPI |
| miCLIP / miCLIP2 (Linder 2015 / Kortel 2021) | iCLIP-style barcodes | R1 5' = m6A -1 (truncation OR C-to-T) | iCLIP L3 | m6A-specific |
| STAMP (Brannan 2021) | NA (no UV) | NA (C-to-U editing) | 10x or bulk RNA-seq adapters | Antibody-free editing-based; preprocess as RNA-seq |

If the read structure differs from the lab's documentation, run `seqkit head -n 1000 R1.fq.gz | seqkit stats -a` and inspect the first 12 bases of 100 random reads. Random-barcode positions show ~25% base composition per position; library/sample barcodes will be fixed across reads.

## Critical Choice: One Adapter Pass vs Two

**One pass (single-end iCLIP, PAR-CLIP):** Single 3' adapter; cutadapt with `-a <L3>` and `-m 18` is sufficient.

**Two passes (eCLIP, paired-end):** The eCLIP library design ligates an inverted X1A/X1B inline adapter that can appear at either end of a short fragment due to read-through. The ENCODE pipeline does pass 1 with the standard 3' adapter, then pass 2 trims a residual 5' adapter that read-through events leave on read 2. Trimming a 5' adapter from R2 is a special case: cutadapt's `-G <ADAPTER>` (uppercase) anchored at R2's 5' end is the right invocation. Do NOT use `-g` (lowercase) for R1 5' trimming - that destroys the truncation site.

**Cutadapt full eCLIP-style invocation:**

```bash
# Pass 1 - 3' adapter (both reads)
cutadapt \
    -a AGATCGGAAGAGCACACGTCT \
    -A AGATCGGAAGAGCGTCGTGTAGGGAAAGAGTGT \
    --quality-base 33 -q 6 \
    -m 18 \
    -o R1.p1.fq.gz -p R2.p1.fq.gz \
    R1.umi.fq.gz R2.umi.fq.gz

# Pass 2 - read-through 5' adapter on R2 only (ENCODE eCLIP)
cutadapt \
    -G GATCGTCGGACTGTAGAACTCTGAAC \
    --quality-base 33 -q 6 \
    -m 18 \
    -o R1.p2.fq.gz -p R2.p2.fq.gz \
    R1.p1.fq.gz R2.p1.fq.gz
```

The `-q 6` is intentionally permissive. Aggressive `-q 20` quality trimming chews back the 5' end of R2 and breaks truncation-based crosslink-site detection. The `-m 18` minimum-length cutoff is non-negotiable: reads shorter than 18 nt are functionally unmappable (multi-map prevalence > 50%, see CIMS analysis discussions in the CLIP review literature) and must be discarded before alignment.

## Per-Protocol Failure Modes

### eCLIP -- 5' end trimmed by mistake

**Trigger:** Pipeline written for generic RNA-seq applied to eCLIP; `fastp` defaults trim both ends; `cutadapt -g` invoked on R2.

**Mechanism:** The 5' end of R2 in eCLIP is the truncation site = crosslink -1. Quality-trim from 5' or untargeted 5' adapter trim discards that exact base.

**Symptom:** Downstream PureCLIP / iCount / CTK CITS analysis fails to call crosslink sites; peak calling still works but the peaks lose nucleotide precision.

**Fix:** Use `-q 6` (3' only by default in cutadapt) and never `--trim_front2` in fastp. Validate by inspecting BAM read 2 5' positions: 60-90% of unique R2 5' positions should map within 100 nt windows around known RBP binding motifs, not be uniformly distributed.

### iCLIP / iCLIP2 -- Demultiplex confusion with UMI

**Trigger:** Multiplexed iCLIP library; user runs `umi_tools extract` with `NNNNNNNNN` (9 N) when the actual prep is `NNNXXXXNN` (5 random + 4 sample barcode).

**Mechanism:** umi_tools treats the entire 9-base prefix as UMI, losing the sample identity in the middle 4 bases. Reads from different samples are merged.

**Symptom:** Library complexity inflated artificially; per-sample read counts seem high but binding profiles look averaged across samples; sample-specific motifs absent.

**Fix:** Demultiplex BEFORE UMI extraction. Use `umi_tools extract --bc-pattern=NNNXXXXNN --extract-method=string --filter-cell-barcode` with a whitelist of the 4 X-base barcodes; or split the FASTQ with `je demultiplex` against the library barcode table first, then `umi_tools extract --bc-pattern=NNNNN` (the 5 surviving random Ns).

### PAR-CLIP -- T->C mistaken for sequencing error

**Trigger:** Standard variant-calling pipeline applied to PAR-CLIP without recognizing the T->C signature.

**Mechanism:** PAR-CLIP's diagnostic mutation is T->C (4SU adduct pairs with G during RT). Upon crosslinking the per-position T->C rate jumps from ~0.5% baseline to 20-50% (see PAR-CLIP literature; Hafner 2010, Spitzer 2014). Pipelines that filter "high-error" reads or apply STAR `--outFilterMismatchNoverLmax 0.04` (4% mismatch ceiling) discard the very reads carrying the signal.

**Symptom:** Loss of 40-70% of PAR-CLIP reads; downstream T->C site calling (PARalyzer, wavClusteR) finds almost nothing.

**Fix:** For PAR-CLIP only, raise the STAR mismatch ceiling to `--outFilterMismatchNoverLmax 0.07` (downstream in clip-alignment); also raise quality-trim tolerance in cutadapt to `-q 6` (already the CLIP default but worth re-confirming for PAR-CLIP). Track T->C rate per nucleotide position with `samtools mpileup` to confirm the signature is preserved post-alignment. **Critical exception summary:** iCLIP/eCLIP/iCLIP2 keep 0.04; PAR-CLIP only raises to 0.07. See clip-seq/clip-alignment for the alignment-stage override.

### Quality trimming too aggressive

**Trigger:** Inherited fastp/Trimmomatic pipeline with `-q 20` or `MINLEN 36`.

**Mechanism:** CLIP fragments are short (20-75 nt insert) and the 3' end carries adapter. Aggressive quality trimming and length filtering destroy 30-60% of usable reads.

**Symptom:** Per-replicate unique-mapped read count < 500k from a ~30M raw library; library complexity calculation crashes from too few reads.

**Fix:** Use `-q 6 -m 18` (cutadapt) or `--qualified_quality_phred 6 --length_required 18` (fastp). Compare retained fraction: a properly preprocessed CLIP library retains ~70-85% of raw reads through trimming.

### Library complexity below threshold

**Trigger:** PCR duplication rate >> 50%; deduplicated unique fragment count < 1M.

**Mechanism:** Low input cells (< 5M), over-amplification (> 25 PCR cycles), or failed IP all produce libraries dominated by a small number of PCR-amplified molecules.

**Symptom:** `preseq lc_extrap` predicts plateau well below 5M unique reads at infinite sequencing depth; `picard CollectLibraryComplexity` reports `ESTIMATED_LIBRARY_SIZE` < 1M; UMI families have median size > 8.

**Fix:** No analytic rescue. Re-prep the library with more input cells and fewer PCR cycles (target 14-18 cycles for eCLIP, 16-20 for iCLIP2). If the dataset must be salvaged, downsample to the unique-fragment fraction and acknowledge the loss of statistical power in differential analyses.

## UMI Deduplication Decision

After alignment, collapse PCR duplicates by `(UMI, position, strand)`. The choice between `unique` and `directional` methods is a precision/recall tradeoff:

| Method | Match rule | Behaviour | When to use |
|--------|-----------|-----------|-------------|
| `--method=unique` | Exact UMI match | Strictest; two UMIs differing by 1 base treated as independent molecules | ENCODE convention; reproducibility against published peaks |
| `--method=directional` | Network of UMIs differing by hamming-1; pick most-abundant | Collapses UMI sequencing errors; slightly fewer unique fragments reported | Highest precision; preferred when UMI sequencing error rate > 1% |
| `--method=cluster` | Connected components within edit distance | Most aggressive collapse | Default umi_tools behaviour pre-2017; not recommended for CLIP |
| `--method=adjacency` | Adjacency clustering | Between unique and directional | Rarely used for CLIP |

For eCLIP and iCLIP, the **ENCODE convention is `--method=unique`** (Van Nostrand 2016); the Yeo lab pipeline uses this exclusively. Directional adds 5-15 minutes runtime for human eCLIP at typical depth and is more conservative on rare UMI sequencing errors but produces slightly different absolute counts.

```bash
samtools index aligned.bam
umi_tools dedup \
    --stdin=aligned.bam \
    --stdout=dedup.bam \
    --method=unique \
    --paired \
    --log=dedup.log
samtools index dedup.bam
```

Without UMIs (e.g., older eCLIP with only sample barcodes, some custom protocols), fall back to `picard MarkDuplicates --REMOVE_DUPLICATES true`. The trade-off: position-only dedup over-collapses (genuinely independent fragments that share start positions are lost) at ~5-15% in deep CLIP libraries; UMI dedup recovers them.

## Pre-Mapping rRNA Filter (Optional but Standard)

eCLIP libraries are 5-30% rRNA reads even after polyA depletion. ENCODE's pipeline pre-maps to a rRNA + RepeatMasked repeat index with bowtie2, then aligns unmapped reads to the genome. This is purely a performance optimization (avoiding STAR's multi-mapper tangle on rRNA) and does not change which reads survive deduplication; it only reorders the alignment stages.

```bash
# Pre-map to rRNA + repeats
bowtie2 -x repbase_repeats -U R1.trim.fq.gz \
    --un-gz R1.norep.fq.gz \
    -p 8 -S /dev/null
# Then align unmapped reads to genome with STAR
```

For non-eCLIP protocols (iCLIP, PAR-CLIP) the rRNA pre-map is optional; the alternative is to filter rRNA-overlapping reads after STAR alignment with `bedtools intersect -v -a aligned.bam -b rRNA.bed`.

## Library Complexity Assessment

```bash
# preseq lc_extrap predicts unique fragments at deeper sequencing
preseq lc_extrap -B -P aligned.bam -o complexity.txt
# Output: TOTAL_READS, EXPECTED_DISTINCT, LOWER_0.95CI, UPPER_0.95CI
# At 100M reads, EXPECTED_DISTINCT >= 10M = good complexity; < 3M = library failed

# picard direct estimate
picard EstimateLibraryComplexity \
    I=aligned.bam \
    O=picard_complexity.txt
# ESTIMATED_LIBRARY_SIZE > 5M = healthy
```

ENCODE eCLIP requires >= 1M unique fragments per replicate (after UMI dedup). Anything below 500k is functionally unusable for genome-wide peak calling. Between 500k and 1M, restrict analysis to high-expression transcripts.

## QC Checkpoints After Preprocessing

| Metric | Target | If below target |
|--------|--------|-----------------|
| Adapter trim retention | >= 70% reads | Adapter pattern wrong; recheck library prep docs |
| Mean read length post-trim | >= 25 nt | RNA fragmentation too aggressive in IP step |
| % reads >= 18 nt | >= 80% | Drop the prep; libraries with > 30% < 18 nt indicate degraded RNA |
| UMI-extract success rate | >= 95% | UMI pattern wrong; reads do not start with random Ns |
| PCR duplication rate | 30-70% (CLIP normal) | < 30% means under-sequenced; > 90% means over-amplified |
| Library complexity (preseq) | >= 1M unique at sequenced depth | Library failed; cannot rescue analytically |

CLIP libraries have HIGH duplication rates (40-70%) by design - the IP enriches a small pool of molecules. This is NOT a problem if UMIs collapse duplicates correctly. A "30% duplication rate" CLIP library typically means either (a) the IP failed (no enrichment, so library looks like RNA-seq) or (b) the library was undersequenced and unique molecules dominate.

## Common Errors

| Error / symptom | Cause | Solution |
|-----------------|-------|----------|
| umi_tools extract: "Read does not match pattern" for > 10% of reads | UMI pattern wrong (5 vs 9 vs 10 nt) | Inspect first 12 bases of 100 reads; rerun extract with correct `--bc-pattern` |
| cutadapt: 95% reads "Too short, filtered" | Adapter sequence wrong; reads are adapter-only after trim | Verify adapter sequence in library prep documentation; check both R1 and R2 adapters |
| All reads aligning to chrM after preprocessing | Pre-map to rRNA skipped; rRNA reads dominate | Add bowtie2 rRNA pre-map OR `samtools view -F 4 -L exclude_chrM_rRNA.bed` post-align |
| umi_tools dedup very slow (> 4h on 30M BAM) | Default `--method=directional` on dense libraries | Switch to `--method=unique` (ENCODE convention) |
| eCLIP truncation positions look uniform across genome | 5' adapter trim destroyed R2 5' end | Re-preprocess; `-g` should never touch R2 5' in eCLIP |
| 30% T->C mismatches in PAR-CLIP fail mapping | STAR mismatch ceiling 0.04 too strict | Raise to `--outFilterMismatchNoverLmax 0.07` for PAR-CLIP only |
| iCLIP2 reads after dedup << expected unique | Demultiplex done with whole UMI; samples merged | Demultiplex by 4 X bases of NNNXXXXNN first, then extract UMI |

## Reconciliation: Preprocessing Tools

| Pattern | Likely cause | Action |
|---------|--------------|--------|
| fastp gives more reads than cutadapt | fastp default polyG trimming kept short reads; cutadapt min-length filtered them | Both correct; pick one and document |
| umi_tools and je-suite give different dedup counts | Different UMI distance metric (unique vs hamming-1 vs network) | Use ENCODE convention `umi_tools --method=unique` for cross-study comparability |
| Library complexity (preseq) vs (picard) disagree | preseq extrapolates non-linearly; picard estimates at sequenced depth only | Trust preseq at sequenced depth, picard for absolute library size |
| Read count after Yeo lab pipeline >> nf-core/clipseq | Yeo includes rRNA reads; nf-core pre-filters | Both correct; downstream peak callers handle this differently |

**Operational rule:** For ENCODE comparability, follow the Yeo lab eCLIP pipeline exactly: `umi_tools extract` (10 N R1) -> `cutadapt -q 6 -m 18` (two-pass) -> `bowtie2 pre-map rRNA` (unmapped to genome) -> `STAR --alignEndsType EndToEnd --outFilterMultimapNmax 1` -> `umi_tools dedup --method=unique`. For iCLIP/iCLIP2, follow the iCount preprocessing tutorial. Document any deviation in methods.

## References

- Van Nostrand EL et al 2016 Nat Methods 13:508 (eCLIP protocol, ENCODE standard)
- Konig J et al 2010 Nat Struct Mol Biol 17:909 (original iCLIP, truncation principle)
- Buchbender A et al 2020 Methods 178:33 (iCLIP2 protocol, library complexity gain)
- Lee FCY et al 2021 bioRxiv 2021.08.27.457890 (iiCLIP / improved iCLIP, motif specificity)
- Hafner M et al 2010 Cell 141:129 (PAR-CLIP, 4SU labeling, T->C signature)
- Zarnegar BJ et al 2016 Nat Methods 13:489 (irCLIP, non-radioactive)
- Ilik IA et al 2020 Nucleic Acids Res 48:e15 (FLASH, fast protocol)
- Smith T et al 2017 Genome Res 27:491 (UMI-tools, network-based dedup)
- Daley T & Smith AD 2013 Nat Methods 10:325 (preseq library complexity)
- West C et al 2023 Wellcome Open Res 8:286 (nf-core/clipseq pipeline)

## Related Skills

- clip-seq/clip-alignment - Downstream STAR/bowtie2 alignment with ENCODE parameters
- clip-seq/clip-qc - Library complexity, FRiP, IDR, read-distribution QC after preprocessing
- clip-seq/crosslink-site-detection - Why preserving the 5' R2 base is critical
- clip-seq/clip-peak-calling - Downstream peak callers consume the dedup BAM
- clip-seq/stamp-antibody-free - STAMP/DART-seq use RNA-seq preprocessing instead of UMI-based CLIP preprocessing
- read-qc/umi-processing - General UMI handling concepts
- read-qc/adapter-trimming - General adapter trimming
- alignment-files/duplicate-handling - Picard MarkDuplicates fallback when UMIs unavailable