# Drosophila MaleCNS perturbomics v2

Whole-CNS in silico perturbation analysis on the MaleCNS connectome. Each annotated cell-type unit is perturbed once and its downstream influence is summarized across biologically annotated effector channels.

The main object is

\[
P_{g,i}=\text{indirect influence of perturbing unit }i\text{ on effector group }g.
\]

The analysis is deliberately unsigned. The connectome provides synaptic weights, but transmitter sign is not sufficiently reliable for all relevant cell classes to make a signed model the primary analysis.

## Paper question

Does the adult fly CNS contain a limited repertoire of perturbation-response programs, such that anatomically distinct cell types can produce similar downstream effector profiles, and does structural centrality fully explain which types exert the largest downstream effects?

## Prespecified analyses

- **H1 — low-dimensional perturbome.** PCA on L1-normalized perturbation signatures; parallel-analysis null determines how many PCs exceed a shuffled-output baseline. PCA loadings identify the effector composition of those programs.
- **H2 — specialist/generalist axis.** Normalized Shannon breadth and Hill q=2 effective output count; superclass differences are tested with Kruskal-Wallis, epsilon-squared, and BH-adjusted pairwise Wilcoxon tests.
- **H3 — degeneracy.** For every active unit, identify its nearest perturbational neighbour after excluding all units of the same annotated type. A matched nearest-neighbour permutation null repeats the same search after shuffling effector channels across perturbations; functional similarity is also compared with structural overlap (in/out-neighbour Jaccard), direct connectivity, superclass identity, and multivariate topology distance.
- **H4 — external DN sanity validation.** Published DN activation phenotypes are compared against predicted perturbation-signature similarity. The test statistic is the mean cosine similarity difference between same-phenotype and different-phenotype DN pairs, with a one-sided phenotype-label permutation test.
- **Topology versus leverage.** Degree, strength, PageRank, HITS hub score, coreness and participation coefficients predict log indirect influence in an out-of-fold ridge model. The residual defines perturbational leverage beyond what those structural quantities predict.

## Primary model

\[
r_\infty=(I-W)^{-1}s
\]

with `W[target, source]` built from synapse counts normalized by total input to the postsynaptic unit. Neuron-pair edges below the synapse threshold are removed **before** aggregation to cell-type units.

The main result uses:

- neuron-pair synapse threshold: **5**;
- spectral radius after rescaling: **0.90**;
- output taxonomy: **fine effector groups**.

`rho = 0.90` is used as the primary operating point because it allows multi-hop propagation without placing the linear system as close to the critical boundary as `rho = 0.99`. The pipeline does not rely on this choice: `06_sensitivity.R` explicitly repeats the analysis across `rho = 0.50, 0.70, 0.90, 0.95, 0.99`.

The principal matrix is indirect influence:

\[
P_{indirect}=B[(I-W)^{-1}-I].
\]

Subtracting `B` removes the direct self-stimulation term for effector units.

## Complete workflow

```text
00_sanity.R (fail-fast core algebra/name checks)
    |
    v
00_download.R
    |
    v
raw MaleCNS annotations + weighted neuron graph
    |
    v
01_prepare.R
    |-- body_to_unit.parquet
    |-- unit_meta.parquet
    |-- effector_membership.parquet
    `-- type_edges.parquet
             |
             v
02_perturbome.R
    |-- perturbome_raw.parquet
    |-- perturbome_indirect.parquet
    `-- model_qc.tsv
             |
     +-------+--------------------+
     |                            |
     v                            v
03_metrics.R                 04_validate_dns.R
H1/H2/H3                     H4
PCA + parallel null          DN pair similarities
breadth tests                permutation test
nearest distinct type
     |
     v
05_topology.R
centrality + communities + participation
ridge topology->influence
residual leverage
structural check of H3 pairs
     |
     v
06_sensitivity.R
rho sweep
synapse-threshold sweep
output-taxonomy sweep
     |
     v
07_figures.R
paper-facing figures
     |
     v
08_summary.R
paper_evidence.tsv
```

## Run

Install once:

```r
install.packages(c(
  "arrow", "DBI", "duckdb", "dplyr", "tidyr", "purrr", "stringr",
  "Matrix", "RSpectra", "irlba", "uwot", "FNN", "igraph", "glmnet",
  "ggplot2", "readr", "tibble"
))
```

Then:

```bash
./run_all.sh 2>&1 | tee v2_run.log
```

For a quicker dry run of the parallel/permutation pieces only, without changing committed defaults:

```bash
HORN_PERMUTATIONS=20 DEGENERACY_PERMUTATIONS=19 DN_PERMUTATIONS=100 ./run_all.sh
```

The paper run should use the defaults (`200` parallel-analysis permutations, `99` degeneracy-null permutations on a fixed 3,000-source subset, and `10,000` DN label permutations).

## Main outputs

### Graph and perturbome

- `data/derived/body_to_unit.parquet`
- `data/derived/unit_meta.parquet`
- `data/derived/effector_membership.parquet`
- `data/derived/type_edges.parquet`
- `results/perturbome_raw.parquet`
- `results/perturbome_indirect.parquet`
- `results/model_qc.tsv`
- `results/package_versions.tsv`

### H1

- `results/perturbome_pca.tsv`
- `results/perturbome_pca_loadings.tsv`
- `results/perturbome_pca_variance.tsv`
- `results/perturbome_parallel_analysis.tsv`
- `results/perturbome_umap.tsv`

### H2

- `results/perturbome_metrics.tsv`
- `results/breadth_by_superclass.tsv`
- `results/breadth_superclass_test.tsv`
- `results/breadth_superclass_pairwise.tsv`

### H3

- `results/degeneracy_pairs.tsv`
- `results/degeneracy_summary.tsv`
- `results/degeneracy_null.tsv`
- `results/degeneracy_null_summary.tsv`
- `results/degeneracy_structural.tsv`
- `results/degeneracy_structural_summary.tsv`

### H4

- `results/dn_validation.tsv`
- `results/dn_validation_summary.tsv`

### Topology / leverage

- `results/topology_metrics.tsv`
- `results/topology_correlations.tsv`
- `results/topology_influence.tsv`
- `results/topology_model_summary.tsv`

### Robustness

- `results/sensitivity/model_sensitivity.tsv`
- `data/derived/sensitivity/type_edges_min*.parquet`

### Consolidated evidence

- `results/paper_evidence.tsv`

## Figures

- `figures/perturbome_atlas.pdf`
- `figures/perturbome_umap.pdf`
- `figures/pca_variance.pdf`
- `figures/pca_parallel_analysis.pdf`
- `figures/breadth_by_superclass.pdf`
- `figures/degeneracy_structure.pdf`
- `figures/topology_vs_influence.pdf`
- `figures/high_leverage_types.pdf`
- `figures/dn_validation.pdf` when the historical DN set is found
- `figures/sensitivity_rho.pdf`
- `figures/sensitivity_synapse_threshold.pdf`
- `figures/sensitivity_effector_taxonomy.pdf`

## Important interpretation rules

1. **Pattern and magnitude are separate.** PCA, UMAP, breadth and cosine similarity use L1-normalized signatures. `total_indirect_influence` is analyzed separately.
2. **Zero-response units stay in the atlas/QC but do not receive fake breadth or Hill-number values.** Those fields are `NA` for inactive perturbations.
3. **Degeneracy means distinct annotated types.** Same-type units split by neuromere/nerve are excluded from the nearest-neighbour claim.
4. **UMAP is visualization only.** No inferential claim is based on UMAP geometry.
5. **High leverage is out-of-fold.** The residual comes from cross-validated ridge predictions, not an in-sample regression fit.
6. **The near-critical regime is a sensitivity condition, not the only model.** `rho = 0.99` is retained explicitly so saturation/mixing can be seen rather than hidden.
7. **No automatic GO decision is encoded.** `paper_evidence.tsv` exposes the quantities needed to decide after the full v2 run.

## External DN sanity set

The coarse phenotype labels in `04_validate_dns.R` are taken from the 2018 optogenetic DN screen by Cande et al. The set contains examples of global locomotor activation, head grooming, abdomen grooming and anterior reaching. It is intentionally a small external sanity test, not the final biological validation dataset.

## References / provenance

- MaleCNS official download: https://male-cns.janelia.org/download/
- Berg et al. (2026), complete male CNS connectome.
- Bates et al. (2026), connectome influence framework.
- Cande et al. (2018), *Optogenetic dissection of descending behavioral control in Drosophila*, eLife.
- Namiki et al. (2018), *The functional organization of descending sensory-motor pathways in Drosophila*, eLife.