# Design Patterns for Discriminating Tests

## Design starts from the prediction

For each test, link:

`candidate → mechanism → prediction → observable → operationalization → design → analysis → interpretation`

Choose the design that can distinguish candidates under realistic uncertainty. Do not select a design merely because it is familiar or available.

## NIH-aligned rigor questions

Where applicable, address:

- rigor of the prior research forming the scientific premise;
- unbiased and well-controlled design;
- relevant biological variables such as sex, age, weight, or health condition;
- authentication and validity of key biological/chemical resources;
- transparent methods, analysis, interpretation, and reporting.

Apply only the elements relevant to the science and explain omissions.

## Core design record

Every design should state:

- study system and target population;
- experimental/observational unit and analysis unit;
- sampling frame and recruitment/selection;
- interventions/exposures and comparator versions;
- allocation, randomization, concealment, and masking;
- outcomes, timing, measurement IDs, and quality control;
- positive, negative, vehicle/sham, procedural, or reference controls as applicable;
- inclusion, exclusion, attrition, and stopping;
- sample-size, precision, or information rationale;
- analysis IDs and estimands;
- safety, ethics, feasibility, data, and regulatory gates;
- replication, transport, and external-validation plan.

## Experimental designs

### Randomized intervention

Useful for causal contrasts when intervention, allocation, and ethics permit.

Check:

- allocation sequence and concealment;
- intervention versions, adherence, contamination, and co-interventions;
- masking of participants, providers, outcome assessors, and analysts where feasible;
- primary estimand and intercurrent events;
- intention-to-treat or other analysis population aligned to the estimand;
- harms, stopping, missing outcomes, and protocol deviations.

Randomization does not solve measurement bias, nonadherence, post-randomization selection, interference, or poor external validity.

### Factorial design

Useful for multiple interventions and interactions.

Check:

- scientific meaning and scale of interaction;
- power/precision for interactions, not only main effects;
- compatibility and safety of combined conditions;
- multiplicity and hierarchy;
- whether sparse combinations undermine interpretation.

### Within-unit or crossover design

Useful when effects are reversible and carryover can be controlled.

Check:

- period and sequence effects;
- washout rationale;
- time trends and learning;
- missing periods;
- whether the condition is stable and intervention reversible.

### Perturbation and rescue

Useful for mechanistic candidates when ethically and technically appropriate.

Check:

- perturbation specificity and off-target effects;
- manipulation check;
- rescue interpretation and overexpression artifacts;
- temporal order;
- orthogonal perturbations and measurements;
- relevant negative and positive controls.

A rescue can still be explained by compensatory or non-specific effects.

### Time-course

Useful when candidates predict different ordering or dynamics.

Check:

- sampling times justified by expected process;
- independent versus repeated units;
- baseline and pre-trend;
- measurement stability across time;
- multiple looks and timepoint multiplicity;
- lag, feedback, and reverse causation.

## Observational designs

### Cross-sectional

Can estimate prevalence and associations at a defined time. It usually cannot establish temporal direction. Explicitly consider selection, reverse causation, survival/prevalence bias, and common-method measurement.

### Cohort/longitudinal

Can establish measured temporal ordering and incidence. It does not eliminate confounding.

Check:

- time zero and eligibility;
- exposure updates and time-varying confounding;
- loss to follow-up and informative censoring;
- competing events;
- immortal-time and delayed-entry risks;
- outcome ascertainment changes.

### Case-control

Efficient for some rare outcomes.

Check:

- source population and control sampling;
- matching implications;
- exposure measurement and recall;
- selection mechanisms;
- correct effect measure and sampling analysis.

### Natural/quasi-experimental

Can strengthen causal identification when an assignment mechanism or discontinuity is credible.

Check:

- assignment mechanism and manipulation;
- continuity, parallel trends, exclusion, or instrument assumptions as applicable;
- anticipation and spillovers;
- bandwidth/window choices;
- placebo/negative-control tests;
- sensitivity to specification and clustering.

The design label alone does not establish identification.

## Computational and theoretical designs

### Simulation

Use to test implications of assumptions, estimator behavior, or model dynamics.

Record:

- data-generating process and parameter ranges;
- rationale for scenarios;
- seeds and software versions;
- performance targets and uncertainty;
- failure cases and sensitivity;
- separation between simulated truth and empirical validity.

Simulation can show consequences within a model, not that the model describes nature.

### Predictive model evaluation

Separate prediction from causation.

Check:

- target population, outcome, time origin, and horizon;
- leakage and preprocessing;
- train/tune/test independence;
- calibration and discrimination;
- uncertainty and subgroup performance;
- temporal/geographic/external validation;
- dataset shift and update policy.

### Secondary-data analysis

Record provenance, data-generating process, inclusion, missingness, transformations, version, and prior analysis exposure. Avoid using the same data to generate and confirm a hypothesis without transparent separation or independent validation.

## Controls

### Positive control

A condition expected to produce a known response. It checks whether the system and measurement can detect a relevant effect.

### Procedural control

Matches handling, timing, delivery, or processing without the target active component.

### Negative control

Should not operate through the target mechanism but should share relevant bias pathways. State assumptions and failure interpretation.

### Null comparator

A comparator representing no intervention or no difference may be useful, but it is not equivalent to a negative control and may not isolate placebo, handling, expectancy, or background trends.

## Measurement validity

Before collecting target outcomes:

- define constructs and proxies;
- verify instrument validity in the target context;
- assess reliability/repeatability;
- calibrate and authenticate resources;
- prespecify detection limits and quality failures;
- plan masking and standardized acquisition;
- define missing/invalid values;
- test cross-site, cross-device, cross-group, or longitudinal comparability where relevant.

A precise measure can be precisely wrong. Technical replicates do not replace independent biological, participant, site, or experimental units.

## Sample size and precision

Do not use universal minima.

Base planning on:

- primary estimand and effect/precision target;
- expected variability and dependence;
- allocation ratio;
- attrition/missingness;
- multiplicity or sequential design;
- model complexity;
- feasibility and ethical burden;
- uncertainty in planning values.

Report assumptions and sensitivity to them. Pilot data may be too unstable for definitive effect-size planning; use external evidence, conservative ranges, or precision-based goals where appropriate.

## Multiplicity

Inventory:

- candidate hypotheses;
- outcomes and timepoints;
- subgroups and interactions;
- model specifications and transformations;
- interim looks and stopping;
- repeated datasets or cohorts.

Prespecify a strategy appropriate to the inferential goal, such as:

- family-wise error control;
- false-discovery-rate control;
- hierarchical/gatekeeping testing;
- multilevel estimation;
- clearly labeled exploratory analysis without confirmatory claims.

Do not report only favorable analyses. Threshold crossing is not a quality score or probability that a candidate is true.

## Missing data and deviations

Define:

- missingness by variable, time, and group;
- reasons and data-collection process;
- primary handling;
- assumptions;
- sensitivity analyses;
- protocol and analysis deviations.

Complete-case analysis is not automatically unbiased. Record deviations without overwriting the original plan.

## Replication and open materials

Following the National Academies terminology:

- reproducibility uses the same data/code/conditions;
- replicability collects new data to address the same question.

Plan:

- code, environment, seeds, and workflow capture;
- provenance and versioning;
- shareable materials and justified restrictions;
- independent replication;
- boundary-condition and transport tests;
- reporting of null and contrary results.

Open sharing remains subject to consent, privacy, community governance, intellectual property, biosecurity, and other controls.

## Intervention reporting

For randomized intervention work:

- use SPIRIT 2025 and its explanation/elaboration for protocol completeness;
- use CONSORT 2025 and its explanation/elaboration for result reporting;
- include trial registration, protocol and statistical-analysis-plan access, outcomes, harms, intervention/comparator details, analysis populations, missing data, and important changes;
- use applicable extensions.

These are reporting guidelines. They do not replace ethics review, trial registration rules, statistical expertise, or regulatory requirements.