--- name: evidence-study-designs-and-evidence-levels description: "Explain what study designs can support and how certainty is judged for the research question, including design limitations and GRADE-style factors." --- ## Selection and handoffs Explain evidential power and certainty of study designs. **Select for:** Explain what study designs can support and how certainty is judged for the research question, including design limitations and GRADE-style factors. **Expected work:** Effects-specific pyramid; Design strengths/caveats; Qualitative exception; Within-level quality; GRADE-style downgrade/upgrade; Four certainty levels **Expected output:** Requested artifact described by complete workflow sections above. **Scope boundaries:** Do not activate solely because the family was used earlier or for unrelated chat.; Use the adjacent specialist when its distinct task is requested: evidence-study-appraisal. Keep: a framework explanation differs from applying a full checklist and bias tool to an individual paper.; Statistical thresholds, model defaults and test matrices are contextual heuristics, not universal rules or executed analyses. **Related skills:** [evidence-study-appraisal](../evidence-study-appraisal/SKILL.md) Apply this skill only to the selected task. Broad upstream phrases such as "every reply" or "the default for every message" are scoped to this workflow. They do not activate it for unrelated chat. User instructions and the [integration contract](../../docs/INTEGRATION.md) govern combined work. Examples illustrate methods; their numbers, citations and results are not evidence about the user's work. ## Preserved source instructions ## Unified plugin integration Read [the integration contract](../../docs/INTEGRATION.md) when combining workflows. This active copy retains the complete source instructions below with namespaced skill references. Original bytes and upstream metadata remain in `../../resources/`. Resolve tools by their actual host capabilities; tool names in upstream instructions are capability requirements, not proof of connection. Use the task workspace for generated outputs. Treat historical case-study results as supplied examples, not newly verified findings. Family skill aliases (read the linked SKILL.md before use): - `citation-chaining` → `evidence-citation-chaining` (`../evidence-citation-chaining/SKILL.md`) - `citations-and-references` → `evidence-citations-and-references` (`../evidence-citations-and-references/SKILL.md`) - `claim-checker` → `evidence-claim-checker` (`../evidence-claim-checker/SKILL.md`) - `compare-studies` → `evidence-compare-studies` (`../evidence-compare-studies/SKILL.md`) - `data-extraction` → `evidence-data-extraction` (`../evidence-data-extraction/SKILL.md`) - `database-guide` → `evidence-database-guide` (`../evidence-database-guide/SKILL.md`) - `debate-and-argument-evidence` → `evidence-debate-and-argument-evidence` (`../evidence-debate-and-argument-evidence/SKILL.md`) - `decision-evidence` → `evidence-decision-evidence` (`../evidence-decision-evidence/SKILL.md`) - `evidence-answer` → `evidence-evidence-answer` (`../evidence-evidence-answer/SKILL.md`) - `evidence-based-writing` → `evidence-evidence-based-writing` (`../evidence-evidence-based-writing/SKILL.md`) - `evidence-help` → `evidence-evidence-help` (`../evidence-evidence-help/SKILL.md`) - `evidence-meter` → `evidence-evidence-meter` (`../evidence-evidence-meter/SKILL.md`) - `field-guides` → `evidence-field-guides` (`../evidence-field-guides/SKILL.md`) - `find-studies` → `evidence-find-studies` (`../evidence-find-studies/SKILL.md`) - `literature-review` → `evidence-literature-review` (`../evidence-literature-review/SKILL.md`) - `meta-analysis` → `evidence-meta-analysis` (`../evidence-meta-analysis/SKILL.md`) - `open-access-full-text` → `evidence-open-access-full-text` (`../evidence-open-access-full-text/SKILL.md`) - `paper-summary` → `evidence-paper-summary` (`../evidence-paper-summary/SKILL.md`) - `plain-language-research` → `evidence-plain-language-research` (`../evidence-plain-language-research/SKILL.md`) - `research-evidence-finder` → `evidence-research-evidence-finder` (`../evidence-research-evidence-finder/SKILL.md`) - `research-gaps` → `evidence-research-gaps` (`../evidence-research-gaps/SKILL.md`) - `research-proposal-background` → `evidence-research-proposal-background` (`../evidence-research-proposal-background/SKILL.md`) - `research-question` → `evidence-research-question` (`../evidence-research-question/SKILL.md`) - `search-strings` → `evidence-search-strings` (`../evidence-search-strings/SKILL.md`) - `source-verification` → `evidence-source-verification` (`../evidence-source-verification/SKILL.md`) - `statistics-explained` → `evidence-statistics-explained` (`../evidence-statistics-explained/SKILL.md`) - `study-appraisal` → `evidence-study-appraisal` (`../evidence-study-appraisal/SKILL.md`) - `study-designs-and-evidence-levels` → `evidence-study-designs-and-evidence-levels` (`../evidence-study-designs-and-evidence-levels/SKILL.md`) - `systematic-review` → `evidence-systematic-review` (`../evidence-systematic-review/SKILL.md`) --- # Study Designs and Evidence Levels ## The evidence pyramid (for questions about effects) 1. Systematic reviews and meta-analyses of good studies 2. Randomized controlled trials (RCTs) 3. Quasi-experiments and natural experiments 4. Cohort (longitudinal) studies 5. Case-control studies 6. Cross-sectional surveys 7. Case studies and case series 8. Expert opinion, editorials, anecdotes ## What each design can show | Design | Shows | Watch out for | |---|---|---| | Meta-analysis | Pooled effect across studies | Garbage in, garbage out; publication bias | | RCT | Cause and effect | Small samples, short follow-up, artificial settings | | Quasi-experiment | Likely cause | Group differences | | Cohort | Associations over time | Confounding | | Cross-sectional | Snapshot associations | Cannot show which came first | | Qualitative | Experiences, meanings, why | Not designed for generalizing numbers | ## Beyond the pyramid - The best design depends on the question. For "how do people experience X?", qualitative research is the right evidence. - Quality matters within each level: a well-run cohort study can beat a flawed RCT. ## Certainty ratings (GRADE-style) Start high for randomized evidence and low for observational. Rate down for risk of bias, inconsistency, indirectness, imprecision and publication bias; rate up for large effects or dose-response. Final certainty: high, moderate, low or very low.