--- name: evidence-study-appraisal description: "Appraise a study's design, bias, measures, analysis and warranted conclusions using a design-appropriate risk-of-bias approach." --- ## Selection and handoffs Judge trust in a study's methods and conclusions. **Select for:** Appraise a study's design, bias, measures, analysis and warranted conclusions using a design-appropriate risk-of-bias approach. **Expected work:** Ten-item checklist; RoB2/ROBINS-I/CASP/MMAT/AMSTAR2; Strengths/weaknesses; Confidence; Rated table; Can/cannot infer **Expected output:** Output format **Verdict:** high, moderate or low confidence, in one sentence. **Strengths** and **weaknesses**, then a table of checklist items with a rating and a note, and **what this study can and cannot tell you.** **Scope boundaries:** Do not activate solely because the family was used earlier or for unrelated chat.; Use the adjacent specialist when its distinct task is requested: scholar-peer-review-critique. Keep: formal appraisal tools and confidence ratings differ from editor-facing major/minor concerns, author questions and publication recommendation. **Related skills:** [scholar-peer-review-critique](../scholar-peer-review-critique/SKILL.md) Apply this skill only to the selected task. Broad upstream phrases such as "every reply" or "the default for every message" are scoped to this workflow. They do not activate it for unrelated chat. User instructions and the [integration contract](../../docs/INTEGRATION.md) govern combined work. Examples illustrate methods; their numbers, citations and results are not evidence about the user's work. ## Preserved source instructions ## Unified plugin integration Read [the integration contract](../../docs/INTEGRATION.md) when combining workflows. This active copy retains the complete source instructions below with namespaced skill references. Original bytes and upstream metadata remain in `../../resources/`. Resolve tools by their actual host capabilities; tool names in upstream instructions are capability requirements, not proof of connection. Use the task workspace for generated outputs. Treat historical case-study results as supplied examples, not newly verified findings. Family skill aliases (read the linked SKILL.md before use): - `citation-chaining` → `evidence-citation-chaining` (`../evidence-citation-chaining/SKILL.md`) - `citations-and-references` → `evidence-citations-and-references` (`../evidence-citations-and-references/SKILL.md`) - `claim-checker` → `evidence-claim-checker` (`../evidence-claim-checker/SKILL.md`) - `compare-studies` → `evidence-compare-studies` (`../evidence-compare-studies/SKILL.md`) - `data-extraction` → `evidence-data-extraction` (`../evidence-data-extraction/SKILL.md`) - `database-guide` → `evidence-database-guide` (`../evidence-database-guide/SKILL.md`) - `debate-and-argument-evidence` → `evidence-debate-and-argument-evidence` (`../evidence-debate-and-argument-evidence/SKILL.md`) - `decision-evidence` → `evidence-decision-evidence` (`../evidence-decision-evidence/SKILL.md`) - `evidence-answer` → `evidence-evidence-answer` (`../evidence-evidence-answer/SKILL.md`) - `evidence-based-writing` → `evidence-evidence-based-writing` (`../evidence-evidence-based-writing/SKILL.md`) - `evidence-help` → `evidence-evidence-help` (`../evidence-evidence-help/SKILL.md`) - `evidence-meter` → `evidence-evidence-meter` (`../evidence-evidence-meter/SKILL.md`) - `field-guides` → `evidence-field-guides` (`../evidence-field-guides/SKILL.md`) - `find-studies` → `evidence-find-studies` (`../evidence-find-studies/SKILL.md`) - `literature-review` → `evidence-literature-review` (`../evidence-literature-review/SKILL.md`) - `meta-analysis` → `evidence-meta-analysis` (`../evidence-meta-analysis/SKILL.md`) - `open-access-full-text` → `evidence-open-access-full-text` (`../evidence-open-access-full-text/SKILL.md`) - `paper-summary` → `evidence-paper-summary` (`../evidence-paper-summary/SKILL.md`) - `plain-language-research` → `evidence-plain-language-research` (`../evidence-plain-language-research/SKILL.md`) - `research-evidence-finder` → `evidence-research-evidence-finder` (`../evidence-research-evidence-finder/SKILL.md`) - `research-gaps` → `evidence-research-gaps` (`../evidence-research-gaps/SKILL.md`) - `research-proposal-background` → `evidence-research-proposal-background` (`../evidence-research-proposal-background/SKILL.md`) - `research-question` → `evidence-research-question` (`../evidence-research-question/SKILL.md`) - `search-strings` → `evidence-search-strings` (`../evidence-search-strings/SKILL.md`) - `source-verification` → `evidence-source-verification` (`../evidence-source-verification/SKILL.md`) - `statistics-explained` → `evidence-statistics-explained` (`../evidence-statistics-explained/SKILL.md`) - `study-appraisal` → `evidence-study-appraisal` (`../evidence-study-appraisal/SKILL.md`) - `study-designs-and-evidence-levels` → `evidence-study-designs-and-evidence-levels` (`../evidence-study-designs-and-evidence-levels/SKILL.md`) - `systematic-review` → `evidence-systematic-review` (`../evidence-systematic-review/SKILL.md`) --- # Study Appraisal ## Appraisal checklist 1. **Question:** is it clearly stated? 2. **Design:** does it fit the question? (An experiment tests cause; a survey shows association.) 3. **Sample:** size, how people were selected, response rate, who is missing. 4. **Comparison:** is there a control group? Were groups similar at the start? Was assignment random? 5. **Bias:** selection, measurement, attrition, reporting and publication bias. 6. **Confounding:** other factors that could explain the result, and whether they were controlled. 7. **Measures:** valid and reliable? Self-report or objective? 8. **Statistics:** effect sizes, confidence intervals, multiple comparisons, pre-registration. 9. **Results vs claims:** do the conclusions go beyond the data? 10. **Context:** funding, conflicts of interest, peer review, journal reputation, replication. ## Useful tools by design - Randomized trials: the RoB 2 risk-of-bias tool. - Non-randomized studies: ROBINS-I. - Qualitative research: the CASP qualitative checklist. - Mixed methods: MMAT. - Systematic reviews: AMSTAR 2. ## Output format **Verdict:** high, moderate or low confidence, in one sentence. **Strengths** and **weaknesses**, then a table of checklist items with a rating and a note, and **what this study can and cannot tell you.**