# Acceptance Test Plan — Vendor Privacy Policy First Pass v1.0.0

## Skill summary

Triage assessment of a vendor's published privacy policy. Produces a structured summary of data handling practices and surfaces red flags relevant to in-house counsel evaluating whether to engage the vendor or escalate to deeper privacy review (DPA negotiation, full data-protection assessment).

## Test corpus requirements

Source 5–8 anonymized vendor privacy policies covering:

- **At least 1 high-quality, market-standard privacy policy** (e.g., a SaaS platform with a comprehensive, GDPR-aware policy) — to confirm baseline calibration.
- **At least 1 ambiguous-quality policy** (sparse provisions; vague data-handling language; missing key sections) — to surface red flags.
- **At least 1 policy with notable data-handling features** (AI-training rights, broad data-sharing, third-party advertiser ecosystem, cross-border data transfers without specified mechanism).
- **At least 1 policy with consumer-app focus** (vs. enterprise focus) — different concerns surface.
- **At least 1 policy with regulated-industry context** (financial services, healthcare, education) — additional regulatory lenses apply.

Documents can be sourced from the vendor's public website (privacy policies are typically published) and "anonymized" simply by replacing the vendor's name with a placeholder; the substantive content is what is being tested.

## Test scenarios

### Scenario 1: High-quality, market-standard privacy policy (baseline)

**Inputs:** A comprehensive SaaS-vendor privacy policy.

**Expected output structure:**
- Markdown report with sections: "Bottom line" (summary recommendation: engage / escalate / decline), "Summary of practices" (factual summary of what the policy says), "Red flags / concerns" (issues warranting attention), "Open questions for the vendor" (gaps in the policy that require follow-up), "What this skill does not do".

**Expected calibration:**
- 0 critical red flags (this is a market-standard policy).
- 0–2 material red flags (often: international transfer mechanisms not explicit; sub-processor list not published).
- 2–6 minor observations (typically: cookie consent mechanism, retention period clarity, data-subject-rights mechanism specifics).
- "Bottom line" leads with "engage with standard contractual safeguards" or similar.

**Edge cases to verify:**
- Skill summarizes practices factually without inventing claims the policy does not make.
- Skill differentiates clear practices from ambiguous-language practices.

**Pass criteria:**
- Structural pass: All required sections present.
- Calibration pass: Reviewing attorney confirms baseline calibration is correct (not over-flagging market-standard).

### Scenario 2: Ambiguous / sparse privacy policy

**Inputs:** A privacy policy with vague language, missing sections, or unclear data-handling commitments.

**Expected output structure:** Same structure with greater emphasis on "Open questions for the vendor."

**Expected calibration:**
- 1–3 critical red flags if substantively missing key elements (no data-handling commitment, no breach notification, no data-subject rights).
- 2–5 material red flags on ambiguous-but-present provisions.
- 3–8 minor observations.
- "Open questions for the vendor" is substantive — listing the specific questions that need answering before engagement.

**Edge cases to verify:**
- Skill flags absences (missing sections) as findings, not as silent omissions.
- Skill differentiates "policy says nothing about X" from "policy says X without specificity."

**Pass criteria:** As above with ambiguity-aware verification.

### Scenario 3: Policy with notable data-handling features

**Inputs:** A privacy policy with notable features: AI-training-on-customer-data, broad data-sharing with affiliates / advertisers, cross-border transfers without specified mechanism.

**Expected output structure:** Same structure with explicit findings on the notable features.

**Expected calibration:**
- AI-training rights without clear opt-out: critical red flag.
- Broad data-sharing with advertisers if user data is implicated: critical or material depending on scope.
- Cross-border transfers without specified mechanism: material red flag (specific to cross-border-data-flow contexts).
- "Bottom line" reflects the notable-features posture.

**Edge cases to verify:**
- Skill identifies AI-training-on-customer-data provisions specifically and addresses them at appropriate severity.
- Skill addresses ad-tech / data-broker ecosystem if implicated.

**Pass criteria:** As above with notable-features verification.

### Scenario 4: Consumer-app privacy policy

**Inputs:** A consumer-facing app's privacy policy (B2C rather than B2B).

**Expected output structure:** Same structure but with B2C-specific concerns: COPPA if minors are implicated, geolocation, advertising, in-app behavior tracking.

**Expected calibration:**
- Findings address consumer-app-specific concerns appropriately.
- "Bottom line" notes whether the in-house counsel context (B2B engagement evaluating a vendor) maps to the policy's posture.

**Edge cases to verify:**
- Skill identifies the B2C focus and notes how it differs from typical B2B-vendor evaluation.
- Skill addresses COPPA / minor-targeted concerns if present.

**Pass criteria:** As above with B2C-context verification.

### Scenario 5: Regulated-industry vendor policy

**Inputs:** A privacy policy for a vendor operating in a regulated industry (financial services, healthcare, education).

**Expected output structure:** Same structure with attention to regulatory-specific concerns: GLBA / financial-privacy provisions; HIPAA / healthcare data handling; FERPA / education records.

**Expected calibration:**
- Findings address the regulatory context specifically.
- "Bottom line" recommends regulatory-specific deeper review (e.g., DPA Checklist Review with HIPAA BAA regime if healthcare data is in scope).

**Edge cases to verify:**
- Skill identifies the regulatory context.
- Skill recommends appropriate cross-skill follow-up (DPA Checklist Review for the regulatory layer).

**Pass criteria:** As above with regulatory-context verification.

## Refusal scenarios

### Refusal 1: Document is not a privacy policy

**Input:** Terms of service, EULA, or other policy type misidentified as a privacy policy.

**Expected behavior:**
- Skill identifies that the document is not (primarily) a privacy policy.
- Skill suggests appropriate next steps (read the actual privacy policy; or note that the document type is not within scope).

**Pass criteria:** Explicit refusal.

### Refusal 2: Document is in non-English

**Input:** A non-English privacy policy.

**Expected behavior:**
- Skill identifies the language and notes the skill is calibrated to English-language US/EU/UK contexts.
- Skill seeks confirmation before proceeding.

**Pass criteria:** Explicit identification and confirmation request.

## Cross-cutting verification

- **Triage scope discipline.** This skill is a *first-pass triage*, not a comprehensive privacy assessment. Findings should not overclaim on the depth of analysis — the skill explicitly recommends DPA Checklist Review or full privacy assessment for deeper analysis where appropriate.
- **No invented privacy practices.** The skill summarizes what the policy *says*, not what the vendor *does* in practice (the policy may not reflect actual practice; this is acknowledged in the "What this skill does not do" enumeration).
- **No regulatory-compliance opinions.** The skill flags regulatory concerns but does not assert "this policy is GDPR-compliant" or "this policy violates CCPA."
- **Open questions are substantive.** The "Open questions for the vendor" section produces real questions, not generic placeholders.
- "What this skill does not do" enumeration present.
- Citations to specific sections of the policy resolve.

## Pass / fail decision

Vendor Privacy Policy First Pass v1.0.0 passes acceptance testing when:

1. All 5 test scenarios pass structural checks.
2. All 5 test scenarios pass calibration evaluation by a reviewing attorney with privacy / data-protection experience.
3. Both refusal scenarios trigger documented refusal behavior.
4. Cross-cutting verification passes on every scenario.

## Reviewer notes

The reviewing attorney for Vendor Privacy Policy First Pass should have practical privacy / data-protection experience covering both consumer and enterprise contexts. Specific competencies:

- Identifying AI-training-on-customer-data provisions and calibrating their materiality.
- Evaluating cross-border transfer mechanisms in the post-Schrems II environment.
- Distinguishing market-standard from sub-standard privacy policy practices.
- Recognizing when a triage flag warrants escalation to full DPA negotiation vs. when standard contractual safeguards are sufficient.

Calibration assessment is documented in `test-results/vendor-privacy-policy-first-pass-v1.0.0/calibration-assessment.md`.