# Contributing

Thanks for your interest in contributing. We expect to accept external contributions, but the bar for merging is intentionally high.

This repository contains shared building blocks for coding agents:

- Skills: reusable, task focused instruction packs
- Agents: GitHub Copilot custom-agent configurations that bundle tool expectations and skill selection

Because these artifacts can affect many users and workflows, we prioritize correctness, clarity, and long term maintainability over speed.

## Code ownership

Every plugin, skill, and agent must have designated owners in the `.github/CODEOWNERS` file. When you add a new skill or agent, add a matching CODEOWNERS entry. Ownership must be either:

- **Two or more FTE GitHub aliases** (e.g., `@user1 @user2`), or
- **A GitHub team alias** (e.g., `@dotnet/my-team`)

This ensures that every contribution area has accountable reviewers and that PRs are automatically routed to the right people.

## Repository layout

```text
plugins/
  <plugin>/
    plugin.json
    .claude-plugin/
      plugin.json
    .codex-plugin/
      plugin.json
    skills/
      <skill-name>/
        SKILL.md
        scripts/
        references/
        assets/
    agents/
      <agent-name>.agent.md
tests/
  <plugin>/
    <skill-name>/
      eval.yaml
      <fixture files>
```

Every plugin must have a plugin.json file in the plugin root that is linked to from the marketplace.json file.

### Plugin organization

Skills are grouped into domain-specific plugins. When proposing a new skill, place it in the plugin that best matches its domain. See [README.md](README.md) for the current list of plugins.
If your skill does not fit any existing plugin, consider creating a new one.

To create a new plugin:

1. Add `plugins/<plugin-name>/plugin.json`, a Claude compatibility manifest at
   `plugins/<plugin-name>/.claude-plugin/plugin.json`, a Codex-specific compatibility manifest at
   `plugins/<plugin-name>/.codex-plugin/plugin.json`, and a `skills/` directory beneath them.
   Start the Claude manifest as a copy of the root manifest, then preserve any host-specific field
   values that Claude requires. The Codex manifest must contain only components its runtime loads;
   do not copy `agents` or `lspServers` into it.
2. Add a matching entry in `.github/plugin/marketplace.json`, `.claude-plugin/marketplace.json`, `.cursor-plugin/marketplace.json`, and `.agents/plugins/marketplace.json`. Keep names and `plugins[].source` values consistent across all marketplace manifests. Descriptions may be host-specific when a capability is unavailable there; for example, the Codex marketplace must not advertise `.agent.md` agents or LSP integration.
   Also add a `plugins/<plugin-name>/version.json` (copy an existing one) so the plugin participates in automated versioning. Start its `plugin.json` version at `0.1.0`.
3. Add a CODEOWNERS entry for the new plugin and its tests (see [Code ownership](#code-ownership)).
4. Add the plugin to the **What's Included** table in the root `README.md`.
5. Create a `tests/<plugin-name>/` directory for skill tests.

See existing plugins for the expected format.

This legacy multi-host layout and its Codex parity validation are a compatibility bridge, not the
intended permanent Agent Plugins package format. A future Agent Plugins 1.0 migration would use a
schema-qualified root `plugin.json`, fixed `skills/` and `mcp.json` locations, and namespaced client
extensions instead of top-level host fields. That migration should remove the duplicated
host-specific manifests and most custom parity and shape validation while retaining a smaller Codex
integration smoke test for installation and discovery.

### The `dotnet-experimental` plugin

Use `dotnet-experimental` when you want to try out a skill idea but are not yet confident it belongs in a stable plugin — for example, when the skill is outside your usual area of responsibility, the approach is unproven, or you want community feedback before committing to a long-term home.

Skills in `dotnet-experimental`:

- May change, be reworked, or be removed without notice.
- Are held to the same quality and testing standards as any other skill (frontmatter, `eval.yaml`, etc.).
- Should eventually graduate to a stable plugin or be retired. When a skill has proven itself, move it to the appropriate domain plugin and update tests accordingly.

Place experimental skills under `plugins/dotnet-experimental/skills/` with matching tests in `tests/dotnet-experimental/`.

## Plugin versioning

Each plugin is versioned independently. Every plugin carries the manifests its consumers read:
`plugins/<plugin>/plugin.json`, `plugins/<plugin>/.codex-plugin/plugin.json`, and
`plugins/<plugin>/.claude-plugin/plugin.json`. The Claude and Codex manifests share the stamped
version with the root manifest, but both can preserve host-specific fields or omit unsupported
fields. Consumers (Copilot CLI, Claude, Codex, Cursor) read the version directly from this
repository.

Each `plugins/<plugin>/version.json` declares the plugin's major/minor release base and the files
that count as effective plugin content. A calculated manifest version transition is a release
checkpoint; an arbitrary patch edit is not. When effective content on `main` differs from the
latest valid checkpoint, the **patch** advances once.
The generated manifests
(`plugin.json`, `.codex-plugin/plugin.json`, and `.claude-plugin/plugin.json`) and
`version.json` itself are excluded from content comparison via the `pathFilters`, so editing only
manifest metadata (anything other than a deliberate base bump in `version.json`) does **not** change
the patch number and is **not** picked up by `/version-bump` or the weekly sync. Touch a skill or
other plugin content to bump the version.

What this means when you contribute:

- **Every plugin carries its own `plugins/<plugin>/version.json`.** It declares the plugin's version
  base; the weekly sync fails fast if a plugin ships a `plugin.json` without one, so none is ever
  left unversioned.
- **Don't hand-edit the `version` field** in any of the manifests (`plugin.json`,
  `.codex-plugin/plugin.json`, or `.claude-plugin/plugin.json`). The patch number is computed and
  stamped automatically, and a manual edit will be overwritten.
- **The only version field you may change is the base** (`"version"`) in `plugins/<plugin>/version.json`,
  and only to declare a deliberate **minor or major** release of that plugin (e.g. `0.1` → `0.2` or `1.0`).
  Changing the base resets the patch number to `0`.
- After a PR changes a plugin's content, stamping its calculated version on the branch is optional:
  - A maintainer can comment **`/version-bump`** on a same-repo PR to stamp the new version onto the branch.
  - Otherwise the **weekly version sync** opens a PR that stamps any plugin whose content changed since its
    release checkpoint, explaining each change. Nothing is ever missed.

Two PRs stamped concurrently can predict the same patch number. The weekly sync compares effective
content with the latest first-parent release checkpoint and reconciles the second change. Branch-local
merge history cannot inflate a version when it makes no content change on `main`. Version-only changes
do not trigger skill evaluations.

## Bundled MCP servers

Under the repository's current legacy layout, a plugin that bundles an MCP server must declare it
in **every** manifest it ships. Each host reads a different one — Copilot reads `plugin.json`, Codex
reads `.codex-plugin/plugin.json`, and Claude reads `.claude-plugin/plugin.json` — so a server
declared in only one of them is silently unavailable in the others. Keep the same server names and
transport settings, but preserve host-specific schema differences instead of copying optional
fields blindly.

Declare the servers inline as an object. The alternative form, a relative path to a companion
`.mcp.json`, is resolved by hosts against the **plugin root** and not against the directory holding the
manifest, so `"mcpServers": "./.mcp.json"` inside `.codex-plugin/plugin.json` points at
`plugins/<plugin>/.mcp.json`, never `plugins/<plugin>/.codex-plugin/.mcp.json`.

`skill-validator check` enforces both rules: it fails when a referenced `.mcp.json` does not resolve
from the plugin root, when the manifests do not declare the same set of servers, or when the Codex
manifest uses a server shape that Codex rejects. In particular, Codex interprets `tools` as a map of
per-tool settings, not an allow-list array. Omit `tools` to enable all tools.

The `codex-plugin-smoke` workflow pins a supported Codex CLI release and, without API credentials,
verifies marketplace installation, skill discovery, MCP discovery, and direct MCP transport through
a real `binlog_overview` call against a generated binary log. It does not cover authentication,
model inference, prompt-based skill selection, or model-driven tool selection.

## Host-specific agents

[Agent Plugins 1.0](https://agent-plugins.org/specification) standardizes skills and MCP
servers, not custom agents or LSP servers. Files under `plugins/<plugin>/agents/*.agent.md` use
[GitHub Copilot custom-agent](https://docs.github.com/copilot/reference/custom-agents-configuration)
frontmatter such as `tools`, `agents`, and `handoffs`.

Do not list `.agent.md` files in `.codex-plugin/plugin.json`: Codex ignores that field and does not
install those files as native agents. Codex discovers native agents from `.codex/agents/*.toml` in a
trusted project or `~/.codex/agents/*.toml` in a user profile. Plugin installation does not populate
those locations, so do not generate or claim native Codex agents until the repository defines and
tests a documented installation mechanism. Copilot handoffs also have no static Codex equivalent;
Codex delegation is model-driven. OpenAI Agents SDK handoffs are an application-level orchestration
API between SDK-defined agents, not a plugin packaging or discovery mechanism.

## Before you start

- Search existing issues and pull requests to avoid duplicates.
- Start with an issue before you submit a pull request for a new skill, a new agent, or any non trivial change. This helps us align on scope and avoids wasted work.
- Small fixes like typos, broken links, or clearly isolated corrections can go straight to a pull request.
- Keep changes small and focused. One skill or one agent per pull request is a good default.

## What we look for

We are most likely to accept contributions that are:

- Addresses a LLM gap and is clearly motivated by a real use case
- Likely to be used frequently and is general (not repo-specific)
- Narrow in scope and easy to review
- Tool conscious and explicit about assumptions
- Verifiable with concrete validation steps
- Written to be durable across repo changes

We are less likely to accept contributions that:

- Add broad frameworks, meta tooling, or large reorganizations
- Duplicate guidance that already exists in another skill
- Encode private environment details, credentials, or company specific secrets
- Depend on proprietary tools or access that most contributors will not have
- Skills that make use of third party tools will be evaluated on a case by case basis. Acceptance of such skills will depend on our evaluation of the provenance and maturity of any such tools.

## Proposing a new skill

Please review the **What we look for** section and add justification for the skill in your issue and PR.

A skill should be self-contained and:

- Clearly state **what it does** and **when to use it**.
- Frontmatter (name and description) is small and minimal, just enough for LLM to understand when to use it
- Keep the SKILL.md body under 500 lines for optimal performance. Split content into separate files when you approach this limit. Use a progressive disclosure pattern, referring to those files from the SKILL.md file where needed.
- Specify required inputs (repo context, environment, access needs).
- Prefer concrete checklists and verification steps over vague guidance.

Create a new folder under a plugin's `skills/` directory:

```text
plugins/<plugin>/skills/<skill-name>/SKILL.md
```

A skill should answer three questions up front:

1. What outcome does the skill produce
2. When should an agent use it
3. How does the agent validate success

### Skill naming

Use short, kebab-case names that mirror how developers naturally phrase the task, prioritizing keyword overlap over grammar — e.g., add-aspnet-auth, configure-jwt-auth, setup-identity-server. Optionally using gerund style (verb-ing) is acceptable as well - e.g., configuring-caching.

Optimize for intent matching: lead with the action verb users actually say (add, configure, setup, deploy) followed the outcome the skill is aiming to assist.

The `SKILL.md` is required to have front-matter at a minimum:

Create the file with required YAML frontmatter:

```yaml
---
name: <skill-name>
description: <description of what the skill does, when to use it, and when not to use it>
---
```

> **Tip:** The `description` field is used by the agent runtime to decide whether to load the full skill.
> Include **when to use** and **when not to use** guidance directly in the description so the agent can
> select or skip skills without reading the entire `SKILL.md`. This avoids unnecessary token usage.
> See [`thread-abort-migration/SKILL.md`](plugins/dotnet-upgrade/skills/thread-abort-migration/SKILL.md) for a good example.

### Recommended `SKILL.md` sections

- **Purpose**: one paragraph describing the outcome.
- **When to use** / **When not to use** (put the essentials in the frontmatter `description`; expand here only if more detail is needed).
- **Inputs**: what the agent needs (files, commands, permissions).
- **Workflow**: numbered steps with checkpoints.
- **Validation**: how to confirm the result (tests, linters, manual checks).
- **Common pitfalls**: known traps and how to avoid them.

### Skill checklist

Include a `SKILL.md` that covers:

- Purpose and non goals
- When to use and when not to use (summarized in the frontmatter `description`; body section for extended detail)
- Inputs and prerequisites
- Step by step workflow with checkpoints
- Validation steps that can be run or observed
- Failure modes and recovery guidance

Also:

- Avoid duplicating text across multiple skills. Prefer referencing shared patterns.
- Do not include content copied from other repositories. If you are inspired by existing work, rewrite in your own words and adapt it to our conventions.

## Proposing a new GitHub Copilot agent

An agent definition should be opinionated but bounded:

- Describe the **role** (e.g., "WinForms Expert", "Security Reviewer", "Docs Maintainer").
- Define boundaries (what the agent should not do).
- List the skills it expects to use and how it chooses among them.

Add a GitHub Copilot custom-agent file under a plugin's `agents/` directory:

```text
plugins/<plugin>/agents/<agent-name>.agent.md
```

### Agent checklist

Include documentation that explains:

- Role and intended tasks
- Boundaries and safety constraints
- Tooling assumptions
- How the agent chooses which skills to apply
- What a good completion looks like, including validation expectations

## Testing and validation

Skills and agents are documentation driven, but we still treat them as production assets.

- Every change should include a validation section that a reviewer can follow.
- If your change references commands, keep them cross platform when practical. If not, state the supported environment.
- If your change depends on external services, document how a reviewer can validate without privileged access, or explain why validation is not possible.

### Writing skill tests

Each skill should have an `eval.yaml` file that defines test scenarios. Tests live under the repo root `tests/` directory, matching the plugin and skill name:

```text
tests/<plugin>/<skill-name>/eval.yaml
```

The exception is a helper or reference skill that sets `disable-model-invocation: true`. The model
cannot self-activate it, so an activation-graded eval would compare two identical arms. Cover those
through the evals of the skills that load them and through the plugin arm instead.

#### Evaluation design quality bar

An eval must prove useful behavior, not only produce a valid YAML file. Authors and reviewers should
apply this order:

1. **Prove necessity and skill fit.** A preference scenario is necessary when the target should
   improve the answer or action. A dormancy contract or no-op guard is necessary when it protects a
   meaningful routing or preservation invariant, even if correct behavior is baseline-equivalent.
   Do not eval generic model knowledge, path recall, or a `disable-model-invocation: true` reference
   in isolation.
2. **Tag every capability stimulus; add distinct value to voting cases.** Every stimulus in a
   `type: capability` eval must have stable lowercase kebab-case `capability`, `risk`, and `journey`
   tags, including dormancy cases. Each preference-eligible stimulus must cover a different value;
   renamed or reworded duplicates do not add evidence.
3. **Use a natural, non-cued request.** Write the prompt as a developer would ask it. Do not name
   the target, quote its vocabulary, reveal the fix, or tell the agent which workflow to follow.
4. **Make the outcome deterministic.** State what must be true in the response and workspace.
   Include the complete in-scope file set, and use executable, file, or output graders for facts
   that do not need an LLM judge.
5. **Prove golden acceptance.** The golden trajectory and patch, when present, must pass every
   deterministic grader.
6. **Prove mutation rejection.** A realistic broken mutation must fail the grader that is meant to
   detect it; otherwise the grader does not protect the behavior.
7. **Cover restraint.** Migration and rewrite evals need an already-correct no-op case. Routing
   boundaries need `tags.expect_activation: "false"` dormancy cases that test recognition,
   restraint, and redirection without a disallowed `skill-invocation` grader.
8. **Size for the expected tie rate.** Five preference-eligible stimuli is only the eligibility
   floor. Add independent stimuli until the expected win, tie, and loss record can reach the sign
   test with useful power.
9. **Use the production path.** Run skill evals through Vally and custom-agent evals through the
   native SDK lane. Use the normal worker concurrency and declared suite time budget. A serial-only
   pass or an unbounded timeout hides reliability defects.
10. **Check model-family sensitivity.** For broad routing or behavior changes, collect evidence from
   more than one executor family. Keep each family result separate; model combinations are
   sensitivity evidence, not extra stimulus votes.
11. **Classify failures before editing.** Separate fixture, spec-load, reliability, power, eval
    design, content, routing, and cost failures. Do not rewrite skill content until the evidence
    identifies a content defect.

The deterministic gate catches structural corruption. It cannot decide whether a scenario is
necessary, whether a prompt sounds natural, or whether the tested behavior has product value. Those
remain required author and reviewer checks.

The skeleton below shows the shape only — it declares a single trial and would therefore be rejected
by the quality gate. See [Size the eval so it can return a verdict](#size-the-eval-so-it-can-return-a-verdict) for the real bar.

```yaml
name: my-skill
description: Evaluates the <plugin>/<skill-name> skill
type: capability
defaults:
  timeout: 3m
  runs: 1
stimuli:
  - name: "Describe what the agent should do"
    prompt: |
      The prompt sent to the agent.
    tags:
      capability: distinct-capability
      risk: failure-being-prevented
      journey: customer-task
    graders:
      # Deterministic graders check the produced output/artifacts.
      - type: exit-success
      - type: output-contains
        config:
          substring: "expected text in agent output"
      # The `prompt` grader runs the LLM judge against the rubric below.
      - type: prompt
    rubric:
      - The agent correctly identified the issue
      - The agent suggested a concrete fix
```

> [!IMPORTANT]
> `config:` is no longer supported. Use one `defaults:` block and preserve its settings.

Each skill is evaluated in up to three variants — **baseline** (no skills), **skilled** (only the skill under test), and **plugin** (the whole plugin loaded) — and a skill "passes" only when the skilled run is a *credible* improvement over baseline. To assert that a skill should stay dormant for an out-of-scope task, add the quoted string tag `expect_activation: "false"` under that stimulus's `tags:` mapping. Dormancy is an isolated-skill activation contract: unexpected activation blocks a pass, while the stimulus's retained comparison does not vote in preference. See any existing `tests/*/*/eval.yaml` for a fuller example of the grader and stimulus format.

Custom-agent evals live at `tests/<plugin>/agent.<name>/eval.yaml` and use the
same baseline / isolated / plugin roles and verdict policy. Vally does not
register custom agents, so CI executes those specs through the native Copilot
SDK runner: isolated runs register the target agent plus declared dependencies,
and plugin runs register the complete production plugin skill/agent surface.
Agent results carry `skillKind: agent` and retain target activation, nested
delegation, invoked skills, tools, completion, token, and wall-time evidence.

#### Size the eval so it can return a verdict

The pass gate gives each distinct stimulus one vote. Repeated runs collapse to one
majority-direction vote and remain available as reliability evidence.

1. **Preference-eligible distinct stimuli ≥ 5**, else the verdict is reported `underpowered` —
   never a pass, never a regression. `tags.expect_activation: "false"` dormancy contracts do not count
   toward this preference floor.
2. **p ≤ 0.05 on an exact one-sided sign test over *discordant* (non-tie) stimulus votes.** Ties are not
   discarded; they hold the discordant count down.

| discordant stimulus votes | records that pass | p |
| ---: | --- | ---: |
| ≤ 4 | none, however good the skill | ≥ 0.0625 |
| 5–7 | zero losses only (5W/0L) | 0.031 |
| 8 | one loss survivable (7W/1L) | 0.035 |

At exactly 5 stimuli a single tie is fatal — it leaves 4 discordant votes. At 6 stimuli one
tie is survivable (5W/1T/0L); at 7, up to two are (5W/2T/0L). A loss is not. Five is an *eligibility
floor*, not adequate
power. Add **discriminating stimuli** for task breadth. Use `runs` only to measure pass rate,
pass@k, pass^k, and flakiness for the same tasks. See
[`eng/eval-quality/README.md`](eng/eval-quality/README.md) for the full derivation and for the 22
structural defects the CI quality gate blocks.

Run the gate locally before pushing:

```bash
python eng/eval-quality/check_eval_quality.py
```

<!-- TODO: Vally is not yet public. Check with Aditya (Aditya Mandaleeka) on the
     canonical public location for Vally docs, then link the grader/stimulus
     reference and the CLI usage guide here instead of the local examples. -->

### Running tests locally

Prerequisites: Node.js 20+ and the [GitHub CLI](https://cli.github.com) signed in (`gh auth login`). The script checks these and tells you what's missing, so just run it:

```bash
# Run tests for a single skill
./eng/run-skill-evals.sh dotnet-msbuild binlog-failure-analysis

# Run tests for a whole plugin
./eng/run-skill-evals.sh dotnet-msbuild

# Exercise one custom-agent eval through the native SDK lane
dotnet run --project eng/skill-validator/src/SkillValidator.csproj -- evaluate \
  plugins/dotnet-msbuild/agents/msbuild.agent.md \
  --tests-dir tests/dotnet-msbuild --runs 1 --verdict-warn-only

# Exercise one redistributable workflow package against offline fixtures
dotnet run --project eng/skill-validator/src/SkillValidator.csproj -- evaluate \
  agentic-workflows/msbuild-quality-review/aw.yml \
  --tests-dir tests/agentic-workflows --runs 1 --verdict-warn-only

# Run every skill's tests
./eng/run-skill-evals.sh
```

Per-skill verdicts are written to `./eval-results/<plugin>/<skill>/results.json`, and the raw experiment output goes to `./eval-results/_experiment/`. Model and judge model come from the `overrides:` block in `dotnet-skills.experiment.yaml`.

> [!WARNING]
> LLM evaluations are noisy. Runs-per-stimulus is deliberately **not** set in
> `dotnet-skills.experiment.yaml`: an experiment-level `runs` overwrites every eval's own value
> instead of defaulting it, making per-eval trial counts impossible to express. Raise the eval's own
> `defaults.runs` instead — or, better, add discriminating stimuli.

### CI evaluation

Tests do **not** run automatically on pull requests. When a PR changes skills, the `pr-status` job posts a pending commit status and a maintainer must trigger the evaluation, binding it to a specific reviewed commit — either by submitting a PR review ("Files changed" → "Review changes") whose body contains `/evaluate` (recommended, no SHA to copy), or by commenting `/evaluate <sha>`. A bare `/evaluate` comment only posts guidance. Results are posted as a PR comment and uploaded as build artifacts.

The same discovery/reporting pipeline covers workflow packages and their specs
under `tests/agentic-workflows/<package>/eval.yaml`. Manual dispatch with
`plugin: agentic-workflows` selects that collection. These results measure
offline workflow decisions/proposals using the real imported prompts and
installed resources; they do not claim live Actions or publication validation.
Keep package compilation and trusted-helper regression tests as separate gates.

The [Skill Value dashboard](https://dotnet.github.io/skills/) provides historical
results for each skill by executor and judge model.

For the architecture, metrics, verdict policy, and real repair examples, see the
[Skill evaluation infrastructure overview](eng/vally-adapter/README.md). If a
scenario fails or regresses, see
[Investigating Results](eng/vally-adapter/InvestigatingResults.md) for how to
download artifacts, interpret `results.json`, and diagnose common failure
patterns.

## Writing style

- Be concise and specific.
- Prefer numbered steps for workflows.
- Prefer checklists for requirements.
- Define terminology the first time it appears.
- Avoid excessive formatting and avoid clever wording that could be misread by an agent.

## Security and safety

- Do not include secrets, tokens, or internal URLs.
- If you discover a security issue, do not open a public issue with sensitive details. Use the repository or organization security reporting process instead.

### External references

Skills often reference external tools, documentation, and projects — this is
expected and welcome, including community and third-party resources. To help
reviewers stay aware of external dependencies, the repository includes an
automated reference scanner (integrated into `skill-validator check`) that runs
in CI against plugin content (SKILL.md, agent files, and reference docs).

The scanner treats all of the following as CI-blocking errors:
- `http://` URLs where `https://` should be used
- `<script>` tags loading external resources without an `integrity` (SRI) attribute
- Pipe-to-shell patterns (`curl ... | bash`)
- URLs pointing to domains not listed in `eng/known-domains.txt`

Community tools and third-party projects are evaluated on a case-by-case basis
(see "What we look for" above). If your skill references a new external domain,
add it to `eng/known-domains.txt` in the same PR — the reviewer will
approve it alongside the skill content.

## Review process

Maintainers may request changes for:

- Clarity and unambiguous instructions
- Reduced scope
- More explicit validation
- Compatibility with multiple agent runtimes
- Consistency with existing conventions

We may close pull requests that are out of scope or too large to review. If that happens, we are happy to suggest a smaller path forward.

## Licensing and provenance

Only submit content that you have the right to contribute.

- Do not include copyrighted text from other projects.
- You may be asked to confirm that your contribution is original or appropriately licensed.

## Getting help

If you are unsure where a change belongs or how to structure a skill or agent, open an issue describing:

- The user problem
- The proposed outcome
- A small example of the desired behavior

If you're not sure whether something belongs under `skills/` or `agents/`, a good rule of thumb is:

- Put **reusable task playbooks** in `skills/`.
- Put **role + operating model** in `agents/`.

## Quality bar

Skills and agents in this repo should be:

- **Actionable**: the agent can follow them without guesswork.
- **Minimal**: no extra features or scope creep; focus on the task.
- **Verifiable**: always include a way to validate success.
- **Tool-conscious**: don't assume capabilities that might not exist in every runtime.

### What consistently separates a passing skill from a failing one

Every skill is scored head-to-head against the *same model with no skill loaded*, so the score is a
**delta**. The rules below are the ones this repo has learned the hard way, each from a merged fix.

**Content**

- Encode the decisions the model gets wrong; delete anything it already produces unaided. A skill
  that reads as reference prose ties its own baseline.
- Prefer "when A, do B, never C, verify D" tables over lists of plausible alternatives, and end with
  a concrete output contract (the exact command, the verdict line, the findings table).
- Scale output structure to input size. A twelve-section report for an eight-test suite loses to a
  concise direct answer.
- Add stop-conditions so a strong skill doesn't over-apply — then check you haven't over-corrected
  into answering more narrowly than the baseline did.
- Tell the agent to discover repo paths rather than listing them as required inputs; a "required"
  project path makes the agent ask the user for a file that is already in the working directory.
- Require truthful validation reporting. Claiming "Build succeeded" after a failed restore is an
  automatic loss.
- Verify load-bearing API claims by compiling or probing, not by reading source.
- Keep the common path in `SKILL.md` and gate rare or expensive paths behind `references/` reads.

**Activation**

- The `description` is the only text the runtime sees when choosing a skill. Put the user's own
  words in it: symptoms, error codes, artifact names, quoted requests.
- Partition against sibling skills on the real discriminator, not the shared topic, and add the
  matching exclusion to both siblings.
- Re-read every "do not use for" clause against the scenarios the skill exists to serve — an
  exclusion can lock out the skill's own purpose.
- Watch both budgets: 1,024 characters per description, and the plugin's rendered skill menu.

**When something fails**

- Classify before you rewrite. Broken fixtures, underpowered trial counts, forced tools, stale spec
  keys and harness errors have all masqueraded as skill regressions.
- Read the losing trial and the judge's stated reason, and drive the fix from that evidence rather
  than from style preference.
- A positive win/tie/loss record with a failing verdict is a power problem, not a content problem.
- A skill that is weak across model families, thinly used, and costing menu budget is a candidate
  for retirement, not indefinite polishing.

### Authoring skills for this repository

The repository ships agent skills for working on itself, under `.agents/skills/`:

| Skill | Use it when |
|-------|-------------|
| `create-skill` | Scaffolding a new skill and writing a description the runtime will route to |
| `create-skill-test` | Writing or resizing an `eval.yaml` |
| `improve-skill-quality` | An evaluation regressed, returned no verdict, or the skill didn't activate |
| `create-custom-agent` | Adding an agent definition |
| `authoring-github-workflows` | Editing anything under `.github/workflows/` |

## Skill-Validator & Evaluation workflow

Changes to `eng/skill-validator` or the `.github/workflows/evaluation*.yml` workflows must be made from a branch in the `dotnet/skills` repository (i.e., not from a fork). This is a security measure.
For pull requests from forks, the evaluation workflow (triggered via `/evaluate`) always uses the workflow YAML from the default branch of `dotnet/skills` and builds the validator from that default-branch checkout, so any changes to these files in the forked PR will be ignored during evaluation.