--- name: prepare-providers-documentation description: > Replace the manual commit-by-commit classification step in `breeze release-management prepare-provider-documentation` with AI-driven classification. For each provider with pending changes, analyze every PR (batched into one sub-agent per provider, not one per PR), pay special attention to potentially breaking changes by inspecting the actual diff, scope multi-provider PRs to the current provider's slice, ask the release manager when uncertain, and apply version bumps + changelog entries. Use during the regular provider release cycle as an alternative to the interactive breeze prompts. license: Apache-2.0 --- # Prepare Providers Documentation (AI-driven) This skill replaces the manual commit-by-commit classification step that the release manager normally performs when running `breeze release-management prepare-provider-documentation`. Instead of asking the release manager to type `d`/`b`/`f`/`x`/`m`/`s`/`v` for each commit, the skill drives the classification itself — inspecting every PR (with extra care for potentially breaking changes), scoping multi-provider PRs to the slice that touched the current provider, and asking the release manager only when genuinely uncertain. The skill keeps the existing breeze tooling as the source of truth for template generation. Claude only owns the classification + version bump + changelog entries; everything else (`__init__.py`, `README.rst`, `pyproject.toml`, `conf.py`, `get_provider_info.py`, `index.rst`) is still regenerated by `breeze release-management prepare-provider-documentation --reapply-templates-only`. > [!IMPORTANT] > This is a release-manager workflow. It mutates `provider.yaml` and > `changelog.rst` for many providers in one pass. Always run on a clean > working tree (or in a dedicated branch) and let the release manager review > the diff before committing. --- ## When to Use This Skill Use during the regular provider release cycle, in place of either of: ```shell breeze release-management prepare-provider-documentation breeze release-management prepare-provider-documentation --incremental-update ``` …when the release manager wants Claude to classify the changes instead of doing it by hand. The skill covers the same scope: classifying changes, bumping versions, generating changelog sections, reapplying templates, and folding new commits into an already-prepared release PR (incremental update). Two entry points: - **Initial run** — classify everything from scratch for a new release. Follow Phases 1–5 below. - **Incremental update** — extend an existing release PR with commits that landed on `main` since the changelog was first generated (typical when rebasing a release PR before merging). Skip ahead to the **Incremental Update** section after Phase 5. Either entry point runs in one of two **release shapes**. Establish which one applies before starting — the incremental flow behaves differently in each: - **Wave** (the default) — the regular cycle that releases *every* provider with pending changes. The provider list is an **output** of the run, so it may legitimately grow between the initial cut and the merge. - **Ad-hoc** — a release deliberately scoped to a fixed provider list (the release manager named a subset, or set `DISTRIBUTIONS_LIST`). The provider list is an **input** and must not grow on its own. Ask the release manager if it is not obvious, and record the answer — Incremental Phases 2 and 3.6 branch on it. Do **not** use this skill for: - `--only-min-version-update` runs (these don't need classification — just run breeze directly). - Releasing from a non-`main` base branch unless you also pass the right `--base-branch` to the breeze invocations described below. - Removing providers (state changes belong in a separate PR). --- ## Inputs You Need Before Starting Ask the release manager (and confirm by reading the answers back) for: 1. **`RELEASE_DATE`** in `YYYY-MM-DD` (or `YYYY-MM-DD_NN`) format, e.g. `2026-04-26`. This is what breeze stamps into `providers/.last_release_date.txt`. 2. **Base branch** — defaults to `main`. Only override when releasing from a provider-specific branch (e.g. `provider-cncf-kubernetes/v4-4`). 3. **Subset of providers**, if any. By default, classify every provider that has pending changes since its last release tag. If the release manager wants a subset (or has set `DISTRIBUTIONS_LIST`), use that list. 4. **Include flags**: whether to include `--include-not-ready-providers` and/or `--include-removed-providers`. Set the environment for the session: ```bash export RELEASE_DATE= # Optional, scopes everything to a subset export DISTRIBUTIONS_LIST=" ..." ``` Make sure the `apache-https-for-providers` git remote exists and is up to date — running breeze the first time below will recreate and fetch it. --- ## Workflow The skill runs in five phases. Mark tasks with `TaskCreate` for each phase and tick them off as you go — the release manager wants to see progress. ### Phase 1 — Discover and pre-classify pending changes (deterministic) The source of truth for "what changed since last release" is the same git query breeze uses internally: commits between the latest release tag for that provider (`providers-/`) and `apache-https-for-providers/`, restricted to the provider's own folders. Run the **deterministic classifier** — it discovers every provider with pending changes **and** pre-classifies each commit with hard-coded, high-confidence rules, flagging only the genuinely ambiguous ones as `needs_llm`. No random answers, nothing to discard: ```bash breeze release-management classify-provider-changes \ --base-branch main \ --output-file /tmp/provider-changes.json # scope to a subset by appending provider ids, e.g. ... amazon cncf.kubernetes ``` The JSON it writes: ```json { "base_branch": "main", "providers": { "amazon": { "current_version": "9.29.0", "commits": [ {"hash": "c2dbd7a75a", "pr": "67987", "subject": "Fix IDC domain S3 path resolution", "classification": "needs_llm", "reason": "no high-confidence deterministic rule matched"}, {"hash": "abc123", "pr": "68087", "subject": "Bump the edge-ui-package-updates group ...", "classification": "misc", "reason": "dependency bump (subject starts with 'Bump')"} ] } } } ``` How to read it: - Providers under `providers` have pending changes (these need attention). - `classification ∈ {documentation, skip, misc}` are **decided by rules — take them as-is**, no sub-agent needed (doc-only → `documentation`, test/example only → `skip`, `Bump …` dependency bump → `misc`). - `classification == needs_llm` → **Phase 3 decides** with a sub-agent. These are the only commits that need LLM analysis. - A provider with a `note`/`error` (e.g. a brand-new provider with no prior release tag) → treat as an **initial release** and classify by hand. > [!NOTE] > The classifier is deliberately conservative: `Fix …`/`Add …` subjects are > **not** auto-classified (an "Add …" can be a breaking change), so they come > back as `needs_llm`. The rules live in `classify_change_deterministically` > (`dev/breeze/src/airflow_breeze/prepare_providers/provider_documentation.py`). The range the classifier reports is not fixed for the whole run — it widens once a provider actually gets a version bump: > [!WARNING] > **A version bump supersedes the provider's doc-only marker — re-run discovery > after Phase 4a.** When `docs/.latest-doc-only-change.txt` is present the > classifier starts the range at that commit, so anything older (earlier doc-only > changes, tooling churn) is hidden for as long as the provider has no pending > release. That is correct while nothing ships — but the moment the provider gets > a real version bump the marker no longer applies, and those commits are part of > what the release publishes. > > Re-run `classify-provider-changes` **after** bumping versions and fold in every > commit that newly appears; otherwise the release silently drops doc-only > entries that were only ever suppressed by the marker. The tell is a provider > whose commit count grows between two runs without any new merges. Observed in > the 2026-08-01 wave: `cohere` gained #69649, `dbt.cloud`/`presto`/`trino` gained > #69478, and `papermill` gained #68322 — all invisible in the first pass. Then regenerate the auto-generated build files (this does **no** classification, so nothing random is produced): ```bash breeze release-management prepare-provider-documentation \ --reapply-templates-only --release-date "$RELEASE_DATE" git checkout -- $(git diff --name-only -- '**/provider.yaml' '**/changelog.rst') ``` This leaves the regenerated build files (`__init__.py`, `README.rst`, `pyproject.toml`, `conf.py`, `get_provider_info.py`, `index.rst`) in place and discards only the changelog/version files Claude is about to rewrite itself. ### Phase 2 — Per-provider commit list For each provider in **Success** from Phase 1, get the same commit list that breeze would have shown. From the repo root: ```bash PROVIDER_ID= # e.g. amazon, cncf.kubernetes PROVIDER_PATH=$(echo "$PROVIDER_ID" | tr '.' '/') # folder path: cncf/kubernetes PROVIDER_TAG=$(echo "$PROVIDER_ID" | tr '.' '-') # tag segment: cncf-kubernetes # Pick the latest *final* release tag. Two gotchas the tag pattern must handle: # * dotted provider ids use HYPHENS in tag names (providers-cncf-kubernetes/), # even though the source folder uses slashes — build the tag prefix from # PROVIDER_TAG, not PROVIDER_PATH; # * skip the sentinel upper-bound tags (providers-/99.98.0, /99.99.0) and rc # tags — git's default version sort orders "1.2.0rc1" AFTER "1.2.0", so a bare # `head -n1` would otherwise select a sentinel or a release candidate. LAST_TAG=$(git tag --list "providers-${PROVIDER_TAG}/*" --sort=-v:refname \ | grep -vE '/99\.9[0-9]\.' | grep -vE 'rc[0-9]+$' | head -n1) git log --pretty=format:'%H %h %cd %s' --date=short \ "${LAST_TAG}..apache-https-for-providers/main" \ -- "providers/${PROVIDER_PATH}/" ``` > [!WARNING] > This git query is a convenience for building the per-provider commit list, but > the **authoritative set is what breeze prints in the Phase 1 "Commit" tables** > for each provider. The tag-based range can still diverge from breeze when a > provider's most recent *final* tag is not the last actually-published release > (for example, a wave commit bumped the version on `main` but the published > baseline is older), which makes breeze include repo-wide commits this query > misses. When the two disagree, trust breeze's list and reconcile against it > before classifying. Capture the full hash, short hash, date, subject, and `#NNNN` PR number for each commit. Note that some old providers also have legacy paths under `airflow/providers//` — include those when present (consult `provider_details.possible_old_provider_paths` semantics by checking the provider's `provider.yaml` history if needed). ### Phase 3 — Classify the PRs (inline, or batched per-provider sub-agents) For each commit, classify it into one of: | Code | Meaning | Version bump | | ---- | ---------------------------------------------- | -------------- | | `d` | Documentation-only | none (patch if combined) | | `b` | Bug fix | patch | | `f` | Feature | minor | | `x` | Breaking change | major | | `m` | Misc (deps, refactors, internal only) | patch | | `s` | Skip (test/CI/example only — no user impact) | none | | `v` | Min Airflow version bump | minor (treated as misc + bump) | #### Take the deterministic classifications from Phase 1 `classify-provider-changes` (Phase 1) already classified every commit it could with hard-coded rules. Read `/tmp/provider-changes.json` and: - Use any commit whose `classification` is `documentation`, `skip`, or `misc` **as-is** — these map to `d`, `s`, `m` respectively; no sub-agent needed. - Only commits with `classification: needs_llm` go to a sub-agent (below). The deterministic rules (doc-only → `d`, test/example-only → `s`, `Bump …` dependency bump → `m`) are exactly the cheap cases — now computed once by breeze (`classify_change_deterministically`) instead of re-derived here. If you ever need the min-Airflow-bump case (`v`), that one is still a `needs_llm` judgement: a sub-agent should flag it when a PR bumps the provider's minimum Airflow version. > [!NOTE] > A `needs_llm` commit can still be classified as `skip` by the LLM. The > deterministic classifier only catches test/example-only changes by looking > at which files were modified in the commit. PRs that change build tooling, > packaging infrastructure, or breeze templates (files under `dev/breeze/`, > `scripts/`, etc.) and only regenerate template-generated files in the > provider slice (e.g. `README.rst`, `index.rst`, `pyproject.toml`) are NOT > caught by the deterministic rules but should still be classified as `skip` > — they have zero user-facing impact on the provider package itself. That rule cuts one way only. Read the next one before applying it. > [!WARNING] > The converse trap: a PR is **not** `skip` merely because its subject > describes tooling. `provider.yaml` is a **source** file, never a regenerated > one — anything it declares (`connection-types`, `conn-fields`, > `ui-field-behaviour`, `extra-links`, `hook-class-names`, dependencies) ships > to users through `get_provider_info.py`, so a change to it is `misc` at > minimum. `get_provider_info.py` moving *because `provider.yaml` moved* is > therefore not "regenerated metadata" in the sense above. Worked example: > `Fix conn-fields check crash for nested provider packages (#70224)` fixes a > prek check under `scripts/ci/prek/`, and in the same commit declares a new > `Verify SSL` connection-form field for `atlassian.jira` — `misc`, not `skip`. > Classify from the provider-scoped diff, never from the subject. #### Classify the `needs_llm` commits — batched per provider, not one agent per PR Only the commits the classifier returned as `needs_llm` still need a sub-agent. Classification is the token-heavy part of this skill, so spend sub-agents sparingly. Do **not** spawn one sub-agent per PR — that is one agent per commit and balloons to hundreds of agents on a normal release wave. Pick the smallest fan-out that fits the volume: - **Few `needs_llm` commits remain (≲ 15 across all providers) → classify inline.** Read each PR and its provider-scoped diff yourself, in this context. Spawn no sub-agents at all. - **More than that → one sub-agent per provider.** Each agent classifies that provider's *entire* remaining `needs_llm` list in a single pass. This is the natural unit: multi-provider PRs are classified independently per provider anyway (see Cross-Cutting Rules), and one provider-scoped agent amortizes the breaking-change checklist across all of that provider's commits instead of paying a fresh agent spin-up per commit. Only split a provider across more than one agent when its remaining list is large (> ~25 commits) — chunk it then. This keeps the sub-agent count at roughly the number of providers with pending changes, not the number of commits. Use the `Explore` agent type — they need read-only access. Brief each sub-agent with its provider and the whole batch of commits it owns: ``` Classify a batch of Apache Airflow provider PRs for ONE provider. Provider: (path: providers//) Commits to classify () — one row per PR: # # …(this provider's full remaining list) For EACH commit above: 1. Read the PR's title, body, and labels: `gh pr view --json title,body,labels,files` 2. Read the diff for the slice of the PR that touched providers// only: `gh pr diff -- 'providers//**'` (When the PR touches multiple providers, you only care about the slice for THIS provider — ignore the others when classifying.) 3. Decide a single classification: - documentation: only docs/comments/typos in the provider slice - bugfix: fixes incorrect behavior, no API changes - feature: adds new capability, parameter, operator, sensor, hook, or extends an existing one in a backwards-compatible way - breaking: see "Breaking-change checklist" below - misc: dependency bumps, internal refactors, type-hint cleanups, no user-visible behavior - skip: only tests/examples/CI for this provider's slice, OR changes that only touch build tooling, packaging infrastructure, or template-generated files (e.g. README.rst, index.rst, pyproject.toml regeneration, flit/sdist config, breeze templates) — even when those files live under the provider path. If the PR's actual code changes are entirely in dev/breeze/ or similar tooling and the provider-scoped diff is limited to regenerated docs/metadata, classify as skip. NEVER skip a slice that touches provider.yaml — that is a source file whose contents ship in get_provider_info.py (connection-types, conn-fields, ui-field-behaviour, extra-links, dependencies), so it is misc at minimum even when the PR's headline change is tooling. - min_airflow_bump: explicitly bumps the minimum Airflow version pin 4. Set BREAKING_RISK to "maybe" whenever the diff has any signal from the breaking-change checklist below, even if you think the author intended otherwise. Output one row per commit and nothing else, in this exact pipe format ( rows for commits): # | | | | A change is only breaking if the thing it breaks was **released**. Removing, renaming, or altering a symbol/behavior that was *introduced in this same unreleased wave* (i.e. the feature was added in one pending commit and changed in another, both after the provider's last release tag) is NOT a breaking change — users never received the old form, so there is nothing to break. Before classifying any removal/rename as breaking, confirm the affected symbol existed at the last released version: `git show providers-/:providers//... | grep ` (or grep the last release tag). If it isn't there, treat the change as part of delivering the new feature (feature/misc), not breaking. A within-wave rename of a brand-new operator or plugin is feature-shaped, not a major bump. Breaking-change checklist (any of these → BREAKING_RISK >= maybe; usually breaking unless clearly behind a deprecation shim) — **each item assumes the affected symbol/behavior shipped in a released version, per the rule above**: * Public class/function/method removed or renamed in the **public interface** of the provider — i.e. files under `providers//src/**/{hooks,operators,sensors,triggers, notifications,decorators,executors}/**`, the provider's top-level package `__init__.py`, plus anything imported by `provider.yaml` (`hook-class-names`, `extra-links`, etc.). Internal helpers (e.g. `utils/`, `_internal/`, `pod_manager.py`, or any module not re-exported from the package or referenced in `provider.yaml`) are NOT breaking on their own. NOT in tests/. * Required parameter added to a public constructor or operator __init__ * Default value of a public parameter changed * Return type or signature of a public method changed * `extra_dejson` / connection-form fields removed or renamed * Behavior change in `execute()`, `poke()`, `get_conn()` that produces different results for the same inputs * Minimum Python or Airflow version bumped (separate: that's min_airflow_bump unless the bump excludes a previously supported version of a provider's hard dependency, in which case it's also breaking) * Removed deprecation: a previously-deprecated symbol is now deleted * Schema change in stored data (xcom, connection, asset metadata, or the serialized state/context of a `BaseTrigger` subclass — deferred tasks survive provider upgrades only if the trigger's `serialize()` payload stays compatible) Do NOT trust the PR title alone — read the diff. A PR titled "Refactor X" that removes a public method is breaking. A PR titled "BREAKING: rename foo" that only renames a private symbol is not. A PR that renames a public class introduced earlier *in this same unreleased wave* is not breaking either — the old name was never released (see the released-only rule above). ``` Collect every sub-agent's rows (and any you classified inline) into one classification table for Phase 3.5. ### Phase 3.5 — Confirm with the release manager Print a per-provider summary in this exact format (so the release manager can scan it quickly): ``` Provider: amazon Current version: 9.12.0 Most-impactful change: feature → next version: 9.13.0 Commits (12): abc1234 d high docs: fix S3 example #65000 def5678 b high Fix retry on transient SQS error #65010 9ab0123 f high Add wait_for_completion to AthenaOperator #65020 4cd5678 x med Remove deprecated S3Hook.list_objects #65030 ⚠ BREAKING 7ef9012 m high Bump aiobotocore to 2.13 #65040 ... Uncertain: 2 commits below — please confirm: 4cd5678 x med Remove deprecated S3Hook.list_objects (#65030) Why: list_objects is documented as deprecated since 8.0.0 but never raised DeprecationWarning, so removal may surprise users. abc4321 ? low "Refactor Athena client" (#65060) Why: PR description says non-breaking but diff changes the default region resolution from env to provider extras. ``` Always escalate to the release manager when: - `CONFIDENCE: low` from any sub-agent. - `BREAKING_RISK: maybe` but the sub-agent classified as anything other than `breaking`. - Same PR appears in multiple providers and got different classifications across them — explain why and let the RM call it. - Most-impactful change is `breaking` (major bump): always reconfirm explicitly before applying. Major bumps are never silent. If the release manager corrects a classification, **save it** in your classification table and re-derive the most-impactful change. ### Phase 4 — Apply classifications For each provider, in order: #### 4a. Bump the version in `provider.yaml` Open `providers//provider.yaml`, find the `versions:` block, and prepend the new version. The bump rule (most-impactful classification across all commits for this provider, computed in Phase 3.5): | Most-impactful | Bump | | ------------------------ | ------------- | | `breaking` | major (X+1.0.0) | | `feature` | minor (X.Y+1.0) | | `min_airflow_bump` | minor (X.Y+1.0) | | `bugfix` | patch (X.Y.Z+1) | | `misc` | patch (X.Y.Z+1) | | `documentation` only | no bump — handle as doc-only (see below) | | `skip` only | no bump — nothing to do | Also update `source-date-epoch:` to the current `int(time.time())`. For **doc-only** providers, do not bump the version. Instead, write the latest commit hash from the doc-only batch into `providers//docs/.latest-doc-only-change.txt` (newline terminated). This is what breeze checks on the next release to know the provider hasn't really changed. #### 4b. Write the changelog entry Open `providers//docs/changelog.rst`. Insert a new section *above* the most recent existing version section. The exact format must match `dev/breeze/src/airflow_breeze/templates/CHANGELOG_TEMPLATE.rst.jinja2` — don't paraphrase it. The skeleton: ```rst .. note:: This release of provider is only available for Airflow X.Y+ as explained in the Apache Airflow providers support policy _. Breaking changes ~~~~~~~~~~~~~~~~ * `` (#NNNN)`` Features ~~~~~~~~ * `` (#NNNN)`` Bug Fixes ~~~~~~~~~ * `` (#NNNN)`` Misc ~~~~ * `` (#NNNN)`` Doc-only ~~~~~~~~ * `` (#NNNN)`` .. Below changes are excluded from the changelog. Move them to appropriate section above if needed. Do not delete the lines(!): * `` (#NNNN)`` ``` Rules: - A `.. note::` block at the top of the version section (directly under the `` underline, before the first `~~~` header) is used in two distinct situations. Include it whenever **either** applies — combine the wording into a single note, or stack two notes, when both do: - **Airflow min-version bump** — when the bump was driven by a `min_airflow_bump` (or by a `breaking` whose breaking aspect *is* the Airflow min bump), use the support-policy wording shown in the skeleton. - **Breaking change** — for **every** `breaking` classification (major bump, including a `0.x` minor that ships a breaking change), add a note explaining *what* breaks and *how users should adapt* (the migration path). Write it from the PR description and the actual diff, not as a restatement of the commit subject — the reader must learn how to react without opening the PR. This mirrors the standing changelog convention ("only add notes … when there are some breaking changes and you want to add an explanation to the users on how they are supposed to deal with them"). The bullet under `Breaking changes` still lists the commit subject as usual; the note is in addition to it, not a replacement. - Drop a section entirely if it has no entries (e.g. no `Breaking changes` section if there were none — don't leave an empty header). - The `.. Below changes are excluded ...` block at the end is required even if empty. Lines under it use the indented ` * ``...``` form (three-space indent, double backticks). - Subjects must be the original commit subject with backticks replaced by single quotes (matches `message_without_backticks`). Don't paraphrase. - **Exception — rewrite subjects written in project-internal language.** The don't-paraphrase rule keeps you honest about *what* shipped; it does not oblige you to publish a subject the reader cannot act on. The changelog is read by users, not by us. Rewrite the entry — keeping every `(#NNNN)` — when the subject is only meaningful inside the project: - **Spell out internal abbreviations.** `KPO`, `DFP`, `TI`, `RTIF`, `OL` and friends are our shorthand. Write ``Add '--min-completed-minutes' to 'cleanup-pods' to prevent KubernetesPodOperator race condition (#70595)``, not ``… to prevent KPO race condition (#70595)``. - **Name what a grouped dependency bump actually changed.** Dependabot group subjects such as ``Bump the fab-ui-package-updates group across 1 directory with 3 updates (#70604)`` tell the reader nothing. Read the PR diff and list the packages with their versions: ``Bump prettier to 3.9.6, stylelint to 17.14.1, webpack to 5.109.0 (#70604)``. A single-package bump still needs its target version: ``Bump eslint to 10.8.0 (#70697)``. - **Describe the user-visible effect, not the internal mechanic.** ``Remove noqa:S101 from production code (#70378)`` names a lint directive; the reader wants ``Mark asserts under 'TYPE_CHECKING' in 'DocumentLoaderOperator' (#70378)``. - **Drop our test vocabulary.** A system test is just an example Dag to the user, so ``Remove hard-coded deferrable crawler run from example_glue system test (#70206)`` becomes ``Remove hard-coded deferrable crawler run from example_glue (#70206)``. Rewrite the *wording*, never the *claim* — do not describe behaviour that did not ship. When a subject is too vague to rewrite honestly, read the PR diff before writing the entry. - **Exception — collapse within-wave "add then rename/rework" chains into one net entry.** When several pending commits are steps toward *one* net change — a feature added in one PR and renamed or reworked in a later PR, both since the last release (the released-only situation from Phase 3) — do **not** list the intermediate steps as separate entries. A reader who never saw the released intermediate form gets no context from ``Add X listener (#a)`` under Features plus ``Rename X to Y (#b)`` under Misc. Write a **single** entry that describes the **net user-facing change** and references **all** related PRs, placed in the section of the most-impactful step. Real example: #68082 added a Kafka listener and #70014 renamed it to the Kafka Event Producer in the same wave → ``Add Kafka Event Producer publishing DagRun and TaskInstance state-change events (#68082, #70014)`` under Features (and *no* separate Misc "Rename …" line). This is the changelog counterpart of the unreleased-feature classification rule: classify the rename as non-breaking (Phase 3) **and** describe only what shipped, naming every PR involved. - **Capitalize the first letter of every entry**, not only after stripping a Conventional Commit prefix. Contributors sometimes write a lowercase subject (`derive keycloak oauth redirect_uri …`) or a pseudo-scope (`cncf-kubernetes: fix …`); the changelog convention is a leading capital, so render them as ``Derive keycloak oauth redirect_uri …`` / ``Cncf-kubernetes: fix …``. - **Strip Conventional Commit prefixes** before writing to the changelog. If the subject starts with a prefix like `feat:`, `fix:`, `chore:`, `docs:`, `refactor:`, `ci:`, `test:`, `perf:`, `build:`, or `style:` (with or without a scope in parentheses, e.g. `fix(amazon):`), remove the prefix and capitalize the first letter of the remaining text. Example: `refactor: Fix _is_http_client_closed ...` → `Fix _is_http_client_closed ...`. Airflow does not use Conventional Commits and these prefixes should not appear in changelogs. - **Send no-PR release-tooling commits to the excluded block**, even when the Phase 1 deterministic classifier labeled them `documentation`. Subjects like ``Prepare … providers release/documentation …`` and ``Hide non-user-facing entries from ad-hoc provider release notes`` (often with no `(#NNNN)` suffix because they were committed directly) are release plumbing, not user-facing changes — a `(#NNNN)`-less line in a visible section reads as a mistake. Put them under the `.. Below changes are excluded …` block. breeze's deterministic classification can even be inconsistent for the same commit across providers, so normalize to excluded. - Always keep the `(#NNNN)` PR suffix (or, for a collapsed chain, the comma-separated list of all involved PRs). #### 4c. Regenerate templates with breeze Once **all** providers have their `provider.yaml` and `changelog.rst` updated, run: ```bash breeze release-management prepare-provider-documentation \ --reapply-templates-only \ --skip-git-fetch \ --release-date "$RELEASE_DATE" ``` This regenerates `__init__.py`, `README.rst`, `pyproject.toml`, `conf.py`, `get_provider_info.py`, and `index.rst` for every provider — picking up the new versions you just wrote. It will not touch `changelog.rst`. > [!NOTE] > `commits.rst` per provider is also stable template content (the actual > commit list is rendered at doc-build time via the > `airflow-providers-commits` directive). It will be regenerated on the > next full release. No action needed here. #### 4d. Resolve `# use next version` inter-provider pins Contributors can defer an inter-provider dependency bump by pinning it in `pyproject.toml` with a trailing `# use next version` comment, instead of hard-coding a version that does not exist yet. Now that the versions are bumped, resolve those pins: ```bash breeze release-management update-providers-next-version ``` This rewrites every `# use next version` dependency to the just-bumped version of the referenced provider and removes the comment. > [!IMPORTANT] > **Run this every time, before opening the PR — even when you believe no > provider uses the comment** (the command is a safe no-op when none do). > Skipping it ships the wave with stale lower bounds on inter-provider > dependencies; once the PR is merged the only remedy is a separate > follow-up PR. This is the "Update versions of dependent providers to the > next version" step in `dev/README_RELEASE_PROVIDERS.md` — it lives between > doc preparation and PR creation, so it is easy to forget when the skill > hands back to the regular release workflow. **Provider dependency-bump CI guard.** Every `>=` bump this produces (and any inter-provider `>=` bump made during the wave, e.g. a `breaking` provider that dependents must now require) trips the `check_provider_dependency_bumps` selective-check (`dev/breeze/src/airflow_breeze/utils/selective_checks.py`), which fails CI with *"Provider dependency version bumps detected that should only be performed by Release Managers!"*. That guard exists to stop **contributors** from silently changing inter-provider `>=` floors; for a release wave the bumps are legitimate. The release PR **must carry the `allow provider dependency bump` label** to bypass it — every prior "Prepare providers release …" PR carries this label. Tell the release manager to add the label to the PR (it re-triggers the check via the `labeled` event); the bumps are not a mistake to revert. ### Phase 5 — Validate Run the same checks the release manager would run: ```bash # RST lint + license headers + ruff on Python files prek run --from-ref main --hook-stage pre-commit # Spot-check that provider.yaml versions parse breeze release-management prepare-provider-documentation \ --reapply-templates-only --skip-git-fetch \ --release-date "$RELEASE_DATE" # idempotent — should be a no-op diff ``` Then `git diff --stat` and walk the release manager through the diff provider-by-provider: - Confirm the version in `provider.yaml` matches the bump rule. - Confirm `changelog.rst` has the right sections populated. - **Check for misplaced top-level note blocks.** Each provider's `changelog.rst` has a standing `.. NOTE TO CONTRIBUTORS:` RST comment near the top (above all version sections). If you detect any `.. note::` directive that is NOT nested under a specific version section (i.e. it appears before the first version header, or between the `Changelog` heading and the first version), notify the release manager immediately — it likely means a breaking-change or min-version note was accidentally written at the wrong indentation level or position. Two variants to check for, both seen in the 2026-08-01 wave: - **Already misplaced on the base branch.** A contributor adds a note for their own change directly under the `Changelog` header instead of inside the version section it belongs to, so it renders page-wide. Move it into the section that ships the change (`google` #70869). - **Pushed out of place by your own prepend.** When a note was sitting above the first version header and you prepend a new section, the note ends up *between* your new excluded block and the previous version — still wrong, but no longer above the first header, so a scan that only looks at the top of the file misses it (`openai` #69506). Check **inside every section you touched** that no `.. note::` appears after the `.. Below changes are excluded …` marker; a note belongs directly under the version underline, before the first `~~~` header. - Confirm Phase 4d ran: no `# use next version` comment remains where the referenced provider was bumped in this wave. - **If any inter-provider `>=` floor changed** (Phase 4d resolved a pin, or a `breaking` provider forced a dependent to require its new major), tell the release manager the PR needs the `allow provider dependency bump` label — otherwise the `check_provider_dependency_bumps` CI check fails with *"Provider dependency version bumps detected that should only be performed by Release Managers!"*. `git diff` the changed `pyproject.toml` files for `apache-airflow-providers-*` `>=` changes and list them for the RM. - **Scan the new changelog sections for these entry defects** — grep the lines you added: (1) a bullet whose text starts with a lowercase letter → capitalize it (Phase 4b); (2) a bullet in a *visible* section (Features / Bug Fixes / Misc / Doc-only) with no `(#NNNN)` suffix → usually no-PR release-tooling that belongs in the excluded block (Phase 4b); (3) an "add then rename" pair for the same feature left as two separate entries → collapse into one net entry naming both PRs (Phase 4b); (4) an internal abbreviation (`KPO`, `DFP`, `TI`, `RTIF`, `OL`) or the phrase "system test" → rewrite in user-facing language (Phase 4b); (5) a dependency bump that names no version, or a Dependabot group subject of the form "… group … with N updates" → replace with the actual packages and versions from the PR diff (Phase 4b). Reviewers reliably catch all of these, so fix them before handing off. - Flag anything where Phase 3.5 had to escalate, so the RM can double-check. Stop here. Do not commit, do not push — the release manager opens the PR themselves following the regular release workflow in `dev/README_RELEASE_PROVIDERS.md`. Make sure Phase 4d (`update-providers-next-version`) has been run before that PR is opened, and that the PR carries the `allow provider dependency bump` label whenever any inter-provider `>=` floor changed (see Phase 4d). --- ## Incremental Update Use this flow when the release PR has already been opened (changelog and version bumps applied via Phases 1–5) and the release manager rebases it to pick up commits that landed on `main` after the original classification. This is the equivalent of `breeze release-management prepare-provider-documentation --incremental-update`, but driven by the same AI classification logic as the initial run. On a **wave**, "extend" is not only about the providers already in the PR: a provider that had nothing pending when the wave was cut can pick up a user-facing change before the PR merges, and the flow below is responsible for surfacing it rather than letting the release ship without it. > [!IMPORTANT] > Run on the **release PR branch** *after* rebasing onto the latest base > branch. Do not start the incremental flow on a clean checkout — it needs > the prior classifications already written into `changelog.rst` to > diff against. ### Incremental Phase 1 — Refresh the apache remote ```bash breeze release-management prepare-provider-documentation \ --reapply-templates-only \ --release-date "$RELEASE_DATE" ``` This re-fetches `apache-https-for-providers/` and regenerates the auto-generated build files for every provider — picking up any upstream template changes that landed since the original PR was opened. It does **not** touch `provider.yaml` or `changelog.rst`. ### Incremental Phase 2 — Detect unrecorded commits across **all** providers > [!IMPORTANT] > On a **wave**, sweep **every** provider — not just the ones already in the > release PR. A provider that had nothing pending when the wave was cut can > acquire its first user-facing change hours later, and iterating only over > providers that already have a new version section can never discover it: it > has no section to extend. That provider then stays invisible for the rest of > the release. This is an observed miss, not a hypothetical one — see the > worked example at the end of this phase. Re-run the deterministic classifier over the full provider set — the same command as Phase 1 of the initial run, with **no** provider ids appended: ```bash breeze release-management classify-provider-changes \ --base-branch main \ --output-file /tmp/provider-changes.json ``` Then reduce its output to commits **not yet recorded** in the matching provider's changelog. A commit is unrecorded when its `#NNNN` PR number appears nowhere in `providers//docs/changelog.rst` — the same predicate breeze uses internally (see the `_generate_new_changelog` append branch in `dev/breeze/src/airflow_breeze/prepare_providers/provider_documentation.py`): ```bash python3 - <<'EOF' import json, pathlib data = json.loads(pathlib.Path("/tmp/provider-changes.json").read_text()) for provider_id, info in sorted(data["providers"].items()): base = pathlib.Path("providers") / provider_id.replace(".", "/") changelog = base / "docs" / "changelog.rst" seen = changelog.read_text() if changelog.exists() else "" doc_only = base / "docs" / ".latest-doc-only-change.txt" doc_hash = doc_only.read_text().strip() if doc_only.exists() else "" for commit in info.get("commits", []): pr = commit.get("pr") # Substring match, NOT "(#NNNN)": a collapsed within-wave entry reads # "(#68082, #70014)", which an exact-suffix match would report as new. if pr and f"#{pr}" in seen: continue if not pr and commit["subject"].replace("`", "'") in seen: continue if doc_hash and commit["hash"].startswith(doc_hash[:10]): continue print(f"{provider_id}\t{commit['hash'][:10]}\t#{pr}\t" f"{commit['classification']}\t{commit['subject']}") EOF ``` Split the result into two buckets. A provider is **already in the wave** iff its `provider.yaml` changed on this branch: ```bash git diff --name-only ..HEAD -- '**/provider.yaml' \ | sed 's|providers/||; s|/provider.yaml||' | tr '/' '.' | sort ``` - **Bucket A — already in the wave.** Fold the unrecorded commits into the existing version section: continue through Incremental Phases 3 → 3.5 → 4a. - **Bucket B — not in the wave.** The provider has unrecorded commits but no new version section, so it is a candidate to *join*. Classify it (Incremental Phase 3), then take it to **Incremental Phase 3.6**. Most Bucket B rows are ordinary noise — repo-wide tooling and test commits that correctly keep a provider out of the release. A Bucket B provider whose unrecorded commits are **all** `skip` stays out silently; escalate only when at least one classifies as something other than `skip`. If a provider has zero unrecorded commits, skip it. > [!NOTE] > **Worked example — the miss this phase exists to prevent.** In the > 2026-07-22 wave, `atlassian.jira` had nothing pending when the wave was cut > at 22:15. PR #70224 merged at 03:41 the next morning, adding a `Verify SSL` > conn-field to `providers/atlassian/jira/provider.yaml`. The branch was > rebased past that commit and an incremental fold-in ran — but because > `atlassian.jira` had no version section, the old per-wave-provider loop > never looked at it, and the provider was still missing from the release PR > a day later. The full sweep above surfaces it; Phase 3.6 asks about it. ### Incremental Phase 3 — Classify the new commits Same logic as Phase 3 of the initial run — including the auto-classify heuristic for docs/test-only changes and the batched classification (inline when few commits remain, otherwise one sub-agent per provider) with the breaking-change checklist. Incremental runs usually have only a handful of new commits, so prefer classifying them inline rather than spawning any sub-agent. The output is a per-provider table mapping each new commit hash to a classification. ### Incremental Phase 3.5 — Decide whether to escalate the version bump Compute the most-impactful classification across **both** the existing classified commits in the changelog **and** the new ones. If the most impactful is now stronger than what's already in `provider.yaml`, the version needs to be re-bumped. The escalation table: | Was bumped to | Now most-impactful is | Action | | ------------- | --------------------- | --------------------------------------- | | patch | `feature` | re-bump to next minor (X.Y+1.0) | | patch | `min_airflow_bump` | re-bump to next minor (X.Y+1.0) | | patch / minor | `breaking` | re-bump to next major (X+1.0.0) | | minor | `feature` | no change — already minor | | anything | `bugfix` or `misc` | no change | A re-bump means: replace the prepended version in `provider.yaml` AND update the version header in `changelog.rst`'s new section to match. **Always confirm a re-bump with the release manager** — explicitly state the old version, the new version, and which incoming commit forced the escalation. Don't silently re-bump. ### Incremental Phase 3.6 — Ask before adding a provider to the wave Every Bucket B provider (Incremental Phase 2) with at least one unrecorded commit classified as something other than `skip` is a provider the release is currently missing. **Never add one silently, and never drop one silently — always ask.** The release manager owns the scope of the release; your job is to make sure the choice is made deliberately rather than by omission. On an **ad-hoc** release the answer is usually "leave it out" — the provider list is a fixed input — but still surface it, so the release manager knows the change exists and needs a later release. Ask once per provider. State what landed, why it is user-facing, when it landed relative to the cut, and the version consequence of each option: > Provider `atlassian.jira` is **not** in this wave, but PR #70224 ("Fix > conn-fields check crash for nested provider packages", merged 2026-07-23 > 03:41 — about 5h after the wave was cut) added a `Verify SSL` conn-field and > `ui-field-behaviour` to its `provider.yaml`. That is shipped metadata: it > changes the Jira connection form. Its other pending commits (#67978, #68991) > are test/template-only. > **Add it to the wave** (3.3.4 → 3.3.5, most-impactful `misc`), or **leave it > out** and let the change ride the next release? Record the answer. If the release manager says **add**, the provider follows the *initial-run* application path rather than the append path — see Incremental Phase 4b. ### Incremental Phase 4 — Apply the new entries #### 4a. Providers already in the wave — append to the existing section For each new commit, insert into the existing latest-version section of `changelog.rst` under the right header: | Classification | Section | | ---------------------- | ---------------------- | | `breaking` | `Breaking changes` | | `feature` | `Features` | | `bugfix` | `Bug Fixes` | | `misc` | `Misc` | | `min_airflow_bump` | `Misc` | | `documentation` | `Doc-only` | | `skip` | excluded block at end | If the section header doesn't exist yet (e.g. previously there were no breaking changes, but a new commit introduced one), create the header above the next existing section, matching the order in `CHANGELOG_TEMPLATE.rst.jinja2`: `Breaking changes` → `Features` → `Bug Fixes` → `Misc` → `Doc-only`. If you re-bumped the version in Incremental Phase 3.5, also add or remove the `.. note::` block about the Airflow min version requirement to match the new bump kind. If a new commit is classified `breaking`, add (or extend) a `.. note::` at the top of the version section explaining what breaks and how users should adapt, exactly as in the breaking-change note rule in the initial run's Phase 4b. #### 4b. Providers joining the wave — apply the initial-run path A provider the release manager confirmed in Incremental Phase 3.6 has no new version section yet, so there is nothing to append to. Run the steps of the **initial run's Phase 4** for that provider only: - **initial-run 4a** — prepend the new version to `provider.yaml` and refresh `source-date-epoch`. Reuse the epoch the rest of the wave already carries (`grep -h source-date-epoch` across the providers this branch bumped) so the release stays on a single value rather than gaining a stray third one. - **initial-run 4b** — write a complete new version section in `changelog.rst`: the classified commits under their headers, **every** `skip` commit in the `.. Below changes are excluded …` block, and a `.. note::` if the bump is breaking or a min-Airflow bump. - **initial-run 4c** — re-run `--reapply-templates-only` so the joining provider's `__init__.py`, `README.rst`, `pyproject.toml`, `index.rst` pick up the new version. Expect a diff for exactly that provider and no other. - **initial-run 4d** — re-run `update-providers-next-version`: a `# use next version` pin pointing at the joining provider must now resolve. ### Incremental Phase 5 — Validate Same as Phase 5 of the initial run plus an extra check: confirm there are no leftover "Please review …" markers from a prior interactive `breeze release-management prepare-provider-documentation --incremental-update` run. If any are present (someone ran the breeze incremental flow before invoking this skill), remove them as part of the final pass. Then walk the diff with the release manager. Also confirm that **every Bucket B provider from Incremental Phase 2 was either added or explicitly declined** by the release manager — none dropped by omission. Re-run the Phase 2 sweep after applying; the only providers it should still report are ones whose unrecorded commits are all `skip`, plus any the release manager deliberately left out. If the incremental run bumped a provider to a *new* version (Incremental Phase 3.5) or added one to the wave (Incremental Phase 3.6), re-run Phase 4d (`update-providers-next-version`) as well — a `# use next version` pin on that provider must resolve to the freshly bumped version before the rebased PR is pushed. --- ## Cross-Cutting Rules ### PRs covering multiple providers When a single PR touches several providers (e.g. `Add Python 3.14 Support (#63520)` touches dozens), classify it **independently per provider**. The same PR can be `feature` in one provider (a real new capability) and `misc` in another (just a constraint bump in `pyproject.toml`). Always scope the diff inspection (whether inline or in a per-provider sub-agent) to the current provider's path: ```bash gh pr diff -- 'providers//**' ``` If the per-provider classifications come back different, do NOT try to "reconcile" them — that's a feature, not a bug. The release manager wants each provider's changelog to reflect what changed in *that* provider. ### Asking the release manager — phrasing When you ask, state your best guess and the alternative explicitly: > Provider `amazon`, commit `4cd5678` ("Remove deprecated `S3Hook.list_objects`" > #65030): I classified this as **breaking** because the symbol is removed > from the public API in `providers/amazon/src/airflow/providers/amazon/aws/hooks/s3.py`, > even though the PR description says "deprecated since 8.0.0". Confirm > **breaking** (major bump 9.x → 10.0.0) or override to **misc** (patch)? Don't ask vague yes/no questions ("is this breaking?"); always offer the two alternatives with the version-bump consequence. ### Things you must NOT do silently - Bump major version without explicit confirmation from the release manager. - Add a provider to the wave — or leave one out — without asking (Incremental Phase 3.6). Scope is the release manager's call, and a provider dropped by omission is as much a silent decision as one added. - Reclassify a commit the RM already confirmed. - Skip commits that don't fit a category — flag them as `?` and ask. - Edit `commits.rst`, `index.rst`, `__init__.py`, `README.rst`, `pyproject.toml`, `conf.py`, `get_provider_info.py` directly. Those are template-generated by breeze. - Run `git add` or `git commit` — the release manager owns the PR. ### When to give up and fall back to interactive breeze If the per-provider commit count is huge (50+) **and** the sub-agents come back with `low` confidence on most of them (typically because the diffs require deep domain knowledge), tell the release manager you're stopping the AI classification and recommend they run the regular interactive `breeze release-management prepare-provider-documentation` for that specific provider. Don't try to power through guesswork — the wrong classification at major-bump granularity is worse than a slower manual run. --- ## References - `dev/breeze/src/airflow_breeze/prepare_providers/provider_documentation.py` — the breeze module this skill replaces (classification + changelog generation). Read this when in doubt about format. - `dev/breeze/src/airflow_breeze/templates/CHANGELOG_TEMPLATE.rst.jinja2` — exact format for the changelog section you write in Phase 4b. - `dev/README_RELEASE_PROVIDERS.md` §"Convert commits to changelog entries and bump provider versions" — the human workflow this skill automates. - `PROVIDERS.rst` §"Upgrading minimum supported version of Airflow" — policy for `min_airflow_bump` classifications.