# content-placeholder-text Detect TODO markers, bracket placeholders, and unfilled template text | | | |---|---| | **Severity** | warning (auto) | | **Autofix** | - | | **Since** | v0.9.0 | | **Category** | [Content Intelligence](content-intelligence.md) | ## Why An LLM cannot distinguish between a deliberate instruction and an unfilled template. A `TODO`, `[Insert API key here]`, or `*TBD*` left in an instruction file will be interpreted literally — the model may try to complete the TODO itself, use a placeholder value as a real credential, or follow a half-written instruction in unpredictable ways. ## Examples **Bad:** ```markdown TODO: add deployment instructions here. Set the API key to [Insert your API key]. *Details to be added* ``` **Good:** ```markdown Deploy with `make deploy-staging`. Set the API key via the `API_KEY` environment variable. ``` ## How to fix Replace each placeholder with the real content it was standing in for. If the content is not ready yet, remove the placeholder entirely — an absent instruction is better than one the model will misinterpret. ## Configuration ```yaml rules: content-placeholder-text: enabled: auto # true | false | auto severity: warning ``` ## Research Basis **Detects TODO markers, bracket placeholders, and unfilled template text** in instruction files (e.g., `TODO`, `FIXME`, `[Insert API key here]`, `*TBD*`). Placeholder text in committed instruction files is unfinished work that the agent treats as real context. An LLM cannot distinguish between a deliberate instruction and an unfilled template — it processes `[Insert your API endpoint here]` as a literal instruction, potentially generating code that references a nonexistent endpoint or asking the user to fill in information that should already be present. This is standard software engineering hygiene applied to a new file type. `TODO` and `FIXME` markers have been tracked by linters such as ESLint's `no-warning-comments` for decades because they indicate incomplete implementation. The same principle applies to instruction files: if the content isn't ready, it shouldn't be in the agent's context. **References:** - [ESLint: no-warning-comments](https://eslint.org/docs/latest/rules/no-warning-comments) — Tracks TODO/FIXME as code quality signals; the same pattern applies to instruction files - [Anthropic: Effective Context Engineering](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents) — "Keep your context informative, yet tight" — placeholder text is uninformative noise that consumes context budget ## Instruction Budget vs. Context Budget skillsaw has two separate budget rules that measure different things: ### `content-instruction-budget` — How many directives? Counts **discrete imperative instructions** per file using regex matching on imperative verb patterns (lines starting with "use", "always", "never", "ensure", etc.). Code blocks are stripped first. | Threshold | Severity | |-----------|----------| | 80–119 instructions | INFO | | 120–150 instructions | WARNING | | 150+ instructions | ERROR | **Why it matters:** The "Curse of Instructions" (ICLR 2025) showed that the probability of following all N instructions equals p^N — exponential decay. At p = 0.99 and N = 150, the probability of following all instructions is only ~22%. The IFScale benchmark confirmed that primacy bias (selectively ignoring later instructions) becomes dominant at 150–200 instructions. This is about **cognitive load on the model** — too many simultaneous directives exceed the model's instruction-following capacity regardless of how many tokens they occupy. ### `context-budget` — How many tokens? Measures **estimated token count** (chars ÷ 4) of each individual file, checked per-file against category-specific thresholds. | Category | Warn | Error | |----------|------|-------| | CLAUDE.md, AGENTS.md, GEMINI.md, QWEN.md | 6,000 | 12,000 | | Instruction files (Cursor, Copilot, Cline, Kiro) | 4,000 | 8,000 | | Skills | 3,000 | 6,000 | | Commands, agents, rules | 2,000 | 4,000 | **Why it matters:** Raw token count determines how much of the context window the file consumes and how severely attention degrades. Levy et al. showed reasoning performance degrades at ~3,000 tokens. Chroma's "Context Rot" study found that attention dilution is **quadratic** in token count — doubling the tokens more than doubles the accuracy loss. This is about **context window consumption** — a single file that's too large will crowd out other context and degrade attention across the board. ### The distinction A file with 50 instructions in 5,000 tokens (verbose prose around each one) has a low instruction budget but high context budget. A file with 200 terse one-line instructions in 2,000 tokens has a high instruction budget but low context budget. Both degrade model performance, but through different mechanisms. | | Instruction Budget | Context Budget | |---|---|---| | **Measures** | Discrete imperative count | Estimated token count | | **Scope** | Per-file | Per-file | | **Degradation mechanism** | Instruction-following capacity | Attention dilution | | **Research basis** | Curse of Instructions (ICLR 2025) | Same Task, More Tokens (ACL 2024) | ## Key Papers (Cross-Cutting) These papers justify multiple rules simultaneously: | Paper | Venue | Rules | |-------|-------|-------| | Liu et al., [Lost in the Middle](https://arxiv.org/abs/2307.03172) | TACL 2024 | critical-position, section-length, cognitive-chunks | | [Curse of Instructions](https://openreview.net/forum?id=R6q67CDBCH) | ICLR 2025 | instruction-budget, contradiction, inconsistent-terminology | | Jaroslawicz et al., [How Many Instructions Can LLMs Follow at Once?](https://arxiv.org/abs/2507.11538) | arXiv 2025 | instruction-budget | | Levy, Jacoby & Goldberg, [Same Task, More Tokens](https://arxiv.org/abs/2402.14848) | ACL 2024 | tautological, redundant-with-tooling, instruction-budget, section-length | | Bsharat et al., [Principled Instructions Are All You Need](https://arxiv.org/abs/2312.16171) | arXiv 2023 | weak-language, negative-only, actionability-score | | [Suppressing Pink Elephants](https://arxiv.org/abs/2402.07896) | arXiv 2024 | negative-only | | Chroma, [Context Rot](https://research.trychroma.com/context-rot) | 2025 | critical-position, instruction-budget, section-length | | [When Prompts Go Wrong](https://arxiv.org/abs/2507.20439) | arXiv 2025 | contradiction | | [Anthropic: Effective Context Engineering](https://www.anthropic.com/engineering/effective-context-engineering-for-ai-agents) | 2025 | tautological, redundant-with-tooling, instruction-budget, broken-internal-reference, placeholder-text | | Wang et al., [LLMs Meet Library Evolution](https://yebof.github.io/assets/pdf/wang2025icse.pdf) | ICSE 2025 | banned-references | *Run `skillsaw explain content-placeholder-text` to see this documentation and the rule's effective configuration in your terminal.*