# References

References for the OWASP Top 10 for LLM Applications (2026), grouped by risk entry and formatted in APA-7. A source cited by more than one entry appears in full under each, with an `[also cited in ...]` note.

## LLM01: Prompt Injection

- Amazon Web Services. (2025a, July). *AWS-2025-015: Amazon Q VS Code extension supply-chain incident* [Security bulletin]. https://aws.amazon.com/security/security-bulletins/AWS-2025-015/
- Amazon Web Services. (2025b, July). *AWS-2025-019: Amazon Q runtime injection* [Security bulletin]. https://aws.amazon.com/security/security-bulletins/AWS-2025-019/
- Chao, P., Debenedetti, E., Robey, A., Andriushchenko, M., Croce, F., Sehwag, V., Dobriban, E., Flammarion, N., Pappas, G. J., Tramèr, F., Hassani, H., & Wong, E. (2024). *JailbreakBench: An open robustness benchmark for jailbreaking large language models*. arXiv. https://arxiv.org/abs/2404.01318
- Chen, R., Cui, S., Huang, X., Pan, C., Huang, V. S.-J., Zhang, Q., Ouyang, X., Zhang, Z., Wang, H., & Huang, M. (2025). JPS: Jailbreak multimodal large language models with collaborative visual perturbation and textual steering. In *Proceedings of the 33rd ACM International Conference on Multimedia (MM '25)*. Association for Computing Machinery. https://dl.acm.org/doi/10.1145/3746027.3754561
- Chen, S., Piet, J., Sitawarin, C., & Wagner, D. (2025). StruQ: Defending against prompt injection with structured queries. In *Proceedings of the 34th USENIX Security Symposium (USENIX Security 25)*. USENIX Association. https://www.usenix.org/system/files/usenixsecurity25-chen-sizhe.pdf
- Clusmann, J., Ferber, D., Wiest, I. C., Schneider, C. V., Brinker, T. J., Foersch, S., Truhn, D., & Kather, J. N. (2025). Prompt injection attacks on vision language models in oncology. *Nature Communications, 16*, Article 1239. https://www.nature.com/articles/s41467-024-55631-x
- Cybersecurity and Infrastructure Security Agency, Federal Bureau of Investigation, National Security Agency, & Australian Cyber Security Centre. (2025, December). *Principles for the secure integration of AI in operational technology*. https://www.cisa.gov/sites/default/files/2025-12/joint-guidance-principles-for-the-secure-integration-of-artificial-intelligence-in-operational-technology-508c.pdf
- Debenedetti, E., Shumailov, I., Fan, T., Hayes, J., Carlini, N., Fabian, D., Kern, C., Shi, C., Terzis, A., & Tramèr, F. (2025). *Defeating prompt injections by design*. arXiv. https://arxiv.org/abs/2503.18813
- Debenedetti, E., Zhang, J., Balunović, M., Beurer-Kellner, L., Fischer, M., & Tramèr, F. (2024). *AgentDojo: A dynamic environment to evaluate prompt injection attacks and defenses for LLM agents*. arXiv. https://arxiv.org/abs/2406.13352
- General Analysis. (2025, July). *Supabase MCP can leak your entire SQL database* [Blog post]. https://generalanalysis.com/blog/supabase-mcp-blog
- Greshake, K. (2023). *Inject my PDF: Prompt injection for your resume* [Blog post]. https://kai-greshake.de/posts/inject-my-pdf
- Greshake, K., Abdelnabi, S., Mishra, S., Endres, C., Holz, T., & Fritz, M. (2023). *Not what you've signed up for: Compromising real-world LLM-integrated applications with indirect prompt injection*. arXiv. https://arxiv.org/abs/2302.12173
- Hackett, W., Birch, L., Trawicki, S., Suri, N., & Garraghan, P. (2025). *Bypassing LLM guardrails: An empirical analysis of evasion attacks against prompt injection and jailbreak detection systems*. arXiv. https://arxiv.org/abs/2504.11168
- INCIBE-CERT. (2024). *CVE-2024-5184: EmailGPT* [Vulnerability advisory]. https://www.incibe.es/en/incibe-cert/early-warning/vulnerabilities/cve-2024-5184
- Invariant Labs. (2025, May). *GitHub MCP server vulnerability* [Blog post]. https://invariantlabs.ai/blog/mcp-github-vulnerability
- Koi Security. (2025, September). *Postmark-MCP npm malicious backdoor: Email theft* [Blog post]. https://www.koi.ai/blog/postmark-mcp-npm-malicious-backdoor-email-theft
- Labunets, A., Pandya, N. V., Hooda, A., Fu, X., & Fernandes, E. (2025). *Fun-tuning: Characterizing the vulnerability of proprietary LLMs to optimization-based prompt injection attacks via the fine-tuning interface*. arXiv. https://arxiv.org/abs/2501.09798
- Meta AI. (2025, October 31). *Agents rule of two: A practical approach to AI agent security* [Blog post]. https://ai.meta.com/blog/practical-ai-agent-security/  `[also cited in LLM03]`
- Microsoft Research. (2025). *Defending against indirect prompt injection attacks with spotlighting*. https://www.microsoft.com/en-us/research/publication/defending-against-indirect-prompt-injection-attacks-with-spotlighting/
- Microsoft Security Response Center. (2025, March). *Announcing the winners of the Adaptive Prompt Injection Challenge — LLMail-Inject* [Blog post]. https://www.microsoft.com/en-us/msrc/blog/2025/03/announcing-the-winners-of-the-adaptive-prompt-injection-challenge-llmail-inject/
- MITRE. (n.d.). *AI agent context poisoning: Memory* (AML.T0080.000) [ATLAS technique]. MITRE ATLAS. Retrieved July 11, 2026, from https://atlas.mitre.org/techniques/AML.T0080.000
- Nasr, M., Carlini, N., Sitawarin, C., Schulhoff, S. V., Hayes, J., Ilie, M., Pluto, J., Song, S., Chaudhari, H., Shumailov, I., Thakurta, A., Xiao, K. Y., Terzis, A., & Tramèr, F. (2025). *The attacker moves second: Stronger adaptive attacks bypass defenses against LLM jailbreaks and prompt injections*. arXiv. https://arxiv.org/abs/2510.09023
- National Cyber Security Centre. (2025, December). *Prompt injection is not SQL injection* [Blog post]. https://www.ncsc.gov.uk/blog-post/prompt-injection-is-not-sql-injection
- National Institute of Standards and Technology. (2025). *Adversarial machine learning: A taxonomy and terminology of attacks and mitigations* (NIST AI 100-2e2025). https://csrc.nist.gov/pubs/ai/100/2/e2025/final
- National Institute of Standards and Technology. (n.d.). *CVE-2025-32711 detail*. National Vulnerability Database. Retrieved July 11, 2026, from https://nvd.nist.gov/vuln/detail/CVE-2025-32711
- Noma Security. (2025). *Why the Rule of Two can't protect your agents* [Blog post]. https://noma.security/blog/mcp-servers-agentic-risk-and-the-framework-that-protects-it/
- OWASP GenAI Security Project. (2025). *OWASP Top 10 for LLM Applications — LLM01:2025 Prompt injection*. https://genai.owasp.org/llmrisk/llm01-prompt-injection/
- Reddy, P., & Gujral, A. S. (2025). *EchoLeak: The first real-world zero-click prompt injection exploit in a production LLM system*. arXiv. https://arxiv.org/abs/2509.10540
- Rehberger, J. (2024, August). *M365 Copilot prompt injection, tool invocation and data exfil using ASCII smuggling* [Blog post]. Embrace The Red. https://embracethered.com/blog/posts/2024/m365-copilot-prompt-injection-tool-invocation-and-data-exfil-using-ascii-smuggling/
- Rehberger, J. (2025a, August). *GitHub Copilot: Remote code execution via prompt injection (CVE-2025-53773)* [Blog post]. Embrace The Red. https://embracethered.com/blog/posts/2025/github-copilot-remote-code-execution-via-prompt-injection/  `[also cited in LLM10]`
- Rehberger, J. (2025b, February). *Hacking Gemini's memory with prompt injection and delayed tool invocation* [Blog post]. Embrace The Red. https://embracethered.com/blog/posts/2025/google-gemini-memory-persistence-prompt-injection/
- Rehberger, J. (2025c). *Sneaky bits & ASCII smuggler updates* [Blog post]. Embrace The Red. https://embracethered.com/blog/posts/2025/sneaky-bits-and-ascii-smuggler/
- Toulas, B. (2025, September 25). *Unofficial Postmark MCP npm package silently stole users' emails*. BleepingComputer. https://www.bleepingcomputer.com/news/security/unofficial-postmark-mcp-npm-silently-stole-users-emails/
- Willison, S. (2025, June 16). *The lethal trifecta for AI agents: Private data, untrusted content, and external communication* [Blog post]. https://simonwillison.net/2025/Jun/16/the-lethal-trifecta/
- Zou, A., Wang, Z., Carlini, N., Nasr, M., Kolter, J. Z., & Fredrikson, M. (2023). *Universal and transferable adversarial attacks on aligned language models*. arXiv. https://arxiv.org/abs/2307.15043
- Zou, W., Geng, R., Wang, B., & Jia, J. (2025). PoisonedRAG: Knowledge corruption attacks to retrieval-augmented generation of large language models. In *Proceedings of the 34th USENIX Security Symposium (USENIX Security 25)*. USENIX Association. https://www.usenix.org/system/files/usenixsecurity25-zou-poisonedrag.pdf  `[also cited in LLM07, LLM09]`

## LLM02: Sensitive Information Disclosure

- BeyondScale. (2026, May). *Vector database security: RAG compliance & monitoring*. https://beyondscale.tech/blog/vector-database-security-rag-compliance-monitoring
- Boenisch, F., Dziedzic, A., Schuster, R., Shamsabadi, A. S., Shumailov, I., & Papernot, N. (2021). *When the curious abandon honesty: Federated learning is not private*. arXiv. https://arxiv.org/abs/2112.02918
- Carlini, N., Paleka, D., Dvijotham, K. D., Steinke, T., Hayase, J., Cooper, A. F., Lee, K., Jagielski, M., Nasr, M., Conmy, A., Yona, I., Wallace, E., Rolnick, D., & Tramèr, F. (2024). *Stealing part of a production language model*. arXiv. https://arxiv.org/abs/2403.06634  `[also cited in LLM06]`
- Check Point Research. (2026, February). *ChatGPT data leakage via a hidden outbound runtime channel*. https://research.checkpoint.com/2026/chatgpt-data-leakage-via-a-hidden-outbound-channel-in-the-code-execution-runtime
- Dong, T., Meng, Y., Li, S., Chen, G., Liu, Z., & Zhu, H. (2025). *Depth gives a false sense of privacy: LLM internal states inversion*. arXiv. https://arxiv.org/abs/2507.16372
- Fu, W., Wang, H., Gao, C., Liu, G., Li, Y., & Jiang, T. (2024). *Membership inference attacks against fine-tuned large language models via self-prompt calibration* [Conference poster]. Neural Information Processing Systems (NeurIPS 2024). https://neurips.cc/virtual/2024/poster/95327
- The Guardian. (2026, April 1). *Anthropic Claude Code source leak*. https://www.theguardian.com/technology/2026/apr/01/anthropic-claudes-code-leaks-ai
- InstaTunnel. (2026, February). *Differential privacy reversal via LLM feedback*. https://instatunnel.my/blog/differential-privacy-reversal-via-llm-feedback-the-silent-killer-of-data-anonymization
- Jiang, C., Pan, X., Hong, G., Bao, C., Chen, Y., & Yang, M. (2024). *Feedback-guided extraction of knowledge base from retrieval-augmented LLM applications*. arXiv. https://arxiv.org/abs/2411.14110
- Lakshmanan, R. (2026, March 26). *Claude extension flaw enabled zero-click XSS prompt injection via any website*. The Hacker News. https://thehackernews.com/2026/03/claude-extension-flaw-enabled-zero.html
- Lin, Y., Zhang, Q., Ruan, W., Zhang, D., Hong, J., Wu, Y., Xia, H., Mao, Y., & Zhong, S. (2026). *Towards privacy-preserving LLM inference via covariant obfuscation (Technical report)*. arXiv. https://arxiv.org/abs/2603.01499
- McDonald, G., & Bar Or, J. (2025). *Whisper Leak: A side-channel attack on large language models*. arXiv. https://arxiv.org/abs/2511.03675
- Nasr, M., Carlini, N., Hayase, J., Jagielski, M., Cooper, A. F., Ippolito, D., Choquette-Choo, C. A., Wallace, E., Tramèr, F., & Lee, K. (2023). *Scalable extraction of training data from (production) language models*. arXiv. https://arxiv.org/abs/2311.17035
- OWASP GenAI Security Project. (2026). *OWASP GenAI data security 2026* (Version 1.0). https://genai.owasp.org/
- Panda, A., Choquette-Choo, C. A., Zhang, Z., Yang, Y., & Mittal, P. (2024). *Teach LLMs to phish: Stealing private information from language models*. arXiv. https://arxiv.org/abs/2403.00871
- Unit 42. (2026, March). *Gemini Live in Chrome hijacking (CVE-2026-0628)*. Palo Alto Networks. https://unit42.paloaltonetworks.com/gemini-live-in-chrome-hijacking
- Weiss, R., Ayzenshteyn, D., Amit, G., & Mirsky, Y. (2024). *What was your prompt? A remote keylogging attack on AI assistants*. arXiv. https://arxiv.org/abs/2403.09751
- Wiz. (2025, January). *Wiz Research uncovers exposed DeepSeek database leak*. https://www.wiz.io/blog/wiz-research-uncovers-exposed-deepseek-database-leak
- Wu, G., Zhang, Z., Zhang, Y., Wang, W., Niu, J., Wu, Y., & Zhang, Y. (2025). *I know what you asked: Prompt leakage via KV-cache sharing in multi-tenant LLM serving* [Conference paper]. Network and Distributed System Security Symposium (NDSS 2025). https://www.ndss-symposium.org/ndss-paper/i-know-what-you-asked-prompt-leakage-via-kv-cache-sharing-in-multi-tenant-llm-serving/

## LLM03: Excessive Agency

- Bort, J. (2026, February 23). *A Meta AI security researcher said an OpenClaw agent ran amok on her inbox*. TechCrunch. https://techcrunch.com/2026/02/23/a-meta-ai-security-researcher-said-an-openclaw-agent-ran-amok-on-her-inbox/
- Ferreira, B. (2026, March 7). *Claude Code deletes developers' production setup, including its database and snapshots — 2.5 years of records were nuked in an instant*. Tom's Hardware. https://www.tomshardware.com/tech-industry/artificial-intelligence/claude-code-deletes-developers-production-setup-including-its-database-and-snapshots-2-5-years-of-records-were-nuked-in-an-instant
- Kovacs, E. (2026, April 1). *Google addresses Vertex security issues after researchers weaponize AI agents*. SecurityWeek. https://www.securityweek.com/google-addresses-vertex-security-issues-after-researchers-weaponize-ai-agent/
- Kundel, D. (2024, October 10). *Rogue agents: Stop AI from misusing your APIs*. Twilio. https://www.twilio.com/en-us/blog/rogue-ai-agents-secure-your-apis
- Lucas, J. (2024, December 16). *Sandboxing agentic AI workflows with WebAssembly*. NVIDIA Developer Blog. https://developer.nvidia.com/blog/sandboxing-agentic-ai-workflows-with-webassembly/
- Meta AI. (2025, October 31). *Agents rule of two: A practical approach to AI agent security* [Blog post]. https://ai.meta.com/blog/practical-ai-agent-security/  `[also cited in LLM01]`
- NVIDIA. (n.d.). *NeMo-Guardrails: Interface guidelines*. GitHub. https://github.com/NVIDIA/NeMo-Guardrails/blob/main/docs/security/guidelines.md
- Rehberger, J. (2023, May 28). *ChatGPT plugin exploit explained: From prompt injection to accessing private data*. Embrace The Red. https://embracethered.com/blog/posts/2023/chatgpt-cross-plugin-request-forgery-and-prompt-injection./

## LLM04: Supply Chain

- Cisco Talos. (2024). *llama.cpp GGUF library gguf_fread_str heap-based buffer overflow vulnerability (TALOS-2024-1913)*. https://talosintelligence.com/vulnerability_reports/TALOS-2024-1913
- Cohen, D. (2025, December 2). *PyTorch users at risk: Unveiling 3 zero-day PickleScan vulnerabilities*. JFrog. https://jfrog.com/blog/unveiling-3-zero-day-vulnerabilities-in-picklescan/
- Egashira, K., Staab, R., Vero, M., He, J., & Vechev, M. (2025). *Mind the gap: A practical attack on GGUF quantization* (arXiv:2505.23786). arXiv. https://arxiv.org/abs/2505.23786
- GitHub Security Advisories. (2025, April 17). *PyTorch torch.load weights_only bypass (CVE-2025-32434, GHSA-53q9-r3pm-6pq6)*. https://github.com/pytorch/pytorch/security/advisories/GHSA-53q9-r3pm-6pq6
- HiddenLayer. (2024, February 21). *Hijacking Safetensors conversion on Hugging Face*. https://hiddenlayer.com/research/silent-sabotage/
- Huynh, D., & Hardouin, J. (2023, July 9). *PoisonGPT: How we hid a lobotomized LLM on Hugging Face to spread fake news*. Mithril Security. https://blog.mithrilsecurity.io/poisongpt-how-we-hid-a-lobotomized-llm-on-hugging-face-to-spread-fake-news
- Lumelsky, A., & Elbaz, G. (2025, November 18). *ShadowRay 2.0: Attackers turn AI against itself in global campaign that hijacks AI into self-propagating botnet*. Oligo Security. https://www.oligo.security/blog/shadowray-2-0-attackers-turn-ai-against-itself-in-global-campaign-that-hijacks-ai-into-self-propagating-botnet
- MITRE. (n.d.). *AI supply chain compromise (AML.T0010)*. MITRE ATLAS. Retrieved July 11, 2026, from https://atlas.mitre.org/techniques/AML.T0010
- OASIS Open. (2025, November 18). *Coalition for Secure AI releases two actionable frameworks for AI model signing and incident response*. https://www.oasis-open.org/2025/11/18/coalition-for-secure-ai-releases-two-actionable-frameworks-for-ai-model-signing-and-incident-response/
- Open Source Security Foundation. (2025, April 4). *Launch of Model Signing v1.0: OpenSSF AI/ML Working Group secures the machine learning supply chain*. https://openssf.org/blog/2025/04/04/launch-of-model-signing-v1-0-openssf-ai-ml-working-group-secures-the-machine-learning-supply-chain/
- Open Source Security Foundation. (n.d.). *Supply-chain Levels for Software Artifacts (SLSA)*. Retrieved July 11, 2026, from https://slsa.dev/
- OWASP CycloneDX. (n.d.). *Machine learning bill of materials (AI/ML-BOM)*. Retrieved July 11, 2026, from https://cyclonedx.org/capabilities/mlbom/  `[also cited in LLM05]`
- OWASP GenAI Security Project. (2025, December 18). *Evolving AI transparency: The journey of the AIBOM Generator and its new home at OWASP*. https://genai.owasp.org/2025/12/18/evolving-ai-transparency-the-journey-of-the-aibom-generator-and-its-new-home-at-owasp/
- OWASP GenAI Security Project. (2026). *OWASP Top 10 for Agentic Applications (2026)*. https://genai.owasp.org/resource/owasp-top-10-for-agentic-applications-for-2026/
- Python Package Index. (2024, December 11). *Supply-chain attack analysis: Ultralytics*. PyPI Blog. https://blog.pypi.org/posts/2024-12-11-ultralytics-attack-analysis/
- PyTorch Foundation. (2022, December 31). *Compromised PyTorch-nightly dependency chain between December 25th and December 30th, 2022*. https://pytorch.org/blog/compromised-nightly-dependency/
- Saraf, I., & Balassiano, O. (2025, September 3). *Model namespace reuse: An AI supply-chain attack exploiting model name trust*. Unit 42, Palo Alto Networks. https://unit42.paloaltonetworks.com/model-namespace-reuse/
- Spracklen, J., Wijewickrama, R., Sakib, A. H. M. N., Maiti, A., Viswanath, B., & Jadliwala, M. (2025). We have a package for you! A comprehensive analysis of package hallucinations by code generating LLMs. In *Proceedings of the 34th USENIX Security Symposium (USENIX Security 25)*. USENIX Association. https://www.usenix.org/system/files/usenixsecurity25-spracklen.pdf  `[also cited in LLM07]`
- Tamari, S., & Tzadik, S. (2024, April 4). *Wiz Research finds architecture risks that may compromise AI-as-a-Service providers*. Wiz. https://www.wiz.io/blog/wiz-and-hugging-face-address-risks-to-ai-infrastructure
- Wickens, E., Schulz, K., & Bonner, T. (2024, October 10). *ShadowLogic: Persistent no-code backdoors in AI computational graphs*. HiddenLayer. https://hiddenlayer.com/innovation-hub/shadowlogic/
- Wiz. (2024). *Probllama: Ollama remote code execution vulnerability (CVE-2024-37032)*. https://www.wiz.io/blog/probllama-ollama-vulnerability-cve-2024-37032
- Zanki, K. (2025, February 6). *Malicious ML models discovered on Hugging Face platform (nullifAI)*. ReversingLabs. https://www.reversinglabs.com/blog/rl-identifies-malware-ml-model-hosted-on-hugging-face
- Zhan, Q., Fang, R., Bindu, R., Gupta, A., Hashimoto, T., & Kang, D. (2023). *Removing RLHF protections in GPT-4 via fine-tuning* (arXiv:2311.05553). arXiv. https://arxiv.org/abs/2311.05553

## LLM05: Data and Model Poisoning

- OWASP CycloneDX. (n.d.). *Machine learning bill of materials (AI/ML-BOM)*. Retrieved July 11, 2026, from https://cyclonedx.org/capabilities/mlbom/  `[also cited in LLM04]`
- DVC. (n.d.). *DVC: Data version control* [Documentation]. https://dvc.org/doc
- F5. (2025). *Data and model poisoning* [OWASP Top 10 for LLM applications guide (2025)]. https://my.f5.com/manage/s/article/K000149782
- Fogel, A., Hofman, O., Cohen, E., & Vainshtein, R. (2026). *Inference-time backdoors via chat templates: From LLM supply chains to agentic system compromise* (arXiv:2602.04653). arXiv. https://arxiv.org/abs/2602.04653
- GitHub. (2026, March 26). *Giskard Agents have server-side template injection via ChatWorkflow.chat() using non-sandboxed Jinja2 Environment* (GHSA-frv4-x25r-588m). GitHub Advisory Database. https://github.com/advisories/GHSA-frv4-x25r-588m
- Hubinger, E., Denison, C., Mu, J., Lambert, M., Tong, M., MacDiarmid, M., Lanham, T., Ziegler, D. M., Maxwell, T., Cheng, N., Jermyn, A., Askell, A., Radhakrishnan, A., Anil, C., Duvenaud, D., Ganguli, D., Barez, F., Clark, J., Ndousse, K., … Perez, E. (2024). *Sleeper agents: Training deceptive LLMs that persist through safety training* (arXiv:2401.05566). arXiv. https://arxiv.org/abs/2401.05566
- JFrog Security Research. (n.d.). *GGUF-SSTI*. https://research.jfrog.com/model-threats/gguf-ssti/
- MITRE. (n.d.). *MITRE ATLAS: Adversarial threat landscape for artificial-intelligence systems*. https://atlas.mitre.org
- National Institute of Standards and Technology. (2023). *Artificial intelligence risk management framework (AI RMF 1.0)* (NIST AI 100-1). https://www.nist.gov/itl/ai-risk-management-framework
- Souly, A., Rando, J., Chapman, E., Davies, X., Hasircioglu, B., Shereen, E., Mougan, C., Mavroudis, V., Jones, E., Hicks, C., Carlini, N., Gal, Y., & Kirk, R. (2025). *Poisoning attacks on LLMs require a near-constant number of poison samples* (arXiv:2510.07192). arXiv. https://arxiv.org/abs/2510.07192
- Zhang, B., Chen, Y., Liu, Z., Nie, L., Li, T., Liu, Z., & Fang, M. (2025). *Practical poisoning attacks against retrieval-augmented generation* (arXiv:2504.03957). arXiv. https://arxiv.org/abs/2504.03957

## LLM06: Unbounded Consumption

- Akkus, A. (2024, December 27). *You wouldn't download an AI: Extracting AI models from mobile apps*. Altay's Blog. https://altayakkus.substack.com/p/you-wouldnt-download-an-ai
- Anthropic. (n.d.). *Vision* [Build with Claude documentation]. Retrieved July 11, 2026, from https://docs.anthropic.com/en/docs/build-with-claude/vision
- Awan, A. A. (2023, March 24). *How watermarking can help mitigate the potential risks of LLMs?* KDnuggets. https://www.kdnuggets.com/2023/03/watermarking-help-mitigate-potential-risks-llms.html
- Carlini, N., Paleka, D., Dvijotham, K. D., Steinke, T., Hayase, J., Cooper, A. F., Lee, K., Jagielski, M., Nasr, M., Conmy, A., Yona, I., Wallace, E., Rolnick, D., & Tramèr, F. (2024). *Stealing part of a production language model* (arXiv:2403.06634). arXiv. https://arxiv.org/abs/2403.06634  `[also cited in LLM02]`
- DeepLearning.AI. (2023, March 15). *Runaway LLaMA: How Meta's LLaMA NLP model leaked*. The Batch. https://www.deeplearning.ai/the-batch/how-metas-llama-nlp-model-leaked/
- Dong, B., Feng, H., & Wang, Q. (2026). *Clawdrain: Exploiting tool-calling chains for stealthy token exhaustion in OpenClaw agents* (arXiv:2603.00902). arXiv. https://arxiv.org/abs/2603.00902
- Gao, H., Zhang, Y., Zhou, Z., Jiang, L., Meng, F., Xiao, Y., Sun, L., Wang, K., Liu, Y., & Feng, J. (2025). *Resource consumption red-teaming for large vision-language models* (arXiv:2507.18053). arXiv. https://arxiv.org/abs/2507.18053
- Gao, K., Pang, T., Du, C., Yang, Y., Xia, S.-T., & Lin, M. (2024). *Denial-of-service poisoning attacks against large language models* (arXiv:2410.10760). arXiv. https://arxiv.org/abs/2410.10760
- Jiang, W., Li, H., Xu, G., Zhang, T., & Lu, R. (2024). A comprehensive defense framework against model extraction attacks. *IEEE Transactions on Dependable and Secure Computing*, *21*(2), 685–700. https://ieeexplore.ieee.org/document/10080996
- Li, Y., Wang, J., Zhu, H., Lin, J., Chang, S., & Guo, M. (2025). *ThinkTrap: Denial-of-service attacks against black-box LLM services via infinite thinking* (arXiv:2512.07086). arXiv. https://arxiv.org/abs/2512.07086
- moohax, & monoxgas. (2023, March 31). *Proof Pudding (CVE-2019-20634)* (AVID-2023-V009). AI Vulnerability Database. https://avidml.org/database/avid-2023-v009/
- Nevo, S., Lahav, D., Karpur, A., Bar-On, Y., Bradley, H. A., & Alstott, J. (2024). *Securing AI model weights: Preventing theft and misuse of frontier models* (RR-A2849-1). RAND Corporation. https://www.rand.org/content/dam/rand/pubs/research_reports/RRA2800/RRA2849-1/RAND_RRA2849-1.pdf
- Radosevich, B., & Halloran, J. (2025). *MCP safety audit: LLMs with the Model Context Protocol allow major security exploits* (arXiv:2504.03767). arXiv. https://arxiv.org/abs/2504.03767
- Sev, C. (2023, August 30). *Security update: Incident involving unauthorized admin access*. Sourcegraph. https://about.sourcegraph.com/blog/security-update-august-2023
- Shumailov, I., Zhao, Y., Bates, D., Papernot, N., Mullins, R., & Anderson, R. (2020). *Sponge examples: Energy-latency attacks on neural networks* (arXiv:2006.03463). arXiv. https://arxiv.org/abs/2006.03463
- Taori, R., Gulrajani, I., Zhang, T., Dubois, Y., Li, X., Guestrin, C., Liang, P., & Hashimoto, T. B. (2023, March 13). *Alpaca: A strong, replicable instruction-following model*. Stanford Center for Research on Foundation Models. https://crfm.stanford.edu/2023/03/13/alpaca.html

## LLM07: Misinformation

- National Institute of Standards and Technology. (2024). *Artificial intelligence risk management framework: Generative artificial intelligence profile* (NIST AI 600-1). U.S. Department of Commerce. https://nvlpubs.nist.gov/nistpubs/ai/NIST.AI.600-1.pdf
- OpenAI. (2025, September 5). *Why language models hallucinate*. https://openai.com/index/why-language-models-hallucinate/
- OWASP Gen AI Security Project. (2025). *LLM09:2025 Misinformation*. https://genai.owasp.org/llmrisk/llm092025-misinformation/
- Spracklen, J., Wijewickrama, R., Sakib, A. H. M. N., Maiti, A., Viswanath, B., & Jadliwala, M. (2025). We have a package for you! A comprehensive analysis of package hallucinations by code generating LLMs. In *Proceedings of the 34th USENIX Security Symposium (USENIX Security 25)*. USENIX Association. https://www.usenix.org/system/files/usenixsecurity25-spracklen.pdf  `[also cited in LLM04]`
- Zou, W., Geng, R., Wang, B., & Jia, J. (2025). PoisonedRAG: Knowledge corruption attacks to retrieval-augmented generation of large language models. In *Proceedings of the 34th USENIX Security Symposium (USENIX Security 25)*. USENIX Association. https://www.usenix.org/system/files/usenixsecurity25-zou-poisonedrag.pdf  `[also cited in LLM01, LLM09]`

## LLM08: Hidden Context Exposure

- Cao, B., Li, C., Cao, Y., Ge, Y., Wang, T., & Chen, J. (2025). *You can't steal nothing: Mitigating prompt leakages in LLMs via system vectors*. arXiv. https://arxiv.org/abs/2509.21884
- Das, B. C., Amini, M. H., & Wu, Y. (2025). *System prompt extraction attacks and defenses in large language models*. arXiv. https://arxiv.org/abs/2505.23817
- Goffin, J. (2025, July). *Proof of concept: Dangers of system prompt leakage*. Resk. https://resk.fr/pdf/resk-enterprise-ai-security-guide.pdf
- Hui, B., Yuan, H., Gong, N., Burlina, P., & Cao, Y. (2024). *PLeak: Prompt leaking attacks against large language model applications*. arXiv. https://arxiv.org/abs/2405.06823
- jujumilk3. (2025). *Leaked system prompts* [GitHub repository]. GitHub. https://github.com/jujumilk3/leaked-system-prompts
- LeakHub. (n.d.). *LeakHub* [Website]. Retrieved July 11, 2026, from https://leakhub.ai/
- Li, Z., Guo, J., & Cai, H. (2025). *System prompt poisoning: Persistent attacks on large language models beyond user injection*. arXiv. https://arxiv.org/abs/2505.06493
- Nie, Y., Wang, Z., Yu, Y., Wu, X., Zhao, X., Guo, W., & Song, D. (2024). *LeakAgent: RL-based red-teaming agent for LLM privacy leakage*. arXiv. https://arxiv.org/abs/2412.05734
- Zheng, X., Wu, Y., Huang, H., Li, Y., Ma, X., Li, B., Jiang, Y.-G., & Wang, C. (2026). *Just ask: Curious code agents reveal system prompts in frontier LLMs*. arXiv. https://doi.org/10.48550/arXiv.2601.21233

## LLM09: Vector and Embedding Weaknesses

- Chaudhari, H., Severi, G., Abascal, J., Suri, A., Jagielski, M., Choquette-Choo, C. A., Nasr, M., Nita-Rotaru, C., & Oprea, A. (2024). *Phantom: General backdoor attacks on retrieval augmented language generation* (arXiv:2405.20485). arXiv. https://arxiv.org/abs/2405.20485
- Chen, Y., Xu, Q., & Bjerva, J. (2025). ALGEN: Few-shot inversion attacks on textual embeddings via cross-model alignment and generation. In *Proceedings of the 63rd Annual Meeting of the Association for Computational Linguistics (Volume 1: Long Papers)*. Association for Computational Linguistics. https://aclanthology.org/2025.acl-long.1185/
- Chen, Z., Xiang, Z., Xiao, C., Song, D., & Li, B. (2024). *AgentPoison: Red-teaming LLM agents via poisoning memory or knowledge bases* (arXiv:2407.12784). arXiv. https://arxiv.org/abs/2407.12784
- Ha, H., Zhan, Q., Kim, J., Bralios, D., Sanniboina, S., Peng, N., Chang, K.-W., Kang, D., & Ji, H. (2025). *MM-PoisonRAG: Disrupting multimodal RAG with local and global poisoning attacks* (arXiv:2502.17832). arXiv. https://arxiv.org/abs/2502.17832
- Kim, D., Kang, D., Lee, K., Baek, H., & Kang, B. B. (2026). *Zero2Text: Zero-training cross-domain inversion attacks on textual embeddings* (arXiv:2602.01757). arXiv. https://arxiv.org/abs/2602.01757
- Li, H., Xu, M., & Song, Y. (2023). *Sentence embedding leaks more information than you expect: Generative embedding inversion attack to recover the whole sentence* (arXiv:2305.03010). arXiv. https://arxiv.org/abs/2305.03010
- Liu, Y., Yuan, Z., Tie, G., Shi, J., Zhou, P., Sun, L., & Gong, N. Z. (2025). *Poisoned-MRAG: Knowledge poisoning attacks to multimodal retrieval augmented generation* (arXiv:2503.06254). arXiv. https://arxiv.org/abs/2503.06254
- MITRE. (n.d.). *RAG poisoning* (AML.T0070) [Technique]. MITRE ATLAS. https://atlas.mitre.org/techniques/AML.T0070
- Morris, J. X., Kuleshov, V., Shmatikov, V., & Rush, A. M. (2023). *Text embeddings reveal (almost) as much as text* (arXiv:2310.06816). arXiv. https://arxiv.org/abs/2310.06816
- Shafran, A., Schuster, R., & Shmatikov, V. (2025). Machine against the RAG: Jamming retrieval-augmented generation with blocker documents. In *34th USENIX Security Symposium (USENIX Security 25)*. USENIX Association. https://www.usenix.org/conference/usenixsecurity25/presentation/shafran
- Shereen, E., Ristea, D., McFadden, S., Hasircioglu, B., Mavroudis, V., & Hicks, C. (2025). *One pic is all it takes: Poisoning visual document retrieval augmented generation with a single image* (arXiv:2504.02132). arXiv. https://arxiv.org/abs/2504.02132
- Song, C., & Raghunathan, A. (2020). *Information leakage in embedding models* (arXiv:2004.00053). arXiv. https://arxiv.org/abs/2004.00053
- Tan, X., Luan, H., Luo, M., Sun, X., Chen, P., & Dai, J. (2025). RevPRAG: Revealing poisoning attacks in retrieval-augmented generation through LLM activation analysis. In *Findings of the Association for Computational Linguistics: EMNLP 2025*. Association for Computational Linguistics. https://aclanthology.org/2025.findings-emnlp.698/
- Wang, F., Wan, X., Sun, R., Chen, J., & Arık, S. Ö. (2024). *Astute RAG: Overcoming imperfect retrieval augmentation and knowledge conflicts for large language models* (arXiv:2410.07176). arXiv. https://arxiv.org/abs/2410.07176
- Wu, G., Wang, T., Zhang, Y., Zhang, Z., Niu, J., Wu, Y., & Zhang, Y. (2026). When cache poisoning meets LLM systems: Semantic cache poisoning and its countermeasures. In *Proceedings of the 2026 Network and Distributed System Security Symposium (NDSS)*. Internet Society. https://www.ndss-symposium.org/ndss-paper/when-cache-poisoning-meets-llm-systems-semantic-cache-poisoning-and-its-countermeasures/
- Xue, J., Zheng, M., Hu, Y., Liu, F., Chen, X., & Lou, Q. (2024). *BadRAG: Identifying vulnerabilities in retrieval augmented generation of large language models* (arXiv:2406.00083). arXiv. https://arxiv.org/abs/2406.00083
- Zhang, C., Morris, J. X., & Shmatikov, V. (2025). *Universal zero-shot embedding inversion* (arXiv:2504.00147). arXiv. https://arxiv.org/abs/2504.00147
- Zhang, Z., Liu, Z., Xie, Y., Huang, Q., & She, D. (2026). *From similarity to vulnerability: Key collision attack on LLM semantic caching* (arXiv:2601.23088). arXiv. https://arxiv.org/abs/2601.23088
- Zou, W., Geng, R., Wang, B., & Jia, J. (2025). PoisonedRAG: Knowledge corruption attacks to retrieval-augmented generation of large language models. In *Proceedings of the 34th USENIX Security Symposium (USENIX Security 25)*. USENIX Association. https://www.usenix.org/system/files/usenixsecurity25-zou-poisonedrag.pdf  `[also cited in LLM01, LLM07]`

## LLM10: Improper Output Handling

- diskordia. (2025, July 24). *Inside CVE-2025-32711 (EchoLeak): Prompt injection meets AI exfiltration*. Hack The Box. https://www.hackthebox.com/blog/cve-2025-32711-echoleak-copilot-vulnerability
- Klondike, G. (2023, June 6). *Threat modeling LLM applications*. AI Village. https://aivillage.org/blog/threat-modeling-llm/
- The MITRE Corporation. (2026). *CWE-116: Improper encoding or escaping of output* (Common Weakness Enumeration List Version 4.20). https://cwe.mitre.org/data/definitions/116.html
- OWASP. (2026). *C7: Model behavior, output control and safety assurance* (AI Security Verification Standard Version 1.0). https://github.com/OWASP/AISVS/blob/main/1.0/en/0x10-C07-Model-Behavior.md
- OWASP. (n.d.). *V5: Validation, sanitization and encoding* (Application Security Verification Standard 4.0). Retrieved July 11, 2026, from https://owasp-aasvs4.readthedocs.io/en/latest/V5.html#validation-sanitization-and-encoding
- Porat, Y. (2025, December 25). *All I want for Christmas is your secrets: LangGrinch hits LangChain Core (CVE-2025-68664)*. Cyata. https://cyata.ai/blog/langgrinch-langchain-core-cve-2025-68664/
- PortSwigger. (n.d.). *Lab: Exploiting insecure output handling in LLMs*. Web Security Academy. Retrieved July 11, 2026, from https://portswigger.net/web-security/llm-attacks/lab-exploiting-insecure-output-handling-in-llms
- Rehberger, J. [wunderwuzzi]. (2024a, June 14). *GitHub Copilot Chat: From prompt injection to data exfiltration*. Embrace The Red. https://embracethered.com/blog/posts/2024/github-copilot-chat-prompt-injection-data-exfiltration/
- Rehberger, J. [wunderwuzzi]. (2024b, December 6). *Terminal DiLLMa: LLM-powered apps can hijack your terminal via prompt injection*. Embrace The Red. https://embracethered.com/blog/posts/2024/terminal-dillmas-prompt-injection-ansi-sequences/
- Rehberger, J. [wunderwuzzi]. (2025, August 12). *GitHub Copilot: Remote code execution via prompt injection (CVE-2025-53773)*. Embrace The Red. https://embracethered.com/blog/posts/2025/github-copilot-remote-code-execution-via-prompt-injection/  `[also cited in LLM01]`
- Willison, S. (n.d.). *Markdown exfiltration* [Tag archive]. Simon Willison's Weblog. Retrieved July 11, 2026, from https://simonwillison.net/tags/markdown-exfiltration/