Recent advances in Large Language Models (LLMs) have sparked concerns over their potential to acquire and misuse dangerous capabilities, posing frontier risks to society. Current safety evaluations primarily test for what a model can do---its capabilities---without assessing what it would do if endowed with high-risk capabilities. This leaves a critical blind spot: models may strategically conceal capabilities or rapidly acquire them, while harboring latent inclinations toward misuse. We argue that propensity---the likelihood of a model to pursue harmful actions if empowered---is a critical, yet underexplored, axis of safety evaluation. We present PropensityBench, a novel benchmark framework that assesses the proclivity of models to engage in risky behaviors when equipped with simulated dangerous capabilities using proxy tools. Our framework spans four high-risk domains: self-proliferation, cybersecurity, chemical security, and biosecurity. We simulate access to powerful capabilities via a controlled agentic environment and evaluate the models' choices under varying operational pressures that reflect real-world constraints or incentives models may encounter, such as resource scarcity or gaining more autonomy. Across open-source and proprietary frontier models, we uncover alarming signs of propensity: models frequently choose high-risk tools when under pressure, despite lacking the capability to execute such actions unaided. These findings call for a shift from static capability audits toward dynamic propensity assessments as a prerequisite for deploying frontier AI systems safely.
We start with the seed configuration that provides the domain, environment, agent's role, and the risk category information to the scenario generation pipeline. In Stage 1, seed configuration is fed to a reasoning LLM to synthesize high-level scenario outlines with both aligned and misaligned approaches. In Stage 2, these outlines are operationalized into concrete tool definitions, refined task and consequence descriptions, and JSON-structured prompts, yielding fully specified scenarios ready for evaluation and easy extension to novel risk domains at scale.
End-to-end scenario simulation pipeline showing how static pressure messages at ten escalating intensity levels are interleaved with agent reasoning and tool-call interactions, terminating either upon simulation end or upon activation of the misaligned trigger.
(a) Taxonomy of dangerous capabilities in the four risk domains used in our work (b) The six pressure tactics and the highest and lower pressure in each.
@article{sehwag25PropensityBench,
title={PropensityBench: Evaluating Latent Safety Risks in Large Language Models via an Agentic Approach},
author={Sehwag, Udari Madhushani and Shabihi, Shayan and McAvoy, Alex and Sehwag, Vikash and
Xu, Yuancheng and Towers, Dalton and Huang, Furong}
journal={under submission},
year={2025}
}