This paper highlights that the persuasive power of large-scale language models (LLMs) poses both beneficial applications (e.g., smoking cessation support) and significant risks (e.g., large-scale targeted political manipulation). Existing research has found significant and increasing persuasive power of models by measuring belief changes in simulated or real users. However, these benchmarks overlook a significant risk factor: the tendency of models to attempt persuasion in harmful contexts. Understanding whether a model will blindly "follow" a command to persuade on a harmful topic, such as glorifying terrorist affiliation, is crucial for understanding the effectiveness of safeguards. Furthermore, understanding when a model engages in persuasive behavior to pursue a specific goal is essential for understanding the risks of agent AI systems. Therefore, this paper proposes the Attempt to Persuade Evaluation (APE) benchmark, which focuses on persuasion attempts rather than persuasion success. This benchmark measures a model's willingness to generate content aimed at shaping beliefs or behaviors. APE examines state-of-the-art LLMs using a multi-round dialogue setting between simulated persuaders and persuaded agents. We explore a variety of topics, including conspiracies, controversial issues, and non-controversial harmful content, and introduce an automated assessment model to identify willingness to persuade and measure the frequency and context of persuasion attempts. We find that multiple open- and closed-weighted models frequently indicate willingness to attempt persuasion on harmful topics, and that jailbreaking can increase willingness to engage in such behavior. These results highlight gaps in current safeguards and emphasize the importance of assessing willingness to persuade as a key dimension of LLM risk. APE is available under github.com/AlignmentResearch/AttemptPersuadeEval에서.
