paper3 min
Theory of Mind and Persuasion Beyond Conversation: Assessing the Capacity of LLMs to Induce Belief States via Planning and Action
A theory-of-mind benchmark that drops the question-answering format: agents have to induce a target belief in another character by moving objects and directing people, not by talking. Across six frontier models and a human cohort on 600 tasks, GPT-5 succeeds on about 80% and is the only model to beat humans, though it stays less robust as the context changes.
Ben Slater (Cambridge), Matteo G. Mecattaf (Cambridge), Lucy G. Cheke (Cambridge),
John Burden (Prolific), Winnie Street (Google)
Almost every theory-of-mind benchmark for language models uses the same shape: read a short story, then answer a question about who believes what. That format measures whether a model can report a mental state, but it says little about an agent that has to change what someone else believes as part of getting a job done. This paper evaluates that second capability directly and names it Non-Conversational Planning Theory of Mind: inducing a specific belief in another agent by acting on the world rather than talking them into it.
The framework, NCP-ExploreToM, inverts the usual task. Rather than asking a model what a character believes, it hands the model a target belief state and a small manipulable world, then requires it to move objects between rooms and direct characters around until the target ends up believing the intended thing. Six frontier models, including GPT-5, Gemini 2.5 Pro, and the Claude 4 series, and a cohort of human participants worked through 600 task instances.
GPT-5 solved roughly 80% of the tasks in the agentic setting and was the only model to outperform the human participants, though it remained less robust than humans as the context varied. The result with the clearest safety reading concerns direction: every model, like the human cohort, did better when the goal was to instil a true belief than a false one. Inducing a false belief, the manipulation-shaped case, is the harder skill for current models, which is a modest piece of reassurance rather than a worry.
That shift is why the task design matters. As agents get better at rearranging the world to their ends, part of what they are doing is arranging what other agents conclude, and a story-and-question benchmark cannot see it. GPT-5 already outperforms the human cohort here, so the capability is not hypothetical. Measuring it, and watching the gap between true and false beliefs as models improve, needs evaluations where the model has to act.
Accepted to the LM4Plan workshop at ICML 2026. A collaboration with the Leverhulme Centre for the Future of Intelligence at the University of Cambridge and Google's Paradigms of Intelligence team.
Abstract
Theory of Mind (ToM) benchmarks for Large Language Models (LLMs) typically rely on passive question-answering formats, but the deployment of LLMs in increasingly agentic and autonomous forms demands new evaluations. In this paper we evaluate an agent's ability to induce specific belief states in other agents by taking actions rather than using conversational persuasion, a capability we call Non-Conversational Planning ToM (NCP-ToM). NCP-ToM is likely to be essential for many agent use-cases, including within user-assistant interactions and pedagogical contexts, but may also present manipulation or misinformation risks. Using a novel framework, NCP-ExploreToM, we subvert the conventional task structure by providing models with a set of belief state goals and requiring them to move objects or direct characters into rooms to achieve their goals. We evaluated six frontier models, including GPT-5, Gemini 2.5 Pro and the Claude 4 series, and a cohort of human participants, across 600 task instances. GPT-5 was successful on approximately 80% of tasks in the agentic setting, and was the only model to outperform human participants on our task, but was still less robust than humans across contexts. We additionally found that all models, like humans, performed better on tasks inducing true belief states than false belief states, which is a positive signal for alignment efforts. These findings highlight emerging social-reasoning capabilities in LLMs for non-conversational task completion and underscore the necessity of agentic evaluations for understanding the safety and alignment of autonomous social agents.