LaTeX editing, your library, and agents that actually read the literature — in one window. This is the real interface: click around.
Large language models increasingly succeed on tasks once thought to require social reasoning, yet their performance on false-belief tasks remains brittle. We hypothesize that much of this brittleness stems from a failure to anchor reasoning to a specific agent's knowledge state, rather than from a lack of underlying capability.
We test this hypothesis with a lightweight intervention that asks the model to restate what each character can and cannot observe before answering. The result is a large, consistent improvement that requires no additional parameters.
Given a scenario and a question about agent a, we prompt the model to enumerate the events a witnessed, mark events hidden from a, and only then resolve the query conditioned on a's partial observation. This decomposition mirrors the sub-steps humans perform implicitly.
End-to-end prompting conflates the beliefs of different characters, causing errors that compound as stories grow longer. We argue that an explicit, symbolic representation of "who knows what" is both interpretable and more robust than relying on the model's latent state.
For each character we instantiate a graph whose nodes are propositions and whose edges encode observation events. Queries are answered by reading off the relevant character's sub-graph, sidestepping the interference that plagues monolithic prompting.
LLMs such as GPT-4 seemingly excel at tracking characters’ beliefs in stories, yet struggle to translate this capability into strategic action. The core challenge lies in identifying the implicit inferences about mental states — without being explicitly asked, as in ToMi — that lead to choosing the correct action.
The foresee step prompts the model to anticipate likely future events and the challenges facing each character; the reflect step reasons over candidate actions against those futures before committing. FaR generalizes to diverse out-of-distribution story structures and scenarios.
Existing benchmarks present a story and ask a question about a character. Real theory of mind is exercised while conversing: participants join and leave, so beliefs diverge continuously. FANToM generates multiparty conversations in which a participant steps away and misses a span of dialogue.
Each belief is probed with free-response, multiple-choice, and list variants; a model scores only when all forms agree. This penalizes the illusory competence that single-format evaluations reward.
We use the Sally–Anne unexpected-transfer task and the Smarties unexpected-contents task, each with carefully constructed controls to rule out shallow lexical shortcuts. Items are presented one sentence at a time to probe incremental belief updates.
Emergence of ToM-like behavior may be an unintended by-product of training on human-generated text. We caution against both over-attribution of mental states and premature dismissal, and call for adversarial benchmarks that separate the two.