Recognition, Simulation, and Refusal: A Contamination-Aware Study of Classic Psychological Effects in LLM Agents
Not provided in the abstract
Abstract
The study presents PsyAgentBench, a benchmark that investigates how LLM agents exhibit human psychological effects under various experimental conditions.
Reality Card
The research demonstrates that LLM agents can exhibit human-like psychological effects through different mechanisms, challenging the notion of a single susceptibility to bias.
The study involved 41,904 trials across five paradigms, revealing significant variations in bias susceptibility based on experimental framing.
The findings highlight issues such as persona dominance and safety selection that may affect the reproducibility of psychological effects in LLMs.
Paper to code
Verified implementation resources so builders can test the paper’s claims instead of stopping at the abstract.