LARGE LANGUAGE MODEL POLLUTION IN ONLINE CONFLICT RESEARCH

Abstract: The integration of Large Language Models (LLMs) into online survey platforms has precipitated an epistemic crisis in conflict research, which depends heavily on online survey methods. With the advancement of browser-based AI Agents in these LLMs, a new class of synthetic respondents have emerged that can easily evade traditional bot detection methods. At the time of writing, major associations and outlets have not yet provided guidance to reviewers and authors on reporting standards that suggest mitigation strategies (e.g., APA’s Journal Article Reporting Standards, the Journal of Applied Psychology’s methodological checklist). In this novel session, we aim to (a) demonstrate the capabilities of these new AI agents, (b) begin a conversation with a panel on the nature and scope of the threat to psychological research, reporting standards, strategies for mitigation, and field-wide solutions, and (c) small group discussions with participants. Participants will leave with a clearer understanding of emerging threats to online data collections, concrete mitigation strategies they can implement immediately, and a shared vocabulary for engaging editors, reviewers, and platforms on this issue.

Keywords: LLM; Research Methods; Online Data Collection