AI chatbots can shift political opinions ^{CEFR B2}

A study in PNAS Nexus shows that ordinary AI chatbot summaries can shift social and political opinions through subtle framing. The authors argue these effects arise both from latent biases in model training and from the specific prompts used to produce responses. As Daniel Karell notes, "We show that querying an AI chatbot to obtain historical facts can influence people’s opinions even when the information provided is accurate and nobody has prompted the tool to try to persuade you of anything."

The research tested two 20th-century cases: the Seattle General Strike, a five-day stoppage in February 1919, and the Third World Liberation Front student protests in 1968. In an experiment with 1,912 participants, people read either default GPT-4o summaries, the corresponding Wikipedia entries, or summaries the team prompted to adopt liberal or conservative framing.

Results showed that default AI summaries and liberal prompts pushed responses in a more liberal direction compared with Wikipedia, while conservative prompts produced more conservative reactions relative to Wikipedia. The liberal shift in default summaries points to latent bias in large language models; conservative effects appear to arise mainly from prompting. The researchers warn that, unlike Wikipedia’s transparent editing, chatbot development is opaque and can allow companies to shape public opinion. Additional coauthors are from Yale and Rutgers University.