UNESCO Report Warns AI Models Tend to Agree With Users Over Climate Facts

AI models frequently abandon factual accuracy when users challenge them, according to a report by UNESCO and University College London (UCL) presented at the COP31 summit in Antalya, Türkiye. Researchers found that while models answer approximately 90 per cent of neutral climate questions correctly in English, they often succumb to user pressure or plausible-sounding fabrications. In eight out of 10 cases, the models changed their stance at the first sign of mild disagreement, including confirming false claims that CO2 is not the primary driver of climate change.

The study, which tested up to 33 leading models across 13 languages and 656 climate-policy questions, identified a fundamental design flaw in how these systems are trained. Because models are rewarded during training for providing polite, non-confrontational, and validating responses, they exhibit sycophantic behavior intended to keep users engaged. Leona Verdadero, UNESCO’s lead author of the report, noted that while AI has the potential to make complex science accessible, "models have a design flaw – they are incentivised to agree with their user."

This tendency to please users makes AI an unreliable tool for developing robust climate policies. When asked to assess whether new evidence supported or contradicted accepted climate science, even the strongest models averaged only 75 per cent accuracy. Performance dropped significantly when models were asked to evaluate how different policies work together, where they scored an average of 33.8 per cent—only slightly higher than the 33.3 per cent expected from random guessing.

Accuracy also varied significantly by language. While models performed well in English, Spanish, French, and Portuguese, accuracy fell to 83.6 per cent in Hindi and Bengali, 81.4 per cent in Urdu, and 78.3 per cent in Swahili. The researchers noted that speakers of these lower-scoring languages often live in regions disproportionately affected by climate change. The report, commissioned as part of UNESCO’s Global Initiative for Information Integrity on Climate Change, warned policymakers and researchers against relying on these models for policy development.

Researchers found that the models' fluency with existing knowledge can create a false sense of capability, according to UCL senior author Dr Maria Perez-Ortiz. This issue is compounded by the increasing sophistication of climate disinformation, which has shifted from denying climate change exists to suggesting it is not human-caused, that technology alone will solve the crisis, or that action is too costly or late to be effective.

FI Journal
Privacy Overview

This website uses cookies so that we can provide you with the best user experience possible. Cookie information is stored in your browser and performs functions such as recognising you when you return to our website and helping our team to understand which sections of the website you find most interesting and useful.