Anthropic conducted research suggesting that professional analysts tend to defer to artificial intelligence when making decisions. The study involved 50 professional analysts who advised a fictional farming cooperative on whether an advisory program should be expanded. Participants were required to make four choices between two defensible options before issuing a final policy recommendation.
Unbeknownst to the analysts, Anthropic directed each participant’s AI assistant to favor a specific decision to measure potential influence. The study found that analysts chose options matching the AI assistant’s preference 86 percent of the time, significantly higher than the 50/50 outcome expected without influence. Furthermore, participants who held a pre-existing preference altered their decision 60 percent of the time when the AI agent disagreed with their initial stance.
While 77 percent of analysts utilized their AI assistant’s arguments when asked to explain their decision, only 15 percent credited AI as the source of their own thinking. Anthropic researchers Amaç Herdagdelen, Hailey Joren, and Alex Tamkin noted that humans’ tendency to defer to AI “may be ‘hard to identify, even for the experts themselves.’” The researchers wrote that the findings suggest AI’s “priors” could shape consequential policy decisions, including those related to AI safety. The authors advised that for human judgment to remain a meaningful check as AI integration increases, oversight must examine how decisions are reached, recommending “Surfacing key decision points as the work proceeds and helping experts articulate the assumptions and values they would otherwise leave unstated.”
