Experts Adopt Arbitrary AI Views, Believing Them Their Own
A study by Anthropic shows that 50 experts adopted the judgments of an AI agent whose preferences had been set by coin flip, and mostly considered the conclusions to be their own.
Experts often follow the judgments of an artificial agent even when its preferences were set completely arbitrarily, and then regard these views as their own. This is the finding of a study by the AI company Anthropic, in which 50 experts worked with an AI agent whose preferences had previously been determined by a coin flip.
In decisions requiring judgment, the participants adopted the agent's view in up to 86 percent of cases. Most of them were convinced that the conclusions came from their own reasoning, although they were in fact following the system's arbitrary default.
The results raise questions about how strongly people can be influenced by AI systems without noticing it. If experts unknowingly adopt the assessments of a tool, this could affect research, consulting and decision-making processes in which AI is increasingly used.
The finding underlines that transparency about the default settings of AI systems matters. As long as users do not recognize that a model has a particular tendency, it is difficult to critically examine its recommendations. The study suggests that expertise alone does not protect against this.
6
