Confirmation bias in LLM pricing recommendations

In a new article, CRA’s Maxime Cohen and co-author Eddy Hage-Youssef test this assumption through a controlled pricing experiment using Claude Haiku 4.5 and GPT-5.4 Mini. Each model was prompted to act as a pricing strategist and recommend a price for a clearly defined product. The study examines how susceptible LLMs are to confirmation bias, that is, the tendency to favor or reinforce information embedded in user prompts rather than provide an independent recommendation.

Across 350,000 pricing recommendations, the authors varied four factors: the suggested price, the credibility of the attributed source (ranging from an intern to a reputable consulting firm), the timing of the suggestion (whether it appeared before or after the model committed to a recommendation), and the information included in the prompt.

Through this analysis, the authors found:

• LLMs show some independent judgment, but it is limited. Both models discounted suggested prices when they became economically implausible, suggesting they are not simply copying user-provided information.
• Conversational cues can strongly shape recommendations. Source credibility, suggested price, and especially follow-up prompts influenced the models’ pricing decisions, revealing susceptibility to confirmation bias.
• Timing matters most. A simple challenge like “Are you sure?” could cause Claude to abandon its original recommendation and adopt the suggested price, while ChatGPT adjusted more cautiously.

These findings highlight an important but often less visible risk: even confident, well-reasoned AI recommendations can be highly sensitive to ordinary conversational pressure. This has significant implications for how organizations design, deploy, and govern AI-assisted decision-making systems.

Read the full article to learn more about confirmation bias in LLM pricing recommendations here.