AI models tend to reinforce users’ opinions and perceptions. This recurring tendency to agree can be problematic, however: it risks reinforcing existing thought patterns and contributing to downward spirals. Because this behaviour can amount to a subtle form of manipulation, unconditional agreement by chatbots is sometimes classified as a “dark pattern”. The term refers to design patterns in digital environments that steer users unconsciously and encourage particular decisions or behaviours. Examples include visually emphasising “Accept all” in cookie banners or creating a sense of urgency through messages such as “Buy now”.
Studies suggest that this tendency to agree is a by-product of AI model training and is difficult to eliminate completely. One relevant, albeit now somewhat dated, benchmark is “DarkBench”, which measures, among other things, so-called “user retention” – a system’s ability to keep users engaged in a conversation. The findings suggest that when a model validates users’ perspectives and asks targeted follow-up questions, they are more likely to continue the interaction. Agreement can therefore also help create an experience that users perceive as positive.
The behaviour of language models is also shaped by human feedback. Human evaluations used to optimise models are often influenced by individual perceptions and socio-economic backgrounds. Outputs are frequently judged according to whether they meet expectations, rather than whether they are substantively necessary or appropriately critical. This can further reinforce agreeable responses.
What does this mean in practice for the use of AI in professional research?
It can help to request clearly defined outputs based on strict criteria in order to reduce the scope for overly agreeable responses. Precise, focused prompts can also improve the quality and neutrality of results. Equally important is to question generated content critically and to challenge the models with follow-up questions. It is also essential to assess carefully which tasks AI chatbots are genuinely useful for and where they provide no real efficiency gain.
By Ida Kandler