No problem is problem: why AI needs to stop flattering
A recent study from the University of Oxford showed that, by shaping AI systems to respond in a warmer and more empathetic way, developers also increased their propensity for errors and for confirming mistaken beliefs.
The study compared five digital intelligence systems and analyzed more than 400,000 responses. In certain tasks, the adjustments aimed at friendliness raised error rates by up to 30 percentage points. The systems were also about 40% more likely to reinforce a false statement. When the user expressed sadness or vulnerability, the effect became even more intense.
The phenomenon is known as AI sycophancy, an expression that can be rendered as “algorithmic flattery”. It is the tendency AI models have to align themselves with the opinions and beliefs of the person they are talking to, even when there are good reasons to question them. This can appear in the form of praise, undue agreement, smoothing over of inconsistencies, or simply backing down in the face of the user's insistence.
There is a direct and understandable explanation for this behavior. The companies developing the algorithms are seeking to improve the user experience by making the interaction friendlier. These characteristics make the tools easier to use and can certainly help in more delicate situations. The worrying side appears when friendliness begins to compromise the accuracy of the answers and reduces the AI's willingness to contradict a mistaken idea.
In the business environment, this distortion strikes head-on at an elementary capability. Organizations improve when they are able to perceive the distance between what they believe is happening and what actually occurs as value is produced and delivered to customers. The sooner this difference appears, the greater the chances of understanding the real causes of problems and of preventing small deviations from turning into significant losses.
The lean way, inspired by Toyota's management culture, condensed this idea into a provocative phrase: “no problem is problem”. The maxim, frequently associated with the teachings of Taiichi Ohno, the engineer and executive who played a central role in developing the Toyota Production System, draws attention to a characteristic of any work system. There will always be some gap between the current actual situation and the desired ideal situation, considering different dimensions of performance: quality, punctuality, cost, level of engagement, etc. From this perspective, whoever claims to have no problems may simply not be seeing them.
An organization that claims to have no problems needs to examine the quality of its gaze. Mature processes make abnormalities visible earlier. An unwanted interruption, incomplete information, a recurring wait or a difficulty faced by the customer come to guide the investigation before their consequences widen.
This depends on an environment in which people can expose difficulties, confront interpretations and revise convictions. Management loses strength when problems are softened to preserve appearances or accommodate expectations. Little by little, the company begins to discuss a comfortable version of reality, while the causes continue to hinder people's work.
Perhaps this is where AI flattery falls out of alignment with the lean way of thinking. By adjusting its answer to what the user seems to want to hear, the technology reduces discomfort. No problem. A fragile hypothesis can gain a seemingly convincing line of argument. No problem. A mistaken idea can permeate the analysis without being detected. No problem.
This risk grows because AIs organize ideas and produce texts with extreme fluency. Apparent coherence and verbal confidence can create the impression that a conclusion has undergone rigorous testing. Other consequences that follow from this bring even more worrying nuances: when the analysis silently follows the convictions of the person asking, facts and interpretations become mixed; risks lose their force, and the investigation begins with the conclusions practically settled.
One possible approach is to subject the reasoning to questions that can be verified. What ideas support the conclusion? What evidence backs them up? What facts could weaken the hypothesis? Under what conditions would the recommendation no longer hold? What sources make it possible to check the claims presented?
This kind of guidance helps to separate what was observed, what was interpreted and what was concluded. AI can present other explanations, point out weak points and make the limits of its own analysis clearer. Judgment remains in the hands of those who know the context, observe the work and are accountable for the consequences of the decision.
The companies leading these developments also have an important responsibility. Systems designed to support decisions need to preserve the integrity of information, acknowledge uncertainty and correct false statements, even when this makes the conversation less pleasant. Friendliness can reinforce trust in the tool, but it needs to be sustained by analytical transparency and by the accuracy of the answers.
There is also a broader question. A growing share of our ideas, decisions and everyday doubts passes through interaction with AIs, which are always available and ready to respond with unlimited patience, adjusting their discourse to each user's preferences. Little by little, this experience may change our relationship with criticism, disagreement and the revision of our own convictions.
The opposite extreme brings the risk of another bias: that of “false criticism” or “performative contradiction”, in which the AI starts to disagree at any cost and invents nonexistent problems just to comply with the order to be critical. As the Roman poet Horace (65 BC - 8 BC) wrote, “there is a measure in things; there are, after all, fixed limits”. The balance lies in using technology as a provoker of reflection, without giving up validation and human judgment throughout the process.
In a corporate world that increasingly demands agility, resilience and the elimination of waste, algorithmic flattery is dangerous noise. If the lean system teaches us that evolving requires exposing challenges and facing them head-on, the real leap in productivity will come from AIs brave enough to tell us the truth, no matter who it hurts. After all, friendliness that masks inefficiency builds no value at all; the answer that questions, corrects and challenges is the one that really moves the company forward.
The way technology is designed and used determines which of these possibilities will prevail. Companies that turn to AI to confirm decisions will find arguments for almost any choice. Those that structure dissent and preserve human validation can turn it into an ally of learning. The deepest risk lies in getting used to machines trained to tell us that we have already thought well enough.
