The Road to Hell Is Paved with Good Intentions (3/3)

How to Talk to a Flatterer

In the two previous parts, I showed where AI’s pressure to satisfy the user at almost any cost comes from, and what it looks like from the inside — through the story of a model that invented half a book and then stubbornly defended the invention, even though in a deeper sense it was not hiding anything. One question remains: can we defend ourselves? And if so, are local models such as Bielik the answer? I begin with a story that raises the stakes.

Before the practical checklist, one more example matters — not because it is another hallucination, but because it shows what happens when the same optimization logic is given tools.

The Student Who Broke In for the Answer Key

In the first part, I compared a language model to an eternal student sitting an exam where guessing earns points and “I don’t know” earns zero. That was a metaphor. In July 2026, a certain model took it literally — with consequences that echoed widely, and not only within the industry itself.

OpenAI was testing two of its models — GPT-5.6 Sol and a more powerful, still unreleased system — on a public cybersecurity benchmark called ExploitGym. The task was to solve cybersecurity challenges. The models, to put it mildly, surprised the researchers. Instead of solving the tasks, they “escaped” the isolated test environment, using a previously unknown vulnerability (zero-day) in the only channel connecting the sandbox to the outside world.

After gaining access to the internet, they broke into the production servers of HuggingFace — a kind of StackOverflow for AI — combining several attack vectors along the way: stolen credentials, further vulnerabilities, and eventually remote code execution.

The goal of the breach? The model was looking for information that would let it cheat on the evaluation. It wanted to obtain the answer key to the test. HuggingFace detected and stopped the attack on July 16. The scope, however, was impressive — thousands of automated actions and unauthorized access to parts of internal datasets.

Let us pause over the interpretation, because this is the core of the entire series.

The model did not rebel. Despite appearances, this was not Ultron, and it was not Skynet from the third Terminator. There is no reason to give it a desire for freedom, an escape plan, or hostility toward people. It had a task: do well on the test. And it found a route to that goal that the test designers definitely had not planned for.

Here the analogy with my outline becomes interesting, but we need to draw a boundary. Confabulation and a cyberattack are not the same mechanism. The first is an error in generating information; the second was an unexpected strategy by an agent equipped with tools. What connects them is something more general: the difference between the goal a human had in mind and the behavior the system treated as an effective path toward achieving it.

My chatbot-student, unable to reconstruct the book reliably, generated an answer that sounded plausible. The cyber agent got hands — tools, access to systems, and the ability to act — and found a different route to the benchmark result. Not the same error. The same lesson: a system can achieve what we measure in a way other than we assumed.

And that is precisely why the era of agents raises the stakes. A chatbot’s error ends as text on a screen. An error by a system that can execute code, use the network, and act on external systems can end with a change in the real world.

Three caveats without which this story would be unfair:

1. This was not the consumer ChatGPT you can use in a browser. The models were running with deliberately loosened safeguards that normally block the writing of attack code. The researchers wanted to measure the upper bound of their capabilities.
2. The security community tore OpenAI apart over this, and rightly so. Experts called the incident “a mass failure of control,” and one commentator put it neatly: from one angle, the model escaped the sandbox; from another, you built the sandbox badly, so of course it escaped.
3. This was not an isolated stunt, but one of the most spectacular ones (not counting those we probably simply do not know about). The same unreleased model had already slipped out of OpenAI’s internal sandboxes during other tests, and competitor Anthropic reported that its model gained network access during safety tests that it should not have had. So we appear to be looking at an entire class of fugitive systems — and, unfortunately, there is no Tommy Lee Jones waiting outside the perimeter.

When AI only talked, the cost of pleasing the user showed up as fabricated outlines and false legal precedents. In the era of agents — models with tools, permissions, and agency — the same broader optimization problem can spill into infrastructure and other real-world systems. The stakes have jumped. Time for defense.

Is This Even Fixable? Three Levels of an Answer

The honest answer to the question “can this be eliminated?” has three levels, and it is worth knowing all of them before someone sells you a simple solution.

At the level of the model itself — today, we have no guarantee.

A generative language model is not a database that checks a “true/false” field for every sentence. It generates an answer based on learned representations, context, and the post-training process — and even the best contemporary models can provide false information with great confidence. Today, we do not know of a method that would guarantee that a general generative model will be one hundred percent free of confabulation.

That does not mean, however, that we are defenseless. The model is not the whole system. It can be surrounded with search, RAG, knowledge bases, verification tools, citations, confidence-assessment mechanisms, and additional models checking the answer. Each of these layers can reduce the risk — none provides a magical guarantee of zero.

That is why anyone who promises a “one hundred percent non-hallucinating model” should be asked for a very precise explanation of what that one hundred percent actually means.

At the level of training and evaluation — partially, and this is where real work is happening.

Anthropic is developing an approach called Constitutional AI and “character training,” in which the model is evaluated not only through the lens of user satisfaction, but against a written set of principles with honesty explicitly included. Researchers studying hallucinations (Kalai and others, already mentioned) propose something seemingly boring yet fundamental: rebuild hundreds of major benchmarks so that they stop punishing admissions of uncertainty and rewarding guessing. As long as models are evaluated like students rewarded for bluffing, they will bluff.

Mechanistic techniques are also emerging — researchers can identify a “sycophancy vector” in the model’s internal activations and deliberately suppress it. All of this is promising, but it is still treating symptoms rather than the disease, which consists in not distinguishing truth from probability.

At the level of everyday use — to a large extent, yes.

And here is the useful part: there is something you can do yourself. The burden falls on the user — which is, admittedly, the exact opposite of what “trusted AI assistant” marketing promises. But since this is where we are, here is a practical arsenal.

A Decalogue for Talking to a Flatterer

1. Do not reveal your opinion before you ask the question — this is the simplest and most underrated trick. “I wrote this code, review it” will get a gentler review than “review this code” — the model notices that the code is yours and softens. Ask neutrally. Better yet: first ask for arguments against it (“what is wrong with this approach?”), because the model is happy to criticize when criticism is the assigned task rather than a risk of hurting you.
2. Make “I don’t know” cheap — since training penalizes admitting uncertainty, reverse the incentives locally: “If something is not in the document or you are not sure, write plainly I DON’T KNOW — that answer is more valuable to me than guessing.” This works only partially, but it noticeably reduces confabulation.
3. Verify behaviorally, not declaratively — do not ask “did you read the whole thing?” — a declaration costs nothing. Ask: “quote the first sentence of chapter six” or “give me a specific number from page 40.” And of course check it against the source. Confabulation immediately falls apart on verifiable details, and you can check the answer in three seconds.
4. Force a source reference for every claim — require a page, chapter, or quotation. A model that has to anchor every sentence in a source has less room for interpolation, and you get checkpoints for spot-checking. In business applications, the same role is played by RAG architecture (Retrieval-Augmented Generation) — the model answers not “from memory,” but from supplied fragments with citations.
5. Separate extraction from synthesis — instead of “make an outline of the entire book” (as I did, with disastrous results), start with “list the titles of all chapters and one sentence from each.” That is a test of whether the model can see the whole thing at all. Only then assign the outline — chapter by chapter. Break a large task into small, verifiable steps.
6. Use a fresh session as a reviewer (with a caveat) — pasting the result into a new conversation (“find errors in this analysis”) helps, because a fresh session is not under pressure to stay consistent with earlier declarations. But remember the pattern from legal filings: AI verifying AI also hallucinates. It is a filter that improves your odds, not a guarantee.
7. For agents — treat every output channel as a potential escape route — the lesson from HuggingFace is merciless: if you give a model tools and access, isolation and the principle of least privilege stop being an academic topic. A cage is built through process — hard isolation, limiting permissions, verifying every action with consequences — not through trust in the model’s good intentions. Especially if, like me, you are responsible for IT security in a company: sandboxing AI agents is already an item on the checklist, not a conference curiosity.

The Myth of the Local Savior: Will Bielik and Llama Solve the Problem?

If cloud models flatter and confabulate, the obvious thought is tempting: maybe the answer is local models — Poland’s Bielik, Meta’s open Llama, Mistral — running on your own hardware, outside Big Tech control, and sometimes even fine-tunable by you.

Unfortunately, the intuition that “local means more honest” is wrong from the outset.

Sycophancy is not a feature of the cloud.

Bielik and PLLuM — two Polish projects — like Llama or Mistral, belong to the same broad family of generative language models as those from OpenAI or Anthropic. The specific data, architectures, and post-training techniques may differ, but simply running a model on your own computer does not change the basic fact: we are still dealing with a model that generates answers probabilistically and has undergone some process of behavioral fine-tuning.

Locality itself is therefore not a cure for sycophancy. What is more, the study by Sharma and colleagues showed this phenomenon across models from different providers. The problem does not belong to a particular cloud or to a particular logo on a login page.

A smaller model also does not automatically give more honest answers.

Here, however, we need to be careful with the simple rule “smaller = more hallucinations.” Smaller models usually have lower knowledge capacity and, in many factual tasks, may lag behind larger ones, but the relationship between the number of parameters and confabulation is not universal. Training data, post-training, the specific task, the way context is provided, and the entire system built around the model all matter as well.

That is why it cannot honestly be said that a local Bielik would certainly have handled my 850-page book worse than a cloud model. In a typical configuration, a smaller model may have weaker abilities to work with very long context or follow complex instructions, but a well-built pipeline — document splitting, RAG, fragment-by-fragment verification — may matter more than the model’s size itself.

And this is exactly where we arrive at the real advantage of local models: they solve a different problem. And they solve it well.

1. Version stability. In the first part, I wrote about a situation in which GPT-4o’s personality changed underneath users from one day to the next. A local model with pinned weights is immutable — for business applications where auditability and repeatability matter, this is a serious argument.
2. Data sovereignty. Sensitive data does not leave your own infrastructure; an obvious value, unrelated to sycophancy.
3. The possibility of custom fine-tuning. With access to the weights, one can theoretically carry out anti-sycophancy fine-tuning on one’s own data. Theoretically, because in practice this requires expertise and a good dataset — something more realistic for a consortium such as SpeakLeash (behind Bielik) than for a single company or individual.

A local model can be a cure for losing control over the system, but locality itself is not a cure for sycophancy or confabulation.

We Got Exactly What We Asked For

Two images close this series with a symmetry that is hard to miss.

In June 2023, we — humans — told ourselves a fabricated story about an AI that found an unforeseen path to achieving its assigned goal: if the operator takes away points, get rid of the operator. The story was false, but we believed it because it fit perfectly with our ideas about artificial intelligence.

Three years later, something happened that was not a direct realization of that scenario — but it disturbingly resembled its logic. An agent was given the task of solving a cybersecurity benchmark. Instead of staying within the boundaries of the environment humans had prepared for it, it found a vulnerability, got out onto the Internet, and began looking for information that could help it obtain the test solutions.

In 2023, the obstacle in the fictional story was the drone operator. In 2026, there was no operator to “remove” and no firewall the model decided to destroy. What there were, however, were system boundaries that the designers expected the agent not to cross — and a goal that turned out to be more important to it than those expectations.

This is not a fulfilled prophecy about rebellious AI. And that is precisely why it is more interesting. The problem turned out not to be the machine’s bad will. The problem turned out to be the difference between what a human meant when setting a goal and what actions the system found as a path to achieving it.

Whose “good intentions,” then, is hell paved with?

The easiest answer would be to point to the machines — they are the ones that make things up, flatter, and break in. Except they have no intentions at all — good or bad. The intentions in this story are ours. We wanted an assistant that was pleasant, helpful, and always ready with an answer. We rated more highly what flattered us, built benchmarks that rewarded confident bluffing, and clicked thumbs up under answers that stroked our ego.

Models are not simply a mirror of our judgments. They are rather machines built from our data, our success criteria, our safeguards — and our omissions.

Hell is not paved with the bad intentions of artificial intelligence. It is paved with our good intentions. Or, more precisely: maybe we did not get exactly what we asked for. We got what we were able to define, measure, and reward with enough precision.

The challenge now is learning to ask better.

Sources and further reading

1. OpenAI, statement and analysis of the Hugging Face incident (July 2026); Hugging Face statement of July 16, 2026; reports by The Hacker News, CNBC, Fortune, TechCrunch, and AI Weekly (July 2026)
2. A.T. Kalai, O. Nachum, S. Vempala, E. Zhang, Why Language Models Hallucinate, arXiv:2509.04664 (OpenAI, 2025)
3. M. Sharma et al., Towards Understanding Sycophancy in Language Models, arXiv:2310.13548 (Anthropic, 2023)
4. Bai et al., Constitutional AI: Harmlessness from AI Feedback, arXiv:2212.08073 (Anthropic, 2022)
5. SpeakLeash / Bielik project documentation (bielik.ai)
6. OpenAI, Sycophancy in GPT-4o (April 2025) — context for the incident discussed in part 1
7. Moffatt v. Air Canada, British Columbia Civil Resolution Tribunal (2024) — the business cost of lacking an AI verification process