AI hallucinations: why AI makes things up, how to limit it
Filip Papranecfounder of Digitaliko
An AI hallucination is an answer the language model made up but presents confidently as fact. A source that does not exist, a wrong number, an invented legal clause. Nothing removes it completely, but in a business you can limit it a lot: the AI answers from your sources, code checks the output and a person decides on important steps.
What is an AI hallucination?
A hallucination is not an error in the classic sense. The model does not crash or print a message. It answers fluently, grammatically and confidently. It is just not true.
Typical examples in a business:
- A made-up source. A link to an article, law or study that does not exist.
- A wrong number. An amount, date or company ID that is not in the document but "fits" the sentence.
- A made-up feature. The AI promises a customer a product feature or a condition you do not offer.
- Invented content. A contract summary with a clause that is not in it.
The trickiest part is that a made-up answer sounds just as sure as a correct one. You cannot tell by the tone. You can find a short definition in the glossary.
Why does AI hallucinate?
A language model does not look for the truth. It looks for the most likely next words. When it lacks data, it fills the gap with something that sounds plausible. Researchers at OpenAI described it like this in 2025:
Like students facing hard exam questions, large language models sometimes guess when uncertain, producing plausible yet incorrect statements instead of admitting uncertainty.
In their view, part of the problem is how models are evaluated: an answer pays off more than "I don't know". For a business, three practical things follow. The model hallucinates most where it does not have your data. It hallucinates on precise details such as numbers, names and dates. And it hallucinates more when you force it to always answer.
Where is a hallucination cheap and where is it expensive?
One question decides it: does the AI output move on without a check? That tells you how many checks you need.
| Use | What a hallucination causes | Cost of the mistake |
|---|---|---|
| Ideas, an outline, a first draft | A person throws it out while reading | Low |
| Internal summary of emails or a meeting | A wrong picture of the situation until someone checks the source | Medium |
| Replies to customers | A promise or information the company will not keep | High |
| Numbers in invoices, quotes and orders | A wrong amount or wrong supplier in the accounts | High |
| Contracts, tax, legal questions | A decision based on a clause that does not exist | Very high |
The lower in the table, the more checks and the less autonomy the AI gets. For the last two rows, the output must not move on without a check by code or by a person.
How to limit hallucinations in a business?
There is no single trick. What works is a combination of layers, where each one catches what the previous one let through:
01
Answer from your sources
The AI gets your documents, price list or orders and answers from them, not from memory (RAG).
02
Allow "I don't know"
When the answer is not in the source, the AI should say so and hand the case to a person.
03
A fixed output format
Instead of free text, exact fields: amount, date, company ID. Those can be checked.
04
A check in code
The line totals add up, the company ID exists, the date makes sense. Code does not get things wrong the way a model does.
05
A person on important steps
Payments, promises to customers, contracts. What the checks cannot verify goes to a person.
How RAG works is explained in the glossary under RAG. For outputs that cannot be checked by a rule, a second model can act as a reviewer. How we do that for code is in LLM as a judge: a reviewer that tries to refute the work.
Example: invoices in our own accounting
In our own accounting, AI reads invoices from emails and records them in the accounting system. The model pulls out the fields: supplier, company ID, amount, VAT and due date. We do not match the supplier by the name the AI read but by the company ID, which is unique.
- Processing one invoice
- 10-15 min → <1 min
- Of invoices go through with no person at all
- 70 %
A person checks the rest. How to choose between OCR and a system with checks is in Invoice data extraction: OCR or a system?
Is your AI solution protected against hallucinations?
Tick what you already have:
When not to use AI for the job
- A mistake costs a lot and the output cannot be checked. For example, interpreting a law without a person who checks it.
- The answer must always be the same. A calculation a fixed rule can do should not be done by a model.
- You have no sources. If the company has no written price list, terms or process, the AI will make them up.
We would rather leave that kind of work to fixed software and use AI only where free text needs reading.
Want AI that does not make things up in your name?
Tell us what work you want to give to AI and where a mistake would be expensive. We get back to you within two business days with a few questions about your work. Then you get a list: what will be done and how you can check it.
How we build AI agents with checks is on the page Custom AI agent for business. If you'd rather talk first, book a 15-minute intro call.
In short
- An AI hallucination is a made-up answer that sounds just as sure as a correct one.
- It happens when the model lacks data and guessing pays off more than "I don't know".
- It is cheap for drafts and expensive for numbers, customer replies and contracts.
- You limit it with sources, a fixed output format, checks in code and a person on important steps.
Frequently asked questions
What is an AI hallucination in simple terms?
An answer the AI made up but presents confidently as fact. A source that does not exist, a wrong number, an invented legal clause or a product feature you do not have.
Why does AI hallucinate?
A language model does not look for the truth but for the most likely next words. When it lacks data, it fills the gap with something that sounds plausible. On top of that, training often rewards an answer more than admitting it does not know.
Can AI hallucinations be removed completely?
No. Newer models hallucinate less, but nobody can guarantee zero. In a business they can be limited a lot: the AI answers from your sources, code checks the output and a person decides on important steps.
Does RAG help against hallucinations?
It helps, because the AI answers from your documents, not from memory. It does not remove hallucinations completely though: the model can misread the source or add what is not in it. So it pays to also check that the answer really is in the source.
Where is an AI hallucination most dangerous in a business?
Wherever the output moves on without a check: numbers in invoices and quotes, replies to customers, contracts, tax and legal questions. For drafts and ideas that a person reads anyway, a hallucination is cheap.
Ask AI about this article
The AI also draws on our other articles and experience from real projects.
ChatGPTClaudePerplexity
