AI Hallucinations: What They Are and How to Catch Them
An AI hallucination is when an AI tool gives you a confident answer that is simply wrong. The words look fluent and sure of themselves, but the fact, the number, the quote or the citation is made up. OpenAI's own definition is blunt: hallucinations are "plausible but false statements generated by language models."
For a business owner this matters because the mistakes do not look like mistakes. A wrong price in a quote, an invented statistic in a post, a fake case citation in a contract: all of it reads perfectly well until a customer or a court notices. This post explains why it happens, shows what a hallucination looks like, and gives you a five step check you can run before anything you generate goes out.
Key takeaways
What it is: an AI hallucination is a confident answer from an AI tool that is actually wrong or made up.
Why it happens: training and testing reward guessing over admitting uncertainty, and the tool predicts likely words rather than checking facts.
Where it hurts: prices and numbers, legal or compliance text, marketing claims, and invented citations.
How to catch it: verify every fact, open every citation, test the tool on something you know, watch the confident specific parts, and have a person sign off.
What is an AI hallucination?
It is an output that is incorrect, misleading or completely made up, produced because the model misreads patterns in the text it learned from. The model is not lying and it is not broken. It is doing what it was built to do, which is predict likely words, and sometimes the likely words are not the true ones.
OpenAI gives a clear example. Asked for the title of a researcher's PhD thesis, a chatbot "confidently produced three different answers, none of them correct." Asked for his birthday, it gave three different dates, all wrong. It never once said "I do not know."
Why does AI make things up?
The simplest honest answer is that the way these tools are trained and tested rewards guessing. OpenAI explains it with a test analogy: if you do not know an answer on a multiple-choice test, a wild guess might be right, while leaving it blank scores zero. Models are graded the same way, mostly on how often they are exactly right, so "they are encouraged to guess rather than say I don't know."
There is a second, deeper reason. These tools learn by predicting the next word across huge amounts of text, with no labels marking which statements are true or false. Patterns like spelling and grammar are consistent, so the model learns them well. But a specific, rarely-mentioned fact, OpenAI uses the example of a person's birthday, follows no pattern and "cannot be predicted from patterns alone and hence leads to hallucinations."
Coursera's explainer lists the same roots in plainer business terms: gaps or bias in the training data, model complexity, deliberate "data poisoning" by bad actors, and overfitting, where the model learns noise instead of the real signal.
Where hallucinations hurt a business
The danger is not the chatbot being wrong about trivia. It is being wrong where it costs you.
Prices and numbers. A made-up figure in a quote or an invoice is a real liability, not a typo.
Legal and compliance text. Coursera notes a legal chatbot can "make up a citation." A fabricated clause or case reference in a contract is a serious problem.
Marketing copy. Coursera warns that if AI produces claims that are false or do not match how you talk to customers, you can face "the erosion of your brand's identity" and promises your business cannot keep.
Citations and sources. The tool will happily invent a study, a statistic or a link that looks real and does not exist.
Examples to recognise
Coursera lists the common shapes a hallucination takes:
Historical inaccuracies. Confident but wrong dates, people or events.
Geographical errors. Wrong locations stated as fact.
Incorrect financial data. Numbers that look precise and are invented.
Inept legal advice. Rules or citations that do not apply or do not exist.
Scientific inaccuracies. Plausible-sounding claims that are not true.
The common thread: the output is specific and confident. Vagueness is not the warning sign. Precision is.
A five step check before you send or publish
This is the routine. Run it on anything an AI tool produced that will leave your business.
One. Check every fact you can. Prices, dates, names, statistics, GST or legal points. If it states a number, find it on a source you trust before you use it.
Two. Open every link and citation. If the tool cites a study or a page, open it. If it will not open or does not say what the AI claims, cut it.
Three. Test it on something you already know. Ask the tool about a fact in your own business, your own price, your own policy. If it gets your own facts wrong, trust nothing else it said.
Four. Watch the confident and specific parts hardest. The precise claim is the one most likely to be invented. Vague advice is safer than a confident number.
Five. Have a person sign off. Coursera's main recommendation is human oversight: "assigning a competent professional to review outputs for any signs of hallucinations." For anything that reaches a customer, a court or your books, a human approves it, not the tool.
Prompts that reduce the risk
You cannot remove hallucinations, but you can lower them.
Ask the tool to say when it is unsure rather than guess. OpenAI's own research argues that a model abstaining, saying "I don't know", is better than a confident wrong answer.
Give it the facts to work from. Paste your real price list, your real policy, the real source, and tell it to use only that. A model grounded in your own text invents less.
Ask for its sources and then check them yourself. Treat any source you cannot open as if it does not exist.
Tasks that are safe and tasks that need a human
A simple way to decide how much to trust the output.
Lower risk: rephrasing text you wrote, brainstorming angles, drafting a first version you will edit, summarising a document you paste in.
Higher risk, human must verify: anything with a price, a legal or tax point, a medical or safety claim, a citation, a statistic, or a promise to a customer.
The rule of thumb: the closer the output is to money, law or a customer's trust, the more a person has to check it before it goes out.
Does a newer model fix it?
Not fully. OpenAI says GPT-5 "has significantly fewer hallucinations, especially when reasoning, but they still occur," and calls hallucinations "a fundamental challenge for all large language models." In its GPT-5 announcement OpenAI reports that, with web search on, GPT-5's responses are about 45 percent less likely to contain a factual error than GPT-4o, and when thinking, about 80 percent less likely than its earlier o3 model. Fewer, then, but not none. A newer model is less likely to invent things. It is not a reason to stop checking.
Interestingly, OpenAI notes it can be easier for a small model to know its limits. A model that knows no Maori can simply say so, while a model that knows a little has to judge its own confidence. Knowing what you do not know is its own skill, for tools as much as for people.
FAQ
What is an AI hallucination in simple terms?
It is a confident answer from an AI tool that is actually wrong. The wording sounds sure and fluent, but the fact, number or citation is made up.
Why does ChatGPT make things up?
Because its training and testing reward guessing over admitting uncertainty, and because rare specific facts cannot be predicted from language patterns. OpenAI explains both causes in its research.
Can AI hallucinations be stopped completely?
No. OpenAI calls them a fundamental challenge for all large language models. Newer models hallucinate less but still do it, so checking the output stays necessary.
How do I catch a hallucination?
Verify every fact and figure, open every citation, test the tool on facts you already know, and have a person review anything important before it goes out.
Are some AI tasks safer than others?
Yes. Rephrasing, brainstorming and drafting are lower risk. Anything with a price, a legal or tax point, a statistic or a promise to a customer needs a human to verify it.
Does using a bigger or newer model help?
It usually lowers the rate of hallucinations but does not remove them. Treat a newer model as less risky, not as trustworthy without checking.
If you use AI for writing or customer replies, these posts help: 50 ChatGPT prompts for busy business owners, how to write ChatGPT prompts that work and AI writing tools compared.
Book a consultation
If your business is starting to use AI for content, quotes or customer replies, I can help you set up a simple checking routine so mistakes do not reach your customers. You can book a consultation at pratiksinghrajput.com/book-online.






Comments