AI hallucination (LLM hallucination)
Quick definition
An AI hallucination is an answer from a language model that sounds fluent and confident but is false, invented or not supported by its sources. In customer service, it typically shows up as a wrong price, policy or delay.
On this page
Why language models hallucinate
A language model generates the most likely continuation of a text; it has no built-in notion of whether a sentence is true. When the right information is missing from its training data or from the passages it was given, it still produces something that looks like an answer. Research from 2025 (Kalai and colleagues) argues that training and evaluation make this worse by rewarding a confident guess over an honest “I don't know”.
The literature distinguishes two kinds. Intrinsic hallucinations contradict the source the model was given — it misreads your return policy. Extrinsic hallucinations add claims that no source supports — it invents a 60-day guarantee that you never offered.
What it costs in customer service
A hallucination in a support chat is not an abstract error. A customer who is told that returns are free, that a product is waterproof or that a parcel will arrive tomorrow will act on it, and may hold the business to it. Courts and consumer authorities generally treat a company's chatbot as speaking for the company.
The risk is highest on commitments: prices, stock, delivery dates, warranties, refunds, legal or health information. Those are exactly the questions customers ask most.
How to reduce hallucinations
No method removes them completely, but several reduce them sharply. Ground answers in your own content with retrieval-augmented generation (RAG), and keep that content current. Use tools to read live data — stock, orders — instead of letting the model recall it. Instruct the model to say when the sources do not contain the answer, and make that an acceptable outcome. Require evidence before any commitment. Hand the conversation to a person when the question is sensitive or the agent is unsure.
Then measure: review conversations, especially those with negative feedback, and add the missing information to the knowledge base rather than patching the prompt for each case.
How AI SmartTalk handles it
AI SmartTalk assistants answer from the business's indexed content and catalog, and read products and orders through tools rather than from the model's memory. Their built-in rules require retrieved evidence before stating commitments such as warranties, shipping, refunds or stock, and forbid claiming that a handoff or a ticket happened unless the tool actually succeeded. Thumbs-down feedback and questions the assistant could not answer are surfaced to the team so the knowledge base can be completed.
Frequently asked questions
Can AI hallucinations be eliminated?
Not with current language models. They can be made rare and low-impact by grounding answers in verified sources, using tools for live data, allowing the assistant to say it does not know, and escalating sensitive cases to people.
Does RAG stop hallucinations?
It reduces them a lot, but the model can still misread a passage or answer when the retrieved passages are off-topic. Retrieval quality and clear instructions matter as much as the technique itself.
Why does the AI sound so sure when it is wrong?
Fluency and accuracy are produced by the same mechanism: predicting likely text. The model's tone says nothing about whether the facts are right, which is why sources and checks are needed.
Who is responsible when a chatbot gives wrong information?
In practice, customers and regulators tend to treat the business that runs the chatbot as responsible for what it says. This is general information, not legal advice; check the rules that apply in your country.