Reliability
Hallucination
When a model produces confident, fluent output that is simply wrong — the failure mode that makes AI risky in business processes.
A model generates plausible text. Usually plausible and true coincide. Sometimes they do not, and the model has no way to tell the difference — it will invent a citation, a part number or a policy with exactly the same confidence it states a fact.
This is not a bug that gets patched. It is inherent to how these systems work, and it is the single biggest reason AI projects fail in regulated or high-consequence settings.
What actually reduces it:
- Grounding the model in real retrieved data rather than its training (RAG).
- Citations back to source, so answers can be checked.
- Structured output validated against a schema, so malformed answers fail loudly.
- A confidence floor — a system that declines to answer when unsure is worth more than one that always has an answer.