AI Hallucination
An AI-generated output that is fluent and plausible but factually wrong or unsupported by the source material.
AI hallucination is the phenomenon where a language model produces an output that reads as authoritative but is either factually incorrect or invented entirely. It happens because language models are optimised for plausibility, not truth, they will compose a confident answer to almost any question, including questions where the model lacks grounding.
In consumer applications, a hallucination is an inconvenience. In regulated financial services, a hallucinated DDQ answer or a fabricated SoA citation is a compliance breach with discoverable consequences.
Hallucination is the single biggest blocker to AI adoption in regulated workflows. Adding documents alone does not solve it: in BackPro’s March 2026 benchmark, standard retrieval invented an answer on 28.3% of 120 compliance questions, more than a model with no documents at all (15.8%). What reduced it to 0.8% was checking that each answer is supported by its source and allowing the system to say the documents do not answer the question.
Take this with you · eBook · PDF, 11 pages
The AI-Native Compliance Advantage
The questions advisers and licensees ask about AI in advice documents, from "will it make things up?" to "who is responsible?", answered with the evidence.
Sent to your inbox. No call, and nothing else unless you ask.