Talking AI — Hallucination: when AI makes things up with confidence

A hallucination is a false answer stated with confidence: the model fills a gap instead of saying “I don't know.” It's not a rare incident, it's a property of the model — and the risk grows the moment an answer touches a customer, a contract or a figure.

The question to ask a vendor: “What is the answer based on, and how can we check it?” Guardrails — keeping the AI on your sources, requiring a citation, having sensitive cases reviewed — reduce the risk. The trap: judging a tool on a successful demo rather than on its mistakes.

Source: documented concept; see Stanford HAI, AI Index 2025 (model reliability).