AI engineering · 32 of 42

Hallucination & Groundedness

Fluent, confident, and not in the source

Scroll

Fluent, confident, and not in the source

A hallucination is a statement the model produces that is not supported by anything it was given. It is usually fluent, plausible and delivered with the same confidence as everything else.

It is not deception and not a bug. The model completes patterns, and a free-postage clause is exactly the kind of thing that usually follows a refund policy.

Groundedness is the narrower, more checkable version: is every claim in the answer traceable to the supplied sources? That can be measured per claim, which is why it makes a better metric than the general question of truth. Asking the model to be more careful is not a control; checking claims against sources is.

Evaluation
FLUENT, CONFIDENT, AND NOT IN THE SOURCE what was retrieved · refunds within 30 days · store credit after that the model what it said · refunds within 30 days · and free return postage plausible, useful, and in no document groundedness asks a narrower question: is every claim in the source? It is not lying and it is not a bug. The model completes patterns, and a free-postage clause is exactly what usually follows a refund policy. The fix is checking claims against sources, not asking it to be more careful.
Two retrieved facts going in and three coming out, the extra one plausible and sourceless.