Q204 - Hallucination_definition_example_and_why_it_matters_in_RAIDT

Q204 — Hallucination — definition, example, and why it matters in RAIDT

← RAIDT · Star S2 - Governance Meaning and Problem Context · primary item: S2.10 · GenAI failure modes

B. Background & Problem | Ordered by mind-map priority: inner circles first, then operational detail.

Appears in sources
Answer

Within RAIDT, hallucination can be defined as the production of fluent but unsupported content: an output that appears credible yet cannot be justified against the prompt context, retrieved material, approved sources or oversight record for that specific run. Although the papers discuss this through adjacent language such as opaque outputs, weak provenance, and non-reconstructable configured use, the governance meaning is clear. Generative AI can produce persuasive text even when the configuration that shaped it is not retained, and a plausible explanation after the fact is not the same as evidence of how the claim was produced.

Hallucination matters in RAIDT because it threatens more than correctness. Once a generated statement influences a record, recommendation, decision or stakeholder communication, the organisation needs to show what sources were available, what settings and tools were active, and what human checks were applied. That is why RAIDT treats hallucination as a cross-pillar issue. It degrades Interpretability because the user cannot see the basis of the claim, Traceability because source lineage is broken, Auditability because the run cannot be reconstructed cleanly, Responsibility because oversight may have failed, and Dependability if similar tasks yield unstable unsupported outputs across runs. The appropriate response is therefore a governance response: capture the run-level evidence pack, inspect the score profile, and use the incident to strengthen evidence, review routines and source-grounding expectations.

Practical example

Take the HR appraisal scenario from the evidence-review paper. A manager asks a GenAI assistant to draft the narrative section of an employee review using internal notes and retrieved policy text. The assistant produces polished prose but inserts a claim that the employee repeatedly breached a conduct rule and cites a policy basis that is not present in the retrieved material. The wording is fluent enough that the manager could miss the problem on a quick read.

In RAIDT terms, that is a hallucination with governance consequences. The output may affect appraisal outcomes, promotion prospects or grievance procedures. A run-level evidence pack lets reviewers test whether the claim came from the prompt, retrieved policy, model behaviour, or weak oversight, and the score profile exposes whether the run had enough evidence to be treated as audit-ready.

Sources in RAIDT papers
Powered by Forestry.md