• qa_deck_100#slide 22 · Uncertainty, hallucination, and over-reliance

In organisational GenAI, uncertainty is a governance problem because outputs can shape records, recommendations, decisions, and stakeholder communications, while behaviour is materially shaped at run time by prompts, retrieved context, tools, adapters, safety settings, and review actions. The RAIDT papers argue that organisation-level policy and model-level documentation are therefore necessary but insufficient. When outputs are probabilistic, sensitive to small prompt changes, or variable across repeated runs, the key governance question is no longer only whether the system is generally acceptable, but whether one configured use can later be reconstructed, reviewed, challenged, and justified. This is why RAIDT treats the run as the unit of governance and requires a run-level evidence pack rather than narrative assurance alone.

Uncertainty also cuts across the five pillars (Responsibility, Auditability, Interpretability, Dependability, Traceability). Responsibility requires explicit purpose, limits, oversight, and uncertainty communication. Auditability and Traceability require preserved evidence of prompts, model deployment, retrieval snapshots, checks, and reviewer decisions. Interpretability requires that outputs and limitations are understandable in context. Dependability is especially important because single outputs cannot establish reliability where repeated runs may vary; RAIDT therefore expects repeat-run evidence, configuration capture, and visibility of dispersion rather than hidden variance. The score profile, using anchors 1=missing / 3=partial / 5=audit-ready, makes this governance readiness inspectable across runs and configurations.

From an Information Systems perspective, uncertainty is a governance issue because governance concerns decision rights, accountability, monitoring, and coordinated action under uncertainty. In RAIDT, influence methods as governance interventions can improve grounding or constrain output structure, but they also alter the risk profile of the configured use. If they are not captured and scored, organisations cannot identify failure patterns, compare runs, support post-incident review, or justify reliance on GenAI in high-stakes work.

A hospital uses a GenAI assistant to draft discharge summaries from clinician notes. Two runs on the same case produce slightly different summaries: one includes a medication warning, while the other omits it and sounds more certain than the underlying evidence supports. In RAIDT terms, that is not merely a technical quality issue; it is a governance problem because the summary may enter the patient record and influence care.

The hospital therefore treats the run as the unit of governance. Its run-level evidence pack records the prompt template, model version, retrieval snapshot from the internal clinical guideline corpus, output hash, uncertainty statement, and clinician review decision. Structured prompting and retrieval augmentation operate as influence methods as governance interventions because they are intended to improve interpretability, traceability, and safer uncertainty communication. If repeat runs still vary materially, the Dependability pillar falls in the score profile, even if the prose looks fluent. That evidence allows the organisation to audit the event, compare configurations, and decide whether the workflow is genuinely audit-ready.

• 11-RAIDT_Academic_Logic_M_v11
• 15-RAIDT-IS-Governance_M_v07