Q116 - What_are_the_main_GenAI_failure_modes_that_motivate_RAIDT
Q116 — What are the main GenAI failure modes that motivate RAIDT?
← RAIDT · Star S2 - Governance Meaning and Problem Context · primary item: S2.10 · GenAI failure modes
Appears in sources
integrated_82#Q2.4
Answer
The main GenAI failure modes motivating RAIDT are hallucination, overconfidence, unstable output, source opacity, hidden configuration drift, missing provenance and over-reliance. This list fits the item definition and is strongly supported by the three papers. The managerial decision-making paper highlights lack of interpretability, overconfidence in opaque outputs, weak calibration under uncertainty, and vulnerability to misinformation or conflicting evidence. The evidence-review and governance papers show why these risks intensify in generative AI: one output is shaped by prompts, retrieval sources, tool chains, provider versions, safety settings and human review actions, many of which vary across runs. The result is not a single 'accuracy problem' but a cluster of socio-technical failure modes that appear in configured use.
RAIDT responds by treating run as the unit of governance and by requiring a run-level evidence pack for each governed use. The five pillars (Responsibility, Auditability, Interpretability, Dependability, Traceability) convert these failure modes into inspectable governance questions. Hallucination and source opacity test Interpretability and Traceability; over-reliance tests Responsibility; unstable output and hidden configuration drift test Dependability; and missing provenance undermines Auditability and Traceability across the board. The score profile then makes those weaknesses comparable across runs using anchors 1=missing / 3=partial / 5=audit-ready. In other words, RAIDT is motivated by the fact that generative-AI failures are often produced by runtime configuration, weak evidence and uncalibrated human-AI collaboration, so governance must examine the configured run rather than only the model or policy layer.
Practical example
A healthcare organisation using GenAI to prepare clinical-administrative summaries could face several failure modes in one run. The system may hallucinate a treatment detail, present uncertain material with excessive confidence, draw on retrieved guidance whose source is not visible to the reviewer, or change behaviour after a provider update without frontline staff noticing. If clinicians begin to accept these outputs routinely because they are fluent and fast, over-reliance compounds the problem.
RAIDT would handle this by attaching a run-level evidence pack to the governed use and rating the result through a score profile. If prompts, source lineage, version identifiers and oversight actions are incomplete, the profile would sit closer to 1=missing or 3=partial rather than 5=audit-ready, signalling that the run is not governance-ready even before any incident escalates.
Sources in RAIDT papers
01-Responsible_AI_for_Managerial_Decision-Making_Under_Uncertainty-V313-RAIDT-Evidence-Review_M_v1015-RAIDT-IS-Governance_M_v07