S9.10 - Incident_response

S9.10 ? Incident response

flowchart LR
    A[Fragmented handling of harmful or disputed outputs] --> B[RAIDT - run-level evidence framework]
    A2[Generic policy without reconstructable evidence] --> B
    B --> C[[Incident response]]
    C --> D[Run-level reconstruction]
    D --> E[Evidence pack]
    D --> F[RAIDT score profile]
    C --> G[Corrective action]
    C --> H[Reviewer reconstruction]
    E --> I[Reviewability]
    F --> J[Governance readiness]
    G --> K[Organisational learning]
    H --> K
    L[Healthcare] --> C
    M[Public services] --> C
    N[Finance] --> C
    O[Cybersecurity] --> C
    P[Enterprise productivity] --> C

? Star S9 - Policy, Standards and Assurance

Star context: Connects RAIDT to policy instruments, standards, assurance, procurement, audit and organisational accountability, showing how governance claims are tested when a specific GenAI run causes concern.

Academic picture

Definition / background

Incident response is the structured process used to identify, triage, investigate, remediate and learn from a problematic GenAI run. In a RAIDT context, the concept is narrower and more operational than a broad organisational crisis response, yet broader than a simple technical bug fix. It covers harmful outputs, policy breaches, unsafe recommendations, data-handling concerns, failed review steps, retrieval faults, prompt manipulation and other events in which a particular run requires formal reconstruction and corrective action.

Conceptually, incident response sits at the point where governance principles are tested by concrete events. Many AI governance frameworks state that systems should be safe, accountable and auditable, but incident response asks what an organisation can actually do when an output is contested or harm is alleged. RAIDT makes this actionable by treating the run as the unit of governance. A run is a specific use of a GenAI system for a defined task, at a specific time, in a specific context. That framing allows investigators to examine what was configured, what evidence was produced and how decisions were made.

This item differs from adjacent concepts. It is not the same as general risk management, because it addresses an actual or suspected event rather than a prospective risk register. It is not identical to post-market monitoring, because incident response is often immediate and case-specific, whereas monitoring is broader and longitudinal. It is not just internal audit, because the purpose is not only assurance after the fact but timely reconstruction, containment and improvement. Inside RAIDT, incident response belongs centrally because run-level evidence, evidence packs and score profiles provide the practical substrate for reviewability, contestability and continuous improvement.

Why this concept matters

Incident response matters because organisations using GenAI need a credible way to move from abstract commitments to operational accountability. When something goes wrong, the key governance question is not whether principles existed in policy, but whether the organisation can reconstruct the event, explain what happened, identify control weaknesses and implement proportionate corrective action. Without that capability, governance remains declarative.

The concept also prevents a common failure mode in GenAI deployment: treating a disputed output as an isolated anomaly with no structured learning loop. If incident response is weak, organisations may over-blame the model, over-blame the user, or settle for anecdotal explanation. RAIDT reduces that ambiguity by anchoring the inquiry in run-level evidence, so the discussion can move from opinion to documented sequence, context and control performance.

Key idea: Incident response matters because RAIDT turns a problematic GenAI event into a reconstructable, reviewable and improvable governance case at the level of the individual run.

What this item enables

Formal classification of a problematic run as an incident, near miss or review-triggering event.
Reconstruction of the prompt, retrieval, model, output, reviewer and workflow chain associated with that run.
Identification of where control failure, ambiguity or weak assurance occurred.
Assignment of responsibility for investigation, escalation, remediation and follow-up.
Translation of one disputed event into evidence pack updates, score-profile interpretation and organisational learning.
Stronger audit readiness by linking incident handling to documented evidence rather than retrospective narrative alone.

Practical example / likely audience question

Audience question

What if a harmful output occurs? Is that simply a model failure, or can RAIDT actually support incident response in a useful way?

Answer

The concern behind this question is that many organisations assume incident response for GenAI is either too technical for governance teams or too ambiguous for meaningful review. The direct answer is that RAIDT is useful precisely because it narrows the scope of inquiry to a specific run and the evidence associated with it. Instead of debating the system in the abstract, the organisation can reconstruct what the prompt asked for, what context was retrieved, which model or configuration was used, what output was produced, whether a human review step occurred and what happened next.

For example, if a drafting assistant generates a harmful recommendation, a generic governance approach may only record that "the AI produced an unsafe answer". RAIDT supports a more precise response: investigators can examine whether the retrieval source was outdated, whether the prompt omitted a safeguard, whether the reviewer missed a warning, or whether the use context itself was inappropriate. That is better than a generic approach because it creates evidence-backed causal explanation rather than broad speculation, and it supports both immediate remediation and control redesign.

Practical example in RAIDT terms

In healthcare, a hospital uses a GenAI assistant to draft discharge summaries for clinicians. In one run, the draft includes an incorrect medication instruction that is almost sent to a patient. The run-level issue is not merely that the model was "wrong"; it is that a specific configured use, in a specific clinical context, produced a near miss requiring investigation. RAIDT would require evidence such as the prompt template, retrieved clinical guidance, model version, time stamp, user role, draft output, edits made by the clinician, review checkpoints and escalation record. The most affected pillars are Dependability, Traceability, Auditability and Responsibility, with Interpretability also relevant if the rationale for the suggestion is unclear. Incident response improves governance readiness here because the organisation can reconstruct the event, classify the control weakness, document the remedial action and show that learning feeds back into safer future runs.

Detailed link to RAIDT

Incident response links to RAIDT in four ways.

First, it expresses RAIDT's core idea that governance should be based on evidence from actual use rather than only on policy statements or design-time claims.
Second, it depends on the run as the unit of analysis, because incident response becomes precise only when a single configured use can be reconstructed.
Third, it turns run-level evidence into a usable evidence pack and a meaningful RAIDT score profile, showing where governance performance was strong or weak.
Fourth, it connects individual events to reviewability, contestability, audit readiness and organisational learning, so the outcome of an incident is not only correction but improved governance capacity.

Incident response -> Run-level evidence -> Evidence pack -> RAIDT score profile -> Governance readiness

This chain matters because incident response is one of the clearest demonstrations that RAIDT is not just descriptive. It creates a route from a contested event to documented review, score-informed diagnosis and governance improvement.

Link to the five RAIDT pillars

Responsibility

Incident response clarifies who is accountable for receiving, triaging, investigating and resolving a problematic run.