S8.07 - Corrective_action

S8.07 — Corrective action

flowchart LR
    A[Weak run / low score / review concern] --> B[RAIDT
run-level evidence framework]
    A2[Informal fixes / undocumented changes] --> B
    B --> C[[Corrective action
evidence-triggered improvement]]
    C --> D[Evidence pack update]
    C --> E[Score profile revision]
    C --> F[Prompt, retrieval, logging,
or human review changes]
    D --> G[Reviewer reconstruction]
    E --> H[Governance readiness]
    F --> H
    I[Healthcare / finance / public services / enterprise workflows] --> C

← Star S8 - Implementation and Operations

Star context: Shows how RAIDT is embedded in day-to-day implementation and governance routines, including how weak runs are diagnosed, reviewed, and improved through practical operational controls.

Academic picture

Definition / background

Corrective action is the structured response taken after a RAIDT review identifies a weakness, incident, evidence gap, or low pillar score in a specific run. In practical terms, it may involve changing prompts, improving retrieval configuration, tightening logging, adding human review, clarifying escalation rules, or altering the surrounding workflow. The important point is that the action is not merely technical repair; it is a documented governance response to evidence generated by the run.

Conceptually, corrective action sits between assessment and improvement. Many AI governance frameworks can identify principles, risks, or control requirements, but they are often less precise about what happens after a weak result is found. RAIDT fills that operational gap by treating the run as the unit of governance. A run generates evidence, that evidence supports review, and the review can trigger a corrective action that is specific enough to be scrutinised, repeated, or contested.

Corrective action is related to, but distinct from, broader terms such as remediation, incident response, or continuous improvement. Incident response is often about immediate containment. Remediation can describe wider repair after harm or non-compliance. Continuous improvement is a general management idea. In RAIDT, corrective action is narrower and more operational: it is the targeted improvement step linked to identifiable weaknesses in run-level evidence.

This concept belongs inside RAIDT because RAIDT does not stop at producing a score profile. Its purpose is to move governance from broad assertions to reviewable intervention. Corrective action therefore connects the run-level evidence pack, the five-pillar profile, and the organisation's next decision about whether to change, rerun, escalate, or gate the system. It is one of the points where RAIDT turns diagnosis into accountable governance practice.

Why this concept matters

Corrective action matters because governance is incomplete if an organisation can detect weakness but cannot show how it responded. A score profile on its own may reveal that a run was poorly evidenced, difficult to interpret, or not dependable enough for the task. Without corrective action, those findings remain descriptive rather than operational.

The concept also prevents a common organisational confusion: the belief that evaluation is valuable only if it certifies success. RAIDT takes the opposite view. Weak runs are informative because they show where evidence, design, or oversight needs to improve. Corrective action turns poor performance, weak documentation, or reviewer concern into a basis for change rather than a dead end.

If corrective action is missing, organisations tend to rely on informal patching, undocumented prompt tweaks, or local workarounds. That weakens audit readiness, makes reviewer reconstruction difficult, and reduces confidence that similar future runs will be governed better. By contrast, documented corrective action supports institutional memory and makes governance cumulative rather than episodic.

Key idea: Corrective action matters because RAIDT is not only about identifying weak runs, but about showing how evidence leads to accountable improvement.

What this item enables

It enables weak run findings to be translated into concrete governance actions rather than left as abstract observations.
It enables a direct connection between evidence gaps, low pillar scores, and specific changes to prompts, retrieval, review steps, or workflow controls.
It enables named accountability by recording who authorised, implemented, or verified the response.
It enables comparison between pre-action and post-action runs so that improvement can be evidenced rather than assumed.
It enables organisational learning by turning one problematic run into a reusable lesson for future deployments.
It enables RAIDT to function as a governance loop, not just a measurement exercise.

Practical example / likely audience question

Audience question

Does RAIDT do anything beyond scoring a run, or does it also tell an organisation what to change when a run performs badly?

Answer

The concern behind this question is that many governance tools diagnose problems but do not support practical follow-through. A supervisor, examiner, or manager may worry that RAIDT produces a neat dashboard while leaving operational teams uncertain about the next step. That would limit its value because diagnosis without action is only partial governance.

The direct answer is that RAIDT is designed to support more than scoring. Its outputs include not only a five-pillar score profile and a run-level evidence pack, but also the identification of evidence gaps, control weaknesses, and review findings that can justify corrective action. In other words, RAIDT provides a basis for deciding what should be changed and why.

A practical example would be an LLM-assisted drafting workflow where reviewers repeatedly find unsupported claims in generated outputs. RAIDT does not simply record that the run scored poorly on Dependability and Traceability. It also makes visible which prompt version was used, what sources were retrieved, what reviewer concerns were raised, and where the evidence trail failed. That makes it possible to define a corrective action such as mandatory citation display, tighter retrieval filters, and an additional human sign-off for high-risk outputs.

RAIDT handles this better than a generic AI governance approach because the corrective action is tied to a specific run and its evidence. A general governance framework might recommend "improve oversight" or "reduce hallucinations" in broad terms. RAIDT can show which run failed, what evidence supported that judgement, which pillars were affected, and what operational changes should be reviewed in the next run.

Practical example in RAIDT terms

Consider a healthcare setting in which a generative AI tool helps draft discharge summaries for clinicians. One particular run produces a summary that omits an important medication change and presents one recommendation without a clear supporting source. During RAIDT review, the run-level evidence shows that retrieval pulled an outdated document set, the prompt did not force explicit source attribution, and the final output was accepted without a mandatory secondary check.

The evidence needed for this assessment includes the prompt version, retrieval configuration, source list, timestamps, user and reviewer actions, the generated output, and reviewer comments explaining the identified weakness. The most affected RAIDT pillars are Dependability, because the output was not sufficiently reliable for the task, Traceability, because the evidential route to the recommendation was weak, and Responsibility, because the workflow did not clearly require an escalation or review step when confidence was low.

Corrective action in this case could include revising the retrieval rules, requiring source-linked output formatting, adding a clinician verification gate for medication changes, and improving logging so that future reviewers can reconstruct the run more easily. Governance readiness improves because the organisation can show not only that the weakness was detected, but also that it was analysed, acted on, and subsequently monitored in a structured way.

Detailed link to RAIDT

Corrective action links to RAIDT in four ways.

First, it connects to RAIDT's core idea that governance should be organised around the run rather than around vague claims about a system in general. Corrective action is meaningful because it is triggered by evidence from a particular configured use in a particular context.

Second, it connects directly to run-level evidence. The reason for acting should be visible in the run record itself: what was prompted, what was retrieved, what was produced, what reviewers observed, and where the weakness appeared.

Third, it connects to the RAIDT evidence pack and score profile. The evidence pack explains why action is needed, while the score profile helps locate which governance dimensions were most affected and therefore where improvement effort should be concentrated.

Fourth, it connects to reviewability, contestability, audit readiness, and organisational learning. A corrective action can be challenged, justified, compared across runs, and used to improve future governance routines. That is how RAIDT supports movement from one-off evaluation to institutional capability.

Corrective action → Run-level evidence → Evidence pack → RAIDT score profile → Governance readiness

In this chain, corrective action is the bridge between identifying a weak run and showing that the organisation can respond in a disciplined, reviewable manner.

Link to the five RAIDT pillars

Corrective action affects all five RAIDT pillars, but it is especially strong in relation to Responsibility, Dependability, and Traceability because these are the pillars most directly involved when weak runs must be diagnosed and improved.

Responsibility

Corrective action supports Responsibility by making clear who must respond to a weak run, who approves the change, and who is accountable for verifying that the issue has been addressed.