Q156 - What_do_gating_monitoring_post-run_review_and_corrective_act

Q156 — What do gating, monitoring, post-run review, and corrective action look like in RAIDT operations?

← RAIDT · Star S8 - Implementation and Operations · primary item: S8.04 · Gating

Appears in sources
Answer

In RAIDT operations, gating is the point-of-use decision rule, monitoring is the longitudinal reading of evidence and scores over time, post-run review is the reconstruction of selected or disputed runs, and corrective action is the governance response to weak pillars. The operating object throughout is the run-level evidence pack, because RAIDT treats the run as the unit of governance. Each run yields a score profile across the five pillars (Responsibility, Auditability, Interpretability, Dependability, Traceability). That profile is used both synchronically, to decide whether a run may proceed, and longitudinally, to detect recurring weaknesses across workflows, vendors, or influence configurations.

Operationally, organisations can sample runs much as auditors sample transactions. Monitoring combines routine score tracking, repeat-run testing for variance, and scrutiny of whether evidence fields remain complete as prompts, retrieval, adapters, or alignment settings change. Post-run review is especially important for high-impact, contested, or anomalous cases because it allows a later reviewer to inspect evidence pointers, oversight actions, and integrity-protected artefacts rather than relying on memory. Corrective action is targeted rather than generic: low auditability or traceability leads to improved logging and provenance capture; low responsibility leads to tighter constraints, explicit escalation rules, or mandatory human review; low dependability leads to stability testing, configuration control, or model and prompt adjustments. Across manual, partial, and higher automation, the mechanics differ, but the same RAIDT logic holds: monitoring and correction are evidence-based, and a model-generated narrative never replaces the evidence pack as the source of truth.

Practical example

In cybersecurity incident triage, a GenAI assistant helps summarise alerts and suggest next steps. Operations staff capture a run-level evidence pack for each significant triage event, including the prompt, model settings, retrieved threat-intelligence references, output, and any analyst override. The gate allows the suggestion to enter the analyst workflow only when the score profile meets the team threshold, especially for dependability and traceability.

Over time, monitoring shows that some runs become unstable when retrieval results conflict. A post-run review reconstructs those cases and finds that retrieval pointers were incomplete. Corrective action is then specific: preserve retrieval snapshots, tighten the prompt to require uncertainty statements when evidence conflicts, and keep human review mandatory until repeat-run stability improves. RAIDT operations therefore look like a control loop rather than a one-off checklist.

Sources in RAIDT papers
Powered by Forestry.md