Q169 - How_does_RAIDT_support_internal_audit

Q169 — How does RAIDT support internal audit?

← RAIDT · Star S9 - Policy, Standards and Assurance · primary item: S9.09 · Internal audit

Appears in sources
Answer

RAIDT supports internal audit by treating the run as the unit of governance, rather than relying only on policy statements, model cards, or high-level control descriptions. Across the papers, the central claim is that internal audit needs inspectable evidence for one configured use in context: what prompt or template was used, which model and tools were active, whether retrieval was enabled, what output was produced, and what human or automated checks followed. The run-level evidence pack therefore gives auditors a bounded record that can be sampled, reconstructed, and challenged later. This directly addresses the policy-to-operation gap identified in the motivation paper, where internal audit is said to need more than a policy declaration and more than a checklist showing that controls exist in principle.

RAIDT then makes review more consistent by scoring each run across the five pillars (Responsibility, Auditability, Interpretability, Dependability, Traceability) and retaining the full score profile rather than collapsing governance to a single headline number. The papers specify anchors 1=missing / 3=partial / 5=audit-ready, which allows internal audit to assess evidence completeness and control quality in a disciplined way and to compare runs across teams, workflows, and vendors. This is valuable for sampling, exception handling, complaint review, and post-incident learning, because repeated low scores reveal systemic weaknesses in logging, provenance, oversight, or explanation. RAIDT also matters because influence methods as governance interventions can materially alter evidence quality and reviewability. Structured prompting, retrieval augmentation, PEFT or LoRA, and preference-based alignment may produce different evidence profiles, so auditors can compare not only outputs but the governability of different configurations.

Practical example

In a healthcare note-summarisation workflow, an internal audit team could sample a set of discharged-patient summaries generated over one month. For each sampled run, the auditor would inspect the run-level evidence pack: prompt template ID, model deployment ID, decoding settings, any retrieval snapshot hash, the output text and hash, and the recorded safety check and human oversight flag. Using the RAIDT score profile, the team could judge whether the summary was reviewable and whether uncertainty or escalation was recorded when results were pending.

If several sampled runs show acceptable clinical wording but weak Auditability or Traceability, the audit finding would not be that the model is universally unsafe. It would be that the workflow is insufficiently governed because evidence is incomplete or reconstruction is too difficult. The organisation could then tighten logging, preserve retrieval snapshots, or require stronger oversight before release. RAIDT therefore helps internal audit move from abstract assurance to targeted remediation based on comparable run-level evidence.

Sources in RAIDT papers
Powered by Forestry.md