Q208 - Compliance-measurement_gap_definition_example_and_why_it_mat
Q208 — Compliance-measurement gap — definition, example, and why it matters in RAIDT
← RAIDT · Star C0 - RAIDT Core, Definition, Values, Claims and Innovation · primary item: C0.06 · Governance readiness
B. Background & Problem | Ordered by mind-map priority: inner circles first, then operational detail.
Appears in sources
workshop_dense_100#slide 31
Answer
The compliance-measurement gap is the problem that governance regimes increasingly demand demonstrable evidence of control, oversight, and traceability, while many GenAI deployments still produce only high-level documentation or output-facing explanations. In the RAIDT papers, the gap appears when organisations can state that they govern AI responsibly, but cannot measure that claim for an individual run because the relevant artefacts were never preserved. This is especially acute in GenAI because deployed behaviour depends on prompt design, retrieved context, tool use, adaptation layers, and alignment settings, all of which may change outcomes materially. Without a replayable record, compliance remains difficult to inspect and governance can drift into compliance theatre.
The reason it matters in RAIDT is that the framework is designed precisely as the measurement hinge between principle and practice. RAIDT supplies a portable governance object, the run-level evidence pack, and evaluates it through a score profile grounded in the five pillars (Responsibility, Auditability, Interpretability, Dependability, Traceability). That makes governance claims testable across standards, audits, procurement, and internal review. It also explains why influence methods as governance interventions must be logged: if retrieval or alignment changes behaviour but is not recorded, measurement breaks down even when a policy appears compliant. RAIDT therefore treats the gap not as a rhetorical issue, but as an operational failure of evidence capture and comparability.
Practical example
A public-service team uses GenAI to advise on benefit eligibility. Its policy says decisions must be traceable and reviewable, yet staff only keep the final answer shown on screen. When a claimant challenges the advice, the team cannot show the exact rule passages retrieved, the prompt version, or whether the system was updated between runs. The organisation may have complied on paper, but it cannot measure or defend that compliance for the disputed case.
With RAIDT, the advice would be stored as a run-level evidence pack and assessed through the relevant pillars. The challenge could then be answered with concrete run evidence rather than general assurances, which is exactly why closing the compliance-measurement gap matters.
Sources in RAIDT papers
08-RAIDT_Foundations_M_V5011-RAIDT_Academic_Logic_M_v11