Q064 - How_should_supervisors_read_trade-offs_when_deciding_whether

Q064 — How should supervisors read trade-offs when deciding whether a run is governance-ready?

← RAIDT · Star S5 - RAIDT Pillars and Scoring · primary item: S5.12 · Trade-offs

Governance readiness depends on whether the profile fits the task, risk, and review burden.

Appears in sources
Answer

Supervisors should read trade-offs through the score profile, not through a single headline number. The scoring paper states that a composite can be reported, but the profile remains primary because trade-offs are often the substantive governance result. A run may be quite interpretable yet still weak on Auditability or Traceability; alternatively, it may be well logged but not sufficiently dependable under repeat runs. For governance-ready judgement, the central question is therefore whether the run-level evidence pack supports justified reliance for the stated task, not whether the output merely appears persuasive.

In practice, supervisors should interpret the anchors 1=missing / 3=partial / 5=audit-ready conservatively, especially in higher-impact settings. A low or middling score in a critical pillar is not an administrative nuisance; it is evidence that some governance claim cannot yet be justified. If retrieval was used without preserved snapshots, high Auditability and Traceability scores are not warranted, even when the answer cites plausible sources. If oversight is unclear, Responsibility cannot be assumed from output quality alone.

This means governance-ready status should be read as a threshold decision tied to context and escalation rules. RAIDT scoring assesses governance readiness of the run as the unit of governance; it does not certify factual correctness or legal compliance by itself. Supervisors should therefore ask whether the score profile is coherent for the risk level, whether weak pillars are tolerable in that workflow, and whether improvement actions or human escalation are required before the run can support operational use.

Practical example

Consider a hospital discharge-summary assistant. A structured prompt and explicit uncertainty statement may raise Interpretability, while retrieval from an internal guideline corpus may improve Traceability. However, if the retrieval snapshot hash is missing, a supervisor should not treat the run as governance-ready for clinical record use merely because the summary reads well. Under RAIDT, the weak pillar matters because the organisation may be unable to reconstruct what evidence informed the text.

A supervisor would therefore read the score profile as a decision aid. Strong Interpretability with partial Auditability and Traceability suggests a useful but not yet audit-ready workflow. The correct response is not to average away the weakness; it is to require the missing retrieval evidence, confirm reviewer sign-off, and then reassess the run-level evidence pack before allowing routine deployment.

Sources in RAIDT papers
Powered by Forestry.md