Q127 - What_is_a_run-level_evidence_pack

Q127 — What is a run-level evidence pack?

← RAIDT · Star C0 - RAIDT Core, Definition, Values, Claims and Innovation · primary item: C0.04 · Evidence pack

Appears in sources
Answer

A run-level evidence pack is RAIDT?s auditable record of a single configured use of a GenAI system. It is stored either as one record or as linked records with stable identifiers, and it exists so that a later reviewer can reconstruct the run without having been present. The papers distinguish this from model-level documentation and from narrative assurance. A model card may describe a system in general terms, but a run-level evidence pack records what actually happened in a specific event: which prompt and template were used, which model and settings were active, whether tools or retrieval were invoked, what output was produced, and what checks or oversight steps followed.

Its function is both evidential and evaluative. Evidentially, it supports replayability, post-incident review, contestability, and audit sampling. Evaluatively, it is the object scored in the RAIDT rubric, allowing a score profile to be assigned to the five pillars (Responsibility, Auditability, Interpretability, Dependability, Traceability). The pack therefore embodies the run as the unit of governance and makes it possible to compare runs across contexts and influence configurations. When the papers discuss complete, review-supporting evidence, they are describing the high end of the anchors 1=missing / 3=partial / 5=audit-ready. In short, a run-level evidence pack is the mechanism by which responsible use becomes reconstructable rather than merely asserted.

Practical example

In finance, a bank may use GenAI to draft an adverse action explanation after a credit refusal. A run-level evidence pack for that event would include the reason-code template version, the prompt and model versions, any policy or safety-filter identifiers, the generated explanation, and the reviewer who checked that the text matched the decision record. If retrieval or a tool was used, those artefacts would also be preserved.

When a customer contests the explanation, the bank can inspect the run-level evidence pack to show which criteria informed the draft and whether the system stayed within the approved template. That makes the run reviewable in a way that a copied paragraph alone never could.

Sources in RAIDT papers
Powered by Forestry.md