Q231 - Review_decision_definition_example_and_why_it_matters_in_RAI

Q231 — Review decision — definition, example, and why it matters in RAIDT

← RAIDT · Star S4 - Evidence Architecture and Artefacts · primary item: S4.16 · Review decision and reviewer notes

D. Evidence Architecture | Ordered by mind-map priority: inner circles first, then operational detail.

Appears in sources
Answer

In RAIDT, a review decision is the documented oversight outcome attached to one configured run: the recorded judgement about whether and how a generated output was used after human review. Because RAIDT defines the run as the unit of governance, the review decision belongs in the run-level evidence pack alongside prompts, configuration provenance, outputs, and traceability fields. The papers do not treat review as a peripheral workflow step. Rather, they present oversight and decision recording as a core element of governance readiness, because it preserves what humans reviewed, what they changed, what they approved, and what was escalated. On that basis, a review decision can indicate that an output was accepted, edited, rejected, escalated, or linked to incident handling, with the associated rationale preserved in reviewer notes.

A concrete RAIDT example would be a cybersecurity alert-triage run. The model produces a recommendation to close an alert as benign; the analyst reviews the evidence, notices that the retrieved indicators are incomplete, and changes the decision to escalate for investigation. The review decision is therefore not merely a status flag. It is the recorded bridge between generated content and organisational action.

Why it matters is straightforward in RAIDT terms. Review decisions make accountability attributable, contestability possible, and governance comparable across runs. They help convert fragmented traces into a governance-ready score profile and support the five pillars (Responsibility, Auditability, Interpretability, Dependability, Traceability). They also align with the papers’ treatment of influence methods as governance interventions: if a prompt, retrieval setup, adapter, or alignment layer repeatedly leads to rejection or escalation, the organisation has evidence for redesign rather than relying on anecdote. In that sense, review decisions are a mechanism for both control and learning, and they are central to anchors 1=missing / 3=partial / 5=audit-ready.

Practical example

In an HR shortlist-justification workflow, a GenAI system drafts reasons for why a candidate appears to meet selection criteria. The recruiter reviews the draft and sees that one justification rests on an ambiguous phrasing in the candidate’s CV. The recruiter removes that claim, records a note that the evidence is insufficient, and marks the output as edited before limited internal use.

If a later complaint alleges unfair treatment, the organisation can reconstruct the run-level evidence pack: prompt template, criteria used, model and adapter identifiers, output, reviewer note, and the review decision itself. That record shows that the organisation did not treat the model’s draft as self-validating. It applied human oversight, constrained reliance, and preserved a reviewable trail. In RAIDT, that is precisely why review decisions matter: they make one disputed organisational use intelligible, contestable, and governable.

Sources in RAIDT papers
Powered by Forestry.md