S3.04 - Evidence_at_point_of_use

S3.04 ? Evidence at point of use

flowchart LR
    A[Retrospective reporting
and memory-based accounts] --> B[RAIDT
run-level evidence framework]
    P[Prompts timestamps
model IDs sources approvals] --> C[[Evidence at point of use]]
    Q[Healthcare finance education
public administration workflows] --> C
    B --> C
    C --> D[Run-level evidence pack]
    C --> E[RAIDT score profile]
    C --> H[Reviewability contestability
audit readiness]
    D --> F[Reviewer reconstruction]
    E --> G[Governance readiness]
    H --> G

← Star S3 - Run-Level Evidence Logic

Star context: Positions evidence capture at the moment of action within RAIDT's run-level proof logic, so that each run can later be reconstructed, compared, challenged, and governed on the basis of artefacts rather than retrospective narrative.

Academic picture

Definition / background

Evidence at point of use means that the relevant artefacts, metadata, decisions, and contextual markers of a generative AI run are captured at the time the run occurs, or as close to that moment as the workflow allows. In RAIDT, this is a foundational operational principle because the framework governs a concrete run rather than an abstract system claim. A run is one configured use of a GenAI system for a specific task, at a specific time, in a specific context; if the evidence is not captured at that point, later claims about what happened become harder to verify and easier to contest.

Conceptually, the idea draws on long-standing governance concerns around contemporaneous records, provenance, audit trails, and evidential integrity. What RAIDT adds is a run-level formulation tailored to generative AI work. The issue is not simply whether an organisation has a policy, a model card, or a general assurance statement. The issue is whether a particular use of a GenAI system can be reconstructed from evidence that was generated during the run itself.

This distinguishes evidence at point of use from retrospective reporting. A report written afterwards may still be useful, but in RAIDT it is secondary evidence unless it points back to the primary artefacts generated in the run. Those artefacts may include the task framing, prompt or instruction set, user identity or role, model and configuration details, timestamps, source materials, retrieved context, output versions, approval actions, and any exceptions or interventions. The closer these are captured to the point of use, the stronger the evidential basis for review.

Within RAIDT, this matters because the run-level evidence pack and the five-pillar score profile depend on evidence quality, not merely on policy aspiration. Responsibility, Auditability, Interpretability, Dependability, and Traceability can only be scored credibly when there is stable run-level evidence to examine. Evidence at point of use therefore belongs inside RAIDT as a core condition for operational governance, not as a peripheral documentation preference.

Why this concept matters

This concept addresses a persistent governance failure in organisational GenAI use: the tendency to explain high-stakes runs after the event without preserving the materials needed to verify those explanations. When that happens, review becomes dependent on memory, convenience, and selective summary. RAIDT avoids this by insisting that evidence should arise from the run itself.

The concept also prevents confusion between documentation and evidence. Many organisations produce documents about AI use, but those documents do not necessarily prove what happened in a given run. Evidence at point of use narrows that gap by treating prompts, retrieved sources, model settings, timestamps, human interventions, and decision checkpoints as primary artefacts rather than optional administrative notes.

If this discipline is missing, several risks appear at once: weak audit readiness, poor contestability, limited reconstruction after incidents, reduced confidence in scoring, and reduced organisational learning. A governance framework may look mature at policy level while remaining fragile in practice because it cannot evidence how a particular run unfolded.

For organisations using GenAI in professional work, the concept turns governance from principle to operation. It supports reviewable use, proportionate oversight, and defensible decisions about whether a run was acceptable, improvable, or unacceptable.

Key idea: evidence at point of use matters because RAIDT can only govern a run credibly if the evidence is captured while that run is actually happening.

What this item captures

The requirement that evidence should be captured during the run, not reconstructed later from memory.
The distinction between primary run artefacts and secondary narrative summaries.
The minimum evidential conditions for reconstructability, challenge, and comparison.
The operational link between evidence capture and the RAIDT evidence pack.
The dependence of the five-pillar score profile on contemporaneous, stable, reviewable artefacts.
The organisational discipline needed to make GenAI use auditable at the level of real work.

Practical example / likely audience question

Audience question

Why not write a report afterward?

Answer

The concern behind this question is understandable: if staff can explain later what they did, it may seem unnecessary to capture detailed evidence during the run itself. The difficulty is that retrospective reporting is vulnerable to omission, simplification, hindsight bias, and unstable identifiers. People may remember the general aim of a run while forgetting the exact prompt wording, retrieved context, model version, approval path, or intermediate output that shaped the final result.

The direct answer is that a report written afterwards is useful only when it points back to the artefacts created during the run. In RAIDT terms, the report is not a substitute for run-level evidence; it is an interpretive layer built on that evidence. Without the underlying artefacts, reviewers cannot reliably reconstruct the run, compare it with similar runs, or challenge whether the account is accurate.

A practical example is a team using a GenAI tool to draft a supplier-risk summary. If the team later writes, "the model suggested moderate risk based on the available documents," that statement is too weak on its own. RAIDT would ask what documents were retrieved, which model and configuration were used, what the prompt requested, what output was first produced, what human edits followed, and who approved the final use. Generic AI governance often stops at policy compliance or broad usage guidance. RAIDT handles the issue better because it ties governance to evidence from the actual run, making post-run explanation accountable to recorded artefacts.

Practical example in RAIDT terms

Consider a healthcare setting where a clinician uses an approved GenAI assistant to draft a discharge summary from structured notes and recent observations. The run-level issue is that the summary may influence patient communication and continuity of care, yet the final text alone does not reveal how it was produced.

Evidence at point of use would require capture of the prompt template, model identifier and version, time of use, clinician role, source records accessed, any retrieval or context window used, draft output, subsequent edits, approval or sign-off step, and the final stored document reference. If an anomaly later appears, reviewers can test whether the model introduced unsupported wording, whether the source record was incomplete, or whether a human editor removed an important warning.

The most affected RAIDT pillars are Auditability and Traceability, with strong implications for Responsibility and Dependability. Auditability improves because the run can be reconstructed. Traceability improves because the final summary is linked to identifiable run artefacts. Responsibility improves because human roles and approvals are visible. Dependability improves because recurrent failure modes can be detected across runs. This makes the organisation more governance-ready than a process in which only the finished document survives.

Detailed link to RAIDT

Evidence at point of use links to RAIDT in four ways.

First, it operationalises RAIDT's core idea that governance should focus on what happened in an actual run rather than on general claims about a system.
Second, it supplies the contemporaneous artefacts that make run-level evidence credible and reviewable.
Third, it strengthens both the evidence pack and the RAIDT score profile by giving assessors stable material on which to base judgement.
Fourth, it supports reviewability, contestability, audit readiness, and organisational learning because later review is anchored in recorded artefacts rather than recollection.

Evidence at point of use ? Run-level evidence ? Evidence pack ? RAIDT score profile ? Governance readiness

In this sense, the item is not merely about logging. It is about creating the evidential conditions under which RAIDT can function as a practical governance framework.

Link to the five RAIDT pillars

Responsibility

Evidence at point of use clarifies who initiated, configured, reviewed, approved, or overrode a run. It therefore supports accountable human oversight rather than diffuse responsibility.