S5.05 - Traceability

S5.05 — Traceability

flowchart LR
    A[Fragmented evidence and weak lineage] --> B[RAIDT
Run-level evidence framework]
    B --> C[[Traceability
Links inputs, sources, versions, tools, outputs, and review decisions]]
    H[Healthcare, finance, public services, research, enterprise work] --> C
    I[Prompts, source IDs, retrieval logs, model IDs, output hashes, approval records] --> C
    C --> D[Evidence pack]
    C --> E[Score profile]
    C --> F[Reviewer reconstruction]
    C --> G[Governance readiness]
    D --> G
    E --> G
    F --> G
    C --> J[Contestability and organisational learning]

← Star S5 - RAIDT Pillars and Scoring

Star context: Positions Traceability within RAIDT's five-pillar scoring model by showing how a run can be reconstructed from linked evidence rather than defended through general claims alone.

Academic picture

Definition / background

Traceability is the extent to which a specific generative AI run can be linked, step by step, to the evidence that shaped it. In RAIDT, this means that outputs should be connected to the task context, prompt version, source materials, retrieval results, model and adapter identifiers, tool settings, human review actions, and final approval or release decisions. A traceable run is therefore one whose evidential chain can be followed rather than guessed.

Conceptually, traceability draws on ideas from audit trails, provenance, records management, software configuration control, and data lineage. In generative AI governance, however, these traditions often remain fragmented. One team may keep prompt logs, another may store retrieved documents, and another may record approvals, yet no one can reconstruct the full run after the fact. RAIDT brings those fragments together by treating the run itself as the governance object.

Traceability is related to, but different from, several nearby concepts. It is not the same as auditability: auditability concerns whether evidence can be inspected and assessed, whereas traceability concerns whether that evidence remains linked across the run chain. It is not the same as interpretability: interpretability helps explain model behaviour or output meaning, whereas traceability helps establish what artefacts, versions, and decisions produced the output in the first place. It is also not reducible to transparency, because transparency can remain broad and principle-level, while traceability demands concrete run-level linkage.

Within RAIDT, Traceability belongs inside the five-pillar score profile because governance readiness depends not only on having evidence, but on being able to connect that evidence coherently. A run-level evidence pack with weak linkage is much less useful for review, challenge, or learning. Traceability therefore supports both the practical construction of the evidence pack and the defensibility of the score profile that summarises run readiness.

Why this concept matters

Traceability solves a frequent governance problem in organisational use of generative AI: evidence may exist, but it is too fragmented to support review. An organisation may retain prompts, outputs, or policy documents, yet still be unable to answer basic questions such as which source informed the response, which model version generated it, whether retrieval changed between runs, or who approved the final text.

When traceability is weak, several risks follow. Errors become harder to investigate, contested outputs become harder to reconstruct, citation claims become harder to verify, and lessons from one run become difficult to transfer into organisational improvement. Weak traceability also makes formal assurance harder because reviewers cannot reliably move from an outcome back to the inputs and decisions that shaped it.

For RAIDT, the importance of Traceability is that it converts governance from assertion to evidence. Rather than saying that a system is documented, controlled, or responsible, the framework asks whether the run leaves behind a connected evidential pathway that others can inspect, challenge, and learn from.

Key idea: Traceability matters because governance becomes reviewable only when a run's evidence remains connected from input to output to decision.

What this item measures

Whether a run's outputs can be linked back to the prompts, sources, retrieved content, model versions, tools, and review decisions that shaped them.
Whether identifiers, timestamps, hashes, and version records are complete enough to reconstruct the run later.
Whether evidence is connected across the run chain rather than stored as isolated artefacts.
Whether human interventions, overrides, approvals, and edits are recorded in a way that preserves lineage.
Whether the resulting evidence is durable enough to support contestability, audit preparation, and organisational learning.

Practical example / likely audience question

Audience question

Is Traceability just another word for keeping system logs?

Answer

That question usually reflects a common misconception: that once an organisation records prompts and outputs, the governance problem is solved. In practice, logs alone are rarely enough. They may capture events, but not the relationships between those events. A reviewer may see that a prompt existed, that an output was generated, and that a reviewer signed off, while still being unable to show which retrieved sources informed the response, whether the model version changed between runs, or how a human editor altered the text before release.

In RAIDT, Traceability is stronger than simple logging because it requires linkage across artefacts. The direct answer is therefore no: logs may contribute to traceability, but they do not automatically provide it. Traceability exists when a later reviewer can move from a particular output to the exact inputs, source identifiers, model configuration, tool chain, and review actions associated with that specific run.

A practical example is a policy team using a retrieval-augmented assistant to draft compliance guidance. If the team stores only prompts and final outputs, it may not be able to explain which policy clauses were retrieved, whether the knowledge base changed that morning, or whether the reviewer corrected a misleading statement before publication. RAIDT handles this better than generic AI governance approaches because it ties the question to a single run and asks for concrete run-level evidence, not general documentation about the system as a whole.

Practical example in RAIDT terms

Consider a healthcare setting in which a hospital uses a generative AI tool to draft discharge advice based on clinician instructions, local policy documents, and a retrieval layer connected to approved guidance. The run-level issue arises when a clinician notices that the discharge advice includes wording that does not align with the patient's medication history and asks how that wording entered the final draft.

In RAIDT terms, the evidence needed would include the task description for that run, the prompt template and its version, the patient-safe source set or policy documents retrieved, retrieval identifiers or hashes, the model and adapter IDs, generation settings, the draft output, any clinician edits, the final approved version, and the reviewer decision that released the text. Without those links, the organisation may know that the tool was used but remain unable to reconstruct the path from instruction to final advice.

This example affects all five pillars. Responsibility is involved because someone must define who reviews and approves the run. Auditability is involved because the evidence must be inspectable. Interpretability is involved because reviewers may need to explain why particular wording appeared. Dependability is involved because repeated safe performance depends on stable controls. Traceability is central because it ties every artefact together and makes the run reconstructable. In governance-readiness terms, stronger traceability turns an isolated output into an evidence-backed decision object.

Detailed link to RAIDT

Traceability links to RAIDT in four ways.

First, it supports RAIDT's core idea that governance should focus on the run, not only on high-level principles or system-wide claims.
Second, it makes the run reviewable by preserving the evidence chain from task context to output and approval.
Third, it strengthens both the evidence pack and the score profile, because linked evidence is easier to assess than disconnected records.
Fourth, it supports contestability, audit readiness, and organisational learning by allowing later reviewers to reconstruct what happened and identify where controls should improve.

Traceability → Run-level evidence → Evidence pack → RAIDT score profile → Governance readiness

In this sense, Traceability is one of the mechanisms by which RAIDT moves governance from broad assurance language to evidentially grounded review.

Link to the five RAIDT pillars

Responsibility

Responsibility concerns who is accountable for defining, checking, approving, and acting on a run. Traceability supports Responsibility by showing which person, team, or role made or authorised each relevant intervention.