S4.11 - Retrieved_document_IDs_and_hashes

S4.11 ? Retrieved document IDs and hashes

flowchart LR
    A[Background problem:
retrieval can change over time] --> B[RAIDT:
run-level evidence framework]
    A2[Traditional limitation:
query logged, retrieved artefact not fully verifiable] --> B
    B --> C[[Retrieved document IDs and hashes]]
    H[Practical fields:
document ID
chunk ID
page reference
corpus version
content hash] --> C
    C --> D[Evidence pack:
retrieval provenance]
    C --> E[RAIDT score profile:
stronger Auditability and Traceability]
    C --> F[Reviewer reconstruction]
    D --> G[Governance readiness]
    E --> G
    F --> G
    G --> I[Organisational learning and contestability]

? Star S4 - Evidence Architecture and Artefacts

Star context: Specifies the concrete fields and artefacts that make a run record inspectable, reconstructable, and reviewable within RAIDT's run-level evidence architecture.

Academic picture

Definition / background

Retrieved document IDs and hashes record the identity and integrity markers of the materials returned by a retrieval step during a GenAI run. In a retrieval-augmented system, the model does not answer from its base model alone; it also draws on documents, passages, chunks, tables, or other indexed artefacts supplied at run time. RAIDT therefore treats the retrieved evidence objects as part of the run record, not as an invisible background process.

A document ID typically names the retrieved artefact within the organisation's corpus, search layer, vector index, content management system, or evidence store. A hash is a compact integrity value derived from the relevant source object, such as the full document, the retrieved chunk, or a canonicalised passage representation. Together, these fields support later checking that the evidence retrieved during the run is the evidence being discussed during review.

This item matters because retrieval introduces a distinctive governance problem: the answer may depend on external materials that can change over time. A query alone does not fully solve this problem. Two runs can use the same query and index but still retrieve different chunks because of corpus updates, ranking changes, ingestion errors, or source edits. For RAIDT, retrieved document IDs and hashes therefore complement S4.10 ? Retrieval query and index ID by documenting what was actually returned, and complement S4.15 ? Output hash by showing what evidence underpinned the output.

Within RAIDT, this belongs inside run-level evidence because a run is the unit of governance. If reviewers cannot reconstruct which evidence artefacts informed the run, then the evidence pack is incomplete and the five-pillar profile risks resting on assertion rather than inspectable records.

Why this concept matters

Retrieved document IDs and hashes solve a practical audit problem: they allow a reviewer to move from a claim that a system used evidence to a demonstrable record of which evidence objects were retrieved. This reduces ambiguity when outputs are contested, when a document repository is updated after the fact, or when an organisation needs to compare two runs that produced materially different answers.

Without this item, retrieval provenance is often reduced to vague statements such as "the model searched the policy library" or "the answer was grounded in internal documents". That is insufficient for responsible governance. It becomes difficult to test whether the model cited the wrong version of a policy, whether an outdated chunk was surfaced, or whether a later reviewer is looking at a source that differs from the one used at the original moment of decision support.

For organisations using GenAI in operational settings, this item helps convert abstract expectations about transparency into a reviewable artefact. It supports challenge, replay, escalation, and post hoc investigation. In RAIDT terms, it moves governance from principles to operational evidence by showing what evidence objects were actually in play at run time.

Key idea: Retrieved document IDs and hashes matter because they make retrieval provenance inspectable at the level of the actual evidence objects used in a specific run.

What this item captures

The specific document, record, file, or corpus object returned by the retrieval step.
The identifier structure used by the retrieval layer, such as document ID, chunk ID, page ID, or repository key.
The integrity value associated with the retrieved artefact, such as a documented content hash.
The basis for checking whether the retrieved source has changed since the run occurred.
The link between retrieval events and the evidence pack used for review, scoring, and contestability.
The minimum provenance needed to compare what was asked for, what was retrieved, and what was ultimately produced.

Practical example / likely audience question

Audience question

Why store document hashes?

Answer

The concern behind this question is usually that document identifiers alone may seem sufficient. In practice, they are not always enough. A document ID may remain stable even when the underlying content changes, when a page is revised, or when a retrieval pipeline re-chunks source material during re-indexing. If RAIDT stored only the identifier, a reviewer might locate the same nominal document later but still fail to verify the exact content state that informed the run.

The direct answer is that hashes help show that the evidence source actually used in the run is the evidence source being reviewed afterwards. For example, suppose a policy assistant retrieved HR-POL-017 from an internal repository during a run in March. By June, the same policy has been updated after a compliance review. The document ID still points to HR-POL-017, but the hash reveals whether the reviewer is looking at the March content or the June revision. That distinction matters if the run supported a decision that is later challenged.

RAIDT handles this better than a generic AI governance approach because it records the issue at run level. Rather than saying only that the system was connected to approved sources, RAIDT asks what was retrieved in this run, how it can be verified, and how that retrieval evidence enters the evidence pack and score profile.

Practical example in RAIDT terms

Consider a healthcare use case in which a hospital uses a retrieval-augmented GenAI assistant to draft discharge guidance for clinicians from internal protocols and medicines information sheets. A particular run retrieves three passages: a local anticoagulation protocol, a renal dosing table, and a discharge checklist. The run-level issue is that the generated answer may later be questioned if a patient incident occurs or if the protocol was updated after the response was generated.

The evidence needed is not only the clinician's prompt and the retrieval query, but also the retrieved document IDs, chunk references, and hashes for the exact materials returned during the run. These records allow a reviewer to test whether the assistant relied on the correct protocol version and whether the retrieved content corresponds to the answer that was given.

In RAIDT terms, Auditability and Traceability are the most directly affected pillars, with Dependability also strengthened because the organisation can evaluate whether the system behaved consistently against the intended evidence base. Responsibility is implicated because clinical governance depends on being able to investigate evidence use. This item improves governance readiness by making the retrieval layer reviewable rather than opaque.

Detailed link to RAIDT

Retrieved document IDs and hashes link to RAIDT in four ways.

First, they support RAIDT's core idea that governance should attach to the run rather than to broad system-level assurances. The item makes a specific retrieval event evidentially visible.

Second, they strengthen run-level evidence by documenting what evidence artefacts were actually returned, not merely what repository or query configuration existed in principle.

Third, they improve the evidence pack and the score profile because reviewers can assess whether retrieval provenance is sufficiently robust for audit, challenge, and replay.

Fourth, they support reviewability, contestability, audit readiness, and organisational learning by allowing later investigators to compare the original retrieval state with the current corpus state and to diagnose source drift or pipeline change.

Retrieved document IDs and hashes -> Run-level evidence -> Evidence pack -> RAIDT score profile -> Governance readiness

This chain matters because retrieval provenance is often where apparently well-governed systems become difficult to reconstruct in practice. RAIDT makes that weakness visible and governable.

Link to the five RAIDT pillars

Responsibility

This item supports Responsibility when an organisation must show that evidence-backed outputs were grounded in identifiable and reviewable source materials. It is especially important when human operators rely on a system in consequential organisational settings.