S5.09 - Composite_vs_profile

S5.09 ? Composite vs profile

flowchart LR
    A[Pressure for one headline score] --> B[RAIDT
Run-level evidence framework]
    A2[Risk: weak pillars hidden by averaging] --> B
    H[Healthcare, finance, public services, enterprise use] --> C
    B --> C[[Composite vs profile
Profile first, composite second]]
    C --> D[Run-level evidence pack]
    C --> E[Five-pillar score profile]
    C --> F[Optional composite summary]
    D --> G[Reviewer reconstruction]
    E --> I[Weak pillar visibility]
    E --> J[Targeted governance improvement]
    F --> K[High-level reporting]
    G --> L[Audit readiness]
    I --> L
    J --> M[Organisational learning]
    B --> N[Evidence over assertion
Reviewability and contestability]
    C --> N

? Star S5 - RAIDT Pillars and Scoring

Star context: Situates the five RAIDT pillars within a scoring logic that keeps governance measurable without collapsing important differences between pillars into a misleadingly simple headline number.

Academic picture

Definition / background

In RAIDT, composite vs profile refers to the distinction between two ways of presenting governance scoring results for a run of a generative AI system. A composite score is a single summary value, typically derived by averaging the five RAIDT pillars. A profile is the full five-pillar pattern showing separate results for Responsibility, Auditability, Interpretability, Dependability, and Traceability.

The distinction matters because the two outputs do different jobs. The composite supports concise communication and coarse comparison. The profile supports judgement, diagnosis, and governance action. RAIDT therefore allows the composite as a shorthand, but treats the profile as the primary result. This is because organisational risk is often driven not by the average condition of a run, but by a weakness in one specific governance dimension.

Conceptually, this item sits at the intersection of scoring theory and governance design. Many assessment systems produce an aggregate measure for reporting convenience, but governance frameworks also need to preserve internal structure so that trade-offs remain visible. In generative AI governance, this is particularly important because a run may be dependable in output stability while still being weak in traceability, or auditable in logging while still lacking interpretability for affected reviewers.

Inside RAIDT, the concept belongs directly to run-level evidence because the score profile is meant to remain anchored to concrete evidence within the run-level evidence pack. The profile is not just a visual summary; it is an organised representation of how the evidence supports or limits governance confidence across the five pillars. In that sense, composite vs profile is not only a reporting choice but a design principle for evidence-based governance.

Why this concept matters

This concept matters because it prevents governance simplification from becoming governance distortion. If a single composite score is treated as the main result, organisations may overlook serious weaknesses that are hidden by stronger scores elsewhere. A run that appears acceptable on average may still be difficult to contest, poorly documented, or operationally fragile.

It also reduces confusion between measurement convenience and governance adequacy. In practice, managers often ask for one number because it feels easier to track. RAIDT accepts that need but avoids letting it dominate the interpretation of readiness. By retaining the profile as the main output, RAIDT keeps attention on the actual shape of governance performance rather than only its arithmetic average.

For organisations using generative AI, that distinction supports better escalation, remediation, and prioritisation. It helps reviewers identify whether a governance problem is about ownership, documentation, explanation, reliability, or reconstruction. This is one of the ways RAIDT moves AI governance away from broad principles and towards operational reviewability.

Key idea: The profile matters more than the composite because governance readiness depends on the pattern of evidence across pillars, not just on the average score.

What this item measures

It measures whether governance readiness is being interpreted as a structured five-pillar pattern rather than a single flattened number.
It measures whether scoring outputs preserve pillar-specific weaknesses that may require targeted intervention.
It measures the difference between summary reporting value and decision-making value.
It measures how far a run-level assessment remains connected to evidence rather than becoming an abstract metric.
It measures whether reviewers can diagnose trade-offs across Responsibility, Auditability, Interpretability, Dependability, and Traceability.

Practical example / likely audience question

Audience question

Why not simplify RAIDT to one composite score if decision-makers usually want a single number?

Answer

The concern behind this question is understandable: organisations often need concise reporting, benchmarking, and dashboard indicators. A composite score can help with that, and RAIDT does not reject it. The problem arises when the composite is treated as the main governance result rather than as a secondary summary.

The direct answer is that a single score can hide meaningful governance weaknesses. Imagine a run that scores strongly on Responsibility, Auditability, Interpretability, and Dependability, but poorly on Traceability because the chain of prompts, model settings, and review steps cannot be reconstructed. Its average may still look respectable. Yet from a governance perspective, that run remains problematic because an important part of the evidence trail is weak.

RAIDT handles this better than a generic AI governance approach because it ties the score to run-level evidence. Instead of saying only that a system is ?moderately governed?, RAIDT shows where the governance condition is strong, where it is weak, and what evidence underpins that judgement. That makes remediation more precise, contestation more credible, and audit preparation more realistic.

Practical example in RAIDT terms

Consider a hospital using a generative AI system to draft discharge summaries for clinicians. One specific run involves a configured prompt template, a named model version, patient-context constraints, staff review procedures, and a timestamped output. The run-level issue is that the generated summary is clinically plausible and the reviewer signs it off, but the underlying prompt revision and approval route were not captured consistently.

The evidence needed includes the exact prompt template used, model and parameter settings, reviewer identity and sign-off, workflow logs, exception notes, and the version history of the template. In RAIDT terms, Dependability may be relatively strong because outputs are stable and reviewed, while Traceability and Auditability may be weaker because the reconstruction trail is incomplete.

If the organisation relied only on a composite score, the run might appear broadly acceptable. The five-pillar profile, however, would show that governance readiness is uneven. That makes the next action clear: improve logging and reconstruction controls rather than assuming the run is fully ready because the average score is adequate. The item therefore strengthens readiness by exposing where evidence-pack improvements are actually required.

Detailed link to RAIDT

Composite vs profile links to RAIDT in four ways.

First, it supports RAIDT?s core idea that generative AI governance should be based on structured evidence rather than broad claims of compliance or responsibility.
Second, it connects directly to the run because the shape of the profile is generated from evidence about one configured use in one real context.
Third, it links to the evidence pack and score profile because the profile is the interpretable surface through which evidence is reviewed, while the composite is only a condensed summary.
Fourth, it strengthens reviewability, contestability, audit readiness, and organisational learning by making weak pillars visible instead of averaging them away.

Composite vs profile ? Run-level evidence ? Evidence pack ? RAIDT score profile ? Governance readiness

This chain matters because RAIDT is not trying merely to score AI systems; it is trying to make governance judgements inspectable, reconstructable, and actionable at the level where real organisational use occurs.

Link to the five RAIDT pillars

Responsibility

A profile makes it easier to see whether responsibility arrangements are genuinely defined or merely assumed. A composite score can obscure a weak assignment of ownership if other pillars score well.