UMNAI
Collection 5

Privacy-Preserving Auditability

Auditability does not require every reviewer to see every raw field. A privacy-preserving design separates the evidence needed for integrity and lineage from the sensitive data needed only for authorised investigation.

By Angelo Dalli8 min read
Evidence layersCumulative reach by roleOPENDEEPUVC + verification statusModel id, hash matchExplanation metadataActive rules, attributionsQuery inputs + outputsRaw fields, predictionIdentity linkagePerson or entity mappingExternalverifierInternalauditorOperatorDatastewardEach role is authorised to a depth; identity linkage requires separation of duties
Layered privacy diagram: protected raw data at the centre, restricted explanation and rule metadata around it, UVC and verification status at the outer layer, with role-based access paths for operator, internal auditor and external verifier.

The tension between evidence and disclosure

A high-impact model may process health, financial, employment or behavioural data. Auditors need enough information to verify how a result was produced, but broad duplication of raw data increases exposure.

Privacy-preserving auditability begins by defining the audit question and disclosing only the evidence required for that question.

Separate evidence layers

A layered record can distinguish:

  • public or broadly shareable metadata: model identifier, UVC schema version and verification result;
  • restricted explanation metadata: active rules, attribution summaries and governed feature references;
  • sensitive query data: raw inputs and detailed outputs;
  • highly restricted identity linkage: information connecting the query to a person or protected entity.

Access can be granted by role and purpose. The retained links must still allow an authorised reviewer to reconstruct the full chain when necessary.

Hashes as integrity references

A Unique Verification Code (UVC) can represent selected data without embedding the original data in the code. A reviewer who possesses the authorised source record can recompute the hash and check whether it matches. The covered result may originate from an Explainable Neural Network (XNN), but the audit design should expose only the fields required for the verification purpose.

The hash itself is not a complete privacy solution. Low-entropy or predictable inputs can create inference risks, and surrounding metadata may reveal sensitive facts. Security design may require additional controls beyond the basic UVC mechanism.

References instead of copies

Where practical, an audit record can retain governed references to protected data rather than duplicating it. The reference should resolve to the exact version used during inference and remain subject to retention, access and deletion policies.

This approach can reduce proliferation, but it creates dependencies on the availability and integrity of the source system. The design should document how an audit proceeds if the source record is archived, corrected or legally deleted.

Encrypted computation

Hybrid Intelligence models are designed to be compatible with homomorphic encryption, in which certain computations can be carried out on encrypted values. In principle, encrypted-computation approaches can reduce the need to expose raw data while performing defined checks.

This is an advanced implementation area. Feasibility depends on the exact algorithm, performance requirements, cryptographic scheme and threat model. Public claims should describe it as a potential integration pattern until the supported implementation and scope have been technically verified.

Independent verification without broad access

An external verifier may receive:

  • the canonicalisation and UVC specification;
  • the stored UVC;
  • controlled access to the required source fields;
  • the model reference; and
  • a verification environment or independent implementation.

Separation of duties can prevent the model operator from being the only party capable of checking its records. Privacy controls should define what the verifier can see, retain and disclose.

Distributed and append-only records

Storing UVCs in an append-only log or distributed ledger can provide an external time-ordered record. Because the UVC is a fingerprint, the underlying sensitive data need not be placed on the ledger.

The design still needs to consider linkage, metadata leakage, correction procedures, governance and the permanence of the chosen store. Immutability is not automatically compatible with every privacy obligation.

Corrections should be represented without silently rewriting the historical record. A governed design can retain the original integrity reference, add a linked correction record and state which version is authoritative for current use. The exact approach must follow the organisation's retention and legal requirements.

Design questions

Before deployment, decide:

  • which fields are covered by the model and query UVCs;
  • who can recompute them;
  • where the source data remains;
  • how identity is separated from the explanation record;
  • what is retained and for how long;
  • how corrections are represented;
  • how access is logged; and
  • which claims a successful verification supports.

Key technical concepts explained

  • Data minimisation: limiting collection and disclosure to information required for a defined purpose.
  • Keyed hash: a hash-based integrity mechanism that also requires a secret key.
  • Separation of duties: allocating creation, operation and verification responsibilities across different actors.
  • Homomorphic encryption: cryptographic techniques that allow defined computation over encrypted data.
  • Append-only log: a record store designed to preserve prior entries rather than overwrite them.

Simplified scenario

A hospital stores raw patient data in its protected clinical system. The XNN audit record retains governed feature references, model and rule identifiers, a restricted attribution summary and a Query UVC. An external reviewer receives controlled access to the required fields and recomputes the UVC without receiving a general export of the patient record.

Why it matters

Transparency and privacy are not opposites when evidence is designed in layers. The goal is to make decisions verifiable by authorised parties while avoiding unnecessary replication or disclosure of sensitive information.

Continue the journey

Keep exploring Technical Insights.