Pillar 1: Auditability

When you can’t see how an answer was formed, you can’t trust it

Auditability is the discipline of being able to trace, verify, and justify how an AI assisted outcome was produced. In traditional software, this is straightforward: deterministic code paths, logs, and reproducible behaviour give you a clear chain of causality. Generative models break that assumption. Their internal processes are opaque, their outputs are non deterministic, and their explanations are narratives rather than evidence. That combination makes auditability one of the defining challenges of trustworthy financial AI.

LLMs and vector systems operate as opaque black boxes. Their internal states, intermediate steps, and decision paths are not observable or reconstructable. You cannot inspect how a specific answer was formed, and you cannot replay the internal reasoning that led to it. This means auditability cannot rely on introspection; it must rely on external verification.

Because the model sits outside the trust boundary, its output must be treated as untrusted input. This is the same posture used in security engineering: anything that originates outside the system of record is untrusted until validated. Fluency, confidence, and coherence do not grant trust. Only verifiability does.

When asked to “show its working,” an LLM generates a post hoc reconstruction, not a trace. The explanation is produced after the answer, using the same generative mechanism that produces the answer itself. It may be plausible, but it is not guaranteed to reflect the internal process. It can be incomplete, incorrect, or entirely fabricated. This is why explanations cannot be treated as audit evidence.

Hallucinations are not defects; they are a direct consequence of how generative models work. The same mechanism that enables generalisation, inference, and creativity also enables confident fabrication. Eliminating hallucinations would eliminate the model’s ability to operate beyond rote retrieval. This is why external validation is mandatory.

When multiple agents or models are chained together without verification, their uncertainties compound. A network of unvalidated agents does not distribute intelligence; it distributes error. Each step amplifies drift, weakens attribution, and erodes auditability. Without controls, the system becomes a multiplier of its own weaknesses.

Auditability, therefore, is not about trusting the model. It is about containing it: defining the trust boundary, validating every output, and ensuring that no decision relies on unverified generative content.