Six Pillars of Trustworthy AI

Where financial AI fails – the risks and challenges on the path to production

Simon Gregory | CTO & Co-Founder
MPhys Physics, University of Warwick

Request your copy of the Executive Summary / Full Report (PDF)

Foreword

In more than a decade of building financial AI systems in production, I have consistently found myself making the same engineering choice: preserve the information, adapt the system. Not the other way around.

I call this the Information First principle. It is not a methodology I arrived at through theory. It is something I discovered through building. Through the accumulated weight of every decision about what to discard, what to flatten, what to sacrifice to fit the constraints of the system. Every time I chose not to sacrifice it, the system got better. Every time I did, something broke downstream. Eventually the pattern became a principle.

That principle deserves its own treatment, and will receive it. But it is the current that runs beneath everything in this document.

The work that produced it predates the current generation of language models. When I began applying that principle to AI systems for tier-1 clients in production, the gaps became immediate and structural. The architecture, the principles, the failure modes each emerged from building financial AI under real regulatory scrutiny, with real consequences for getting it wrong. When language models arrived, they validated the principles by failing in exactly the ways I had already designed against. These principles were tested in production at tier-1 institutions under conditions designed to challenge them. They held, and survived multiple adversarial evaluations. The Six Pillars are what I found during this process.

Three Axioms

The Six Pillars are six expressions of three foundational requirements: axioms that any trustworthy financial AI system must satisfy. Violate any one of them and the others cannot hold.

Evidence Integrity
A system must maximise and preserve the integrity of the evidence it operates on.

Provenance Integrity
A system must preserve the lineage of its evidence and verify it architecturally, such that every output surfaces its authoritative sources for human review. Without this, nothing downstream can be trusted.

Determinism
Non‑determinism is a property, not a feature. In finance, it must be contained by deterministic architecture.

What are The Six Pillars?

Each pillar describes a structural failure mode. A way that financial AI systems break in production that cannot be fixed by reaching for a more capable model. The model reasons within an evidential world constructed by the surrounding architecture. Together the pillars form a diagnostic framework: a map of where that evidential world breaks, and why trust collapses with it.

The pillars are ordered deliberately. Auditability establishes the requirement for inspectability. Authority and Provenance define what legitimate evidence looks like and how its lineage must be preserved. Context Integrity and Temporal Integrity describe how that evidence can be corrupted in retrieval. Determinism closes the loop: it is the architectural property that makes all the others hold under pressure.

The framework doesn’t name any products or describe any features. It can serve as an evaluation rubric for any financial AI system, including the ones I’ve built over the years.

Read individually, each pillar identifies a failure. Together they form a containment architecture, the only structural response to the non-determinism that is inherent in these systems. One that exists, and works in production.

Financial AI earns trust only when its reasoning is constrained, inspectable, and replayable.
Outside that boundary, it isn’t really a system, it’s uncontrolled behaviour.

The Six Pillars: Foreword / What are the Six Pillars?
Where financial AI fails – the risks and challenges on the path to production

Pillar 1: Auditability
When you can’t see how an answer was formed, you can’t trust it

Pillar 2: Authority
When AI can’t tell who is allowed to speak, relevance replaces legitimacy

Pillar 3: Provenance
When you can’t see the lineage, the system invents it

Pillar 4: Context Integrity
When the evidential world breaks, the model hallucinates the missing structure

Pillar 5: Temporal Integrity
When time collapses, financial reasoning collapses with it

Pillar 5: Temporal Integrity
When time collapses, financial reasoning collapses with it

The Six Pillars: Conclusion
GenAI is a different kind of system. The only viable response is deterministic architecture.

Next > | Pillar 1: Auditability

Request your copy of the Executive Summary / Full Report (PDF)

Includes Risk Register of over 25 risks

If you would like a copy of the Executive Summaries or Full Report request your copy using the form below:

This contact form is deactivated because you refused to accept Google reCaptcha service which is necessary to validate any messages sent by the form.

Six Pillars of Trustworthy Financial AI

Financial AI earns trust only when its reasoning is constrained, inspectable, and replayable. Outside that boundary, it isn’t really a system – it’s uncontrolled behaviour.

Simon Gregory  |  CTO & Co-Founder

Pillar 1: Auditability
When you can’t see how an answer was formed, you can’t trust it

Pillar 2: Authority
When AI can’t tell who is allowed to speak, relevance replaces legitimacy

Pillar 3: Provenance
When you can’t see the lineage, the system invents it

Pillar 4: Context Integrity
When the evidential world breaks, the model hallucinates the missing structure

Pillar 5: Temporal Integrity
When time collapses, financial reasoning collapses with it

Pillar 6: Determinism
When behaviour is unstable, trust must come from the architecture, not the model

Pillar 1: Auditability

When you can’t see how an answer was formed, you can’t trust it

Auditability is the discipline of being able to trace, verify, and justify how an AI assisted outcome was produced. In traditional software, this is straightforward: deterministic code paths, logs, and reproducible behaviour give you a clear chain of causality. Generative models break that assumption. Their internal processes are opaque, their outputs are non deterministic, and their explanations are narratives rather than evidence. That combination makes auditability one of the defining challenges of trustworthy financial AI.

LLMs and vector systems operate as opaque black boxes. Their internal states, intermediate steps, and decision paths are not observable or reconstructable. You cannot inspect how a specific answer was formed, and you cannot replay the internal reasoning that led to it. This means auditability cannot rely on introspection; it must rely on external verification.

Because the model sits outside the trust boundary, its output must be treated as untrusted input. This is the same posture used in security engineering: anything that originates outside the system of record is untrusted until validated. Fluency, confidence, and coherence do not grant trust. Only verifiability does.

When asked to “show its working,” an LLM generates a post hoc reconstruction, not a trace. The explanation is produced after the answer, using the same generative mechanism that produces the answer itself. It may be plausible, but it is not guaranteed to reflect the internal process. It can be incomplete, incorrect, or entirely fabricated. This is why explanations cannot be treated as audit evidence.

Hallucinations are not defects; they are a direct consequence of how generative models work. The same mechanism that enables generalisation, inference, and creativity also enables confident fabrication. Eliminating hallucinations would eliminate the model’s ability to operate beyond rote retrieval. This is why external validation is mandatory.

When multiple agents or models are chained together without verification, their uncertainties compound. A network of unvalidated agents does not distribute intelligence; it distributes error. Each step amplifies drift, weakens attribution, and erodes auditability. Without controls, the system becomes a multiplier of its own weaknesses.

Auditability, therefore, is not about trusting the model. It is about containing it: defining the trust boundary, validating every output, and ensuring that no decision relies on unverified generative content.

Next > | Pillar 2: Authority

News & Insights

© 2025 Limeglass – All Rights Reserved.
Privacy Policy