Insights · Engineering
The AI data paradox: the safer you make the data, the worse the AI gets
Enterprise data is valuable because it’s specific, yet that specificity is the first thing protection tools strip out. Privacy needs designing in.
- Published
- Updated
- Reading time
- 3 min read
- By
- Tom Fairey, Founder and CEO

AI engineers spend enormous amounts of time tweaking small things to achieve bigger gains. Energy is poured into analysing model performance, rewriting prompts, tuning retrieval and building elaborate evals. But in enterprise settings, model performance is often most affected by what happens before that work even begins, by what goes into the pipeline first.
Data isn’t something that a system solely consumes; it is part of the architecture itself. It is as structural as the model or prompt – the blueprint for the structure. In most enterprises, where data is so sensitive, this blueprint arrives with sections missing.
Enterprises have the most delicate pain point. The data is richest here because it is proprietary, intrinsically tied to each organisation. And this richness of context requires the most cleaning. Compliance and security teams, working to either internal or external policy, run data through protection tools that detect sensitive information and then redact, mask, or tokenise it. Their job is to make sure nothing sensitive is exposed or leaked, not to check whether the data is still useful.
A paradox is created: an enterprise’s dataset is valuable because of its specificity and its representation of a company’s customers, successes, failures. Yet the data carrying these signals is the first to go, leaving a mottled blueprint.
If data is a blueprint, then redaction is a black marker taken to the plans. The room layout survives, but the dimensions, load-bearing walls and materials are incomplete or damaged. Even with the best builder, or developer, the building will still stand on shaky foundations. You can’t expect it to be the most beautiful, sturdiest building on the block if it was built from partial plans.
If data is a blueprint, then redaction is a black marker taken to the plans.
The richer the data, the more aggressively it is stripped. This produces technically safer but sterile data that is rarely useful for generating insight. This cost is measurable: a recent study from the University of Bonn and Fraunhofer IAIS found that a model scored 0.73 on a reasoning benchmark using the original text, but this dropped to 0.52 once personal details were redacted. Replacing them with consistent pseudonyms brought it back to 0.70.
Starting a pipeline with incomplete records puts a burden on developers and engineers to improve other aspects and fix a problem that started before them.
Every option to preserve safety gives something up. Self-hosted models keep data in-house but demand infrastructure and talent many teams lack or cannot budget for. Aggregated data smooths away the edge cases that are often the most insightful, and doesn’t work when single records need to be reasoned with.
An adaptive approach would not look at “what is sensitive?” but “what needs to be kept for the model to perform best?” Identity and context might seem inseparable, and in many ways they are. But providing relevant or equivalent context while protecting individuals behind data is one way models can be given a more complete blueprint.
Models, prompts, and retrieval are already seen as things to design. Privacy measures should be seen like this too. A requirement or restriction that has to be adhered to can be an opportunity for innovation and creativity.
The combination of a safety requirement and the need to stay attuned to an organisation’s specific data produces fertile ground for new approaches – and this shouldn’t fall only on engineers. Look at data as part of the system architecture, and its treatment becomes a crucial first step in achieving high performance.
Sources
On the Impact of Anonymisation on Large Language Models
Tobias Deußer et al., University of Bonn and Fraunhofer IAIS (arXiv), September 2026
Reasoning benchmark scores for original, redacted and pseudonymised text.
Written by
Tom Fairey
Founder and CEO
Tom Fairey is the Founder and CEO of Nakato. He brings years of experience working with enterprises on moving AI projects out of pilot purgatory and into production. He brought the company together and leads Nakato.
More insights

Engineering ·
How can teams test for bypasses around an AI de-identification layer?
A de-identification layer protects only the traffic routed through it. Test every route a prompt can take, not just the detector.
5 min read

Research ·
What is a semantic twin?
A semantic twin is a fictional stand-in for personal data that keeps what an AI task needs, so the model can reason with it, then swaps back in the answer.
10 min read

Regulation ·
Anonymisation vs pseudonymisation: what’s the difference?
Pseudonymised data is still personal data; properly anonymised isn't. Here’s how the law tells them apart in the UK, EU and US.
7 min read