Masking, tokenisation and semantic pseudonymisation for LLMs

When companies want to use AI tools hosted by third parties with data that contains personal information, three techniques exist to remove identifying details before a sensitive prompt hits a hosted model. Masking deletes each detected value and leaves a generic label. Tokenisation swaps each value for an indexed label such as PERSON_1 and keeps a key so the original can be put back. Semantic pseudonymisation swaps in a realistic stand-in of the same kind, so the model can still reason about it.

This article outlines the differences between each process. The three differ in whether the original can be restored, how well the AI does its job with altered data, and how much context still reaches the model provider. All three depend first on detection: finding the personal data and working out what type it is.

What survives the substitution

Masking leaves a symbol where the value was. Vats and colleagues give a standard form: “Tom lives in Chicago” becomes “[MASK] lives in [MASK]”. They report that such markers break the meaning and flow of the sentence.

Tokenisation gives each distinct value its own label, with a type and number, and stores the pairing somewhere the sender controls. The model learns that a person and a place were mentioned, but nothing more about either. Legally, this already counts as pseudonymisation: the data can't be tied to a person without extra information.

Semantic pseudonymisation substitutes a different value of the same kind in place of the sensitive original. Yermilov and colleagues at Grammarly turn “Sarah works at The Times in London with Rachel and David” into “Sophie works at Manchester Evening News in Manchester with Emma and Tom”. The grammar survives, and so does the link between a newspaper and its city. The simplest tools only match type – a name for a name. More sensitive ones, such as Nakato's Reversible Semantic Pseudonymisation (RSP), also keep relationships between values, like the gap between two dates.

A worked example

An insurer uses a hosted AI tool to triage claims. Its policy covers physiotherapy only if treatment starts within 12 months of the injury. This claim comes in:

The gap is 628 days, about 20 months, so the claim should be flagged. The job matters too: a manual worker with a knee injury is a typical employers' liability case.

Masking gives “[NAME], born [DOB], is a [JOB TITLE]… injured… on [DATE] and began physiotherapy on [DATE].” Both dates carry the same label, so the model can't tell which came first, let alone the gap. The best it can do is ask for the missing details.

Tokenisation gives “[PERSON_1], born [DATE_1]… on [DATE_2]… on [DATE_3].” The model can now tell the dates apart, but a label has no value to subtract. The claim goes through unchecked or back to a human.

Type substitution, the simplest semantic approach, swaps in random values of the right type: “Mark Ellis, born 02/11/1989, is an accountant… injured… on 14/08/1976 and began physiotherapy on 26/01/2015.” The model has values to reason with, but they're wrong. The gap is now 38 years, the injury predates the birth, and an accountant's knee reads as a different claim. Format-preserving tokens, common in payments, behave the same way.

Context-aware substitution, such as RSP, shifts every date by the same number of days and swaps the job for a comparable one: “David Carter, born 14/04/1967, is a scaffolder… injured… on 14/03/2022 and began physiotherapy on 02/12/2023.” The gap is still 628 days and the job is still manual. The retained context means that this claim is flagged correctly. The real details are restored on return to the user, within the company’s own domain.

Preserving intervals has a privacy cost: exact gaps between events can help re-identify someone when combined with other data. But with other indirectly identifying details changed, this risk reduces.

How much usefulness each method keeps

The more meaning or context a substitute keeps, the better the AI performs, but the ranking shifts by task. Utility here simply means: does the task still get the right answer after the data has been changed?

Masking costs more the more you blank. Vats and colleagues trained a small language model on conversation transcripts with names and other details masked. They scored it on perplexity, which measures how surprised the model is by real text: lower is better, like a golf score. Trained on the original text, the model scored 37.3. With 12.5% of words masked (one word in eight), it scored 120.1, roughly three times as confused. Filling the blanks with guessed words of the right type brought that down to 65.3. When only 1.7% of words were masked and then filled, it scored 38.5, almost as good as the original. The takeaway: the more you blank out, the more the model loses.

Realistic substitutes kept most of the score. Hou and colleagues tested a set-up closer to real use: a small local model found and replaced private details, and a larger model did the task. Across 1,000 test items per task, the substituted text kept over 95% of the original score on question answering and summarisation. Simply put, if the model got 100 answers right on the real text, it got about 95 right on the disguised text. Translation fared worst, at about 85.

Nakato's own benchmark compared five replacement methods, all fed by the same detector, so differences come from the replacement alone. Each figure below is retention: the protected score divided by the unprotected score. 100% means no loss at all.

The question tasks score the share of answers right. The summaries are scored by ROUGE-L, which checks how much of a human-written summary the model's version reproduces, in order. Each test used 280 to 300 cases, so gaps under two points are noise, not real differences.

On contract questions, where the answer hinges on who is party to what, tokens and fake names cost the most. Plain redaction did better than either. On summaries, tokenisation and fake names caught up with RSP. Nakato was the only method close to the unprotected baseline on all three. This was a vendor-run test with one model family (Gemini 2.5), judged by the same model, with no confidence intervals and no measure of cost or speed.

Detection sets the ceiling, context sets the floor

Every method only protects what the detector finds. Whatever it misses reaches the model unchanged.

Hou and colleagues measured how many true private details each detector caught. The best caught 65.7%: about two in three, so roughly one in three names, places or organisations went to the model untouched. Nakato's detector was the best of five it tested on four benchmarks, catching 99% of identifying characters against Presidio’s 71% in one test (the AI4Privacy benchmark). A single prompt designed to make the model repeat identifiers then pulled out 1.5% of them, three in every 200, a degree below the next best pipeline and Presidio (7.8%, GLiNER; 22% respectively).

But even perfect detection doesn't stop a model guessing from what's left. Patsakis and Lykousas gave GPT-3.5 descriptions of 20 UK public figures with names removed. It named the person correctly about 73 times in 100; human readers managed about 26. After stripping every word the model itself flagged as a giveaway, it still identified 59 in 100 of the texts it had identified before. The subjects were celebrities, which helps the guesser, but anyone holding background data about your customers can do the same. Staab and colleagues similarly found models can infer traits such as location, income and sex from context, and that masking does little to stop it. These are called quasi-identifiers, which can identify an individual when in combination, but don’t indicate anything serious when mentioned alone.

Running it in practice

The lookup table requires careful handling. Tokenisation and reversible pseudonymisation both need a stored mapping from original to substitute. Microsoft's open-source Presidio project shows a simple version: a counter per entity type, a lookup table held by the client, and a reverse step that restores the originals. Its own notes warn the logic isn't safe to run in parallel, because the mapping must be shared across workers. Any home-built scheme inherits the same problem.

Consistency matters across a conversation. A substitute has to stay the same every time a value reappears. Tools that treat each message in isolation can also miss risk that builds up as different details are revealed over several turns.

Language matters. Every language has grammatical idiosyncrasies that substitutes have to align with. In languages such as Arabic, a substitute name must agree with case, number and gender in the sentence, which makes good surrogates harder to produce.

The control point decides coverage. RSP detects details inside the customer's environment using small local models, substitutes them, sends the changed text out and restores the originals in the reply. The mapping never leaves. It runs as a sidecar next to an application or as a shared gateway. If a claims team calls the AI through that gateway, those requests are protected. A desktop tool calling the vendor directly, or a request in a format the tool doesn't parse, goes around it – the real question is which route every relevant call actually takes.

Which method should I use?

Pick by task, then test on your own data, because the ranking changes between tasks.

• Masking suits pipelines where the value plays no part in the result and nothing has to be restored, such as coarse classification. Test it on anything that needs to know who is who.
• Tokenisation suits extraction, reformatting and summaries, where you need an exact round trip and the model copies structure more than it reasons about entities.
• Semantic pseudonymisation such as RSP is best when an entity's properties drive the answer, like dates in a claim or parties in a contract, or when more complex reasoning is required from AI tools.

When judging a vendor and their chosen de-identification method, run two evaluations. First, measure detection on your own records against your own definitions of personal data. Second, compare task output against an unprotected baseline on the same prompts, using the metric your application is judged on. Start with your longest records, which give a recipient the most surrounding context to work with.

No technique fully stops a model inferring identity from context, so pair it with data minimisation. Legally, the GDPR recognises pseudonymisation as reducing re-identification risk, but not removing it totally. The CJEU's September 2025 ruling in EDPS v SRB suggests data a provider cannot re-identify may not be personal data for that provider, but it remains personal data for you. Keep any mapping within your domain and take legal advice.