How can teams test for bypasses around an AI de-identification layer?

A de-identification layer protects the traffic that passes through it, but nothing else. Most evaluations test the detector; the harder question is whether text has other ways to reach a model or a log. Direct provider calls, disabled keys, logging-only modes and embedding pipelines never appear in a detector benchmark. Before any of that, though, the layer has to sit somewhere all the relevant traffic actually passes through.

Coverage belongs to the deployment

Sidecar or gateway, a layer sees only the requests routed to it. Presidio's LiteLLM sample makes the dependency plain: masking runs as a callback inside the proxy, and traffic flows from the app, through the proxy, to the provider. A service holding its own provider key never touches that proxy.

One reasonable response is that this is a network problem, not a de-identification one. Block egress to providers except through the gateway and route testing shrinks to checking a firewall rule. That works where egress is tightly controlled, less well where teams use several providers, embedding APIs or SaaS tools with built-in models, each another endpoint to allowlist or forget. The enforcement around the control point is usually where the gaps are.

How much does a wrong label matter?

In an example Presidio uses, “John Smith AHV number is 756.3026.0705.92. Zip code: 1334023” is logged with the name masked as PERSON, the AHV number as AHV_NUMBER and the zip code as US_DRIVER_LICENSE. Detection fired; the type was wrong.

For plain redaction this may not matter much: the value is hidden either way. Substitution designs are different. A wrong type yields a substitute of the wrong shape, a licence number where a postcode belonged, and the answer degrades in ways a pass/fail redaction check won't catch. The stronger a tool's utility claim, the more its label accuracy needs separate evidence.

NIST's 2023 guidance for federal agencies notes that not every tool that masks personal information can actually de-identify it. It observes that accuracy tends to fall as techniques get more aggressive, and that data accurate for one purpose may not be for another. Its remedy, a standard with measurable performance levels plus re-identification studies, is demanding for a small team, but a vendor's own benchmark says little about your specific transcripts. It's important to test with your own sample.

What a route test can show

Picture a US insurer's claims assistant, with claim text on five routes: the agent web app through the gateway, a nightly batch summariser on a service account, an indexer that sends claim notes to a hosted embedding API, the observability pipeline, and a replay tool that re-sends production prompts.

Seed one record with a name, a postcode and a fake claim reference no recogniser matches, send it down each route, and check three places: the proxy debug log, where masking lines begin with the mechanism's name; the provider's request record, where one exists; and the log sink.

Suppose the web app's payload arrives transformed, but the summariser still uses a key left over from the pilot. LiteLLM lets an administrator switch masking off for an individual key by setting “pii” to “false”, and this one was issued that way, so the name goes out intact. The indexer has its own credential and never appears in the proxy logs. The log sink shows masked text while the provider got raw content, because the proxy was left in a logging-only mode that masks what goes to observability tools but not the request itself. The claim reference survives everywhere because no recogniser covers it. On the way back, the replay tool shares the agent's key, so restored outputs land in an evaluation store outside the environment holding the mapping.

Nothing malfunctioned. Every leak was a configuration choice, which is arguably worse: the fix lies in ownership (who issued that key, who decided an embedding call wasn't a model call) rather than in a better detector.

What a clean result doesn't prove

A clean route test describes one configuration on one day. NIST frames privacy controls as something to review against the risks, the intended uses and the ways controls could fail, which implies re-testing as the system changes. Request formats are easy to miss: a layer that parses chat messages may pass a tool-call argument straight through.

There is also a limit no route test reaches. NIST quotes HHS guidance that even properly applied HIPAA de-identification leaves a risk of identification that is very small but not zero, and notes that a data release is hard, if not impossible, to recall, as are prompts already sent to a provider. Keeping the substitution mapping in your own environment protects the key, not the narrative around it. A claim note about an unusual accident in a small town may identify someone with every name swapped, the combination risk the Evidence Act asks agencies to account for. Details like these are quasi-identifiers, and Nakato's detection is geared towards identifying such contextual identifiers too.

Advocates of pseudonymisation would answer that a very small residual risk is a fair price for what teams gain. Often that is right, but someone should make the call knowingly, treating the route test as evidence rather than a guarantee.

What to do

Taking all these possible lapses of awareness into account, readers can follow a short sequence of actions, listed roughly in order of how much each one closes off:

1. Make the layer the only way out: block direct egress to model and embedding endpoints except through it.
2. List every credential that can reach a provider, give each an owner, and retire pilot keys with masking turned off.
3. Run a seeded route test down every route, including logging, embeddings and replay, and check restored output as well as outbound text.
4. Measure label accuracy on your own transcripts, not just whether something was replaced.
5. Re-run the test whenever a provider, key, log sink, retrieval source or request format changes.
6. Write down the residual risk you are accepting, and who signed off on it.