Insights · Regulation
What should a customer-data AI release checklist include?
A fill-in checklist for approving AI that uses customer data: five sections, each with a question, the evidence to attach and a red flag.
- Published
- Reading time
- 7 min read
- By
- Nakato
In this article
Our article about how teams can safely send customer data to an LLM outlined the steps needed and the factors to consider when on the way to approving an AI tool that needs to use customer data. This piece gives you a concrete release checklist that covers five things: the job the model does, every route the data takes, what gets kept afterwards, what could still identify someone, and what reopens the approval.
Each section has a question that has to be answered, the evidence that should be attached and a red flag to watch for. An approval built on “the tool masks names” will miss most of what goes wrong.
This can be used for your purposes: tables to fill in, one copy per task. Read the reasoning behind each item, and how the law treats pseudonymised data in the UK, EU and US, in How can teams safely send customer data to an LLM? first.
At the end of the piece is a filled-in example. If an answer in the evidence column is vague, like “we think so” or “the vendor said”, treat it as blank. Specificity means safety.
1. The task
| Question | Evidence to attach | Red flag |
|---|---|---|
| What exactly will the model do, and who uses the result? | A single sentence with the task, output and its reader | General statements like “support use cases” or “increased productivity” |
| Which data fields does it read? | A list of fields, with a reason each one is needed | Whole records sent “for context” |
| What happens when a field is missing? | The fallback, written down | No answer, so the model guesses and might take unwanted routes |
| Which jobs are not allowed? | A short list of forbidden uses | Nothing listed, so anything unlisted is treated as fine |
2. The route
| Question | Evidence to attach | Red flag |
|---|---|---|
| Which apps, scripts and tools will send this data to a model? | A list, each with a named owner | A list made from memory, with browser extensions and chat assistants unchecked |
| Which of them pass through the privacy checkpoint? | A test request from each route, showing it was caught | “It's all behind the gateway”, with no test |
| What does the text look like when it leaves? | Five real edited samples from the live route | Only the vendor's demo examples |
| Who can turn the protection off? | Every setting or access key that relaxes it, and who holds each | Anyone can issue keys, or no one knows |
The last row matters more than it might seem. Some tools have a mode that cleans your own logs while the AI provider might still receive the original text. A review that only checks logs will pass this tool, but not look at what the provider is receiving.
3. What's retained by third-party providers
| Question | Evidence to attach | Red flag |
|---|---|---|
| How long does the AI provider store what you send and what it returns? | The clause in your contract | An answer from a sales call or the provider's website |
| Does the provider use either to train or improve its models? | The clause in your contract | “Not by default”, with no clause |
| Which other companies handle the data for the provider, and in which countries? | The provider's list of subcontractors | No list, or no locations |
| What do your own logs store: the original text or the edited version? | A section of a real log entry | Logs and provider checked separately, never side by side |
| Where is the table that links stand-ins back to real details, and who can open it? | Its location and the named people with access | A shared drive, or “the engineering team” |
4. What's still identifiable
| Question | Evidence to attach | Red flag |
|---|---|---|
| Who receives the edited text, and what else might they know? | A short description of the realistic recipient | “The provider can't see the lookup table, so it's anonymous” |
| Can someone with ordinary access match de-identified records back to real people? | Who tried, how many records, how many they matched, and what was changed as a result | Never tested, or only short, simple records tested |
| Who authorised that test? | Written sign-off from whoever owns the data | No sign-off: in the UK, re-identifying data without the owner's consent can be an offence (more on this) |
| Does the output still do the job? | Results from edited and original text compared on the two or three things that matter. Create your own mini benchmark | “It looked fine” |
| What is the data's legal status, for you and for the recipient? | One paragraph, reviewed by your data protection lead | “It's anonymised”: if you can reverse it, it isn't |
Test the re-identification ability with the longest, most unusual records. These carry the most detail for someone to piece together, and can be heavy with quasi-identifiers that link an individual in combination.
5. What reopens the review
An approval with no end date drifts. Either approval should be scheduled for consistent review, or triggers should be agreed. Ideally, name a party who is responsible for noticing when this happens.
| Trigger | Who watches for it |
|---|---|
| A new AI provider, model or model version | |
| A new app or tool sending this data | |
| Any change to privacy settings or access keys | |
| A new data field, data source or task | |
| A stand-in showing up in an answer a user sees |
Final sign-off
| Item | Answer |
|---|---|
| Approved task | |
| Model and version covered | |
| Owner of the remaining risk | |
| Approval expires on | |
| Date of next re-test |
An example checklist
Using a full example to demonstrate this process. A broadband provider wants an outside AI model to turn notes from customers’ cancellation calls into a monthly summary of why people leave, for the leadership team.
The task. The model produces themes across all cancellations, but doesn’t comment on individual customers. It reads the call agent’s free-text notes and the customer’s package details. It doesn’t get direct identifiers like names, addresses or account numbers. Two uses are forbidden in writing: deciding which individual customers to offer discounts to, and scoring call agents.
The route. Notes are exported from the customer service system and pass through the company’s privacy checkpoint before reaching the model. While listing routes, the team finds that one team leader has been pasting notes into a chat assistant in their browser. That route goes around the checkpoint, so it's shut off before approval.
What's kept. The contract says the provider deletes inputs after 30 days and doesn't train on them. The lookup table stays in the customer service system, and two named analysts can open it.
What's still identifiable. One analyst reads 30 edited notes and tries to match them to real customers. She matches four. One reads, roughly, “customer since 1994, the only house on the street with fibre, called 14 times in March about evening dropouts”. The name and address have been swapped, but the length of time as a customer, the street detail and the call count point to one household. The fix is to replace the year with a band (“20+ years”), drop the street detail, and generalise the call count as “repeated calls”. With these changes, she matches none on a second test. The team also notes that call notes record customers’ complaints in their own words. This is significant: the EU’s top court has said opinions count as personal data when the person behind them can be identified.
Does the tool still work? The team compares the model’s themes from the de-identified notes against themes from the originals. The top five reasons for leaving match and are in the same order. The de-identification tool works.
Sign-off. The head of customer service owns the remaining risk. The approval covers one model version and expires in six months, or sooner if any trigger in section 5 fires.
Further reading
- How can teams safely send customer data to an LLM?: the reasoning behind each section, and how the law treats pseudonymised data in the UK, EU and US.
- Swapping names isn't enough: the evidence a UK data protection committee expects.
- Masking, tokenisation and semantic pseudonymisation: what do they do?: how each method changes data, and how much usefulness it keeps.
- How can teams test for bypasses around an AI de-identification layer?: how to find the routes in section 2 that go around your checkpoint.
Sources
Case C-413/23 P, European Data Protection Supervisor v Single Resolution Board
Court of Justice of the European Union, 4 September 2025
Cited for: personal opinions count as personal data when their author can be identified.
Data Protection Act 2018, section 171
UK Parliament
The offence of re-identifying de-identified personal data without the controller's consent.
More insights
Regulation ·
How can teams safely send customer data to an LLM?
Approve one task and one data flow, keep the key that reverses the twins in your own environment, and test the detector on your own text.
8 min read
Regulation ·
Swapping names isn’t enough: getting pseudonymised data approved for an LLM
Pseudonymised data stays personal data while you hold the key. A committee approving it for an LLM needs written evidence on seven points.
6 min read
Research ·
How good are LLMs at PII detection?
Recent benchmarks show LLMs miss many rare or ambiguous names and over-flag others. They work best alongside string-matching and classifiers.
4 min read