Pseudonymisation of training data does not count as anonymisation under the EDPB

According to the EDPB, training data is only anonymous when records can no longer be singled out, linked or used for meaningful inference, for which removing names or pseudonymisation does not suffice. Anyone training generative AI must also test the trained model for memorisation and extraction before the data counts as anonymous.

In its explanation of anonymisation and pseudonymisation, the European Data Protection Board (EDPB) explains a sharp distinction: with pseudonymisation, linkage to a person remains possible, whereas with anonymisation that is reasonably no longer the case. In an accompanying statement on anonymisation and web scraping for generative AI, the EDPB applies that framework in the generative-AI context, including to training-data assessment. A recent study in the ACM Digital Library adds an important insight in November 2026: anonymity claims about synthetic or transformed training data must be assessed from the model, not only from the underlying dataset.

What do the three EDPB criteria for anonymisation mean concretely for AI training data?

The EDPB’s framework uses three cumulative criteria. Data is only anonymous when records cannot be singled out (singling out), cannot be linked to external sources (linkability) and cannot lead to meaningful inferences about a person (inference). Because the criteria are cumulative, failure to meet any one of them means the data should not be treated as anonymous without further analysis. Importantly, this test is context-dependent: who can access the data, with what means and which additional datasets, partly determines the outcome. That framework is separate from any other obligations that may apply under the EU AI Act, including data-governance requirements for high-risk AI systems.

• No singling out: no individual record referring to a single person can be isolated from the data.
• No linkability: records cannot be connected to other data that does lead to identification.
• No meaningful inference: nothing about a specific person can be derived from the data with reasonable certainty.

Why is testing only the dataset not enough for generative AI?

With generative AI, the risk does not end at the dataset. Information can end up in the model weights during training and be reproduced later. The French CNIL states in its analysis of the GDPR status of an AI model that a model can only be considered anonymous when identification from the model parameters is highly unlikely and personal data cannot be extracted via model queries.

The OECD describes in its report on user control and privacy protections in AI that data can be memorised in model weights and that removing it from a trained model can require substantial technical effort and may not be straightforward. According to the ACM study and the CNIL analysis, a model-centric assessment should include privacy attacks such as memorisation and extraction, including query-based extraction; the article’s recommended protocol can additionally test for regurgitation.

In our assessment, this is where the most important shift lies: anyone who only cleans up the dataset and skips the model test bases an anonymity claim on an incomplete picture and leaves it vulnerable to an extraction investigation.

What does this mean for directors, lawyers and CISOs who want to demonstrate anonymity?

Our analysis: because the EDPB tests anonymity cumulatively on three criteria, an organisation that only removes direct identifiers loses its anonymity claim as soon as a supervisory authority or researcher demonstrates linkability or inference; In our analysis, a defensible claim should be supported by an explicit threat model recorded before use of the training set, including the attacker, access rights and external datasets. In our analysis, because the CNIL and the OECD show that personal data can remain in model weights, the evidentiary assessment should extend from the data specialist to the whole AI chain.; In our analysis, a lawyer could consider including in processor and procurement contracts a clause requiring both white-box and query-based extraction tests on the delivered model. Because the anonymity test is context-dependent, an anonymity claim changes along with the access context; In our analysis, the CISO should record who may query the model and under which rights, and should reassess that assumption if access is widened. Our analysis: because new models, sources and attack techniques can overtake the assumptions, a one-off determination of anonymity ages; In our analysis, the board should consider assigning a periodic reassessment, with a recorded moment at which the claim is tested again. Keep in mind the distinction with pseudonymisation under the GDPR and the duties that still apply: separately stored keys, hashes or restricted access rights reduce the risk, but do not automatically make data anonymous.

Which testing steps belong to a defensible anonymity claim?

A defensible claim rests on a combined dataset and model test, with recorded assumptions. None of the sources mentioned offers a guarantee that a model never reproduces information after transformation; they substantiate an evidentiary obligation. Our recommended verification protocol is to record the following steps, and set them side by side as with a DPIA that assesses the whole AI chain rather than the dataset alone.

1. Describe the access context: which attacker, which rights, which means and which external datasets have been taken into account.
2. Test the dataset for singling out, linkability to external sources and meaningful inference.
3. Test the trained model separately for white-box identification from the parameters and for memorisation, regurgitation and query-based extraction.
4. Document the transformation, model version, the attack tests carried out and the remaining uncertainty.
5. Reassess the claim periodically in light of new models, new sources or new attack techniques.

Because the place where the model runs does not automatically remove the extraction risks, it is useful to understand why running locally does not automatically protect sensitive data. More background on this topic is available in our hub on AI privacy and the GDPR in professional practice.

Sources: The article draws on the EDPB guidance on anonymisation, the CNIL analysis of the GDPR status of AI models, an OECD report and a peer-reviewed ACM study on model-centric privacy attacks.