Book a call (opens Google Calendar)
Book a call (opens Google Calendar) hey@pronovahq.com

The 5 Biggest Engineering Problems with GDPR-Compliant AI

By Izzy Aly, Co-Founder, Systems and Architecture

GDPR fines have hit EUR 6.11 billion. Legal policies don't stop data leaks, architecture does. Five engineering failures in "compliant" AI, and the fixes.

Your legal team handed you a 40-page GDPR policy. Your DPO signed off. Your privacy page looks great. And none of it matters, because your LLM just echoed a customer's home address back to the wrong user.

Here's the uncomfortable truth: 88% of organisations now use AI in at least one business function, but only 8% have implemented a comprehensive AI governance framework (McKinsey, The State of AI in 2025; Economist Impact / Kyocera survey of 639 executives, late 2025). That's not a policy gap. That's an engineering gap.

I'm not going to rehash legal checklists. These are the five biggest engineering failures I see in "GDPR-compliant" LLM or agentic AI systems, and the architectures that fix them. If you got the AI mandate but don't write the code, the "How to Fix It" section under each problem is what to hand your engineer.

1. Your PII Detection Has Blind Spots You Don't Know About

Most teams bolt on a basic PII scanner and call it done. A regex catches email addresses. Maybe a Named Entity Recognition (NER) model flags obvious names. But production data is messy, multilingual, and full of edge cases that deterministic rules miss.

A customer writes, "My daughter Sophie starts school at Karlstadsskolan in August," in a support chat. That's a minor's name, a school name, and an implied location, none of which your regex will catch. That prompt goes to your LLM provider's API, gets logged in their system, and now you've sent a child's personal data to a third party without consent.

The gap between "we have PII detection" and "our PII detection actually works" is where fines live. GDPR fines have reached EUR 6.11 billion across 2,685 recorded cases as of the CMS GDPR Enforcement Tracker Report's March 2026 cut-off (CMS Law), and the report cites a EUR 5 million fine issued by the Italian DPA in an AI-related case.

How to Fix It

Production-grade PII detection needs three layers working together:

  • NER models (spaCy, Hugging Face transformers) for names, locations, and organisations.
  • Pattern matching (regex + checksum validation) for IBANs, credit card numbers, and email addresses.
  • Allow-lists for terms that look like PII but aren't (e.g., your company's support address).

This runs as a middleware layer between your app and your LLM provider, so nothing crosses the network without passing the PII firewall. The combination matters: a July 2025 study combining rule-based NLP with NER models on financial documents reported 94.7% precision and a 91.1% F1 score on synthetic test data (Scientific Reports, 2025). On multilingual data, a hybrid regex-plus-context-aware-LLM approach outperformed fine-tuned NER models by 82% and zero-shot LLMs by 17% in weighted F1 score across 13 low-resource locales (arXiv, October 2025).

A three-layer PII firewall between the app and the LLM provider A prompt leaves the app and passes through PII middleware before it reaches the LLM provider API. The middleware has three layers: NER models for names, locations and organisations; pattern matching with regex and checksum validation for IBANs, card numbers and emails; and allow-lists for terms that look like PII but are not. The response comes back through the same middleware before it reaches the end user. Example: "My daughter Sophie starts school at Karlstadsskolan in August" contains a minor's name, a school name and an implied location, and regex alone misses all three. Hybrid rule-based NLP plus NER reported 94.7% precision and 91.1% F1 (Scientific Reports, 2025); hybrid regex plus context-aware LLM beat fine-tuned NER by 82% and zero-shot LLMs by 17% in weighted F1 across 13 low-resource locales (arXiv, 2025). PII FIREWALL: EVERY CALL, BOTH DIRECTIONS App support chat End user clean response PII middleware scans prompts out and responses back NER models names, locations, organisations Pattern matching checksums: IBANs, cards, emails Allow-lists look-alikes, e.g. support address LLM provider API “My daughter Sophie starts school at Karlstadsskolan in August” A minor’s name, a school name and an implied location. Regex alone misses all three. Rule-based NLP + NER: 94.7% precision, 91.1% F1 (Scientific Reports, 2025). Regex + context-aware LLM: +82% weighted F1 vs fine-tuned NER (arXiv, 2025).
A three-layer PII firewall between the app and the LLM provider A prompt leaves the app and passes through PII middleware before it reaches the LLM provider API. The middleware has three layers: NER models for names, locations and organisations; pattern matching with regex and checksum validation for IBANs, card numbers and emails; and allow-lists for terms that look like PII but are not. The response comes back through the same middleware before it reaches the end user. PII FIREWALL, BOTH DIRECTIONS App End user PII middleware scans both ways NER models names, places, orgs Patterns IBANs, cards, emails Allow-lists known look-alikes LLM provider API
Fig. 1 — Three detection layers in one middleware, scanning prompts out and responses back. Source: Scientific Reports, 2025; arXiv, 2025.

Scan both sides of the call: prompts before they leave your system and responses before they reach the user. Track precision and recall, and alert on drift; otherwise your "PII firewall" silently turns into a sieve.

2. Data Deletion Requests Break When Models Memorise Data

You cannot DELETE FROM model_weights WHERE user_id = 456.

A user exercises their Right to Erasure under Article 17. Simple enough: delete their data. Except their data was used to fine-tune your model three months ago, and it's now entangled in billions of parameters.

Machine unlearning is still experimental: techniques like gradient ascent on target data can degrade the whole model or leave residual traces, and you can't retrain a 70B parameter model every time a user deletes their account. If you've fine-tuned on user data, you may be unable to honour deletion requests, which means you're non-compliant by design.

How to Fix It

Use the Erasure-Safe RAG Pattern.

Fine-tune for style, tone, format, and reasoning patterns, never for knowledge that contains personal data. Separate your model's reasoning from its knowledge using Retrieval-Augmented Generation (RAG), and store user data in a vector database where you can apply normal CRUD operations.

When a deletion request comes in, run a four-step loop:

  1. Query the vector DB for all chunks tied to the user_id.
  2. Delete the matched vectors.
  3. Run a verification query (expect zero results).
  4. Log the event to a compliance ledger.
Erasure on fine-tuned weights versus the Erasure-Safe RAG deletion loop Left, a dead end: user data fine-tuned into model weights has no delete path. DELETE FROM model_weights is not possible, and unlearning degrades the model or leaves residual traces. Right, the Erasure-Safe RAG pattern: an Article 17 erasure request triggers a four-step loop on the vector database. One, query the vector DB for all chunks tied to the user_id. Two, delete the matched vectors. Three, run a verification query and expect zero results. Four, log the event to a compliance ledger that a regulator or DPO can check. FINE-TUNED ON USER DATA ERASURE-SAFE RAG Fine-tuned weights no delete path DELETE FROM model_weights not possible Unlearning degrades the model or leaves traces Erasure request Article 17 01 Query vector DB chunks for user_id 02 Delete vectors matched chunks 03 Verify expect zero results 04 Log to ledger provable record Keep personal data out of the weights and erasure becomes a database operation.
Erasure on fine-tuned weights versus the Erasure-Safe RAG deletion loop An Article 17 erasure request triggers a four-step loop on the vector database: query the vector DB for all chunks tied to the user_id, delete the matched vectors, run a verification query and expect zero results, then log the event to a compliance ledger. Fine-tuned model weights have no delete path. ERASURE-SAFE RAG Erasure request Article 17 01 Query vector DB by user_id 02 Delete vectors matched 03 Verify zero results 04 Log to ledger provable Fine-tuned weights no delete path
Fig. 2 — Keep personal data out of the weights and erasure becomes a four-step database operation with a ledger entry as proof.

When a regulator or DPO asks, "Can you prove you deleted this user's data?" you point to a verifiable query plan, not hand-waving about model weights.

3. "Privacy by Design" Is Treated as a Checkbox, Not an Architecture

Article 25 mandates Privacy by Design (PbD). Most teams treat it as a document they write before launch. It's an architectural requirement, and getting it wrong is expensive: IBM's 2025 Cost of a Data Breach Report found that 13% of organisations reported breaches of AI models or applications, and 97% of those breached organisations lacked proper AI access controls at the time of the incident (IBM, 2025).

The most common failure we see is engineers assuming vector embeddings are anonymous because they're arrays of floating-point numbers. They aren't: high-dimensional embeddings can be inverted to reconstruct the original text or infer sensitive attributes. Treat them as anonymous and you get "leakage via relevance," where an unauthorised but relevant document surfaces in a search result just because it semantically matches the query.

How to Fix It

Build access control into your retrieval layer, not around it. The engineering pattern here is "Filter First, Search Second":

  1. Extract the user's permissions from their session (role, department, region).
  2. Apply metadata filters to your vector DB before running the semantic search.
  3. Only return chunks the user is authorised to see, and only assemble those into the LLM context window.

Log the filters you applied and the index used, so you can prove unauthorised content never entered the context window. Under the hood, this means row-level security and tenant isolation on your vector store, so one internal user can't see another user's data just because the embedding is "similar."

Search then filter versus Filter First, Search Second Top row, the failure: a query runs semantic search over the whole vector index, and a relevant but unauthorised document lands in the LLM context window, which is leakage via relevance. Bottom row, Filter First, Search Second: one, extract permissions from the session (role, department, region); two, apply metadata filters to the vector DB before the search; three, run the search over permitted chunks only and send only those into the context window; four, log the filters applied and the index used. IBM's 2025 Cost of a Data Breach Report found 13% of organisations reported breaches of AI models or applications, and 97% of those lacked proper AI access controls. SEARCH, THEN FILTER Query any user Semantic search over the whole index Unauthorised doc “leakage via relevance” FILTER FIRST, SEARCH SECOND 01 Permissions role, department, region 02 Metadata filter applied before the search 03 Search permitted chunks into context 04 Log filters applied, index used 13% of organisations reported breaches of AI models or applications. 97% of those lacked proper AI access controls. Source: IBM, 2025.
Search then filter versus Filter First, Search Second The failure: semantic search over the whole index lets a relevant but unauthorised document leak into the context window. Filter First, Search Second: extract permissions from the session (role, department, region), apply metadata filters before the search, search only the permitted chunks and send them into the context window, then log the filters applied and the index used. SEARCH, THEN FILTER Whole-index search unauthorised doc leaks FILTER FIRST, SEARCH SECOND 01 Permissions role, dept, region 02 Metadata filter before search 03 Search permitted chunks only 04 Log filters + index used
Fig. 3 — Filter on session permissions before the semantic search, then log what was applied. 13% of organisations reported breaches of AI models or applications; 97% of those lacked proper AI access controls. Source: IBM, 2025.

4. Audit Trails Don't Survive a Regulator's First Question

Article 30 requires a Record of Processing Activities (ROPA). Most teams keep it in a static spreadsheet, and their logging captures application errors but misses compliance events. The ROPA documents processing at a policy level. When a Data Protection Authority (DPA) shows up, they want the technical evidence underneath it: the exact lineage of a decision, and proof that PII was masked before it hit a third-party API. If you can't prove it with logs, it didn't happen.

How to Fix It

Automate your ROPA through your CI/CD pipeline using tools that scan for data sinks and third-party flows. For AI-specific logging, you need to capture:

  • The sanitised prompt sent to the LLM, proving PII was masked before it left your boundary.
  • Classification tags for what was detected ("2 emails, 1 IP address").
  • The policy/config version active at the time.
  • Consent/legal-basis verification results.
  • Retention/deletion events showing data was removed when the policy said it should be.

Set retention periods for compliance logs (e.g., 3-5 years) and enforce them. Store the logs in an append-only, tamper-evident system (think WORM storage or hash-chained ledgers). If logs can be edited, they aren't evidence; they're fiction.

5. Compliance Degrades Silently Between Audits

Compliance is a signal you monitor, not a state you achieve. Models get updated. Data sources change. A PII detection model with 94% precision at deployment can drop meaningfully after months of distribution shift. Fewer than half of organisations (48%) monitor their production AI systems for accuracy, drift, or misuse at all, and small companies trail further behind (Pacific AI / Gradient Flow 2025 AI Governance Survey, 351 respondents). If you aren't monitoring, you won't know you're leaking data until the fine arrives.

How to Fix It

Privacy needs SLOs and dashboards, not just Confluence pages. Monitor:

  • PII detection rate trends: a sudden spike or drop means something changed upstream.
  • Guardrail latency: if your PII scanner adds 800ms to every request, developers will quietly route around it.
  • Privacy incident rate: how often PII shows up where it shouldn't (outputs, non-PII logs).
  • Jailbreak attempts: adversarial prompts that try to extract PII need real-time flagging.

Set SLOs (e.g., minimum detection precision, maximum guardrail latency) and alert when you blow past them.

What we do on client builds

On the agents and connectors we build:

  • The agent runs in your own account with Anthropic, OpenAI or Vercel. If your data can't leave your control, we use only AI models that keep none of it, or run on your own cloud.
  • You decide which data the agent can see.
  • Personal data stays out of the weights. It lives in a store you can query and delete from, not in a fine-tuned model.
  • Every action is logged: who asked, what changed, and when.
  • Nothing important happens without your sign-off. Deletes and bulk changes show a preview and wait for a person to confirm.
  • The agent proves itself first. It only acts on its own once its work on real cases matches your team's.

The Pattern Is Clear: Compliance Is an Engineering Problem

Every problem on this list comes back to the gap between what your policies say and what your systems actually do. Legal documents don't prevent data leakage. Architecture does. Policies don't prove compliance. Monitoring does.

These are solvable engineering problems. RAG architectures make deletion provable. PII firewalls make masking deterministic. Automated ROPA keeps documentation in sync with reality.

You don't have to rebuild your architecture from scratch. If you want someone to build the first governed piece of it with you, book a scoping call.

Topics
Author
Izzy Aly · Co-Founder, Systems and Architecture
Share

Keep reading

All posts →