Skip to main content
INFRASOFT

Infrasoft Engineering Team6 min read

GDPR-compliant data pipelines: an architecture primer for B2B teams

GDPR obligations such as erasure, retention and lawful basis are architecture constraints, not policy text. This primer maps them to concrete pipeline decisions: EU data residency, encryption, DPAs with subprocessors, pseudonymisation zones and deletion propagation.

Most teams treat GDPR as a legal exercise: a privacy policy, a cookie banner, a signed DPA in a drawer. That approach fails the first time an erasure request has to reach a data warehouse, or the first time an enterprise client’s auditor asks where exactly a backup lives. The regulation makes concrete demands on how data moves through your systems — what enters a pipeline, how long it persists, who can read it, and whether you can prove all of it.

This primer translates the main obligations into architecture decisions. It is written for technical decision-makers in B2B companies that move personal data at scale — CRM records, user events, transaction histories — and who want pipelines that pass an audit rather than policies that promise one.

Four obligations that live in the architecture

Most of the GDPR text does not concern engineers. Four obligations do, because they constrain schemas and topology directly.

  • Lawful basis propagation. Every record of personal data was collected under a specific legal basis — consent, contract, legitimate interest. That basis must travel with the data, or be resolvable from it, so downstream systems can act when it changes. If consent is withdrawn in the CRM, the warehouse and every derived table need to know.
  • Data minimisation. A pipeline should carry the fields the processing purpose requires and drop the rest at ingestion. Copying full source records because storage is cheap multiplies your compliance surface with every table.
  • Retention limits. Each dataset needs a defined retention period and an automated job that enforces it. Retention handled by ad-hoc scripts, or not at all, is one of the most common audit findings.
  • Right to erasure. When a data subject asks, you must delete their personal data from every store that holds it, within one month. This is a distributed-systems problem, and the hardest of the four to retrofit.

EU data residency: pin the regions, then check the edges

All three major clouds operate EU regions — Frankfurt, Paris, Amsterdam and Dublin among them — and pinning primary storage to one of them is straightforward. The risk sits at the edges: cross-region backup replication, disaster-recovery copies, log aggregation or error-tracking SaaS hosted in the United States, support tooling that exports production data. A residency claim is only as strong as the least careful component.

Any transfer outside the EEA requires a valid legal mechanism — an adequacy decision, or standard contractual clauses backed by a transfer impact assessment. For most B2B platforms, the simplest defensible position is to keep personal data in EU regions end to end: primary stores, replicas, backups and the observability stack. It removes an entire category of questions from every security review your clients will run on you.

Encryption: standard practice, applied completely

Encryption in transit means TLS 1.2 or higher on every hop, including service-to-service traffic inside your own network — internal traffic is where it is most often missing. Encryption at rest is a default on managed cloud storage, but defaults use provider-managed keys. Customer-managed keys through the cloud KMS give you rotation, per-environment separation and the ability to revoke access, and they are what enterprise security questionnaires increasingly expect.

For high-sensitivity fields, add field-level encryption in the application layer. It also enables crypto-shredding: destroy the key for a data subject, and their data becomes unreadable in every copy that key protected — including backups and archives you cannot practically rewrite. For append-only or immutable stores, this is often the only workable erasure mechanism.

Subprocessors and DPAs: the contract layer under the pipeline

Every third party that touches personal data on your behalf is a processor or subprocessor and needs a data processing agreement under Article 28. In a typical pipeline that list is longer than teams expect: the cloud provider, the managed ETL or streaming service, the warehouse vendor, monitoring and error-tracking tools — stack traces and log lines carry personal data more often than not — and any enrichment API.

Keep a live subprocessor register that maps each vendor to the pipeline component it serves, the data categories it sees and the region it runs in. Make signing the DPA part of the definition of done for adding any tool to the pipeline. Enterprise clients in France, Germany and the Netherlands will ask for this register during procurement; producing it in a day rather than a month is a credibility signal.

Pattern: pseudonymisation zones

Split the pipeline into two zones. An identified zone holds direct identifiers — names, email addresses, phone numbers — in a small number of tightly access-controlled stores. A pseudonymised zone, where analytics, reporting and machine learning happen, carries stable tokens instead of identifiers, with a vault mapping tokens back to identities for the few workflows that need them.

The benefit is blast-radius reduction: most of your infrastructure, and most of your team’s access, never touches a direct identifier, and a breach of the analytical zone discloses far less. Be precise about the limits — pseudonymised data is still personal data under GDPR, so the pattern reduces risk and audit scope; it does not exempt anything.

Pattern: deletion propagation and audit trails

Treat erasure as an event, not a script. An erasure request enters at a single point, is validated, and is published as a deletion event on your internal bus. Every store holding personal data — operational databases, warehouse, search indexes, caches, downstream SaaS via API — subscribes, executes its own deletion and reports completion. A coordinator tracks the fan-out and closes the request only when every consumer has confirmed, which gives you a defensible answer to the question of how you know it was deleted everywhere.

Backups need an explicit, documented policy: either deletion is re-applied automatically when a backup is restored, or backup retention is short enough that expiry does the work. Alongside deletion, maintain an append-only audit log of processing activity — consent changes, erasure confirmations, retention runs, access to the identified zone. The accountability principle in Article 5(2) means being able to demonstrate compliance, and this log is the demonstration.

Where to start

Begin with a data map: which systems hold personal data, which fields, under which basis, for how long. Then work through the layers in order of leverage — residency and encryption baseline first, retention automation next, erasure propagation after that, pseudonymisation zones where the sensitivity justifies them. Retrofitting is measured in months on a typical B2B stack; building these properties into a new pipeline costs a fraction of that.

We design and operate GDPR-compliant data pipelines as part of our Cloud Infrastructure & Data Processing work, with EU data residency and DPAs handled as standard. If you want a second pair of eyes on your current architecture, an audit is where we would start too.

All insights
contact@infrasoftdev.com

Let’s talk about your technical operations

A 30-minute introductory call with a senior engineer — no sales script, no obligation. We listen, we ask questions, and we tell you honestly whether we can help.