Loading…
Compliance
The simplest way to reduce GDPR Article 5, HIPAA, and CCPA risk in an AI system is to prevent personal data from reaching the LLM in the first place. A PII redaction layer sits between your data and your LLM API calls, detecting and replacing personal identifiers before they are included in any prompt. If the data never reaches the model, the model cannot leak it, and the DPA coverage gap with your LLM vendor becomes much smaller.
We build custom PII detection and redaction pipelines using spaCy, Microsoft Presidio, and GLiNER for custom entity types. Standard pipelines cover common identifiers: names, emails, phone numbers, SSNs. Healthcare pipelines target all 18 HIPAA Safe Harbor identifiers. Reversible tokenisation lets you reconstruct original context post-inference without ever sending the original values to the LLM.
Tell us what you're building.
Five components that form a complete PII redaction layer for an AI pipeline. Each is independently deployable or combined into a full redaction stack.
Detects names, dates of birth, addresses, phone numbers, email addresses, Social Security Numbers, and account numbers using a combination of NER models and pattern-matching. We use spaCy's en_core_web_trf transformer model as the base recogniser, supplemented by GLiNER for zero-shot detection of entity types not in the default training set. Regex patterns handle high-precision identifiers (SSN format, US phone formats, email addresses) where NER confidence is secondary.
The HIPAA Safe Harbor de-identification standard at 45 CFR §164.514(b)(2) specifies 18 identifiers that must be removed before data is considered de-identified. Our healthcare pipeline explicitly targets all 18: names, geographic data smaller than state level, dates (except year), phone and fax numbers, email addresses, SSNs, medical record numbers, health plan beneficiary numbers, account numbers, certificate/licence numbers, vehicle identifiers, device identifiers, URLs, IP addresses, biometric identifiers, full-face photographs, and any other unique identifying number.
Rather than permanently deleting PII, reversible tokenisation replaces each detected entity with a consistent pseudonym ([PERSON_1], [DATE_1], [PHONE_1]) that persists across the prompt. Post-inference, the original values can be restored using a local lookup table that never leaves your infrastructure. This allows the LLM to reason about relationships between entities while the actual PII stays within your environment.
Off-the-shelf NER models miss domain-specific identifiers: medical record numbers (MRNs), policy numbers, student IDs, financial account identifiers, and internal customer IDs. We add custom recognisers using GLiNER for zero-shot entity types and spaCy's EntityRuler for pattern-based identifiers. Custom entity types are documented with their detection pattern and tested against held-out samples before deployment.
Every redaction event is logged with: entity type detected, confidence score, source document reference, and the redaction action taken (replaced, masked, or skipped). Logs are structured JSON written to your logging infrastructure. This provides the evidence trail needed for GDPR Article 5 data minimisation compliance, HIPAA audit controls at §164.312(b), and SOC 2 processing integrity criteria PI1.2.
The specific engineering deliverables for a PII redaction engagement. Beyond the detection models, the integration and operational pieces are often where the real work is.
A middleware component that sits between your data layer and your LLM API calls. Input text passes through the redaction pipeline before prompt construction; the redacted text is used in the prompt; LLM output is returned with optional re-identification applied post-inference. The component is designed to be inserted into existing pipelines with minimal application code changes.
We evaluate spaCy, Microsoft Presidio, and GLiNER against your specific data types and language patterns, select the combination with the best precision-recall tradeoff for your use case, and configure confidence thresholds to balance coverage against false positive rate. For healthcare data, we use Presidio's pre-built healthcare analysers as a starting point. For financial data, we build custom recognisers.
PII redaction at high sensitivity produces false positives, benign text flagged as PII, which degrades LLM response quality. We set up a monitoring pipeline that samples redaction events, flags anomalous false positive rates, and alerts when entity types are over-triggering. Threshold tuning is an ongoing process in the first weeks after deployment.
For use cases requiring reversible tokenisation, we implement the post-inference re-identification step: parsing LLM output for pseudonym tokens, looking up originals in the local token store, and reconstructing the response with original values restored. Token stores are encrypted at rest and scoped to a single session or document to limit blast radius if a token store is compromised.
Documentation of the redaction pipeline for compliance purposes: entity types covered, models used, confidence thresholds, false positive rate from testing, and the mapping to specific regulatory requirements (HIPAA 18 identifiers, GDPR Article 5, CCPA categories). This documentation feeds directly into your privacy impact assessment and audit evidence packages.
We prefer open-source, locally-deployed models over cloud-based detection APIs for inline pipeline redaction. Sending data containing PII to a cloud detection API to detect PII can recreate the compliance exposure you are trying to eliminate.
Cloud-based options (AWS Macie, Google Cloud DLP) are appropriate for scanning data at rest in cloud storage, where the detection service is in the same cloud environment as the data.
Open-source PII detection and anonymisation framework from Microsoft. Pre-built analysers for common entity types. Used as the primary orchestration layer.
Industrial-strength NLP library. The en_core_web_trf transformer model provides the base NER capability. Used for entity extraction within the Presidio pipeline.
Zero-shot named entity recognition model for custom entity types. Handles domain-specific identifiers that are not in standard NER training sets.
Managed PII detection for data at rest in S3. Useful for scanning training datasets and data lakes before ingestion into AI pipelines.
Managed PII detection API. High throughput option for teams already in GCP who need a managed service rather than a self-hosted model.
PII redaction solves a specific problem. We are direct about when it is not the right solution.
Teams who can avoid using PII in their AI workflows entirely
The cleanest solution to PII risk in AI systems is to not include PII in AI inputs in the first place, redesigning the workflow so the AI task can be accomplished without personal data. If that is achievable for your use case, it is a better solution than redaction. Redaction adds latency, has a non-zero false negative rate, and requires ongoing maintenance. We will tell you if we think your use case can be restructured to avoid PII entirely.
Teams requiring real-time sub-100ms redaction at very high throughput
Transformer-based NER models add 20–80ms of latency per redaction call depending on text length and hardware. For most AI applications this is acceptable. For high-frequency real-time chat applications processing thousands of concurrent messages, the latency and infrastructure cost may be prohibitive. Contact us to discuss architectural options. There are approaches using lighter models and caching that can reduce latency significantly, but they involve precision tradeoffs.
Anyone expecting 100% detection accuracy
No automated PII detection system achieves 100% recall. Well-tuned systems on structured data can reach 95–99% recall for common entity types. Novel formats, obfuscated identifiers, and out-of-distribution data reduce accuracy. We document the expected false negative rate for each entity type, test against held-out samples from your actual data, and design the system to minimise (not eliminate) PII exposure.