Healthcare AI projects stall on compliance questions more than any other kind. Teams either overestimate what's needed ("we have to self-host everything") or underestimate it ("our cloud provider is HIPAA-compliant, so we're covered"). Both assumptions lead to problems.
This post covers what HIPAA actually requires of an AI system, what questions to ask your LLM provider, and the architectural decisions that determine whether you're actually compliant.
What HIPAA requires (and what it doesn't say about AI)
HIPAA doesn't mention AI, LLMs, or machine learning. It regulates how covered entities (healthcare providers, health plans, clearinghouses) and their business associates handle Protected Health Information (PHI).
The core requirements that matter for AI systems:
- Business Associate Agreements (BAA) — any vendor that processes PHI on your behalf must sign a BAA accepting shared HIPAA responsibility
- Minimum necessary standard — you should only access, use, or disclose the minimum PHI needed for a given purpose
- Audit controls — you must have systems to track who accessed PHI and when
- Transmission security — PHI transmitted electronically must be encrypted in transit
- Access controls — only authorized users and systems should access PHI
Critically: HIPAA compliance is about meeting these requirements, not about any specific technology choice. A cloud-hosted LLM system can be HIPAA-compliant. A self-hosted system can be HIPAA non-compliant. What matters is the controls.
What counts as PHI in an LLM prompt
PHI includes any individually identifiable health information. In an AI system context, this includes:
Obvious PHI:
- Patient name + diagnosis
- Medical record numbers
- Insurance ID numbers
- Dates of service combined with patient name
Less obvious PHI:
- IP addresses combined with health information
- Voice recordings where health conditions are mentioned
- Free-text clinical notes (even without an obvious identifier, if the content is specific enough to identify a patient)
- Email addresses in a medical context
Generally not PHI:
- De-identified data (if properly de-identified under HIPAA Safe Harbor or Expert Determination)
- Aggregate statistics that can't identify individuals
- General medical knowledge queries without patient-specific information
The practical implication: if a clinician types "patient John Smith, DOB 1982, presenting with chest pain" into your AI system, that's PHI from the moment it enters your system, regardless of what the AI does with it afterward.
The BAA landscape for LLM providers
Signing a BAA with an LLM provider doesn't automatically make your system HIPAA-compliant — it means the provider has accepted shared responsibility for protecting PHI that passes through their system.
OpenAI: offers a BAA via the ChatGPT Enterprise product and for some API customers. The standard API does not come with a BAA. OpenAI's BAA covers their processing but does not cover OpenAI using your data to train their models (this is excluded under the enterprise agreement).
Anthropic: provides HIPAA-covered access to Claude via AWS Bedrock or Google Cloud Vertex AI. The BAA is with AWS/Google, not Anthropic directly. Claude accessed through Anthropic's direct API does not come with a BAA by default.
Azure OpenAI: Microsoft Azure is HIPAA-eligible and Microsoft will sign a BAA. GPT-5 accessed through Azure OpenAI, with data residency configured appropriately, is a common path for healthcare AI.
Google Vertex AI: HIPAA-eligible. Gemini and other models accessed through Vertex AI with appropriate configuration are BAA-covered.
Important nuance: "HIPAA eligible" is not the same as "automatically HIPAA compliant." The BAA creates shared responsibility. Your architecture, configuration, and controls determine whether you're actually meeting the requirements.
Self-hosting vs managed API for PHI
The most common question we get: "Do we have to self-host to be HIPAA compliant?"
No. But self-hosting does offer specific advantages for healthcare:
| Consideration | Managed API (Azure/AWS/GCP) | Self-hosted | |---|---|---| | BAA availability | Yes (from cloud provider) | N/A (you own it) | | Data never leaves your infra | No | Yes | | Fine-tuning on PHI | Generally prohibited or restricted | Possible | | Implementation complexity | Low-medium | High | | Ongoing maintenance | Low | High | | Cost at scale | API pricing | Infra + operations |
For most healthcare AI use cases — internal tools, clinical decision support, documentation assistance — managed API with a BAA is the right path. Self-hosting makes sense when:
- Your compliance team requires data never leave your infrastructure
- You need to fine-tune on PHI-containing datasets
- You have extremely high volume where API pricing exceeds self-host infra costs
Audit logging requirements
HIPAA requires audit controls — records of who accessed PHI and when. For an AI system, this means logging:
- User identity for every request
- Timestamp
- What PHI was included in the request (at minimum, a patient identifier if one was present)
- The action taken (query type, response given)
- Any downstream system calls made by the AI
HIPAA's 6-year retention requirement (§164.316(b)(2)) applies to required documentation — policies, procedures, and records — rather than to audit logs specifically; the Security Rule doesn't set a fixed audit-log retention period. In practice, retaining logs for 6 years to match the documentation baseline is a common, defensible standard. They must be protected from unauthorized modification (immutable log storage — write-once S3 buckets, Azure Immutable Blob Storage, or equivalent).
AI systems add a specific audit challenge: when a clinician asks a general question ("what are the treatment options for stage 3 NSCLC"), that might not involve PHI. When they ask "what are treatment options for patient John Smith in room 412," it does. Your logging infrastructure needs to capture the difference.
The three biggest HIPAA mistakes in AI systems
1. Assuming the cloud provider's compliance covers the application.
Your cloud provider being HIPAA-eligible means their infrastructure meets HIPAA requirements. It says nothing about your application. If your AI system logs PHI to a publicly accessible endpoint, encrypts data with a weak key, or has no access controls, you're non-compliant regardless of where it runs.
2. Putting PHI in logs without audit protection.
Development teams log everything. For good reason — debugging AI systems requires seeing the actual inputs and outputs. But production logs containing PHI need the same protection as any other PHI store: access controls, retention policies, immutability. The number of healthcare AI systems we've seen with patient data in plaintext development logs is significant.
3. Not scoping the BAA to the specific product.
LLM providers often have HIPAA-covered and non-HIPAA-covered products within the same company. Anthropic via AWS Bedrock is covered; Anthropic's direct API is not. OpenAI's standard API is not covered; Azure OpenAI may be. Read the BAA carefully and confirm it covers the specific product, API endpoint, and data processing you're doing.
HIPAA-eligible architecture pattern
A compliant architecture for a healthcare AI system typically looks like:
- Application layer — user-facing interface with authentication (SSO, MFA) and role-based access controls
- API layer — request handling with PHI detection, audit logging to immutable storage
- AI processing — LLM calls via HIPAA-covered cloud API (Azure OpenAI, AWS Bedrock, Vertex AI) with data residency configured
- Data layer — encrypted storage, access controls, retention policies
De-identify where you can. If a use case doesn't require patient-identifiable data to work (many analytics and research use cases), de-identify before it enters the AI layer. Less PHI in the system means less compliance surface area.
If you're building healthcare AI and want to understand what compliance requirements apply to your specific use case, the contact page is a good place to start. These projects have distinct architecture requirements that are worth discussing before you start building.