Does Zero Data Retention Satisfy HIPAA, and What a BAA Covers That ZDR Does Not

Within a HIPAA compliance framework, Zero Data Retention limits provider-side storage after inference, while a Business Associate Agreement establishes the vendor’s legal obligations around protected health information. A defensible healthcare AI deployment also needs controls before transmission, including PHI minimization, authorized access, and audit records that do not expose sensitive content.

Why ZDR and a BAA are not the same control

Teams make a mistake when they treat a provider’s Zero Data Retention promise as permission to send it PHI. A team may establish that prompts will not be stored and consider the vendor review complete. But storage is only one part of the relationship. A vendor that receives or processes PHI on a covered entity’s behalf has obligations even when it discards the data immediately after responding.

Under 45 CFR § 164.504(e), a vendor must have a Business Associate Agreement if it creates, receives, maintains, or transmits PHI for a covered entity. Without the required agreement, non-retention does not cure the violation. HIPAA does not, however, mandate ZDR. A vendor may retain PHI when the BAA permits it and the data is secured, so a retention setting and an agreement answer different procurement questions.

Training opt-out introduces another distinction that reviews often miss. It prevents inputs from being used for model improvement, but it does not prevent temporary storage or logging. With ZDR, the contract and settings remove the retention window that would otherwise apply to the provider. Neither control sets out who is legally responsible at each step of PHI processing.

What ZDR actually covers — and where its boundary ends

ZDR governs a specific provider’s inference infrastructure. Under that arrangement, the provider processes the prompt in memory, returns a response, and discards the prompt and output without writing them to persistent storage. For standard API traffic, providers usually have a retention window to monitor for abuse. ZDR removes that window for covered requests, giving teams a documented basis for stating that the upstream provider does not persist their inputs.

That is a valuable but bounded protection. The application calling the provider can still retain transcripts or prompts on its own servers, even when its upstream requests qualify for ZDR. An auditor asking whether the model host retains a prompt is therefore asking a different question from one asking whether the healthcare application retains it.

The same boundary applies across the rest of the pipeline. Vector databases, embedding APIs, orchestration tools, and monitoring platforms may each handle information outside the model provider’s infrastructure. ZDR with one provider does not cover those other links. The relevant unit of review is the data path.

The brief’s 2026 provider terms identify Anthropic’s Covered Models, Claude Fable 5 and Mythos 5, as retaining data for up to 30 days and being excluded from standard ZDR agreements. Teams must check whether every model and API surface their workflow uses is eligible, because account-level ZDR may not cover every request.

What a BAA obligates a vendor to do that ZDR never touches

A BAA establishes enforceable obligations throughout the vendor’s relationship with PHI. 45 CFR § 164.504(e)(2) requires the agreement to address permitted uses and disclosures as well as how long data remains in storage. It also requires safeguards aligned with the HIPAA Security Rule. Those safeguards include administrative, physical, and technical controls, so the vendor has duties that go beyond inference.

The agreement also reaches the vendor’s subcontractors. The same restrictions must apply to any model host, cloud provider, or annotation contractor that handles PHI. A direct agreement does not automatically protect the chain. The BAA must also address incident reporting, return or destruction of PHI when the relationship ends, and access to books and records by HHS upon request.

A Business Associate must notify the Covered Entity within 60 days of discovering a breach of unsecured PHI, though the agreement may require it to do so sooner. ZDR creates no equivalent notification process on its own. It limits retention, but it does not establish who must report a breach or which obligations extend to subcontractors.

The two controls are complementary rather than interchangeable. A BAA can restrict retention without creating a technical barrier to storage, while ZDR implements non-retention without supplying the full legal framework. The agreement still does not certify the whole workflow. Explicitly excluded product surfaces remain outside its scope, and workforce training, minimum-necessary enforcement, audit logging, and risk analysis remain the covered entity’s responsibility.

How major providers implement ZDR and BAA eligibility in practice

Provider eligibility depends on the product surface and contract. OpenAI makes BAAs available for healthcare workloads on the OpenAI API and for sales-managed ChatGPT Enterprise/Edu accounts, including ChatGPT for Healthcare. Its API BAA covers only ZDR-eligible endpoints. Assistants, Threads, Vector Stores, Files, Batch, and model-customization features retain application state until explicitly deleted, and enabling ZDR elsewhere on an account does not change that behavior. ChatGPT Free, Plus, Pro, and self-serve Business, formerly Team, are not BAA-eligible.

Anthropic makes BAAs available for the Claude API on paid contracts, HIPAA-ready Claude Enterprise plans, and Claude in Microsoft Foundry. Enterprise activation requires the Primary Owner to open Organization Settings, then Data and Privacy, then HIPAA Compliance, and accept the BAA. Anthropic describes that activation as a one-way decision. Its exclusions include Claude Free, Pro, Max, Team, Claude in Chrome, third-party connectors, and research previews. Claude Code is excluded unless ZDR or the HIPAA configuration for local mode is separately enabled through an Anthropic account team.

Google Workspace with Gemini on Business/Enterprise tiers became HIPAA-eligible under Google’s BAA in late 2024. Consumer Gemini is not eligible when used through personal Google accounts. Vertex AI is the primary compliant path for API PHI workloads. The distinction is between covered services and consumer access.

Microsoft delivers its HIPAA BAA through the Microsoft Online Services Data Protection Addendum. It covers M365 Copilot under commercial/enterprise tenants and Azure OpenAI Service. Microsoft 365 Family and Personal subscriptions are not BAA-covered, regardless of the feature used. Consumer tiers across these providers are therefore not usable for PHI workflows.

A vendor may offer a BAA that leaves out the endpoint, feature, or subscription a team plans to use. Vendor posture also changes over time. Before teams treat a workflow as covered, they should check each provider’s compliance page for what its BAA now covers, what is eligible for ZDR, and how to turn it on.

Why PHI can still reach a provider unguarded even when both ZDR and a BAA are in place

A provider may receive unnecessary PHI even with a signed BAA and enabled ZDR. Their protections do not automatically minimize an outbound prompt. Transmission remains a separate point of exposure. A workflow can use a covered endpoint with non-retention enabled and still send more identifying information than the task requires.

45 CFR §§ 164.502(b) and 164.514(d) set out the minimum-necessary requirements for PHI. Removing identifiers wherever possible is therefore part of the compliance obligation. Safe Harbor and Expert Determination are methods for de-identification. The architecture must address what leaves the application as well as what the recipient does afterward.

Logging is a concrete way that this architecture can fail. A team may redact PII from an outbound prompt but preserve the original prompt in a debugging or audit log. The sensitive content has not disappeared from the workflow. It now sits in a log store that the model provider’s BAA does not cover, so redaction at the provider boundary has left another exposure unresolved.

The subcontractor chain presents a related problem. A vector database, embedding service, orchestration tool, or monitoring platform that handles PHI needs its own agreement that covers that use and must be set up correctly. The transition from contracts to architecture requires attention to every component that receives sensitive content.

The architectural response is to detect PHI or PII at a centralized chokepoint before forwarding a request. Redaction, masking, or hashing can sanitize the prompt, with sensitive values restored on the response where the mechanism supports it. Audit records need a different treatment. They should retain the model, team, guardrail event, and detected data category without reproducing the raw content that redaction was intended to protect.

What an LLM gateway adds to a ZDR-plus-BAA posture

An LLM gateway provides a place to enforce those transmission controls across providers. When every outbound API call passes through it, the gateway can detect sensitive information and use the same redaction policy before the provider gets the request. Masking, synthetic replacement, or hashing becomes part of the request path rather than a convention that each application must remember.

Audit integrity belongs at the same boundary. Compliance-grade records need enough information to reconstruct events for OCR review, including the model used, team, trace ID, detected data category, and guardrail outcome. They do not need the sensitive text itself. Metadata-only auditing can preserve timestamps, token counts, policy outcomes, and anomaly flags without turning observability into another store of PHI.

Identity controls make those records attributable to authorized users. RBAC scoped by team, project, and model, integrated with existing identity providers through SSO, can restrict who submits PHI-touching requests. Access and redaction solve different problems. An authorized user still needs minimization controls, while a sanitized prompt does not remove the need to establish who sent it.

Provider fallback makes centralized enforcement especially useful. A gateway can route to another provider during an outage without a code change, and gateway-level redaction applies consistently to the providers it routes to. Contract scope still governs the fallback destination. Consistent sanitization does not make an otherwise uncovered provider eligible to receive any PHI that remains.

For teams evaluating a HIPAA-compliant AI gateway, the selection criteria follow directly from these gaps. The gateway needs an applicable BAA when it handles PHI, redaction before upstream transmission, access controls, and audit logs that do not retain raw sensitive content. Managed delivery can remove proxy infrastructure maintenance. It does not remove the need to review the gateway’s own retention policy or the agreements covering its upstream routes.

What a defensible HIPAA posture for AI actually requires

A vendor review should begin with the complete PHI path and the agreements covering it. Signed BAAs are needed for every provider and AI subprocessor that touches PHI, including vector databases, embedding APIs, orchestration tools, and monitoring platforms. Product scope must match actual use. Consumer products such as ChatGPT Plus, Claude.ai Pro, and consumer Gemini do not qualify as parts of a compliant PHI workflow.

For a deployment adopting ZDR, the technical review should establish that it is enabled on every BAA-covered endpoint the workflow uses. Account-level assurances are not sufficient evidence of endpoint coverage. Encryption must also be addressed independently of retention. The review should require TLS 1.2+ in transit and AES-256 at rest, with confirmation through the provider’s SOC 2 Type II report.

The application review should establish where PHI minimization occurs before transmission and how policy enforces it. A centralized chokepoint should centralize decisions about redaction or de-identification and apply them. Identity and logging require their own evidence. Unique user IDs, MFA, RBAC, automatic logoff, and per-request audit records retained according to OCR guidance should be part of that review.

The incident process should also spell out what the provider must report and who must be notified downstream. It must account for the requirement that a Business Associate notify a Covered Entity within 60 days of discovering a breach of unsecured PHI. Shorter contractual windows also need to be operationally workable. A completed review should leave the team with a defined PHI route, covered services, verified retention settings, enforced transmission controls, and a documented response process.