No BAA No Deal Why PHI Use Cases Require a Gateway That Will Sign
Patient data sent to AI models requires a signed BAA from every service in the data path.
A HIPAA Compliant AI Gateway must enforce signed BAA coverage across every PHI-handling provider and logging path. To prevent sensitive customer data from reaching OpenAI or another model provider, the gateway must redact PHI and PII before forwarding requests and restrict routing to covered endpoints. For a hosted gateway that receives PHI, no BAA means no deal.
Why a BAA is a structural constraint, not a legal formality
A BAA defines the permitted architecture before engineering selects the components. Under HIPAA, an entity that creates, receives, maintains, or transmits PHI on behalf of a covered entity is a business associate, and a signed BAA is required before sharing PHI with it. Sending patient data to a model provider’s API is such a disclosure. The agreement therefore governs a technical boundary.
That boundary extends to every service handling the data. A gateway, logging destination, or fallback provider that receives PHI needs a BAA that covers it, even when the primary model provider already has coverage. No AI product is inherently HIPAA compliant, and no vendor is “HIPAA certified”. Compliance belongs to the organization’s implementation and governance, so the contract and the controls must describe the same data path.
The consumer app trap and the enterprise BAA landscape in 2026
BAA coverage attaches to a product and deployment path. ChatGPT Enterprise offers a BAA, while ChatGPT Plus and Team do not. Those uncovered products, along with the uncovered Claude.ai consumer tier, must not receive PHI.
The enterprise landscape in 2026 offers several covered paths. Anthropic offers BAAs on paid contracts through its first-party API and Claude Enterprise, while Claude deployments through AWS Bedrock or Google Vertex AI rely on AWS or Google agreements respectively. OpenAI Enterprise direct and Azure OpenAI Service provide distinct paths, with the latter under the Microsoft BAA. Google Gemini is available through Gemini Enterprise Agent Platform, formerly Vertex AI, under the Google Cloud BAA. AWS Bedrock falls under the AWS BAA and hosts models including Claude, Llama, and Mistral.
DeepSeek’s hosted API does not offer a BAA. Its compliant deployment path is self-hosting inside a HIPAA-aligned VPC. Even among covered services, “HIPAA eligible” is not interchangeable with “HIPAA compliant”. Eligibility establishes that a provider will sign an agreement, while the team remains responsible for the surrounding safeguards.
What the BAA requirement actually touches beyond the model call itself
A covered inference call does not cover every artifact produced around it. Response caches, trace exports, and observability tools become PHI stores when they record patient information from prompts or completions. Each needs a BAA that covers it or must run inside the controlled environment. These artifacts are not merely engineering exhaust.
Embeddings made from PHI need the same protection as the data they come from. A vector store such as Pinecone, Weaviate, or pgvector on AWS RDS must therefore use a HIPAA-eligible configuration. Audit logs are also PHI when they hold patient data. A fallback route expands the boundary again, because a provider reached only during an outage still receives a disclosure.
Proposed rules would make that wider inventory more consequential. The HIPAA Security Rule NPRM published in December 2024 would eliminate the distinction between “addressable” and “required” safeguards, making encryption at rest, MFA, vulnerability scanning, and asset inventories unambiguously required for ePHI systems, including model artifacts and training data. HHS now targets July 2027 for finalization and classifies the proposal as a long-term action. It remains a proposal.
How can a gateway enforce BAA scope and redact sensitive data before it reaches a model?
The gateway is the common enforcement point between applications and providers. Without it, each application team implements authentication, redaction, model access, and logging independently, creating room for inconsistent controls. A shared gateway applies those policies before requests leave for a provider. Its value is architectural rather than cosmetic. It turns BAA scope into a routing constraint that application code cannot simply override.
HIPAA’s technical safeguards under 45 CFR 164.312 provide the control framework. Unique, scoped credentials support person or entity authentication, while access policies limit which endpoints each caller can reach. PHI and PII redaction on each request helps meet the minimum necessary standard before the prompt reaches the model. Records for each request help meet audit controls by showing caller identity, model selection, PHI detection, and timing. BAA-scoped allowlists, covered fallback chains, and zero-data-retention headers where supported translate contractual restrictions into runtime policy.
A hosted service that receives PHI is a business associate and needs a BAA. A self-hosted or in-VPC deployment keeps the vendor outside the data path when the vendor receives no PHI. A fully managed gateway that signs a BAA offers another route for teams that cannot operate the infrastructure themselves. In every case, the enforcement point and the agreement must cover the actual traffic.
How fallback logic becomes a compliance control in a PHI-handling gateway
An outage does not suspend the BAA requirement. A fallback chain built solely for availability can send PHI to an uncovered provider as soon as the primary endpoint fails. The covered entity remains responsible for that disclosure even when the routing was automatic.
The gateway must allow fallback only to another BAA-covered provider or a self-hosted model inside the covered environment. It must also record the routing decision in the audit log. Cost and quality constraints belong in the same policy, since an unplanned switch can change both the expense and the suitability of the response. Coverage is the prerequisite for considering those tradeoffs.
A model-specific interruption illustrates why endpoint policy needs this precision. When particular model versions become unavailable while others remain accessible, applications hardcoded to those IDs face a different problem from applications with configured fallback chains. Provider-agnostic routing makes switching possible, but does not itself make the destination permissible. Only a fallback chain restricted to covered endpoints addresses the PHI disclosure risk.
What the gateway must do with audit logs when those logs contain PHI
Prompt and completion logs containing patient data are PHI stores. Treating them as generic observability data leaves the system’s most easily overlooked copy of patient information outside its intended controls. The gateway must govern what enters those records and where they go. Log policy belongs at the request boundary. It should be configurable per key or workload so content restrictions follow the traffic being handled.
A useful per-request record captures timestamp, caller identity, model called, PHI detection outcome, action taken, and response metadata. The gateway must restrict logged fields and redact logged content according to policy, while exporting only to BAA-covered destinations. Splunk, Datadog, Sumo Logic, and Microsoft Sentinel are possible destinations under appropriate agreements.
A developer who can query raw patient prompts through an unrestricted dashboard has access to a PHI store regardless of the gateway’s other safeguards. RBAC must also control access at the log destination, where logs must be kept for at least the six years specified by OCR guidance. Complete, controlled, retained records make it possible to demonstrate that policies operated on actual requests.
The six technical safeguards a gateway must satisfy to support a defensible HIPAA posture
Six properties help teams check controls for PHI workloads under 45 CFR 164.312, with BAA scope and deployment as part of that check. The first is BAA coverage across the hosted data path. For a gateway vendor that receives PHI, willingness to sign is a binary filter. The second is deployment model. Self-hosting changes who receives data, while managed hosting places contractual obligations on another business associate and changes who operates the infrastructure.
The third property is PHI and PII detection on each request, with redaction before it goes to the provider. Its purpose is to reduce disclosure without depending on each developer to sanitize prompts correctly. The fourth is BAA-scoped provider access and fallback, enforced per key with routing decisions logged. Primary and alternate routes need the same boundary. Otherwise, an outage activates a disclosure path that ordinary request testing may never exercise.
The fifth property is attributable access through SSO, SCIM, RBAC, and per-app or per-user virtual keys. Shared provider API keys defeat attribution and make it impossible to identify the specific caller responsible for a disclosure. The sixth is governed request logging, including configurable content policy, covered export destinations, and enforced retention. Together, these properties assess whether a gateway can support a defensible implementation. They do not turn a product purchase into HIPAA compliance.
How available gateways compare on the criteria that actually matter for PHI
Bifrost by Maxim AI runs inside the organization’s infrastructure, including VM, Kubernetes, in-VPC, or air-gapped deployments, without sending PHI or telemetry to the vendor. Compliance depends on the team’s deployment and configuration. Kong AI Gateway supports a hybrid arrangement with a vendor-managed control plane and self-managed data plane, but a fully self-hosted control plane is not currently available for its AI Gateway entity model. The available material leaves Kong’s BAA status publicly unresolved, adding to distinctions that affect the data boundary, while self-managed infrastructure also assigns deployment, patching, scaling, and incident response to the team.
Azure API Management and Google Apigee fall under their respective cloud BAAs and fit teams already invested in Azure or GCP. Because these API management platforms are built for general use rather than as purpose-built LLM gateways, teams need to do more configuration for PHI redaction, model-specific controls, and model routing. Cloudflare AI Gateway offers a hosted alternative. Cloudflare enters into BAAs with Enterprise customers handling PHI, and AI Gateway is within its HIPAA attestation scope. There is no in-VPC option, and higher log volume requires a paid Workers plan, with Logpush export subject to a monthly cap before additional per-log charges.
Concentrate AI provides a fully managed gateway with a BAA available on the paid plan, requiring no proxy for the customer to deploy or maintain. Its agreement does not by itself establish coverage for every upstream route. Concentrate AI’s audit-log documentation distinguishes request traffic from administrative audit logs, so per-request evidence requires separate evaluation. Aptible LLM Gateway is another managed option, arising from Aptible’s digital health compliance work since 2013. Across these options, a gateway that permits fallback to an uncovered provider fails the routing criterion regardless of its other capabilities.
What a PHI-handling team needs to verify before sending the first request
Before launch, the team should check that coverage matches where the data actually goes, beyond what the architecture diagram shows. That means the gateway vendor, every reachable model provider including fallbacks, and every logging or observability destination. That check should also cover vector stores and caches that receive prompt or response content. Coverage must precede production traffic. A signed agreement for the primary provider leaves the remaining hops unresolved.
The team should then check that each provider has turned on zero data retention, even if the provider offers it. The provider agreement and, where possible, the provider’s SOC 2 Type II report should support that confirmation. Redaction also needs request-level evidence showing what was detected, what was removed, and that the removed content did not reach the provider. To test fallback, the team should send a test request to an intentionally unavailable provider and confirm that the alternate endpoint is covered and the routing decision is logged.
Finally, the team should check the log store that receives the records. Its BAA coverage, six-year retention policy, and RBAC limits on who can read raw prompts need to match the gateway’s policy. This connects the deployment back to OCR’s enforcement concern, namely whether safeguards operate on real PHI. The defensible evidence is an end-to-end covered path with controls that can be demonstrated in production.
