HIPAA Compliant LLM API: What to Demand From a Vendor

Most LLM vendor evaluations I see stop at one question: do you sign a BAA? It is a good filter and a low bar. HIPAA is more specific than that, and the specifics are what your security reviewer will actually read.

This is the checklist I would bring to a procurement conversation. It is drawn from the regulation itself, not from vendor marketing.

What you are actually buying

When you send PHI to an inference API, the vendor becomes a business associate under HIPAA. The regulation defines that broadly: a person who creates, receives, maintains, or transmits protected health information to perform a function on your behalf - explicitly including "data analysis, processing or administration."

That definition covers an inference provider. It also covers the provider's subcontractors, which is where chains get long.

Two consequences follow. Your vendor is directly liable under the HIPAA Rules for unauthorized uses and disclosures, not merely liable through your contract. And you need a written agreement in place before the first request carries PHI, not after.

The contract terms HIPAA requires

The regulation at 45 CFR 164.504(e)(2) states what a business associate contract must contain. In plain terms, it has to:

• Establish the permitted and required uses and disclosures of PHI by the vendor, and not authorize anything you could not do yourself.
• Require the vendor to not use or further disclose the information except as the contract allows or the law requires.
• Require appropriate safeguards, including compliance with the Security Rule for electronic PHI.
• Require breach reporting to you, including breaches of unsecured PHI.
• Require the same restrictions to flow down to subcontractors.
• Require the vendor to support individual rights - access, amendment, and an accounting of disclosures.
• Require the vendor to make its internal practices available to HHS for compliance review.
• Require return or destruction of PHI at termination, or, if that is not feasible, to extend the protections and limit further use.
• Authorize you to terminate the contract if the vendor violates a material term.

If a vendor's agreement is missing the last two, that is not a nitpick. The termination clause is your protection and the destruction clause is your exit.

The technical safeguards to verify

The Security Rule's technical safeguards live at 45 CFR 164.312. There are five standards, and it is worth knowing which implementation specifications are Required and which are Addressable:

"Addressable" does not mean optional. It means the vendor must assess whether the measure is reasonable and appropriate, implement it if so, or document why not and implement an equivalent alternative. A vendor that cannot explain its addressable decisions has not done the analysis.

The checklist

Bring these to the call. The pattern of answers tells you more than any individual one.

On the agreement

• Will you sign a BAA, and can I see the actual text before we commit?
• Does it name a retention posture, or only describe one in a policy?
• Which subprocessors touch the data, and will you flow the same terms down?
• What happens to PHI at termination - return, destroy, on what timeline?

On the data path

• Which endpoints and features are in scope, and which are excluded?
• What is the default retention if we configure nothing?
• Is your zero-retention configuration contractual or a dashboard setting?
• Do you train on customer data, and is that in the agreement?

On the controls

• How is access controlled, and how are personnel identified?
• What is logged, for how long, and can we export it?
• Is encryption applied in transit and at rest, and with what key management?
• What can you show me for a security review - a SOC 2 report, a pen test summary, a subprocessor list?

On operations

• Who is our named security contact, and what is the breach notification path?
• What is the SLA, and what happens during a provider outage?
• Can we get a data-processing addendum covering residency if we need it?

The failure modes to watch for

Three patterns predict trouble.

The compliance page that is not a contract. "HIPAA compliant" on a marketing site, with no executed agreement available. This is common and it is the reason the phrase has a bad reputation among engineers.

The BAA that covers the platform but not the model. Some agreements cover infrastructure while excluding the inference service itself, or exclude specific endpoints and beta features. Ask which endpoints are in scope and write down the answer.

The retention answer that changes. A support thread saying "we don't retain your data" is not a commitment. If it is not in the agreement, assume it can change with a product update.

A note on what a BAA does not do

Signing one does not make you compliant. OpenAI states this about its own agreement: "Accepting a BAA and enabling HIPAA compliance support do not, by themselves, make your application HIPAA compliant." That is true of every vendor, including us.

A BAA is the permission to start. Your safeguards, your access policies, and your own audit trail are the rest of the program. The HIPAA compliance guide walks through the full control set, and Zero Data Retention covers the retention question in detail.

OpenMed Router signs a BAA covering inference across the open-weight model catalog, with zero-retention isolated inference and AES-256 encryption in transit and at rest.

Evaluating vendors right now? Join the waitlist.