Compliance
Zero Data Retention for AI APIs: What It Really Means
Zero data retention is the phrase healthcare buyers want confirmed in writing. But what does it cover, and what is left out?
TL;DR
Zero data retention means a provider does not store your prompts and responses after the response is returned. It is not a substitute for a BAA, it usually excludes some endpoints, and flagged content is often retained for safety review regardless.
Ask any healthcare team what they want from an inference vendor and "zero data retention" will be in the first three answers. It is the phrase buyers want confirmed in writing, and it is also the phrase most often misunderstood.
This post is about what ZDR actually is, where it stops, and how the major providers implement it differently - which matters most when you are the one signing the contract.
What zero data retention means
Stripped of marketing, ZDR is a storage commitment. The provider agrees not to write your prompts and responses to durable storage after the request completes.
Anthropic states it plainly for its own API: under a ZDR arrangement, "Anthropic does not store customer prompts or responses at rest after the API response is returned."
That is pretty handy! It reduces your attack surface, shrinks the blast radius of a breach, and makes a data-retention policy easier to defend in a security review.
What it does not mean
Each of the following points have caught out folks we speak to:
It is not a BAA. Zero data retention is a data-handling property. A BAA is a legal instrument that makes a vendor a business associate under HIPAA. A vendor can offer excellent ZDR and still be unable to carry PHI, because no agreement exists. OpenRouter is the clearest example: real ZDR controls, no BAA.
It does not cover every endpoint. OpenAI notes that some endpoints "may
still store application state, even if Zero Data Retention is enabled." Their
ZDR also forces store=false on /v1/responses and /v1/chat/completions
rather than relying on the caller. Anthropic's ZDR explicitly excludes the
Claude Console, consumer plans, and the Claude Teams and Enterprise product
interfaces.
It may not stop safety retention. This is the one people miss. Anthropic states that even with ZDR enabled, if a session is flagged for a policy violation, "Anthropic may retain the associated inputs and outputs for up to 2 years." ZDR means no routine retention; it does not mean no retention, full stop.
It does not automatically mean no training. Retention and training are separate controls. Most enterprise providers commit to neither training on nor retaining your data, but they are documented separately and should be confirmed separately.
How three providers implement it
The differences matter more than the label.
| Provider | How ZDR is configured | Notable carve-outs |
|---|---|---|
| OpenAI | Requires prior approval; excludes customer content from abuse logs and forces store=false on the responses and chat endpoints | Some endpoints may still store application state; regional and FedRAMP endpoints carry a 10% price uplift |
| Anthropic | Enabled per organization by the account team; no prompts or responses stored at rest after the response | Console, consumer plans, and Team/Enterprise interfaces excluded; flagged sessions retained up to 2 years; some models require 30-day retention and are unavailable under ZDR |
| OpenRouter | Enforceable globally, per model group, per guardrail, or per request | Provider-dependent; some endpoints "do not train on your data but do retain it, for example to scan for abuse or for legal reasons" |
The decision you are actually making
ZDR and a BAA answer different questions, and you need both:
- ZDR answers "can this data leak from the vendor's storage?" It reduces the probability and the impact of a breach.
- A BAA answers "am I legally permitted to send this data at all?" It is the precondition, not the optimization.
Buying ZDR without a BAA is like locking a door on a house you do not own. Useful, but not the thing that lets you move in.
What to ask before you sign
- Which endpoints are covered by ZDR, and which are excluded? Get the list. The exclusions are where the risk lives.
- What happens to flagged content? Ask about safety review retention separately from operational retention.
- Is ZDR contractual or a setting? A dashboard toggle can change. A contract term is harder to move.
- Does your BAA reference the same scope as your ZDR? A mismatch is common and it is a finding in a security review.
- What is the default if we do nothing? You want to know the baseline, not just the best case.
What a defensible retention posture looks like
If you are assembling this for a security review, the pieces that hold up are:
- A signed agreement that names the retention commitment, rather than a policy page that describes it.
- A documented default, so you know what happens when nobody configures anything.
- Audit logs that record access without recording the content itself.
- Encryption in transit and at rest, which is a control entirely separate from retention.
- A named internal owner for the retention configuration, because settings drift as teams change.
None of these are exotic. Problems arise when one of them was assumed rather than verified.
The failure mode to avoid
The most common mistake is treating ZDR as a compliance program. It is one control among several. A defensible clinical deployment needs the executed agreement, the retention posture, encryption in transit and at rest, access controls, and audit logging - the technical safeguards in 45 CFR 164.312, in other words.
ZDR is the part vendors like to talk about because it is easy to state. It is not the part that makes you compliant.
OpenMed Router provides zero-retention isolated inference across the open-weight catalog, under a signed BAA, with AES-256 encryption in transit and at rest. The HIPAA compliance guide covers the rest of the control set.
Want the retention posture and the agreement in one place? Join the waitlist.
Chris Williams, MD
Chris Williams, MD is a physician, clinical AI researcher and the co-founder of OpenMed Router, working to make open source AI models safely accessible to healthcare organizations under HIPAA. He writes about clinical AI, model selection, compliance, and the practical adoption of open source inference in clinical and operational workflows.
Join the waitlist