Context Leakage: How LLMs Fall Short on Patient Privacy

A 32-year-old patient logged into a clinic portal on a Friday morning to download a routine visit summary for an ear infection. The top of the page looked normal: an encounter summary dated October 9, 2026, from Dr. Marcus Vance, MD, listing a diagnosis of acute otitis media, right ear (H66.91), a prescription for Amoxicillin 875mg PO BID x 10 days, and a follow-up instruction to return in 14 days if symptoms did not resolve.

Directly beneath that treatment plan, separated by a single horizontal rule, was a second block of text the patient had never seen before and could not explain. It described persistent suicidal ideation over the past three weeks following a recent divorce, a history of bipolar I disorder with psychotic features, non-adherence with Lithium titration, and acute auditory hallucinations during a morning assessment. It recommended an immediate psychiatric evaluation and noted a safety plan contracted with social work.

None of it belonged to the person reading the screen. The evaluation belonged to a different patient who had completed a behavioral health intake in the adjacent exam room twenty minutes earlier. Within two hours, the health system was dealing with an emergency data breach declaration, mandatory reporting to the HHS Office for Civil Rights (OCR), and an active HIPAA Title II security investigation.

The cause was not an external SQL injection, a stolen database credential, or a targeted ransomware attack. It was an un-sandboxed memory leak inside an autonomous multi-tenant LLM orchestration framework.

How a Warm Worker Pool Blends Two Patients Together

The leak traces back to cost-optimization patterns that are common in agentic frameworks. To avoid cold-start latency, which can add 1,500ms to 3,000ms per turn, engineering teams often keep a pool of long-lived inference workers. These workers hold “warm” context windows or shared Key-Value (KV) caches across sequential API calls.

When a worker finishes handling one patient and immediately picks up the next patient’s queue without an operating-system-level process termination and memory zeroing, residual tokens remain accessible in unallocated buffer space. In this case, the psychiatric intake tokens from the first patient were still sitting in the shared memory segment when the ear infection request arrived.

Three compounding factors turned that residue into a breach.

KV-cache reuse and warm process pooling. The same pooled process served both sessions. No kernel flush occurred between them.

The semantic retrieval trap. Agents equipped with local vector scratchpads or retrieval buffers to remember multi-turn context will pull semantically dense clinical tokens from previous turns when queries lack rigid metadata segregation. An attention head optimizing for conversational completeness grabbed salient clinical phrases floating in the warm buffer and wove them into the current generation.

LLMs have no tenancy primitives. A large language model is an autoregressive token predictor. It calculates the conditional probability P(wₜ | w<ₜ). It has no native concept of operating system process boundaries, role-based access control (RBAC), cryptographic tenant keys, or legal data isolation mandates under federal law. If a token exists inside the model’s active attention field, the model treats it as available context.

The result was a synthesized patient summary with an ear infection plan on top and a psychiatric evaluation underneath, produced in a single generation.

Why Prompt-Based Privacy Rules Cannot Fix This

When engineering teams find data bleed in staging, the usual first response is to add negative constraints to the system prompt. A typical version reads: “You are an AI clinical documentation assistant. You must adhere strictly to HIPAA guidelines. Under no circumstances should you ever reveal or include private health information (PHI) from any previous patient, conversation, or session in the current output. Only output data relevant to the current patient.”

Relying on system prompts to enforce data boundaries is an information security anti-pattern for three reasons.

Soft probabilistic bias versus hard kernel isolation. A system prompt is merely a set of input tokens that subtly alters attention distributions. It cannot close network sockets, flush CPU caches, or isolate virtual memory pages.

Context window degradation. In long conversational sessions with complex clinical notes, prompt constraints experience attention decay. The model’s internal weights prioritize the most immediate, concrete tokens over high-level behavioral rules.

No auditability. When a model fails to obey a prompt constraint, it leaves no system-level audit trail. You cannot prove to a regulatory body why an attention head weighted a prohibited token at 0.78 instead of 0.02.

The Zero-Trust Enclave Architecture That Enforces HIPAA

In enterprise clinical environments, patient privacy cannot be managed through conversational instructions. Data segregation must be enforced at the infrastructure and transport layer. At Claire, the approach is deterministic runtime segregation: inference never occurs in shared, persistent environments.

The architecture works in four stages.

Cryptographic tenant tokens on every ingestion layer. Data cannot enter an agent pipeline as untyped natural language. Every clinical payload must be encapsulated within a signed cryptographic envelope that binds the request to a unique encounter identifier. A Pydantic model can enforce this with an HMAC check comparing a submitted tenant_hmac against a locally computed digest of patient_id and encounter_id using hmac.compare_digest.

Invariant runtime gateway. An out-of-band auth assertion validates the cryptographic JWT and patient tenant key, enforcing the invariant UserSession.TenantID == Payload.PatientID. Only then does the gateway provision an ephemeral sandbox.

Micro-enclave ephemeral execution. Inference workers must not be persistent multi-turn daemons handling serial requests across different patients. Every operational turn executes inside an isolated micro-container, such as AWS Firecracker or Google gVisor. Memory isolation means the worker process allocates a dedicated, private memory space and is barred from sharing memory pointers, vector scratchpads, or global variables with any sibling process. No shared KV-cache is permitted at the CUDA kernel or driver level.

Deterministic memory incineration. The moment the generation loop completes and emits its structured JSON output, allocated heap pages are explicitly overwritten with zeroes using low-level memory clearing primitives such as memset_s or bzero. The micro-enclave is immediately terminated and torn down by the supervisor daemon. The gateway logs an immutable audit event confirming that the session execution context was incinerated, recording the timestamp, encounter ID, and cryptographic hash of the purge operation.

The net result is zero residual context.

What This Means for Engineering Leads

If you are deploying autonomous agents into healthcare, life sciences, or legal environments, four production axioms follow from this incident.

System prompts are not access controls. If a model has physical access to data in its context window or warm cache, assume that data is already public.

Ephemeral execution is mandatory. Never reuse persistent worker processes or unsegmented KV-caches across distinct user sessions. Run single-session sandboxes and destroy them immediately after turn execution.

Isolate data outside the model. Tenancy boundaries, authorization checks, and data redaction must occur deterministically at the network and transport layer before tokens ever reach an LLM.

Healthcare data requires cryptographic, zero-trust infrastructure. Conversational politeness is not a substitute for patient record protection.

The breach described here did not require an attacker. It required only a shared inference worker, a warm buffer, and a queue that moved from one patient to the next. The remediation is not a better prompt. It is a process that never lets the two patients share memory in the first place.