When a team starts using AI for real work, someone eventually asks the question that’s on everyone’s mind: if we paste a confidential document into this thing, does the model learn it? Could bits of it show up in someone else’s chat next week?

The short answer is no. Once a model finishes training, it doesn’t learn from the prompts you send it, and it doesn’t remember you after it replies. Everything that feels like memory in a modern AI tool comes from the software built around the model: databases, search indexes, and logs that work the same way they have for decades. Once you separate the model from the system around it, the privacy risks get much easier to understand and mitigate.

How a model answers a question

Stripped of the mystique, a large language model is a static file of numbers called weights. Those weights are produced during training, long before you ever type a prompt, and once training finishes they are frozen. When you send a message, the model reads your text, calculates a reply token by token (think: word by word), and stops. Nothing in that calculation writes back to the weights. The model that answers your second question is bit-for-bit identical to the one that answered your first, and also identical to the copy answering a stranger’s question five minutes or five weeks later.

In software terms, running a model is stateless. Every call starts from scratch.

If the model has no memory of its own, how does a chat assistant follow a twenty-minute conversation, or seem to remember you across sessions? Because the application around it keeps notes and feeds them back in on every turn. Each time you hit send, the app bundles up any saved context about you along with the entire conversation so far, and ships the whole package to the model as a single fresh request. It’s like working with a sharp consultant who has complete short-term amnesia: every time they step into the room, someone hands them the file, they read it from page one, add their reply at the bottom, and walk back out. Their expertise stays in their head, but the file stays in the room.

Where the memory lives

Of course, that file still has to sit somewhere between turns. That job belongs to the surrounding application — the databases, caches, and services wrapped around the model. When data lingers after a model interaction, this is where it lives:

  • Conversation history. The threads in your sidebar are ordinary database rows on the provider’s servers, kept until you delete them or a retention rule clears them out.
  • Saved “memories.” When an assistant remembers that you work in healthcare or prefer metric units, the model hasn’t learned who you are. The app simply saved a short text note on your profile and quietly tacks it onto the top of every new chat.
  • Search and retrieval indexes. Tools that answer questions about your internal documents store copies of those files, along with numerical representations called embeddings, in a database so relevant passages can be pulled into future prompts.
  • Prompt caches. To avoid recalculating the same long document on every follow-up question, providers often cache the intermediate math from a prompt in memory for a few minutes to an hour, scoped to your account.
  • Abuse and application logs. Providers sometimes retain prompts for a window to review safety flags, and the application sitting in front of the model usually keeps its own request logs and analytics — occasionally in a third-party observability tool that never came up during the privacy review.

None of this is new territory. It’s rows in a database, files in an index, and lines in a log — things we already know how to govern with encryption, access controls, data residency, and short retention windows.

When prompts do become training data

There is one genuine way your words can end up inside a model: if the vendor saves your conversations and feeds them into the training run for a future model. Training is the one phase that modifies weights, and models do occasionally memorize verbatim passages from their training data — researchers have been able to coax real phone numbers, code, and paragraphs of text out of production models for years.

Even then, nothing you type today leaks into the model answering people this afternoon. The risk is that under permissive terms, today’s prompt becomes part of next year’s training set.

Whether that happens comes down to paperwork rather than architecture. Free and consumer-tier chat accounts often allow the vendor to train on conversations unless you opt out. Commercial API accounts and business tiers from the major labs generally exclude customer prompts from training by default, and enterprise agreements can go further with zero-retention endpoints that process a request in memory and drop it immediately. The catch in most workplaces is simpler than any of that: the policy on paper says one thing, while staff are quietly pasting work into personal accounts on the free tier.

Running the model yourself

Because a trained model is just a static file of weights and a piece of software that does matrix math, there is no technical reason it has to live in a vendor’s cloud at all.

Over the last few years, open-weights releases — Llama, Mistral, Qwen, Gemma, and OpenAI’s gpt-oss among them — have put strong models within reach of ordinary hardware. A capable small model runs well on a modern laptop or a single workstation GPU; a larger one fits on a modest server in your own rack or virtual private cloud. When the weights sit on machines you control, prompts never leave your network.

The trade-off is that the largest proprietary “frontier” models still hold a clear lead on hard reasoning, nuanced writing, and complex code, and most teams would rather not babysit physical servers if they don’t have to. In practice, you rarely have to choose one or the other across the board.

Five ways to draw the boundary

Depending on what a system handles, there are five practical ways to set it up, ordered roughly from easiest to most isolated:

  1. Direct API under commercial or enterprise terms. Prompts go to the model vendor, covered by a contract that forbids training on your data and caps or eliminates retention. You trust the provider to adhere to its own terms. For low-to-moderate sensitivity work, this is usually enough.
  2. Proprietary models hosted in your own cloud account. AWS Bedrock, Azure, and Google Cloud Vertex all let you call major commercial models inside your own tenancy, pinned to a region you choose (including Canadian regions), under your own IAM roles and audit logs. The same tenant-and-landlord boundaries from an earlier note apply here.
  3. Sensitivity routing. A small model running on your own hardware inspects each request first. Everyday tasks go out to a frontier API; anything touching patient charts, personnel files, or privileged legal work is routed to a local model that never phones home.
  4. A local masking proxy. Before a prompt leaves your network, a local pass strips out names and identifiers and swaps in placeholders. The external model works on the scrubbed text, and your proxy puts the real names back on the way in.
  5. Self-hosted open weights. The model runs entirely on hardware you own or lease, with no external API calls at all — trading the top end of model capability and some operational upkeep for complete control of the boundary.

A sixth option is starting to take shape alongside these: running cloud models inside hardware-encrypted enclaves so that not even the host operator can inspect prompts in memory, verified through cryptographic attestation. Apple’s Private Cloud Compute works this way, and the major cloud and GPU vendors are rolling out similar primitives for enterprise workloads.

Inside a masking proxy

Option four deserves a closer look, because it solves a problem teams run into constantly: wanting a frontier model’s writing or reasoning skill on text that contains personal identifiers.

Take a clinic that wants help turning rough visit notes into a clean specialist referral letter. A local masking proxy handles the request in five steps:

  1. The clinician’s notes — patient name, health card number, date of birth, clinical history — stay inside the clinic’s own environment and hit the proxy first.
  2. Pattern-matching rules catch structured fields like health card numbers, phone numbers, and postal codes. A small local model reads the prose to catch the messier identifiers that rules miss: names, employers, relatives, or a town mentioned mid-sentence.
  3. Each sensitive span is replaced with a consistent token like [PATIENT_1] or [HOSPITAL_1], and the lookup table stays in local memory for the life of the request.
  4. Only the scrubbed notes go out to the frontier model, which drafts the referral letter around those placeholders without ever seeing who the patient is.
  5. When the draft comes back, the proxy swaps the real details back into place, drops the lookup table, and returns the finished letter.

Because the external model is stateless, the vendor only ever sees a single de-identified draft — and under zero-retention terms, doesn’t keep even that.

What masking and local models don’t solve

Patterns like this are useful, but they are also easy to oversell if you treat them as a silver bullet. Five caveats matter in practice:

Scrubbing names isn’t the same as anonymizing. Latanya Sweeney famously demonstrated that five-digit ZIP code, birth date, and sex are enough to uniquely identify most Americans. A rare diagnosis in a small town or an unusual career history can point straight to one person even after every proper noun is stripped out. A good proxy generalizes as well as redacts — swapping a birth date for an age band, or a street address for a region — and still shouldn’t be treated as mathematically anonymous.

Sometimes the subject matter itself is the secret. If you are summarizing an unannounced acquisition, a patent draft, or a litigation strategy, replacing the company names with [COMPANY_A] and [COMPANY_B] doesn’t make the text safe to send over the wire. When the substance of the document is what’s confidential, it belongs in your own cloud tenancy or on your own hardware.

Redaction models miss things. Rules catch formatted numbers reliably, but the local model hunting for names and context in free text is probabilistic; sooner or later, an unusual spelling or an indirect reference will slip past it. Masking cuts exposure dramatically, which is valuable as a layer of defence, not a licence to ignore the terms on the downstream API.

Embeddings aren’t just abstract numbers. Teams often treat a vector database as harmless because it only stores arrays of floating-point numbers. In practice, researchers have shown that embeddings can be inverted to recover a surprising amount of the source phrases. A retrieval index needs the same access controls and encryption as the documents that fed it.

Running a model locally doesn’t clean up the rest of the stack. A self-hosted model still sits inside an application with chat databases, request logs, and backups — except now your team is the one responsible for locking them down and expiring old records.

Questions worth asking before you ship

Once you set aside the worry that a live model is quietly absorbing everything you type, evaluating an AI tool looks a lot like evaluating any other piece of software. Do our contract terms forbid training on our prompts? How long does the vendor hold request logs and cached context, and in which jurisdiction? What does the application layer store in its own database, its vector index, and its monitoring tools? And for the workloads where even a zero-retention external call is too much exposure, which parts can we mask or run on weights we host ourselves?

The model itself forgets the moment it finishes replying. The real work is making sure the system around it only remembers what you intended it to keep.