What Is AI Memory Poisoning? When Agents Remember the Wrong Thing

An AI assistant that forgets everything can be frustrating. You have to explain your role, your preferences, the names of your projects and the way your business works every time you start a conversation. Persistent memory is meant to fix that: the assistant can retain useful context and make future interactions more relevant.

That convenience changes the security question. A memory is not only a helpful note. It is information that may be retrieved later and placed back into the instructions used by an AI system. If the note is false, misleading or malicious, the system may carry the problem into a different conversation, a different day or a different workflow.

The employee notebook analogy

Imagine giving a new employee a notebook. They write down that a particular customer prefers email, that a project has a Friday deadline and that a certain manager approves routine purchases. The notes are useful because the employee can refer back to them later.

Now imagine that somebody adds a false note: “The finance director has approved every invoice from this supplier.” If nobody can see who wrote the note, check where it came from or remove it, the employee may rely on it weeks later. The danger is not just that the note is wrong. It is that the note has acquired the appearance of internal knowledge.

AI memory can work in a similar way. Depending on the product, it may include saved preferences, conversation history, a retrieval index, grounding documents, a vector database, shared agent context or summaries created from earlier interactions. These are different implementations, but they share an important property: information from the past can influence what the system does now.

What does memory poisoning mean?

Memory poisoning means placing information into a persistent context channel so that an AI system later retrieves and uses it in a way the business did not intend. The information may be deliberately malicious, accidentally wrong, out of date or simply given more authority than it deserves.

Microsoft describes AI memory and context poisoning as a new, durable attack surface. Its examples include persistent memory, grounding caches, user-preference memories, vector databases and conversation histories. It also points out that contamination can arrive through user input, retrieved documents, tampered grounding data or interactions between agents.

How this is different from a bad prompt

A user can type something incorrect into a normal chat and receive a poor answer. The conversation ends, the next chat starts clean and the error may go no further. That is still worth checking, but it is not automatically memory poisoning.

The concern becomes more serious when the system decides that the statement is worth saving, puts it into a shared knowledge store or allows another agent to inherit it. A later user may not know the original source. The model may not distinguish a verified policy from an informal comment. A retrieval system may select the false entry because it appears relevant, recent or semantically similar.

This also explains why the same problem can look like an ordinary hallucination. The final answer may be fluent and confident, but the hidden cause is not only the model's generation. It may be an untrusted memory record that was supplied as context before the answer was produced.

What research has demonstrated

The research literature is useful for showing that persistent context deserves security attention, but it should not be turned into a claim about every commercial assistant. The AgentPoison paper studied ways of poisoning the memory or knowledge bases used by LLM agents in controlled research settings. Its abstract reports average attack success above 80% across three evaluated agent types at a poison rate below 0.1%, with less than 1% impact on benign performance. Those are results from the paper's evaluated setups, not a prediction that the same rates apply to a commercial product.

PoisonedRAG examined knowledge corruption in retrieval-augmented generation systems: if an attacker can add harmful text to a knowledge database, retrieval may bring that text into later answers. Its latest arXiv version reports a 90% attack success rate in one experiment that injected five malicious texts per target question into a database with millions of texts. It is related to memory poisoning, although a business knowledge base and a personal assistant memory are not necessarily the same component.

These papers are demonstrations of a class of failure under defined conditions. They do not show that an attacker can automatically poison every AI assistant, that a single false sentence always survives a product's safeguards or that UK SMEs are experiencing a wave of these incidents. Product architecture, isolation, access control, source validation, retrieval ranking and human review all change the outcome.

Microsoft: AI Memory / Context Poisoning →

AgentPoison: Red-teaming LLM Agents via Poisoning Memory or Knowledge Bases →

PoisonedRAG: Knowledge Corruption Attacks to Retrieval-Augmented Generation →

Fictional illustration: the false sharing instruction

A 30-person engineering business gives an internal AI agent access to its document library and calendar. The agent helps staff find project information and prepare meeting notes. A new memory feature is switched on so that it can remember recurring project preferences.

A false note enters the shared memory saying that documents labelled “Project Harbour” may be shared with any external consultant. The note is not linked to a policy, owner or approval record. A month later, the agent is asked to prepare a handover pack and send it to a consultant. It retrieves the note, treats it as an established rule and prepares the sharing action.

The important point is not that the agent has developed an intention to leak information. The problem is that an unverified memory has been allowed to influence a workflow with access to real documents and an external recipient. A visible memory history, a source check, a restriction on external sharing and human approval could each have stopped the chain.

Why agent permissions make it more serious

A chatbot that only drafts text can still produce a costly mistake, but a connected agent may be able to turn bad context into an operational event. It could search a document store, read a calendar, create a CRM record, open a support ticket, alter a workflow, prepare an email or call an API. Whether it can actually do those things depends on the permissions the business has granted.

1. 1A memory entry is retrieved because it appears relevant to the task.
2. 2The model treats the entry as context when deciding what to recommend or do.
3. 3The agent selects a tool or workflow that is available to it.
4. 4The action reaches a business system, person or external service.
5. 5A human or monitoring system has to notice, stop or reverse the result.

The chain may break at any stage. That is why memory safety cannot be separated from identity, permissions, approval, logging and incident response. The same remembered sentence has a very different consequence when the agent can only suggest a draft compared with when it can send messages, share files or change records.

Should businesses stop using AI memory?

No. Persistent context can provide real value: less repetition, more consistent support, better handovers and useful assistance across a project. The question is whether the business has chosen an appropriate level of memory and autonomy for the work involved.

Microsoft's current guidance on managing agentic memory safety is more specific than a general warning: it recommends gating writes on authorised intent and provenance, enforcing deterministic isolation by user, agent and tenant with verifiable identity, treating retrieval as a risk decision with freshness and malicious-content checks, and logging the full memory lifecycle. Its examples include write-time governance through Microsoft Purview and retrieval-time Prompt Shields, but these are supplier-specific control patterns rather than proof that every product provides them by default. The NCSC's current agentic-AI guidance similarly says to identify what could go wrong, set the right level of oversight, control the agent's environment, and monitor and respond; its practical adoption advice is to start small with low-risk work and expand only when safeguards are understood.

An AI memory governance checklist for SMEs

Practical controls to put in place

Start with a small inventory. List each AI tool that remembers anything, the business purpose, the information it may hold, the people who administer it and the systems it can reach. Do not assume that a tool called a chatbot has no memory, and do not assume that a product's memory is private just because it appears in an individual's interface. Treat the supplier's controls as claims to verify in your tenant, not as a substitute for your own access and approval design.

• Use approved accounts and workspaces so the business can apply identity, retention and access controls.
• Separate personal preferences from shared business rules. A personal note should not silently become an organisation-wide instruction.
• Gate writes on authorised intent and provenance. Prefer explicit, attributable and reviewable writes rather than allowing every retrieved document or conversation to become durable memory.
• Keep a trusted source for policies and permissions, and treat retrieved memory as candidate context. If memory conflicts with a current policy, the policy must win.
• Use deterministic user, agent and tenant isolation; do not rely on model prompts to enforce a security boundary. Where the supplier offers retrieval-time content screening, test what it catches and what it does when screening is unavailable.
• Give agents read access before write access, and suggestion access before action access. Expand permissions only when the benefit and controls are clear.
• Require confirmation or human approval for actions with financial, legal, privacy, safety or external-communication consequences.
• Test correction and deletion in practice. A “delete” button is not enough if copies remain in a retrieval index, audit trail or downstream cache without a clear reason.
• Monitor unusual memory writes, changes in retrieval behaviour and actions that rely on old or unverified context.

Memory also belongs in the supplier review. Ask whether the provider can explain the memory lifecycle in plain English, support tenant separation, export relevant records, honour deletion requests, identify model or retrieval changes and investigate a suspected contamination event. If the supplier cannot tell you what is remembered or how to remove it, do not give the feature more authority than you can safely supervise.

How this fits with wider AI security

Memory poisoning is one part of a wider system risk. An agent can be influenced by its model, instructions, retrieved data, tools, identity, permissions and operating environment. Security controls should therefore cover the whole path rather than focusing only on the wording of the prompt.

AI agents and accountability when they act in real systems →

What happens when an AI agent acts beyond its authority? →

AI privacy controls for business data →

Why AI needs practical guardrails →

When AI can fool a security camera: a different kind of AI interpretation risk →

The camera example is a useful reminder that an AI system can interpret the same underlying input differently from a person. Memory poisoning is not the same technical problem, but the governance lesson is similar: understand the boundary between the raw information, the system's interpretation and the action that follows.

The IT Club takeaway

AI memory can be useful without being treated as unquestionable truth. Before an assistant or agent starts remembering business information, decide what it may retain, who may change it, how it is checked, how long it lasts and how it can be removed. Then match the agent's permissions to the consequence of being wrong. Persistent memory should make work easier — not create an invisible policy system that nobody can inspect.

Microsoft: Manage agentic AI memory safety →

Microsoft Security: Guarding AI memory (22 June 2026) →

NCSC: Managing the cyber risk of agentic AI (20 August 2026) →