COMPEL Glossary / memory-poisoning
Memory Poisoning
Memory poisoning is an attack targeting AI agents with persistent memory, where an adversary manipulates what the agent remembers to permanently alter its behavior across future sessions.
What this means in practice
Unlike prompt injection (which affects a single conversation), memory poisoning persists because the corrupted information is stored in the agent's long-term memory system and retrieved to influence all subsequent interactions. For example, an adversary might manipulate interactions to make the agent 'remember' a false policy that leads to unauthorized actions in future sessions. Memory poisoning is extremely difficult to detect because the agent behaves consistently with its (corrupted) memories. Defenses include memory validation, anomaly detection in memory patterns, periodic memory audits, and architectural controls that separate critical operational instructions from experience-derived memories. In the COMPEL Agent Governance layer, memory hygiene practices are a required governance dimension.
Why it matters
Memory poisoning is an attack targeting AI agents with persistent memory, corrupting stored information to permanently alter agent behavior across all future sessions. Unlike prompt injection affecting a single conversation, memory poisoning persists indefinitely and is extremely difficult to detect because the agent behaves consistently with its corrupted memories. Organizations deploying agents with learning capabilities face a unique security risk that traditional cybersecurity practices do not address.
How COMPEL uses it
In the COMPEL Agent Governance layer, memory hygiene practices are a required governance dimension. During Model, memory validation and anomaly detection controls are designed for agents with persistent memory. The Produce stage implements architectural controls separating critical operational instructions from experience-derived memories. The Evaluate stage conducts periodic memory audits to detect corruption, and incident response procedures cover memory poisoning scenarios.
Related Terms
Other glossary terms mentioned in this entry's definition and context.