Context Engineering: Memory for AI Agents

Large language models are stateless, so an agent's apparent memory exists only because developers decide what to store, when to load it, and when to discard it. Context engineering treats the finite context window as a scarce resource and designs retrieval, memory, tools and prompts around it. Agent memory is the external storage that survives session boundaries, while context engineering selects what gets loaded from that storage. Sources in this brief describe working, episodic, semantic, procedural, entity and workflow memory, plus operations such as writing, selecting, compressing and isolating context. This brief covers the core concepts, a memory pipeline, common failure modes, and design guidance for developers and AI agents building memory-aware systems.

Key concepts

Context engineering

The discipline of designing the architecture that feeds an LLM the right information at the right time, covering retrieval, memory, tools and prompts, so the model spends attention on high-signal tokens. S2 · weaviate.ioS3 · mem0.ai

Context window

The maximum input a model can consider at once, measured in tokens. It includes inputs, outputs, tool calls and retrieved documents, and acts as the model's short-term memory. S2 · weaviate.io

Agent memory

External storage that survives session boundaries, such as files, databases and vector stores. Context engineering then selects what is loaded from it into the next session's window. S8 · augmentcode.com

Short-term versus long-term memory

Short-term memory persists within a session: conversation history, temporary variables, workflow state. Long-term memory is stored outside the window, such as user profiles and project context, for retrieval across sessions. S1 · medium.comS2 · weaviate.io

Forgetting and staleness

Forgetting is designed in as a feature, using TTLs, decay, freshness metadata and staleness detection, so agents do not act on facts that have stopped being true. S5 · hidekazu-konishi.com

Context failures

Common failures that derail agents: poisoning, distraction, confusion and clash. Each has mitigation techniques within context engineering. S6 · microsoft.github.io

A memory pipeline for agents

1. Capture informationCollect inputs worth remembering from interactions, such as user input, system instructions, and retrieved documents, as the raw context sources.
2. Encode itStructure information before it enters the model: plain text or JSON schemas, embeddings or symbolic forms, summaries or verbatim content.
3. Store itPersist information outside the context window, in stores such as vector databases or relational and JSON databases, so it survives across sessions.
4. Organize itStructure stored memory by type, such as working, episodic, semantic, procedural, entity and workflow memory, and apply summarization, compaction and indexing.
5. Retrieve what is relevantPull only relevant items into the window at each step, using relevance scoring, recency weighting and importance thresholds.

Why agents need engineered memory

Large language models are stateless: every invocation starts from zero, knows only what sits in its context window, and forgets everything when the call returns. By default an LLM has only short-term memory, the current context window, which holds instructions, inputs, outputs, tool calls and retrieved documents. Everything an agent remembers exists because someone decided to store it, load it at the right moment, and eventually remove it. The sources attribute many production failures to context rather than the model: when an agent gives irrelevant responses or forgets previous interactions, the underlying LLM is usually working fine and the problem lies in the information being fed to it. Augment Code frames the split as agent memory, which determines what survives between sessions, and context engineering, which determines what is loaded into the next session's finite window.

Memory types and where they live

Oracle's session material lists working, episodic, semantic, procedural, entity and workflow memory, alongside short-term and long-term memory, summarization, compaction and semantic indexing. Medium's guide describes writing context as storing information outside the window for later retrieval; short-term memory persists within a session, covering conversation history, temporary variables and workflow state, while long-term storage holds items such as user profiles and project context. Weaviate says reliable agents combine the window with long-term memory retrieved from external stores like vector databases and selectively decide what to include at each step. Augment Code names three persistence layers: static context files for team-wide knowledge, agent memory for session-learned decisions with human curation, and living specs for evolving implementation intent. The design guide on hidekazu-konishi.com adds that forgetting should be designed in from the start through TTLs, decay, freshness metadata and staleness detection.

Context operations and failure modes

Microsoft's lesson frames four strategies for managing context: write, select, compress and isolate. The Medium guide similarly describes four fundamental operations, including writing context to persistent storage, reading relevant information into the window, and compressing context to reduce redundancy and verbosity while keeping essential information. Mem0 lists selection and filtering as deciding what to include and what not to include, using relevance scoring, recency weighting and importance thresholds, and says context poisoning and overload are addressed through adaptive memory updates and intelligent relevance filtering. Microsoft names poisoning, distraction, confusion and clash as common context failures to mitigate. Augment Code warns that adding more memory without engineering how it loads produces agents that drown in stale context, while precise context engineering without populated memory produces agents that start fresh every session.

Prompt engineering vs context engineering

Timeline

1. 2026-06
   Date of documentation underlying the agent memory design guide.hidekazu-konishi.com

Common pitfalls

• Confusing agent memory with context engineering, or neglecting either, yields agents that forget critical decisions and resuggest rejected patterns.
• Treating memory as an afterthought leaves agents repeating corrected mistakes or acting on facts that stopped being true; these are memory design problems, not model problems.
• Irrelevant information can hurt performance through distraction or confusion, so loading everything into the window is counterproductive.
• Prompt engineering alone cannot fix the fundamental limitation of a disconnected model that lacks access to private data or earlier conversation.

Key takeaways

• Treat the context window as a scarce resource and design retrieval, memory and tools around it.
• Decide deliberately what to store, when to load it, and when to forget it.
• Separate persistent agent memory from context selection, and invest in both.

FAQ

How this page is made. Built from the full text of 8 sources found with the AISA Tavily API (advanced search), refreshed weekly. A language model organizes what those sources state into the blocks above; each block links the sources it draws on, and figures that do not appear in the sources are removed.