Three Layers of Agent Memory: Working, Session, and Long-Term
Three Layers of Agent Memory: Working, Session, and Long-Term
Agent memory is not one thing. Treating it as a single flat file leads to the common failure mode where short-term notes crowd out important long-term facts, or permanent preferences get deleted during cleanup. The fix is separating memory into three distinct layers.
Layer 1: Working Memory
Working memory is the current context window. It contains the active task, recent tool outputs, and immediate conversation history. This is the most valuable memory because it is the most relevant - everything in working memory directly applies to what the agent is doing right now.
Properties: Ephemeral, high-relevance, limited capacity (context window size), automatically managed by the model.
Layer 2: Session Summaries
When a session ends, the key decisions, outcomes, and unresolved issues get compressed into a summary. The next session loads these summaries to understand recent history without replaying entire conversations.
Properties: Semi-persistent (kept for days to weeks), medium-relevance, manually or automatically generated, stored as structured notes.
Good session summaries include:
- What was accomplished
- What decisions were made and why
- What was attempted but failed
- What is left to do
Layer 3: Long-Term Facts
These are stable truths about the project, the user, and the environment. Project architecture, user preferences, API credentials locations, known bugs and workarounds. They change rarely and apply broadly.
Properties: Persistent (kept for months), variable relevance (depends on the task), manually curated, stored in CLAUDE.md or similar files.
The Interaction Between Layers
The layers feed each other:
- Working memory generates session summaries
- Session summaries surface patterns that become long-term facts
- Long-term facts load into working memory at session start
The key design decision is the promotion and demotion criteria. When does a session observation become a long-term fact? When does a long-term fact become stale enough to demote? Getting these thresholds right is the difference between an agent that learns and one that hoards outdated information.
Implementation
Most developers implement this with three files: the context window (automatic), a sessions/ directory with dated summaries, and a CLAUDE.md for long-term facts. Simple, debuggable, and effective.
Fazm is an open source macOS AI agent. Open source on GitHub.