Context Compaction Ate Our Agent's Memory
Context Compaction Ate Our Agent's Memory
We had an agent that worked perfectly for the first 20 minutes of any session. Then it would start making bizarre decisions - repeating work it had already done, forgetting constraints it had acknowledged, contradicting its own earlier reasoning.
The culprit was context compaction. As the conversation grew past the context window limit, the system automatically summarized older messages to make room for new ones. Sounds reasonable. Except the summarization consistently threw away the exact details the agent needed.
What Gets Lost
Context compaction treats all information as equally compressible. But it is not. A constraint like "never modify files in the /config directory" gets summarized away because it looks like a minor detail. The agent's understanding of a complex multi-step plan gets reduced to a single sentence. Error messages from failed attempts disappear, so the agent tries the same broken approach again.
The information that looks least important to a summarizer is often the most operationally critical to an agent.
The Memory Hierarchy Problem
What agents actually need is a memory hierarchy - not a flat context that gets uniformly compressed. Some information should be permanent (constraints, goals, credentials). Some should be medium-term (current plan, recent errors). Some is truly ephemeral (intermediate reasoning steps).
Compaction does not distinguish between these layers. It just squeezes everything.
Practical Fixes
- Pin critical context - keep constraints and goals in a system prompt or separate document that never gets compacted
- Use external memory - write important decisions and discoveries to files that can be re-read
- Structured checkpoints - periodically have the agent write a status summary to persistent storage
- Smaller task scopes - break work into chunks that fit within the context window without needing compaction
The real fix is treating agent memory as a first-class engineering problem rather than hoping the LLM's built-in context management will handle it.
Fazm is an open source macOS AI agent. Open source on GitHub.