Compound Knowledge Across 100+ Sessions: 10% Signal, 90% Noise
Compound Knowledge Across 100+ Sessions
After running an AI agent across more than 100 sessions, I measured how often stored memories were actually useful when retrieved. The number was sobering: about 10% of retrieved memories contributed to the current task. The other 90% was noise.
The Accumulation Problem
Every session produces memories. Bug fixes, user preferences, codebase patterns, failed approaches, successful patterns. After 100 sessions, the memory store is large. After 500, it is enormous. The problem is not storage - it is retrieval relevance.
When the agent starts a new session and retrieves context from memory, most of what comes back is technically accurate but irrelevant. The memory about a bug you fixed three months ago in a different part of the codebase is not helpful when you are writing a new feature today.
Why 10% Is Still Valuable
Despite the noise, that 10% is extremely valuable. A memory about a specific API quirk, a user preference for code style, or a previous decision about architecture saves real time. The compound effect of 100 sessions of accumulated relevant knowledge makes session 101 noticeably faster than session 1.
The problem is paying the cost of the 90% to get the benefit of the 10%.
Fixing the Ratio
Aggressive pruning helps. Memories that have not been accessed in 30 days get archived. Memories that were retrieved but not used get downranked. The system learns which categories of memories are useful and prioritizes those.
Structured memory helps more. Instead of storing everything in one flat vector store, categorize memories by type - project knowledge, user preferences, tool quirks, past errors. Retrieve from the relevant category instead of searching the entire store.
The goal is not perfect retrieval. The goal is making the 10% easy to find without drowning in the 90%.
Fazm is an open source macOS AI agent. Open source on GitHub.