How AI Remembers and Why It Forgets — and How LifeOS Fights Context Rot
Nadia Makarevich of developerway.com published a great investigation, "How AI Remembers and Why It Forgets: Part 1. The Context Problem" — running real experiments on Claude and GPT to show exactly where and why LLMs start forgetting. Reading it felt personal, because the problem she dissects is the exact problem that made me build LifeOS, my open-source AI memory companion. These are my notes on the piece, and how its findings map onto the architecture decisions I made.
The core idea: context is the only thing an LLM remembers about you. There are no sessions, no recall, no memory on the model's side — every "memory" feature you've ever seen is engineering that decides what to inject into the prompt, and the naive strategy of injecting everything actively makes the model dumber.
Memory is a workaround, not a capability
The article demystifies AI "memory" in one move: when your assistant remembers your favourite food, that fact was stored in a database and literally appended to the system prompt of your next conversation. The model itself is a stateless function — text in, text out. Everything that feels like memory is clever context assembly done by the tooling around the model.
Context Rot: bigger windows don't save you
The most valuable part is her experiments. Models today advertise 1M+ token context windows, but performance degrades far earlier. At ~46k tokens she saw context pollution — the model pulling in facts from adjacent, irrelevant documents. At ~130k tokens (65% of the window) it dropped articles from the middle of a list — the U-shaped degradation curve — misattributed numbers, and even hallucinated an entire extra article that leaked in from training data. The takeaway she leaves you with: the longer the chat, the higher the chance of corruption, and once context is corrupted, don't argue with the robot — start a new session.
This is the exact problem LifeOS is built around
LifeOS exists because a lifelong AI companion can't keep its history in the chat — conversation context can't grow forever, and as the article shows, it degrades long before it overflows. So LifeOS never relies on a long-running context. Instead, every conversation is converted into structured long-term memory outside the model:
- Extract, don't accumulate. After each conversation, LifeOS mines it for a summary, people, places, goals, tasks, mood and importance — then stores the diary entry, metadata and embeddings in Postgres + pgvector. The raw chat never has to live in context again.
- Selective RAG, not "inject everything". Retrieval runs only when historical context is actually needed, and it's routed: career questions pull career memories, planning pulls goals and recent entries, emotional support pulls the mood timeline. Small, relevant context — exactly the antidote to the pollution her experiments demonstrate.
- A living profile instead of a growing transcript. A compact, continuously-updated user profile (goals, interests, relationships, recurring challenges) is loaded before any diary retrieval — a few hundred tokens standing in for years of history, keeping the prompt far away from the rot zone.
- Hybrid search over embeddings. Metadata filters narrow the candidate set before vector search ranks it, so what reaches the context window is the needle, not the haystack.
My takeaways as a builder
- Treat the context window as a budget, not a bucket. The question is never "does it fit?" but "is every token in here earning its place?"
- Structure beats volume. Extracted facts, summaries and embeddings retrieved on demand outperform raw transcripts stuffed into the prompt — in quality, latency and cost.
- Design for forgetting. The model will forget; your system shouldn't. Persist memory outside the model and rebuild minimal context per request.
Part 1 ends where the engineering begins — context solutions, tools and agents are promised for Part 2. If you build anything conversational, the whole piece is worth your time.