Why we stopped using RAG
and built a hippocampus instead.

We need to start with an admission: we used RAG for two years. We built on it, we shipped with it, we defended it to clients who asked whether it was the right architecture. For a long time, we believed the problems we were seeing were implementation problems — that if we tuned the chunking, improved the embeddings, optimised the retrieval pipeline, things would eventually work the way we needed them to.

They didn't. And the more we dug into why, the more we realised the problems weren't fixable. They were architectural. RAG was built to answer questions from documents. We were trying to make it the long-term memory of an intelligent agent. Those are fundamentally different problems, and no amount of engineering can bridge the gap.

This article is the honest account of what broke, what we tried, and what we built instead. It is also an argument that the entire enterprise AI industry is building on a foundation that will have to be replaced — and that the replacement is biological in design.

What RAG actually does —
and what it was designed for.

Retrieval-augmented generation was introduced in a 2020 Facebook AI Research paper as a technique for grounding language model outputs in external knowledge. The core idea is elegant: instead of training all the knowledge into the model's weights, store it in a retrieval system and fetch the relevant pieces at query time. You get factual grounding. You get updateable knowledge. You avoid the computational cost of constant retraining.

For its intended purpose — answering factual questions from a knowledge base — RAG works well. Ask a question, retrieve relevant documents, generate an answer. The architecture is well-suited to this. It is fast, scalable, and understandable.

The problem is that enterprise AI agents don't primarily need to answer factual questions from a knowledge base. They need to:

• Remember the context of previous conversations with specific users
• Learn from corrections and apply them permanently
• Understand relationships between concepts — not just find similar text
• Build expertise about a specific domain through use
• Improve over time without requiring retraining

RAG does none of these things. It was never designed to. Every session starts cold. Every correction evaporates when the context window closes. The embeddings computed at deployment are the same embeddings used in month twelve. The system learns nothing, adapts to nothing, and improves at nothing — because learning, adaptation, and improvement are not in its design.

The four things that broke
in production.

We didn't arrive at this conclusion from theory. We arrived at it from watching four specific failure modes appear repeatedly across production deployments, regardless of how carefully we had engineered the RAG pipeline.

Failure 1: Session amnesia

Every conversation started from zero. A user who had spent thirty minutes teaching an agent their preferences in one session found, in the next session, that the agent had no recollection of any of it. We tried storing conversation summaries in the vector store. We tried hierarchical memory structures. We tried fine-tuning on conversation history. None of it provided the seamless, persistent memory that the use cases required.

The root cause is structural: a vector store is a snapshot of knowledge at a point in time. It doesn't have a native mechanism for updating based on a conversation. Adding persistent memory to RAG requires bolting on an entirely separate system — and that bolted-on system is always a compromise.

Failure 2: Correction evaporation

In every enterprise deployment, users correct agents. "Don't use bullet points." "Always show financial data in tables." "When you mention Supplier A, note that we had quality issues with them in 2023." These corrections are often critical — they encode institutional knowledge that doesn't exist anywhere in the documents.

With RAG, every correction requires either a retraining cycle (slow, expensive, requires engineering) or a workaround that degrades over time (appending to system prompts, adding to retrieval context — neither of which behaves like genuine learning). We watched teams spend significant engineering time trying to make corrections stick, and saw the user trust in their agents erode each time a correction was forgotten.

Failure 3: Similarity is not understanding

The semantic embedding of "Q4 revenue targets" is not proximate in vector space to "open purchase commitments" or "headcount freeze implications" — even though an experienced finance professional immediately understands these concepts as connected. Similarity search finds text that looks like the query. It doesn't understand what the query is actually about.

In practice this means agents retrieve documents that are textually similar to the question but miss the conceptually relevant context. As the knowledge base grows, this problem compounds — more documents means more opportunities for textually similar but conceptually irrelevant retrieval to crowd out the right answer.

Failure 4: The knowledge plateau

Every RAG deployment performs at deployment-time capability. Month one, month six, month twelve — the same accuracy, the same errors, the same limitations. The system cannot improve from use, because nothing about its architecture supports improvement from use. This is not a bug. It is the correct behaviour for a document retrieval system.

For agents deployed in enterprise contexts — where the use case is to build institutional expertise, not just answer questions — this plateau is unacceptable. You are not paying for a search engine. You are paying for something that gets better at your specific domain over time. RAG cannot provide that.

What we tried before
going back to biology.

Before building HippoFabric we spent six months trying to solve these problems within the RAG paradigm. This is worth documenting because the failure of these approaches clarified exactly what the architectural requirements were.

Attempt 1: Hierarchical memory with summarisation

The idea: create a layered memory system where recent conversations are stored verbatim, older ones are summarised, and ancient history is compressed into themes. Each layer is stored as vectors and retrieved at query time.

The problem: summarisation loses critical specifics. The correction "always check two exchange rate sources" becomes "use reliable data sources" in summarisation — precise behavioral guidance degraded into vague principle. Retrieval across layers was inconsistent. The system was complex and brittle. And it still didn't learn from corrections — it just stored them more efficiently until they eventually dropped out of context.

Attempt 2: Knowledge graph augmentation

The idea: add a knowledge graph on top of the vector store to capture relationships between entities. Use graph traversal to surface related concepts that wouldn't appear in similarity search.

The problem: the graph had to be manually constructed and maintained. It didn't learn from interactions. It didn't weight connections by how often they were relevant together in practice. It was static in the same way the vector store was static — better at representing relationships, but still frozen at construction time. And it doubled the complexity without halving the fundamental limitation.

Attempt 3: Fine-tuning loops

The idea: periodically fine-tune the model on interaction data, incorporating corrections and preferences into the model weights themselves.

The problem: fine-tuning is expensive, slow, and requires engineering involvement every cycle. It causes catastrophic forgetting — new fine-tuning runs risk overwriting previous learnings. It doesn't allow for targeted correction of specific behaviors. And it still doesn't solve the session memory problem, because fine-tuning changes model behaviour but doesn't give the model access to user-specific context.

Going back to biology —
the insight that changed everything.

The turning point came when we stopped asking "how do we fix the RAG architecture?" and started asking "what architecture does the problem actually require?" And when we framed it that way, the answer was obvious — we needed something that worked like biological memory.

Consider what the human hippocampus does. It doesn't store memories as embeddings and retrieve them by similarity. It stores concepts as nodes and relationships between concepts as weighted edges. When you think about "procurement," your brain doesn't search a vector space for semantically similar tokens. It activates a network of connected concepts — suppliers, contracts, pricing, colleagues, past experiences — and spreading activation carries the relevant context to the surface.

That activation is weighted. Concepts that have fired together many times have stronger connections. Concepts that co-occurred in negative contexts have inhibitory weights. The graph is not static — every experience adjusts the weights, strengthening some connections and weakening others. This is Hebbian learning: "neurons that fire together, wire together."

And critically: the hippocampus runs a consolidation process during sleep. The day's experiences are replayed, patterns are extracted, schemas are strengthened. The brain improves without new external input. This offline processing is why sleeping on a problem genuinely helps — memory is not just storage, it is an active processing system.

HippoFabric implements these three biological mechanisms in software — weighted concept graphs, Hebbian learning from interactions, and offline sleep consolidation — and makes them available as a simple API that any agent can use as its memory layer. Here is what each mechanism provides that RAG cannot:

Spreading activation — understanding over retrieval

When an agent calls brain.think("Q4 tracking"), HippoFabric doesn't search for similar text. It activates the "Q4 tracking" node and allows activation to spread through the concept graph — reaching "revenue pipeline," "budget variance," "open POs," "headcount implications" — because those concepts have co-activated with "Q4 tracking" in previous relevant contexts. The system surfaces related ideas, not similar text. This is a fundamentally different and more powerful operation.

Hebbian learning — improvement from use

Every time two concepts co-activate in a meaningful context, the edge between them is strengthened. Every correction — "always check two exchange rate sources" — immediately creates a strong inhibitory connection on the incorrect behavior and a reinforcing connection on the correct one. No retraining. No engineering cycle. The correction is in the graph from the moment it's made, permanent across all future sessions.

Sleep consolidation — compounding without input

HippoFabric runs an offline consolidation cycle that replays interaction patterns, strengthens high-signal edges, and crystallises emerging schemas. The brain improves overnight — the agent that starts Monday is demonstrably smarter than the agent that finished Friday, without any new interactions having occurred. This is the compounding effect that makes the value of a HippoFabric deployment grow over time rather than plateau.

What this looks like
in practice.

The API difference between RAG and HippoFabric reflects the architectural difference. With RAG, you are querying a retrieval system. With HippoFabric, you are activating a brain.

The brain.think() difference in depth

The depth parameter in brain.think() controls how many hops of spreading activation are performed. At depth=1, you get directly connected concepts. At depth=3, you get concepts that are three relational steps away — which is often where the most valuable associative context lives. A RAG system has no equivalent: it retrieves the top-k most similar documents regardless of relational distance.

In a procurement context: brain.think("Supplier A contract renewal", depth=3) activates not just the contract documents, but the quality incident history, the relationship notes from the account manager, the benchmark pricing from comparable suppliers, and the institutional memory of why Supplier A's terms were renegotiated two years ago. None of this is textually similar to "Supplier A contract renewal" — but all of it is conceptually relevant.

What the benchmarks
actually showed.

We submitted HippoFabric to LongMemEval — the ICLR 2025 gold standard benchmark for AI memory evaluation, specifically designed to test multi-session reasoning and long-term memory retention. The results confirmed what we had seen in production deployments but needed independent verification to claim publicly.

The 90.6% figure is significant not just because it is higher, but because of what it measures. LongMemEval tests specifically the scenarios where RAG fails: remembering user-specific context across multiple sessions, applying corrections from one session to future sessions, and reasoning about relationships between concepts that were established in different prior conversations.

These are precisely the failure modes we documented internally. The benchmark validates that HippoFabric solves the problem that RAG cannot — not that it is a marginally better retrieval system, but that it succeeds at a fundamentally different task.

The speed figure is worth noting separately. HippoFabric's spreading activation operates at 0.46 seconds — ten times faster than the slowest competitors and faster than most RAG pipelines at equivalent context depth. This is because graph traversal with precomputed weights is computationally much cheaper than embedding generation and nearest-neighbour search at scale. As the knowledge base grows, RAG gets slower. HippoFabric's traversal time scales with graph diameter, not graph size.

When RAG still makes sense —
being honest about tradeoffs.

We want to be direct about this: RAG is not wrong. It is the right tool for the right problem. If your use case is document question-answering — a user asking questions from a defined knowledge base, where session context doesn't matter and persistent learning isn't required — RAG is excellent. It is well-understood, widely supported, and has a mature ecosystem.

The argument we are making is narrower: RAG is the wrong architecture for agents that need to build genuine expertise in a domain, maintain persistent relationships with users, and improve from production use. For those use cases — which describe the majority of enterprise AI agent deployments — HippoFabric is the right architecture.

The conclusion we didn't
want to reach — but had to.

We didn't want to rebuild our memory architecture. We had significant investment in our RAG pipeline — the tooling, the embeddings, the retrieval optimisations, the team's familiarity with how it worked. Replacing it was expensive and risky. We made the decision reluctantly, after exhausting the alternatives, because the evidence was unambiguous.

Every production agent that needs to genuinely learn from its users, maintain expertise in a domain, and improve over time will eventually hit the same walls we hit. The architecture that was designed for document retrieval cannot be engineered into something that behaves like biological memory. The scaffolding will always fail eventually.

The enterprise AI industry is, right now, building largely on RAG. This is understandable — it works for the demo, it ships fast, and the limitations only become apparent in sustained production use. But the reckoning is coming. The organisations that recognise it early and build on the right foundation will have a compounding advantage that their competitors will find very difficult to close.

The hippocampus has been solving the long-term memory problem for 500 million years. It turns out the right answer was biological all along.