The Memory Ladder: Which Kind of Memory an AI Agent Needs, and When

Why memory is a decision, not a feature

"Give it memory" is the most common request Sardar Ghaffarhears from students and clients, and the least specific. Memory of what? For how long? For whom? An agent that remembers the last two messages, an agent that can read the whole menu, an agent that knows a customer's usual order, and an agent that shares what it knows with three other agents are four different builds with four different costs — and four different things that can go wrong. The ladder turns "give it memory" into a question with an answer: which rung?

The rule is the important part. Pick the lowest rung that solves the actual problem. Beginners build too high — a vector database for a one-page price list — and pay in complexity, cost and bugs. Every rung you do not need is one you do not have to secure.

The five rungs

Rung 5  SHARED MEMORY     → several agents / channels, one business's customers
        ↑
Rung 4  PERSISTENT STORE  → this customer, across sessions, forever
        ↑
Rung 3  RETRIEVAL (RAG)   → the business's knowledge base, searched on demand
        ↑
Rung 2  FILES             → a document attached for this session
        ↑
Rung 1  CONTEXT WINDOW    → what has been said in this conversation

RUNG 1Context window — persists: this conversation only

What has been said so far in the current chat — the last few messages back and forth. Every agent has this for free. It is enough for the agent to remember that the customer asked for a large chai two messages ago, and it disappears the moment the conversation ends. Use it for: Following a single conversation. Café example: The café agent remembers the order it is building while the customer is still typing.

RUNG 2Files — persists: this session

A document handed to the agent for one session — a PDF menu pasted in, a price list uploaded to a Claude Project. The agent can read all of it, but it is loaded whole, every time, so it suits documents of a few pages, not a catalogue. Use it for: A small, stable document for one conversation or one project. Café example: The Module 2 exercise: a one-page menu in a Claude Project so the JD can be tested.

RUNG 3Retrieval (RAG) — persists: across all conversations — the business's knowledge

A knowledge base the agent searches on demand — menu, policies, FAQs, product pages — chunked and indexed so it fetches only the relevant part, then answers from it. This is what stops an agent inventing prices: it reads the answer instead of guessing. Use it for: A large or changing body of business facts that every customer might ask about. Café example: The café agent answers "do you have anything without dairy?" from the menu file, and the owner updates the menu without touching the agent.

RUNG 4Persistent store — persists: across sessions, per customer, forever

A database row per customer: their name, their number, their last three orders, the booking they cancelled. The agent writes to it during a conversation and reads from it the next time the same person messages. This is the rung that turns a good agent into one customers feel knows them. Use it for: Returning customers; anything that must survive between one conversation and the next. Café example: "Same as last time, Ahmed?" — the café agent recalls a returning customer's usual order.

RUNG 5Shared memory — persists: across agents and channels, for one business

One memory that several agents read and write — the sales agent on WhatsApp, the booking agent on the website, the phone agent — so a customer who booked on one channel is known on the others. It needs an owner (who may write what), rules for conflicts, and care about what one agent is allowed to see. This is how the agency's AI Sales Team runs Zara, Adam and Maya on one number. Use it for: More than one agent, or more than one channel, serving the same customers. Café example: Module 14: one deployment serving two cafés, each with its own memory, no leakage between them.

How to choose the rung

Write one line per thing the agent must remember, with for how long and for whom. The answer to those two questions is the rung. Then build that rung and nothing above it, and add one Rule to the job description: answer only from what was retrieved or stored, and say so when the answer is not there. Climb a rung only when a real conversation fails for lack of memory — that failure names the rung to add next.

The café agent, rung by rung

1. Module 2 — rung 2. A one-page menu in a Claude Project, so the job description can be tested.
2. Module 7 — rung 1. The agent holds an order in the conversation while the customer adds items.
3. Module 9 — rungs 3 and 4. The full menu and policies move into a knowledge base the agent searches; returning customers get a record. Five grounded answers, memory across two sessions.
4. Module 14 — rung 5. One deployment serves two cafés; each café's memory is its own, and nothing leaks between them.

Notice that the agent reached a paying-customer standard at rungs 3 and 4. That is where most AI employees live, and it is what Module 9 of AI Agent Mastery teaches you to build.