AI Agent Memory Explained

A language model remembers nothing between API calls. Every reply is produced from only the text you send in that request. AI agent memory is the set of techniques that decide what goes into that text: the recent conversation, saved facts about the user, past events and learned instructions. This guide explains short-term and long-term memory, shows both in Python, and tells you which one your agent needs.

By Abdul Rahman Azam, founder of AI Season · Updated

Learn it live with us →

Key facts

Agent memory is Code that stores information and puts the right part of it back into the model's context
Short-term memory The current conversation or task, kept in the message list or a checkpoint
Long-term memory Facts, events and preferences saved across sessions in a database or vector store
Main limit The context window: every remembered token costs money and attention
Common tools LangGraph checkpointers and stores, SDK sessions, vector databases, plain SQL
Memory vs RAG RAG retrieves documents you wrote; memory retrieves what the agent learned from use

What is AI agent memory?

AI agent memory is the system around a language model that saves information from earlier steps or conversations and loads the relevant part into the prompt when it is needed. The model itself is stateless. If your agent seems to remember your name, that is because your code sent your name again.

Memory therefore is a context management problem. You cannot send everything forever, because the context window is limited and long prompts are slower, costlier and less accurate. Good memory design chooses what to keep, what to summarise and what to look up only on demand.

Types of AI agent memory

TypeWhat it holdsWhere it livesExample
Short-term (working)Messages and tool results of the current taskThe message list, a checkpointThe last ten turns of a chat
SemanticFacts about the user or the worldKey-value store, database, vector store"Prefers answers in Urdu"
EpisodicWhat happened in past runsLogs, vector store"Last week the refund tool failed for order 812"
ProceduralHow to do the jobSystem prompt, rules files"Always confirm before sending email"

Most beginner agents need only the first row. Add long-term memory when users return and expect the agent to know them.

Short-term memory: keeping a conversation

The simplest memory is the message list. Append every user message, model reply and tool result, and send the whole list each time.

from openai import OpenAI

client = OpenAI()
messages = [{"role": "system", "content": "You are a helpful study assistant."}]

def chat(user_text: str) -> str:
    messages.append({"role": "user", "content": user_text})
    reply = client.chat.completions.create(model="gpt-4o-mini", messages=messages)
    answer = reply.choices[0].message.content
    messages.append({"role": "assistant", "content": answer})
    return answer

chat("My name is Hira and I am learning LangGraph.")
print(chat("What am I learning?"))  # LangGraph: it is still in the list
short_term.py: memory is just the message list

This breaks when the list outgrows the context window or the process restarts. Frameworks fix the restart problem with persistence. In LangGraph a checkpointer saves the graph state after each step under a thread ID, so the conversation survives restarts and can be resumed later.

from langchain.chat_models import init_chat_model
from langgraph.checkpoint.memory import InMemorySaver
from langgraph.graph import START, MessagesState, StateGraph

model = init_chat_model("openai:gpt-4o-mini", temperature=0)

def call_model(state: MessagesState):
    return {"messages": [model.invoke(state["messages"])]}

builder = StateGraph(MessagesState)
builder.add_node("model", call_model)
builder.add_edge(START, "model")
graph = builder.compile(checkpointer=InMemorySaver())

config = {"configurable": {"thread_id": "hira-1"}}
graph.invoke({"messages": [("user", "My name is Hira.")]}, config)
result = graph.invoke({"messages": [("user", "What is my name?")]}, config)
print(result["messages"][-1].content)
langgraph_memory.py: one thread ID, one remembered conversation

InMemorySaver is for learning. In production use a database-backed checkpointer such as the SQLite or Postgres savers so memory survives a redeploy.

Managing a long conversation

  • Trim: keep the system prompt and the last N messages. Cheap, but old details are lost.
  • Summarise: when the history passes a token limit, ask the model to compress the older part into a short summary and keep that instead.
  • Offload: store large tool results, such as a long file or web page, outside the context and keep only a reference the agent can re-open.
  • Isolate: give a sub-task to a sub-agent with its own fresh context and return only the result.

Never cut a tool result away from the tool call that produced it. APIs reject histories where the two are separated.

Long-term memory: remembering across sessions

Long-term memory has two halves: writing (deciding what is worth saving) and reading (finding it later). The simplest working design gives the agent two tools, one to save a fact and one to search saved facts, and stores them per user.

import json, pathlib

STORE = pathlib.Path("memories.json")

def _load() -> dict:
    return json.loads(STORE.read_text()) if STORE.exists() else {}

def save_memory(user_id: str, fact: str) -> str:
    """Save one durable fact about the user, e.g. a preference or goal."""
    data = _load()
    data.setdefault(user_id, []).append(fact)
    STORE.write_text(json.dumps(data, indent=2))
    return "saved"

def recall_memories(user_id: str) -> list[str]:
    """Return everything remembered about this user."""
    return _load().get(user_id, [])

# At the start of each session, load memories into the system prompt:
facts = recall_memories("hira")
system_prompt = "You are a study assistant. Known about this user: " + "; ".join(facts)
long_term.py: a file-based memory you can swap for a database

When there are too many memories to load them all, store each one with an embedding in a vector database and retrieve only the few most similar to the current message. LangGraph ships this as a store with namespaces per user; managed options such as Mem0 and Zep do the extraction and search for you.

Agent memory vs RAG

RAGAgent memory
SourceDocuments you prepared: PDFs, policies, product dataWhat the agent learned from conversations and tasks
ChangesWhen you re-index documentsContinuously, as the agent runs
ScopeUsually shared by all usersUsually private to one user or thread
RiskOut-of-date documentsSaving wrong, sensitive or injected information

They use the same retrieval tools and often sit side by side. See RAG for AI agents for the document half.

Mistakes to avoid

  • Saving everything. Memory full of small talk makes retrieval worse. Save durable facts only.
  • Never updating. A user who moved from Lahore to Dubai should not have both cities remembered. Overwrite, do not only append.
  • Mixing users. Always key memory by user or thread ID. A leak between users is a privacy incident.
  • Trusting memory as instructions. Text saved from a web page or email can carry a prompt injection. Treat memories as data.
  • No way to forget. Give users a way to view and delete what is stored about them.

How AI Season teaches agent memory

Memory appears three times in the AI Season curriculum: in the first session as the message list, in the design patterns session as context engineering inside the agent harness, and in the LangGraph session as checkpoints and stores. Students build an agent that remembers a user across restarts. Cohort 02 starts 1st January 2027.

Frequently asked questions

Do AI agents have memory?

Not by themselves. The language model is stateless, so an agent remembers only what its surrounding code stores and sends back in the prompt. Memory is built by the developer, not given by the model.

What is the difference between short-term and long-term memory in AI agents?

Short-term memory is the current conversation or task, usually the message list or a saved checkpoint. Long-term memory is information kept across sessions, such as user preferences, stored in a database or vector store and retrieved when relevant.

Is agent memory the same as RAG?

No. RAG retrieves from documents you prepared in advance. Agent memory stores and retrieves what the agent learned while working. They share the same retrieval techniques and are often used together.

How does LangGraph handle memory?

LangGraph uses a checkpointer to save the graph state per thread ID, which gives short-term memory that survives restarts, and a store for long-term memories shared across threads.

What is the context window and why does it matter for memory?

The context window is the maximum amount of text a model can read in one request. Memory must fit inside it, so agents trim, summarise or retrieve selectively instead of sending the full history.