How NEMORI Enables AI Agents to Remember What Matters.
NEMORI distills 20 chat messages into episodic and semantic memory, helping LLM agents retain facts and context across conversations.
Key Takeaways
- NEMORI converts a 20-message chat buffer into organized long-term memory for AI assistants, preserving facts, context, and updates across conversations.
- It groups messages into coherent episodes, creates cues and narratives with embeddings, and keeps original messages attached for episodic memory.
- It distills episodes into standalone semantic facts through prediction-error comparison, extracting durable information while ignoring temporary or stylistic details.
- New facts are reconciled with stored knowledge, allowing additions, merges, or replacements when contradictions or updates occur.
- Retrieval combines relevant episodes and facts to give assistants a smaller, more useful context instead of relying on full chat history.
NEMORI Turns 20 AI Chat Messages Into Long-Term Memory
Meta Description: NEMORI processes 20-message chat buffers into episodic and semantic memories, helping AI assistants retain facts, context, and updates across conversations.
Tags: NEMORI, AI memory, LLM agents, episodic memory, semantic memory
Featured image: abstract, friendly illustration of numbered chat messages forming memory cards, with no real people, logos, or text.
Why Chat History Alone Is Not Enough
NEMORI is a memory framework for language-model assistants. It takes the stream of chat messages and builds a second layer of organized memory around the model. The framework processes a buffer of 20 messages, groups them into events, and separates what happened from what should become a durable fact. For people building LLM agents, and for users who expect assistants to remember decisions, preferences, and changes, NEMORI tries to make continuity less dependent on rereading the entire history. The method is introduced in What Deserves Memory: Adaptive Memory Distillation for LLM Agents.
Human conversation works through small follow-ups. A friend mentions a new job, and a week later another question can refer back to it without a full recap. An assistant should do something similar: carry forward a role change, a tool choice, or a decision made in an earlier chat. But raw logs are not memory. A conversation can contain repetition, temporary concerns, updates to earlier statements, and details that only make sense with context. Some should be retrieved directly; others should be stored with the experience around them. NEMORI keeps the language model itself unchanged and grows the information stored around it.
How NEMORI Builds Episodes From Messages
The system first puts incoming messages into a temporary buffer. Each user message and each assistant message is a separate element with sender, content, and timestamp. In the paper’s main setup, processing starts when the buffer contains 20 messages. The messages are numbered by position, from m1 to m20.
An example from the paper uses May 10, 2026. Alice tells the assistant that she now works at Nova and started yesterday. She also mentions a possible holiday next month. The assistant asks about her role. Alice replies that she is a backend engineer and that her team uses Go because deployment is simpler. The rest of the buffered exchange moves through holiday planning and then debugging a SQL query.
NEMORI then asks a language model to group the numbered messages into coherent episodes. A group is a set of messages about the same event or discussion. The model returns indices that identify the messages. The job discussion may use [1, 3, 4], while holiday messages use [2, 5, 6, 7] and the SQL debugging conversation uses [8, 9, 10, 11, 12, 13, 14, 15, 16, 17, 18, 19, 20]. Indices can be nonconsecutive because topics can be interleaved.
For each group, the program retrieves the original messages using those indices. Because the message labels start at 1 and list positions in code start at 0, the system subtracts one to fetch each selected message. It then creates a short description called a cue and a narrative that connects the statements. In Alice’s job episode, the narrative resolves “yesterday” into May 9 and records that she joined Nova as a backend engineer and that her team uses Go to simplify deployment. The raw messages stay attached. The cue and narrative are turned into an embedding, a numerical representation used to search related meanings later.
If a related episode already exists, NEMORI can check whether the new discussion continues it, combine the narratives, keep the source messages, and update the embedding. The result is stored in episodic memory.
How Episodes Become Standalone Facts
The next stage asks what new knowledge the experience adds. NEMORI searches semantic memory using the episode embedding. In the example, the assistant already knows two facts: Alice currently works at Acme, and Alice prefers Python. It then gives the cue and those known facts to a language model, while withholding the full narrative and raw messages. The model predicts the episode from limited clues. It might say Alice discussed changes at work and her experience using Python. That prediction misses Nova, the joining date, and the team’s use of Go.
The gap between prediction and reality is the signal. A second language-model call compares the prediction with the original conversation and extracts useful differences. The result may include three new facts: Alice currently works at Nova as a backend engineer, Alice joined Nova on May 9, 2026, and Alice’s team uses Go because it simplifies deployment. The paper calls this process prediction error distillation. The extraction prompt favors specific information useful for future interactions, while pleasantries, conversational style, and temporary reactions generally do not become separate entries.
Before saving, NEMORI reconciles each candidate with related stored facts. It can add a new entry, combine overlapping knowledge, or replace information invalidated by a contradiction. Alice’s current employer can move from Acme to Nova. Her preference for Python can remain, because a team’s use of Go and an individual’s preference for Python describe different things.
After the update, the assistant has two views of the same experience. Episodic memory keeps the job discussion, its narrative, and the original messages. Semantic memory keeps standalone facts about Alice’s employer, role, joining date, and team tools. When a later question arrives, the system searches both stores. A question about when Alice joined Nova can retrieve the date. A request for help with a work service can retrieve the Go choice and the reason behind it. The answer model receives a smaller context: relevant episodes, relevant facts, and selected original messages when needed.
What Readers Should Take From the Architecture
The main practical value is preparation. Some work happens before the next question arrives: messages are grouped, events are clarified, and possible knowledge gaps are examined. That can reduce the need to stuff every answer with a long transcript. It also preserves different levels of detail. A simple query may need one semantic fact. A more involved query may need the surrounding episode or the original conversation.
The process still depends on model judgment. A prediction can miss information the system already knows, or it can guess a new detail correctly. That means prediction error is a useful signal, not a perfect measure of what will matter later. Keeping the source conversation gives the system evidence it can revisit.
The paper presents benchmark evidence on LoCoMo with GPT-4o-mini, describing encouraging results and an average outcome, but the available text does not give the full number. For readers, the safer takeaway is not a performance claim but the design pattern: long-term memory for LLM agents needs a structure that can separate experience from facts, update facts when contradictions appear, and retain context when facts are not enough.

Comments (0)