Reasoning Over Episodic Memory
The frontier in agent memory is not better storage, it is retrieval that thinks. Three ICLR 2026 submissions point the same direction: time-aware episodic memory, retrieval that iterates instead of looking up, and experience distilled into strategy. Together they sketch what memory looks like when it stops being a filing cabinet.
TL;DR. Most memory systems treat retrieval as a lookup: one query, one ranked list, done. The most interesting agent-memory submissions at ICLR 2026 treat it as reasoning. REMem builds a time-aware episodic graph and retrieves from it iteratively, with tools, across multiple hops. Memory-T1 uses reinforcement learning to teach agents temporal reasoning over noisy multi-session histories, selecting and grounding evidence chronologically instead of just fetching it. ReasoningBank goes a step further up the stack and distills both successful and failed experience into reusable strategies, memory about how things went rather than what happened. The common thread: episodic memory, the record of experience in time, is where memory and reasoning stop being separate subsystems. Retrieval that cannot iterate answers the question you asked. Retrieval that reasons discovers the question you should have asked.
There is a moment familiar to anyone who has watched an agent fail at a multi-session task. The information it needed was in memory. Retrieval even surfaced some of it. And the agent still got the answer wrong, because the answer was not in any single memory. It was in the relationship between several memories laid out in time: this happened, then that changed, then this earlier thing stopped being true. The agent had the evidence and lacked the reasoning to assemble it.
I used to file that failure under retrieval quality. The papers coming out of this year's ICLR submission pool convinced me it deserves a better name. The failure is architectural: we built memory as storage plus lookup, when the problems agents actually face require memory plus reasoning, fused.
Episodic Memory Is the Underrated Type
Semantic memory, the store of facts, gets most of the attention in the memory conversation, probably because facts are what RAG retrieves and RAG is the mental model everyone arrives with. But the type doing the heavy lifting in agent work is episodic: the record of what happened, when, in what context, in what order.
The reason I keep coming back to episodes is that every agent failure I investigate has a history attached to it. Agents act, action generates history, and history is episodic by nature. What did I try. What worked. What was true at the time I decided this. Facts float free of time, which is why semantic stores age into inconsistency without anyone noticing. Episodes are anchored in it. Time is not metadata on an episode, it is the organizing dimension, a point I have argued at length in Time Is the Missing Dimension.
REMem, one of the submissions that prompted this essay, takes that seriously in a way I find validating: it uses a time-aware hybrid episodic graph as its memory substrate. But the storage choice is only half of what makes the paper interesting.
Retrieval That Iterates
The other half is how REMem reads its own memory: iteratively, with tools, across multiple hops. Retrieve something, look at it, decide what to look for next, repeat. That loop sounds small. It is not. It is the difference between a lookup and an investigation.
Single-shot retrieval carries a hidden assumption: that the user's query is already the right key into the store. For simple questions it is. For the questions that matter to agents, the query is usually just the entry point, and the real key emerges only after you see the first round of results. The migration is blocked. Blocked by what? A licensing issue. Resolved? Check the vendor decision. Each answer reshapes the next question. A single embedding lookup may miss that final memory, because the path to it can run through intermediate discoveries.
We have found the same thing in our own work on multi-hop retrieval: letting the retrieval process reason about its next step, rather than fanning out blindly, is where recall on connection-shaped questions actually comes from. The engineering challenge, and I will leave it at this altitude, is keeping an iterating retriever on a leash, because a process that decides its own next query is a process that can wander. Budgets and discipline matter as much as cleverness. But the direction is not in doubt for me, and seeing it emerge independently in the literature only firms that up.
Memory-T1 attacks the same fusion from the training side, using reinforcement learning to teach temporal reasoning over multi-session histories. What I find notable is the problem framing: the challenge is not fetching memories, it is selecting temporal evidence from a noisy history and keeping the chronology consistent while reasoning over it. To me, that makes it look as much like a reasoning benchmark as a memory benchmark, which is exactly my point.
Memory About How, Not Just What
ReasoningBank adds the layer I think gets overlooked most: memory of outcomes. It distills an agent's experience, and crucially both the successes and the failures, into reusable strategies. Not a log of what happened, a residue of what worked.
This is the leap from episodic memory to something like judgment. A raw episode says: I tried X in situation Y and it failed. A distilled strategy says: in situations shaped like Y, do not start with X. The first is a record. The second is advice from your own past. The promise is that accumulating reusable strategies could help an agent improve across a class of tasks, not merely remember a specific instance, which is what everyone actually hears in the phrase self-improving agent, minus the mysticism.
The part I want to underline is the inclusion of failures. I worry that success-only distillation produces survivorship-biased strategy, confident advice from a past that conveniently forgot its mistakes. An agent that remembers what did not work, and why, holds the more valuable half of experience. Humans know this. We just tend to build memory systems that do not.
The Shape of the Frontier
Put the three papers side by side and a composite picture emerges, one I would bet on as the shape of the next couple of years in agent memory.
The substrate is episodic and time-aware, because agents generate history and history is temporal. Retrieval over that substrate is iterative and tool-driven, because the questions that matter require assembly across hops, not lookup. And above the raw episodes sits a distillation layer that converts experience into strategy, so the agent's past improves its future rather than merely occupying storage.
Notice what that picture is not. It is not a bigger context window, which I have already argued is not memory at all, and it is not only a better embedding model, because better embeddings alone do not turn a one-shot lookup into iterative investigation. What strikes me instead is how much the three pieces need each other. Iterative retrieval is only worth running over a substrate that preserves time, and distilled strategies are only trustworthy while the episodes underneath them survive to be checked. Every stage of that picture is buildable machinery, and the ICLR crop is the field starting to build it in the open.
Memory used to be the boring layer under the interesting agent. The dependency has quietly inverted. Reasoning is becoming something memory does, not something done to memory afterward, and the systems that internalize that first will be the ones whose agents seem inexplicably better at long-horizon work. It will not be inexplicable. It will be episodic memory with a reasoner inside, and a year from now I expect that to be the default assumption rather than the frontier.