Curing my AI Agent’s Amnesia

AI agents forget past runs, leading to repeated outputs. Learn how to fix memory, avoid context rot, and build smarter automations.

AI AGENTS

March 24, 2026

Dallas Inman

Summarize with AI:

My AI agent had amnesia. Here's what I found when I tried to fix it.

I use OpenClaw to handle parts of my daily workflow. One of the things I set up early on was a weekly competitor research job that runs automatically, checks what our competitors are doing, and drops a summary into my Discord server. No manual effort, just results waiting for me. For the first few weeks it was great.

Then the reports started feeling familiar. Competitors I'd already read about, details I'd already seen, findings that were identical to last week's. The job couldn't tell me what was new because it had no idea what it had reported before. Every run started from scratch with the same prompt and the same scope, which meant I was reading the same output on a loop. It couldn't flag a new feature or a pricing change because it had no concept of what was there last time.

I started checking my other scheduled jobs and found the same pattern everywhere. Morning briefings that couldn't distinguish new information from yesterday's news. A watchdog monitor running against no baseline. Nine of my fourteen automated jobs were operating with zero memory between runs. While I was digging through the file system I also found 61 orphan session files from old runs just sitting on disk, referenced by nothing. Cleaning those up was pure housekeeping, but it made me realize that nobody had thought about how any of these sessions should be managed over time.

My first instinct was the obvious one: give the jobs their history back and let them remember everything. But giving an agent all of its history doesn't actually make it smarter. Research from Chroma tested 18 frontier models and found that every single one produced worse output as context length increased, a phenomenon they call "context rot" that kicks in well before you hit the context window limit. Andrej Karpathy frames the solution as "context engineering," which is the art of filling the context window with just the right information for the next step. Not everything, not nothing, just what the job actually needs.

So the question became how much memory each job actually needs, and I audited all fourteen to find out. Three distinct patterns emerged. About a third were truly stateless, things like reminders and infrastructure tasks and one-shot syncs that don't need any memory at all. Another third only needed the previous run, daily briefings and watchdog monitors where one prior output is enough to generate a meaningful diff. The remaining third needed a rolling window of a few weeks to spot trends and track changes over time. The most important finding was that none of the fourteen needed full history, but nine of them needed something and were getting nothing.

The fix was a per-job config with three modes. Prior run output gets injected as a prefix before execution, and the agent doesn't know the difference. It just sees a slightly longer prompt that includes what it found last time. No compaction, no summarization, no LLM deciding what to throw away. Just a clean rule per job: this one keeps zero, this one keeps one, this one keeps four. The competitive intel job immediately started producing diffs like "this competitor added a new integration since last week," which was the whole point.

I was happy with the fix until I shared it with a teammate who's designing our Workflows product. He pointed out that for any resumable chat session, if you don't have some form of compaction you'll eventually fail hard as the context keeps growing with each turn. Not degrade gracefully, just fail. And he was right. My per-job config works for scheduled jobs because I control the inputs and know when execution ends. Interactive sessions are a different problem entirely because the context grows with every message and you can't cap it upfront when you don't know when the conversation will stop.

The analogy that helped me think about it clearly is packing a golf bag versus managing your bag during a round. Configuration is deciding before the round which fourteen clubs to carry based on the course layout. Compaction is realizing mid-round that your bag is too heavy and figuring out what to leave behind without losing something you'll need on the back nine. They're two tools for two fundamentally different problems.

This is still largely unsolved in the industry. Most agent frameworks either persist everything and hit context rot, or persist nothing and lose continuity. It gets even more interesting when you combine both patterns, because a workflow that runs on a schedule with clean configured memory might also need to hand off to an interactive session when a user opens a chat to ask about the last run. Getting that handoff right is the next frontier.

In my last post I talked about building an AI system that maintains my task list. This one is about what I found when I looked at how the jobs behind that system were actually running. At Airbyte, we connect AI agents to the data they need, and it turns out the "how much data" question matters just as much for the agent's own memory as it does for the sources it reads from.

This is the second post in Deployed with Dallas, a series on how I've been building AI into my day-to-day as a forward deployed engineer at Airbyte. More posts to come soon!

Try Airbyte Agents

Airbyte connects to your tools and turns your business data into living context. Every agent you build gets the full picture.

Try it free

About the Author

Dallas Inman

Dallas Inman is a forward deployed engineer at Airbyte.

Try Airbyte Agents

Be among the first to explore our new platform and get access to our latest features.

Try it free

Curing my AI Agent’s Amnesia

Try Airbyte Agents

Try Airbyte Agents

Related posts