VI EN
AI Agents & Skills

Harnesses for long-running agents: keeping memory across context windows

Binh 3 min read Đọc bài này bằng tiếng Việt

TL;DR

Agents on long projects fail by doing too much, declaring victory early and skipping real tests. The fix is external memory: a JSON feature list, progress file, startup script and Git, one feature per session, tested like a real user.

Give an agent a project that runs for hours or days, such as building an entire web app, and the biggest problem isn't intelligence but memory. Each new session starts with an empty context; compaction helps but isn't enough. Anthropic shared how it designed a "harness" that lets agents work effectively across many context windows, based on an experiment building a claude.ai clone.

How do agents fail?

  • Doing too much at once: running out of context mid-feature and leaving half-finished, undocumented code for the next session.
  • Declaring victory too early: a later session sees existing progress and assumes the project is done.
  • Marking features done without real testing: only unit tests or curl, no end-to-end check.

A two-part solution

AgentWhen it runsJob
Initializer agentOnce, at project startSets up the environment, writes the feature list, a startup script, the first commit
Coding agentEvery later sessionWorks one feature at a time, leaves clean, mergeable code

The artefacts that hold memory

  • Feature list as JSON: over 200 end-to-end features, all initially marked failing. Agents may only flip a status flag. JSON was chosen because models corrupt it less than Markdown.
  • Progress file: a log of what each session did.
  • init.sh: a script that starts the dev server, so agents don't have to work out how to run the app.
  • Git history: descriptive commits that allow reverting to a working state.

The per-session routine

1. Check the working directory
2. Read the progress file and git log
3. Run init.sh + a basic end-to-end test
   (catch bugs the previous session left)
4. Pick ONE failing feature, build it, test it like a real user
5. Flip its status, update progress, commit

Testing "like a real user" with browser automation (Puppeteer MCP) made a clear difference, catching bugs that weren't visible in the code. A remaining limitation: Claude can't see native browser alert dialogs, so features relying on them were buggier.

Risks and caveats

  • Structured state beats model memory: don't hope the model "remembers"; write it down in machine-readable files.
  • One unit of work per session: many things per session is a recipe for half-done code.
  • Open questions: whether one general agent or several specialised agents (testing, QA, cleanup) work better is still unclear, and the approach has mainly been validated on web development.

How to apply it

  • Dev: For long Claude Code projects, create features.json and PROGRESS.md, and have Claude read them at the start of each session and update them at the end.
  • Tech Lead: Define "done" as tested end to end, not just unit-tested; provide a standard startup script.
  • PM: Write requirements as a list of individually testable features; it's both the agent's spec and the progress measure.

Sources & references

  1. Effective harnesses for long-running agents — Anthropic Engineering
More on AI Agents & Skills