Lesson 4: Structured State: How an Agent Remembers Where a Task Stands

Learning goals:

• Explain why burying task progress in conversational prose is unreliable, and why it has to become structured state
• State the full lifecycle a todo goes through, from creation to removal
• Distinguish the cost of starting over from scratch versus resuming from where the work was interrupted
• Decide whether a piece of task state should live in the session history or be written into a separate checkpoint

Prerequisites: Finish Lesson 3 and understand the two modes of external memory | Previous: Lesson 3 << | Next: Lesson 5 >>

After a Process Restart, Does the Agent Still Know Which Step It Reached

An agent is working through a multi-step task: refactoring a module, which breaks down into four steps — "update the type definitions," "update the call sites," "run the tests," "update the docs." It has just finished the second step when the process is interrupted by an unexpected restart. When it comes back up, it's facing the same task description. Should it redo step one from scratch, or does it know it already finished the first two steps and can pick up straight from step three?

The answer comes down to one thing: whether the task's progress was recorded as structured state, rather than scattered across a pile of conversational prose. If "the type definitions are already updated" is just a line of natural language in one of the model's earlier replies, buried among dozens of messages, the host code has no reliable way to pull the concrete fact "which step are we on" out of it. But if that progress is expressed as a todo list with fixed fields — where every task carries an explicit status marker — the host code can read it straight off: steps one and two are completed, step three hasn't started.

That's the core question this lesson tackles. Lessons 1, 2, and 3 were all about how to manage content — conversation history, memory files. This lesson is about how to express progress, so that an agent or its host application, after an interruption, knows exactly how far the task got.

The Todo Lifecycle: Created, Activated, Completed, Removed

Take Claude Code's task-tracking tool as the example. The official docs lay out the full lifecycle a todo goes through during execution — four steps:¹

1. Created: Claude adds the todo as pending when it identifies a task
2. Activated: Claude sets the todo to in_progress when it starts the work
3. Completed: Claude marks it completed when the task finishes successfully
4. Removed: Claude deletes a todo it no longer needs by setting status: "deleted" in a TaskUpdate call¹

These four steps aren't the vague "I'm done" or "I'm working on it" of natural language. They're four explicit status values: pending, in_progress, completed, and deleted for removal. Every status change happens through an explicit tool call, not through the model tossing off a remark in its reply text.

The docs also spell out how this mechanism actually shows up in the conversation: "In a session that has the task-tracking tools, Claude keeps a written todo list, updating each item's status as it works. You see each change in the message stream as a structured tool call."¹ That sentence pins down the key distinction: progress isn't passively "reflected" in the conversation, it's actively written as a tool call that can be identified and parsed on its own. That's the real difference between structured state and a progress description scattered through prose.

Checkpoints: Making Recovery Something Other Than Starting Over

Once you have a structured todo list, the next question is: where does the list itself live? If it exists only in this session's conversation history, then the moment the session truly ends (not a brief interruption but a full close, like the "once the session ends, everything in the window is gone" idea from Lesson 3), the progress record disappears with it. At bottom, the API is stateless: "The Messages API is stateless, which means that you always send the full conversational history to the API."²

That's the problem checkpoints solve: write the state of a task at some point in time — which steps are done, which step is current, which steps remain — into a piece of data, and persist it somewhere outside the session's lifetime. Checkpoints use the same underlying mechanism as the external memory from Lesson 3 (write a file, read it back later); the difference is that a checkpoint doesn't store "knowledge worth remembering," it stores "how far the task got" — the kind of state you can use directly to resume execution.

With checkpoints in place, recoverability finally holds: after a process restart, the agent doesn't have to guess "where was I." It reads the most recent checkpoint, sees "steps one and two are completed, step three is in_progress," and continues from step three instead of redoing steps one and two.

The cost comparison here is concrete. In the refactoring task, the "update the type definitions" step, if it's idempotent (running it again produces the same result), only costs wasted time when redone from scratch. But if some step is a non-idempotent operation like "insert a migration record into the database," starting over could insert two duplicate records and even corrupt data. What a checkpoint saves isn't just time — it's the risk of accidentally re-running that kind of non-idempotent operation.

Structured State Exists to Make Interruption Survivable

Back to the scenario that opened this lesson: after a process restart, should the agent start over or resume from where it was interrupted? Now we can answer it clearly — it depends on whether two things were done. Was the task's progress expressed as structured state (rather than scattered through conversational prose), and was that state written into a checkpoint (rather than living only in this one session's history)? You need both. With structured state but no checkpoint, the state still disappears when the session ends; with a checkpoint but no structured state, what got written into the checkpoint is itself vague natural language, and reading it back still won't tell you reliably which step the task reached.

This lesson was about how to express and preserve progress. The next lesson turns to another question that matters just as much: could this preserved state and memory become a target for attackers — what happens if an attacker can write content into a checkpoint or a memory file?

Recap

• Task progress can only be read reliably by the host code once it becomes structured state; a progress description scattered through natural-language replies can't be parsed stably into "which step we're on right now"
• The full lifecycle of a todo is four steps — created (pending), activated (in_progress), completed (completed), removed (deleted) — and each step happens through an explicit structured tool call that can be observed in the message stream
• Structured and persistent are two different things: structured solves "can a program read it," a checkpoint solves "is the state still there after a process restart or a session ends" — you need both
• A checkpoint writes a task's state at a point in time to somewhere outside the session's lifetime, turning recovery into "continue from where it was interrupted" rather than "start over," and especially avoiding the accidental re-run of non-idempotent operations
• Structured state, like external memory, becomes its own attack target once persisted — the topic of the next lesson; the more you preserve, the more boundaries you have to hold

>> Lesson 5: The Boundaries and Safety of Memory