Lesson 2: Managing Conversation History: Append, Truncate, Summarize

Learning goals:

• Explain why conversation history only grows and never shrinks by default
• Say what truncation throws away, what it keeps, and what structure it can break
• Tell apart the problems that compaction and tool-result clearing each solve
• Judge which mechanism to reach for based on window usage and the kind of content that's bloating it

Prerequisites: finish Lesson 1 and understand what a context window is made of | Prev: Lesson 1 << | Next: Lesson 3 >>

Append is the default: why history keeps growing

Lesson 1 made the point that the history the model sees is whatever the host app re-sends every turn. So how does it actually get sent? The plainest implementation is append: when a turn ends, you tack the new messages from that turn (the user's words, the model's reply, tool calls and their results) onto the end of the existing messages array, and next turn you send the whole array back out as-is.

The official docs put this default plainly: as the conversation moves forward, each user message and model reply piles up in the context window, and every earlier turn is kept in full. "As the conversation advances through turns, each user message and assistant response accumulates within the context window, and previous turns are preserved completely."¹ Nobody is actively deleting anything, so history only climbs — ten turns in, the window holds all ten turns' worth of content, not a summary of the latest turn and not an auto-filtered set of highlights.

In a short conversation this is a non-issue. But for an Agent that runs a long time, the problem snowballs: every tool call's full arguments and full return value get stuffed into history, and a task that repeatedly reads files and runs commands can easily push the messages array into the tens of thousands of tokens after a few dozen turns. Lesson 1 covered that the window has a hard capacity ceiling, and the fuller it gets the closer you are to hitting it. The subtler cost is context rot — the longer and messier the history, the harder it is for the model to find the one line in there that actually matters right now¹. Let history grow unchecked and you eventually pay both bills.

Truncation: the simplest and bluntest option

The most direct response is truncation: when the window is nearly full, cut the oldest batch of messages outright and keep only the most recent N turns. This is the easiest thing to build — no extra model call, no summary format to design. A single line of messages.slice(-N) does it.

But what truncation drops is gone for good. If the batch you cut contained a key constraint the user stated back in turn 3 ("budget stays under $5,000") and the Agent is now at turn 40 about to place an order, that information simply vanishes. The model won't know it once "saw" and then "forgot" it — it just behaves as though it was never told.

Truncation has a more hidden trap too, one that ties directly back to the round-trip protocol from the previous course, Agent Tool Calling: Getting Agents to Actually Do Things, Lesson 2 "The Full Round-Trip of a Tool Call": if you truncate by naively slicing to "the last N messages," you can easily cut in the middle of a tool_use / tool_result pair — keeping the assistant message that fired the call but slicing off the tool_result message that came right after it. Send that history to the model and the protocol itself is broken. The docs are explicit that "Tool result blocks must immediately follow their corresponding tool use blocks in the message history."² An error like "tool_use ids were found without tool_result blocks immediately after" is the signal that the pairing is broken² — the model sees that it "started a call" but never gets that call's result, and the next request fails outright.

Compaction: squeeze the window down to one summary

Truncation's problem is that it discards whole stretches. Is there a way to free up space without throwing information away entirely? That's the problem compaction solves. The official Cookbook defines it this way: "Compaction distills the contents of a context window into a high-fidelity summary, letting the agent continue with minimal performance degradation when the conversation gets long."³

Unlike truncation's "delete a whole stretch," compaction is a "rewrite the whole thing": earlier conversation history is compressed into a single high-fidelity summary that replaces the long run of raw messages and stays at the front of the window. What the summary keeps is "what happened and what was concluded"; what it drops is the word-for-word raw dialogue detail.

The docs spell out the parameters of this mechanism. There's a default trigger threshold — compaction fires automatically when window usage hits 150K tokens; the threshold is configurable but can't go below 50K tokens, a server-enforced floor^{3 4}. Each trigger is a discrete replacement: the big stretch of history gets swapped for the summary, and new messages keep appending normally after it. This isn't a one-time event — the docs are explicit that a long conversation can compact more than once, and "The last compaction block reflects the final state of the prompt, replacing content prior to it with the generated summary."⁴ When it compacts again, the earlier compaction block is folded into the new summary along with the rest of history; compaction is a whole-transcript operation where "user messages, assistant messages, tool calls, tool results, even prior compaction blocks are all flattened into the summary."³

Compaction isn't free. The act of compacting costs an extra model call (the summarizer model runs)³, and however carefully written, the summary is a lossy version of the original — "The summary preserves key decisions and facts but may drop specific numbers or exact phrasing."³ If a later step happens to depend on a tiny detail that got summarized away (the exact spelling of some variable, say), that detail may be gone. That's also why compaction fits the coarse-grained problem of "the overall context got too big" rather than serving as a cure-all for every kind of history bloat.

Tool-result clearing: clear only the part that goes stale

A big contributor to history bloat is tool calls themselves. Every time an Agent reads a file or runs a command, the full return value gets stuffed into history — read a file a few thousand lines long and those thousands of lines sit in the messages array as-is, even ten turns later when nobody needs the detail anymore. The Cookbook calls this out directly: "Tool-result clearing addresses the bloat from tool use itself. As an agent pulls in tools and calls them, the results pile up, and deciding how much of that tool output to keep becomes an increasingly important part of managing context."³

Tool-result clearing is the mechanism aimed squarely at this: it "drops old, re-fetchable results while keeping the record that the call happened."³ That's the key distinction — clearing drops the concrete content the tool returned (those few thousand lines of file content), but it doesn't erase the record that "the Agent called read_file with this path." If that content is needed again later, the Agent knows what tool it called and what arguments it passed, and can decide whether to call it again to fetch the content back.

Its trigger threshold and retention policy have clear defaults too: clearing fires when window usage hits 100K tokens, and by default keeps the full results of the most recent 3 tool calls, clearing out older tool results³. The 100K trigger is lower than compaction's 150K, which fits its role — deal first with tool output, the part that "bloats easiest and is easiest to re-fetch," and if that isn't enough, hand the overall window off to compaction.

Choosing among the three: a mental model

We now have two mechanisms, and adding the external memory that Lesson 3 covers makes three. The Cookbook gives a compact mental model that sorts out their division of labor: "compaction compresses the whole window when it grows too large, clearing drops stale re-fetchable data inside the window, and memory moves information out of the window so it survives across sessions."³

Their priorities and use cases aren't competing — they're layered:

• Tool-result clearing handles "this content is still in the window but it's gone stale, and dropping it is fine because it can be re-fetched" — the most targeted, the least costly.
• Compaction handles "the whole window has grown too large," regardless of where the content came from, rewriting it all into one summary — broader reach, but lossy and it costs an extra model call.
• Memory (the next lesson's topic) handles "this information shouldn't only live in this one conversation, it needs to last into the next session" — it isn't solving "the window can't hold everything" at all, but "once this conversation ends, everything in the window disappears."

Back to the question this lesson opened with: history only grows because nobody actively clears it. Truncation, compaction, and tool-result clearing are three ways of clearing it at different costs and for different situations — which one you pick depends on what you want to keep and how much you're willing to pay to keep it.

Recap

• Conversation history only grows by default: each turn's messages pile up in the window, earlier turns are kept in full, and with nobody actively clearing it, it climbs without limit
• Truncation is the simplest, but what it drops is irreversible, and if the cut point lands in the middle of a tool_use / tool_result pair, it breaks the tool call's protocol structure
• Compaction rewrites the whole window's history into one high-fidelity summary, firing by default at 150K tokens (the threshold can't go below 50K, server-enforced), at the cost of being lossy and one extra model call; a long conversation may compact more than once, with earlier summary blocks folded into the new summary^{3 4}
• Tool-result clearing only drops stale, re-fetchable tool output while keeping the call record, firing by default at 100K tokens and keeping the last 3 calls' results — more targeted than compaction
• The three have different jobs: clearing handles stale re-fetchable data, compaction handles an overall window that's too large, memory handles surviving across sessions — which one you pick depends on the specific source of the bloat and whether you can afford to lose detail

>> Lesson 3: External Memory: Files and Retrieval