Long-Running Sessions: Compression, Notes and Memory Files, Subagent Isolation and Handoff Summaries

The craft of compaction: what to keep, what to drop, how to verify

Start with the trigger. The most common mistake is triggering by turn count: compact every 20 turns. That rule will fail, because turn count is not proportional to token count — one tool result outweighs twenty turns of small talk. What to watch is the window-occupancy rate.

An implementation you can use as is:

// Trigger on occupancy, not turn count: one tool result outweighs twenty turns of small talk
export function shouldCompact(turns, window, threshold) {
  return estimateTokens(turnsToText(turns)) / window >= threshold
}
 
// Splitting: the last keepTurns turns must be kept verbatim.
// Summarize the thing currently in progress and the model instantly loses its referent,
// asking "which order do you mean" on the very next line
export function splitKeepAndFold(turns, keepTurns) {
  const cut = Math.max(0, turns.length - keepTurns)
  return { fold: turns.slice(0, cut), keep: turns.slice(cut) }
}

# Trigger on occupancy, not turn count: one tool result outweighs twenty turns of small talk
def should_compact(turns, window: int, threshold: float) -> bool:
    return estimate_tokens(turns_to_text(turns)) / window >= threshold
 
 
# Splitting: the last keep_turns turns must be kept verbatim.
# Summarize the thing currently in progress and the model instantly loses its referent,
# asking "which order do you mean" on the very next line
def split_keep_and_fold(turns, keep_turns: int):
    cut = max(0, len(turns) - keep_turns)
    return {"fold": turns[:cut], "keep": turns[cut:]}

Next, what the summary should contain. A freeform "please summarize the conversation above" produces a passage that reads smoothly and turns an order number into "the relevant order." Identifiers are what a summary loses most easily, because they are linguistically unimportant and engineeringly everything. So the summary prompt has to do two things: section the output, and demand identifiers be kept verbatim by name. Today uses three sections:

• Settled conclusions: facts already confirmed that will not change, listed one per line, keeping the original identifiers such as order numbers, amounts, tracking numbers, and policy clause numbers.
• Open items: what is unfinished and has to be continued.
• Hard constraints: rules that must not be broken, plus any firm requirement the user stated.

Last comes verification, which is the real point of today. The lab presets four facts that must still be findable after compaction and checks each one afterwards. Under the default configuration 2,199 tokens compact to 701, a ratio of 68.1%, with all four surviving; drop the retained turns from 6 to 1 and the ratio rises to 87.3%, but the fact that the user asked for an answer before 18:00 on Friday disappears outright — because it lives only in the verbatim text of the last few turns, and because it does not look like a conclusion linguistically, so a summarizer will not go fishing for it.

Light compaction first: clear tool results before reaching for a summary

Settle one thing before writing any summary code: much of the time you do not need a summary at all.

Yesterday's profile already showed that history is often not the largest of the four pieces. What actually fills the window in a session is usually the pile of accumulated tool results — and they have an advantage a summary does not: old tool results can simply be deleted, because the model read them long ago and already used them. The raw JSON of an order lookup has no value once the model has said "your order has shipped."

That is light compaction: no model call, no semantic understanding, purely positional replacement of old tool results with a placeholder note. It is cheap, deterministic, reversible (the placeholder keeps the ref), and it loses no linguistic information.

Claude's API turns this strategy into a managed capability: with it enabled, input over a configured threshold automatically clears the earliest tool results, triggering by default at a hundred thousand tokens, retaining the last 3 tool calls, and reporting in the response how many were cleared and how many tokens were saved. It also cooperates with the memory tool — before clearing fires, the model first receives a nudge to write anything still needed into a memory file, and only then does the clearing happen.

So the right order is clear tool results first, and summarize history only if that is not enough. Plenty of people do it the other way because summarizing sounds more sophisticated; but a summary costs an extra model call and carries the risk of losing information, and what it treats is usually not the bulk.

There is also a caching trade-off here that few anticipate: clearing breaks the cached prefix. The cleared tool results sit in the middle of the message sequence, so touching them means everything after has to be written into the cache again. Clearing must therefore be neither too frequent nor too small — the managed capability offers a "clear at least this many tokens before acting" parameter precisely to guarantee that the money this clearing saves outweighs the cost of rebuilding the cache. Set that floor in a hand-written strategy too, or you get a program clearing and rebuilding the cache every turn: a pretty token count and a bigger bill.

Notes and memory files: put state outside the window

Compaction solves "it will not fit in the window," but it has a ceiling: the summary is itself in the window, and given a long enough session, the summary of summaries fills up too.

The other route is writing state outside the window. As it works, the model actively writes important information into files — an open-items list, confirmed facts, the next step — and reads it back when needed. That practice has a name, structured note-taking, also called agent memory.

Its division of labor with compaction is clear:

Anthropic published a telling example: a model playing Pokémon kept persistent notes and, thousands of steps in, still maintained exact counts, drew maps, and remembered its long-term strategy — none of which compaction could have preserved, because they are incrementally accumulated state rather than summarizable conclusions. They also turned the capability into a public beta memory tool, letting a model maintain a file-based knowledge base of its own.

The notes route has one more benefit compaction cannot offer: crossing sessions. A compaction's output lives in this session's context and vanishes when the session ends; notes written to a file can be read back next time you start. So for any task done half today and continued tomorrow, notes are not optional but mandatory — otherwise every morning starts by restating yesterday's conclusions, and every restatement loses something.

To decide which route, one question suffices: is this thing's state a conclusion or a ledger? Conclusions can be summarized; a ledger has to go in a file — you cannot summarize a book of transactions into one sentence and still expect it to reconcile.

Subagent isolation: each carries its own suitcase

The third route is more radical: use a different suitcase.

The main agent plans, and when it meets a subtask needing extensive exploration it starts a subagent. The subagent has its own clean context window in which it can burn tens of thousands of tokens searching, trying, and reading files, and when it is done it passes back only a condensed summary to the main thread, typically one or two thousand tokens. The main agent's context gained only those one or two thousand tokens from start to finish, and never saw a word of the tens of thousands in between.

Anthropic's multi-agent research system is built that way: a lead agent sets strategy and dispatches, and several subagents explore in parallel with their own separate context windows. That architecture scored 90.2% higher than a single agent on their internal research evaluation.

But its cost has to be stated plainly, and it is a big number: agent-style applications already use roughly 4 times the tokens of chat, and multi-agent systems roughly 15 times. Subagent isolation does not buy savings; it spends money to keep the main thread's context clean. So its applicability is narrow:

1. The subtask involves heavy exploration but its output condenses to one page — "find every place in this codebase that calls a given endpoint," say.
2. The subtasks are independent and can run in parallel — isolation only makes serial subtasks slower and dearer.
3. The main thread genuinely does not need to see the intermediate process — if the main agent will ask follow-up questions about details, isolation has manufactured an information gap.

Fail any one of the three and use compaction honestly. Do not reach for multi-agent because it sounds more impressive; a 15-times bill is not an abstraction.

What a handoff summary should contain

That one page a subagent passes back determines whether the whole architecture holds up. It is not "summarize what you did" but handoff material that lets the main thread keep making decisions without reviewing the process. The same three sections as compaction suffice, with stricter requirements per section:

// The three sections of a handoff summary: an output contract for the subagent, not a suggestion to it
export const HANDOFF_CONTRACT = `When finished, pass back only these three sections. Do not narrate the process.
 
Settled conclusions: facts you verified that can be relied on directly. Each must carry a source
identifier (file path, order number, URL) so the main thread can re-check it for itself rather than
trusting your paraphrase.
Open items: what you did not finish or lacked permission to do, each stating where it is stuck.
Hard constraints: limits you discovered that the main thread must respect from here on. Write "none"
if there are none; never omit this section.`
 
export function validateHandoff(text) {
  const missing = ['Settled conclusions', 'Open items', 'Hard constraints'].filter(
    (s) => !text.includes(s)
  )
  if (missing.length > 0) throw new Error(`handoff summary is missing sections: ${missing.join(', ')}`)
  return text
}

# The three sections of a handoff summary: an output contract for the subagent, not a suggestion to it
HANDOFF_CONTRACT = """When finished, pass back only these three sections. Do not narrate the process.
 
Settled conclusions: facts you verified that can be relied on directly. Each must carry a source
identifier (file path, order number, URL) so the main thread can re-check it for itself rather than
trusting your paraphrase.
Open items: what you did not finish or lacked permission to do, each stating where it is stuck.
Hard constraints: limits you discovered that the main thread must respect from here on. Write "none"
if there are none; never omit this section."""
 
 
def validate_handoff(text: str) -> str:
    missing = [
        s for s in ("Settled conclusions", "Open items", "Hard constraints") if s not in text
    ]
    if missing:
        raise ValueError("handoff summary is missing sections: " + ", ".join(missing))
    return text

"Each must carry a source identifier" is the most important line in the whole contract. A conclusion without an identifier cannot be re-checked, leaving the main thread to either believe everything or redo everything, both of which are bad. With identifiers, the main thread can spot-verify the one line it doubts, at a controllable cost.

One last easily overlooked point: the main agent's own plan should also be written outside the window. Late in a long task, once the context is truncated or compacted, the first thing lost is often that initial plan — which is exactly the thing that should never be lost. Writing the plan into a file or memory first is insurance too cheap to skip.

Source Reading

Hands-On Lab

This lab's MOCK=1 uses a local extractive summarizer whose shape matches model output, so triggering, splitting, assembling, and checking all genuinely run offline. Put your attention on the fifth criterion: watching a specific fact vanish because of compaction teaches more than reading "compaction loses information" ten times.

1. Get the solution's MOCK=1 pnpm start running and read the occupancy rate, the compression ratio, and the four survival checks.
2. Go back to the starter and complete exercise 1's trigger criterion, verifying it refuses to compact with the threshold at 0.9.
3. Complete exercise 2's splitting so the last 6 turns return verbatim, and watch that final deadline requirement reappear.
4. Complete exercise 3's three-section summary prompt and exercise 4's survival check, confirming the check genuinely verifies rather than being vacuously true.
5. Run once with retained turns at 1, note which fact was lost and why it was that one, then think of a change that would rescue it.

Interview Questions

Today's 3 questions are in the question bank below, weighted toward triggering and verifying compaction, how to write external memory, and the design of subagent isolation and handoff summaries. Expand each one and read the analysis before the key points — practicing the derivation beats memorizing bullets. The high-frequency labels for the domestic and overseas markets are there so you can pick by target market.

Checklist and Tomorrow

• Implement a compression strategy, and verify it with the token count before and after plus a key-information retention check
• Name which tasks compression, notes-and-memory files, and subagent isolation each suit
• Write a handoff summary template, and explain why a subagent should only pass back a summary, not its full process
• Explain why tool results should be cleared before summarizing history is considered
• All 5 acceptance criteria of the lab pass, including watching one fact vanish to compaction
• Answer at least 2 of the 3 interview questions without looking at the key points

Tomorrow (D5) we close the loop: turning these four days' techniques into a measurable cycle. You will work out a task's full token bill (including how caching rewrites it, where a counter-intuitive conclusion is waiting), define two utilization metrics, triage against four failure modes, and finish with a context engineering interview deep dive. The first four days taught techniques; tomorrow teaches which technique when — and when to stop.