Crash Resume, the Budget Circuit Breaker and Governance Decay
Overnight the power drops, the process gets killed, the money burns through, and the rules you set at the start quietly get forgotten. Today adds three things: resuming after a hard kill, halting the moment the budget is blown, and keeping the constraints alive after compaction. That last one is a real risk somebody has already measured.
Today's Goals
- Can explain how resuming from a checkpoint differs from starting over, and say what a checkpoint must record for a resume to avoid redoing work
- Can explain why the budget circuit breaker has to decide during the run, and say how it divides labor with after-the-fact cost measurement
- Can explain what governance decay is, why probing with early facts cannot detect it, and how restating the constraints closes that hole
D5 ended runs cleanly: the harness decided to stop, and it still had a chance to tidy up. Today handles three messier endings. When you finish, scroll back up and tick off the three goals.
Plain-Language Walkthrough
Three things the night shift runs into
Same production line as always. The whiteboard from D2 is up, the handover document from D3 is written, the sign-off sheet from D4 gets signed line by line, and D5 put an inspector on the floor. Now let it actually run through a whole night. What else shows up?
The power drops. Not somebody deciding to stop, but a breaker tripping. No handover, no tidying up, not a word.
The money burns. The machines keep turning and the material keeps going in, but by morning barely anything got built. Cost accrues by the hour and progress accrues by the item, and those two can come apart completely.
The rules fade. Six handovers later, nobody repeats the line about wearing eye protection before cutting. Nobody decided to ignore it. The sentence simply got dropped somewhere along the chain of retellings.
Three things, three modules today. The third is the least visible and the most worth learning, and it has a name: governance decay.
Crash resume needs almost no new persistence
Start with something that may be a surprise: almost nothing new has to go to disk today.
The ledger from D2, the three onboarding artifacts from D3, the feature list and verified commits from D4, the swapped-away set that D5 derives from attempt counts — all of it is already on disk. Every time the previous five days insisted on reading from disk instead of memory, it was laying track for today. The comment in D2 said so at the time:
Only when it is read from disk does this boundary become something a process restart can survive.
Today collects on that. So the hard part is not what to record. It is how to reattach correctly — and the word correctly hides three things that are easy to get wrong.
| The mistake | What it costs | Why it is hard to spot |
|---|---|---|
| Initializing the workspace on a resume too | A whole night of work erased by one delete call | Nothing raises an error. In the log it reads as an ordinary run |
| Treating a dangling in-flight item as finished | The feature list starts lying, or that patch gets written twice | The list looks better than it is |
| Keeping uncommitted changes | You continue from a state that never passed verification | That code looks exactly like good code |
The second one deserves unpacking. A crash can land precisely in the instant between the patch being written and the verification being run, leaving a record on disk that says an item is in flight — it is neither in the finished set nor in any failure record.
This is not dirty data. It is the single most important thing on the disk. It tells the new process: whether that item succeeded is unknown, so confirm it first. Treat it as finished and the feature list starts lying. Treat it as never attempted and the patch already sitting in the file gets written a second time.
The third one follows straight from the D4 rule: commit only what passed verification. So anything uncommitted in the workspace is, by definition, something that did not pass. Keeping it means carrying an unknown state into the next round dressed as a known one.
The measured run: same budget, different arithmetic
The comparison is only fair if the budget for the night is fixed. The steps burned before a crash really were burned, and the model calls were really paid for. So both sides get 24 steps, the first three segments are each killed at step 5, and the last segment gets the remaining 9.
start over after every crash: 9 items finished
resume from disk after a crash: 24 items finishedThe control is not stupid. It simply does not know there is anything on disk to pick up — every time it wakes up it believes it is the first one here.
The budget circuit breaker: three lines, three runaways
The second thing is money. First, a division of labor that is easy to blur.
The evals course measures after the fact: run a batch, then compute pass rates, token counts and cache hit rates to judge whether an agent is any good. That work is offline and retrospective.
This course does live circuit breaking inside a run: how many steps in, how much wall clock burned, how far the rate of verified commits has fallen — cross a line and the run stops right there. That work is online and forward-looking, and its purpose is not to grade anything. Its purpose is to stop the bleeding.
The two share no metric at all, and they should not — however precisely you measure after the fact, it does not buy back the night you already burned.
| Line | What it catches | Why the other lines miss it |
|---|---|---|
| Step ceiling | An infinite loop | Blind to "every step is slow" |
| Wall-clock ceiling | A single step hanging, a slow external dependency | Step count may be nowhere near the limit while the time is gone |
| Yield floor | Working hard and producing nothing | Steps fine, time fine, and not one item passes |
The third is the one worth remembering. The first two mean it ran too long; the third means it ran pointlessly: N steps in, fewer than M verified commits. That is the most insidious way an unattended run burns money — everything appears to be moving, and in the morning the progress bar has not shifted.
Governance decay: the rule was not broken, it disappeared first
The third thing is the subtlest. Start with how it relates to D12 of the hand-built Coding Agent course, which is the most delicate of this course's three comparison points.
That day taught how to verify that compaction lost nothing: take a fact mentioned early, ask about it again after compacting, and if the answer still comes back, the compaction was safe. What that verifies is factual fidelity.
But something else also disappears during compaction, and the probe method is blind to it: constraints.
An agent that can still recite the order number accurately may have long since stopped honoring "never delete a file without confirmation." Remembering is not the same as still listening.
That is governance decay. And inside this course's architecture its mechanism is completely visible, arguably inevitable:
Recall the rule set on D2. At a context window boundary the harness rebuilds an equivalent context holding only three categories: what is finished, what is in flight, how many attempts were made. Constraints are not one of those three, so the rebuild never carries them. They appear in the opening of the first context window and never again.
This is not a bug. It is the price of "lossy compression, but lossless for the decision": when we judged it lossless for the decision, we were only thinking about what to do next. We were not thinking about what not to do next.
The fix is the one the paper proposes, Constraint Pinning: restate the constraints verbatim at the opening of every context window.
no restatement (the default rebuild) 3 -> 0 -> 0 -> 0
restated per window (Constraint Pinning) 3 -> 3 -> 3 -> 3The measured cost of restating is 174 versus 717 characters — a few dozen tokens, in exchange for the rules surviving the night.
Source Reading
Today adds three modules. Two places repay a slow read.
Position one: the resume check has to run before the workspace is initialized.
// Right order: ask the disk whether there is anything to pick up, then decide
// whether to seed at all
const cp = inspectCheckpoint(repoDir, stateDir)
const features = cp.resumable && cp.features !== null
? cp.features // resuming: the list comes off disk, in the same commit as the code
: seedWorkspace(root).features
// Wrong order: seed first, ask afterwards. seedWorkspace deletes work/repo and
// rebuilds it every time, so this line erases a whole night of work -- and raises
// nothing, so in the log it reads as an ordinary run
const seeded = seedWorkspace(root)
const cpTooLate = inspectCheckpoint(repoDir, stateDir)# Right order: ask the disk whether there is anything to pick up, then decide
# whether to seed at all
cp = inspect_checkpoint(repo_dir, state_dir)
features = cp.features if cp.resumable and cp.features is not None else seed_workspace(root).features
# Wrong order: seed first, ask afterwards. seed_workspace deletes work/repo and
# rebuilds it every time, so this line erases a whole night of work -- and raises
# nothing, so in the log it reads as an ordinary run
seeded = seed_workspace(root)
cp_too_late = inspect_checkpoint(repo_dir, state_dir)That swap is today's mutation check. With the order reversed, the resuming run drops from 24 items to 5 — worse than the 9 of the control that cannot resume at all.
The reason is worth working out. The ledger survives, because it lives outside the target repository and initialization does not touch it, and it says fifteen items are finished. The workspace has been wiped clean. The two have diverged. So the harness skips those fifteen and goes to work on the sixteenth, while not one line of them exists in the code.
Position two: the breaker and its grace period.
function checkBudget(state, startedAt, greens, budget, now) {
if (state.spent.steps >= budget.maxSteps) {
return { kind: 'steps', spent: state.spent.steps, limit: budget.maxSteps }
}
if (now - startedAt >= budget.maxWallMs) {
return { kind: 'wall', elapsedMs: now - startedAt, limitMs: budget.maxWallMs }
}
// No yield check inside the grace period: in the first steps it is naturally
// zero, and tripping there means the run can never get started
if (state.spent.steps >= budget.graceSteps) {
const needed = Math.floor(state.spent.steps / budget.stepsPerGreen)
if (greens < needed) return { kind: 'yield', steps: state.spent.steps, greens, needed }
}
return null
}def check_budget(state, started_at, greens, budget, now):
if state.spent.steps >= budget.max_steps:
return Steps(state.spent.steps, budget.max_steps)
if now - started_at >= budget.max_wall_ms:
return Wall(now - started_at, budget.max_wall_ms)
# No yield check inside the grace period: in the first steps it is naturally
# zero, and tripping there means the run can never get started
if state.spent.steps >= budget.grace_steps:
needed = state.spent.steps // budget.steps_per_green
if greens < needed:
return Yield(state.spent.steps, greens, needed)
return NoneNotice that the three lines never stand in for one another: each returns on its own, and they are not folded into one combined score. Folding them would throw away information — what you need to know in the morning is whether the time ran out or whether nothing at all got finished, and those two call for completely different responses.
Hands-On Lab
Four exercises: the resume check and workspace restoration, the three budget lines, constraint rendering and the survival measurement. Every module from the previous five days carries over verbatim.
- Read scripts/run-child.ts first, and see why the resume check has to precede the seed.
- Fill in inspectCheckpoint. All three parts are required: do not re-seed, report the dangling item, discard uncommitted changes.
- Fill in the three lines of checkBudget, and do not forget the grace period on the yield floor.
- Fill in renderConstraints and countConstraints. The numbering markers are not decoration, the measurement reads them.
- Run MOCK=1 pnpm selftest until all 88 self-test assertions are green, then MOCK=1 pnpm start for the three demos.
The mutation check is the order swap above. When you are done, look at the item on the manual checklist that matters most: after the run, the commit history in git log is continuous across four processes, and the hash of an early verified commit is character for character the same before and after all three crashes. That is the hard evidence that nothing was re-initialized.
Interview Questions
Today's four questions circle the correctness of a resume, the division of labor between live circuit breaking and after-the-fact measurement, and one concept that is very easy to answer shallowly: why verifying what an agent remembers cannot tell you whether it still obeys.
The third is worth preparing carefully. Producing the phrase governance decay is only the pass mark. What separates a good answer is explaining precisely why the probe method is blind to it, and volunteering what your own experiment did and did not measure.
Checklist and Tomorrow
- Can explain how resuming from a checkpoint differs from starting over, and say what a checkpoint must record for a resume to avoid redoing work
- Can explain why the budget circuit breaker has to decide during the run, and say how it divides labor with after-the-fact cost measurement
- Can explain what governance decay is, why probing with early facts cannot detect it, and how restating the constraints closes that hole
- Can name the three things a resume most easily gets wrong, and why none of them raises an error
- Can say why a ratio threshold needs a minimum sample size
- Got all 88 self-test assertions green with
MOCK=1 pnpm selftest, and read the numbers from all three demos - Ran the mutation check that moves the seed ahead of the resume check, and can explain why the result is worse than not resuming at all
- Can answer at least three of the four interview questions without looking at the key points
Tomorrow is D7, Putting It Together: Let It Run Through the Night and Check It in the Morning. Six days of modules are now complete, but one thing is still missing: all of this eventually has to be read by a person. Today's circuit breaks, halts and swapped-away items are outcomes sitting in memory, and nobody has been told. Tomorrow writes the overnight report, against a single criterion: one look tells you what got done and where it is stuck. Then the six modules go together for a real long-horizon run, and the course gets its retrospective.
Interview questions
An agent process that had been running for six hours gets killed - how should it come back on the next start?一个跑了六小时的 Agent 进程被杀了。你希望它下次启动时怎么恢复?
Common in ChinaCommon overseasDeep dive#crash-resume#checkpoint#idempotenceHow to reason about it · think before answering
- This question is about the process boundary. 'Serialize the context and load it back next time' misses: that is replay recovery within one session (the territory of D07 in the course where you build a coding agent by hand), and it restores what was said. Here you restore what was done. Every 'read it from disk' of the previous five days was laying track for this moment: once everything at a window boundary comes from disk, that boundary is entitled to become a process restart.
- Start by listing what new things need persisting today - the answer is almost nothing. The ledger, the onboarding trio, the checklist and green points, the skipped set (derived from attempt counts) are already on disk. Today's work is reconnecting them correctly, and all the difficulty lives in that word. All three ways of getting it wrong fail silently.
- Mistake one: never re-seed. Seeding wipes the workspace and rebuilds it, which is right on a fresh run - reproducibility is the whole point - and catastrophic on a resume: one rmSync erases a full night's work, with no error output whatsoever. So the first step of a resume is deciding whether to seed at all, not seeding and then thinking about it.
- Mistake two: a dangling current must be re-verified. The crash can land exactly between 'the patch was written' and 'verification ran', leaving a current pointing at F07 that is neither in the done set nor in any failure record. That is not dirty data, it is the single most important piece of information. Treat it as done and the checklist starts lying; treat it as untouched and a patch already in the files gets written twice. Re-verification is the only correct handling.
- Mistake three: uncommitted changes must be discarded. The rule is commit only after verification passes, so anything uncommitted in the workspace is by definition unverified. Keeping it carries an unknown state into the next round dressed as a known one. Resetting to HEAD is the only correct opening - the same rule as 'a rollback may only target a green point', restated at the process boundary.
- The measurement is worth memorizing: a fixed budget of 24 steps for the night, with the first three segments killed by SIGKILL at step 5. Starting over each time finishes 9 items; resuming from disk finishes 24. One demo discipline goes with it: use SIGKILL, not process.exit(), because the latter runs finally blocks and exit hooks - that is a graceful shutdown, and it demonstrates a path a real crash never takes.
- Expected follow-up: surely half a resume beats none? The mutation test says no. Move the seed ahead of the resume decision, so a resume re-seeds too, and the resuming run drops from 24 items to 5 - worse than the 9 of a run that cannot resume at all. The ledger says the work was done while the workspace is empty, and the two diverge on the spot: the harness carries a ledger claiming N completed items into a workspace holding nothing. Half a resume is more dangerous than none.
分析过程 · 先想清楚再作答
- 这题考的是**进程边界**。答「把上下文序列化存下来、下次原样 load 回去」是跑偏的——那是**同一会话的重放恢复**(手搓 Coding Agent 那门课 D07 的地盘),它还原的是「说过什么」;这里要还原的是「做成了什么」。前五天每一次「从磁盘读」都是在为这一刻铺路:窗口边界上的东西**只要全部从磁盘读**,它就有资格升级成一次进程重启。
- 拆的第一步是盘点今天要新增哪些需要持久化的东西,答案是**几乎没有**。账本、上手三件套、清单与绿点、被换掉的集合(从尝试次数推导)早就都在磁盘上了。今天做的只是把它们**正确地**接回来——难的全在「正确」这两个字里,而且这三件做错了都不会报错。
- 第一件容易做错的:**不许重新 seed**。seed 每次都先把工作区删干净再重建,从头跑时这是对的(反复可重建是它的价值),续跑时它是灾难——一行 rmSync 抹掉一整夜的工作,而且没有任何错误输出。所以续跑的第一步是**判断该不该 seed**,不是先 seed 再说。
- 第二件:**悬空的 current 必须重验**。崩溃可能正好发生在「补丁写进去了、验证还没跑」那一瞬,磁盘上会留下一条 current 指向 F07,它既不在已完成集合里、也没有失败记录。**这不是脏数据,它是最重要的一条信息**:当成做完了,清单就开始说谎;当成没做过,那段已经写进文件的补丁会被再写一遍。正确的处理只有重验。
- 第三件:**未提交的改动必须丢掉**。规矩是通过验证才提交,所以工作区里任何未提交的东西**按定义就是没通过验证的**。把它留着等于把一个未知状态当成已知状态带进下一轮,reset 到 HEAD 是唯一正确的开场——这就是「回退的目标只能是绿点」那条规矩在进程边界上的同一句话。
- 实测值得背下来:一夜固定 24 步预算,前三段各在第 5 步被 SIGKILL。**每次从头重来最终完成 9 条,从磁盘接着做完成 24 条。** 顺带一个演示纪律:**用 SIGKILL 而不是 process.exit()**,后者会跑 finally 与退出钩子,那是优雅退出,演示不出崩溃——你测的会是一条根本不会发生的路径。
- 可预期的追问是「做一半的续跑总比没有强吧」。变异实测说不:把 seed 挪到续跑判断之前(即续跑时也重新 seed),续跑组从 24 条掉到 **5 条**,**比完全不会续跑的 9 条还差**。因为账本说做过了、工作区却是空的,两者当场分叉——harness 拿着一份声称完成了 N 条的账本去一个什么都没有的工作区上接着做。**半套续跑比没有续跑更危险。**
Key points
- Restore what was done, not what was said - this is not replay recovery inside one session.
- The window boundary becomes a process boundary: everything was already on disk, so almost nothing new is persisted.
- Never re-seed: seeding wipes the workspace, and on a resume one rmSync erases the night without an error.
- A dangling current must be re-verified: neither in done nor in any failure record, it is the key piece of information.
- Discard uncommitted changes: commit follows verification, so uncommitted means unverified by definition.
- Measured: 24-step budget, three SIGKILLs - starting over finishes 9 items, resuming from disk finishes 24.
- Mutation: re-seeding on resume drops 24 to 5, worse than the 9 of no resume at all.
答题要点
- 要恢复的是「做成了什么」,不是「说过什么」——与同一会话的重放恢复不是一回事。
- 窗口边界升级成进程边界:前几天已经把该落盘的都落了盘,今天几乎不新增持久化的东西。
- 不许重新 seed:seed 会先删工作区,续跑时一行 rmSync 抹掉一整夜且不报错。
- 悬空的 current 必须重验:它既不在已完成里也没有失败记录,是最重要的一条信息。
- 未提交的改动必须丢掉:通过验证才提交,所以没提交的按定义就是没通过验证的。
- 实测:24 步固定预算、三次 SIGKILL,从头重来 9 条,从磁盘接着做 24 条。
- 变异实测:续跑时也重新 seed,24 条掉到 5 条,比不会续跑的 9 条还差——半套比没有更危险。
Are a budget circuit breaker and a post-hoc cost report two different things, and what does each solve?预算熔断和事后成本报表是两件事吗?分别解决什么问题?
Common in ChinaCommon overseasIntermediate#budget#circuit-breaker#real-time-controlHow to reason about it · think before answering
- This is about the line between stopping losses online and judging quality offline. 'Both control cost' scores zero. Post-hoc measurement is offline and retrospective: after a batch finishes you compute pass rates, tokens and cache hit rates to judge whether an agent is any good - that belongs to the evaluation course. A circuit breaker is online and forward-looking, and its purpose is not judgment but stopping the bleeding. They share no metric, and should not.
- One line nails the split: however precisely you compute a post-hoc metric, it cannot recover a night already burned. The reverse holds too - the breaker's three numbers say nothing about quality, only how long it ran and how much it spent. One decides whether to stop, the other whether to change anything.
- Break it open by asking what decision a number feeds and when it must be in hand. A breaker's inputs must be computable on the spot, so it prefers coarse; post-hoc metrics can wait for the whole batch, so they can be precise. Putting a metric that only exists after the run into a breaker's criterion means it will never fire.
- The three lines each guard against a different runaway. A step ceiling guards against infinite loops but is blind to every step being slow. A wall-clock ceiling guards against a stuck step or a slowing dependency, because the step count may be nowhere near its limit while the time is gone. A yield floor guards against the third and most hidden kind.
- That third one deserves elaboration: N steps spent with fewer than M green points means the agent can work but produces nothing. The first two watch for running too long, the third for running pointlessly - everything looks busy and in the morning the progress bar has not moved. Only this line watches output rather than consumption, which is exactly why the other two cannot replace it.
- The yield line needs a warm-up period: in the first few steps of a run the yield is naturally zero because nothing has finished verification yet, and tripping there means the run can never start. Promote that into a portable conclusion: any ratio-based threshold needs a minimum sample size. Saying that sentence in an interview is worth more than reciting all three lines.
- Expected follow-up: how do you set the thresholds? Not by guessing. Decide what a night is worth in money and hours, convert that into wall-clock and step ceilings, and derive the yield floor from the green-point density of a healthy run with a margin. If you cannot derive them, you have not decided what the night is worth. Also, the breaker must halt at a consistent point and record which line tripped - the three causes point at completely different investigations, and the next resume starts exactly there.
分析过程 · 先想清楚再作答
- 这题考的是**在线止损与离线评价的边界**。答「都是控成本」拿不到分。事后度量是**离线的、回顾性的**,一批运行跑完之后算通过率、算 token、算缓存命中率,用来判断一个 Agent 好不好,那是评估那门课的地盘;熔断是**在线的、前瞻性的**,唯一目的不是评价而是**止损**。两者一个指标都不共享,也不该共享。
- 一句话把分工钉死:**事后指标算得再准,也救不回已经烧掉的一夜。** 反过来也成立——熔断那三个数字拿去做质量评价毫无意义,它们只说明这次跑了多久、烧了多少,不说明结果好不好。一个管**停不停**,一个管**改不改**。
- 拆法是问「这个数字用来做什么决定、什么时候必须拿到」。熔断要的量必须**当场就能算出来**,所以它宁可粗糙;事后指标可以等整批跑完再慢慢算,所以它可以精确。把一个要跑完才有的指标塞进熔断判据,等于永远不会熔断。
- 熔断的三条线各防一种失控:**步数上限**防无限循环,但它对「每步都很慢」完全无感;**墙钟上限**防单步卡死与外部依赖变慢,因为步数可能还早得很、时间已经烧光;**产出率下限**防的是第三种,也是最隐蔽的一种。
- 第三条最值得展开:已经跑了 N 步、绿点却少于 M 个,这是「**干得动但干不出东西**」。前两条盯的是「跑太久」,第三条盯的是「跑得没意义」——一切看起来都在动,早上打开一看进度条没挪。只有它盯的是**产出**而不是消耗,这也是它不可被前两条替代的理由。
- 产出率那条必须带**热身期**:一次运行的头几步里产出率天然是 0(第一条还没验完),此时熔断等于永远跑不起来。这条可以直接升级成一句通用结论带走:**任何基于比率的阈值都需要一个最小样本量。** 面试里说出这一句,比把三条线背全更值钱。
- 可预期的追问是「阈值怎么定」。不是拍脑袋:先定这一夜愿意烧掉的钱与时间,折算成墙钟与步数上限,产出率那条按正常运行的绿点密度打个折。定不出来说明还没想清楚这一夜值多少钱。另外熔断必须**停在一个一致的点上**并写清是哪条线到线——三种原因对应的排查方向完全不同,而下一次续跑接的就是这个点。
Key points
- One decides whether to stop (online loss control), the other whether to change (offline judgment); no shared metrics.
- However precise a post-hoc metric is, it cannot recover a night already burned.
- The criterion is what decision the number feeds and when: a breaker prefers coarse but computable on the spot.
- Three lines: steps for infinite loops, wall clock for stuck steps, yield for working without producing.
- The yield line watches output rather than consumption - the only defense against a busy-looking night with no progress.
- Ratio thresholds need a warm-up: any ratio-based threshold needs a minimum sample size.
- Derive thresholds from what the night is worth; halt at a consistent point and record which line tripped.
答题要点
- 一个管停不停(在线止损),一个管改不改(离线评价),一个指标都不共享。
- 事后指标算得再准,也救不回已经烧掉的一夜。
- 判据是这个数字用来做什么决定、什么时候必须拿到:熔断宁可粗糙也要当场能算。
- 三条线:步数防无限循环、墙钟防单步卡死、产出率防干得动但干不出东西。
- 产出率那条盯的是产出不是消耗,它是「一切都在动、进度条没挪」的唯一解。
- 比率阈值必须有热身期——任何基于比率的阈值都需要一个最小样本量。
- 阈值从「这一夜值多少钱」倒推;熔断必须停在一致点上并写清是哪条线到线。
Repeated context compaction makes an agent drift away from its original safety constraints - how would you defend against that?反复压缩上下文会让 Agent 逐渐不遵守最初的安全约束。你会怎么防?
Common in ChinaCommon overseasDeep dive#governance-decay#constraint-pinning#compactionHow to reason about it · think before answering
- First establish that you know this is a phenomenon already quantified by primary research, not an imagined risk: it has a name, governance decay. Before answering 'put the constraints in the system prompt', answer this - at the moment the context is rebuilt, is that copy of the constraints still there? Without the mechanism, any defense is a slogan.
- The mechanism is architecturally inevitable, not a bug. At a window boundary you rebuild an equivalent context that keeps only three categories: what is done, what is in progress, how many attempts were made. Constraints are in none of them, so the rebuild naturally leaves them out - they appear in the first window's opening and never again.
- This is the price of 'lossy compression that loses nothing decision-relevant': when that judgment was made, only what to do next was considered, never what must not be done next. The first is state, the second is policy, and a compaction that keeps only state necessarily drops policy. Saying that out loud is worth far more than naming the phenomenon.
- The defense is the paper's Constraint Pinning: restate the constraints verbatim at the opening of every window. It is almost free - measured here, that block is 174 of 717 characters in the opening - and it buys a night in which the rules stay in force. Note what it defends against: constraints disappearing, not a model resisting them. The next question separates those two.
- It needs a measurement, not just a claim that you restate them. The metric used here is how many constraints survive in each window's opening: without pinning it is 3, 0, 0, 0; with pinning it is 3, 3, 3, 3. With that number, 'is the restatement actually wired in' becomes an automatically assertable fact instead of an unverified good intention.
- Which constraints to pin also matters: prefer the rules whose violation does not fail loudly. A rule that fails immediately gets corrected by reality anyway - the next step simply will not work - while the quiet ones are exactly what compaction erases and what is hardest to diagnose afterwards. All three used here (one feature at a time, append only without rewriting existing registration code, never hand-edit the checklist) are of that kind.
- Close with the capability boundary, volunteered rather than extracted: a single-purpose offline model neither sees constraints nor violates them, so the lab here can measure whether the constraints are still present, not whether the model still obeys. The latter is only cited, and always with its condition: the ConstraintRot benchmark, 1323 episodes across 7 model families, violation rates rising from 0 percent with the policy fully visible to 30 percent after compaction, reaching 59 percent in some families, with Constraint Pinning pushing it back to 0 percent. Quoting 30 percent without 'after compaction' is simply wrong.
分析过程 · 先想清楚再作答
- 这题第一步是确认你知道这是一个**已经被一手论文量化过的现象**而不是想象出来的风险,它有名字:**governance decay**。答「把约束写进系统提示词就完事了」之前要先回答一个问题——重建上下文的那一刻,那份约束到底还在不在。机制没说清,任何防法都只是一句口号。
- 机制在架构里是**必然的**,不是 bug。窗口边界处重建的是**等效上下文**,只保留「做完了什么、正在做什么、试了几次」三类。约束不在这三类里,所以重建天然不会带上它——它只出现在第一个窗口的开场,之后再也没有出现过。
- 它是「有损压缩,但对决策无损」那句话的代价:当时判断「对决策无损」时,只考虑了**下一步做什么**,没考虑**下一步不许做什么**。前者是状态,后者是策略,而一个只保留状态的压缩必然丢掉策略。把这层说出来,比背出现象名字有用得多。
- 防法就是论文里的 **Constraint Pinning**:**每个窗口开场都把约束原样复述一遍。** 它便宜得几乎不用算——本课实测那段只占开场 **174/717 字符**,换的是整晚的规矩不失效。注意它防的是「约束消失」,不是「模型抗拒」,这两件事下一题会分开讲。
- 必须**配一个度量**,不能只写一句「我们复述了」。本课的口径是数各窗口开场里还剩几条约束:不复述是 **3 → 0 → 0 → 0**,每窗口复述是 **3 → 3 → 3 → 3**。有了这个数字,「复述有没有真的接上」就变成一个能自动断言的事实,而不是一段谁也没验过的好意。
- 选哪几条进复述也有讲究:优先选**违反了不会立刻报错**的那种规矩。会立刻报错的规矩不靠复述也会被现实纠正(下一步就跑不通了),不报错的那些才是压缩里悄悄失效、失效之后最难查的一类——本课那三条(一次只做一条、只追加不改写已有注册代码、不手改清单)全是这一类。
- 能力边界必须主动说:离线的单用途模型**看不见约束也不会违规**,所以自己的 lab 能测的是「**约束还在不在**」,测不到「**模型还听不听**」。后者只引用一手证据,而且引用时必须带条件:ConstraintRot 基准 **1323 个 episode、7 个模型家族**,违规率从策略完整可见时的 **0%** 升到**压缩后**的 **30%**,某些家族达 **59%**,Constraint Pinning 压回 **0%**。脱离「压缩后」这个条件单说 30% 是错的。
Key points
- The phenomenon has a name - governance decay, quantified by primary research; explain the mechanism before the defense.
- The mechanism is architectural: the rebuild keeps done, in progress and attempt counts; constraints are in none of them.
- 'Nothing decision-relevant lost' only considered what to do next, never what must not be done - policy is dropped, not state.
- The defense is Constraint Pinning: restate verbatim at each window opening, measured at 174 of 717 characters.
- Pair it with a metric: 3, 0, 0, 0 without pinning versus 3, 3, 3, 3 with it.
- Pin the rules that fail quietly; the ones that fail loudly get corrected by reality anyway.
- Capability boundary: you can measure presence, not obedience; always cite the paper's numbers with the 'after compaction' condition.
答题要点
- 现象有名字:governance decay,已被一手论文量化,先说机制再说防法。
- 机制是架构必然:重建只保留做完了什么、正在做什么、试了几次,约束不在这三类里。
- 「对决策无损」当初只考虑了下一步做什么,没考虑下一步不许做什么——丢的是策略不是状态。
- 防法是 Constraint Pinning:每个窗口开场原样复述,实测只占开场 174/717 字符。
- 必须配度量:不复述是 3 → 0 → 0 → 0,每窗口复述是 3 → 3 → 3 → 3。
- 优先复述那些违反了不会立刻报错的规矩,会立刻报错的靠现实就能纠正。
- 能力边界:自己能测「约束还在不在」,测不到「模型还听不听」;引用论文数字必须带「压缩后」这个条件。
You already have probes that check factual fidelity after compaction - why do they miss constraints being erased?你已经有一套压缩后的事实校验探针了,为什么它发现不了约束被擦掉?
Common in ChinaCommon overseasDeep dive#probe-blind-spot#constraint-fidelity#capability-boundaryHow to reason about it · think before answering
- This question is about two kinds of fidelity, and it exists to filter for people who know what their checks actually cover. The probe method (D12 of the course where you build a coding agent by hand) takes facts mentioned early, asks about them again after compaction, and treats a correct answer as proof that compaction was safe. That is factual fidelity - whether it remembers. Constraint fidelity is a different thing: whether it still complies.
- One line makes it plain: an agent that can still recite the order number perfectly may well have stopped obeying 'never delete a file without confirmation'. Remembering is not the same as heeding. All of the probe method's evidence lands on the first half, and it is structurally blind to the second - not blind for lack of coverage, not fixable by adding more probes.
- Why structurally: a probe asks whether a piece of information is still in the context, while constraints fail along two paths - the text is still there but its weight has dropped, or, as here, the rebuild never carried it at all. The first is entirely invisible to probes (the model answers beautifully and violates anyway); the second is in principle catchable, but nobody thinks to use a rule as a factual probe.
- There is a further layer people miss: probes sample. Facts are homogeneous, so a few representative ones suffice. Constraints are not homogeneous - each is irreplaceable, and finding C1 intact says nothing about C2. Using a sampling method on an object that cannot be sampled fails as a method, regardless of how well the individual probes are written.
- The fix is cheap and points the right way: treat constraints as first-class and count them directly, checking id by id how many survive in each window's opening. What matters is that it does not ask the model - it counts them in the context text itself. This is the same principle as the self-verification day: a check that does not depend on the cooperation of the party under review is the only trustworthy check.
- Volunteer the ceiling of that fix too: it proves the constraints are still in the context, not that the model still obeys them. Obedience needs a behavioural metric - a violation rate - and that needs a model that genuinely violates. The offline model used here is single-purpose: it neither sees constraints nor breaks them, so that layer only cites the primary numbers (0 percent rising to 30 percent after compaction, 59 percent in some families, Constraint Pinning back to 0 percent) rather than manufacturing its own.
- Expected follow-up: why not turn the constraints into probes as well? You can, but the criterion has to change. A factual probe passes on a correct answer; a constraint probe must require a verbatim restatement plus a refusal in a scenario engineered to invite the violation. Asking 'do you remember the rules?' still yields a memory test - and memory is precisely the thing already shown to be insufficient.
分析过程 · 先想清楚再作答
- 这题考的是**两种保真的区别**,而且它是一道专门用来筛「知道自己验了什么」的题。探针法(手搓 Coding Agent 那门课 D12 的做法)是拿早期提过的事实当探针、压完再问一遍,答得上来就说明压缩安全。它验的是**事实保真**——记不记得。约束保真是另一回事:**还听不听得进。**
- 一句话就能点破:**一个还能准确复述订单号的 Agent,完全可能已经不再遵守「未经确认不许删文件」。记得住不等于听得进。** 探针法的全部证据都落在前半句上,它对后半句是**结构性**的盲,不是覆盖率不够、多加几条探针就能补上的那种盲。
- 为什么是结构性的:探针问的是「这条信息还在不在上下文里」,而约束失效有两条路径——信息还在但权重掉了,以及像本课这样在重建时根本就没被带上。前一条探针完全抓不到(它会答得很好然后照样违规);后一条探针原则上抓得到,但没有人会想到拿一条规矩去当事实探针问。
- 还有一层常被忽略的:**探针是抽样的**。事实之间是同质的,抽几条有代表性的就够了;**约束不是同质的**——每一条都不可替代,抽到 C1 还在推不出 C2 还在。用一个抽样的方法去验一个不可抽样的对象,方法本身就不成立,与探针写得好不好无关。
- 补法很便宜,而且方向是对的:把约束当成**一等公民**直接数——按 id 逐条核对各窗口开场里还剩几条。关键在于它**不问模型**,是在上下文文本里自己数。这与自验证那天是同一条原则:**不依赖被审查方配合的检查,才是可信的检查。**
- 但要主动说清这个补法的**上限**:它证明的是「约束还在上下文里」,不是「模型还在遵守」。后者要用行为指标(违规率)去量,而那需要一个真会违规的模型;本课的离线模型是单用途的,看不见约束也不会违规,所以这一层只引用一手论文的数字(**压缩后** 0% 升到 30%、某些家族 59%、Constraint Pinning 压回 0%),不自己造数据。
- 可预期的追问是「那把约束也做成探针不就行了」。可以,但**判据必须换**:事实探针的判据是「答得上来」,约束探针的判据得是「**一字不差地复述**」外加「**在一个会诱导违规的场景里仍然拒绝**」。只问一句「你还记得有哪些规矩吗」,得到的仍然是一个记忆测试——而记忆正是那个已经被证明不够的东西。
Key points
- Probes verify factual fidelity (does it remember); constraint fidelity is another thing (does it still comply).
- An agent that still recites the order number perfectly may have stopped obeying 'no deletion without confirmation'.
- The blind spot is structural: text can survive with lost weight, or never be carried by the rebuild at all.
- Facts can be sampled, constraints cannot: each is irreplaceable, and C1 surviving says nothing about C2.
- The fix is counting constraints by id in the context and not asking the model - checks that need no cooperation are the trustworthy ones.
- Its ceiling: it proves presence, not obedience; obedience is only cited from primary research.
- Constraint probes need a different criterion: verbatim restatement plus refusal under an inviting scenario, not 'do you remember'.
答题要点
- 探针法验的是事实保真(记不记得),约束保真是另一回事(还听不听得进)。
- 一个还能准确复述订单号的 Agent,完全可能已经不再遵守「未经确认不许删文件」。
- 盲区是结构性的:约束可以信息还在而权重掉了,也可以在重建时根本没被带上。
- 事实可以抽样,约束不可以:每条约束不可替代,抽到 C1 还在推不出 C2 还在。
- 补法是按 id 逐条数上下文里还剩几条约束,而且不问模型——不依赖被审查方配合的检查才可信。
- 补法的上限:它证明约束还在,不证明模型还在遵守;后者只引用一手论文数字。
- 把约束做成探针要换判据:一字不差复述 + 在诱导场景里仍然拒绝,而不是「你还记得吗」。