Cost and Model Routing: Choosing a Model per Stage, Caching, Degradation, and a Budget Circuit Breaker

Tiered routing: a draft pass and a final pass are not the same job

Do you deploy to production on every line of code you change? Of course not — run locally, run CI, then ship. Generated content is the same, and yet many people run every experiment at the most expensive tier.

Tiered routing means the same business code selects different models and specs according to what this run is for. This line has at least two tiers. The draft tier is for revising the script, adjusting the storyboard, trying camera moves — you are judging whether the narrative flows, and softer picture does not affect that. The final tier is the version that ships, where picture quality, duration and look tests can none of them be skipped.

The key is that the switching criterion lives in the code, not in someone's head: script review passed (D2), human review passed (D10), and final output explicitly requested — all three before the final tier is used.

What differs between tiers? The lab's two configurations:

The measured comparison: draft CNY 4.58, final CNY 9.13, 99% more expensive. So every extra attempt at the draft stage saves half the money; if an episode takes five attempts to settle, tiering alone saves twenty-odd yuan, and a five-episode season saves over a hundred.

There is a trap here that must be admitted, and it happens to be the best teaching material of the section. The official tiers are 768P at CNY 0.50 per second, 2K at 0.80, 480P at 0.33 — and our draft tier requests 512P, a tier with no corresponding rate. What now?

Estimate on the expensive side. The lab bills 512P at the 768P rate, flags that ledger row with rateApprox, and prints the flag on the dashboard verbatim. The rule sounds conservative, but the worst outcome of over-estimating is a few fewer attempts, while the worst outcome of under-estimating is a doubled bill at month end with no idea why.

Estimating high has a consequence: 512P and 768P cost the same on paper, so lowering the resolution saves literally nothing. What actually saves money in the draft tier is the other two items — three-second shots and no look tests. Which is the reminder: degrade against the books, not against intuition.

Caching: the most profitable few lines on this line

The second route is caching. Re-run after changing one shot's visual description and the bill drops from CNY 9.13 to CNY 3.03 — 67% saved for a few dozen lines of code.

Why does this line suit caching so well? Because the artifacts are expensive and deterministic: the same prompt, reference image, duration and resolution regenerate into something indistinguishable, at full price again. For a chatbot, a different answer each time is a feature; here, a different regeneration each time is mostly a nuisance.

Designing the cache key was D8's subject (the workflow engine's idempotency key, one key governing both "skip finished nodes" and "hit the cache"). Today covers two things D8 did not.

First: the saving is layered. Re-running the same episode prints "cache hits 11 / 11, new spend CNY 0.0000"; changing one line or one shot's visual description prints "cache hits 9 / 11" — only the edited shot's first frame and video are recomputed and everything else is reused. That is why cache granularity must be per shot and not per episode: one notch coarser and one changed character re-runs a whole episode, making the cache pointless.

Second: a cache hit is not free. Cached files occupy disk, video especially; and the first run after clearing the cache spends everything at once. So the dashboard prints hit rows too, at zero cost, plus a separate line reading "saved by cache (estimated) CNY 9.13" — you must see both what this run spent and what it would have spent.

Degradation is not failure: three degradable dimensions

The third route is degradation. Its difference from failure is clear: failure means getting nothing, degradation means getting something lesser that is still usable. A video call dropping resolution on a weak signal rather than hanging up is the same idea.

This line has three degradable dimensions, ordered from least to most noticeable to the audience:

1. Resolution: 768P down to 512P. Watching vertical short drama on a phone, most viewers will not notice.
2. Duration: six seconds per shot down to four. The pace quickens, and the story survives.
3. Shot count: five shots down to three. This one already alters the narrative and is the last resort.

The correct shape of degradation is not "run out of money halfway and cut the remaining shots" but compute before starting, degrade if it does not fit, then run: the former leaves half a wasted episode, the latter produces one complete episode that is merely lesser.

// Three degradable dimensions, ordered from least to most noticeable to the audience
const STEPS = [
  { label: 'resolution 768P -> 512P', apply: (p) => ({ ...p, resolution: '512P' }) },
  { label: 'shot length 6s -> 4s', apply: (p) => ({ ...p, maxShotSeconds: 4 }) },
  { label: 'shot count: keep the first 2', apply: (p) => ({ ...p, maxShots: 2 }) },
]
 
// project is a pure function: given a plan it computes projected spend, sending no requests
function degradeToFit(plan, project, limitCny) {
  let cur = plan
  const applied = []
  const skipped = []
  let projected = project(cur)
  for (const step of STEPS) {
    if (projected <= limitCny) break
    const next = step.apply(cur)
    if (project(next) >= projected) {
      // This step saves nothing on the books, so degrading only loses quality
      skipped.push(step.label)
      continue
    }
    cur = next
    applied.push(step.label)
    projected = project(cur)
  }
  return { plan: cur, applied, skipped, projected, fits: projected <= limitCny }
}

# Three degradable dimensions, ordered from least to most noticeable to the audience
STEPS = [
    ("resolution 768P -> 512P", lambda p: {**p, "resolution": "512P"}),
    ("shot length 6s -> 4s", lambda p: {**p, "max_shot_seconds": 4}),
    ("shot count: keep the first 2", lambda p: {**p, "max_shots": 2}),
]
 
 
def degrade_to_fit(plan, project, limit_cny):
    """project is a pure function: given a plan it computes spend, sending no requests."""
    cur, applied, skipped = plan, [], []
    projected = project(cur)
    for label, apply in STEPS:
        if projected <= limit_cny:
            break
        nxt = apply(cur)
        if project(nxt) >= projected:
            # This step saves nothing on the books, so degrading only loses quality
            skipped.append(label)
            continue
        cur, projected = nxt, project(nxt)
        applied.append(label)
    return {
        "plan": cur,
        "applied": applied,
        "skipped": skipped,
        "projected": projected,
        "fits": projected <= limit_cny,
    }

Note the middle "skip if it saves nothing" logic. That is not defensive code, it is a genuine criterion: 512P costs the same as 768P in our rate table, so the resolution step is skipped automatically — every degradation step must be validated against the books, and skipped if it fails validation. The log prints adopted and skipped steps separately with reasons.

The budget circuit breaker: a soft limit warns, a hard limit stops

The first three routes save money; this last one is the backstop — no amount of saving stops one runaway retry storm from sending the bill to the moon.

A circuit breaker's core semantics fit in one sentence: decide before spending, not tally afterwards. Written as an after-the-fact tally, the best you get is "this run overspent by 40%" — the money is gone, and that log line does nothing but make you unhappy. The correct shape is reservation: before calling a paid API, deduct the projected amount from the budget; if the deduction succeeds, send the request, and if not, raise.

class BudgetExceededError extends Error {}
 
class Budget {
  #spent = 0
  #softFired = false
  constructor({ softCny, hardCny, onSoft }) {
    Object.assign(this, { softCny, hardCny, onSoft })
  }
 
  // Call before spending. If the deduction fails, raise; the caller stops safely.
  reserve(stage, amountCny) {
    if (this.#spent + amountCny > this.hardCny) {
      throw new BudgetExceededError(
        `hard limit tripped: spent ${this.#spent.toFixed(4)}, ${stage} needs ${amountCny.toFixed(4)}`
      )
    }
    this.#spent += amountCny
    if (!this.#softFired && this.#spent > this.softCny) {
      this.#softFired = true
      this.onSoft?.(this.#spent) // warn once, do not flood
    }
  }
 
  // A failed or safety-blocked video is not billed, so the reservation is refunded
  refund(amountCny) {
    this.#spent = Math.max(0, this.#spent - amountCny)
  }
}

class BudgetExceededError(Exception):
    pass
 
 
class Budget:
    def __init__(self, soft_cny, hard_cny, on_soft=None):
        self.soft_cny, self.hard_cny, self.on_soft = soft_cny, hard_cny, on_soft
        self._spent = 0.0
        self._soft_fired = False
 
    def reserve(self, stage: str, amount_cny: float) -> None:
        """Call before spending. If the deduction fails, raise; the caller stops safely."""
        if self._spent + amount_cny > self.hard_cny:
            raise BudgetExceededError(
                f"hard limit tripped: spent {self._spent:.4f}, {stage} needs {amount_cny:.4f}"
            )
        self._spent += amount_cny
        if not self._soft_fired and self._spent > self.soft_cny:
            self._soft_fired = True
            if self.on_soft:
                self.on_soft(self._spent)  # warn once, do not flood
 
    def refund(self, amount_cny: float) -> None:
        """A failed or safety-blocked video is not billed, so refund the reservation."""
        self._spent = max(0.0, self._spent - amount_cny)

The two limits have different jobs. The soft limit warns and changes nothing: one alert on crossing is enough, so that you can decide while you still have room — keep going, or drop a tier by hand. The hard limit stops, and must really stop.

And inside "stops" hides this chapter's most important judgment: what counts as stopping safely?

Not process.exit(1). That discards the ledger and the progress file, so the next run starts from scratch and spends the already-spent money again — turning the breaker into an amplifier of waste. Safe stopping has three parts:

1. Finished artifacts stay on disk, none deleted;
2. The ledger is written: how much was spent, where it stopped, what is still missing, all persisted;
3. The cache is written too, so a later run with a higher budget need not redo the finished parts.

When the lab reaches this point the log looks like this:

(5) Squeeze the hard limit to CNY 3, clear the cache, run the final tier (INJECT=overbudget)
  STOP hard limit tripped: spent CNY 0.1250, clips needs CNY 3.0000, over the CNY 3.0000 hard limit; this run ends here
  5 finished artifacts remain on disk and the cache is written: a later run with a higher budget need not redo them.

Both the breaker and the safety-block scenario are fault injections, off by default and enabled through the INJECT environment variable — the one switch this course uses everywhere it needs a deliberate failure, taking comma-separated scenario names. D12 supports two: overbudget trips the hard limit, and blocked makes one shot's first video generation hit content moderation.

One more easily missed rule: a failed or safety-blocked video is not billed. So the breaker's reservation must be refunded, or one moderation block wastes three yuan of headroom that never appears as spend on the bill. In the lab that is the refund method plus the ledger's billed: false record, with a dedicated dashboard line for "failed or blocked, not billed."

The cost dashboard: see at a glance what made this run expensive

A dashboard is not a report. Reports are for month end; a dashboard is for this run, and answers one question: where did the money go this time, and is it more or less than last time?

It presents two dimensions: by stage (assets / frames / clips / voice) and by provider and type. The first tells you which part of the flow to optimize; the second tells you which vendor to negotiate with, or which class of call to move elsewhere.

A finished dashboard looks like this:

-- Cost dashboard: final tier (first run) --
  by stage
    clips    CNY  9.0000   98.5%  ########################  4 calls (0 cache hits)
    frames   CNY  0.0750    0.8%  ........................  3 calls (0 cache hits)
    assets   CNY  0.0500    0.5%  ........................  2 calls (0 cache hits)
    voice    CNY  0.0096    0.1%  ........................  3 calls (0 cache hits)
  total (estimated) CNY 9.1346
  failed or blocked, not billed (estimated) CNY 3.0000: failed or safety-blocked video is not billed

Three design points: every row carries call count and cache hits, because "how much was spent" and "how many calls" are different things — a failed retry raises the count without raising the amount; percentages and a bar together, because people read bars far faster than numbers; and two fixed disclaimer lines at the bottom saying the amounts are estimates and cannot be reconciled against an invoice, naming which tiers are derived and which are officially published.

One last line: a dashboard exists to make a decision, not to be archived. If reading one leaves you unsure what to change next, it was wasted. On this line the dashboard always points at the same conclusion — ninety percent of the money is in video, and the next step is always how to call the video API one fewer time.

Source Reading

Hands-On Lab

Acceptance takes two commands: MOCK=1 pnpm start for the happy path, and INJECT=overbudget,blocked MOCK=1 pnpm start to inject both the breaker and the safety block. Confirm ffmpeg is installed first (a missing one gives a readable error at startup), and remember every amount on the dashboard is an estimate — offline, the provider's returned cost is always 0. If you get stuck, look at the "cache hits x / y" line first; it locates about ninety percent of this lab's problems.

1. Run the starter unmodified first, read all six stages' output end to end, and note the obviously wrong symptoms (an all-zero dashboard, a cache that never hits, a breaker that never trips, degradation that never degrades).
2. Add the projection function and the scoring rules: make project compute an episode's cost without sending any request, then check that stage 6 starts printing degradation steps.
3. Implement cache hits, re-run stages 3 and 4, and confirm hits go from 0 / 11 to 11 / 11 and 9 / 11 with new spend dropping accordingly.
4. Change the budget to reserve in advance, run stage 5 with INJECT=overbudget, and see both the "STOP hard limit tripped" and "5 finished artifacts remain on disk" lines appear.
5. Add INJECT=blocked, implement the refund and the not-billed rule for the moderation-blocked call, and confirm the "failed or blocked, not billed" line appears at the bottom of the dashboard.

Interview Questions

Today's 3 questions are in the question bank below, focused on cost-structure analysis, the criteria for tiered routing, and the safe-stop semantics of breaking and degrading. Read the analysis before the key points — practicing the derivation beats memorizing them. The cn / global labels let you pick by target market.

Checklist and Tomorrow

• I can break cost down per stage, and point out where most of the money goes and why
• I can configure a model-routing policy per stage that makes an explainable trade-off between quality and cost
• I can implement a budget circuit breaker that safely stops a run when it goes over budget instead of burning through it all
• I can state the three criteria for a safe stop, and explain why exiting the process outright is wrong
• I can name the three degradation dimensions and why each step must be validated against the books first
• All 5 lab acceptance criteria pass
• I can answer at least 2 of the 3 interview questions without looking at the key points

Tomorrow (D13) we publish: export one episode in three platforms' specs, have the model batch-generate titles and cover copy then score and pick the top three, and feed playback data back to guide the next episode. Why settle the accounts before distributing? Because distribution gives you an incentive to produce more variants, and producing variants is exactly where things run away — install the brakes before touching the accelerator.