Dayward AI
中文
Week 2 · D13About 6 hours

Cron Scheduling (Central Scheduler → Stream Delivery) + Cost Metering (Token → USD Ledger, Usage Report)

Implement a centralized cron scheduler that delivers scheduled tasks into the message bus, and build a cost-metering system that converts token usage into a USD ledger.

Today's goals 0/3

Sign in to tick these off and save your progress.

Today's Goals

  1. Implement a central scheduler that delivers tasks into the message bus on a cron schedule
  2. Implement a token-to-USD cost ledger, recording cost per call
  3. Produce a simple usage report that summarizes cost by user or by time

Yesterday fitted the agent with long-term memory and left a note in passing: every memory written calls an embedding, which is a new expense, and so far nobody is watching that bill. Today does both jobs — the agent goes from motionless until spoken to, to working on its own schedule, while every cent spent is recorded in a ledger as it goes.

Plain-Language Walkthrough

One person writes the duty roster; the staff do not each set an alarm

A company has three front-desk staff, and who starts when is written on the roster the office manager keeps. Change that to "each person sets their own phone alarm and opens up when it rings" and the result is not better punctuality but three people crowding the same door every morning, doing one job three times.

Since D10 you have had more than one worker: sharding and leases guarantee that one user's messages land on one worker, and the worker process itself is multi-replica, which D14 scales to 3. Write "run the daily digest task at 9am" into the worker's startup code and that line executes once per process — one task executed 3 times, users receiving 3 identical digests, and you paying three times.

Compute the money and it lands harder. Suppose 1,000 users in the support scenario have enabled a daily order summary, each running an agent at 9am: about 3,000 input tokens per request (system prompt plus tool definitions plus the day's order data) and about 500 output. At this course's standard prices, openai/gpt-4o-mini inputs at 0.15 dollars per million tokens and outputs at 0.60:

  • Input cost per call: 3000 / 1000000 * 0.15 = 0.00045 dollars
  • Output cost per call: 500 / 1000000 * 0.60 = 0.0003 dollars
  • Per call: 0.00075 dollars
  • Per month: 0.00075 * 1000 users * 30 days = 22.5 dollars

Twenty-two and a half dollars a month sounds harmless. But with three workers each running their own cron it becomes 67.5 — and the extra 45 dollars buys nothing but duplicate nuisance messages. Nor does it stop there: the day D14 moves replicas from 3 to 10, the bill and the nuisance both go up tenfold with no alert to anybody, because from each process's own point of view it merely executed dutifully on schedule.

So the correct shape gathers "who should be executed when" into one place, with workers responsible for working rather than for deciding when:

TextText
      +-------------+  cron matches  +-----------+  consumer group   +----------+
      |  scheduler  | -------------) | koda:runs | ----------------) | Worker 1 |
      | (single)    |  one XADD      |  the bus  | one message, one  +----------+
      +-------------+                +-----------+ consumer          | Worker 2 |
        does one thing:                                              +----------+
        decide what to send now                                      | Worker 3 |
                                                                     +----------+

The bus in that diagram is D9's: the scheduler is simply another producer for it, and the worker side does not change one line. A scheduled task is not a new kind of execution, it is a change of who presses the button — the user pressed it before and a clock presses it now. Grasp that and the amount of code today is startlingly small.

But handing the decision to one process immediately invites the question: what if the scheduler itself dies? If it restarts across the 9am minute, does it skip or duplicate? And if you run two scheduler instances for availability, are you not back to one task delivered twice?

The cron expression and a minimal scheduler

Fix "when" first. A cron expression is five space-separated fields, left to right minute, hour, day of month, month, day of week:

PositionMeaningRange
1stminute0-59
2ndhour0-23
3rdday of month1-31
4thmonth1-12
5thday of week0-6, where 0 is Sunday

Each field takes three forms: an asterisk is any value, a number is equal to this value (several separated by commas), and an asterisk plus a slash plus a number is every n units. A few examples:

ExpressionMeaning
*/5 * * * *every 5 minutes
0 9 * * *9:00 every day
0 9 * * 19:00 every Monday
0 0 1 * *00:00 on the 1st of every month
30 8,20 * * *8:30 and 20:30 every day

Cron's finest granularity is one minute; there is no seconds field (Quartz's six-field form is an extension, not the standard). So the scheduler's main loop is one sentence: wake once a minute, match that minute against the task table, and publish a message for every hit.

What genuinely needs care is what the idempotency key is anchored to. The wrong version uses Date.now() as part of the key: two scheduler instances' clocks never align to the millisecond, one wakes at 09:00:00.120 and the other at 09:00:00.480, the computed keys differ, and deduplication fails entirely. The correct approach uses the minute the trigger was scheduled for, that is the timestamp with seconds and milliseconds truncated — whoever wakes at whatever instant within that minute, the key is the same string.

scheduler.js
// Only *, comma-separated numbers, and */n are supported: enough, and readable at a glance
function matchField(field, value) {
  if (field === '*') return true
  if (field.startsWith('*/')) return value % Number(field.slice(2)) === 0
  return field.split(',').some((part) => Number(part) === value)
}
 
function matches(expr, d) {
  const [minute, hour, dom, mon, dow] = expr.split(' ')
  return (
    matchField(minute, d.getUTCMinutes()) &&
    matchField(hour, d.getUTCHours()) &&
    matchField(dom, d.getUTCDate()) &&
    matchField(mon, d.getUTCMonth() + 1) &&
    matchField(dow, d.getUTCDay())
  )
}
 
async function tick(now) {
  // The key step: truncate seconds and milliseconds. The idempotency key must be anchored
  // to the minute the trigger was scheduled for, not to the instant I woke up - two
  // scheduler instances never wake at the same instant.
  const slot = new Date(Math.floor(now.getTime() / 60000) * 60000)
  const stamp = slot.toISOString().slice(0, 16) + 'Z'
  for (const task of TASKS) {
    if (!matches(task.cron, slot)) continue
    await bus.publish('koda:runs', {
      idempotencyKey: `cron:${task.id}:${stamp}`,
      userId: task.userId,
      input: task.input,
    })
  }
}

Now answer the three questions from the end of the previous section. Their answer is the same one: the idempotency_key unique constraint on D8's runs table. The worker takes a message and runs the usual insert into runs ... on conflict do nothing, and a conflict means somebody already created a run for this task in this minute, so it XACKs without executing. This is the second use of D8's idempotency key: D9 used it to block duplicate consumption from at-least-once delivery, and today it blocks duplicate triggering from multiple scheduler instances or a restart replay. One unique constraint, two entirely different sources of duplication.

The skip-versus-duplicate trade-off follows: a scheduler crashing and coming back 90 seconds later has already passed that minute, and to backfill, have it look back N minutes on startup and match minute by minute. With the idempotency key as a backstop, duplicate publishing is harmless, which is exactly the premise that makes "better to over-send than under-send" workable. At-least-once plus idempotency is the least troublesome pairing in distributed systems — chasing exactly-once first and adding idempotency later usually does neither well.

What one call cost: converting tokens to dollars

The roster is done; turn to the ledger page.

An expense ledger's rule is to record on the spot: who, when, why, and how much, none optional, recorded at the moment of spending rather than reconstructed from memory at month end. Cost metering is identical, except the how-much column needs one conversion first — a model API returns token counts and a bill is denominated in dollars.

The conversion is one multiplication. This course's standard prices are this table:

ItemPrice
Chat model openai/gpt-4o-mini input0.15 dollars per million tokens
Chat model openai/gpt-4o-mini output0.60 dollars per million tokens
Embedding model openai/text-embedding-3-small0.02 dollars per million tokens

Note that input and output are not the same price, with output usually three or four times more. That gap decides optimization direction directly: "add 500 tokens to the system prompt" and "let the model say 500 more tokens" are not the same order of cost.

Price D12's memory writing and yesterday's hook proves its worth. Same 1,000 users, assume 5 memories settle per user per day at 400 characters per chunk, using this course's conservative one-character-per-token convention:

  • Embedding tokens per day: 1000 * 5 * 400 = 2,000,000 tokens
  • Cost per day: 2000000 / 1000000 * 0.02 = 0.04 dollars
  • Cost per month: 0.04 * 30 = 1.2 dollars

Against the same users' 22.5 dollars of monthly chat spend, embeddings are about five percent. So the conclusion is not that embeddings are expensive, it is that embeddings are cheap enough for you to forget they exist — until somebody ships "write a memory for every message," volume grows twentyfold, this line becomes 24 dollars, and it exceeds the chat itself. A team with no ledger spends two weeks finding where that money came from.

At the code layer there is really only one thing to watch: do not represent money as floating point.

ledger.js
// The price table is configuration, not a constant: when a vendor changes prices you edit
// one table, and you must be able to tell which version an old charge used
const PRICING = {
  'openai/gpt-4o-mini': { promptPerM: 0.15, completionPerM: 0.6 },
  'openai/text-embedding-3-small': { promptPerM: 0.02, completionPerM: 0 },
}
 
// Six decimal places: one call is often under a ten-thousandth of a dollar, and rounding
// to 2 places turns every row into 0. JS has no decimal type, so convert to a fixed-point
// string immediately and let the numeric(12,6) column accumulate exactly - never reduce
// a pile of amounts on the JS side.
function costUsd(model, promptTokens, completionTokens) {
  const price = PRICING[model]
  if (!price) throw new Error(`model with no registered price: ${model}`)
  const micro = promptTokens * price.promptPerM + completionTokens * price.completionPerM
  return (micro / 1_000_000).toFixed(6)
}
 
async function recordUsage(db, entry) {
  await db.query(
    `insert into usage_ledger
       (id, user_id, run_id, model, kind, prompt_tokens, completion_tokens, cost_usd, created_at)
     values ($1, $2, $3, $4, $5, $6, $7, $8, now())`,
    [randomUUID(), entry.userId, entry.runId, entry.model, entry.kind,
     entry.promptTokens, entry.completionTokens,
     costUsd(entry.model, entry.promptTokens, entry.completionTokens)],
  )
}

Three of the four versions stress the same thing: Python uses Decimal, Java BigDecimal, Swift Decimal, and only JS has no native decimal type, so its strategy is to convert to a fixed-point string immediately and never accumulate money in memory. That is not fussiness — summing a hundred thousand amounts in double precision and finding the total disagrees with the row-by-row sum is the most classic and least explainable ticket a cost system produces.

Which fields a ledger needs to deserve the name

A ledger is not a log. A log is for your own investigations and can be deleted; a ledger is for reconciliation, for finance, and for answering "why did this month rise 40%," so its field design has one criterion: any charge must trace back through the fields to who, which execution, which model, and how many tokens.

D13 adds one table beside D8's three, with only these fields:

SQLSQL
create table usage_ledger (
  id                text           primary key,
  user_id           text           not null,
  run_id            text,                        -- nullable: not every charge belongs to a run
  model             text           not null,
  kind              text           not null,     -- 'chat' | 'embedding'
  prompt_tokens     integer        not null,
  completion_tokens integer        not null,
  cost_usd          numeric(12, 6) not null,     -- fixed point, not float
  created_at        timestamptz    not null default now()
);
 
create index usage_ledger_user_id_idx    on usage_ledger (user_id);
create index usage_ledger_created_at_idx on usage_ledger (created_at);

Why each of them:

  • user_id and run_id are two different trace lines. The former answers whose account this charge goes on, the latter which execution it belongs to. run_id may be null, because system-level spend such as a nightly bulk reindex of memories belongs to no user execution and still needs an owner.
  • model stores the one actually used at the time. D4's fallback lands the same business on different providers, and you must be able to compute how much of last month went on downgraded traffic.
  • kind separates chat from embedding. Their magnitudes, growth curves, and optimizations differ completely, and summing them together is not a summary at all.
  • prompt_tokens and completion_tokens are stored separately. The unit prices differ fourfold, so a single total can never be converted back to an amount, and it hides whether the prompt is too long or the model too verbose.
  • cost_usd uses numeric(12, 6) and is stored redundantly rather than recomputed per query. Prices change. Today's 0.15 input price may be 0.10 in six months, and you cannot have the first half of the year's historical bills change with it. An amount is frozen at the moment of writing, which is the essential difference between a ledger and a report.

Usage report: one ledger, three questions

With a ledger, a report is a few group by statements. What genuinely needs thought is which question you are answering — pick the wrong dimension and nobody reads however pretty the report. Three cuts cover most needs:

SQLSQL
-- By user: answers "who spent the most this month", used for pricing tiers and abuse checks
select user_id,
       sum(cost_usd)                                as cost,
       sum(prompt_tokens + completion_tokens)       as tokens,
       count(*)                                     as calls
from usage_ledger
where created_at >= date_trunc('month', now())
group by user_id
order by cost desc
limit 20;
 
-- By day: answers "when did it start rising", used to pin the release that caused it
select date_trunc('day', created_at) as day, kind, sum(cost_usd) as cost
from usage_ledger
where created_at >= now() - interval '30 days'
group by 1, 2
order by 1;
 
-- By model: answers "are we overusing the expensive tier", used to verify D4's tiered routing
select model, kind, count(*) as calls, sum(cost_usd) as cost
from usage_ledger
where created_at >= date_trunc('month', now())
group by model, kind
order by cost desc;

The three dimensions map to three completely different actions: by user is a commercial action (who should pay more, who is abusing it); by day is a troubleshooting action (align against the release timeline to find the culprit); by model is an optimization action (verify that downgrading actually saved money). The admin script in today's lab does the first, printing something like:

TextText
=== usage report 2026-09 (by user) ===
user_id   calls  prompt_tokens  completion_tokens  cost_usd
u-1       6      4800           720                $0.000996
u-2       2      1600           240                $0.000332
------------------------------------------------------------
TOTAL     8      6400           960                $0.001328

Feeding cost data back into capacity planning

The last step, and the one that turns a ledger from a finance toy into an engineering tool: convert cost into unit-economics metrics and use them to constrain the system.

An absolute amount carries no information. "We spent 22.5 dollars this month" says neither cheap nor expensive; what you need is three numbers with denominators:

MetricFormulaAnswers
Cost per executionmonth's total cost / month's run counthow much each added feature made one conversation cost
Cost per user per monthmonth's total cost / month's active userswhether pricing covers cost
Output per dollarmonth's completed business actions / month's total costwhether the system is worth continued investment

Compute the first from this chapter's numbers: 22.5 dollars / 30,000 runs = 0.00075 dollars each. Its use is as a ruler. A genuinely common example: on D5 you registered 10 tools, and by this course's convention each definition occupies 100 to 150 tokens in the context, so 10 is 1,000 to 1,500 — precisely half of those 3,000 input tokens. Which means half of each call's input cost pays for a tool list the model mostly does not use — at 1,500 tokens, 1500 / 1000000 * 0.15 = 0.000225 dollars each, or 6.75 dollars a month over 30,000 calls. Change 5 low-frequency tools to load per scenario and you save about 3.4 dollars a month, 15% of total chat cost. That judgment is not guesswork, it was measured off the ledger.

With a ruler you can build a guardrail. The most practical is a budget guard: before a call, check this user's spend this month and downgrade or refuse past a threshold.

budget.js
const MONTHLY_LIMIT_USD = 5
 
// The return value is which tier to use rather than true/false - refusing shuts the user
// out, while downgrading only makes them slower and less clever, and the experience gap
// is enormous
async function pickTier(db, userId) {
  const spent = await db.monthlySpend(userId)
  if (spent >= MONTHLY_LIMIT_USD) return 'blocked'
  if (spent >= MONTHLY_LIMIT_USD * 0.8) return 'cheap'
  return 'default'
}

One layer above is capacity planning: plot daily cost and daily call volume as two lines and, with cost per execution as the ratio, you can spot a signal like "user count is flat and per-call cost rose first" a week or two early — which usually means a release lengthened the prompt or a tool's response body ballooned. A production-grade IM Agent platform keeps that curve on the on-call dashboard alongside error rate and latency, because cost is a health indicator that moves before failures do: an abnormal rise in per-call cost often precedes a timeout alert by several days.

Source Reading

Hands-On Lab

🧪 D13 lab: a cron tool plus a cost ledger plus a simple admin page

Code location: labs/agent-30days/day-13-cron-and-billing

Acceptance criteria:

  1. All five self-checks under MOCK=1 SELFTEST=1 pnpm start pass with exit code 0 (starter/ as-is passes only check 3).
  2. Check 1 proves cron matching: across the 15 minutes from 08:58 to 09:12, */5 * * * * hits 3 times and 0 9 * * * hits once.
  3. Check 2 proves idempotency: replaying those 15 minutes twice publishes 8 messages, of which only 4 actually execute and 4 are blocked by the idempotency key.
  4. Check 3 proves one cron tick is consumed once: two workers consume together and each run leaves exactly 2 ledger rows (1 chat plus 1 embedding).
  5. Checks 4 and 5 prove the ledger arithmetic: 8 rows total 0.001328 dollars, and pnpm report prints 0.000996 for u-1 and 0.000332 for u-2.

starter/ has four exercise points cut out of it and needs zero external services under MOCK=1: Redis and Postgres are in-memory implementations (not stubs — consumer-group semantics and the idempotency unique constraint are genuinely implemented), and the clock is injectable, so the self-check fast-forwards those 15 minutes within a second rather than making you wait. All four blanks default to something that runs but is visibly wrong — */n never matches, the idempotency key uses Date.now(), amounts are always 0, the report does not group by user — so run it as-is first and read those 4 failures. With Docker installed, docker compose up -d in the lab root, then rerun the self-check with REDIS_URL and DATABASE_URL and the infrastructure line changes from memory to real while the results are identical: not one line of business code changed.

  1. Complete matchField's */n branch and rerun, watching check 1 turn green with hit counts of exactly 3 and 1.
  2. Change the idempotency key from Date.now() to the scheduled minute, and watch check 2 turn green: 8 publishes, 4 runs.
  3. Read check 3's trace and confirm the two workers have different consumer names and each run appears in only one of their logs — this one is already scaffolded, a direct reuse of D9's consumer group, and the only check starter/ passes as-is.
  4. Implement costUsd, multiplying prompt and completion by their prices before dividing by a million, and verify 0.001328 by hand.
  5. Complete the group-by-user summary, then run pnpm report and see both users' monthly cost summing to check 4's figure.

Interview Questions

Today's four questions are in the bank below, weighted toward distributed scheduled tasks, cost control, and observability. Expand a question and read the analysis before the key points — the follow-up on question 4 about what the idempotency key should be anchored to is where this chapter gets pressed hardest, so do not skip it. Each is tagged for the China-domestic or overseas market so you can prioritize by where you are applying.

Checklist and Tomorrow

  • Implement a central scheduler that delivers tasks into the message bus on a cron schedule
  • Implement a token-to-USD cost ledger, recording cost per call
  • Produce a simple usage report that summarizes cost by user or by time
  • Say why the idempotency key is anchored to the scheduled minute rather than the current instant
  • Compute, without notes, that 1,000 users once a day at 3,000 input and 500 output tokens costs 22.5 dollars a month
  • All 5 self-check criteria of the lab pass
  • Answer at least 3 of the 4 interview questions without looking at the key points

Tomorrow (D14) closes out W2. As of today the scheduler, the bus, the worker, the database, and the ledger all run in one terminal on your machine, and you notice any problem by reading logs with your own eyes. Going live means three new problems: running 3 worker replicas from one image, replacing a process during a release without cutting off an in-flight run, and knowing whether a worker is genuinely working or has quietly died. Tomorrow fills those in with compose replicas, graceful shutdown, and heartbeats, then strings D8 through D13 into one line for a retrospective — get the system working before discussing how to operate it, and the order cannot be reversed.

Interview questions

  • When a service runs multiple replicas, why not let each replica start its own cron? What would you do instead?服务部署了多个实例，定时任务为什么不能让每个实例各自起一个 cron？你会怎么做？
    Common in ChinaCommon overseasBasic#scheduling#distributed-systems#cost

    How to reason about it · think before answering

    1. The hinge is the phrase multiple replicas. Saying it would run twice is only the symptom; the interviewer wants the business and dollar consequence.
    2. Make the cost concrete: three replicas each running cron means the job fires three times, users get three identical pushes, and you pay for three model calls. The multiplier tracks replica count, so scaling to ten makes both the bill and the spam tenfold, with no alert firing, because from each process's own point of view it ran exactly once.
    3. Give the right shape: move the decision of who runs when into one central scheduler whose only job, on a cron match, is to publish a task message onto the bus; the execution side keeps using a consumer group so one message reaches exactly one consumer. The key insight is that a scheduled task is not a new execution path, it just swaps the user for a clock as the thing pressing the button, so the worker code stays untouched.
    4. Volunteer the obvious follow-up: doesn't the scheduler become a single point of failure? Two layers. It is stateless, so a crash costs you a few minutes of task delay; if you truly need HA, run two instances and dedupe on the idempotency key at publish time rather than bolting a distributed lock onto the scheduler.
    5. Close with sizing: a central scheduler plus a bus is enough at modest volume. At high volume, or when tasks have dependencies, teams move to a dedicated workflow scheduler with dependency graphs, retry policy and backfill, but the underlying central-decision-plus-queue shape is identical.
    6. Expect: the scheduler was down for 90 seconds and skipped a minute — now what? Replay the last N minutes on startup, one minute at a time. The idempotency key makes redundant publishes harmless, which is exactly what makes at-least-once plus idempotency the easy combination.

    分析过程 · 先想清楚再作答

    1. 题眼在「多个实例」四个字。只答「会重复执行」拿不到分，因为那是现象；面试官想看你能不能把现象换算成业务后果和钱。
    2. 先把重复的代价说具体：3 个副本各起 cron，同一个任务被执行 3 次，用户收到 3 份一样的推送，你付 3 份模型调用的钱。而且这个倍数会跟着副本数走——扩容到 10 个副本，账单和骚扰量一起变成十倍，却不会触发任何告警，因为从每个进程自己的视角看它只是老实地执行了一次。
    3. 然后给出正确的形状：把「谁该在什么时候被执行」收进一个中心调度器，它命中 cron 之后只做一件事——往消息总线投递一条任务消息；执行侧照旧靠消费组分摊，一条消息只会被一个消费者拿到。关键认知是「定时任务不是一种新的执行方式，只是把按按钮的人从用户换成了钟表」，所以执行侧一行代码都不用改。
    4. 接着主动补上「那调度器自己不就成单点了吗」——这是必被追问的一句。答案分两层：调度器无状态、崩了拉起来就行，短暂不可用的代价只是几分钟内的任务延迟；真要高可用就起两个实例，靠投递时的幂等键去重，而不是靠给调度器加分布式锁。
    5. 最后点一句选型：任务量不大时中心调度器加消息总线足够；量大或者任务本身有依赖关系时，业界会换成专门的调度框架（带任务依赖、重试策略、补数），但底层的「中心决定 + 队列分发」结构是一样的。
    6. 可以预期的追问：调度器崩溃 90 秒，中间跨过的那一分钟怎么办？答启动时回看最近 N 分钟逐分钟重放，因为有幂等键兜底，重复投递无害——这正是 at-least-once 加幂等这组搭配能成立的地方。

    Key points

    • Per-replica cron means the job runs N times: N duplicate pushes, N times the model spend, scaling linearly with replica count and silently
    • The right shape is a central scheduler that publishes one message to the bus on a cron match, with a consumer group ensuring exactly one worker picks it up
    • A scheduled task is not a new execution path — only the trigger changed from a user to a clock, so worker code is unchanged
    • The scheduler is stateless: restart on crash, and if you need HA run two and dedupe on the idempotency key rather than adding a distributed lock
    • Missed minutes are recovered by replaying the last N minutes at startup, which is safe because the idempotency key absorbs duplicates

    答题要点

    • 每个实例各自起 cron 等于同一个任务被执行 N 次：用户收到 N 份重复推送，模型调用花 N 倍的钱，倍数随副本数线性增长且不会触发告警
    • 正确形状是中心调度器命中 cron 后只往消息总线投递一条消息，执行侧靠消费组保证一条消息只被一个 Worker 拿到
    • 定时任务不是新的执行路径，只是把触发者从用户换成了钟表，所以 Worker 侧不需要任何改动
    • 调度器是无状态的，崩了拉起来即可；需要高可用就起两个实例靠投递时的幂等键去重，不要给它加分布式锁
    • 崩溃期间跨过的时间点靠启动时回看最近 N 分钟重放补上，幂等键保证重复投递无害
  • How would you design a token cost metering and ledger system from scratch?让你从零设计一套 token 成本计量和台账系统，你会怎么做？
    Common in ChinaCommon overseasIntermediate#cost#observability#data-modeling

    How to reason about it · think before answering

    1. This question tests whether you have ever reconciled a bill. The discriminators are the numeric type you store money in, and whether cost is stored or computed at query time. A design missing either gets rejected by finance within a quarter.
    2. Set the criterion first: a ledger is not a log. Logs exist for debugging and can be dropped; a ledger has to reconcile against the vendor invoice and answer why the bill grew 40% this month, so every charge must trace back to who, which run, which model, and how many tokens. Every field falls out of that.
    3. Then walk the fields with reasons: user_id says whose budget it hits; run_id says which execution it belongs to and is nullable because some spend is system-level batch work; model records the one actually used, since fallback routes the same workload to different providers; kind separates chat from embedding because their volumes and growth curves differ completely; prompt_tokens and completion_tokens are stored separately because input and output differ three- to four-fold in price, and a single total can neither reproduce the amount nor tell you whether the prompt is bloated or the model is verbose.
    4. Now the two judgments that show experience. First, money uses fixed-point: numeric in the database, Decimal or BigDecimal in code, never accumulated in binary floats, or the total will diverge from the sum of rows after a hundred thousand entries. Second, cost is computed at write time and stored redundantly, not recomputed from the current price table — prices change, and history must not change with them. That is the essential difference between a ledger and a report.
    5. Volunteer the timing and transaction boundary: record at the moment you receive the usage field, not at business success, because failed calls still cost money and a fallback spans two or three billable calls per business operation. Ledger writes need not share the business transaction — losing a row costs fractions of a cent, while locking the ledger table stalls user conversations — so write asynchronously with retries and a uniqueness constraint on run id plus call index. The exception is quota enforcement: if the product caps spend, the decrement must be transactional or concurrent requests will blow through the cap.
    6. Expect: what happens to history when the vendor changes prices? The price table itself needs effective dates and a version, and the ledger stores both the computed amount and the price version, so recomputation and audit both have a basis.

    分析过程 · 先想清楚再作答

    1. 这题在考「你有没有真的对过账」。区分度在两个地方：金额用什么类型存，以及金额是冗余存还是查询时现算。答不到这两点的方案，上线三个月就会被财务打回来。
    2. 先立判据：台账不是日志。日志是给排查问题用的，删了就删了；台账要拿去对账、要回答「这个月为什么涨了 40%」，所以每一笔钱都必须能追回到「谁、因为哪一次执行、用哪个模型、花了多少 token」。字段设计全部由这条判据推出来。
    3. 然后给字段和理由，一一对应：user_id 回答该算谁头上、run_id 回答属于哪次执行（允许为空，因为有系统级批量开销）、model 存调用当时那一个（fallback 会让同一段业务落到不同模型上）、kind 区分 chat 和 embedding（两者量级和增长曲线完全不同）、prompt_tokens 与 completion_tokens 分开存（输入输出单价差三到四倍，只存 total 就算不回金额，也看不出是提示词太长还是模型太啰嗦）。
    4. 接着是两个最能体现经验的判断。第一，金额用定点类型：数据库用 numeric，代码里用 Decimal 或 BigDecimal，绝不用双精度浮点累加，否则十万条之后总额和逐条相加对不上。第二，cost_usd 要在写入那一刻算好并冗余存，不要查询时用当前价格表现算——价格会变，历史账单不能跟着一起变，这是台账和报表最本质的区别。
    5. 还要主动说记账的时机和事务边界：记账放在「拿到 usage 字段」那一刻，而不是「业务成功」那一刻，因为失败的调用同样产生费用，尤其 fallback 会一次业务跨两三次收费调用。台账写入不必和业务同事务（丢一条只是几厘钱，锁住台账表却会卡住用户对话），可以异步加重试，用 run_id 加调用序号做唯一约束防重；但如果产品有额度限制，配额扣减必须同事务，否则用户能靠并发把额度刷穿。
    6. 可以预期的追问：厂商调价了历史数据怎么办？答案是价格表本身要有生效时间和版本号，台账里既存算好的金额也可以存价格版本，这样重算和审计都有依据。

    Key points

    • A ledger is not a log: every charge must trace to a user, a run, a model and a token count, and the schema follows from that
    • Store prompt and completion tokens separately, since input and output prices differ three- to four-fold and a single total can neither reproduce the amount nor localize the problem
    • Use fixed-point money (numeric in the database, Decimal or BigDecimal in code); float accumulation makes totals disagree with the sum of rows
    • Compute cost at write time and store it, rather than recomputing from today's price table, so history stays stable when prices change
    • Record at the moment usage is returned, not at business success — failed calls and fallbacks still cost money; ledger writes can be async with retries, but quota decrements must be transactional

    答题要点

    • 台账不是日志：每一笔钱要能追回到谁、哪一次 run、哪个模型、多少 token，字段设计全由这条判据推出
    • prompt_tokens 与 completion_tokens 必须分开存，因为输入输出单价差三到四倍，只存 total 既算不回金额也看不出问题出在哪一侧
    • 金额用定点类型（数据库 numeric、代码 Decimal/BigDecimal），不要用浮点累加，否则总额和逐条相加对不上
    • cost_usd 在写入那一刻算好并冗余存，不要查询时按当前价格现算——价格会变，历史账单不能跟着变
    • 记账时机是拿到 usage 字段那一刻而不是业务成功那一刻，失败调用和 fallback 同样产生费用；台账可异步写入加重试，但配额扣减必须和业务同事务
  • Which dimensions should a usage report for an LLM product cover, and what decision does each one drive?一份 LLM 应用的 usage report 通常要覆盖哪些维度？这些维度分别用来做什么决策？
    Common in ChinaCommon overseasIntermediate#observability#cost#reporting

    How to reason about it · think before answering

    1. The trap is listing dimensions: by user, by day, by model, by feature. Length signals you have not thought about it. The discriminator is the second half — which action each dimension drives. No action means you built reports but never used one.
    2. Give three primary dimensions with their action type: by user is a commercial action (who to reprice, who is abusing, whether tiering covers cost); by day is a debugging action (align with the release timeline to find which deploy stepped the cost up); by model and call kind is an optimization action (did tiered routing actually save money, is embedding volume running away). Three dimensions, three different dashboard audiences.
    3. Then go up a level: absolute dollars carry no information. What matters are unit-economics ratios with a denominator — cost per run (monthly cost over run count), cost per active user per month, and business actions completed per dollar. The first two say whether pricing covers cost; the third says whether the system deserves further investment.
    4. Prove you have used it with a concrete pattern: cost per run is a ruler. If user count is flat but cost per run climbs, it is almost always a deploy that lengthened the prompt or a tool whose response body grew. That signal usually appears days before latency alerts, which is why mature teams put the cost curve next to error rate and latency on the on-call dashboard.
    5. Add the dimension most people miss: failures and fallbacks. Failed calls are still billed, and a fallback spans two or three billable calls per business operation. Without slicing that out, your gap against the vendor invoice concentrates exactly during incidents, when you most need cost clarity.
    6. Expect: how fresh does the report need to be? Tier it — daily rollups can run offline, but quota and budget guardrails need near-real-time month-to-date totals, usually from an incrementally updated per-user monthly summary table rather than scanning the detail rows on every request.

    分析过程 · 先想清楚再作答

    1. 这题最容易答成罗列维度：按用户、按天、按模型、按功能……列得越全越显得没想过。区分度在后半句——每个维度对应的是哪一类行动。列不出行动，说明你只做过报表没用过报表。
    2. 先给三个主维度和它们各自的行动类型：按用户切是商业动作（谁该涨价、谁在滥用、定价分层能不能覆盖成本）；按天切是排障动作（对齐发布时间线，找出是哪次上线让成本跳了台阶）；按模型和调用类型切是优化动作（验证分层路由有没有真省到钱、embedding 的量是不是失控了）。三个维度对应三个不同的看板受众。
    3. 然后升一层，指出绝对金额没有信息量，真正有用的是带分母的单位经济学指标：每次执行成本（当月总成本除以 run 数）、每用户月成本（除以活跃用户数）、每美元产出（完成的业务动作数除以总成本）。前两个用来判断定价能不能覆盖成本，第三个用来判断这套系统值不值得继续投入。
    4. 举一个能落地的用法证明你真用过：每次执行成本这个比值是把尺子。如果用户数没涨而单次成本涨了，几乎一定是某次上线让提示词变长了，或者某个工具的返回体膨胀了——这个信号通常比超时告警早好几天出现，所以成熟团队会把成本曲线和错误率、延迟并排挂在值班大盘上。
    5. 最后补一个大多数人会漏的维度：失败与降级。失败的调用照样收费，fallback 会让一次业务操作跨两三次收费调用。报表里不单独切出这一块，你和厂商账单的差额就会恰好集中在故障期，也就是最需要看清成本的时候。
    6. 可以预期的追问：报表要做到什么实时度？答案是分层——按天的汇总离线跑就够，但配额和预算护栏需要近实时的当月累计，通常用一张按用户按月的汇总表增量更新，而不是每次请求都扫一遍明细。

    Key points

    • By user drives commercial decisions, by day drives debugging, and by model or call kind drives optimization — three dimensions, three audiences
    • Absolute dollars say nothing; use ratios with a denominator: cost per run, cost per active user per month, and business actions per dollar
    • Cost per run is a ruler: flat users with rising per-run cost usually means a longer prompt or a bloated tool response, and it shows days before latency alerts
    • Slice out failed and fallback calls, or your gap against the vendor invoice concentrates during incidents
    • Tier the freshness: daily rollups offline, near-real-time month-to-date totals from an incremental summary table for budget guardrails

    答题要点

    • 按用户切是商业动作（定价分层、异常账号），按天切是排障动作（对齐发布找成本跳变），按模型和调用类型切是优化动作（验证分层路由、盯 embedding 用量）
    • 绝对金额没有信息量，要看带分母的指标：每次执行成本、每用户月成本、每美元产出
    • 每次执行成本是把尺子：用户数没涨而单次成本涨了，通常是提示词变长或工具返回体膨胀，比超时告警早好几天出现
    • 必须单独切出失败与降级的开销，否则和厂商账单的差额会集中在故障期
    • 实时度要分层：按天汇总可离线跑，预算护栏需要近实时的当月累计，用增量汇总表而不是每次扫明细
  • How do you keep a cron job from being published or executed twice, and how should the idempotency key be built?怎么保证一个 cron 任务不会被重复投递或重复执行？幂等键应该怎么构造？
    Common in ChinaCommon overseasDeep dive#idempotency#scheduling#distributed-systems

    How to reason about it · think before answering

    1. The hinge is that publishing and executing are two separate problems. Most candidates answer half: either only the consumer group (which stops duplicate execution) or only a lock (which stops duplicate publishing, and imperfectly). A complete answer names the duplicate sources on both sides plus one backstop that covers both.
    2. Enumerate the sources: duplicate publishes come from multiple scheduler instances, from replay after a scheduler restart, and from the bus's own at-least-once semantics. Duplicate executions come from a worker crashing mid-processing and the message being reclaimed by another consumer. The two need different treatment.
    3. State the core conclusion: do not reach for a distributed lock, use a uniqueness constraint in the data layer. A lease only gives you probable mutual exclusion — in the instant when the TTL expires while the previous holder is merely stuck in GC, both schedulers believe they hold it and both publish. A uniqueness constraint is evaluated at the final insert, so no matter how many times upstream published, the table gains exactly one row. Do not escalate a problem solvable by a constraint into a distributed coordination problem.
    4. Then the key construction, which is where people fail: the key must be the task id plus the scheduled minute, never the current instant. Two scheduler clocks never align to the millisecond; one wakes at 09:00:00.120 and the other at 09:00:00.480, so keys built from now differ and dedup collapses. Truncate seconds and milliseconds and every instance computes the same string for that minute. In code this is an insert with on conflict do nothing; a conflict means the execution already exists, so ack the message and skip.
    5. Scope it honestly: this guarantees one execution per trigger point, not that side effects inside the execution happen once. If the run sends an SMS or charges a card, those side effects need their own idempotency keys, because the worker can crash after sending and before writing status. Making that distinction earns points.
    6. Expect: what about missed triggers? Prefer over-publishing to under-publishing — replay the last N minutes at startup and let the idempotency key absorb duplicates. At-least-once plus idempotency is the easiest combination in distributed systems; chasing exactly-once first and adding idempotency later usually achieves neither.

    分析过程 · 先想清楚再作答

    1. 题眼在「投递」和「执行」是两件事。很多人只答一半：要么只说消费组保证一条消息一个消费者（那只挡住了执行侧的重复），要么只说加锁（那只挡住了投递侧，还挡不干净）。完整答案要说清两侧各自的重复来源，以及一个能同时兜住的兜底。
    2. 先拆重复的来源：投递侧的重复来自多个调度器实例、调度器重启后的补发重放、以及消息总线本身的至少一次语义；执行侧的重复来自 Worker 处理到一半崩溃后消息被 XAUTOCLAIM 转交给别人。这两类重复用不同手段挡效率完全不同。
    3. 再给核心结论：不要用分布式锁去做互斥，用数据层的唯一约束做去重。原因是锁只能提供「大概率互斥」——租约到期而前任进程其实只是 GC 卡住的那一瞬间，两个调度器都会认为自己持有，各发一次；而唯一约束是在最终落库那一步判断的，无论上游发了几次，任务表里只会多一行。能在唯一约束上解决的问题，不要升级成分布式协调问题。
    4. 然后回答幂等键怎么构造，这是最容易翻车的一步：键必须是「任务 id 加计划触发的那一分钟」，绝不能用当前时刻。两个调度器实例的时钟不可能对齐到毫秒，一个在 09:00:00.120 醒来、另一个在 09:00:00.480 醒来，用 now 算出来的键不一样，去重完全失效。把秒和毫秒截掉之后，无论谁在这一分钟里的哪一刻醒来，算出的键都是同一个字符串。落到代码上就是 insert 加 on conflict do nothing，冲突说明已经有人建过这次执行，直接 ack 掉不执行。
    5. 补一句作用范围：这套只保证「同一个触发点只产生一次执行」，不保证「执行内部的副作用只发生一次」。如果这次执行要发短信、要扣款，那些副作用还得各自带自己的幂等键，因为 Worker 可能在发完短信之后、写完状态之前崩掉。这一层区分是加分项。
    6. 可以预期的追问：那漏发怎么办？答宁可多发不可少发——调度器启动时回看最近 N 分钟逐分钟重放，重复投递被幂等键吃掉。at-least-once 加幂等是分布式系统里最省心的一组搭配，反过来先追求 exactly-once 再补幂等，通常两头都做不好。

    Key points

    • Duplicate publishing and duplicate execution are separate: the former comes from multiple schedulers, restart replay and at-least-once delivery; the latter from a crashed worker's message being reclaimed
    • Dedupe with a database uniqueness constraint rather than a distributed lock: when a lease expires while the holder is only GC-stalled, both schedulers publish, whereas the constraint admits exactly one row
    • Build the key from the task id plus the scheduled minute, never the current instant — instances never wake at the same millisecond, so a now-based key defeats dedup entirely
    • In code this is an insert with on conflict do nothing; on conflict, ack the message and skip execution
    • This guarantees one execution per trigger, not once-only side effects — SMS or payments inside the run need their own keys; prefer over-publishing and let at-least-once plus idempotency absorb it

    答题要点

    • 投递重复和执行重复是两件事：前者来自多调度器实例、重启重放和总线的至少一次语义，后者来自 Worker 崩溃后消息被转交
    • 用数据层唯一约束去重，不要用分布式锁互斥：租约过期而前任还活着的瞬间两个调度器都会各发一次，而唯一约束在落库那一步只放行一条
    • 幂等键必须是任务 id 加计划触发的那一分钟，不能用当前时刻——两个实例的醒来时刻永远不同，用 now 会让去重完全失效
    • 落到代码上是 insert 加 on conflict do nothing，冲突就直接 ack 不执行
    • 这只保证一个触发点一次执行，执行内部的发短信、扣款等副作用要各自带幂等键；宁可多发不可少发，靠 at-least-once 加幂等兜底

Comments