Dayward AI
中文
Week 2 · D8About 6 hours

Why Split Gateway and Worker; Postgres Table Design (sessions/runs/messages) + Drizzle

Understand why a production-grade agent splits its intake layer from its execution layer, and use Drizzle to create mini-koda's first batch of tables.

Today's goals 0/3

Sign in to tick these off and save your progress.

Today's Goals

  1. Draw a three-part gateway / message-bus / worker architecture and state each part's responsibility
  2. Define the sessions, runs, and messages tables with Drizzle and generate a migration
  3. Explain why a stateless gateway and idempotent writes are prerequisites for horizontal scaling

Yesterday closed on a promise: the in-memory session Map becomes three Postgres tables today — sessions, runs, messages. But today is not only a change of storage. First we split D7's carry-everything service down the middle, and then we decide where the state goes. This is the foundation for everything else this week.

Plain-Language Walkthrough

A check-in desk does not stop because one aircraft loads slowly

An airport check-in desk does remarkably little: verify the document, weigh the bag, print the boarding pass, push the bag onto the belt. Thirty seconds per passenger. The agent does not care what time your case reaches the hold, or whether that aircraft is still waiting for a tug. The heavy work happens on the apron, where the loading crew carries cartloads of bags into the hold and can grind on for half an hour when things go badly. The two sides are joined by one baggage belt.

The entire value of that division is one sentence: the desk's throughput is unaffected by the apron's pace. Even if one aircraft's loading goes badly wrong, the queue still advances one passenger every thirty seconds.

D7's service is precisely the opposite: one process is both the desk and the loading crew. It runs beautifully on your machine and has three ways to die in production.

First, a restart loses every in-flight execution. D6 already wrote sessions to a local file, which persisted what was said; but that file holds nothing about how far the currently running execution got. Restart the service and there is not even a body to find, and the user waits for a reply that will never come. D6 solved history; today solves execution itself, and they are not the same problem.

Second, two instances each store their own. You run two instances to handle concurrency, the load balancer sends the user's second request to the other one, its Map is empty, and the agent asks blankly what they were just saying.

Third, and subtlest: one slow request drags down the whole machine. An agent loop with tool calls in it taking thirty seconds is unremarkable, and for those thirty seconds it holds a connection, memory, and a buffer for the tens of thousands of tokens in its context. Run a dozen of those on one machine at once and incoming health checks start timing out; the orchestrator declares the machine dead, pulls it out and restarts it — taking those dozen in-flight executions down with it.

So the first test for a production-grade agent service is: anything that can be done in the worker should not sit in the gateway. The gateway does four things only: authenticate, rate-limit, persist, deliver. What those four share is a bounded cost measured in milliseconds, which does not stretch because the model is slow today.

gateway.js
// The gateway's POST /chat: do the four things and return, never run the agent loop here
app.post('/chat', async (request, reply) => {
  const { sessionId, message, clientMessageId } = request.body
  if (!sessionId || !message) {
    return reply.code(400).send({ error: 'sessionId and message are required' })
  }
 
  // 1. auth and 2. rate limiting are handled by upstream plugins, leaving 3. persist
  const key = idempotencyKeyFor(sessionId, clientMessageId, message)
  const run = await store.createRun({ sessionId, input: message, idempotencyKey: key })
 
  // 4. deliver to the message bus - empty today; D9 wires the bus up
  reply.code(202) // 202 Accepted: I took it and I am working on it, not 200 "here is the result"
  return { runId: run.id, status: run.status }
})

Notice the status code. 200 means the work is finished and here is the result; 202 means I took it and I am working on it — after the split the gateway holds no result, so 202 is all it can honestly say. That is not pedantry, it decides how the frontend is written: having received a 202 it must take the runId and subscribe for the result, so one request becomes submit plus subscribe.

That is the split's bill: you pay one extra round trip and one subscription endpoint, and the desk never queues. Whether it is worth it depends on whether one execution of your agent takes three hundred milliseconds or thirty seconds. Do not split a three-hundred-millisecond scenario; and the moment you use tool calls, thirty seconds is normal.

Stateless does not mean no state, it means the state is not in the process

Now the other half of the desk's benefit. You can get a boarding pass at any desk, because your itinerary lives in the system rather than in one agent's head. That is the precise meaning of stateless: not the absence of state, but state that does not remain on the process handling the request.

Statelessness has three direct consequences, each a prerequisite for horizontal scaling:

  • Adding a machine adds throughput. A new instance needs no warm-up and no data sync; attach it to the load balancer and it works immediately.
  • Any one of them can be killed at any time. Rolling deploys, preemptible instances, machine failure — pulling one out affects nobody, because no user's data exists only there.
  • No sticky sessions. D7's Map forces a load-balancer rule pinning a user to one machine, and once that rule exists, reassignment during a scale-up cuts existing users' sessions.

The worker's position is the opposite: it is stateful — but be precise about what it holds. Not user data, but one execution's progress (which round it is on, which tools it called, and from D10 a lease as well). User data lives in Postgres throughout, and what the worker holds is work in progress. The difference is in the consequences: a gateway can be killed freely, while a killed worker must first account for the execution in its hands, which is what D14's graceful shutdown deals with.

Drawn out, the three parts look like this:

TextText
                    +----------------+
   HTTP / SSE       |  Gateway  x N  |  stateless: any instance handles any request
   user -----------)| auth  ratelimit|
                    | persist deliver|
                    +---+--------+---+
                        | write  | read
                        |        v
                        |   +------------------+
                        |   |    Postgres      |
                        |   |  sessions        |
                        |   |  runs            |
                        |   |  messages        |
                        |   +------------------+
                        v            ^
                +----------------+   | write
                |  message bus   |   |
                |  (filled on D9)|   |
                +-------+--------+   |
                        | take       |
                        v            |
                +----------------+   |
                |  Worker   x M  |---+  stateful: holds one execution's progress
                |  agent loop    |
                +----------------+

Cost, as always. D7 read a history out of a Map in microseconds; with Postgres every round adds at least two database round trips, one to read the history and one to write a message, each a millisecond or two in the same data center. Three orders of magnitude slower, and the absolute value is still far below the few hundred milliseconds of one model call. That tax has to be paid, because what it buys is "add a machine and serve more people." As for how D7's SSE connection hanging off the gateway lines up with generation over on the worker — that is D11's subject. Today we only move the state out.

Three tables: a long-lived container, one execution, immutable facts

Stay with check-in. The system actually holds three records at three granularities: this passenger's itinerary (long-lived, following you all the way), this flight's loading work order (this one job, with a start and an end, which may fail or be redone), and the bag-by-bag manifest (each entry an accomplished fact, never edited once recorded).

An agent service's three tables are those three things:

  • sessions — a conversation's container, long-lived, one row per user session.
  • runs — one execution. The user says something, the system runs a round, and that round is a run. It has a lifecycle: queued, running, finished, failed, cancelled.
  • messages — immutable facts. What the user said and what the agent replied, never modified once written.

The most common beginner move is to skip run and hang messages directly off the session. You cannot skip it: without the run layer you have nowhere to answer whether this execution finished. Retries, timeouts, cancellation, and cost attribution all need an entity representing one execution to hang from.

SQLSQL
create table sessions (
  id          text primary key,
  user_id     text not null,
  title       text not null,
  created_at  timestamptz not null default now(),
  updated_at  timestamptz not null default now()
);
create index sessions_user_id_idx on sessions (user_id);
 
create table runs (
  id                text primary key,
  session_id        text not null references sessions (id),
  status            text not null,
  idempotency_key   text not null unique,   -- the final arbiter of idempotency, next section
  input             text not null,
  error             text,
  prompt_tokens     integer not null default 0,
  completion_tokens integer not null default 0,
  created_at        timestamptz not null default now(),
  updated_at        timestamptz not null default now()
);
create index runs_session_id_idx on runs (session_id);
 
create table messages (
  id          text primary key,
  session_id  text not null references sessions (id),
  run_id      text references runs (id),
  role        text not null,
  content     text not null,
  seq         integer not null,
  created_at  timestamptz not null default now(),
  unique (run_id, seq)                        -- the basis for ordering and idempotent replay
);
create index messages_session_id_idx on messages (session_id);

Three design decisions deserve their own note:

The primary key is text, not an auto-incrementing integer. Because this id has to exist before the write — the gateway needs to put runId into that 202 response body immediately. An auto-increment key is unknown until the database has inserted the row, so you would have to write first and answer second, wedging a database round trip into the user's waiting path for nothing. Generate a UUID or a ULID application-side and the gateway can fix the id the moment a request arrives, which also leaves room for sharding later.

Index only the columns that are genuinely queried. Today there are two query paths: list a user's sessions (sessions.user_id) and load a session's history (messages.session_id). Surplus indexes are not free; each one slows writes.

unique(run_id, seq) is this week's most important constraint. seq starts at 0, is contiguous, and skips nothing, and it is the sole basis on which D11's resumption picks up from the right position. Today you write that line of SQL almost casually, but from D11 onwards the entire ordered-delivery chain rests on it.

Two things about runs: the state machine and the idempotency key

An execution is not instantaneous; it has a lifespan, so it needs status. This week fixes this set:

StatusMeaningWho writes it
pendingpersisted, waiting to be takengateway
runningthe worker took it and is running the agent loopworker
streamingit has started emitting textworker
donefinished normallyworker
failedretries exhausted, definitively failedworker
cancelledcancelled by the user, or merged away by a newer messagegateway or worker

The normal path is pending to running to streaming to done, with failed and cancelled as the abnormal exits. Note that pending may go straight to failed — a repeatedly failing message may be declared dead before the run ever starts, and D9 uses that edge when handling poison messages; forbidding it means the act of recording a failure itself throws an illegal-transition error, in the one place you least want strictness. What matters is not which statuses exist, but explicitly forbidding illegal transitions. Take the most painful example: a done run pushed back to running by some late duplicate message, so the worker runs again and overwrites the reply — and the user watches an answer they already received turn into a different answer. Write a few-line transition allowlist and that class of incident disappears.

The second thing is the idempotency key. At-least-once delivery is a message bus's default semantics (D9 covers it fully), and with users double-clicking and clients retrying after a timeout, the same sentence arriving at the gateway twice is inevitable rather than exceptional. The check-in desk's countermeasure is settling against the boarding-pass number: hand over your document twice and the system recognizes the same itinerary and does not print two passes.

The idempotency key is that number. It must be determined by the request's content rather than being random — a random UUID differs every time, which is the same as having no idempotency. A sensible derivation hashes the sessionId, the client message id, and the message content together; when the client has no message id, fall back to the content plus a coarse time window.

run.js
import { createHash } from 'node:crypto'
 
// The idempotency key must be determined by content: a random value differs every time,
// which is the same as no idempotency
export function idempotencyKeyFor(sessionId, clientMessageId, message) {
  const material = clientMessageId ?? message
  return createHash('sha256').update(`${sessionId}:${material}`).digest('hex')
}
 
// The transition allowlist: anything not listed here is refused
const ALLOWED = new Map([
  // pending can go straight to failed: a poison message may be declared dead before the
  // run ever starts (D9 uses this)
  ['pending', new Set(['running', 'cancelled', 'failed'])],
  ['running', new Set(['streaming', 'done', 'failed', 'cancelled'])],
  ['streaming', new Set(['done', 'failed', 'cancelled'])],
  ['done', new Set()], // terminal: pushing done back to running overwrites a delivered reply
  ['failed', new Set()],
  ['cancelled', new Set()],
])
 
export function canTransition(from, to) {
  return ALLOWED.get(from)?.has(to) ?? false
}

Having computed the key, what actually makes idempotency hold is that unique constraint in the database, and in SQL it is one statement:

SQLSQL
insert into runs (id, session_id, status, idempotency_key, input)
values ($1, $2, 'pending', $3, $4)
on conflict (idempotency_key) do nothing
returning id;

On a conflict the statement returns zero rows, meaning somebody inserted first, so you read that existing run back and return its id to the user — two requests, one run, one runId.

Drizzle: write the schema once and derive both SQL and types from it

Copy the same information into several places and an edit will eventually miss one. Your name on a driving license, a passport, and a bank record: if each holds its own copy, changing it means three separate visits and missing one leaves them inconsistent. The right shape is for all of them to derive from one ledger.

Creating tables traditionally means writing it three times: a create-table SQL file, an ORM model, and a TypeScript interface. Three hand-written copies drift eventually — somebody adds a column and forgets the type, the compiler says nothing, and it blows up at runtime. Drizzle's approach: write the schema once, let it generate the migration SQL, and let it infer the types of your queries.

schema.ts
// Drizzle: the schema is an ordinary TS declaration, and migrations and types derive from it
export const runs = pgTable(
  'runs',
  {
    id: text('id').primaryKey(),
    sessionId: text('session_id')
      .notNull()
      .references(() => sessions.id),
    status: text('status').notNull().$type<RunStatus>(), // narrowed to a union, not any string
    idempotencyKey: text('idempotency_key').notNull().unique(),
    input: text('input').notNull(),
    error: text('error'), // nullable, so the query result's type is string | null
    promptTokens: integer('prompt_tokens').notNull().default(0),
    completionTokens: integer('completion_tokens').notNull().default(0),
    createdAt: timestamp('created_at', { withTimezone: true }).notNull().defaultNow(),
    updatedAt: timestamp('updated_at', { withTimezone: true }).notNull().defaultNow(),
  },
  (t) => [index('runs_session_id_idx').on(t.sessionId)]
)

The four versions look very different and do one thing: declare the table structure as a type in the host language, so unique constraints, nullability, and foreign keys exist in exactly one place. After that a query's return value carries its type, and a wrong column name is a compile error rather than a production incident.

Declared, you generate the migration. Drizzle's pipeline is two commands:

BashBash
pnpm drizzle-kit generate    # diff the schema against existing migrations, emit numbered SQL
pnpm drizzle-kit migrate     # run them in order, recording them in __drizzle_migrations

The point is that the first command produces files that go into version control and through code review. Plenty of people reach for the "just sync the database to look like the schema" mode (Drizzle calls it push), which is fine to play with locally and a disaster in production: no replayable history, no moment for review, and no way to say which version the live database is stopped at. A migration is an ordered chain, not a snapshot.

Getting Postgres up: compose and "actually usable"

The last step is bringing the database up, and here is the trap everybody hits: a container that has started is not a container that is usable. Throwing the terminal's main breaker does not mean the baggage system is turning, and after the Postgres process starts it still spends a few seconds initializing, during which connecting gets you a refusal. depends_on by default waits only for the container to start, not for it to be ready, so the gateway jumps the gun, the first query errors, and the process exits — three failures in ten local runs, and randomly red in CI.

The fix is a health check plus a conditional dependency:

YAMLYAML
services:
  postgres:
    image: pgvector/pgvector:pg16
    environment:
      POSTGRES_USER: koda
      POSTGRES_PASSWORD: koda
      POSTGRES_DB: koda
    ports: ['5508:5432']
    healthcheck:
      test: ['CMD-SHELL', 'pg_isready -U koda -d koda']
      interval: 2s
      timeout: 3s
      retries: 15
    volumes: [koda-pg:/var/lib/postgresql/data]
 
  gateway:
    build: .
    environment:
      DATABASE_URL: postgres://koda:koda@postgres:5432/koda
    depends_on:
      postgres:
        condition: service_healthy    # wait for usable, not for started
    ports: ['3000:3000']
 
volumes:
  koda-pg:

The image is pgvector/pgvector:pg16 rather than the official postgres:16 — it is the official image with the vector extension installed, which costs nothing now and spares you swapping images and reloading data when D12 builds long-term memory. A dependency you can foresee is far cheaper installed a day early than migrated a day late.

The gateway needs its own health endpoint, and it must distinguish two kinds. A liveness probe answers only whether the process is still there; a readiness probe answers whether traffic may be sent right now. The difference only shows under failure: the database wobbles, readiness goes red, this instance temporarily takes no traffic and returns automatically when things recover. Use a liveness probe there instead and the orchestrator kills and restarts the process, which is slower.

Source Reading

Hands-On Lab

🧪 D8 lab: the mini-koda monorepo skeleton plus PG in docker plus migrations

Code location: labs/agent-30days/day-08-mini-koda-skeleton

Acceptance criteria:

  1. All six self-checks under MOCK=1 SELFTEST=1 pnpm start pass (starter/ as-is passes 1, 2 and 6 and fails 3, 4 and 5, so run it once and remember what that looks like).
  2. Check 3: sending the same client message id twice returns the same runId both times, and the runs table holds one row rather than two.
  3. Check 4: moving pending to running is allowed, and moving done back to running is refused with an explicit error.
  4. Check 5: writing a message with seq 0 twice for the same runId is stopped by the unique constraint, and the messages table still holds one row.
  5. Check 6: the worker side reads the pending run the gateway just persisted (read-only today, no execution); pnpm typecheck passes with no any.

This lab is the week's foundation and the next six days grow on top of it. The directories split into four layers, gateway / worker / shared / infra — the text says these are three independent packages in production, and the lab expresses the same thing with directories so each lab remains one independently installable project. Under MOCK=1 the storage is in-memory, and it is not a stub: unique constraints, composite unique constraints, and the state machine are genuinely implemented over Maps and arrays — so you can see idempotency actually work without installing Docker. To run against a real database, docker compose up -d and set DATABASE_URL; the same business code runs and only the implementation under src/infra/ changes.

  1. Create the four layers gateway / worker / shared / infra, then run MOCK=1 SELFTEST=1 pnpm start as-is and see what the failures on checks 3, 4 and 5 look like.
  2. Bring up Postgres with docker compose (image pgvector/pgvector:pg16, with a pg_isready health check) and put DATABASE_URL into .env.
  3. Define the three tables with Drizzle in shared, run drizzle-kit generate to emit the migration file, migrate it into the database you just started, and confirm in the database that both unique constraints really exist.
  4. Complete idempotencyKeyFor and the in-memory unique-constraint check, then rerun the self-checks: 3 and 5 pass, and sending the same message twice produces one runId.
  5. Complete the state-machine allowlist until check 4 passes, then run the seed script and use the worker-side read-only query to confirm it sees the same pending run — executing it is tomorrow's job.

Interview Questions

Today's four questions are in the bank below, weighted toward the monolith-versus-split trade-off, stateless services, idempotent writes, and the primary-key and index design of the three tables. Expand a question and read the analysis before the key points — the follow-up on question 3, on why check-then-insert is not idempotency, is where this chapter gets pressed hardest, so do not skip it.

Checklist and Tomorrow

  • Draw a three-part gateway / message-bus / worker architecture and state each part's responsibility
  • Define the sessions, runs, and messages tables with Drizzle and generate a migration
  • Explain why a stateless gateway and idempotent writes are prerequisites for horizontal scaling
  • Say who writes each of the run's six statuses, and why illegal transitions must be explicitly forbidden
  • All 5 acceptance criteria of the lab pass, with all six self-checks green
  • Answer at least 3 of the 4 interview questions without looking at the key points

Tomorrow (D9) we fill in the empty box in the middle of the three-part diagram: Redis Streams as the message bus, so the gateway persists a run and then delivers it into the stream, and a worker fishes it out, genuinely executes it, and writes back to the messages table. Why not do both today? Because where the state goes and how the message travels are two independent decisions, and learning them together leaves you unable to tell a table-design mistake from a delivery-semantics mistake when something breaks. The idempotency_key fixed today goes to work immediately tomorrow — a message bus is at-least-once delivery, and it is the only thing standing between you and duplicate consumption.

Interview questions

  • Why do production agent services usually split a gateway from workers, and when should you not split?为什么生产级 Agent 服务通常要把 Gateway 和 Worker 拆开？什么情况下不该拆？
    Common in ChinaCommon overseasBasic#architecture#scalability

    How to reason about it · think before answering

    1. The hinge is the second half. Answering only 'decoupling and scalability' sounds copied from a textbook; the interviewer wants to know which concrete symptom forced you to split, and what splitting costs.
    2. Offer a reusable chain: one agent run is long and unpredictable (model latency plus several tool calls, seconds to tens of seconds), while the ingress path carries all traffic and must stay in the millisecond range. Put workloads three orders of magnitude apart in the same process and the slow one starves the fast one.
    3. Make the symptom concrete: a single process running a dozen long executions saturates connections and memory, health checks start timing out, the orchestrator declares the instance dead and restarts it, and every in-flight run dies with it. That story lands harder than any abstract argument.
    4. Then state the rule: anything a worker can do should not live in the gateway, which keeps only auth, rate limiting, persistence and dispatch — four steps with bounded latency. After the split the stateless gateway scales with traffic while worker concurrency is tuned against model quota; the two curves were never the same.
    5. Volunteer the cost, which is where candidates separate: the contract becomes 202 instead of 200 so clients need a second subscribe round trip, you now operate a bus and a runs table, tracing spans more hops, and local development needs more processes. So do not split when a run takes a few hundred milliseconds, uses no tools, and serves modest traffic.
    6. Expect the follow-up: could a thread pool or child processes do instead? They ease starvation but fix neither 'restart loses in-flight work' nor 'two instances cannot see each other's state', because the root cause is state living inside the process, not the concurrency model.

    分析过程 · 先想清楚再作答

    1. 题眼在后半句。只答「解耦、可扩展」是从架构书上抄来的，面试官想知道你有没有被某个具体现象逼着拆过——所以答案里必须出现「什么现象」和「不拆的代价」。
    2. 先给一条可复用的推导链：Agent 的一次执行是长耗时且时长不可预测的（模型响应加上多轮工具调用，几秒到几十秒），而接入层要承载全部流量、必须是毫秒级的短请求；把两种时长量级差三个数量级的工作放进同一个进程，慢的那一类必然会挤占快的那一类的资源。
    3. 把现象说具体：单进程时一台机器同时跑十几次长执行，连接与内存被占满，新来的健康检查开始超时，编排系统判定实例已死并重启它——正在跑的执行全部陪葬。这个「健康检查被自己的业务拖挂」的故事比任何抽象论证都有说服力。
    4. 然后给判据：能在 Worker 做的不放 Gateway，接入层只留鉴权、限流、落库、投递这四件耗时确定的事。拆开之后 Gateway 无状态可以任意扩缩，Worker 的并发度可以按模型配额单独调，两者的扩容曲线本来就不一样。
    5. 主动说代价，这是区分度所在：接口语义从 200 变成 202，客户端要多一次订阅往返；系统里多了一条总线和一张 runs 表，可观测性和排障链路都变长；本地开发要起更多进程。所以单次执行只有几百毫秒、没有工具调用、日活很小的场景不该拆——那时候拆分带来的复杂度远大于收益。
    6. 可以预期的追问：不拆但用线程池或者子进程行不行？答案是能缓解「挤占」但解决不了「重启即丢失」和「多实例状态不共享」，因为那两件事的根因是状态在进程里，不是并发模型不对。

    Key points

    • A run takes seconds to tens of seconds while ingress requests are millisecond-scale; in one process the long work starves the short work
    • Three concrete failure modes: restarts lose in-flight runs, multiple instances hold separate state, and long runs stall health checks so the orchestrator kills a healthy instance
    • The rule is that anything a worker can do stays out of the gateway, which keeps only auth, rate limiting, persistence and dispatch
    • After splitting, gateways scale on traffic and workers scale on model quota — two independent curves
    • Costs: a 202 contract plus a subscribe round trip, an extra bus and table to operate, longer traces; skip the split for sub-second runs with no tool calls

    答题要点

    • 一次 Agent 执行是几秒到几十秒的长任务，接入层是毫秒级短请求，两者同进程时长任务必然挤占短请求的资源
    • 单进程的三个具体死法：重启丢掉在途执行、多实例状态各存各的、长执行把健康检查拖超时导致实例被误杀
    • 判据是「能在 Worker 做的不放 Gateway」，接入层只留鉴权、限流、落库、投递
    • 拆开后 Gateway 无状态按流量扩容、Worker 按模型配额扩容，两条曲线可以独立调
    • 代价是接口从 200 变 202、多一次订阅往返、排障链路变长；单次执行仅几百毫秒且无工具调用的场景不该拆
  • What makes a service stateless, what does that mean for horizontal scaling, and are workers stateful?什么是无状态服务？它对水平扩展意味着什么？Worker 算不算有状态？
    Common in ChinaCommon overseasIntermediate#stateless#scalability

    How to reason about it · think before answering

    1. The trap is reading the word literally. Many candidates say 'it stores nothing', which is wrong — stateless services write to databases all day. The discriminator is whether you can define it precisely.
    2. One sentence does it: stateless means state does not live in the process handling the request, so any instance can serve any request. Turn it into a self-check: kill a random instance — does any user's data exist only there? Only 'no' is stateless.
    3. Derive three scaling consequences: a new instance needs no warm-up or data sync and starts serving the moment it joins the load balancer; any instance can be killed at will, which is what makes rolling deploys and spot instances viable; and no sticky sessions are needed, whereas stickiness means rebalancing during a scale-up cuts existing conversations.
    4. Answer the worker half carefully: it holds execution progress, not user data — which turn it is on, which tools it called, and later a lease. User data always lives in the database. So 'stateful' here means 'holding unfinished work', and the consequence is that you cannot kill it freely: drain first, refuse new work, let the current run finish.
    5. Expect the follow-up: does an in-memory cache break statelessness? It depends on whether losing it causes wrong behavior. A pure accelerator that only costs latency is fine; the moment a user's session exists only in one machine's memory you are silently relying on stickiness, and the next scale-up will prove it.

    分析过程 · 先想清楚再作答

    1. 这题的陷阱是字面理解。很多人答成「不保存任何数据」，那是错的——无状态服务当然会写数据库。区分度在于你能不能给出准确定义。
    2. 准确定义只有一句：无状态指的是**状态不留在处理请求的那个进程身上**，因此任意一台实例都能处理任意一个请求。把它翻译成一个自检问题就很好用：随便杀掉一台实例，有没有任何用户的数据只存在于那台机器上？答「没有」才是无状态。
    3. 再推出水平扩展的三个后果：新实例不需要预热或同步数据，接上负载均衡立刻能干活；任意实例可以随时被杀，滚动发布和抢占式实例才成立；不需要会话粘连，而粘连一旦存在，扩容时的重新分配就会打断老用户的会话。
    4. Worker 那一问要答得有分寸：它持有的不是用户数据，而是一次执行的进度（跑到第几轮、调了哪些工具、后面还会加上一个租约）。用户数据始终在数据库里。所以说它有状态，指的是「手上有活没交代完」，后果是不能随便杀——必须优雅停机，先拒绝新任务再等手头的跑完。
    5. 可以预期的追问：内存缓存算不算破坏了无状态？答案是看丢了会不会出错。纯粹用于加速、丢了只是变慢的缓存不破坏无状态；一旦某个用户的会话只存在于某台机器的内存里，你就已经在偷偷依赖粘连了，扩容那天必然出事。

    Key points

    • Stateless means the state does not live in the request-handling process, so any instance serves any request — not that nothing is stored
    • Self-check: kill any instance and ask whether any user's data existed only there
    • Three scaling prerequisites: no warm-up, any instance disposable, no sticky sessions
    • Workers are stateful in the sense of holding run progress, not user data, so they need graceful drain rather than a hard kill
    • A pure accelerator cache is fine; in-memory data that is the only copy is implicit stickiness

    答题要点

    • 无状态的准确含义是状态不留在处理请求的进程里，任意实例都能处理任意请求，而不是「不存数据」
    • 自检方法：随便杀一台实例，是否有用户的数据只存在于那一台上
    • 水平扩展的三个前提：新实例无需预热、任意实例可被随时杀掉、不需要会话粘连
    • Worker 的有状态指的是持有一次执行的进度而不是用户数据，后果是必须优雅停机而不能随便杀
    • 只加速、丢失只降速的缓存不破坏无状态；承载唯一副本的内存数据等于隐式的会话粘连
  • With at-least-once delivery, how do you guarantee a redelivered message does not create two runs?消息总线是至少一次投递，同一条消息被重复投递时，怎么保证不会产生两条 run？
    Common in ChinaCommon overseasDeep dive#idempotency#database#reliability

    How to reason about it · think before answering

    1. This question is about which layer idempotency lives in. Anyone who answers 'check whether it exists, then insert' has usually just failed it — that is exactly the answer being screened out.
    2. State the premise: duplicates are not accidents. The bus is at-least-once, clients retry on timeout, users double-click. The same message arriving twice is certain, so the goal is not to prevent duplicates but to make duplicates produce the same result.
    3. Then derive the key: idempotency needs a key derived from request content. A random UUID differs every time and buys nothing; hash the session id, the client message id and the message body together, falling back to content plus a coarse time bucket when the client has no id.
    4. Land it in storage: put a unique constraint on that column in the runs table, write the insert as on-conflict-do-nothing, and when it returns zero rows read back the existing run and return the same run id. Two requests, one run, one id.
    5. Explain why check-then-insert fails, which is the whole point: two gateway instances can query, both see nothing, and both insert. The window between the two statements cannot be closed in application code, it is too narrow to reproduce under load tests, and it leaks a few bad rows every day in production. The database's unique constraint has to be the final arbiter; the application-level check only saves a wasted insert.
    6. Expect the follow-up: what about duplicate execution on the consumer side? The unique constraint gives you one run, but a worker can still receive it twice, so status changes need conditional updates (move to running only if the current status is pending) plus an explicit transition whitelist that blocks a finished run from being pushed back to running and overwriting a reply the user already saw.

    分析过程 · 先想清楚再作答

    1. 这题在考幂等的落点在哪一层。凡是答「在代码里先查一下有没有，没有再插入」的，基本当场结束——因为那正是这题想筛掉的答案。
    2. 先把前提摊开：重复不是意外。总线是至少一次语义、客户端会超时重发、用户会手抖双击，同一句话到达两次是必然事件。所以设计目标不是「避免重复到达」，而是「重复到达时结果相同」。
    3. 然后给推导：幂等需要一个由请求内容决定的键。随机 UUID 每次都不同，等于没有幂等；正确取法是把会话 id、客户端消息 id、消息内容拼起来做哈希，客户端没有消息 id 时退用内容加一个粗粒度时间窗。
    4. 结论落在存储层：在 runs 表的这一列上加唯一约束，插入写成「冲突就什么都不做」，返回零行时回查那条已有的 run，把同一个 runId 返回给用户。两次请求、一条 run、一个 runId。
    5. 解释为什么「先查后插」不行，这是本题的分水岭：两个 Gateway 实例可以同时查、同时发现没有、同时插入，这两步之间有一个应用层拦不住的时间窗；它窄到压测复现不出来，上线后每天漏几条。**幂等的最终裁判必须是数据库的唯一约束**，应用层的判断只是为了少一次插入尝试。
    6. 可以预期的追问：那消费侧的重复执行呢？答：唯一约束保证了只有一条 run，但 Worker 可能重复拿到同一条 run，所以状态迁移也要带条件更新（只有当前状态是 pending 时才能改成 running），并且用一个显式的迁移白名单挡住「已完成的 run 被推回运行中」这种会覆盖用户已收到回复的情况。

    Key points

    • Redelivery is certain, so the goal is identical outcomes on duplicates, not preventing duplicates
    • The idempotency key must be derived from request content — session id plus client message id plus body, hashed; a random UUID buys nothing
    • Put a unique constraint on that column, insert with on-conflict-do-nothing, and read back the existing run when zero rows return
    • Check-then-insert races under concurrency; the window between the statements cannot be closed in application code, so the unique constraint must be the final arbiter
    • On the consumer side add conditional status updates and a transition whitelist so a finished run is never re-run or overwritten

    答题要点

    • 重复投递是必然事件，设计目标是「重复到达时结果相同」，不是「避免重复」
    • 幂等键必须由请求内容决定：会话 id 加客户端消息 id 加内容做哈希，随机 UUID 等于没有幂等
    • 在 runs 的幂等键列上建唯一约束，插入用「冲突就什么都不做」，零行时回查已有 run 返回同一个 runId
    • 先查后插在并发下必然出双份，两条语句之间的时间窗应用层拦不住，幂等的最终裁判是数据库唯一约束
    • 消费侧还要用条件更新加状态迁移白名单，避免同一条 run 被重复执行或把已完成的回复覆盖掉
  • How would you design primary keys and indexes for sessions, runs and messages, and why avoid auto-increment ids?sessions / runs / messages 这三张表你会怎么设计主键与索引？为什么不用自增主键？
    Common in ChinaCommon overseasIntermediate#database#schema-design#idempotency

    How to reason about it · think before answering

    1. It looks like a trivia question, but every choice sits on a concrete constraint. The test is whether you can say what breaks if you choose otherwise.
    2. Start with why three tables rather than one: the grains differ. A session is a long-lived container, a run has a lifecycle and can fail and be retried, a message is an immutable fact. Without the run layer there is nowhere to answer 'did this finish', 'should we retry', or 'what did this turn cost'.
    3. Use text primary keys generated in the application (UUID or ULID), because the gateway must put the id into the 202 response before the row is written. Auto-increment ids are only known after the insert, which parks a round trip in the user's wait path and cannot be pre-allocated across instances. A bonus is that sharding later needs no renumbering.
    4. Index by query path, not by instinct: sessions need an index on user_id to list a user's conversations, messages need one on session_id to load history, and foreign key columns need indexes or deleting a parent row triggers a full scan. Extra indexes are not free — each one slows writes.
    5. The two unique constraints carry the design: a unique idempotency key on runs blocks duplicate delivery, and a composite unique on run id plus sequence in messages both fixes output ordering for one run and lets a reconnect replay idempotently by sequence. The sequence must start at zero and never skip, otherwise resume cannot find the cut point.
    6. Expect the follow-up: ULID or UUIDv4? Choose ULID or UUIDv7 — they are time-ordered so inserts land at the right edge of the B-tree, whereas random UUIDv4 scatters writes, splits pages and hurts cache hit rates. Mentioning this shows you have watched write performance.

    分析过程 · 先想清楚再作答

    1. 这题看着像八股，其实每一个选择背后都有一个具体约束。判断标准是：你能不能为每个决定说出「不这么做会发生什么」。
    2. 先讲为什么是三张表而不是一张：粒度不同。会话是长期容器，一次执行有生命周期且可能失败重来，消息是不可变事实。少了「一次执行」这一层，你就没有地方回答「这次跑完没有」「该不该重试」「这轮花了多少钱」。
    3. 主键选文本型的应用侧 id（UUID 或 ULID），理由是接入层必须在写库之前就把 id 放进 202 响应体返回给客户端；自增主键要等数据库插完才知道值，那次往返就被卡在用户的等待路径上，而且多实例无法预分配。附带好处是将来分库分表不用重编号。
    4. 索引按查询路径建，不按直觉建：按用户拉会话列表要 sessions 的 user_id 索引，按会话拉历史要 messages 的 session_id 索引，外键列本身要索引否则删除父行会全表扫。多余的索引不是免费的，每个都让写入变慢。
    5. 两条唯一约束才是这套设计的灵魂：runs 的幂等键唯一，挡住重复投递；messages 的「run id 加序号」复合唯一，既保证同一次执行的输出顺序稳定，又让断线重连可以按序号幂等回放。序号要从 0 开始、连续、不跳号，否则续传就找不到断点。
    6. 可以预期的追问：ULID 和 UUIDv4 选哪个？答 ULID 或 UUIDv7——它们按时间有序，插入时集中在 B 树右端，不像 UUIDv4 那样随机分布导致页分裂和缓存命中率下降。这个细节能直接体现你关心过写入性能。

    Key points

    • Three tables for three grains: a long-lived session, a run with a lifecycle, and immutable messages; without runs you cannot answer completion, retry or cost questions
    • Application-generated text ids, because the gateway must return the run id in the 202 before the write, and auto-increment ids cannot be pre-allocated across instances
    • Index the real query paths — user_id on sessions, session_id on messages, plus foreign key columns; extra indexes slow writes
    • Two unique constraints carry the design: a unique idempotency key on runs, and a composite unique on run id plus sequence in messages for ordering and idempotent replay
    • Prefer time-ordered ids such as ULID or UUIDv7 over random UUIDv4 to avoid page splits and cache misses

    答题要点

    • 三张表对应三种粒度：会话是长期容器、run 是一次有生命周期的执行、message 是不可变事实；少了 run 就无法回答是否跑完、该不该重试、花了多少钱
    • 主键用应用侧生成的文本 id，因为 Gateway 要在写库之前把 runId 放进 202 响应里，自增主键必须等插入完成且无法跨实例预分配
    • 索引按实际查询路径建：sessions 的 user_id、messages 的 session_id、以及外键列；多余索引会拖慢写入
    • 两条唯一约束是灵魂：runs 的幂等键唯一挡重复投递，messages 的「run id 加序号」复合唯一保证保序与幂等回放
    • id 优先选 ULID 或 UUIDv7 这类时间有序的方案，避免随机 UUID 造成的页分裂与缓存失效

Comments