Dayward AI
中文
Week 1 · D4About 5 hours

Chunking Strategies: Five Approaches — Fixed, Recursive, Structure-Based, Parent-Child, and Semantic — and Choosing by Evaluation, Not Intuition

Chunking is the step in the whole pipeline most often decided by gut feel, yet it affects results the most. Today implement five mainstream chunking approaches one by one, explain where each fits and fails, cover what overlap and contextual chunk headers are for, then hold all five up against the same ruler with a fixed set of questions.

Today's goals 0/3

Sign in to tick these off and save your progress.

Today's Goals

  1. Explain the assumptions and failure scenarios of fixed-length, recursive, structure-based, parent-child, and semantic chunking
  2. Explain what problems overlap and contextual chunk headers each remedy, and state the cost of each
  3. Design a set of questions that quantifies retrieval hit rate across different chunking approaches, and decide with data instead of intuition

Yesterday you unified PDF, HTML, and Markdown sources into one stream of nodes with heading paths, clearing the parsing stage. Today answers the question immediately after: at what granularity and where should that stream be cut so it can be retrieved? Come back and tick off the three goals.

Plain-Language Walkthrough

One: chopping vegetables — too large and the seasoning cannot get in, too small and you cannot taste what the dish is

My family's braised pork has an iron rule: cut the meat into two-centimetre cubes. Cut it at five and the outside is salty while the inside is still pale, with the sauce unable to get in; cut it at half a centimetre and it takes the flavor, but after braising the pot is full of shreds and you cannot tell belly from loin in what you pick up.

Chunking faces the same contradiction with different names on each end.

Too large and the seasoning cannot get in. Retrieval compares the question against a chunk. A two-thousand-character chunk may contain one sentence relevant to the question with over nineteen hundred characters of noise diluting that signal. And even when retrieved, what goes into the model's context is those two thousand characters — you paid a whole chunk's price for one sentence and took up room another chunk could have used.

Too small and you cannot taste what the dish is. A thirty-character chunk reads "the retention period is 30 days, after which it is permanently deleted." Precise in itself, and whose retention period? The recycle bin's? Logs? Backups? The moment the chunk was cut, it lost contact with its context and became an isolated statement nobody can place. Retrieval finds it and the model cannot explain it.

So chunking is never picking a number but finding a position between two quantities pulling against each other: signal-to-noise ratio and completeness. Which is why online advice like "just use 512 tokens with 50 overlap" all looks right and none of it works quite right — it omits the preconditions: what kind of document, what kind of question, and how much context budget retrieval was given.

Today's job is concrete: implement five mainstream approaches behind one interface, cut the same corpus with each, and measure their hit rates with the same set of questions. After today, how large a chunk should be is not a matter of opinion for you but the output of one command.

Two: fixed length and recursive — the two simplest, and what a separator priority list is worth

Fixed length is the most direct: ignore all structure and cut after 400 characters. Its assumption is that importance is uniformly distributed across the text, so where you cut does not matter. That assumption roughly holds for novels and chat logs and fails completely for technical documentation.

Its problem is measured clearly in our lab: 30 documents cut into 60 chunks with 48.3% of chunks ending mid-sentence. Nearly half the chunks start and end in the middle of a sentence.

The only remedy is overlap: let adjacent chunks share a tail so the split sentence is complete in at least one of them.

fixed.js
// Fixed-length chunking: stride = size - overlap, so adjacent chunks share `overlap` characters
export function chunkFixed(text, size = 400, overlap = 80) {
  const chunks = []
  const step = Math.max(1, size - overlap)
  for (let start = 0; start < text.length; start += step) {
    const piece = text.slice(start, start + size).trim()
    if (piece) chunks.push(piece)
    if (start + size >= text.length) break // the last chunk already reaches the end; do not cut another tail
  }
  return chunks
}

Recursive splitting takes a different tack: since a cut has to happen, cut where it costs least. It prepares a separator table ordered from strongest to weakest semantic boundary — blank line, newline, full stop, semicolon, comma, and finally a hard cut — cuts with the strongest first, cuts any still-over-long piece with the next one down, and retreats level by level.

That table's order is its entire wisdom. On the same corpus with the same 400-character limit, recursive produces 61 chunks, essentially the same count as fixed length, while the mid-sentence rate falls from 48.3% to 1.6%. Same chunk count, same cost, only a different place to cut.

Three: cutting by document structure — heading levels are the semantic boundaries the author already drew

The first two guess where the boundaries are. But for a document with heading levels there is nothing to guess — the author drew them for you the moment they wrote each level-two heading.

Structure-based cutting takes headings as chunk boundaries directly: one section per chunk, with the heading path recorded in the chunk's metadata along the way. On our corpus it produces 153 chunks averaging 102 characters, with a mid-sentence rate of 2.0%. The count is two and a half times the others because a section is naturally shorter than 400 characters.

structure.js
// Cut by heading level: keep a heading stack, and on a level-N heading truncate to N-1 and push
export function splitSections(markdown) {
  const sections = []
  const stack = []
  let buffer = []
  let path = []
 
  const flush = () => {
    const text = buffer.join('\n').trim()
    if (text) sections.push({ headingPath: [...path], text })
    buffer = []
  }
 
  for (const line of markdown.split('\n')) {
    const heading = /^(#{1,6})\s+(.*)$/.exec(line)
    if (heading) {
      flush() // the previous section ends here
      stack.length = Math.min(stack.length, heading[1].length - 1)
      stack[heading[1].length - 1] = heading[2].trim()
      path = stack.filter(Boolean)
      continue
    }
    buffer.push(line)
  }
  flush()
  return sections
}

This is the best value of the five: nearly free and close to semantic chunking in effect. But it has a hard precondition — yesterday's parsing did not lose the structure. The moment a parser flattens headings into ordinary paragraphs, this road ends. That is why day three worked so hard to preserve the heading path: structure is the one thing you get free at parse time, and once lost it can never be recovered.

One degeneration to guard against: structure-based cutting produces wildly uneven chunk lengths. Our corpus's longest section is 415 characters and its shortest is a dozen. So the implementation needs an upper limit, falling back to recursive splitting past it, or one over-long section takes you straight back into the too-large pit.

Four: parent-child chunking — small chunks to retrieve, large chunks to feed the model

The first three approaches all assume one thing: the unit retrieved and the unit placed in the context are the same thing. Parent-child chunking separates the two for the first time.

The reasoning is not complex. Retrieval likes small chunks for their signal-to-noise ratio, and models like large chunks for their complete context. So cut both: small chunks go into the index to be found, and once found the small chunk is not given to the model — instead its whole section is filled back in via parentId. Retrieval at two-hundred-character precision, generation with eight-hundred-character completeness.

The cost is equally direct. The index gains a mapping layer to maintain; a document update recomputes both sets; and more seriously, the context arithmetic — with the same 600-token budget, parent-child fits 4.8 chunks on average, but each new child chunk may drag a whole parent section in, so the budget drains faster than it looks. It ties with structure-based cutting on our corpus (both 7 of 8), because the corpus's sections are not long and parent-child's advantage has not yet come into play.

How a parent-child index is built, how backfill works, and how the cost is computed is day eleven's complete subject. Today only remember the concept: small chunks retrieve, large chunks backfill, and the retrieval unit need not be the context unit.

Five: semantic and proposition chunking — where the cost is and whether it is worth it

The first four all guess where the topic changed by rule. Semantic chunking does not guess — it embeds sentence by sentence, and where the cosine similarity between adjacent sentences drops is where the topic changed.

Proposition chunking goes further: have a model rewrite each passage into independently standing statements, one chunk each. It can restore a sentence like "its retention period is 30 days" into "the recycle bin's retention period is 30 days," giving the best retrieval and the highest price — every passage goes through a model, and the rewriting itself may introduce factual errors.

semantic.js
// Take the breakpoint threshold from a percentile rather than hard-coding an absolute value
export function findBreakpoints(vectors, percentile = 0.25) {
  const sims = vectors.slice(1).map((v, i) => cosine(vectors[i], v))
  const sorted = [...sims].sort((a, b) => a - b)
  // Break only at this document's least-alike 25%: the threshold adapts to the document
  const threshold = sorted[Math.floor(sorted.length * percentile)] ?? -1
  return sims.map((sim, i) => (sim <= threshold ? i + 1 : -1)).filter((i) => i > 0)
}

Note that the most important line above is how threshold is computed. Do not hard-code the similarity threshold as an absolute value. Different embedding models have entirely different similarity distributions: with one, any two sentences sit above 0.8; with another, adjacent sentences of one passage reach only 0.4. Hard-code 0.55 and a model change turns it into break everywhere or break nowhere. A percentile is adaptive: whatever the distribution, break only at this document's least-alike 25%.

As for whether it is worth it, our lab's answer is not on this corpus: semantic chunking produces 90 chunks with a hit rate of 7 of 8, exactly the same as the free structure-based approach, at the price of a full embedding pass. That does not mean semantic chunking is useless; it means when your documents already have clear heading levels, the author did the semantic chunking for you free of charge. Its real place is unstructured long text: meeting transcripts, support call logs, continuous paragraphs out of a scan.

Six: contextual chunk headers — giving an isolated chunk a line saying who it is

Back to the first section's example: nobody knows what the "the retention period is 30 days" chunk is about.

A contextual chunk header prefixes each chunk with a line locating it, and the combined text is what gets indexed. The cheap version composes it from day three's heading path: "Drive and File Management, Recycle Bin: the retention period is 30 days…". The expensive version has a model read the whole document and write a line for each chunk — the approach in Anthropic's contextual retrieval post, whose implementation and full evaluation are day eleven's, with today needing only awareness that the technique exists.

But today must state its cost first, and this cost is one we genuinely measured: adding heading-path headers to the fixed-length variant left the hit rate unchanged, raised index tokens by 10.4%, and pushed the answer document's average rank back from 2.88 to 3.25.

State the precondition first: this is a result under pure keyword retrieval (BM25), and the vector side's conclusion is quite possibly the opposite. Why the regression here? Because a header dilutes keyword retrieval. Every chunk of a document gets the same heading prefixed, so that heading's terms go from a strong signal appearing in a few chunks to a weak signal appearing everywhere, lowering their own inverse document frequency. A header's real benefit is on the vector side, where it gives an isolated chunk a location in semantic space. So the technique's correct use is alongside hybrid search, and adding it to pure keyword retrieval alone is a negative return.

That fact is itself today's most memorable lesson: a technique with striking results in somebody else's blog post may be negative in your pipeline. The only way to tell is running the numbers.

Seven: overlap is not better for being larger

Finally, the parameter most often mindlessly enlarged.

Overlap does remedy something — it makes a split sentence complete in at least one place. And its three bills are all real:

The first is storage and tokens. Raising overlap from 0 to 80 (a fifth of a 400-character chunk) took total index tokens from 15,641 to 18,002, a 15% rise. That money is storage in a vector store, comparison volume on every retrieval, and possibly context cost at generation.

The second is redundancy in results. The larger the overlap, the more alike adjacent chunks are, and the likelier the top results are three versions of the same passage. You believe you gave the model three pieces of evidence and you gave it one, three times. That problem is essentially unsolvable before reranking.

The third is blurred citation location. The same sentence appears in two chunks, so which does the model cite? On day six, doing citation validation, that becomes an edge case to handle.

The rule-of-thumb range is 10% to 20% of the chunk length, but that range means start here rather than use this. The real practice is always: adjust one step, run an evaluation, read the three bills, and decide whether to keep it.

Source Reading

Hands-On Lab

🧪 D4 lab: five chunking strategies implemented side by side, with a hit rate comparison over one set of questions

Code location: labs/rag-14days/day-04-chunking-strategies

Acceptance criteria:

  1. MOCK=1 pnpm start prints three tables — chunk-level statistics, the hit rate comparison, and before and after adding headers — and writes chunking-report.json.
  2. In the first table, fixed length's mid-sentence rate is near half (48.3%) and recursive's is only 1.6% — with essentially the same chunk count, the whole difference is where the cut lands.
  3. In the second table, fixed and recursive score 6 of 8 while structure-based, parent-child, and semantic score 7 of 8; with the same 600-token budget, the first two fit only 1.2 chunks on average and the latter three fit 2.5 to 4.8.
  4. q06 (who approves a failover and what it is called) is missed by all five approaches — chunking does not solve multi-hop.
  5. In the third table, adding headers leaves the hit rate unchanged, raises index tokens by 10.4%, and pushes the answer document's average rank back from 2.88 to 3.25.

Before starting, confirm one thing: the ten questions in eval/questions-10.json are the course-wide golden set, and day eight expands them to 20 while reusing these ten verbatim. So do not change them now; run against them and get a firm grip on the same-questions ruler. When stuck, read the README's three evaluation-criteria lines, which answer eight out of ten questions.

  1. Complete the five chunkers' gaps (exercises 1 through 4) and run once for the first table: recursive's mid-sentence rate should fall from 48.3% to 1.6% and structure-based's chunk count should go from 30 to 153.
  2. Read the second table's average-chunks-fitted column and understand why the comparison ruler must be a token budget rather than taking the top few chunks.
  3. Complete exercise 5's contextual chunk header and compare the third table's two rows: how much the hit rate, the rank, and the tokens each changed.
  4. Change FIXED_OVERLAP from 80 to 0 and then to 160, running each, recording index tokens and hit rate, and verifying that overlap is not better for being larger.
  5. Write down your choice, and after it add one line stating the preconditions under which it holds: what kind of documents, what kind of questions, and how large a context budget.

Interview Questions

Today's 4 questions are in the question bank below, weighted toward the trade-off between chunk granularity and retrieval precision, the benefit of a parent-child structure, and evaluation-driven parameter choice. Expand each one and read the analysis before the key points — practicing the derivation beats memorizing bullets. The high-frequency labels for the domestic and overseas markets are there so you can pick by target market.

Checklist and Tomorrow

  • Explain the assumptions and failure scenarios of fixed-length, recursive, structure-based, parent-child, and semantic chunking
  • Explain what problems overlap and contextual chunk headers each remedy, and state the cost of each
  • Design a set of questions that quantifies retrieval hit rate across different chunking approaches, and decide with data instead of intuition
  • Say why the comparison must happen under one token budget rather than by taking the same top five chunks
  • All 5 acceptance criteria of the lab pass, with the numbers in chunking-report.json matching the walkthrough
  • Answer at least 3 of the 4 interview questions without looking at the key points

Tomorrow (D5) attention shifts from how it is cut to where it is stored and how it is queried quickly: the structure and tuning of hierarchical navigable small world graphs and inverted file indexes, cutting memory with quantization, and the most trip-prone filtered query. The order is deliberate — today you took chunk counts from 30 documents to hundreds or thousands of entries, so the index's scale problem genuinely begins, and only then does index selection have concrete numbers to discuss.

Interview questions

  • How do you decide on chunk size? Name two metrics you would look at, and one counterexample.你怎么决定切块大小？说出你会看的两个指标和一个反例。
    Common in ChinaCommon overseasIntermediate#chunking#evaluation

    How to reason about it · think before answering

    1. The question is about method, not about a number. Answering with a specific default (512 tokens, 1000 characters) already loses it — the interviewer wants to hear that you have a procedure.
    2. State the tension first: large chunks dilute the signal and cost context; small chunks lose the surrounding meaning so the model cannot use them. The two metrics you name should map onto those two failure modes.
    3. Metric one is retrieval-side hit rate: did a document that actually answers the question make it into the context. Metric two is generation-side usability, cheaply proxied by the fraction of chunks that end mid-sentence, and more seriously by faithfulness and whether citations resolve.
    4. Add the point that separates candidates: both metrics must be compared under the same token budget, never under a fixed top-k. With fixed k, bigger chunks simply buy more text and win for the wrong reason.
    5. Make the counterexample concrete: raising chunk size from 400 to 1200 characters can lift hit rate purely because whole short documents now fit in one chunk, which means retrieval stopped doing anything and you are back to stuffing full documents. The metric improved while the system got worse.
    6. Expect the follow-up: where do you start on day one. Pick the strategy from the document type first (structural splitting whenever headings exist), start around 300 to 500 characters with 10 to 20 percent overlap, then build a golden set immediately and iterate. A starting point is not a conclusion.

    分析过程 · 先想清楚再作答

    1. 这题的题眼是「怎么决定」，不是「多大合适」。答一个具体数字（512 token、1000 字符）就已经输了——面试官想看的是你有没有一套定法，而不是你记得住哪个默认值。
    2. 先把矛盾摆出来：块大则信噪比低、上下文贵，块小则单块缺语境、模型答不出所以然。切块大小就是在这两头之间找位置，所以两个指标必须分别对应这两头。
    3. 第一个指标是检索侧的命中率——答案文档有没有进上下文。第二个是生成侧的可用性，最省事的代理指标是切碎率，也就是有多少块结尾停在半句话上；再往前一步就是忠实度和引用是否可定位。
    4. 关键补一句：两个指标必须在**同一个 token 预算**下比，不能按「取前 k 块」比。k 固定时块越大塞进去的字越多，大块切法会赢在买得多而不是切得准上。这一句往往是这道题的区分点。
    5. 反例要具体。最好用的一个是：把块从 400 字调到 1200 字，命中率不降反升——但那是因为一整篇短文档被当成一块塞了进去，检索其实什么都没做，等于退化成了全文投喂。指标涨了，系统更差了。
    6. 可预期的追问是「那你第一次上手时从哪个数字起步」。答：先按文档类型选切法（有标题层级就按结构切），块长从 300 到 500 字起步、重叠取一到两成，然后立刻建一组标准问题跑评估，用两三轮迭代把它调到位。起步值是起步值，不是结论。

    Key points

    • Choose the strategy from the document type first, then tune length: split on headings whenever the structure survives parsing.
    • Watch two metrics: retrieval hit rate on one side, mid-sentence break rate (then faithfulness and citation resolvability) on the other.
    • Compare under an equal token budget, never a fixed top-k, or larger chunks win by buying more text.
    • Counterexample: hit rate rises after enlarging chunks because whole documents now fit in one chunk and retrieval has effectively stopped working.
    • Start near 300 to 500 characters with 10 to 20 percent overlap, then iterate against a fixed question set instead of guessing.

    答题要点

    • 先按文档类型选切法，再调长度：有标题层级就按结构切，没有结构才谈固定长度或语义。
    • 看两个指标：检索侧的命中率，生成侧的切碎率（进一步是忠实度与引用可定位性）。
    • 两个指标必须在同一个 token 预算下比，不能按「取前 k 块」比，否则大块只是买得更多。
    • 反例：块调大后命中率上升，但那是因为整篇被当成一块，检索退化成全文投喂。
    • 起步值 300 到 500 字、重叠一到两成，然后靠一组固定问题迭代，不靠直觉定稿。
  • What does parent-child chunking buy you, and when does it slow the system down instead?父子切块的收益是什么？它在什么情况下反而会拖慢系统？
    Common in ChinaCommon overseasIntermediate#chunking#parent-child

    How to reason about it · think before answering

    1. This question checks whether you know that the retrieval unit and the context unit can be two different things. Without that sentence, everything else is recitation.
    2. State the benefit compactly: small chunks go into the index so they are easy to match, and once a child is hit you follow the parent pointer and hand the model the whole section. You stop trading precision against completeness.
    3. Derive the slowdown from the costs. First, the context budget: every new child may drag in an entire parent, so an equal budget holds fewer distinct pieces and result diversity drops.
    4. Second, the write path: two levels to maintain, both recomputed on every document update, and chunk ids become harder to keep stable, which makes incremental sync noticeably more complex.
    5. Third, the condition under which the benefit disappears: when sections are already short, the parent and the child are nearly the same text, so you paid for two indexes and bought nothing. Parent-child suits long sections and deep hierarchies, not already fine-grained knowledge bases.
    6. Expect the follow-up: how is this different from simply using bigger chunks. Bigger chunks put the noise into the index; parent-child puts the noise only into the context. What gets matched stays short and clean.

    分析过程 · 先想清楚再作答

    1. 这题考的是你有没有意识到「检索单位」和「上下文单位」可以是两个东西。答不出这句话，后面说什么都是复述。
    2. 收益一句话说清：小块进索引，信噪比高、容易被找到；命中之后顺着父指针把整节回填给模型，语境完整。精度和完整度这次不用二选一。
    3. 拖慢的场景要从代价一条条推。第一条是上下文预算：每命中一个新子块可能拖进来一整个父节，同样的 token 预算装不下几条，检索结果的多样性反而变差。
    4. 第二条是写入侧：父子两套都要维护，文档更新时两边都要重算，块 id 的稳定性也更难保证，增量同步的复杂度明显上升。
    5. 第三条是收益消失的条件：当文档本身的小节就不长时，父块和子块差不多大，你付了两套索引的钱，什么也没多买到。所以父子切块适合长节、深层级的文档，不适合结构本来就细碎的知识库。
    6. 可预期的追问是「那和直接把块切大有什么区别」。答：切大是把噪声一起放进索引，父子是只把噪声放进上下文、不放进索引——被检索的那一段始终是干净的短文本，这是本质区别。

    Key points

    • The core idea is decoupling the retrieval unit from the context unit: small chunks get found, large chunks get understood.
    • The payoff is precision and completeness at the same time instead of trading one for the other.
    • Cost one: a single hit can drag in a whole parent, so an equal context budget holds fewer distinct results and diversity suffers.
    • Cost two: two index levels to maintain and recompute, which makes incremental sync on document updates considerably harder.
    • It stops paying off when sections are already short, because parent and child are nearly identical and you bought nothing for the extra cost.

    答题要点

    • 核心是把检索单位和上下文单位拆开：小块负责被找到，大块负责被读懂。
    • 收益是精度与完整度同时拿到，不用在信噪比和语境之间二选一。
    • 代价一：一次命中可能拖进整个父节，同样的上下文预算装得下的条数变少，结果多样性下降。
    • 代价二：父子两套索引都要维护与重算，文档更新时增量同步的复杂度明显上升。
    • 失效场景：文档小节本来就短时父子块差不多大，多付一套成本却没多买到东西。
  • What overlap ratio would you use, and what concretely goes wrong when the overlap is too large?重叠区设成块长的百分之多少合适？重叠过大会带来什么具体问题？
    Common in ChinaCommon overseasBasic#chunking#overlap

    How to reason about it · think before answering

    1. This is a giveaway question, but the marks are in the second half, not the percentage. Stopping at 'usually ten to twenty percent' reads like someone who has never run it.
    2. Say what overlap is patching: fixed-length splitting cuts sentences in half, and overlap guarantees the broken sentence survives intact in at least one of the two neighbors. It is a patch for careless splitting, not an optimization of its own.
    3. That yields the first conclusion: with structural or recursive splitting the boundaries already land on semantic positions, so the need for overlap drops sharply and can legitimately be zero. The ratio question is meaningless without naming the strategy.
    4. Give three concrete costs. Storage and tokens: at 400-character chunks, moving overlap from 0 to 80 grows total index tokens by roughly fifteen percent, which is storage cost in the vector store and comparison work at query time.
    5. Retrieval redundancy: the more neighbors overlap, the more likely the top results are three versions of the same passage. You think you handed the model three pieces of evidence; you handed it one, three times. Nothing fixes this before reranking.
    6. Citation resolution: when a sentence lives in two chunks, which one does the model cite. Expect the follow-up on deduplication: merge at the result layer using a content fingerprint or longest common substring, not by tweaking the chunker.

    分析过程 · 先想清楚再作答

    1. 这是一道送分题，但送分点不在那个百分比上，而在后半句。只答「一般一到两成」就停住的人，面试官会认为他没跑过。
    2. 先说清重叠在补救什么：固定长度切法会把句子从中间切开，重叠让被切开的那句话至少在相邻两块之一里是完整的。它是给「乱切」打的补丁，不是一个独立的优化。
    3. 由此推出第一个结论：如果你用的是按结构切或递归切，边界本来就落在语义位置上，重叠的必要性会大幅下降，甚至可以是零。**重叠比例这个问题的前提是切法**，脱开切法谈比例就是背数字。
    4. 过大的代价要说三笔，越具体越好。存储与 token：块长 400、重叠从 0 加到 80，索引 token 会涨一成半左右，这笔钱在向量库是存储费、在检索时是比对量。
    5. 检索冗余：相邻块越像，前几名越可能是同一段话的三个版本，你以为给了模型三条证据，其实是一条说了三遍。这一条在重排之前基本无解。
    6. 引用定位：同一句话出现在两个块里，模型标出处该标哪一个，这会直接变成引用校验环节要处理的边界情况。可预期的追问就是「那你怎么去重」，答按内容指纹或最长公共子串在结果层合并，而不是在切块层想办法。

    Key points

    • Ten to twenty percent of chunk length is the working range, but that number assumes fixed-length splitting.
    • With structural or recursive splitting the boundaries are already semantic, so overlap can be small or zero.
    • Cost one: index tokens and storage grow noticeably; at 400-character chunks, an 80-character overlap adds roughly fifteen percent.
    • Cost two: neighboring chunks become near-duplicates, so the top results are several versions of one passage and the evidence diversity is illusory.
    • Cost three: a sentence spanning two chunks complicates citation attribution and forces result-level deduplication.

    答题要点

    • 经验区间是块长的一到两成，但这个数字的前提是你用的是固定长度切法。
    • 按结构或递归切时边界本来就在语义位置上，重叠可以很小甚至为零。
    • 过大代价一：索引 token 与存储明显上涨，块长 400 时重叠加到 80 大约涨一成半。
    • 过大代价二：相邻块高度相似，检索前几名变成同一段话的多个版本，证据多样性是假的。
    • 过大代价三：同一句话跨块出现，引用标注和去重都要额外处理。
  • Semantic chunking costs considerably more than recursive splitting. How would you prove to your team that the money is well spent?语义切分比递归切分贵不少，你怎么向团队证明这笔钱值得花？
    Common in ChinaCommon overseasDeep dive#chunking#evaluation#cost

    How to reason about it · think before answering

    1. This looks like a technical question but it tests whether you can run a controlled technical argument. Launching into how semantic chunking works answers a different question.
    2. Step one is to concede that it may well not be worth it. The gain comes from documents that have no usable structure; if your knowledge base is well-formed documents, the authors' heading hierarchy already did the semantic split for free and the money is likely wasted.
    3. Step two is translating 'worth it' into three measurable numbers: how much the metric moved (hit rate on the same golden set under the same token budget), how much latency moved (chunking is offline, but the end-to-end update path changes), and how much it costs (the initial full embedding pass plus recomputation amortised over update frequency).
    4. Step three is the control. Recursive splitting is the baseline, semantic chunking the treatment, and they must share the corpus, the questions, the context budget and the retriever. Change one variable only; a two-variable experiment proves nothing.
    5. Step four is a decision threshold rather than an impression. For example: below three points of hit-rate gain, no; above five points with recomputation inside the monthly budget, yes; in between, roll it out on one document class first. Fix the threshold before you run the numbers, or you will quietly bend it to fit them.
    6. Expect the follow-up: is there a cheaper way to the same gain. Yes — try structural splitting first, since it is free and often nearly as good, and if the structure really is unusable, apply semantic chunking only to the high-value subset rather than the whole corpus.

    分析过程 · 先想清楚再作答

    1. 这题表面问技术，实际考的是你会不会做一次带对照组的技术论证。上来就讲语义切分原理的人，答的是另一道题。
    2. 第一步是先承认它可能不值。语义切分的收益来自「文档没有可用的结构」；如果知识库是结构良好的文档，作者的标题层级已经免费替你做完了语义切分，这时候花的钱大概率打水漂。**先说清适用前提，再谈证明，这一步就把大多数候选人区分开了。**
    3. 第二步是把「值不值」翻译成可测的三笔账：指标涨了多少（同一批标准问题、同一个 token 预算下的命中率）、延迟涨了多少（切块是离线的，但更新链路的端到端时间会变）、钱涨了多少（首次全量 embedding 的费用，加上按更新频率折算的重算费用）。只报第一笔的论证不成立。
    4. 第三步是设计对照。递归切分是基线，语义切分是实验组，两组必须用同一份语料、同一批问题、同一个上下文预算、同一个检索器，只改切法这一个变量。改两个变量的实验，结论一文不值。
    5. 第四步是给决策一个门槛，而不是给一个感想。比如：命中率相对基线提升低于三个百分点就不上；提升超过五个百分点且重算成本在月度预算内就上；中间地带先在一类文档上灰度。**门槛要在跑数字之前定好**，否则你会不自觉地去迁就已经跑出来的结果。
    6. 可预期的追问是「有没有更便宜的办法拿到同样的收益」。答有：先试按结构切，它零成本且效果常常接近；结构确实不可用时，再考虑只对高价值的那一部分文档做语义切分，而不是全量上。

    Key points

    • Start with the precondition: the gain comes from documents without usable structure, so on well-formed documents it usually is not worth it.
    • Translate 'worth it' into three numbers — hit rate, latency, and cost. Reporting only the first is not an argument.
    • Run a controlled comparison: same corpus, same golden set, same context budget, same retriever, with the splitting strategy as the only variable.
    • Fix the decision threshold before running the numbers so you cannot bend it to fit the result afterwards.
    • Try free structural splitting first, and if semantic chunking is genuinely needed, apply it to the high-value subset rather than the entire corpus.

    答题要点

    • 先讲适用前提：语义切分的收益来自文档没有可用结构，结构良好的文档上它大概率不值。
    • 把「值不值」翻译成三笔账：命中率涨多少、延迟涨多少、钱涨多少，只报第一笔不算论证。
    • 做对照实验：同语料、同问题集、同上下文预算、同检索器，只改切法一个变量。
    • 决策门槛必须在跑数字之前定好，避免事后迁就结果。
    • 先试零成本的按结构切；确需语义切分时也优先只覆盖高价值文档，而不是全量上。

Comments