Streaming Output and Terminal Rendering: Hand-Writing an SSE Parser, Incremental Markdown, and an Interruptible Typewriter

Today's Goals

1. Hand-write an SSE parser with no library dependency, and explain why it must split on blank lines rather than on lines
2. Distinguish the gateway layer's streaming chunks from the loop layer's semantic events, and explain what trouble this boundary saves
3. Implement incremental Markdown rendering and interruptible output in the terminal, handling a stream that disconnects mid-way

Yesterday's buffer-then-print black screen comes apart today. When you finish, scroll back up and tick the three goals off.

Plain-Language Walkthrough

Let the new hire think out loud instead of vanishing for an hour

Back to that new hire. You give them a question; they sit silently at their desk and hand you a well-written report an hour later. Are you satisfied? Probably not. You would rather they thought out loud: "let me look at this module... hmm, the logic here seems wrong... let me check the tests." You know which step they are on at all times and can stop them the moment they drift.

Models are the same, and their silence lasts longer than you expect. The numbers below were measured on 7 September 2026 against an OpenAI-compatible gateway (https://api.n1n.ai/v1), on the same roughly two-hundred-character answer, three runs per model, reported as ranges:

Read the table correctly, not as a leaderboard: the delta count shows only magnitude and the first-token latency only a relative relationship; neither column is reproducible — they depend on the gateway's implementation, the load at that moment and your network, and another run gives different numbers. The table says one thing only: in a buffer-then-print implementation, that last row's dozen-odd seconds is completely silent black screen, and the only judgment available to the user is "has this thing died?" Its total time is not ten times the first row's; it simply thought for a long while before speaking.

So: streaming is not a nicety, it is the only remedy for time to first token. The total time does not change at all, and the user sees characters moving in the second second. That is also why yesterday's gateway interface kept only a streaming method — nothing in a Coding Agent has any use for the non-streaming one.

A counterintuitive bonus, used in the second half of today: streaming makes stopping midway possible. Dozens of deltas per turn are dozens of places to cut in; the buffer-then-print implementation offers only "wait for it to finish."

What SSE looks like: an event's boundary is a blank line, not a newline

Server-Sent Events is a text protocol plain enough to be endearing. The raw bytes the gateway emits, decoded, look roughly like this:

data: {"choices":[{"delta":{"content":"Streaming"}}]}
 
data: {"choices":[{"delta":{"content":" is not"}}]}
 
: this is a comment line, usually a heartbeat keep-alive
 
data: {"choices":[],"usage":{"prompt_tokens":24,"completion_tokens":88}}
 
data: [DONE]

Three rules, each corresponding to a real incident:

1. Events are separated by blank lines, and one event may have several data: lines. Per the specification, several data lines in one event are joined with newlines into a single payload before parsing. So the parser frames on blank lines (that is, two consecutive newlines), not on single newlines. Reading line by line "happens to work" on most gateways because they send one data line per event — until one day one does not.
2. A line starting with a colon is a comment; drop it. It is usually a heartbeat, preventing an intermediate proxy from killing an idle connection. Feeding it to a JSON parser blows up.
3. [DONE] is not JSON. Pass it into the parse function and you get a syntax error.

And one easier to step on than all three: network chunking has nothing to do with event boundaries. What you get from the response body is byte chunks; one event may span two of them, one chunk may hold five events, and even a multi-byte character's bytes can be split across two chunks. So you must keep a buffer and carry an incomplete tail into the next chunk.

data: {"choices":[{"delta":{"content":"Hi"}}]}
data: {"choices":[{"delta":{"content":" there"}}]}
data: [DONE]

(empty)

const decoder = new TextDecoder()
let buffer = ''
let closed = false
 
for await (const chunk of res.body as unknown as AsyncIterable<Uint8Array>) {
  // stream: true lets the decoder handle multi-byte characters split across chunks
  buffer += decoder.decode(chunk, { stream: true })
  buffer = buffer.replace(/\r\n/g, '\n') // normalize CRLF, then look only for blank lines
  let boundary = buffer.indexOf('\n\n')
  while (boundary !== -1) {
    const rawEvent = buffer.slice(0, boundary)
    buffer = buffer.slice(boundary + 2)
    for (const payload of dataLines(rawEvent)) {
      if (payload === '[DONE]') return
      for (const delta of parseChunk(payload)) {
        if (delta.type === 'finish') closed = true
        yield delta
      }
    }
    boundary = buffer.indexOf('\n\n')
  }
}
// The byte stream ended without a finish and without [DONE]: that is a truncated stream
if (!closed) throw new StreamTruncatedError()
 
/** Several data lines in one event join with newlines; comment lines are dropped */
function dataLines(rawEvent: string): string[] {
  const lines = rawEvent
    .split('\n')
    .filter((line) => line.startsWith('data:'))
    .map((line) => line.slice(5).trimStart())
  return lines.length ? [lines.join('\n')] : []
}

# The Python version frames at the byte level: httpx has aiter_lines, but it splits on
# lines and cannot give you the blank-line boundary, so we keep our own buffer here.
buffer = b""
closed = False
 
async for chunk in res.aiter_bytes():
    buffer += chunk.replace(b"\r\n", b"\n")
    while b"\n\n" in buffer:
        raw_event, buffer = buffer.split(b"\n\n", 1)
        for payload in data_lines(raw_event.decode("utf-8", "ignore")):
            if payload == "[DONE]":
                return
            for delta in parse_chunk(payload):
                if isinstance(delta, FinishDelta):
                    closed = True
                yield delta
 
if not closed:
    raise StreamTruncatedError()
 
 
def data_lines(raw_event: str) -> list[str]:
    """Several data lines in one event join into one payload; comment lines are dropped"""
    lines = [
        line.removeprefix("data:").lstrip()
        for line in raw_event.split("\n")
        if line.startswith("data:")
    ]
    return ["\n".join(lines)] if lines else []

Both versions check one thing outside the loop: did we see a terminator? That is truncation detection, and it is the step most home-made clients miss — relying on "the iteration ended naturally" means that when a proxy cuts the stream, the user sees half a sentence and a normal prompt and assumes that is what the model said.

Delta granularity is an implementation detail; never treat it as a contract

That table's delta ranges hide another trap. Three runs of the same model give three different counts, and different models differ by more than fivefold. Delta granularity is an implementation detail of the gateway and the model: the magnitude is stable, the exact number is not.

Hence a rule: the render layer must not depend on the shape of a delta. Concretely, all of the following are wrong:

• assuming one delta is one word, or one complete token
• assuming a delta never crosses a newline, and treating deltas as lines
• assuming Markdown markers (backticks, asterisks) are never split
• using delta counts as a progress bar

The right approach treats a delta as "some more characters arrived," and nothing else. This course's offline script uses a fixed four characters per delta precisely to force this into the open: offline, asking "tell me about streaming" splits a 133-character answer into 34 deltas — a reproducible number, because we did the splitting. A real gateway gives you whatever it gives you, and you have to catch it.

The two-layer event model: why gateway deltas and loop events must stay separate

Yesterday defined two type families: the deltas the gateway emits and the semantic events the loop emits. Today they pay interest for the first time, so the boundary deserves spelling out.

flowchart LR
  A[SSE byte chunks] -->|framing| B[gateway: one event at a time]
  B -->|translate the wire format| C[deltas: text, tool, usage, finish]
  C -->|add semantics| D[loop: semantic events]
  D -->|subscribe| E[render: characters on screen]
  D -->|subscribe| F[session: the persisted log]

The same thing is described completely differently at each layer. The gateway layer says "the payload has a content field whose value is these two words"; the loop layer says "the model is producing text." That looks like rewording, and the difference is which one changes: wire shapes change with the gateway, semantics do not.

For today's two ways of ending, the boundary saves obvious trouble. A cut stream is a gateway-layer matter (the byte stream ended without a terminator); a user pressing cancel is a keyboard matter, and their origins are entirely different. At the semantic layer they unify into two events — an error carrying a "can this be retried" judgment, and a done carrying a finish reason. The render layer knows only those two and never needs to know where they came from.

for await (const delta of provider.stream({ messages, tools, signal })) {
  // The interrupt check sits between deltas: dozens of chances to stop per turn
  if (signal?.aborted) {
    if (assistantText) messages.push({ role: 'assistant', content: assistantText })
    yield { type: 'done', reason: 'aborted' }
    return
  }
  if (delta.type === 'text') {
    assistantText += delta.text
    yield { type: 'text', delta: delta.text } // wire shape stops here; outside sees semantics
  }
}

async for delta in provider.stream(ChatRequest(messages, tools, signal)):
    # The interrupt check sits between deltas: dozens of chances to stop per turn
    if signal.aborted:
        if assistant_text:
            messages.append(Message(role="assistant", content=assistant_text))
        yield DoneEvent(reason="aborted")
        return
    match delta:
        case TextDelta(text=text):
            assistant_text += text
            yield TextEvent(delta=text)  # wire shape stops here; outside sees semantics

Note that messages.push: an interrupt must still write the received text back into the message array. Content the user has seen must not vanish — they would think the program ate it, and when they ask about "that approach you just mentioned" the model has no idea it said anything. Truncation is the same: it is not "this turn did not happen," it is "this turn did not finish."

The hard part of incremental Markdown: code fences and inline markers get split

Now the render layer. Printing character by character is easy; the difficulty is that the model outputs Markdown and Markdown markers get split across deltas.

The most annoying example: the model outputs a fenced code block. Three backticks get split into at least two deltas, so for one frame the screen holds two backticks. If your renderer colors that as inline code, the user sees a flash of color that reverts. A dozen such flickers in one answer look like a broken program.

The fix has two plain steps.

One: treat the line as the smallest settled unit. Until the newline arrives, the line may still change, so display it verbatim; once the newline arrives, the line is fixed and only then do you apply styling (bold headings, color inline code, toggle code mode on a fence line). On a real terminal, implement it as "return to the line start, clear the line, redraw," and the user sees one line growing.

Two: give unsettled lines a criterion. Not every partial line is dangerous; only three cases must be shown verbatim: a line holding just one or two backticks (it may be growing into a fence), an odd total of backticks (unclosed inline code), and a line ending on an asterisk (unclosed emphasis).

/** Could later characters still change this line's meaning? If so, show it verbatim */
export function looksUnsettled(line: string): boolean {
  const trimmed = line.trimStart()
  if (/^`{1,2}$/.test(trimmed)) return true // may be growing into a fence
  if (((line.match(/`/g) ?? []).length) % 2 === 1) return true // unclosed inline code
  if (/\*{1,2}$/.test(line)) return true // unclosed emphasis
  return false
}
 
push(delta: string): void {
  for (const ch of delta) {
    if (ch === '\n') this.commitLine() // newline arrived, this line is settled
    else this.line += ch
  }
  if (this.tty) this.redraw() // no cursor outside a TTY: degrade to plain appending
}

import re
 
UNCLOSED_EMPHASIS = re.compile(r"\*{1,2}$")
MAYBE_FENCE = re.compile(r"^`{1,2}$")
 
 
def looks_unsettled(line: str) -> bool:
    """Could later characters still change this line's meaning? If so, show it verbatim"""
    return bool(
        MAYBE_FENCE.match(line.lstrip())      # may be growing into a fence
        or line.count("`") % 2               # unclosed inline code
        or UNCLOSED_EMPHASIS.search(line)    # unclosed emphasis
    )
 
 
def push(self, delta: str) -> None:
    for ch in delta:
        if ch == "\n":
            self.commit_line()               # newline arrived, this line is settled
        else:
            self.line += ch
    if self.tty:
        self.redraw()                        # no cursor outside a TTY: plain appending

And a problem only writing it reveals: there is no cursor in a pipe. The escape sequences for returning to the line start and clearing the line are pure noise in a non-interactive terminal (CI, a pipe, another Agent's shell) and make logs unreadable. So the renderer branches on whether standard output is a terminal: a real terminal redraws the current line, a pipe appends plainly with no styling. The cost is that the typewriter effect is invisible in a pipe — which is why this lab's acceptance rests on the self-test's assertions (delta count, verbatim equality with the concatenated text after stripping escapes, zero mis-styled frames), not on how pretty the colors are.

Offline in this lab, asking "what does the parser look like" splits the fenced answer into 44 deltas, triggers 51 redraw frames, zero mis-styled frames and one recognized code block. All four numbers are reproducible, because we wrote both the script and the splitting rule.

Interrupting and finishing: what a cut stream and a cancel each must leave behind

Finally, finishing. The two abnormal endings are handled differently, with one shared floor: not one character already shown to the user may be lost.

The stream is cut mid-way. The criterion was given above: the byte stream ended without a finish event. Three things follow — settle the received text (do not leave half a line on screen), add a plain-language sentence ("the stream was cut mid-way; the part received above has been kept"), and write that segment into the message array. Note that the error object carries a "can this be retried" flag: truncation and rate limiting are retryable; a malformed parameter gives the same result on the hundredth retry. Today only classifies; how to back off and how many times is day six.

The user presses cancel. That is not an exception but a normal early ending. It is implemented with a cancellation controller: create one when a turn begins, pass its signal down to the gateway (which aborts the request itself against a real gateway), and have the loop check it between deltas. Today the chain reaches only the gateway; once day four adds child processes, day six shows you the full cancellation chain — key, signal, request, child process — where one broken link means a hung command that cannot be killed.

Source Reading

Hands-On Lab

Today's lab finalizes the gateway layer and the offline script engine: those two files freeze today and are copied verbatim for the next nineteen days. The starter leaves four exercises; unmodified it passes three checks and fails three, and completing the exercises should turn everything green.

1. Replace the gateway parser with the blank-line-framing version: handle multi-line data, drop comment lines, stop at the terminator, and check on stream end whether a finish event was ever seen.
2. Translate deltas into semantic events so the render layer can see no wire field at all. Afterwards, searching the render layer's code for gateway field names like content or delta should find nothing.
3. Implement incremental rendering: settle lines, show unsettled lines verbatim, toggle code mode on fences, and branch on whether standard output is a terminal into redraw or plain appending.
4. Run INJECT=truncated and confirm the half sentence stays on screen with a plain sentence beneath it and no stack trace.
5. Run the self-test: MOCK=1 SELFTEST=1 pnpm start should print 6/6 passed, with the delta count, redraw frames and mis-styled frames all reproducible numbers.

Acceptance is five ticks: the self-test prints 6/6 passed; the long answer splits into 34 deltas and matches the concatenated text verbatim after stripping escapes; the fenced answer produces zero mis-styled frames; an interrupt yields done with aborted and keeps the received characters in the message array; and INJECT=truncated produces a plain-language notice.

Interview Questions

Today's three questions test whether you have written a parser and a renderer yourself, not whether you know what SSE is:

1. What edge cases trip up a hand-written SSE parser? Why can it not read line by line?
2. When a streaming response is cut mid-way, how should the client handle it? What does a retry cost?
3. What is hard about incremental Markdown rendering in a terminal? How would you weigh correctness against immediacy?

Full bilingual prompts, analyses and key points are in this course's day-two question bank. Question two is the one most often answered incompletely — most people say "retry" and cannot name the two real costs: how many tokens a retry spends, and what happens to the half segment already printed.

Checklist and Tomorrow

• I can hand-write a blank-line-framing parser and say how multi-line data, comment lines and the terminator are each handled
• I can explain why network chunking is unrelated to event boundaries, and what the buffer solves
• I know delta granularity is an implementation detail, and can name three wrong ways to depend on delta shape
• I can explain what the gateway-delta versus semantic-event boundary saves in each of today's two endings
• I can state the three criteria for an unsettled line, and why a pipe degrades to plain appending
• The self-test prints 6/6 passed, and both truncation and interruption preserve what was already received

Tomorrow is D3, "The Tool Protocol and the Read-Only Trio: Schema Design, Fragment Merging, and Result Truncation." Today handled only text deltas, but the same stream carries another kind — tool calls. Merging those is far harder: the arguments are a chopped-up JSON string that is only valid once joined, and the index's starting point cannot be assumed (we measured two different starting points from the same model across two runs). Master streaming before tools, in that order: a fragment-merging bug is invisible on a text stream.