boxmaker/docs/plans/M2a/07-llama-assemble.md

M2a task 07: assembling a streamed completion

Branch: m2a (run git switch m2a; git status --short must be empty, otherwise stop) Commit subject: Add the assembler that folds streamed chunks into a completion

Goal

The server streams a completion as many small JSON chunks. Fold them into one Completion, and turn each chunk into the ChatEvents it stands for. No I/O in this task: the input is the text of one data: payload at a time.

Context

Look at a recording first: grep -m3 '^data' crates/loopd/tests/fixtures/http/tool_call.http | cut -c1-400

One chunk (shortened). Fields not shown here also arrive, and newer servers add more:

{"id":"chatcmpl-abc","choices":[{"index":0,"finish_reason":null,
  "delta":{"reasoning_content":"I should"}}],
 "timings":{"cache_n":0,"prompt_n":41,"predicted_n":39,"prompt_ms":246.4},
 "prompt_progress":{"total":312,"cache":0,"processed":278,"time_ms":370}}

The kinds of delta, one example each:

Delta	Means
{"role":"assistant","content":null}	nothing; the first chunks look like this
{"reasoning_content":"I should"}	thinking text
{"content":"It says"}	answer text
{"tool_calls":[{"index":0,"id":"x","type":"function","function":{"name":"read_file","arguments":"{"}}]}	the first piece of tool call 0: id, name, and the start of its arguments
{"tool_calls":[{"index":0,"function":{"arguments":"\"path\""}}]}	a later piece: more argument text only
{} with "finish_reason":"stop"	the end; also "tool_calls" and "length"

This is the server's format, not ours: unknown fields must be ignored. Do not put deny_unknown_fields on the structs that parse chunks. One test sends extra fields at every level.

Files

• Copy: crates/loopd/tests/assemble.rs, crates/loopd/tests/fixtures/expected/ (7 files)
• Create: crates/loopd/src/llama/assemble.rs
• Modify: crates/loopd/src/llama/mod.rs (add pub mod assemble;), docs/implementer-log.md

Interfaces

Consumes from task 06: ChatEvent, Completion, FinishReason, InferError, Timings.

Produces, in crates/loopd/src/llama/assemble.rs:

#[derive(Default)]
pub struct Assembler { /* private */ }
impl Assembler {
    pub fn new() -> Self;
    /// Takes one `data:` payload (JSON text) and returns the events it stands for, in order.
    pub fn push(&mut self, data: &str) -> Result<Vec<ChatEvent>, InferError>;
    pub fn id(&self) -> Option<&str>;          // the completion id, once a chunk has carried one
    pub fn reasoning_tokens(&self) -> u64;     // tokens generated while the model was reasoning
    pub fn in_reasoning(&self) -> bool;        // true while the latest text was reasoning
    pub fn is_finished(&self) -> bool;         // a finish_reason has arrived
    pub fn finish(self, thinking_capped: bool) -> Result<Completion, InferError>;
}

Rules the tests check:

1. For one chunk, events come in this order: Progress (if prompt_progress is present), then for each choice: Reasoning, Content, then one ToolCallDelta per entry in tool_calls. Empty or null text produces no event. A ToolCallDelta carries id and name as they were in that piece (None in later pieces) and that piece's arguments text ("" if absent).
2. content and reasoning_content of the completion are the concatenation of the event texts, or None if there never was any. A tool call's arguments is the concatenation of its pieces; its id and name come from the first piece that has them.
3. timings is the latest timings object seen.
4. reasoning_tokens: whenever a chunk carries reasoning text, set it to the predicted_n of the latest timings (update timings first). Everything generated up to then was reasoning. It stops changing when the answer starts. in_reasoning is true after reasoning text, and false after answer text or a tool-call piece.
5. finish fails with StreamClosedEarly if no finish_reason arrived, and with Protocol if no chunk carried an id, or a tool call never got an id or a name.
6. push fails with Protocol for: text that is not a JSON object of this shape, a finish_reason other than the three above, and a tool call index that skips ahead (index 5 when only calls 0 and 1 exist). The index comes from the server: convert it with try_from and use get_mut, never [].

The expected results in tests/fixtures/expected/*.json were produced from the same recordings by a separate script, so the test checks your code against an independent reading of the same bytes.

Steps

• 1. Copy.

git switch m2a
cp docs/plans/M2a/files/crates/loopd/tests/assemble.rs crates/loopd/tests/
cp -r docs/plans/M2a/files/crates/loopd/tests/fixtures/expected crates/loopd/tests/fixtures/

• 2. See the test fail. cargo test -p loopd --test assemble. Expected: it does not compile.
• 3. Write assemble.rs and register the module. Run cargo fmt --all.
• 4. See the test pass. cargo test -p loopd --test assemble. Expected: 7 passed.
• 5. Run the gate. make gate. Expected last line: gate: ok.
• 6. Log and commit. git add crates/loopd docs/implementer-log.md && git commit

Done when

• cargo test -p loopd --test assemble reports 7 passed; make gate prints gate: ok.
• grep -n deny_unknown_fields crates/loopd/src/llama/assemble.rs prints nothing.

Stop and report if

• Your result differs from a file in fixtures/expected/ and you believe the file is wrong.