Add M2a plan: thirteen tasks, tests, fake server and recordings
The tasks build the inference path: emsha-backed SHA-256, inferproxy, config, a hand-written HTTP and SSE client, request building, delta assembly, the chat state machine, the thinking cap, the slot gate with retry, the startup self-test and on-device verification. Everything the tasks copy in was checked against a private reference implementation: the gate passes after each task in order, the timing tests pass repeatedly under CPU load, and the reference passes the self-test and all four device checks on straylight. Expected results for the recorded streams were derived by a separate script. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
@@ -0,0 +1,114 @@
|
||||
# M2a task 07: assembling a streamed completion
|
||||
|
||||
**Branch:** `m2a` (run `git switch m2a`; `git status --short` must be empty, otherwise stop)
|
||||
**Commit subject:** `Add the assembler that folds streamed chunks into a completion`
|
||||
|
||||
## Goal
|
||||
|
||||
The server streams a completion as many small JSON chunks. Fold them into one `Completion`, and
|
||||
turn each chunk into the `ChatEvent`s it stands for. No I/O in this task: the input is the text of
|
||||
one `data:` payload at a time.
|
||||
|
||||
## Context
|
||||
|
||||
Look at a recording first:
|
||||
`grep -m3 '^data' crates/loopd/tests/fixtures/http/tool_call.http | cut -c1-400`
|
||||
|
||||
One chunk (shortened). Fields not shown here also arrive, and newer servers add more:
|
||||
|
||||
```json
|
||||
{"id":"chatcmpl-abc","choices":[{"index":0,"finish_reason":null,
|
||||
"delta":{"reasoning_content":"I should"}}],
|
||||
"timings":{"cache_n":0,"prompt_n":41,"predicted_n":39,"prompt_ms":246.4},
|
||||
"prompt_progress":{"total":312,"cache":0,"processed":278,"time_ms":370}}
|
||||
```
|
||||
|
||||
The kinds of delta, one example each:
|
||||
|
||||
| Delta | Means |
|
||||
|---|---|
|
||||
| `{"role":"assistant","content":null}` | nothing; the first chunks look like this |
|
||||
| `{"reasoning_content":"I should"}` | thinking text |
|
||||
| `{"content":"It says"}` | answer text |
|
||||
| `{"tool_calls":[{"index":0,"id":"x","type":"function","function":{"name":"read_file","arguments":"{"}}]}` | the first piece of tool call 0: id, name, and the start of its arguments |
|
||||
| `{"tool_calls":[{"index":0,"function":{"arguments":"\"path\""}}]}` | a later piece: more argument text only |
|
||||
| `{}` with `"finish_reason":"stop"` | the end; also `"tool_calls"` and `"length"` |
|
||||
|
||||
**This is the server's format, not ours: unknown fields must be ignored.** Do not put
|
||||
`deny_unknown_fields` on the structs that parse chunks. One test sends extra fields at every level.
|
||||
|
||||
## Files
|
||||
|
||||
- Copy: `crates/loopd/tests/assemble.rs`, `crates/loopd/tests/fixtures/expected/` (7 files)
|
||||
- Create: `crates/loopd/src/llama/assemble.rs`
|
||||
- Modify: `crates/loopd/src/llama/mod.rs` (add `pub mod assemble;`), `docs/implementer-log.md`
|
||||
|
||||
## Interfaces
|
||||
|
||||
Consumes from task 06: `ChatEvent`, `Completion`, `FinishReason`, `InferError`, `Timings`.
|
||||
|
||||
Produces, in `crates/loopd/src/llama/assemble.rs`:
|
||||
|
||||
```rust
|
||||
#[derive(Default)]
|
||||
pub struct Assembler { /* private */ }
|
||||
impl Assembler {
|
||||
pub fn new() -> Self;
|
||||
/// Takes one `data:` payload (JSON text) and returns the events it stands for, in order.
|
||||
pub fn push(&mut self, data: &str) -> Result<Vec<ChatEvent>, InferError>;
|
||||
pub fn id(&self) -> Option<&str>; // the completion id, once a chunk has carried one
|
||||
pub fn reasoning_tokens(&self) -> u64; // tokens generated while the model was reasoning
|
||||
pub fn in_reasoning(&self) -> bool; // true while the latest text was reasoning
|
||||
pub fn is_finished(&self) -> bool; // a finish_reason has arrived
|
||||
pub fn finish(self, thinking_capped: bool) -> Result<Completion, InferError>;
|
||||
}
|
||||
```
|
||||
|
||||
Rules the tests check:
|
||||
|
||||
1. For one chunk, events come in this order: `Progress` (if `prompt_progress` is present), then
|
||||
for each choice: `Reasoning`, `Content`, then one `ToolCallDelta` per entry in `tool_calls`.
|
||||
Empty or null text produces no event. A `ToolCallDelta` carries `id` and `name` as they were in
|
||||
that piece (`None` in later pieces) and that piece's `arguments` text (`""` if absent).
|
||||
2. `content` and `reasoning_content` of the completion are the concatenation of the event texts,
|
||||
or `None` if there never was any. A tool call's `arguments` is the concatenation of its pieces;
|
||||
its `id` and `name` come from the first piece that has them.
|
||||
3. `timings` is the latest `timings` object seen.
|
||||
4. `reasoning_tokens`: whenever a chunk carries reasoning text, set it to the `predicted_n` of the
|
||||
latest timings (update timings first). Everything generated up to then was reasoning. It stops
|
||||
changing when the answer starts. `in_reasoning` is true after reasoning text, and false after
|
||||
answer text or a tool-call piece.
|
||||
5. `finish` fails with `StreamClosedEarly` if no `finish_reason` arrived, and with `Protocol` if no
|
||||
chunk carried an `id`, or a tool call never got an id or a name.
|
||||
6. `push` fails with `Protocol` for: text that is not a JSON object of this shape, a
|
||||
`finish_reason` other than the three above, and a tool call `index` that skips ahead (index 5
|
||||
when only calls 0 and 1 exist). The index comes from the server: convert it with `try_from`
|
||||
and use `get_mut`, never `[]`.
|
||||
|
||||
The expected results in `tests/fixtures/expected/*.json` were produced from the same recordings by
|
||||
a separate script, so the test checks your code against an independent reading of the same bytes.
|
||||
|
||||
## Steps
|
||||
|
||||
- [ ] **1. Copy.**
|
||||
|
||||
```sh
|
||||
git switch m2a
|
||||
cp docs/plans/M2a/files/crates/loopd/tests/assemble.rs crates/loopd/tests/
|
||||
cp -r docs/plans/M2a/files/crates/loopd/tests/fixtures/expected crates/loopd/tests/fixtures/
|
||||
```
|
||||
|
||||
- [ ] **2. See the test fail.** `cargo test -p loopd --test assemble`. Expected: it does not compile.
|
||||
- [ ] **3. Write `assemble.rs`** and register the module. Run `cargo fmt --all`.
|
||||
- [ ] **4. See the test pass.** `cargo test -p loopd --test assemble`. Expected: `7 passed`.
|
||||
- [ ] **5. Run the gate.** `make gate`. Expected last line: `gate: ok`.
|
||||
- [ ] **6. Log and commit.** `git add crates/loopd docs/implementer-log.md && git commit`
|
||||
|
||||
## Done when
|
||||
|
||||
- `cargo test -p loopd --test assemble` reports 7 passed; `make gate` prints `gate: ok`.
|
||||
- `grep -n deny_unknown_fields crates/loopd/src/llama/assemble.rs` prints nothing.
|
||||
|
||||
## Stop and report if
|
||||
|
||||
- Your result differs from a file in `fixtures/expected/` and you believe the file is wrong.
|
||||
Reference in New Issue
Block a user