Review M2a: accept with two follow-up tasks; record models and lessons

The branch passes every check, including make verify-device on
straylight and repeated timing runs under load. Reading and probing
found that inferproxy does not pass an upstream close on to a client
that is still sending, and that the chunked body reader delivers a
stream only when the caller's buffer fills or the stream ends. Both
were also gaps in the tasks and tests, so tasks 14 and 15 carry the
fixes with new tests checked against the reference.

The Model column is corrected: tasks 04 to 06 and 08 to 13 were Ornith.
Lessons gain four implementer tips and five task-writing tips; three
rules are promoted to AGENTS.md.

Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
2026-09-18 15:43:06 -07:00
co-authored by Claude Fable 5.1
parent ad7bce88a4
commit 58c7738721
8 changed files with 283 additions and 18 deletions
+23 -7
View File
@@ -16,12 +16,16 @@ How it is used:
| # | Tip | Evidence | In AGENTS.md | Seen again |
|---|---|---|---|---|
| I1 | When a rule says "every" or "everywhere", finish by listing each place it could apply (every struct and enum in the file, every script, every branch) and check them one by one. An example in the task shows one place; the rule covers all of them. | M1 finding 1: `deny_unknown_fields` was put on both enums in task 06, where the task's example showed it, and on neither struct. In task 03, where the task said "works on structs and on enums", every type got it. | yes | |
| I2 | A check must fail when it cannot do its job: missing input, unreadable file, a tool that errors. Never throw errors away (`2>/dev/null`, `\|\| true`, an ignored `Result`). | M1 finding 4: all three gate scripts passed when `crates/` did not exist, and hid `find` and `awk` errors. | yes | |
| I3 | Report every problem found, not only the first. | M1 finding 8: `check-lines.sh` exits at the first long file. | yes | |
| I4 | Run the formatter before the gate, and let it decide ordering. | M1 finding 7: three first-gate failures were rustfmt reordering `lib.rs`. | yes | |
| I5 | Log every attempt, including one you abandon. | M1 review: the task 01 row mentions files "from a prior attempt" that has no row of its own. | yes | |
| I6 | Prefer a conversion that can fail (`u32::try_from`) to a cast that is only safe because of a check somewhere else (`as u32`). | M1 finding 6: bounded `as` casts in `frame.rs`. Correct today, but the safety depends on a line ten lines away. | no (already implied by the code rules) | |
| I1 | When a rule says "every" or "everywhere", finish by listing each place it could apply (every struct and enum in the file, every script, every branch) and check them one by one. An example in the task shows one place; the rule covers all of them. | M1 finding 1: `deny_unknown_fields` was put on both enums in task 06, where the task's example showed it, and on neither struct. In task 03, where the task said "works on structs and on enums", every type got it. | yes | M2a: no. All six config structs have it; none of the server-format structs do. |
| I2 | A check must fail when it cannot do its job: missing input, unreadable file, a tool that errors. Never throw errors away (`2>/dev/null`, `\|\| true`, an ignored `Result`). | M1 finding 4: all three gate scripts passed when `crates/` did not exist, and hid `find` and `awk` errors. | yes | M2a: no recurrence. |
| I3 | Report every problem found, not only the first. | M1 finding 8: `check-lines.sh` exits at the first long file. | yes | M2a: not exercised. |
| I4 | Run the formatter before the gate, and let it decide ordering. | M1 finding 7: three first-gate failures were rustfmt reordering `lib.rs`. | yes | M2a: no fmt failures; the first-gate failures were clippy. |
| I5 | Log every attempt, including one you abandon. | M1 review: the task 01 row mentions files "from a prior attempt" that has no row of its own. | yes | M2a: partly. Task 11's `stopped` row was overwritten by its later `done` row. |
| I6 | Prefer a conversion that can fail (`u32::try_from`) to a cast that is only safe because of a check somewhere else (`as u32`). | M1 finding 6: bounded `as` casts in `frame.rs`. Correct today, but the safety depends on a line ten lines away. | no (already implied by the code rules) | M2a: no `as` casts at all. |
| I7 | A `read` returns as soon as it has any data to give. It blocks only when it has none. Never keep reading to fill the caller's buffer. | M2a finding 2: the chunked reader delivered a whole stream at its end. | yes | |
| I8 | When forwarding between two connections, pass a close on in both directions. Do not rely on the client half-closing first. | M2a finding 1. | no (specific to inferproxy) | |
| I9 | Never end a turn by describing what you are about to do. Do it, then report. | Two Ornith turns in M2a ended with a "## Objective" plan and no tool call. | yes | |
| I10 | If a tool you were told to use does not exist, stop and say so. Do not invent a command in its place. | Laguna as coordinator ran `opencodec`, which does not exist, and then diagnosed its own typo. | yes | |
## Tips for writing tasks
@@ -35,6 +39,11 @@ How it is used:
| T6 | Do not tell the implementer how to lay out what a formatter controls. | M1 finding 7. |
| T7 | A follow-up task works well as: what the reviewer observed, which part was the task's fault, a failing test to copy in, and the new rules. | Tasks 08 and 09 each passed the gate on the first run, and 09 handled six forms its self-test did not contain. |
| T8 | Vet a dependency by testing it against an independent implementation before naming it in a spec, whoever wrote it. Include the boundary cases of its algorithm. | The owner's `emsha` 1.0.3 passed its own tests and hashed every message of length 63 mod 64 wrongly. A 90-case differential run against `sha256sum` found it in minutes. Same shape as T5: an author's tests share the author's blind spots. |
| T9 | State the contract of a standard trait the implementer must honour, in the task, even when it seems obvious. | M2a finding 2: task 04 said what `Body` reads but not when `read` must return. Two implementations, the implementer's and the reference, both got it wrong in different degrees. |
| T10 | Test *when* streamed data is delivered, not only *what*. A test that compares final bytes cannot see buffering. | M2a finding 2 passed 15 tests that only compared bytes. |
| T11 | Make the fake client behave like the real one. If the real client never half-closes, no test client may half-close either. | M2a finding 1: all test clients half-closed; `loopd` does not. |
| T12 | A weak model should not coordinate other sessions. Use the shell driver (`tools/run-plan.sh`), whose checks are code. | The `opencodec` episode. |
| T13 | Set the sampling defaults on the server for coding agents (`temp`, `top-p`, `top-k`, `reasoning-budget`). OpenCode sends none, so the server's default temperature of 1.0 applies, and thinking has no cap. | Five Ornith turns ran 16k tokens of thinking to the output limit and produced nothing. |
## What worked and should be kept
@@ -48,7 +57,7 @@ How it is used:
- One task, one fresh session, one commit, with only listed paths staged. The history reads as the
plan.
## What we have seen of this implementer so far
## What we have seen of the implementers so far
Nine tasks, all in M1, all small and pinned by tests, so treat these as first impressions.
@@ -60,3 +69,10 @@ Nine tasks, all in M1, all small and pinned by tests, so treat these as first im
- Given a clear defect report and a failing test, fixes the defect minimally and generalises.
- Gate on the first run: 6 of 9 tasks. Wall time per task, including the owner's turnaround: about
10 to 55 minutes.
M2a added two more models. Ornith-1.5-35B-A3B did nine tasks: it found and fixed a real bug in its
own chunked reader, wrote module docs everywhere, and stopped correctly when a prerequisite was
missing; its two failure modes were runaway thinking to the output limit and ending a turn with a
plan instead of a tool call. GLM-5.3 did one task, the assembler, correctly and quickly, with the
best comments of the milestone. Laguna did three and a half tasks in M2a with more nudging than in
M1, and failed as a coordinator. First-gate pass rate for the milestone: 8 of 13.
+54 -9
View File
@@ -18,16 +18,16 @@ reviewer adds findings under "Reviews" once per milestone.
| M2a/01-proto-sha256 | 2026-09-17 | done | 1 | pass | none | Added crates/proto/src/hash.rs wrapping emsha 1.0.4 (HashError, Sha256 with new/update/finish, sha256, Default); re-exported from lib.rs, added emsha workspace dep and dependencies.md row. One compile fix: finish needed `mut self` to call finalize. 4 hash tests pass, `make gate` prints `gate: ok`. | Laguna S 2.1 |
| M2a/02-inferproxy | 2026-09-17 | done | 3 | fail | none | Implemented Limits/TokenBucket/OpenGuard/serve/handle/forward in lib.rs (std only: the inference server's JSON is upstream's concern) and arg parsing + stale-socket handling in main.rs; TokenBucket counts thousandths of a token in u128 so partial refills stay additive and `saturating_duration_since` on an earlier `now` never underflows, capping at burst; serve refuses by rate-limit then open-connection limit; handle drops its OpenGuard before closing the client so a freshly-finished client is not refused on re-connect (rule 4). The scaffold `proto` dependency in `crates/inferproxy/Cargo.toml` was left untouched since the task's Modify/Copy lists cover only lib.rs, main.rs, the two test files and this log. First `make gate` failed on `clippy::map_clone` (`main.rs` used `.map(String::clone)`); switched to `.cloned()` and re-ran, then re-ran once more after restoring the proto scaffold — both pass. Hand test against straylight returned `{"status":"ok"}`; `forward.rs` passed 6/6 ten runs in a row. | Laguna S 2.1 |
| M2a/03-loopd-config | 2026-09-17 | done | 1 | pass | none | Wrote crates/loopd/src/config.rs: Config + Infer/Slots/Expect/Sampling/Limits with deny_unknown_fields on all six and struct-level `#[serde(deny_unknown_fields, default)]` on Sampling and Limits; manual Default impls for the two; hand-written Display/std::error::Error ConfigError named by file. Everywhere check: all six structs (Infer, Slots, Expect, Sampling, Limits, Config) carry deny_unknown_fields. One local fix: Display used `{path}` on a PathBuf and failed to build, switched to `path.display()`. 6 config tests pass; `make gate` prints `gate: ok`. | Laguna S 2.1 |
| M2a/04-loopd-http | 2026-09-18 | done | 2 | fail | none | Added `pub mod http;` to crates/loopd/src/lib.rs and wrote crates/loopd/src/http.rs (423 lines): Request/Head/HttpError, send (exact header order, no `Content-Length` for GET), read_head (buffers across timeouts; `Timeout`/`Closed`/`TooLarge`/`Malformed`), parse_status+parse_head (`HTTP/1.1`/`HTTP/1.0`, status 100-599, lowercased names, duplicates kept, trimmed values), body (chunked/length/close; chunk extensions + trailers skipped), and read_capped. Two fixes: the chunk `Data` phase jumped to `Crlf` on `take==want`, but `want` was capped by the caller's buffer so it switched mid-chunk and returned `malformed chunk` on the recorded fixture — changed to switch on `chunk_remaining==0`; `read_length` reads straight into the caller buffer instead of an 8192 temp slice that would panic for readers larger than 8192. First gate failed on three clippy lints (needless borrows of `format!` results and `map_or` -> `is_some_and`), fixed on the second run. All 15 tests pass and `make gate` prints `gate: ok`. | Laguna S 2.1 |
| M2a/05-loopd-sse | 2026-09-18 | done | 2 | fail | none | Added `pub mod sse;` to crates/loopd/src/lib.rs and wrote crates/loopd/src/sse.rs: SseItem (Data, Done), SseError (Io, Timeout, Truncated, LineTooLong, NotUtf8) with Display/std::error::Error, and Events<R> which reads a line in bounded 4096-byte chunks, skipping blank/comment/other-field lines and stripping `data:` plus one leading space, decoding UTF-8 only once a whole line has arrived. Two fixes: `drain(..pos)` left the newline in the buffer so blank lines never advanced — changed to `drain(..=pos)` and pop the endings; `process_line` returns `Ok(None)` for a skipped line, which collided with `next_item`'s "stream ended" `Ok(None)` — restructured so a skip continues the loop and only a clean EOF sets `ended`. Both LineTooLong checks (mid-arrival and whole-read) verified by checking the accumulated length before reading and the finished line length. All 7 tests pass; first gate run failed on an unused import, fixed. | Laguna S 2.1 |
| M2a/06-llama-request | 2026-09-18 | done | 1 | pass | none | Added `pub mod llama;` to crates/loopd/src/lib.rs and wrote crates/loopd/src/llama/mod.rs (ChatMessage, ToolSchema, ChatRequest, ChatEvent, FinishReason, Timings with the server's deserialize shape, Completion, InferError with Display/std::error::Error, and a Client holding the config) and crates/loopd/src/llama/request.rs. build_body serializes the body from `#[derive(Serialize)]` structs so key order is fixed; each message kind is serialized with its own struct (the assistant renders `content` as `""` when `None`, and leaves `reasoning_content`/`tool_calls` out when absent; the top-level `tools` array is omitted when empty; `type` comes from `#[serde(rename = "type")]`); the first `cargo build` after writing the structs missed the `role` field on every message struct, caught by the failing test compile, added. 5 request tests pass; `make gate` prints `gate: ok`. | Laguna S 2.1 |
| M2a/04-loopd-http | 2026-09-18 | done | 2 | fail | none | Added `pub mod http;` to crates/loopd/src/lib.rs and wrote crates/loopd/src/http.rs (423 lines): Request/Head/HttpError, send (exact header order, no `Content-Length` for GET), read_head (buffers across timeouts; `Timeout`/`Closed`/`TooLarge`/`Malformed`), parse_status+parse_head (`HTTP/1.1`/`HTTP/1.0`, status 100-599, lowercased names, duplicates kept, trimmed values), body (chunked/length/close; chunk extensions + trailers skipped), and read_capped. Two fixes: the chunk `Data` phase jumped to `Crlf` on `take==want`, but `want` was capped by the caller's buffer so it switched mid-chunk and returned `malformed chunk` on the recorded fixture — changed to switch on `chunk_remaining==0`; `read_length` reads straight into the caller buffer instead of an 8192 temp slice that would panic for readers larger than 8192. First gate failed on three clippy lints (needless borrows of `format!` results and `map_or` -> `is_some_and`), fixed on the second run. All 15 tests pass and `make gate` prints `gate: ok`. | Laguna S 2.1 (abandoned after two sessions), then Ornith-1.5-35B-A3B |
| M2a/05-loopd-sse | 2026-09-18 | done | 2 | fail | none | Added `pub mod sse;` to crates/loopd/src/lib.rs and wrote crates/loopd/src/sse.rs: SseItem (Data, Done), SseError (Io, Timeout, Truncated, LineTooLong, NotUtf8) with Display/std::error::Error, and Events<R> which reads a line in bounded 4096-byte chunks, skipping blank/comment/other-field lines and stripping `data:` plus one leading space, decoding UTF-8 only once a whole line has arrived. Two fixes: `drain(..pos)` left the newline in the buffer so blank lines never advanced — changed to `drain(..=pos)` and pop the endings; `process_line` returns `Ok(None)` for a skipped line, which collided with `next_item`'s "stream ended" `Ok(None)` — restructured so a skip continues the loop and only a clean EOF sets `ended`. Both LineTooLong checks (mid-arrival and whole-read) verified by checking the accumulated length before reading and the finished line length. All 7 tests pass; first gate run failed on an unused import, fixed. | Ornith-1.5-35B-A3B |
| M2a/06-llama-request | 2026-09-18 | done | 1 | pass | none | Added `pub mod llama;` to crates/loopd/src/lib.rs and wrote crates/loopd/src/llama/mod.rs (ChatMessage, ToolSchema, ChatRequest, ChatEvent, FinishReason, Timings with the server's deserialize shape, Completion, InferError with Display/std::error::Error, and a Client holding the config) and crates/loopd/src/llama/request.rs. build_body serializes the body from `#[derive(Serialize)]` structs so key order is fixed; each message kind is serialized with its own struct (the assistant renders `content` as `""` when `None`, and leaves `reasoning_content`/`tool_calls` out when absent; the top-level `tools` array is omitted when empty; `type` comes from `#[serde(rename = "type")]`); the first `cargo build` after writing the structs missed the `role` field on every message struct, caught by the failing test compile, added. 5 request tests pass; `make gate` prints `gate: ok`. | Ornith-1.5-35B-A3B |
| M2a/07-llama-assemble | 2026-09-18 | done | 3 | fail | none | Wrote crates/loopd/src/llama/assemble.rs (Assembler plus server-format Chunk/Choice/Delta/ToolCallPiece/FunctionPiece/PromptProgress structs with no deny_unknown_fields) and registered `pub mod assemble;`. Accumulation: text goes through get_or_insert_with so content/reasoning stay None until a non-empty piece arrives; tool-call pieces land by index via u32::try_from then usize::try_from and get_mut, a new call only at exactly the end, a skip-ahead or an out-of-range index is Protocol; timings update before the reasoning-token count reads predicted_n; finish checks finish_reason (StreamClosedEarly), then id, then every tool call has id and name. First gate failed on two clippy collapsible_if lints; rewrote the two nested ifs as edition-2024 let-chains and re-ran. All 7 assemble tests pass; `make gate` prints `gate: ok`. | GLM-5.3 (z.ai, default settings) |
| M2a/08-llama-info | 2026-09-18 | done | 2 | fail | none | Wrote crates/loopd/src/llama/info.rs and registered `pub mod info;`. `call` is one exchange: open socket, set read timeout to `liveness_ms`, send, read head, `read_capped` with `MAX_BODY`; non-200 returns `InferError::Http { status, error_text(&bytes) }`, everything else maps through `map_http` (Connect->Connect, Timeout->Stalled, Closed->StreamClosedEarly, else->Protocol). `error_text` reads the full body via `from_utf8_lossy` then backs up from byte 4096 to a char boundary, so a cut mid-character does not panic. `props` reads `chat_template`, `total_slots`, and `default_generation_settings.n_ctx` from the JSON (unknown fields ignored); `slots` deserializes `Vec<SlotInfo>`; `tokenize` POSTs `{"model","content"}` via `serde_json` and returns `tokens.len()`. `cache_outcome` uses `saturating_add` and `current.cache_n + CACHE_TOLERANCE >= expected`. All 7 tests pass; first gate run failed on rustfmt import order, fixed with `cargo fmt --all`. | ? |
| M2a/11-llama-gate-retry | 2026-09-18 | done | 3 | fail | none | Prerequisite `chat` (M2a/09) now exists, so the task was possible. Implemented `SlotGate` in gate.rs: per-slot holder + a `VecDeque` of arrival tickets, `notify_all`, a woken waiter takes the slot only if free and its ticket is at the front (and claims it by setting holder), `Drop` frees and wakes; mutex/condvar poison recovered via `unwrap_or_else(...into_inner)`, no `unwrap`. Implemented `chat_with_retry` + `is_retryable` (all nine variants, a new one is a compile error) + `backoff_ms` in retry.rs: base is `schedule[retry-1]` or last or 0, jitter clamped and computed in `i128` so `u64::MAX` never overflows, jitter from sub-second nanos. `chat` acquires the gate for `req.slot` and maps `GateFull`->`InferError::Busy`; `Client` gained a `gate` field. First gate failed on three clippy lints (derivable `Default`, `or_insert_with`->`or_default`), fixed. One logic bug caught by `waiters_are_served_in_order`: `take_if_front` claimed the ticket but not `holder`, letting two permits overlap — set holder on claim. All 79 loopd tests pass; retry 13/13 over ten runs; `make gate` prints `gate: ok`. | ? |
| M2a/09-llama-chat | 2026-09-18 | done | 1 | pass | none | Wrote crates/loopd/src/llama/chat.rs (`chat`, with the head wait in a separate `wait_for_head`) and registered `pub mod chat;`. `chat` builds the body (build error -> Protocol), opens and POSTs, then `wait_for_head` loops `read_head` at `poll_ms`: a Timeout is classified Idle/Busy/Unavailable by `received_any` then a `slots()` poll, emits `Waiting { slot_busy }` on every poll, keeps a per-state `since` that resets on state change, and returns `WaitTimeout`/`LoadTimeout`/`Stalled` at the right limits; 200 streams via `Events`+`Assembler` mapping `Timeout->Stalled`, `Truncated->StreamClosedEarly`, else `Protocol`, then `finish(false)`; non-200 returns `Http { status, error_text }`. All 13 chat tests pass five runs in a row. Table-to-test map: state (table 1) Busy -> a_busy_slot_is_waited_out / a_slot_that_stays_busy_is_a_wait_timeout, Idle-not-processing -> only_the_requests_own_slot_counts_as_busy, Unavailable -> an_unavailable_server_is_a_load_timeout, Idle-received_any -> a_slow_but_steady_stream_is_not_a_stall (turn1 head is 738 B, pieces are 3140/8=392 B, so the head-wait times out with a partial head); thresholds (table 2) -> a_slot_that_stays_busy_is_a_wait_timeout / an_unavailable_server_is_a_load_timeout / only_the_requests_own_slot_counts_as_busy; stream (table 3) Data/Done/None -> the recorded completion and trickle tests, Timeout -> silence_in_mid_stream_is_a_stall, Truncated -> a_stream_that_is_cut_is_closed_early_wherever_the_cut_falls, other -> garbage_in_the_stream_is_a_protocol_error; clock-restart -> the_wait_clocks_restart_when_the_state_changes. One path fix: info and request live under crate::llama, not crate::, so the imports use crate::llama::{info::..., request::...}. | ? |
| M2a/10-llama-cap | 2026-09-18 | done | 1 | pass | none | Added `Client::end_reasoning` to info.rs: POSTs `{"id","action":"reasoning_end","model"}` to `/v1/chat/completions/control` via `call`, reads `success` as a bool from the server's JSON (ignoring `message`), non-200 stays an Err through `call`, a missing/non-bool `success` is Protocol. Threaded the cap into `chat` step 4: after passing a chunk's events on, when `assembler.in_reasoning()`, a local `cap_at: Option<u64>` holds where the cap fired (None while it has not fired); on `tokens >= thinking_cap` it calls `end_reasoning(assembler.id())` once, remembers `tokens` and emits `ThinkingCapped` on `Ok(true)`, returns `ThinkingOverrun` on `Ok(false)`/`Err`, and after firing returns `ThinkingOverrun` once `tokens >= at + thinking_overrun`; `finish(cap_at.is_some())`. The `the_allowance_is_exact` test passes with `>=` in both rows (63 is not `20+44`, and is `>= 20+43`). One guard: a reasoning chunk with no id at cap time is Protocol rather than a panic. 6 cap tests + 13 chat tests pass; `make gate` prints `gate: ok`. | ? |
| M2a/12-selftest | 2026-09-18 | done | 1 | pass | none |
| M2a/13-verify-device | 2026-09-18 | done | 1 | pass | none | Wrote no library code. Put the two given files in place (`crates/loopd/tests/device.rs`, and a `Makefile` whose only difference from the old one is the new `verify-device` target), confirmed `make gate` prints `gate: ok` with `device` at `0 passed; 0 failed; 4 ignored`, and `curl http://straylight:11434/health` returned `{"status":"ok"}`. `make verify-device` ran the four `#[ignore]` checks one at a time against the real server on slot 0: self-test, capped thinking block, a request surviving its own proxy being killed and restarted, and a second turn reusing the cache of a first turn that contained thinking. All 4 passed in ~21s (well under two minutes). | ? | Wrote crates/loopd/src/selftest.rs (SelfTestError with Display/std::error::Error/From<InferError>, and `run` doing the three checks in order) and registered `pub mod selftest;` in lib.rs; rewrote main.rs into `loopd selftest --config <path>`. Check 1 calls props() and compares chat-template sha256, then n_ctx, then total_slots, a sha256 error being Hash so a wrong server is sent no prompt; check 2 posts one read_file tool with chat_with_retry and requires finish_reason ToolCalls, a first call named read_file whose arguments parse as JSON carrying a string path, wrapping an InferError as Infer; check 3 runs turn 1 then an extension of it and maps a cache Loss to CacheMiss. The copied test's cache_outcome/CacheOutcome live at crate::llama::info, so the import follows that. main.rs parses args as &[&str] via a two-step String->&str collect; unknown/missing args are exit 2 and a config load failure is `loopd: <error>` exit 1. Real server via inferproxy against straylight: minimal.toml gave three step lines and `selftest: ok` exit 0; setting slots=3 gave `selftest: FAILED: slot count: expected 3, got 2` exit 1. | ? |
| M2a/08-llama-info | 2026-09-18 | done | 2 | fail | none | Wrote crates/loopd/src/llama/info.rs and registered `pub mod info;`. `call` is one exchange: open socket, set read timeout to `liveness_ms`, send, read head, `read_capped` with `MAX_BODY`; non-200 returns `InferError::Http { status, error_text(&bytes) }`, everything else maps through `map_http` (Connect->Connect, Timeout->Stalled, Closed->StreamClosedEarly, else->Protocol). `error_text` reads the full body via `from_utf8_lossy` then backs up from byte 4096 to a char boundary, so a cut mid-character does not panic. `props` reads `chat_template`, `total_slots`, and `default_generation_settings.n_ctx` from the JSON (unknown fields ignored); `slots` deserializes `Vec<SlotInfo>`; `tokenize` POSTs `{"model","content"}` via `serde_json` and returns `tokens.len()`. `cache_outcome` uses `saturating_add` and `current.cache_n + CACHE_TOLERANCE >= expected`. All 7 tests pass; first gate run failed on rustfmt import order, fixed with `cargo fmt --all`. | Ornith-1.5-35B-A3B |
| M2a/11-llama-gate-retry | 2026-09-18 | done | 3 | fail | none | Prerequisite `chat` (M2a/09) now exists, so the task was possible. Implemented `SlotGate` in gate.rs: per-slot holder + a `VecDeque` of arrival tickets, `notify_all`, a woken waiter takes the slot only if free and its ticket is at the front (and claims it by setting holder), `Drop` frees and wakes; mutex/condvar poison recovered via `unwrap_or_else(...into_inner)`, no `unwrap`. Implemented `chat_with_retry` + `is_retryable` (all nine variants, a new one is a compile error) + `backoff_ms` in retry.rs: base is `schedule[retry-1]` or last or 0, jitter clamped and computed in `i128` so `u64::MAX` never overflows, jitter from sub-second nanos. `chat` acquires the gate for `req.slot` and maps `GateFull`->`InferError::Busy`; `Client` gained a `gate` field. First gate failed on three clippy lints (derivable `Default`, `or_insert_with`->`or_default`), fixed. One logic bug caught by `waiters_are_served_in_order`: `take_if_front` claimed the ticket but not `holder`, letting two permits overlap — set holder on claim. All 79 loopd tests pass; retry 13/13 over ten runs; `make gate` prints `gate: ok`. | Ornith-1.5-35B-A3B |
| M2a/09-llama-chat | 2026-09-18 | done | 1 | pass | none | Wrote crates/loopd/src/llama/chat.rs (`chat`, with the head wait in a separate `wait_for_head`) and registered `pub mod chat;`. `chat` builds the body (build error -> Protocol), opens and POSTs, then `wait_for_head` loops `read_head` at `poll_ms`: a Timeout is classified Idle/Busy/Unavailable by `received_any` then a `slots()` poll, emits `Waiting { slot_busy }` on every poll, keeps a per-state `since` that resets on state change, and returns `WaitTimeout`/`LoadTimeout`/`Stalled` at the right limits; 200 streams via `Events`+`Assembler` mapping `Timeout->Stalled`, `Truncated->StreamClosedEarly`, else `Protocol`, then `finish(false)`; non-200 returns `Http { status, error_text }`. All 13 chat tests pass five runs in a row. Table-to-test map: state (table 1) Busy -> a_busy_slot_is_waited_out / a_slot_that_stays_busy_is_a_wait_timeout, Idle-not-processing -> only_the_requests_own_slot_counts_as_busy, Unavailable -> an_unavailable_server_is_a_load_timeout, Idle-received_any -> a_slow_but_steady_stream_is_not_a_stall (turn1 head is 738 B, pieces are 3140/8=392 B, so the head-wait times out with a partial head); thresholds (table 2) -> a_slot_that_stays_busy_is_a_wait_timeout / an_unavailable_server_is_a_load_timeout / only_the_requests_own_slot_counts_as_busy; stream (table 3) Data/Done/None -> the recorded completion and trickle tests, Timeout -> silence_in_mid_stream_is_a_stall, Truncated -> a_stream_that_is_cut_is_closed_early_wherever_the_cut_falls, other -> garbage_in_the_stream_is_a_protocol_error; clock-restart -> the_wait_clocks_restart_when_the_state_changes. One path fix: info and request live under crate::llama, not crate::, so the imports use crate::llama::{info::..., request::...}. | Ornith-1.5-35B-A3B |
| M2a/10-llama-cap | 2026-09-18 | done | 1 | pass | none | Added `Client::end_reasoning` to info.rs: POSTs `{"id","action":"reasoning_end","model"}` to `/v1/chat/completions/control` via `call`, reads `success` as a bool from the server's JSON (ignoring `message`), non-200 stays an Err through `call`, a missing/non-bool `success` is Protocol. Threaded the cap into `chat` step 4: after passing a chunk's events on, when `assembler.in_reasoning()`, a local `cap_at: Option<u64>` holds where the cap fired (None while it has not fired); on `tokens >= thinking_cap` it calls `end_reasoning(assembler.id())` once, remembers `tokens` and emits `ThinkingCapped` on `Ok(true)`, returns `ThinkingOverrun` on `Ok(false)`/`Err`, and after firing returns `ThinkingOverrun` once `tokens >= at + thinking_overrun`; `finish(cap_at.is_some())`. The `the_allowance_is_exact` test passes with `>=` in both rows (63 is not `20+44`, and is `>= 20+43`). One guard: a reasoning chunk with no id at cap time is Protocol rather than a panic. 6 cap tests + 13 chat tests pass; `make gate` prints `gate: ok`. | Ornith-1.5-35B-A3B |
| M2a/12-selftest | 2026-09-18 | done | 1 | pass | Ornith-1.5-35B-A3B |
| M2a/13-verify-device | 2026-09-18 | done | 1 | pass | none | Wrote no library code. Put the two given files in place (`crates/loopd/tests/device.rs`, and a `Makefile` whose only difference from the old one is the new `verify-device` target), confirmed `make gate` prints `gate: ok` with `device` at `0 passed; 0 failed; 4 ignored`, and `curl http://straylight:11434/health` returned `{"status":"ok"}`. `make verify-device` ran the four `#[ignore]` checks one at a time against the real server on slot 0: self-test, capped thinking block, a request surviving its own proxy being killed and restarted, and a second turn reusing the cache of a first turn that contained thinking. All 4 passed in ~21s (well under two minutes). | ? | Wrote crates/loopd/src/selftest.rs (SelfTestError with Display/std::error::Error/From<InferError>, and `run` doing the three checks in order) and registered `pub mod selftest;` in lib.rs; rewrote main.rs into `loopd selftest --config <path>`. Check 1 calls props() and compares chat-template sha256, then n_ctx, then total_slots, a sha256 error being Hash so a wrong server is sent no prompt; check 2 posts one read_file tool with chat_with_retry and requires finish_reason ToolCalls, a first call named read_file whose arguments parse as JSON carrying a string path, wrapping an InferError as Infer; check 3 runs turn 1 then an extension of it and maps a cache Loss to CacheMiss. The copied test's cache_outcome/CacheOutcome live at crate::llama::info, so the import follows that. main.rs parses args as &[&str] via a two-step String->&str collect; unknown/missing args are exit 2 and a config load failure is `loopd: <error>` exit 1. Real server via inferproxy against straylight: minimal.toml gave three step lines and `selftest: ok` exit 0; setting slots=3 gave `selftest: FAILED: slot count: expected 3, got 2` exit 1. | Ornith-1.5-35B-A3B |
## Reviews
@@ -105,3 +105,48 @@ Remaining, recorded and not worth a task:
|---|---|---|
| 8 | low | `check-lines.sh` stops at the first file that is too long, so a second one is only reported after the first is fixed. |
| 9 | low | A quoted key (`"brokerd" = { path = "…" }`) is not seen by either dependency script. The task did not list that form and the reference scripts miss it too. It does not happen by accident. The robust fix is to ask `cargo metadata`, which needs a JSON parser the gate does not have. |
### M2a, tasks 01 to 13 — reviewed 2026-09-18 by the design model (Claude)
**Verdict: accepted, with two follow-up tasks (14, 15).** Nothing has to be redone. Models: tasks
01 to 03 Laguna S 2.1; 04 started by Laguna and finished by Ornith-1.5-35B-A3B; 05, 06 and 08 to 13
Ornith; 07 GLM-5.3.
Checklist from `docs/plans/M2a/README.md`:
| Check | Result |
|---|---|
| Fourteen implementer commits (thirteen tasks, plus task 11's correct early stop), each with the trailer | pass |
| All 38 copied files byte-identical to the plan | pass |
| No change to the brief, specs, plans, `CLAUDE.md`, `deny.toml`; `AGENTS.md` changed only by the reviewer | pass |
| `make gate` | `gate: ok`, 151 tests, 4 ignored (the reference had the same numbers) |
| `make audit` | `advisories ok` |
| `make verify-device` against straylight | 4 passed in 19 s |
| Every timing suite 12 times under heavy CPU load | no failure |
| No `unwrap`, `expect`, `panic!`, `#[allow]`, `unsafe` or `as` cast in library code | pass |
| `deny_unknown_fields` on all six config structs and on none of the server-format structs | pass |
Process: the first gate run passed in 8 of 13 tasks. The failures were clippy lints, one
compile fix, and the chunked-body bug the implementer found and fixed itself in task 04.
Findings. "Implementer" means the task said it and the code missed it; "task" means the task or
its tests were silent or wrong. Both follow-ups are one of each.
| # | Severity | Owner | Finding | Fix |
|---|---|---|---|---|
| 1 | medium | implementer (Laguna, task 02) and task | `inferproxy` closes towards the client only after the client has stopped sending. `loopd`'s client never does, so a server that closes or dies mid-answer is reported as `Stalled` after the liveness limit, not `StreamClosedEarly` at once. Task 02 rule 3 said "close both"; every test client half-closed, so the tests could not see it. Confirmed with a probe: no EOF within 3 s of the upstream closing. | Task 14; new test in `forward.rs` |
| 2 | medium | implementer (Ornith, task 04) and task | The chunked body reader returns only when the caller's buffer is full or the stream ends. Measured: with events 300 ms apart, all seven were delivered when the last arrived. The cap fires up to about twenty chunks late, M2b's text would arrive in lumps, and bytes already copied are lost when a later read in the same call times out. Task 04 never stated `Read`'s contract, and no test looked at delivery timing. The reference implementation held data for a chunk's trailing CRLF, a smaller form of the same fault; fixed in the reference too. | Task 15; new test in `http.rs` |
| 3 | low | implementer (Ornith, task 04) | The commit subject is `M2a/04-loopd-http` instead of the one the task gave; the log row says "deviations: none". | Noted |
| 4 | low | implementer (Ornith, task 11) | The `stopped` row from the first attempt was overwritten by the `done` row instead of a new row being added. The stop itself was correct: `chat` did not exist yet because the owner started task 11 out of order. | Noted; the rule is restated in `AGENTS.md` |
| 5 | nit | implementer | `http.rs` does not retry a read that fails with `Interrupted`; `sse.rs` does. | When next touched |
What was good: no panics anywhere in 2,300 lines that parse peer input; the gate and retry
modules are exactly as specified, with an exhaustive match in `is_retryable`; every file has a
module doc comment, which M1's did not; the assembler (GLM) survived ten probes outside its tests;
Ornith found and fixed a real chunked-framing bug in its own code during task 04.
Observed in the sessions (from OpenCode's database, not the log): of 259 Ornith turns, 5 ran the
thinking block to OpenCode's 16,384-token output limit and produced nothing, and 2 ended by
describing a plan instead of calling a tool; both look like "the model stopped" to the owner.
Laguna, asked to coordinate the tasks, invented a command-line tool (`opencodec`) rather than
stop when the subagent tool it was told to use was not available.
+52
View File
@@ -0,0 +1,52 @@
# M2a task 14: pass the upstream's close on to the client (review follow-up)
**Branch:** `m2a` (run `git switch m2a`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `Close the client side when the upstream closes, in inferproxy`
## Goal
The M2a review found that `inferproxy` only closes towards the client after the client has stopped
sending. `loopd`'s HTTP client never stops sending on its own: it keeps its side open until it
drops the connection. So when `llama-server` closes, or dies in the middle of an answer, the client
is not told; it sits until its own liveness timeout and reports `Stalled` instead of
`StreamClosedEarly`, thirty seconds late. A body that runs to the close would hang the same way.
Task 02, rule 3, said: "When the server stops sending, the exchange is over: close both." The
current code waits for both copy threads to finish instead. This was partly the plan's fault: every
test client in `forward.rs` half-closed after sending, so the gap could not show. The new test
sends without half-closing.
## Files
- Copy (replacing the old one): `crates/inferproxy/tests/forward.rs`
- Modify: `crates/inferproxy/src/lib.rs`, `docs/implementer-log.md`
## Required behaviour
- When the server-to-client copy ends, for any reason, shut down the client connection (both
directions) and the server connection, so that the client's read returns 0 at once and the
client-to-server copy thread ends. Do this **after** releasing the open-connection place, as
rule 4 of task 02 requires.
- Everything else from task 02 still holds, including the half-close when the client stops
sending first.
## Steps
- [ ] **1. Copy.** `git switch m2a`, then
`cp docs/plans/M2a/files/crates/inferproxy/tests/forward.rs crates/inferproxy/tests/`
- [ ] **2. See it fail.** `cargo test -p inferproxy --test forward`. Expected: 1 of 7 fails,
`the_upstreams_close_reaches_a_client_that_is_still_sending`, with a timeout.
- [ ] **3. Fix `lib.rs`.** Run `cargo fmt --all`.
- [ ] **4. See it pass.** `cargo test -p inferproxy --test forward`, ten times in a row. Expected:
`7 passed` every time.
- [ ] **5. Run the gate.** `make gate`. Expected last line: `gate: ok`.
- [ ] **6. Log and commit.** `git add crates/inferproxy docs/implementer-log.md && git commit`
## Done when
- `cargo test -p inferproxy --test forward` reports 7 passed, ten runs in a row; `make gate` prints
`gate: ok`; `cmp crates/inferproxy/tests/forward.rs docs/plans/M2a/files/crates/inferproxy/tests/forward.rs` prints nothing.
## Stop and report if
- Passing the new test breaks `forwards_both_ways_and_passes_the_half_close_on`.
+60
View File
@@ -0,0 +1,60 @@
# M2a task 15: deliver streamed data as it arrives (review follow-up)
**Branch:** `m2a` (run `git switch m2a`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `Return chunked body data as soon as it is available`
## Goal
The M2a review found that the chunked body reader in `http.rs` keeps reading until the caller's
buffer is full or the stream ends. With an 8 KiB buffer that means a streamed completion reaches
`loopd` in bursts of about twenty events, and a short completion arrives all at once at its end.
The reviewer measured it: with events sent 300 ms apart, every event was delivered at the moment
the last one arrived. Three things follow: the thinking cap fires up to twenty chunks late, the
text a user sees in M2b would come in lumps, and bytes already copied into the caller's buffer are
lost when a later read in the same call times out.
Task 04 did not state the contract of `std::io::Read`, which is the plan's fault. The contract is:
**a `read` returns as soon as it has any data to give. It blocks only when it has none.** The test
you copy in sends a recording in pieces 300 ms apart and requires the first data within 200 ms and
the rest spread out over the run.
## Files
- Copy (replacing the old one): `crates/loopd/tests/http.rs`
- Modify: `crates/loopd/src/http.rs`, `docs/implementer-log.md`
## Required behaviour
In the chunked framing, `Body::read`:
1. Returns as soon as it has copied at least one byte of chunk data into the caller's buffer, even
if the buffer is not full and even if the chunk's trailing `\r\n` has not arrived yet. Consume
that `\r\n` at the start of the **next** call.
2. Reads from the socket only when it has no data to give: at a chunk-size line, at a pending
`\r\n`, or at the trailers.
3. Everything else from task 04 still holds: the same bytes come out however they arrive, errors
and limits are unchanged, and the 15 earlier tests still pass.
The length and close framings already behave this way.
## Steps
- [ ] **1. Copy.** `git switch m2a`, then
`cp docs/plans/M2a/files/crates/loopd/tests/http.rs crates/loopd/tests/`
- [ ] **2. See it fail.** `cargo test -p loopd --test http`. Expected: 1 of 16 fails,
`streamed_data_is_delivered_as_it_arrives`, saying the reader is buffering the stream.
- [ ] **3. Fix `read_chunked`.** Run `cargo fmt --all`.
- [ ] **4. See it pass.** `cargo test -p loopd`. Expected: `http` 16 passed and every other file as
before (the `chat`, `cap`, `sse` and `selftest` tests read through this code).
- [ ] **5. Run the gate.** `make gate`. Expected last line: `gate: ok`.
- [ ] **6. Log and commit.** `git add crates/loopd docs/implementer-log.md && git commit`
## Done when
- `cargo test -p loopd --test http` reports 16 passed; `make gate` prints `gate: ok`;
`cmp crates/loopd/tests/http.rs docs/plans/M2a/files/crates/loopd/tests/http.rs` prints nothing.
## Stop and report if
- `the_result_does_not_depend_on_how_the_bytes_arrive` fails after the change: the fix has broken
the framing somewhere.
+4 -2
View File
@@ -1,7 +1,7 @@
# M2a implementation plan: the inference path
> **For the implementing model:** do not work from this file. The owner gives you one task file at
> a time (`01-…` to `13-…`). This file is the index for the owner and the reviewer.
> a time (`01-…` to `15-…`). This file is the index for the owner and the reviewer.
**Goal:** `loopd` can hold a correct, robust conversation with `llama-server` through a Unix
socket: requests built from typed input, streams reassembled exactly, every kind of silence and
@@ -28,7 +28,7 @@ fake server replaying responses recorded from straylight.
use and newer builds add more, so structs that parse the server's responses must not use
`deny_unknown_fields`. Each task says which kind it is dealing with.
- Branch `m2a`. One task, one fresh OpenCode session, one commit. Run `cargo fmt --all` before the
gate. Review happens once, after task 13.
gate. Review happened once, after task 13; tasks 14 and 15 are its follow-ups.
## Tasks
@@ -47,6 +47,8 @@ fake server replaying responses recorded from straylight.
| 11 | `11-llama-gate-retry.md` | `SlotGate`, `chat_with_retry` | `loopd/tests/retry.rs` |
| 12 | `12-selftest.md` | `loopd::selftest`, `loopd selftest --config` | `loopd/tests/selftest.rs` |
| 13 | `13-verify-device.md` | `make verify-device` against straylight | `loopd/tests/device.rs` |
| 14 | `14-inferproxy-close.md` | Review follow-up: `inferproxy` closes towards the client when the upstream closes | updated `inferproxy/tests/forward.rs` |
| 15 | `15-http-streaming.md` | Review follow-up: chunked body data is returned as soon as it is available | updated `loopd/tests/http.rs` |
`files/` holds everything the tasks copy into place: tests, the fake server
(`loopd/tests/support/mod.rs`), recordings (`fixtures/http/*.http`), expected results derived from
@@ -154,3 +154,44 @@ fn an_unreachable_upstream_closes_the_client() {
let path = start("127.0.0.1:1".to_string(), Limits::default());
assert!(was_refused(&path));
}
/// An upstream that answers at once and closes, whether or not the client has finished sending.
/// This is what `llama-server` does, and what a server that dies mid-answer looks like.
fn answer_and_close_upstream() -> String {
let listener = TcpListener::bind("127.0.0.1:0").unwrap();
let addr = listener.local_addr().unwrap().to_string();
thread::spawn(move || {
for stream in listener.incoming() {
let mut stream = stream.unwrap();
let mut first = [0u8; 1];
let _ = stream.read(&mut first);
stream.write_all(b"answer").unwrap();
// dropping `stream` closes it
}
});
addr
}
#[test]
fn the_upstreams_close_reaches_a_client_that_is_still_sending() {
// The HTTP client in loopd never half-closes: it keeps its sending side open until it drops
// the connection. When the server closes, the proxy must close towards the client at once,
// or the client only learns of a dead server from its own timeout.
let path = start(answer_and_close_upstream(), Limits::default());
let mut s = UnixStream::connect(&path).unwrap();
s.set_read_timeout(Some(Duration::from_secs(2))).unwrap();
s.write_all(b"request").unwrap();
let started = std::time::Instant::now();
let mut got = Vec::new();
let result = s.read_to_end(&mut got);
assert!(
result.is_ok(),
"the read must end with EOF, not a timeout: {result:?}"
);
assert_eq!(got, b"answer");
assert!(
started.elapsed() < Duration::from_millis(1000),
"EOF took {:?}",
started.elapsed()
);
}
@@ -308,3 +308,46 @@ fn a_missing_socket_is_a_connect_error() {
let e: Box<dyn std::error::Error> = Box::new(HttpError::Timeout);
assert!(!e.to_string().is_empty());
}
/// A streamed body is read while it arrives. A `read` that already has data must not block
/// waiting for more, or every event is delivered one buffer late and the last ones only when
/// the stream ends.
#[test]
fn streamed_data_is_delivered_as_it_arrives() {
let server = FakeServer::start();
let reply = Reply::fixture("turn1");
let piece = reply.offset_after_events(1);
// About one event per piece, 300 ms apart: the whole stream takes over a second.
server.route("/v1/chat/completions", vec![reply.trickle(piece, 300)]);
let mut conn = Connection::open(&server.socket).unwrap();
conn.set_read_timeout(Duration::from_secs(5)).unwrap();
conn.send(&Request {
method: "POST",
path: "/v1/chat/completions",
body: Some(b"{}"),
})
.unwrap();
let head = conn.read_head().unwrap();
let started = std::time::Instant::now();
let mut body = conn.body(&head).unwrap();
let mut arrivals = Vec::new();
let mut buf = [0u8; 8192];
loop {
let n = std::io::Read::read(&mut body, &mut buf).unwrap();
if n == 0 {
break;
}
arrivals.push(started.elapsed());
}
let first = arrivals.first().copied().unwrap_or_default();
let last = arrivals.last().copied().unwrap_or_default();
assert!(
first < Duration::from_millis(200),
"the first data came only after {first:?}"
);
assert!(
last - first >= Duration::from_millis(600),
"everything arrived within {:?} of the first read: the reader is buffering the stream",
last - first
);
}