M4a review: accepted; checked end to end against the owner's Mattermost

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-09-24 01:21:27 -07:00
co-authored by Claude Opus 5.5
parent 09b074c121
commit 131b3677fd
2 changed files with 57 additions and 1 deletions
+7 -1
View File
@@ -21,7 +21,7 @@ How it is used:
| I3 | Report every problem found, not only the first. | M1 finding 8: `check-lines.sh` exits at the first long file. | yes | M2a: not exercised. |
| I4 | Run the formatter before the gate, and let it decide ordering. | M1 finding 7: three first-gate failures were rustfmt reordering `lib.rs`. | yes | M2a: no fmt failures; the first-gate failures were clippy. |
| I5 | Log every attempt, including one you abandon. | M1 review: the task 01 row mentions files "from a prior attempt" that has no row of its own. | yes | M2a: partly. Task 11's `stopped` row was overwritten by its later `done` row. M2b: no recurrence; all ten rows correct, one with a stray pipe. |
| I6 | Prefer a conversion that can fail (`u32::try_from`) to a cast that is only safe because of a check somewhere else (`as u32`). | M1 finding 6: bounded `as` casts in `frame.rs`. Correct today, but the safety depends on a line ten lines away. | no (already implied by the code rules) | M2a: no `as` casts at all. |
| I6 | Prefer a conversion that can fail (`u32::try_from`) to a cast that is only safe because of a check somewhere else (`as u32`). | M1 finding 6: bounded `as` casts in `frame.rs`. Correct today, but the safety depends on a line ten lines away. | no (already implied by the code rules) | M2a: no `as` casts at all. M4a: yes, bounded casts in `http.rs` and `handshake.rs`, where the task named no conversion; none where a skeleton comment gave the `try_from`. |
| I7 | A `read` returns as soon as it has any data to give. It blocks only when it has none. Never keep reading to fill the caller's buffer. | M2a finding 2: the chunked reader delivered a whole stream at its end. | yes | M2b: no recurrence. |
| I8 | When forwarding between two connections, pass a close on in both directions. Do not rely on the client half-closing first. | M2a finding 1. | no (specific to inferproxy) | |
| I9 | Never end a turn by describing what you are about to do. Do it, then report. | Two Ornith turns in M2a ended with a "## Objective" plan and no tool call. | yes | M2b: not seen; all ten tasks ran unattended to a commit. |
@@ -104,3 +104,9 @@ gate 4 of 10, and four low findings in review, two of which are the "everywhere"
a new shape (paths instead of types). Its code reads well and follows the numbered steps in the
tasks closely; where the task did not walk a path, the rule was not applied there. Task 11 fixed all four in one run and reported its one
deviation, which the task itself had caused.
M4a was Ornith alone, fifteen tasks through the driver: first gate 12 of 15, four review findings, none
above low. Given a skeleton whose comments name the calls and expressions, it writes exactly that,
and its code agreed with the reference on every fuzzed input. Given a `todo!()` with many branches,
or a task that needs the whole crate's API, it plans in prose until the turn runs out; given a
comment that contradicts its function, it notices but deliberates rather than stopping (T29).
+50
View File
@@ -519,3 +519,53 @@ address of its own), and M3a findings 17, 18 and 20. Owner foot-guns noted by th
and not fixed: a grant path that overlaps a container path (`/bin`, `/tmp`, `/run/egress`), and a
`home` containing `:`.
### M4a, tasks 01 to 15 — reviewed 2026-09-24 by the design model (Claude)
Accepted. All work by Ornith-1.5-35B-A3B through `tools/run-plan.sh`. `gatewayd` does what the
spec says, and against the owner's Mattermost, with the owner sending the messages, every claim of
the milestone held.
| Check | Result |
|---|---|
| 15 task commits, each with the trailer; 3 plan commits by the design model during the run | pass |
| All given tests, fixtures, `deny.toml` and the runbook entries identical to the plan | pass |
| Protected files (`AGENTS.md`, `CLAUDE.md`, `docs/design.md`, specs, `Makefile`, `scripts/`) | untouched |
| `make gate` | `gate: ok`, 762 tests, the count the plan gave |
| Differential fuzz against the reference (scratch, not kept): 40,000 random frame streams in random pieces, 20,000 HTTP responses, 50,000 strings for `named`, 300 answers for `split_answer` | frames, encoding, names and splits identical; HTTP differs only in leniency (finding 3) |
| Banned constructs in new library code | no `unwrap`, `expect`, `panic!` or `#[allow]`; bounded `as` casts in `http.rs` and `handshake.rs` (finding 1) |
**Against the owner's server** (2026-09-24; this branch's binaries on the owner's machine,
`inferproxy`, `brokerd` without `[runner]` and `loopd` against Ornith slot 0 on straylight,
`gatewayd` against `https://straylight.scylla-hammerhead.ts.net`):
| Claim | Seen |
|---|---|
| A direct message is answered in its thread, typing shown meanwhile | "What is 17 × 23?" answered "391" in a second; the owner saw the bot typing |
| A thread is one session; a new top-level message is a new one | three threads, three `mm-<root>` sessions in `loopd` |
| Messages sent during a turn go together as the next turn | three replies sent during a long answer reached `loopd` as one turn, joined by blank lines |
| A long answer is split | a 4,000-word essay came back as two posts, in order |
| Commands | `!approve 1` answered with the M4b notice and never reached `loopd`; `!!approve is just a word` reached it as `!approve is just a word` |
| In a channel: only posts that name the bot, or replies in its thread naming nobody else | a plain post ignored; a post naming the bot and an unnamed reply in its thread answered; a reply naming `@agent-bot` ignored |
| Anyone else gets nothing | a direct message from the second account: no reply, no typing; one log line with the post and user ids, no text |
| A restart reports the cut-off turn and answers what came meanwhile | `gatewayd` stopped mid-turn; "Are you back?" sent while it was down; at restart, "interrupted: …" in the cut-off thread, then "Are you back?" answered; the cut-off answer is in `loopd`'s session log |
| No listening port | `ss`: no listening socket; one outbound connection, to 443 |
| The token comes from an encrypted systemd credential and is never printed | `systemd-creds --user encrypt`, then `systemd-run --user -p LoadCredentialEncrypted=…`: connected; the token is in no output of any run |
| # | Severity | Owner | Finding | Fix |
|---|---|---|---|---|
| 1 | nit | implementer (06, 07) | `as` casts on values already bounded a few lines up: five in `http.rs` (chunk sizes, `MAX_BODY`), several in `handshake.rs`'s base64. None can lose data; tip I6 prefers `try_from`. | With the next change to those files |
| 2 | low | spec, reference and implementer (04) | A secret file is checked with `symlink_metadata` and then opened by path, so someone who can write to its directory could swap it for a symbolic link in between. The reference had the same (tip T5). Opening once with `O_NOFOLLOW` and checking the open file closes it. | A follow-up task, if the owner wants it |
| 3 | nit | implementer (06) | The HTTP reader is more lenient than the reference: two spaces in the status line, and a bare `\n` ending a chunk line, are accepted. Every hostile case is still refused before any allocation. | None needed |
| 4 | nit | plan (11) | The skeleton's `Pending.resume` was read only in a `todo!()` whose comment did not mention it; the implementer wrote `resume: true` directly, then removed the unused field and reported it (tip T26). | None needed |
What was good: after the skeletons were split, every task finished on its first session; the
decoder, the HTTP reader and the state file follow their comments exactly and agree with the
reference on every fuzzed input; the one deviation was reported.
**The run.** 12 of 15 tasks passed the gate on their first run. Three sessions ended with nothing
written, all three from the plan: task 08's `header` was one `todo!()` with a dozen branches (tip
T25); task 14 made the model read the whole crate to learn its calls (T25, a call table); and task
14's second session found `now_ms`'s comment on `post`, put there by the design model's script (tip
T29). It saw the contradiction and would not guess, which was right, but it deliberated instead of
stopping.