M4a review: accepted; checked end to end against the owner's Mattermost
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
@@ -21,7 +21,7 @@ How it is used:
|
||||
| I3 | Report every problem found, not only the first. | M1 finding 8: `check-lines.sh` exits at the first long file. | yes | M2a: not exercised. |
|
||||
| I4 | Run the formatter before the gate, and let it decide ordering. | M1 finding 7: three first-gate failures were rustfmt reordering `lib.rs`. | yes | M2a: no fmt failures; the first-gate failures were clippy. |
|
||||
| I5 | Log every attempt, including one you abandon. | M1 review: the task 01 row mentions files "from a prior attempt" that has no row of its own. | yes | M2a: partly. Task 11's `stopped` row was overwritten by its later `done` row. M2b: no recurrence; all ten rows correct, one with a stray pipe. |
|
||||
| I6 | Prefer a conversion that can fail (`u32::try_from`) to a cast that is only safe because of a check somewhere else (`as u32`). | M1 finding 6: bounded `as` casts in `frame.rs`. Correct today, but the safety depends on a line ten lines away. | no (already implied by the code rules) | M2a: no `as` casts at all. |
|
||||
| I6 | Prefer a conversion that can fail (`u32::try_from`) to a cast that is only safe because of a check somewhere else (`as u32`). | M1 finding 6: bounded `as` casts in `frame.rs`. Correct today, but the safety depends on a line ten lines away. | no (already implied by the code rules) | M2a: no `as` casts at all. M4a: yes, bounded casts in `http.rs` and `handshake.rs`, where the task named no conversion; none where a skeleton comment gave the `try_from`. |
|
||||
| I7 | A `read` returns as soon as it has any data to give. It blocks only when it has none. Never keep reading to fill the caller's buffer. | M2a finding 2: the chunked reader delivered a whole stream at its end. | yes | M2b: no recurrence. |
|
||||
| I8 | When forwarding between two connections, pass a close on in both directions. Do not rely on the client half-closing first. | M2a finding 1. | no (specific to inferproxy) | |
|
||||
| I9 | Never end a turn by describing what you are about to do. Do it, then report. | Two Ornith turns in M2a ended with a "## Objective" plan and no tool call. | yes | M2b: not seen; all ten tasks ran unattended to a commit. |
|
||||
@@ -104,3 +104,9 @@ gate 4 of 10, and four low findings in review, two of which are the "everywhere"
|
||||
a new shape (paths instead of types). Its code reads well and follows the numbered steps in the
|
||||
tasks closely; where the task did not walk a path, the rule was not applied there. Task 11 fixed all four in one run and reported its one
|
||||
deviation, which the task itself had caused.
|
||||
|
||||
M4a was Ornith alone, fifteen tasks through the driver: first gate 12 of 15, four review findings, none
|
||||
above low. Given a skeleton whose comments name the calls and expressions, it writes exactly that,
|
||||
and its code agreed with the reference on every fuzzed input. Given a `todo!()` with many branches,
|
||||
or a task that needs the whole crate's API, it plans in prose until the turn runs out; given a
|
||||
comment that contradicts its function, it notices but deliberates rather than stopping (T29).
|
||||
|
||||
@@ -519,3 +519,53 @@ address of its own), and M3a findings 17, 18 and 20. Owner foot-guns noted by th
|
||||
and not fixed: a grant path that overlaps a container path (`/bin`, `/tmp`, `/run/egress`), and a
|
||||
`home` containing `:`.
|
||||
|
||||
|
||||
### M4a, tasks 01 to 15 — reviewed 2026-09-24 by the design model (Claude)
|
||||
|
||||
Accepted. All work by Ornith-1.5-35B-A3B through `tools/run-plan.sh`. `gatewayd` does what the
|
||||
spec says, and against the owner's Mattermost, with the owner sending the messages, every claim of
|
||||
the milestone held.
|
||||
|
||||
| Check | Result |
|
||||
|---|---|
|
||||
| 15 task commits, each with the trailer; 3 plan commits by the design model during the run | pass |
|
||||
| All given tests, fixtures, `deny.toml` and the runbook entries identical to the plan | pass |
|
||||
| Protected files (`AGENTS.md`, `CLAUDE.md`, `docs/design.md`, specs, `Makefile`, `scripts/`) | untouched |
|
||||
| `make gate` | `gate: ok`, 762 tests, the count the plan gave |
|
||||
| Differential fuzz against the reference (scratch, not kept): 40,000 random frame streams in random pieces, 20,000 HTTP responses, 50,000 strings for `named`, 300 answers for `split_answer` | frames, encoding, names and splits identical; HTTP differs only in leniency (finding 3) |
|
||||
| Banned constructs in new library code | no `unwrap`, `expect`, `panic!` or `#[allow]`; bounded `as` casts in `http.rs` and `handshake.rs` (finding 1) |
|
||||
|
||||
**Against the owner's server** (2026-09-24; this branch's binaries on the owner's machine,
|
||||
`inferproxy`, `brokerd` without `[runner]` and `loopd` against Ornith slot 0 on straylight,
|
||||
`gatewayd` against `https://straylight.scylla-hammerhead.ts.net`):
|
||||
|
||||
| Claim | Seen |
|
||||
|---|---|
|
||||
| A direct message is answered in its thread, typing shown meanwhile | "What is 17 × 23?" answered "391" in a second; the owner saw the bot typing |
|
||||
| A thread is one session; a new top-level message is a new one | three threads, three `mm-<root>` sessions in `loopd` |
|
||||
| Messages sent during a turn go together as the next turn | three replies sent during a long answer reached `loopd` as one turn, joined by blank lines |
|
||||
| A long answer is split | a 4,000-word essay came back as two posts, in order |
|
||||
| Commands | `!approve 1` answered with the M4b notice and never reached `loopd`; `!!approve is just a word` reached it as `!approve is just a word` |
|
||||
| In a channel: only posts that name the bot, or replies in its thread naming nobody else | a plain post ignored; a post naming the bot and an unnamed reply in its thread answered; a reply naming `@agent-bot` ignored |
|
||||
| Anyone else gets nothing | a direct message from the second account: no reply, no typing; one log line with the post and user ids, no text |
|
||||
| A restart reports the cut-off turn and answers what came meanwhile | `gatewayd` stopped mid-turn; "Are you back?" sent while it was down; at restart, "interrupted: …" in the cut-off thread, then "Are you back?" answered; the cut-off answer is in `loopd`'s session log |
|
||||
| No listening port | `ss`: no listening socket; one outbound connection, to 443 |
|
||||
| The token comes from an encrypted systemd credential and is never printed | `systemd-creds --user encrypt`, then `systemd-run --user -p LoadCredentialEncrypted=…`: connected; the token is in no output of any run |
|
||||
|
||||
| # | Severity | Owner | Finding | Fix |
|
||||
|---|---|---|---|---|
|
||||
| 1 | nit | implementer (06, 07) | `as` casts on values already bounded a few lines up: five in `http.rs` (chunk sizes, `MAX_BODY`), several in `handshake.rs`'s base64. None can lose data; tip I6 prefers `try_from`. | With the next change to those files |
|
||||
| 2 | low | spec, reference and implementer (04) | A secret file is checked with `symlink_metadata` and then opened by path, so someone who can write to its directory could swap it for a symbolic link in between. The reference had the same (tip T5). Opening once with `O_NOFOLLOW` and checking the open file closes it. | A follow-up task, if the owner wants it |
|
||||
| 3 | nit | implementer (06) | The HTTP reader is more lenient than the reference: two spaces in the status line, and a bare `\n` ending a chunk line, are accepted. Every hostile case is still refused before any allocation. | None needed |
|
||||
| 4 | nit | plan (11) | The skeleton's `Pending.resume` was read only in a `todo!()` whose comment did not mention it; the implementer wrote `resume: true` directly, then removed the unused field and reported it (tip T26). | None needed |
|
||||
|
||||
What was good: after the skeletons were split, every task finished on its first session; the
|
||||
decoder, the HTTP reader and the state file follow their comments exactly and agree with the
|
||||
reference on every fuzzed input; the one deviation was reported.
|
||||
|
||||
**The run.** 12 of 15 tasks passed the gate on their first run. Three sessions ended with nothing
|
||||
written, all three from the plan: task 08's `header` was one `todo!()` with a dozen branches (tip
|
||||
T25); task 14 made the model read the whole crate to learn its calls (T25, a call table); and task
|
||||
14's second session found `now_ms`'s comment on `post`, put there by the design model's script (tip
|
||||
T29). It saw the contradiction and would not guess, which was right, but it deliberated instead of
|
||||
stopping.
|
||||
|
||||
Reference in New Issue
Block a user