kyle/boxmaker
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Hand over the M3a plan: 22 tasks, their files, and the check record

Browse Source 
Task files, the files they copy in (byte-identical to the reference on
m3a-ref), each area's check record, and a README with the per-task
table of what each check exposed. The handoff note is done with.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
kyle
2026-09-18 23:45:43 -07:00
co-authored by Claude Opus 5
parent 69f0a0a218
commit e3f37da232
180 changed files with 17219 additions and 292 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/plans/M3a/checks-e.md
+138
View File
@@ -0,0 +1,138 @@
# M3a checks, area E: the broker (tasks 10 to 15) and end to end (task 22)
What each task's given tests were checked against before hand-over, and what doing so exposed.
Branch `m3a-ref-e` (worktree `~/src/boxmaker-ref-e`), built on the merge of areas A to D
(`f9195ec`). Every task here had a full reference implementation, as the plan decided.
## What ran
| Task | Check | Tests | First run against the reference |
|---|---|---|---|
| 10 runner | reference | `runner` 8, doctests 2 (9 in `brokerd` with task 07's 7) | all passed |
| 11 approvals | reference | `approvals` 7 | all passed |
| 12 ledger | reference | `ledger` 11, `ledger_answer` 9 | all passed |
| 13 broker | reference | `broker` 9, `broker_pending` 5, `broker_sequence` 2 | all passed after two test-side compile fixes |
| 14 admin | reference | `admin` 12 | all passed |
| 15 serve | reference, through the binary | `serve` 9 | all passed |
| 22 end to end | reference (the real `brokerd` binary) | `end_to_end` 1 | passed |
| 17 `BrokerPort::call` (area C's skeleton) | a reference body, as a measurement; **not handed over** | `broker_port` 15, `broker_port_bad` 2 | all 17 passed first time |
**Nothing only compiled.** With the `BrokerPort` body written, `make gate` passes in full on this
branch, the end-to-end line included.
**Runs.** Every area E suite, and `broker_port`, `broker_port_bad` and `end_to_end`, ten times in a
row idle and ten times under sixteen busy loops on eight cores: 20 of 20 green, no flakes. The
timing bounds in area C's `broker_port` (250 to 1,100 ms for a 300 ms timeout; 550 to 1,700 ms for
a pending call nobody answers) held under that load.
**Task order.** For each of tasks 10 to 15, the reference tree was cut back to what the
implementer will have at that task (later modules empty, `main.rs` the stub) and that task's tests
and all earlier ones were compiled. Each compiled. This found one plan defect (below).
**Mutation check.** Fourteen deliberate defects, one at a time; each broke at least one given
test: no stop after a failed append; state not raised by the result's label; no last look before
running; a gone requester not taking its own entry; the re-decision ignoring the current state;
shell mounts read-only; expiry at `>` instead of `>=`; a found socket directory left at 0755;
`RunSpec` fields public (the `compile_fail` doctest); grant problems printed on every call; state
checked before grants; a refusal recorded without the owner's reason; result content passed on
when its record fails; the pending frame sent with `final: true`.
## Spec defects and gaps (not applied to the spec; proposed wording)
1. **Section 5, write order: denials before `decide` are recorded too.** Section 3 says "the first
two need no `Decision`-like guard: anyone may deny", which reads as if `grants_invalid` and
`state_unreadable` skip the record. The reference records every tool request, those included.
Proposed: "Every tool request gets a `Decision` record, including one denied with
`grants_invalid` or `state_unreadable` before `decide` runs. A message of a forbidden kind is
not a tool request and gets none."
2. **Section 5, records: the state of a session whose state cannot be read.** `Decision`,
`Approval` and a failed `Result` carry `taint`/`untrusted` (or `taint_after`), and the spec does
not say what they hold when the file is damaged. Chosen: `secret` and `true`. Proposed: "When
the session's state cannot be read, records carry `taint: secret` and `untrusted: true`:
`brokerd` does not know how sensitive the session is."
3. **Section 5, step 3: a result whose state cannot be read or written leaves no `Result`.** The
answer is `failed`, "the result could not be recorded", and nothing more is written, so
`bxctl audit verify` lists the call as running or unfinished. Proposed, appended to step 3:
"Nothing is written for it, so the log shows the call as unfinished; the state entry of the
runbook explains that."
4. **Section 6, approve: the re-decision's mode.** `approve_result.outcome` and the `Approval`
record say `ask` or `allowed`, but `redecide` returns a `Decision`, which does not carry the
mode. The ledger looks the matched grant up in the set it passed. It works; a cleaner fix for
M3b is for `redecide` to return the mode with the decision. Proposed: "The outcome is `ask` if
the grant matched now is an `ask` grant, `allowed` if it is `auto`."
5. **Section 6, refuse: a refusal whose record cannot be written.** The table says `ok {}`. The
call is denied either way, but `bxctl` would report success for something that was not
recorded. Chosen: `error internal`, detail "the refusal could not be recorded; the call is
denied; see docs/runbook.md#audit-unavailable" (`bxctl refuse` prints it and exits 1). Proposed:
"If the `Approval` record of a refusal cannot be written, the answer is `error internal` with
a pointer to `audit-unavailable`; the call is denied with `audit_unavailable`."
6. **Section 6, lost connection: the pending frame itself cannot be sent.** Chosen: the thread
tries to take its own entry at once, exactly as when its one-second look finds `loopd` gone.
Proposed: add "The same holds if the pending frame cannot be sent."
7. **Section 6: the request frame has no read timeout.** A client that connects and sends nothing
holds a thread for ever. Only `loopd` can reach `broker.sock` and only the owner `admin.sock`,
so it is not fixed in M3a. Worth a line in section 14, or a timeout in M3b.
8. **Section 12, end to end: the recorded model calls `read_file` directly**, not through
`call_tool` (the `tool_call` recording). `loopd` forwards both the same way, so the test proves
the same thing. Proposed: "The scripted model calls `read_file` with no grant".
9. **Section 2, "Threads and locks": where the expiry lives.** Expiry answers entries the same way
`approve` does, so it is `admin::expire_due`, called by `serve`'s thread; `approvals` stays a
plain table (it comes before the ledger in task order).
## Plan defects found by the checks
1. **The test rig broke task order.** The first `support/rig.rs` held both the ledger rig and the
broker client, so task 12's tests would not have compiled before task 13's `broker` module
existed. Found by the task-order simulation above; the client moved to `support/client.rs`,
which only task 13's and 14's tests include.
2. **`files/Makefile` could not serve both task 21 and task 22.** With the end-to-end line in it,
task 21's gate would fail on a test target that does not exist yet. Task 21 now copies
`files/Makefile-task21` (area D's version, byte for byte); `files/Makefile` is task 22's. **The
parent should confirm this, since task 21 is area D's.**
3. **The editing tool decoded a backslash-u escape again**, this time in a test comment
(`admin.rs`). The escaped string itself is built from pieces and checked at run time
(`escaped.contains("u002f")`); the comment was rewritten without an escape.
4. `Verdict::Run` holds a `Box<Decision>`: clippy's `large_enum_variant` rejects it unboxed. The
handoff's design had `Run(Decision)`.
## Area C's `BrokerPort`, measured
About 130 lines, written from task 17's exit list. All 17 compile-only tests passed on the first
run and on all 20 later runs, idle and loaded. **The body found nothing the skeleton and the
desk-checked exit list had missed.** The end-to-end test now exercises the same port against the
real `brokerd`.
## The straylight check
`tools/check-m3a-device.sh` (copied by task 22, never run by the implementer). It reads
`GET /slots?model=ornith-1.5-35b-a3b` first and stops unless slot 0 reports
`"is_processing": false`; `loopd` uses slot 0 for the turn. It starts a private `inferproxy`,
`brokerd serve` and `loopd serve` on a temporary home with one `ask` grant for `read_file`, runs
`bxctl chat --say`, waits for the approval in `bxctl approvals`, checks that the block names the
grant and the call, approves it, and checks the audit log verifies with `decision`, `approval` and
`result` records. **It has not run**: this area had no access to straylight. Its `jq` test was
checked on sample `/slots` answers only (a first version used `jq -e`, which exits 1 on `false`,
the idle answer, and would have refused every run).
`docs/egress.md` lists no development call to straylight, although `verify-device` and this
script both make one. Proposed row: "Development | `make verify-device`,
`tools/check-m3a-device.sh` | straylight's `llama-server` | Checks against the real model, through
a private `inferproxy`; the script also reads `/slots` with `curl`."
## Runbook
No new anchor. The `audit-unavailable` entry gained the new messages that point at it (the
"earlier write failed" line on every later call, "the result could not be recorded", the refusal
error, and the panicked-ledger variant). Carry that edit to `master`. `check-runbook.sh` passes on
this branch.
## For the process record
| Found by | Defects |
|---|---|
| Writing the reference | 7 spec gaps (1 to 7 above) |
| Writing the tests and fixtures | 1 spec inaccuracy (8), 1 tool defect (the escape) |
| Simulating the implementer's tree per task | 2 plan defects (the rig, the Makefile) |
| Running the tests against the reference | 0 test defects beyond compile errors |
| The `BrokerPort` body | 0 |
| Mutation check | 14 of 14 mutants caught |