boxmaker/docs/plans/M3a/checks-e.md

M3a checks, area E: the broker (tasks 10 to 15) and end to end (task 22)

What each task's given tests were checked against before hand-over, and what doing so exposed. Branch m3a-ref-e (worktree ~/src/boxmaker-ref-e), built on the merge of areas A to D (f9195ec). Every task here had a full reference implementation, as the plan decided.

What ran

Task	Check	Tests	First run against the reference
10 runner	reference	runner 8, doctests 2 (9 in brokerd with task 07's 7)	all passed
11 approvals	reference	approvals 7	all passed
12 ledger	reference	ledger 11, ledger_answer 9	all passed
13 broker	reference	broker 9, broker_pending 5, broker_sequence 2	all passed after two test-side compile fixes
14 admin	reference	admin 12	all passed
15 serve	reference, through the binary	serve 9	all passed
22 end to end	reference (the real brokerd binary)	end_to_end 1	passed
17 BrokerPort::call (area C's skeleton)	a reference body, as a measurement; not handed over	broker_port 15, broker_port_bad 2	all 17 passed first time

Nothing only compiled. With the BrokerPort body written, make gate passes in full on this branch, the end-to-end line included.

Runs. Every area E suite, and broker_port, broker_port_bad and end_to_end, ten times in a row idle and ten times under sixteen busy loops on eight cores: 20 of 20 green, no flakes. The timing bounds in area C's broker_port (250 to 1,100 ms for a 300 ms timeout; 550 to 1,700 ms for a pending call nobody answers) held under that load.

Task order. For each of tasks 10 to 15, the reference tree was cut back to what the implementer will have at that task (later modules empty, main.rs the stub) and that task's tests and all earlier ones were compiled. Each compiled. This found one plan defect (below).

Mutation check. Fourteen deliberate defects, one at a time; each broke at least one given test: no stop after a failed append; state not raised by the result's label; no last look before running; a gone requester not taking its own entry; the re-decision ignoring the current state; shell mounts read-only; expiry at > instead of >=; a found socket directory left at 0755; RunSpec fields public (the compile_fail doctest); grant problems printed on every call; state checked before grants; a refusal recorded without the owner's reason; result content passed on when its record fails; the pending frame sent with final: true.

Spec defects and gaps (not applied to the spec; proposed wording)

1. Section 5, write order: denials before decide are recorded too. Section 3 says "the first two need no Decision-like guard: anyone may deny", which reads as if grants_invalid and state_unreadable skip the record. The reference records every tool request, those included. Proposed: "Every tool request gets a Decision record, including one denied with grants_invalid or state_unreadable before decide runs. A message of a forbidden kind is not a tool request and gets none."
2. Section 5, records: the state of a session whose state cannot be read. Decision, Approval and a failed Result carry taint/untrusted (or taint_after), and the spec does not say what they hold when the file is damaged. Chosen: secret and true. Proposed: "When the session's state cannot be read, records carry taint: secret and untrusted: true: brokerd does not know how sensitive the session is."
3. Section 5, step 3: a result whose state cannot be read or written leaves no Result. The answer is failed, "the result could not be recorded", and nothing more is written, so bxctl audit verify lists the call as running or unfinished. Proposed, appended to step 3: "Nothing is written for it, so the log shows the call as unfinished; the state entry of the runbook explains that."
4. Section 6, approve: the re-decision's mode. approve_result.outcome and the Approval record say ask or allowed, but redecide returns a Decision, which does not carry the mode. The ledger looks the matched grant up in the set it passed. It works; a cleaner fix for M3b is for redecide to return the mode with the decision. Proposed: "The outcome is ask if the grant matched now is an ask grant, allowed if it is auto."
5. Section 6, refuse: a refusal whose record cannot be written. The table says ok {}. The call is denied either way, but bxctl would report success for something that was not recorded. Chosen: error internal, detail "the refusal could not be recorded; the call is denied; see docs/runbook.md#audit-unavailable" (bxctl refuse prints it and exits 1). Proposed: "If the Approval record of a refusal cannot be written, the answer is error internal with a pointer to audit-unavailable; the call is denied with audit_unavailable."
6. Section 6, lost connection: the pending frame itself cannot be sent. Chosen: the thread tries to take its own entry at once, exactly as when its one-second look finds loopd gone. Proposed: add "The same holds if the pending frame cannot be sent."
7. Section 6: the request frame has no read timeout. A client that connects and sends nothing holds a thread for ever. Only loopd can reach broker.sock and only the owner admin.sock, so it is not fixed in M3a. Worth a line in section 14, or a timeout in M3b.
8. Section 12, end to end: the recorded model calls read_file directly, not through call_tool (the tool_call recording). loopd forwards both the same way, so the test proves the same thing. Proposed: "The scripted model calls read_file with no grant".
9. Section 2, "Threads and locks": where the expiry lives. Expiry answers entries the same way approve does, so it is admin::expire_due, called by serve's thread; approvals stays a plain table (it comes before the ledger in task order).

Plan defects found by the checks

1. The test rig broke task order. The first support/rig.rs held both the ledger rig and the broker client, so task 12's tests would not have compiled before task 13's broker module existed. Found by the task-order simulation above; the client moved to support/client.rs, which only task 13's and 14's tests include.
2. files/Makefile could not serve both task 21 and task 22. With the end-to-end line in it, task 21's gate would fail on a test target that does not exist yet. Task 21 now copies files/Makefile-task21 (area D's version, byte for byte); files/Makefile is task 22's. The parent should confirm this, since task 21 is area D's.
3. The editing tool decoded a backslash-u escape again, this time in a test comment (admin.rs). The escaped string itself is built from pieces and checked at run time (escaped.contains("u002f")); the comment was rewritten without an escape.
4. Verdict::Run holds a Box<Decision>: clippy's large_enum_variant rejects it unboxed. The handoff's design had Run(Decision).

Area C's BrokerPort, measured

About 130 lines, written from task 17's exit list. All 17 compile-only tests passed on the first run and on all 20 later runs, idle and loaded. The body found nothing the skeleton and the desk-checked exit list had missed. The end-to-end test now exercises the same port against the real brokerd.

The straylight check

tools/check-m3a-device.sh (copied by task 22, never run by the implementer). It reads GET /slots?model=ornith-1.5-35b-a3b first and stops unless slot 0 reports "is_processing": false; loopd uses slot 0 for the turn. It starts a private inferproxy, brokerd serve and loopd serve on a temporary home with one ask grant for read_file, runs bxctl chat --say, waits for the approval in bxctl approvals, checks that the block names the grant and the call, approves it, and checks the audit log verifies with decision, approval and result records. It has not run: this area had no access to straylight. Its jq test was checked on sample /slots answers only (a first version used jq -e, which exits 1 on false, the idle answer, and would have refused every run).

docs/egress.md lists no development call to straylight, although verify-device and this script both make one. Proposed row: "Development | make verify-device, tools/check-m3a-device.sh | straylight's llama-server | Checks against the real model, through a private inferproxy; the script also reads /slots with curl."

Runbook

No new anchor. The audit-unavailable entry gained the new messages that point at it (the "earlier write failed" line on every later call, "the result could not be recorded", the refusal error, and the panicked-ledger variant). Carry that edit to master. check-runbook.sh passes on this branch.

For the process record

Found by	Defects
Writing the reference	7 spec gaps (1 to 7 above)
Writing the tests and fixtures	1 spec inaccuracy (8), 1 tool defect (the escape)
Simulating the implementer's tree per task	2 plan defects (the rig, the Makefile)
Running the tests against the reference	0 test defects beyond compile errors
The BrokerPort body	0
Mutation check	14 of 14 mutants caught