Files
boxmaker/docs/specs/2026-09-18-m3a-decision-path.md
kyleandClaude Opus 5.5 f0b39a4766 Record the second review of task 23 and the findings first left out
The review table gains findings 16 to 20, which the two review agents
reported and the first write-up dropped. The independent review of the fix
commits, and what was changed for it, is recorded; task 23's claims about
its tests are corrected. The spec and decisions record the day-long cap, the
ttl_ms bound, the socket-directory rule and the listener's retry.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-22 21:54:28 -07:00

991 lines
61 KiB
Markdown

# M3a design: the decision path
Status: draft for owner review, 2026-09-18; revised the same day after a design review (the
"Spec review of M3a" rows in `docs/decisions.md`), and again after the plan's checks (the "M3a plan
checks" rows there). M3 is split in two (`docs/decisions.md`). M3a is
everything that decides whether a tool call may run and records it: grant files and matching,
session taint, the audit log, approvals through `bxctl`, the two `brokerd` sockets, and `loopd`'s
real tool port. Tools themselves do not run in M3a: the runner is a trait whose only
implementation refuses. M3b adds the Podman runner, the four tools and the egress proxy. Where
this document and `docs/design.md` disagree, the brief wins. The threat model and data classes are
in `docs/specs/2026-09-17-pre-m1-design.md`; section 6 of this document replaces its socket table.
## 1. What M3a proves
| Claim | Checked by |
|---|---|
| No grant, an expired grant, too much taint, or arguments outside the grant each end in a denial with the right reason | Policy table tests; property test against an oracle |
| One invalid grant file denies every call | Loading tests |
| Nothing runs without a decision record on disk first | Audit-before-action tests with a failing writer |
| Any edit, deletion, reordering or gap in the audit log is found, with its file and line | Tampering suite, through `bxctl audit verify` and `brokerd`'s startup |
| `loopd` cannot approve: the tool socket refuses every admin message | Socket tests |
| An approval only lets through a call that is still `ask` when approved | Approval tests |
| A denial reaches the model as a plain tool result and the turn goes on | End-to-end test with the real `brokerd` binary; on straylight with Ornith |
| Every fail-closed state's message names a runbook entry, and every entry named exists | A test per state for the pointer; the gate script for the entry |
Out of scope: running tools, containers, the egress proxy, tool timeouts (all M3b); secrets
(moved to M4); approvals over Mattermost (M4); job-scoped grants (M5).
## 2. Components
```
loopd ── run/loop-broker/broker.sock ──┐
brokerd ── Runtime (M3a: refuses; M3b: podman)
bxctl ── run/owner-broker/admin.sock ──┘ │
├── grants/*.toml (read only)
├── <home>/broker/sessions/<id>.json
└── <home>/audit/YYYY-MM-DD.jsonl
```
### `brokerd` modules
Each is one file under 500 lines with one purpose. Only `serve` starts threads.
| Module | Purpose |
|---|---|
| `config` | `brokerd.toml` into a typed `Config`. Unknown keys are errors. |
| `grants` | Reads `grants/*.toml` into a `GrantSet`, applying the loading rules in section 3. |
| `args` | Parses each tool's arguments into typed values; checks paths and URLs for form. |
| `policy` | `decide` and `redecide`. The only place a `Decision` is built. Already exists from M1. |
| `state` | Each session's taint and untrusted flag, read and written as files. |
| `audit` | The audit writer: lock, chain, sync, rollover, startup check, recovery, accepted breaks. |
| `approvals` | The table of pending approvals and their expiry. In memory only. |
| `ledger` | The audit writer and the session state behind one lock, and the three steps that hold it (section 2, "Threads and locks"). |
| `runner` | `Runtime`, `RunSpec`, and `run(decision, runtime)`. The only place a `RunSpec` is built. |
| `broker` | Handles one `broker.sock` connection: one tool request from decision to answer. |
| `admin` | Handles one `admin.sock` connection: one admin request. |
| `serve` | Startup, the two listeners, one thread per connection. |
`brokerd serve --config <path> [--accept-break]`. `brokerd` gains `serde`, `serde_json` and
`toml`, all already vetted.
### Threads and locks
`serve` starts one thread per connection and one expiry thread. Expiry answers an entry the way
`approve` does, so it lives in `admin` (`expire_due`), called by that thread; `approvals` stays a
plain table. They share two things, each behind
its own `Mutex`, and no thread ever holds both at once:
- **The ledger**: the audit writer and every session state file. A thread holds it for each of
these steps as a whole, and never while a tool runs or an approval is awaited:
- read the session's state, decide, write the `Decision` record;
- read the state, re-decide, write the `Approval` record;
- read the state, raise it, write the state file, write the `Result` record.
Without it two results for one session could each read `private`, and the second write would
put the taint back down; and two records could take the same `seq`.
- **The pending table** (section 6). An entry is answered by whoever removes it from the table
while holding the table's lock: `approve`, `refuse`, the expiry thread, or the waiting thread
when it finds its connection closed. Everyone else finds the entry gone. This one rule settles
every race between them.
A poisoned table lock is recovered with `into_inner`, as in `loopd`. A poisoned ledger lock is
not: a thread panicked part-way through a write, so the writer's idea of the chain's head may be
wrong. Every call is then denied with `audit_unavailable` until `brokerd` is restarted, and the
startup check puts the tail right. The same holds after any failed audit write: the writer does
not write again in this process.
### Configuration
```toml
# brokerd.toml
[paths]
home = "/var/lib/boxmaker" # default: BOXMAKER_HOME, then /var/lib/boxmaker
grants = "/etc/boxmaker/grants"
[sockets]
broker = "/var/lib/boxmaker/run/loop-broker/broker.sock"
admin = "/var/lib/boxmaker/run/owner-broker/admin.sock"
[approvals]
ttl_ms = 900000 # 15 min; 1 to 86400000 (a day), else a config error
```
A socket path that is absent or empty means the default under `home`, as in `loopd`.
## 3. Grants
### Files
One grant per file, `grants/<id>.toml`. The id is the file stem: 1 to 64 characters of
`[a-z0-9-]`. Other files in the directory (not ending in `.toml`) are ignored; a `.toml` file with
a bad stem is an invalid grant. A grants directory that is missing or cannot be read makes the set
invalid; an empty directory is a valid empty set. The fields are `proto::Grant` as defined in M1.
```toml
# grants/notes-read.toml
tool = "read_file"
mode = "auto"
max_taint = "secret"
result_class = "private"
untrusted = false
expires = "2027-01-01T00:00:00.000Z"
[constraints]
paths = ["/home/kyle/notes"]
```
### Loading
`brokerd` reads every grant file at the start of every decision. There is no cache and no reload
signal: a change applies to the next call.
A grant file is invalid if any of these holds. The error names the file and, where the parser
gives one, the line and field.
1. It cannot be read, is not UTF-8, or is not valid TOML for `proto::Grant` (unknown fields
included).
2. Its stem is not a valid id.
3. `tool` is not one of `read_file`, `write_file`, `shell`, `http_fetch`.
4. `secret` is set: "secrets are not supported until M4".
5. `constraints.patterns` is not empty: "patterns are not supported".
6. A constraint is present that the tool does not take, or a required one is missing:
| Tool | `paths` | `hosts` |
|---|---|---|
| `read_file` | required | must be empty |
| `write_file` | required | must be empty |
| `shell` | optional | must be empty |
| `http_fetch` | must be empty | required |
7. A path in `paths` is not a valid absolute path (section 3, "Paths"), or is `/`: "a grant of the
whole file system is not supported".
8. A host in `hosts` is not a valid host pattern (section 3, "Hosts").
9. `mode` is `deny` and `max_taint` is not `secret`: "a deny grant must apply at every taint".
**If any file is invalid, the whole set is invalid**, and every call is denied with
`grants_invalid` until it is fixed. `brokerd` prints each problem, once per distinct set of
problems, followed by `see docs/runbook.md#grants-invalid`. The reason: a mistyped `deny` grant
that was skipped would silently turn into an allow wherever another grant matches.
Rule 9 has the same reason. A grant stops applying when the session's taint passes its
`max_taint`, so a `deny` grant with `max_taint = "private"` would stop denying as soon as the
session read a secret, and whatever `auto` grant lay behind it would take over: the model's own
actions would have removed the owner's prohibition. Rule 7 rejects `/` because M3b mounts a grant
path at the same path inside the container, where `/` would replace the tool image itself. M3b
may reserve further prefixes for the same reason.
### Matching
For a request to tool `T`, with the session's current taint `S`:
1. Candidates are the grants whose `tool` is `T`.
2. A candidate is ruled out by the first of these that holds: it has expired (`now >= expires`);
`S > max_taint`; its constraints do not cover the arguments.
3. Among those left, the most restrictive mode wins: `deny`, then `ask`, then `auto`. Within that
mode the winner is the grant with the longest matched path (section 3, "Paths"); grants with no
matched path (`http_fetch`, and `shell` without `cwd`) all tie. Remaining ties go to the lowest
grant id in byte order. The winner is the grant that is recorded and, in M3b, the one whose
paths are mounted.
4. The result's label does not come from the winner alone. `result_class` is the highest among
all the grants left after step 2, and `untrusted` is true if any of them says so.
5. If none is left, the reason is `grant_expired` if some candidate was ruled out only by expiry;
otherwise `taint_too_high` if some candidate was ruled out only by taint; otherwise `no_grant`.
"Only by expiry" means the grant would have matched had it not expired: taint and constraints both
pass. Likewise for taint.
Step 4 exists because grants overlap. With `a-home` (`paths = ["/home/kyle"]`, `result_class =
"private"`) and `b-keys` (`paths = ["/home/kyle/keys"]`, `result_class = "secret"`), both `auto`,
a read of `/home/kyle/keys/id` matches both. Whichever wins, the result is `secret`. The
longest-path rule then picks `b-keys`, so the mount is the narrower one.
Two consequences for whoever writes grants:
- An expired `deny` grant no longer denies. `expires` on a `deny` grant means "forbid this until
then".
- An `ask` grant with a lower `max_taint` than an `auto` grant over the same arguments drops out
when taint rises, and the call then runs without asking. Give the `ask` grant the higher
`max_taint`. Loading does not check this.
The outcome is one of: allowed by grant `g` (`auto`); ask, under grant `g`; denied by grant `g`
(`denied_by_grant`); denied with no grant (`no_grant`, `grant_expired`, `taint_too_high`).
### What each tool takes, and what its constraints cover
Arguments are the model's JSON object, parsed strictly (unknown fields rejected). Arguments that
do not parse or fail the form checks below are denied with `invalid_arguments` before matching.
| Tool | Arguments | Covered when |
|---|---|---|
| `read_file` | `{ "path": "<abs>" }` | `path` is inside one of the grant's `paths` |
| `write_file` | `{ "path": "<abs>", "content": "<text>" }` | `path` is inside one of the grant's `paths` and is not the grant path itself |
| `shell` | `{ "command": "<text>", "cwd": "<abs>" }`, `cwd` optional | the grant has no `paths` and `cwd` is absent, or `cwd` is inside one of the grant's `paths` |
| `http_fetch` | `{ "url": "https://…" }` | the URL's host matches one of the grant's `hosts` |
`command` and `content` are not inspected. The container is the boundary for what a command can do.
### Paths
A valid path, in a grant or an argument: starts with `/`; is at most 4,096 bytes; contains no NUL;
has no empty component (no `//`, no trailing `/` except the root itself), no `.` and no `..`
component. Paths are compared as they are, never normalised, and matching is by whole components:
| Grant path | Requested | Result |
|---|---|---|
| `/home/kyle/notes` | `/home/kyle/notes/a.md` | inside |
| `/home/kyle/notes` | `/home/kyle/notes` | inside (the directory itself) |
| `/home/kyle/notes` | `/home/kyle/notes2/a.md` | not inside |
| `/home/kyle/notes` | `/home/kyle/notes/../.ssh/id` | `invalid_arguments` (`..`) |
| `/home/kyle/notes` | `notes/a.md` | `invalid_arguments` (relative) |
| `/home/kyle/notes` | `/home/kyle//notes/./a.md` | `invalid_arguments` (`//`, `.`) |
| `/` | anything | the grant file is invalid (loading rule 7) |
`brokerd` does not resolve symlinks. In M3b only the matched grant directory is mounted, at the
same path, so a symlink pointing outside it points at nothing inside the container.
When several of a grant's paths contain the argument, the longest one is the matched path. For
`write_file`, a grant path equal to the argument does not count: with `paths = ["/s", "/s/out"]`,
a write to `/s/out` is covered through `/s`, and `/s` is the matched path.
### Hosts
A host name: 1 to 253 bytes, lowercase, at least two labels separated by `.`, each label 1 to 63
bytes of `[a-z0-9-]` not starting or ending with `-`, and the last label starts with a letter,
which excludes every spelling of an IPv4 address (`127.0.0.1`, `127.1`, `10.0.0.0x1`). A host
pattern in a grant is a host name, or
`*.` followed by a host name.
| Grant host | URL host | Result |
|---|---|---|
| `example.com` | `example.com` | matches |
| `example.com` | `www.example.com` | no match |
| `*.example.com` | `www.example.com`, `a.b.example.com` | matches |
| `*.example.com` | `example.com` | no match (list both to allow both) |
A valid `url`: at most 2,048 bytes; starts with `https://`; the host is a host name as above (so no
IP literals, no uppercase, no `[`); no userinfo (`@` before the path); no port, or port `443`;
then optionally `/` and a path, query or fragment of printable ASCII other than space. Anything
else is `invalid_arguments`.
### The policy functions
`policy` does no I/O and reads no clock: everything it needs is an argument, which is what lets
the property test drive it.
```rust
pub struct SessionState { pub taint: DataClass, pub untrusted: bool }
pub struct Denial {
pub reason: DenyReason,
pub grant: Option<String>, // the `deny` grant, for `denied_by_grant` only
pub grant_sha256: Option<Hash32>,
}
pub enum Outcome { Allowed(Decision), Ask(Ask), Denied(Denial) }
pub fn decide(request: ToolRequest, grants: &GrantSet, state: SessionState, now: Timestamp)
-> Outcome;
pub fn redecide(ask: Ask, grants: &GrantSet, state: SessionState, now: Timestamp)
-> Result<Decision, Denial>;
```
- `Ask` is built like `Decision`: private fields, no `Clone`, no `Deserialize`, a `compile_fail`
doctest. It holds the request, the parsed arguments and the grant that asked. **`decide` never
returns a `Decision` for an `ask` grant**, and `redecide` is the only thing that turns an `Ask`
into a `Decision`, so "this call was approved" is a fact about the types, not about the order
of statements in `broker`.
- `redecide` runs the same matching again. `ask` and `auto` both give a `Decision`; anything else
gives the `Denial`, whose grant and file hash the `Approval` record needs.
- `broker` checks in this order, and the first that applies is the answer: the grant set is
invalid (`grants_invalid`); the session's state cannot be read (`state_unreadable`); then
`decide`, inside which: the tool is not one of the four (`no_grant`, arguments not parsed); the
arguments are not valid (`invalid_arguments`); matching. The first two need no `Decision`-like
guard: anyone may deny. They are still recorded (section 5, "Write order").
## 4. Session state
`<home>/broker/sessions/<id>.json`, written only by `brokerd`:
```json
{"taint":"private","untrusted":false}
```
- A session with no file is at `private`, trusted. The file is created at its first result.
- After a tool result, taint becomes the higher of its current value and the result's
`result_class`, and an `untrusted` result sets the flag (both as combined in section 3, step 4).
Neither ever goes down. The read, the comparison and the write happen under the ledger lock
(section 2). A `failed` call changes nothing: its message is `brokerd`'s own text (section 7).
- Writes are atomic: write `<id>.json.tmp`, `fsync`, rename over `<id>.json`, `fsync` the directory.
- A file that exists but cannot be read or parsed, or that says `public` (a session is never below
`private`, so `brokerd` did not write it), is an error: every call for that session is
denied with `state_unreadable` and `brokerd` prints `see docs/runbook.md#broker-state-damaged`.
- The brief's State list names this file (P14, applied 2026-09-18).
- `loopd` has no access to `<home>/broker/`. The `class` and `untrusted` values it logs are a copy
for its own use and never an input to policy.
## 5. The audit log
### Records
Replaces the M1 `AuditRecord` (nothing has written one yet). Every record has a header and an
event:
```rust
pub struct AuditRecord { pub seq: u64, pub time: Timestamp, pub prev: Hash32, pub event: AuditEvent }
#[serde(tag = "type", rename_all = "snake_case")]
pub enum AuditEvent {
Decision {
session: SessionId, call: CallId, tool: String,
arguments: String, // in full, as received
outcome: DecisionRecord,
grant: Option<String>, grant_sha256: Option<Hash32>, // the matched grant and its file's hash
taint: DataClass, untrusted: bool, // the session's state when decided
},
Approval {
session: SessionId, call: CallId, decision: u64, // seq of the Decision record
answer: ApprovalAnswer, // approved | refused | expired
by: Option<String>, // "bxctl"; from M4 a Mattermost user id
post: Option<String>, // from M4, the Mattermost post id
reason: Option<String>, // the owner's text, for refusals
outcome: DecisionRecord, // the re-decision (section 6); denied if refused or expired
grant: Option<String>, grant_sha256: Option<Hash32>, // as matched by the re-decision;
taint: DataClass, untrusted: bool, // the session's state at the re-decision
}, // for refused and expired: no grant, the state as it is
Result {
session: SessionId, call: CallId, decision: u64,
status: ResultStatus, // result | failed
class: DataClass, untrusted: bool, truncated: bool,
bytes: u64, sha256: Hash32, // of the content (or of the failure message)
taint_after: DataClass,
},
Recovery { torn_bytes: u64, torn_sha256: Hash32 },
AcceptedBreak { file: String, line: u64, last_good: Hash32 },
}
#[serde(tag = "outcome", rename_all = "snake_case")]
pub enum DecisionRecord { Allowed {}, Ask {}, Denied { reason: DenyReason } }
```
`Allowed {}` and `Ask {}` are empty struct variants, not unit variants: serde does not apply
`deny_unknown_fields` to a unit variant of an internally tagged enum, so with `Allowed` the text
`{"outcome":"allowed","x":1}` would decode. The JSON is the same either way.
`DecisionRecord` keeps the name the brief uses for the plain record of a decision; its M1 shape
(with `grant` inside and an `Approved` variant) is replaced, since approval is now its own event.
`event` is a nested object: `{"seq":…,"time":…,"prev":…,"event":{"type":"decision",…}}`. It is not
flattened: `#[serde(flatten)]` does not work with `deny_unknown_fields`.
Results are recorded by hash and size, never content, so reading a secret does not by itself copy
it into the audit log. Arguments are recorded in full, exactly as received; they are at most one
frame (1 MiB). A session that has read a secret can put it in a later call's arguments (a
`write_file` content, a `shell` command), so the log is not free of secrets: a record is as
sensitive as the `taint` it carries, and the files are mode 0600 for that reason. For a `failed`
result, `class` and `untrusted` are the label the result would have had, and `taint_after` equals
the taint before.
### The chain
- A record is one line of JSON written by serde, then `\n`. The line and its `\n` go out in one
`write_all`, followed by the sync; whatever the record allows happens only after the sync
returns. A line without its newline was therefore never acted on.
- `hash(line)` is `proto::sha256` of the line's bytes without the `\n`.
- `prev` is the hash of the previous line; the first record ever written has `prev` all zeros and
`seq` 0.
- `seq` goes up by one per record with no gaps, across files.
- The file is `audit/YYYY-MM-DD.jsonl` by the UTC date of the record's `time`. The first record of
a new day starts a new file and chains from the last line of the previous file. Files are created
with mode 0600; the directory is `fsync`ed after a file is created.
- The writer never goes back. If a record's date is earlier than the latest file's (the clock was
stepped back over midnight), the record goes in the latest file. Files are verified in name
order, so a record appended to an older file would break the chain. The verifier reports the
backwards time as a clock warning.
- `Recovery` and `AcceptedBreak` records always go in the latest file, whatever their date: they
belong next to the lines they describe.
- `brokerd` is the only writer. It holds an exclusive lock on `audit/.lock` for its whole life
and `fsync`s after every record. The lock is `std::fs::File::try_lock`, stable since Rust 1.89
(checked in the std docs of 1.98.1; the workspace minimum is 1.95), so it needs neither `libc`
nor `unsafe`. The task must check its error type on docs.rs before use.
### Write order
For every tool request:
1. Decide (section 3). Write the `Decision` record and sync it. **If this write fails, nothing
runs**: the answer is `denied` with `audit_unavailable`, and `brokerd` prints
`see docs/runbook.md#audit-unavailable`. A failed write may have left part of a line, so the
writer does not write again: every later call gets the same denial until `brokerd` is
restarted, and the startup check recovers the tail.
2. If `ask`: wait (section 6). Write the `Approval` record. If it fails, the call does not run and
the answer is `denied` with `audit_unavailable`.
3. If allowed: run. Update the session state file. Write the `Result` record. Then answer `loopd`.
If the state write or the `Result` write fails, the answer is `failed` with the message "the
result could not be recorded", and the content is not sent: content that raised the taint must
never reach the model unless the raised taint is on disk.
Denials are answered after step 1 (or 2); nothing else is written for them. Every tool request
gets a `Decision` record, including one denied with `grants_invalid` or `state_unreadable` before
`decide` runs. A message of a forbidden kind is not a tool request and gets none.
When the session's state cannot be read, records carry `taint: secret` and `untrusted: true`:
`brokerd` does not know how sensitive the session is.
In step 3, a failed state write (or a state that cannot be read) leaves no `Result` record, so the
log shows the call as unfinished; the `broker-state-damaged` runbook entry says so.
### Verification
`proto::audit::ChainVerifier` is a pure state machine, shared by `brokerd` and `bxctl`:
```rust
pub struct ChainVerifier { /* … */ }
impl ChainVerifier {
pub fn new() -> Self; // expects seq 0, prev zero
pub fn resume(next_seq: u64, prev: Hash32) -> Self; // continue from a known point
pub fn file(&mut self, name: &str); // starts the next file
pub fn line(&mut self, bytes: &[u8], has_newline: bool); // false only for a file's last line
pub fn feed(&mut self, name: &str, content: &[u8]); // file, then each of its lines
pub fn finish(self) -> ChainReport;
}
pub struct ChainReport {
pub records: u64, pub head: Option<Hash32>, pub next_seq: u64,
pub failure: Option<ChainFailure>, // the first one since the last accepted break
pub recoveries: Vec<Location>, pub accepted_breaks: Vec<Location>,
pub abandoned: Vec<u64>, // seq of Ask decisions with no Approval after them
pub unfinished: Vec<u64>, // seq of decisions allowed, at once or by approval,
// with no Result after them
pub clock_warnings: Vec<Location>, // time went backwards
pub torn_tail: Option<TornTail>, // the last line of the last file needs recovery
}
pub struct ChainFailure { // file, line (1-based), what; and what an
pub file: String, pub line: u64, pub what: String, // AcceptedBreak appended now must carry
pub last_good: Hash32, pub break_prev: Hash32, pub break_seq: u64,
pub tail_torn: bool, // the last line fed has no newline
}
pub struct TornTail { // and what its Recovery record must carry
pub at: Location, pub has_newline: bool, pub bytes: u64, pub sha256: Hash32,
pub recovery_prev: Hash32, pub recovery_seq: u64,
}
```
The report carries what the writer must put in a `Recovery` or `AcceptedBreak` record, so that no
consumer computes `seq` or `prev` a second time. Both lists name the *decision's* `seq`, for an
approved call too: `Approval.decision` and `Result.decision` both refer to it, and it is the
approval id the owner saw.
A line fails if it does not parse as an `AuditRecord`, its `seq` is not the next one, or its
`prev` is not the hash of the line before. Files are fed in name order; a file whose first record
does not chain from the previous file's last line fails at its line 1. Two cases are not failures:
- **Torn tail.** The last line of the last file has no newline, or has one but does not parse (a
crash between ending a torn line and writing its `Recovery`). It is reported in `torn_tail` and
is not checked further. Treating an unparseable last line this way gives up nothing: whoever
could damage the last line could as easily remove its newline.
- **Recovered line.** A line immediately followed by a `Recovery` record whose `torn_bytes` and
`torn_sha256` describe exactly that line, whose `prev` is the hash of the line before it, and
whose `seq` follows that line's. "Immediately followed" spans files, though the writer never
produces that. **It makes no difference whether the recovered line parses**: a crash can cut a
record exactly before its newline, leaving complete JSON. A recovered line is not a record and
nothing may refer to its `seq`, which the `Recovery` record reuses. Reported in `recoveries`.
A `Recovery` record that describes no line (its hash and length match nothing before it) is a
failure at the `Recovery` record.
The verifier therefore holds each line back until it has seen the next one, and judges it then (or
at `finish`).
An `AcceptedBreak` record is accepted only if all of these hold:
- its `file` and `line` name the first failure found since the last accepted break (or the start);
- its `last_good` is the hash of the last line that verified before that failure (all zeros if
none did);
- its `prev` is the hash of the line immediately before the break record;
- its `seq` is the `seq` the failing line should have had, plus the number of lines from the
failure line up to the line before the break record (so a failure at line 7 with the break
record at line 10 adds 3). The count is of lines, not of what they say: a damaged
region must not choose the counter's value. After a deletion a later `seq` can therefore repeat
one inside the region; those lines are not vouched for, and a reference to a `seq` means the
latest record before it that has it.
The verifier then clears that failure, reports the break, and continues with the break record as
the new head. Lines between the failure and the break record are read only to look for the break
record and are not checked, so one break covers every failure before it.
A verifier started with `resume` has not seen the earlier files. When it meets an `AcceptedBreak`
whose `file` sorts before the first file it was given, it checks only `prev`, accepts it and
continues from it; the full verification judges the rest. This holds inside a failed region too:
if the latest file has damage before such a break record, the resumed verifier is already in a
region when it meets the break, and the break still clears it. Otherwise a correctly accepted
break would stop every later start while `--accept-break` said "nothing to accept".
### Startup
1. Take the lock. If it is held: print "brokerd is already running"
and `see docs/runbook.md#brokerd-already-running`, exit 1.
2. Without `--accept-break`: verify the latest file, resumed from the last line of the file before
it (if any). If that line does not parse as a record, there is no point to resume from, and the
whole log is verified instead. (Treating it as the failure would deadlock: after a break at
that very line is accepted, every later start would fail on it again.) Otherwise the earlier
files are not re-read; `bxctl audit verify` does that. Damage in an older file is therefore not
seen by an ordinary start, by design. With `--accept-break`: verify every file, exactly as
`bxctl audit verify` does, because the break to accept must be the first failure of the whole
log, and it may lie in an older file that the short check never reads.
3. If the report has a failure: without `--accept-break`, write nothing, print the failure's file,
line and what, and `see docs/runbook.md#audit-chain-broken`, and exit 1. With `--accept-break`,
end a torn final line with `\n` if there is one, then append an `AcceptedBreak` record with the
four values the verifier will check (above; lines are counted across files). Print what was
accepted.
4. Otherwise, if the report has a torn tail: write `\n` after the torn bytes if it is missing,
then a `Recovery` record chained from the line before the torn one. Print "audit: recovered a
torn final line" and `see docs/runbook.md#audit-recovered`.
5. `--accept-break` with no failure is an error: "nothing to accept". Exit 2. It writes nothing,
not even the recovery of a torn tail; the next start without the flag does that.
An empty latest file (created, then a crash before its first record) is not a failure and needs
no recovery: the next record is its line 1 and chains from the file before.
One crash is not recovered automatically: a crash while the `Recovery` record itself is being
written leaves a line that no `Recovery` describes. That is a chain failure, and the runbook entry
says so. It takes two crashes within two consecutive one-line writes.
Nothing is truncated, rewritten or deleted, ever.
## 6. Sockets and messages
### Two sockets
| Socket | Directory | Client | Accepts |
|---|---|---|---|
| `broker.sock` | `run/loop-broker/` | `loopd` | `tool_request` |
| `admin.sock` | `run/owner-broker/` | `bxctl` | `approvals`, `approve`, `refuse`, `check_grants` |
`brokerd` creates each directory if it is missing and sets its mode to 0700 whether it made it or
found it (a failure to do so is a startup error), removes a stale socket file, binds, and sets the
socket to 0600. It refuses a socket whose directory is `/` or a symbolic link, since the mode
change would land on `/` or on the link's target (added after the M3a review). Once serving, a
listener that runs out of file descriptors or memory pauses and retries; any other `accept`
failure stops `brokerd` (`see docs/runbook.md#brokerd-listener-lost`). All of this comes after the audit lock is taken (section 5, "Startup"): the lock
is what proves the socket file is stale and not another `brokerd`'s.
Any other message kind on a socket is answered with `error` `forbidden`, and the connection is
closed. `brokerd` prints the kind and the socket, with
`see docs/runbook.md#socket-forbidden`: nothing in the harness sends a wrong kind, so it means a
bug or a component doing what it should not.
In M3 all roles run as the owner's user, so the directory permissions do not yet keep `loopd` out of
`admin.sock`; `loopd` simply has no code that sends admin messages. M7 mounts only
`run/loop-broker/` into `loopd`'s container. This gap is recorded in section 11.
### Tool requests, on `broker.sock`
One connection per request, as on `loop.sock`:
```
→ tool_request { session, call, tool, arguments }
← tool_response { status: pending_approval, approval, expires } final: false (ask only)
← tool_response { status: result | failed | denied, … } final: true
```
`approval` becomes a `u64`: the `seq` of the call's `Decision` record.
### Approvals
- An `ask` call is added to the pending table with `expires` = now + `ttl_ms`, or the grant's
`expires` if that is earlier. The entry holds the `Ask`, what `approvals` lists, and the sending
half of a `std::sync::mpsc` channel. The connection's thread sends the pending frame and waits
on the receiving half for a verdict: run this `Decision`, or deny with this reason.
- **Whoever takes the entry out of the table answers it** (section 2), and does all of the
answering: the admin thread for `approve` and `refuse`, the expiry thread for expiry. That
thread takes the ledger lock, writes the `Approval` record, sends the verdict, and only then
answers `bxctl`. The waiting thread writes nothing for an approval; it receives the verdict and
either runs the call or sends the denial.
- **Approve.** The admin thread decides again (`redecide`) with the grants and session state as
they are now. An `ask` outcome lets the call run, and so does `auto` (the owner has since
allowed it outright); the grant matched now is the one used and recorded. The outcome is `ask`
if the grant matched now is an `ask` grant, `allowed` if it is `auto`. (`redecide` does not
return the mode; the ledger looks the grant up in the set it passed. M3b may have `redecide`
return it.) Any denial, from a
`deny` grant or from no grant still matching, denies the call with that reason. The `Approval`
record carries `approved` and the re-decision. If that record cannot be written, the verdict and
the `approve_result` are both `denied` with `audit_unavailable`.
- **Refuse.** Denied with `approval_refused`. If the refusal's `Approval` record cannot be
written, the call is denied with `audit_unavailable` and the answer to `bxctl` is `error`
`internal`, "the refusal could not be recorded", with a pointer to `audit-unavailable`.
- **Expiry.** A thread checks the table every second. An expired approval is denied with
`approval_expired`.
- **Lost connection.** While it waits, the connection's thread wakes every second
(`recv_timeout`) and checks its socket: with a read timeout of 10 ms set, a `read` that returns
`Ok(0)` means `loopd` has gone; a `WouldBlock` or `TimedOut` error means it is still there; any
bytes are a protocol error and count as gone. (`UnixStream::peek` would be the natural call, but
it is nightly-only; a zero read timeout is an error.) If `loopd` has gone, the thread tries to
take its own entry out of the table. If it gets it, nothing is written, and
`bxctl audit verify` reports the decision as abandoned. If the entry is already gone, someone
is answering it: the thread waits for the verdict. The same holds if the pending frame cannot be
sent.
- **One more check before running.** On a verdict to run, the thread checks its socket once more.
If `loopd` has gone, the call does not run, and a `Result` record with status `failed` (message
"the requester went away") closes the call in the log. `loopd` can still go away between this
check and the run, and the call then runs with nobody waiting for it; that is accepted.
- **Restart.** The table is in memory only. After a restart there are no pending approvals.
### Admin messages, on `admin.sock`
| Request | Answer (final) |
|---|---|
| `approvals {}` | `approval_list { items: [PendingApproval] }` |
| `approve { approval }` | `approve_result { outcome: DecisionRecord }` — the re-decision |
| `refuse { approval, reason }` | `ok {}` |
| `check_grants {}` | `grants_report { problems: [GrantProblem] }`, empty when valid |
```rust
pub struct PendingApproval {
pub approval: u64, pub session: SessionId, pub call: CallId, pub tool: String,
pub arguments: String, pub grant: String, pub taint: DataClass,
pub created: Timestamp, pub expires: Timestamp,
}
pub struct GrantProblem { pub file: String, pub line: Option<u64>, pub problem: String }
```
An unknown or already answered `approval` is `error` `no_such_approval`.
`PendingApproval.arguments` is not the string `loopd` sent. It is the parsed arguments serialised
again by `serde_json` from the typed value in `args`, fields in the order of section 3's table,
compact, with an absent `cwd` left out (`"cwd": null` in the request means absent).
The owner approves what policy matched: in the raw string `"\u002fetc"` and `"/etc"` look
different and mean the same, and the re-serialised form shows both as `/etc`. The audit log keeps
the raw string.
## 7. The runner seam
```rust
pub trait Runtime: Send + Sync {
fn run(&self, spec: &RunSpec) -> Result<RunOutput, RunError>;
}
pub struct RunSpec { // fields private; built only in runner.rs
tool: ToolName, // ReadFile | WriteFile | Shell | HttpFetch
arguments: ToolArgs, // the typed arguments from `args`
mounts: Vec<Mount>, // { path, writable }
egress: Option<Vec<String>>, // the grant's host patterns; None = no network
}
pub struct RunOutput { pub content: String, pub truncated: bool }
pub enum RunError { Failed(String), Unavailable(String) }
pub fn run(decision: Decision, runtime: &dyn Runtime) -> ToolResponse;
```
`RunSpec` has public getters and no public constructor; a `compile_fail` doctest proves it, like
`Decision`'s. `Decision` gains the parsed arguments, the winning grant (id, file hash, the matched
path or hosts) and the result's label (`result_class` and `untrusted`, combined over every matching
grant as in section 3, step 4), all set by `policy`.
What `run` puts in the spec, from the decision:
| Tool | `mounts` | `egress` |
|---|---|---|
| `read_file` | the matched grant path, read-only | none |
| `write_file` | the matched grant path, writable | none |
| `shell` | every path of the grant, writable | none |
| `http_fetch` | none | the grant's `hosts` |
**A `RunError`'s text is never tool output.** `loopd` logs a `failed` result as `public` and
trusted, and a failure does not raise taint, so the message must be one of a fixed set of
sentences written in `brokerd` or `toolkit` ("the tool timed out", "the container could not
start"). Nothing a tool printed, read from a file or received from the network goes in it. In M3b
a command that exits non-zero is a `result`, labelled like any other, with its output and exit
status as the content.
M3a's production runtime is `Refusing`: every call is `RunError::Unavailable("the runner arrives
in M3b")`, which becomes `failed` for `loopd`. Tests use a recording fake. Limits (timeout, memory,
processes, output size) are M3b's.
## 8. `loopd` changes
- **`BrokerPort`** implements `ToolPort` over `broker.sock`. `ToolPort::call` gains a callback:
`fn call(&self, req: &ToolRequest, on_pending: &mut dyn FnMut(&Pending)) -> ToolResponse`.
`BrokerPort` waits for the first frame until `[broker] timeout_ms` (default 120,000) after the
call began, and after a pending frame until the frame's `expires` plus `timeout_ms`, which
leaves a call approved at the last moment the same time to run as any other. The wait before
that `timeout_ms` is never more than a day (`MAX_PENDING_WAIT`), whatever `expires` says, and
`brokerd` refuses a `ttl_ms` over a day, so the two agree (added after the M3a review: a far
`expires` parked a turn for ever). These are
deadlines, not per-read socket timeouts: a peer that trickles bytes must not hold a turn for
ever. (Unlike the inference path's liveness rule, this is a total limit.) M3b must keep its tool
time limit under `timeout_ms`. If the socket cannot be reached, closes early or times out, the
answer is `Failed { "the tool broker is unavailable" }`, and `loopd` prints
`see docs/runbook.md#broker-unavailable`. The turn goes on.
- The envelope `id` of a tool request is `request.call.0`, and every answer must carry it. Also
failures of the port, each with its own fixed text and no runbook pointer unless it says so: a
request too large for one frame ("the request is too large for the tool broker"); a pending
frame as the final answer ("the tool broker gave no final answer"); an `expires` so far off
that the deadline cannot be computed is a broker that is unavailable. `timeout_ms = 0` is not
refused at load; every call then fails closed as unavailable.
- **`clock` moves into `loopd`**, beside `find_tool` and `call_tool`: the time is not authority and
needs no broker. The core tools stay `clock`, `find_tool`, `call_tool`. The registry's
discoverable tools become `read_file`, `write_file`, `shell`, `http_fetch`, with the argument
schemas from section 3. `echo` stays in `FakeTools` and in the test registry `Registry::m2b()`,
which recorded conversations and tests find it through; `loopd serve` uses `Registry::m3a()`.
The `tools` array is the same for both (`clock`, `find_tool`, `call_tool`), so this changes only
what `find_tool` finds.
- **Denials become tool results** the model can explain, written by `loopd` (`class` public,
`untrusted` false):
| Reason | Content |
|---|---|
| `no_grant` | Denied: no grant allows this call. |
| `grant_expired` | Denied: the grant for this call has expired. |
| `taint_too_high` | Denied: this session has seen data too sensitive for this call. |
| `denied_by_grant` | Denied: a grant forbids this call. |
| `approval_refused` | Denied: the owner refused this call. |
| `approval_expired` | Denied: the approval request expired without an answer. |
| `invalid_arguments` | Denied: the arguments are not valid for this tool. |
| `grants_invalid` | Denied: the grant files have an error; the owner has been told. |
| `audit_unavailable` | Denied: the audit log cannot be written; the owner has been told. |
| `state_unreadable` | Denied: this session's broker state is damaged; the owner has been told. |
- **Two new turn events.** `approval_pending { approval, tool, expires }` is sent when the pending
frame arrives. It is a notice that something is waiting, not a description of it: `loopd` does
not know the grant or the taint, and what the owner is shown must not come from `loopd`
(section 9). `tool_denied { name, reason }` is sent before the `tool_result` event of a denied
call, so the owner sees the reason itself and not only the model's account of it. In both
events the tool is the one `brokerd` decides on (`request.tool`), not `call_tool`;
`tool_call_started` and `tool_result` keep the name the model called.
- **Config**: `[broker] socket` and `[broker] timeout_ms`. If `socket` is absent, `loopd serve`
uses no port and every tool call except the core ones fails with "no tool broker is
configured"; `loopd` prints that once at startup with `see docs/runbook.md#broker-unavailable`.
- **Runbook pointers** on the existing fail-closed messages: the failed self-test
(`loopd-selftest-failed`), a damaged session log (`session-log-damaged`), an unreadable
`memory/core.md` (`core-memory-unreadable`).
## 9. `bxctl` changes
- `bxctl approvals [--admin-socket <path>]` lists pending approvals:
```
41 2 min ago expires in 13 min session chat-1758… grant shell-scratch taint private
shell {"command":"rm -rf /home/kyle/scratch/build","cwd":"/home/kyle/scratch"}
```
- `bxctl approve <id>` prints `approved 41: runs` (exit status 0) or
`approved 41: denied (<reason>)` (exit status 1). `bxctl refuse <id> [--reason <text>]` prints
`refused 41`. For `no_such_approval` both print `41: no such approval (already answered or
expired)` and exit 1.
- `bxctl grants check` prints each problem as `<file>:<line>: <problem>`, or `grants: ok`; exit
status 1 if there are problems.
- `bxctl audit verify [--home <path>]` reads `<home>/audit/` itself (no daemon) and prints
`audit: ok, <n> records, head <hex>`, then any recoveries, accepted breaks, approvals "pending or
abandoned", calls "running or unfinished", clock warnings and a torn final line, one per line
(the exact lines are fixed in task 19; a torn tail is also what a `brokerd` mid-write looks
like, so it is not a failure); or the failure as `<file>:<line>: <what>` and the runbook pointer,
and exit status 1. The two double names are because `bxctl` reads the
files without asking `brokerd`: an approval still waiting and a call still running look the same
on disk as ones a crash cut off.
- `bxctl chat`: on an `approval_pending` event it sends `approvals` on `admin.sock`, finds the
entry with that id, and prints the same two-line block as `bxctl approvals`. **Everything in the
block comes from `brokerd`, nothing from the event**: a compromised `loopd` must not choose what
the owner sees. If the id is not in the list it prints `approval 41 is no longer pending` and
does not ask. In the interactive mode it then asks `type 41 to approve, anything else refuses: `
on stderr and reads one line from stdin; exactly the id sends `approve`, anything else sends
`refuse`. The id, not `y`, because lines typed while the turn ran are still waiting in stdin,
and `bxctl` cannot discard them without a terminal library: a stray line must never approve.
`--say` shows the block and does not ask; the owner answers with `bxctl approve` from another
terminal. `--json` prints the event as JSON and nothing else, and never contacts `brokerd`. The
chat lines and the approval's answer are read from one buffered reader on stdin. `bxctl chat`
gains `--admin-socket`. The exact texts of the block, the spans and the one-line failures
(`brokerd` unreachable, an approval expired before the answer, end of input at the question,
which refuses) are fixed in tasks 18 and 20.
- A `tool_denied` event prints `[denied <name>: <reason>]`, and for `grants_invalid`,
`audit_unavailable` and `state_unreadable` the runbook pointer (`grants-invalid`,
`audit-unavailable`, `broker-state-damaged`) on the next line.
- **Arguments are printed as data.** Each of these characters is printed as `\uXXXX`, four
lowercase hex digits: U+0000 to U+001F, U+007F to U+009F, U+200B to U+200F, U+2028 to U+202E,
U+2060 to U+2069, and U+FEFF. The first two ranges are the control characters, so nothing the
model wrote can move the cursor or change colours. The rest are the invisible and
direction-changing characters: with U+202E in a path, the path displayed would differ from the
path that was matched. The arguments are JSON, and inside a JSON string `\uXXXX` is the standard
spelling of the same character, so the printed text is still exactly the arguments and cannot
be forged: a real backslash in a value is already `\\` there.
- **Model text is printed the same way** from M3a: `bxctl chat` applies the same escaping to
everything the model wrote (reasoning, content, tool names, and the answer on stdout), except
that newline and tab pass through, and writes `ESC[0m` before an
approval block. Until now it wrote model text to the terminal as it came, and a colour or
conceal sequence in it would have carried into the block that follows.
- The admin socket defaults to `<BOXMAKER_HOME>/run/owner-broker/admin.sock`.
## 10. `proto` changes
| Change | Detail |
|---|---|
| `AuditRecord`, `AuditEvent`, `DecisionRecord`, `ApprovalAnswer`, `ResultStatus` | Section 5; the M1 audit fixtures are replaced |
| `audit::ChainVerifier`, `ChainReport`, `ChainFailure`, `Location` | Section 5 |
| `ToolResponse::PendingApproval.approval` | `String` becomes `u64` |
| `DenyReason` | gains `grants_invalid`, `audit_unavailable`, `invalid_arguments`, `state_unreadable` |
| `ErrorCode` | gains `forbidden`, `no_such_approval` |
| `Message` | gains `approvals`, `approval_list`, `approve`, `approve_result`, `refuse`, `ok`, `check_grants`, `grants_report` |
| `PendingApproval`, `GrantProblem` | Section 6 |
| `TurnEvent` | gains `approval_pending { approval, tool, expires }` and `tool_denied { name, reason }` |
All of them reject unknown fields. Every new message kind gets a byte-exact fixture.
## 11. The runbook
`docs/runbook.md` has one entry for every state in which the harness refuses to work or withholds
something, and for events the owner should understand when they see them. Each entry is a level-2
heading whose text is its anchor (`## grants-invalid`), with five parts: what you see, why the
system refuses, how to confirm, how to fix, and how to check the fix.
Every message for such a state ends with `see docs/runbook.md#<anchor>`. A gate script,
`scripts/check-runbook.sh`, collects every `docs/runbook.md#<anchor>` in the `*.rs` files under
`crates/` (tests included, `target/` excluded) and fails if `docs/runbook.md` lacks the heading
`## <anchor>`. An anchor must be written out in the source: a pointer whose anchor the script
cannot read (built with `format!`, or a placeholder in a comment) is an error. It also fails if it
finds no pointer at all, since that means the search is broken. It has a self-test like the other
gate scripts.
## 12. Testing
Tests and fixtures are given to the implementer. How each was checked first (a reference
implementation, the oracle, or a compiling skeleton) is in section 15.
- **Policy tables.** Every row of every table in section 3 is a case, plus: mode precedence among
three matching grants; the reason order when one candidate is expired and another has too much
taint; expiry exactly at `expires`; the longest matched path winning within a mode; ties by
grant id; the `a-home` and `b-keys` example labelled `secret` whichever id sorts first; an
`untrusted = true` grant among the matches setting the flag when the winner says false; an
expired `deny` no longer denying.
- **Loading.** One case per rule in "Loading", each with the problem text it must contain; one
invalid file among valid ones denies a call a valid file would allow.
- **Property test.** A seeded xorshift generator (no crate) makes grant sets, session taints and
requests. A separate, deliberately naive oracle in the test file says what should happen; every
case must agree. Two properties over single decisions, with restrictiveness ordered allowed,
ask, denied: adding a `deny` grant to a set never makes any outcome less restrictive; and a
call that is `denied_by_grant` at one taint is `denied_by_grant` at every higher taint. Over
sequences of calls: taint never goes down; every `Result` record follows a
`Decision` for the same call; the fake runtime sees a call only after `allowed` or an approval.
The seed is printed on failure.
- **Audit.** The writer: chain across a day boundary, `seq` across files, sync on each record, the
lock refusing a second writer. The tampering suite, each a fixture directory: a changed byte in a
middle line; a deleted line; two lines swapped; a `seq` gap; a file that does not chain from the
one before; a middle line cut short. Each must fail at the right file and line in `ChainVerifier`, and in
`brokerd`'s startup when the damaged file is the latest (an ordinary start reads only that one). Not failures: a torn tail (startup writes a `Recovery`,
and the next verify reports it); an accepted break (reported by every later verify). Failures:
an `AcceptedBreak` naming the wrong line, or with the wrong `last_good`, `prev` or `seq`; a
`Recovery` whose hash or length does not match the line before it, or that describes no line; a
line no `Recovery` describes followed by a torn `Recovery`.
- **Audit edges**, each a fixture or a scripted writer test: a torn tail that is complete JSON
lacking only its newline (recovered, and the chain verifies afterwards: this is the case a
"does not parse" rule gets wrong); an unparseable last line that has its newline (recovered
without a second newline); a torn first line of a file; an empty latest file; a clock stepped
back over midnight (the record goes in the latest file, one clock warning, the chain verifies);
a line torn on one day and recovered on the next (the `Recovery` is in the torn line's file); a
break in an older file (plain startup succeeds, `bxctl audit verify` fails, `--accept-break`
accepts it, both verify afterwards, and so does the next plain startup through `resume`); two
failures before one break record (one break covers both); a region holding a line with `seq`
18446744073709551615 (the break's `seq` is unaffected).
- **Unfinished and abandoned.** An allowed `Decision` with no `Result` is listed in `unfinished`;
an `Ask` with no `Approval` in `abandoned`; neither is a failure.
- **Audit before action.** A writer that fails on demand: the answer is `audit_unavailable` and
the fake runtime's count is 0; the next call is denied the same way though the writer would now
succeed. A state writer that fails: `failed`, and the content is absent.
- **Concurrency.** Eight threads, two sessions, fifty calls each through the `brokerd` library
with the recording runtime, half of the grants labelled `secret`: the log verifies, no `seq`
repeats, and each session's `taint_after` never goes down from one `Result` to the next.
- **Sockets.** Each admin kind on `broker.sock` and `tool_request` on `admin.sock` is `forbidden`.
Socket mode 0600; directory mode 0700, both when `brokerd` makes it and when it finds it at 0755.
- **Approvals.** Approve; refuse; expire with `ttl_ms` 100; approve after the grant file is
removed (denied, `no_grant`); approve after the taint rose past `max_taint` (denied,
`taint_too_high`); an unknown id; a dropped `loopd` connection while pending (removed, nothing
written, reported as abandoned); `approve` and `refuse` sent together from two threads, a
hundred times (exactly one `Approval` record each time, and the other caller gets
`no_such_approval`); an approval whose `Approval` record cannot be written (both sides get
`audit_unavailable`, runtime count 0); `loopd` gone at the check before running (a `failed`
`Result`, runtime count 0). The `Approval` record names the grant matched at the re-decision
when that differs from the first.
- **Pointers.** For each fail-closed state (`grants_invalid`, `audit_unavailable`,
`state_unreadable`, a broken chain, a recovered tail, a held lock, a forbidden message, the
broker unreachable, no broker configured), a test that its message ends with
`see docs/runbook.md#<anchor>`.
- **Runner seam.** The recording fake shows the mounts and egress of section 7 for each tool, and no
egress for anything but `http_fetch`. `compile_fail` for `RunSpec`.
- **`loopd`.** `BrokerPort` against a fake `brokerd`: result, denial text, pending event, broker
gone, a broker that accepts and never answers (times out after `timeout_ms`), a final frame that
arrives after `expires` but within `timeout_ms` of it (delivered). The turn loop with
`approval_pending` and `tool_denied`.
- **`bxctl`.** `approvals` output with an argument full of escape sequences and one with U+202E,
U+200B and U+2066; `\u002fetc` in the request shown as `/etc`; `approve`, `refuse`,
`grants check` and `audit verify` against fakes and fixture directories. `chat` against a fake
`loopd` and a fake `brokerd`: when the event says `read_file` and `brokerd`'s entry says `shell`,
the block says `shell`;
`y` refuses and the id approves; a line already waiting in stdin refuses; an id missing from the
list prints "no longer pending" and asks nothing; model content holding `ESC[8m` comes out
escaped.
- **End to end, in two processes.** A test in `loopd` starts the real `brokerd` binary (its path
comes from `BOXMAKER_BROKERD`; the test is `#[ignore]`d without it, and `make gate` builds the
workspace and then runs it with the variable set) on a temporary home, and runs `loopd`'s turn
loop against the fake llama server with a `BrokerPort` on that `brokerd`'s socket. The scripted
model calls `read_file` with no grant; its next request contains "Denied: no
grant allows this call."; the audit directory, read with `proto::ChainVerifier`, verifies and
holds one record, a `Decision` with `no_grant`. It cannot be one process: that would make `loopd` depend on
`brokerd`, even as a dev-dependency, and no crate may depend on another role's crate
(`scripts/check-crate-deps.sh` checks dev-dependencies too). Two processes over the real socket
is also the better test.
- **The runbook script**, with its self-test.
**On straylight** (a scripted check, not part of `verify-device`): the script first reads
`GET /slots?model=ornith-1.5-35b-a3b` and stops if slot 0 is busy, then uses slot 0 only, as
`verify-device` does. `brokerd serve` and `loopd serve`
with a test home holding one `ask` grant for `read_file`. A `bxctl chat` asks Ornith to read a file
in that directory; the approval block appears; approving it gives the M3a runner's failure, which
the model reports. The audit log verifies, with `Decision`, `Approval` and `Result` records.
## 13. What M3b needs from this
M3b implements `Runtime` with Podman and adds nothing to policy. Its tests: golden `podman`
argument lists per `RunSpec`; on device, a container with no egress reaches neither the tailnet,
the host nor the internet; `http_fetch` reaches an allowed host and is refused for another and for
a redirect to another; `write_file` cannot write outside its mount; a symlink to `~/.ssh` in a
granted directory reads nothing; memory, process and time limits hold; no container with
Boxmaker's label remains after a call. Measurements behind these choices, 2026-09-18 on straylight:
rootless Podman 5.8.6 with crun starts a `--network=none` container in 40 to 80 ms; such a
container reaches neither `100.100.100.100` nor the host; static `curl` in it fetches HTTPS through
a SOCKS5 proxy on a mounted Unix socket (`--proxy socks5h://localhost/<path>`), and a proxy that
accepts only allowlisted host names refuses other hosts and IP literals.
## 14. Threat model notes
- M3 runs every role as the owner's user. A compromised `loopd` could connect to `admin.sock` and
approve its own calls, and a container escape is the owner's user. Both close in M7 (separate
users and containers; `admin.sock` is never mounted into `loopd`). Accepted for M3, because the
main adversary is injected text, and `loopd` has no code path that sends an admin message.
- The pending-approval table is in memory. A `brokerd` crash loses pending approvals; the calls
fail and are reported as abandoned. Nothing runs without an approval record.
- Results are audited by hash. Proving what a result was needs the content from elsewhere (the
session log holds it, capped).
- Taint is kept per session id, and `loopd` chooses the id. Injected text cannot change it, but a
compromised `loopd` can read a secret under one id and send it out under a fresh one, which
starts at `private`. Taint therefore contains the model, not a compromised `loopd`; what
contains that is the grants themselves (`ask` on anything that leaves the host). Accepted for
v0. A host-wide taint floor would close it and is not proposed here.
- `brokerd` sets no read timeout on a request frame, so a client that connects and sends nothing
holds a thread until it goes. Only `loopd` can reach `broker.sock` and only the owner
`admin.sock`. Accepted for M3a; M3b should add a timeout.
- Deleting `<home>/broker/sessions/<id>.json` puts a session back to `private` and nothing
notices; the audit log, by contrast, shows tampering. Only the owner's user can do it, and the
`Result` records still hold every `taint_after`. Accepted until M7 gives `brokerd` its own user.
- The deny reasons tell the model something about grants `loopd` cannot see: `grant_expired` and
`taint_too_high` reveal that a grant covers those arguments. Each probe is a call, so each is in
the audit log. Accepted: the reasons are worth more to the owner, through the model's
explanation, than their absence would cost an attacker.
## 15. How the work is handed over
As in M2: small closed tasks with given tests, driven by `tools/run-plan.sh`, reviewed once after
the last M3a task. No task mixes policy with plumbing. Every task that adds a fail-closed path
lists its exits and its runbook anchor (tips T14 and T16).
What changes from M2 is how the given tests are checked before they are handed over (decision of
2026-09-18, tip T17):
| Check | Used for | What it proves |
|---|---|---|
| Reference implementation | `proto::audit` (the verifier), `brokerd::audit` (the writer and startup), `brokerd::approvals` | The tests pass, and the spec can be implemented as written. It also generates the tampering fixtures, whose hashes must be real. |
| Oracle | `brokerd::policy`, `args`, `grants` | The property test carries its own naive second implementation. |
| Skeleton | Everything else | The given tests compile against the task's signatures with `todo!()` bodies. |
The reference and the skeletons live in a git worktree at `~/src/boxmaker-ref` on an unpushed
branch, not in `/tmp`. The plan's README records, for each task, which check it had and what that
check exposed. In the event the table above did not hold: the approval handoff could not run
without working `config`, `args`, `grants`, `policy` and `state`, so those got a minimal
reference too, and the broker (tasks 10 to 15) a full one; skeletons were confined to `loopd` and
`bxctl`, whose bodies were later written as a measurement and found nothing more.