diff --git a/.gitignore b/.gitignore index 42e21a3..c04b756 100644 --- a/.gitignore +++ b/.gitignore @@ -1,2 +1,3 @@ /target/ /spike/out/ +/.state/ diff --git a/AGENTS.md b/AGENTS.md new file mode 100644 index 0000000..2bb0f9d --- /dev/null +++ b/AGENTS.md @@ -0,0 +1,73 @@ +# AGENTS.md + +Boxmaker is a personal agent harness in Rust: a Cargo workspace of small daemons that talk over +Unix sockets. You are implementing it one task at a time. + +## How you work + +1. The owner gives you one task file, `docs/plans/M/NN-name.md`. Read it fully. Do that task and + nothing else. Do not start the next task. +2. Do the steps in order. Where a step shows a command and its expected output, run it and compare. +3. Do not read `docs/design.md` or other plans unless the task tells you to. The task file quotes + what you need. +4. If something in the task is impossible, contradictory, or fails twice in the same way, **stop**. + Do not improvise, do not change a test, do not weaken a check. Add your row to + `docs/implementer-log.md` with status `stopped`, say what you tried and what happened, commit + only that file, and tell the owner. + +## Files you must never edit + +- `docs/design.md`, `docs/decisions.md`, `docs/inference-contract.md`, `docs/specs/`, `docs/plans/` +- Anything a task told you to copy from `docs/plans/**/files/`: tests, fixtures, `Makefile`, + `deny.toml`, `scripts/test-gate-scripts.sh`. If a copied test fails, your code is wrong. +- `AGENTS.md`, `CLAUDE.md` + +## Code rules + +- Rust stable, edition 2024, `rust-version = "1.95"`. No `unsafe`. No async runtime. +- No dependency that the task file does not name. All external dependencies are declared in the + root `Cargo.toml` under `[workspace.dependencies]`, used with `name.workspace = true`, and have a + row in `docs/dependencies.md`. +- `proto` depends on no workspace crate. Every other crate depends on `proto` only, never on + another role crate. +- No source file over 500 lines. +- Library code never panics on input: no `unwrap`, `expect`, `panic!`, indexing that can go out of + bounds, or `as` casts that can lose data on untrusted values. Tests may use them. +- Errors are plain enums that implement `std::fmt::Display` and `std::error::Error`. No `anyhow`, + no `thiserror`. +- Do not silence a lint with `#[allow(...)]` unless the task says so. Fix the code. +- Keep struct fields and enum variants in the order the task gives. The order is the wire format. +- Comments say why, not what. Match the amount of commenting you see in the task's examples. + +## The gate + +`make gate` must print `gate: ok` before a task is done. It runs offline: rustfmt, clippy with +warnings denied, all tests, cargo-deny, and the scripts in `scripts/`. After you add a dependency, +run `cargo build` once so that `Cargo.lock` is updated, then run the gate. + +Useful while working: `cargo test -p --test ` runs one test file, and +`cargo test -p ` runs tests whose name contains ``. + +## Git + +- Work on the branch the task names. One task is one commit. +- Stage only the paths the task lists: `git add ...`. Never `git add -A` or `git add .`. +- Never push, amend, rebase, reset, or switch to another branch. +- Commit message: the subject line the task gives, a blank line, then this trailer: + `Implemented-By: Laguna S 2.1 (OpenCode)` + +## The implementer log + +Before you commit, add one row to the table in `docs/implementer-log.md` and include the file in +the commit. Be honest: the log is how the owner judges the process, and a wrong row is worse than a +bad one. + +| Column | What to write | +|---|---| +| Task | The task file name, for example `M1/03-proto-wire` | +| Date | Today's date, `YYYY-MM-DD` | +| Status | `done` or `stopped` | +| Gate runs | How many times you ran `make gate` | +| First gate | `pass` or `fail` for the first run | +| Deviations | Anything you did that the task did not say, or `none` | +| Notes | Problems you hit and how you solved them, in one or two sentences | diff --git a/CLAUDE.md b/CLAUDE.md index 668c73d..c178872 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -13,7 +13,10 @@ for `AGENTS.md` and `Cargo.toml` to see whether M1 has started. which also lists proposed changes that are not yet applied. - `docs/inference-contract.md` holds the M0 measurements from straylight. Where it and the brief disagree, the measurements are newer. -- `docs/specs/` holds design specs; `docs/plans/M/` holds the task files Laguna works from. +- `docs/specs/` holds design specs; `docs/plans/M/` holds the task files Laguna works from, + and `files/` under it holds the tests and fixtures the tasks copy in. `AGENTS.md` holds the + implementer's standing rules; its code rules apply to any code written here too. +- `docs/implementer-log.md` is Laguna's own record, one row per task. Reviews go at the bottom. - `spike/` is throwaway measurement code, not harness code. Roles: implementation is done by Laguna S 2.1 (served by straylight) through OpenCode on this diff --git a/docs/implementer-log.md b/docs/implementer-log.md new file mode 100644 index 0000000..781b7a1 --- /dev/null +++ b/docs/implementer-log.md @@ -0,0 +1,9 @@ +# Implementer log + +Kept by the implementing model, one row per task. The column meanings are in `AGENTS.md`. The +reviewer adds findings under "Reviews" once per milestone. + +| Task | Date | Status | Gate runs | First gate | Deviations | Notes | +|---|---|---|---|---|---|---| + +## Reviews diff --git a/docs/milestones.md b/docs/milestones.md index cc3911e..d454204 100644 --- a/docs/milestones.md +++ b/docs/milestones.md @@ -34,7 +34,9 @@ own commit before building on it. ## Session 1 prompt (M0 spike + M1 skeleton) -As written in the kickoff pack. See `docs/decisions.md` for what has changed since. +As written in the kickoff pack, kept for the record. It is superseded: M0 is done +(`docs/inference-contract.md`), and M1 is planned in `docs/plans/M1/` from +`docs/specs/2026-09-17-pre-m1-design.md`. ```text ```text diff --git a/docs/plans/M1/01-workspace-and-gate.md b/docs/plans/M1/01-workspace-and-gate.md new file mode 100644 index 0000000..7b42b1d --- /dev/null +++ b/docs/plans/M1/01-workspace-and-gate.md @@ -0,0 +1,185 @@ +# M1 task 01: workspace and gate + +**Branch:** `m1` (create it: `git switch -c m1`) +**Commit subject:** `Add Cargo workspace, crate skeletons and the gate` + +## Goal + +Create the Cargo workspace with seven empty crates, and the gate that every later task must pass. +At the end `make gate` prints `gate: ok`. + +## Context + +From the design brief: "Rust stable, Cargo workspace. Crates: `proto` (shared types), `loopd`, +`brokerd`, `gatewayd`, `inferproxy`, `toolkit` (tool container entrypoints), `bxctl` (owner CLI). +No source file over 500 lines. No crate depends on another role's crate, only on `proto`. +Dependencies are few and justified in `docs/dependencies.md`. No outbound call not listed in +`docs/egress.md`." + +Every crate except `proto` is a library with a thin `main.rs`, so that doctests can run later. + +## Files + +- Copy (never edit afterwards): `Makefile`, `deny.toml`, `scripts/test-gate-scripts.sh` +- Create: `Cargo.toml`, `Cargo.lock` (generated), `crates//Cargo.toml` for all seven crates, + `crates//src/lib.rs` for all seven, `crates//src/main.rs` for the six that are not + `proto` +- Create: `scripts/check-lines.sh`, `scripts/check-crate-deps.sh`, `scripts/check-dep-docs.sh` +- Create: `docs/dependencies.md`, `docs/egress.md` +- Modify: `docs/implementer-log.md` + +## Steps + +- [ ] **1. Branch and copy the given files.** + +```sh +git switch -c m1 +mkdir -p scripts +cp docs/plans/M1/files/Makefile docs/plans/M1/files/deny.toml . +cp docs/plans/M1/files/scripts/test-gate-scripts.sh scripts/ +``` + +Read `Makefile` and `scripts/test-gate-scripts.sh`. The second one is the test for the three +scripts you write in step 4: it shows exactly which inputs must pass and which must fail. + +- [ ] **2. Write the root `Cargo.toml`** with exactly this content: + +```toml +[workspace] +resolver = "3" +members = [ + "crates/proto", + "crates/loopd", + "crates/brokerd", + "crates/gatewayd", + "crates/inferproxy", + "crates/toolkit", + "crates/bxctl", +] + +[workspace.package] +edition = "2024" +rust-version = "1.95" +publish = false + +[workspace.lints.rust] +unsafe_code = "forbid" + +[workspace.dependencies] +proto = { path = "crates/proto" } +``` + +- [ ] **3. Write the seven crates.** `crates/loopd/Cargo.toml` is: + +```toml +[package] +name = "loopd" +version = "0.1.0" +edition.workspace = true +rust-version.workspace = true +publish.workspace = true + +[lints] +workspace = true + +[dependencies] +proto.workspace = true +``` + +The other five role crates are the same with their own `name`. `crates/proto/Cargo.toml` is the +same with `name = "proto"` and an empty `[dependencies]` section. + +Each `src/lib.rs` is one doc comment line and nothing else: + +| Crate | `src/lib.rs` | +|---|---| +| `proto` | `//! Shared data types and the frame codec for Boxmaker. No policy and no I/O beyond frames.` | +| `loopd` | `//! The agent loop: sessions, prompt assembly and memory. It holds no authority.` | +| `brokerd` | `//! The broker: the only role that holds authority.` | +| `gatewayd` | `//! The Mattermost channel. Outbound connections only.` | +| `inferproxy` | ``//! Forwards bytes between `infer.sock` and the llama-server router. It logs nothing.`` | +| `toolkit` | `//! Entry points that run inside tool containers.` | +| `bxctl` | `//! The owner's command-line tool.` | + +Each `src/main.rs` (not for `proto`) follows this pattern, with its own name and milestone +(`loopd` M2, `brokerd` M3, `gatewayd` M4, `inferproxy` M2, `toolkit` M3, `bxctl` M2): + +```rust +fn main() { + eprintln!("loopd: not implemented until M2"); + std::process::exit(2); +} +``` + +Run `cargo build`. Expected: it finishes without warnings and creates `Cargo.lock`. + +- [ ] **4. Write the three gate scripts.** POSIX `sh` only: no bash features, no Python, no `jq`. +Each takes an optional `ROOT` argument that defaults to `.`, prints one line per problem to stderr +starting with the script's name, and exits 1 if there was any problem, 0 otherwise. + +- `scripts/check-lines.sh [ROOT]`: fails if any `*.rs` file under `ROOT/crates` has more than 500 + lines. Files under any `target/` directory are ignored. Test files count. +- `scripts/check-crate-deps.sh [ROOT]`: the workspace crates are the directories in `ROOT/crates`. + Fails if `proto` depends on any workspace crate, or if any other crate depends on a workspace + crate other than `proto`. Look at every manifest section whose name contains `dependencies` + (so `[dev-dependencies]` counts). A dependency is any key in such a section, however it is + written: `x.workspace = true`, `x = { path = "…" }`, `x = "1"`. +- `scripts/check-dep-docs.sh [ROOT]`: fails if a key in `[workspace.dependencies]` of + `ROOT/Cargo.toml` that is not a workspace crate has no table row starting ``| `name` |`` in + `ROOT/docs/dependencies.md`. It also fails if any dependency line in any + `ROOT/crates/*/Cargo.toml` lacks `workspace = true`. + +Run `sh scripts/test-gate-scripts.sh`. Expected: `test-gate-scripts: ok`. If it reports failures, +fix your scripts, never the test. + +- [ ] **5. Write the two documents.** `docs/dependencies.md`: + +```markdown +# Dependencies + +Every external crate has a row here. `scripts/check-dep-docs.sh` enforces it. + +| Crate | Version | Used by | Why | +|---|---|---|---| +``` + +`docs/egress.md`: + +```markdown +# Egress + +Every outbound network call the project makes, at run time or in development. Nothing else is +allowed. + +| When | From | To | What | +|---|---|---|---| +| Development | `cargo` | crates.io | Downloading the crates listed in `docs/dependencies.md` | +| Development | `make audit` | github.com/rustsec/advisory-db | The RustSec advisory database, fetched by `cargo deny check advisories` | +``` + +- [ ] **6. Run the gate.** `make gate`. Expected last line: `gate: ok`. At this stage `cargo deny` + prints `license-not-encountered` warnings, because there are no dependencies yet. They are + expected and they go away in task 02. Do not edit `deny.toml`. + +- [ ] **7. Log and commit.** Add your row to `docs/implementer-log.md`, then: + +```sh +git add Cargo.toml Cargo.lock Makefile deny.toml crates scripts docs/dependencies.md docs/egress.md docs/implementer-log.md +git status --short +git commit +``` + +`git status --short` must show nothing untracked except ignored files. Use the commit subject at +the top of this file and the trailer from `AGENTS.md`. + +## Done when + +- `make gate` prints `gate: ok`. +- `cmp Makefile docs/plans/M1/files/Makefile` and the same for `deny.toml` and + `scripts/test-gate-scripts.sh` print nothing. +- One new commit on `m1`. + +## Stop and report if + +- `cargo deny` is not installed (`cargo deny --version` fails). +- `make gate` fails in a step that is not one of your scripts and you cannot see why. diff --git a/docs/plans/M1/02-proto-values.md b/docs/plans/M1/02-proto-values.md new file mode 100644 index 0000000..5805e51 --- /dev/null +++ b/docs/plans/M1/02-proto-values.md @@ -0,0 +1,157 @@ +# M1 task 02: `proto` identifiers and values + +**Branch:** `m1` (run `git switch m1`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `Add validated identifiers, Hash32, Timestamp and DataClass to proto` + +## Goal + +Add the small validated value types that every later type is built from. They are defined by +`crates/proto/tests/ids.rs`, which you copy in and must not edit. + +## Context + +From the spec: "Identifiers. Constructors and Deserialize validate; invalid input is an error, +never a panic." A `SessionId` becomes a directory name, so it must never contain `/` or `.`. +Every value has exactly one spelling in JSON, because log files are compared and hashed as bytes. + +## Files + +- Copy: `crates/proto/tests/ids.rs` +- Create: `crates/proto/src/ids.rs`, `crates/proto/src/class.rs` +- Modify: `Cargo.toml`, `crates/proto/Cargo.toml`, `crates/proto/src/lib.rs`, + `docs/dependencies.md`, `docs/implementer-log.md`, `Cargo.lock` (generated) + +## Interfaces + +Produces, all re-exported from the crate root (`proto::SessionId` and so on): + +```rust +// crates/proto/src/ids.rs +pub enum ValueError { SessionId, Hash32, Timestamp } // Debug, Clone, Copy, PartialEq, Eq; Display; std::error::Error + +pub struct SessionId(String); // 1 to 64 bytes, each one of a-z, 0-9 or '-' +impl SessionId { + pub fn new(s: &str) -> Result; // Err(ValueError::SessionId) + pub fn as_str(&self) -> &str; +} + +pub struct Epoch(pub u32); // JSON: a plain number +pub struct CallId(pub u64); // JSON: a plain number + +pub struct Hash32([u8; 32]); // JSON: exactly 64 lowercase hex characters +impl Hash32 { + pub const ZERO: Hash32; // 32 zero bytes + pub fn from_bytes(bytes: [u8; 32]) -> Self; + pub fn as_bytes(&self) -> &[u8; 32]; + pub fn to_hex(&self) -> String; + pub fn from_hex(s: &str) -> Result; // Err(ValueError::Hash32); uppercase is an error +} + +pub struct Timestamp(u64); // Unix milliseconds; JSON: "2026-09-17T08:05:00.000Z" +impl Timestamp { + pub fn from_unix_millis(ms: u64) -> Self; + pub fn unix_millis(&self) -> u64; + pub fn now() -> Self; + pub fn to_rfc3339(&self) -> String; // always three decimals and a final Z + pub fn parse(s: &str) -> Result; // Err(ValueError::Timestamp) +} + +// crates/proto/src/class.rs +pub enum DataClass { Public, Private, Secret } // JSON: "public", "private", "secret" +``` + +Derives: `SessionId` gets `Debug, Clone, PartialEq, Eq, Hash`. `Epoch`, `CallId` and `Timestamp` +get `Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash`. `Hash32` gets +`Debug, Clone, Copy, PartialEq, Eq, Hash`. `DataClass` gets +`Debug, Clone, Copy, PartialEq, Eq, PartialOrd, Ord, Hash`, with the variants in the order shown so +that `Public < Private < Secret`. All of them get `Serialize, Deserialize`. + +Rules the tests check: + +- `Timestamp::parse` accepts only the spelling `to_rfc3339` produces. Parse the text, format the + result again, and reject the input if the two strings differ. +- Deserializing must validate. A JSON string that `new`, `from_hex` or `parse` would reject must + fail to deserialize. + +## API notes (verified on docs.rs, 2026-09-17) + +- serde container attributes: `#[serde(try_from = "String", into = "String")]` makes a type + serialize through `From for String` and deserialize through `TryFrom for T`; the + error type must implement `Display`. `into` requires `Clone`. `#[serde(transparent)]` makes a + one-field struct serialize as that field. `#[serde(rename_all = "lowercase")]` on an enum. +- `humantime::format_rfc3339_millis(t: std::time::SystemTime) -> humantime::Rfc3339Timestamp`, + which implements `Display`. +- `humantime::parse_rfc3339(s: &str) -> Result`. +- `std::time::UNIX_EPOCH + std::time::Duration::from_millis(ms)` gives a `SystemTime`; + `t.duration_since(UNIX_EPOCH)` gives a `Result`; `Duration::as_millis()` is `u128`. + +## Steps + +- [ ] **1. Copy the test and add the dependencies.** + +```sh +git switch m1 +mkdir -p crates/proto/tests +cp docs/plans/M1/files/crates/proto/tests/ids.rs crates/proto/tests/ +``` + +Add to `[workspace.dependencies]` in the root `Cargo.toml`: + +```toml +serde = { version = "1.0.229", features = ["derive"] } +serde_json = "1.0.151" +humantime = "2.4.0" +``` + +Add to `[dependencies]` in `crates/proto/Cargo.toml`: + +```toml +serde.workspace = true +serde_json.workspace = true +humantime.workspace = true +``` + +Add these rows to the table in `docs/dependencies.md`: + +```markdown +| `serde` | 1.0.229 | `proto` | Derives serialization for every shared type. MIT OR Apache-2.0. | +| `serde_json` | 1.0.151 | `proto` | JSON for frames and log files. MIT OR Apache-2.0. | +| `humantime` | 2.4.0 | `proto` | RFC 3339 timestamps. No dependencies of its own. MIT OR Apache-2.0. | +``` + +- [ ] **2. See the test fail.** `cargo test -p proto --test ids`. Expected: it does not compile, + because `proto::SessionId` and the other names do not exist. + +- [ ] **3. Write `class.rs` and `ids.rs`,** and in `lib.rs` add below the doc comment: + +```rust +pub mod class; +pub mod ids; + +pub use class::DataClass; +pub use ids::{CallId, Epoch, Hash32, SessionId, Timestamp, ValueError}; +``` + +Write the hex encoding and decoding yourself; do not add a hex crate. + +- [ ] **4. See the test pass.** `cargo test -p proto --test ids`. Expected: `11 passed; 0 failed`. + +- [ ] **5. Run the gate.** `make gate`. Expected last line: `gate: ok`. + +- [ ] **6. Log and commit.** + +```sh +git add Cargo.toml Cargo.lock crates/proto docs/dependencies.md docs/implementer-log.md +git commit +``` + +## Done when + +- `cargo test -p proto --test ids` reports 11 passed. +- `make gate` prints `gate: ok`. +- `cmp crates/proto/tests/ids.rs docs/plans/M1/files/crates/proto/tests/ids.rs` prints nothing. + +## Stop and report if + +- A test in `ids.rs` seems wrong to you. Do not edit it. +- `humantime` does not have the two functions named above. diff --git a/docs/plans/M1/03-proto-wire.md b/docs/plans/M1/03-proto-wire.md new file mode 100644 index 0000000..e6649a9 --- /dev/null +++ b/docs/plans/M1/03-proto-wire.md @@ -0,0 +1,130 @@ +# M1 task 03: `proto` IPC messages + +**Branch:** `m1` (run `git switch m1`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `Add IPC envelope and tool messages to proto` + +## Goal + +Add the message types that roles send each other. They are defined by +`crates/proto/tests/wire.rs` and the byte-exact JSON files in +`crates/proto/tests/fixtures/wire/`. Each fixture must decode to the expected value, and the value +must encode back to exactly the fixture's bytes. + +## Context + +From the spec: "Unknown fields are rejected everywhere. Enums are tagged with explicit strings. No +floats. No free-form maps." An envelope looks like this: + +```json +{"v":1,"id":7,"final":true,"msg":{"kind":"tool_request","body":{"session":"mm-thread-42","call":3,"tool":"read_file","arguments":"{\"path\":\"/etc/hosts\"}"}}} +``` + +`arguments` is the model's JSON object carried as a string. `proto` never parses it. + +## Files + +- Copy: `crates/proto/tests/wire.rs`, `crates/proto/tests/fixtures/wire/` (6 files) +- Create: `crates/proto/src/wire.rs` +- Modify: `crates/proto/src/lib.rs`, `docs/implementer-log.md` + +## Interfaces + +Consumes from task 02: `proto::SessionId`, `proto::CallId`, `proto::Timestamp`, `proto::DataClass`. + +Produces, in `crates/proto/src/wire.rs`, re-exported from the crate root. **Keep fields and +variants in exactly this order**; serde writes JSON in declaration order and the fixtures depend on +it. + +```rust +pub const PROTOCOL_VERSION: u32 = 1; + +pub struct Envelope { pub v: u32, pub id: u64, pub r#final: bool, pub msg: Message } + +// JSON: {"kind":"tool_request","body":{...}} ; kinds are snake_case +pub enum Message { ToolRequest(ToolRequest), ToolResponse(ToolResponse), Error(WireError) } + +pub struct WireError { pub code: ErrorCode, pub detail: String } + +// JSON: "bad_frame", "bad_version", "bad_message", "internal" +pub enum ErrorCode { BadFrame, BadVersion, BadMessage, Internal } + +pub struct ToolRequest { pub session: SessionId, pub call: CallId, pub tool: String, pub arguments: String } + +// JSON: {"status":"result","content":...} ; the tag sits beside the fields; statuses are snake_case +pub enum ToolResponse { + PendingApproval { approval: String, expires: Timestamp }, + Result { content: String, class: DataClass, untrusted: bool, truncated: bool }, + Failed { message: String }, + Denied { reason: DenyReason }, +} + +// JSON: "no_grant", "grant_expired", "taint_too_high", "denied_by_grant", "approval_refused", "approval_expired" +pub enum DenyReason { NoGrant, GrantExpired, TaintTooHigh, DeniedByGrant, ApprovalRefused, ApprovalExpired } +``` + +Derives: every type gets `Debug, Clone, PartialEq, Eq, Serialize, Deserialize`. `ErrorCode` and +`DenyReason` also get `Copy`. + +Rules the tests check: an unknown field is an error in the envelope, beside `kind` and `body`, +inside a body, and inside a `ToolResponse` variant. A missing field is an error, including +`final`. An unknown `kind` or `status` is an error. + +## API notes (serde 1.0, verified 2026-09-17) + +- `#[serde(deny_unknown_fields)]` works on structs and on enums. +- Adjacently tagged enum: `#[serde(tag = "kind", content = "body")]`. +- Internally tagged enum (tag beside the variant's own fields): `#[serde(tag = "status")]`. +- `#[serde(rename_all = "snake_case")]` renames variants: `ToolRequest` becomes `tool_request`. +- Several attributes can share one `#[serde(...)]`, separated by commas. +- A field named `r#final` serializes as `final`. No rename is needed. + +## Steps + +- [ ] **1. Copy the test and fixtures.** + +```sh +git switch m1 +mkdir -p crates/proto/tests/fixtures +cp docs/plans/M1/files/crates/proto/tests/wire.rs crates/proto/tests/ +cp -r docs/plans/M1/files/crates/proto/tests/fixtures/wire crates/proto/tests/fixtures/ +``` + +- [ ] **2. See the test fail.** `cargo test -p proto --test wire`. Expected: it does not compile. + +- [ ] **3. Write `wire.rs`,** and add to `lib.rs`: + +```rust +pub mod wire; + +pub use wire::{ + DenyReason, Envelope, ErrorCode, Message, PROTOCOL_VERSION, ToolRequest, ToolResponse, + WireError, +}; +``` + +Keep the `pub mod` lines in `lib.rs` in alphabetical order, and the `pub use` lines after them in +alphabetical order by module. + +- [ ] **4. See the test pass.** `cargo test -p proto --test wire`. Expected: `9 passed; 0 failed`. + +- [ ] **5. Run the gate.** `make gate`. Expected last line: `gate: ok`. + +- [ ] **6. Log and commit.** + +```sh +git add crates/proto docs/implementer-log.md +git commit +``` + +## Done when + +- `cargo test -p proto --test wire` reports 9 passed. +- `make gate` prints `gate: ok`. +- `diff -r crates/proto/tests/fixtures/wire docs/plans/M1/files/crates/proto/tests/fixtures/wire` + and `cmp crates/proto/tests/wire.rs docs/plans/M1/files/crates/proto/tests/wire.rs` print nothing. + +## Stop and report if + +- A fixture cannot be matched byte for byte with the field order given above. +- `deny_unknown_fields` does not reject one of the cases in the test with the enum + representations named above. diff --git a/docs/plans/M1/04-proto-frame.md b/docs/plans/M1/04-proto-frame.md new file mode 100644 index 0000000..d2bf7d8 --- /dev/null +++ b/docs/plans/M1/04-proto-frame.md @@ -0,0 +1,112 @@ +# M1 task 04: `proto` frame codec + +**Branch:** `m1` (run `git switch m1`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `Add length-prefixed frame codec to proto` + +## Goal + +Read and write one envelope on a byte stream. This is the only I/O in `proto`. It is the first code +that touches bytes from another process, so it must never trust a length it has not checked. + +## Context + +From the spec: "A 4-byte big-endian unsigned length `N`, then `N` bytes of UTF-8 JSON holding one +envelope. `MAX_FRAME` is 1,048,576. A reader that sees `N == 0` or `N > MAX_FRAME` returns an error +without reading or allocating the body." And: "`v` is the protocol version, 1. A reader that sees +another value" rejects the frame. + +## Files + +- Copy: `crates/proto/tests/frame.rs`, `crates/proto/tests/fixtures/frame/tool_request.bin` +- Create: `crates/proto/src/frame.rs` +- Modify: `crates/proto/src/lib.rs`, `docs/implementer-log.md` + +## Interfaces + +Consumes from task 03: `proto::Envelope`, `proto::PROTOCOL_VERSION`. + +Produces, in `crates/proto/src/frame.rs`, re-exported from the crate root: + +```rust +pub const MAX_FRAME: usize = 1_048_576; + +#[derive(Debug)] +pub enum FrameError { + Closed, // the stream ended before the first byte of a frame + Empty, // the length prefix was 0 + TooLarge(usize), // the length prefix, or an outgoing body, was larger than MAX_FRAME + BadVersion(u32), // the envelope's v was not PROTOCOL_VERSION + Json(serde_json::Error), // the body was not a valid envelope, or could not be encoded + Io(std::io::Error), // any other I/O failure, including a stream that ends mid-frame +} +// FrameError implements Display (a short message per variant) and std::error::Error. + +pub fn write_frame(w: &mut W, env: &Envelope) -> Result<(), FrameError>; +pub fn read_frame(r: &mut R) -> Result; +``` + +Rules the tests check: + +- `write_frame` encodes first. If the body is larger than `MAX_FRAME` it returns `TooLarge(len)` + and writes **nothing**. Otherwise it writes the 4-byte length, the body, and flushes. +- `read_frame` reads exactly 4 length bytes. `Read::read` may return fewer bytes than asked for, + even one at a time; keep reading until you have all 4. If the stream ends before the first of + them, return `Closed`. If it ends after 1 to 3 of them, return `Io`. +- It checks `N == 0` (`Empty`) and `N > MAX_FRAME` (`TooLarge(N)`) **before** it allocates a + buffer or reads the body. One test uses a reader that panics if the body is read. +- A body of exactly `MAX_FRAME` bytes is allowed. +- A stream that ends inside the body is `Io`. A body that is not a valid envelope is `Json`. +- After decoding, `v != PROTOCOL_VERSION` is `BadVersion(v)`. + +## API notes + +- `u32::from_be_bytes([u8; 4])` and `u32::to_be_bytes()`. +- `serde_json::to_vec(&T) -> Result, serde_json::Error>` and + `serde_json::from_slice::(&[u8])`. +- `Read::read_exact` returns `ErrorKind::UnexpectedEof` when the stream ends early, which is what + you want for the body. It cannot tell a clean close from a cut-off header, so do not use it for + the 4 length bytes. +- A `read` that fails with `ErrorKind::Interrupted` should be retried. +- Build an `io::Error` from a kind with `std::io::Error::from(std::io::ErrorKind::UnexpectedEof)`. + +## Steps + +- [ ] **1. Copy the test and fixture.** + +```sh +git switch m1 +cp docs/plans/M1/files/crates/proto/tests/frame.rs crates/proto/tests/ +cp -r docs/plans/M1/files/crates/proto/tests/fixtures/frame crates/proto/tests/fixtures/ +``` + +- [ ] **2. See the test fail.** `cargo test -p proto --test frame`. Expected: it does not compile. + +- [ ] **3. Write `frame.rs`,** and add to `lib.rs` (alphabetical order, as before): + +```rust +pub mod frame; + +pub use frame::{FrameError, MAX_FRAME, read_frame, write_frame}; +``` + +- [ ] **4. See the test pass.** `cargo test -p proto --test frame`. Expected: `13 passed; 0 failed`. + +- [ ] **5. Run the gate.** `make gate`. Expected last line: `gate: ok`. + +- [ ] **6. Log and commit.** + +```sh +git add crates/proto docs/implementer-log.md +git commit +``` + +## Done when + +- `cargo test -p proto --test frame` reports 13 passed. +- `make gate` prints `gate: ok`. +- `cmp crates/proto/tests/frame.rs docs/plans/M1/files/crates/proto/tests/frame.rs` prints nothing. +- `frame.rs` contains no `unwrap`, `expect`, `panic!` or unchecked indexing. + +## Stop and report if + +- `write_matches_the_fixture_byte_for_byte` fails although `cargo test -p proto --test wire` passes. diff --git a/docs/plans/M1/05-proto-grant.md b/docs/plans/M1/05-proto-grant.md new file mode 100644 index 0000000..22fa2cb --- /dev/null +++ b/docs/plans/M1/05-proto-grant.md @@ -0,0 +1,152 @@ +# M1 task 05: `proto` grant files + +**Branch:** `m1` (run `git switch m1`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `Add Grant, Mode and Constraints to proto` + +## Goal + +Add the data type for a grant file. A grant is a TOML file the owner writes to allow one kind of +tool call. This task only parses grants. Deciding whether a grant matches a request is `brokerd`'s +job in a later milestone; write no matching logic. + +## Context + +From the design brief: "A grant is a file the owner writes: tool, argument constraints (paths, +hosts, patterns), allowed data classes, expiry, mode (`auto`, `ask`, `deny`)." + +From the spec: `max_taint` is "the highest session taint under which the grant applies". +`result_class` is the data class of the tool's results and defaults to `private`. `untrusted` +says whether results may have been written by someone other than the owner and defaults to `true`. +A typing mistake in a grant file must be an error, never silently ignored: a misspelt +`max_tiant` that fell back to a default would widen what the grant allows. + +A full grant file: + +```toml +tool = "http_fetch" +mode = "ask" +max_taint = "secret" +result_class = "public" +untrusted = false +expires = "2026-12-31T00:00:00.000Z" +secret = "example-api-token" + +[constraints] +paths = ["/home/kyle/notes/**"] +hosts = ["example.com", "api.example.com"] +patterns = ["^GET "] +``` + +## Files + +- Copy: `crates/proto/tests/grant.rs`, `crates/proto/tests/fixtures/grant/` (6 files) +- Create: `crates/proto/src/grant.rs` +- Modify: `Cargo.toml`, `crates/proto/Cargo.toml`, `crates/proto/src/lib.rs`, + `docs/dependencies.md`, `docs/implementer-log.md`, `Cargo.lock` (generated) + +## Interfaces + +Consumes from task 02: `proto::DataClass`, `proto::Timestamp`. + +Produces, in `crates/proto/src/grant.rs`, re-exported from the crate root: + +```rust +// TOML and JSON: "auto", "ask", "deny" +pub enum Mode { Auto, Ask, Deny } + +pub struct Grant { + pub tool: String, // required + pub mode: Mode, // required + pub max_taint: DataClass, // required + pub result_class: DataClass, // default: DataClass::Private + pub untrusted: bool, // default: true + pub expires: Option, // default: None, meaning no expiry + pub secret: Option, // default: None + pub constraints: Constraints, // default: all three lists empty +} + +pub struct Constraints { // also derives Default + pub paths: Vec, // default: empty + pub hosts: Vec, // default: empty + pub patterns: Vec, // default: empty +} +``` + +Derives: all three get `Debug, Clone, PartialEq, Eq, Serialize, Deserialize`. `Mode` also gets +`Copy`. `Constraints` also gets `Default`. + +Rules the tests check: `tool`, `mode` and `max_taint` are required. Unknown keys are errors in the +grant and in `[constraints]`. An unknown mode is an error. `expires` is written as a quoted string +in the `Timestamp` spelling from task 02, not as a bare TOML date. + +## API notes (verified on docs.rs, 2026-09-17) + +- `toml::from_str::(&str) -> Result`. Only the test uses the `toml` crate, + so it is a dev-dependency. `grant.rs` itself uses only serde. +- `#[serde(default)]` on a field uses `Default::default()` when the key is missing. + `#[serde(default = "function_name")]` calls a function in the same module instead; use it for + `result_class` and `untrusted`, whose defaults are not the types' `Default`. +- `#[serde(deny_unknown_fields)]` and `#[serde(rename_all = "lowercase")]` as in earlier tasks. + +## Steps + +- [ ] **1. Copy the test and fixtures, add the dev-dependency.** + +```sh +git switch m1 +cp docs/plans/M1/files/crates/proto/tests/grant.rs crates/proto/tests/ +cp -r docs/plans/M1/files/crates/proto/tests/fixtures/grant crates/proto/tests/fixtures/ +``` + +Add to `[workspace.dependencies]` in the root `Cargo.toml`: + +```toml +toml = "1.1.6" +``` + +Add a new section at the end of `crates/proto/Cargo.toml`: + +```toml +[dev-dependencies] +toml.workspace = true +``` + +Add this row to `docs/dependencies.md`: + +```markdown +| `toml` | 1.1.6 | `proto` tests; `brokerd` and `bxctl` from M3 | Parses grant files. MIT OR Apache-2.0. | +``` + +- [ ] **2. See the test fail.** `cargo test -p proto --test grant`. Expected: it does not compile. + +- [ ] **3. Write `grant.rs`,** and add to `lib.rs` (alphabetical order): + +```rust +pub mod grant; + +pub use grant::{Constraints, Grant, Mode}; +``` + +- [ ] **4. See the test pass.** `cargo test -p proto --test grant`. Expected: `4 passed; 0 failed`. + +- [ ] **5. Run the gate.** `make gate`. Expected last line: `gate: ok`. If `cargo deny` reports a + licence that is not allowed or a duplicate crate version, stop and report; do not edit + `deny.toml`. + +- [ ] **6. Log and commit.** + +```sh +git add Cargo.toml Cargo.lock crates/proto docs/dependencies.md docs/implementer-log.md +git commit +``` + +## Done when + +- `cargo test -p proto --test grant` reports 4 passed. +- `make gate` prints `gate: ok`. +- `diff -r crates/proto/tests/fixtures/grant docs/plans/M1/files/crates/proto/tests/fixtures/grant` + prints nothing. + +## Stop and report if + +- `cargo deny` rejects `toml` or one of its dependencies. diff --git a/docs/plans/M1/06-proto-records.md b/docs/plans/M1/06-proto-records.md new file mode 100644 index 0000000..fd04876 --- /dev/null +++ b/docs/plans/M1/06-proto-records.md @@ -0,0 +1,131 @@ +# M1 task 06: `proto` audit and session log records + +**Branch:** `m1` (run `git switch m1`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `Add audit and session log record types to proto` + +## Goal + +Add the record types for the two log files. Each log is JSONL: one JSON object per line. This task +adds only the types. Writing files, hashing and locking come in later milestones. + +## Context + +From the design brief: "Audit log: append-only JSONL, hash-chained, one record per decision +including denials." and "`sessions//.jsonl` — append-only session log". The assistant's +messages are stored "exactly as the server returned" them, because the next request must repeat +them byte for byte or the inference server's cache is lost. + +The hash chain hashes the exact bytes of each stored line, so a record must always encode to the +same bytes. That is why the fixtures are compared byte for byte in both directions. + +One audit line: + +```json +{"seq":1,"time":"2026-09-17T08:05:01.250Z","prev":"000102…1e1f","session":"mm-thread-42","call":2,"tool":"shell","arguments":"{\"command\":\"df -h\"}","session_taint":"private","decision":{"outcome":"approved","grant":"shell-ask","approver":"u8f3k2","post":"p9x7"}} +``` + +## Files + +- Copy: `crates/proto/tests/records.rs`, `crates/proto/tests/fixtures/records/` (2 files) +- Create: `crates/proto/src/audit.rs`, `crates/proto/src/log.rs` +- Modify: `crates/proto/src/lib.rs`, `docs/implementer-log.md` + +## Interfaces + +Consumes from tasks 02 and 03: `CallId`, `DataClass`, `DenyReason`, `Epoch`, `Hash32`, `SessionId`, +`Timestamp`. + +Produces, re-exported from the crate root. **Keep fields and variants in exactly this order.** + +```rust +// crates/proto/src/audit.rs + +// JSON: {"outcome":"allowed","grant":"…"} ; the tag sits beside the fields; outcomes are snake_case +pub enum DecisionRecord { + Allowed { grant: String }, + Approved { grant: String, approver: String, post: Option }, + Denied { reason: DenyReason, grant: Option }, +} + +pub struct AuditRecord { + pub seq: u64, + pub time: Timestamp, + pub prev: Hash32, + pub session: SessionId, + pub call: CallId, + pub tool: String, + pub arguments: String, + pub session_taint: DataClass, + pub decision: DecisionRecord, +} + +// crates/proto/src/log.rs + +pub struct ToolCall { pub id: String, pub name: String, pub arguments: String } + +// JSON: {"type":"user","time":"…","content":"…"} ; the tag sits beside the fields; types are snake_case +pub enum LogRecord { + SessionStart { time: Timestamp, session: SessionId, epoch: Epoch, slot: u32, baseline: Hash32 }, + User { time: Timestamp, content: String }, + Assistant { time: Timestamp, content: Option, reasoning_content: Option, + tool_calls: Vec }, + ToolResult { time: Timestamp, call: CallId, tool_call_id: String, content: String, + class: DataClass, untrusted: bool, truncated: bool }, + CacheLoss { time: Timestamp, expected: u64, got: u64 }, + EpochEnd { time: Timestamp, next: Epoch, summary: String }, +} +``` + +Derives: every type gets `Debug, Clone, PartialEq, Eq, Serialize, Deserialize`. + +Rules the tests check: a `None` is written as `null`, never left out (so do not use +`skip_serializing_if`). Unknown fields and unknown `type` or `outcome` values are errors. + +## API notes + +Internally tagged enums and `deny_unknown_fields` work as in task 03: +`#[serde(tag = "type", rename_all = "snake_case", deny_unknown_fields)]`. + +## Steps + +- [ ] **1. Copy the test and fixtures.** + +```sh +git switch m1 +cp docs/plans/M1/files/crates/proto/tests/records.rs crates/proto/tests/ +cp -r docs/plans/M1/files/crates/proto/tests/fixtures/records crates/proto/tests/fixtures/ +``` + +- [ ] **2. See the test fail.** `cargo test -p proto --test records`. Expected: it does not compile. + +- [ ] **3. Write `audit.rs` and `log.rs`,** and add to `lib.rs` (alphabetical order): + +```rust +pub mod audit; +pub mod log; + +pub use audit::{AuditRecord, DecisionRecord}; +pub use log::{LogRecord, ToolCall}; +``` + +- [ ] **4. See the test pass.** `cargo test -p proto --test records`. Expected: `3 passed; 0 failed`. + +- [ ] **5. Run the gate.** `make gate`. Expected last line: `gate: ok`. + +- [ ] **6. Log and commit.** + +```sh +git add crates/proto docs/implementer-log.md +git commit +``` + +## Done when + +- `cargo test -p proto --test records` reports 3 passed. +- `cargo test -p proto` reports 40 passed in total across the five test files. +- `make gate` prints `gate: ok`. +- `diff -r crates/proto/tests docs/plans/M1/files/crates/proto/tests` prints nothing. + +## Stop and report if + +- A fixture line cannot be matched byte for byte with the field order given above. diff --git a/docs/plans/M1/07-brokerd-decision.md b/docs/plans/M1/07-brokerd-decision.md new file mode 100644 index 0000000..0e4ea1f --- /dev/null +++ b/docs/plans/M1/07-brokerd-decision.md @@ -0,0 +1,178 @@ +# M1 task 07: `Decision` in `brokerd` + +**Branch:** `m1` (run `git switch m1`; `git status --short` must be empty, otherwise stop) +**Commit subject:** `Add Decision, decide and the runner stub to brokerd` + +## Goal + +Put the project's central safety rule into types: a tool can only run if policy produced a +`Decision`, and nothing outside `brokerd`'s policy module can make one. This task adds the type, +a policy function that denies everything, and a runner stub. Grants are matched in M3, not here. + +## Context + +From the design brief: "Authority is encoded in types: a tool cannot execute without a `Decision` +value that only `brokerd`'s policy module can construct. `Decision` is defined in `brokerd`, has a +private field and does not implement `Deserialize`, so no other crate or wire message can produce +one." Also: "No matching grant means deny." + +The proof is two `compile_fail` doctests: examples that must **fail** to compile. A third doctest +uses the same setup through the allowed path and must compile and pass. That third one shows the +first two fail because of `Decision`'s privacy and not because of a mistake in the example. + +## Files + +- Create: `crates/brokerd/src/policy.rs`, `crates/brokerd/src/runner.rs` +- Modify: `crates/brokerd/src/lib.rs`, `docs/implementer-log.md` + +## Interfaces + +Consumes from `proto`: `ToolRequest`, `ToolResponse`, `DenyReason`, `SessionId`, `CallId`. + +Produces: + +```rust +// crates/brokerd/src/policy.rs +#[derive(Debug)] // and nothing else: not Clone, not Serialize, not Deserialize +pub struct Decision { request: ToolRequest, grant: String } // both fields private + +impl Decision { + fn new(request: ToolRequest, grant: String) -> Self; // private: no `pub` + pub fn request(&self) -> &ToolRequest; + pub fn grant(&self) -> &str; +} + +/// Until M3 there are no grants, so every request is denied with DenyReason::NoGrant. +pub fn decide(request: ToolRequest) -> Result; + +// crates/brokerd/src/runner.rs +/// Takes the Decision by value, so one decision cannot run a tool twice. +pub fn run(decision: Decision) -> ToolResponse; // returns Failed { message: "no tool runner until M3" } +``` + +`Decision::new` is not called by non-test code until M3, so the compiler reports it as dead code. +This is the one place in M1 where a lint attribute is allowed. Put exactly this on `new`: + +```rust +#[cfg_attr(not(test), expect(dead_code, reason = "grant matching arrives in M3"))] +``` + +## Steps + +- [ ] **1. Write the module documentation first.** `crates/brokerd/src/policy.rs` starts with + exactly this doc comment. It contains the three doctests. + +````rust +//! Policy decisions. `Decision` can only be constructed in this module. +//! +//! Code outside this module cannot build a `Decision` with a struct literal, because its fields +//! are private: +//! +//! ```compile_fail +//! let request = proto::ToolRequest { +//! session: proto::SessionId::new("s1").unwrap(), +//! call: proto::CallId(1), +//! tool: "read_file".to_string(), +//! arguments: "{}".to_string(), +//! }; +//! let _ = brokerd::policy::Decision { request, grant: "g".to_string() }; +//! ``` +//! +//! Nor with the constructor, because it is private to this module: +//! +//! ```compile_fail +//! let request = proto::ToolRequest { +//! session: proto::SessionId::new("s1").unwrap(), +//! call: proto::CallId(1), +//! tool: "read_file".to_string(), +//! arguments: "{}".to_string(), +//! }; +//! let _ = brokerd::policy::Decision::new(request, "g".to_string()); +//! ``` +//! +//! The same setup compiles when it goes through `decide`, which proves the two examples above +//! fail because of `Decision` and not because of a mistake in the setup: +//! +//! ``` +//! let request = proto::ToolRequest { +//! session: proto::SessionId::new("s1").unwrap(), +//! call: proto::CallId(1), +//! tool: "read_file".to_string(), +//! arguments: "{}".to_string(), +//! }; +//! assert_eq!(brokerd::policy::decide(request).unwrap_err(), proto::DenyReason::NoGrant); +//! ``` +```` + +- [ ] **2. Write the unit tests** at the end of `policy.rs`, exactly: + +```rust +#[cfg(test)] +mod tests { + use super::*; + use proto::{CallId, SessionId}; + + fn request() -> ToolRequest { + ToolRequest { + session: SessionId::new("s1").unwrap(), + call: CallId(1), + tool: "read_file".to_string(), + arguments: "{}".to_string(), + } + } + + #[test] + fn no_grants_means_deny() { + assert_eq!(decide(request()).unwrap_err(), DenyReason::NoGrant); + } + + #[test] + fn decision_exposes_request_and_grant() { + let d = Decision::new(request(), "g1".to_string()); + assert_eq!(d.request().tool, "read_file"); + assert_eq!(d.grant(), "g1"); + } +} +``` + +- [ ] **3. Register the modules and see the tests fail.** Add to `crates/brokerd/src/lib.rs`, below + the doc comment: + +```rust +pub mod policy; +pub mod runner; +``` + +Create `runner.rs` as an empty file for now. Run `cargo test -p brokerd`. Expected: it does not +compile, because `Decision`, `decide`, `ToolRequest` and `DenyReason` are not defined in `policy.rs`. + +- [ ] **4. Write `Decision`, `decide` and `run`** as described under Interfaces. + +- [ ] **5. See the tests pass.** `cargo test -p brokerd`. Expected: 2 unit tests pass, and under + `Doc-tests brokerd` 3 pass, two of them marked `compile fail`. + +- [ ] **6. Check that the doctests guard what they claim.** Temporarily change `fn new` to + `pub fn new` and run `cargo test -p brokerd --doc`. Expected: one failure saying + `Test compiled successfully, but it's marked compile_fail`. Change it back and run the command + again. Expected: 3 pass. Write in your log row that you did this check and what you saw. + +- [ ] **7. Run the gate.** `make gate`. Expected last line: `gate: ok`. + +- [ ] **8. Log and commit.** + +```sh +git add crates/brokerd docs/implementer-log.md +git commit +``` + +## Done when + +- `cargo test -p brokerd` passes 2 unit tests and 3 doctests. +- `Decision` derives only `Debug`, and `new` has no `pub`. +- `make gate` prints `gate: ok`. +- `git log --oneline master..m1` shows one commit per task. + +## Stop and report if + +- The `expect(dead_code)` attribute itself causes a warning or an error. +- A `compile_fail` doctest passes in step 6 even with `pub fn new`. diff --git a/docs/plans/M1/README.md b/docs/plans/M1/README.md new file mode 100644 index 0000000..20c2b43 --- /dev/null +++ b/docs/plans/M1/README.md @@ -0,0 +1,63 @@ +# M1 implementation plan: workspace, `proto`, gate + +> **For the implementing model:** do not work from this file. The owner gives you one task file at +> a time (`01-…` to `07-…`). This file is the index for the owner and the reviewer. + +**Goal:** A Cargo workspace whose gate passes, with the real shared types in `proto` and a +`Decision` type in `brokerd` that code outside its policy module cannot construct. + +**Architecture:** Seven crates under `crates/`. `proto` holds data types and the frame codec and +depends on no workspace crate. Every role crate is a library with a thin `main.rs` and depends only +on `proto`. Behaviour is pinned by test files and byte-exact fixtures that the design model wrote +and verified against a private reference implementation; the implementer writes the code. + +**Tech stack:** Rust stable (edition 2024, `rust-version = "1.95"`), `serde` 1.0.229, +`serde_json` 1.0.151, `humantime` 2.4.0, `toml` 1.1.6 (tests only), `cargo-deny` 0.20.2, POSIX `sh`. + +**Spec:** `docs/specs/2026-09-17-pre-m1-design.md`, sections 4, 5 and 8. Brief: `docs/design.md`. + +## Global constraints + +- No `unsafe` (`unsafe_code = "forbid"` as a workspace lint). No async runtime. +- No source file over 500 lines. No role crate depends on another role crate. +- Every external dependency is in `[workspace.dependencies]` and has a row in + `docs/dependencies.md`. Crates are `publish = false`. +- Unknown fields are rejected everywhere. Field order is the wire format. +- `make gate` runs offline and must print `gate: ok` at the end of every task. +- Branch `m1`. One task, one fresh OpenCode session, one commit. Review happens once, after task 07. + +## Tasks + +| # | File | Delivers | Tests that define it | +|---|---|---|---| +| 01 | `01-workspace-and-gate.md` | Workspace, seven crates, `Makefile`, `deny.toml`, three gate scripts, `docs/dependencies.md`, `docs/egress.md` | `scripts/test-gate-scripts.sh` | +| 02 | `02-proto-values.md` | `SessionId`, `Epoch`, `CallId`, `Hash32`, `Timestamp`, `ValueError`, `DataClass` | `tests/ids.rs` | +| 03 | `03-proto-wire.md` | `Envelope`, `Message`, `ToolRequest`, `ToolResponse`, `WireError`, `ErrorCode`, `DenyReason` | `tests/wire.rs`, `fixtures/wire/` | +| 04 | `04-proto-frame.md` | `read_frame`, `write_frame`, `FrameError`, `MAX_FRAME` | `tests/frame.rs`, `fixtures/frame/` | +| 05 | `05-proto-grant.md` | `Grant`, `Mode`, `Constraints` | `tests/grant.rs`, `fixtures/grant/` | +| 06 | `06-proto-records.md` | `DecisionRecord`, `AuditRecord`, `ToolCall`, `LogRecord` | `tests/records.rs`, `fixtures/records/` | +| 07 | `07-brokerd-decision.md` | `brokerd::policy::Decision`, `decide`, `brokerd::runner::run` | doctests and unit tests in `policy.rs` | + +`files/` holds everything the tasks copy into place. Tests for a later task do not compile until +that task's types exist, which is why they are copied task by task and not all at once. + +## For the owner: running a task + +In `~/src/boxmaker`, start a fresh OpenCode session with Laguna S 2.1 and send: + +> Read `docs/plans/M1/01-workspace-and-gate.md` and do exactly that task. + +Then the next file in a new session, and so on. If a session ends with a `stopped` row in +`docs/implementer-log.md`, do not start the next task. + +## For the reviewer: after task 07 + +1. `git log --oneline master..m1` shows seven commits (plus any `stopped` rows), each with the + `Implemented-By` trailer. +2. Copied files are unchanged: + `for f in $(cd docs/plans/M1/files && find . -type f); do cmp "docs/plans/M1/files/$f" "$f"; done` +3. `git diff master..m1 --stat -- docs/design.md docs/specs docs/plans AGENTS.md CLAUDE.md` is empty. +4. `make gate` prints `gate: ok`. `make audit` passes. +5. Read every non-test source file against its task: field order, no panics in library code, no + `#[allow]`, no dependency the tasks did not name, error types as specified. +6. Write findings under "Reviews" in `docs/implementer-log.md`. diff --git a/docs/plans/M1/files/Makefile b/docs/plans/M1/files/Makefile new file mode 100644 index 0000000..a9c14f3 --- /dev/null +++ b/docs/plans/M1/files/Makefile @@ -0,0 +1,22 @@ +# Boxmaker gate. `make gate` must pass before any work is called done. It needs no network. + +.PHONY: gate audit verify-device + +gate: + cargo fmt --all --check + cargo clippy --workspace --all-targets --locked --offline -- -D warnings + cargo test --workspace --locked --offline + cargo deny --offline check bans licenses sources + sh scripts/check-lines.sh + sh scripts/check-crate-deps.sh + sh scripts/check-dep-docs.sh + sh scripts/test-gate-scripts.sh + @echo "gate: ok" + +# Fetches the RustSec advisory database. Listed in docs/egress.md. +audit: + cargo deny check advisories + +# Checks that need straylight. Filled in from M2. +verify-device: + @echo "verify-device: nothing to check until M2" diff --git a/docs/plans/M1/files/crates/proto/tests/fixtures/frame/tool_request.bin b/docs/plans/M1/files/crates/proto/tests/fixtures/frame/tool_request.bin new file mode 100644 index 0000000..fd3b2c1 Binary files /dev/null and b/docs/plans/M1/files/crates/proto/tests/fixtures/frame/tool_request.bin differ diff --git a/docs/plans/M1/files/crates/proto/tests/fixtures/grant/bad_mode.toml b/docs/plans/M1/files/crates/proto/tests/fixtures/grant/bad_mode.toml new file mode 100644 index 0000000..6b32325 --- /dev/null +++ b/docs/plans/M1/files/crates/proto/tests/fixtures/grant/bad_mode.toml @@ -0,0 +1,3 @@ +tool = "read_file" +mode = "always" +max_taint = "private" diff --git a/docs/plans/M1/files/crates/proto/tests/fixtures/grant/full.toml b/docs/plans/M1/files/crates/proto/tests/fixtures/grant/full.toml new file mode 100644 index 0000000..7bcc968 --- /dev/null +++ b/docs/plans/M1/files/crates/proto/tests/fixtures/grant/full.toml @@ -0,0 +1,12 @@ +tool = "http_fetch" +mode = "ask" +max_taint = "secret" +result_class = "public" +untrusted = false +expires = "2026-12-31T00:00:00.000Z" +secret = "example-api-token" + +[constraints] +paths = ["/home/kyle/notes/**"] +hosts = ["example.com", "api.example.com"] +patterns = ["^GET "] diff --git a/docs/plans/M1/files/crates/proto/tests/fixtures/grant/minimal.toml b/docs/plans/M1/files/crates/proto/tests/fixtures/grant/minimal.toml new file mode 100644 index 0000000..03e3920 --- /dev/null +++ b/docs/plans/M1/files/crates/proto/tests/fixtures/grant/minimal.toml @@ -0,0 +1,3 @@ +tool = "read_file" +mode = "auto" +max_taint = "private" diff --git a/docs/plans/M1/files/crates/proto/tests/fixtures/grant/missing_max_taint.toml b/docs/plans/M1/files/crates/proto/tests/fixtures/grant/missing_max_taint.toml new file mode 100644 index 0000000..5535611 --- /dev/null +++ b/docs/plans/M1/files/crates/proto/tests/fixtures/grant/missing_max_taint.toml @@ -0,0 +1,2 @@ +tool = "read_file" +mode = "auto" diff --git a/docs/plans/M1/files/crates/proto/tests/fixtures/grant/unknown_constraint.toml b/docs/plans/M1/files/crates/proto/tests/fixtures/grant/unknown_constraint.toml new file mode 100644 index 0000000..5d9a414 --- /dev/null +++ b/docs/plans/M1/files/crates/proto/tests/fixtures/grant/unknown_constraint.toml @@ -0,0 +1,6 @@ +tool = "read_file" +mode = "auto" +max_taint = "private" + +[constraints] +path = ["/etc/**"] diff --git a/docs/plans/M1/files/crates/proto/tests/fixtures/grant/unknown_field.toml b/docs/plans/M1/files/crates/proto/tests/fixtures/grant/unknown_field.toml new file mode 100644 index 0000000..aca9ced --- /dev/null +++ b/docs/plans/M1/files/crates/proto/tests/fixtures/grant/unknown_field.toml @@ -0,0 +1,4 @@ +tool = "read_file" +mode = "auto" +max_taint = "private" +max_tiant = "secret" diff --git a/docs/plans/M1/files/crates/proto/tests/fixtures/records/audit.jsonl b/docs/plans/M1/files/crates/proto/tests/fixtures/records/audit.jsonl new file mode 100644 index 0000000..9a1ad17 --- /dev/null +++ b/docs/plans/M1/files/crates/proto/tests/fixtures/records/audit.jsonl @@ -0,0 +1,4 @@ +{"seq":0,"time":"2026-09-17T08:05:00.000Z","prev":"0000000000000000000000000000000000000000000000000000000000000000","session":"mm-thread-42","call":1,"tool":"read_file","arguments":"{\"path\":\"/etc/hosts\"}","session_taint":"private","decision":{"outcome":"allowed","grant":"read-etc"}} +{"seq":1,"time":"2026-09-17T08:05:01.250Z","prev":"000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f","session":"mm-thread-42","call":2,"tool":"shell","arguments":"{\"command\":\"df -h\"}","session_taint":"private","decision":{"outcome":"approved","grant":"shell-ask","approver":"u8f3k2","post":"p9x7"}} +{"seq":2,"time":"2026-09-17T08:05:02.000Z","prev":"ffeeddccbbaa99887766554433221100ffeeddccbbaa99887766554433221100","session":"cron-morning","call":1,"tool":"consult","arguments":"{\"question\":\"hi\"}","session_taint":"secret","decision":{"outcome":"denied","reason":"taint_too_high","grant":"consult-private"}} +{"seq":3,"time":"2026-09-17T08:05:03.000Z","prev":"ffeeddccbbaa99887766554433221100ffeeddccbbaa99887766554433221100","session":"cron-morning","call":2,"tool":"rm_rf","arguments":"{}","session_taint":"secret","decision":{"outcome":"denied","reason":"no_grant","grant":null}} diff --git a/docs/plans/M1/files/crates/proto/tests/fixtures/records/session.jsonl b/docs/plans/M1/files/crates/proto/tests/fixtures/records/session.jsonl new file mode 100644 index 0000000..b02c91a --- /dev/null +++ b/docs/plans/M1/files/crates/proto/tests/fixtures/records/session.jsonl @@ -0,0 +1,7 @@ +{"type":"session_start","time":"2026-09-17T08:05:00.000Z","session":"mm-thread-42","epoch":0,"slot":0,"baseline":"000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f"} +{"type":"user","time":"2026-09-17T08:05:01.000Z","content":"What is in /etc/hosts?"} +{"type":"assistant","time":"2026-09-17T08:05:03.000Z","content":null,"reasoning_content":"The user wants a file.\nI will read it.","tool_calls":[{"id":"call_a1","name":"read_file","arguments":"{\"path\":\"/etc/hosts\"}"}]} +{"type":"tool_result","time":"2026-09-17T08:05:04.000Z","call":1,"tool_call_id":"call_a1","content":"127.0.0.1 localhost\n","class":"private","untrusted":true,"truncated":false} +{"type":"assistant","time":"2026-09-17T08:05:06.000Z","content":"It maps localhost to 127.0.0.1.","reasoning_content":null,"tool_calls":[]} +{"type":"cache_loss","time":"2026-09-17T09:00:00.000Z","expected":30695,"got":0} +{"type":"epoch_end","time":"2026-09-17T12:00:00.000Z","next":1,"summary":"Looked at /etc/hosts."} diff --git a/docs/plans/M1/files/crates/proto/tests/fixtures/wire/error.json b/docs/plans/M1/files/crates/proto/tests/fixtures/wire/error.json new file mode 100644 index 0000000..0d08341 --- /dev/null +++ b/docs/plans/M1/files/crates/proto/tests/fixtures/wire/error.json @@ -0,0 +1 @@ +{"v":1,"id":0,"final":true,"msg":{"kind":"error","body":{"code":"bad_version","detail":"expected 1"}}} diff --git a/docs/plans/M1/files/crates/proto/tests/fixtures/wire/tool_request.json b/docs/plans/M1/files/crates/proto/tests/fixtures/wire/tool_request.json new file mode 100644 index 0000000..d0f3bea --- /dev/null +++ b/docs/plans/M1/files/crates/proto/tests/fixtures/wire/tool_request.json @@ -0,0 +1 @@ +{"v":1,"id":7,"final":true,"msg":{"kind":"tool_request","body":{"session":"mm-thread-42","call":3,"tool":"read_file","arguments":"{\"path\":\"/etc/hosts\"}"}}} diff --git a/docs/plans/M1/files/crates/proto/tests/fixtures/wire/tool_response_denied.json b/docs/plans/M1/files/crates/proto/tests/fixtures/wire/tool_response_denied.json new file mode 100644 index 0000000..ae6f72b --- /dev/null +++ b/docs/plans/M1/files/crates/proto/tests/fixtures/wire/tool_response_denied.json @@ -0,0 +1 @@ +{"v":1,"id":9,"final":true,"msg":{"kind":"tool_response","body":{"status":"denied","reason":"taint_too_high"}}} diff --git a/docs/plans/M1/files/crates/proto/tests/fixtures/wire/tool_response_failed.json b/docs/plans/M1/files/crates/proto/tests/fixtures/wire/tool_response_failed.json new file mode 100644 index 0000000..9ccade2 --- /dev/null +++ b/docs/plans/M1/files/crates/proto/tests/fixtures/wire/tool_response_failed.json @@ -0,0 +1 @@ +{"v":1,"id":8,"final":true,"msg":{"kind":"tool_response","body":{"status":"failed","message":"exit status 2"}}} diff --git a/docs/plans/M1/files/crates/proto/tests/fixtures/wire/tool_response_pending.json b/docs/plans/M1/files/crates/proto/tests/fixtures/wire/tool_response_pending.json new file mode 100644 index 0000000..a1148c9 --- /dev/null +++ b/docs/plans/M1/files/crates/proto/tests/fixtures/wire/tool_response_pending.json @@ -0,0 +1 @@ +{"v":1,"id":7,"final":false,"msg":{"kind":"tool_response","body":{"status":"pending_approval","approval":"ap-0001","expires":"2026-09-17T08:35:00.000Z"}}} diff --git a/docs/plans/M1/files/crates/proto/tests/fixtures/wire/tool_response_result.json b/docs/plans/M1/files/crates/proto/tests/fixtures/wire/tool_response_result.json new file mode 100644 index 0000000..c133caa --- /dev/null +++ b/docs/plans/M1/files/crates/proto/tests/fixtures/wire/tool_response_result.json @@ -0,0 +1 @@ +{"v":1,"id":7,"final":true,"msg":{"kind":"tool_response","body":{"status":"result","content":"127.0.0.1 localhost\n","class":"private","untrusted":true,"truncated":false}}} diff --git a/docs/plans/M1/files/crates/proto/tests/frame.rs b/docs/plans/M1/files/crates/proto/tests/frame.rs new file mode 100644 index 0000000..d49edbd --- /dev/null +++ b/docs/plans/M1/files/crates/proto/tests/frame.rs @@ -0,0 +1,176 @@ +//! Tests for the frame codec. Do not edit these or the fixtures. + +use proto::{ + CallId, Envelope, FrameError, MAX_FRAME, Message, SessionId, ToolRequest, read_frame, + write_frame, +}; +use std::io::{Cursor, Read}; + +fn fixture_bytes(name: &str) -> Vec { + let path = format!("{}/tests/fixtures/frame/{name}", env!("CARGO_MANIFEST_DIR")); + std::fs::read(&path).unwrap_or_else(|e| panic!("{path}: {e}")) +} + +fn request(arguments: String) -> Envelope { + let body = ToolRequest { + session: SessionId::new("mm-thread-42").unwrap(), + call: CallId(3), + tool: "read_file".to_string(), + arguments, + }; + Envelope { + v: 1, + id: 7, + r#final: true, + msg: Message::ToolRequest(body), + } +} + +/// Serves `head`, then panics if anyone reads further. +struct HeaderOnly { + head: Cursor>, +} + +impl Read for HeaderOnly { + fn read(&mut self, buf: &mut [u8]) -> std::io::Result { + let n = self.head.read(buf)?; + assert!( + n > 0, + "the reader asked for the body of a frame it should have rejected" + ); + Ok(n) + } +} + +/// Hands out one byte per call, to catch codecs that assume `read` fills the buffer. +struct OneByteAtATime(Cursor>); + +impl Read for OneByteAtATime { + fn read(&mut self, buf: &mut [u8]) -> std::io::Result { + let end = buf.len().min(1); + self.0.read(&mut buf[..end]) + } +} + +#[test] +fn max_frame_is_one_mebibyte() { + assert_eq!(MAX_FRAME, 1_048_576); +} + +#[test] +fn write_matches_the_fixture_byte_for_byte() { + let mut out = Vec::new(); + write_frame(&mut out, &request(r#"{"path":"/etc/hosts"}"#.to_string())).unwrap(); + assert_eq!(out, fixture_bytes("tool_request.bin")); + assert_eq!(&out[..4], &[0, 0, 0, 159]); +} + +#[test] +fn read_decodes_the_fixture() { + let mut input = Cursor::new(fixture_bytes("tool_request.bin")); + let env = read_frame(&mut input).unwrap(); + assert_eq!(env, request(r#"{"path":"/etc/hosts"}"#.to_string())); +} + +#[test] +fn two_frames_in_a_row_then_closed() { + let mut bytes = fixture_bytes("tool_request.bin"); + bytes.extend(fixture_bytes("tool_request.bin")); + let mut input = Cursor::new(bytes); + assert!(read_frame(&mut input).is_ok()); + assert!(read_frame(&mut input).is_ok()); + assert!(matches!(read_frame(&mut input), Err(FrameError::Closed))); +} + +#[test] +fn short_reads_are_handled() { + let mut input = OneByteAtATime(Cursor::new(fixture_bytes("tool_request.bin"))); + assert!(read_frame(&mut input).is_ok()); +} + +#[test] +fn zero_length_is_empty() { + let mut input = Cursor::new(vec![0, 0, 0, 0]); + assert!(matches!(read_frame(&mut input), Err(FrameError::Empty))); +} + +#[test] +fn oversize_length_is_rejected_without_reading_the_body() { + let n = (MAX_FRAME as u32) + 1; + let mut input = HeaderOnly { + head: Cursor::new(n.to_be_bytes().to_vec()), + }; + match read_frame(&mut input) { + Err(FrameError::TooLarge(got)) => assert_eq!(got, MAX_FRAME + 1), + other => panic!("expected TooLarge, got {other:?}"), + } + let mut input = HeaderOnly { + head: Cursor::new(vec![0xff, 0xff, 0xff, 0xff]), + }; + assert!(matches!( + read_frame(&mut input), + Err(FrameError::TooLarge(4_294_967_295)) + )); +} + +#[test] +fn exactly_max_frame_is_accepted_by_the_length_check() { + // A body of MAX_FRAME bytes of spaces is not valid JSON, so the error must be Json, not TooLarge. + let mut bytes = (MAX_FRAME as u32).to_be_bytes().to_vec(); + bytes.extend(std::iter::repeat_n(b' ', MAX_FRAME)); + assert!(matches!( + read_frame(&mut Cursor::new(bytes)), + Err(FrameError::Json(_)) + )); +} + +#[test] +fn truncated_header_and_body_are_io_errors() { + let full = fixture_bytes("tool_request.bin"); + let mut header_cut = Cursor::new(full[..2].to_vec()); + assert!(matches!( + read_frame(&mut header_cut), + Err(FrameError::Io(_)) + )); + let mut body_cut = Cursor::new(full[..full.len() - 1].to_vec()); + assert!(matches!(read_frame(&mut body_cut), Err(FrameError::Io(_)))); +} + +#[test] +fn garbage_body_is_a_json_error() { + let mut bytes = vec![0, 0, 0, 3]; + bytes.extend(b"{{{"); + assert!(matches!( + read_frame(&mut Cursor::new(bytes)), + Err(FrameError::Json(_)) + )); +} + +#[test] +fn other_protocol_versions_are_rejected() { + let mut env = request("{}".to_string()); + env.v = 2; + let mut out = Vec::new(); + write_frame(&mut out, &env).unwrap(); + assert!(matches!( + read_frame(&mut Cursor::new(out)), + Err(FrameError::BadVersion(2)) + )); +} + +#[test] +fn oversize_envelopes_are_not_written() { + let mut out = Vec::new(); + let err = write_frame(&mut out, &request("x".repeat(MAX_FRAME))).unwrap_err(); + assert!(matches!(err, FrameError::TooLarge(n) if n > MAX_FRAME)); + assert!( + out.is_empty(), + "nothing may be written when the envelope is too large" + ); +} + +#[test] +fn frame_error_is_a_std_error_with_a_message() { + let e: Box = Box::new(FrameError::Empty); + assert!(!e.to_string().is_empty()); +} diff --git a/docs/plans/M1/files/crates/proto/tests/grant.rs b/docs/plans/M1/files/crates/proto/tests/grant.rs new file mode 100644 index 0000000..088b40b --- /dev/null +++ b/docs/plans/M1/files/crates/proto/tests/grant.rs @@ -0,0 +1,67 @@ +//! Tests for grant files. Do not edit these or the fixtures. + +use proto::{Constraints, DataClass, Grant, Mode, Timestamp}; + +fn parse(name: &str) -> Result { + let path = format!("{}/tests/fixtures/grant/{name}", env!("CARGO_MANIFEST_DIR")); + let text = std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("{path}: {e}")); + toml::from_str(&text) +} + +#[test] +fn minimal_grant_gets_safe_defaults() { + let want = Grant { + tool: "read_file".to_string(), + mode: Mode::Auto, + max_taint: DataClass::Private, + result_class: DataClass::Private, + untrusted: true, + expires: None, + secret: None, + constraints: Constraints::default(), + }; + assert_eq!(parse("minimal.toml").unwrap(), want); + assert_eq!(Constraints::default().paths, Vec::::new()); +} + +#[test] +fn full_grant() { + let want = Grant { + tool: "http_fetch".to_string(), + mode: Mode::Ask, + max_taint: DataClass::Secret, + result_class: DataClass::Public, + untrusted: false, + expires: Some(Timestamp::parse("2026-12-31T00:00:00.000Z").unwrap()), + secret: Some("example-api-token".to_string()), + constraints: Constraints { + paths: vec!["/home/kyle/notes/**".to_string()], + hosts: vec!["example.com".to_string(), "api.example.com".to_string()], + patterns: vec!["^GET ".to_string()], + }, + }; + assert_eq!(parse("full.toml").unwrap(), want); +} + +#[test] +fn mistakes_in_grant_files_are_errors() { + for name in [ + "unknown_field.toml", + "unknown_constraint.toml", + "bad_mode.toml", + "missing_max_taint.toml", + ] { + assert!(parse(name).is_err(), "{name} was accepted"); + } +} + +#[test] +fn modes_are_lowercase() { + for (mode, text) in [ + (Mode::Auto, "auto"), + (Mode::Ask, "ask"), + (Mode::Deny, "deny"), + ] { + assert_eq!(serde_json::to_string(&mode).unwrap(), format!("\"{text}\"")); + } +} diff --git a/docs/plans/M1/files/crates/proto/tests/ids.rs b/docs/plans/M1/files/crates/proto/tests/ids.rs new file mode 100644 index 0000000..9291f66 --- /dev/null +++ b/docs/plans/M1/files/crates/proto/tests/ids.rs @@ -0,0 +1,170 @@ +//! Tests for identifiers and primitive values. Do not edit: these define the required behaviour. + +use proto::{CallId, DataClass, Epoch, Hash32, SessionId, Timestamp, ValueError}; + +#[test] +fn session_id_accepts_lowercase_digits_and_hyphen() { + for ok in ["a", "mm-thread-42", "0", "a-b-c", &"x".repeat(64)] { + assert_eq!(SessionId::new(ok).unwrap().as_str(), ok); + } +} + +#[test] +fn session_id_rejects_everything_else() { + let too_long = "x".repeat(65); + for bad in [ + "", + "A", + "a b", + "a/b", + "../etc", + "a.b", + "a_b", + "é", + "a\n", + too_long.as_str(), + ] { + assert_eq!( + SessionId::new(bad), + Err(ValueError::SessionId), + "accepted {bad:?}" + ); + } +} + +#[test] +fn session_id_json_is_a_plain_string_and_is_validated() { + let id = SessionId::new("mm-thread-42").unwrap(); + assert_eq!(serde_json::to_string(&id).unwrap(), r#""mm-thread-42""#); + assert_eq!( + serde_json::from_str::(r#""mm-thread-42""#).unwrap(), + id + ); + assert!(serde_json::from_str::(r#""../etc""#).is_err()); + assert!(serde_json::from_str::("42").is_err()); +} + +#[test] +fn epoch_and_call_id_are_plain_numbers() { + assert_eq!(serde_json::to_string(&Epoch(3)).unwrap(), "3"); + assert_eq!( + serde_json::to_string(&CallId(18446744073709551615)).unwrap(), + "18446744073709551615" + ); + assert_eq!(serde_json::from_str::("7").unwrap(), CallId(7)); + assert!(serde_json::from_str::("-1").is_err()); + assert!(serde_json::from_str::("1.5").is_err()); + assert!(serde_json::from_str::("4294967296").is_err()); +} + +#[test] +fn hash32_hex_round_trip() { + let mut bytes = [0u8; 32]; + for (i, b) in bytes.iter_mut().enumerate() { + *b = i as u8; + } + let h = Hash32::from_bytes(bytes); + let hex = "000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f"; + assert_eq!(h.to_hex(), hex); + assert_eq!(Hash32::from_hex(hex).unwrap(), h); + assert_eq!(h.as_bytes(), &bytes); + assert_eq!(serde_json::to_string(&h).unwrap(), format!("\"{hex}\"")); + assert_eq!( + serde_json::from_str::(&format!("\"{hex}\"")).unwrap(), + h + ); + assert_eq!(Hash32::ZERO.to_hex(), "0".repeat(64)); +} + +#[test] +fn hash32_rejects_wrong_length_uppercase_and_non_hex() { + let upper = "000102030405060708090A0B0C0D0E0F101112131415161718191a1b1c1d1e1f"; + let non_hex = "g00102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f"; + let non_ascii = format!("é{}", "0".repeat(62)); + for bad in [ + "", + "00", + &"0".repeat(63), + &"0".repeat(65), + upper, + non_hex, + non_ascii.as_str(), + ] { + assert_eq!( + Hash32::from_hex(bad), + Err(ValueError::Hash32), + "accepted {bad:?}" + ); + } +} + +#[test] +fn timestamp_has_one_spelling() { + let t = Timestamp::from_unix_millis(1_789_632_300_000); + assert_eq!(t.unix_millis(), 1_789_632_300_000); + assert_eq!(t.to_rfc3339(), "2026-09-17T08:05:00.000Z"); + assert_eq!(Timestamp::parse("2026-09-17T08:05:00.000Z").unwrap(), t); + assert_eq!( + serde_json::to_string(&t).unwrap(), + r#""2026-09-17T08:05:00.000Z""# + ); + assert_eq!( + Timestamp::from_unix_millis(0).to_rfc3339(), + "1970-01-01T00:00:00.000Z" + ); + assert_eq!( + Timestamp::from_unix_millis(1_789_632_300_007).to_rfc3339(), + "2026-09-17T08:05:00.007Z" + ); +} + +#[test] +fn timestamp_rejects_other_spellings() { + for bad in [ + "2026-09-17T08:05:00Z", + "2026-09-17T08:05:00.0Z", + "2026-09-17T08:05:00.000000Z", + "2026-09-17 08:05:00.000Z", + "2026-09-17T08:05:00.000+00:00", + "2026-09-17t08:05:00.000z", + "2026-09-17", + "", + "now", + ] { + assert_eq!( + Timestamp::parse(bad), + Err(ValueError::Timestamp), + "accepted {bad:?}" + ); + assert!(serde_json::from_str::(&format!("\"{bad}\"")).is_err()); + } + assert!(serde_json::from_str::("1789632300000").is_err()); +} + +#[test] +fn timestamp_now_is_after_2026() { + assert!(Timestamp::now() > Timestamp::parse("2026-01-01T00:00:00.000Z").unwrap()); +} + +#[test] +fn data_class_is_ordered_and_lowercase() { + assert!(DataClass::Public < DataClass::Private); + assert!(DataClass::Private < DataClass::Secret); + assert_eq!(DataClass::Private.max(DataClass::Secret), DataClass::Secret); + assert_eq!( + serde_json::to_string(&DataClass::Secret).unwrap(), + r#""secret""# + ); + assert_eq!( + serde_json::from_str::(r#""public""#).unwrap(), + DataClass::Public + ); + assert!(serde_json::from_str::(r#""Public""#).is_err()); + assert!(serde_json::from_str::(r#""internal""#).is_err()); +} + +#[test] +fn value_error_is_a_std_error_with_a_message() { + let e: Box = Box::new(ValueError::Hash32); + assert!(!e.to_string().is_empty()); +} diff --git a/docs/plans/M1/files/crates/proto/tests/records.rs b/docs/plans/M1/files/crates/proto/tests/records.rs new file mode 100644 index 0000000..2449d00 --- /dev/null +++ b/docs/plans/M1/files/crates/proto/tests/records.rs @@ -0,0 +1,166 @@ +//! Tests for audit and session log records against JSONL fixtures. Do not edit these or the fixtures. + +use proto::{ + AuditRecord, CallId, DataClass, DecisionRecord, DenyReason, Epoch, Hash32, LogRecord, + SessionId, Timestamp, ToolCall, +}; +use serde::{Serialize, de::DeserializeOwned}; +use std::fmt::Debug; + +const SEQ_HEX: &str = "000102030405060708090a0b0c0d0e0f101112131415161718191a1b1c1d1e1f"; +const REV_HEX: &str = "ffeeddccbbaa99887766554433221100ffeeddccbbaa99887766554433221100"; + +fn ts(s: &str) -> Timestamp { + Timestamp::parse(s).unwrap() +} + +/// Line `i` of the fixture must decode to `want[i]`, and `want[i]` must encode to exactly that line. +fn check(name: &str, want: &[T]) { + let path = format!( + "{}/tests/fixtures/records/{name}", + env!("CARGO_MANIFEST_DIR") + ); + let text = std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("{path}: {e}")); + let lines: Vec<&str> = text.lines().collect(); + assert_eq!(lines.len(), want.len(), "{name}: number of lines"); + for (i, (line, want)) in lines.iter().zip(want).enumerate() { + let got: T = serde_json::from_str(line).unwrap_or_else(|e| panic!("{name}:{}: {e}", i + 1)); + assert_eq!(&got, want, "{name}:{}: decoded value", i + 1); + assert_eq!( + &serde_json::to_string(want).unwrap(), + line, + "{name}:{}: encoded bytes", + i + 1 + ); + } +} + +/// A record for session `session`; each test case overrides the fields it cares about. +fn audit(seq: u64, time: &str, prev: Hash32, session: &str) -> AuditRecord { + AuditRecord { + seq, + time: ts(time), + prev, + session: SessionId::new(session).unwrap(), + call: CallId(1), + tool: String::new(), + arguments: "{}".to_string(), + session_taint: DataClass::Private, + decision: DecisionRecord::Denied { + reason: DenyReason::NoGrant, + grant: None, + }, + } +} + +#[test] +fn audit_records() { + let seq_hash = Hash32::from_hex(SEQ_HEX).unwrap(); + let rev_hash = Hash32::from_hex(REV_HEX).unwrap(); + let want = [ + AuditRecord { + tool: "read_file".to_string(), + arguments: r#"{"path":"/etc/hosts"}"#.to_string(), + decision: DecisionRecord::Allowed { + grant: "read-etc".to_string(), + }, + ..audit(0, "2026-09-17T08:05:00.000Z", Hash32::ZERO, "mm-thread-42") + }, + AuditRecord { + call: CallId(2), + tool: "shell".to_string(), + arguments: r#"{"command":"df -h"}"#.to_string(), + decision: DecisionRecord::Approved { + grant: "shell-ask".to_string(), + approver: "u8f3k2".to_string(), + post: Some("p9x7".to_string()), + }, + ..audit(1, "2026-09-17T08:05:01.250Z", seq_hash, "mm-thread-42") + }, + AuditRecord { + tool: "consult".to_string(), + arguments: r#"{"question":"hi"}"#.to_string(), + session_taint: DataClass::Secret, + decision: DecisionRecord::Denied { + reason: DenyReason::TaintTooHigh, + grant: Some("consult-private".to_string()), + }, + ..audit(2, "2026-09-17T08:05:02.000Z", rev_hash, "cron-morning") + }, + AuditRecord { + call: CallId(2), + tool: "rm_rf".to_string(), + session_taint: DataClass::Secret, + ..audit(3, "2026-09-17T08:05:03.000Z", rev_hash, "cron-morning") + }, + ]; + check("audit.jsonl", &want); +} + +#[test] +fn session_log_records() { + let want = [ + LogRecord::SessionStart { + time: ts("2026-09-17T08:05:00.000Z"), + session: SessionId::new("mm-thread-42").unwrap(), + epoch: Epoch(0), + slot: 0, + baseline: Hash32::from_hex(SEQ_HEX).unwrap(), + }, + LogRecord::User { + time: ts("2026-09-17T08:05:01.000Z"), + content: "What is in /etc/hosts?".to_string(), + }, + LogRecord::Assistant { + time: ts("2026-09-17T08:05:03.000Z"), + content: None, + reasoning_content: Some("The user wants a file.\nI will read it.".to_string()), + tool_calls: vec![ToolCall { + id: "call_a1".to_string(), + name: "read_file".to_string(), + arguments: r#"{"path":"/etc/hosts"}"#.to_string(), + }], + }, + LogRecord::ToolResult { + time: ts("2026-09-17T08:05:04.000Z"), + call: CallId(1), + tool_call_id: "call_a1".to_string(), + content: "127.0.0.1 localhost\n".to_string(), + class: DataClass::Private, + untrusted: true, + truncated: false, + }, + LogRecord::Assistant { + time: ts("2026-09-17T08:05:06.000Z"), + content: Some("It maps localhost to 127.0.0.1.".to_string()), + reasoning_content: None, + tool_calls: vec![], + }, + LogRecord::CacheLoss { + time: ts("2026-09-17T09:00:00.000Z"), + expected: 30695, + got: 0, + }, + LogRecord::EpochEnd { + time: ts("2026-09-17T12:00:00.000Z"), + next: Epoch(1), + summary: "Looked at /etc/hosts.".to_string(), + }, + ]; + check("session.jsonl", &want); +} + +#[test] +fn unknown_fields_and_types_are_rejected() { + let user = r#"{"type":"user","time":"2026-09-17T08:05:01.000Z","content":"hi"}"#; + assert!(serde_json::from_str::(user).is_ok()); + let extra = user.replacen("\"content\"", "\"role\":\"user\",\"content\"", 1); + assert!(serde_json::from_str::(&extra).is_err()); + let unknown_type = user.replacen("\"user\"", "\"system\"", 1); + assert!(serde_json::from_str::(&unknown_type).is_err()); + + let decision = r#"{"outcome":"allowed","grant":"g"}"#; + assert!(serde_json::from_str::(decision).is_ok()); + let extra = decision.replacen("\"grant\"", "\"why\":\"\",\"grant\"", 1); + assert!(serde_json::from_str::(&extra).is_err()); +} diff --git a/docs/plans/M1/files/crates/proto/tests/wire.rs b/docs/plans/M1/files/crates/proto/tests/wire.rs new file mode 100644 index 0000000..c8df146 --- /dev/null +++ b/docs/plans/M1/files/crates/proto/tests/wire.rs @@ -0,0 +1,177 @@ +//! Tests for IPC messages against byte-exact fixtures. Do not edit these or the fixtures. + +use proto::{ + CallId, DataClass, DenyReason, Envelope, ErrorCode, Message, SessionId, Timestamp, ToolRequest, + ToolResponse, WireError, +}; + +fn fixture(name: &str) -> String { + let path = format!("{}/tests/fixtures/wire/{name}", env!("CARGO_MANIFEST_DIR")); + let text = std::fs::read_to_string(&path).unwrap_or_else(|e| panic!("{path}: {e}")); + text.trim_end_matches('\n').to_string() +} + +/// The fixture must decode to `want`, and `want` must encode to exactly the fixture's bytes. +fn check(name: &str, want: Envelope) { + let text = fixture(name); + let got: Envelope = serde_json::from_str(&text).unwrap_or_else(|e| panic!("{name}: {e}")); + assert_eq!(got, want, "{name}: decoded value"); + assert_eq!( + serde_json::to_string(&want).unwrap(), + text, + "{name}: encoded bytes" + ); +} + +fn response(id: u64, r#final: bool, body: ToolResponse) -> Envelope { + Envelope { + v: 1, + id, + r#final, + msg: Message::ToolResponse(body), + } +} + +#[test] +fn tool_request() { + let body = ToolRequest { + session: SessionId::new("mm-thread-42").unwrap(), + call: CallId(3), + tool: "read_file".to_string(), + arguments: r#"{"path":"/etc/hosts"}"#.to_string(), + }; + check( + "tool_request.json", + Envelope { + v: 1, + id: 7, + r#final: true, + msg: Message::ToolRequest(body), + }, + ); +} + +#[test] +fn tool_response_pending() { + let body = ToolResponse::PendingApproval { + approval: "ap-0001".to_string(), + expires: Timestamp::parse("2026-09-17T08:35:00.000Z").unwrap(), + }; + check("tool_response_pending.json", response(7, false, body)); +} + +#[test] +fn tool_response_result() { + let body = ToolResponse::Result { + content: "127.0.0.1 localhost\n".to_string(), + class: DataClass::Private, + untrusted: true, + truncated: false, + }; + check("tool_response_result.json", response(7, true, body)); +} + +#[test] +fn tool_response_failed() { + let body = ToolResponse::Failed { + message: "exit status 2".to_string(), + }; + check("tool_response_failed.json", response(8, true, body)); +} + +#[test] +fn tool_response_denied() { + let body = ToolResponse::Denied { + reason: DenyReason::TaintTooHigh, + }; + check("tool_response_denied.json", response(9, true, body)); +} + +#[test] +fn error_message() { + let body = WireError { + code: ErrorCode::BadVersion, + detail: "expected 1".to_string(), + }; + check( + "error.json", + Envelope { + v: 1, + id: 0, + r#final: true, + msg: Message::Error(body), + }, + ); +} + +#[test] +fn deny_reasons_and_error_codes_are_snake_case() { + let reasons = [ + (DenyReason::NoGrant, "no_grant"), + (DenyReason::GrantExpired, "grant_expired"), + (DenyReason::TaintTooHigh, "taint_too_high"), + (DenyReason::DeniedByGrant, "denied_by_grant"), + (DenyReason::ApprovalRefused, "approval_refused"), + (DenyReason::ApprovalExpired, "approval_expired"), + ]; + for (value, text) in reasons { + assert_eq!( + serde_json::to_string(&value).unwrap(), + format!("\"{text}\"") + ); + } + let codes = [ + (ErrorCode::BadFrame, "bad_frame"), + (ErrorCode::BadVersion, "bad_version"), + (ErrorCode::BadMessage, "bad_message"), + (ErrorCode::Internal, "internal"), + ]; + for (value, text) in codes { + assert_eq!( + serde_json::to_string(&value).unwrap(), + format!("\"{text}\"") + ); + } +} + +#[test] +fn unknown_and_missing_fields_are_rejected() { + let good = fixture("tool_request.json"); + assert!(serde_json::from_str::(&good).is_ok()); + let bad = [ + // extra field in the envelope + good.replacen("{\"v\":1,", "{\"v\":1,\"extra\":0,", 1), + // extra field beside kind and body + good.replacen( + "\"kind\":\"tool_request\",", + "\"kind\":\"tool_request\",\"x\":1,", + 1, + ), + // extra field in the body + good.replacen("\"call\":3,", "\"call\":3,\"priority\":9,", 1), + // missing field in the body + good.replacen("\"call\":3,", "", 1), + // missing `final` + good.replacen("\"final\":true,", "", 1), + // unknown kind + good.replacen("tool_request", "tool_demand", 1), + // invalid session id inside a message + good.replacen("mm-thread-42", "../../etc", 1), + ]; + for text in bad { + assert!( + serde_json::from_str::(&text).is_err(), + "accepted {text}" + ); + } +} + +#[test] +fn unknown_field_in_a_response_variant_is_rejected() { + let good = fixture("tool_response_denied.json"); + let bad = good.replacen("\"reason\":", "\"note\":\"x\",\"reason\":", 1); + assert!(serde_json::from_str::(&good).is_ok()); + assert!(serde_json::from_str::(&bad).is_err()); + let bad_status = good.replacen("denied", "refused", 1); + assert!(serde_json::from_str::(&bad_status).is_err()); +} diff --git a/docs/plans/M1/files/deny.toml b/docs/plans/M1/files/deny.toml new file mode 100644 index 0000000..b081bd2 --- /dev/null +++ b/docs/plans/M1/files/deny.toml @@ -0,0 +1,25 @@ +# cargo-deny configuration. `make gate` runs bans, licenses and sources offline. +# `make audit` runs advisories, which fetches the RustSec database. + +[graph] +all-features = true + +[licenses] +allow = ["MIT", "Apache-2.0", "Unicode-3.0"] +confidence-threshold = 0.9 + +[licenses.private] +ignore = true + +[bans] +multiple-versions = "deny" +wildcards = "deny" +allow-wildcard-paths = true + +[sources] +unknown-registry = "deny" +unknown-git = "deny" +allow-registry = ["https://github.com/rust-lang/crates.io-index"] + +[advisories] +yanked = "deny" diff --git a/docs/plans/M1/files/scripts/test-gate-scripts.sh b/docs/plans/M1/files/scripts/test-gate-scripts.sh new file mode 100644 index 0000000..f51d127 --- /dev/null +++ b/docs/plans/M1/files/scripts/test-gate-scripts.sh @@ -0,0 +1,81 @@ +#!/bin/sh +# Self-test for the three gate scripts. It builds small fake trees in a temporary +# directory and checks that each script passes the good tree and fails the bad ones. +# Do not edit: this file defines the required behaviour of the scripts. +set -eu +here=$(cd "$(dirname "$0")" && pwd) +tmp=$(mktemp -d) +trap 'rm -rf "$tmp"' EXIT +fails=0 + +expect() { # expect pass|fail NAME SCRIPT ROOT + want="$1"; name="$2"; script="$3"; root="$4" + if sh "$here/$script" "$root" >/dev/null 2>&1; then got=pass; else got=fail; fi + if [ "$got" != "$want" ]; then + echo "test-gate-scripts: $name: expected $want, got $got" >&2 + fails=$((fails + 1)) + fi +} + +manifest() { # manifest DIR NAME [DEPENDENCY-LINES...] + dir="$1"; name="$2"; shift 2 + mkdir -p "$dir/src" + { + printf '[package]\nname = "%s"\nversion = "0.1.0"\n\n[dependencies]\n' "$name" + for line in "$@"; do printf '%s\n' "$line"; done + } > "$dir/Cargo.toml" +} + +tree() { # tree ROOT: a good workspace with proto, loopd and brokerd + root="$1" + mkdir -p "$root/docs" + printf '[workspace]\nmembers = ["crates/*"]\n\n[workspace.dependencies]\nproto = { path = "crates/proto" }\nserde = { version = "1", features = ["derive"] }\n' > "$root/Cargo.toml" + printf '# Dependencies\n\n| Crate | Why |\n|---|---|\n| `serde` | types |\n' > "$root/docs/dependencies.md" + manifest "$root/crates/proto" proto 'serde.workspace = true' + manifest "$root/crates/loopd" loopd 'proto.workspace = true' + manifest "$root/crates/brokerd" brokerd 'proto = { workspace = true }' 'serde.workspace = true' +} + +lines() { # lines N FILE + mkdir -p "$(dirname "$2")" + i=0; : > "$2" + while [ "$i" -lt "$1" ]; do echo "// line" >> "$2"; i=$((i + 1)); done +} + +# check-lines.sh +tree "$tmp/l-ok"; lines 500 "$tmp/l-ok/crates/loopd/src/lib.rs" +expect pass "500 lines is allowed" check-lines.sh "$tmp/l-ok" +tree "$tmp/l-bad"; lines 501 "$tmp/l-bad/crates/loopd/src/deep/mod.rs" +expect fail "501 lines in a nested file" check-lines.sh "$tmp/l-bad" +tree "$tmp/l-test"; lines 501 "$tmp/l-test/crates/proto/tests/big.rs" +expect fail "501 lines in a test file" check-lines.sh "$tmp/l-test" +tree "$tmp/l-tgt"; lines 501 "$tmp/l-tgt/crates/proto/target/debug/gen.rs" +expect pass "files under target/ are ignored" check-lines.sh "$tmp/l-tgt" + +# check-crate-deps.sh +tree "$tmp/c-ok" +expect pass "roles depend only on proto" check-crate-deps.sh "$tmp/c-ok" +tree "$tmp/c-role"; manifest "$tmp/c-role/crates/loopd" loopd 'proto.workspace = true' 'brokerd.workspace = true' +expect fail "role depends on another role" check-crate-deps.sh "$tmp/c-role" +tree "$tmp/c-tbl"; manifest "$tmp/c-tbl/crates/loopd" loopd 'brokerd = { path = "../brokerd" }' +expect fail "role depends on another role by path" check-crate-deps.sh "$tmp/c-tbl" +tree "$tmp/c-dev"; printf '\n[dev-dependencies]\nbrokerd.workspace = true\n' >> "$tmp/c-dev/crates/loopd/Cargo.toml" +expect fail "role dev-depends on another role" check-crate-deps.sh "$tmp/c-dev" +tree "$tmp/c-proto"; manifest "$tmp/c-proto/crates/proto" proto 'loopd.workspace = true' +expect fail "proto depends on a role" check-crate-deps.sh "$tmp/c-proto" + +# check-dep-docs.sh +tree "$tmp/d-ok" +expect pass "every dependency is documented" check-dep-docs.sh "$tmp/d-ok" +tree "$tmp/d-miss"; printf 'rand = "0.9"\n' >> "$tmp/d-miss/Cargo.toml" +expect fail "workspace dependency without a docs row" check-dep-docs.sh "$tmp/d-miss" +tree "$tmp/d-prose"; printf 'rand = "0.9"\n' >> "$tmp/d-prose/Cargo.toml"; printf '\nWe do not use rand.\n' >> "$tmp/d-prose/docs/dependencies.md" +expect fail "a mention in prose is not a table row" check-dep-docs.sh "$tmp/d-prose" +tree "$tmp/d-loose"; manifest "$tmp/d-loose/crates/loopd" loopd 'proto.workspace = true' 'rand = "0.9"' +expect fail "crate declares a dependency outside the workspace table" check-dep-docs.sh "$tmp/d-loose" + +if [ "$fails" -ne 0 ]; then + echo "test-gate-scripts: $fails failure(s)" >&2 + exit 1 +fi +echo "test-gate-scripts: ok"