diff --git a/CLAUDE.md b/CLAUDE.md index b554738..668c73d 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -13,10 +13,11 @@ for `AGENTS.md` and `Cargo.toml` to see whether M1 has started. which also lists proposed changes that are not yet applied. - `docs/inference-contract.md` holds the M0 measurements from straylight. Where it and the brief disagree, the measurements are newer. +- `docs/specs/` holds design specs; `docs/plans/M/` holds the task files Laguna works from. - `spike/` is throwaway measurement code, not harness code. -Roles: implementation is done by Laguna S 2.1 through OpenCode on straylight, which reads -`AGENTS.md`. Design, specs, measurement and review are done here. Write plans for Laguna as small +Roles: implementation is done by Laguna S 2.1 (served by straylight) through OpenCode on this +machine, which reads `AGENTS.md`. Builds are deployed to straylight afterwards. Design, specs, measurement and review are done here. Write plans for Laguna as small closed tasks: exact paths, given type signatures, verified crate APIs, tests specified first, and "stop and report" instead of judgement calls. The model the harness serves is Ornith-1.5-35B-A3B. diff --git a/docs/decisions.md b/docs/decisions.md index 38fb608..509cb2a 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -6,6 +6,7 @@ Newest first. A decision that changes `docs/design.md` lands in the same commit | Date | Decision | Reason | |---|---|---| +| 2026-09-17 | Tool results are untrusted by default: each grant has `untrusted`, default true, and `brokerd` tracks a per-session untrusted flag beside taint. Review of Laguna's work is once per milestone, and Laguna keeps `docs/implementer-log.md`. | Owner's choices during the pre-M1 design review. Details in `docs/specs/2026-09-17-pre-m1-design.md`. | | 2026-09-17 | Threat model: the main adversary is injected text steering the model. Secondary: any one role process or tool container is compromised, and the goal is containment. `brokerd` trusts nothing `loopd` reports beyond the request itself and tracks session taint on its own. | Owner's choice. It matches the role split the brief already has. | | 2026-09-17 | Mattermost and the tailnet are trusted. A Mattermost compromise presumes the whole machine is compromised and is out of scope. Forged approvals are therefore out of scope, and there is no per-grant approval-path field. An accidental secret or PII leak into Mattermost is an accepted risk in v0. | Owner: Mattermost sits behind a single-user tailnet with ACLs. | | 2026-09-17 | Data classes: every session starts at `private`. A `secret` result raises it to `secret`. `public` is a provenance label and never lowers a session. The grant that authorises a call names the class of its results (default `private`); tools never label their own output. A grant's allowed data classes are the highest session taint under which it applies. Unattended egress is controlled by grant mode, not by lowering the floor. | `memory/core.md` is in every baseline, so no session is really public. Job-scoped grants are left to M5. | @@ -15,13 +16,14 @@ Newest first. A decision that changes `docs/design.md` lands in the same commit | 2026-09-17 | Boxmaker uses the shared `llama-server` router and the shared Ornith instance. No dedicated instance. `inferproxy` stays. | The owner runs coding agents against the same router, and a second resident copy of Ornith does not fit beside Laguna. | | 2026-09-17 | The harness runs on straylight, the same host as `llama-server`. | Owner's choice. One host to secure, and no inference traffic crosses the tailnet. | | 2026-09-17 | Container runtime is rootless Podman. | It is already the standard runtime on straylight and the owner's other hosts. | -| 2026-09-17 | Implementation is done by Laguna S 2.1 through OpenCode on straylight. Design, specs, measurement and review are done by a stronger model. Ornith-1.5-35B-A3B remains the model the harness serves. | The owner wants to test a local model on real implementation work. Plans must be written as small closed tasks with tests specified up front. | +| 2026-09-17 | Implementation is done by Laguna S 2.1, served by straylight, through OpenCode on the owner's development machine; builds are then deployed to straylight. Design, specs, measurement and review are done by a stronger model. Ornith-1.5-35B-A3B remains the model the harness serves. | The owner wants to test a local model on real implementation work. Plans must be written as small closed tasks with tests specified up front. | | 2026-09-17 | M0 is run by the design model, not by Laguna. | M0 is measurement and interpretation, and its findings bind the design. | ## Open | Decision | Needed by | |---|---| +| Repository licence. Crates are `publish = false` until it is chosen. | before publishing | | Audit integrity beyond a self-contained hash chain (for example anchoring the chain head outside `brokerd`). Deferred by the owner on 2026-09-17 to a later revision. | after v0 | | Secret store backend, and where the v0 file's key lives. straylight has no secrets manager today. | M3 | | Whether cloud-led sessions are ever allowed, and for which data classes. | M6 | diff --git a/docs/specs/2026-09-17-pre-m1-design.md b/docs/specs/2026-09-17-pre-m1-design.md new file mode 100644 index 0000000..9e50787 --- /dev/null +++ b/docs/specs/2026-09-17-pre-m1-design.md @@ -0,0 +1,281 @@ +# Pre-M1 design: threat model, data classes, IPC, shared types, workflow + +Status: draft for owner review, 2026-09-17. It fills the gaps in `docs/design.md` that M1 depends +on. Where this document and the brief disagree, the brief wins and this document is wrong. +Decisions behind it are recorded in `docs/decisions.md`. + +## 1. Threat model + +**Main adversary: injected text.** Any content the model reads can try to steer it: web pages, +files, tool output, recalled notes. The model is treated as possibly hostile at every tool call. + +**Secondary: one compromised component.** Any single role process or tool container may be +compromised. The goal is containment, not prevention: + +- Grants are the ceiling. No grant means deny, and no component can write grants. +- `brokerd` trusts nothing `loopd` reports except the request itself. It computes session taint + and the untrusted flag from the tool results it has returned. +- Tool containers never label their own output. + +**Trusted:** the owner, the host's kernel and container runtime, the owner's Mattermost server and +the tailnet. `gatewayd` only ever receives content the owner wrote. + +**Out of scope for v0:** + +- A compromised Mattermost server. It presumes the whole machine is compromised. Forged approvals + are therefore out of scope. +- Accidental secret or PII leaks into Mattermost, for example in an approval request. Accepted. +- Audit integrity against a compromised `brokerd` (anchoring the chain head elsewhere). Deferred. + +## 2. Data classes and provenance + +Two independent properties are tracked for every session. `brokerd` tracks both. + +**Taint** (what may leave the host). `DataClass` is ordered `public < private < secret`. + +1. Every session starts at `private`, because `memory/core.md` and the owner's messages are in it. +2. A tool result labelled `secret` raises the session to `secret`. Taint never goes down. +3. `public` is a provenance label on results and notes. It never lowers a session. +4. The grant that authorises a call names the class of its results (`result_class`, default + `private`). +5. A grant's `max_taint` is the highest session taint under which the grant applies. A session at + `secret` cannot use a grant whose `max_taint` is `private`. This one rule covers `consult` + payloads and `http_fetch` URLs alike. +6. Unattended egress is controlled by grant mode (`auto` or `ask`), never by lowering the floor. + Grants scoped to one scheduled job are left to M5. + +**Untrusted** (what may have steered the model). A boolean per session, false at the start. + +1. Each grant has `untrusted`, default `true`. The owner sets it to `false` only for sources that + only the owner writes, such as their own notes directory. +2. A result from a grant with `untrusted = true` sets the session's flag. It never clears. +3. M5 uses the flag to mark notes written by such sessions. + +## 3. Approvals + +1. `brokerd` composes the approval request: tool, full arguments, matched grant, session taint, + expiry. Arguments are shown as quoted data, never as prose. +2. Over Mattermost (M4), only a reply or reaction from an allowlisted user ID on that approval post + counts. `gatewayd` routes it to `brokerd`. It never reaches `loopd` as a user message. +3. `bxctl` can approve locally through `broker.sock` (M3). This path stays after M4. +4. The audit record stores who approved and, for Mattermost, the post ID. + +## 4. IPC + +**Transport.** Unix stream sockets. Blocking I/O with threads. No async runtime. + +**Frame.** A 4-byte big-endian unsigned length `N`, then `N` bytes of UTF-8 JSON holding one +envelope. `MAX_FRAME` is 1,048,576. A reader that sees `N == 0` or `N > MAX_FRAME` returns an error +without reading or allocating the body, and the connection is closed. + +**Envelope.** + +```json +{"v":1,"id":7,"final":true,"msg":{"kind":"tool_request","body":{}}} +``` + +- `v` is the protocol version, 1. A reader that sees another value replies with an `error` message + and closes. +- `id` is chosen by the requester. Every frame of the response carries the same `id`. +- `final` is always present. A response may span several frames; only the last has `final: true`. + Requests are single frames with `final: true`. +- `msg` is an adjacently tagged enum. M1 defines `tool_request`, `tool_response` and `error`. Later + milestones add kinds. + +**Strictness.** Unknown fields are rejected everywhere (`deny_unknown_fields`). Enums are tagged +with explicit strings. No floats. No free-form maps. Byte-exact fixture files under +`crates/proto/tests/fixtures/` are the wire specification. + +**Sockets.** Each daemon listens on one socket named after itself. + +| Socket | Listener | Clients | Carries | +|---|---|---|---| +| `infer.sock` | `inferproxy` | `loopd` | Plain HTTP, forwarded byte for byte. Not framed. | +| `broker.sock` | `brokerd` | `loopd`, `bxctl` | Tool requests; local approvals and admin | +| `loop.sock` | `loopd` | `bxctl chat` (M2), `gatewayd` (M4) | The channel protocol, the same for both | +| `gateway.sock` | `gatewayd` | `brokerd` | Approval requests and answers | + +**Peer authentication** is by file permissions. Each pair of roles has its own directory under +`run/` that only those two can enter. In M7 each becomes a shared volume. No tokens or handshakes. + +## 5. Shared types in `proto` + +`proto` holds data types and the frame codec. It holds no policy and performs no I/O except +reading and writing frames on a caller-supplied stream. All types derive `Debug, Clone, PartialEq, +Eq, Serialize, Deserialize` and reject unknown fields. + +```rust +// Identifiers. Constructors and Deserialize validate; invalid input is an error, never a panic. +pub struct SessionId(String); // 1 to 64 chars of [a-z0-9-]; used as a directory name +pub struct Epoch(pub u32); +pub struct CallId(pub u64); // assigned by loopd, increasing within a session +pub struct Hash32([u8; 32]); // JSON: 64 lowercase hex chars +pub struct Timestamp(u64); // Unix milliseconds; JSON: "2026-09-17T08:05:00.000Z" + +#[serde(rename_all = "lowercase")] +pub enum DataClass { Public, Private, Secret } // derives PartialOrd, Ord in this order + +// IPC +pub struct Envelope { pub v: u32, pub id: u64, pub r#final: bool, pub msg: Message } +#[serde(tag = "kind", content = "body", rename_all = "snake_case")] +pub enum Message { ToolRequest(ToolRequest), ToolResponse(ToolResponse), Error(WireError) } +pub struct WireError { pub code: ErrorCode, pub detail: String } +pub enum ErrorCode { BadFrame, BadVersion, BadMessage, Internal } // snake_case strings + +pub struct ToolRequest { + pub session: SessionId, pub call: CallId, pub tool: String, + pub arguments: String, // the model's JSON object, carried unparsed +} +#[serde(tag = "status", rename_all = "snake_case")] +pub enum ToolResponse { + PendingApproval { approval: String, expires: Timestamp }, // never final + Result { content: String, class: DataClass, untrusted: bool, truncated: bool }, + Failed { message: String }, // the tool ran and failed; not a policy decision + Denied { reason: DenyReason }, +} +pub enum DenyReason { NoGrant, GrantExpired, TaintTooHigh, DeniedByGrant, + ApprovalRefused, ApprovalExpired } // snake_case strings + +// Grants: one grant per file, grants/.toml. The id is the file stem. +#[serde(rename_all = "lowercase")] +pub enum Mode { Auto, Ask, Deny } +pub struct Grant { + pub tool: String, + pub mode: Mode, + pub max_taint: DataClass, + pub result_class: DataClass, // default private + pub untrusted: bool, // default true + pub expires: Option, // absent means no expiry + pub secret: Option, // name of one secret to inject + pub constraints: Constraints, // default empty +} +pub struct Constraints { pub paths: Vec, pub hosts: Vec, pub patterns: Vec } +``` + +`Grant` is in `proto` so that `bxctl` can check grant files with the parser `brokerd` uses. +Matching semantics for `Constraints` are specified in M3 and implemented only in `brokerd`. + +```rust +// Audit +#[serde(tag = "outcome", rename_all = "snake_case")] +pub enum DecisionRecord { + Allowed { grant: String }, + Approved { grant: String, approver: String, post: Option }, + Denied { reason: DenyReason, grant: Option }, +} +pub struct AuditRecord { + pub seq: u64, pub time: Timestamp, pub prev: Hash32, + pub session: SessionId, pub call: CallId, pub tool: String, + pub arguments: String, // in full + pub session_taint: DataClass, pub decision: DecisionRecord, +} + +// Session log +pub struct ToolCall { pub id: String, pub name: String, pub arguments: String } +#[serde(tag = "type", rename_all = "snake_case")] +pub enum LogRecord { + SessionStart { time: Timestamp, session: SessionId, epoch: Epoch, slot: u32, baseline: Hash32 }, + User { time: Timestamp, content: String }, + Assistant { time: Timestamp, content: Option, reasoning_content: Option, + tool_calls: Vec }, + ToolResult { time: Timestamp, call: CallId, tool_call_id: String, content: String, + class: DataClass, untrusted: bool, truncated: bool }, + CacheLoss { time: Timestamp, expected: u64, got: u64 }, + EpochEnd { time: Timestamp, next: Epoch, summary: String }, +} +``` + +`Assistant` stores the fields the chat template reads, exactly as the server returned them. M0 +replayed these fields and got full cache hits. + +**`Decision` is not in `proto`.** It is defined in `brokerd::policy` with a private field, and it +implements neither `Deserialize` nor `Clone`. Only `brokerd::policy` can construct one, and the +container runner takes it by value. A `compile_fail` doctest in `brokerd` proves that code outside +the module cannot build one. For doctests to run, every role crate is a library with a thin +`main.rs`. + +## 6. Log files + +Both logs are JSONL written by serde from the types above, through one writer module per log. + +**Audit log, `audit/YYYY-MM-DD.jsonl`.** + +1. One `AuditRecord` per line, for every decision including denials. +2. `hash(record)` is SHA-256 over the exact bytes of the stored line, without the trailing newline. + A verifier hashes stored bytes. It never parses and re-serialises. +3. `prev` is the hash of the previous record. The first record ever written has 32 zero bytes. The + first record of a new file continues from the last record of the previous file. +4. `brokerd` is the only writer. It holds a file lock and calls `fsync` after each record. +5. On start, a torn final line is kept, a recovery record is appended after it, and the chain + continues from the last complete record. Nothing is truncated silently. + +The writer, the hash and the recovery record type are built in M3. + +**Session log, `sessions//.jsonl`.** One `LogRecord` per line, append only. Built in M2. + +## 7. Runtime layout + +- `BOXMAKER_HOME`, default `/var/lib/boxmaker`: `sessions/`, `memory/`, `audit/`, `index/`, and + `run/` with one socket directory per pair of roles. +- `/etc/boxmaker/config.toml` and `/etc/boxmaker/grants/*.toml`. +- In development everything points at `.state/` in the repo, which is git-ignored. +- M1 documents this. M7 adds the NixOS module that creates users and directories. + +## 8. Workspace and gate (M1) + +- Crates live in `crates/`: `proto`, `loopd`, `brokerd`, `gatewayd`, `inferproxy`, `toolkit`, + `bxctl`. Role crates depend only on `proto`. Edition 2024, `rust-version = "1.95"`, + `publish = false`, `unsafe_code = "forbid"` as a workspace lint. +- Code is written and the gate is run on the owner's development machine (rustup, rustc 1.98.1 on + 2026-09-17), then deployed to straylight. `rust-version` is 1.95 because that is the rustc in + straylight's nixpkgs, where M7 will build the deployed binaries. `cargo-deny` 0.20.2 is installed + on the development machine with `cargo install --locked cargo-deny`. A Nix package and dev shell + are left to M7. +- `make gate`, offline: `cargo fmt --check`; `cargo clippy --workspace --all-targets -- -D + warnings`; `cargo test --workspace`; `cargo deny check bans licenses sources`; and three scripts: + no source file over 500 lines, no role crate depends on another role crate, every dependency in + any `Cargo.toml` is named in `docs/dependencies.md`. +- `make audit` runs `cargo deny check advisories`. It fetches the advisory database, so it is not + part of the offline gate and is listed in `docs/egress.md` as development-time egress. +- `make verify-device` is created empty in M1 and filled from M2. + +Dependencies verified on 2026-09-17 (crates.io and docs.rs): + +| Crate | Version | Used for | Notes | +|---|---|---|---| +| `serde` (derive) | 1.0.229 | All `proto` types | MIT OR Apache-2.0 | +| `serde_json` | 1.0.151 | Frames, logs | MIT OR Apache-2.0 | +| `humantime` | 2.4.0 | RFC 3339 timestamps | No runtime dependencies. `format_rfc3339_millis(SystemTime) -> Rfc3339Timestamp`, `parse_rfc3339(&str) -> Result` | +| `toml` | 1.1.6 | Grant files; a dev-dependency of `proto` in M1 | MIT OR Apache-2.0 | + +`trybuild` was considered for the compile-fail test and rejected: it brings a large +dev-dependency tree and its expected-output files differ between compiler versions. A +`compile_fail` doctest needs no dependency. + +## 9. How the implementing model gets the work + +Implementation is done by Laguna S 2.1. The model is served by straylight; OpenCode runs on the +owner's development machine in `~/src/boxmaker`, where the code is written and the gate is run. +Builds are deployed to straylight afterwards. + +1. `AGENTS.md` holds the standing rules and stays under about 1,500 tokens, because OpenCode loads + it in every session. +2. Each task is one file, `docs/plans/M/NN-name.md`, self-contained and under about 3,000 + tokens: goal, exact paths, given signatures, verified crate versions and API snippets, the + tests to write first, commands with expected output, a "done when" list and a "stop and report + if" list. It quotes the part of the brief it needs. +3. One task is one fresh OpenCode session and one commit, on a branch named for the milestone. The + owner starts each session with: "Read `docs/plans/M1/NN-name.md` and do exactly that task." +4. Fixture files that define the wire format are written by the design model and committed with + the plan. For the security-critical parts of M3 the tests are too. +5. Laguna keeps `docs/implementer-log.md`: one row per task with the date, how many gate runs it + took, whether the first run passed, any deviation from the task, and anything it stopped on. +6. Review is once per milestone, by the design model: the branch is checked against the plan, and + the gate is run on the development machine. Findings go back as follow-up tasks for Laguna. Trivial or + blocking findings may be fixed directly, and the log says so. + +## 10. Not covered here + +Channel and approval message bodies (M2 to M4). Grant constraint matching, the tool container +contract and image pinning (M3). The fake `llama-server` for offline tests, the hand-written HTTP +client, and who writes the compaction summary (M2). The repository licence (open decision).