boxmaker/docs/design.md

# Boxmaker design brief

Boxmaker is a sovereign personal agent harness. The name is from *Count Zero*: the remnant AI in
the Straylight cores that patiently assembles small boxes from fragments.

This brief is binding. Changes to it land as their own commit and are recorded in
`docs/decisions.md`. Milestones and session prompts are in `docs/milestones.md`.

Crate names and llama.cpp server parameters below are from memory or secondhand research.
Verify each one against primary docs before use.

## Goal

A personal agent harness that maximizes sovereignty: local inference by default, no ambient
authority, state in plain files the owner can read, a codebase small enough to audit.

## Target environment

- Inference host: `straylight`, AMD Strix Halo, 128 GB unified memory, of which the GPU may
  address 104 GiB. The harness runs on the same host.
- Serving: llama.cpp `llama-server` in router mode. One endpoint, one child server per model, at
  most two models loaded. The owner's coding agents and Open WebUI use the same router and the
  same Ornith instance. Details and measurements are in `docs/inference-contract.md`.
- Model: Ornith-1.5-35B-A3B (Qwen3.5-family MoE, hybrid linear/full attention, Qwen XML tool calls
  with thinking blocks). As deployed: Q4_K_M weights, q8_0 KV cache, two slots of 131,072 tokens.
  Weights and cache are unloaded after six idle hours, or when a third model is loaded.
- Memory is not abundant: Laguna S 2.1 (69 GB) and Ornith (22 GB) are normally both loaded.
  Generation is fast. **Prompt processing is slow and cache invalidation is the dominant cost.**
  Hybrid-attention models cannot partially rewind their cache; M0 confirmed it.
- Measured in M0 with the GPU otherwise idle: prompt processing 1,170 tokens/s at depth 0 and 669
  at 32k depth; generation 69 tokens/s falling to 60. A second active session on Ornith roughly
  halves generation speed. An uncached 32k-token prompt took 36 s, so a 64k re-read costs well
  over a minute.
- Deployment: rootless containers on a host that is already a tailnet node. Containers do not get
  their own tailnet identities, so tailnet ACLs cannot tell one role from another. Isolation
  between roles is done with container networking on the host (see "Network isolation"). Tailnet
  ACLs restrict what the host as a whole may reach.
- First channel: the owner's existing self-hosted Mattermost server, reached over the tailnet.

## Non-goals (v0)

Plugin registry, skill marketplace, web dashboard, multi-user, voice, more than one chat channel,
cloud-led orchestration as the default, in-process third-party code of any kind.

## Roles (separate binaries, narrow interfaces)

| Role | Purpose | Holds | Can reach | Never holds |
|---|---|---|---|---|
| `gatewayd` | Carries messages between the owner's Mattermost and the harness. | Mattermost bot token, allowlist, message queue | Mattermost server (outbound only), `loopd` socket | Tool secrets, grants |
| `loopd` | Runs the conversation: builds each prompt, calls the model, and asks `brokerd` for every tool call. | Session logs, prompt assembly | Four Unix sockets: `gatewayd`, `brokerd`, `inferproxy`, `memoryd` | Any credential, any network interface |
| `inferproxy` | Lets `loopd` reach the model without having a network. | The `llama-server` address | `llama-server` only | Anything else; it forwards bytes and logs nothing |
| `brokerd` | Decides whether each tool call may run, runs it in a fresh container, and records the decision. | Grants (read-only), secrets, audit log | Container runtime, `gatewayd` socket (for approvals) | Conversation history |
| `memoryd` | Keeps the memory files and their index, syncs finished turns to Honcho, and runs every memory tool call `brokerd` has decided. | `memory/`, `index/`, Honcho's address and workspace list (and its key, if any) | Honcho only; sockets from `loopd` (finished turns in, the memory files out once per epoch) and from `brokerd` (memory tool calls); `sessions/` read-only, for reindexing | Grants, tool secrets, the Mattermost token |
| tool containers | Do one approved tool call and then cease to exist. | Only what one call was granted | Only what one call was granted | Anything persistent |

v0 runs all roles on one host. The interfaces must not assume co-location with `llama-server`.

## Network isolation

The host's tailnet identity is shared by every container that has a network, so a container with
ordinary outbound networking can reach the whole tailnet and the internet as the host.

1. `loopd` runs with no network namespace interfaces except loopback (`--network=none`). Its only
   I/O is four Unix sockets on a shared volume.
2. `inferproxy` is a forwarder of about 100 lines: one Unix socket in, one fixed upstream out. The
   upstream is the shared router's TCP listener on the host. It is required: `loopd` has no
   network, and the router stays on TCP for its other clients.
3. `gatewayd` has outbound network only. It opens no listening TCP port. Approvals are commands
   (`!approve`, `!deny`) replied in the approval's thread, which arrive over the Mattermost
   WebSocket, so no callback URL is needed.
4. `memoryd` reaches Honcho and nothing else. It does not index sessions above a configured data
   class.
5. Tool containers get no network unless the grant names hosts. Granted network access goes through
   an allowlisting egress proxy that `brokerd` configures per call.
6. Tailnet ACLs (kept in `deploy/`) limit the host's node to `llama-server`, Mattermost and Honcho,
   plus whatever tool grants need. This is defence in depth, not the role boundary.

## Channel: Mattermost

- Bot account with a token held in the `SecretStore`. REST API for posting, WebSocket for events.
  Implement directly over HTTP and WebSocket; do not adopt a community SDK without checking its
  maintenance. Fetch the Mattermost API reference before writing any request.
- Identity is the Mattermost user ID. The allowlist is a config file of user IDs. Direct messages
  only by default; channels must be allowlisted by ID. Authentication is delegated to the server.
- Long tasks: post a placeholder, edit or reply when finished. Threads map to sessions.
- Mattermost stores messages unencrypted in its database. That is acceptable because the server is
  the owner's, and it is recorded in `docs/egress.md`.

## Inference contract (the part existing harnesses get wrong)

1. **Baseline budget.** System prompt, tool schemas and the three memory files at session start:
   3,000 tokens or less, measured with the server's tokenizer (`/tokenize`), enforced by a test.
   That test needs straylight, so it runs in `make verify-device`, not in the offline `make gate`.
2. **Append-only.** The request for turn N+1 is a strict extension of the request for turn N.
   Nothing volatile (time, heartbeat notes, memory refreshes) goes anywhere but the newest message.
   Enforced by a property test on the serialized message array. The session log stores each
   assistant message exactly as the server returned it (`content`, `reasoning_content`,
   `tool_calls`) and replays it unchanged. Ornith's template keeps every thinking block, so
   dropping or editing one changes the prefix.
3. **No in-place pruning.** Tool results are size-capped when first appended, never trimmed later.
4. **Compaction is an epoch change.** After a turn, when that turn's `cache_n + prompt_n` is at
   or above a configured fraction of `n_ctx` (default 0.85), it writes a summary and starts a new
   epoch whose prefix is the baseline plus the summary. The session is not in another turn when
   this runs. The old log is kept. A quiet session stays on its current epoch.
5. **Pinned slots, not reserved.** Every request carries `id_slot`, and a session always uses the
   same slot: moving it costs a full re-read. Slot assignment is configuration. With the two slots
   deployed today, main sessions (one per Mattermost thread) share one slot, and subagents and
   scheduled runs share the other. The server is shared, so another client can take a harness
   slot and the router can unload Ornith. An evicted session is normally restored from the
   server's host-RAM prompt cache when it returns on its slot; an unloaded model is not. Cache
   loss is therefore an expected event: `loopd` detects it (`cache_n` far below the previous
   request's total), records it in the session log and carries on. It is never an error.
6. **Progressive disclosure.** The `tools` array is fixed for the whole epoch. The template renders
   it at the top of the prompt, so any change re-reads everything. Only a small core tool set has
   schemas in it. Everything else is found through `find_tool`, which returns schemas as an
   ordinary tool result, and is called through one fixed `call_tool(name, arguments)` meta-tool.
   `brokerd` applies grants to the target tool, not to `call_tool`. Never prompt the model to call
   a tool that is not in the `tools` array: the server's grammar forces such a call into a
   declared tool.
7. **Liveness, not deadlines.** Streaming always, with `return_progress: true` so that prompt
   processing produces events. The timeout is "no bytes for N seconds", never a total-request
   deadline. Progress events count as bytes. Before the first byte, a request may be queued behind
   another client on its slot or waiting for the router to load the model. A separate, longer
   limit covers that wait, and the liveness timer starts at the first byte. M2 measures what the
   stream sends while a request is queued.
8. **Runaway control.** Per-turn thinking-token cap, per-turn tool-iteration cap, detection of
   repeated identical tool calls. `loopd` enforces the thinking cap: it counts streamed reasoning
   tokens and ends the block with the server's `reasoning_control` mechanism
   (`POST /v1/chat/completions/control`, action `reasoning_end`). M2 exercises this on straylight
   before relying on it.
9. **Startup self-test.** On boot `loopd` checks: tool-call round trip parses, turn-2 prompt
   processing count shows a cache hit, configured context matches what the server reports.
   It refuses to start if any check fails.
10. **Serving settings are recorded, not owned.** The launch flags live in the owner's NixOS
    configuration (`~/src/nixos` on straylight). This repo records the expected values: chat
    template hash, per-slot context, slot count, q8_0 KV cache, no speculative decoding. The
    startup self-test compares them with `/props` and `/slots`. Sampling settings (temp 0.6,
    top-p 0.95, top-k 20) are sent with every request, because the server's default temperature
    is 1.0.

Settled by M0 (`docs/inference-contract.md`): `loopd` uses the server's chat-completions endpoint
with server-side tool parsing. It does not render the template in-process. Cache reuse held across
a multi-turn tool conversation and tool parsing had no failures in 20 trials.

## Authority contract

1. `loopd` has no authority. Every tool call goes to `brokerd` as a request.
2. A **grant** is a file the owner writes: tool, argument constraints (paths, hosts, patterns),
   allowed data classes, expiry, mode (`auto`, `ask`, `deny`). `brokerd` reads grants and cannot
   write them. `loopd` cannot see the directory.
3. No matching grant means deny. `ask` routes an approval request to the owner through `gatewayd`
   and the turn suspends until answered or expired.
4. Each approved call runs in a fresh rootless container: no network unless granted, only granted
   paths mounted, only the granted secret injected. Tool containers run from one image built from
   source by Nix and named by digest; nothing is pulled at call time.
5. **Audit log**: append-only JSONL, hash-chained, one record per decision including denials.
6. **Data classes**: every tool result is labelled `public`, `private`, or `secret`. A session's
   taint is the union of labels it has seen. The cloud-consult tool is refused for tainted sessions
   unless a grant allows that class out. Every outbound cloud payload is logged in full.
7. Scheduled jobs are re-validated against current grants on every run.

## Cloud consult (M6)

- One tool, `consult`, behind `brokerd` like any other. The endpoint is any OpenAI-compatible URL,
  set in config. Nothing in the code is specific to one vendor.
- Candidate as of 2026-09-16: GLM-5.3-Flash (open weights, MIT, 320B total / 18B active). It does
  not run usefully on the inference host, so it is cloud-only here. Hosting choices: Z.ai's
  international API (Singapore entity, Singapore law, states it does not store API content), or a
  third-party host chosen by jurisdiction with zero-data-retention routing. Z.ai's subscription
  coding plan is restricted to its officially supported tools; use pay-as-you-go.
- Call it at temperature 0.6 or lower. Community tests show malformed tool calls at the default
  of 1.0. Prefer giving `consult` no tools at all: it answers a self-contained question in text.
- Hosted GLM always thinks. Treat `consult` as slow and asynchronous.
- Payloads are built from an explicit allowlist of fields, never from raw session history.

## Memory (M5)

Memory is what carries continuity across epoch compaction, so it is required. It must not break
the inference contract, and it must not become a way for injected text to persist.

1. **Identity and experience.** Memory is two things. **Identity** is who the agent is and what
   it holds true about its work and its owner: three files read once per epoch into the baseline,
   in this order after the system prompt: `memory/soul.md`, the persona, written only by the owner
   (at most 4,000 characters); `memory/memory.md`, the agent's core notes on its work (at most
   2,200 characters); and `memory/user.md`, its core notes on the owner (at most 1,375
   characters), each shown with how full it is. **Experience** is what it has seen or been told:
   `memory/notes.md` (append-only) and the session logs, never in the prompt, found with
   `recall`, each item labelled with its source and trust. Not everything remembered is believed.
   The agent writes memory with a `memory` tool, without asking: `note` records something seen,
   in any session; `add`, `replace`, `remove` and `consolidate` (one edit marking every entry of a
   file kept, rewritten, moved or dropped) edit identity, and only a session that has not read
   untrusted content may use them. In a session that has, an identity `add` becomes a `note` and
   the other edits are refused. What leaves an identity file goes to `notes.md` as moved or
   dropped; nothing is deleted. A write takes effect at the next epoch, never mid-session. A
   write over a file's limit is refused with a pointer to `consolidate`.
2. **Everything else is retrieved, not injected.** A `recall(query, k)` tool returns size-capped
   snippets, each with a source tag (`identity` for entries moved out of identity, `memory` for
   trusted experience, `memory-tainted` for experience from sessions or workspaces that are not
   trusted), where it came from, and its class. An `ask_memory` tool asks Honcho's dialectic.
   Results are appended like any tool result, so the cache prefix is untouched.
3. **Local index.** SQLite FTS5 over the session logs and `memory/notes.md`, kept by `memoryd`
   and rebuildable with `bxctl reindex`. It needs no network and no model, so it still answers
   when Honcho is down. Session logs contribute the owner's messages, the assistant's replies,
   tool calls by name and arguments, and `ToolSummary` records for tools that opt in (a web
   fetch), labelled as the tool result was; raw tool results are not indexed. `memoryd` counts
   `recall` hits per note and when each was last found. Retrieval sits behind a `Retriever` trait.
4. **Honcho.** The fleet's memory service; it and the models it calls run on the owner's hosts.
   Each Boxmaker writes finished turns (the speaker's message and its own final reply, from their
   own peers) to two workspaces of its own: a trusted one for turns from sessions that saw no
   untrusted content, an untrusted one for the rest. Nothing above `private` is written, and a
   session that reached `secret` is never synced; `memoryd` enforces this itself, asking
   `brokerd` for the session's state when the turn ends. Every trusted memory write is mirrored
   there as a conclusion at once; untrusted notes go, as messages, to one workspace the whole
   fleet shares. The owner and other people are observed peers; the agent is not, so Honcho draws
   no conclusions from its replies. Honcho's own extraction, summaries and consolidation replace a
   flush job of ours. Honcho is not rebuilt from the logs: what it holds wrongly is corrected by
   writing a contrary fact, or, for a conclusion `memoryd` wrote, by deleting it.
5. **Workspaces and trust.** `memoryd` reads other agents' trusted workspaces, which the owner
   marks trusted or untrusted (untrusted is the default), and the fleet workspace, which is always
   untrusted. Results from an untrusted workspace are marked untrusted and taint the session.
   Honcho's conclusions carry no per-entry provenance, so the filter is applied when writing and
   the label is per workspace.
6. **Provenance on recall.** Recalled text is presented as data with its source and data class,
   never placed in the system prompt, and taints the session with that class; a `memory-tainted` hit
   marks the session untrusted.
7. **The tools go through `brokerd`** and run in `memoryd`. Their policy is built in, and no grant
   file can change it: `recall`, `ask_memory` and `memory` are all `auto`. What protects the
   prompt is which sessions may edit identity (item 1), plus a check on every write (no control
   or invisible characters, no chat-template markup, a short list of instruction phrases), run
   again when the prompt is built. Every write is journalled in `memory/edits/`, listed by
   `bxctl memory log`, and the latest edit of a file can be undone. An owner setting stages
   identity writes for review instead (off by default).
8. **No memory text leaves the owner's hosts.** Honcho and the models it calls run on the owner's
   hosts, and each is listed in `docs/egress.md`.

## State

Files are the source of truth. SQLite is allowed only for rebuildable indexes and queues.

- `sessions/<id>/<epoch>.jsonl` — append-only session log
- `memory/soul.md`, `memory/memory.md`, `memory/user.md`, `memory/notes.md`, `memory/edits/` — owner-readable memory
- `index/` — FTS5, kept by `memoryd`; disposable and rebuildable from `memory/` and `sessions/`
- `grants/*.toml` — owner-written
- `audit/*.jsonl` — hash-chained
- `broker/sessions/<id>.json` — each session's taint and untrusted flag, written only by
  `brokerd`; it can be rebuilt from the audit log's `result` records
- Secrets: behind a `SecretStore` trait with three backends, chosen per secret: a systemd credential
  (encrypted at rest; the key is the host's TPM and host key, not stored beside it), an environment
  variable, or a file that only the owner can read. The credential backend is the default; the
  other two put the secret in plaintext, in the process environment or on disk, and are for
  portability; a file secret prints a warning at startup. None in config, none in the repo.

## Code constraints

- Rust stable, Cargo workspace. One crate per role in the table above (`loopd`, `brokerd`,
  `gatewayd`, `inferproxy`, `memoryd`), plus three that are not roles:

  | Crate | Purpose |
  |---|---|
  | `bxcore` | Defines the data every role shares and the frame format they exchange it in, and holds the code roles share: the secret store and the sanitizer. Called `proto` until M5b. |
  | `toolkit` | Holds the programs that run inside tool containers. |
  | `bxctl` | Is the owner's command line: chat, local approvals, checks and maintenance. |
- No source file over 500 lines. No crate depends on another role's crate, only on `bxcore`.
- Authority is encoded in types: a tool cannot execute without a `Decision` value that only
  `brokerd`'s policy module can construct. `Decision` is defined in `brokerd`, has a private
  field and does not implement `Deserialize`, so no other crate or wire message can produce one.
  `bxcore` carries a plain `DecisionRecord` for the audit log and the wire.
- Dependencies are few and justified in `docs/dependencies.md`. `cargo-deny` runs in the gate.
- `make gate` runs offline. Checks that need straylight run in `make verify-device`.
- No telemetry, no update checks, no outbound call not listed in `docs/egress.md`.