boxmaker/docs/design.md

# Boxmaker design brief

Boxmaker is a sovereign personal agent harness. The name is from *Count Zero*: the remnant AI in
the Straylight cores that patiently assembles small boxes from fragments.

This brief is binding. Changes to it land as their own commit and are recorded in
`docs/decisions.md`. Milestones and session prompts are in `docs/milestones.md`.

Crate names and llama.cpp server parameters below are from memory or secondhand research.
Verify each one against primary docs before use.

## Goal

A personal agent harness that maximizes sovereignty: local inference by default, no ambient
authority, state in plain files the owner can read, a codebase small enough to audit.

## Target environment

- Inference host: `straylight`, AMD Strix Halo, 128 GB unified memory, llama.cpp `llama-server`,
  ~128k context configured.
- Model: Ornith-1.5-35B-A3B (Qwen3.5-family MoE, hybrid linear/full attention, ~20 KB/token KV
  at f16, Qwen XML tool calls with thinking blocks).
- Memory is abundant. Generation is fast. **Prompt processing is slow and cache invalidation is
  the dominant cost.** Hybrid-attention models cannot partially rewind their cache.
- Reference numbers for the same model family on Strix Halo (Qwen3.6-35B-A3B Q4, llama.cpp,
  community benchmark grid, May 2026): prompt processing ~1,100 tokens/s at depth 0 and ~700 at
  32k depth; generation ~60 tokens/s falling to ~49. M0 replaces these with measurements from
  straylight. At these rates an uncached 18k-token prompt costs 16–25 s and a 64k re-read costs
  well over a minute.
- Deployment: rootless containers on a host that is already a tailnet node. Containers do not get
  their own tailnet identities, so tailnet ACLs cannot tell one role from another. Isolation
  between roles is done with container networking on the host (see "Network isolation"). Tailnet
  ACLs restrict what the host as a whole may reach.
- First channel: the owner's existing self-hosted Mattermost server, reached over the tailnet.

## Non-goals (v0)

Plugin registry, skill marketplace, web dashboard, multi-user, voice, more than one chat channel,
cloud-led orchestration as the default, in-process third-party code of any kind.

## Roles (separate binaries, narrow interfaces)

| Role | Holds | Can reach | Never holds |
|---|---|---|---|
| `gatewayd` | Mattermost bot token, allowlist, message queue | Mattermost server (outbound only), `loopd` socket | Tool secrets, grants |
| `loopd` | Session logs, prompt assembly, memory files | Three Unix sockets: `gatewayd`, `brokerd`, `inferproxy` | Any credential, any network interface |
| `inferproxy` | The `llama-server` address | `llama-server` only | Anything else; it forwards bytes and logs nothing |
| `brokerd` | Grants (read-only), secrets, audit log | Container runtime, `gatewayd` socket (for approvals) | Conversation history |
| tool containers | Only what one call was granted | Only what one call was granted | Anything persistent |

v0 runs all roles on one host. The interfaces must not assume co-location with `llama-server`.

## Network isolation

The host's tailnet identity is shared by every container that has a network, so a container with
ordinary outbound networking can reach the whole tailnet and the internet as the host.

1. `loopd` runs with no network namespace interfaces except loopback (`--network=none`). Its only
   I/O is three Unix sockets on a shared volume.
2. `inferproxy` is a forwarder of about 100 lines: one Unix socket in, one fixed upstream out. If
   M0 shows `llama-server` can listen on a Unix socket and it runs on the same host, drop
   `inferproxy` and mount that socket instead.
3. `gatewayd` has outbound network only. It opens no listening TCP port. v0 approvals use replies
   or emoji reactions, which arrive over the Mattermost WebSocket, so no callback URL is needed.
4. Tool containers get no network unless the grant names hosts. Granted network access goes through
   an allowlisting egress proxy that `brokerd` configures per call.
5. Tailnet ACLs (kept in `deploy/`) limit the host's node to `llama-server` and Mattermost, plus
   whatever tool grants need. This is defence in depth, not the role boundary.

## Channel: Mattermost

- Bot account with a token held in the `SecretStore`. REST API for posting, WebSocket for events.
  Implement directly over HTTP and WebSocket; do not adopt a community SDK without checking its
  maintenance. Fetch the Mattermost API reference before writing any request.
- Identity is the Mattermost user ID. The allowlist is a config file of user IDs. Direct messages
  only by default; channels must be allowlisted by ID. Authentication is delegated to the server.
- Long tasks: post a placeholder, edit or reply when finished. Threads map to sessions.
- Mattermost stores messages unencrypted in its database. That is acceptable because the server is
  the owner's, and it is recorded in `docs/egress.md`.

## Inference contract (the part existing harnesses get wrong)

1. **Baseline budget.** System prompt plus tool schemas at session start: 3,000 tokens or less,
   measured with the server's tokenizer, enforced by a test.
2. **Append-only.** The request for turn N+1 is a strict extension of the request for turn N.
   Nothing volatile (time, heartbeat notes, memory refreshes) goes anywhere but the newest message.
   Enforced by a property test on the serialized message array. The session log stores each
   assistant message exactly as the server returned it (`content`, `reasoning_content`,
   `tool_calls`) and replays it unchanged. Ornith's template keeps every thinking block, so
   dropping or editing one changes the prefix.
3. **No in-place pruning.** Tool results are size-capped when first appended, never trimmed later.
4. **Compaction is an epoch change.** It happens only when the session is idle, writes a summary,
   and starts a new epoch whose prefix is the baseline plus the summary. The old log is kept.
5. **Pinned slots.** Main session, subagents, and scheduled runs each use their own server slot so
   they do not evict each other's cache. Verify the pinning mechanism on the deployed build.
6. **Progressive disclosure.** The `tools` array is fixed for the whole epoch. The template renders
   it at the top of the prompt, so any change re-reads everything. Only a small core tool set has
   schemas in it. Everything else is found through `find_tool`, which returns schemas as an
   ordinary tool result, and is called through one fixed `call_tool(name, arguments)` meta-tool.
   `brokerd` applies grants to the target tool, not to `call_tool`. Never prompt the model to call
   a tool that is not in the `tools` array: the server's grammar forces such a call into a
   declared tool.
7. **Liveness, not deadlines.** Streaming always, with `return_progress: true` so that prompt
   processing produces events. The timeout is "no bytes for N seconds", never a total-request
   deadline. Progress events count as bytes.
8. **Runaway control.** Per-turn thinking-token cap, per-turn tool-iteration cap, detection of
   repeated identical tool calls. `loopd` enforces the thinking cap: it counts streamed reasoning
   tokens and ends the block with the server's `reasoning_control` mechanism
   (`POST /v1/chat/completions/control`, action `reasoning_end`). M2 exercises this on straylight
   before relying on it.
9. **Startup self-test.** On boot `loopd` checks: tool-call round trip parses, turn-2 prompt
   processing count shows a cache hit, configured context matches what the server reports.
   It refuses to start if any check fails.
10. **Serving settings are part of the repo.** Launch flags, chat template hash, sampling settings
    (temp 0.6, top-p 0.95, top-k 20), f16 KV cache, no speculative decoding.

Open question settled by measurement in M0: use the server's chat-completions endpoint with
server-side tool parsing, or render the template in-process for byte-exact prefix control. Default
to chat-completions. Switch only if the M0 cache measurements fail.

## Authority contract

1. `loopd` has no authority. Every tool call goes to `brokerd` as a request.
2. A **grant** is a file the owner writes: tool, argument constraints (paths, hosts, patterns),
   allowed data classes, expiry, mode (`auto`, `ask`, `deny`). `brokerd` reads grants and cannot
   write them. `loopd` cannot see the directory.
3. No matching grant means deny. `ask` routes an approval request to the owner through `gatewayd`
   and the turn suspends until answered or expired.
4. Each approved call runs in a fresh rootless container: no network unless granted, only granted
   paths mounted, only the granted secret injected.
5. **Audit log**: append-only JSONL, hash-chained, one record per decision including denials.
6. **Data classes**: every tool result is labelled `public`, `private`, or `secret`. A session's
   taint is the union of labels it has seen. The cloud-consult tool is refused for tainted sessions
   unless a grant allows that class out. Every outbound cloud payload is logged in full.
7. Scheduled jobs are re-validated against current grants on every run.

## Cloud consult (M6)

- One tool, `consult`, behind `brokerd` like any other. The endpoint is any OpenAI-compatible URL,
  set in config. Nothing in the code is specific to one vendor.
- Candidate as of 2026-09-16: GLM-5.3-Flash (open weights, MIT, 320B total / 18B active). It does
  not run usefully on the inference host, so it is cloud-only here. Hosting choices: Z.ai's
  international API (Singapore entity, Singapore law, states it does not store API content), or a
  third-party host chosen by jurisdiction with zero-data-retention routing. Z.ai's subscription
  coding plan is restricted to its officially supported tools; use pay-as-you-go.
- Call it at temperature 0.6 or lower. Community tests show malformed tool calls at the default
  of 1.0. Prefer giving `consult` no tools at all: it answers a self-contained question in text.
- Hosted GLM always thinks. Treat `consult` as slow and asynchronous.
- Payloads are built from an explicit allowlist of fields, never from raw session history.

## Memory (M5)

Memory is what carries continuity across epoch compaction, so it is required. It must not break
the inference contract, and it must not become a way for injected text to persist.

1. **Core block.** `memory/core.md`, owner-curated, 300 tokens or less, part of the baseline.
   Edits take effect at the next epoch, never mid-session. The agent can propose changes; only an
   `ask` approval or the owner's editor applies them.
2. **Everything else is retrieved, not injected.** A `recall(query, k)` tool returns size-capped
   snippets with file, line and provenance. Results are appended like any tool result, so the
   cache prefix is untouched.
3. **Notes.** `memory/notes/YYYY-MM-DD.md`, append-only, written by a `remember` tool. Each entry
   records its session, time, data-class label and whether the session had seen untrusted content.
4. **Flush before compaction.** When a session goes idle and is about to change epoch, a job on
   the scheduled slot extracts durable facts from the log into notes. The main slot is not used.
5. **Provenance on recall.** Recalled text is presented as data with its provenance, never placed
   in the system prompt. Entries written from sessions that saw untrusted content are marked as
   such when recalled. Recalled entries carry their data-class label and taint the session.
6. **Both tools go through `brokerd`** under ordinary grants: `recall` and note-writing `auto`,
   core and topic edits `ask`.
7. **Index.** Lexical search first: SQLite FTS5 over the Markdown files, rebuildable with
   `bxctl reindex`. Retrieval sits behind a `Retriever` trait.
8. **Embeddings are a later, additive index** (M5b), never a store. Local model only, served by a
   separate `llama-server` embedding instance so chat slots are unaffected. Vectors live in a flat
   file or SQLite table keyed by chunk hash and embedding-model hash; a model change triggers a
   rebuild. Combine with lexical results by rank fusion. Add M5b when logged `recall` misses show
   lexical search failing, not before.
9. **No vector database service.** Search is exact brute-force cosine, in-process. Revisit only if
   the index passes about one million chunks or measured `recall` latency exceeds one second, and
   then prefer an embedded, file-backed approximate index over a server.

## State

Files are the source of truth. SQLite is allowed only for rebuildable indexes and queues.

- `sessions/<id>/<epoch>.jsonl` — append-only session log
- `memory/core.md`, `memory/notes/*.md`, `memory/topics/*.md` — owner-readable memory
- `index/` — FTS5 and, later, vectors; disposable and rebuildable from `memory/`
- `grants/*.toml` — owner-written
- `audit/*.jsonl` — hash-chained
- Secrets: behind a `SecretStore` trait. v0 backend is an encrypted file whose key is not stored
  beside it. No plaintext secrets on disk, none in config, none in the repo.

## Code constraints

- Rust stable, Cargo workspace. Crates: `proto` (shared types), `loopd`, `brokerd`, `gatewayd`,
  `inferproxy`, `toolkit` (tool container entrypoints), `bxctl` (owner CLI).
- No source file over 500 lines. No crate depends on another role's crate, only on `proto`.
- Authority is encoded in types: a tool cannot execute without a `Decision` value that only
  `brokerd`'s policy module can construct.
- Dependencies are few and justified in `docs/dependencies.md`. `cargo-deny` runs in the gate.
- No telemetry, no update checks, no outbound call not listed in `docs/egress.md`.