From 4a510fde08656c262bf96f20721ebd98fadca273 Mon Sep 17 00:00:00 2001 From: "K. Isom" Date: Wed, 23 Sep 2026 17:42:45 -0700 Subject: [PATCH] M4a spec: approved, with a file-secret warning and multi-agent channels A secret read from a file warns at startup. In allowed channels and group messages, Boxmaker answers only posts that name it, or replies in its own threads that name nobody else; one Boxmaker per machine, each its own bot. Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/decisions.md | 1 + docs/specs/2026-09-23-m4a-gateway.md | 81 ++++++++++++++++++++++------ 2 files changed, 65 insertions(+), 17 deletions(-) diff --git a/docs/decisions.md b/docs/decisions.md index 11587c8..9a35ff4 100644 --- a/docs/decisions.md +++ b/docs/decisions.md @@ -6,6 +6,7 @@ Newest first. A decision that changes `docs/design.md` lands in the same commit | Date | Decision | Reason | |---|---|---| +| 2026-09-23 | M4a spec approved with two changes: a secret read from a file prints a warning at startup (and still starts); and in allowed channels and group messages shared with other agents, Boxmaker answers a post only if it names this bot, or it is a reply in one of this Boxmaker's threads that names no other user (`@channel`, `@here`, `@all` name nobody). One Boxmaker per machine, each with its own bot account; nothing is shared between them. | Owner's notes. The owner runs several agents in channels, possibly one Boxmaker per machine. A post's `from_bot` property is user-settable in v11.11.0, so bots are told apart by the allowlist, not by that property. | | 2026-09-23 | M4 is split: M4a is conversations over Mattermost (`gatewayd`, the secret store), M4b is approvals over Mattermost. Spec `docs/specs/2026-09-23-m4a-gateway.md`. | As with M2 and M3: the authority-bearing half is reviewed separately, and the open question about long approvals lands in M4b. | | 2026-09-23 | `gatewayd` reaches Mattermost over TLS, by default at the tailnet name `https://straylight.scylla-hammerhead.ts.net`; plain HTTP stays possible by URL. TLS is `rustls` 0.23 with `ring` and `rustls-native-certs` (the host's trust store, plus an optional CA file); `deny.toml` gains ISC and BSD-3-Clause. | Owner: portability, and a bigger picture that favours the tailnet name over host loopback. The host's store keeps a privately installed CA trusted; `webpki-roots` would not, and `aws-lc-rs` is a larger C build for nothing needed. | | 2026-09-23 | Our own blocking WebSocket client (RFC 6455), adversarially tested; SHA-1 for its handshake check lives in `proto`. | `tungstenite` would add about ten crates for a few hundred lines of protocol. | diff --git a/docs/specs/2026-09-23-m4a-gateway.md b/docs/specs/2026-09-23-m4a-gateway.md index 172745b..637da76 100644 --- a/docs/specs/2026-09-23-m4a-gateway.md +++ b/docs/specs/2026-09-23-m4a-gateway.md @@ -1,6 +1,7 @@ # M4a design: `gatewayd`, conversations over Mattermost -Status: draft for owner review, 2026-09-23. M4 is split in two (`docs/decisions.md`): M4a is the +Status: approved by the owner, 2026-09-23, with two changes folded in (a warning for a file +secret; channels shared with other agents, section 7). M4 is split in two (`docs/decisions.md`): M4a is the conversation path (this document), M4b is approvals over Mattermost (summarised in section 12, specified after M4a is built). Where this document and `docs/design.md` disagree, the brief wins; the one change it needs, P15 on secrets, is proposed in `docs/decisions.md`. @@ -24,12 +25,18 @@ source and server source at the server's version, v11.11.0: `{"channel_id", "parent_id"}` and shows the user as typing, in that thread. - The web client does not post a message that starts with `/`: it treats it as a slash command. Commands to Boxmaker therefore start with `!` (section 7). +- A post's `from_bot` property is settable by any user (the server's own comment, v11.11.0), so it + does not say who is a bot. Mentions are added to events per connection; `gatewayd` finds its own + `@username` in the message text instead. +- The owner runs several agents in Mattermost channels, and may run one Boxmaker on each of several + machines (at most one per machine), each with its own bot account. ## 1. What M4a proves | Claim | Checked by | |---|---| | A direct message from an allowlisted user becomes a turn, and the answer is posted in its thread | Tests against a fake Mattermost; on straylight | +| In an allowed channel shared with other agents, Boxmaker answers only posts that name it, or replies in its own threads that name nobody else | Tests; on straylight in a channel | | Anyone else gets no response at all: no post, no typing | Tests; on straylight with a second user | | A thread is one session; a new top-level message is a new session | Tests | | Messages sent while a turn runs are sent together as the next turn | Tests | @@ -39,9 +46,9 @@ source and server source at the server's version, v11.11.0: | TLS verifies: an unknown CA or a wrong name is refused | Tests with a test-only CA | | Secrets come from a systemd credential, an environment variable or an owner-only file, and never appear in a log | Tests | -Out of scope: approvals over Mattermost (M4b), channels other than direct messages unless -allowlisted by id, files and images, edits and deletions of posts (an edited message is not -re-sent), memory and scheduled jobs (M5). +Out of scope: approvals over Mattermost (M4b), channels not allowlisted by id, the earlier posts of +a thread Boxmaker is first named in (only the naming post reaches it), files and images, edits and +deletions of posts (an edited message is not re-sent), memory and scheduled jobs (M5). ## 2. Components @@ -94,7 +101,7 @@ credential = "mattermost-token" # exactly one of: credential, env, file [allow] users = ["abcdefghijklmnopqrstuvwxyz"] # Mattermost user ids (26 characters) -channels = [] # non-direct channels allowed, by id; default none +channels = [] # channels and group messages allowed, by id; default none [loop] socket = "" # empty: /run/loop/loop.sock @@ -113,6 +120,9 @@ dead_after_ms = 60000 # no traffic at all for this long: reconnec channel id is 26 characters of `[a-z0-9]`. An empty `allow.users` is a configuration error: a gateway that answers nobody is a mistake. +Allowlisting another agent's user id lets that agent talk to Boxmaker. Two agents that allowlist +each other will answer each other without end; nothing prevents it, so do not. + ## 4. Secrets The brief's `SecretStore`, with three backends, chosen per secret (P15): @@ -123,7 +133,12 @@ The brief's `SecretStore`, with three backends, chosen per secret (P15): | `env = ""` | the environment variable | unset or empty | | `file = ""` | the file | not an absolute path; a symbolic link; not a regular file; not owned by the user `gatewayd` runs as; any group or other permission bit set (anything but 0600 or 0400) | -The value has one trailing newline removed; an empty value is refused. `Secret` holds it in a +The value has one trailing newline removed; an empty value is refused. + +A secret read from a **file** is in plaintext on disk. `gatewayd` starts, and prints at startup, for +each such secret: `gatewayd: warning: secret is read in plaintext from ; a systemd +credential keeps it encrypted at rest (see docs/runbook.md#secret-in-a-file)`. It is a warning, +not a refusal: there are deployments where a file is right. `Secret` holds it in a `zeroize::Zeroizing`, has no `Display`, and its `Debug` prints `Secret(…)`: it cannot reach a log by accident. `Secret::expose(&self) -> &str` is the only way to the text, used only to build the `Authorization` header. @@ -177,21 +192,45 @@ ignored. Then, in this order: 1. Ignore it if `user_id` is the bot's own (from `GET /api/v4/users/me` at startup), if `type` is not empty (a system message), or if its id was handled already (section 9). -2. Ignore it, **silently**, unless `user_id` is in `allow.users` **and** the channel is a direct - message (`data.channel_type == "D"`) or its id is in `allow.channels`. No reply, no typing, no - log line with the message's text (one line with the user id and post id is enough). -3. The session: its root is `root_id`, or the post's own `id` when `root_id` is empty; the session +2. Ignore it, **silently**, unless `user_id` is in `allow.users`. No reply, no typing, no log line + with the message's text (one line with the user id and post id is enough). +3. Is it for this Boxmaker? By the channel: + - A direct message with the bot (`data.channel_type == "D"`): yes, always. + - A channel or a group message (`"O"`, `"P"`, `"G"`) whose id is in `allow.channels`: yes if + the message **names this bot**, or if it is a reply in a thread this Boxmaker already has a + session for (section 9, `threads`) **and names no other user**. Otherwise ignore it, + silently. `@channel`, `@here` and `@all` name nobody: every agent would answer them. + - Any other channel: ignore it, silently. + + "Names" means `@` in the message, compared case-insensitively, where the username is + followed by the end of the message or a character that cannot be part of a username (anything + but `a-z`, `0-9`, `.`, `-`, `_`), and a trailing `.` is not part of it. This bot's username + comes from `GET /api/v4/users/me`. "Names another user" is any other `@` of that shape + except `channel`, `here` and `all`. + + Examples in a channel shared with an agent called Hermes, with this bot `boxmaker-straylight`: + + | Post | For Boxmaker? | + |---|---| + | `@boxmaker-straylight summarise the audit log` (top level) | yes: a new session rooted here | + | a reply in that thread: `and the older files?` | yes: its thread, nobody else named | + | a reply in that thread: `@hermes what do you think?` | no | + | `@boxmaker-straylight @hermes compare notes` | yes (and Hermes answers too) | + | `@boxmaker-straylightx hello` | no: a different name | + | `@channel standup in five` | no | + +4. The session: its root is `root_id`, or the post's own `id` when `root_id` is empty; the session id is `mm-` (29 characters of `[a-z0-9-]`, a valid `SessionId`). A top-level message is a new session (`resume: false`); a reply resumes (`resume: true`). A reply in a thread `loopd` does not know (a thread that began before Boxmaker) is created on `no_such_session`, as `bxctl chat --session` does. -4. Commands. A message whose first character is `!`: +5. Commands. A message whose first character is `!`: - `!!…` is not a command: one `!` is removed and the rest goes on as a message. - `!approve …` and `!deny …` are M4b's; in M4a they are answered in the thread with "approvals over Mattermost arrive in M4b; use `bxctl approvals`". - Anything else is answered with "unknown command; the commands are !approve and !deny". - A command is never sent to `loopd`. -5. Otherwise the message joins its session's queue. A queue already holding `queue` messages +6. Otherwise the message joins its session's queue. A queue already holding `queue` messages answers "busy: too many messages are waiting in this conversation" in the thread and drops it. ## 8. Turns and delivery @@ -226,12 +265,16 @@ every change: ```json {"channels": {"": 1758650000000}, "recent": ["", …], + "threads": ["", …], "in_flight": [{"session": "mm-…", "channel": "", "root": ""}]} ``` - `channels`: the `create_at` of the last post handled in each channel. - `recent`: the ids of the last 500 posts handled, so a post seen twice (live and in a catch-up) is handled once. +- `threads`: the roots of the threads this Boxmaker has a session for, in channels and group + messages (direct messages need none). Added when a naming post starts or joins a thread; the + newest 5,000 are kept. - `in_flight`: turns sent to `loopd` and not yet answered. A file that exists but cannot be read or parsed stops `gatewayd`, with @@ -239,7 +282,7 @@ A file that exists but cannot be read or parsed stops `gatewayd`, with **Connecting** (at start and after every loss): `GET /users/me`; open the WebSocket; wait for `hello`. Then **catch up**: for every channel in `channels`, and the direct channel with every -allowlisted user (`POST /channels/direct`), fetch the posts since the last one handled +allowlisted user (`POST /channels/direct`), and every channel in `allow.channels`, fetch the posts since the last one handled (`GET /channels/{id}/posts?since=…`) and handle them in `create_at` order, before any live event. On a first start, `channels` is empty and nothing is caught up: history is not answered. @@ -254,7 +297,7 @@ turn and logged it; the answer is in the session log. ## 10. Runbook -New entries: `secret-unavailable`, `mattermost-unreachable` (connecting keeps failing), +New entries: `secret-unavailable`, `secret-in-a-file` (the warning), `mattermost-unreachable` (connecting keeps failing), `mattermost-auth-failed`, `gateway-state-damaged`, `loop-unavailable`, `gatewayd-start-failed` (configuration). Every message for these states ends with its pointer. @@ -274,18 +317,22 @@ Offline, as tasks with given tests, against fakes: fixtures (long expiry, labelled test keys); the client reaches a local TLS server through `ca_file`; an unknown CA and a name mismatch are refused. - **HTTP**: `Content-Length`, chunked, the size caps, 429 with `X-Ratelimit-Reset`. -- **Secrets**: each backend, each refusal in section 4's table, and that `Debug` shows no value. +- **Secrets**: each backend, each refusal in section 4's table, the warning for a file secret (and + none for the other two), and that `Debug` shows no value. - **The gateway, against a fake Mattermost** (test support, with a reference implementation): a scripted HTTP and WebSocket server that records every request and pushes events; and a fake `loopd` on a Unix socket. Every rule of sections 7 to 9: the allowlist (nothing at all for anyone - else, and nothing for non-direct channels), own posts ignored, sessions from roots, the burst + else, and nothing for channels not allowed), every row of the naming table and the naming rule's + edges (case, trailing punctuation, a longer name that starts with this one, `@channel`), a reply + in a joined thread with and without another name, own posts ignored, sessions from roots, the burst joined into one turn, the queue limit, typing sent while a turn runs and stopped after, splitting a long answer, commands and `!!`, catch-up after a reconnect, `recent` preventing a second answer, the interrupted reply after a restart, and each runbook pointer. On straylight, by the design model, once the owner has a bot account and a second Mattermost user: a direct message is answered in its thread while Boxmaker shows as typing; the second user gets -nothing; a burst is one turn; messages sent while `gatewayd` was stopped are answered when it +nothing; in an allowed channel with another agent, Boxmaker answers only when named or in its own +thread; a burst is one turn; messages sent while `gatewayd` was stopped are answered when it starts; a restart mid-turn gives the interrupted reply; `ss -ltnp` shows no port for `gatewayd`; the token comes from a systemd credential.