M4a spec: approved, with a file-secret warning and multi-agent channels

A secret read from a file warns at startup. In allowed channels and group
messages, Boxmaker answers only posts that name it, or replies in its own
threads that name nobody else; one Boxmaker per machine, each its own bot.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-09-23 17:42:45 -07:00
co-authored by Claude Opus 5.5
parent 7e3783faae
commit 4a510fde08
2 changed files with 65 additions and 17 deletions
+1
View File
@@ -6,6 +6,7 @@ Newest first. A decision that changes `docs/design.md` lands in the same commit
| Date | Decision | Reason | | Date | Decision | Reason |
|---|---|---| |---|---|---|
| 2026-09-23 | M4a spec approved with two changes: a secret read from a file prints a warning at startup (and still starts); and in allowed channels and group messages shared with other agents, Boxmaker answers a post only if it names this bot, or it is a reply in one of this Boxmaker's threads that names no other user (`@channel`, `@here`, `@all` name nobody). One Boxmaker per machine, each with its own bot account; nothing is shared between them. | Owner's notes. The owner runs several agents in channels, possibly one Boxmaker per machine. A post's `from_bot` property is user-settable in v11.11.0, so bots are told apart by the allowlist, not by that property. |
| 2026-09-23 | M4 is split: M4a is conversations over Mattermost (`gatewayd`, the secret store), M4b is approvals over Mattermost. Spec `docs/specs/2026-09-23-m4a-gateway.md`. | As with M2 and M3: the authority-bearing half is reviewed separately, and the open question about long approvals lands in M4b. | | 2026-09-23 | M4 is split: M4a is conversations over Mattermost (`gatewayd`, the secret store), M4b is approvals over Mattermost. Spec `docs/specs/2026-09-23-m4a-gateway.md`. | As with M2 and M3: the authority-bearing half is reviewed separately, and the open question about long approvals lands in M4b. |
| 2026-09-23 | `gatewayd` reaches Mattermost over TLS, by default at the tailnet name `https://straylight.scylla-hammerhead.ts.net`; plain HTTP stays possible by URL. TLS is `rustls` 0.23 with `ring` and `rustls-native-certs` (the host's trust store, plus an optional CA file); `deny.toml` gains ISC and BSD-3-Clause. | Owner: portability, and a bigger picture that favours the tailnet name over host loopback. The host's store keeps a privately installed CA trusted; `webpki-roots` would not, and `aws-lc-rs` is a larger C build for nothing needed. | | 2026-09-23 | `gatewayd` reaches Mattermost over TLS, by default at the tailnet name `https://straylight.scylla-hammerhead.ts.net`; plain HTTP stays possible by URL. TLS is `rustls` 0.23 with `ring` and `rustls-native-certs` (the host's trust store, plus an optional CA file); `deny.toml` gains ISC and BSD-3-Clause. | Owner: portability, and a bigger picture that favours the tailnet name over host loopback. The host's store keeps a privately installed CA trusted; `webpki-roots` would not, and `aws-lc-rs` is a larger C build for nothing needed. |
| 2026-09-23 | Our own blocking WebSocket client (RFC 6455), adversarially tested; SHA-1 for its handshake check lives in `proto`. | `tungstenite` would add about ten crates for a few hundred lines of protocol. | | 2026-09-23 | Our own blocking WebSocket client (RFC 6455), adversarially tested; SHA-1 for its handshake check lives in `proto`. | `tungstenite` would add about ten crates for a few hundred lines of protocol. |
+64 -17
View File
@@ -1,6 +1,7 @@
# M4a design: `gatewayd`, conversations over Mattermost # M4a design: `gatewayd`, conversations over Mattermost
Status: draft for owner review, 2026-09-23. M4 is split in two (`docs/decisions.md`): M4a is the Status: approved by the owner, 2026-09-23, with two changes folded in (a warning for a file
secret; channels shared with other agents, section 7). M4 is split in two (`docs/decisions.md`): M4a is the
conversation path (this document), M4b is approvals over Mattermost (summarised in section 12, conversation path (this document), M4b is approvals over Mattermost (summarised in section 12,
specified after M4a is built). Where this document and `docs/design.md` disagree, the brief wins; specified after M4a is built). Where this document and `docs/design.md` disagree, the brief wins;
the one change it needs, P15 on secrets, is proposed in `docs/decisions.md`. the one change it needs, P15 on secrets, is proposed in `docs/decisions.md`.
@@ -24,12 +25,18 @@ source and server source at the server's version, v11.11.0:
`{"channel_id", "parent_id"}` and shows the user as typing, in that thread. `{"channel_id", "parent_id"}` and shows the user as typing, in that thread.
- The web client does not post a message that starts with `/`: it treats it as a slash command. - The web client does not post a message that starts with `/`: it treats it as a slash command.
Commands to Boxmaker therefore start with `!` (section 7). Commands to Boxmaker therefore start with `!` (section 7).
- A post's `from_bot` property is settable by any user (the server's own comment, v11.11.0), so it
does not say who is a bot. Mentions are added to events per connection; `gatewayd` finds its own
`@username` in the message text instead.
- The owner runs several agents in Mattermost channels, and may run one Boxmaker on each of several
machines (at most one per machine), each with its own bot account.
## 1. What M4a proves ## 1. What M4a proves
| Claim | Checked by | | Claim | Checked by |
|---|---| |---|---|
| A direct message from an allowlisted user becomes a turn, and the answer is posted in its thread | Tests against a fake Mattermost; on straylight | | A direct message from an allowlisted user becomes a turn, and the answer is posted in its thread | Tests against a fake Mattermost; on straylight |
| In an allowed channel shared with other agents, Boxmaker answers only posts that name it, or replies in its own threads that name nobody else | Tests; on straylight in a channel |
| Anyone else gets no response at all: no post, no typing | Tests; on straylight with a second user | | Anyone else gets no response at all: no post, no typing | Tests; on straylight with a second user |
| A thread is one session; a new top-level message is a new session | Tests | | A thread is one session; a new top-level message is a new session | Tests |
| Messages sent while a turn runs are sent together as the next turn | Tests | | Messages sent while a turn runs are sent together as the next turn | Tests |
@@ -39,9 +46,9 @@ source and server source at the server's version, v11.11.0:
| TLS verifies: an unknown CA or a wrong name is refused | Tests with a test-only CA | | TLS verifies: an unknown CA or a wrong name is refused | Tests with a test-only CA |
| Secrets come from a systemd credential, an environment variable or an owner-only file, and never appear in a log | Tests | | Secrets come from a systemd credential, an environment variable or an owner-only file, and never appear in a log | Tests |
Out of scope: approvals over Mattermost (M4b), channels other than direct messages unless Out of scope: approvals over Mattermost (M4b), channels not allowlisted by id, the earlier posts of
allowlisted by id, files and images, edits and deletions of posts (an edited message is not a thread Boxmaker is first named in (only the naming post reaches it), files and images, edits and
re-sent), memory and scheduled jobs (M5). deletions of posts (an edited message is not re-sent), memory and scheduled jobs (M5).
## 2. Components ## 2. Components
@@ -94,7 +101,7 @@ credential = "mattermost-token" # exactly one of: credential, env, file
[allow] [allow]
users = ["abcdefghijklmnopqrstuvwxyz"] # Mattermost user ids (26 characters) users = ["abcdefghijklmnopqrstuvwxyz"] # Mattermost user ids (26 characters)
channels = [] # non-direct channels allowed, by id; default none channels = [] # channels and group messages allowed, by id; default none
[loop] [loop]
socket = "" # empty: <home>/run/loop/loop.sock socket = "" # empty: <home>/run/loop/loop.sock
@@ -113,6 +120,9 @@ dead_after_ms = 60000 # no traffic at all for this long: reconnec
channel id is 26 characters of `[a-z0-9]`. An empty `allow.users` is a configuration error: a channel id is 26 characters of `[a-z0-9]`. An empty `allow.users` is a configuration error: a
gateway that answers nobody is a mistake. gateway that answers nobody is a mistake.
Allowlisting another agent's user id lets that agent talk to Boxmaker. Two agents that allowlist
each other will answer each other without end; nothing prevents it, so do not.
## 4. Secrets ## 4. Secrets
The brief's `SecretStore`, with three backends, chosen per secret (P15): The brief's `SecretStore`, with three backends, chosen per secret (P15):
@@ -123,7 +133,12 @@ The brief's `SecretStore`, with three backends, chosen per secret (P15):
| `env = "<VAR>"` | the environment variable | unset or empty | | `env = "<VAR>"` | the environment variable | unset or empty |
| `file = "<absolute path>"` | the file | not an absolute path; a symbolic link; not a regular file; not owned by the user `gatewayd` runs as; any group or other permission bit set (anything but 0600 or 0400) | | `file = "<absolute path>"` | the file | not an absolute path; a symbolic link; not a regular file; not owned by the user `gatewayd` runs as; any group or other permission bit set (anything but 0600 or 0400) |
The value has one trailing newline removed; an empty value is refused. `Secret` holds it in a The value has one trailing newline removed; an empty value is refused.
A secret read from a **file** is in plaintext on disk. `gatewayd` starts, and prints at startup, for
each such secret: `gatewayd: warning: secret <name> is read in plaintext from <path>; a systemd
credential keeps it encrypted at rest (see docs/runbook.md#secret-in-a-file)`. It is a warning,
not a refusal: there are deployments where a file is right. `Secret` holds it in a
`zeroize::Zeroizing<String>`, has no `Display`, and its `Debug` prints `Secret(…)`: it cannot reach `zeroize::Zeroizing<String>`, has no `Display`, and its `Debug` prints `Secret(…)`: it cannot reach
a log by accident. `Secret::expose(&self) -> &str` is the only way to the text, used only to build a log by accident. `Secret::expose(&self) -> &str` is the only way to the text, used only to build
the `Authorization` header. the `Authorization` header.
@@ -177,21 +192,45 @@ ignored. Then, in this order:
1. Ignore it if `user_id` is the bot's own (from `GET /api/v4/users/me` at startup), if `type` is 1. Ignore it if `user_id` is the bot's own (from `GET /api/v4/users/me` at startup), if `type` is
not empty (a system message), or if its id was handled already (section 9). not empty (a system message), or if its id was handled already (section 9).
2. Ignore it, **silently**, unless `user_id` is in `allow.users` **and** the channel is a direct 2. Ignore it, **silently**, unless `user_id` is in `allow.users`. No reply, no typing, no log line
message (`data.channel_type == "D"`) or its id is in `allow.channels`. No reply, no typing, no with the message's text (one line with the user id and post id is enough).
log line with the message's text (one line with the user id and post id is enough). 3. Is it for this Boxmaker? By the channel:
3. The session: its root is `root_id`, or the post's own `id` when `root_id` is empty; the session - A direct message with the bot (`data.channel_type == "D"`): yes, always.
- A channel or a group message (`"O"`, `"P"`, `"G"`) whose id is in `allow.channels`: yes if
the message **names this bot**, or if it is a reply in a thread this Boxmaker already has a
session for (section 9, `threads`) **and names no other user**. Otherwise ignore it,
silently. `@channel`, `@here` and `@all` name nobody: every agent would answer them.
- Any other channel: ignore it, silently.
"Names" means `@<username>` in the message, compared case-insensitively, where the username is
followed by the end of the message or a character that cannot be part of a username (anything
but `a-z`, `0-9`, `.`, `-`, `_`), and a trailing `.` is not part of it. This bot's username
comes from `GET /api/v4/users/me`. "Names another user" is any other `@<name>` of that shape
except `channel`, `here` and `all`.
Examples in a channel shared with an agent called Hermes, with this bot `boxmaker-straylight`:
| Post | For Boxmaker? |
|---|---|
| `@boxmaker-straylight summarise the audit log` (top level) | yes: a new session rooted here |
| a reply in that thread: `and the older files?` | yes: its thread, nobody else named |
| a reply in that thread: `@hermes what do you think?` | no |
| `@boxmaker-straylight @hermes compare notes` | yes (and Hermes answers too) |
| `@boxmaker-straylightx hello` | no: a different name |
| `@channel standup in five` | no |
4. The session: its root is `root_id`, or the post's own `id` when `root_id` is empty; the session
id is `mm-<root>` (29 characters of `[a-z0-9-]`, a valid `SessionId`). A top-level message is a id is `mm-<root>` (29 characters of `[a-z0-9-]`, a valid `SessionId`). A top-level message is a
new session (`resume: false`); a reply resumes (`resume: true`). A reply in a thread `loopd` new session (`resume: false`); a reply resumes (`resume: true`). A reply in a thread `loopd`
does not know (a thread that began before Boxmaker) is created on `no_such_session`, as does not know (a thread that began before Boxmaker) is created on `no_such_session`, as
`bxctl chat --session` does. `bxctl chat --session` does.
4. Commands. A message whose first character is `!`: 5. Commands. A message whose first character is `!`:
- `!!…` is not a command: one `!` is removed and the rest goes on as a message. - `!!…` is not a command: one `!` is removed and the rest goes on as a message.
- `!approve …` and `!deny …` are M4b's; in M4a they are answered in the thread with - `!approve …` and `!deny …` are M4b's; in M4a they are answered in the thread with
"approvals over Mattermost arrive in M4b; use `bxctl approvals`". "approvals over Mattermost arrive in M4b; use `bxctl approvals`".
- Anything else is answered with "unknown command; the commands are !approve and !deny". - Anything else is answered with "unknown command; the commands are !approve and !deny".
- A command is never sent to `loopd`. - A command is never sent to `loopd`.
5. Otherwise the message joins its session's queue. A queue already holding `queue` messages 6. Otherwise the message joins its session's queue. A queue already holding `queue` messages
answers "busy: too many messages are waiting in this conversation" in the thread and drops it. answers "busy: too many messages are waiting in this conversation" in the thread and drops it.
## 8. Turns and delivery ## 8. Turns and delivery
@@ -226,12 +265,16 @@ every change:
```json ```json
{"channels": {"<channel id>": 1758650000000}, "recent": ["<post id>", ], {"channels": {"<channel id>": 1758650000000}, "recent": ["<post id>", ],
"threads": ["<root post id>", ],
"in_flight": [{"session": "mm-…", "channel": "<id>", "root": "<id>"}]} "in_flight": [{"session": "mm-…", "channel": "<id>", "root": "<id>"}]}
``` ```
- `channels`: the `create_at` of the last post handled in each channel. - `channels`: the `create_at` of the last post handled in each channel.
- `recent`: the ids of the last 500 posts handled, so a post seen twice (live and in a catch-up) is - `recent`: the ids of the last 500 posts handled, so a post seen twice (live and in a catch-up) is
handled once. handled once.
- `threads`: the roots of the threads this Boxmaker has a session for, in channels and group
messages (direct messages need none). Added when a naming post starts or joins a thread; the
newest 5,000 are kept.
- `in_flight`: turns sent to `loopd` and not yet answered. - `in_flight`: turns sent to `loopd` and not yet answered.
A file that exists but cannot be read or parsed stops `gatewayd`, with A file that exists but cannot be read or parsed stops `gatewayd`, with
@@ -239,7 +282,7 @@ A file that exists but cannot be read or parsed stops `gatewayd`, with
**Connecting** (at start and after every loss): `GET /users/me`; open the WebSocket; wait for **Connecting** (at start and after every loss): `GET /users/me`; open the WebSocket; wait for
`hello`. Then **catch up**: for every channel in `channels`, and the direct channel with every `hello`. Then **catch up**: for every channel in `channels`, and the direct channel with every
allowlisted user (`POST /channels/direct`), fetch the posts since the last one handled allowlisted user (`POST /channels/direct`), and every channel in `allow.channels`, fetch the posts since the last one handled
(`GET /channels/{id}/posts?since=…`) and handle them in `create_at` order, before any live event. (`GET /channels/{id}/posts?since=…`) and handle them in `create_at` order, before any live event.
On a first start, `channels` is empty and nothing is caught up: history is not answered. On a first start, `channels` is empty and nothing is caught up: history is not answered.
@@ -254,7 +297,7 @@ turn and logged it; the answer is in the session log.
## 10. Runbook ## 10. Runbook
New entries: `secret-unavailable`, `mattermost-unreachable` (connecting keeps failing), New entries: `secret-unavailable`, `secret-in-a-file` (the warning), `mattermost-unreachable` (connecting keeps failing),
`mattermost-auth-failed`, `gateway-state-damaged`, `loop-unavailable`, `gatewayd-start-failed` `mattermost-auth-failed`, `gateway-state-damaged`, `loop-unavailable`, `gatewayd-start-failed`
(configuration). Every message for these states ends with its pointer. (configuration). Every message for these states ends with its pointer.
@@ -274,18 +317,22 @@ Offline, as tasks with given tests, against fakes:
fixtures (long expiry, labelled test keys); the client reaches a local TLS server through fixtures (long expiry, labelled test keys); the client reaches a local TLS server through
`ca_file`; an unknown CA and a name mismatch are refused. `ca_file`; an unknown CA and a name mismatch are refused.
- **HTTP**: `Content-Length`, chunked, the size caps, 429 with `X-Ratelimit-Reset`. - **HTTP**: `Content-Length`, chunked, the size caps, 429 with `X-Ratelimit-Reset`.
- **Secrets**: each backend, each refusal in section 4's table, and that `Debug` shows no value. - **Secrets**: each backend, each refusal in section 4's table, the warning for a file secret (and
none for the other two), and that `Debug` shows no value.
- **The gateway, against a fake Mattermost** (test support, with a reference implementation): a - **The gateway, against a fake Mattermost** (test support, with a reference implementation): a
scripted HTTP and WebSocket server that records every request and pushes events; and a fake scripted HTTP and WebSocket server that records every request and pushes events; and a fake
`loopd` on a Unix socket. Every rule of sections 7 to 9: the allowlist (nothing at all for anyone `loopd` on a Unix socket. Every rule of sections 7 to 9: the allowlist (nothing at all for anyone
else, and nothing for non-direct channels), own posts ignored, sessions from roots, the burst else, and nothing for channels not allowed), every row of the naming table and the naming rule's
edges (case, trailing punctuation, a longer name that starts with this one, `@channel`), a reply
in a joined thread with and without another name, own posts ignored, sessions from roots, the burst
joined into one turn, the queue limit, typing sent while a turn runs and stopped after, splitting joined into one turn, the queue limit, typing sent while a turn runs and stopped after, splitting
a long answer, commands and `!!`, catch-up after a reconnect, `recent` preventing a second a long answer, commands and `!!`, catch-up after a reconnect, `recent` preventing a second
answer, the interrupted reply after a restart, and each runbook pointer. answer, the interrupted reply after a restart, and each runbook pointer.
On straylight, by the design model, once the owner has a bot account and a second Mattermost user: On straylight, by the design model, once the owner has a bot account and a second Mattermost user:
a direct message is answered in its thread while Boxmaker shows as typing; the second user gets a direct message is answered in its thread while Boxmaker shows as typing; the second user gets
nothing; a burst is one turn; messages sent while `gatewayd` was stopped are answered when it nothing; in an allowed channel with another agent, Boxmaker answers only when named or in its own
thread; a burst is one turn; messages sent while `gatewayd` was stopped are answered when it
starts; a restart mid-turn gives the interrupted reply; `ss -ltnp` shows no port for `gatewayd`; starts; a restart mid-turn gives the interrupted reply; `ss -ltnp` shows no port for `gatewayd`;
the token comes from a systemd credential. the token comes from a systemd credential.