# M4a design: `gatewayd`, conversations over Mattermost Status: draft for owner review, 2026-09-23. M4 is split in two (`docs/decisions.md`): M4a is the conversation path (this document), M4b is approvals over Mattermost (summarised in section 12, specified after M4a is built). Where this document and `docs/design.md` disagree, the brief wins; the one change it needs, P15 on secrets, is proposed in `docs/decisions.md`. Facts this rests on, checked on 2026-09-23 against the owner's server and against Mattermost's API source and server source at the server's version, v11.11.0: - Mattermost runs in a rootless Podman container on straylight, plain HTTP on `127.0.0.1:8065`; Tailscale Serve terminates TLS for `https://straylight.scylla-hammerhead.ts.net` and proxies to it. Bot accounts are enabled. Another bot (the owner's Hermes gateway) uses the same server. - REST: `Authorization: Bearer `; `GET /api/v4/users/me`; `POST /api/v4/posts` (with `root_id` for a reply in a thread); `GET /api/v4/channels/{id}/posts?since=`; `POST /api/v4/channels/direct` (a direct channel between two user ids). Rate limits are reported in `X-Ratelimit-Limit`, `-Remaining` and `-Reset`; over the limit is status 429. A post holds at most 16,383 characters (`PostMessageMaxRunesV2`). - WebSocket: `/api/v4/websocket`, RFC 6455, authenticated by the same header (or an `authentication_challenge` message). Events are `{"event", "data", "broadcast", "seq"}`, starting with `hello`. `posted` carries the post as a **JSON string** in `data.post`, and `data.channel_type` (`"D"` for a direct message). Requests are `{"action", "seq", "data"}`, answered by `{"status", "seq_reply"}`; `user_typing` takes `{"channel_id", "parent_id"}` and shows the user as typing, in that thread. - The web client does not post a message that starts with `/`: it treats it as a slash command. Commands to Boxmaker therefore start with `!` (section 7). ## 1. What M4a proves | Claim | Checked by | |---|---| | A direct message from an allowlisted user becomes a turn, and the answer is posted in its thread | Tests against a fake Mattermost; on straylight | | Anyone else gets no response at all: no post, no typing | Tests; on straylight with a second user | | A thread is one session; a new top-level message is a new session | Tests | | Messages sent while a turn runs are sent together as the next turn | Tests | | Messages sent while `gatewayd` was down are answered when it comes back; a turn cut off by a restart is reported | Tests; on straylight | | `gatewayd` has no listening port | On straylight (`ss -ltnp`) | | The WebSocket client survives hostile input without a panic or an unchecked allocation | Adversarial tests | | TLS verifies: an unknown CA or a wrong name is refused | Tests with a test-only CA | | Secrets come from a systemd credential, an environment variable or an owner-only file, and never appear in a log | Tests | Out of scope: approvals over Mattermost (M4b), channels other than direct messages unless allowlisted by id, files and images, edits and deletions of posts (an edited message is not re-sent), memory and scheduled jobs (M5). ## 2. Components ``` Mattermost ── HTTPS (or HTTP) ── gatewayd ── loop.sock ── loopd REST + WebSocket │ └── /gateway/state.json ``` `gatewayd` opens **no listening socket** in M4a. (M4b adds `gateway.sock`, a Unix socket for `brokerd`; still no TCP port.) ### Modules Each is one file under 500 lines with one purpose. | Module | Purpose | |---|---| | `config` | `gatewayd.toml` into a typed `Config`; unknown keys are errors | | `secrets` | The `SecretStore` backends and the `Secret` type (section 4) | | `net` | A connected stream: TCP, or TCP with TLS through `rustls` (section 5) | | `http` | HTTP/1.1 requests over a `net` stream: `Content-Length` and chunked bodies, size caps | | `ws` | The WebSocket client: handshake, frames, ping and close (section 6) | | `mm` | Mattermost's JSON: the calls above and the events, typed | | `sessions` | Which post belongs to which session; the queue per session; commands (section 7) | | `deliver` | Turns on `loop.sock`, typing while they run, posting the answer (section 8) | | `state` | The state file: last post seen per channel, recent post ids, turns in flight (section 9) | | `serve` | Startup, the event loop, reconnecting | `proto` gains `sha1` (only for the WebSocket handshake check), with the standard vectors. Base64 is small and lives in `ws`. ### Dependencies `gatewayd` gains `rustls` 0.23 (`default-features = false`, features `ring`, `std`, `tls12`), `rustls-native-certs` 0.8, `zeroize` 1, and `serde`, `serde_json`, `toml` (already vetted). On Linux, `rustls` with `ring` and native roots is 12 crates, measured 2026-09-23; `ring` builds C and assembly. `deny.toml` gains the licences ISC and BSD-3-Clause (owner's approval, 2026-09-23). ## 3. Configuration ```toml # gatewayd.toml [mattermost] url = "https://straylight.scylla-hammerhead.ts.net" # https or http; no path ca_file = "/etc/boxmaker/extra-ca.pem" # optional; added to the system's roots [secrets.mattermost_token] credential = "mattermost-token" # exactly one of: credential, env, file [allow] users = ["abcdefghijklmnopqrstuvwxyz"] # Mattermost user ids (26 characters) channels = [] # non-direct channels allowed, by id; default none [loop] socket = "" # empty: /run/loop/loop.sock [paths] home = "/var/lib/boxmaker" # default: BOXMAKER_HOME, then /var/lib/boxmaker [limits] queue = 20 # messages waiting per session typing_every_ms = 3000 ping_every_ms = 30000 dead_after_ms = 60000 # no traffic at all for this long: reconnect ``` `url` must be `http://` or `https://` with a host and an optional port, nothing else. A user id or channel id is 26 characters of `[a-z0-9]`. An empty `allow.users` is a configuration error: a gateway that answers nobody is a mistake. ## 4. Secrets The brief's `SecretStore`, with three backends, chosen per secret (P15): | Form | Reads | Refused when | |---|---|---| | `credential = ""` | `$CREDENTIALS_DIRECTORY/`, which systemd fills for a service with `LoadCredentialEncrypted=` | the variable is unset (not run by systemd with credentials), the file is missing or unreadable | | `env = ""` | the environment variable | unset or empty | | `file = ""` | the file | not an absolute path; a symbolic link; not a regular file; not owned by the user `gatewayd` runs as; any group or other permission bit set (anything but 0600 or 0400) | The value has one trailing newline removed; an empty value is refused. `Secret` holds it in a `zeroize::Zeroizing`, has no `Display`, and its `Debug` prints `Secret(…)`: it cannot reach a log by accident. `Secret::expose(&self) -> &str` is the only way to the text, used only to build the `Authorization` header. Every refusal names the secret and the reason, never the value, and ends with `see docs/runbook.md#secret-unavailable`. `gatewayd` does not start without its token. For the owner: `systemd-creds --user encrypt --name=mattermost-token - ` encrypts the token to the machine's TPM and host key (checked on straylight, systemd 260, 2026-09-23), and a user service gets `LoadCredentialEncrypted=mattermost-token:`. ## 5. The connection: TLS or not `net::connect(url, ca_file) -> Stream`: for `http`, a `TcpStream`; for `https`, a `TcpStream` wrapped in `rustls::StreamOwned`, verifying the server's name against the host in `url` and its chain against the system's roots (`rustls-native-certs`) plus `ca_file` if given. Nothing can turn verification off. A failure to load the system roots when `ca_file` is unset is an error. Both the REST calls and the WebSocket run over a `Stream`. REST calls use one connection per request (`Connection: close`), as `loopd`'s HTTP client does; the WebSocket keeps one connection. Every read has a deadline; a response header over 16 KiB or a body over 4 MiB is an error. ## 6. The WebSocket client Our own, against RFC 6455, blocking, one connection. - **Handshake.** `GET /api/v4/websocket` with `Upgrade: websocket`, `Connection: Upgrade`, `Sec-WebSocket-Version: 13`, a `Sec-WebSocket-Key` of 16 bytes from `/dev/urandom` in base64, and the `Authorization` header. The answer must be status 101 with `Upgrade: websocket` and a `Sec-WebSocket-Accept` equal to base64(SHA-1(key + `258EAFA5-E914-47DA-95CA-C5AB0DC85B11`)). Anything else is an error. - **Reading.** Frames of opcodes text (1), continuation (0), close (8), ping (9), pong (10). A frame from the server that is masked, sets a reserved bit, uses another opcode, is a control frame over 125 bytes or fragmented, is a continuation with nothing to continue, or a new data frame while a message is unfinished, ends the connection. A message over 1 MiB, counted from the length fields **before** anything is allocated, ends it too; so does a 64-bit length with the top bit set. Text must be UTF-8. - **Writing.** Every frame we send is masked with 4 fresh bytes from `/dev/urandom`. A ping is answered with a pong carrying the same payload. We send a ping every `ping_every_ms`; no bytes at all for `dead_after_ms` means the connection is dead. - **Closing.** A close frame is answered with a close frame, then the connection is closed. Any ending of the connection leads to reconnecting (section 9), never to a panic. ## 7. Sessions, the allowlist and commands For each `posted` event: parse `data.post` (a JSON string) into `id`, `user_id`, `channel_id`, `root_id`, `message`, `create_at`, `type`. Mattermost's JSON is not ours: unknown fields are ignored. Then, in this order: 1. Ignore it if `user_id` is the bot's own (from `GET /api/v4/users/me` at startup), if `type` is not empty (a system message), or if its id was handled already (section 9). 2. Ignore it, **silently**, unless `user_id` is in `allow.users` **and** the channel is a direct message (`data.channel_type == "D"`) or its id is in `allow.channels`. No reply, no typing, no log line with the message's text (one line with the user id and post id is enough). 3. The session: its root is `root_id`, or the post's own `id` when `root_id` is empty; the session id is `mm-` (29 characters of `[a-z0-9-]`, a valid `SessionId`). A top-level message is a new session (`resume: false`); a reply resumes (`resume: true`). A reply in a thread `loopd` does not know (a thread that began before Boxmaker) is created on `no_such_session`, as `bxctl chat --session` does. 4. Commands. A message whose first character is `!`: - `!!…` is not a command: one `!` is removed and the rest goes on as a message. - `!approve …` and `!deny …` are M4b's; in M4a they are answered in the thread with "approvals over Mattermost arrive in M4b; use `bxctl approvals`". - Anything else is answered with "unknown command; the commands are !approve and !deny". - A command is never sent to `loopd`. 5. Otherwise the message joins its session's queue. A queue already holding `queue` messages answers "busy: too many messages are waiting in this conversation" in the thread and drops it. ## 8. Turns and delivery One worker per session with a non-empty queue. When the session has no turn running, the worker takes **every** message waiting, joins them with a blank line (`"\n\n"`), in the order they were posted, and sends one `turn` on `loop.sock`. While the turn runs, `gatewayd` sends `user_typing` over the WebSocket with the channel and the root as `parent_id`, every `typing_every_ms`. Turn events are otherwise not shown: no thinking, no status. One exception: an `approval_pending` event is posted once in the thread as "waiting for approval : approve or deny it with `bxctl` (Mattermost approvals arrive in M4b)". When the turn ends: - `turn_done` → the content is posted in the thread. Over 16,000 characters, it is split into several replies, at the last newline before the limit (or at a character boundary if there is none), in order. - An `error` frame → "Error: : " in the thread, with the runbook pointer when the detail carries one. - `loop.sock` cannot be reached, or closes early → "Boxmaker's loop is not running (see docs/runbook.md#loop-unavailable)" in the thread, and the messages are dropped, not retried. Then, if more messages arrived during the turn, the worker starts the next turn with them. Several sessions can run turns at once; `loopd` serialises them at the model. ## 9. State, reconnecting and restarts `/gateway/state.json`, written atomically (write, sync, rename, sync the directory) after every change: ```json {"channels": {"": 1758650000000}, "recent": ["", …], "in_flight": [{"session": "mm-…", "channel": "", "root": ""}]} ``` - `channels`: the `create_at` of the last post handled in each channel. - `recent`: the ids of the last 500 posts handled, so a post seen twice (live and in a catch-up) is handled once. - `in_flight`: turns sent to `loopd` and not yet answered. A file that exists but cannot be read or parsed stops `gatewayd`, with `see docs/runbook.md#gateway-state-damaged`; a missing file is a first start. **Connecting** (at start and after every loss): `GET /users/me`; open the WebSocket; wait for `hello`. Then **catch up**: for every channel in `channels`, and the direct channel with every allowlisted user (`POST /channels/direct`), fetch the posts since the last one handled (`GET /channels/{id}/posts?since=…`) and handle them in `create_at` order, before any live event. On a first start, `channels` is empty and nothing is caught up: history is not answered. **Losing the connection**: reconnect after 1, 2, 5, 10, then every 30 seconds, printing one line per attempt. A REST call answered 429 waits until `X-Ratelimit-Reset`; a 5xx is retried twice; a 401 or 403 stops `gatewayd` with `see docs/runbook.md#mattermost-auth-failed`, since retrying with a wrong token would not help. **Restarting**: for every entry left in `in_flight`, `gatewayd` posts "interrupted: gatewayd restarted before the answer arrived; ask again" in its thread and removes it. `loopd` finished the turn and logged it; the answer is in the session log. ## 10. Runbook New entries: `secret-unavailable`, `mattermost-unreachable` (connecting keeps failing), `mattermost-auth-failed`, `gateway-state-damaged`, `loop-unavailable`, `gatewayd-start-failed` (configuration). Every message for these states ends with its pointer. ## 11. Testing Offline, as tasks with given tests, against fakes: - **`proto::sha1` and base64**: FIPS 180 vectors and RFC 4648 vectors; lengths 55, 56, 63, 64 and 65; each also fed in pieces. - **WebSocket**: the handshake (the exact request; RFC 6455's example key and accept; refusal of a wrong accept, a status other than 101, a missing `Upgrade`, oversized headers). Frames: seeded random frame streams checked against a deliberately naive decoder in the test; every hostile frame in section 6, each ending the connection cleanly with no panic; masking of our frames; ping and pong; fragments reassembled; the close handshake; a server that trickles one byte at a time, and one that goes silent. - **TLS**: a test-only CA and server certificate generated once with `openssl` and committed as fixtures (long expiry, labelled test keys); the client reaches a local TLS server through `ca_file`; an unknown CA and a name mismatch are refused. - **HTTP**: `Content-Length`, chunked, the size caps, 429 with `X-Ratelimit-Reset`. - **Secrets**: each backend, each refusal in section 4's table, and that `Debug` shows no value. - **The gateway, against a fake Mattermost** (test support, with a reference implementation): a scripted HTTP and WebSocket server that records every request and pushes events; and a fake `loopd` on a Unix socket. Every rule of sections 7 to 9: the allowlist (nothing at all for anyone else, and nothing for non-direct channels), own posts ignored, sessions from roots, the burst joined into one turn, the queue limit, typing sent while a turn runs and stopped after, splitting a long answer, commands and `!!`, catch-up after a reconnect, `recent` preventing a second answer, the interrupted reply after a restart, and each runbook pointer. On straylight, by the design model, once the owner has a bot account and a second Mattermost user: a direct message is answered in its thread while Boxmaker shows as typing; the second user gets nothing; a burst is one turn; messages sent while `gatewayd` was stopped are answered when it starts; a restart mid-turn gives the interrupted reply; `ss -ltnp` shows no port for `gatewayd`; the token comes from a systemd credential. ## 12. M4b in summary Agreed on 2026-09-23, to be specified after M4a is built: - `brokerd` opens a connection to `gateway.sock` (a Unix socket `gatewayd` listens on) for each approval, with the approval's id, tool, arguments, grant, taint and expiry. `gatewayd` posts it, answers with the post's id, and keeps the connection open until the owner answers; the final frame is the answer (approved or refused, the Mattermost user id, the post id, a reason). If `brokerd` closes the connection (answered in `bxctl`, expired, requester gone, restarted), the post is edited to "no longer pending". `gatewayd` never reaches `admin.sock`. - The approval is posted in the asking session's thread when it is a Mattermost session, otherwise in the owner's direct messages. Arguments escaped as in `bxctl`, in a code fence longer than any run of backticks inside them. - An answer counts only if it is from an allowlisted user, after the approval post, while pending, and either a ✅ or ❌ reaction on the approval post itself, or a reply in its thread of exactly `!approve ` or `!deny [reason]`. - Arguments too long for one post are split into up to 4 numbered parts, the last repeating the id, the count and the SHA-256 of the whole; only an answer on the last part counts. Longer than that: one post with the size and the hash, answerable only in `bxctl`. - `bxctl` gains `deny` as another name for `refuse`. ## 13. Threat model notes - `gatewayd` holds the bot token. With the environment or file backend the token is in plaintext (in `/proc//environ` to the same user, or on disk under an owner-only mode); the credential backend keeps it encrypted at rest. The owner chooses per deployment (P15). - Only allowlisted user ids are heard. Mattermost and the tailnet are trusted (decision of 2026-09-17), so a post's `user_id` is believed. - The model's answer is posted as it is. Mattermost renders markdown; a hostile answer can format itself, but reaches only the owner, who asked. - M4b gives `gatewayd` the power to answer pending approvals, as `bxctl` has; it cannot create one, widen a grant or read one. - M7 must let `gatewayd`'s container reach the tailnet name (and, if `url` is loopback HTTP, the host's loopback).