boxmaker/docs/specs/2026-09-23-m4a-gateway.md

M4a design: gatewayd, conversations over Mattermost

Status: draft for owner review, 2026-09-23. M4 is split in two (docs/decisions.md): M4a is the conversation path (this document), M4b is approvals over Mattermost (summarised in section 12, specified after M4a is built). Where this document and docs/design.md disagree, the brief wins; the one change it needs, P15 on secrets, is proposed in docs/decisions.md.

Facts this rests on, checked on 2026-09-23 against the owner's server and against Mattermost's API source and server source at the server's version, v11.11.0:

• Mattermost runs in a rootless Podman container on straylight, plain HTTP on 127.0.0.1:8065; Tailscale Serve terminates TLS for https://straylight.scylla-hammerhead.ts.net and proxies to it. Bot accounts are enabled. Another bot (the owner's Hermes gateway) uses the same server.
• REST: Authorization: Bearer <token>; GET /api/v4/users/me; POST /api/v4/posts (with root_id for a reply in a thread); GET /api/v4/channels/{id}/posts?since=<ms>; POST /api/v4/channels/direct (a direct channel between two user ids). Rate limits are reported in X-Ratelimit-Limit, -Remaining and -Reset; over the limit is status 429. A post holds at most 16,383 characters (PostMessageMaxRunesV2).
• WebSocket: /api/v4/websocket, RFC 6455, authenticated by the same header (or an authentication_challenge message). Events are {"event", "data", "broadcast", "seq"}, starting with hello. posted carries the post as a JSON string in data.post, and data.channel_type ("D" for a direct message). Requests are {"action", "seq", "data"}, answered by {"status", "seq_reply"}; user_typing takes {"channel_id", "parent_id"} and shows the user as typing, in that thread.
• The web client does not post a message that starts with /: it treats it as a slash command. Commands to Boxmaker therefore start with ! (section 7).

1. What M4a proves

Claim	Checked by
A direct message from an allowlisted user becomes a turn, and the answer is posted in its thread	Tests against a fake Mattermost; on straylight
Anyone else gets no response at all: no post, no typing	Tests; on straylight with a second user
A thread is one session; a new top-level message is a new session	Tests
Messages sent while a turn runs are sent together as the next turn	Tests
Messages sent while gatewayd was down are answered when it comes back; a turn cut off by a restart is reported	Tests; on straylight
gatewayd has no listening port	On straylight (ss -ltnp)
The WebSocket client survives hostile input without a panic or an unchecked allocation	Adversarial tests
TLS verifies: an unknown CA or a wrong name is refused	Tests with a test-only CA
Secrets come from a systemd credential, an environment variable or an owner-only file, and never appear in a log	Tests

Out of scope: approvals over Mattermost (M4b), channels other than direct messages unless allowlisted by id, files and images, edits and deletions of posts (an edited message is not re-sent), memory and scheduled jobs (M5).

2. Components

Mattermost  ── HTTPS (or HTTP) ──  gatewayd  ── loop.sock ──  loopd
             REST + WebSocket          │
                                       └── <home>/gateway/state.json

gatewayd opens no listening socket in M4a. (M4b adds gateway.sock, a Unix socket for brokerd; still no TCP port.)

Modules

Each is one file under 500 lines with one purpose.

Module	Purpose
config	gatewayd.toml into a typed Config; unknown keys are errors
secrets	The SecretStore backends and the Secret type (section 4)
net	A connected stream: TCP, or TCP with TLS through rustls (section 5)
http	HTTP/1.1 requests over a net stream: Content-Length and chunked bodies, size caps
ws	The WebSocket client: handshake, frames, ping and close (section 6)
mm	Mattermost's JSON: the calls above and the events, typed
sessions	Which post belongs to which session; the queue per session; commands (section 7)
deliver	Turns on loop.sock, typing while they run, posting the answer (section 8)
state	The state file: last post seen per channel, recent post ids, turns in flight (section 9)
serve	Startup, the event loop, reconnecting

proto gains sha1 (only for the WebSocket handshake check), with the standard vectors. Base64 is small and lives in ws.

Dependencies

gatewayd gains rustls 0.23 (default-features = false, features ring, std, tls12), rustls-native-certs 0.8, zeroize 1, and serde, serde_json, toml (already vetted). On Linux, rustls with ring and native roots is 12 crates, measured 2026-09-23; ring builds C and assembly. deny.toml gains the licences ISC and BSD-3-Clause (owner's approval, 2026-09-23).

3. Configuration

# gatewayd.toml
[mattermost]
url = "https://straylight.scylla-hammerhead.ts.net"   # https or http; no path
ca_file = "/etc/boxmaker/extra-ca.pem"                  # optional; added to the system's roots

[secrets.mattermost_token]
credential = "mattermost-token"      # exactly one of: credential, env, file

[allow]
users = ["abcdefghijklmnopqrstuvwxyz"]   # Mattermost user ids (26 characters)
channels = []                            # non-direct channels allowed, by id; default none

[loop]
socket = ""                          # empty: <home>/run/loop/loop.sock

[paths]
home = "/var/lib/boxmaker"           # default: BOXMAKER_HOME, then /var/lib/boxmaker

[limits]
queue = 20                           # messages waiting per session
typing_every_ms = 3000
ping_every_ms = 30000
dead_after_ms = 60000                # no traffic at all for this long: reconnect

url must be http:// or https:// with a host and an optional port, nothing else. A user id or channel id is 26 characters of [a-z0-9]. An empty allow.users is a configuration error: a gateway that answers nobody is a mistake.

4. Secrets

The brief's SecretStore, with three backends, chosen per secret (P15):

Form	Reads	Refused when
credential = "<name>"	$CREDENTIALS_DIRECTORY/<name>, which systemd fills for a service with LoadCredentialEncrypted=	the variable is unset (not run by systemd with credentials), the file is missing or unreadable
env = "<VAR>"	the environment variable	unset or empty
file = "<absolute path>"	the file	not an absolute path; a symbolic link; not a regular file; not owned by the user gatewayd runs as; any group or other permission bit set (anything but 0600 or 0400)

The value has one trailing newline removed; an empty value is refused. Secret holds it in a zeroize::Zeroizing<String>, has no Display, and its Debug prints Secret(…): it cannot reach a log by accident. Secret::expose(&self) -> &str is the only way to the text, used only to build the Authorization header.

Every refusal names the secret and the reason, never the value, and ends with see docs/runbook.md#secret-unavailable. gatewayd does not start without its token.

For the owner: systemd-creds --user encrypt --name=mattermost-token - <path> encrypts the token to the machine's TPM and host key (checked on straylight, systemd 260, 2026-09-23), and a user service gets LoadCredentialEncrypted=mattermost-token:<path>.

5. The connection: TLS or not

net::connect(url, ca_file) -> Stream: for http, a TcpStream; for https, a TcpStream wrapped in rustls::StreamOwned<ClientConnection, TcpStream>, verifying the server's name against the host in url and its chain against the system's roots (rustls-native-certs) plus ca_file if given. Nothing can turn verification off. A failure to load the system roots when ca_file is unset is an error.

Both the REST calls and the WebSocket run over a Stream. REST calls use one connection per request (Connection: close), as loopd's HTTP client does; the WebSocket keeps one connection. Every read has a deadline; a response header over 16 KiB or a body over 4 MiB is an error.

6. The WebSocket client

Our own, against RFC 6455, blocking, one connection.

• Handshake. GET /api/v4/websocket with Upgrade: websocket, Connection: Upgrade, Sec-WebSocket-Version: 13, a Sec-WebSocket-Key of 16 bytes from /dev/urandom in base64, and the Authorization header. The answer must be status 101 with Upgrade: websocket and a Sec-WebSocket-Accept equal to base64(SHA-1(key + 258EAFA5-E914-47DA-95CA-C5AB0DC85B11)). Anything else is an error.
• Reading. Frames of opcodes text (1), continuation (0), close (8), ping (9), pong (10). A frame from the server that is masked, sets a reserved bit, uses another opcode, is a control frame over 125 bytes or fragmented, is a continuation with nothing to continue, or a new data frame while a message is unfinished, ends the connection. A message over 1 MiB, counted from the length fields before anything is allocated, ends it too; so does a 64-bit length with the top bit set. Text must be UTF-8.
• Writing. Every frame we send is masked with 4 fresh bytes from /dev/urandom. A ping is answered with a pong carrying the same payload. We send a ping every ping_every_ms; no bytes at all for dead_after_ms means the connection is dead.
• Closing. A close frame is answered with a close frame, then the connection is closed.

Any ending of the connection leads to reconnecting (section 9), never to a panic.

7. Sessions, the allowlist and commands

For each posted event: parse data.post (a JSON string) into id, user_id, channel_id, root_id, message, create_at, type. Mattermost's JSON is not ours: unknown fields are ignored. Then, in this order:

1. Ignore it if user_id is the bot's own (from GET /api/v4/users/me at startup), if type is not empty (a system message), or if its id was handled already (section 9).
2. Ignore it, silently, unless user_id is in allow.users and the channel is a direct message (data.channel_type == "D") or its id is in allow.channels. No reply, no typing, no log line with the message's text (one line with the user id and post id is enough).
3. The session: its root is root_id, or the post's own id when root_id is empty; the session id is mm-<root> (29 characters of [a-z0-9-], a valid SessionId). A top-level message is a new session (resume: false); a reply resumes (resume: true). A reply in a thread loopd does not know (a thread that began before Boxmaker) is created on no_such_session, as bxctl chat --session does.
4. Commands. A message whose first character is !:
   • !!… is not a command: one ! is removed and the rest goes on as a message.
   • !approve … and !deny … are M4b's; in M4a they are answered in the thread with "approvals over Mattermost arrive in M4b; use bxctl approvals".
   • Anything else is answered with "unknown command; the commands are !approve and !deny".
   • A command is never sent to loopd.
5. Otherwise the message joins its session's queue. A queue already holding queue messages answers "busy: too many messages are waiting in this conversation" in the thread and drops it.

8. Turns and delivery

One worker per session with a non-empty queue. When the session has no turn running, the worker takes every message waiting, joins them with a blank line ("\n\n"), in the order they were posted, and sends one turn on loop.sock.

While the turn runs, gatewayd sends user_typing over the WebSocket with the channel and the root as parent_id, every typing_every_ms. Turn events are otherwise not shown: no thinking, no status. One exception: an approval_pending event is posted once in the thread as "waiting for approval : approve or deny it with bxctl (Mattermost approvals arrive in M4b)".

When the turn ends:

• turn_done → the content is posted in the thread. Over 16,000 characters, it is split into several replies, at the last newline before the limit (or at a character boundary if there is none), in order.
• An error frame → "Error: : " in the thread, with the runbook pointer when the detail carries one.
• loop.sock cannot be reached, or closes early → "Boxmaker's loop is not running (see docs/runbook.md#loop-unavailable)" in the thread, and the messages are dropped, not retried.

Then, if more messages arrived during the turn, the worker starts the next turn with them.

Several sessions can run turns at once; loopd serialises them at the model.

9. State, reconnecting and restarts

<home>/gateway/state.json, written atomically (write, sync, rename, sync the directory) after every change:

{"channels": {"<channel id>": 1758650000000}, "recent": ["<post id>", …],
 "in_flight": [{"session": "mm-…", "channel": "<id>", "root": "<id>"}]}

• channels: the create_at of the last post handled in each channel.
• recent: the ids of the last 500 posts handled, so a post seen twice (live and in a catch-up) is handled once.
• in_flight: turns sent to loopd and not yet answered.

A file that exists but cannot be read or parsed stops gatewayd, with see docs/runbook.md#gateway-state-damaged; a missing file is a first start.

Connecting (at start and after every loss): GET /users/me; open the WebSocket; wait for hello. Then catch up: for every channel in channels, and the direct channel with every allowlisted user (POST /channels/direct), fetch the posts since the last one handled (GET /channels/{id}/posts?since=…) and handle them in create_at order, before any live event. On a first start, channels is empty and nothing is caught up: history is not answered.

Losing the connection: reconnect after 1, 2, 5, 10, then every 30 seconds, printing one line per attempt. A REST call answered 429 waits until X-Ratelimit-Reset; a 5xx is retried twice; a 401 or 403 stops gatewayd with see docs/runbook.md#mattermost-auth-failed, since retrying with a wrong token would not help.

Restarting: for every entry left in in_flight, gatewayd posts "interrupted: gatewayd restarted before the answer arrived; ask again" in its thread and removes it. loopd finished the turn and logged it; the answer is in the session log.

10. Runbook

New entries: secret-unavailable, mattermost-unreachable (connecting keeps failing), mattermost-auth-failed, gateway-state-damaged, loop-unavailable, gatewayd-start-failed (configuration). Every message for these states ends with its pointer.

11. Testing

Offline, as tasks with given tests, against fakes:

• proto::sha1 and base64: FIPS 180 vectors and RFC 4648 vectors; lengths 55, 56, 63, 64 and 65; each also fed in pieces.
• WebSocket: the handshake (the exact request; RFC 6455's example key and accept; refusal of a wrong accept, a status other than 101, a missing Upgrade, oversized headers). Frames: seeded random frame streams checked against a deliberately naive decoder in the test; every hostile frame in section 6, each ending the connection cleanly with no panic; masking of our frames; ping and pong; fragments reassembled; the close handshake; a server that trickles one byte at a time, and one that goes silent.
• TLS: a test-only CA and server certificate generated once with openssl and committed as fixtures (long expiry, labelled test keys); the client reaches a local TLS server through ca_file; an unknown CA and a name mismatch are refused.
• HTTP: Content-Length, chunked, the size caps, 429 with X-Ratelimit-Reset.
• Secrets: each backend, each refusal in section 4's table, and that Debug shows no value.
• The gateway, against a fake Mattermost (test support, with a reference implementation): a scripted HTTP and WebSocket server that records every request and pushes events; and a fake loopd on a Unix socket. Every rule of sections 7 to 9: the allowlist (nothing at all for anyone else, and nothing for non-direct channels), own posts ignored, sessions from roots, the burst joined into one turn, the queue limit, typing sent while a turn runs and stopped after, splitting a long answer, commands and !!, catch-up after a reconnect, recent preventing a second answer, the interrupted reply after a restart, and each runbook pointer.

On straylight, by the design model, once the owner has a bot account and a second Mattermost user: a direct message is answered in its thread while Boxmaker shows as typing; the second user gets nothing; a burst is one turn; messages sent while gatewayd was stopped are answered when it starts; a restart mid-turn gives the interrupted reply; ss -ltnp shows no port for gatewayd; the token comes from a systemd credential.

12. M4b in summary

Agreed on 2026-09-23, to be specified after M4a is built:

• brokerd opens a connection to gateway.sock (a Unix socket gatewayd listens on) for each approval, with the approval's id, tool, arguments, grant, taint and expiry. gatewayd posts it, answers with the post's id, and keeps the connection open until the owner answers; the final frame is the answer (approved or refused, the Mattermost user id, the post id, a reason). If brokerd closes the connection (answered in bxctl, expired, requester gone, restarted), the post is edited to "no longer pending". gatewayd never reaches admin.sock.
• The approval is posted in the asking session's thread when it is a Mattermost session, otherwise in the owner's direct messages. Arguments escaped as in bxctl, in a code fence longer than any run of backticks inside them.
• An answer counts only if it is from an allowlisted user, after the approval post, while pending, and either a ✅ or ❌ reaction on the approval post itself, or a reply in its thread of exactly !approve <id> or !deny <id> [reason].
• Arguments too long for one post are split into up to 4 numbered parts, the last repeating the id, the count and the SHA-256 of the whole; only an answer on the last part counts. Longer than that: one post with the size and the hash, answerable only in bxctl.
• bxctl gains deny as another name for refuse.

13. Threat model notes

• gatewayd holds the bot token. With the environment or file backend the token is in plaintext (in /proc/<pid>/environ to the same user, or on disk under an owner-only mode); the credential backend keeps it encrypted at rest. The owner chooses per deployment (P15).
• Only allowlisted user ids are heard. Mattermost and the tailnet are trusted (decision of 2026-09-17), so a post's user_id is believed.
• The model's answer is posted as it is. Mattermost renders markdown; a hostile answer can format itself, but reaches only the owner, who asked.
• M4b gives gatewayd the power to answer pending approvals, as bxctl has; it cannot create one, widen a grant or read one.
• M7 must let gatewayd's container reach the tailnet name (and, if url is loopback HTTP, the host's loopback).