boxmaker/docs/plans/M4a/14-gatewayd-serve.md

M4a task 14: the serve loop

Branch: m4a (run git switch m4a; git status --short must be empty, otherwise stop) Commit subject: gatewayd: serve, the event loop, typing, catch-up and reconnecting

Goal

The loop that ties tasks 03 to 13 together (spec sections 8 and 9):

• Connecting, at start and after every loss: GET /users/me, open the WebSocket, wait for hello, then log gatewayd: connected to <url> as <username>. A connection that fails is tried again after 1, 2, 5, 10, then every 30 seconds, with one log line per attempt: gatewayd: cannot reach <url>: <error>; trying again in <n> s, then a newline and see docs/runbook.md#mattermost-unreachable. A refused token (401 or 403) is never tried again: run returns Stop::Auth.
• After a restart (the first connection only): every turn left in flight gets interrupted: gatewayd restarted before the answer arrived; ask again in its thread. A later reconnect must not do this: those turns are still running.
• Catching up: the direct channel with each allowed user, and each allowed channel, from its mark. A channel without a mark is marked "now" and not caught up: history is not answered.
• Each post, live or caught up: posts in channels we do not track are not recorded at all. A tracked post is recorded as handled before it is acted on, so after a crash it is not answered twice (the in-flight record reports it instead). Then it is routed (task 11): a stranger is logged by user id and post id only, never with the text.
• Turns run on their own threads (task 13), recorded in flight while they run. When one ends, its session's waiting messages start the next.
• Typing: every typing_every_ms, user_typing for every thread with a turn running.

Files

• Copy: crates/gatewayd/tests/serve.rs, crates/gatewayd/tests/serve_restart.rs, crates/gatewayd/tests/support/fake_mm.rs, crates/gatewayd/tests/support/gateway.rs, and the skeletons crates/gatewayd/src/serve/mod.rs and crates/gatewayd/src/serve/handle.rs
• Modify: crates/gatewayd/src/lib.rs (pub mod serve;), docs/implementer-log.md

Before anything else

Your first two actions are steps 1 and 2 below: copy the files, and see the tests fail. Do not read the other modules of gatewayd or proto first. Everything the functions you write call is in the table below, with its signature, and the comment above each todo!() gives the code to write, down to the expressions. Write each function as its comment says, run cargo check -p gatewayd, and go on to the next. Do not weigh other ways to write it.

The skeletons

serve/mod.rs, written: the pointers, INTERRUPTED, Log, Stop (Auth, State, Start, Asked) and its Display, Tuning and its default, the Gateway struct, and the glue: run (the connect-or-back-off loop), connect (users/me, the WebSocket, the wait for hello), Gateway::new and Gateway::event_loop (finish turns, send typing, wait for an event, handle it). To fill: From<StateError> for Stop, backoff, sleep_unless, Gateway::connected.

serve/handle.rs, written: the glue catch_up (which channels). To fill: now_ms, post, tracked, finished, typing, start, handle_post, catch_up_channel.

Twelve functions, none more than about twenty lines. run takes the token already loaded and a stop flag, so the tests need no secrets and can end it; task 15's main passes a flag that is never set.

What the functions call

Call	Signature (from the earlier tasks)
self.client.create_post	(&self, channel: &str, root: &str, message: &str) -> Result<Post, MmError>
self.client.posts_since	(&self, channel: &str, since: i64) -> Result<Since, MmError>; Since { posts: Vec<Post>, full: bool }
self.state.seen / knows_thread	(&self, id: &str) -> bool
self.state.handled	(&mut self, post_id: &str, channel: &str, create_at: i64) -> Result<(), StateError>
self.state.since	(&self, channel: &str) -> Option<i64>
self.state.mark	(&mut self, channel: &str, at: i64) -> Result<(), StateError>
self.state.join_thread	(&mut self, root: &str) -> Result<(), StateError>
self.state.start_turn	(&mut self, turn: InFlight) -> Result<(), StateError>; InFlight { session, channel, root }, all String
self.state.end_turn	(&mut self, session: &str) -> Result<(), StateError>
self.state.take_in_flight	(&mut self) -> Result<Vec<InFlight>, StateError>
self.router.route	(&self, post: &Post, channel_type: &str, known: &dyn Fn(&str) -> bool) -> Route
Router::new	(me_id: &str, me_name: &str, users: &[String], channels: &[String]) -> Router
self.queues.push	(&mut self, message: Message) -> Pushed
self.queues.finish	(&mut self, session: &SessionId) -> Option<Batch>
self.queues.threads	(&self) -> Vec<Thread>; Thread { channel, root }
deliver	(poster: &dyn Poster, socket: &Path, batch: &Batch, log: &dyn Fn(&str)); Client is a Poster
typing	(seq: u64, channel: &str, parent: &str) -> String
ws.send_text	(&mut self, text: &str) -> Result<(), WsError>
self.log	Arc<dyn Fn(&str) + Send + Sync>: call it as (self.log)(&line)

? turns a StateError into a Stop through the From you write first.

Steps

• 1. Copy. git switch m4a, then mkdir -p crates/gatewayd/src/serve && cp docs/plans/M4a/files/crates/gatewayd/src/serve/mod.rs docs/plans/M4a/files/crates/gatewayd/src/serve/handle.rs crates/gatewayd/src/serve/, cp docs/plans/M4a/files/crates/gatewayd/tests/serve.rs docs/plans/M4a/files/crates/gatewayd/tests/serve_restart.rs crates/gatewayd/tests/ and cp docs/plans/M4a/files/crates/gatewayd/tests/support/fake_mm.rs docs/plans/M4a/files/crates/gatewayd/tests/support/gateway.rs crates/gatewayd/tests/support/. Add pub mod serve; to lib.rs.
• 2. See it fail. cargo test -p gatewayd --no-fail-fast --test serve --test serve_restart. Expected: it compiles; serve 6 fail; serve_restart 5 fail and 2 pass (a damaged state file and a refused token stop run before anything you write is called).
• 3. Fill mod.rs first (from, backoff, sleep_unless, connected), then handle.rs (now_ms, post, tracked, finished, typing, start, handle_post, catch_up_channel). cargo check -p gatewayd after each function.
• 4. See it pass. cargo test -p gatewayd --test serve --test serve_restart, five times. Expected: 6 and 7 passed each time, in about 2 s.
• 5. Run the gate. cargo fmt --all, then make gate. Expected last line: gate: ok.
• 6. Log and commit. git add crates/gatewayd docs/implementer-log.md Cargo.lock && git commit

Done when

• Both suites pass five times running; make gate prints gate: ok.

Stop and report if

• A comment above a todo!() does not fit its function, or contradicts a test. The comments were written by hand and one was wrong before; stop at once and quote it, rather than guess.
• A test passes only sometimes, or a test takes 5 s or more (that is a wait that timed out, not a pass).
• A written function (run, connect, Gateway::new, event_loop, catch_up) seems to need a change. They are given; report instead.