boxmaker/docs/plans/M4a/14-gatewayd-serve.md

# M4a task 14: the serve loop

**Branch:** `m4a` (run `git switch m4a`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `gatewayd: serve, the event loop, typing, catch-up and reconnecting`

## Goal

The loop that ties tasks 03 to 13 together (spec sections 8 and 9):

- **Connecting**, at start and after every loss: `GET /users/me`, open the WebSocket, wait for
  `hello`, then log `gatewayd: connected to <url> as <username>`. A connection that fails is tried
  again after 1, 2, 5, 10, then every 30 seconds, with one log line per attempt:
  `gatewayd: cannot reach <url>: <error>; trying again in <n> s`, then a newline and
  `see docs/runbook.md#mattermost-unreachable`. A refused token (401 or 403) is never tried again:
  `run` returns `Stop::Auth`.
- **After a restart** (the first connection only): every turn left in flight gets
  `interrupted: gatewayd restarted before the answer arrived; ask again` in its thread. A later
  reconnect must not do this: those turns are still running.
- **Catching up**: the direct channel with each allowed user, and each allowed channel, from its
  mark. A channel without a mark is marked "now" and not caught up: history is not answered.
- **Each post**, live or caught up: posts in channels we do not track are not recorded at all. A
  tracked post is recorded as handled **before** it is acted on, so after a crash it is not
  answered twice (the in-flight record reports it instead). Then it is routed (task 11): a stranger
  is logged by user id and post id only, **never with the text**.
- **Turns** run on their own threads (task 13), recorded in flight while they run. When one ends,
  its session's waiting messages start the next.
- **Typing**: every `typing_every_ms`, `user_typing` for every thread with a turn running.

## Files

- Copy: `crates/gatewayd/tests/serve.rs`, `crates/gatewayd/tests/serve_restart.rs`,
  `crates/gatewayd/tests/support/fake_mm.rs`, `crates/gatewayd/tests/support/gateway.rs`, and the
  skeletons `crates/gatewayd/src/serve/mod.rs` and `crates/gatewayd/src/serve/handle.rs`
- Modify: `crates/gatewayd/src/lib.rs` (`pub mod serve;`), `docs/implementer-log.md`

## Before anything else

Your **first two actions** are steps 1 and 2 below: copy the files, and see the tests fail. Do not
read the other modules of `gatewayd` or `proto` first. Everything the functions you write call is
in the table below, with its signature, and the comment above each `todo!()` gives the code to
write, down to the expressions. Write each function as its comment says, run
`cargo check -p gatewayd`, and go on to the next. Do not weigh other ways to write it.

## The skeletons

`serve/mod.rs`, written: the pointers, `INTERRUPTED`, `Log`, `Stop` (`Auth`, `State`, `Start`,
`Asked`) and its `Display`, `Tuning` and its default, the `Gateway` struct, and the glue: **`run`**
(the connect-or-back-off loop), **`connect`** (`users/me`, the WebSocket, the wait for `hello`),
**`Gateway::new`** and **`Gateway::event_loop`** (finish turns, send typing, wait for an event,
handle it). To fill: `From<StateError> for Stop`, `backoff`, `sleep_unless`, `Gateway::connected`.

`serve/handle.rs`, written: the glue **`catch_up`** (which channels). To fill: `now_ms`, `post`,
`tracked`, `finished`, `typing`, `start`, `handle_post`, `catch_up_channel`.

Twelve functions, none more than about twenty lines. `run` takes the token already loaded and a
`stop` flag, so the tests need no secrets and can end it; task 15's `main` passes a flag that is
never set.

## What the functions call

| Call | Signature (from the earlier tasks) |
|---|---|
| `self.client.create_post` | `(&self, channel: &str, root: &str, message: &str) -> Result<Post, MmError>` |
| `self.client.posts_since` | `(&self, channel: &str, since: i64) -> Result<Since, MmError>`; `Since { posts: Vec<Post>, full: bool }` |
| `self.state.seen` / `knows_thread` | `(&self, id: &str) -> bool` |
| `self.state.handled` | `(&mut self, post_id: &str, channel: &str, create_at: i64) -> Result<(), StateError>` |
| `self.state.since` | `(&self, channel: &str) -> Option<i64>` |
| `self.state.mark` | `(&mut self, channel: &str, at: i64) -> Result<(), StateError>` |
| `self.state.join_thread` | `(&mut self, root: &str) -> Result<(), StateError>` |
| `self.state.start_turn` | `(&mut self, turn: InFlight) -> Result<(), StateError>`; `InFlight { session, channel, root }`, all `String` |
| `self.state.end_turn` | `(&mut self, session: &str) -> Result<(), StateError>` |
| `self.state.take_in_flight` | `(&mut self) -> Result<Vec<InFlight>, StateError>` |
| `self.router.route` | `(&self, post: &Post, channel_type: &str, known: &dyn Fn(&str) -> bool) -> Route` |
| `Router::new` | `(me_id: &str, me_name: &str, users: &[String], channels: &[String]) -> Router` |
| `self.queues.push` | `(&mut self, message: Message) -> Pushed` |
| `self.queues.finish` | `(&mut self, session: &SessionId) -> Option<Batch>` |
| `self.queues.threads` | `(&self) -> Vec<Thread>`; `Thread { channel, root }` |
| `deliver` | `(poster: &dyn Poster, socket: &Path, batch: &Batch, log: &dyn Fn(&str))`; `Client` is a `Poster` |
| `typing` | `(seq: u64, channel: &str, parent: &str) -> String` |
| `ws.send_text` | `(&mut self, text: &str) -> Result<(), WsError>` |
| `self.log` | `Arc<dyn Fn(&str) + Send + Sync>`: call it as `(self.log)(&line)` |

`?` turns a `StateError` into a `Stop` through the `From` you write first.

## Steps

- [ ] **1. Copy.** `git switch m4a`, then
  `mkdir -p crates/gatewayd/src/serve && cp docs/plans/M4a/files/crates/gatewayd/src/serve/mod.rs docs/plans/M4a/files/crates/gatewayd/src/serve/handle.rs crates/gatewayd/src/serve/`,
  `cp docs/plans/M4a/files/crates/gatewayd/tests/serve.rs docs/plans/M4a/files/crates/gatewayd/tests/serve_restart.rs crates/gatewayd/tests/`
  and
  `cp docs/plans/M4a/files/crates/gatewayd/tests/support/fake_mm.rs docs/plans/M4a/files/crates/gatewayd/tests/support/gateway.rs crates/gatewayd/tests/support/`.
  Add `pub mod serve;` to `lib.rs`.
- [ ] **2. See it fail.** `cargo test -p gatewayd --no-fail-fast --test serve --test serve_restart`.
  Expected: it compiles; `serve` 6 fail; `serve_restart` 5 fail and 2 pass (a damaged state file
  and a refused token stop `run` before anything you write is called).
- [ ] **3. Fill `mod.rs` first** (`from`, `backoff`, `sleep_unless`, `connected`), **then
  `handle.rs`** (`now_ms`, `post`, `tracked`, `finished`, `typing`, `start`, `handle_post`,
  `catch_up_channel`). `cargo check -p gatewayd` after each function.
- [ ] **4. See it pass.** `cargo test -p gatewayd --test serve --test serve_restart`, five times.
  Expected: 6 and 7 passed each time, in about 2 s.
- [ ] **5. Run the gate.** `cargo fmt --all`, then `make gate`. Expected last line: `gate: ok`.
- [ ] **6. Log and commit.** `git add crates/gatewayd docs/implementer-log.md Cargo.lock && git commit`

## Done when

- Both suites pass five times running; `make gate` prints `gate: ok`.

## Stop and report if

- A comment above a `todo!()` does not fit its function, or contradicts a test. The comments were
  written by hand and one was wrong before; stop at once and quote it, rather than guess.
- A test passes only sometimes, or a test takes 5 s or more (that is a wait that timed out, not a
  pass).
- A written function (`run`, `connect`, `Gateway::new`, `event_loop`, `catch_up`) seems to need a
  change. They are given; report instead.