M4a tasks 14 and 15: glue written, calls tabled, after task 14's session wrote nothing

Task 14's session read the crate to learn the APIs and planned `connect` in prose until it was
cut off. `connect`, `Gateway::new` and `catch_up` are now written; the task lists every call with
its signature and makes the copy and the failing test the first actions. Task 15's comment is
rewritten one step per item. Both checked to fail, then pass when filled from their comments.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

docs/plans/M4a/14-gatewayd-serve.md

@@ -33,21 +33,54 @@ The loop that ties tasks 03 to 13 together (spec sections 8 and 9):
   skeletons `crates/gatewayd/src/serve/mod.rs` and `crates/gatewayd/src/serve/handle.rs`
 - Modify: `crates/gatewayd/src/lib.rs` (`pub mod serve;`), `docs/implementer-log.md`
 
+## Before anything else
+
+Your **first two actions** are steps 1 and 2 below: copy the files, and see the tests fail. Do not
+read the other modules of `gatewayd` or `proto` first. Everything the functions you write call is
+in the table below, with its signature, and the comment above each `todo!()` gives the code to
+write, down to the expressions. Write each function as its comment says, run
+`cargo check -p gatewayd`, and go on to the next. Do not weigh other ways to write it.
+
 ## The skeletons
 
 `serve/mod.rs`, written: the pointers, `INTERRUPTED`, `Log`, `Stop` (`Auth`, `State`, `Start`,
-`Asked`) and its `Display`, `Tuning` and its default, the `Gateway` struct, and the two functions
-that are glue: **`run`** (the connect-or-back-off loop) and **`Gateway::event_loop`** (finish turns,
-send typing, wait for an event, handle it). Read both first: they call everything you write.
-To fill: `From<StateError> for Stop`, `backoff`, `sleep_unless`, `connect`, `Gateway::new`,
-`Gateway::connected`.
+`Asked`) and its `Display`, `Tuning` and its default, the `Gateway` struct, and the glue: **`run`**
+(the connect-or-back-off loop), **`connect`** (`users/me`, the WebSocket, the wait for `hello`),
+**`Gateway::new`** and **`Gateway::event_loop`** (finish turns, send typing, wait for an event,
+handle it). To fill: `From<StateError> for Stop`, `backoff`, `sleep_unless`, `Gateway::connected`.
 
-`serve/handle.rs`, all to fill: `now_ms`, `post`, `tracked`, `handle_post`, `start`, `finished`,
-`typing`, `catch_up`.
+`serve/handle.rs`, written: the glue **`catch_up`** (which channels). To fill: `now_ms`, `post`,
+`tracked`, `finished`, `typing`, `start`, `handle_post`, `catch_up_channel`.
 
-Each `todo!()` has its steps above it, with the exact log lines. `run` takes the token already
-loaded and a `stop` flag, so the tests need no secrets and can end it; task 15's `main` passes a
-flag that is never set.
+Twelve functions, none more than about twenty lines. `run` takes the token already loaded and a
+`stop` flag, so the tests need no secrets and can end it; task 15's `main` passes a flag that is
+never set.
+
+## What the functions call
+
+| Call | Signature (from the earlier tasks) |
+|---|---|
+| `self.client.create_post` | `(&self, channel: &str, root: &str, message: &str) -> Result<Post, MmError>` |
+| `self.client.posts_since` | `(&self, channel: &str, since: i64) -> Result<Since, MmError>`; `Since { posts: Vec<Post>, full: bool }` |
+| `self.state.seen` / `knows_thread` | `(&self, id: &str) -> bool` |
+| `self.state.handled` | `(&mut self, post_id: &str, channel: &str, create_at: i64) -> Result<(), StateError>` |
+| `self.state.since` | `(&self, channel: &str) -> Option<i64>` |
+| `self.state.mark` | `(&mut self, channel: &str, at: i64) -> Result<(), StateError>` |
+| `self.state.join_thread` | `(&mut self, root: &str) -> Result<(), StateError>` |
+| `self.state.start_turn` | `(&mut self, turn: InFlight) -> Result<(), StateError>`; `InFlight { session, channel, root }`, all `String` |
+| `self.state.end_turn` | `(&mut self, session: &str) -> Result<(), StateError>` |
+| `self.state.take_in_flight` | `(&mut self) -> Result<Vec<InFlight>, StateError>` |
+| `self.router.route` | `(&self, post: &Post, channel_type: &str, known: &dyn Fn(&str) -> bool) -> Route` |
+| `Router::new` | `(me_id: &str, me_name: &str, users: &[String], channels: &[String]) -> Router` |
+| `self.queues.push` | `(&mut self, message: Message) -> Pushed` |
+| `self.queues.finish` | `(&mut self, session: &SessionId) -> Option<Batch>` |
+| `self.queues.threads` | `(&self) -> Vec<Thread>`; `Thread { channel, root }` |
+| `deliver` | `(poster: &dyn Poster, socket: &Path, batch: &Batch, log: &dyn Fn(&str))`; `Client` is a `Poster` |
+| `typing` | `(seq: u64, channel: &str, parent: &str) -> String` |
+| `ws.send_text` | `(&mut self, text: &str) -> Result<(), WsError>` |
+| `self.log` | `Arc<dyn Fn(&str) + Send + Sync>`: call it as `(self.log)(&line)` |
+
+`?` turns a `StateError` into a `Stop` through the `From` you write first.
 
 ## Steps
 
@@ -58,11 +91,11 @@ flag that is never set.
   `cp docs/plans/M4a/files/crates/gatewayd/tests/support/fake_mm.rs docs/plans/M4a/files/crates/gatewayd/tests/support/gateway.rs crates/gatewayd/tests/support/`.
   Add `pub mod serve;` to `lib.rs`.
 - [ ] **2. See it fail.** `cargo test -p gatewayd --no-fail-fast --test serve --test serve_restart`.
-  Expected: it compiles; `serve` 6 fail; `serve_restart` 6 fail and 1 passes (a damaged state file
-  stops `run` before anything you write is called).
-- [ ] **3. Fill `mod.rs` first** (`from`, `backoff`, `sleep_unless`, `Gateway::new`, `connect`,
-  `connected`), **then `handle.rs`** (`now_ms`, `post`, `tracked`, `finished`, `typing`, `start`,
-  `handle_post`, `catch_up`). `cargo check -p gatewayd` after each function.
+  Expected: it compiles; `serve` 6 fail; `serve_restart` 5 fail and 2 pass (a damaged state file
+  and a refused token stop `run` before anything you write is called).
+- [ ] **3. Fill `mod.rs` first** (`from`, `backoff`, `sleep_unless`, `connected`), **then
+  `handle.rs`** (`now_ms`, `post`, `tracked`, `finished`, `typing`, `start`, `handle_post`,
+  `catch_up_channel`). `cargo check -p gatewayd` after each function.
 - [ ] **4. See it pass.** `cargo test -p gatewayd --test serve --test serve_restart`, five times.
   Expected: 6 and 7 passed each time, in about 2 s.
 - [ ] **5. Run the gate.** `cargo fmt --all`, then `make gate`. Expected last line: `gate: ok`.
@@ -76,4 +109,5 @@ flag that is never set.
 
 - A test passes only sometimes, or a test takes 5 s or more (that is a wait that timed out, not a
   pass).
-- `run` or `event_loop` seems to need a change. They are given; report instead.
+- A written function (`run`, `connect`, `Gateway::new`, `event_loop`, `catch_up`) seems to need a
+  change. They are given; report instead.

docs/plans/M4a/README.md

@@ -78,6 +78,18 @@ At the end of task 15: about 762 tests (650 before task 01).
   skeletons were checked in a scratch copy of the branch: they compile and fail their tests, and
   filled literally from their comments they pass (7 of 7, 10 of 10 five times), clippy clean, gate
   ok. Resume from task 08.
+- 2026-09-23, task 14: tasks 08 to 13 committed, each on its first session. Task 14's session
+  wrote nothing, not even the copies: it read twenty source files to learn the APIs, then planned
+  `connect`'s wait for `hello` in prose ("let me stop designing", and it did not) until it was cut
+  off. Twelve-plus `todo!()`s across two files and the whole crate's API were too much for one
+  turn. The design model wrote `connect`, `Gateway::new` and `catch_up` as glue (leaving
+  `catch_up_channel`), gave `handle_post` and `start` their borrow- and move-sensitive lines
+  verbatim, added a table of every call the helpers make with its signature (checked against the
+  branch), and made copying and seeing the tests fail the first two actions. Task 15's `serve`
+  comment, whose numbered steps had run together, was rewritten one step per item. Both were
+  checked in a scratch copy of the branch at task 14's start: the skeletons compile and fail
+  (`serve` 6, `serve_restart` 5 with 2 passing, `main` 4 with 1 passing), and filled from their
+  comments they pass (five runs), clippy clean, gate ok. Resume from task 14.
 
 ## Running it
 

docs/plans/M4a/files/crates/gatewayd/src/main.rs

@@ -25,13 +25,20 @@ fn main() -> ExitCode {
 }
 
 fn serve(path: &Path) -> ExitCode {
-    // Each failure prints one line (plus its pointer) and returns ExitCode::from(1):
-    // 1. `Config::load`: "gatewayd: <error>\n<START_FAILED>". 2. `token_source`: "gatewayd: <path>:
-    //    <why>\n<START_FAILED>".
-    // 3. `secrets::load(MATTERMOST_TOKEN, &source, &|k| std::env::var_os(k))`: "gatewayd: <error>"
-    //    (it carries its pointer). Print the warning, if any, as it is. 4. Create the state file
-    //    directory, recursive, 0700: "gatewayd: cannot prepare <dir>: <e>\n<START_FAILED>". 5.
-    //    `run` with Tuning::default(), a log that prints each line to standard error, and a stop
-    //    flag that is never set; print the Stop it returns.
+    // Each failure prints its message with `eprintln!` and returns `ExitCode::from(1)`.
+    // 1. `let config = match Config::load(path) { ... }`: an Err(e) prints
+    //    "gatewayd: {e}\n{START_FAILED}".
+    // 2. `let source = match config.token_source() { ... }`: an Err(why) prints
+    //    "gatewayd: {}: {why}\n{START_FAILED}" with `path.display()`.
+    // 3. `let loaded = match secrets::load(MATTERMOST_TOKEN, &source, &|k| std::env::var_os(k))`:
+    //    an Err(e) prints "gatewayd: {e}" (the error carries its own pointer).
+    //    Then `if let Some(warning) = &loaded.warning { eprintln!("{warning}"); }`.
+    // 4. The state file's directory:
+    //    `let dir = config.state_path().parent().map(Path::to_path_buf).unwrap_or_default();`
+    //    `std::fs::DirBuilder::new().recursive(true).mode(0o700).create(&dir)`: an Err(e) prints
+    //    "gatewayd: cannot prepare {}: {e}\n{START_FAILED}" with `dir.display()`.
+    // 5. `let log: gatewayd::serve::Log = Arc::new(|line: &str| eprintln!("{line}"));`
+    //    `let stop = run(config, loaded.secret, Tuning::default(), log, &AtomicBool::new(false));`
+    //    `eprintln!("{stop}");` and `ExitCode::from(1)`.
     todo!()
 }

docs/plans/M4a/files/crates/gatewayd/src/serve/handle.rs

@@ -21,58 +21,101 @@ fn now_ms() -> i64 {
 impl Gateway {
     /// Post in a thread; a failure is logged, not fatal.
     pub(super) fn post(&self, channel: &str, root: &str, text: &str) {
-        // `self.client.create_post`; an error is logged, "gatewayd: cannot post in <channel>
-        // (thread <root>): <error>".
+        // `SystemTime::now().duration_since(UNIX_EPOCH).map(|d| d.as_millis()).unwrap_or(0)`, then
+        // `i64::try_from(ms).unwrap_or(i64::MAX)`.
         todo!()
     }
 
     /// Is this channel one whose posts `gatewayd` keeps track of?
     fn tracked(&self, channel: &str, channel_type: &str) -> bool {
-        // channel_type "D", or the channel is in allow.channels.
+        // `channel_type == "D" || self.config.allow.channels.iter().any(|c| c == channel)`.
         todo!()
     }
 
     /// One new post, live or caught up.
     pub(super) fn handle_post(&mut self, post: &Post, channel_type: &str) -> Result<(), Stop> {
-        // 1. Not tracked, or seen -> nothing. 2. `state.handled(...)?` before anything else. 3.
-        //    Route it: NotAllowed -> log "gatewayd: ignored post <id> from <user>: not allowed"
-        //    (never the message); other Ignore -> nothing; Reply -> post it; Queue -> join the
-        //    thread when joins_thread, then push: Start -> `start`, Waiting -> nothing, Full ->
-        //    post BUSY.
+        // 1. `if !self.tracked(&post.channel_id, channel_type) || self.state.seen(&post.id)`:
+        //    return Ok(()).
+        // 2. `self.state.handled(&post.id, &post.channel_id, post.create_at)?;` before anything
+        //    else: after a crash, a post is never answered twice.
+        // 3. Route it, with the state answering "is this thread ours?":
+        //    `let state = &self.state;`
+        //    `let route = self.router.route(post, channel_type, &|root| state.knows_thread(root));`
+        //    then `match route`:
+        //    - Route::Ignore(Ignored::NotAllowed): log
+        //      "gatewayd: ignored post <post.id> from <post.user_id>: not allowed"
+        //      (never the text).
+        //    - Route::Ignore(_): nothing.
+        //    - Route::Reply { thread, text }: `self.post(&thread.channel, &thread.root, &text)`.
+        //    - Route::Queue(message): if `message.joins_thread`,
+        //      `self.state.join_thread(&message.thread.root)?`; then
+        //      `match self.queues.push(message)`:
+        //      Pushed::Start(batch) -> `self.start(batch)?`; Pushed::Waiting -> nothing;
+        //      Pushed::Full(thread) -> `self.post(&thread.channel, &thread.root, BUSY)`.
+        // 4. Ok(()).
         todo!()
     }
 
     /// Record the turn as in flight and run it on its own thread.
     fn start(&mut self, batch: Batch) -> Result<(), Stop> {
-        // `state.start_turn` (session, channel, root). Spawn with std::thread::Builder: `deliver`
-        // with the client, the loop socket, the batch and the log, then send the session on
-        // done_tx. If spawning fails: log "gatewayd: cannot start a thread for <session>: <e>",
-        // post LOOP_DOWN in the thread, and send the session on done_tx.
+        // 1. `self.state.start_turn(InFlight { session: batch.session.as_str().to_string(),
+        //    channel: batch.thread.channel.clone(), root: batch.thread.root.clone() })?;`
+        // 2. Clone what the thread takes, before the `move`:
+        //    `let (client, socket, log, done) = (Arc::clone(&self.client),
+        //    self.loop_socket.clone(), Arc::clone(&self.log), self.done_tx.clone());`
+        //    `let (session, thread) = (batch.session.clone(), batch.thread.clone());`
+        // 3. `let spawned = std::thread::Builder::new().spawn(move || {
+        //    deliver(client.as_ref(), &socket, &batch, &|line| log(line));
+        //    let _ = done.send(batch.session); });`
+        // 4. `if let Err(e) = spawned`: log "gatewayd: cannot start a thread for <session>: <e>"
+        //    (`session.as_str()`), `self.post(&thread.channel, &thread.root, LOOP_DOWN)`, and
+        //    `let _ = self.done_tx.send(session);` so the session does not stay busy.
+        // 5. Ok(()).
         todo!()
     }
 
     /// Turns that ended: out of flight, and the next batch of each session started.
     pub(super) fn finished(&mut self) -> Result<(), Stop> {
-        // For each session on done_rx (try_recv, never blocking): `end_turn`, then `queues.finish`;
-        // a batch it returns is started.
+        // `while let Ok(session) = self.done_rx.try_recv() {` (never blocks)
+        //    `self.state.end_turn(session.as_str())?;`
+        //    `if let Some(batch) = self.queues.finish(&session) { self.start(batch)?; }` `}`
+        // Then Ok(()).
         todo!()
     }
 
     /// Show this bot as typing in every thread with a turn running.
     pub(super) fn typing(&mut self, ws: &mut Ws) -> Result<(), WsError> {
-        // For each of `queues.threads()`: seq += 1, send `typing(seq, channel, root)` as text.
+        // `for thread in self.queues.threads() { self.seq += 1;
+        //    ws.send_text(&typing(self.seq, &thread.channel, &thread.root))?; }` then Ok(()).
         todo!()
     }
 
     /// Answer what arrived while `gatewayd` was away: the direct channel with each allowed user,
-    /// and each allowed channel. A channel seen for the first time starts from now.
+    /// and each allowed channel. Written for you: it is the glue.
     pub(super) fn catch_up(&mut self) -> Result<(), Stop> {
-        // Channels: the direct channel with each allowed user ("D"; an error is logged "gatewayd:
-        // no direct channel with <user>: <e>" and skipped), then each allowed channel ("O"). For
-        // each: no mark -> `mark(channel, now_ms())` and skip; else `posts_since(channel, mark)`
-        // (an error is logged "gatewayd: cannot catch up <channel>: <e>" and skipped); when full,
-        // log "gatewayd: <channel>: too many posts to catch up; some may be missed"; `handle_post`
-        // each post in order.
+        let mut channels = Vec::new();
+        for user in self.config.allow.users.clone() {
+            match self.client.direct_channel(&self.me.id, &user) {
+                Ok(id) => channels.push((id, "D")),
+                Err(e) => (self.log)(&format!("gatewayd: no direct channel with {user}: {e}")),
+            }
+        }
+        channels.extend(self.config.allow.channels.iter().map(|c| (c.clone(), "O")));
+        for (channel, channel_type) in channels {
+            self.catch_up_channel(&channel, channel_type)?;
+        }
+        Ok(())
+    }
+
+    /// One channel's posts since its mark. A channel seen for the first time starts from now.
+    fn catch_up_channel(&mut self, channel: &str, channel_type: &str) -> Result<(), Stop> {
+        // 1. `let Some(since) = self.state.since(channel) else { ... }`: with no mark,
+        //    `self.state.mark(channel, now_ms())?` and return Ok(()).
+        // 2. `self.client.posts_since(channel, since)`: an Err(e) is logged
+        //    "gatewayd: cannot catch up <channel>: <e>" and returns Ok(()).
+        // 3. When `found.full`, log "gatewayd: <channel>: too many posts to catch up; some may be
+        //    missed".
+        // 4. `for post in &found.posts { self.handle_post(post, channel_type)?; }` and Ok(()).
         todo!()
     }
 }

docs/plans/M4a/files/crates/gatewayd/src/serve/mod.rs

@@ -60,7 +60,7 @@ impl std::fmt::Display for Stop {
 
 impl From<StateError> for Stop {
     fn from(e: StateError) -> Stop {
-        // Stop::State(e).
+        // `Stop::State(e)`.
         todo!()
     }
 }
@@ -107,14 +107,16 @@ pub(crate) struct Gateway {
 
 /// The wait before attempt `n` (from 0) after a loss.
 pub fn backoff(tuning: &Tuning, n: usize) -> Duration {
-    // tuning.backoff[n], or its last entry when n is past the end (30 s if the list is empty). No
-    // indexing.
+    // `let last = tuning.backoff.last().copied().unwrap_or(Duration::from_secs(30));` then
+    // `tuning.backoff.get(n).copied().unwrap_or(last)`.
     todo!()
 }
 
 /// Sleep for `d`, in short steps, unless `stop` is set.
 fn sleep_unless(stop: &AtomicBool, d: Duration) {
-    // Sleep in steps of at most 20 ms until `d` has passed or `stop` is set.
+    // `let until = Instant::now() + d;` then, while `!stop.load(Ordering::SeqCst)` and
+    // `Instant::now() < until`, sleep
+    // `Duration::from_millis(20).min(until.saturating_duration_since(Instant::now()))`.
     todo!()
 }
 
@@ -167,28 +169,69 @@ pub fn run(config: Config, token: Secret, tuning: Tuning, log: Log, stop: &Atomi
 
 /// Who we are, and a WebSocket that has said hello.
 fn connect(config: &Config, client: &Client, tuning: &Tuning) -> Result<(Me, Ws), MmError> {
-    // 1. `client.me()?`. 2. Timing from limits (ping_every_ms, dead_after_ms). 3. Open /dev/urandom
-    //    (an error is Net("/dev/urandom: <e>")). 4. `Ws::open(client.connector(),
-    //    client.token().expose(), timing, ..)`.
-    // 5. Poll with tuning.poll until a text that `parse_event`s to Hello, for at most dead_after.
-    //    Every WebSocket error, and no hello in time, is Net("websocket: <why>").
-    todo!()
+    let me = client.me()?;
+    let timing = Timing {
+        ping_every: Duration::from_millis(config.limits.ping_every_ms),
+        dead_after: Duration::from_millis(config.limits.dead_after_ms),
+    };
+    let random =
+        File::open("/dev/urandom").map_err(|e| MmError::Net(format!("/dev/urandom: {e}")))?;
+    let net = |e: String| MmError::Net(format!("websocket: {e}"));
+    let mut ws = Ws::open(
+        client.connector(),
+        client.token().expose(),
+        timing,
+        Box::new(random),
+    )
+    .map_err(|e| net(e.to_string()))?;
+    // The server says hello first; anything before it is skipped, and silence is an error.
+    let until = Instant::now() + timing.dead_after;
+    while Instant::now() < until {
+        if let Some(text) = ws.poll(tuning.poll).map_err(|e| net(e.to_string()))?
+            && matches!(parse_event(&text), Ok(Event::Hello))
+        {
+            return Ok((me, ws));
+        }
+    }
+    Err(net("no hello from the server".to_string()))
 }
 
 impl Gateway {
     fn new(config: Config, client: Arc<Client>, state: State, log: Log) -> Gateway {
-        // A done channel, a Router with empty ids (set on connect), an empty Me, loop_socket from
-        // config, Queues with limit limits.queue (usize::try_from), seq 0, restarted false.
-        todo!()
+        let (done_tx, done_rx) = mpsc::channel();
+        let limit = usize::try_from(config.limits.queue).unwrap_or(usize::MAX);
+        Gateway {
+            // Both are set on each connection, from `users/me`.
+            router: Router::new("", "", &[], &[]),
+            me: Me {
+                id: String::new(),
+                username: String::new(),
+            },
+            loop_socket: config.loop_socket(),
+            queues: Queues::new(limit),
+            config,
+            client,
+            state,
+            done_tx,
+            done_rx,
+            log,
+            seq: 0,
+            restarted: false,
+        }
     }
 
     /// A connection is up: say so, route as this user, and on the first one, answer the turns a
     /// restart cut off.
     fn connected(&mut self, me: Me) -> Result<(), Stop> {
-        // Log exactly "gatewayd: connected to <url> as <username>". A new Router from me and the
-        // allow lists; store me. The first time only (`restarted`): for each turn `take_in_flight`
-        // gives back, post INTERRUPTED in its channel and root. A later reconnect must not: those
-        // turns are still running.
+        // 1. Log exactly "gatewayd: connected to <url> as <username>"
+        //    (`self.config.mattermost.url`, `me.username`), with `(self.log)(&line)`.
+        // 2. `let allow = &self.config.allow;`
+        //    `self.router = Router::new(&me.id, &me.username, &allow.users, &allow.channels);`
+        //    then `self.me = me;`.
+        // 3. The first time only (`if !self.restarted { self.restarted = true; ... }`): for each
+        //    `turn` in `self.state.take_in_flight()?`, `self.post(&turn.channel, &turn.root,
+        //    INTERRUPTED)`. A later reconnect must not: those turns are still running.
+        // 4. Ok(()).
         todo!()
     }