M4a tasks 08 and 09: finer skeletons after task 08's session wrote nothing

Task 08's `header` was one todo!() with a dozen branches; Ornith planned it in its head until the
turn ran out (tip T25). `next_message` and `header` are now written as glue over seven small
helpers, and task 09's `poll` over four, each checked to pass when filled from its comments.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>

docs/plans/M4a/08-gatewayd-ws-frames.md

@@ -34,19 +34,24 @@ Written: `MAX_MESSAGE`, the opcode constants, `Incoming` (`Text`, `Ping`, `Pong`
 `Close(Option<u16>, String)`), `Decoder { buf, partial }` with `new` and `feed`, and the private
 `Header { fin, opcode, header_len, payload_len }`.
 
-To fill, in this order, each with its steps above it:
+Also written, because they are the glue: **`Decoder::next_message`** (take a whole frame out of
+`buf` and act on it) and **`Decoder::header`** (read the first two bytes, then call the checks and
+`length` below). Read both first: they call everything you write.
 
-1. `Decoder::header(&self) -> Result<Option<Header>, WsError>`: the next frame's header once all of
-   it is in `buf`, checked against every rule above that does not need the payload. `Ok(None)`
-   means "wait for more bytes". It only reads `buf`; it removes nothing.
-2. `close(payload) -> Result<Incoming, WsError>`.
-3. `Decoder::next_message(&mut self)`: uses `header`, and when the whole frame is in `buf`, takes it
-   out (`drain`) and acts on it. Control frames may come between the frames of a text message and
-   are returned at once.
-4. `encode(opcode, payload, mask) -> Vec<u8>`.
+To fill, **one at a time, in this order**, each with its steps above it. Each is a few lines:
 
-No indexing that can go out of bounds, no `as` casts: read lengths with `get(2..4)` and
-`u16::from_be_bytes`, and convert with `usize::try_from` / `u8::try_from`.
+1. `check_first_bytes(b0, b1)`: the reserved bits, the mask bit, the opcode.
+2. `check_control(fin, payload_len)`.
+3. `Decoder::length(&self, short)`: the three length forms. `Ok(None)` means "wait for more bytes".
+4. `Decoder::check_data(&self, opcode, payload_len)`: the rules that depend on `partial`.
+5. `Decoder::data_frame(&mut self, opcode, fin, payload)`: a text or continuation payload.
+6. `close(payload)`.
+7. `encode(opcode, payload, mask)`.
+
+The comment above each `todo!()` gives the constructs to use, down to the expressions for the
+indexing and the conversions. Use them as written: they are checked against the tests, and the
+indexing in them cannot go out of bounds. Do not weigh other ways to write the same thing. Write
+the function, run `cargo check -p gatewayd`, go on to the next.
 
 ## Steps
 
@@ -55,7 +60,7 @@ No indexing that can go out of bounds, no `as` casts: read lengths with `get(2..
   Add `pub mod frame;` to `crates/gatewayd/src/ws/mod.rs`.
 - [ ] **2. See it fail.** `cargo test -p gatewayd --test ws_frame`. Expected: it compiles and 7
   tests fail.
-- [ ] **3. Fill the four functions**, `cargo check -p gatewayd` after each.
+- [ ] **3. Fill the seven functions** in the order above, `cargo check -p gatewayd` after each.
 - [ ] **4. See it pass.** `cargo test -p gatewayd --test ws_frame`. Expected: 7 passed. The last
   test feeds 300 random streams, half of them with bits flipped, in random pieces, and compares
   your decoder with a simple one written inside the test: they must return the same messages, and

docs/plans/M4a/09-gatewayd-ws-conn.md

@@ -32,18 +32,25 @@ impl Ws {
                 mut random: Box<dyn Read + Send>) -> Result<Ws, WsError>;
     fn send(&mut self, opcode: u8, payload: &[u8]) -> Result<(), WsError>;
     pub fn send_text(&mut self, text: &str) -> Result<(), WsError>;
-    pub fn poll(&mut self, wait: Duration) -> Result<Option<String>, WsError>;
+    pub fn poll(&mut self, wait: Duration) -> Result<Option<String>, WsError>;   // written
+    fn take_messages(&mut self) -> Result<Option<String>, WsError>;
+    fn keep_alive(&mut self, now: Instant) -> Result<(), WsError>;
+    fn read_timeout(&self, until: Instant, now: Instant) -> Duration;
+    fn read_some(&mut self, timeout: Duration) -> Result<(), WsError>;
     pub fn close(mut self);
 }
 pub fn host_header(server: &ServerUrl) -> String;
 ```
 
-All are `todo!()`, each with its steps above it. `poll` is the one with the most in it: it returns
+**`poll` is written for you**: it is the glue, and it calls the four helpers after it. It returns
 the next text message, or `None` after about `wait` with none, and does the pinging, the pong
 answers and the dead-peer check on the way. Its read timeout is always the time to the **next
 thing it must do** (the end of `wait`, the next ping, or the dead-after limit), so a quiet
-connection neither spins nor oversleeps. Write it as its comment says, step by step; a helper
-function for step 1 is fine.
+connection neither spins nor oversleeps. Read it first.
+
+Everything else is `todo!()`, each a few lines, with the steps and the expressions to use above it.
+Use them as written; do not weigh other ways to write the same thing. Write the function, run
+`cargo check -p gatewayd`, go on to the next.
 
 `random` gives the handshake key and a fresh 4-byte mask for every frame we send (in `gatewayd`,
 `/dev/urandom`).
@@ -55,8 +62,8 @@ function for step 1 is fine.
   Add `pub mod conn;` to `crates/gatewayd/src/ws/mod.rs`.
 - [ ] **2. See it fail.** `cargo test -p gatewayd --test ws_conn`. Expected: it compiles and 10
   tests fail.
-- [ ] **3. Fill `host_header`, `send`, `send_text`, `open`, `close`, then `poll`**,
-  `cargo check -p gatewayd` after each.
+- [ ] **3. Fill `host_header`, `send`, `send_text`, `open`, `close`, then `take_messages`,
+  `keep_alive`, `read_timeout`, `read_some`**, `cargo check -p gatewayd` after each.
 - [ ] **4. See it pass.** `cargo test -p gatewayd --test ws_conn`, five times. Expected: 10 passed
   each time, in under a second.
 - [ ] **5. Run the gate.** `cargo fmt --all`, then `make gate`. Expected last line: `gate: ok`.

docs/plans/M4a/README.md

@@ -65,6 +65,20 @@ checked to compile against its task's tests and fail them.
 
 At the end of task 15: about 762 tests (650 before task 01).
 
+## Changes during the run
+
+- 2026-09-23, task 08: tasks 01 to 07 committed; task 08's session ended with nothing written. It
+  planned all four `todo!()`s in its head, most of it deliberating how to avoid indexing, and was
+  cut off (tip T25, again: `Decoder::header` has a dozen branches and was one `todo!()`). The
+  tree also had `handshake.rs` with its imports in 2021-edition order, which no logged command
+  explains; it was reverted. The attempt is saved in `.state/runs/M4a/08-first-attempt.diff`.
+  The design model split the skeleton: `next_message` and `header` are now written as glue, over
+  seven helpers of a few lines each whose comments give the exact expressions. The same was done
+  for task 09's `poll` (glue over four helpers) before it could stop the same way. Both new
+  skeletons were checked in a scratch copy of the branch: they compile and fail their tests, and
+  filled literally from their comments they pass (7 of 7, 10 of 10 five times), clippy clean, gate
+  ok. Resume from task 08.
+
 ## Running it
 
 ```sh

docs/plans/M4a/files/crates/gatewayd/src/ws/conn.rs

@@ -58,19 +58,65 @@ impl Ws {
 
     /// The next text message, or `None` after about `wait` with none. Pings are answered and sent
     /// here; a close frame is answered and ends the connection (`Closed`); silence past the
-    /// dead-after limit is `Dead`.
+    /// dead-after limit is `Dead`. Written for you: it is the glue.
     pub fn poll(&mut self, wait: Duration) -> Result<Option<String>, WsError> {
-        // Loop, with `until = now + wait`:
-        // 1. Every whole message the decoder has: Text -> return it; Ping(p) -> send PONG with p;
-        //    Pong -> nothing; Close(code, _) -> send CLOSE with the code as 2 bytes (or empty),
-        //    ignore that error, return Closed.
-        // 2. Silence since last_heard >= dead_after -> Dead.
-        // 3. Since last_ping >= ping_every -> send an empty PING, last_ping = now.
-        // 4. now >= until -> Ok(None).
-        // 5. Read timeout: the least of (until - now), (last_ping + ping_every - now) and
-        //    (last_heard + dead_after - now), at least 1 ms. Read into a 16 KiB buffer: 0 bytes ->
-        //    Closed; n bytes -> feed them, last_heard = now; WouldBlock, TimedOut or Interrupted ->
-        //    go round; any other error -> Io.
+        let until = Instant::now() + wait;
+        loop {
+            if let Some(text) = self.take_messages()? {
+                return Ok(Some(text));
+            }
+            let now = Instant::now();
+            self.keep_alive(now)?;
+            if now >= until {
+                return Ok(None);
+            }
+            let timeout = self.read_timeout(until, now);
+            self.read_some(timeout)?;
+        }
+    }
+
+    /// Act on every whole message the decoder holds, until one is text.
+    fn take_messages(&mut self) -> Result<Option<String>, WsError> {
+        // `while let Some(message) = self.decoder.next_message()? { match message { ... } }`:
+        // - Text(text) -> return Ok(Some(text)).
+        // - Ping(payload) -> `self.send(PONG, &payload)?`.
+        // - Pong(_) -> nothing.
+        // - Close(code, _) -> the reply is the code as 2 big-endian bytes, or empty:
+        //   `code.map(|c| c.to_be_bytes().to_vec()).unwrap_or_default()`. Then
+        //   `let _ = self.send(CLOSE, &reply);` (the peer may be gone) and Err(WsError::Closed).
+        // After the loop: Ok(None).
+        todo!()
+    }
+
+    /// The dead-peer check, and our ping when one is due.
+    fn keep_alive(&mut self, now: Instant) -> Result<(), WsError> {
+        // 1. `now.duration_since(self.last_heard) >= self.timing.dead_after` -> Err(WsError::Dead).
+        // 2. `now.duration_since(self.last_ping) >= self.timing.ping_every` ->
+        //    `self.send(PING, b"")?` and `self.last_ping = now`.
+        // 3. Ok(()).
+        todo!()
+    }
+
+    /// How long the next read may wait: until the soonest of the end of `wait`, the next ping and
+    /// the dead-after limit, and never less than 1 ms.
+    fn read_timeout(&self, until: Instant, now: Instant) -> Duration {
+        // `(self.last_ping + self.timing.ping_every).saturating_duration_since(now)` is the time to
+        // the next ping; the same with last_heard and dead_after; and
+        // `until.saturating_duration_since(now)`. The least of the three (`.min`), then
+        // `.max(Duration::from_millis(1))`.
+        todo!()
+    }
+
+    /// One read, at most `timeout` long, fed to the decoder.
+    fn read_some(&mut self, timeout: Duration) -> Result<(), WsError> {
+        // 1. `self.stream.set_read_timeout(Some(timeout))?`.
+        // 2. `let mut buf = [0u8; 16 * 1024];` and `match self.stream.read(&mut buf)`:
+        //    - Ok(0) -> Err(WsError::Closed).
+        //    - Ok(n) -> `self.decoder.feed(buf.get(..n).unwrap_or_default())`,
+        //      `self.last_heard = Instant::now()`, Ok(()).
+        //    - Err(e) whose `e.kind()` is WouldBlock, TimedOut or Interrupted -> Ok(()): nothing
+        //      came, and `poll` goes round.
+        //    - any other Err(e) -> Err(WsError::Io(e)).
         todo!()
     }
 

docs/plans/M4a/files/crates/gatewayd/src/ws/frame.rs

@@ -50,42 +50,143 @@ impl Decoder {
     }
 
     /// The next whole message, `None` if more bytes are needed, or the error that ends the
-    /// connection. After an error, do not call again.
+    /// connection. After an error, do not call again. Written for you: it is the glue.
     pub fn next_message(&mut self) -> Result<Option<Incoming>, WsError> {
-        // Loop: `self.header()?`, None -> Ok(None). If the buffer holds less than header_len +
-        // payload_len, Ok(None). Otherwise take the payload out and drain the frame from the
-        // buffer, then by opcode: PING -> Ping, PONG -> Pong, CLOSE -> `close(&payload)`. TEXT
-        // starts a new message, CONTINUATION extends `partial`; with fin the whole message must be
-        // UTF-8 (else Protocol) and is returned as Text; without fin it is kept in `partial` and
-        // the loop goes on. Any other opcode is Protocol.
+        loop {
+            let Some(header) = self.header()? else {
+                return Ok(None);
+            };
+            let total = header.header_len.saturating_add(header.payload_len);
+            if self.buf.len() < total {
+                return Ok(None);
+            }
+            let payload: Vec<u8> = self
+                .buf
+                .get(header.header_len..total)
+                .unwrap_or_default()
+                .to_vec();
+            self.buf.drain(..total);
+            match header.opcode {
+                PING => return Ok(Some(Incoming::Ping(payload))),
+                PONG => return Ok(Some(Incoming::Pong(payload))),
+                CLOSE => return close(&payload).map(Some),
+                _ => {
+                    // A text or continuation frame: a whole message, or wait for the next frame.
+                    if let Some(text) = self.data_frame(header.opcode, header.fin, payload)? {
+                        return Ok(Some(Incoming::Text(text)));
+                    }
+                }
+            }
+        }
+    }
+
+    /// The next frame's header, once all of it is in `buf`, checked against every rule that does
+    /// not need the payload. It only reads `buf`. Written for you: it is the glue.
+    fn header(&self) -> Result<Option<Header>, WsError> {
+        let (Some(&b0), Some(&b1)) = (self.buf.first(), self.buf.get(1)) else {
+            return Ok(None);
+        };
+        check_first_bytes(b0, b1)?;
+        let fin = b0 & 0x80 != 0;
+        let opcode = b0 & 0x0F;
+        let Some((header_len, payload_len)) = self.length(b1 & 0x7F)? else {
+            return Ok(None);
+        };
+        if opcode & 0x8 != 0 {
+            check_control(fin, payload_len)?;
+        } else {
+            self.check_data(opcode, payload_len)?;
+        }
+        Ok(Some(Header {
+            fin,
+            opcode,
+            header_len,
+            payload_len,
+        }))
+    }
+
+    /// The header's length and the payload's, from the 7-bit length `short` in byte 1 and the
+    /// bytes after it. `Ok(None)` while those bytes have not all arrived.
+    fn length(&self, short: u8) -> Result<Option<(usize, usize)>, WsError> {
+        // Three forms, a `match short`:
+        // - 0..=125: Ok(Some((2, usize::from(short)))).
+        // - 126: `let Some(bytes) = self.buf.get(2..4) else { return Ok(None) };` then
+        //   `let len = u16::from_be_bytes([bytes[0], bytes[1]]);` (`bytes` has exactly 2 bytes, so
+        //   this indexing cannot fail). len < 126 is Protocol("a length not in its shortest
+        //   form"). Else Ok(Some((4, usize::from(len)))).
+        // - 127: `let Some(bytes) = self.buf.get(2..10) else { return Ok(None) };`, then
+        //   `let mut b = [0u8; 8]; b.copy_from_slice(bytes); let len = u64::from_be_bytes(b);`.
+        //   len >> 63 != 0 is Protocol("a 64-bit length with its top bit set"); len <= 0xFFFF is
+        //   Protocol("a length not in its shortest form"). Else
+        //   Ok(Some((10, usize::try_from(len).unwrap_or(usize::MAX)))).
         todo!()
     }
 
-    /// The next frame's header, once it is all here, checked against every rule that does not need
-    /// the payload.
-    fn header(&self) -> Result<Option<Header>, WsError> {
-        // Return Ok(None) while the bytes the header needs have not all arrived. The rules, each an
-        // error: a reserved bit (0x70 of byte 0) is Protocol; the mask bit (0x80 of byte 1) is
-        // Protocol; an opcode that is not CONTINUATION, TEXT, CLOSE, PING or PONG is Protocol.
-        // Length 126: a u16 in the next 2 bytes, below 126 is Protocol (not the shortest form).
-        // 127: a u64 in the next 8 bytes; top bit set is Protocol; <= 0xFFFF is Protocol. Control
-        // frames (opcode & 0x8): not fin, or over 125 bytes, is Protocol. Data frames: TEXT while
-        // `partial` is Some, or CONTINUATION while it is None, is Protocol; a payload over
-        // MAX_MESSAGE minus what `partial` holds is TooLarge. All of it before any allocation.
+    /// The rules for a text or continuation frame that need what came before.
+    fn check_data(&self, opcode: u8, payload_len: usize) -> Result<(), WsError> {
+        // 1. TEXT while `self.partial` is Some: Protocol("a new message inside an unfinished one").
+        // 2. CONTINUATION while `self.partial` is None: Protocol("a continuation with nothing to
+        //    continue").
+        // 3. `let so_far = self.partial.as_ref().map_or(0, Vec::len);` then
+        //    payload_len > MAX_MESSAGE.saturating_sub(so_far) is TooLarge.
+        // 4. Ok(()).
+        todo!()
+    }
+
+    /// A text or continuation frame's payload, taken out of `buf`: the whole message when `fin`
+    /// is set, or `None` when more frames must come.
+    fn data_frame(
+        &mut self,
+        opcode: u8,
+        fin: bool,
+        payload: Vec<u8>,
+    ) -> Result<Option<String>, WsError> {
+        // 1. The message so far: `Vec::new()` for TEXT, `self.partial.take().unwrap_or_default()`
+        //    for CONTINUATION. Append `payload` to it.
+        // 2. Not `fin`: `self.partial = Some(message);` and Ok(None).
+        // 3. `fin`: `String::from_utf8(message)`; an error is Protocol("text is not UTF-8"); else
+        //    Ok(Some(text)).
         todo!()
     }
 }
 
+/// The rules on the first two bytes alone.
+fn check_first_bytes(b0: u8, b1: u8) -> Result<(), WsError> {
+    // In this order, each a Protocol error: b0 & 0x70 != 0 ("a reserved bit is set");
+    // b1 & 0x80 != 0 ("a frame from the server is masked"); the opcode b0 & 0x0F is not one of
+    // CONTINUATION, TEXT, CLOSE, PING, PONG ("opcode <n>"). Then Ok(()).
+    todo!()
+}
+
+/// The rules for a control frame (close, ping, pong).
+fn check_control(fin: bool, payload_len: usize) -> Result<(), WsError> {
+    // Not `fin`: Protocol("a fragmented control frame"). payload_len > 125: Protocol("a control
+    // frame over 125 bytes"). Else Ok(()).
+    todo!()
+}
+
+/// A close frame's payload.
 fn close(payload: &[u8]) -> Result<Incoming, WsError> {
-    // Empty: Close(None, ""). One byte: Protocol. Otherwise a big-endian u16 code and a UTF-8
-    // reason (else Protocol).
+    // A `match payload` on slice patterns, no indexing:
+    // - `[]` -> Ok(Incoming::Close(None, String::new())).
+    // - `[_]` -> Protocol("a close frame of one byte").
+    // - `[a, b, reason @ ..]` -> the code is `u16::from_be_bytes([*a, *b])`; the reason is
+    //   `String::from_utf8(reason.to_vec())`, whose error is Protocol("a close reason that is not
+    //   UTF-8"). Ok(Incoming::Close(Some(code), reason)).
     todo!()
 }
 
 /// One whole frame from us: FIN set, masked with `mask`.
 pub fn encode(opcode: u8, payload: &[u8], mask: [u8; 4]) -> Vec<u8> {
-    // Byte 0: 0x80 | opcode. Byte 1: 0x80 | length, where the length is the 7-bit form below 126,
-    // 126 then a u16 up to 0xFFFF, else 127 then a u64. Then the 4 mask bytes, then each payload
-    // byte XOR mask[i % 4]. No `as` casts: use try_from.
+    // 1. `let mut out = vec![0x80 | (opcode & 0x0F)];` and `let len = payload.len();`.
+    // 2. The length, with the mask bit 0x80 set in its first byte, in three branches:
+    //    `if len < 126`: push `0x80 | u8::try_from(len).unwrap_or(0)`.
+    //    `else if let Ok(len16) = u16::try_from(len)`: push `0x80 | 126`, then
+    //    `extend_from_slice(&len16.to_be_bytes())`.
+    //    `else`: push `0x80 | 127`, then
+    //    `extend_from_slice(&u64::try_from(len).unwrap_or(u64::MAX).to_be_bytes())`.
+    //    (`unwrap_or` never uses its fallback here; it is how to convert without a panic.)
+    // 3. `out.extend_from_slice(&mask);`
+    // 4. `out.extend(payload.iter().zip(mask.iter().cycle()).map(|(b, m)| b ^ m));` and return out.
     todo!()
 }