kyle/boxmaker
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

M4a tasks 08 and 09: finer skeletons after task 08's session wrote nothing

Browse Source 
Task 08's `header` was one todo!() with a dozen branches; Ornith planned it in its head until the
turn ran out (tip T25). `next_message` and `header` are now written as glue over seven small
helpers, and task 09's `poll` over four, each checked to pass when filled from its comments.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
kyle
2026-09-23 20:37:48 -07:00
co-authored by Claude Opus 5.5
parent d56fcf8611
commit 02681069b6
6 changed files with 227 additions and 54 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/implementer-lessons.md
+1 -1
View File
@@ -59,7 +59,7 @@ How it is used:
| T22 | List the fail-closed states a task creates, with their runbook anchors, in the task itself. A script can check that a pointer names an existing entry; nothing can check for a pointer that was never written. | M3a finding 3: four startup failures in `serve` and `main` print no pointer, and the spec's own list of pointers omitted them. |
| T23 | A test that writes a script and then runs it must hold a lock shared by every test in its binary that starts a process. Otherwise another test's fork can hold the new script open for writing at that moment, and running it fails with "Text file busy" (ETXTBSY), about once in seven runs. Give the lock in the support file and say why. | M3b plan checks: the fake `curl` tests failed 8 times in 40 until every forking test took `serial()`; then 0 in 60. |
| T24 | A task that changes a `Cargo.toml` must stage `Cargo.lock` in its `git add` line. Better, put `Cargo.lock` in every task's `git add`; it is a no-op when unchanged. | M3b task 04: committed correctly, left the lock behind, and the driver stopped on an unclean tree. |
| T25 | Size a task by the largest function the model must hold in one turn, not by the task. Ornith writes one function with a few branches well; a function with half a dozen branches and threads (M3b's `run`) it plans in its head until the turn runs out, with nothing written. Give such a task a compiling skeleton with the big function already written as glue over small `todo!()` helpers, and say to fill one at a time with `cargo check` between. | M3b task 11: four sessions. Two wrote nothing; a whole-file skeleton got five of six functions; the finer skeleton finished it in ten minutes, and tasks 12 and 13 followed without a stop. |
| T25 | Size a task by the largest function the model must hold in one turn, not by the task. Ornith writes one function with a few branches well; a function with half a dozen branches and threads (M3b's `run`) it plans in its head until the turn runs out, with nothing written. Give such a task a compiling skeleton with the big function already written as glue over small `todo!()` helpers, and say to fill one at a time with `cargo check` between. | M3b task 11: four sessions. Two wrote nothing; a whole-file skeleton got five of six functions; the finer skeleton finished it in ten minutes, and tasks 12 and 13 followed without a stop. M4a task 08: again, a skeleton whose `header` was one `todo!()` with a dozen branches; nothing written. Size every `todo!()`, not only the task: the largest comment above a `todo!()` is a fair proxy, and one over about six lines needs splitting. |
| T26 | Replay each task's end state on its own, **and** read the task file against the reference for anything the reference has that the task does not ask for. The replay proves the tests can pass; only the reading finds a field the reference reads through a getter the task never mentions. | M3b task 11: the task stored `egress_dir` without the reference's getter, so the field was never read and clippy failed; the replay passed because the reference had the getter. |
| T27 | Every wait in a given test has a limit (`recv_timeout`, a deadline loop, `is_finished` before `join`). A test that waits forever on a broken implementation hangs the driver instead of failing, and the implementer cannot tell a hang from slowness. | M4a planning: breaking the reference's ping made `ws_conn` hang on `rx.recv()`; retrying a refused token made the serve test hang on `join()`. Both now fail within 5 s. |
docs/plans/M4a/08-gatewayd-ws-frames.md
+17 -12
View File
@@ -34,19 +34,24 @@ Written: `MAX_MESSAGE`, the opcode constants, `Incoming` (`Text`, `Ping`, `Pong`
`Close(Option<u16>, String)`), `Decoder { buf, partial }` with `new` and `feed`, and the private
`Header { fin, opcode, header_len, payload_len }`.
To fill, in this order, each with its steps above it:
Also written, because they are the glue: **`Decoder::next_message`** (take a whole frame out of
`buf` and act on it) and **`Decoder::header`** (read the first two bytes, then call the checks and
`length` below). Read both first: they call everything you write.
1. `Decoder::header(&self) -> Result<Option<Header>, WsError>`: the next frame's header once all of
it is in `buf`, checked against every rule above that does not need the payload. `Ok(None)`
means "wait for more bytes". It only reads `buf`; it removes nothing.
2. `close(payload) -> Result<Incoming, WsError>`.
3. `Decoder::next_message(&mut self)`: uses `header`, and when the whole frame is in `buf`, takes it
out (`drain`) and acts on it. Control frames may come between the frames of a text message and
are returned at once.
4. `encode(opcode, payload, mask) -> Vec<u8>`.
To fill, **one at a time, in this order**, each with its steps above it. Each is a few lines:
No indexing that can go out of bounds, no `as` casts: read lengths with `get(2..4)` and
`u16::from_be_bytes`, and convert with `usize::try_from` / `u8::try_from`.
1. `check_first_bytes(b0, b1)`: the reserved bits, the mask bit, the opcode.
2. `check_control(fin, payload_len)`.
3. `Decoder::length(&self, short)`: the three length forms. `Ok(None)` means "wait for more bytes".
4. `Decoder::check_data(&self, opcode, payload_len)`: the rules that depend on `partial`.
5. `Decoder::data_frame(&mut self, opcode, fin, payload)`: a text or continuation payload.
6. `close(payload)`.
7. `encode(opcode, payload, mask)`.
The comment above each `todo!()` gives the constructs to use, down to the expressions for the
indexing and the conversions. Use them as written: they are checked against the tests, and the
indexing in them cannot go out of bounds. Do not weigh other ways to write the same thing. Write
the function, run `cargo check -p gatewayd`, go on to the next.
## Steps
@@ -55,7 +60,7 @@ No indexing that can go out of bounds, no `as` casts: read lengths with `get(2..
Add `pub mod frame;` to `crates/gatewayd/src/ws/mod.rs`.
- [ ] **2. See it fail.** `cargo test -p gatewayd --test ws_frame`. Expected: it compiles and 7
tests fail.
- [ ] **3. Fill the four functions**, `cargo check -p gatewayd` after each.
- [ ] **3. Fill the seven functions** in the order above, `cargo check -p gatewayd` after each.
- [ ] **4. See it pass.** `cargo test -p gatewayd --test ws_frame`. Expected: 7 passed. The last
test feeds 300 random streams, half of them with bits flipped, in random pieces, and compares
your decoder with a simple one written inside the test: they must return the same messages, and
docs/plans/M4a/09-gatewayd-ws-conn.md
+13 -6
View File
@@ -32,18 +32,25 @@ impl Ws {
mut random: Box<dyn Read + Send>) -> Result<Ws, WsError>;
fn send(&mut self, opcode: u8, payload: &[u8]) -> Result<(), WsError>;
pub fn send_text(&mut self, text: &str) -> Result<(), WsError>;
pub fn poll(&mut self, wait: Duration) -> Result<Option<String>, WsError>;
pub fn poll(&mut self, wait: Duration) -> Result<Option<String>, WsError>; // written
fn take_messages(&mut self) -> Result<Option<String>, WsError>;
fn keep_alive(&mut self, now: Instant) -> Result<(), WsError>;
fn read_timeout(&self, until: Instant, now: Instant) -> Duration;
fn read_some(&mut self, timeout: Duration) -> Result<(), WsError>;
pub fn close(mut self);
}
pub fn host_header(server: &ServerUrl) -> String;
```
All are `todo!()`, each with its steps above it. `poll` is the one with the most in it: it returns
**`poll` is written for you**: it is the glue, and it calls the four helpers after it. It returns
the next text message, or `None` after about `wait` with none, and does the pinging, the pong
answers and the dead-peer check on the way. Its read timeout is always the time to the **next
thing it must do** (the end of `wait`, the next ping, or the dead-after limit), so a quiet
connection neither spins nor oversleeps. Write it as its comment says, step by step; a helper
function for step 1 is fine.
connection neither spins nor oversleeps. Read it first.
Everything else is `todo!()`, each a few lines, with the steps and the expressions to use above it.
Use them as written; do not weigh other ways to write the same thing. Write the function, run
`cargo check -p gatewayd`, go on to the next.
`random` gives the handshake key and a fresh 4-byte mask for every frame we send (in `gatewayd`,
`/dev/urandom`).
@@ -55,8 +62,8 @@ function for step 1 is fine.
Add `pub mod conn;` to `crates/gatewayd/src/ws/mod.rs`.
- [ ] **2. See it fail.** `cargo test -p gatewayd --test ws_conn`. Expected: it compiles and 10
tests fail.
- [ ] **3. Fill `host_header`, `send`, `send_text`, `open`, `close`, then `poll`**,
`cargo check -p gatewayd` after each.
- [ ] **3. Fill `host_header`, `send`, `send_text`, `open`, `close`, then `take_messages`,
`keep_alive`, `read_timeout`, `read_some`**, `cargo check -p gatewayd` after each.
- [ ] **4. See it pass.** `cargo test -p gatewayd --test ws_conn`, five times. Expected: 10 passed
each time, in under a second.
- [ ] **5. Run the gate.** `cargo fmt --all`, then `make gate`. Expected last line: `gate: ok`.
docs/plans/M4a/README.md
+14
View File
@@ -65,6 +65,20 @@ checked to compile against its task's tests and fail them.
At the end of task 15: about 762 tests (650 before task 01).
## Changes during the run
- 2026-09-23, task 08: tasks 01 to 07 committed; task 08's session ended with nothing written. It
planned all four `todo!()`s in its head, most of it deliberating how to avoid indexing, and was
cut off (tip T25, again: `Decoder::header` has a dozen branches and was one `todo!()`). The
tree also had `handshake.rs` with its imports in 2021-edition order, which no logged command
explains; it was reverted. The attempt is saved in `.state/runs/M4a/08-first-attempt.diff`.
The design model split the skeleton: `next_message` and `header` are now written as glue, over
seven helpers of a few lines each whose comments give the exact expressions. The same was done
for task 09's `poll` (glue over four helpers) before it could stop the same way. Both new
skeletons were checked in a scratch copy of the branch: they compile and fail their tests, and
filled literally from their comments they pass (7 of 7, 10 of 10 five times), clippy clean, gate
ok. Resume from task 08.
## Running it
```sh
docs/plans/M4a/files/crates/gatewayd/src/ws/conn.rs
+58 -12
View File
@@ -58,19 +58,65 @@ impl Ws {
/// The next text message, or `None` after about `wait` with none. Pings are answered and sent
/// here; a close frame is answered and ends the connection (`Closed`); silence past the
/// dead-after limit is `Dead`.
/// dead-after limit is `Dead`. Written for you: it is the glue.
pub fn poll(&mut self, wait: Duration) -> Result<Option<String>, WsError> {
// Loop, with `until = now + wait`:
// 1. Every whole message the decoder has: Text -> return it; Ping(p) -> send PONG with p;
// Pong -> nothing; Close(code, _) -> send CLOSE with the code as 2 bytes (or empty),
// ignore that error, return Closed.
// 2. Silence since last_heard >= dead_after -> Dead.
// 3. Since last_ping >= ping_every -> send an empty PING, last_ping = now.
// 4. now >= until -> Ok(None).
// 5. Read timeout: the least of (until - now), (last_ping + ping_every - now) and
// (last_heard + dead_after - now), at least 1 ms. Read into a 16 KiB buffer: 0 bytes ->
// Closed; n bytes -> feed them, last_heard = now; WouldBlock, TimedOut or Interrupted ->
// go round; any other error -> Io.
let until = Instant::now() + wait;
loop {
if let Some(text) = self.take_messages()? {
return Ok(Some(text));
}
let now = Instant::now();
self.keep_alive(now)?;
if now >= until {
return Ok(None);
}
let timeout = self.read_timeout(until, now);
self.read_some(timeout)?;
}
}
/// Act on every whole message the decoder holds, until one is text.
fn take_messages(&mut self) -> Result<Option<String>, WsError> {
// `while let Some(message) = self.decoder.next_message()? { match message { ... } }`:
// - Text(text) -> return Ok(Some(text)).
// - Ping(payload) -> `self.send(PONG, &payload)?`.
// - Pong(_) -> nothing.
// - Close(code, _) -> the reply is the code as 2 big-endian bytes, or empty:
// `code.map(|c| c.to_be_bytes().to_vec()).unwrap_or_default()`. Then
// `let _ = self.send(CLOSE, &reply);` (the peer may be gone) and Err(WsError::Closed).
// After the loop: Ok(None).
todo!()
}
/// The dead-peer check, and our ping when one is due.
fn keep_alive(&mut self, now: Instant) -> Result<(), WsError> {
// 1. `now.duration_since(self.last_heard) >= self.timing.dead_after` -> Err(WsError::Dead).
// 2. `now.duration_since(self.last_ping) >= self.timing.ping_every` ->
// `self.send(PING, b"")?` and `self.last_ping = now`.
// 3. Ok(()).
todo!()
}
/// How long the next read may wait: until the soonest of the end of `wait`, the next ping and
/// the dead-after limit, and never less than 1 ms.
fn read_timeout(&self, until: Instant, now: Instant) -> Duration {
// `(self.last_ping + self.timing.ping_every).saturating_duration_since(now)` is the time to
// the next ping; the same with last_heard and dead_after; and
// `until.saturating_duration_since(now)`. The least of the three (`.min`), then
// `.max(Duration::from_millis(1))`.
todo!()
}
/// One read, at most `timeout` long, fed to the decoder.
fn read_some(&mut self, timeout: Duration) -> Result<(), WsError> {
// 1. `self.stream.set_read_timeout(Some(timeout))?`.
// 2. `let mut buf = [0u8; 16 * 1024];` and `match self.stream.read(&mut buf)`:
// - Ok(0) -> Err(WsError::Closed).
// - Ok(n) -> `self.decoder.feed(buf.get(..n).unwrap_or_default())`,
// `self.last_heard = Instant::now()`, Ok(()).
// - Err(e) whose `e.kind()` is WouldBlock, TimedOut or Interrupted -> Ok(()): nothing
// came, and `poll` goes round.
// - any other Err(e) -> Err(WsError::Io(e)).
todo!()
}
docs/plans/M4a/files/crates/gatewayd/src/ws/frame.rs
+124 -23
View File
@@ -50,42 +50,143 @@ impl Decoder {
}
/// The next whole message, `None` if more bytes are needed, or the error that ends the
/// connection. After an error, do not call again.
/// connection. After an error, do not call again. Written for you: it is the glue.
pub fn next_message(&mut self) -> Result<Option<Incoming>, WsError> {
// Loop: `self.header()?`, None -> Ok(None). If the buffer holds less than header_len +
// payload_len, Ok(None). Otherwise take the payload out and drain the frame from the
// buffer, then by opcode: PING -> Ping, PONG -> Pong, CLOSE -> `close(&payload)`. TEXT
// starts a new message, CONTINUATION extends `partial`; with fin the whole message must be
// UTF-8 (else Protocol) and is returned as Text; without fin it is kept in `partial` and
// the loop goes on. Any other opcode is Protocol.
loop {
let Some(header) = self.header()? else {
return Ok(None);
};
let total = header.header_len.saturating_add(header.payload_len);
if self.buf.len() < total {
return Ok(None);
}
let payload: Vec<u8> = self
.buf
.get(header.header_len..total)
.unwrap_or_default()
.to_vec();
self.buf.drain(..total);
match header.opcode {
PING => return Ok(Some(Incoming::Ping(payload))),
PONG => return Ok(Some(Incoming::Pong(payload))),
CLOSE => return close(&payload).map(Some),
_ => {
// A text or continuation frame: a whole message, or wait for the next frame.
if let Some(text) = self.data_frame(header.opcode, header.fin, payload)? {
return Ok(Some(Incoming::Text(text)));
}
}
}
}
}
/// The next frame's header, once all of it is in `buf`, checked against every rule that does
/// not need the payload. It only reads `buf`. Written for you: it is the glue.
fn header(&self) -> Result<Option<Header>, WsError> {
let (Some(&b0), Some(&b1)) = (self.buf.first(), self.buf.get(1)) else {
return Ok(None);
};
check_first_bytes(b0, b1)?;
let fin = b0 & 0x80 != 0;
let opcode = b0 & 0x0F;
let Some((header_len, payload_len)) = self.length(b1 & 0x7F)? else {
return Ok(None);
};
if opcode & 0x8 != 0 {
check_control(fin, payload_len)?;
} else {
self.check_data(opcode, payload_len)?;
}
Ok(Some(Header {
fin,
opcode,
header_len,
payload_len,
}))
}
/// The header's length and the payload's, from the 7-bit length `short` in byte 1 and the
/// bytes after it. `Ok(None)` while those bytes have not all arrived.
fn length(&self, short: u8) -> Result<Option<(usize, usize)>, WsError> {
// Three forms, a `match short`:
// - 0..=125: Ok(Some((2, usize::from(short)))).
// - 126: `let Some(bytes) = self.buf.get(2..4) else { return Ok(None) };` then
// `let len = u16::from_be_bytes([bytes[0], bytes[1]]);` (`bytes` has exactly 2 bytes, so
// this indexing cannot fail). len < 126 is Protocol("a length not in its shortest
// form"). Else Ok(Some((4, usize::from(len)))).
// - 127: `let Some(bytes) = self.buf.get(2..10) else { return Ok(None) };`, then
// `let mut b = [0u8; 8]; b.copy_from_slice(bytes); let len = u64::from_be_bytes(b);`.
// len >> 63 != 0 is Protocol("a 64-bit length with its top bit set"); len <= 0xFFFF is
// Protocol("a length not in its shortest form"). Else
// Ok(Some((10, usize::try_from(len).unwrap_or(usize::MAX)))).
todo!()
}
/// The next frame's header, once it is all here, checked against every rule that does not need
/// the payload.
fn header(&self) -> Result<Option<Header>, WsError> {
// Return Ok(None) while the bytes the header needs have not all arrived. The rules, each an
// error: a reserved bit (0x70 of byte 0) is Protocol; the mask bit (0x80 of byte 1) is
// Protocol; an opcode that is not CONTINUATION, TEXT, CLOSE, PING or PONG is Protocol.
// Length 126: a u16 in the next 2 bytes, below 126 is Protocol (not the shortest form).
// 127: a u64 in the next 8 bytes; top bit set is Protocol; <= 0xFFFF is Protocol. Control
// frames (opcode & 0x8): not fin, or over 125 bytes, is Protocol. Data frames: TEXT while
// `partial` is Some, or CONTINUATION while it is None, is Protocol; a payload over
// MAX_MESSAGE minus what `partial` holds is TooLarge. All of it before any allocation.
/// The rules for a text or continuation frame that need what came before.
fn check_data(&self, opcode: u8, payload_len: usize) -> Result<(), WsError> {
// 1. TEXT while `self.partial` is Some: Protocol("a new message inside an unfinished one").
// 2. CONTINUATION while `self.partial` is None: Protocol("a continuation with nothing to
// continue").
// 3. `let so_far = self.partial.as_ref().map_or(0, Vec::len);` then
// payload_len > MAX_MESSAGE.saturating_sub(so_far) is TooLarge.
// 4. Ok(()).
todo!()
}
/// A text or continuation frame's payload, taken out of `buf`: the whole message when `fin`
/// is set, or `None` when more frames must come.
fn data_frame(
&mut self,
opcode: u8,
fin: bool,
payload: Vec<u8>,
) -> Result<Option<String>, WsError> {
// 1. The message so far: `Vec::new()` for TEXT, `self.partial.take().unwrap_or_default()`
// for CONTINUATION. Append `payload` to it.
// 2. Not `fin`: `self.partial = Some(message);` and Ok(None).
// 3. `fin`: `String::from_utf8(message)`; an error is Protocol("text is not UTF-8"); else
// Ok(Some(text)).
todo!()
}
}
/// The rules on the first two bytes alone.
fn check_first_bytes(b0: u8, b1: u8) -> Result<(), WsError> {
// In this order, each a Protocol error: b0 & 0x70 != 0 ("a reserved bit is set");
// b1 & 0x80 != 0 ("a frame from the server is masked"); the opcode b0 & 0x0F is not one of
// CONTINUATION, TEXT, CLOSE, PING, PONG ("opcode <n>"). Then Ok(()).
todo!()
}
/// The rules for a control frame (close, ping, pong).
fn check_control(fin: bool, payload_len: usize) -> Result<(), WsError> {
// Not `fin`: Protocol("a fragmented control frame"). payload_len > 125: Protocol("a control
// frame over 125 bytes"). Else Ok(()).
todo!()
}
/// A close frame's payload.
fn close(payload: &[u8]) -> Result<Incoming, WsError> {
// Empty: Close(None, ""). One byte: Protocol. Otherwise a big-endian u16 code and a UTF-8
// reason (else Protocol).
// A `match payload` on slice patterns, no indexing:
// - `[]` -> Ok(Incoming::Close(None, String::new())).
// - `[_]` -> Protocol("a close frame of one byte").
// - `[a, b, reason @ ..]` -> the code is `u16::from_be_bytes([*a, *b])`; the reason is
// `String::from_utf8(reason.to_vec())`, whose error is Protocol("a close reason that is not
// UTF-8"). Ok(Incoming::Close(Some(code), reason)).
todo!()
}
/// One whole frame from us: FIN set, masked with `mask`.
pub fn encode(opcode: u8, payload: &[u8], mask: [u8; 4]) -> Vec<u8> {
// Byte 0: 0x80 | opcode. Byte 1: 0x80 | length, where the length is the 7-bit form below 126,
// 126 then a u16 up to 0xFFFF, else 127 then a u64. Then the 4 mask bytes, then each payload
// byte XOR mask[i % 4]. No `as` casts: use try_from.
// 1. `let mut out = vec![0x80 | (opcode & 0x0F)];` and `let len = payload.len();`.
// 2. The length, with the mask bit 0x80 set in its first byte, in three branches:
// `if len < 126`: push `0x80 | u8::try_from(len).unwrap_or(0)`.
// `else if let Ok(len16) = u16::try_from(len)`: push `0x80 | 126`, then
// `extend_from_slice(&len16.to_be_bytes())`.
// `else`: push `0x80 | 127`, then
// `extend_from_slice(&u64::try_from(len).unwrap_or(u64::MAX).to_be_bytes())`.
// (`unwrap_or` never uses its fallback here; it is how to convert without a panic.)
// 3. `out.extend_from_slice(&mask);`
// 4. `out.extend(payload.iter().zip(mask.iter().cycle()).map(|(b, m)| b ^ m));` and return out.
todo!()
}