boxmaker/docs/plans/M5c1/01-wire-speaker-state.md

# M5c-1 task 01: a turn's speaker, `SessionUpdated`'s turn, and the session-state messages

**Branch:** `m5c1` (run `git switch m5c1`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `bxcore: a turn's speaker, SessionUpdated's turn, the session-state messages`
**Log row:** the Task cell is `M5c1/01-wire-speaker-state`.

## Goal

M5c sends finished turns to Honcho, a memory server shared by a fleet of agents. Before any of
that can be built, the messages between the roles must carry three new things (M5c spec,
sections 5, 7 and 11). This task changes the types in `bxcore` only; the behaviour comes in later
tasks.

1. **Who spoke a turn.** `Turn` gains `speaker: Option<String>`, and the session log's `User`
   record gains the same field. Both are written only when they are `Some`, and a frame or a log
   line without the field reads as `None`. `None` is the safe side: a turn without a speaker is
   never sent to Honcho. So `turn.json` and every log written before M5c keep their bytes.
2. **Which turn ended.** `SessionUpdated`, which `loopd` sends to `memoryd` after each turn, gains
   `turn: u64`. It is **required**: a frame without it is refused, never read as turn 0.
3. **The session's state.** `memoryd` will ask `brokerd` for a session's state on a new socket.
   Two new messages: `SessionState { session }` and the answer `SessionStateReply { class,
   untrusted }`. Both fields of the answer are required.

```json
{"kind":"turn","body":{"session":"…","content":"…","resume":true,"speaker":"kyle"}}
{"type":"user","time":"…","content":"…","speaker":"kyle"}
{"kind":"session_updated","body":{"session":"…","turn":3}}
{"kind":"session_state","body":{"session":"…"}}
{"kind":"session_state_reply","body":{"class":"private","untrusted":true}}
```

The other crates build these types, so they must change to keep compiling, but only as far as
that. Where they build a `Turn` or a `User` record they write `speaker: None`; `loopd` sends
`turn: 0`; `brokerd` gives the two new kinds their names. Later tasks put real values there:
task 02 (`loopd`: the speaker in the log, the turn's number), task 03 (`gatewayd`), task 04
(`bxctl chat --speaker`), task 05 (`brokerd` answers `SessionState`), task 09 (`memoryd` reads
`turn`). Until then nothing reads the new fields, so nothing behaves differently.

## Every place the changed types appear

This is the complete list. Each one is covered by an edit below or by a given file.

| Place | What happens there |
|---|---|
| `crates/bxcore/src/wire.rs`: `Turn`, `Message`, the `use` of `crate::memory` | gains `speaker`; two new variants (edits W1 to W3) |
| `crates/bxcore/src/log.rs`: `LogRecord::User` | gains `speaker` (edit L1) |
| `crates/bxcore/src/memory.rs`: `SessionUpdated` | gains `turn`; the two new structs after it (edit M1) |
| `crates/bxcore/src/lib.rs` | exports the two new structs (edit X1) |
| `crates/brokerd/src/broker.rs`, `kind` | two new arms, and its comment's count (edits B1, B2) |
| `crates/gatewayd/src/deliver.rs`, `one_turn` | `speaker: None` (edit G1) |
| `crates/bxctl/src/chat.rs`, `run_turn` | `speaker: None` (edit C1) |
| `crates/loopd/src/turn.rs`, `run_turn` | `speaker: None` on the `User` record (edit T1) |
| `crates/loopd/src/memory.rs`, `MemoryPort::updated` | `turn: 0` (edit U1) |
| `crates/memoryd/src/index.rs`, the `User` arm | the pattern gains `..`: the speaker is not indexed (edit I1) |
| `crates/loopd/src/baseline.rs`, `crates/loopd/tests/support/turn.rs`, `crates/loopd/tests/neutralise.rs` | patterns with `..`; no change |
| `crates/loopd/src/channel.rs`, `crates/memoryd/src/serve.rs`, `crates/gatewayd/tests/support/*`, `crates/bxctl/tests/support/mod.rs` | take a `Turn` or a `SessionUpdated` apart by name or pass it on; no change |
| tests that build a `Turn`, a `User` record or a `SessionUpdated`, in `bxcore`, `brokerd`, `bxctl`, `loopd`, `memoryd` | new versions are given |

## Files

- Copy, from `docs/plans/M5c1/files/` (the same path under it):
  - new: `crates/bxcore/tests/fixtures/wire/turn_speaker.json`,
    `crates/bxcore/tests/fixtures/wire/session_state.json`,
    `crates/bxcore/tests/fixtures/wire/session_state_reply.json`,
    `crates/bxcore/tests/fixtures/records/session_speaker.jsonl`
  - **new versions** of (they replace the files in the tree):
    `crates/bxcore/tests/fixtures/wire/session_updated.json`;
    `crates/bxcore/tests/turn_wire.rs`, `memory_wire.rs`, `records.rs`, `strict.rs`;
    `crates/brokerd/tests/broker.rs`; `crates/bxctl/tests/chat.rs`;
    `crates/loopd/tests/baseline.rs`, `channel.rs`, `instance.rs`, `memory.rs`,
    `neutralise_replay.rs`, `session.rs`, `turn_done_untrusted.rs`;
    `crates/memoryd/tests/serve.rs`, `crates/memoryd/tests/support/logs.rs`. Your `loopd`
    `memory.rs` is the given `memory.rs-task01`: the given `memory.rs` is task 02's. Your `bxctl`
    `chat.rs` is the given `chat.rs-task01`: the given `chat.rs` is task 04's.
  - Do not copy anything under `docs/plans/M5c1/files/crates/memoryd/tests/fixtures/honcho/`:
    it is for later tasks.
- Modify: `crates/bxcore/src/wire.rs`, `crates/bxcore/src/log.rs`, `crates/bxcore/src/memory.rs`,
  `crates/bxcore/src/lib.rs`, `crates/brokerd/src/broker.rs`, `crates/gatewayd/src/deliver.rs`,
  `crates/bxctl/src/chat.rs`, `crates/loopd/src/turn.rs`, `crates/loopd/src/memory.rs`,
  `crates/memoryd/src/index.rs`, `docs/implementer-log.md`.
- Do not touch: any other source file, any `Cargo.toml`, any file under `crates/gatewayd/tests`.

## Facts you need

- **The optional fields.** `#[serde(default, skip_serializing_if = "Option::is_none")]` on an
  `Option` field does what it says: a missing field reads as `None`, and `None` is not written.
  An unknown key is still refused under `deny_unknown_fields`. It works the same on a field of a
  struct (`Turn`) and on a field of a variant of an internally tagged enum (`LogRecord::User`);
  `ToolResponse::Denied`'s `detail` already uses it. (`serde` reads a missing `Option` field as
  `None` even without `default`; the attribute says so in so many words, as `detail`'s does.)
- **Why `speaker` is optional.** A missing speaker means "not synced", which is the safe way to
  be wrong. And `loopd` replays each session's log byte for byte to the inference server: a log
  written before M5c must read as it is, and a record without a speaker must be written with the
  bytes it had before. The given `neutralise_replay.rs` and `session.rs` check those bytes.
- **Why `turn` is required.** A `u64` field without `#[serde(default)]` is required: decoding a
  `session_updated` without it fails with ``missing field `turn` ``. A default would read a
  missing turn as turn 0, and `memoryd` would later record the wrong turn. `loopd` and `memoryd`
  are built from this one workspace and deployed together, and `SessionUpdated` refuses unknown
  fields, so an older reader refuses the new frame anyway. So: no `#[serde(default)]`, no
  `skip_serializing_if`. The same holds for `SessionStateReply`'s `class` and `untrusted`: a
  default would read a missing field as a clean session. `PROTOCOL_VERSION` does not change.
- **The kinds.** `Message` is `#[serde(tag = "kind", content = "body", rename_all =
  "snake_case")]`, so the variants `SessionState` and `SessionStateReply` are written as
  `"kind":"session_state"` and `"kind":"session_state_reply"` with no attribute of their own.
- **No checks on the name.** `bxcore` does not check a speaker's name. `memoryd` checks it
  against Honcho's pattern in a later task (M5c spec, section 3). A given test sends
  `first.last` and expects it back as it is.
- **Two types named `SessionState`.** `brokerd` has its own `brokerd::policy::SessionState`
  (`taint`, `untrusted`). The new `bxcore::SessionState` is a message. They do not meet in this
  task: `broker.rs` names the new one only as the pattern `Message::SessionState(_)`, which needs
  no import. Do not add `SessionState` to any `use` line in `brokerd`.
- **Placeholders.** `turn: 0` in `loopd` is harmless until task 02: `memoryd` answers
  `SessionUpdated` by reading the session's log, and nothing reads `turn` before task 09.
- The calls and attributes the edits use:

  | Item | What it does |
  |---|---|
  | `#[serde(default, skip_serializing_if = "Option::is_none")]` | on `speaker` (W3, L1): missing reads as `None`; `None` is not written |
  | `#[serde(deny_unknown_fields)]` | on each new struct (M1): an unknown key is refused, as on every message |
  | `LogRecord::User { time, content, .. }` | a pattern that ignores the fields it does not name (I1) |

## The edits, exactly

Nothing else in these files changes. Each quoted "before" block occurs exactly once in its file.
Keep the fields and variants in the order shown: it is the wire format.

### `crates/bxcore/src/wire.rs`

**W1.** The lines

```rust
use crate::memory::{
    MemoryAdmin, MemoryAdminReply, MemoryBlockReply, MemoryCall, Reindexed, SessionUpdated,
};
```

become

```rust
use crate::memory::{
    MemoryAdmin, MemoryAdminReply, MemoryBlockReply, MemoryCall, Reindexed, SessionState,
    SessionStateReply, SessionUpdated,
};
```

**W2.** At the end of `enum Message`, the lines

```rust
    MemoryAdmin(MemoryAdmin),
    MemoryAdminReply(MemoryAdminReply),
}
```

become

```rust
    MemoryAdmin(MemoryAdmin),
    MemoryAdminReply(MemoryAdminReply),
    SessionState(SessionState),
    SessionStateReply(SessionStateReply),
}
```

**W3.** The lines

```rust
// JSON: {"kind":"turn","body":{"session":"…","content":"…","resume":false}}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct Turn {
    pub session: SessionId,
    pub content: String,
    pub resume: bool,
}
```

become

```rust
// JSON: {"kind":"turn","body":{"session":"…","content":"…","resume":false,"speaker":"kyle"}}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct Turn {
    pub session: SessionId,
    pub content: String,
    pub resume: bool,
    /// Who spoke the turn, when one person did (M5c spec, section 11). Written only when present,
    /// and a frame without it reads as `None`, the safe side: a turn without a speaker is never
    /// synced to Honcho.
    #[serde(default, skip_serializing_if = "Option::is_none")]
    pub speaker: Option<String>,
}
```

### `crates/bxcore/src/log.rs`

**L1.** In `enum LogRecord`, the lines

```rust
    User {
        time: Timestamp,
        content: String,
    },
```

become

```rust
    User {
        time: Timestamp,
        content: String,
        /// The turn's speaker (M5c spec, section 11). Written only when present, so a log written
        /// before M5c reads as it is, and a turn without a speaker is logged as before.
        #[serde(default, skip_serializing_if = "Option::is_none")]
        speaker: Option<String>,
    },
```

### `crates/bxcore/src/memory.rs`

**M1.** The lines

```rust
/// `loopd` → `memoryd`: this session's log has new records.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SessionUpdated {
    pub session: SessionId,
}
```

become

```rust
/// `loopd` → `memoryd`: this session's log has new records, those of a turn that has ended.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SessionUpdated {
    pub session: SessionId,
    /// The ended turn's number in the session (M5c spec, section 5). It has no default: a frame
    /// without it is refused, never read as turn 0.
    pub turn: u64,
}

/// `memoryd` → `brokerd`, on `state.sock`: the session's state now (M5c spec, section 7).
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SessionState {
    pub session: SessionId,
}

/// `brokerd` → `memoryd`: the session's class, and whether it has read untrusted content. Neither
/// field has a default: a reply without one is refused, never read as a clean session.
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct SessionStateReply {
    pub class: DataClass,
    pub untrusted: bool,
}
```

`DataClass` and `SessionId` are imported at the top of the file already.

### `crates/bxcore/src/lib.rs`

**X1.** The lines

```rust
pub use memory::{
    ConsolidateEntry, EntryAction, MemoryAdmin, MemoryAdminReply, MemoryArgs, MemoryBlockReply,
    MemoryCall, MemoryCommand, MemoryFile, RecallArgs, Reindexed, SessionUpdated,
};
```

become

```rust
pub use memory::{
    ConsolidateEntry, EntryAction, MemoryAdmin, MemoryAdminReply, MemoryArgs, MemoryBlockReply,
    MemoryCall, MemoryCommand, MemoryFile, RecallArgs, Reindexed, SessionState, SessionStateReply,
    SessionUpdated,
};
```

### `crates/brokerd/src/broker.rs`

**B1.** The line

```rust
/// The snake_case wire name of a message: all twenty-six kinds, none with a leading underscore.
```

becomes

```rust
/// The snake_case wire name of a message: all twenty-eight kinds, none with a leading underscore.
```

**B2.** At the end of `kind`, the lines

```rust
        Message::MemoryAdminReply(_) => "memory_admin_reply",
    }
```

become

```rust
        Message::MemoryAdminReply(_) => "memory_admin_reply",
        Message::SessionState(_) => "session_state",
        Message::SessionStateReply(_) => "session_state_reply",
    }
```

### `crates/gatewayd/src/deliver.rs`

**G1.** In `one_turn`, the lines

```rust
        content: batch.text.clone(),
        resume,
    };
```

become

```rust
        content: batch.text.clone(),
        resume,
        // Set in M5c-1 task 03. Until then no turn has a speaker, and none is synced.
        speaker: None,
    };
```

### `crates/bxctl/src/chat.rs`

**C1.** In `run_turn`, the lines

```rust
        content: content.to_string(),
        resume,
    };
```

become

```rust
        content: content.to_string(),
        resume,
        // Set in M5c-1 task 04, from `--speaker`. Until then no turn has a speaker.
        speaker: None,
    };
```

### `crates/loopd/src/turn.rs`

**T1.** In `run_turn`, the lines

```rust
            time: Timestamp::now(),
            content: neutralise(content).into_owned(),
        })
```

become

```rust
            time: Timestamp::now(),
            content: neutralise(content).into_owned(),
            // Set in M5c-1 task 02, from the turn. `None` is not written: the record's bytes
            // are as before.
            speaker: None,
        })
```

### `crates/loopd/src/memory.rs`

**U1.** In `MemoryPort::updated`, the lines

```rust
        match self.ask(Message::SessionUpdated(SessionUpdated {
            session: session.clone(),
        }))? {
```

become

```rust
        match self.ask(Message::SessionUpdated(SessionUpdated {
            session: session.clone(),
            // Set in M5c-1 task 02. `memoryd` does not read it before M5c-1 task 09.
            turn: 0,
        }))? {
```

### `crates/memoryd/src/index.rs`

**I1.** The line

```rust
            LogRecord::User { time, content } if !content.trim().is_empty() => {
```

becomes

```rust
            LogRecord::User { time, content, .. } if !content.trim().is_empty() => {
```

## Steps

- [ ] **1. Copy the given files.** From the repository root, twenty commands:

  ```sh
  cp docs/plans/M5c1/files/crates/bxcore/tests/fixtures/wire/turn_speaker.json \
    crates/bxcore/tests/fixtures/wire/
  cp docs/plans/M5c1/files/crates/bxcore/tests/fixtures/wire/session_state.json \
    crates/bxcore/tests/fixtures/wire/
  cp docs/plans/M5c1/files/crates/bxcore/tests/fixtures/wire/session_state_reply.json \
    crates/bxcore/tests/fixtures/wire/
  cp docs/plans/M5c1/files/crates/bxcore/tests/fixtures/wire/session_updated.json \
    crates/bxcore/tests/fixtures/wire/
  cp docs/plans/M5c1/files/crates/bxcore/tests/fixtures/records/session_speaker.jsonl \
    crates/bxcore/tests/fixtures/records/
  cp docs/plans/M5c1/files/crates/bxcore/tests/turn_wire.rs crates/bxcore/tests/
  cp docs/plans/M5c1/files/crates/bxcore/tests/memory_wire.rs crates/bxcore/tests/
  cp docs/plans/M5c1/files/crates/bxcore/tests/records.rs crates/bxcore/tests/
  cp docs/plans/M5c1/files/crates/bxcore/tests/strict.rs crates/bxcore/tests/
  cp docs/plans/M5c1/files/crates/brokerd/tests/broker.rs crates/brokerd/tests/
  cp docs/plans/M5c1/files/crates/bxctl/tests/chat.rs-task01 \
    crates/bxctl/tests/chat.rs
  cp docs/plans/M5c1/files/crates/loopd/tests/baseline.rs crates/loopd/tests/
  cp docs/plans/M5c1/files/crates/loopd/tests/channel.rs crates/loopd/tests/
  cp docs/plans/M5c1/files/crates/loopd/tests/instance.rs crates/loopd/tests/
  cp docs/plans/M5c1/files/crates/loopd/tests/memory.rs-task01 \
    crates/loopd/tests/memory.rs
  cp docs/plans/M5c1/files/crates/loopd/tests/neutralise_replay.rs crates/loopd/tests/
  cp docs/plans/M5c1/files/crates/loopd/tests/session.rs crates/loopd/tests/
  cp docs/plans/M5c1/files/crates/loopd/tests/turn_done_untrusted.rs crates/loopd/tests/
  cp docs/plans/M5c1/files/crates/memoryd/tests/serve.rs crates/memoryd/tests/
  cp docs/plans/M5c1/files/crates/memoryd/tests/support/logs.rs crates/memoryd/tests/support/
  ```

  Then `git status --short` lists 16 modified files and 4 new ones, all under `crates/*/tests/`.
- [ ] **2. See them fail.** Two commands:
  - `cargo test -p bxcore --test strict`: two tests fail:
    `envelopes_reject_unknown_keys_at_every_depth`, with `turn_speaker.json: fixture must decode`,
    and `speaker_log_records_reject_unknown_keys_at_every_depth`, with
    `session_speaker.jsonl:2: fixture must decode`.
  - `cargo build --tests --keep-going -p bxcore -p brokerd -p bxctl -p loopd -p memoryd`: fifteen
    test targets do not compile: `memory_wire`, `records` and `turn_wire` in `bxcore`; `broker`
    in `brokerd`; `chat` in `bxctl`; `baseline`, `channel`, `instance`, `memory`,
    `neutralise_replay`, `session` and `turn_done_untrusted` in `loopd`; `index`, `recall` and
    `serve` in `memoryd`. Every error is about a name this task adds: `E0559` (``variant
    `LogRecord::User` has no field named `speaker` ``), `E0560` (``struct `bxcore::Turn` has no
    field named `speaker` `` or ``struct `bxcore::SessionUpdated` has no field named `turn` ``),
    `E0432` (unresolved imports `bxcore::SessionState`, `bxcore::SessionStateReply`) or `E0599`
    (no variant `SessionState` or `SessionStateReply` for enum `Message`).
- [ ] **3. `bxcore`.** Make edits W1, W2, W3, L1, M1 and X1. `cargo test -p bxcore`: every suite
  passes, among them `turn_wire` (9 tests), `memory_wire` (5), `records` (4) and `strict` (6).
- [ ] **4. The other crates.** `cargo build --workspace --keep-going` now fails in exactly six
  places: `kind` in `brokerd` (`E0004`, non-exhaustive patterns); `one_turn` in `gatewayd`,
  `run_turn` in `bxctl` and `run_turn` in `loopd` (`E0063`, missing field `speaker`);
  `MemoryPort::updated` in `loopd` (`E0063`, missing field `turn`); and the `User` arm in
  `memoryd`'s `index.rs` (`E0027`, pattern does not mention field `speaker`). Make edits B1, B2,
  G1, C1, T1, U1 and I1. Then `cargo test -p brokerd -p gatewayd -p bxctl -p loopd -p memoryd`:
  every suite passes.
- [ ] **5. Check.** Three commands:
  - `grep -c 'serde(default' crates/bxcore/src/wire.rs` prints `2`: `detail` of
    `ToolResponse::Denied` and `speaker` of `Turn`. Nothing on `turn`, `class` or `untrusted`.
  - `git diff --stat -- crates/gatewayd/tests crates/inferproxy crates/toolkit` prints nothing.
  - `git diff --stat -- crates/*/src` lists exactly the ten source files of "Modify".
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, with a shell timeout of 600000 ms.
  Expected last line: `gate: ok`, with 1130 tests: 1122 before this task, plus 8 new ones (two
  in `bxcore/tests/turn_wire.rs`, three in `memory_wire.rs`, one in `records.rs`, one in
  `strict.rs`, one in `brokerd/tests/broker.rs`; the other new versions keep their counts).
- [ ] **7. Log and commit.** Add your row to `docs/implementer-log.md`; its Task cell is
  `M5c1/01-wire-speaker-state`. `git status --short` lists the ten sources, the sixteen replaced
  test files and your log as modified, and the four new fixtures. Stage with
  `git add crates/bxcore crates/brokerd crates/gatewayd crates/bxctl crates/loopd crates/memoryd docs/implementer-log.md Cargo.lock`,
  then commit as `AGENTS.md` says. `Cargo.lock` does not change in this task; the `git add` is
  harmless.

## Done when

- `make gate` prints `gate: ok` with 1130 tests.
- `git diff HEAD~1 --stat` lists only the ten source files of "Modify", the twenty copied files
  and `docs/implementer-log.md`.

## Stop and report if

- A "before" block of any edit is not in its file exactly as shown, or is there more than once.
- Step 2 shows a different result: other tests failing in `strict`, other test targets, or an
  error of another kind.
- Step 4's build fails anywhere other than the six places it names.
- After step 3 or 4, a test fails. Do not change a test or a fixture: stop, and give the failing
  tests' names and their messages.
- The fix seems to need a change anywhere else: another source file, a test under
  `crates/gatewayd/tests`, a default for `turn`, `class` or `untrusted`, or a check on the
  speaker's name. It does not; stop and say why you think so.
- A test in `inferproxy` or `toolkit` fails in the gate: check once whether it fails without your
  change, then stop and report both results.
- The gate passes with a count other than 1130.
- A comment in an edit does not fit the code under it. Quote it and stop.