boxmaker/docs/plans/M5d1/04-memoryd-honcho-epochs.md

# M5d-1 task 04: Honcho reads every epoch file

**Branch:** `m5d1`. Check it: `git branch --show-current` prints `m5d1` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `memoryd: Honcho concatenates every epoch of a session`
**Log row:** the Task cell is `M5d1/04-memoryd-honcho-epochs`.

## Goal

`memoryd` keys a turn as `Item::Turn { session, turn }` with no epoch (spec 3.2). After
compaction, turn 3 may live in `1.jsonl`. `log_path` still names `0.jsonl` (existing tests pin
that path). A new `read_session` concatenates every `<n>.jsonl` in epoch order. `TurnSync`
walks that concatenation, and the worker's `send_turn` does too, so a turn in epoch 1 is
decided and sent.

`turn_records` / `turns` do not change: they already count `User` records on a slice.

## Every place that changes

| Place | What happens there |
|---|---|
| `crates/memoryd/src/honcho/turns.rs` | `epoch_logs`; `read_session`; `records` and `resume` use it |
| `crates/memoryd/src/honcho/worker.rs` | `send_turn` reads the concatenated log (W1–W3) |
| `crates/memoryd/tests/honcho_epochs.rs` | new, given |
| everything else | **no change**: not `log_path`'s return value, not `decide.rs` |

## Files

- Copy, from `docs/plans/M5d1/files/` (the same path under it): the new test
  `crates/memoryd/tests/honcho_epochs.rs`.
- Modify: `crates/memoryd/src/honcho/turns.rs`, `crates/memoryd/src/honcho/worker.rs`,
  `docs/implementer-log.md`.
- Do not touch: any other file.

## Facts you need

- **`read_records` / `LogError`** already exist in `turns.rs`. `read_session` returns
  `Result<Vec<LogRecord>, LogError>` so a damaged line still names its file and
  `LOG_RUNBOOK`.
- **Listing.** Under `sessions/<id>/`, every file named `<n>.jsonl` where `n` parses as `u32`,
  sorted by `n`. A missing session directory is empty, not an error (same as a missing
  `0.jsonl` today).
- **`log_path` stays** `.../0.jsonl`. Do not change its body. `readable` in the worker still
  names that path when the concatenated read succeeds, so existing recovery log lines stay
  stable.
- **`send_turn` today** calls `read_turn(&log_path(...), turn)`, which only sees epoch 0. After
  this task it loads `read_session`, then `turn_records` / `user_text` / `final_reply` from
  `crate::honcho::decide` (already `pub`). `read_turn` maps a damaged line to `Ok(None)` (the
  turn is marked failed, kind `log`). Keep that: `LogError::Damaged` is `Unsent::Local("log")`;
  an I/O error stays `Unsent::Unreadable` and is retried.
- `turns.rs` after this task stays under 500 lines (it is 332 now).
- The items the edits use. This table is complete.

  | Item | What it does |
  |---|---|
  | `read_records(&Path) -> Result<Vec<LogRecord>, LogError>` | one file, already in `turns.rs` |
  | `turn_records(&[LogRecord], u64) -> Option<&[LogRecord]>` | `decide.rs` |
  | `user_text(&[LogRecord]) -> (&str, Option<&str>)` | `decide.rs` |
  | `final_reply(&[LogRecord]) -> Option<&str>` | `decide.rs` |
  | `str::strip_suffix(".jsonl")`, `str::parse::<u32>` | epoch numbers |
  | `Unsent::Unreadable(String)` | worker, `e.to_string()` |

## The edits, exactly

### `crates/memoryd/src/honcho/turns.rs`

**H1.** The doc comment on `log_path`, the lines

```rust
/// The path of `session`'s log under `sessions`: epoch 0, the only one `loopd` writes.
pub fn log_path(sessions: &Path, session: &SessionId) -> PathBuf {
```

become

```rust
/// The path of `session`'s epoch-0 log under `sessions`. Later epochs are sibling files;
/// `read_session` concatenates them. Existing tests pin this path.
pub fn log_path(sessions: &Path, session: &SessionId) -> PathBuf {
```

**H1b.** After `read_records` (the function that ends `Ok(records)` / `}`), insert these two
functions. They sit below `LogError` and `read_records`, so both names resolve.

```rust
/// Every `<n>.jsonl` of `session`, sorted by `n`. A missing directory is empty.
pub fn epoch_logs(sessions: &Path, session: &SessionId) -> Result<Vec<PathBuf>, LogError> {
    let dir = sessions.join(session.as_str());
    let entries = match std::fs::read_dir(&dir) {
        Ok(e) => e,
        Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(Vec::new()),
        Err(e) => return Err(LogError::Io(format!("{}: {e}", dir.display()))),
    };
    let mut found: Vec<(u32, PathBuf)> = Vec::new();
    for entry in entries {
        let entry = match entry {
            Ok(e) => e,
            Err(e) => return Err(LogError::Io(format!("{}: {e}", dir.display()))),
        };
        let name = entry.file_name();
        let Some(name) = name.to_str() else { continue };
        let Some(stem) = name.strip_suffix(".jsonl") else { continue };
        let Ok(n) = stem.parse::<u32>() else { continue };
        found.push((n, entry.path()));
    }
    found.sort_by_key(|(n, _)| *n);
    Ok(found.into_iter().map(|(_, p)| p).collect())
}

/// Every epoch's records, in epoch order. A missing session has none.
pub fn read_session(
    sessions: &Path,
    session: &SessionId,
) -> Result<Vec<LogRecord>, LogError> {
    let mut out = Vec::new();
    for path in epoch_logs(sessions, session)? {
        out.extend(read_records(&path)?);
    }
    Ok(out)
}
```

The next function after this insert is `finished`. Do not change `finished`.

**H2.** In `resume`, the line

```rust
            if let Ok(records) = read_log(&log_path(&self.sessions, &session)) {
```

becomes

```rust
            if let Ok(records) = read_session(&self.sessions, &session) {
```

**H3.** In `records`, the line

```rust
        match read_log(&log_path(&self.sessions, session)) {
```

becomes

```rust
        match read_session(&self.sessions, session).map_err(|e| e.to_string()) {
```

`read_log` already mapped `LogError` through `Display`; this keeps the logged line the same
shape (`memoryd: {why}; {LOG_RUNBOOK}`).

### `crates/memoryd/src/honcho/worker.rs`

**W1.** The `use` line

```rust
use crate::honcho::turns::{TurnSync, log_path};
```

becomes

```rust
use crate::honcho::decide::{final_reply, turn_records, user_text};
use crate::honcho::turns::{LogError, TurnSync, log_path, read_session};
```

**W2.** The body of `send_turn`, the lines

```rust
        let path = log_path(&self.sessions, session);
        let read = read_turn(&path, turn).map_err(Unsent::Unreadable)?;
        self.readable(&path);
        let (text, reply) = read.ok_or(Unsent::Local("log"))?;
```

become

```rust
        let path = log_path(&self.sessions, session);
        let records = match read_session(&self.sessions, session) {
            Ok(records) => {
                self.readable(&path);
                records
            }
            Err(LogError::Damaged(_)) => return Err(Unsent::Local("log")),
            Err(e) => return Err(Unsent::Unreadable(e.to_string())),
        };
        let slice = turn_records(&records, turn).ok_or(Unsent::Local("log"))?;
        let (text, _) = user_text(slice);
        let reply = final_reply(slice).ok_or(Unsent::Local("log"))?;
```

**W3.** The `batch` call, the line

```rust
        let messages = batch(&text, &reply, speaker, &self.agent, session, turn);
```

becomes

```rust
        let messages = batch(text, reply, speaker, &self.agent, session, turn);
```

`text` and `reply` are already `&str`; clippy denies the extra borrows (`needless_borrow`).
The rest of `send_turn` is unchanged (`honcho.ok_or`, `MAX_BATCH`, `add_messages`).

## Steps

- [ ] **1. Copy the given test.**

  ```sh
  cp docs/plans/M5d1/files/crates/memoryd/tests/honcho_epochs.rs crates/memoryd/tests/
  ```

- [ ] **2. See it fail.** `cargo build --tests --keep-going -p memoryd`, timeout 600000 ms: test
  target `honcho_epochs` does not compile (`E0432` unresolved import
  `memoryd::honcho::turns::read_session`). Every other target compiles.
- [ ] **3. The edits.** H1, H1b, H2, H3, W1, W2, W3. `cargo check -p memoryd`
  prints no error and no warning. `wc -l crates/memoryd/src/honcho/turns.rs` and
  `wc -l crates/memoryd/src/honcho/worker.rs` are each ≤ 500.
- [ ] **4. The tests.** `cargo test -p memoryd --test honcho_epochs --test honcho_turns --test
  honcho_worker`: every test passes, 6 in `honcho_epochs` and the others as before.
- [ ] **5. Check.** `git diff --stat` lists exactly `crates/memoryd/src/honcho/turns.rs` and
  `crates/memoryd/src/honcho/worker.rs`.
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, timeout 600000 ms. Last line
  `gate: ok`. Test count **1484** (1478 before, plus 6).
- [ ] **7. Log and commit.** Task cell `M5d1/04-memoryd-honcho-epochs`. Stage with
  `git add crates/memoryd docs/implementer-log.md Cargo.lock`, then commit as `AGENTS.md` says.

## Done when

- `make gate` prints `gate: ok` with 1484 tests.
- `git diff HEAD~1 --stat` lists only the two `honcho` sources, `honcho_epochs.rs`, and the log.

## Stop and report if

- The branch check fails; a "before" block is not found exactly once; step 2 differs; a file
  exceeds 500 lines; a test fails; the gate count is not 1484.
- You think `log_path` should return every file. It must still return `0.jsonl`.

## What the given tests pin

Two users in epoch 0 and one in epoch 1 is `turns() == 3` and `turn_records(..., 3)` is the
epoch-1 user; epoch 0 alone still reads; a damaged `1.jsonl` names that file; `TurnSync` decides
turn 3 from epoch 1; `send_turn` posts turn 3 from epoch 1 (three worker steps, oldest first);
a missing session directory is empty.