boxmaker/docs/plans/M5d1/README.md

# M5d-1 implementation plan: epochs and compaction

> For the implementer: each task file is one closed task for one fresh session. Read `AGENTS.md`,
> then the task file, and do exactly that task. The driver is `tools/run-plan.sh`.

**Goal:** a live session can start a new epoch. `loopd` writes `N.jsonl` / `N.baseline.json` for
`N > 0`, turn numbers and `untrusted()` span every epoch, compaction runs after a turn whose last
`Usage` is at or above a configured fraction of `expect.n_ctx`, the summary call uses the
background slot, and `memoryd`'s Honcho walker reads every epoch file of a session.

**Architecture:** `loopd` gains a `[compact]` table and a `compact` module (when to compact, the
summary call, writing the next epoch). `Session::open` loads the highest complete epoch and the
carried summary. `messages` for epoch `n > 0` inserts that carry as a `ChatMessage::User` that is
never a `LogRecord::User`. Honcho concatenates epoch logs in order so `Item::Turn` numbers stay
monotonic. Heartbeat, cron, `ToolSummary`, and tiering are M5d-2.

**Tech stack:** Rust 1.98.1, edition 2024; no new crate.

**Spec:** `docs/specs/2026-10-03-m5d-scheduler.md`, sections 3, 5 (`[compact]` without using
`summarise`), 7 (the M5d-1 tests), and 8 (`session-compact-failed`).

## The owner's decisions (2026-10-03)

1. Compaction runs after a turn, when that turn's `cache_n + prompt_n` is at or above a configured
   fraction of `expect.n_ctx` (default 0.85). A quiet session is not a reason to compact.
2. If a live request is `exceed_context_size_error`, compact and retry that user message once in
   the new epoch.
3. The summary call is on the background slot, with no tools, thinking on. A context-full summary
   falls back to the epoch's `User` texts, newest last, cut to `summary_chars`.
4. The carried summary is not a `LogRecord::User`, so it is not a Honcho turn.

## What the code already has, and what it lacks

- `LogRecord::EpochEnd { time, next, summary }` is in `bxcore` and unused. `messages` already
  ignores it. This plan writes it.
- `Session::open` reads only `0.baseline.json` and `0.jsonl`. `turns()` and `untrusted()` look at
  that file only. `next_call` is recovered from that file only.
- `run_turn` always sends `slots.main`. Live turns stay on the session's live slot (main, from
  `SessionStart`). Only the summary call uses `slots.background`. Changing live turns to the
  session's slot is M5d-2.
- `memoryd`'s indexer already walks every `<epoch>.jsonl`. Honcho's `log_path` is still
  `sessions/<id>/0.jsonl`.
- `Runtime` has no `MemorySource`. Compaction asks `memoryd` for a fresh block from `channel.rs`
  (and `compact::finish_turn`), so given `turn` tests that build a `Runtime` do not change.
- `[compact] summarise` is stored from this plan so a spec-complete `loopd.toml` loads; nothing
  in M5d-1 reads the list. M5d-2 opts in `http_fetch`.

## Global constraints

- Roles depend on `bxcore` only. No `unsafe`, no async runtime, no new crate.
- No file the implementer reads holds a chat-template tag whole (T33). The carry heading is built
  from pieces and named in words.
- Every path a given test creates is removed by a guard (T44); every process a test starts has a
  deadline.
- The fail-closed state `session-compact-failed` has a runbook entry, added by the task that adds
  its pointer (T49).
- Each task gives its baseline and its gate count with what the count includes (T50, T53); the
  branch line is a check, never `git switch` (T51).
- `Cargo.lock` is in the `git add` line (T24). Gate timeout 600000 ms (T40).
- Commit trailer from the driver.

## Tasks

Every task below has been written, replayed from its text and checked; the gate count is the passed
total of `cargo test --workspace` plus one, for the ignored `end_to_end` test the gate runs on
its own. Before task 01 it is **1456** (straylight, rustc 1.98.1 / clippy 0.1.98, 48a229ceae;
master `3726ae5` after the M5d plans). End-state `make gate` on the reference is **1508**.

| # | Delivers | Crates | Gate after | Tests added | Mutations tried, caught |
|---|---|---|---|---|---|
| 01 | `[compact]` on `loopd.toml`: `threshold` 0.85, `summary_chars` 4000, `summarise` default `["http_fetch"]` (stored, unused); `Config::compact_prompt` is `compact.md` beside `system.md` | loopd | 1464 | 8 | 5, 5 |
| 02 | `Session::open` loads the highest epoch that has both a baseline and a log, the carry from the previous `EpochEnd`, `turns` / `untrusted` / `next_call` over every epoch; persist writes the current epoch | loopd | 1473 | 9 | 5, 4 (`log_path` after `open` does not move the already-open `File`) |
| 03 | `messages_with_carry`: epoch `n > 0` is the new system message, then one `ChatMessage::User` (fixed heading plus carry, not a log record), then the new epoch's records; `run_turn` uses `session.messages()` | loopd | 1478 | 5 | 2, 2 |
| 04 | Honcho concatenates every `<epoch>.jsonl` in epoch order; `turn_records` of turn 3 is the epoch-1 user; `send_turn` reads the concatenated log | memoryd | 1484 | 6 | 3, 3 |
| 05 | `loopd::compact`: `due`, `fallback_summary`; starter `config/compact.md` | loopd | 1490 | 6 | 3, 3 |
| 06 | Summary call on the background slot, no tools, thinking on; tag-neutralised and capped; context-full uses the fallback; `session-compact-failed` pointer and runbook entry | loopd | 1496 | 6 | 3, 3 |
| 07 | Writing the next epoch: `EpochEnd`, fresh baseline from `memoryd`, `SessionStart` on the live slot; resume if `EpochEnd` is already the last record | loopd | 1502 | 6 | 2, 2 |
| 08 | After a turn: compact when `due`; on `SessionFull`, compact and retry the user message once; compaction holds the session busy | loopd | 1508 | 6 | 2, 2 (`finish_turn` after `drop(held)` is wiring in `channel.rs`; the given tests call `finish_turn` directly) |

After task 08, the design model runs the M5d-1 device check (spec 7) on straylight: a `bxctl`
session writes a distinctive fact into `memory.md`, continues until compaction with
`[compact] threshold` lowered, and the next turn's baseline contains the fact; `recall` of a
phrase from epoch 0 still hits; `TurnDone.untrusted` stays true across the epoch change after a
tainted recall.

## Known limits

These are known, accepted for M5d-1, and not bugs to fix in a task:

- **Live turns still use `slots.main`.** `SessionStart.slot` is recorded and copied onto the new
  epoch; `run_turn` does not read it yet. Isolated sessions on the background slot are M5d-2.
- **`[compact] summarise` is unused.** Unknown tool names are not a config error until M5d-2.
- **No idle path exists.** Compaction is only called after a turn or on `SessionFull`. There is
  no timer to test.
- **A crash after `EpochEnd` and before the next files exist** reopens epoch 0 with `EpochEnd` at
  the end of the log. The next compact resumes without writing a second `EpochEnd` (task 07).
- **A `SessionFull` retry logs the user message twice**, once in the old epoch (Honcho skips it:
  no reply) and once in the new epoch. Turn numbers stay monotonic.

## Before hand-over

Work through `docs/plans/CHECKLIST.md` with a reference in a scratch worktree with its own build
directory, replayed from its task text; mutations; `/tmp` compared around a gate run; scratch
files in the worktree's `.state/`; the reference moved to straylight as a bundle before the run
(T18). Task files name their log row (T47).

## Running it

The reference (`m5d1-ref`, tasks 01 to 08) is moved to straylight as a bundle and deleted here
before the run (T18). Then, from the repository root, with the OpenCode TUI closed:

```sh
git switch -c m5d1 master
OPENCODE_MODEL=straylight/ornith-1.5-35b-a3b RESUME=1 \
  SLOTS_URL='http://straylight:11434/slots?model=ornith-1.5-35b-a3b' \
  tools/run-plan.sh docs/plans/M5d1 01
```

After task 08, the gate count is 1508. Then the device check above.