kyleandGrok 4.6
3726ae585f
Eight closed tasks for epochs and compaction, then ten for the scheduler, ToolSummary, and tiers. Given tests first. Gate 1456 → 1507 → 1572. Co-Authored-By: Grok 4.6 <noreply@x.ai>
262 lines
9.9 KiB
Markdown
262 lines
9.9 KiB
Markdown
# M5d-1 task 08: compact after a turn, and retry once on `SessionFull`
|
|
|
|
**Branch:** `m5d1`. Check it: `git branch --show-current` prints `m5d1` and `git status --short`
|
|
is empty; otherwise stop. Never switch branch.
|
|
**Commit subject:** `loopd: compact after a full turn and retry SessionFull once`
|
|
**Log row:** the Task cell is `M5d1/08-loopd-compact-turn`.
|
|
|
|
## Goal
|
|
|
|
After `run_turn` returns, `finish_turn` decides whether to compact (spec 3.3):
|
|
|
|
- A successful turn whose last `Usage` is `due` is compacted before the caller returns. If
|
|
compact fails, the turn stays successful and the session stays on its epoch; `loopd` logs the
|
|
compact error with the runbook pointer.
|
|
- `TurnError::SessionFull` is compacted, then that user message is `run_turn`'d once in the new
|
|
epoch. If compact fails, the error stays `SessionFull`. The retry is not compacted again.
|
|
- Any other `TurnError` is returned as it is. No compact.
|
|
|
|
`Runtime` does not gain `MemorySource` (given `turn` tests build a `Runtime` without one).
|
|
`finish_turn` takes `memory` as its own argument.
|
|
|
|
The channel holds the session busy for this whole call: `handle` runs `finish_turn` **before**
|
|
`drop(held)`. A compact is not an idle timer and does not start after the busy set releases.
|
|
|
|
Live turns still send `slots.main` (task 03). Only the summary call uses the background slot.
|
|
|
|
## Every place that changes
|
|
|
|
| Place | What happens there |
|
|
|---|---|
|
|
| `crates/loopd/src/compact.rs` | `finish_turn` and its two helpers (edit F1) |
|
|
| `crates/loopd/src/channel.rs` | `handle` calls `finish_turn` before `drop(held)` (edits H1, H2) |
|
|
| `crates/loopd/tests/compact_turn.rs` | new, given |
|
|
| everything else | **no change**: not `turn.rs`, not `Runtime`, not existing `turn` / `channel` tests |
|
|
|
|
## Files
|
|
|
|
- Copy, from `docs/plans/M5d1/files/` (the same path under it): the new test
|
|
`crates/loopd/tests/compact_turn.rs`.
|
|
- Modify: `crates/loopd/src/compact.rs`, `crates/loopd/src/channel.rs`,
|
|
`docs/implementer-log.md`.
|
|
- Do not touch: any other file.
|
|
|
|
## Facts you need
|
|
|
|
- **`due` looks at `session.last_usage()`**, which is the current epoch's last `Usage`. After a
|
|
successful `run_turn` that record is there. After `SessionFull`, `run_turn` has already
|
|
appended the `User` and returned before any `Usage`; compact runs because of the error, not
|
|
because of `due`.
|
|
- **Compact helper.** `summarise` then `commit`. Log failures with
|
|
|
|
```text
|
|
loopd: could not compact session <id>: <CompactError Display>
|
|
```
|
|
|
|
`Display` already includes the runbook pointer, matching the runbook's example line.
|
|
- **Retry.** `run_turn` appends the user message again in the new epoch. The old epoch keeps its
|
|
`User` (Honcho skips it: no reply). `turns()` is 2. That is accepted (README, known limits).
|
|
- **Existing channel tests stay green.** Their `n_ctx` is 131072 and fixture `prompt_n` is tens,
|
|
so `due` is false. `SessionFull` without `compact.md` in `Home` stays `SessionFull` (the
|
|
successful-turn-when-compact-fails behaviour, on the error path).
|
|
- **`compact.rs` stays ≤ 500 lines.** After this task it is about 250.
|
|
- **`channel.rs` stays ≤ 500 lines.** After this task it is about 290.
|
|
- The items the edits use. This table is complete.
|
|
|
|
| Item | What it does |
|
|
|---|---|
|
|
| `due` | task 05 |
|
|
| `summarise` / `commit` | tasks 06, 07 |
|
|
| `run_turn` | retry the same `text` and `speaker` |
|
|
| `Session::last_usage` | current epoch |
|
|
| `Config.compact.threshold`, `expect.n_ctx` | the `due` arguments |
|
|
| `MemorySource` | `finish_turn`'s extra argument; channel passes `&*ctx.memory` |
|
|
| `Held` | already in `channel.rs`; drop it after `finish_turn` |
|
|
|
|
## The edits, exactly
|
|
|
|
Each quoted "before" block occurs exactly once after task 07 (`channel.rs` is still as on
|
|
master, except rustfmt).
|
|
|
|
### `crates/loopd/src/compact.rs`
|
|
|
|
**F1.** Extend the crate uses. After task 07 they include `MemorySource`, `Session`, `Config`,
|
|
`Registry`. Add:
|
|
|
|
```rust
|
|
use crate::turn::{Runtime, TurnError, TurnOutcome, run_turn};
|
|
use bxcore::TurnEvent;
|
|
```
|
|
|
|
Keep rustfmt's order of `use` lines; if it moves them, accept that.
|
|
|
|
Then after `commit`, add these three functions whole. There is no `todo!()`.
|
|
|
|
```rust
|
|
/// After `run_turn`: compact when the last `Usage` is due, or on `SessionFull` (then retry the
|
|
/// user message once). Compact failure on a successful turn keeps the success; on `SessionFull`
|
|
/// keeps `SessionFull`. Other errors skip compact.
|
|
pub fn finish_turn(
|
|
session: &mut Session,
|
|
rt: &Runtime<'_>,
|
|
memory: &dyn MemorySource,
|
|
text: &str,
|
|
speaker: Option<&str>,
|
|
on_event: &mut dyn FnMut(&TurnEvent),
|
|
outcome: Result<TurnOutcome, TurnError>,
|
|
) -> Result<TurnOutcome, TurnError> {
|
|
match outcome {
|
|
Err(TurnError::SessionFull) => match compact_session(session, rt, memory) {
|
|
Ok(()) => run_turn(session, rt, text, speaker, on_event),
|
|
Err(e) => {
|
|
log_compact(session, &e);
|
|
Err(TurnError::SessionFull)
|
|
}
|
|
},
|
|
Ok(done) => {
|
|
if let Some(usage) = session.last_usage()
|
|
&& due(&usage, rt.cfg.compact.threshold, rt.cfg.expect.n_ctx)
|
|
&& let Err(e) = compact_session(session, rt, memory)
|
|
{
|
|
log_compact(session, &e);
|
|
}
|
|
Ok(done)
|
|
}
|
|
Err(e) => Err(e),
|
|
}
|
|
}
|
|
|
|
fn compact_session(
|
|
session: &mut Session,
|
|
rt: &Runtime<'_>,
|
|
memory: &dyn MemorySource,
|
|
) -> Result<(), CompactError> {
|
|
let summary = summarise(rt.cfg, rt.client, session)?;
|
|
commit(session, rt.cfg, rt.registry, memory, summary)
|
|
}
|
|
|
|
fn log_compact(session: &Session, e: &CompactError) {
|
|
eprintln!(
|
|
"loopd: could not compact session {}: {e}",
|
|
session.id().as_str()
|
|
);
|
|
}
|
|
```
|
|
|
|
The `Ok(done)` arm uses `&& let Err(e) = ...` so a compact failure is logged and the original
|
|
`done` is still returned. Do not `?` that error.
|
|
|
|
### `crates/loopd/src/channel.rs`
|
|
|
|
**H1.** The `run_turn` call and the `drop(held)` that follows it. The lines
|
|
|
|
```rust
|
|
let outcome = run_turn(
|
|
&mut session,
|
|
&runtime,
|
|
&turn.content,
|
|
turn.speaker.as_deref(),
|
|
&mut |event| {
|
|
let _ = write_frame(
|
|
&mut stream,
|
|
&Envelope {
|
|
v: PROTOCOL_VERSION,
|
|
id: request.id,
|
|
r#final: false,
|
|
msg: Message::TurnEvent(event.clone()),
|
|
},
|
|
);
|
|
},
|
|
);
|
|
|
|
// 5. Release the session before the final frame, so the next turn does not find it busy.
|
|
drop(held);
|
|
```
|
|
|
|
become
|
|
|
|
```rust
|
|
let mut emit = |event: &bxcore::TurnEvent| {
|
|
let _ = write_frame(
|
|
&mut stream,
|
|
&Envelope {
|
|
v: PROTOCOL_VERSION,
|
|
id: request.id,
|
|
r#final: false,
|
|
msg: Message::TurnEvent(event.clone()),
|
|
},
|
|
);
|
|
};
|
|
|
|
let outcome = run_turn(
|
|
&mut session,
|
|
&runtime,
|
|
&turn.content,
|
|
turn.speaker.as_deref(),
|
|
&mut emit,
|
|
);
|
|
let outcome = crate::compact::finish_turn(
|
|
&mut session,
|
|
&runtime,
|
|
&*ctx.memory,
|
|
&turn.content,
|
|
turn.speaker.as_deref(),
|
|
&mut emit,
|
|
outcome,
|
|
);
|
|
|
|
// 5. Release the session before the final frame, so the next turn does not find it busy.
|
|
// Compaction ran inside the hold: a second turn on this session waits until the new
|
|
// epoch is on disk (or compact failed and the epoch is unchanged).
|
|
drop(held);
|
|
```
|
|
|
|
**H2.** Do not add `MemorySource` to `Runtime`. Do not change `turn_code`. The `memory.updated`
|
|
block after `drop(held)` stays: a successful compact-and-retry still reports `session.turns()`,
|
|
which now spans epochs.
|
|
|
|
## Steps
|
|
|
|
- [ ] **1. Copy the given test.**
|
|
|
|
```sh
|
|
cp docs/plans/M5d1/files/crates/loopd/tests/compact_turn.rs crates/loopd/tests/
|
|
```
|
|
|
|
- [ ] **2. See it fail.** `cargo build --tests --keep-going -p loopd`, timeout 600000 ms: test
|
|
target `compact_turn` does not compile (`E0599` no function `finish_turn`). Every other
|
|
target compiles, including `turn` and `channel`.
|
|
- [ ] **3. The edits.** F1, H1. `cargo check -p loopd` prints no error and no warning.
|
|
`wc -l crates/loopd/src/compact.rs` and `wc -l crates/loopd/src/channel.rs` each print a
|
|
number ≤ 500.
|
|
- [ ] **4. The tests.** `cargo test -p loopd --test compact_turn --test turn --test channel
|
|
--test compact_advance`: every test passes, 6 in `compact_turn` and the others as before.
|
|
- [ ] **5. Check.** `git diff --stat` lists exactly `crates/loopd/src/compact.rs` and
|
|
`crates/loopd/src/channel.rs`. `git status --short` shows `compact_turn.rs` as new.
|
|
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, timeout 600000 ms. Last line
|
|
`gate: ok`. Test count **1507** (1501 before, plus 6).
|
|
- [ ] **7. Log and commit.** Task cell `M5d1/08-loopd-compact-turn`. Stage with
|
|
`git add crates/loopd docs/implementer-log.md Cargo.lock`, then commit as `AGENTS.md` says.
|
|
|
|
## Done when
|
|
|
|
- `make gate` prints `gate: ok` with 1507 tests.
|
|
- `git diff HEAD~1 --stat` lists only `crates/loopd/src/compact.rs`,
|
|
`crates/loopd/src/channel.rs`, `crates/loopd/tests/compact_turn.rs` and the log.
|
|
|
|
## Stop and report if
|
|
|
|
- The branch check fails; a "before" block is not found exactly once; step 2 differs; `cargo
|
|
check` warns; a test fails; the gate count is not 1507; a file exceeds 500 lines; you add
|
|
`MemorySource` to `Runtime`; you compact inside `run_turn`; you call `finish_turn` after
|
|
`drop(held)`; you add an idle timer; you compact a `TurnError` other than `SessionFull`; you
|
|
retry `SessionFull` more than once.
|
|
|
|
## What the given tests pin
|
|
|
|
Usage 849/1000 leaves epoch 0 (one chat); 850/1000 compact before return (live slot then
|
|
background, no tools on the summary); `SessionFull` compact-and-retry (three chats, user logged
|
|
in both epochs, `turns() == 2`); `SessionFull` stays `SessionFull` when compact fails; a
|
|
successful due turn stays successful when `compact.md` is missing (epoch stays 0); another
|
|
infer error is not compacted.
|