boxmaker/docs/plans/M5d1/08-loopd-compact-turn.md

M5d-1 task 08: compact after a turn, and retry once on SessionFull

Branch: m5d1. Check it: git branch --show-current prints m5d1 and git status --short is empty; otherwise stop. Never switch branch. Commit subject: loopd: compact after a full turn and retry SessionFull once Log row: the Task cell is M5d1/08-loopd-compact-turn.

Goal

After run_turn returns, finish_turn decides whether to compact (spec 3.3):

• A successful turn whose last Usage is due is compacted before the caller returns. If compact fails, the turn stays successful and the session stays on its epoch; loopd logs the compact error with the runbook pointer.
• TurnError::SessionFull is compacted, then that user message is run_turn'd once in the new epoch. If compact fails, the error stays SessionFull. The retry is not compacted again.
• Any other TurnError is returned as it is. No compact.

Runtime does not gain MemorySource (given turn tests build a Runtime without one). finish_turn takes memory as its own argument.

The channel holds the session busy for this whole call: handle runs finish_turn before drop(held). A compact is not an idle timer and does not start after the busy set releases.

Live turns still send slots.main (task 03). Only the summary call uses the background slot.

Every place that changes

Place	What happens there
crates/loopd/src/compact.rs	finish_turn and its two helpers (edit F1)
crates/loopd/src/channel.rs	handle calls finish_turn before drop(held) (edits H1, H2)
crates/loopd/tests/compact_turn.rs	new, given
everything else	no change: not turn.rs, not Runtime, not existing turn / channel tests

Files

• Copy, from docs/plans/M5d1/files/ (the same path under it): the new test crates/loopd/tests/compact_turn.rs.
• Modify: crates/loopd/src/compact.rs, crates/loopd/src/channel.rs, docs/implementer-log.md.
• Do not touch: any other file.

Facts you need

• due looks at session.last_usage(), which is the current epoch's last Usage. After a successful run_turn that record is there. After SessionFull, run_turn has already appended the User and returned before any Usage; compact runs because of the error, not because of due.

• Compact helper. summarise then commit. Log failures with

  loopd: could not compact session <id>: <CompactError Display>
  

  Display already includes the runbook pointer, matching the runbook's example line.

• Retry. run_turn appends the user message again in the new epoch. The old epoch keeps its User (Honcho skips it: no reply). turns() is 2. That is accepted (README, known limits).

• Existing channel tests stay green. Their n_ctx is 131072 and fixture prompt_n is tens, so due is false. SessionFull without compact.md in Home stays SessionFull (the successful-turn-when-compact-fails behaviour, on the error path).

• compact.rs stays ≤ 500 lines. After this task it is about 250.

• channel.rs stays ≤ 500 lines. After this task it is about 290.

• The items the edits use. This table is complete.

  Item	What it does
  due	task 05
  summarise / commit	tasks 06, 07
  run_turn	retry the same text and speaker
  Session::last_usage	current epoch
  Config.compact.threshold, expect.n_ctx	the due arguments
  MemorySource	finish_turn's extra argument; channel passes &*ctx.memory
  Held	already in channel.rs; drop it after finish_turn

The edits, exactly

Each quoted "before" block occurs exactly once after task 07 (channel.rs is still as on master, except rustfmt).

crates/loopd/src/compact.rs

F1. Extend the crate uses. After task 07 they include MemorySource, Session, Config, Registry. Add:

use crate::turn::{Runtime, TurnError, TurnOutcome, run_turn};
use bxcore::TurnEvent;

Keep rustfmt's order of use lines; if it moves them, accept that.

Then after commit, add these three functions whole. There is no todo!().

/// After `run_turn`: compact when the last `Usage` is due, or on `SessionFull` (then retry the
/// user message once). Compact failure on a successful turn keeps the success; on `SessionFull`
/// keeps `SessionFull`. Other errors skip compact.
pub fn finish_turn(
    session: &mut Session,
    rt: &Runtime<'_>,
    memory: &dyn MemorySource,
    text: &str,
    speaker: Option<&str>,
    on_event: &mut dyn FnMut(&TurnEvent),
    outcome: Result<TurnOutcome, TurnError>,
) -> Result<TurnOutcome, TurnError> {
    match outcome {
        Err(TurnError::SessionFull) => match compact_session(session, rt, memory) {
            Ok(()) => run_turn(session, rt, text, speaker, on_event),
            Err(e) => {
                log_compact(session, &e);
                Err(TurnError::SessionFull)
            }
        },
        Ok(done) => {
            if let Some(usage) = session.last_usage()
                && due(&usage, rt.cfg.compact.threshold, rt.cfg.expect.n_ctx)
                && let Err(e) = compact_session(session, rt, memory)
            {
                log_compact(session, &e);
            }
            Ok(done)
        }
        Err(e) => Err(e),
    }
}

fn compact_session(
    session: &mut Session,
    rt: &Runtime<'_>,
    memory: &dyn MemorySource,
) -> Result<(), CompactError> {
    let summary = summarise(rt.cfg, rt.client, session)?;
    commit(session, rt.cfg, rt.registry, memory, summary)
}

fn log_compact(session: &Session, e: &CompactError) {
    eprintln!(
        "loopd: could not compact session {}: {e}",
        session.id().as_str()
    );
}

The Ok(done) arm uses && let Err(e) = ... so a compact failure is logged and the original done is still returned. Do not ? that error.

crates/loopd/src/channel.rs

H1. The run_turn call and the drop(held) that follows it. The lines

    let outcome = run_turn(
        &mut session,
        &runtime,
        &turn.content,
        turn.speaker.as_deref(),
        &mut |event| {
            let _ = write_frame(
                &mut stream,
                &Envelope {
                    v: PROTOCOL_VERSION,
                    id: request.id,
                    r#final: false,
                    msg: Message::TurnEvent(event.clone()),
                },
            );
        },
    );

    // 5. Release the session before the final frame, so the next turn does not find it busy.
    drop(held);

become

    let mut emit = |event: &bxcore::TurnEvent| {
        let _ = write_frame(
            &mut stream,
            &Envelope {
                v: PROTOCOL_VERSION,
                id: request.id,
                r#final: false,
                msg: Message::TurnEvent(event.clone()),
            },
        );
    };

    let outcome = run_turn(
        &mut session,
        &runtime,
        &turn.content,
        turn.speaker.as_deref(),
        &mut emit,
    );
    let outcome = crate::compact::finish_turn(
        &mut session,
        &runtime,
        &*ctx.memory,
        &turn.content,
        turn.speaker.as_deref(),
        &mut emit,
        outcome,
    );

    // 5. Release the session before the final frame, so the next turn does not find it busy.
    //    Compaction ran inside the hold: a second turn on this session waits until the new
    //    epoch is on disk (or compact failed and the epoch is unchanged).
    drop(held);

H2. Do not add MemorySource to Runtime. Do not change turn_code. The memory.updated block after drop(held) stays: a successful compact-and-retry still reports session.turns(), which now spans epochs.

Steps

• 1. Copy the given test.

  cp docs/plans/M5d1/files/crates/loopd/tests/compact_turn.rs crates/loopd/tests/
  

• 2. See it fail. cargo build --tests --keep-going -p loopd, timeout 600000 ms: test target compact_turn does not compile (E0599 no function finish_turn). Every other target compiles, including turn and channel.

• 3. The edits. F1, H1. cargo check -p loopd prints no error and no warning. wc -l crates/loopd/src/compact.rs and wc -l crates/loopd/src/channel.rs each print a number ≤ 500.

• 4. The tests. cargo test -p loopd --test compact_turn --test turn --test channel --test compact_advance: every test passes, 6 in compact_turn and the others as before.

• 5. Check. git diff --stat lists exactly crates/loopd/src/compact.rs and crates/loopd/src/channel.rs. git status --short shows compact_turn.rs as new.

• 6. Run the gate. cargo fmt --all, then make gate, timeout 600000 ms. Last line gate: ok. Test count 1508 (1502 before, plus 6).

• 7. Log and commit. Task cell M5d1/08-loopd-compact-turn. Stage with git add crates/loopd docs/implementer-log.md Cargo.lock, then commit as AGENTS.md says.

Done when

• make gate prints gate: ok with 1508 tests.
• git diff HEAD~1 --stat lists only crates/loopd/src/compact.rs, crates/loopd/src/channel.rs, crates/loopd/tests/compact_turn.rs and the log.

Stop and report if

• The branch check fails; a "before" block is not found exactly once; step 2 differs; cargo check warns; a test fails; the gate count is not 1508; a file exceeds 500 lines; you add MemorySource to Runtime; you compact inside run_turn; you call finish_turn after drop(held); you add an idle timer; you compact a TurnError other than SessionFull; you retry SessionFull more than once.

What the given tests pin

Usage 849/1000 leaves epoch 0 (one chat); 850/1000 compact before return (live slot then background, no tools on the summary); SessionFull compact-and-retry (three chats, user logged in both epochs, turns() == 2); SessionFull stays SessionFull when compact fails; a successful due turn stays successful when compact.md is missing (epoch stays 0); another infer error is not compacted.