boxmaker/docs/plans/M5d1/07-loopd-compact-advance.md

M5d-1 task 07: writing the next epoch

Branch: m5d1. Check it: git branch --show-current prints m5d1 and git status --short is empty; otherwise stop. Never switch branch. Commit subject: loopd: write the next epoch from a compaction summary Log row: the Task cell is M5d1/07-loopd-compact-advance.

Goal

Given a summary string, commit starts the next epoch (spec 3.5):

1. Append EpochEnd { next, summary } to the current log, unless that log already ends with an EpochEnd (a compact that stopped after the end record and before the next files).
2. Ask memoryd for a fresh block; write <n>.baseline.json from assemble_with (so identity edits applied in the old epoch land here, and registry.core_schemas() is the new tools array).
3. Create <n>.jsonl with SessionStart { epoch: n, slot: the session's live slot, baseline: hash of the new file }.
4. The in-memory Session switches: new baseline, records that are only that SessionStart, carry = Some(summary), prior_users / prior_untrusted accumulated from the old records.

A crash after step 1 and before the next files exist reopens epoch 0 with EpochEnd last (task 02). The next commit resumes: it uses the summary already in that EpochEnd as the carry and does not write a second EpochEnd. The summary argument is ignored in that case.

Every place that changes

Place	What happens there
crates/loopd/src/session.rs	begin_epoch (edit S1)
crates/loopd/src/compact.rs	commit (edit C1)
crates/loopd/tests/compact_advance.rs	new, given
everything else	no change: not turn.rs, not channel.rs

Files

• Copy, from docs/plans/M5d1/files/ (the same path under it): the new test crates/loopd/tests/compact_advance.rs.
• Modify: crates/loopd/src/session.rs, crates/loopd/src/compact.rs, docs/implementer-log.md.
• Do not touch: any other file.

Facts you need

• begin_epoch is pub(crate) on Session so compact.rs can switch the private fields. Tests call compact::commit, never begin_epoch.
• Order inside begin_epoch. Write <n>.baseline.json (overwrite is fine on resume), create_new the log file, then set epoch / baseline / carry / empty records / the new File, then persist SessionStart. log_path uses self.epoch, so epoch must be next before that persist. Accumulate prior_users and prior_untrusted from the old records before clearing them. Do not reset next_call or slot.
• Resume. If session.records().last() is EpochEnd { next, summary }, skip the append; next and summary (cloned) are what begin_epoch gets. Otherwise append EpochEnd with next = Epoch(session.epoch().0.saturating_add(1)) and the summary argument, then begin_epoch.
• Memory. memory.block().map_err(|e| CompactError::Failed { detail: e })?. Then Baseline::assemble_with(cfg, registry, &block). assemble_with already calls registry.core_schemas(), so a new core tool lands at this boundary. Map assemble and session errors to Failed { detail: e.to_string() }.
• NoMemory.block() is Ok(""). The given test the_new_baseline_takes_a_fresh_memory_block uses a MemorySource that returns a distinctive sentence.
• session.rs stays ≤ 500 lines. After task 02 it is about 430; begin_epoch is about 40.
• compact.rs stays ≤ 500 lines. After this task it is about 180.
• The items the edits use. This table is complete.

  Item	What it does
  Baseline::assemble_with	new system + memory block + registry.core_schemas()
  Baseline::to_json / hash	the file and SessionStart.baseline
  OpenOptions::create_new(true).append(true)	the new log, as create does
  count_users / any_untrusted	task 02, module-private in session.rs
  Session::persist	private; begin_epoch is in the same impl
  MemorySource::block	fresh text; Err is Failed
  Timestamp::now	EpochEnd and SessionStart
  Epoch(n.saturating_add(1))	the next number

The edits, exactly

Each quoted "before" block occurs exactly once after task 06 (session.rs is as task 02 left it).

crates/loopd/src/session.rs

S1. log_path, the lines

    fn log_path(&self) -> PathBuf {
        self.dir.join(format!("{}.jsonl", self.epoch.0))
    }
}

become

    fn log_path(&self) -> PathBuf {
        self.dir.join(format!("{}.jsonl", self.epoch.0))
    }

    /// Write `<n>.baseline.json` and `<n>.jsonl` with `SessionStart`, and switch this session
    /// onto that epoch. The live slot and `next_call` stay. Callers pass the carry that
    /// `messages_with_carry` will insert.
    pub(crate) fn begin_epoch(
        &mut self,
        next: Epoch,
        baseline: Baseline,
        carry: String,
    ) -> Result<(), SessionError> {
        let n = next.0;
        let baseline_path = self.dir.join(format!("{n}.baseline.json"));
        let json = baseline.to_json().map_err(SessionError::Encode)?;
        std::fs::write(&baseline_path, json)
            .map_err(|e| SessionError::Io(baseline_path.clone(), e))?;
        let log_path = self.dir.join(format!("{n}.jsonl"));
        let log = OpenOptions::new()
            .create_new(true)
            .append(true)
            .open(&log_path)
            .map_err(|e| SessionError::Io(log_path.clone(), e))?;
        self.prior_users = self.prior_users.saturating_add(count_users(&self.records));
        self.prior_untrusted = self.prior_untrusted || any_untrusted(&self.records);
        self.epoch = next;
        self.baseline = baseline;
        self.carry = Some(carry);
        self.records = Vec::new();
        self.log = log;
        let hash = self.baseline.hash().map_err(SessionError::Baseline)?;
        self.persist(LogRecord::SessionStart {
            time: Timestamp::now(),
            session: self.id.clone(),
            epoch: next,
            slot: self.slot,
            baseline: hash,
        })
    }
}

The helpers after the impl (complete_epochs and the rest) stay where they are. Do not move them.

crates/loopd/src/compact.rs

C1. After summarise (the function that ends the match client.chat_with_retry / }), add the use items this function needs by extending the existing use crate:: group, then add commit. The current crate uses (after task 06) are:

use crate::baseline::messages_with_carry;
use crate::config::Config;
use crate::llama::{ChatMessage, ChatRequest, Client};
use crate::session::Session;
use crate::turn::is_context_full;

They become

use crate::baseline::{messages_with_carry, Baseline};
use crate::config::Config;
use crate::llama::{ChatMessage, ChatRequest, Client};
use crate::memory::MemorySource;
use crate::session::Session;
use crate::tools::Registry;
use crate::turn::is_context_full;

And the bxcore use

use bxcore::{LogRecord, Usage};

becomes

use bxcore::{Epoch, LogRecord, Timestamp, Usage};

Then this function, whole, after summarise:

/// Append `EpochEnd` (unless it is already last), rebuild the baseline from `memory`, and
/// switch `session` onto the next epoch. When the current log already ends with `EpochEnd`,
/// that record's summary is the carry and the argument is ignored.
pub fn commit(
    session: &mut Session,
    cfg: &Config,
    registry: &Registry,
    memory: &dyn MemorySource,
    summary: String,
) -> Result<(), CompactError> {
    let (next, summary) = match session.records().last() {
        Some(LogRecord::EpochEnd { next, summary, .. }) => (*next, summary.clone()),
        _ => {
            let next = Epoch(session.epoch().0.saturating_add(1));
            session
                .append(LogRecord::EpochEnd {
                    time: Timestamp::now(),
                    next,
                    summary: summary.clone(),
                })
                .map_err(|e| CompactError::Failed {
                    detail: e.to_string(),
                })?;
            (next, summary)
        }
    };
    let block = memory.block().map_err(|e| CompactError::Failed { detail: e })?;
    let baseline = Baseline::assemble_with(cfg, registry, &block).map_err(|e| {
        CompactError::Failed {
            detail: e.to_string(),
        }
    })?;
    session
        .begin_epoch(next, baseline, summary)
        .map_err(|e| CompactError::Failed {
            detail: e.to_string(),
        })?;
    Ok(())
}

If rustfmt splits the map_err closures, accept that.

Steps

• 1. Copy the given test.

  cp docs/plans/M5d1/files/crates/loopd/tests/compact_advance.rs crates/loopd/tests/
  

• 2. See it fail. cargo build --tests --keep-going -p loopd, timeout 600000 ms: test target compact_advance does not compile (E0599 no function commit). Every other target compiles.

• 3. The edits. S1, then C1. cargo check -p loopd prints no error and no warning. wc -l crates/loopd/src/session.rs and wc -l crates/loopd/src/compact.rs each print a number ≤ 500.

• 4. The tests. cargo test -p loopd --test compact_advance --test session_epochs --test compact_summary: every test passes, 6 in compact_advance, 9 in session_epochs, 6 in compact_summary.

• 5. Check. git diff --stat lists exactly crates/loopd/src/session.rs and crates/loopd/src/compact.rs. git status --short shows compact_advance.rs as new.

• 6. Run the gate. cargo fmt --all, then make gate, timeout 600000 ms. Last line gate: ok. Test count 1501 (1495 before, plus 6).

• 7. Log and commit. Task cell M5d1/07-loopd-compact-advance. Stage with git add crates/loopd docs/implementer-log.md Cargo.lock, then commit as AGENTS.md says.

Done when

• make gate prints gate: ok with 1501 tests.
• git diff HEAD~1 --stat lists only crates/loopd/src/session.rs, crates/loopd/src/compact.rs, crates/loopd/tests/compact_advance.rs and the log.

Stop and report if

• The branch check fails; a "before" block is not found exactly once; step 2 differs; cargo check warns; a test fails; the gate count is not 1501; a file exceeds 500 lines; you write a second EpochEnd on resume; you reset slot or next_call; you log the summary as a LogRecord::User.

What the given tests pin

commit writes EpochEnd on the old log, a new baseline file, and a new log whose SessionStart has the next epoch; the new baseline contains a fresh memory block; reopen loads the new epoch and its carry; a second commit leaves epoch 0 with one EpochEnd and writes the next on epoch 1; resume after EpochEnd without next files uses that summary as carry and does not write a second end; the new SessionStart keeps the live slot (including a non-default 7).