kyle/boxmaker
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: M5d-1 and M5d-2 implementation plans for Ornith

Browse Source 
Eight closed tasks for epochs and compaction, then ten for the scheduler,
ToolSummary, and tiers. Given tests first. Gate 1456 → 1507 → 1572.

Co-Authored-By: Grok 4.6 <noreply@x.ai>
This commit is contained in:
kyle
2026-10-02 21:54:37 -07:00
co-authored by Grok 4.6
parent 069cd1686a
commit 3726ae585f
48 changed files with 7319 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
docs/plans/M5d1/01-loopd-compact-config.md
+209
View File
@@ -0,0 +1,209 @@
# M5d-1 task 01: `[compact]` on `loopd.toml`
**Branch:** `m5d1`. Check it: `git branch --show-current` prints `m5d1` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `loopd: [compact] threshold, summary_chars and summarise`
**Log row:** the Task cell is `M5d1/01-loopd-compact-config`.
## Goal
M5d-1 compact after a turn whose last `Usage` is at or above a fraction of `expect.n_ctx` (spec
3.3, 5). This task adds only the `[compact]` table so later tasks can read it:
- `threshold` (float, default `0.85`)
- `summary_chars` (default `4000`)
- `summarise` (list of tool names, default `["http_fetch"]`)
Nothing in this task uses the values. `summarise` is stored so a spec-complete `loopd.toml`
loads; M5d-2 reads it. Unknown keys in `[compact]` are errors, as in every table we define.
`Config::compact_prompt` is `compact.md` beside `[baseline] system` (the same directory join
`load` already does for `system.md`). The starter file is copied in task 05.
## Every place that changes
This is the complete list.
| Place | What happens there |
|---|---|
| `crates/loopd/src/config.rs` | `Compact`; `Config.compact`; `Config::compact_prompt` (edits C1 to C3) |
| `crates/loopd/tests/config_compact.rs` | new, given |
| everything else | **no change**: not `session.rs`, not the existing `config.rs` tests or fixtures |
## Files
- Copy, from `docs/plans/M5d1/files/` (the same path under it): the new test
`crates/loopd/tests/config_compact.rs`.
- Modify: `crates/loopd/src/config.rs`, `docs/implementer-log.md`.
- Do not touch: any other file.
## Facts you need
- **`Config` already has `#[serde(default)]` on optional tables** (`sampling`, `limits`, `paths`,
`channel`, `loop`, `baseline`, `broker`, `memory`). `[compact]` is the same: missing means
`Compact::default()`.
- **`threshold` is `f64`.** `Sampling.temperature` is already `f64` on `Config`, so `PartialEq`
on `Config` still compiles. Do not add a range check: the spec does not name one.
- **`summarise` default.** `vec!["http_fetch".to_string()]`. An explicit `summarise = []` is
allowed and is empty. Unknown names are not checked here (M5d-2).
- **`compact_prompt`.** `self.baseline.system.with_file_name("compact.md")`. After `load`,
`baseline.system` is already joined to the config file's directory, so the prompt follows it
into a subdirectory when `system` does (`prompts/agent.md` → `prompts/compact.md`).
- The items the edits use. This table is complete; `serde` is already a dependency of `loopd`.
| Item | What it does |
|---|---|
| `#[serde(deny_unknown_fields, default)]` | on `Compact`: missing table uses `Default`; unknown keys error |
| `Path::with_file_name` | `compact.md` beside `system.md` |
| `Vec<String>` | `summarise` |
## The edits, exactly
Nothing else in this file changes. Each quoted "before" block occurs exactly once in its file
when its turn comes (make them in order). Keep the fields in the order shown.
### `crates/loopd/src/config.rs`
**C1.** After the `Memory` struct (the lines)
```rust
pub struct Memory {
pub socket: std::path::PathBuf,
}
```
become
```rust
pub struct Memory {
pub socket: std::path::PathBuf,
}
/// `[compact]`: when a session starts a new epoch, and how long a summary may be (M5d spec,
/// section 5). `summarise` is stored here and unused until M5d-2.
#[derive(Debug, Clone, PartialEq, Deserialize)]
#[serde(deny_unknown_fields, default)]
pub struct Compact {
pub threshold: f64,
pub summary_chars: usize,
pub summarise: Vec<String>,
}
impl Default for Compact {
fn default() -> Self {
Self {
threshold: 0.85,
summary_chars: 4_000,
summarise: vec!["http_fetch".to_string()],
}
}
}
```
**C2.** The lines
```rust
#[serde(default)]
pub memory: Memory,
}
```
become
```rust
#[serde(default)]
pub memory: Memory,
#[serde(default)]
pub compact: Compact,
}
```
**C3.** After `channel_socket` (the lines)
```rust
pub fn channel_socket(&self) -> std::path::PathBuf {
if self.channel.socket.as_os_str().is_empty() {
self.paths.home.join("run/loop/loop.sock")
} else {
self.channel.socket.clone()
}
}
}
```
become
```rust
pub fn channel_socket(&self) -> std::path::PathBuf {
if self.channel.socket.as_os_str().is_empty() {
self.paths.home.join("run/loop/loop.sock")
} else {
self.channel.socket.clone()
}
}
/// `compact.md` beside the system prompt: the summary call's last user message.
pub fn compact_prompt(&self) -> std::path::PathBuf {
self.baseline.system.with_file_name("compact.md")
}
}
```
## Steps
- [ ] **1. Copy the given test.** From the repository root, one command:
```sh
cp docs/plans/M5d1/files/crates/loopd/tests/config_compact.rs crates/loopd/tests/
```
Then `git status --short` lists one new file, `crates/loopd/tests/config_compact.rs`.
- [ ] **2. See it fail.** `cargo build --tests --keep-going -p loopd`, with a shell timeout of
600000 ms: one test target, `config_compact`, does not compile, with `E0609` (`no field
\`compact\` on type \`Config\``) and `E0599` (`no method named \`compact_prompt\`` /
`no variant or associated item named \`default\` for \`Compact\``). Every other target
compiles.
- [ ] **3. The edits.** Make edits C1 to C3, in order. `cargo check -p loopd` prints no error and
no warning.
- [ ] **4. The tests.** `cargo test -p loopd --test config_compact --test config`: every test
passes, 8 in `config_compact` and the others as before.
- [ ] **5. Check.** `git diff --stat` lists exactly `crates/loopd/src/config.rs`.
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, with a shell timeout of 600000 ms.
Expected last line: `gate: ok`. The test count is **1464** after this task, 1456 before it
(1455 passed in `cargo test --workspace`, plus 1), and the 8 new ones are the tests of
`config_compact.rs`. To count, add up the `N passed` of every `test result:` line the gate's
`cargo test --workspace` printed, and add 1. Never run the gate again only to count.
- [ ] **7. Log and commit.** Add your row to `docs/implementer-log.md`; its Task cell is
`M5d1/01-loopd-compact-config`. Stage with
`git add crates/loopd docs/implementer-log.md Cargo.lock`, then commit as `AGENTS.md` says.
`Cargo.lock` does not change in this task; the `git add` is harmless.
## Done when
- `make gate` prints `gate: ok` with 1464 tests.
- `git diff HEAD~1 --stat` lists only `crates/loopd/src/config.rs`,
`crates/loopd/tests/config_compact.rs` and `docs/implementer-log.md`.
## Stop and report if
- The branch check at the top fails.
- A "before" block is not in its file exactly as shown when its turn comes, or is there more
than once.
- Step 2 shows a different result: another target failing, or other errors.
- `cargo check` in step 3 prints an error or a warning. Quote it, and stop.
- A test fails in step 4. Do not change the test: stop, and give the failing tests' names and
their messages.
- The fix seems to need a change anywhere else: a fixture, `config.rs` tests, a range check on
`threshold`. It does not; stop and say why you think so.
- A test outside `config_compact.rs` fails in the gate: check once whether it fails without your
change (`git stash -u`, run it, `git stash pop`), then stop and report both results.
- The gate passes with a count other than 1464.
- A comment in an edit does not fit the code under it. Quote it and stop.
## What the given tests pin
Each of these changes to the finished code makes at least one given test fail: the default
threshold moved; `summary_chars` default moved; `summarise` default emptied or given another
name; unknown keys accepted; a misspelt `summary_char` accepted; a partial table wiping the
other defaults; `compact_prompt` using `home` or the config file's name instead of `system.md`'s
directory; `Compact::default` disagreeing with a missing table.
docs/plans/M5d1/02-loopd-session-epochs.md
+513
View File
@@ -0,0 +1,513 @@
# M5d-1 task 02: `Session::open` loads the current epoch
**Branch:** `m5d1`. Check it: `git branch --show-current` prints `m5d1` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `loopd: open the highest epoch and carry its summary`
**Log row:** the Task cell is `M5d1/02-loopd-session-epochs`.
## Goal
A session directory may have `0.jsonl` and, after compaction, `1.baseline.json` / `1.jsonl` and
so on (spec 3.1). `Session::open` loads the highest epoch that has **both** a baseline file and a
log. For epoch `n > 0` the previous log must end with `EpochEnd { next: n, summary }`; that
summary is the carry. `turns()` and `untrusted()` span every epoch. `next_call` is one more than
the highest `ToolResult` call in any epoch. `append` writes the current epoch's log.
`records()` stays the current epoch only: later tasks replay that file (plus the carry) as the
request.
## Every place that changes
This is the complete list.
| Place | What happens there |
|---|---|
| `crates/loopd/src/session.rs` | `NoCarry`; fields `epoch`, `slot`, `carry`, `prior_users`, `prior_untrusted`; `open`; `turns`; `untrusted`; `log_path`; accessors; helpers |
| `crates/loopd/tests/session_epochs.rs` | new, given |
| everything else | **no change**: not `session.rs`'s existing tests, not `turn.rs`, not Honcho |
## Files
- Copy, from `docs/plans/M5d1/files/` (the same path under it): the new test
`crates/loopd/tests/session_epochs.rs`.
- Modify: `crates/loopd/src/session.rs`, `docs/implementer-log.md`.
- Do not touch: any other file.
## Facts you need
- **`Epoch` is `pub struct Epoch(pub u32)`** in `bxcore`. Format a file name with
`format!("{}.jsonl", epoch.0)` and `format!("{}.baseline.json", epoch.0)`.
- **A complete epoch** is both `<n>.baseline.json` and `<n>.jsonl` in the session directory.
A baseline without a log, or a log without a baseline, does not count. The highest complete
`n` is the current epoch. If none is complete, `open` is `NotFound`, as today when `0.jsonl`
is missing.
- **Carry.** For current epoch `n > 0`, read `(n-1).jsonl`. Among its records, the last
`EpochEnd` whose `next` is `Epoch(n)` gives `summary`. If that file is missing, or has no such
record, `SessionError::NoCarry { path }` naming `(n-1).jsonl`. Epoch 0 has `carry: None`.
- **`records` on the live `Session`** is only the current epoch's log, so `messages` in task 03
can replay it. `prior_users` / `prior_untrusted` are counted from every **earlier** complete
epoch while `open` walks them. `turns()` is `prior_users` plus the `User` records in
`records`. `untrusted()` is `prior_untrusted` or any untrusted `ToolResult` in `records`.
- **`next_call`** is one more than the highest `ToolResult.call.0` in **every** complete epoch
(current included). A call id of 4 in epoch 0 and 2 in epoch 1 continues at 5.
- **`slot`** is the `slot` on the current epoch's `SessionStart`. `create` stores the `slot`
argument. If the current log has no `SessionStart`, use `0`.
- **`append` / `persist`** already write `self.log_path()`. Change `log_path` to the current
epoch's file; `create` still starts at `0.jsonl`.
- **A last `EpochEnd` on epoch 0 with no epoch-1 files** means compact stopped halfway: `open`
stays on epoch 0, `carry` is `None`, and the `EpochEnd` is in `records`. Task 07 resumes that.
- The torn-log checks stay as they are, per file, with the line number of that file.
- `session.rs` must stay at or under 500 lines (`wc -l`). After these edits it is about 430.
- The items the edits use. This table is complete.
| Item | What it does |
|---|---|
| `std::fs::read_dir` | list `<n>.jsonl` names |
| `str::strip_suffix(".jsonl")`, `str::parse::<u32>` | the epoch number |
| `Path::exists` | the matching baseline |
| `u64::try_from(count).unwrap_or(u64::MAX)` | `turns`, as today |
| `CallId(highest + 1)` | `next_call`, as today, over every epoch |
| `OpenOptions::new().append(true).open` | the current log |
## The edits, exactly
Nothing else in this file changes. Each quoted "before" block occurs exactly once. Keep struct
fields in the order shown.
### `crates/loopd/src/session.rs`
**S1.** The module doc comment, the lines
```rust
//! A session on disk: `sessions/<id>/0.baseline.json` and `0.jsonl`. The log is the state —
//! create, open, append and resume all live here, and nothing is durable until `append` syncs it.
```
become
```rust
//! A session on disk: `sessions/<id>/<epoch>.baseline.json` and `<epoch>.jsonl`. The log is the
//! state — create, open, append and resume all live here, and nothing is durable until `append`
//! syncs it. Epoch 0 is the first; later epochs appear after compaction.
```
**S2.** After `Encode(serde_json::Error),` in `SessionError`, add a variant (the lines)
```rust
Encode(serde_json::Error),
}
```
become
```rust
Encode(serde_json::Error),
NoCarry {
path: PathBuf,
},
}
```
**S3.** In `Display`, the lines
```rust
SessionError::Encode(err) => write!(f, "{err}"),
}
```
become
```rust
SessionError::Encode(err) => write!(f, "{err}"),
SessionError::NoCarry { path } => write!(
f,
"{}: compacted epoch is missing its EpochEnd; see docs/runbook.md#session-log-damaged",
path.display()
),
}
```
**S4.** The struct, the lines
```rust
pub struct Session {
id: SessionId,
dir: PathBuf,
baseline: Baseline,
records: Vec<LogRecord>,
log: File,
next_call: CallId,
}
```
become
```rust
pub struct Session {
id: SessionId,
dir: PathBuf,
baseline: Baseline,
records: Vec<LogRecord>,
log: File,
next_call: CallId,
epoch: Epoch,
slot: u32,
carry: Option<String>,
prior_users: u64,
prior_untrusted: bool,
}
```
**S5.** In `create`, the lines
```rust
let mut session = Session {
id,
dir,
baseline,
records: Vec::new(),
log,
next_call: CallId(1),
};
```
become
```rust
let mut session = Session {
id,
dir,
baseline,
records: Vec::new(),
log,
next_call: CallId(1),
epoch: Epoch(0),
slot,
carry: None,
prior_users: 0,
prior_untrusted: false,
};
```
**S6.** Replace the whole `open` function. The lines from `pub fn open(home: &Path, id: SessionId)`
through the `Ok(Session {` block that ends `next_call: CallId(highest + 1),` and `})` `}` become
the function below. Fill any `todo!()` from its comment, then delete the comment. Each comment
is at most six lines.
```rust
pub fn open(home: &Path, id: SessionId) -> Result<Session, SessionError> {
let dir = Session::dir_for(home, &id);
let epochs = complete_epochs(&dir)?;
let Some(&n) = epochs.last() else {
return Err(SessionError::NotFound(id));
};
let epoch = Epoch(n);
let baseline_path = dir.join(format!("{n}.baseline.json"));
let baseline = Baseline::load(&baseline_path).map_err(SessionError::Baseline)?;
let log_path = dir.join(format!("{n}.jsonl"));
let records = load_records(&log_path)?;
let (carry, prior_users, prior_untrusted) = priors(&dir, n)?;
let mut highest = max_call(&records);
for earlier in &epochs[..epochs.len() - 1] {
let path = dir.join(format!("{earlier}.jsonl"));
highest = highest.max(max_call(&load_records(&path)?));
}
let slot = live_slot(&records, 0);
let log = OpenOptions::new()
.append(true)
.open(&log_path)
.map_err(|e| SessionError::Io(log_path.clone(), e))?;
Ok(Session {
id,
dir,
baseline,
records,
log,
next_call: CallId(highest + 1),
epoch,
slot,
carry,
prior_users,
prior_untrusted,
})
}
```
**S7.** After `pub fn records(&self)`, add accessors (the lines)
```rust
pub fn records(&self) -> &[LogRecord] {
&self.records
}
```
become
```rust
pub fn records(&self) -> &[LogRecord] {
&self.records
}
pub fn epoch(&self) -> Epoch {
self.epoch
}
pub fn slot(&self) -> u32 {
self.slot
}
pub fn carry(&self) -> Option<&str> {
self.carry.as_deref()
}
```
**S8.** `untrusted` and `turns`, the lines
```rust
pub fn untrusted(&self) -> bool {
self.records.iter().any(|record| {
matches!(
record,
LogRecord::ToolResult {
untrusted: true,
..
}
)
})
}
/// How many turns the session has had: its `User` records, one per turn that reached the
/// log, a failed turn's too. Counted from `records`, which `open` reads back whole, so the
/// count goes on after a reopen or a restart; right after a turn it is that turn's number,
/// the first being 1. Epoch 0 is a session's only log until compaction is built.
pub fn turns(&self) -> u64 {
let users = self
.records
.iter()
.filter(|record| matches!(record, LogRecord::User { .. }))
.count();
u64::try_from(users).unwrap_or(u64::MAX)
}
```
become
```rust
pub fn untrusted(&self) -> bool {
self.prior_untrusted || any_untrusted(&self.records)
}
/// How many turns the session has had: its `User` records across every epoch, a failed
/// turn's too. The first `User` of the session is turn 1 for its life.
pub fn turns(&self) -> u64 {
self.prior_users
.saturating_add(count_users(&self.records))
}
```
**S9.** `log_path`, the lines
```rust
fn log_path(&self) -> PathBuf {
self.dir.join("0.jsonl")
}
}
```
become
```rust
fn log_path(&self) -> PathBuf {
self.dir.join(format!("{}.jsonl", self.epoch.0))
}
}
/// Epoch numbers that have both `<n>.jsonl` and `<n>.baseline.json`, sorted.
fn complete_epochs(dir: &Path) -> Result<Vec<u32>, SessionError> {
let entries = match std::fs::read_dir(dir) {
Ok(e) => e,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(Vec::new()),
Err(e) => return Err(SessionError::Io(dir.to_path_buf(), e)),
};
let mut out = Vec::new();
for entry in entries {
let entry = entry.map_err(|e| SessionError::Io(dir.to_path_buf(), e))?;
let name = entry.file_name();
let Some(name) = name.to_str() else { continue };
let Some(stem) = name.strip_suffix(".jsonl") else { continue };
let Ok(n) = stem.parse::<u32>() else { continue };
if dir.join(format!("{n}.baseline.json")).exists() {
out.push(n);
}
}
out.sort_unstable();
Ok(out)
}
fn load_records(path: &Path) -> Result<Vec<LogRecord>, SessionError> {
let text = match std::fs::read_to_string(path) {
Ok(t) => t,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => {
return Err(SessionError::Io(path.to_path_buf(), e));
}
Err(e) => return Err(SessionError::Io(path.to_path_buf(), e)),
};
let mut records = Vec::new();
if text.is_empty() {
return Ok(records);
}
for (idx, line) in text.split_inclusive('\n').enumerate() {
let line_no = idx + 1;
if !line.ends_with('\n') {
return Err(SessionError::Torn {
path: path.to_path_buf(),
line: line_no,
why: "last line does not end in a newline".to_string(),
});
}
match serde_json::from_str::<LogRecord>(line) {
Ok(record) => records.push(record),
Err(why) => {
return Err(SessionError::Torn {
path: path.to_path_buf(),
line: line_no,
why: why.to_string(),
});
}
}
}
Ok(records)
}
/// Carry from epoch `n-1`, and user / untrusted counts from every epoch before `n`.
fn priors(dir: &Path, n: u32) -> Result<(Option<String>, u64, bool), SessionError> {
let mut prior_users = 0u64;
let mut prior_untrusted = false;
let mut carry = None;
if n == 0 {
return Ok((None, 0, false));
}
for earlier in 0..n {
let path = dir.join(format!("{earlier}.jsonl"));
let records = match load_records(&path) {
Ok(r) => r,
Err(SessionError::Io(path, e)) if e.kind() == std::io::ErrorKind::NotFound => {
return Err(SessionError::NoCarry { path });
}
Err(e) => return Err(e),
};
prior_users = prior_users.saturating_add(count_users(&records));
prior_untrusted = prior_untrusted || any_untrusted(&records);
if earlier + 1 == n {
carry = carry_from(&records, Epoch(n)).map(str::to_string);
if carry.is_none() {
return Err(SessionError::NoCarry { path });
}
}
}
Ok((carry, prior_users, prior_untrusted))
}
fn carry_from(records: &[LogRecord], next: Epoch) -> Option<&str> {
records.iter().rev().find_map(|record| match record {
LogRecord::EpochEnd {
next: n,
summary,
..
} if *n == next => Some(summary.as_str()),
_ => None,
})
}
fn count_users(records: &[LogRecord]) -> u64 {
let users = records
.iter()
.filter(|record| matches!(record, LogRecord::User { .. }))
.count();
u64::try_from(users).unwrap_or(u64::MAX)
}
fn any_untrusted(records: &[LogRecord]) -> bool {
records.iter().any(|record| {
matches!(
record,
LogRecord::ToolResult {
untrusted: true,
..
}
)
})
}
fn max_call(records: &[LogRecord]) -> u64 {
let mut highest = 0u64;
for record in records {
if let LogRecord::ToolResult { call, .. } = record {
highest = highest.max(call.0);
}
}
highest
}
fn live_slot(records: &[LogRecord], fallback: u32) -> u32 {
records
.iter()
.find_map(|record| match record {
LogRecord::SessionStart { slot, .. } => Some(*slot),
_ => None,
})
.unwrap_or(fallback)
}
```
The `open` body in S6 is given whole: copy it, do not leave a `todo!()` in `open`. The helpers
in S9 are given whole too.
## Steps
- [ ] **1. Copy the given test.** From the repository root:
```sh
cp docs/plans/M5d1/files/crates/loopd/tests/session_epochs.rs crates/loopd/tests/
```
- [ ] **2. See it fail.** `cargo build --tests --keep-going -p loopd`, timeout 600000 ms: test
target `session_epochs` does not compile (`E0599` no method `epoch` / `carry` / `slot`, and/or
`E0433` `NoCarry`). Every other target compiles. Existing `session` tests still compile.
- [ ] **3. The edits.** Make S1 to S9 in order. `cargo check -p loopd` prints no error and no
warning. `wc -l crates/loopd/src/session.rs` prints a number ≤ 500.
- [ ] **4. The tests.** `cargo test -p loopd --test session_epochs --test session`: every test
passes, 9 in `session_epochs` and the others as before.
- [ ] **5. Check.** `git diff --stat` lists exactly `crates/loopd/src/session.rs`.
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, timeout 600000 ms. Last line
`gate: ok`. Test count **1473** (1464 before, plus 9).
- [ ] **7. Log and commit.** Task cell `M5d1/02-loopd-session-epochs`. Stage with
`git add crates/loopd docs/implementer-log.md Cargo.lock`, then commit as `AGENTS.md` says.
## Done when
- `make gate` prints `gate: ok` with 1473 tests.
- `git diff HEAD~1 --stat` lists only `crates/loopd/src/session.rs`,
`crates/loopd/tests/session_epochs.rs` and `docs/implementer-log.md`.
## Stop and report if
- The branch check fails.
- A "before" block is not found exactly once.
- Step 2 shows a different result.
- `cargo check` warns or errors, or `session.rs` exceeds 500 lines.
- A test fails in step 4. Do not change the test.
- An existing `session` test fails: stop and report. Those tests still create and open epoch 0
only.
- The gate count is not 1473.
- A comment does not fit the code under it.
## What the given tests pin
Opening a directory with epochs 0 and 1 loads 1 and the carry from 0's `EpochEnd`; a missing
`EpochEnd` on epoch 0 when epoch 1 files exist is `NoCarry` naming `0.jsonl` with the damaged-log
runbook pointer; epoch 0 has no carry and keeps its `SessionStart` slot; two `User`s in epoch 0
and one in epoch 1 is `turns() == 3` while `records()` has one `User`; an untrusted `ToolResult`
in epoch 0 taints after compact; `next_call` is one more than the highest call in any epoch;
`append` after opening epoch 1 writes `1.jsonl`; a `1.baseline.json` without `1.jsonl` is ignored;
an `EpochEnd` on epoch 0 without the next files stays on epoch 0.
docs/plans/M5d1/03-loopd-carry-messages.md
+185
View File
@@ -0,0 +1,185 @@
# M5d-1 task 03: the carried summary in the next epoch's prefix
**Branch:** `m5d1`. Check it: `git branch --show-current` prints `m5d1` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `loopd: replay a carried summary as an unlogged user message`
**Log row:** the Task cell is `M5d1/03-loopd-carry-messages`.
## Goal
For epoch `n > 0`, the request prefix is the new system message, then one `ChatMessage::User`
whose content is a fixed heading plus the carry, then the new epoch's records (spec 3.5). That
user message is **not** a `LogRecord::User`: it is not a turn, not reported to Honcho, and not
counted by `turns()`. Epoch 0 is unchanged (`carry` is `None`).
`run_turn` must send this prefix, or a compacted session's next live turn would omit the
summary.
## Every place that changes
| Place | What happens there |
|---|---|
| `crates/loopd/src/baseline.rs` | `CARRY_HEADING`; `messages_with_carry`; `messages` calls it with `None` |
| `crates/loopd/src/turn.rs` | the request uses `messages_with_carry` and `session.carry()` |
| `crates/loopd/tests/carry_messages.rs` | new, given |
| everything else | **no change**: not the existing `baseline` tests (they still call `messages`) |
## Files
- Copy, from `docs/plans/M5d1/files/` (the same path under it): the new test
`crates/loopd/tests/carry_messages.rs`.
- Modify: `crates/loopd/src/baseline.rs`, `crates/loopd/src/turn.rs`,
`docs/implementer-log.md`.
- Do not touch: any other file.
## Facts you need
- **The heading** is built from pieces so no file holds a chat-template tag whole (T33):
```rust
pub const CARRY_HEADING: &str = concat!("Earlier ", "turns, summarised:");
```
The user content is `format!("{CARRY_HEADING}\n\n{carry}")` when `carry` is `Some`. `None`
inserts nothing.
- **`messages` stays.** Existing tests and `limits.rs` call `messages(baseline, records)`. Make
it `messages_with_carry(baseline, records, None)` so those tests do not change.
- **`run_turn` today** (one place):
```rust
messages: messages(baseline, session.records()),
```
It must pass `session.carry()`. `Session::carry` is `Option<&str>` from task 02.
- Insert the carry user **after** the system message and **before** walking `records`.
- The items the edits use. This table is complete.
| Item | What it does |
|---|---|
| `concat!` | heading from pieces |
| `Session::carry(&self) -> Option<&str>` | task 02 |
| `ChatMessage::User { content }` | the carry, and log users |
## The edits, exactly
### `crates/loopd/src/baseline.rs`
**B1.** After the `Baseline` impl, replace `messages`. The lines
```rust
/// The system message, then each log record replayed in order. Every record variant is named, so a
/// new one is a compile error rather than a silent skip.
pub fn messages(baseline: &Baseline, records: &[LogRecord]) -> Vec<ChatMessage> {
let mut out = vec![ChatMessage::System {
content: baseline.system.clone(),
}];
for record in records {
```
become
```rust
/// The heading in front of a carried summary. Built from pieces (T33).
pub const CARRY_HEADING: &str = concat!("Earlier ", "turns, summarised:");
/// The system message, then each log record replayed in order. Every record variant is named, so a
/// new one is a compile error rather than a silent skip.
pub fn messages(baseline: &Baseline, records: &[LogRecord]) -> Vec<ChatMessage> {
messages_with_carry(baseline, records, None)
}
/// Like `messages`, and when `carry` is `Some` inserts one user message after the system
/// prompt: the heading, a blank line, the summary. That user is not a `LogRecord::User`.
pub fn messages_with_carry(
baseline: &Baseline,
records: &[LogRecord],
carry: Option<&str>,
) -> Vec<ChatMessage> {
let mut out = vec![ChatMessage::System {
content: baseline.system.clone(),
}];
if let Some(summary) = carry {
out.push(ChatMessage::User {
content: format!("{CARRY_HEADING}\n\n{summary}"),
});
}
for record in records {
```
The rest of the `match` is unchanged; do not touch it.
### `crates/loopd/src/turn.rs`
**T1.** The `use` line
```rust
use crate::baseline::messages;
```
becomes
```rust
use crate::baseline::messages_with_carry;
```
**T2.** The lines
```rust
let request = ChatRequest {
slot: rt.cfg.slots.main,
messages: messages(baseline, session.records()),
tools: baseline.tools.clone(),
thinking: true,
};
```
become
```rust
let request = ChatRequest {
slot: rt.cfg.slots.main,
messages: messages_with_carry(baseline, session.records(), session.carry()),
tools: baseline.tools.clone(),
thinking: true,
};
```
## Steps
- [ ] **1. Copy the given test.**
```sh
cp docs/plans/M5d1/files/crates/loopd/tests/carry_messages.rs crates/loopd/tests/
```
- [ ] **2. See it fail.** `cargo build --tests --keep-going -p loopd`, timeout 600000 ms: test
target `carry_messages` does not compile (`E0432` unresolved import
`loopd::baseline::{CARRY_HEADING, messages_with_carry}`). Every other target compiles.
- [ ] **3. The edits.** B1, T1, T2. `cargo check -p loopd` prints no error and no warning.
- [ ] **4. The tests.** `cargo test -p loopd --test carry_messages --test baseline --test turn`:
every test passes, 5 in `carry_messages` and the others as before.
- [ ] **5. Check.** `git diff --stat` lists exactly `crates/loopd/src/baseline.rs` and
`crates/loopd/src/turn.rs`.
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, timeout 600000 ms. Last line
`gate: ok`. Test count **1478** (1473 before, plus 5).
- [ ] **7. Log and commit.** Task cell `M5d1/03-loopd-carry-messages`. Stage with
`git add crates/loopd docs/implementer-log.md Cargo.lock`, then commit as `AGENTS.md` says.
## Done when
- `make gate` prints `gate: ok` with 1478 tests.
- `git diff HEAD~1 --stat` lists only `crates/loopd/src/baseline.rs`, `crates/loopd/src/turn.rs`,
`crates/loopd/tests/carry_messages.rs` and `docs/implementer-log.md`.
## Stop and report if
- The branch check fails; a "before" block is not found exactly once; step 2 differs; `cargo
check` warns; a test fails; the gate count is not 1478; a comment does not fit its code.
- You think `messages` should insert the carry. It must not: existing tests call `messages` and
would see an extra user. `run_turn` is the one call that must pass `session.carry()`.
## What the given tests pin
Epoch 0 with `None` matches `messages`; a carry is a user after the system prompt and before the
epoch's users; the carry is not in the log records; the heading is the `concat!` pieces; the
first request of epoch 1 is not a prefix-extension of epoch 0's messages.
docs/plans/M5d1/04-memoryd-honcho-epochs.md
+232
View File
@@ -0,0 +1,232 @@
# M5d-1 task 04: Honcho reads every epoch file
**Branch:** `m5d1`. Check it: `git branch --show-current` prints `m5d1` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `memoryd: Honcho concatenates every epoch of a session`
**Log row:** the Task cell is `M5d1/04-memoryd-honcho-epochs`.
## Goal
`memoryd` keys a turn as `Item::Turn { session, turn }` with no epoch (spec 3.2). After
compaction, turn 3 may live in `1.jsonl`. `log_path` still names `0.jsonl` (existing tests pin
that path). A new `read_session` concatenates every `<n>.jsonl` in epoch order. `TurnSync`
walks that concatenation, and the worker's `send_turn` does too, so a turn in epoch 1 is
decided and sent.
`turn_records` / `turns` do not change: they already count `User` records on a slice.
## Every place that changes
| Place | What happens there |
|---|---|
| `crates/memoryd/src/honcho/turns.rs` | `epoch_logs`; `read_session`; `records` and `resume` use it |
| `crates/memoryd/src/honcho/worker.rs` | `send_turn` reads the concatenated log |
| `crates/memoryd/tests/honcho_epochs.rs` | new, given |
| everything else | **no change**: not `log_path`'s return value, not `decide.rs` |
## Files
- Copy, from `docs/plans/M5d1/files/` (the same path under it): the new test
`crates/memoryd/tests/honcho_epochs.rs`.
- Modify: `crates/memoryd/src/honcho/turns.rs`, `crates/memoryd/src/honcho/worker.rs`,
`docs/implementer-log.md`.
- Do not touch: any other file.
## Facts you need
- **`read_records` / `LogError`** already exist in `turns.rs`. `read_session` returns
`Result<Vec<LogRecord>, LogError>` so a damaged line still names its file and
`LOG_RUNBOOK`.
- **Listing.** Under `sessions/<id>/`, every file named `<n>.jsonl` where `n` parses as `u32`,
sorted by `n`. A missing session directory is empty, not an error (same as a missing
`0.jsonl` today).
- **`log_path` stays** `.../0.jsonl`. Do not change its body. `readable` in the worker still
names that path when the concatenated read succeeds, so existing recovery log lines stay
stable.
- **`send_turn` today** calls `read_turn(&log_path(...), turn)`, which only sees epoch 0. After
this task it loads `read_session`, then `turn_records` / `user_text` / `final_reply` from
`crate::honcho::decide` (already `pub`).
- `turns.rs` after this task stays under 500 lines (it is 332 now).
- The items the edits use. This table is complete.
| Item | What it does |
|---|---|
| `read_records(&Path) -> Result<Vec<LogRecord>, LogError>` | one file, already in `turns.rs` |
| `turn_records(&[LogRecord], u64) -> Option<&[LogRecord]>` | `decide.rs` |
| `user_text(&[LogRecord]) -> (&str, Option<&str>)` | `decide.rs` |
| `final_reply(&[LogRecord]) -> Option<&str>` | `decide.rs` |
| `str::strip_suffix(".jsonl")`, `str::parse::<u32>` | epoch numbers |
| `Unsent::Unreadable(String)` | worker, `e.to_string()` |
## The edits, exactly
### `crates/memoryd/src/honcho/turns.rs`
**H1.** The doc comment on `log_path`, the lines
```rust
/// The path of `session`'s log under `sessions`: epoch 0, the only one `loopd` writes.
pub fn log_path(sessions: &Path, session: &SessionId) -> PathBuf {
```
become
```rust
/// The path of `session`'s epoch-0 log under `sessions`. Later epochs are sibling files;
/// `read_session` concatenates them. Existing tests pin this path.
pub fn log_path(sessions: &Path, session: &SessionId) -> PathBuf {
```
**H1b.** After `read_records` (the function that ends `Ok(records)` / `}`), insert these two
functions. They sit below `LogError` and `read_records`, so both names resolve.
```rust
/// Every `<n>.jsonl` of `session`, sorted by `n`. A missing directory is empty.
pub fn epoch_logs(sessions: &Path, session: &SessionId) -> Result<Vec<PathBuf>, LogError> {
let dir = sessions.join(session.as_str());
let entries = match std::fs::read_dir(&dir) {
Ok(e) => e,
Err(e) if e.kind() == std::io::ErrorKind::NotFound => return Ok(Vec::new()),
Err(e) => return Err(LogError::Io(format!("{}: {e}", dir.display()))),
};
let mut found: Vec<(u32, PathBuf)> = Vec::new();
for entry in entries {
let entry = match entry {
Ok(e) => e,
Err(e) => return Err(LogError::Io(format!("{}: {e}", dir.display()))),
};
let name = entry.file_name();
let Some(name) = name.to_str() else { continue };
let Some(stem) = name.strip_suffix(".jsonl") else { continue };
let Ok(n) = stem.parse::<u32>() else { continue };
found.push((n, entry.path()));
}
found.sort_by_key(|(n, _)| *n);
Ok(found.into_iter().map(|(_, p)| p).collect())
}
/// Every epoch's records, in epoch order. A missing session has none.
pub fn read_session(
sessions: &Path,
session: &SessionId,
) -> Result<Vec<LogRecord>, LogError> {
let mut out = Vec::new();
for path in epoch_logs(sessions, session)? {
out.extend(read_records(&path)?);
}
Ok(out)
}
```
The next function after this insert is `finished`. Do not change `finished`.
**H2.** In `resume`, the line
```rust
if let Ok(records) = read_log(&log_path(&self.sessions, &session)) {
```
becomes
```rust
if let Ok(records) = read_session(&self.sessions, &session) {
```
**H3.** In `records`, the line
```rust
match read_log(&log_path(&self.sessions, session)) {
```
becomes
```rust
match read_session(&self.sessions, session).map_err(|e| e.to_string()) {
```
`read_log` already mapped `LogError` through `Display`; this keeps the logged line the same
shape (`memoryd: {why}; {LOG_RUNBOOK}`).
### `crates/memoryd/src/honcho/worker.rs`
**W1.** The `use` line
```rust
use crate::honcho::turns::{TurnSync, log_path};
```
becomes
```rust
use crate::honcho::decide::{final_reply, turn_records, user_text};
use crate::honcho::turns::{TurnSync, log_path, read_session};
```
**W2.** The body of `send_turn`, the lines
```rust
let path = log_path(&self.sessions, session);
let read = read_turn(&path, turn).map_err(Unsent::Unreadable)?;
self.readable(&path);
let (text, reply) = read.ok_or(Unsent::Local("log"))?;
```
become
```rust
let path = log_path(&self.sessions, session);
let records = match read_session(&self.sessions, session) {
Ok(records) => {
self.readable(&path);
records
}
Err(e) => return Err(Unsent::Unreadable(e.to_string())),
};
let slice = turn_records(&records, turn).ok_or(Unsent::Local("log"))?;
let (text, _) = user_text(slice);
let reply = final_reply(slice).ok_or(Unsent::Local("log"))?;
```
The rest of `send_turn` is unchanged (`honcho.ok_or`, `batch`, `MAX_BATCH`, `add_messages`).
`text` is `&str` from `user_text`; `reply` is `&str` from `final_reply`. `batch` already takes
`&str`.
## Steps
- [ ] **1. Copy the given test.**
```sh
cp docs/plans/M5d1/files/crates/memoryd/tests/honcho_epochs.rs crates/memoryd/tests/
```
- [ ] **2. See it fail.** `cargo build --tests --keep-going -p memoryd`, timeout 600000 ms: test
target `honcho_epochs` does not compile (`E0432` unresolved import
`memoryd::honcho::turns::read_session`). Every other target compiles.
- [ ] **3. The edits.** H1, H1b, H2, H3, W1, W2. `cargo check -p memoryd`
prints no error and no warning. `wc -l crates/memoryd/src/honcho/turns.rs` and
`wc -l crates/memoryd/src/honcho/worker.rs` are each ≤ 500.
- [ ] **4. The tests.** `cargo test -p memoryd --test honcho_epochs --test honcho_turns --test
honcho_worker`: every test passes, 5 in `honcho_epochs` and the others as before.
- [ ] **5. Check.** `git diff --stat` lists exactly `crates/memoryd/src/honcho/turns.rs` and
`crates/memoryd/src/honcho/worker.rs`.
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, timeout 600000 ms. Last line
`gate: ok`. Test count **1483** (1478 before, plus 5).
- [ ] **7. Log and commit.** Task cell `M5d1/04-memoryd-honcho-epochs`. Stage with
`git add crates/memoryd docs/implementer-log.md Cargo.lock`, then commit as `AGENTS.md` says.
## Done when
- `make gate` prints `gate: ok` with 1483 tests.
- `git diff HEAD~1 --stat` lists only the two `honcho` sources, `honcho_epochs.rs`, and the log.
## Stop and report if
- The branch check fails; a "before" block is not found exactly once; step 2 differs; a file
exceeds 500 lines; a test fails; the gate count is not 1483.
- You think `log_path` should return every file. It must still return `0.jsonl`.
## What the given tests pin
Two users in epoch 0 and one in epoch 1 is `turns() == 3` and `turn_records(..., 3)` is the
epoch-1 user; epoch 0 alone still reads; a damaged `1.jsonl` names that file; `TurnSync` decides
turn 3 from epoch 1; a missing session directory is empty.
docs/plans/M5d1/05-loopd-compact-due.md
+130
View File
@@ -0,0 +1,130 @@
# M5d-1 task 05: when to compact, and the fallback summary
**Branch:** `m5d1`. Check it: `git branch --show-current` prints `m5d1` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `loopd: compaction threshold and the fallback summary`
**Log row:** the Task cell is `M5d1/05-loopd-compact-due`.
## Goal
A session is due to compact when the current epoch's last `Usage` has
`cache_n + prompt_n` at or above `threshold × expect.n_ctx` (spec 3.3). There is no idle timer.
If the summary call is context-full (task 06), the fallback is the epoch's `User` texts, newest
last, cut to `summary_chars` on a character boundary (spec 3.4).
This task is those two pure functions, the `compact` module, and the starter `config/compact.md`.
No inference, no disk besides copying the starter.
## Every place that changes
| Place | What happens there |
|---|---|
| `crates/loopd/src/compact.rs` | new: `due`, `fallback_summary`, `cap_chars` |
| `crates/loopd/src/lib.rs` | `pub mod compact;` |
| `config/compact.md` | new, given starter |
| `crates/loopd/tests/compact_due.rs` | new, given |
| everything else | **no change** |
## Files
- Copy, from `docs/plans/M5d1/files/` (the same path under it): the new test
`crates/loopd/tests/compact_due.rs` and the starter `config/compact.md`.
- Create: `crates/loopd/src/compact.rs` (skeleton below).
- Modify: `crates/loopd/src/lib.rs`, `docs/implementer-log.md`.
- Do not touch: any other file.
## Facts you need
- **Depth is saturating.** `cache_n.saturating_add(prompt_n) as f64 >= threshold * (n_ctx as f64)`.
0.85 × 262144 is 222822.4, so 222822 is not due and 222823 is. 0.85 × 1000 is 850 exactly.
- **Cast.** `u64 as f64` is exact for these magnitudes (n_ctx is 262144). Do not use `as` on
untrusted indexes; these are our own config and the server's counts.
- **Fallback join.** `"\n\n"` between texts. Newest last: reverse the `User` contents, then join,
then cap.
- **Cap.** `s.chars().take(n).collect::<String>()`. `n == 0` is empty. A multi-byte character
is not split.
- **`config/compact.md`** is copied like `config/system.md`. Task 06 reads it from
`Config::compact_prompt`.
- The items the edits use. This table is complete.
| Item | What it does |
|---|---|
| `bxcore::Usage` | `cache_n`, `prompt_n` |
| `LogRecord::User { content, .. }` | fallback texts |
| `str::chars`, `Iterator::take` | the cap |
## Skeleton: `crates/loopd/src/compact.rs`
Write this file, then fill the two `todo!()` bodies from their comments and delete the
comments. Each comment is at most six lines.
```rust
//! Compaction: when a session starts a new epoch, the summary call, and writing the next files
//! (M5d spec, section 3). Later tasks fill the call and the write.
use bxcore::{LogRecord, Usage};
/// True when `cache_n + prompt_n` is at or above `threshold * n_ctx`.
pub fn due(usage: &Usage, threshold: f64, n_ctx: u64) -> bool {
// saturating_add the two counts; compare as f64 to `threshold * (n_ctx as f64)`.
todo!()
}
/// The epoch's `User` texts, newest last, joined with `"\n\n"`, then `cap_chars`.
pub fn fallback_summary(records: &[LogRecord], summary_chars: usize) -> String {
// Collect `&str` contents of every `User`, reverse, join `"\n\n"`, then `cap_chars`.
todo!()
}
pub(crate) fn cap_chars(s: &str, n: usize) -> String {
s.chars().take(n).collect()
}
```
### `crates/loopd/src/lib.rs`
**L1.** After `pub mod channel;` add `pub mod compact;` (keep rustfmt's order; if rustfmt
moves the line, accept that).
## Steps
- [ ] **1. Copy the given files.**
```sh
cp docs/plans/M5d1/files/crates/loopd/tests/compact_due.rs crates/loopd/tests/
cp docs/plans/M5d1/files/config/compact.md config/compact.md
```
- [ ] **2. Write the skeleton** at `crates/loopd/src/compact.rs` and make L1. See it fail:
`cargo test -p loopd --test compact_due`, timeout 600000 ms: the two `due` tests and the
three `fallback` tests panic on `todo!()` (or fail their asserts if you filled the bodies
early). If the test target does not compile (`E0432` unresolved `loopd::compact`), the
`mod` line is missing.
- [ ] **3. Fill `due` and `fallback_summary`.** Delete each instruction comment. `cargo check
-p loopd` prints no error and no warning.
- [ ] **4. The tests.** `cargo test -p loopd --test compact_due`: 6 passed.
- [ ] **5. Check.** `git diff --stat` lists `crates/loopd/src/lib.rs`,
`crates/loopd/src/compact.rs`, `config/compact.md`. `git status --short` shows
`compact_due.rs` as new.
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, timeout 600000 ms. Last line
`gate: ok`. Test count **1489** (1483 before, plus 6).
- [ ] **7. Log and commit.** Task cell `M5d1/05-loopd-compact-due`. Stage with
`git add crates/loopd config/compact.md docs/implementer-log.md Cargo.lock`, then commit
as `AGENTS.md` says.
## Done when
- `make gate` prints `gate: ok` with 1489 tests.
- `git diff HEAD~1 --stat` lists only `crates/loopd/src/lib.rs`, `crates/loopd/src/compact.rs`,
`crates/loopd/tests/compact_due.rs`, `config/compact.md` and the log.
## Stop and report if
- The branch check fails; step 2 fails to compile for a reason other than the missing module
or `todo!()`; a test fails; the gate count is not 1489; you add an idle timer. There is none.
## What the given tests pin
849/1000 is not due, 850 is; 222822/262144 is not due, 222823 is; threshold 1.0 is the full
window; fallback is newest-last with `"\n\n"`; the cap is in characters on a boundary; no
users is empty.
docs/plans/M5d1/06-loopd-compact-summary.md
+213
View File
@@ -0,0 +1,213 @@
# M5d-1 task 06: the summary call, and `session-compact-failed`
**Branch:** `m5d1`. Check it: `git branch --show-current` prints `m5d1` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `loopd: compact summary on the background slot`
**Log row:** the Task cell is `M5d1/06-loopd-compact-summary`.
## Goal
Compaction asks for a summary of the current epoch on the **background** slot, with **no tools**,
thinking on (spec 3.4). The last user message is `compact.md` (task 01's `compact_prompt`),
trimmed at the end. The completion is tag-neutralised and cut to `summary_chars`.
If that call is context-full (`exceed_context_size_error`), use task 05's `fallback_summary`.
If `compact.md` cannot be read, or the call fails for any other reason, return
`CompactError::Failed` whose `Display` ends with the runbook pointer
`see docs/runbook.md#session-compact-failed`. There is no fallback for a missing prompt.
This task does not write the next epoch. Task 07 does.
## Every place that changes
| Place | What happens there |
|---|---|
| `crates/loopd/src/compact.rs` | `CompactError`; `summarise` (edits E1, E2) |
| `docs/runbook.md` | append the given `session-compact-failed` entry |
| `crates/loopd/tests/compact_summary.rs` | new, given |
| everything else | **no change**: not `turn.rs`, not `channel.rs`, not `session.rs` |
## Files
- Copy, from `docs/plans/M5d1/files/` (the same path under it): the new test
`crates/loopd/tests/compact_summary.rs`.
- Append, from `docs/plans/M5d1/files/runbook-session-compact-failed.md`, to `docs/runbook.md`.
- Modify: `crates/loopd/src/compact.rs`, `docs/runbook.md`, `docs/implementer-log.md`.
- Do not touch: any other file. `config/compact.md` is already in the tree from task 05.
## Facts you need
- **The request.** `ChatRequest { slot: cfg.slots.background, messages, tools: Vec::new(),
thinking: true }`. Empty `tools` is omitted from the JSON (`skip_serializing_if` on the body).
The given test asserts `body.get("tools").is_none()` and `id_slot == 1` (the test config's
background slot).
- **The messages.** `messages_with_carry(session.baseline(), session.records(), session.carry())`,
then one `ChatMessage::User` whose content is `compact.md` after `trim_end`. Epoch 0 has no
carry; later epochs keep the carried summary in the prefix so the model summarises what the
session actually sends.
- **Read the prompt** with `std::fs::read_to_string(&cfg.compact_prompt())`. An I/O error is
`Failed { detail: format!("{}: {e}", path.display()) }` so the path (and `compact.md`) appears
in the message.
- **The call.** `client.chat_with_retry(&request, &mut |_| {})`. A 400 that is not context-full
is not retried (`is_retryable` is already false for it).
- **Context-full.** `crate::turn::is_context_full(&e)` then `fallback_summary(session.records(),
cfg.compact.summary_chars)`. Do not fall back for a missing file or any other `InferError`.
- **The text.** `completion.content.unwrap_or_default()`, then `bxcore::sanitize::neutralise`,
then `cap_chars(..., cfg.compact.summary_chars)`. Neutralise first so a tag is not split by
the cap.
- **`Display`.** `"{detail}; see docs/runbook.md#session-compact-failed"`. The given tests box
the error as `dyn Error` and check `to_string()`.
- **`compact.rs` stays ≤ 500 lines.** After this task it is about 120.
- The items the edits use. This table is complete.
| Item | What it does |
|---|---|
| `Config::compact_prompt` | task 01, `compact.md` beside `system.md` |
| `messages_with_carry` | task 03 |
| `Client::chat_with_retry` | same as a live turn |
| `turn::is_context_full` | the one 400 `exceed_context_size_error` |
| `bxcore::sanitize::neutralise` | tags to a visible look-alike |
| `fallback_summary`, `cap_chars` | task 05 |
| `ChatRequest.tools: Vec::new()` | omits `tools` on the wire |
## The edits, exactly
Nothing else in `compact.rs` changes. Each quoted "before" block occurs exactly once after
task 05. Keep the fields in the order shown.
### `crates/loopd/src/compact.rs`
**E1.** The module doc and the `use` line. The lines
```rust
//! Compaction: when a session starts a new epoch, the summary call, and writing the next files
//! (M5d spec, section 3). Later tasks fill the call and the write.
use bxcore::{LogRecord, Usage};
```
become
```rust
//! Compaction: when a session starts a new epoch, the summary call, and writing the next files
//! (M5d spec, section 3). Task 07 writes the next files; task 08 calls this after a turn.
use bxcore::sanitize::neutralise;
use bxcore::{LogRecord, Usage};
use crate::baseline::messages_with_carry;
use crate::config::Config;
use crate::llama::{ChatMessage, ChatRequest, Client};
use crate::session::Session;
use crate::turn::is_context_full;
/// Why a summary call or a next-epoch write failed. Every variant's `Display` ends with the
/// runbook pointer `session-compact-failed`.
#[derive(Debug)]
pub enum CompactError {
Failed { detail: String },
}
impl std::fmt::Display for CompactError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
CompactError::Failed { detail } => write!(
f,
"{detail}; see docs/runbook.md#session-compact-failed"
),
}
}
}
impl std::error::Error for CompactError {}
```
**E2.** After `cap_chars` (the function that ends `s.chars().take(n).collect()` / `}`), add
`summarise`. Copy this function whole. There is no `todo!()`.
```rust
/// One completion on the background slot: no tools, thinking on, last user `compact.md`.
/// Context-full uses `fallback_summary`. A missing prompt, or any other infer error, is
/// `CompactError::Failed` with the runbook pointer. No fallback for a missing prompt.
pub fn summarise(cfg: &Config, client: &Client, session: &Session) -> Result<String, CompactError> {
let path = cfg.compact_prompt();
let prompt = std::fs::read_to_string(&path).map_err(|e| CompactError::Failed {
detail: format!("{}: {e}", path.display()),
})?;
let mut messages = messages_with_carry(session.baseline(), session.records(), session.carry());
messages.push(ChatMessage::User {
content: prompt.trim_end().to_string(),
});
let request = ChatRequest {
slot: cfg.slots.background,
messages,
tools: Vec::new(),
thinking: true,
};
match client.chat_with_retry(&request, &mut |_| {}) {
Ok(completion) => {
let text = completion.content.unwrap_or_default();
let text = neutralise(&text);
Ok(cap_chars(text.as_ref(), cfg.compact.summary_chars))
}
Err(e) if is_context_full(&e) => {
Ok(fallback_summary(session.records(), cfg.compact.summary_chars))
}
Err(e) => Err(CompactError::Failed {
detail: e.to_string(),
}),
}
}
```
## Steps
- [ ] **1. Copy the given test.**
```sh
cp docs/plans/M5d1/files/crates/loopd/tests/compact_summary.rs crates/loopd/tests/
```
- [ ] **2. See it fail.** `cargo build --tests --keep-going -p loopd`, timeout 600000 ms: test
target `compact_summary` does not compile (`E0432` unresolved import `loopd::compact::CompactError`
and/or `E0599` no function `summarise`). Every other target compiles. `compact_due` still
compiles.
- [ ] **3. The edits.** E1, E2. `cargo check -p loopd` prints no error and no warning.
`wc -l crates/loopd/src/compact.rs` prints a number ≤ 500.
- [ ] **4. The tests.** `cargo test -p loopd --test compact_summary --test compact_due`: every
test passes, 6 in `compact_summary` and 6 in `compact_due`.
- [ ] **5. The runbook.** From the repository root:
```sh
{ echo; cat docs/plans/M5d1/files/runbook-session-compact-failed.md; } >> docs/runbook.md
```
Then `grep -c '^## session-compact-failed$' docs/runbook.md` prints `1`.
- [ ] **6. Check.** `git diff --stat` lists `crates/loopd/src/compact.rs` and `docs/runbook.md`.
`git status --short` shows `compact_summary.rs` as new.
- [ ] **7. Run the gate.** `cargo fmt --all`, then `make gate`, timeout 600000 ms. Last line
`gate: ok`. Test count **1495** (1489 before, plus 6).
- [ ] **8. Log and commit.** Task cell `M5d1/06-loopd-compact-summary`. Stage with
`git add crates/loopd docs/runbook.md docs/implementer-log.md Cargo.lock`, then commit as
`AGENTS.md` says.
## Done when
- `make gate` prints `gate: ok` with 1495 tests.
- `git diff HEAD~1 --stat` lists only `crates/loopd/src/compact.rs`,
`crates/loopd/tests/compact_summary.rs`, `docs/runbook.md` and the log.
## Stop and report if
- The branch check fails; a "before" block is not found exactly once; step 2 differs; `cargo
check` warns; a test fails; the gate count is not 1495; a comment does not fit its code;
`compact.rs` exceeds 500 lines; you add a fallback for a missing `compact.md`; you send tools
on the summary request; you use `slots.main` for the summary.
## What the given tests pin
The summary request is `id_slot` of the background slot, omits `tools`, has thinking on, and its
last user is `compact.md` after `trim_end`; the completion is capped to `summary_chars`; a
context-full summary is the epoch's `User` texts; tags in the completion are neutralised; a
missing `compact.md` names the file and the runbook; a non-context-full infer error is
`CompactError::Failed` with the same pointer.
docs/plans/M5d1/07-loopd-compact-advance.md
+268
View File
@@ -0,0 +1,268 @@
# M5d-1 task 07: writing the next epoch
**Branch:** `m5d1`. Check it: `git branch --show-current` prints `m5d1` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `loopd: write the next epoch from a compaction summary`
**Log row:** the Task cell is `M5d1/07-loopd-compact-advance`.
## Goal
Given a summary string, `commit` starts the next epoch (spec 3.5):
1. Append `EpochEnd { next, summary }` to the current log, unless that log already ends with an
`EpochEnd` (a compact that stopped after the end record and before the next files).
2. Ask `memoryd` for a fresh block; write `<n>.baseline.json` from `assemble_with` (so identity
edits applied in the old epoch land here, and `registry.core_schemas()` is the new tools
array).
3. Create `<n>.jsonl` with `SessionStart { epoch: n, slot: the session's live slot, baseline:
hash of the new file }`.
4. The in-memory `Session` switches: new baseline, records that are only that `SessionStart`,
`carry = Some(summary)`, `prior_users` / `prior_untrusted` accumulated from the old records.
A crash after step 1 and before the next files exist reopens epoch 0 with `EpochEnd` last (task
02). The next `commit` resumes: it uses the summary already in that `EpochEnd` as the carry and
does not write a second `EpochEnd`. The summary argument is ignored in that case.
## Every place that changes
| Place | What happens there |
|---|---|
| `crates/loopd/src/session.rs` | `begin_epoch` (edit S1) |
| `crates/loopd/src/compact.rs` | `commit` (edit C1) |
| `crates/loopd/tests/compact_advance.rs` | new, given |
| everything else | **no change**: not `turn.rs`, not `channel.rs` |
## Files
- Copy, from `docs/plans/M5d1/files/` (the same path under it): the new test
`crates/loopd/tests/compact_advance.rs`.
- Modify: `crates/loopd/src/session.rs`, `crates/loopd/src/compact.rs`,
`docs/implementer-log.md`.
- Do not touch: any other file.
## Facts you need
- **`begin_epoch` is `pub(crate)`** on `Session` so `compact.rs` can switch the private fields.
Tests call `compact::commit`, never `begin_epoch`.
- **Order inside `begin_epoch`.** Write `<n>.baseline.json` (overwrite is fine on resume),
`create_new` the log file, then set `epoch` / `baseline` / `carry` / empty `records` / the new
`File`, **then** `persist` `SessionStart`. `log_path` uses `self.epoch`, so `epoch` must be
`next` before that persist. Accumulate `prior_users` and `prior_untrusted` from the **old**
`records` before clearing them. Do not reset `next_call` or `slot`.
- **Resume.** If `session.records().last()` is `EpochEnd { next, summary }`, skip the append;
`next` and `summary` (cloned) are what `begin_epoch` gets. Otherwise append `EpochEnd` with
`next = Epoch(session.epoch().0.saturating_add(1))` and the summary argument, then
`begin_epoch`.
- **Memory.** `memory.block().map_err(|e| CompactError::Failed { detail: e })?`. Then
`Baseline::assemble_with(cfg, registry, &block)`. `assemble_with` already calls
`registry.core_schemas()`, so a new core tool lands at this boundary. Map assemble and session
errors to `Failed { detail: e.to_string() }`.
- **`NoMemory.block()`** is `Ok("")`. The given test `the_new_baseline_takes_a_fresh_memory_block`
uses a `MemorySource` that returns a distinctive sentence.
- **`session.rs` stays ≤ 500 lines.** After task 02 it is about 430; `begin_epoch` is about 40.
- **`compact.rs` stays ≤ 500 lines.** After this task it is about 180.
- The items the edits use. This table is complete.
| Item | What it does |
|---|---|
| `Baseline::assemble_with` | new system + memory block + `registry.core_schemas()` |
| `Baseline::to_json` / `hash` | the file and `SessionStart.baseline` |
| `OpenOptions::create_new(true).append(true)` | the new log, as `create` does |
| `count_users` / `any_untrusted` | task 02, module-private in `session.rs` |
| `Session::persist` | private; `begin_epoch` is in the same impl |
| `MemorySource::block` | fresh text; `Err` is `Failed` |
| `Timestamp::now` | `EpochEnd` and `SessionStart` |
| `Epoch(n.saturating_add(1))` | the next number |
## The edits, exactly
Each quoted "before" block occurs exactly once after task 06 (session.rs is as task 02 left it).
### `crates/loopd/src/session.rs`
**S1.** `log_path`, the lines
```rust
fn log_path(&self) -> PathBuf {
self.dir.join(format!("{}.jsonl", self.epoch.0))
}
}
```
become
```rust
fn log_path(&self) -> PathBuf {
self.dir.join(format!("{}.jsonl", self.epoch.0))
}
/// Write `<n>.baseline.json` and `<n>.jsonl` with `SessionStart`, and switch this session
/// onto that epoch. The live slot and `next_call` stay. Callers pass the carry that
/// `messages_with_carry` will insert.
pub(crate) fn begin_epoch(
&mut self,
next: Epoch,
baseline: Baseline,
carry: String,
) -> Result<(), SessionError> {
let n = next.0;
let baseline_path = self.dir.join(format!("{n}.baseline.json"));
let json = baseline.to_json().map_err(SessionError::Encode)?;
std::fs::write(&baseline_path, json)
.map_err(|e| SessionError::Io(baseline_path.clone(), e))?;
let log_path = self.dir.join(format!("{n}.jsonl"));
let log = OpenOptions::new()
.create_new(true)
.append(true)
.open(&log_path)
.map_err(|e| SessionError::Io(log_path.clone(), e))?;
self.prior_users = self.prior_users.saturating_add(count_users(&self.records));
self.prior_untrusted = self.prior_untrusted || any_untrusted(&self.records);
self.epoch = next;
self.baseline = baseline;
self.carry = Some(carry);
self.records = Vec::new();
self.log = log;
let hash = self.baseline.hash().map_err(SessionError::Baseline)?;
self.persist(LogRecord::SessionStart {
time: Timestamp::now(),
session: self.id.clone(),
epoch: next,
slot: self.slot,
baseline: hash,
})
}
}
```
The helpers after the `impl` (`complete_epochs` and the rest) stay where they are. Do not move
them.
### `crates/loopd/src/compact.rs`
**C1.** After `summarise` (the function that ends the `match client.chat_with_retry` / `}`), add
the `use` items this function needs by extending the existing `use crate::` group, then add
`commit`. The current crate uses (after task 06) are:
```rust
use crate::baseline::messages_with_carry;
use crate::config::Config;
use crate::llama::{ChatMessage, ChatRequest, Client};
use crate::session::Session;
use crate::turn::is_context_full;
```
They become
```rust
use crate::baseline::{messages_with_carry, Baseline};
use crate::config::Config;
use crate::llama::{ChatMessage, ChatRequest, Client};
use crate::memory::MemorySource;
use crate::session::Session;
use crate::tools::Registry;
use crate::turn::is_context_full;
```
And the bxcore use
```rust
use bxcore::{LogRecord, Usage};
```
becomes
```rust
use bxcore::{Epoch, LogRecord, Timestamp, Usage};
```
Then this function, whole, after `summarise`:
```rust
/// Append `EpochEnd` (unless it is already last), rebuild the baseline from `memory`, and
/// switch `session` onto the next epoch. When the current log already ends with `EpochEnd`,
/// that record's summary is the carry and the argument is ignored.
pub fn commit(
session: &mut Session,
cfg: &Config,
registry: &Registry,
memory: &dyn MemorySource,
summary: String,
) -> Result<(), CompactError> {
let (next, summary) = match session.records().last() {
Some(LogRecord::EpochEnd { next, summary, .. }) => (*next, summary.clone()),
_ => {
let next = Epoch(session.epoch().0.saturating_add(1));
session
.append(LogRecord::EpochEnd {
time: Timestamp::now(),
next,
summary: summary.clone(),
})
.map_err(|e| CompactError::Failed {
detail: e.to_string(),
})?;
(next, summary)
}
};
let block = memory.block().map_err(|e| CompactError::Failed { detail: e })?;
let baseline = Baseline::assemble_with(cfg, registry, &block).map_err(|e| {
CompactError::Failed {
detail: e.to_string(),
}
})?;
session
.begin_epoch(next, baseline, summary)
.map_err(|e| CompactError::Failed {
detail: e.to_string(),
})?;
Ok(())
}
```
If rustfmt splits the `map_err` closures, accept that.
## Steps
- [ ] **1. Copy the given test.**
```sh
cp docs/plans/M5d1/files/crates/loopd/tests/compact_advance.rs crates/loopd/tests/
```
- [ ] **2. See it fail.** `cargo build --tests --keep-going -p loopd`, timeout 600000 ms: test
target `compact_advance` does not compile (`E0599` no function `commit`). Every other target
compiles.
- [ ] **3. The edits.** S1, then C1. `cargo check -p loopd` prints no error and no warning.
`wc -l crates/loopd/src/session.rs` and `wc -l crates/loopd/src/compact.rs` each print a
number ≤ 500.
- [ ] **4. The tests.** `cargo test -p loopd --test compact_advance --test session_epochs --test
compact_summary`: every test passes, 6 in `compact_advance`, 9 in `session_epochs`, 6 in
`compact_summary`.
- [ ] **5. Check.** `git diff --stat` lists exactly `crates/loopd/src/session.rs` and
`crates/loopd/src/compact.rs`. `git status --short` shows `compact_advance.rs` as new.
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, timeout 600000 ms. Last line
`gate: ok`. Test count **1501** (1495 before, plus 6).
- [ ] **7. Log and commit.** Task cell `M5d1/07-loopd-compact-advance`. Stage with
`git add crates/loopd docs/implementer-log.md Cargo.lock`, then commit as `AGENTS.md` says.
## Done when
- `make gate` prints `gate: ok` with 1501 tests.
- `git diff HEAD~1 --stat` lists only `crates/loopd/src/session.rs`,
`crates/loopd/src/compact.rs`, `crates/loopd/tests/compact_advance.rs` and the log.
## Stop and report if
- The branch check fails; a "before" block is not found exactly once; step 2 differs; `cargo
check` warns; a test fails; the gate count is not 1501; a file exceeds 500 lines; you write a
second `EpochEnd` on resume; you reset `slot` or `next_call`; you log the summary as a
`LogRecord::User`.
## What the given tests pin
`commit` writes `EpochEnd` on the old log, a new baseline file, and a new log whose
`SessionStart` has the next epoch; the new baseline contains a fresh memory block; reopen loads
the new epoch and its carry; a second commit leaves epoch 0 with one `EpochEnd` and writes the
next on epoch 1; resume after `EpochEnd` without next files uses that summary as carry and does
not write a second end; the new `SessionStart` keeps the live slot (including a non-default 7).
docs/plans/M5d1/08-loopd-compact-turn.md
+261
View File
@@ -0,0 +1,261 @@
# M5d-1 task 08: compact after a turn, and retry once on `SessionFull`
**Branch:** `m5d1`. Check it: `git branch --show-current` prints `m5d1` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `loopd: compact after a full turn and retry SessionFull once`
**Log row:** the Task cell is `M5d1/08-loopd-compact-turn`.
## Goal
After `run_turn` returns, `finish_turn` decides whether to compact (spec 3.3):
- A successful turn whose last `Usage` is `due` is compacted before the caller returns. If
compact fails, the turn stays successful and the session stays on its epoch; `loopd` logs the
compact error with the runbook pointer.
- `TurnError::SessionFull` is compacted, then that user message is `run_turn`'d once in the new
epoch. If compact fails, the error stays `SessionFull`. The retry is not compacted again.
- Any other `TurnError` is returned as it is. No compact.
`Runtime` does not gain `MemorySource` (given `turn` tests build a `Runtime` without one).
`finish_turn` takes `memory` as its own argument.
The channel holds the session busy for this whole call: `handle` runs `finish_turn` **before**
`drop(held)`. A compact is not an idle timer and does not start after the busy set releases.
Live turns still send `slots.main` (task 03). Only the summary call uses the background slot.
## Every place that changes
| Place | What happens there |
|---|---|
| `crates/loopd/src/compact.rs` | `finish_turn` and its two helpers (edit F1) |
| `crates/loopd/src/channel.rs` | `handle` calls `finish_turn` before `drop(held)` (edits H1, H2) |
| `crates/loopd/tests/compact_turn.rs` | new, given |
| everything else | **no change**: not `turn.rs`, not `Runtime`, not existing `turn` / `channel` tests |
## Files
- Copy, from `docs/plans/M5d1/files/` (the same path under it): the new test
`crates/loopd/tests/compact_turn.rs`.
- Modify: `crates/loopd/src/compact.rs`, `crates/loopd/src/channel.rs`,
`docs/implementer-log.md`.
- Do not touch: any other file.
## Facts you need
- **`due` looks at `session.last_usage()`**, which is the current epoch's last `Usage`. After a
successful `run_turn` that record is there. After `SessionFull`, `run_turn` has already
appended the `User` and returned before any `Usage`; compact runs because of the error, not
because of `due`.
- **Compact helper.** `summarise` then `commit`. Log failures with
```text
loopd: could not compact session <id>: <CompactError Display>
```
`Display` already includes the runbook pointer, matching the runbook's example line.
- **Retry.** `run_turn` appends the user message again in the new epoch. The old epoch keeps its
`User` (Honcho skips it: no reply). `turns()` is 2. That is accepted (README, known limits).
- **Existing channel tests stay green.** Their `n_ctx` is 131072 and fixture `prompt_n` is tens,
so `due` is false. `SessionFull` without `compact.md` in `Home` stays `SessionFull` (the
successful-turn-when-compact-fails behaviour, on the error path).
- **`compact.rs` stays ≤ 500 lines.** After this task it is about 250.
- **`channel.rs` stays ≤ 500 lines.** After this task it is about 290.
- The items the edits use. This table is complete.
| Item | What it does |
|---|---|
| `due` | task 05 |
| `summarise` / `commit` | tasks 06, 07 |
| `run_turn` | retry the same `text` and `speaker` |
| `Session::last_usage` | current epoch |
| `Config.compact.threshold`, `expect.n_ctx` | the `due` arguments |
| `MemorySource` | `finish_turn`'s extra argument; channel passes `&*ctx.memory` |
| `Held` | already in `channel.rs`; drop it after `finish_turn` |
## The edits, exactly
Each quoted "before" block occurs exactly once after task 07 (`channel.rs` is still as on
master, except rustfmt).
### `crates/loopd/src/compact.rs`
**F1.** Extend the crate uses. After task 07 they include `MemorySource`, `Session`, `Config`,
`Registry`. Add:
```rust
use crate::turn::{Runtime, TurnError, TurnOutcome, run_turn};
use bxcore::TurnEvent;
```
Keep rustfmt's order of `use` lines; if it moves them, accept that.
Then after `commit`, add these three functions whole. There is no `todo!()`.
```rust
/// After `run_turn`: compact when the last `Usage` is due, or on `SessionFull` (then retry the
/// user message once). Compact failure on a successful turn keeps the success; on `SessionFull`
/// keeps `SessionFull`. Other errors skip compact.
pub fn finish_turn(
session: &mut Session,
rt: &Runtime<'_>,
memory: &dyn MemorySource,
text: &str,
speaker: Option<&str>,
on_event: &mut dyn FnMut(&TurnEvent),
outcome: Result<TurnOutcome, TurnError>,
) -> Result<TurnOutcome, TurnError> {
match outcome {
Err(TurnError::SessionFull) => match compact_session(session, rt, memory) {
Ok(()) => run_turn(session, rt, text, speaker, on_event),
Err(e) => {
log_compact(session, &e);
Err(TurnError::SessionFull)
}
},
Ok(done) => {
if let Some(usage) = session.last_usage()
&& due(&usage, rt.cfg.compact.threshold, rt.cfg.expect.n_ctx)
&& let Err(e) = compact_session(session, rt, memory)
{
log_compact(session, &e);
}
Ok(done)
}
Err(e) => Err(e),
}
}
fn compact_session(
session: &mut Session,
rt: &Runtime<'_>,
memory: &dyn MemorySource,
) -> Result<(), CompactError> {
let summary = summarise(rt.cfg, rt.client, session)?;
commit(session, rt.cfg, rt.registry, memory, summary)
}
fn log_compact(session: &Session, e: &CompactError) {
eprintln!(
"loopd: could not compact session {}: {e}",
session.id().as_str()
);
}
```
The `Ok(done)` arm uses `&& let Err(e) = ...` so a compact failure is logged and the original
`done` is still returned. Do not `?` that error.
### `crates/loopd/src/channel.rs`
**H1.** The `run_turn` call and the `drop(held)` that follows it. The lines
```rust
let outcome = run_turn(
&mut session,
&runtime,
&turn.content,
turn.speaker.as_deref(),
&mut |event| {
let _ = write_frame(
&mut stream,
&Envelope {
v: PROTOCOL_VERSION,
id: request.id,
r#final: false,
msg: Message::TurnEvent(event.clone()),
},
);
},
);
// 5. Release the session before the final frame, so the next turn does not find it busy.
drop(held);
```
become
```rust
let mut emit = |event: &bxcore::TurnEvent| {
let _ = write_frame(
&mut stream,
&Envelope {
v: PROTOCOL_VERSION,
id: request.id,
r#final: false,
msg: Message::TurnEvent(event.clone()),
},
);
};
let outcome = run_turn(
&mut session,
&runtime,
&turn.content,
turn.speaker.as_deref(),
&mut emit,
);
let outcome = crate::compact::finish_turn(
&mut session,
&runtime,
&*ctx.memory,
&turn.content,
turn.speaker.as_deref(),
&mut emit,
outcome,
);
// 5. Release the session before the final frame, so the next turn does not find it busy.
// Compaction ran inside the hold: a second turn on this session waits until the new
// epoch is on disk (or compact failed and the epoch is unchanged).
drop(held);
```
**H2.** Do not add `MemorySource` to `Runtime`. Do not change `turn_code`. The `memory.updated`
block after `drop(held)` stays: a successful compact-and-retry still reports `session.turns()`,
which now spans epochs.
## Steps
- [ ] **1. Copy the given test.**
```sh
cp docs/plans/M5d1/files/crates/loopd/tests/compact_turn.rs crates/loopd/tests/
```
- [ ] **2. See it fail.** `cargo build --tests --keep-going -p loopd`, timeout 600000 ms: test
target `compact_turn` does not compile (`E0599` no function `finish_turn`). Every other
target compiles, including `turn` and `channel`.
- [ ] **3. The edits.** F1, H1. `cargo check -p loopd` prints no error and no warning.
`wc -l crates/loopd/src/compact.rs` and `wc -l crates/loopd/src/channel.rs` each print a
number ≤ 500.
- [ ] **4. The tests.** `cargo test -p loopd --test compact_turn --test turn --test channel
--test compact_advance`: every test passes, 6 in `compact_turn` and the others as before.
- [ ] **5. Check.** `git diff --stat` lists exactly `crates/loopd/src/compact.rs` and
`crates/loopd/src/channel.rs`. `git status --short` shows `compact_turn.rs` as new.
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, timeout 600000 ms. Last line
`gate: ok`. Test count **1507** (1501 before, plus 6).
- [ ] **7. Log and commit.** Task cell `M5d1/08-loopd-compact-turn`. Stage with
`git add crates/loopd docs/implementer-log.md Cargo.lock`, then commit as `AGENTS.md` says.
## Done when
- `make gate` prints `gate: ok` with 1507 tests.
- `git diff HEAD~1 --stat` lists only `crates/loopd/src/compact.rs`,
`crates/loopd/src/channel.rs`, `crates/loopd/tests/compact_turn.rs` and the log.
## Stop and report if
- The branch check fails; a "before" block is not found exactly once; step 2 differs; `cargo
check` warns; a test fails; the gate count is not 1507; a file exceeds 500 lines; you add
`MemorySource` to `Runtime`; you compact inside `run_turn`; you call `finish_turn` after
`drop(held)`; you add an idle timer; you compact a `TurnError` other than `SessionFull`; you
retry `SessionFull` more than once.
## What the given tests pin
Usage 849/1000 leaves epoch 0 (one chat); 850/1000 compact before return (live slot then
background, no tools on the summary); `SessionFull` compact-and-retry (three chats, user logged
in both epochs, `turns() == 2`); `SessionFull` stays `SessionFull` when compact fails; a
successful due turn stays successful when `compact.md` is missing (epoch stays 0); another
infer error is not compacted.
docs/plans/M5d1/README.md
+119
View File
@@ -0,0 +1,119 @@
# M5d-1 implementation plan: epochs and compaction
> For the implementer: each task file is one closed task for one fresh session. Read `AGENTS.md`,
> then the task file, and do exactly that task. The driver is `tools/run-plan.sh`.
**Goal:** a live session can start a new epoch. `loopd` writes `N.jsonl` / `N.baseline.json` for
`N > 0`, turn numbers and `untrusted()` span every epoch, compaction runs after a turn whose last
`Usage` is at or above a configured fraction of `expect.n_ctx`, the summary call uses the
background slot, and `memoryd`'s Honcho walker reads every epoch file of a session.
**Architecture:** `loopd` gains a `[compact]` table and a `compact` module (when to compact, the
summary call, writing the next epoch). `Session::open` loads the highest complete epoch and the
carried summary. `messages` for epoch `n > 0` inserts that carry as a `ChatMessage::User` that is
never a `LogRecord::User`. Honcho concatenates epoch logs in order so `Item::Turn` numbers stay
monotonic. Heartbeat, cron, `ToolSummary`, and tiering are M5d-2.
**Tech stack:** Rust 1.98.1, edition 2024; no new crate.
**Spec:** `docs/specs/2026-10-03-m5d-scheduler.md`, sections 3, 5 (`[compact]` without using
`summarise`), 7 (the M5d-1 tests), and 8 (`session-compact-failed`).
## The owner's decisions (2026-10-03)
1. Compaction runs after a turn, when that turn's `cache_n + prompt_n` is at or above a configured
fraction of `expect.n_ctx` (default 0.85). A quiet session is not a reason to compact.
2. If a live request is `exceed_context_size_error`, compact and retry that user message once in
the new epoch.
3. The summary call is on the background slot, with no tools, thinking on. A context-full summary
falls back to the epoch's `User` texts, newest last, cut to `summary_chars`.
4. The carried summary is not a `LogRecord::User`, so it is not a Honcho turn.
## What the code already has, and what it lacks
- `LogRecord::EpochEnd { time, next, summary }` is in `bxcore` and unused. `messages` already
ignores it. This plan writes it.
- `Session::open` reads only `0.baseline.json` and `0.jsonl`. `turns()` and `untrusted()` look at
that file only. `next_call` is recovered from that file only.
- `run_turn` always sends `slots.main`. Live turns stay on the session's live slot (main, from
`SessionStart`). Only the summary call uses `slots.background`. Changing live turns to the
session's slot is M5d-2.
- `memoryd`'s indexer already walks every `<epoch>.jsonl`. Honcho's `log_path` is still
`sessions/<id>/0.jsonl`.
- `Runtime` has no `MemorySource`. Compaction asks `memoryd` for a fresh block from `channel.rs`
(and `compact::finish_turn`), so given `turn` tests that build a `Runtime` do not change.
- `[compact] summarise` is stored from this plan so a spec-complete `loopd.toml` loads; nothing
in M5d-1 reads the list. M5d-2 opts in `http_fetch`.
## Global constraints
- Roles depend on `bxcore` only. No `unsafe`, no async runtime, no new crate.
- No file the implementer reads holds a chat-template tag whole (T33). The carry heading is built
from pieces and named in words.
- Every path a given test creates is removed by a guard (T44); every process a test starts has a
deadline.
- The fail-closed state `session-compact-failed` has a runbook entry, added by the task that adds
its pointer (T49).
- Each task gives its baseline and its gate count with what the count includes (T50, T53); the
branch line is a check, never `git switch` (T51).
- `Cargo.lock` is in the `git add` line (T24). Gate timeout 600000 ms (T40).
- Commit trailer from the driver.
## Tasks
Every task below has been written to be replayed from its text. The gate count is the passed
total of `cargo test --workspace` plus one, for the ignored `end_to_end` test the gate runs on
its own. Before task 01 it is **1456** (talos, rustc 1.98.1, clippy of the same release; master
`069cd16`, docs-only commits after the M5c-2 review).
| # | Delivers | Crates | Gate after | Tests added |
|---|---|---|---|---|
| 01 | `[compact]` on `loopd.toml`: `threshold` 0.85, `summary_chars` 4000, `summarise` default `["http_fetch"]` (stored, unused); `Config::compact_prompt` is `compact.md` beside `system.md` | loopd | 1464 | 8 |
| 02 | `Session::open` loads the highest epoch that has both a baseline and a log, the carry from the previous `EpochEnd`, `turns` / `untrusted` / `next_call` over every epoch; persist writes the current epoch | loopd | 1473 | 9 |
| 03 | `messages_with_carry`: epoch `n > 0` is the new system message, then one `ChatMessage::User` (fixed heading plus carry, not a log record), then the new epoch's records; `run_turn` uses `session.messages()` | loopd | 1478 | 5 |
| 04 | Honcho concatenates every `<epoch>.jsonl` in epoch order; `turn_records` of turn 3 is the epoch-1 user; `send_turn` reads the concatenated log | memoryd | 1483 | 5 |
| 05 | `loopd::compact`: `due`, `fallback_summary`; starter `config/compact.md` | loopd | 1489 | 6 |
| 06 | Summary call on the background slot, no tools, thinking on; tag-neutralised and capped; context-full uses the fallback; `session-compact-failed` pointer and runbook entry | loopd | 1495 | 6 |
| 07 | Writing the next epoch: `EpochEnd`, fresh baseline from `memoryd`, `SessionStart` on the live slot; resume if `EpochEnd` is already the last record | loopd | 1501 | 6 |
| 08 | After a turn: compact when `due`; on `SessionFull`, compact and retry the user message once; compaction holds the session busy | loopd | 1507 | 6 |
After task 08, the design model runs the M5d-1 device check (spec 7) on straylight: a `bxctl`
session writes a distinctive fact into `memory.md`, continues until compaction with
`[compact] threshold` lowered, and the next turn's baseline contains the fact; `recall` of a
phrase from epoch 0 still hits; `TurnDone.untrusted` stays true across the epoch change after a
tainted recall.
## Known limits
These are known, accepted for M5d-1, and not bugs to fix in a task:
- **Live turns still use `slots.main`.** `SessionStart.slot` is recorded and copied onto the new
epoch; `run_turn` does not read it yet. Isolated sessions on the background slot are M5d-2.
- **`[compact] summarise` is unused.** Unknown tool names are not a config error until M5d-2.
- **No idle path exists.** Compaction is only called after a turn or on `SessionFull`. There is
no timer to test.
- **A crash after `EpochEnd` and before the next files exist** reopens epoch 0 with `EpochEnd` at
the end of the log. The next compact resumes without writing a second `EpochEnd` (task 07).
- **A `SessionFull` retry logs the user message twice**, once in the old epoch (Honcho skips it:
no reply) and once in the new epoch. Turn numbers stay monotonic.
## Before hand-over
Work through `docs/plans/CHECKLIST.md` with a reference in a scratch worktree with its own build
directory, replayed from its task text; mutations; `/tmp` compared around a gate run; scratch
files in the worktree's `.state/`; the reference moved to straylight as a bundle before the run
(T18). Task files name their log row (T47).
## Running it
The reference (`m5d1-ref`, tasks 01 to 08) is moved to straylight as a bundle and deleted here
before the run (T18). Then, from the repository root, with the OpenCode TUI closed:
```sh
git switch -c m5d1 master
OPENCODE_MODEL=straylight/ornith-1.5-35b-a3b RESUME=1 \
SLOTS_URL='http://straylight:11434/slots?model=ornith-1.5-35b-a3b' \
tools/run-plan.sh docs/plans/M5d1 01
```
After task 08, the gate count is 1507. Then the device check above.
docs/plans/M5d1/files/config/compact.md
+8
View File
@@ -0,0 +1,8 @@
Summarise this conversation for the next epoch of the same session.
Write a short account of:
- what the owner asked for and decided
- facts that were established
- open threads and anything still waiting
Do not use tools. Do not address the owner. Do not invent facts that are not in the messages.
docs/plans/M5d1/files/crates/loopd/tests/carry_messages.rs
+120
View File
@@ -0,0 +1,120 @@
//! Tests for the carried summary in the next epoch's prefix. Do not edit.
use bxcore::{Epoch, Hash32, LogRecord, SessionId, Timestamp};
use loopd::baseline::{Baseline, CARRY_HEADING, messages, messages_with_carry};
use loopd::llama::ChatMessage;
fn ts() -> Timestamp {
Timestamp::parse("2026-09-18T08:00:00.000Z").unwrap()
}
fn user(content: &str) -> LogRecord {
LogRecord::User {
time: ts(),
content: content.to_string(),
speaker: None,
}
}
fn start(epoch: u32) -> LogRecord {
LogRecord::SessionStart {
time: ts(),
session: SessionId::new("a").unwrap(),
epoch: Epoch(epoch),
slot: 0,
baseline: Hash32::ZERO,
}
}
#[test]
fn epoch_0_matches_messages_without_a_carry() {
let b = Baseline {
system: "sys".to_string(),
tools: vec![],
};
let records = vec![start(0), user("hi")];
assert_eq!(
messages_with_carry(&b, &records, None),
messages(&b, &records)
);
}
#[test]
fn a_carry_is_a_user_message_after_the_system_prompt() {
let b = Baseline {
system: "new-sys".to_string(),
tools: vec![],
};
let records = vec![start(1), user("next")];
let got = messages_with_carry(&b, &records, Some("the summary"));
assert_eq!(
got,
vec![
ChatMessage::System {
content: "new-sys".to_string(),
},
ChatMessage::User {
content: format!("{CARRY_HEADING}\n\nthe summary"),
},
ChatMessage::User {
content: "next".to_string(),
},
]
);
}
#[test]
fn the_carry_is_not_taken_from_a_log_user_record() {
let b = Baseline {
system: "sys".to_string(),
tools: vec![],
};
let records = vec![start(1)];
let got = messages_with_carry(&b, &records, Some("carried"));
assert_eq!(got.len(), 2);
assert!(
!records.iter().any(|r| matches!(r, LogRecord::User { .. })),
"the carry is not in the log"
);
}
#[test]
fn the_heading_is_built_from_pieces() {
assert_eq!(CARRY_HEADING, concat!("Earlier ", "turns, summarised:"));
assert!(
!CARRY_HEADING.contains('<'),
"the heading is words, not a tag"
);
}
#[test]
fn epoch_1_is_not_an_extension_of_epoch_0() {
let old = Baseline {
system: "old-sys".to_string(),
tools: vec![],
};
let new = Baseline {
system: "new-sys".to_string(),
tools: vec![],
};
let epoch0 = vec![start(0), user("first")];
let epoch1 = vec![start(1), user("second")];
let prev = messages(&old, &epoch0);
let next = messages_with_carry(&new, &epoch1, Some("sum of first"));
assert!(
next.len() >= 2,
"system plus carry, then the new epoch's user"
);
assert_ne!(next[0], prev[0], "the system message changed");
assert!(
!next.starts_with(&prev[..]),
"the new prefix is not an extension of the old messages"
);
match &next[1] {
ChatMessage::User { content } => {
assert!(content.contains("sum of first"), "{content}");
assert!(!content.contains("second"), "{content}");
}
other => panic!("{other:?}"),
}
}
docs/plans/M5d1/files/crates/loopd/tests/compact_advance.rs
+201
View File
@@ -0,0 +1,201 @@
//! Tests for writing the next epoch. Do not edit.
mod support;
use bxcore::{Epoch, LogRecord, SessionId};
use loopd::baseline::Baseline;
use loopd::compact;
use loopd::memory::{MemorySource, NoMemory};
use loopd::session::Session;
use loopd::tools::Registry;
use support::Home;
struct Block(&'static str);
impl MemorySource for Block {
fn block(&self) -> Result<String, String> {
Ok(self.0.to_string())
}
fn updated(&self, _session: &SessionId, _turn: u64) -> Result<(), String> {
Ok(())
}
}
fn id(s: &str) -> SessionId {
SessionId::new(s).unwrap()
}
fn session(home: &Home) -> (loopd::config::Config, Session) {
let cfg = home.config(std::path::Path::new("/tmp/unused.sock"));
let baseline = Baseline::assemble(&cfg, &Registry::m2b()).unwrap();
let s = Session::create(&home.dir, id("a"), baseline, cfg.slots.main).unwrap();
(cfg, s)
}
#[test]
fn commit_writes_epoch_end_a_new_baseline_and_a_new_log() {
let home = Home::new();
let (cfg, mut s) = session(&home);
s.append(LogRecord::User {
time: bxcore::Timestamp::parse("2026-09-18T08:00:00.000Z").unwrap(),
content: "hi".to_string(),
speaker: None,
})
.unwrap();
compact::commit(
&mut s,
&cfg,
&Registry::m2b(),
&NoMemory,
"the sum".to_string(),
)
.unwrap();
assert_eq!(s.epoch(), Epoch(1));
assert_eq!(s.carry(), Some("the sum"));
assert_eq!(s.slot(), cfg.slots.main);
assert_eq!(s.turns(), 1, "the old user still counts");
assert_eq!(
s.records()
.iter()
.filter(|r| matches!(r, LogRecord::User { .. }))
.count(),
0
);
let old = home.records("a");
assert!(
matches!(old.last(), Some(LogRecord::EpochEnd { next, summary, .. }) if *next == Epoch(1) && summary == "the sum")
);
assert!(home.dir.join("sessions/a/1.baseline.json").is_file());
let new_log = home.read("sessions/a/1.jsonl");
assert!(new_log.contains("\"epoch\":1"), "{new_log}");
}
#[test]
fn the_new_baseline_takes_a_fresh_memory_block() {
let home = Home::new();
let (cfg, mut s) = session(&home);
compact::commit(
&mut s,
&cfg,
&Registry::m2b(),
&Block("I take my tea strong."),
"sum".to_string(),
)
.unwrap();
assert!(
s.baseline().system.contains("I take my tea strong."),
"{}",
s.baseline().system
);
let on_disk = home.read("sessions/a/1.baseline.json");
assert!(on_disk.contains("I take my tea strong."), "{on_disk}");
}
#[test]
fn reopen_loads_the_new_epoch() {
let home = Home::new();
let (cfg, mut s) = session(&home);
compact::commit(
&mut s,
&cfg,
&Registry::m2b(),
&NoMemory,
"carry".to_string(),
)
.unwrap();
drop(s);
let s = Session::open(&home.dir, id("a")).unwrap();
assert_eq!(s.epoch(), Epoch(1));
assert_eq!(s.carry(), Some("carry"));
}
#[test]
fn a_second_commit_does_not_write_another_epoch_end_on_the_old_log() {
let home = Home::new();
let (cfg, mut s) = session(&home);
compact::commit(
&mut s,
&cfg,
&Registry::m2b(),
&NoMemory,
"first".to_string(),
)
.unwrap();
let ends = home
.records("a")
.iter()
.filter(|r| matches!(r, LogRecord::EpochEnd { .. }))
.count();
assert_eq!(ends, 1);
compact::commit(
&mut s,
&cfg,
&Registry::m2b(),
&NoMemory,
"second".to_string(),
)
.unwrap();
assert_eq!(s.epoch(), Epoch(2));
let ends = home
.records("a")
.iter()
.filter(|r| matches!(r, LogRecord::EpochEnd { .. }))
.count();
assert_eq!(ends, 1, "epoch 0 still has one EpochEnd");
let mid = home.read("sessions/a/1.jsonl");
assert!(mid.contains("\"type\":\"epoch_end\""), "{mid}");
}
#[test]
fn resume_after_epoch_end_without_next_files_does_not_write_a_second_end() {
let home = Home::new();
let (cfg, mut s) = session(&home);
s.append(LogRecord::EpochEnd {
time: bxcore::Timestamp::now(),
next: Epoch(1),
summary: "already".to_string(),
})
.unwrap();
drop(s);
let mut s = Session::open(&home.dir, id("a")).unwrap();
assert_eq!(s.epoch(), Epoch(0));
compact::commit(
&mut s,
&cfg,
&Registry::m2b(),
&NoMemory,
"ignored".to_string(),
)
.unwrap();
let ends = home
.records("a")
.iter()
.filter(|r| matches!(r, LogRecord::EpochEnd { .. }))
.count();
assert_eq!(ends, 1);
assert_eq!(s.epoch(), Epoch(1));
assert_eq!(
s.carry(),
Some("already"),
"the summary already in EpochEnd is the carry"
);
}
#[test]
fn the_new_session_start_keeps_the_live_slot() {
let home = Home::new();
let cfg = home.config(std::path::Path::new("/tmp/unused.sock"));
let baseline = Baseline::assemble(&cfg, &Registry::m2b()).unwrap();
let mut s = Session::create(&home.dir, id("a"), baseline, 7).unwrap();
compact::commit(&mut s, &cfg, &Registry::m2b(), &NoMemory, "s".to_string()).unwrap();
assert_eq!(s.slot(), 7);
match &s.records()[0] {
LogRecord::SessionStart { slot, epoch, .. } => {
assert_eq!(*slot, 7);
assert_eq!(*epoch, Epoch(1));
}
other => panic!("{other:?}"),
}
}
docs/plans/M5d1/files/crates/loopd/tests/compact_due.rs
+79
View File
@@ -0,0 +1,79 @@
//! Tests for when compaction is due and for the fallback summary. Do not edit.
use bxcore::{Epoch, Hash32, LogRecord, SessionId, Timestamp, Usage};
use loopd::compact::{due, fallback_summary};
fn usage(cache_n: u64, prompt_n: u64) -> Usage {
Usage {
cache_n,
prompt_n,
predicted_n: 0,
reasoning_tokens: 0,
thinking_capped: false,
}
}
fn user(content: &str) -> LogRecord {
LogRecord::User {
time: Timestamp::parse("2026-09-18T08:00:00.000Z").unwrap(),
content: content.to_string(),
speaker: None,
}
}
#[test]
fn due_at_the_threshold_and_not_below() {
assert!(!due(&usage(400, 449), 0.85, 1_000), "849 < 850");
assert!(due(&usage(400, 450), 0.85, 1_000), "850 == 0.85 * 1000");
assert!(due(&usage(0, 851), 0.85, 1_000));
assert!(!due(&usage(0, 0), 0.85, 1_000));
}
#[test]
fn due_uses_this_agents_n_ctx() {
// 0.85 * 262144 = 222822.4, so 222823 is the first integer at or above.
assert!(!due(&usage(0, 222_822), 0.85, 262_144));
assert!(due(&usage(0, 222_823), 0.85, 262_144));
assert!(due(&usage(200_000, 22_823), 0.85, 262_144));
}
#[test]
fn a_threshold_of_one_is_the_full_window() {
assert!(!due(&usage(99, 0), 1.0, 100));
assert!(due(&usage(100, 0), 1.0, 100));
}
#[test]
fn fallback_is_user_texts_newest_last() {
let records = vec![
LogRecord::SessionStart {
time: Timestamp::parse("2026-09-18T08:00:00.000Z").unwrap(),
session: SessionId::new("a").unwrap(),
epoch: Epoch(0),
slot: 0,
baseline: Hash32::ZERO,
},
user("first"),
user("second"),
user("third"),
];
assert_eq!(
fallback_summary(&records, 4_000),
"third\n\nsecond\n\nfirst"
);
}
#[test]
fn fallback_is_cut_to_summary_chars_on_a_character_boundary() {
let records = vec![user("café"), user("ab")];
assert_eq!(fallback_summary(&records, 3), "ab\n");
assert_eq!(fallback_summary(&records, 1), "a");
assert_eq!(fallback_summary(&records, 0), "");
let long = user("ååå");
assert_eq!(fallback_summary(&[long], 2), "åå");
}
#[test]
fn fallback_with_no_users_is_empty() {
assert_eq!(fallback_summary(&[], 100), "");
}
docs/plans/M5d1/files/crates/loopd/tests/compact_summary.rs
+147
View File
@@ -0,0 +1,147 @@
//! Tests for the compaction summary call. Do not edit.
mod support;
use bxcore::SessionId;
use loopd::baseline::Baseline;
use loopd::compact::{self, CompactError};
use loopd::llama::Client;
use loopd::session::Session;
use loopd::tools::Registry;
use support::{FakeServer, Home, Reply};
const CHAT: &str = "/v1/chat/completions";
const PROMPT: &str = "Summarise this conversation for the next epoch.\n";
fn sse(content: &str, cache_n: u64, prompt_n: u64) -> Reply {
let content_json = serde_json::to_string(content).unwrap();
let data1 = format!(
r#"data: {{"choices":[{{"finish_reason":null,"index":0,"delta":{{"content":{content_json}}}}}],"id":"sum","timings":{{"cache_n":{cache_n},"prompt_n":{prompt_n},"predicted_n":1}}}}"#
);
let data2 = format!(
r#"data: {{"choices":[{{"finish_reason":"stop","index":0,"delta":{{}}}}],"id":"sum","timings":{{"cache_n":{cache_n},"prompt_n":{prompt_n},"predicted_n":1}}}}"#
);
let body = format!("{data1}\n\n{data2}\n\ndata: [DONE]\n\n");
Reply::raw(format!(
"HTTP/1.1 200 OK\r\nContent-Type: text/event-stream\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}",
body.len()
))
}
fn session(home: &Home, cfg: &loopd::config::Config) -> Session {
let baseline = Baseline::assemble(cfg, &Registry::m2b()).unwrap();
Session::create(
&home.dir,
SessionId::new("a").unwrap(),
baseline,
cfg.slots.main,
)
.unwrap()
}
#[test]
fn the_summary_call_is_on_the_background_slot_with_no_tools_and_thinking() {
let home = Home::new();
home.write("compact.md", PROMPT);
let server = FakeServer::start();
let cfg = home.config(&server.socket);
server.route(CHAT, vec![sse("the sum", 0, 10)]);
let client = Client::new(cfg.clone());
let s = session(&home, &cfg);
let text = compact::summarise(&cfg, &client, &s).unwrap();
assert_eq!(text, "the sum");
let sent = server.requests_to(CHAT);
assert_eq!(sent.len(), 1);
let body = sent[0].json();
assert_eq!(body["id_slot"], 1);
assert!(body.get("tools").is_none(), "{body}");
assert_eq!(body["chat_template_kwargs"]["enable_thinking"], true);
let messages = body["messages"].as_array().unwrap();
let last = messages.last().unwrap();
assert_eq!(last["role"], "user");
assert_eq!(last["content"], PROMPT.trim_end());
}
#[test]
fn the_summary_is_neutralised_and_capped() {
let home = Home::new();
home.write("compact.md", PROMPT);
let server = FakeServer::start();
let mut cfg = home.config(&server.socket);
cfg.compact.summary_chars = 4;
server.route(CHAT, vec![sse("abcdefghij", 0, 10)]);
let client = Client::new(cfg.clone());
let s = session(&home, &cfg);
let text = compact::summarise(&cfg, &client, &s).unwrap();
assert_eq!(text, "abcd");
}
#[test]
fn a_context_full_summary_uses_the_fallback() {
let home = Home::new();
home.write("compact.md", PROMPT);
let server = FakeServer::start();
let cfg = home.config(&server.socket);
server.route(CHAT, vec![Reply::fixture("context_full")]);
let client = Client::new(cfg.clone());
let mut s = session(&home, &cfg);
s.append(bxcore::LogRecord::User {
time: bxcore::Timestamp::parse("2026-09-18T08:00:00.000Z").unwrap(),
content: "remember the tea".to_string(),
speaker: None,
})
.unwrap();
let text = compact::summarise(&cfg, &client, &s).unwrap();
assert_eq!(text, "remember the tea");
}
#[test]
fn tags_in_the_completion_are_neutralised() {
let home = Home::new();
home.write("compact.md", PROMPT);
let server = FakeServer::start();
let cfg = home.config(&server.socket);
let tag = concat!("<", "think", ">");
server.route(CHAT, vec![sse(&format!("hello {tag} world"), 0, 10)]);
let client = Client::new(cfg.clone());
let s = session(&home, &cfg);
let text = compact::summarise(&cfg, &client, &s).unwrap();
assert!(!text.contains(&tag), "{text}");
assert!(text.contains("hello"), "{text}");
}
#[test]
fn a_missing_prompt_file_names_the_runbook() {
let home = Home::new();
let server = FakeServer::start();
let cfg = home.config(&server.socket);
let client = Client::new(cfg.clone());
let s = session(&home, &cfg);
let err = compact::summarise(&cfg, &client, &s).unwrap_err();
let e: Box<dyn std::error::Error> = Box::new(err);
assert!(e.to_string().contains("compact.md"), "{e}");
assert!(
e.to_string()
.ends_with("see docs/runbook.md#session-compact-failed"),
"{e}"
);
}
#[test]
fn an_inference_error_that_is_not_context_full_is_a_compact_failure() {
let home = Home::new();
home.write("compact.md", PROMPT);
let server = FakeServer::start();
let cfg = home.config(&server.socket);
server.route(CHAT, vec![Reply::fixture("bad_request")]);
let client = Client::new(cfg.clone());
let s = session(&home, &cfg);
let err = compact::summarise(&cfg, &client, &s).unwrap_err();
assert!(matches!(err, CompactError::Failed { .. }), "{err:?}");
let e: Box<dyn std::error::Error> = Box::new(err);
assert!(
e.to_string()
.ends_with("see docs/runbook.md#session-compact-failed"),
"{e}"
);
}
docs/plans/M5d1/files/crates/loopd/tests/compact_turn.rs
+257
View File
@@ -0,0 +1,257 @@
//! Tests for compacting after a turn and retrying a full context. Do not edit.
mod support;
use bxcore::{Epoch, LogRecord, SessionId, TurnEvent};
use loopd::baseline::Baseline;
use loopd::compact;
use loopd::llama::Client;
use loopd::memory::NoMemory;
use loopd::session::Session;
use loopd::tools::Registry;
use loopd::turn::{Runtime, TurnError, run_turn};
use support::{FakeServer, Home, Reply};
const CHAT: &str = "/v1/chat/completions";
const PROMPT: &str = "Summarise this conversation for the next epoch.\n";
fn sse(content: &str, cache_n: u64, prompt_n: u64) -> Reply {
let content_json = serde_json::to_string(content).unwrap();
let data1 = format!(
r#"data: {{"choices":[{{"finish_reason":null,"index":0,"delta":{{"content":{content_json}}}}}],"id":"t","timings":{{"cache_n":{cache_n},"prompt_n":{prompt_n},"predicted_n":1}}}}"#
);
let data2 = format!(
r#"data: {{"choices":[{{"finish_reason":"stop","index":0,"delta":{{}}}}],"id":"t","timings":{{"cache_n":{cache_n},"prompt_n":{prompt_n},"predicted_n":1}}}}"#
);
let body = format!("{data1}\n\n{data2}\n\ndata: [DONE]\n\n");
Reply::raw(format!(
"HTTP/1.1 200 OK\r\nContent-Type: text/event-stream\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}",
body.len()
))
}
struct World {
home: Home,
server: FakeServer,
cfg: loopd::config::Config,
client: Client,
port: support::ScriptedPort,
registry: Registry,
}
impl World {
fn new() -> World {
let home = Home::new();
home.write("compact.md", PROMPT);
let server = FakeServer::start();
let mut cfg = home.config(&server.socket);
cfg.expect.n_ctx = 1_000;
cfg.compact.threshold = 0.85;
let client = Client::new(cfg.clone());
World {
home,
server,
cfg,
client,
port: support::ScriptedPort::new(vec![]),
registry: Registry::m2b(),
}
}
fn session(&self) -> Session {
let baseline = Baseline::assemble(&self.cfg, &self.registry).unwrap();
Session::create(
&self.home.dir,
SessionId::new("a").unwrap(),
baseline,
self.cfg.slots.main,
)
.unwrap()
}
fn runtime(&self) -> Runtime<'_> {
Runtime {
cfg: &self.cfg,
client: &self.client,
port: &self.port,
registry: &self.registry,
}
}
fn finish(
&self,
session: &mut Session,
text: &str,
outcome: Result<loopd::turn::TurnOutcome, TurnError>,
) -> Result<loopd::turn::TurnOutcome, TurnError> {
let mut events = Vec::new();
compact::finish_turn(
session,
&self.runtime(),
&NoMemory,
text,
None,
&mut |e: &TurnEvent| events.push(e.clone()),
outcome,
)
}
}
#[test]
fn usage_under_the_threshold_leaves_the_epoch() {
let w = World::new();
w.server.route(CHAT, vec![sse("ok", 400, 449)]);
let mut s = w.session();
let mut events = Vec::new();
let outcome = run_turn(&mut s, &w.runtime(), "hi", None, &mut |e| {
events.push(e.clone())
});
let outcome = w.finish(&mut s, "hi", outcome).unwrap();
assert_eq!(outcome.content, "ok");
assert_eq!(s.epoch(), Epoch(0));
assert_eq!(w.server.requests_to(CHAT).len(), 1);
}
#[test]
fn usage_at_the_threshold_compacts_before_the_turn_returns() {
let w = World::new();
w.server
.route(CHAT, vec![sse("ok", 400, 450), sse("the sum", 0, 10)]);
let mut s = w.session();
let mut events = Vec::new();
let outcome = run_turn(&mut s, &w.runtime(), "hi", None, &mut |e| {
events.push(e.clone())
});
w.finish(&mut s, "hi", outcome).unwrap();
assert_eq!(s.epoch(), Epoch(1));
assert_eq!(s.carry(), Some("the sum"));
let sent = w.server.requests_to(CHAT);
assert_eq!(sent.len(), 2);
assert_eq!(sent[0].json()["id_slot"], 0);
assert_eq!(sent[1].json()["id_slot"], 1);
assert!(sent[1].json().get("tools").is_none());
let old = w.home.records("a");
assert!(matches!(
old.last(),
Some(LogRecord::EpochEnd { next, .. }) if *next == Epoch(1)
));
}
#[test]
fn session_full_compacts_and_retries_the_user_message_once() {
let w = World::new();
w.server.route(
CHAT,
vec![
Reply::fixture("context_full"),
sse("the sum", 0, 10),
sse("retried", 0, 20),
],
);
let mut s = w.session();
let mut events = Vec::new();
let outcome = run_turn(&mut s, &w.runtime(), "please fit", None, &mut |e| {
events.push(e.clone())
});
assert!(matches!(outcome, Err(TurnError::SessionFull)));
let done = w.finish(&mut s, "please fit", outcome).unwrap();
assert_eq!(done.content, "retried");
assert_eq!(s.epoch(), Epoch(1));
let users: Vec<_> = s
.records()
.iter()
.filter_map(|r| match r {
LogRecord::User { content, .. } => Some(content.as_str()),
_ => None,
})
.collect();
assert_eq!(users, ["please fit"]);
let old_users = w
.home
.records("a")
.iter()
.filter(|r| matches!(r, LogRecord::User { .. }))
.count();
assert_eq!(old_users, 1);
assert_eq!(s.turns(), 2);
let sent = w.server.requests_to(CHAT);
assert_eq!(sent.len(), 3);
assert_eq!(sent[1].json()["id_slot"], 1);
assert_eq!(sent[2].json()["id_slot"], 0);
}
#[test]
fn session_full_stays_session_full_when_compact_fails() {
let w = World::new();
w.server.route(
CHAT,
vec![
Reply::fixture("context_full"),
Reply::fixture("bad_request"),
],
);
let mut s = w.session();
let mut events = Vec::new();
let outcome = run_turn(&mut s, &w.runtime(), "x", None, &mut |e| {
events.push(e.clone())
});
let err = w.finish(&mut s, "x", outcome).unwrap_err();
assert!(matches!(err, TurnError::SessionFull), "{err:?}");
assert_eq!(s.epoch(), Epoch(0));
}
#[test]
fn a_successful_turn_stays_successful_when_compact_fails() {
let home = Home::new();
let server = FakeServer::start();
let mut cfg = home.config(&server.socket);
cfg.expect.n_ctx = 1_000;
cfg.compact.threshold = 0.85;
let client = Client::new(cfg.clone());
server.route(CHAT, vec![sse("ok", 400, 450)]);
let baseline = Baseline::assemble(&cfg, &Registry::m2b()).unwrap();
let mut s = Session::create(
&home.dir,
SessionId::new("a").unwrap(),
baseline,
cfg.slots.main,
)
.unwrap();
let port = support::ScriptedPort::new(vec![]);
let registry = Registry::m2b();
let rt = Runtime {
cfg: &cfg,
client: &client,
port: &port,
registry: &registry,
};
let mut events = Vec::new();
let outcome = run_turn(&mut s, &rt, "hi", None, &mut |e| events.push(e.clone()));
let done = compact::finish_turn(
&mut s,
&rt,
&NoMemory,
"hi",
None,
&mut |e| events.push(e.clone()),
outcome,
)
.unwrap();
assert_eq!(done.content, "ok");
assert_eq!(s.epoch(), Epoch(0), "compact.md is missing, so no epoch");
}
#[test]
fn finish_turn_does_not_compact_an_error_other_than_session_full() {
let w = World::new();
w.server.route(CHAT, vec![Reply::fixture("bad_request")]);
let mut s = w.session();
let mut events = Vec::new();
let outcome = run_turn(&mut s, &w.runtime(), "x", None, &mut |e| {
events.push(e.clone())
});
let err = w.finish(&mut s, "x", outcome).unwrap_err();
assert!(matches!(err, TurnError::Infer(_)), "{err:?}");
assert_eq!(s.epoch(), Epoch(0));
assert_eq!(w.server.requests_to(CHAT).len(), 1);
}
docs/plans/M5d1/files/crates/loopd/tests/config_compact.rs
+119
View File
@@ -0,0 +1,119 @@
//! Tests for `[compact]` on `loopd.toml`. Do not edit.
use loopd::config::Config;
use std::path::{Path, PathBuf};
fn fixture(name: &str) -> PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR"))
.join("tests/fixtures/config")
.join(name)
}
fn minimal() -> String {
std::fs::read_to_string(fixture("minimal.toml")).unwrap()
}
#[test]
fn missing_table_gets_the_documented_defaults() {
let c = Config::load(&fixture("minimal.toml")).unwrap();
assert_eq!(c.compact.threshold, 0.85);
assert_eq!(c.compact.summary_chars, 4_000);
assert_eq!(c.compact.summarise, vec!["http_fetch".to_string()]);
}
#[test]
fn the_table_overrides_every_default() {
let text = format!(
"{}\n[compact]\nthreshold = 0.5\nsummary_chars = 100\nsummarise = [\"clock\"]\n",
minimal()
);
let c = Config::parse(&text).unwrap();
assert_eq!(c.compact.threshold, 0.5);
assert_eq!(c.compact.summary_chars, 100);
assert_eq!(c.compact.summarise, vec!["clock".to_string()]);
}
#[test]
fn a_partial_table_keeps_the_other_defaults() {
let text = format!("{}\n[compact]\nthreshold = 0.9\n", minimal());
let c = Config::parse(&text).unwrap();
assert_eq!(c.compact.threshold, 0.9);
assert_eq!(c.compact.summary_chars, 4_000);
assert_eq!(c.compact.summarise, vec!["http_fetch".to_string()]);
}
#[test]
fn unknown_keys_are_errors() {
let text = minimal();
let bad = format!("{text}\n[compact]\nzz_unknown = 1\n");
assert!(
Config::parse(&bad).is_err(),
"[compact] accepted an unknown key"
);
let misspelt = format!("{text}\n[compact]\nsummary_char = 10\n");
assert!(
Config::parse(&misspelt).is_err(),
"[compact] accepted a misspelt key"
);
}
#[test]
fn empty_summarise_is_allowed() {
let text = format!("{}\n[compact]\nsummarise = []\n", minimal());
let c = Config::parse(&text).unwrap();
assert!(c.compact.summarise.is_empty());
}
#[test]
fn compact_prompt_is_compact_md_beside_system_md() {
let c = Config::load(&fixture("minimal.toml")).unwrap();
assert_eq!(c.compact_prompt(), fixture("compact.md"));
let home = Scratch::new();
let cfg_path = home.dir.join("loopd.toml");
std::fs::write(&cfg_path, minimal()).unwrap();
std::fs::write(home.dir.join("system.md"), "sys\n").unwrap();
let c = Config::load(&cfg_path).unwrap();
assert_eq!(c.compact_prompt(), home.dir.join("compact.md"));
}
struct Scratch {
dir: PathBuf,
}
impl Scratch {
fn new() -> Scratch {
let dir = std::env::temp_dir().join(format!(
"loopd-compact-cfg-{}-{}",
std::process::id(),
std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_nanos()
));
std::fs::create_dir_all(&dir).unwrap();
Scratch { dir }
}
}
impl Drop for Scratch {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.dir);
}
}
#[test]
fn compact_prompt_follows_an_overridden_system_path() {
let text = std::fs::read_to_string(fixture("m2b.toml")).unwrap();
let c = Config::load(&fixture("m2b.toml")).unwrap();
assert!(
text.contains("prompts/agent.md"),
"m2b fixture names a prompt in a subdirectory"
);
assert_eq!(c.compact_prompt(), fixture("prompts/compact.md"));
}
#[test]
fn defaults_match_the_struct_default() {
let c = Config::load(&fixture("minimal.toml")).unwrap();
assert_eq!(c.compact, loopd::config::Compact::default());
}
docs/plans/M5d1/files/crates/loopd/tests/session_epochs.rs
+255
View File
@@ -0,0 +1,255 @@
//! Tests for multi-epoch session open, turns, trust and call ids. Do not edit.
mod support;
use bxcore::{CallId, DataClass, Epoch, LogRecord, SessionId, Timestamp};
use loopd::baseline::Baseline;
use loopd::session::{Session, SessionError};
use support::Home;
fn baseline() -> Baseline {
Baseline {
system: "sys".to_string(),
tools: vec![],
}
}
fn later() -> Baseline {
Baseline {
system: "sys-epoch-1".to_string(),
tools: vec![],
}
}
fn id(s: &str) -> SessionId {
SessionId::new(s).unwrap()
}
fn ts() -> Timestamp {
Timestamp::parse("2026-09-18T08:00:00.000Z").unwrap()
}
fn user(content: &str) -> LogRecord {
LogRecord::User {
time: ts(),
content: content.to_string(),
speaker: None,
}
}
fn write_jsonl(home: &Home, session: &str, epoch: u32, records: &[LogRecord]) {
let text: String = records
.iter()
.map(|r| format!("{}\n", serde_json::to_string(r).unwrap()))
.collect();
home.write(&format!("sessions/{session}/{epoch}.jsonl"), &text);
}
fn write_baseline(home: &Home, session: &str, epoch: u32, b: &Baseline) {
home.write(
&format!("sessions/{session}/{epoch}.baseline.json"),
&b.to_json().unwrap(),
);
}
fn start(session: &str, epoch: u32, slot: u32, b: &Baseline) -> LogRecord {
LogRecord::SessionStart {
time: ts(),
session: id(session),
epoch: Epoch(epoch),
slot,
baseline: b.hash().unwrap(),
}
}
fn epoch_end(next: u32, summary: &str) -> LogRecord {
LogRecord::EpochEnd {
time: ts(),
next: Epoch(next),
summary: summary.to_string(),
}
}
fn tool_result(call: u64, untrusted: bool) -> LogRecord {
LogRecord::ToolResult {
time: ts(),
call: CallId(call),
tool_call_id: format!("c{call}"),
content: "out".to_string(),
class: DataClass::Public,
untrusted,
truncated: false,
}
}
#[test]
fn open_loads_the_highest_complete_epoch_and_its_carry() {
let home = Home::new();
let mut s = Session::create(&home.dir, id("a"), baseline(), 3).unwrap();
s.append(user("one")).unwrap();
s.append(user("two")).unwrap();
s.append(epoch_end(1, "carry-text")).unwrap();
drop(s);
write_baseline(&home, "a", 1, &later());
write_jsonl(&home, "a", 1, &[start("a", 1, 3, &later()), user("three")]);
let s = Session::open(&home.dir, id("a")).unwrap();
assert_eq!(s.epoch(), Epoch(1));
assert_eq!(s.slot(), 3);
assert_eq!(s.carry(), Some("carry-text"));
assert_eq!(s.baseline(), &later());
assert_eq!(s.records().len(), 2);
assert!(matches!(s.records()[1], LogRecord::User { ref content, .. } if content == "three"));
}
#[test]
fn a_missing_carry_on_epoch_1_is_an_error() {
let home = Home::new();
let s = Session::create(&home.dir, id("a"), baseline(), 0).unwrap();
drop(s);
write_baseline(&home, "a", 1, &later());
write_jsonl(&home, "a", 1, &[start("a", 1, 0, &later())]);
match Session::open(&home.dir, id("a")) {
Err(SessionError::NoCarry { ref path, .. }) => {
assert!(path.ends_with("0.jsonl"), "{path:?}");
}
other => panic!("{other:?}"),
}
let e: Box<dyn std::error::Error> =
Box::new(Session::open(&home.dir, id("a")).map(|_| ()).unwrap_err());
assert!(e.to_string().contains("0.jsonl"), "{e}");
assert!(
e.to_string()
.contains("see docs/runbook.md#session-log-damaged"),
"{e}"
);
}
#[test]
fn epoch_0_has_no_carry() {
let home = Home::new();
let s = Session::create(&home.dir, id("a"), baseline(), 7).unwrap();
drop(s);
let s = Session::open(&home.dir, id("a")).unwrap();
assert_eq!(s.epoch(), Epoch(0));
assert_eq!(s.carry(), None);
assert_eq!(s.slot(), 7);
}
#[test]
fn turns_count_users_across_every_epoch() {
let home = Home::new();
let mut s = Session::create(&home.dir, id("a"), baseline(), 0).unwrap();
s.append(user("one")).unwrap();
s.append(user("two")).unwrap();
s.append(epoch_end(1, "sum")).unwrap();
drop(s);
write_baseline(&home, "a", 1, &later());
write_jsonl(&home, "a", 1, &[start("a", 1, 0, &later()), user("three")]);
let s = Session::open(&home.dir, id("a")).unwrap();
assert_eq!(s.turns(), 3);
assert_eq!(
s.records()
.iter()
.filter(|r| matches!(r, LogRecord::User { .. }))
.count(),
1,
"records() is the current epoch only"
);
}
#[test]
fn untrusted_reads_every_epoch() {
let home = Home::new();
let mut s = Session::create(&home.dir, id("a"), baseline(), 0).unwrap();
s.append(tool_result(1, true)).unwrap();
s.append(epoch_end(1, "sum")).unwrap();
drop(s);
write_baseline(&home, "a", 1, &later());
write_jsonl(
&home,
"a",
1,
&[start("a", 1, 0, &later()), tool_result(2, false)],
);
let s = Session::open(&home.dir, id("a")).unwrap();
assert!(s.untrusted(), "an untrusted result in epoch 0 still taints");
assert!(
!s.records().iter().any(|r| matches!(
r,
LogRecord::ToolResult {
untrusted: true,
..
}
)),
"the current epoch's records look clean"
);
}
#[test]
fn call_ids_continue_across_epochs() {
let home = Home::new();
let mut s = Session::create(&home.dir, id("a"), baseline(), 0).unwrap();
s.append(tool_result(4, false)).unwrap();
s.append(epoch_end(1, "sum")).unwrap();
drop(s);
write_baseline(&home, "a", 1, &later());
write_jsonl(
&home,
"a",
1,
&[start("a", 1, 0, &later()), tool_result(2, false)],
);
let mut s = Session::open(&home.dir, id("a")).unwrap();
assert_eq!(
s.next_call(),
CallId(5),
"one more than the highest call in any epoch"
);
}
#[test]
fn append_writes_the_current_epoch_file() {
let home = Home::new();
let mut s = Session::create(&home.dir, id("a"), baseline(), 0).unwrap();
s.append(epoch_end(1, "sum")).unwrap();
drop(s);
write_baseline(&home, "a", 1, &later());
write_jsonl(&home, "a", 1, &[start("a", 1, 0, &later())]);
let mut s = Session::open(&home.dir, id("a")).unwrap();
s.append(user("after")).unwrap();
let text = home.read("sessions/a/1.jsonl");
assert!(text.contains("after"), "{text}");
let old = home.read("sessions/a/0.jsonl");
assert!(!old.contains("after"), "{old}");
}
#[test]
fn an_incomplete_next_epoch_is_ignored() {
let home = Home::new();
let mut s = Session::create(&home.dir, id("a"), baseline(), 0).unwrap();
s.append(user("still-here")).unwrap();
drop(s);
write_baseline(&home, "a", 1, &later());
let s = Session::open(&home.dir, id("a")).unwrap();
assert_eq!(s.epoch(), Epoch(0));
assert_eq!(s.carry(), None);
assert_eq!(s.turns(), 1);
}
#[test]
fn epoch_end_without_the_next_files_stays_on_the_old_epoch() {
let home = Home::new();
let mut s = Session::create(&home.dir, id("a"), baseline(), 0).unwrap();
s.append(user("x")).unwrap();
s.append(epoch_end(1, "sum")).unwrap();
drop(s);
let s = Session::open(&home.dir, id("a")).unwrap();
assert_eq!(s.epoch(), Epoch(0));
assert_eq!(s.carry(), None);
assert_eq!(s.turns(), 1);
assert!(
matches!(s.records().last(), Some(LogRecord::EpochEnd { next, .. }) if *next == Epoch(1))
);
}
docs/plans/M5d1/files/crates/memoryd/tests/honcho_epochs.rs
+181
View File
@@ -0,0 +1,181 @@
//! Honcho reads every epoch file of a session, in order. Do not edit.
#[path = "support/logs.rs"]
mod logs;
#[path = "support/tmp.rs"]
mod tmp;
#[path = "support/turns.rs"]
mod turns;
use std::cell::RefCell;
use bxcore::{Epoch, LogRecord, SessionId, Timestamp};
use logs::{assistant, write_log};
use memoryd::honcho::decide::{turn_records, turns};
use memoryd::honcho::ledger::{Action, Item, Ledger, Record, Workspace};
use memoryd::honcho::turns::{LOG_RUNBOOK, TurnSync, read_session};
use tmp::TempDir;
use turns::{clean, sid, spoken};
const LEDGER: &str = "honcho.jsonl";
fn epoch_end(summary: &str) -> LogRecord {
LogRecord::EpochEnd {
time: Timestamp::parse("2026-09-28T10:00:00.000Z").unwrap(),
next: Epoch(1),
summary: summary.to_string(),
}
}
fn write_epoch(home: &std::path::Path, session: &str, epoch: u32, records: &[LogRecord]) {
let dir = home.join("sessions").join(session);
std::fs::create_dir_all(&dir).unwrap();
let path = dir.join(format!("{epoch}.jsonl"));
let text: String = records
.iter()
.map(|r| format!("{}\n", serde_json::to_string(r).unwrap()))
.collect();
std::fs::write(path, text).unwrap();
}
struct World {
sync: TurnSync,
answer: RefCell<Result<bxcore::SessionStateReply, String>>,
dir: TempDir,
}
impl World {
fn new(tag: &str) -> World {
let dir = TempDir::new(tag);
std::fs::create_dir_all(dir.path().join("memory")).unwrap();
let mut ledger = Ledger::open(&dir.path().join("memory").join(LEDGER)).unwrap();
ledger
.append(Record::Began {
time: Timestamp::parse("2026-09-28T09:00:00.000Z").unwrap(),
})
.unwrap();
World {
sync: TurnSync::new(ledger, "boxmaker0", &dir.path().join("sessions")),
answer: RefCell::new(Ok(clean())),
dir,
}
}
fn updated(&mut self, session: &str, turn: u64) {
let ask = |s: &SessionId| {
let _ = s;
self.answer.borrow().clone()
};
let log = |_line: &str| {};
let now = Timestamp::parse("2026-09-28T10:00:00.000Z").unwrap();
self.sync.updated(&sid(session), turn, &ask, now, &log);
}
fn action(&self, session: &str, turn: u64) -> Option<Action> {
let item = Item::Turn {
session: sid(session),
turn,
};
let path = self.dir.path().join("memory").join(LEDGER);
let text = std::fs::read_to_string(path).unwrap_or_default();
let key = serde_json::to_string(&item).unwrap();
let line = text.lines().find(|l| l.contains(&key))?;
let value: serde_json::Value = serde_json::from_str(line).unwrap();
Some(serde_json::from_value(value["action"].clone()).unwrap())
}
}
fn to(workspace: Workspace, peer: &str) -> Action {
Action::To {
workspace,
peer: peer.to_string(),
}
}
#[test]
fn concatenated_logs_count_three_turns() {
let dir = TempDir::new("honcho-epochs-count");
let home = dir.path();
let epoch0 = vec![
spoken("one", Some("kyle")),
assistant(2, Some("ok"), &[]),
spoken("two", Some("kyle")),
assistant(3, Some("ok"), &[]),
epoch_end("sum"),
];
write_log(home, "mm-a", &epoch0);
write_epoch(
home,
"mm-a",
1,
&[
spoken("three", Some("kyle")),
assistant(4, Some("ok"), &[]),
],
);
let records = read_session(&home.join("sessions"), &sid("mm-a")).unwrap();
assert_eq!(turns(&records), 3);
let turn3 = turn_records(&records, 3).unwrap();
assert!(
matches!(turn3[0], LogRecord::User { ref content, .. } if content == "three"),
"{turn3:?}"
);
}
#[test]
fn epoch_0_alone_still_reads() {
let dir = TempDir::new("honcho-epochs-zero");
let home = dir.path();
write_log(home, "mm-a", &[spoken("only", Some("kyle"))]);
let records = read_session(&home.join("sessions"), &sid("mm-a")).unwrap();
assert_eq!(turns(&records), 1);
}
#[test]
fn a_damaged_later_epoch_is_an_error_naming_that_file() {
let dir = TempDir::new("honcho-epochs-damaged");
let home = dir.path();
write_log(home, "mm-a", &[spoken("one", Some("kyle"))]);
let path = home.join("sessions/mm-a/1.jsonl");
std::fs::write(&path, "{\"type\":\"zz\"}\n").unwrap();
let err = read_session(&home.join("sessions"), &sid("mm-a")).unwrap_err();
assert!(err.to_string().contains("1.jsonl"), "{err}");
assert!(
err.to_string().contains(LOG_RUNBOOK) || err.to_string().contains("session-log-damaged"),
"{err}"
);
}
#[test]
fn turn_sync_decides_a_turn_that_lives_in_epoch_1() {
let mut w = World::new("honcho-epochs-sync");
write_log(
w.dir.path(),
"mm-a",
&[
spoken("one", Some("kyle")),
assistant(2, Some("ok"), &[]),
spoken("two", Some("kyle")),
assistant(3, Some("ok"), &[]),
epoch_end("sum"),
],
);
write_epoch(
w.dir.path(),
"mm-a",
1,
&[
spoken("three", Some("kyle")),
assistant(4, Some("ok"), &[]),
],
);
w.updated("mm-a", 3);
assert_eq!(w.action("mm-a", 3), Some(to(Workspace::Trusted, "kyle")));
}
#[test]
fn a_missing_session_directory_is_empty() {
let dir = TempDir::new("honcho-epochs-missing");
let records = read_session(&dir.path().join("sessions"), &sid("mm-none")).unwrap();
assert!(records.is_empty());
}
docs/plans/M5d1/files/runbook-session-compact-failed.md
+35
View File
@@ -0,0 +1,35 @@
## session-compact-failed
**What you see.** `loopd` logs that it could not compact a session, then this entry. The session
stays on its current epoch. A turn that was already too large to fit (`session_full`) stays
`session_full`. A turn that finished stays finished; the next turn on that session may be
`session_full` if it would not fit.
```text
loopd: could not compact session chat-0-0: compact.md: No such file or directory (os error 2); see docs/runbook.md#session-compact-failed
```
**Why.** Compaction needs a summary of the current epoch (a completion on the background slot
with `compact.md` as the last user message, or the fallback of that epoch's owner texts). If the
prompt file cannot be read, the summary call fails for a reason other than a full context, or
writing the next epoch's files fails, `loopd` refuses to guess a prefix. It does not start a new
epoch from an empty summary when the model call failed.
**Confirm.**
```sh
ls -l "$(dirname "$BOXMAKER_HOME/../config")/compact.md" 2>/dev/null
ls -l "$BOXMAKER_HOME/sessions/<id>"/
tail -n 5 "$BOXMAKER_HOME/sessions/<id>/0.jsonl"
```
`compact.md` lives next to `system.md` (the path in `loopd.toml`'s `[baseline] system`). A last
line of type `epoch_end` without `<n>.baseline.json` and `<n>.jsonl` is a compact that stopped
halfway; the next compact resumes it.
**Fix.** Restore `compact.md` beside `system.md` (the starter is `config/compact.md` in the
repository). Check that `inferproxy` still answers. If the session directory is mid-write, do not
delete `EpochEnd`; the next compact continues from it.
**Check.** The next turn on that session either runs on the current epoch, or compact succeeds
and the next turn's log is `<n>.jsonl` with `epoch` n.
docs/plans/M5d2/01-loopd-session-slot.md
+124
View File
@@ -0,0 +1,124 @@
# M5d-2 task 01: `run_turn` sends `session.slot()`
**Branch:** `m5d2`. Check it: `git branch --show-current` prints `m5d2` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `loopd: live turns use the session's slot`
**Log row:** the Task cell is `M5d2/01-loopd-session-slot`.
## Goal
Heartbeat and cron run on the background slot (spec 4). Live Mattermost and `bxctl chat`
sessions keep `cfg.slots.main`. Today `run_turn` always sends `slots.main`. This task makes it
send `session.slot()`: the slot stored on that session's `SessionStart` (M5d-1 task 02).
Nothing else changes. Compaction's summary call already uses `slots.background`. The scheduler
does not exist yet.
## Every place that changes
| Place | What happens there |
|---|---|
| `crates/loopd/src/turn.rs` | `ChatRequest.slot` is `session.slot()` (edit T1) |
| `crates/loopd/tests/session_slot.rs` | new, given |
| everything else | **no change**: not `session.rs`, not `compact.rs`, not channel |
## Files
- Copy, from `docs/plans/M5d2/files/` (the same path under it): the new test
`crates/loopd/tests/session_slot.rs`.
- Modify: `crates/loopd/src/turn.rs`, `docs/implementer-log.md`.
- Do not touch: any other file.
## Facts you need
- **`Session::slot()`** returns the `u32` stored on create / on the current epoch's
`SessionStart`. It is already `pub` (M5d-1 task 02).
- **`messages_with_carry`** stays as it is. Only the `slot:` line changes.
- **Existing turn tests** create sessions with `cfg.slots.main` (0). They still send 0.
- The items the edits use. This table is complete.
| Item | What it does |
|---|---|
| `Session::slot` | the live slot for this session |
| `ChatRequest.slot` | becomes `id_slot` on the wire |
## The edits, exactly
Each quoted "before" block occurs exactly once after M5d-1 task 08.
### `crates/loopd/src/turn.rs`
**T1.** The lines
```rust
let request = ChatRequest {
slot: rt.cfg.slots.main,
messages: messages_with_carry(baseline, session.records(), session.carry()),
tools: baseline.tools.clone(),
thinking: true,
};
```
become
```rust
let request = ChatRequest {
slot: session.slot(),
messages: messages_with_carry(baseline, session.records(), session.carry()),
tools: baseline.tools.clone(),
thinking: true,
};
```
## Steps
- [ ] **1. Copy the given test.** From the repository root, one command:
```sh
cp docs/plans/M5d2/files/crates/loopd/tests/session_slot.rs crates/loopd/tests/
```
Then `git status --short` lists one new file, `crates/loopd/tests/session_slot.rs`.
- [ ] **2. See it fail.** `cargo test -p loopd --test session_slot`, with a shell timeout of
600000 ms: the four tests compile (they call `session.slot()`, which exists) and
`a_session_created_on_the_background_slot_sends_that_slot` fails: `id_slot` is `0`, not `1`.
`a_main_slot_session_still_sends_main` may pass. `the_request_slot_follows_session_start_not_the_config_main`
fails: `id_slot` is `0`, not `7`.
- [ ] **3. The edit.** Make edit T1. `cargo check -p loopd` prints no error and no warning.
- [ ] **4. The tests.** `cargo test -p loopd --test session_slot --test turn --test channel
--test compact_turn`: every test passes, 4 in `session_slot` and the others as before.
- [ ] **5. Check.** `git diff --stat` lists exactly `crates/loopd/src/turn.rs`.
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, with a shell timeout of 600000 ms.
Expected last line: `gate: ok`. The test count is **1511** after this task, 1507 before it,
and the 4 new ones are the tests of `session_slot.rs`. To count, add up the `N passed` of
every `test result:` line the gate's `cargo test --workspace` printed, and add 1. Never run
the gate again only to count.
- [ ] **7. Log and commit.** Add your row to `docs/implementer-log.md`; its Task cell is
`M5d2/01-loopd-session-slot`. Stage with
`git add crates/loopd docs/implementer-log.md Cargo.lock`, then commit as `AGENTS.md` says.
`Cargo.lock` does not change in this task; the `git add` is harmless.
## Done when
- `make gate` prints `gate: ok` with 1511 tests.
- `git diff HEAD~1 --stat` lists only `crates/loopd/src/turn.rs`,
`crates/loopd/tests/session_slot.rs` and `docs/implementer-log.md`.
## Stop and report if
- The branch check at the top fails.
- A "before" block is not in its file exactly as shown, or is there more than once.
- Step 2 shows a different result: the tests do not compile, or a different assertion fails.
- `cargo check` in step 3 prints an error or a warning. Quote it, and stop.
- A test fails in step 4. Do not change the test: stop, and give the failing tests' names and
their messages.
- The fix seems to need a change anywhere else. It does not; stop and say why you think so.
- A test outside `session_slot.rs` fails in the gate: check once whether it fails without your
change (`git stash -u`, run it, `git stash pop`), then stop and report both results.
- The gate passes with a count other than 1511.
## What the given tests pin
A session created with `cfg.slots.background` sends that slot; one created with `cfg.slots.main`
still sends main; a session created with slot `7` sends `7` even when `cfg.slots.main` is `0`; a
heartbeat turn between two main turns does not change the main session's message prefix.
docs/plans/M5d2/02-bxcore-tool-summary.md
+165
View File
@@ -0,0 +1,165 @@
# M5d-2 task 02: `LogRecord::ToolSummary`
**Branch:** `m5d2`. Check it: `git branch --show-current` prints `m5d2` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `bxcore: ToolSummary log record; messages ignore it`
**Log row:** the Task cell is `M5d2/02-bxcore-tool-summary`.
## Goal
A tool that opts in gets a second record, for the index only (spec 4.3):
```text
LogRecord::ToolSummary { time, call, tool_call_id, content, class, untrusted }
```
JSON type `tool_summary`. `messages()` / `messages_with_carry` ignore it, so the epoch's prefix
does not change. `support/turn.rs` `types` names it (T19: this task ships its own copy).
Nothing summarises yet. That is task 08.
## Every place that changes
| Place | What happens there |
|---|---|
| `crates/bxcore/src/log.rs` | the variant, after `ToolResult` (edit L1) |
| `crates/loopd/src/baseline.rs` | ignore arm (edit B1) |
| `crates/loopd/tests/support/turn.rs` | replaced, given |
| `crates/bxcore/tests/tool_summary.rs` | new, given |
| `crates/loopd/tests/tool_summary_messages.rs` | new, given |
| everything else | **no change**: not `index.rs` (task 09), not `turn.rs` |
`memoryd/src/index.rs` `index_record` already has `_ => {}`. It compiles. Summaries are skipped
until task 09.
## Files
- Copy, from `docs/plans/M5d2/files/` (the same path under it): the new tests
`crates/bxcore/tests/tool_summary.rs` and
`crates/loopd/tests/tool_summary_messages.rs`, and the replacement
`crates/loopd/tests/support/turn.rs`.
- Modify: `crates/bxcore/src/log.rs`, `crates/loopd/src/baseline.rs`,
`docs/implementer-log.md`.
- Do not touch: any other file.
## Facts you need
- **Field order is the wire format.** `time`, `call`, `tool_call_id`, `content`, `class`,
`untrusted`. No `truncated`: a summary is already capped.
- **Unknown fields are errors**, as on every `LogRecord` variant (`deny_unknown_fields` is on
the enum).
- **`types` is exhaustive.** Adding the variant without the replacement `support/turn.rs` fails
to compile `turn` / `limits` tests. Copy the given file over the existing one. The only
intended change is the `ToolSummary => "tool_summary"` arm. `Setup::turn` still returns
`(Result<TurnOutcome, TurnError>, Vec<TurnEvent>)`.
- **`messages` names every variant.** The new arm is empty, like `Usage` / `CacheLoss` /
`EpochEnd`.
- The items the edits use. This table is complete.
| Item | What it does |
|---|---|
| `CallId` | already imported in `log.rs` |
| `DataClass` | already imported in `log.rs` |
## The edits, exactly
Each quoted "before" block occurs exactly once after task 01.
### `crates/bxcore/src/log.rs`
**L1.** After the `ToolResult` variant (the lines)
```rust
truncated: bool,
},
Usage {
```
become
```rust
truncated: bool,
},
ToolSummary {
time: Timestamp,
call: CallId,
tool_call_id: String,
content: String,
class: DataClass,
untrusted: bool,
},
Usage {
```
Keep the fields in that order.
### `crates/loopd/src/baseline.rs`
**B1.** After the `ToolResult` arm (the lines)
```rust
} => out.push(ChatMessage::Tool {
tool_call_id: tool_call_id.clone(),
content: content.clone(),
}),
LogRecord::Usage { .. } => {}
```
become
```rust
} => out.push(ChatMessage::Tool {
tool_call_id: tool_call_id.clone(),
content: content.clone(),
}),
LogRecord::ToolSummary { .. } => {}
LogRecord::Usage { .. } => {}
```
## Steps
- [ ] **1. Copy the given files.** From the repository root:
```sh
cp docs/plans/M5d2/files/crates/bxcore/tests/tool_summary.rs crates/bxcore/tests/
cp docs/plans/M5d2/files/crates/loopd/tests/tool_summary_messages.rs crates/loopd/tests/
cp docs/plans/M5d2/files/crates/loopd/tests/support/turn.rs crates/loopd/tests/support/turn.rs
```
- [ ] **2. See them fail.** `cargo build --tests --keep-going -p bxcore -p loopd`, timeout
600000 ms: `tool_summary` and `tool_summary_messages` do not compile (`E0599` no variant
`ToolSummary`). `support/turn.rs` does not compile for the same reason until L1. After L1
and before B1, `baseline.rs` fails (`non-exhaustive patterns`). Do L1 then B1 in step 3;
this step is the first `cargo build` against the given tests only, which is E0599.
- [ ] **3. The edits.** Make L1 then B1. `cargo check -p bxcore -p loopd` prints no error and
no warning.
- [ ] **4. The tests.** `cargo test -p bxcore --test tool_summary --test records` and
`cargo test -p loopd --test tool_summary_messages --test turn --test baseline --test
session_slot`: every test passes, 3 in `tool_summary`, 3 in `tool_summary_messages`.
- [ ] **5. Check.** `git diff --stat` lists `crates/bxcore/src/log.rs`,
`crates/loopd/src/baseline.rs`, `crates/loopd/tests/support/turn.rs`.
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, timeout 600000 ms. Last line
`gate: ok`. Test count **1517** (1511 before, plus 6).
- [ ] **7. Log and commit.** Task cell `M5d2/02-bxcore-tool-summary`. Stage with
`git add crates/bxcore crates/loopd docs/implementer-log.md Cargo.lock`, then commit as
`AGENTS.md` says.
## Done when
- `make gate` prints `gate: ok` with 1517 tests.
- `git diff HEAD~1 --stat` lists only `crates/bxcore/src/log.rs`,
`crates/bxcore/tests/tool_summary.rs`, `crates/loopd/src/baseline.rs`,
`crates/loopd/tests/tool_summary_messages.rs`, `crates/loopd/tests/support/turn.rs` and the
log.
## Stop and report if
- The branch check fails; a "before" block is not found exactly once; step 2 differs; `cargo
check` warns; a test fails; the gate count is not 1517; you replay a summary as a `User` or
`Tool` message; you add `truncated`; you change `Setup::turn`'s return type.
## What the given tests pin
JSON type is `tool_summary`; unknown fields are refused; field order is the struct order;
`messages()` with and without a summary are equal; a summary is not a tool message; inserting a
summary between a result and a later user does not break prefix extension.
docs/plans/M5d2/03-loopd-heartbeat-config.md
+218
View File
@@ -0,0 +1,218 @@
# M5d-2 task 03: `[heartbeat]` and unknown `[compact] summarise` names
**Branch:** `m5d2`. Check it: `git branch --show-current` prints `m5d2` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `loopd: [heartbeat] table and summarise name check`
**Log row:** the Task cell is `M5d2/03-loopd-heartbeat-config`.
## Goal
`[heartbeat]` on `loopd.toml` (spec 4.1, 5): `every_s` (default `0`, disables) and `prompt`
(default `heartbeat.md`, next to the config file). Unknown keys are errors.
Unknown names in `[compact] summarise` are a config error (spec 4.3). `Config::check_summarise`
takes `&Registry`. `loopd` calls it after `load` with `Registry::m5c()`. `parse` still succeeds
with an unknown name: the given tests call `check_summarise` themselves.
The starter `config/heartbeat.md` is copied like `config/system.md`. Nothing ticks yet.
## Every place that changes
| Place | What happens there |
|---|---|
| `crates/loopd/src/config.rs` | `Heartbeat`; field; `load` joins the prompt; `heartbeat_prompt`; `check_summarise` |
| `crates/loopd/src/main.rs` | after each `Config::load`, `check_summarise(&Registry::m5c())` |
| `config/heartbeat.md` | new, given starter |
| `crates/loopd/tests/config_heartbeat.rs` | new, given |
| everything else | **no change** |
## Files
- Copy, from `docs/plans/M5d2/files/` (the same path under it): the new test
`crates/loopd/tests/config_heartbeat.rs` and the starter `config/heartbeat.md`.
- Modify: `crates/loopd/src/config.rs`, `crates/loopd/src/main.rs`,
`docs/implementer-log.md`.
- Do not touch: any other file.
## Facts you need
- **`every_s` is `u64`.** `0` means off. No range check.
- **`prompt` after `load`.** Join it to the config file's parent, the same way `load` already
joins `baseline.system`. `heartbeat_prompt()` returns `self.heartbeat.prompt.clone()`. After
`parse` (no file), the path stays relative.
- **`check_summarise`.** For each name in `self.compact.summarise`, `registry.get(name)` must be
`Some`. Empty list is allowed. The error `String` contains both `summarise` and the unknown
name. `Registry::get` is already `pub`. `http_fetch` and `clock` are in `m5c()`.
- **`main.rs`.** Both `run_selftest` and `run_serve` load a config. After a successful `load`,
if `check_summarise` fails, print `loopd: {e}` and return exit 1, as for a load error.
- **`#[serde(deny_unknown_fields, default)]`** on `Heartbeat`, like `Compact`.
- The items the edits use. This table is complete.
| Item | What it does |
|---|---|
| `Registry::get` | known tool names |
| `Registry::m5c` | what `loopd serve` uses |
| `Path::join` | prompt beside the config file |
## The edits, exactly
Each quoted "before" block occurs exactly once after M5d-1 (compact is already on `Config`).
### `crates/loopd/src/config.rs`
**H1.** After the `Compact` `Default` impl (the closing `}` of that impl, then the `Config`
struct's `memory` / `compact` fields). After
```rust
#[serde(default)]
pub compact: Compact,
}
```
become
```rust
#[serde(default)]
pub compact: Compact,
#[serde(default)]
pub heartbeat: Heartbeat,
}
```
**H2.** After the `Compact` `Default` impl, add `Heartbeat` (before the `Config` struct if that
is where `Compact` sits; keep `Heartbeat` next to `Compact`). The new type:
```rust
/// `[heartbeat]`: how often the standing `heartbeat` session runs (M5d spec, section 4.1).
/// `every_s` 0 disables it.
#[derive(Debug, Clone, PartialEq, Eq, Deserialize)]
#[serde(deny_unknown_fields, default)]
pub struct Heartbeat {
pub every_s: u64,
pub prompt: std::path::PathBuf,
}
impl Default for Heartbeat {
fn default() -> Self {
Self {
every_s: 0,
prompt: std::path::PathBuf::from("heartbeat.md"),
}
}
}
```
If `Compact` sits just above `Config`, put `Heartbeat` immediately after `Compact`'s `Default`
impl and **then** do H1. If a "before" block is not found exactly once, stop.
**H3.** In `load`, the lines
```rust
if let Some(parent) = path.parent() {
config.baseline.system = parent.join(&config.baseline.system);
}
Ok(config)
```
become
```rust
if let Some(parent) = path.parent() {
config.baseline.system = parent.join(&config.baseline.system);
config.heartbeat.prompt = parent.join(&config.heartbeat.prompt);
}
Ok(config)
```
**H4.** After `compact_prompt` (the lines)
```rust
pub fn compact_prompt(&self) -> std::path::PathBuf {
self.baseline.system.with_file_name("compact.md")
}
}
```
become
```rust
pub fn compact_prompt(&self) -> std::path::PathBuf {
self.baseline.system.with_file_name("compact.md")
}
/// The heartbeat prompt, already joined to the config file's directory by `load`.
pub fn heartbeat_prompt(&self) -> std::path::PathBuf {
self.heartbeat.prompt.clone()
}
/// Unknown `[compact] summarise` names are a config error (M5d spec, section 4.3).
pub fn check_summarise(&self, registry: &crate::tools::Registry) -> Result<(), String> {
for name in &self.compact.summarise {
if registry.get(name).is_none() {
return Err(format!("[compact] summarise: unknown tool {name}"));
}
}
Ok(())
}
}
```
### `crates/loopd/src/main.rs`
**M1.** In `run_selftest`, after a successful `Config::load` and before `Client::new`, add:
```rust
if let Err(e) = cfg.check_summarise(&loopd::tools::Registry::m5c()) {
eprintln!("loopd: {e}");
return ExitCode::from(1);
}
```
`Registry` is already imported in `main.rs`. Use `Registry::m5c()`.
**M2.** The same check in `run_serve`, after a successful `load` and before the instance lock.
If `main.rs` already has `use loopd::tools::{Registry, ToolPort};`, write `Registry::m5c()`.
## Steps
- [ ] **1. Copy the given files.**
```sh
cp docs/plans/M5d2/files/crates/loopd/tests/config_heartbeat.rs crates/loopd/tests/
cp docs/plans/M5d2/files/config/heartbeat.md config/heartbeat.md
```
- [ ] **2. See it fail.** `cargo build --tests --keep-going -p loopd`, timeout 600000 ms: test
target `config_heartbeat` does not compile (`E0609` no field `heartbeat`, `E0599` no method
`heartbeat_prompt` / `check_summarise`). Every other target compiles.
- [ ] **3. The edits.** H1 to H4, M1, M2, in order. `cargo check -p loopd` prints no error and
no warning.
- [ ] **4. The tests.** `cargo test -p loopd --test config_heartbeat --test config --test
config_compact`: every test passes, 8 in `config_heartbeat`.
- [ ] **5. Check.** `git diff --stat` lists `crates/loopd/src/config.rs` and
`crates/loopd/src/main.rs`. `git status --short` shows the new test and `config/heartbeat.md`.
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, timeout 600000 ms. Last line
`gate: ok`. Test count **1525** (1517 before, plus 8).
- [ ] **7. Log and commit.** Task cell `M5d2/03-loopd-heartbeat-config`. Stage with
`git add crates/loopd config/heartbeat.md docs/implementer-log.md Cargo.lock`, then commit
as `AGENTS.md` says.
## Done when
- `make gate` prints `gate: ok` with 1525 tests.
- `git diff HEAD~1 --stat` lists only `crates/loopd/src/config.rs`, `crates/loopd/src/main.rs`,
`crates/loopd/tests/config_heartbeat.rs`, `config/heartbeat.md` and the log.
## Stop and report if
- The branch check fails; a "before" block is not found exactly once; step 2 differs; `cargo
check` warns; a test fails; the gate count is not 1525; you join `heartbeat.prompt` onto
`system.md`'s directory instead of the config file's parent; you reject an empty `summarise`
list; you check names inside `parse`.
## What the given tests pin
A missing table is disabled and `heartbeat_prompt` is beside the config file; overrides replace
defaults; unknown keys error; default `http_fetch` is known to `m5c`; an unknown name errors
with `summarise` and that name; empty is allowed; `http_fetch` and `clock` together are known.
docs/plans/M5d2/04-loopd-cron.md
+192
View File
@@ -0,0 +1,192 @@
# M5d-2 task 04: five-field UTC cron matcher
**Branch:** `m5d2`. Check it: `git branch --show-current` prints `m5d2` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `loopd: five-field UTC cron matcher`
**Log row:** the Task cell is `M5d2/04-loopd-cron`.
## Goal
`loopd::cron::due(schedule, Timestamp) -> Result<bool, CronError>` (spec 4.2). Five fields:
minute hour day-of-month month day-of-week. Each field is `*`, a single number, `n-m`, or
`*/n`. Day-of-week `0` is Sunday. No cron crate. DOM and DOW are AND (README known limits).
Nothing loads job files yet.
## Every place that changes
| Place | What happens there |
|---|---|
| `crates/loopd/src/cron.rs` | new: `due`, `CronError` |
| `crates/loopd/src/lib.rs` | `pub mod cron;` |
| `crates/loopd/tests/cron.rs` | new, given |
| everything else | **no change** |
## Files
- Copy, from `docs/plans/M5d2/files/` (the same path under it): the new test
`crates/loopd/tests/cron.rs`.
- Create: `crates/loopd/src/cron.rs` (skeleton below).
- Modify: `crates/loopd/src/lib.rs`, `docs/implementer-log.md`.
- Do not touch: any other file.
## Facts you need
- **Civil fields from RFC 3339.** `Timestamp::to_rfc3339` is `YYYY-MM-DDTHH:MM:SS.mmmZ`.
Minute = `rfc[14..16]`, hour = `rfc[11..13]`, day = `rfc[8..10]`, month = `rfc[5..7]`,
parsed as `u32`. If a slice is missing, `CronError::Fields` naming the schedule.
- **Weekday.** `0` is Sunday. `(unix_millis() / 1000 / 86400 + 4) % 7` as `u32` (`1970-01-01`
was Thursday, so `+ 4`). `unix_millis` is `pub` on `Timestamp`.
- **Ranges.** Inclusive. `*/n` matches when `value % n == 0` (multiples from 0). `n == 0` in
a step is `Fields`.
- **Bounds.** Minute 0–59, hour 0–23, DOM 1–31, month 1–12, DOW 0–6. A number outside its
field's range is `Fields`. Four or six fields, `@daily`, and any other token are `Fields`.
- **`Display`.** Names the schedule, e.g. contains `x x x x x` when that was the input. Do not
add a runbook pointer; this error is used by job loading in task 05.
- **Split.** `split_whitespace`. Five pieces or `Fields`.
- rustfmt decides `mod` order in `lib.rs`. After `pub mod compact;` alphabetically `cron`
comes after `config` and before `http`. Insert where rustfmt will put it; run `cargo fmt`
before the gate.
- The items the edits use. This table is complete.
| Item | What it does |
|---|---|
| `Timestamp::to_rfc3339` | civil fields |
| `Timestamp::unix_millis` | weekday |
| `str::parse::<u32>` | numbers |
## Skeleton
`crates/loopd/src/cron.rs`:
```rust
//! Five-field UTC crontab matcher (M5d spec, section 4.2). No cron crate.
use bxcore::Timestamp;
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum CronError {
Fields(String),
}
impl std::fmt::Display for CronError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
CronError::Fields(schedule) => {
write!(f, "cron schedule {schedule:?} is not five UTC fields")
}
}
}
}
impl std::error::Error for CronError {}
/// Whether `schedule` matches `now` in UTC.
pub fn due(schedule: &str, now: Timestamp) -> Result<bool, CronError> {
let fields: Vec<&str> = schedule.split_whitespace().collect();
if fields.len() != 5 {
return Err(CronError::Fields(schedule.to_string()));
}
let rfc = now.to_rfc3339();
let minute = number(&rfc, 14, 16, schedule)?;
let hour = number(&rfc, 11, 13, schedule)?;
let day = number(&rfc, 8, 10, schedule)?;
let month = number(&rfc, 5, 7, schedule)?;
let dow = {
let days = now.unix_millis() / 1000 / 86_400;
((days + 4) % 7) as u32
};
Ok(match_field(fields[0], minute, 0, 59, schedule)?
&& match_field(fields[1], hour, 0, 23, schedule)?
&& match_field(fields[2], day, 1, 31, schedule)?
&& match_field(fields[3], month, 1, 12, schedule)?
&& match_field(fields[4], dow, 0, 6, schedule)?)
}
fn number(rfc: &str, start: usize, end: usize, schedule: &str) -> Result<u32, CronError> {
let piece = rfc
.get(start..end)
.ok_or_else(|| CronError::Fields(schedule.to_string()))?;
piece
.parse()
.map_err(|_| CronError::Fields(schedule.to_string()))
}
fn match_field(
field: &str,
value: u32,
min: u32,
max: u32,
schedule: &str,
) -> Result<bool, CronError> {
let err = || CronError::Fields(schedule.to_string());
if field == "*" {
return Ok(true);
}
if let Some(step) = field.strip_prefix("*/") {
let n: u32 = step.parse().map_err(|_| err())?;
if n == 0 {
return Err(err());
}
return Ok(value.is_multiple_of(n));
}
if let Some((a, b)) = field.split_once('-') {
let lo: u32 = a.parse().map_err(|_| err())?;
let hi: u32 = b.parse().map_err(|_| err())?;
if lo < min || hi > max || lo > hi {
return Err(err());
}
return Ok(value >= lo && value <= hi);
}
let n: u32 = field.parse().map_err(|_| err())?;
if n < min || n > max {
return Err(err());
}
Ok(n == value)
}
```
`u32::is_multiple_of` is stable in 1.98. Do not use `%` with a comment that `n` was checked;
the skeleton already returns `Fields` for `n == 0`.
If `is_multiple_of` is not in scope on `u32` in this toolchain, use `value % n == 0` after the
`n == 0` check, and stop only if both fail to compile.
## Steps
- [ ] **1. Copy the given test.**
```sh
cp docs/plans/M5d2/files/crates/loopd/tests/cron.rs crates/loopd/tests/
```
- [ ] **2. See it fail.** `cargo build --tests --keep-going -p loopd`, timeout 600000 ms: test
target `cron` does not compile (`E0432` unresolved import `loopd::cron`). Every other
target compiles.
- [ ] **3. The module.** Write `cron.rs` from the skeleton. Add `pub mod cron;` to `lib.rs`
where rustfmt will put it. `cargo check -p loopd` prints no error and no warning.
- [ ] **4. The tests.** `cargo test -p loopd --test cron`: every test passes, 10 tests.
- [ ] **5. Check.** `git diff --stat` lists `crates/loopd/src/cron.rs` and
`crates/loopd/src/lib.rs`.
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, timeout 600000 ms. Last line
`gate: ok`. Test count **1535** (1525 before, plus 10).
- [ ] **7. Log and commit.** Task cell `M5d2/04-loopd-cron`. Stage with
`git add crates/loopd docs/implementer-log.md Cargo.lock`, then commit as `AGENTS.md` says.
## Done when
- `make gate` prints `gate: ok` with 1535 tests.
- `git diff HEAD~1 --stat` lists only `crates/loopd/src/cron.rs`, `crates/loopd/src/lib.rs`,
`crates/loopd/tests/cron.rs` and the log.
## Stop and report if
- The branch check fails; step 2 differs; `cargo check` warns; a test fails; the gate count is
not 1535; you add a dependency; you treat DOM and DOW as OR; you accept `@daily` or a sixth
field; `cron.rs` exceeds 500 lines.
## What the given tests pin
`0 7 * * *` matches that minute only; `* * * * *` matches; `*/5` is multiples from 0; `0-10`
is inclusive; Sunday is 0 on 2026-10-04; the 1st of the month; four or six fields error;
minute 60, hour 24, DOW 7 error; `@daily` errors; `Display` names the schedule.
docs/plans/M5d2/05-loopd-jobs.md
+219
View File
@@ -0,0 +1,219 @@
# M5d-2 task 05: job files and the run ledger
**Branch:** `m5d2`. Check it: `git branch --show-current` prints `m5d2` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `loopd: job files, run ledger, jobs-invalid`
**Log row:** the Task cell is `M5d2/05-loopd-jobs`.
## Goal
Owner-written `{home}/jobs/<name>.toml` and a run ledger `{home}/jobs/runs.jsonl` (spec 4.2).
`loopd` reads job files and cannot write them. Unknown keys, a name that is not `[a-z0-9-]+`,
or a name so long that `job-<name>-<yyyymmdd>t<hhmm>` is not a `SessionId`, are
`jobs-invalid`. A missing jobs directory is an empty list.
Nothing ticks yet. `loopd serve` refuses to start when `load` errors, so a misspelt job cannot
sit silently.
## Every place that changes
| Place | What happens there |
|---|---|
| `crates/loopd/src/jobs.rs` | new: `Job`, `Run`, `RunStatus`, `load`, `session_id`, `append`, `close_leftovers`, `live_session` |
| `crates/loopd/src/lib.rs` | `pub mod jobs;` |
| `crates/loopd/src/main.rs` | `run_serve` calls `jobs::load` after `check_summarise` |
| `docs/runbook.md` | append the given `jobs-invalid` entry |
| `crates/loopd/tests/jobs.rs` | new, given |
| everything else | **no change** |
## Files
- Copy, from `docs/plans/M5d2/files/` (the same path under it): the new test
`crates/loopd/tests/jobs.rs`.
- Append, from `docs/plans/M5d2/files/runbook-jobs-invalid.md`, to `docs/runbook.md`.
- Create: `crates/loopd/src/jobs.rs` (skeleton below).
- Modify: `crates/loopd/src/lib.rs`, `crates/loopd/src/main.rs`, `docs/implementer-log.md`.
- Do not touch: any other file.
## Facts you need
- **`name` from the file stem.** `mail.toml` → `mail`. The stem must match `^[a-z0-9-]+$`
(bytes, nonempty) and `session_id(name, now)` must succeed. `Mail.toml` is `JobsError::Name`.
- **`session_id`.** `format!("job-{name}-{}t{}", ymd, hm)` with `ymd` = `rfc[0..4]+rfc[5..7]+rfc[8..10]`
and `hm` = `rfc[11..13]+rfc[14..16]`, all from `now.to_rfc3339()`, then
`SessionId::new`. `SessionId` is `[a-z0-9-]`, max 64. `"n".repeat(50)` is too long (`job-` +
50 + `-` + 8 + `t` + 4 = 68).
- **`load`.** `read_dir`. `ErrorKind::NotFound` → `Ok(vec![])`. Any other IO error is
`JobsError::Invalid` naming the directory. Skip names that do not end in `.toml`. Sort by
`name` so the order is stable. Parse with `#[serde(deny_unknown_fields)]` on a private
`File { schedule, prompt }`.
- **`RunStatus`.** `started` / `finished` / `failed` (`rename_all = "snake_case"`).
- **`append`.** Create the parent directory (0700 is fine; do not weaken a tighter mode). One
JSON object, then a newline. Open append, write, `sync_all`.
- **`live_session`.** Walk every line of the ledger (missing file → `Ok(None)`). The last
`Run` whose `name` matches: `Started` → `Some(session)`, anything else → `None`.
- **`close_leftovers`.** Names whose last run is `Started` get one more `Run` with
`status: Failed` and the same `session`, `time: now`. Return how many names were closed.
A missing file is `Ok(0)`.
- **`Display` for `Invalid`.** `"{path}: {detail}; see docs/runbook.md#jobs-invalid"`. The
given unknown-keys test checks `ends_with("see docs/runbook.md#jobs-invalid")` and that
the path's file name is in the string.
- **`run_serve`.** After `check_summarise`, `jobs::load(&cfg.paths.home.join("jobs"))`. On
`Err`, print `loopd: {e}` and return exit 1. Do not create the directory here.
- A file that exists but cannot be read is an error. Only a missing directory is empty.
- The items the edits use. This table is complete.
| Item | What it does |
|---|---|
| `toml::from_str` | already a `loopd` dependency |
| `SessionId::new` | the 64-byte alphabet check |
| `serde` `deny_unknown_fields` | unknown keys |
## Skeleton
`crates/loopd/src/jobs.rs` — fill every function; no `todo!()`. Keep the types in this order.
```rust
//! Owner-written job files and the run ledger (M5d spec, section 4.2).
use std::path::{Path, PathBuf};
use bxcore::{SessionId, Timestamp};
use serde::{Deserialize, Serialize};
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Job {
pub name: String,
pub schedule: String,
pub prompt: String,
}
#[derive(Debug, Clone, Copy, PartialEq, Eq, Serialize, Deserialize)]
#[serde(rename_all = "snake_case")]
pub enum RunStatus {
Started,
Finished,
Failed,
}
#[derive(Debug, Clone, PartialEq, Eq, Serialize, Deserialize)]
#[serde(deny_unknown_fields)]
pub struct Run {
pub time: Timestamp,
pub name: String,
pub session: SessionId,
pub status: RunStatus,
}
#[derive(Debug)]
pub enum JobsError {
Name(String),
Invalid { path: PathBuf, detail: String },
}
impl std::fmt::Display for JobsError {
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
match self {
JobsError::Name(name) => write!(
f,
"job name {name:?} is not a valid job; see docs/runbook.md#jobs-invalid"
),
JobsError::Invalid { path, detail } => write!(
f,
"{}: {detail}; see docs/runbook.md#jobs-invalid",
path.display()
),
}
}
}
impl std::error::Error for JobsError {}
#[derive(Deserialize)]
#[serde(deny_unknown_fields)]
struct File {
schedule: String,
prompt: String,
}
pub fn session_id(name: &str, now: Timestamp) -> Result<SessionId, JobsError> {
// Build job-<name>-<yyyymmdd>t<hhmm> from now.to_rfc3339() as in Facts. SessionId::new
// maps a failure to JobsError::Name(name.to_string()).
let _ = (name, now);
Err(JobsError::Name(name.to_string()))
}
pub fn load(dir: &Path) -> Result<Vec<Job>, JobsError> {
let _ = dir;
Ok(Vec::new())
}
pub fn append(path: &Path, run: &Run) -> Result<(), JobsError> {
let _ = (path, run);
Ok(())
}
pub fn live_session(path: &Path, name: &str) -> Result<Option<SessionId>, JobsError> {
let _ = (path, name);
Ok(None)
}
pub fn close_leftovers(path: &Path, now: Timestamp) -> Result<usize, JobsError> {
let _ = (path, now);
Ok(0)
}
```
Replace the four stub bodies in the same task. Do not leave a stub that returns `Ok(None)` /
`Ok(0)` / `Ok(vec![])` once you have written the real body. `session_id` must not stay on the
`Err` path.
## Steps
- [ ] **1. Copy the given test.**
```sh
cp docs/plans/M5d2/files/crates/loopd/tests/jobs.rs crates/loopd/tests/
```
- [ ] **2. See it fail.** `cargo build --tests --keep-going -p loopd`, timeout 600000 ms: test
target `jobs` does not compile (`E0432` unresolved `loopd::jobs`). Every other target
compiles.
- [ ] **3. The module.** Write `jobs.rs` from the skeleton, then fill the four bodies one at a
time (`session_id`, `load`, `append`, `live_session` / `close_leftovers`). Add `pub mod jobs;`
to `lib.rs`. Wire `run_serve` as in Facts. `cargo check -p loopd` prints no error and no
warning. `wc -l crates/loopd/src/jobs.rs` prints a number ≤ 500.
- [ ] **4. The tests.** `cargo test -p loopd --test jobs`: every test passes, 8 tests.
- [ ] **5. The runbook.** From the repository root:
```sh
{ echo; cat docs/plans/M5d2/files/runbook-jobs-invalid.md; } >> docs/runbook.md
```
Then `grep -c '^## jobs-invalid$' docs/runbook.md` prints `1`.
- [ ] **6. Check.** `git diff --stat` lists `crates/loopd/src/jobs.rs`,
`crates/loopd/src/lib.rs`, `crates/loopd/src/main.rs`, `docs/runbook.md`.
- [ ] **7. Run the gate.** `cargo fmt --all`, then `make gate`, timeout 600000 ms. Last line
`gate: ok`. Test count **1543** (1535 before, plus 8).
- [ ] **8. Log and commit.** Task cell `M5d2/05-loopd-jobs`. Stage with
`git add crates/loopd docs/runbook.md docs/implementer-log.md Cargo.lock`, then commit as
`AGENTS.md` says.
## Done when
- `make gate` prints `gate: ok` with 1543 tests.
- `git diff HEAD~1 --stat` lists only `crates/loopd/src/jobs.rs`, `crates/loopd/src/lib.rs`,
`crates/loopd/src/main.rs`, `crates/loopd/tests/jobs.rs`, `docs/runbook.md` and the log.
## Stop and report if
- The branch check fails; step 2 differs; `cargo check` warns; a test fails; the gate count is
not 1543; you add a cron crate; a missing jobs directory is an error; `Display` for `Invalid`
does not end with the runbook pointer; `jobs.rs` exceeds 500 lines; you write a job file.
## What the given tests pin
A valid `mail.toml` loads; unknown keys name the file and the runbook; `Mail.toml` is `Name`;
`session_id` is `job-mail-20261003t0700`; a 50-character name is `Name`; a missing directory is
empty; leftover `started` becomes `failed` and is no longer live; a `started` without a close
is live until `finished`.
docs/plans/M5d2/06-loopd-scheduler-heartbeat.md
+216
View File
@@ -0,0 +1,216 @@
# M5d-2 task 06: heartbeat tick
**Branch:** `m5d2`. Check it: `git branch --show-current` prints `m5d2` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `loopd: heartbeat tick on the background slot`
**Log row:** the Task cell is `M5d2/06-loopd-scheduler-heartbeat`.
## Goal
A `Scheduler` whose `tick(&Context, now)` runs the standing `heartbeat` session when
`every_s` has elapsed since the last heartbeat turn, or since `Scheduler::new(started)` if
none (spec 4.1). The user text is `It is <RFC 3339 UTC>.\n\n` plus the prompt file. Slot is
background. Speaker is `None`. Skip if the session is busy. Skip and log once if the prompt
cannot be read (`heartbeat-prompt-unreadable`).
Tests call `tick(ctx, now)` with injected time. No sleeps (T27).
## Every place that changes
| Place | What happens there |
|---|---|
| `crates/loopd/src/channel.rs` | `Held` is `pub`; `Context::hold` |
| `crates/loopd/src/scheduler.rs` | new: `Scheduler`, `TickWork`, `tick` |
| `crates/loopd/src/lib.rs` | `pub mod scheduler;` |
| `crates/loopd/src/main.rs` | a thread that `tick`s once a second, before `channel::serve` |
| `docs/runbook.md` | append the given `heartbeat-prompt-unreadable` entry |
| `crates/loopd/tests/scheduler_heartbeat.rs` | new, given |
| everything else | **no change**: not cron job runs (task 07) |
## Files
- Copy, from `docs/plans/M5d2/files/` (the same path under it): the new test
`crates/loopd/tests/scheduler_heartbeat.rs`.
- Append, from `docs/plans/M5d2/files/runbook-heartbeat-prompt-unreadable.md`, to
`docs/runbook.md`.
- Create: `crates/loopd/src/scheduler.rs` (skeleton below).
- Modify: `crates/loopd/src/channel.rs`, `crates/loopd/src/lib.rs`,
`crates/loopd/src/main.rs`, `docs/implementer-log.md`.
- Do not touch: any other file.
## Facts you need
- **`Context::hold`.** Same insert as `handle`: if the id is already in `busy`, `Err(())`;
else insert and return `Held`. `Held` must be `pub` so the given test can keep it. `Drop`
already removes the id.
- **Due.** `every_s == 0` → no heartbeat work. Else `now.unix_millis().saturating_sub(last.unix_millis()) >= every_s.saturating_mul(1000)`
where `last` is `last_heartbeat.unwrap_or(started)`. `07:00` to `07:01` with `every_s = 60`
is due. `07:01` then `07:01:30` is not.
- **Prompt first, then hold.** If `read_to_string(cfg.heartbeat_prompt())` fails, do not hold,
push `HeartbeatUnreadable`, log
`loopd: {path}: {e}; see docs/runbook.md#heartbeat-prompt-unreadable` only while
`heartbeat_unreadable_logged` is false, then set that flag. Do not update `last_heartbeat`.
A directory at the prompt path is the given-test stand-in for an unreadable file (T27: no
`chmod 000`).
- **Session id `heartbeat`.** `Session::open`; `NotFound` → `Session::create` with
`Baseline::assemble(&ctx.cfg, &ctx.registry)` and `ctx.cfg.slots.background`. Other open
errors: log and skip (do not panic).
- **The turn.** User text `format!("It is {}.\n\n{file}", now.to_rfc3339())` where `file` is
the prompt contents as read (no extra trim beyond what the file has). `run_turn` then
`compact::finish_turn` with `&*ctx.memory`, speaker `None`, `on_event` a no-op. Then
`last_heartbeat = Some(now)`, push `Heartbeat`.
- **`TickWork`.** Include `Job` / `SkippedJob` now so task 07 does not change the type.
Task 06 never pushes them.
- **`close_leftovers`.** On the first `tick` only, call
`jobs::close_leftovers(&ctx.cfg.paths.home.join("jobs").join("runs.jsonl"), now)` and ignore
a missing-file `Ok(0)`. A leftover that is currently `hold`ing its session is **not** closed:
skip names whose `live_session` session is in `busy` (or: close only when `hold` of that
session would succeed, then immediately `drop` that hold). Task 07's live-skip test depends
on this. If `close_leftovers` has no busy-set filter, add a loop here: for each live name,
if `hold(session)` is `Ok`, drop it and treat as leftover (append Failed yourself or call a
helper). Simplest: **do not call `close_leftovers` blindly**. Walk: if `live_session` is
`Some(id)` and `hold(id)` is `Ok`, drop the hold and append a `Failed` for that name (or
call `close_leftovers` only when no job session is held). The given leftover test does not
hold; the live-skip test does. Closing only leftovers that are not busy is required.
- **Main thread.** After `Context::new` and before `channel::serve`:
```rust
let ctx_sched = Arc::clone(&ctx);
std::thread::spawn(move || {
let mut sched = loopd::scheduler::Scheduler::new(bxcore::Timestamp::now());
loop {
sched.tick(&ctx_sched, bxcore::Timestamp::now());
std::thread::sleep(std::time::Duration::from_secs(1));
}
});
```
That sleep is production only. Tests never sleep.
- **`channel.rs` stays ≤ 500 lines.** `hold` is about 15 lines.
- The items the edits use. This table is complete.
| Item | What it does |
|---|---|
| `Session::slot` / `create` / `open` | standing session |
| `run_turn` / `finish_turn` | the pulse |
| `Baseline::assemble` | new session |
| `jobs::close_leftovers` / `live_session` / `append` | leftovers |
| `Timestamp::unix_millis` / `to_rfc3339` | due and the time line |
## The edits, exactly
### `crates/loopd/src/channel.rs`
**C1.** `struct Held` becomes `pub struct Held`.
**C2.** Inside `impl Context`, after `new`, add:
```rust
/// Mark `id` busy. `Err(())` if it already is. Tests and the scheduler use this so a skip
/// is not a sleep (T27).
pub fn hold(&self, id: SessionId) -> Result<Held<'_>, ()> {
let mut busy = self.busy.lock().unwrap_or_else(|p| p.into_inner());
if busy.contains(&id) {
return Err(());
}
busy.insert(id.clone());
drop(busy);
Ok(Held { ctx: self, id })
}
```
### `crates/loopd/src/scheduler.rs`
Write this module. Fill `tick`. Cron jobs stay a comment until task 07: after heartbeat, do
not iterate job files yet.
```rust
//! Isolated heartbeat (and, from task 07, cron) turns (M5d spec, section 4).
use crate::channel::Context;
use bxcore::{SessionId, Timestamp};
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum TickWork {
Heartbeat,
SkippedBusy { session: SessionId },
HeartbeatUnreadable,
Job { name: String, session: SessionId },
SkippedJob { name: String },
}
pub struct Scheduler {
started: Timestamp,
last_heartbeat: Option<Timestamp>,
leftovers_closed: bool,
heartbeat_unreadable_logged: bool,
}
impl Scheduler {
pub fn new(started: Timestamp) -> Scheduler {
Scheduler {
started,
last_heartbeat: None,
leftovers_closed: false,
heartbeat_unreadable_logged: false,
}
}
pub fn tick(&mut self, ctx: &Context, now: Timestamp) -> Vec<TickWork> {
let _ = (ctx, now);
Vec::new()
}
}
```
Replace the `tick` body in this task. `leftovers_closed` is set true after the first leftover
pass so it runs once.
## Steps
- [ ] **1. Copy the given test.**
```sh
cp docs/plans/M5d2/files/crates/loopd/tests/scheduler_heartbeat.rs crates/loopd/tests/
```
- [ ] **2. See it fail.** `cargo build --tests --keep-going -p loopd`, timeout 600000 ms: test
target `scheduler_heartbeat` does not compile (`E0432` unresolved `loopd::scheduler`,
`E0599` no method `hold`). Every other target compiles.
- [ ] **3. The edits.** C1, C2, `scheduler.rs`, `pub mod scheduler;`, main thread. Fill
`tick`. `cargo check -p loopd` prints no error and no warning.
`wc -l crates/loopd/src/scheduler.rs` and `wc -l crates/loopd/src/channel.rs` each ≤ 500.
- [ ] **4. The tests.** `cargo test -p loopd --test scheduler_heartbeat --test channel --test
session_slot`: every test passes, 6 in `scheduler_heartbeat`.
- [ ] **5. The runbook.**
```sh
{ echo; cat docs/plans/M5d2/files/runbook-heartbeat-prompt-unreadable.md; } >> docs/runbook.md
```
`grep -c '^## heartbeat-prompt-unreadable$' docs/runbook.md` prints `1`.
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, timeout 600000 ms. Last line
`gate: ok`. Test count **1549** (1543 before, plus 6).
- [ ] **7. Log and commit.** Task cell `M5d2/06-loopd-scheduler-heartbeat`. Stage with
`git add crates/loopd docs/runbook.md docs/implementer-log.md Cargo.lock`, then commit as
`AGENTS.md` says.
## Done when
- `make gate` prints `gate: ok` with 1549 tests.
- `git diff HEAD~1 --stat` lists only `crates/loopd/src/channel.rs`,
`crates/loopd/src/scheduler.rs`, `crates/loopd/src/lib.rs`, `crates/loopd/src/main.rs`,
`crates/loopd/tests/scheduler_heartbeat.rs`, `docs/runbook.md` and the log.
## Stop and report if
- The branch check fails; a "before" block is not found exactly once; step 2 differs; `cargo
check` warns; a test fails; the gate count is not 1549; you sleep in a test; you post to
Mattermost; you send `slots.main`; you set a speaker; you `chmod 000`; `Held` stays private.
## What the given tests pin
`every_s` 0 does nothing; a due heartbeat sends `id_slot` 1 with a time line and the file
text, session `heartbeat` has `slot() == 1`; a second tick before `every_s` does not run; a
held session is `SkippedBusy` with no request; a directory at the prompt path is
`HeartbeatUnreadable` with no request; the user record has `speaker: None`.
docs/plans/M5d2/07-loopd-scheduler-cron.md
+107
View File
@@ -0,0 +1,107 @@
# M5d-2 task 07: cron tick
**Branch:** `m5d2`. Check it: `git branch --show-current` prints `m5d2` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `loopd: cron tick, new session per run`
**Log row:** the Task cell is `M5d2/07-loopd-scheduler-cron`.
## Goal
After the heartbeat work, `tick` loads `{home}/jobs/*.toml` and runs each job whose schedule
matches `now` (spec 4.2). A new session `job-<name>-<yyyymmdd>t<hhmm>` on the background slot.
Skip the tick when a previous run of that name is still live. Leftover `started` from a dead
process becomes `failed` on the first tick (already started in task 06: only when that session
is not busy). Speaker `None`. Same time-line prefix as heartbeat.
## Every place that changes
| Place | What happens there |
|---|---|
| `crates/loopd/src/scheduler.rs` | after heartbeat, iterate jobs (edit T1) |
| `crates/loopd/tests/scheduler_cron.rs` | new, given |
| everything else | **no change**: not `TickWork`, not `hold`, not heartbeat |
## Files
- Copy, from `docs/plans/M5d2/files/` (the same path under it): the new test
`crates/loopd/tests/scheduler_cron.rs`.
- Modify: `crates/loopd/src/scheduler.rs`, `docs/implementer-log.md`.
- Do not touch: any other file.
## Facts you need
- **Load.** `jobs::load(&ctx.cfg.paths.home.join("jobs"))`. On `Err`, log `loopd: {e}` and
skip cron this tick (the process stays up: README known limits). On `Ok`, for each `Job` in
order:
1. `cron::due(&job.schedule, now)`. `Err` → log and skip that job. `Ok(false)` → skip.
2. Ledger path `{home}/jobs/runs.jsonl`. If `live_session` is `Some(_)`, push
`SkippedJob { name }` and continue. (Task 06 has already closed leftovers that were not
busy.)
3. `session_id(&job.name, now)`. `Err` → log and skip.
4. `ctx.hold(session.clone())`. `Err` → `SkippedJob`. `Ok(held)` → create the session
(`Session::create` with `assemble` and `slots.background`; if `Exists`, `open` instead),
`jobs::append` `Started`, run `run_turn` then `finish_turn` with user text
`format!("It is {}.\n\n{}", now.to_rfc3339(), job.prompt)`, speaker `None`, then
`append` `Finished` or `Failed` (turn `Err` → `Failed`). `drop(held)`. Push
`TickWork::Job { name, session }`.
- **Two due minutes.** Two ticks at 06:59 and 07:00 with `* * * * *` make two session ids.
- **Not due.** `0 7 * * *` at 06:59 pushes nothing.
- **`scheduler.rs` stays ≤ 500 lines.** After this task it is about 200.
- The items the edits use. This table is complete.
| Item | What it does |
|---|---|
| `jobs::load` / `session_id` / `append` / `live_session` | task 05 |
| `cron::due` | task 04 |
| `Context::hold` | task 06 |
| `run_turn` / `finish_turn` | same as heartbeat |
## The edits, exactly
There is no quoted "before" for a unique heartbeat-only block: you add a loop after the
heartbeat section of `tick`, before `work` is returned. If `tick` currently ends with
`work` after heartbeat, insert the job loop there. If you cannot find a single place to
insert without touching heartbeat, stop.
Do not change heartbeat behaviour. Do not add a sleep.
## Steps
- [ ] **1. Copy the given test.**
```sh
cp docs/plans/M5d2/files/crates/loopd/tests/scheduler_cron.rs crates/loopd/tests/
```
- [ ] **2. See it fail.** `cargo test -p loopd --test scheduler_cron`, timeout 600000 ms: the
tests compile (`TickWork::Job` already exists) and `a_due_job_starts_a_new_session_on_the_background_slot`
fails: `work` is empty, not a `Job`. `a_job_not_due_does_not_run` may pass.
- [ ] **3. The edit.** Add the job loop. `cargo check -p loopd` prints no error and no
warning. `wc -l crates/loopd/src/scheduler.rs` prints a number ≤ 500.
- [ ] **4. The tests.** `cargo test -p loopd --test scheduler_cron --test scheduler_heartbeat
--test jobs --test cron`: every test passes, 6 in `scheduler_cron`.
- [ ] **5. Check.** `git diff --stat` lists exactly `crates/loopd/src/scheduler.rs`.
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, timeout 600000 ms. Last line
`gate: ok`. Test count **1555** (1549 before, plus 6).
- [ ] **7. Log and commit.** Task cell `M5d2/07-loopd-scheduler-cron`. Stage with
`git add crates/loopd docs/implementer-log.md Cargo.lock`, then commit as `AGENTS.md` says.
## Done when
- `make gate` prints `gate: ok` with 1555 tests.
- `git diff HEAD~1 --stat` lists only `crates/loopd/src/scheduler.rs`,
`crates/loopd/tests/scheduler_cron.rs` and the log.
## Stop and report if
- The branch check fails; step 2 differs (tests do not compile, or a different assertion
fails); `cargo check` warns; a test fails; the gate count is not 1555; you reuse a session
across runs; you send `slots.main`; you set a speaker; you skip leftovers that are not busy;
a heartbeat test fails.
## What the given tests pin
A due job creates `job-mail-…` on slot 1 with the time line and the prompt; a job not due
does nothing; a live previous run (Started + `hold`) is `SkippedJob` with no request; leftover
`started` is `failed` on the first tick (the due job may then run); speaker is `None`; two due
minutes make two sessions.
docs/plans/M5d2/08-loopd-tool-summary.md
+257
View File
@@ -0,0 +1,257 @@
# M5d-2 task 08: opted-in tool-output summaries
**Branch:** `m5d2`. Check it: `git branch --show-current` prints `m5d2` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `loopd: background ToolSummary for opted-in results`
**Log row:** the Task cell is `M5d2/08-loopd-tool-summary`.
## Goal
After a turn that logged an opted-in `ToolResult` with no matching `ToolSummary`, run a
no-tools completion on the background slot and append `ToolSummary` with the result's labels
(spec 4.3). `messages()` already ignores it (task 02). `http_fetch` opts in via
`[compact] summarise`.
`summarise::missing` is the function. `finish_turn` calls it on `Ok` and on `SessionFull`,
**before** compaction, so the current epoch still holds the `ToolResult` (README known
limits: v1 runs this inside the turn's hold).
The starter `config/summarise.md` is copied like `config/heartbeat.md`.
## Every place that changes
| Place | What happens there |
|---|---|
| `crates/loopd/src/summarise.rs` | new: `missing` |
| `crates/loopd/src/lib.rs` | `pub mod summarise;` |
| `crates/loopd/src/compact.rs` | `finish_turn` calls `missing` before compact (edit F1) |
| `config/summarise.md` | new, given starter |
| `crates/loopd/tests/tool_summary_call.rs` | new, given |
| everything else | **no change** |
## Files
- Copy, from `docs/plans/M5d2/files/` (the same path under it): the new test
`crates/loopd/tests/tool_summary_call.rs` and the starter `config/summarise.md`.
- Create: `crates/loopd/src/summarise.rs` (function below).
- Modify: `crates/loopd/src/lib.rs`, `crates/loopd/src/compact.rs`,
`docs/implementer-log.md`.
- Do not touch: any other file.
## Facts you need
- **Which results.** Walk `session.records()` once. From each `Assistant`, remember
`tool_calls` by `id → name`. From each `ToolSummary`, remember `call: CallId`. Then for
each `ToolResult` whose `tool_call_id` maps to a name in `cfg.compact.summarise` and whose
`call` is not already summarised, queue it. Do not append while iterating `records()`.
- **The call.** `messages_with_carry(session.baseline(), session.records(), session.carry())`,
then one `ChatMessage::User` whose content is
`format!("{}\n\n{}", prompt.trim_end(), crate::compact::cap_chars(result_content, cfg.r#loop.tool_result_cap))`
where `prompt` is `read_to_string` of
`cfg.baseline.system.with_file_name("summarise.md")`. Missing file: return `Err` naming
the path (do not guess). `ChatRequest { slot: cfg.slots.background, tools: Vec::new(),
thinking: true, messages }`. `client.chat_with_retry(&request, &mut |_| {})`.
- **The record.** Completion `content.unwrap_or_default()`, `neutralise`, `cap_chars` to
`cfg.compact.summary_chars`. Append `LogRecord::ToolSummary { time: Timestamp::now(),
call, tool_call_id, content, class, untrusted }` copying `call`, `tool_call_id`, `class`,
`untrusted` from that `ToolResult`.
- **Context-full / other infer errors.** Skip that result (log `loopd: tool summary: {e}`),
continue the queue. Do not fail the turn.
- **`finish_turn`.** At the start of the `Ok(done)` arm and the `Err(SessionFull)` arm, before
`compact_session`:
```rust
if let Err(e) = crate::summarise::missing(rt.cfg, rt.client, session) {
eprintln!("loopd: tool summary: {e}");
}
```
A compact-due turn summarises first, then compacts. The last given test goes through
`finish_turn` (not `missing` alone) and expects a `ToolSummary`.
- **`cap_chars` is `pub(crate)`** in `compact.rs` (M5d-1 task 05).
- **`tool_call.http` calls `read_file`.** The last given test sets
`summarise = ["read_file"]`. Default `["http_fetch"]` would leave it alone.
- rustfmt `mod` order: `summarise` after `sse` / `session` as rustfmt decides.
- The items the edits use. This table is complete.
| Item | What it does |
|---|---|
| `messages_with_carry` | prefix plus carry |
| `ChatRequest.tools: Vec::new()` | omits `tools` on the wire |
| `neutralise` | `bxcore::sanitize` |
| `cap_chars` | character cap |
| `Session::append` | the new record |
## The function
`crates/loopd/src/summarise.rs`:
```rust
//! Opted-in tool-output summaries (M5d spec, section 4.3).
use crate::baseline::messages_with_carry;
use crate::compact::cap_chars;
use crate::config::Config;
use crate::llama::{ChatMessage, ChatRequest, Client};
use crate::session::Session;
use bxcore::sanitize::neutralise;
use bxcore::{CallId, DataClass, LogRecord, Timestamp, ToolCall};
use std::collections::{HashMap, HashSet};
/// Append a `ToolSummary` for each opted-in `ToolResult` in this epoch that does not yet have
/// one. Inference uses the background slot and no tools.
pub fn missing(
cfg: &Config,
client: &Client,
session: &mut Session,
) -> Result<(), String> {
let path = cfg.baseline.system.with_file_name("summarise.md");
let prompt = std::fs::read_to_string(&path)
.map_err(|e| format!("{}: {e}", path.display()))?;
let mut names: HashMap<String, String> = HashMap::new();
let mut done: HashSet<u64> = HashSet::new();
let mut pending: Vec<(CallId, String, String, DataClass, bool)> = Vec::new();
for record in session.records() {
match record {
LogRecord::Assistant { tool_calls, .. } => {
for ToolCall { id, name, .. } in tool_calls {
names.insert(id.clone(), name.clone());
}
}
LogRecord::ToolSummary { call, .. } => {
done.insert(call.0);
}
LogRecord::ToolResult {
call,
tool_call_id,
content,
class,
untrusted,
..
} => {
if done.contains(&call.0) {
continue;
}
let Some(name) = names.get(tool_call_id) else {
continue;
};
if !cfg.compact.summarise.iter().any(|n| n == name) {
continue;
}
pending.push((
*call,
tool_call_id.clone(),
content.clone(),
*class,
*untrusted,
));
}
_ => {}
}
}
for (call, tool_call_id, content, class, untrusted) in pending {
let mut messages = messages_with_carry(session.baseline(), session.records(), session.carry());
messages.push(ChatMessage::User {
content: format!(
"{}\n\n{}",
prompt.trim_end(),
cap_chars(&content, cfg.r#loop.tool_result_cap)
),
});
let request = ChatRequest {
slot: cfg.slots.background,
messages,
tools: Vec::new(),
thinking: true,
};
let text = match client.chat_with_retry(&request, &mut |_| {}) {
Ok(completion) => completion.content.unwrap_or_default(),
Err(e) => {
eprintln!("loopd: tool summary: {e}");
continue;
}
};
let text = cap_chars(neutralise(&text).as_ref(), cfg.compact.summary_chars);
session
.append(LogRecord::ToolSummary {
time: Timestamp::now(),
call,
tool_call_id,
content: text,
class,
untrusted,
})
.map_err(|e| e.to_string())?;
}
Ok(())
}
```
`CallId` is a tuple struct `CallId(pub u64)` (already used as `CallId(1)` in tests). If
`call.0` does not compile, stop and report.
### `crates/loopd/src/compact.rs`
**F1.** The `Ok(done)` arm. The lines
```rust
Ok(done) => {
if let Some(usage) = session.last_usage()
&& due(&usage, rt.cfg.compact.threshold, rt.cfg.expect.n_ctx)
&& let Err(e) = compact_session(session, rt, memory)
{
log_compact(session, &e);
}
Ok(done)
}
```
become the same, with the `missing` call inserted as the first statement of the arm (see
Facts). The `Err(TurnError::SessionFull)` arm gets the same `missing` call before
`compact_session`.
## Steps
- [ ] **1. Copy the given files.**
```sh
cp docs/plans/M5d2/files/crates/loopd/tests/tool_summary_call.rs crates/loopd/tests/
cp docs/plans/M5d2/files/config/summarise.md config/summarise.md
```
- [ ] **2. See it fail.** `cargo build --tests --keep-going -p loopd`, timeout 600000 ms: test
target `tool_summary_call` does not compile (`E0432` unresolved `loopd::summarise`). Every
other target compiles.
- [ ] **3. The edits.** `summarise.rs`, `pub mod summarise;`, F1. `cargo check -p loopd`
prints no error and no warning. `wc -l crates/loopd/src/summarise.rs` and
`wc -l crates/loopd/src/compact.rs` each ≤ 500.
- [ ] **4. The tests.** `cargo test -p loopd --test tool_summary_call --test
tool_summary_messages --test compact_turn --test turn`: every test passes, 6 in
`tool_summary_call`.
- [ ] **5. Check.** `git diff --stat` lists `crates/loopd/src/summarise.rs`,
`crates/loopd/src/lib.rs`, `crates/loopd/src/compact.rs`.
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, timeout 600000 ms. Last line
`gate: ok`. Test count **1561** (1555 before, plus 6).
- [ ] **7. Log and commit.** Task cell `M5d2/08-loopd-tool-summary`. Stage with
`git add crates/loopd config/summarise.md docs/implementer-log.md Cargo.lock`, then commit
as `AGENTS.md` says.
## Done when
- `make gate` prints `gate: ok` with 1561 tests.
- `git diff HEAD~1 --stat` lists only `crates/loopd/src/summarise.rs`,
`crates/loopd/src/lib.rs`, `crates/loopd/src/compact.rs`,
`crates/loopd/tests/tool_summary_call.rs`, `config/summarise.md` and the log.
## Stop and report if
- The branch check fails; a "before" block is not found exactly once; step 2 differs; `cargo
check` warns; a test fails; the gate count is not 1561; you send tools on the summary
request; you send `slots.main`; you replay the summary in `messages()`; you summarise a
name not in the list; you summarise twice; you compact before summarising.
## What the given tests pin
An opted-in result is summarised on slot 1 with no `tools` key; labels are copied; a name not
in the list is left alone; a second `missing` does not call again; `messages()` is unchanged
after a summary; `finish_turn` after a `read_file` turn appends a `ToolSummary` on slot 1.
docs/plans/M5d2/09-memoryd-summary-index.md
+141
View File
@@ -0,0 +1,141 @@
# M5d-2 task 09: index `ToolSummary` as kind `summary`
**Branch:** `m5d2`. Check it: `git branch --show-current` prints `m5d2` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `memoryd: index ToolSummary as kind summary`
**Log row:** the Task cell is `M5d2/09-memoryd-summary-index`.
## Goal
`memoryd` indexes `LogRecord::ToolSummary` content as kind `summary`, with that record's
`class` and `untrusted` (spec 4.3, P26). `recall` finds it and tags `memory` or
`memory-tainted` as for any other session hit. Raw `ToolResult` bodies stay unindexed.
`index_record` today has `_ => {}`. It compiles. This task adds the arm.
## Every place that changes
| Place | What happens there |
|---|---|
| `crates/memoryd/src/index.rs` | `ToolSummary` arm (edit I1); module doc |
| `crates/memoryd/tests/tool_summary_index.rs` | new, given |
| everything else | **no change**: not `recall.rs` headers (role `"summary"` uses the existing "the assistant said" branch) |
## Files
- Copy, from `docs/plans/M5d2/files/` (the same path under it): the new test
`crates/memoryd/tests/tool_summary_index.rs`.
- Modify: `crates/memoryd/src/index.rs`, `docs/implementer-log.md`.
- Do not touch: any other file. Do not edit `tests/support/logs.rs`.
## Facts you need
- **Kind `summary`.** Set `kind: "summary".to_string()` on the `Source`, not the log's
`"session"`. Role `"summary"`. Time from the record. `class` / `untrusted` from the
`ToolSummary`, not from `base` (the rolling session label). Skip when
`summary.class > self.max_class`, even if `base.class` is still `Private`. Skip when
`content.trim()` is empty.
- **The existing `if base.class > self.max_class` stays.** A secret `ToolResult` raises the
rolling label before the next record, so a following secret summary is also skipped that
way. Still check the summary's own class.
- **Do not add `ToolSummary` to the rolling-label update** in `index_log` (that loop still
looks at `ToolResult` only). The result already carried the same labels.
- **Recall headers.** `kind` `"summary"` is not `moved` / `dropped` / `noted`, so the tag is
`memory` or `memory-tainted`. Role `"summary"` is not `owner` or `call`, so the line says
`the assistant said`. The given tests look for `memory · session mm-a` and
`memory-tainted`.
- The items the edits use. This table is complete.
| Item | What it does |
|---|---|
| `self.add` | FTS row plus `sources` |
| `Source` | kind / role / class / untrusted |
## The edits, exactly
Each quoted "before" block occurs exactly once.
### `crates/memoryd/src/index.rs`
**I1.** After the `Assistant` arm, before `_ => {}`. The lines
```rust
}
_ => {}
}
Ok(())
}
```
become
```rust
}
LogRecord::ToolSummary {
time,
content,
class,
untrusted,
..
} if !content.trim().is_empty() && *class <= self.max_class => {
let src = Source {
kind: "summary".to_string(),
time: time.to_rfc3339(),
role: "summary".to_string(),
class: *class,
untrusted: *untrusted,
..base.clone()
};
self.add(content, &src)?;
}
_ => {}
}
Ok(())
}
```
`DataClass` already implements `Ord` (`Public < Private < Secret`). If `<=` does not
compile, stop.
Optionally update the module doc "never a tool's result" to say summaries of opted-in
results are indexed. A comment that does not fit its code: stop.
## Steps
- [ ] **1. Copy the given test.**
```sh
cp docs/plans/M5d2/files/crates/memoryd/tests/tool_summary_index.rs crates/memoryd/tests/
```
- [ ] **2. See it fail.** `cargo test -p memoryd --test tool_summary_index`, timeout 600000
ms: the tests compile (`ToolSummary` exists) and `a_tool_summary_is_found_as_kind_summary`
fails: `(no matches for "tea")` or the unique phrase is missing. `the_raw_tool_result_is_still_not_indexed`
may pass (results were already unindexed).
- [ ] **3. The edit.** I1. `cargo check -p memoryd` prints no error and no warning.
- [ ] **4. The tests.** `cargo test -p memoryd --test tool_summary_index --test recall`: every
test passes, 5 in `tool_summary_index`.
- [ ] **5. Check.** `git diff --stat` lists exactly `crates/memoryd/src/index.rs`.
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, timeout 600000 ms. Last line
`gate: ok`. Test count **1566** (1561 before, plus 5).
- [ ] **7. Log and commit.** Task cell `M5d2/09-memoryd-summary-index`. Stage with
`git add crates/memoryd docs/implementer-log.md Cargo.lock`, then commit as `AGENTS.md`
says.
## Done when
- `make gate` prints `gate: ok` with 1566 tests.
- `git diff HEAD~1 --stat` lists only `crates/memoryd/src/index.rs`,
`crates/memoryd/tests/tool_summary_index.rs` and the log.
## Stop and report if
- The branch check fails; a "before" block is not found exactly once; step 2 differs; `cargo
check` warns; a test fails; the gate count is not 1566; you index `ToolResult` content; you
leave `kind` as `"session"`; a secret summary is searchable on a private index.
## What the given tests pin
A summary's content is found and tagged `memory · session …`; the raw tool body is not; an
untrusted summary taints the recall (`memory-tainted`); a trusted summary does not taint; a
secret summary is skipped when `max_class` is `Private`.
docs/plans/M5d2/10-memoryd-tiers.md
+227
View File
@@ -0,0 +1,227 @@
# M5d-2 task 10: `MemoryCommand::Tiers` and `bxctl memory tiers`
**Branch:** `m5d2`. Check it: `git branch --show-current` prints `m5d2` and `git status --short`
is empty; otherwise stop. Never switch branch.
**Commit subject:** `memoryd: Tiers listing; bxctl memory tiers`
**Log row:** the Task cell is `M5d2/10-memoryd-tiers`.
## Goal
`MemoryCommand::Tiers` lists each identity entry and each note with how many times `recalls/`
named it and when it was last found (spec 4.4). Untrusted notes are included and marked.
Identity entries list with 0 hits (README known limits). `bxctl memory tiers` prints that
listing. The starter `config/jobs/tier.toml.example` documents a weekly job; nothing in
`loopd` special-cases it.
## Every place that changes
| Place | What happens there |
|---|---|
| `crates/bxcore/src/memory.rs` | `MemoryCommand::Tiers` after `Sync` |
| `crates/memoryd/src/tiers.rs` | new: `list` |
| `crates/memoryd/src/lib.rs` | `pub mod tiers;` |
| `crates/memoryd/src/serve/admin.rs` | `Tiers` arm |
| `crates/bxctl/src/memory.rs` | `["tiers"]` |
| `crates/bxctl/src/cli.rs` | `USAGE` names `tiers` |
| `config/jobs/tier.toml.example` | new, given starter |
| `crates/memoryd/tests/tiers.rs` | new, given |
| `crates/bxctl/tests/cli_memory_tiers.rs` | new, given |
| everything else | **no change**: not `cli_memory.rs` (T19: new file) |
## Files
- Copy, from `docs/plans/M5d2/files/` (the same path under it): the new tests
`crates/memoryd/tests/tiers.rs` and `crates/bxctl/tests/cli_memory_tiers.rs`, and the
starter `config/jobs/tier.toml.example`.
- Create: `crates/memoryd/src/tiers.rs` (function below).
- Modify: `crates/bxcore/src/memory.rs`, `crates/memoryd/src/lib.rs`,
`crates/memoryd/src/serve/admin.rs`, `crates/bxctl/src/memory.rs`,
`crates/bxctl/src/cli.rs`, `docs/implementer-log.md`.
- Do not touch: any other file. Do not edit `crates/bxctl/tests/cli_memory.rs`.
## Facts you need
- **`MemoryCommand` is `rename_all = "snake_case"`.** JSON is `"tiers"`. Add the variant after
`Sync`, with a doc comment. `admin.rs` match is exhaustive: add the arm in this task or it
does not compile.
- **Listing format, exact.** One block per identity entry, then one per note, in file order:
```text
identity memory.md hits 0 last -
I like tea.
identity user.md hits 0 last -
The owner is Kyle.
note session mm-a hits 2 last 2026-10-03T07:30:00.000Z private trusted
launch window moved.
note session mm-c hits 0 last - private untrusted
example.org says the launch moved.
```
Two spaces before the text line. `hits` and `last` for identity are always `0` and `-`.
For a note, `hits` is how many `recalls/*.jsonl` lines contain a hit with
`kind == "noted"`, `session == note.session`, `record ==` that note's index in
`notes::parse` (from 0). `last` is the maximum `time` string among those lines, or `-`.
Class is `public` / `private` / `secret` (`notes::class_name`). Trust is `untrusted` when
`Kind::Noted { untrusted: true }`, else `trusted`. Moved/dropped notes: still list them as
`note` using the same line, class `private`, `trusted` (they have no untrusted flag).
- **`recalls/`.** `store.path("recalls")`. Missing directory: all counts 0. A file that
exists but cannot be read is an error. Ignore a line that does not parse. Deserialize:
```rust
#[derive(Deserialize)]
struct Logged {
time: String,
hits: Vec<Hit>,
}
#[derive(Deserialize)]
struct Hit {
kind: String,
session: String,
record: i64,
}
```
(extra fields in the file are ignored: this is our format but old lines may grow; the given
tests use exactly those fields plus `session` / `query` / `epoch` on the line. Use
`deny_unknown_fields` **off** so `epoch` and `query` do not fail.)
- **Identity.** `store.entries(MemoryFile::Memory)` then `User`. Empty file: no identity
lines for that file.
- **`list` returns `Result<String, String>`.** Trailing newline on the last block is fine;
`bxctl` adds another when printing `MemoryAdminReply.text` (`writeln!`). The given bxctl
test includes a trailing newline in the reply text and expects it plus `writeln!`'s
newline.
- **`admin` arm.** `MemoryCommand::Tiers => tiers::list(&self.store, now)` — `now` is unused
by `list` today; keep the signature `list(store, now)` so the given tests compile.
- **`bxctl`.** `["tiers"] => Ask::Admin(MemoryCommand::Tiers, String::new())`. `["tiers", _]`
is usage. USAGE: add `| tiers` on the `bxctl memory` line.
- rustfmt `mod` order in `memoryd/src/lib.rs`.
- The items the edits use. This table is complete.
| Item | What it does |
|---|---|
| `Store::entries` | identity |
| `notes::parse` / `class_name` | notes |
| `Store::path("recalls")` | the jsonl directory |
| `MemoryFile::name` | `memory.md` / `user.md` |
## The function
`crates/memoryd/src/tiers.rs` — fill `list`. No `todo!()`.
```rust
//! The tiering listing: identity entries and notes with recall-hit counts (M5d spec, 4.4).
use bxcore::{MemoryFile, Timestamp};
use serde::Deserialize;
use crate::files::Store;
use crate::notes::{self, Kind};
#[derive(Deserialize)]
struct Logged {
time: String,
hits: Vec<Hit>,
}
#[derive(Deserialize)]
struct Hit {
kind: String,
session: String,
record: i64,
}
/// One listing for `bxctl memory tiers` and for the tiering job's user text.
pub fn list(store: &Store, _now: Timestamp) -> Result<String, String> {
let _ = store;
Ok(String::new())
}
```
Replace the stub body in this task.
### `crates/bxcore/src/memory.rs`
**M1.** After `Sync,` in `MemoryCommand`:
```rust
/// Identity entries and notes with recall-hit counts (M5d spec, section 4.4).
Tiers,
```
### `crates/memoryd/src/serve/admin.rs`
**A1.** After the `Sync` arm, before the closing of the `match`:
```rust
MemoryCommand::Tiers => crate::tiers::list(&self.store, now),
```
### `crates/bxctl/src/memory.rs`
**B1.** After `["sync"] => ...`, before `_ => return Err(UsageError)`:
```rust
["tiers"] => Ask::Admin(MemoryCommand::Tiers, String::new()),
```
### `crates/bxctl/src/cli.rs`
**U1.** The `bxctl memory` USAGE line. The line
```text
bxctl memory log [N] | undo <id> | pending | accept <id> | reject <id> | sync [--memory-socket PATH]
```
becomes
```text
bxctl memory log [N] | undo <id> | pending | accept <id> | reject <id> | sync | tiers [--memory-socket PATH]
```
## Steps
- [ ] **1. Copy the given files.**
```sh
cp docs/plans/M5d2/files/crates/memoryd/tests/tiers.rs crates/memoryd/tests/
cp docs/plans/M5d2/files/crates/bxctl/tests/cli_memory_tiers.rs crates/bxctl/tests/
mkdir -p config/jobs
cp docs/plans/M5d2/files/config/jobs/tier.toml.example config/jobs/tier.toml.example
```
- [ ] **2. See them fail.** `cargo build --tests --keep-going -p bxcore -p memoryd -p bxctl`,
timeout 600000 ms: `tiers` does not compile (`E0432` unresolved `memoryd::tiers`);
`cli_memory_tiers` does not compile (`E0599` no variant `Tiers`). After M1 alone,
`admin.rs` fails (`non-exhaustive patterns`). Do M1, A1, `tiers.rs`, B1, U1 together in
step 3. This step is the first build against the given tests.
- [ ] **3. The edits.** M1, `tiers.rs` with a real `list`, `pub mod tiers;`, A1, B1, U1.
`cargo check -p bxcore -p memoryd -p bxctl` prints no error and no warning.
`wc -l crates/memoryd/src/tiers.rs` prints a number ≤ 500.
- [ ] **4. The tests.** `cargo test -p memoryd --test tiers` (4) and
`cargo test -p bxctl --test cli_memory_tiers --test cli_memory --test cli_memory_sync`
(2 new): every test passes.
- [ ] **5. Check.** `git diff --stat` lists the files in "Every place that changes" except
the new tests and starter (those are untracked until `git add`).
- [ ] **6. Run the gate.** `cargo fmt --all`, then `make gate`, timeout 600000 ms. Last line
`gate: ok`. Test count **1572** (1566 before, plus 6).
- [ ] **7. Log and commit.** Task cell `M5d2/10-memoryd-tiers`. Stage with
`git add crates/bxcore crates/memoryd crates/bxctl config/jobs docs/implementer-log.md
Cargo.lock`, then commit as `AGENTS.md` says.
## Done when
- `make gate` prints `gate: ok` with 1572 tests.
- `git diff HEAD~1 --stat` lists only the files this task names plus the log.
## Stop and report if
- The branch check fails; a "before" block is not found exactly once; step 2 differs; `cargo
check` warns; a test fails; the gate count is not 1572; you edit `cli_memory.rs`; identity
entries show a non-zero hit count; untrusted notes are omitted; you add a cron crate; you
special-case the name `tier` in `loopd`.
## What the given tests pin
Identity lines are `hits 0 last -` with the entry text; notes join `recalls/` on
`(noted, session, record)` and take the latest `time`; untrusted notes are listed and marked;
a hit on note 1 does not count for note 0; `bxctl memory tiers` parses and prints the reply.
docs/plans/M5d2/README.md
+132
View File
@@ -0,0 +1,132 @@
# M5d-2 implementation plan: the scheduler
> For the implementer: each task file is one closed task for one fresh session. Read `AGENTS.md`,
> then the task file, and do exactly that task. The driver is `tools/run-plan.sh`.
**Goal:** heartbeat and cron run as isolated sessions on the background slot; live turns use the
session's slot; opted-in tool results get a `ToolSummary` that `messages()` ignores and `recall`
indexes; the tiering listing is an ordinary admin command. Answers stay in the session log.
Nothing is posted to Mattermost.
**Architecture:** `run_turn` reads `session.slot()`. `loopd` gains `[heartbeat]`, a cron matcher,
job-file loading, a scheduler thread that shares `Context` and the busy set, and a no-tools
summary call for `http_fetch`. `bxcore` gains `LogRecord::ToolSummary` and `MemoryCommand::Tiers`.
`memoryd` indexes summaries and answers `Tiers` from `recalls/`. `bxctl memory tiers` prints that
listing. No new crate.
**Tech stack:** Rust 1.98.1, edition 2024; no cron crate; no new dependency.
**Spec:** `docs/specs/2026-10-03-m5d-scheduler.md`, sections 4, 5, 6, 7 (the M5d-2 tests), and 8
(`jobs-invalid`, `heartbeat-prompt-unreadable`). M5d-1 (epochs and compaction) is already on the
branch this plan starts from.
## The owner's decisions (2026-10-03)
1. Heartbeat and cron answers stay in the session log. They are not posted to Mattermost.
2. Five-field UTC crontab, `{home}/jobs/<name>.toml`, a new session each run, a tick skipped
when that job is still live.
3. Tool summaries on the background slot; a `ToolSummary` log record with the result's labels,
FTS-indexed; `http_fetch` opts in via `[compact] summarise`.
## What the code already has, and what it lacks
- M5d-1 writes epochs, compact after a turn, Honcho over every epoch file. Live turns still send
`slots.main`. `[compact] summarise` is stored and unused. Unknown names are not a config error.
- `LogRecord` has no `ToolSummary`. `messages()` names every variant. `support/turn.rs` `types`
is exhaustive.
- `loopd serve` binds the channel and blocks in `channel::serve`. There is no scheduler thread.
- `MemoryCommand` has no `Tiers`. `bxctl memory` has no `tiers`.
- `SessionId` is `[a-z0-9-]`, max 64. Job ids `job-<name>-<yyyymmdd>t<hhmm>` must fit.
## Global constraints
- Roles depend on `bxcore` only. No `unsafe`, no async runtime, no new crate.
- No file the implementer reads holds a chat-template tag whole (T33).
- Every path a given test creates is removed by a guard (T44); no test depends on timing (T27):
the scheduler is `tick(now)`, never a sleep.
- Fail-closed states `jobs-invalid` and `heartbeat-prompt-unreadable` have runbook entries,
added by the task that adds their pointer (T49).
- Each task gives its baseline and its gate count (T50, T53); the branch line is a check, never
`git switch` (T51).
- `Cargo.lock` is in the `git add` line (T24). Gate timeout 600000 ms (T40).
- A later task that changes a shared given file ships its own copy (T19): task 02 replaces
`crates/loopd/tests/support/turn.rs` so `types` names `ToolSummary`.
## Tasks
Every task below has been written to be replayed from its text. The gate count is the passed
total of `cargo test --workspace` plus one, for the ignored `end_to_end` test the gate runs on
its own. Before task 01 it is **1507** (talos, rustc 1.98.1; M5d-1 task 08).
| # | Delivers | Crates | Gate after | Tests added |
|---|---|---|---|---|
| 01 | `run_turn` sends `session.slot()`; a session created on the background slot has `id_slot` of that slot; `mm-*` stays main | loopd | 1511 | 4 |
| 02 | `LogRecord::ToolSummary`; `messages()` ignores it; `types` names it; JSON round-trip | bxcore, loopd | 1517 | 6 |
| 03 | `[heartbeat]` (`every_s` 0, `prompt` `heartbeat.md`); unknown `[compact] summarise` names are a config error; starter `heartbeat.md` | loopd | 1525 | 8 |
| 04 | Five-field UTC cron matcher: `*`, `n`, `n-m`, `*/n`; day-of-week 0 is Sunday | loopd | 1535 | 10 |
| 05 | `{home}/jobs/<name>.toml`, `runs.jsonl`, `jobs-invalid` pointer and runbook | loopd | 1543 | 8 |
| 06 | Heartbeat tick: time line, standing session `heartbeat`, skip if busy, unreadable prompt logged once | loopd | 1549 | 6 |
| 07 | Cron tick: new session per run, skip if previous live, leftover `started` becomes `failed` | loopd | 1555 | 6 |
| 08 | After a turn, opted-in `ToolResult` without a `ToolSummary` gets a background summary appended with the result's labels | loopd | 1561 | 6 |
| 09 | `memoryd` indexes `ToolSummary` as kind `summary`; `recall` finds it and taints when untrusted | memoryd | 1566 | 5 |
| 10 | `MemoryCommand::Tiers`, `bxctl memory tiers`, starter `config/jobs/tier.toml.example` | bxcore, memoryd, bxctl | 1572 | 6 |
After task 10, the design model runs the M5d-2 device check (spec 7) on straylight: a heartbeat
turn on the background slot leaves a following main-slot turn's `cache_n` near its previous
total; a `jobs/tier.toml` for the check promotes only a trusted note.
## Review Focus
These are implied by the spec and pinned in the named task. A reviewer who only reads the
happy-path tests will miss them.
- A live `mm-*` turn still sends `slots.main` after heartbeat used the background slot (task 01).
- Inserting a `ToolSummary` between a `ToolResult` and a later `User` does not break prefix
extension (task 02).
- An unknown `[compact] summarise` name is a config error; an empty list is not (task 03).
- DOM and DOW are AND; `@daily` and a sixth field are errors (task 04).
- A leftover `started` whose session is currently held is not closed as `failed` (tasks 06, 07).
- `finish_turn` summarises before it compacts, so the `ToolResult` is still in the current epoch
(task 08).
- A secret `ToolSummary` is skipped when the index is private (task 09).
- Untrusted notes appear in `Tiers` and are marked; they cannot be omitted (task 10).
## Known limits
These are known, accepted for M5d-2, and not bugs to fix in a task:
- **No Mattermost delivery.** Heartbeat and cron answers are in `sessions/<id>/` only. The owner
reads them with `bxctl chat --session` or by opening the log.
- **DOM and DOW are AND.** Both restricted (`0 7 1 * 0`) matches only a Sunday that is also the
1st. Standard Vixie cron would OR them. The spec does not ask for OR.
- **Tool-summary inference runs inside `finish_turn`, before compaction, while the session is
still held.** That is how compaction waits for the summary, and how `missing` still sees the
current epoch's `ToolResult`. A second live turn on that session waits. The summary request
itself uses the background slot and omits `tools`. A later split that infers without the hold
is out of scope.
- **Job files are checked at start.** A file that becomes invalid after `loopd` is running is
skipped that tick and logged; it does not stop the process.
- **Identity entries in `Tiers` list with 0 hits** unless a `recalls/` line names that exact
index row. Notes carry counts from `recalls/`. That is enough for the tiering prompt.
## Before hand-over
Work through `docs/plans/CHECKLIST.md` with a reference in a scratch worktree with its own build
directory, replayed from its task text; mutations; `/tmp` compared around a gate run; the
reference moved to straylight as a bundle before the run (T18). Task files name their log row
(T47).
## Running it
The reference (`m5d2-ref`, tasks 01 to 10) is moved to straylight as a bundle and deleted here
before the run (T18). Then, from the repository root, with the OpenCode TUI closed, after M5d-1
has landed on master:
```sh
git switch -c m5d2 master
OPENCODE_MODEL=straylight/ornith-1.5-35b-a3b RESUME=1 \
SLOTS_URL='http://straylight:11434/slots?model=ornith-1.5-35b-a3b' \
tools/run-plan.sh docs/plans/M5d2 01
```
After task 10, the gate count is 1572. Then the device check above.
docs/plans/M5d2/files/config/heartbeat.md
+4
View File
@@ -0,0 +1,4 @@
It is time for a heartbeat.
Look at recent notes and the current identity. Write a note only if something needs the owner, or
if a standing fact should move. Do not use tools that fetch. Do not address the owner.
docs/plans/M5d2/files/config/jobs/tier.toml.example
+6
View File
@@ -0,0 +1,6 @@
# Copy to {BOXMAKER_HOME}/jobs/tier.toml and edit the schedule.
# Five-field UTC crontab. This example is Sunday 07:00.
schedule = "0 7 * * 0"
prompt = """Read the listing in this message. Promote only trusted notes into identity
(memory.md / user.md) with add or consolidate. Do not promote an untrusted note.
Do not fetch. Write a note of what you changed, or that nothing needed changing."""
docs/plans/M5d2/files/config/summarise.md
+2
View File
@@ -0,0 +1,2 @@
Summarise this tool result for later recall. Keep facts, names, numbers and URLs. Drop boilerplate
and navigation. Do not use tools. Do not address the owner. Do not invent facts.
docs/plans/M5d2/files/crates/bxcore/tests/tool_summary.rs
+52
View File
@@ -0,0 +1,52 @@
//! Tests for `LogRecord::ToolSummary`. Do not edit.
use bxcore::{CallId, DataClass, LogRecord, Timestamp};
fn ts() -> Timestamp {
Timestamp::parse("2026-09-18T08:00:00.000Z").unwrap()
}
#[test]
fn tool_summary_round_trips() {
let rec = LogRecord::ToolSummary {
time: ts(),
call: CallId(3),
tool_call_id: "call_x".to_string(),
content: "the page is about tea".to_string(),
class: DataClass::Public,
untrusted: true,
};
let line = serde_json::to_string(&rec).unwrap();
assert!(line.contains("\"type\":\"tool_summary\""), "{line}");
let back: LogRecord = serde_json::from_str(&line).unwrap();
assert_eq!(back, rec);
}
#[test]
fn unknown_fields_are_refused() {
let line = r#"{"type":"tool_summary","time":"2026-09-18T08:00:00.000Z","call":3,"tool_call_id":"c","content":"x","class":"public","untrusted":false,"extra":1}"#;
assert!(serde_json::from_str::<LogRecord>(line).is_err());
}
#[test]
fn field_order_is_the_struct_order() {
let rec = LogRecord::ToolSummary {
time: ts(),
call: CallId(1),
tool_call_id: "c".to_string(),
content: "x".to_string(),
class: DataClass::Private,
untrusted: false,
};
let line = serde_json::to_string(&rec).unwrap();
let time = line.find("\"time\"").unwrap();
let call = line.find("\"call\"").unwrap();
let id = line.find("\"tool_call_id\"").unwrap();
let content = line.find("\"content\"").unwrap();
let class = line.find("\"class\"").unwrap();
let untrusted = line.find("\"untrusted\"").unwrap();
assert!(
time < call && call < id && id < content && content < class && class < untrusted,
"{line}"
);
}
docs/plans/M5d2/files/crates/bxctl/tests/cli_memory_tiers.rs
+86
View File
@@ -0,0 +1,86 @@
//! Tests for `bxctl memory tiers`. Do not edit.
#[path = "support/temp.rs"]
mod temp;
use std::os::unix::net::UnixListener;
use std::path::{Path, PathBuf};
use std::sync::mpsc;
use std::time::Duration;
use bxcore::{Envelope, MemoryAdminReply, MemoryCommand, Message, read_frame, write_frame};
use bxctl::cli::{Command, UsageError, parse};
use bxctl::memory::{Ask, run};
fn args(line: &str) -> Vec<String> {
line.split_whitespace().map(str::to_string).collect()
}
fn home() -> PathBuf {
PathBuf::from("/srv/bx")
}
fn fake(socket: &Path, answer: Message) -> mpsc::Receiver<Message> {
let listener = UnixListener::bind(socket).unwrap();
let (tx, rx) = mpsc::channel();
std::thread::spawn(move || {
let (mut stream, _) = listener.accept().unwrap();
let env = read_frame(&mut stream).unwrap();
tx.send(env.msg).unwrap();
let reply = Envelope {
v: 1,
id: env.id,
r#final: true,
msg: answer,
};
write_frame(&mut stream, &reply).unwrap();
});
rx
}
#[test]
fn memory_tiers_parses() {
let h = home();
assert_eq!(
parse(&args("memory tiers"), &h),
Ok(Command::Memory {
socket: PathBuf::from("/srv/bx/run/owner-memory/admin.sock"),
ask: Ask::Admin(MemoryCommand::Tiers, String::new())
})
);
assert_eq!(
parse(&args("memory --memory-socket /r/m.sock tiers"), &h),
Ok(Command::Memory {
socket: PathBuf::from("/r/m.sock"),
ask: Ask::Admin(MemoryCommand::Tiers, String::new())
})
);
assert_eq!(parse(&args("memory tiers extra"), &h), Err(UsageError));
}
#[test]
fn memory_tiers_prints_the_reply() {
let d = temp::TempDir::new("bx-tiers");
let socket = d.join("admin.sock");
let listing = "identity memory.md hits 0 last -\n I like tea.\n";
let seen = fake(
&socket,
Message::MemoryAdminReply(MemoryAdminReply {
text: listing.to_string(),
}),
);
let mut out = Vec::new();
run(
&socket,
&Ask::Admin(MemoryCommand::Tiers, String::new()),
Duration::from_secs(5),
&mut out,
)
.unwrap();
assert_eq!(String::from_utf8(out).unwrap(), format!("{listing}\n"));
match seen.recv().unwrap() {
Message::MemoryAdmin(m) => {
assert_eq!((m.command, m.arg.as_str()), (MemoryCommand::Tiers, ""))
}
other => panic!("{other:?}"),
}
}
docs/plans/M5d2/files/crates/loopd/tests/config_heartbeat.rs
+110
View File
@@ -0,0 +1,110 @@
//! Tests for `[heartbeat]` and unknown `[compact] summarise` names. Do not edit.
use loopd::config::Config;
use loopd::tools::Registry;
use std::path::{Path, PathBuf};
fn fixture(name: &str) -> PathBuf {
Path::new(env!("CARGO_MANIFEST_DIR"))
.join("tests/fixtures/config")
.join(name)
}
fn minimal() -> String {
std::fs::read_to_string(fixture("minimal.toml")).unwrap()
}
struct Scratch {
dir: PathBuf,
}
impl Scratch {
fn new() -> Scratch {
let n = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_nanos();
let dir = std::env::temp_dir().join(format!("loopd-hb-{}-{n}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
Scratch { dir }
}
}
impl Drop for Scratch {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.dir);
}
}
#[test]
fn missing_heartbeat_table_is_disabled() {
let c = Config::load(&fixture("minimal.toml")).unwrap();
assert_eq!(c.heartbeat.every_s, 0);
assert_eq!(
c.heartbeat_prompt(),
fixture("minimal.toml")
.parent()
.unwrap()
.join("heartbeat.md")
);
}
#[test]
fn the_table_overrides_every_default() {
let text = format!(
"{}\n[heartbeat]\nevery_s = 60\nprompt = \"pulse.md\"\n",
minimal()
);
let c = Config::parse(&text).unwrap();
assert_eq!(c.heartbeat.every_s, 60);
assert_eq!(c.heartbeat.prompt, PathBuf::from("pulse.md"));
}
#[test]
fn unknown_keys_are_errors() {
let bad = format!("{}\n[heartbeat]\nzz_unknown = 1\n", minimal());
assert!(Config::parse(&bad).is_err());
}
#[test]
fn heartbeat_prompt_is_beside_the_config_file() {
let home = Scratch::new();
let cfg_path = home.dir.join("loopd.toml");
std::fs::write(&cfg_path, minimal()).unwrap();
std::fs::write(home.dir.join("system.md"), "sys\n").unwrap();
let c = Config::load(&cfg_path).unwrap();
assert_eq!(c.heartbeat_prompt(), home.dir.join("heartbeat.md"));
}
#[test]
fn default_summarise_http_fetch_is_known_to_m5c() {
let c = Config::parse(&minimal()).unwrap();
c.check_summarise(&Registry::m5c()).unwrap();
}
#[test]
fn an_unknown_summarise_name_is_an_error() {
let text = format!("{}\n[compact]\nsummarise = [\"no_such_tool\"]\n", minimal());
let c = Config::parse(&text).unwrap();
let err = c.check_summarise(&Registry::m5c()).unwrap_err();
assert!(err.contains("no_such_tool"), "{err}");
assert!(err.contains("summarise"), "{err}");
}
#[test]
fn empty_summarise_is_allowed() {
let text = format!("{}\n[compact]\nsummarise = []\n", minimal());
let c = Config::parse(&text).unwrap();
c.check_summarise(&Registry::m5c()).unwrap();
}
#[test]
fn http_fetch_is_known_and_clock_is_known() {
let text = format!(
"{}\n[compact]\nsummarise = [\"http_fetch\", \"clock\"]\n",
minimal()
);
let c = Config::parse(&text).unwrap();
c.check_summarise(&Registry::m5c()).unwrap();
}
docs/plans/M5d2/files/crates/loopd/tests/cron.rs
+85
View File
@@ -0,0 +1,85 @@
//! Tests for the five-field UTC cron matcher. Do not edit.
use bxcore::Timestamp;
use loopd::cron::{self, CronError};
fn ts(s: &str) -> Timestamp {
Timestamp::parse(s).unwrap()
}
fn due(schedule: &str, at: &str) -> bool {
cron::due(schedule, ts(at)).unwrap()
}
#[test]
fn a_daily_hour_matches_that_minute_only() {
assert!(due("0 7 * * *", "2026-10-03T07:00:00.000Z"));
assert!(!due("0 7 * * *", "2026-10-03T07:01:00.000Z"));
assert!(!due("0 7 * * *", "2026-10-03T06:00:00.000Z"));
}
#[test]
fn every_minute_matches() {
assert!(due("* * * * *", "2026-10-03T00:00:00.000Z"));
assert!(due("* * * * *", "2026-10-03T23:59:00.000Z"));
}
#[test]
fn a_step_matches_multiples_from_zero() {
assert!(due("*/5 * * * *", "2026-10-03T00:00:00.000Z"));
assert!(due("*/5 * * * *", "2026-10-03T00:05:00.000Z"));
assert!(!due("*/5 * * * *", "2026-10-03T00:06:00.000Z"));
}
#[test]
fn a_range_is_inclusive() {
assert!(due("0-10 * * * *", "2026-10-03T00:00:00.000Z"));
assert!(due("0-10 * * * *", "2026-10-03T00:10:00.000Z"));
assert!(!due("0-10 * * * *", "2026-10-03T00:11:00.000Z"));
}
#[test]
fn sunday_is_zero() {
// 2026-10-04 is a Sunday.
assert!(due("0 7 * * 0", "2026-10-04T07:00:00.000Z"));
// 2026-10-03 is a Saturday.
assert!(!due("0 7 * * 0", "2026-10-03T07:00:00.000Z"));
}
#[test]
fn the_first_of_the_month() {
assert!(due("0 7 1 * *", "2026-10-01T07:00:00.000Z"));
assert!(!due("0 7 1 * *", "2026-10-02T07:00:00.000Z"));
}
#[test]
fn four_or_six_fields_are_errors() {
assert!(matches!(
cron::due("* * * *", ts("2026-10-03T00:00:00.000Z")),
Err(CronError::Fields(_))
));
assert!(matches!(
cron::due("* * * * * *", ts("2026-10-03T00:00:00.000Z")),
Err(CronError::Fields(_))
));
}
#[test]
fn an_out_of_range_number_is_an_error() {
assert!(cron::due("60 * * * *", ts("2026-10-03T00:00:00.000Z")).is_err());
assert!(cron::due("* 24 * * *", ts("2026-10-03T00:00:00.000Z")).is_err());
assert!(cron::due("* * * * 7", ts("2026-10-03T00:00:00.000Z")).is_err());
}
#[test]
fn nicknames_are_errors() {
let err = cron::due("@daily", ts("2026-10-03T00:00:00.000Z")).unwrap_err();
assert!(matches!(err, CronError::Fields(_)), "{err:?}");
}
#[test]
fn display_names_the_schedule() {
let err = cron::due("x x x x x", ts("2026-10-03T00:00:00.000Z")).unwrap_err();
let s = err.to_string();
assert!(s.contains("x x x x x"), "{s}");
}
docs/plans/M5d2/files/crates/loopd/tests/jobs.rs
+151
View File
@@ -0,0 +1,151 @@
//! Tests for job files and the run ledger. Do not edit.
use bxcore::{SessionId, Timestamp};
use loopd::jobs::{self, JobsError, RunStatus};
fn ts() -> Timestamp {
Timestamp::parse("2026-10-03T07:00:00.000Z").unwrap()
}
struct Scratch {
dir: std::path::PathBuf,
}
impl Scratch {
fn new() -> Scratch {
let n = std::time::SystemTime::now()
.duration_since(std::time::UNIX_EPOCH)
.unwrap()
.as_nanos();
let dir = std::env::temp_dir().join(format!("loopd-jobs-{}-{n}", std::process::id()));
let _ = std::fs::remove_dir_all(&dir);
std::fs::create_dir_all(&dir).unwrap();
Scratch { dir }
}
}
impl Drop for Scratch {
fn drop(&mut self) {
let _ = std::fs::remove_dir_all(&self.dir);
}
}
#[test]
fn a_valid_job_file_loads() {
let home = Scratch::new();
let jobs = home.dir.join("jobs");
std::fs::create_dir(&jobs).unwrap();
std::fs::write(
jobs.join("mail.toml"),
"schedule = \"0 7 * * *\"\nprompt = \"Check mail.\"\n",
)
.unwrap();
let loaded = jobs::load(&jobs).unwrap();
assert_eq!(loaded.len(), 1);
assert_eq!(loaded[0].name, "mail");
assert_eq!(loaded[0].schedule, "0 7 * * *");
assert_eq!(loaded[0].prompt, "Check mail.");
}
#[test]
fn unknown_keys_are_errors_with_the_runbook() {
let home = Scratch::new();
let jobs = home.dir.join("jobs");
std::fs::create_dir(&jobs).unwrap();
std::fs::write(
jobs.join("mail.toml"),
"schedule = \"* * * * *\"\nprompt = \"x\"\nwhen = \"now\"\n",
)
.unwrap();
let err = jobs::load(&jobs).unwrap_err();
let s = err.to_string();
assert!(s.contains("mail.toml"), "{s}");
assert!(s.ends_with("see docs/runbook.md#jobs-invalid"), "{s}");
}
#[test]
fn a_bad_name_is_an_error() {
let home = Scratch::new();
let jobs = home.dir.join("jobs");
std::fs::create_dir(&jobs).unwrap();
std::fs::write(
jobs.join("Mail.toml"),
"schedule = \"* * * * *\"\nprompt = \"x\"\n",
)
.unwrap();
let err = jobs::load(&jobs).unwrap_err();
assert!(matches!(err, JobsError::Name(_)), "{err:?}");
}
#[test]
fn session_id_fits_and_is_lowercase() {
let id = jobs::session_id("mail", ts()).unwrap();
assert_eq!(id.as_str(), "job-mail-20261003t0700");
SessionId::new(id.as_str()).unwrap();
}
#[test]
fn a_name_too_long_for_a_session_id_is_an_error() {
let long = "n".repeat(50);
let err = jobs::session_id(&long, ts()).unwrap_err();
assert!(matches!(err, JobsError::Name(_)), "{err:?}");
}
#[test]
fn a_missing_jobs_directory_is_empty() {
let home = Scratch::new();
let loaded = jobs::load(&home.dir.join("jobs")).unwrap();
assert!(loaded.is_empty());
}
#[test]
fn leftover_started_is_closed_as_failed() {
let home = Scratch::new();
let path = home.dir.join("runs.jsonl");
let session = jobs::session_id("mail", ts()).unwrap();
jobs::append(
&path,
&jobs::Run {
time: ts(),
name: "mail".to_string(),
session: session.clone(),
status: RunStatus::Started,
},
)
.unwrap();
let n = jobs::close_leftovers(&path, ts()).unwrap();
assert_eq!(n, 1);
assert!(jobs::live_session(&path, "mail").unwrap().is_none());
}
#[test]
fn a_started_run_without_a_close_is_live() {
let home = Scratch::new();
let path = home.dir.join("runs.jsonl");
let session = jobs::session_id("mail", ts()).unwrap();
jobs::append(
&path,
&jobs::Run {
time: ts(),
name: "mail".to_string(),
session: session.clone(),
status: RunStatus::Started,
},
)
.unwrap();
assert_eq!(
jobs::live_session(&path, "mail").unwrap().as_ref(),
Some(&session)
);
jobs::append(
&path,
&jobs::Run {
time: ts(),
name: "mail".to_string(),
session,
status: RunStatus::Finished,
},
)
.unwrap();
assert!(jobs::live_session(&path, "mail").unwrap().is_none());
}
docs/plans/M5d2/files/crates/loopd/tests/scheduler_cron.rs
+186
View File
@@ -0,0 +1,186 @@
//! Tests for the cron tick. Do not edit.
mod support;
use bxcore::{SessionId, Timestamp};
use loopd::channel::Context;
use loopd::jobs::{self, RunStatus};
use loopd::llama::Client;
use loopd::memory::NoMemory;
use loopd::scheduler::{Scheduler, TickWork};
use loopd::tools::Registry;
use std::sync::Arc;
use support::{FakeServer, Home, Reply, ScriptedPort};
const CHAT: &str = "/v1/chat/completions";
fn sse() -> Reply {
let data1 = r#"data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"done"}}],"id":"j","timings":{"cache_n":0,"prompt_n":10,"predicted_n":1}}"#;
let data2 = r#"data: {"choices":[{"finish_reason":"stop","index":0,"delta":{}}],"id":"j","timings":{"cache_n":0,"prompt_n":10,"predicted_n":1}}"#;
let body = format!("{data1}\n\n{data2}\n\ndata: [DONE]\n\n");
Reply::raw(format!(
"HTTP/1.1 200 OK\r\nContent-Type: text/event-stream\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}",
body.len()
))
}
fn ts(s: &str) -> Timestamp {
Timestamp::parse(s).unwrap()
}
fn world(schedule: &str) -> (Home, FakeServer, Arc<Context>) {
let home = Home::new();
home.write("compact.md", "Summarise.\n");
home.write(
"jobs/mail.toml",
&format!("schedule = \"{schedule}\"\nprompt = \"Check mail.\"\n"),
);
let server = FakeServer::start();
let cfg = home.config(&server.socket);
server.route(CHAT, vec![sse(), sse()]);
let ctx = Arc::new(Context::new(
cfg.clone(),
Client::new(cfg),
Box::new(ScriptedPort::new(vec![])),
Registry::m2b(),
Box::new(NoMemory),
));
(home, server, ctx)
}
#[test]
fn a_due_job_starts_a_new_session_on_the_background_slot() {
let (home, server, ctx) = world("0 7 * * *");
let mut sched = Scheduler::new(ts("2026-10-03T06:00:00.000Z"));
let work = sched.tick(&ctx, ts("2026-10-03T07:00:00.000Z"));
let want = jobs::session_id("mail", ts("2026-10-03T07:00:00.000Z")).unwrap();
assert_eq!(
work,
vec![TickWork::Job {
name: "mail".to_string(),
session: want.clone()
}]
);
let sent = server.requests_to(CHAT);
assert_eq!(sent.len(), 1);
assert_eq!(sent[0].json()["id_slot"], 1);
let last = sent[0].json()["messages"]
.as_array()
.unwrap()
.last()
.unwrap();
let content = last["content"].as_str().unwrap();
assert!(
content.starts_with("It is 2026-10-03T07:00:00.000Z."),
"{content}"
);
assert!(content.contains("Check mail."), "{content}");
let s = loopd::session::Session::open(&home.dir, want).unwrap();
assert_eq!(s.slot(), 1);
}
#[test]
fn a_job_not_due_does_not_run() {
let (_home, server, ctx) = world("0 7 * * *");
let mut sched = Scheduler::new(ts("2026-10-03T06:00:00.000Z"));
let work = sched.tick(&ctx, ts("2026-10-03T06:59:00.000Z"));
assert!(work.is_empty());
assert!(server.requests_to(CHAT).is_empty());
}
#[test]
fn a_live_previous_run_skips_the_tick() {
let (home, server, ctx) = world("* * * * *");
let ledger = home.dir.join("jobs/runs.jsonl");
let previous = jobs::session_id("mail", ts("2026-10-03T06:59:00.000Z")).unwrap();
jobs::append(
&ledger,
&jobs::Run {
time: ts("2026-10-03T06:59:00.000Z"),
name: "mail".to_string(),
session: previous.clone(),
status: RunStatus::Started,
},
)
.unwrap();
let held = ctx.hold(previous).expect("hold");
let mut sched = Scheduler::new(ts("2026-10-03T06:00:00.000Z"));
let work = sched.tick(&ctx, ts("2026-10-03T07:00:00.000Z"));
assert_eq!(
work,
vec![TickWork::SkippedJob {
name: "mail".to_string()
}]
);
assert!(server.requests_to(CHAT).is_empty());
drop(held);
}
#[test]
fn leftover_started_is_failed_on_the_first_tick() {
let (home, _server, ctx) = world("* * * * *");
let ledger = home.dir.join("jobs/runs.jsonl");
let previous = jobs::session_id("mail", ts("2026-10-03T06:59:00.000Z")).unwrap();
jobs::append(
&ledger,
&jobs::Run {
time: ts("2026-10-03T06:59:00.000Z"),
name: "mail".to_string(),
session: previous.clone(),
status: RunStatus::Started,
},
)
.unwrap();
let mut sched = Scheduler::new(ts("2026-10-03T07:00:00.000Z"));
sched.tick(&ctx, ts("2026-10-03T07:00:00.000Z"));
let text = home.read("jobs/runs.jsonl");
assert!(
text.contains("failed"),
"leftover started was closed: {text}"
);
assert!(text.contains(previous.as_str()), "{text}");
}
#[test]
fn a_cron_turn_has_no_speaker() {
let (home, _server, ctx) = world("0 7 * * *");
let mut sched = Scheduler::new(ts("2026-10-03T06:00:00.000Z"));
sched.tick(&ctx, ts("2026-10-03T07:00:00.000Z"));
let id = jobs::session_id("mail", ts("2026-10-03T07:00:00.000Z")).unwrap();
let recs = {
let text = home.read(&format!("sessions/{}/0.jsonl", id.as_str()));
text.lines()
.map(|l| serde_json::from_str::<bxcore::LogRecord>(l).unwrap())
.collect::<Vec<_>>()
};
let user = recs
.iter()
.find_map(|r| match r {
bxcore::LogRecord::User { speaker, .. } => Some(speaker),
_ => None,
})
.unwrap();
assert_eq!(user, &None);
}
#[test]
fn two_due_minutes_make_two_sessions() {
let (home, server, ctx) = world("* * * * *");
let mut sched = Scheduler::new(ts("2026-10-03T06:58:00.000Z"));
sched.tick(&ctx, ts("2026-10-03T06:59:00.000Z"));
sched.tick(&ctx, ts("2026-10-03T07:00:00.000Z"));
assert_eq!(server.requests_to(CHAT).len(), 2);
let a = jobs::session_id("mail", ts("2026-10-03T06:59:00.000Z")).unwrap();
let b = jobs::session_id("mail", ts("2026-10-03T07:00:00.000Z")).unwrap();
assert_ne!(a, b);
assert!(
home.dir
.join(format!("sessions/{}/0.jsonl", a.as_str()))
.is_file()
);
assert!(
home.dir
.join(format!("sessions/{}/0.jsonl", b.as_str()))
.is_file()
);
}
docs/plans/M5d2/files/crates/loopd/tests/scheduler_heartbeat.rs
+148
View File
@@ -0,0 +1,148 @@
//! Tests for the heartbeat tick. Do not edit.
mod support;
use bxcore::{SessionId, Timestamp};
use loopd::channel::Context;
use loopd::llama::Client;
use loopd::memory::NoMemory;
use loopd::scheduler::{self, Scheduler, TickWork};
use loopd::tools::Registry;
use std::sync::Arc;
use support::{FakeServer, Home, Reply, ScriptedPort};
const CHAT: &str = "/v1/chat/completions";
const PROMPT: &str = "Look around.\n";
fn sse() -> Reply {
let data1 = r#"data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"noted"}}],"id":"h","timings":{"cache_n":0,"prompt_n":10,"predicted_n":1}}"#;
let data2 = r#"data: {"choices":[{"finish_reason":"stop","index":0,"delta":{}}],"id":"h","timings":{"cache_n":0,"prompt_n":10,"predicted_n":1}}"#;
let body = format!("{data1}\n\n{data2}\n\ndata: [DONE]\n\n");
Reply::raw(format!(
"HTTP/1.1 200 OK\r\nContent-Type: text/event-stream\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}",
body.len()
))
}
fn ts(s: &str) -> Timestamp {
Timestamp::parse(s).unwrap()
}
fn world(every_s: u64) -> (Home, FakeServer, Arc<Context>) {
let home = Home::new();
home.write("heartbeat.md", PROMPT);
home.write("compact.md", "Summarise.\n");
let server = FakeServer::start();
let mut cfg = home.config(&server.socket);
cfg.heartbeat.every_s = every_s;
cfg.heartbeat.prompt = home.dir.join("heartbeat.md");
server.route(CHAT, vec![sse()]);
let ctx = Arc::new(Context::new(
cfg.clone(),
Client::new(cfg),
Box::new(ScriptedPort::new(vec![])),
Registry::m2b(),
Box::new(NoMemory),
));
(home, server, ctx)
}
#[test]
fn every_s_zero_does_nothing() {
let (_home, server, ctx) = world(0);
let mut sched = Scheduler::new(ts("2026-10-03T07:00:00.000Z"));
let work = sched.tick(&ctx, ts("2026-10-03T08:00:00.000Z"));
assert!(work.is_empty());
assert!(server.requests_to(CHAT).is_empty());
}
#[test]
fn a_due_heartbeat_runs_on_the_background_slot_with_a_time_line() {
let (home, server, ctx) = world(60);
let mut sched = Scheduler::new(ts("2026-10-03T07:00:00.000Z"));
let work = sched.tick(&ctx, ts("2026-10-03T07:01:00.000Z"));
assert_eq!(work, vec![TickWork::Heartbeat]);
let sent = server.requests_to(CHAT);
assert_eq!(sent.len(), 1);
assert_eq!(sent[0].json()["id_slot"], 1);
let last = sent[0].json()["messages"]
.as_array()
.unwrap()
.last()
.unwrap();
assert_eq!(last["role"], "user");
let content = last["content"].as_str().unwrap();
assert!(
content.starts_with("It is 2026-10-03T07:01:00.000Z."),
"{content}"
);
assert!(content.contains("Look around."), "{content}");
let s = loopd::session::Session::open(&home.dir, SessionId::new("heartbeat").unwrap()).unwrap();
assert_eq!(s.slot(), 1);
}
#[test]
fn a_second_tick_before_every_s_does_not_run() {
let (_home, server, ctx) = world(60);
let mut sched = Scheduler::new(ts("2026-10-03T07:00:00.000Z"));
sched.tick(&ctx, ts("2026-10-03T07:01:00.000Z"));
let work = sched.tick(&ctx, ts("2026-10-03T07:01:30.000Z"));
assert!(work.is_empty());
assert_eq!(server.requests_to(CHAT).len(), 1);
}
#[test]
fn a_busy_heartbeat_session_is_skipped() {
let (_home, server, ctx) = world(60);
let held = ctx
.hold(SessionId::new("heartbeat").unwrap())
.expect("hold");
let mut sched = Scheduler::new(ts("2026-10-03T07:00:00.000Z"));
let work = sched.tick(&ctx, ts("2026-10-03T07:01:00.000Z"));
assert_eq!(
work,
vec![TickWork::SkippedBusy {
session: SessionId::new("heartbeat").unwrap()
}]
);
assert!(server.requests_to(CHAT).is_empty());
drop(held);
}
#[test]
fn an_unreadable_prompt_is_skipped_and_names_the_runbook() {
let home = Home::new();
let path = home.dir.join("heartbeat.md");
std::fs::create_dir(&path).unwrap();
let server = FakeServer::start();
let mut cfg = home.config(&server.socket);
cfg.heartbeat.every_s = 60;
cfg.heartbeat.prompt = path;
let ctx = Arc::new(Context::new(
cfg.clone(),
Client::new(cfg),
Box::new(ScriptedPort::new(vec![])),
Registry::m2b(),
Box::new(NoMemory),
));
let mut sched = Scheduler::new(ts("2026-10-03T07:00:00.000Z"));
let work = sched.tick(&ctx, ts("2026-10-03T07:01:00.000Z"));
assert_eq!(work, vec![TickWork::HeartbeatUnreadable]);
assert!(server.requests_to(CHAT).is_empty());
}
#[test]
fn heartbeat_has_no_speaker() {
let (home, _server, ctx) = world(60);
let mut sched = Scheduler::new(ts("2026-10-03T07:00:00.000Z"));
sched.tick(&ctx, ts("2026-10-03T07:01:00.000Z"));
let recs = home.records("heartbeat");
let user = recs
.iter()
.find_map(|r| match r {
bxcore::LogRecord::User { speaker, .. } => Some(speaker),
_ => None,
})
.unwrap();
assert_eq!(user, &None);
}
docs/plans/M5d2/files/crates/loopd/tests/session_slot.rs
+170
View File
@@ -0,0 +1,170 @@
//! Tests that a live turn uses the session's slot. Do not edit.
mod support;
use bxcore::SessionId;
use loopd::baseline::Baseline;
use loopd::llama::Client;
use loopd::session::Session;
use loopd::tools::Registry;
use loopd::turn::{Runtime, run_turn};
use support::{FakeServer, Home, Reply};
const CHAT: &str = "/v1/chat/completions";
fn sse() -> Reply {
let data1 = r#"data: {"choices":[{"finish_reason":null,"index":0,"delta":{"content":"ok"}}],"id":"t","timings":{"cache_n":0,"prompt_n":10,"predicted_n":1}}"#;
let data2 = r#"data: {"choices":[{"finish_reason":"stop","index":0,"delta":{}}],"id":"t","timings":{"cache_n":0,"prompt_n":10,"predicted_n":1}}"#;
let body = format!("{data1}\n\n{data2}\n\ndata: [DONE]\n\n");
Reply::raw(format!(
"HTTP/1.1 200 OK\r\nContent-Type: text/event-stream\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}",
body.len()
))
}
fn runtime<'a>(
cfg: &'a loopd::config::Config,
client: &'a Client,
port: &'a support::ScriptedPort,
registry: &'a Registry,
) -> Runtime<'a> {
Runtime {
cfg,
client,
port,
registry,
}
}
#[test]
fn a_session_created_on_the_background_slot_sends_that_slot() {
let home = Home::new();
let server = FakeServer::start();
let cfg = home.config(&server.socket);
server.route(CHAT, vec![sse()]);
let client = Client::new(cfg.clone());
let port = support::ScriptedPort::new(vec![]);
let registry = Registry::m2b();
let baseline = Baseline::assemble(&cfg, &registry).unwrap();
let mut s = Session::create(
&home.dir,
SessionId::new("heartbeat").unwrap(),
baseline,
cfg.slots.background,
)
.unwrap();
let mut events = Vec::new();
run_turn(
&mut s,
&runtime(&cfg, &client, &port, &registry),
"pulse",
None,
&mut |e| events.push(e.clone()),
)
.unwrap();
let sent = server.requests_to(CHAT);
assert_eq!(sent.len(), 1);
assert_eq!(sent[0].json()["id_slot"], cfg.slots.background);
}
#[test]
fn a_main_slot_session_still_sends_main() {
let home = Home::new();
let server = FakeServer::start();
let cfg = home.config(&server.socket);
server.route(CHAT, vec![sse()]);
let client = Client::new(cfg.clone());
let port = support::ScriptedPort::new(vec![]);
let registry = Registry::m2b();
let baseline = Baseline::assemble(&cfg, &registry).unwrap();
let mut s = Session::create(
&home.dir,
SessionId::new("mm-thread").unwrap(),
baseline,
cfg.slots.main,
)
.unwrap();
let mut events = Vec::new();
run_turn(
&mut s,
&runtime(&cfg, &client, &port, &registry),
"hi",
None,
&mut |e| events.push(e.clone()),
)
.unwrap();
assert_eq!(
server.requests_to(CHAT)[0].json()["id_slot"],
cfg.slots.main
);
}
#[test]
fn the_request_slot_follows_session_start_not_the_config_main() {
let home = Home::new();
let server = FakeServer::start();
let mut cfg = home.config(&server.socket);
cfg.slots.main = 0;
cfg.slots.background = 1;
server.route(CHAT, vec![sse()]);
let client = Client::new(cfg.clone());
let port = support::ScriptedPort::new(vec![]);
let registry = Registry::m2b();
let baseline = Baseline::assemble(&cfg, &registry).unwrap();
let mut s = Session::create(&home.dir, SessionId::new("a").unwrap(), baseline, 7).unwrap();
let mut events = Vec::new();
run_turn(
&mut s,
&runtime(&cfg, &client, &port, &registry),
"x",
None,
&mut |e| events.push(e.clone()),
)
.unwrap();
assert_eq!(server.requests_to(CHAT)[0].json()["id_slot"], 7);
}
#[test]
fn a_heartbeat_turn_between_two_main_turns_does_not_change_the_main_prefix() {
let home = Home::new();
let server = FakeServer::start();
let cfg = home.config(&server.socket);
server.route(CHAT, vec![sse(), sse(), sse()]);
let client = Client::new(cfg.clone());
let port = support::ScriptedPort::new(vec![]);
let registry = Registry::m2b();
let mut main = Session::create(
&home.dir,
SessionId::new("mm-a").unwrap(),
Baseline::assemble(&cfg, &registry).unwrap(),
cfg.slots.main,
)
.unwrap();
let mut beat = Session::create(
&home.dir,
SessionId::new("heartbeat").unwrap(),
Baseline::assemble(&cfg, &registry).unwrap(),
cfg.slots.background,
)
.unwrap();
let rt = runtime(&cfg, &client, &port, &registry);
let mut events = Vec::new();
run_turn(&mut main, &rt, "one", None, &mut |e| events.push(e.clone())).unwrap();
run_turn(&mut beat, &rt, "pulse", None, &mut |e| {
events.push(e.clone())
})
.unwrap();
run_turn(&mut main, &rt, "two", None, &mut |e| events.push(e.clone())).unwrap();
let sent = server.requests_to(CHAT);
assert_eq!(sent.len(), 3);
assert_eq!(sent[0].json()["id_slot"], 0);
assert_eq!(sent[1].json()["id_slot"], 1);
assert_eq!(sent[2].json()["id_slot"], 0);
let first = sent[0].json()["messages"].as_array().unwrap().clone();
let third = sent[2].json()["messages"].as_array().unwrap();
assert!(
third.len() > first.len(),
"the main session's second turn is a longer prefix"
);
assert_eq!(&third[..first.len()], &first[..]);
}
docs/plans/M5d2/files/crates/loopd/tests/support/turn.rs
+98
View File
@@ -0,0 +1,98 @@
//! A turn-loop setup shared by the `turn` and `limits` tests: a home, a fake server, a scripted
//! port and a registry. Included with `#[path]` because it needs `loopd::turn`, which the earlier
//! tasks' tests must not depend on. Do not edit.
#![allow(dead_code)]
use crate::support::{FakeServer, Home, ScriptedPort};
use bxcore::{LogRecord, SessionId, ToolResponse, TurnEvent};
use loopd::baseline::Baseline;
use loopd::llama::Client;
use loopd::session::Session;
use loopd::tools::Registry;
use loopd::turn::{Runtime, TurnError, run_turn};
pub struct Setup {
pub home: Home,
pub server: FakeServer,
pub cfg: loopd::config::Config,
pub client: Client,
pub port: ScriptedPort,
pub registry: Registry,
}
pub fn setup(replies: Vec<ToolResponse>) -> Setup {
let home = Home::new();
let server = FakeServer::start();
let cfg = home.config(&server.socket);
let client = Client::new(cfg.clone());
Setup {
home,
server,
cfg,
client,
port: ScriptedPort::new(replies),
registry: Registry::m2b(),
}
}
impl Setup {
pub fn session(&self, id: &str) -> Session {
let baseline = Baseline::assemble(&self.cfg, &self.registry).unwrap();
Session::create(
&self.home.dir,
SessionId::new(id).unwrap(),
baseline,
self.cfg.slots.main,
)
.unwrap()
}
pub fn runtime(&self) -> Runtime<'_> {
Runtime {
cfg: &self.cfg,
client: &self.client,
port: &self.port,
registry: &self.registry,
}
}
pub fn turn(
&self,
session: &mut Session,
text: &str,
) -> (Result<loopd::turn::TurnOutcome, TurnError>, Vec<TurnEvent>) {
let mut events = Vec::new();
let result = run_turn(session, &self.runtime(), text, None, &mut |e| {
events.push(e.clone())
});
(result, events)
}
}
/// The recordings were made in separate conversations, so their timings do not line up and the
/// loop rightly reports cache losses between them. Most tests are not about that, so `types`
/// and `records` leave `CacheLoss` out; one test checks it on purpose.
pub fn without_cache_loss(records: &[LogRecord]) -> Vec<LogRecord> {
records
.iter()
.filter(|r| !matches!(r, LogRecord::CacheLoss { .. }))
.cloned()
.collect()
}
pub fn types(records: &[LogRecord]) -> Vec<&'static str> {
without_cache_loss(records)
.iter()
.map(|r| match r {
LogRecord::SessionStart { .. } => "start",
LogRecord::User { .. } => "user",
LogRecord::Assistant { .. } => "assistant",
LogRecord::Usage { .. } => "usage",
LogRecord::ToolResult { .. } => "tool_result",
LogRecord::ToolSummary { .. } => "tool_summary",
LogRecord::CacheLoss { .. } => "cache_loss",
LogRecord::EpochEnd { .. } => "epoch_end",
})
.collect()
}
docs/plans/M5d2/files/crates/loopd/tests/tool_summary_call.rs
+241
View File
@@ -0,0 +1,241 @@
//! Tests for the opted-in tool-output summary. Do not edit.
mod support;
use bxcore::{CallId, DataClass, LogRecord, SessionId, ToolCall};
use loopd::baseline::Baseline;
use loopd::compact;
use loopd::llama::Client;
use loopd::memory::NoMemory;
use loopd::session::Session;
use loopd::summarise;
use loopd::tools::Registry;
use loopd::turn::{Runtime, run_turn};
use support::{FakeServer, Home, Reply, ScriptedPort};
const CHAT: &str = "/v1/chat/completions";
const SUMMARISE: &str = "Summarise this tool result.\n";
fn sse(content: &str) -> Reply {
let content_json = serde_json::to_string(content).unwrap();
let data1 = format!(
r#"data: {{"choices":[{{"finish_reason":null,"index":0,"delta":{{"content":{content_json}}}}}],"id":"s","timings":{{"cache_n":0,"prompt_n":10,"predicted_n":1}}}}"#
);
let data2 = r#"data: {"choices":[{"finish_reason":"stop","index":0,"delta":{}}],"id":"s","timings":{"cache_n":0,"prompt_n":10,"predicted_n":1}}"#;
let body = format!("{data1}\n\n{data2}\n\ndata: [DONE]\n\n");
Reply::raw(format!(
"HTTP/1.1 200 OK\r\nContent-Type: text/event-stream\r\nContent-Length: {}\r\nConnection: close\r\n\r\n{body}",
body.len()
))
}
fn ts() -> bxcore::Timestamp {
bxcore::Timestamp::parse("2026-09-18T08:00:00.000Z").unwrap()
}
fn assistant(name: &str) -> LogRecord {
LogRecord::Assistant {
time: ts(),
content: None,
reasoning_content: None,
tool_calls: vec![ToolCall {
id: "call_f".to_string(),
name: name.to_string(),
arguments: "{}".to_string(),
}],
}
}
fn result(untrusted: bool) -> LogRecord {
LogRecord::ToolResult {
time: ts(),
call: CallId(1),
tool_call_id: "call_f".to_string(),
content: "the fetched page is about tea".to_string(),
class: DataClass::Public,
untrusted,
truncated: false,
}
}
fn session(home: &Home, cfg: &loopd::config::Config) -> Session {
let registry = Registry::m2b();
Session::create(
&home.dir,
SessionId::new("a").unwrap(),
Baseline::assemble(cfg, &registry).unwrap(),
cfg.slots.main,
)
.unwrap()
}
#[test]
fn an_opted_in_result_gets_a_summary_on_the_background_slot() {
let home = Home::new();
home.write("summarise.md", SUMMARISE);
let server = FakeServer::start();
let mut cfg = home.config(&server.socket);
cfg.compact.summarise = vec!["http_fetch".to_string()];
server.route(CHAT, vec![sse("tea page")]);
let client = Client::new(cfg.clone());
let mut s = session(&home, &cfg);
s.append(assistant("http_fetch")).unwrap();
s.append(result(false)).unwrap();
summarise::missing(&cfg, &client, &mut s).unwrap();
let sent = server.requests_to(CHAT);
assert_eq!(sent.len(), 1);
assert_eq!(sent[0].json()["id_slot"], 1);
assert!(sent[0].json().get("tools").is_none());
assert!(
s.records().iter().any(|r| matches!(r, LogRecord::ToolSummary { content, call, untrusted, .. } if content == "tea page" && *call == CallId(1) && !*untrusted)),
"{:?}",
s.records()
);
}
#[test]
fn the_summary_copies_the_result_labels() {
let home = Home::new();
home.write("summarise.md", SUMMARISE);
let server = FakeServer::start();
let mut cfg = home.config(&server.socket);
cfg.compact.summarise = vec!["http_fetch".to_string()];
server.route(CHAT, vec![sse("x")]);
let client = Client::new(cfg.clone());
let mut s = session(&home, &cfg);
s.append(assistant("http_fetch")).unwrap();
s.append(result(true)).unwrap();
summarise::missing(&cfg, &client, &mut s).unwrap();
match s.records().last() {
Some(LogRecord::ToolSummary {
class,
untrusted,
tool_call_id,
..
}) => {
assert_eq!(*class, DataClass::Public);
assert!(*untrusted);
assert_eq!(tool_call_id, "call_f");
}
other => panic!("{other:?}"),
}
}
#[test]
fn a_tool_not_in_summarise_is_left_alone() {
let home = Home::new();
home.write("summarise.md", SUMMARISE);
let server = FakeServer::start();
let mut cfg = home.config(&server.socket);
cfg.compact.summarise = vec!["http_fetch".to_string()];
let client = Client::new(cfg.clone());
let mut s = session(&home, &cfg);
s.append(assistant("clock")).unwrap();
s.append(result(false)).unwrap();
summarise::missing(&cfg, &client, &mut s).unwrap();
assert!(server.requests_to(CHAT).is_empty());
assert!(
!s.records()
.iter()
.any(|r| matches!(r, LogRecord::ToolSummary { .. }))
);
}
#[test]
fn a_result_that_already_has_a_summary_is_not_summarised_again() {
let home = Home::new();
home.write("summarise.md", SUMMARISE);
let server = FakeServer::start();
let mut cfg = home.config(&server.socket);
cfg.compact.summarise = vec!["http_fetch".to_string()];
server.route(CHAT, vec![sse("once")]);
let client = Client::new(cfg.clone());
let mut s = session(&home, &cfg);
s.append(assistant("http_fetch")).unwrap();
s.append(result(false)).unwrap();
summarise::missing(&cfg, &client, &mut s).unwrap();
summarise::missing(&cfg, &client, &mut s).unwrap();
assert_eq!(server.requests_to(CHAT).len(), 1);
assert_eq!(
s.records()
.iter()
.filter(|r| matches!(r, LogRecord::ToolSummary { .. }))
.count(),
1
);
}
#[test]
fn messages_stay_the_same_after_a_summary() {
let home = Home::new();
home.write("summarise.md", SUMMARISE);
let server = FakeServer::start();
let mut cfg = home.config(&server.socket);
cfg.compact.summarise = vec!["http_fetch".to_string()];
server.route(CHAT, vec![sse("tea page")]);
let client = Client::new(cfg.clone());
let mut s = session(&home, &cfg);
s.append(assistant("http_fetch")).unwrap();
s.append(result(false)).unwrap();
let before = loopd::baseline::messages(s.baseline(), s.records());
summarise::missing(&cfg, &client, &mut s).unwrap();
let after = loopd::baseline::messages(s.baseline(), s.records());
assert_eq!(before, after);
}
#[test]
fn after_a_turn_an_opted_in_call_is_summarised() {
let home = Home::new();
home.write("summarise.md", SUMMARISE);
home.write("compact.md", "Summarise.\n");
let server = FakeServer::start();
let mut cfg = home.config(&server.socket);
cfg.compact.summarise = vec!["read_file".to_string()];
server.route(
CHAT,
vec![
Reply::fixture("tool_call"),
Reply::fixture("plain"),
sse("hosts file"),
],
);
let client = Client::new(cfg.clone());
let port = ScriptedPort::new(vec![support::ok_result("127.0.0.1 localhost\n")]);
let registry = Registry::m2b();
let mut s = Session::create(
&home.dir,
SessionId::new("a").unwrap(),
Baseline::assemble(&cfg, &registry).unwrap(),
cfg.slots.main,
)
.unwrap();
let rt = Runtime {
cfg: &cfg,
client: &client,
port: &port,
registry: &registry,
};
let mut events = Vec::new();
let outcome = run_turn(&mut s, &rt, "hosts?", None, &mut |e| events.push(e.clone()));
compact::finish_turn(
&mut s,
&rt,
&NoMemory,
"hosts?",
None,
&mut |e| events.push(e.clone()),
outcome,
)
.unwrap();
assert!(
s.records()
.iter()
.any(|r| matches!(r, LogRecord::ToolSummary { .. })),
"{:?}",
s.records()
);
assert_eq!(
server.requests_to(CHAT).last().unwrap().json()["id_slot"],
1
);
}
docs/plans/M5d2/files/crates/loopd/tests/tool_summary_messages.rs
+88
View File
@@ -0,0 +1,88 @@
//! Tests that `messages()` ignores `ToolSummary`. Do not edit.
use bxcore::{CallId, DataClass, Epoch, Hash32, LogRecord, SessionId, Timestamp};
use loopd::baseline::{Baseline, messages};
use loopd::llama::ChatMessage;
use loopd::llama::ToolSchema;
fn ts() -> Timestamp {
Timestamp::parse("2026-09-18T08:00:00.000Z").unwrap()
}
fn baseline() -> Baseline {
Baseline {
system: "sys".to_string(),
tools: vec![ToolSchema {
name: "clock".to_string(),
description: "t".to_string(),
parameters: serde_json::json!({"type": "object", "properties": {}}),
}],
}
}
fn user(content: &str) -> LogRecord {
LogRecord::User {
time: ts(),
content: content.to_string(),
speaker: None,
}
}
fn result() -> LogRecord {
LogRecord::ToolResult {
time: ts(),
call: CallId(1),
tool_call_id: "c".to_string(),
content: "raw body".to_string(),
class: DataClass::Public,
untrusted: false,
truncated: false,
}
}
fn summary() -> LogRecord {
LogRecord::ToolSummary {
time: ts(),
call: CallId(1),
tool_call_id: "c".to_string(),
content: "a short account".to_string(),
class: DataClass::Public,
untrusted: false,
}
}
#[test]
fn messages_ignore_tool_summary() {
let start = LogRecord::SessionStart {
time: ts(),
session: SessionId::new("a").unwrap(),
epoch: Epoch(0),
slot: 0,
baseline: Hash32::ZERO,
};
let without = messages(&baseline(), &[start.clone(), user("hi"), result()]);
let with = messages(&baseline(), &[start, user("hi"), result(), summary()]);
assert_eq!(without, with);
assert!(
!with
.iter()
.any(|m| matches!(m, ChatMessage::User { content } if content == "a short account")),
"{with:?}"
);
}
#[test]
fn a_summary_is_not_replayed_as_a_tool_message() {
let msgs = messages(&baseline(), &[summary()]);
assert_eq!(msgs.len(), 1, "only the system message");
assert!(matches!(msgs[0], ChatMessage::System { .. }));
}
#[test]
fn inserting_a_summary_does_not_break_prefix_extension() {
let a = vec![user("hi"), result()];
let b = vec![user("hi"), result(), summary(), user("next")];
let ma = messages(&baseline(), &a);
let mb = messages(&baseline(), &b);
assert_eq!(&mb[..ma.len()], &ma[..]);
}
docs/plans/M5d2/files/crates/memoryd/tests/tiers.rs
+122
View File
@@ -0,0 +1,122 @@
//! Tests for `MemoryCommand::Tiers`: identity entries and notes with recall-hit counts. Do not edit.
#[path = "support/tmp.rs"]
mod tmp;
use bxcore::{DataClass, MemoryFile, Timestamp};
use memoryd::files::Store;
use memoryd::tiers;
use tmp::TempDir;
fn now() -> Timestamp {
Timestamp::parse("2026-10-03T08:00:00.000Z").unwrap()
}
struct Setup {
dir: TempDir,
store: Store,
}
fn setup(tag: &str) -> Setup {
let dir = TempDir::new(tag);
let starter = dir.write("etc/soul.md", "Soul.\n");
let store = Store::new(&dir.path().join("memory"));
store.init(&starter).unwrap();
Setup { dir, store }
}
fn write_recall(store: &Store, line: &str) {
std::fs::create_dir_all(store.path("recalls")).unwrap();
store
.append("recalls/2026-10.jsonl", &format!("{line}\n"))
.unwrap();
}
#[test]
fn identity_entries_list_with_zero_hits() {
let s = setup("tiers-id");
s.store
.write_entries(MemoryFile::Memory, &["I like tea.".to_string()])
.unwrap();
s.store
.write_entries(MemoryFile::User, &["The owner is Kyle.".to_string()])
.unwrap();
let text = tiers::list(&s.store, now()).unwrap();
assert!(
text.contains("identity memory.md hits 0 last -"),
"{text}"
);
assert!(text.contains("I like tea."), "{text}");
assert!(text.contains("identity user.md hits 0 last -"), "{text}");
assert!(text.contains("The owner is Kyle."), "{text}");
}
#[test]
fn notes_carry_counts_from_recalls() {
let s = setup("tiers-notes");
std::fs::write(
s.store.path("notes.md"),
"§ noted 2026-09-22T18:20Z in session mm-a · private\nlaunch window moved.\n",
)
.unwrap();
write_recall(
&s.store,
r#"{"time":"2026-10-03T07:00:00.000Z","session":"mm-now","query":"launch","hits":[{"kind":"noted","session":"mm-a","epoch":0,"record":0}]}"#,
);
write_recall(
&s.store,
r#"{"time":"2026-10-03T07:30:00.000Z","session":"mm-now","query":"window","hits":[{"kind":"noted","session":"mm-a","epoch":0,"record":0}]}"#,
);
let text = tiers::list(&s.store, now()).unwrap();
assert!(
text.contains(
"note session mm-a hits 2 last 2026-10-03T07:30:00.000Z private trusted"
),
"{text}"
);
assert!(text.contains("launch window moved."), "{text}");
}
#[test]
fn untrusted_notes_are_listed_and_marked() {
let s = setup("tiers-untrusted");
std::fs::write(
s.store.path("notes.md"),
"§ noted 2026-09-22T18:20Z in session mm-c · private · untrusted\nexample.org says the launch moved.\n",
)
.unwrap();
let text = tiers::list(&s.store, now()).unwrap();
assert!(
text.contains("note session mm-c hits 0 last - private untrusted"),
"{text}"
);
assert!(
text.contains("example.org says the launch moved."),
"{text}"
);
}
#[test]
fn a_recall_of_a_different_note_does_not_count() {
let s = setup("tiers-other");
std::fs::write(
s.store.path("notes.md"),
"§ noted 2026-09-22T18:20Z in session mm-a · private\nfirst note.\n§ noted 2026-09-22T18:21Z in session mm-b · private\nsecond note.\n",
)
.unwrap();
write_recall(
&s.store,
r#"{"time":"2026-10-03T07:00:00.000Z","session":"mm-now","query":"second","hits":[{"kind":"noted","session":"mm-b","epoch":0,"record":1}]}"#,
);
let text = tiers::list(&s.store, now()).unwrap();
assert!(
text.contains("note session mm-a hits 0 last - private trusted"),
"{text}"
);
assert!(
text.contains(
"note session mm-b hits 1 last 2026-10-03T07:00:00.000Z private trusted"
),
"{text}"
);
}
docs/plans/M5d2/files/crates/memoryd/tests/tool_summary_index.rs
+191
View File
@@ -0,0 +1,191 @@
//! Tests that `ToolSummary` is indexed as kind `summary` and found by `recall`. Do not edit.
#[path = "support/logs.rs"]
mod logs;
#[path = "support/tmp.rs"]
mod tmp;
use bxcore::{CallId, DataClass, LogRecord, RecallArgs, Timestamp};
use logs::{assistant, result, user, write_log};
use memoryd::files::Store;
use memoryd::index::Index;
use memoryd::recall::recall;
use tmp::TempDir;
fn now() -> Timestamp {
Timestamp::parse("2026-09-24T21:00:00.000Z").unwrap()
}
fn summary(minute: u32, content: &str, untrusted: bool) -> LogRecord {
LogRecord::ToolSummary {
time: logs::at(minute),
call: CallId(1),
tool_call_id: "c0".to_string(),
content: content.to_string(),
class: DataClass::Public,
untrusted,
}
}
struct Setup {
dir: TempDir,
store: Store,
index: Index,
}
fn setup(tag: &str) -> Setup {
let dir = TempDir::new(tag);
let starter = dir.write("etc/soul.md", "Soul.\n");
let store = Store::new(&dir.path().join("memory"));
store.init(&starter).unwrap();
let index = Index::open(&dir.path().join("index/memory.sqlite3"), DataClass::Private).unwrap();
Setup { dir, store, index }
}
fn ask(s: &Setup, query: &str) -> memoryd::recall::Recalled {
let args = RecallArgs {
query: query.to_string(),
k: 10,
};
recall(&s.index, &s.store, &args, "mm-now", now()).unwrap()
}
#[test]
fn a_tool_summary_is_found_as_kind_summary() {
let s = setup("summary-found");
write_log(
s.dir.path(),
"mm-a",
&[
user(2, "fetch the launch page"),
assistant(
3,
None,
&[("http_fetch", r#"{"url":"https://example.org"}"#)],
),
result(4, "raw html body about tea", DataClass::Public, false),
summary(5, "the page is about tea and the launch window", false),
],
);
s.index
.catch_up(&s.dir.path().join("sessions"), &s.store.path("notes.md"))
.unwrap();
let got = ask(&s, "tea");
assert!(
got.text
.contains("the page is about tea and the launch window"),
"{got:?}"
);
assert!(
got.text.contains("memory · session mm-a"),
"tagged as a session hit: {}",
got.text
);
assert!(!got.untrusted);
}
#[test]
fn the_raw_tool_result_is_still_not_indexed() {
let s = setup("summary-no-raw");
write_log(
s.dir.path(),
"mm-a",
&[
user(2, "fetch"),
assistant(3, None, &[("http_fetch", "{}")]),
result(4, "UNIQUE_RAW_BODY_XYZ", DataClass::Public, false),
summary(5, "a short account of the page", false),
],
);
s.index
.catch_up(&s.dir.path().join("sessions"), &s.store.path("notes.md"))
.unwrap();
let got = ask(&s, "UNIQUE_RAW_BODY_XYZ");
assert!(
got.text.contains("(no matches"),
"raw tool result stays unindexed: {}",
got.text
);
}
#[test]
fn an_untrusted_summary_taints_the_recall() {
let s = setup("summary-taint");
write_log(
s.dir.path(),
"mm-a",
&[
user(2, "fetch"),
assistant(3, None, &[("http_fetch", "{}")]),
result(4, "page", DataClass::Public, true),
summary(5, "the fetched page mentions the launch window", true),
],
);
s.index
.catch_up(&s.dir.path().join("sessions"), &s.store.path("notes.md"))
.unwrap();
let got = ask(&s, "launch");
assert!(got.untrusted, "{got:?}");
assert!(got.text.contains("memory-tainted"), "{}", got.text);
}
#[test]
fn a_trusted_summary_does_not_taint() {
let s = setup("summary-trusted");
write_log(
s.dir.path(),
"mm-a",
&[
user(2, "fetch"),
assistant(3, None, &[("http_fetch", "{}")]),
result(4, "page", DataClass::Public, false),
summary(5, "the fetched page mentions the launch window", false),
],
);
s.index
.catch_up(&s.dir.path().join("sessions"), &s.store.path("notes.md"))
.unwrap();
let got = ask(&s, "launch");
assert!(!got.untrusted, "{got:?}");
assert!(got.text.contains("memory · session mm-a"), "{}", got.text);
assert!(!got.text.contains("memory-tainted"), "{}", got.text);
}
#[test]
fn a_secret_summary_is_skipped_when_the_index_is_private() {
let s = setup("summary-secret");
write_log(
s.dir.path(),
"mm-a",
&[
user(2, "fetch"),
assistant(3, None, &[("http_fetch", "{}")]),
LogRecord::ToolResult {
time: logs::at(4),
call: CallId(1),
tool_call_id: "c0".to_string(),
content: "secret page",
class: DataClass::Secret,
untrusted: false,
truncated: false,
},
LogRecord::ToolSummary {
time: logs::at(5),
call: CallId(1),
tool_call_id: "c0".to_string(),
content: "SECRET_SUMMARY_PHRASE",
class: DataClass::Secret,
untrusted: false,
},
],
);
s.index
.catch_up(&s.dir.path().join("sessions"), &s.store.path("notes.md"))
.unwrap();
let got = ask(&s, "SECRET_SUMMARY_PHRASE");
assert!(
got.text.contains("(no matches"),
"secret summary above max_class: {}",
got.text
);
}
docs/plans/M5d2/files/runbook-heartbeat-prompt-unreadable.md
+25
View File
@@ -0,0 +1,25 @@
## heartbeat-prompt-unreadable
**What you see.** `loopd` is serving, but heartbeat turns do not run. It logs this line once,
until the file reads:
```text
loopd: heartbeat.md: Permission denied (os error 13); see docs/runbook.md#heartbeat-prompt-unreadable
```
**Why.** `[heartbeat] prompt` is next to the config file (default `heartbeat.md`). The scheduler
skips the tick rather than send an empty or guessed user message. A missing file is the same:
restore the starter from `config/heartbeat.md` in the repository.
**Confirm.**
```sh
ls -l "$(dirname "$LOOPD_CONFIG")/heartbeat.md"
```
**Fix.** Restore mode `0644` (or whatever the owner uses for the other prompts) and content.
The next tick after a successful read logs nothing extra and runs the standing `heartbeat`
session.
**Check.** `sessions/heartbeat/0.jsonl` gains a new `User` whose text starts with `It is ` and an
RFC 3339 timestamp.
docs/plans/M5d2/files/runbook-jobs-invalid.md
+24
View File
@@ -0,0 +1,24 @@
## jobs-invalid
**What you see.** `loopd` does not start. It prints a line naming `{home}/jobs/<name>.toml` and
this entry:
```text
loopd: jobs/mail.toml: unknown field `when`, expected `schedule` or `prompt`; see docs/runbook.md#jobs-invalid
```
**Why.** Job files are owner-written. `loopd` reads them and cannot write them. Unknown keys, a
name that is not `[a-z0-9-]+`, a name so long that `job-<name>-<yyyymmdd>t<hhmm>` is not a valid
session id (64 bytes, `[a-z0-9-]`), a schedule that is not five UTC fields, or a missing
`prompt`, all refuse start so a misspelt job cannot sit silently.
**Confirm.**
```sh
ls -l "$BOXMAKER_HOME/jobs/"
```
**Fix.** Rename or edit the named file so it matches the example in `config/jobs/tier.toml.example`.
Then start `loopd` again.
**Check.** `loopd serve` prints `serving on` and `{home}/jobs/runs.jsonl` exists (empty is fine).