docs: M5d-2 task 11, failed heartbeat still advances last_heartbeat
Review finding 1: last_heartbeat and TickWork::Heartbeat were gated on outcome.is_ok(). The given test fails today with left: [] / right: [Heartbeat]. Co-Authored-By: Grok 4.6 <noreply@x.ai>
This commit is contained in:
@@ -34,9 +34,9 @@ sections 3, 8, 9, 10 and 13) is done, reviewed in `docs/implementer-log.md` and
|
|||||||
straylight against rift; no finding is open. M5d (scheduler and compaction, spec
|
straylight against rift; no finding is open. M5d (scheduler and compaction, spec
|
||||||
`docs/specs/2026-10-03-m5d-scheduler.md`) is split: M5d-1 (epochs and compaction) is done,
|
`docs/specs/2026-10-03-m5d-scheduler.md`) is split: M5d-1 (epochs and compaction) is done,
|
||||||
reviewed in `docs/implementer-log.md` and checked on straylight (spec 7); no finding is open.
|
reviewed in `docs/implementer-log.md` and checked on straylight (spec 7); no finding is open.
|
||||||
M5d-2 (heartbeat, cron, ToolSummary, live-slot isolation) is done on branch `m5d2` at `3bea6c2`,
|
M5d-2 (heartbeat, cron, ToolSummary, live-slot isolation) is done on branch `m5d2` at `85f70de`,
|
||||||
reviewed in `docs/implementer-log.md`; finding 1 (heartbeat `last_heartbeat` only on Ok) is open
|
reviewed in `docs/implementer-log.md`; finding 1 is task 11 (`docs/plans/M5d2/11-loopd-heartbeat-failed-last.md`).
|
||||||
to fix before land; the spec 7 device check on straylight is not yet run.
|
The spec 7 device check on straylight is not yet run.
|
||||||
P22 (proposed, not applied) makes Boxmaker a fleet of
|
P22 (proposed, not applied) makes Boxmaker a fleet of
|
||||||
independent agents, one per host, deployed with Nix in a last milestone, M8; `flake.nix` builds
|
independent agents, one per host, deployed with Nix in a last milestone, M8; `flake.nix` builds
|
||||||
every binary, the tool image and a dev shell. Work happens on titan (macOS, no Nix); build and run
|
every binary, the tool image and a dev shell. Work happens on titan (macOS, no Nix); build and run
|
||||||
|
|||||||
@@ -0,0 +1,138 @@
|
|||||||
|
# M5d-2 task 11: a failed heartbeat still advances `last_heartbeat`
|
||||||
|
|
||||||
|
**Branch:** `m5d2`. Check it: `git branch --show-current` prints `m5d2` and `git status --short`
|
||||||
|
is empty; otherwise stop. Never switch branch.
|
||||||
|
**Commit subject:** `loopd: a failed heartbeat still advances last_heartbeat`
|
||||||
|
**Log row:** the Task cell is `M5d2/11-loopd-heartbeat-failed-last`.
|
||||||
|
|
||||||
|
**First action.** Run this before any Read of crate source. Do not `ls` `crates/loopd/tests/`.
|
||||||
|
Do not Read `tests/support/`, `session.rs`, `turn.rs`, `compact.rs`, `jobs.rs`, or
|
||||||
|
`docs/implementer-log.md` (except the first 12 lines at the log step). The quoted block is
|
||||||
|
the only edit.
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cp docs/plans/M5d2/files/crates/loopd/tests/scheduler_heartbeat_failed.rs crates/loopd/tests/
|
||||||
|
```
|
||||||
|
|
||||||
|
## Goal
|
||||||
|
|
||||||
|
After `run_turn` and `finish_turn` on the standing `heartbeat` session, always set
|
||||||
|
`last_heartbeat = Some(now)` and push `TickWork::Heartbeat`, including when the outcome is
|
||||||
|
`Err` (M5d-2 review finding 1; task 06's "The turn" fact). A failed pulse must not be due
|
||||||
|
again on the next 1 s tick.
|
||||||
|
|
||||||
|
An unreadable prompt still does not update `last_heartbeat`. A busy skip still does not.
|
||||||
|
Cron `Finished` / `Failed` is unchanged.
|
||||||
|
|
||||||
|
## Every place that changes
|
||||||
|
|
||||||
|
| Place | What happens there |
|
||||||
|
|---|---|
|
||||||
|
| `crates/loopd/src/scheduler.rs` | after the heartbeat turn, bookkeeping is unconditional (edit H1) |
|
||||||
|
| `crates/loopd/tests/scheduler_heartbeat_failed.rs` | new, given |
|
||||||
|
| everything else | **no change** |
|
||||||
|
|
||||||
|
## Files
|
||||||
|
|
||||||
|
- Copy, from `docs/plans/M5d2/files/` (the same path under it): the new test
|
||||||
|
`crates/loopd/tests/scheduler_heartbeat_failed.rs`.
|
||||||
|
- Modify: `crates/loopd/src/scheduler.rs`, `docs/implementer-log.md`.
|
||||||
|
- Do not touch: any other file. Do not edit `crates/loopd/tests/scheduler_heartbeat.rs`.
|
||||||
|
|
||||||
|
## Facts you need
|
||||||
|
|
||||||
|
- **H1.** In `heartbeat`, the quoted block is found exactly once. Replace it with the after
|
||||||
|
block. `finish_turn` still runs. `drop(held)` still runs. Then `last_heartbeat` and
|
||||||
|
`Heartbeat`, with no `if outcome.is_ok()`.
|
||||||
|
- **Unreadable prompt / Busy.** Those `return` paths stay as they are. They never reach H1.
|
||||||
|
- **`Reply::fixture("bad_request")`** is HTTP 400. It is not retried. The given test does
|
||||||
|
not sleep (T27).
|
||||||
|
|
||||||
|
## The edit (H1)
|
||||||
|
|
||||||
|
In `crates/loopd/src/scheduler.rs`, this block (exactly once):
|
||||||
|
|
||||||
|
```rust
|
||||||
|
let text = format!("It is {}.\n\n{file}", now.to_rfc3339());
|
||||||
|
let mut emit = |_event: &TurnEvent| {};
|
||||||
|
let outcome = run_turn(&mut session, &runtime, &text, None, &mut emit);
|
||||||
|
let outcome = compact::finish_turn(
|
||||||
|
&mut session,
|
||||||
|
&runtime,
|
||||||
|
&*ctx.memory,
|
||||||
|
&text,
|
||||||
|
None,
|
||||||
|
&mut emit,
|
||||||
|
outcome,
|
||||||
|
);
|
||||||
|
if outcome.is_ok() {
|
||||||
|
self.last_heartbeat = Some(now);
|
||||||
|
work.push(TickWork::Heartbeat);
|
||||||
|
}
|
||||||
|
drop(held);
|
||||||
|
```
|
||||||
|
|
||||||
|
becomes:
|
||||||
|
|
||||||
|
```rust
|
||||||
|
let text = format!("It is {}.\n\n{file}", now.to_rfc3339());
|
||||||
|
let mut emit = |_event: &TurnEvent| {};
|
||||||
|
let outcome = run_turn(&mut session, &runtime, &text, None, &mut emit);
|
||||||
|
let _ = compact::finish_turn(
|
||||||
|
&mut session,
|
||||||
|
&runtime,
|
||||||
|
&*ctx.memory,
|
||||||
|
&text,
|
||||||
|
None,
|
||||||
|
&mut emit,
|
||||||
|
outcome,
|
||||||
|
);
|
||||||
|
drop(held);
|
||||||
|
self.last_heartbeat = Some(now);
|
||||||
|
work.push(TickWork::Heartbeat);
|
||||||
|
```
|
||||||
|
|
||||||
|
## Steps
|
||||||
|
|
||||||
|
- [ ] **1. Copy the given test.**
|
||||||
|
|
||||||
|
```sh
|
||||||
|
cp docs/plans/M5d2/files/crates/loopd/tests/scheduler_heartbeat_failed.rs crates/loopd/tests/
|
||||||
|
```
|
||||||
|
|
||||||
|
- [ ] **2. See it fail.** `cargo test -p loopd --test scheduler_heartbeat_failed`, timeout
|
||||||
|
600000 ms. Expected: one test fails,
|
||||||
|
`a_failed_heartbeat_turn_still_advances_last_heartbeat`, with `left: []` and
|
||||||
|
`right: [Heartbeat]`. Every other loopd test target is untouched. Do not open the test
|
||||||
|
to find out why.
|
||||||
|
- [ ] **3. Make H1.** The before block is found exactly once. `cargo fmt --all`.
|
||||||
|
`cargo check -p loopd` prints no error and no warning.
|
||||||
|
- [ ] **4. See it pass.** `cargo test -p loopd --test scheduler_heartbeat_failed --test
|
||||||
|
scheduler_heartbeat`: every test passes, 1 in `scheduler_heartbeat_failed`, 6 in
|
||||||
|
`scheduler_heartbeat`.
|
||||||
|
- [ ] **5. Run the gate.** `cargo fmt --all`, then `make gate`, timeout 600000 ms. Last line
|
||||||
|
`gate: ok`. Test count **1575** (1574 before, plus 1). After `gate: ok`, do not add up
|
||||||
|
`test result:` lines.
|
||||||
|
- [ ] **6. Log and commit.** Task cell `M5d2/11-loopd-heartbeat-failed-last`. Read only the
|
||||||
|
first 12 lines of `docs/implementer-log.md`; insert the row under the header. Stage with
|
||||||
|
`git add crates/loopd/src/scheduler.rs crates/loopd/tests/scheduler_heartbeat_failed.rs
|
||||||
|
docs/implementer-log.md Cargo.lock`, then commit as `AGENTS.md` says.
|
||||||
|
|
||||||
|
## Done when
|
||||||
|
|
||||||
|
- `make gate` prints `gate: ok` with 1575 tests.
|
||||||
|
- `git diff HEAD~1 --stat` lists only `crates/loopd/src/scheduler.rs`,
|
||||||
|
`crates/loopd/tests/scheduler_heartbeat_failed.rs` and the log.
|
||||||
|
|
||||||
|
## Stop and report if
|
||||||
|
|
||||||
|
- The branch check fails; the before block is not found exactly once; step 2 differs (a
|
||||||
|
different test fails, or it passes); `cargo check` warns; a test in
|
||||||
|
`scheduler_heartbeat` fails; the gate's last line is not `gate: ok`; you edit the
|
||||||
|
unreadable-prompt path; you change cron `Finished` / `Failed`.
|
||||||
|
|
||||||
|
## What the given tests pin
|
||||||
|
|
||||||
|
A due heartbeat whose inference returns HTTP 400 still pushes `Heartbeat` and is not due
|
||||||
|
again 30 s later (`every_s` 60). The existing six heartbeat tests stay green: `every_s` 0,
|
||||||
|
busy skip, unreadable prompt, and a successful second tick before `every_s`.
|
||||||
@@ -70,8 +70,9 @@ its own. Before task 01 it is **1508** (straylight, rustc 1.98.1 / clippy 0.1.98
|
|||||||
| 08 | After a turn, opted-in `ToolResult` without a `ToolSummary` gets a background summary appended with the result's labels | loopd | 1563 | 6 | 2, 2 |
|
| 08 | After a turn, opted-in `ToolResult` without a `ToolSummary` gets a background summary appended with the result's labels | loopd | 1563 | 6 | 2, 2 |
|
||||||
| 09 | `memoryd` indexes `ToolSummary` as kind `summary`; `recall` finds it and taints when untrusted | memoryd | 1568 | 5 | 1, 1 |
|
| 09 | `memoryd` indexes `ToolSummary` as kind `summary`; `recall` finds it and taints when untrusted | memoryd | 1568 | 5 | 1, 1 |
|
||||||
| 10 | `MemoryCommand::Tiers`, `bxctl memory tiers`, starter `config/jobs/tier.toml.example` | bxcore, memoryd, bxctl | 1574 | 6 | 1, 1 |
|
| 10 | `MemoryCommand::Tiers`, `bxctl memory tiers`, starter `config/jobs/tier.toml.example` | bxcore, memoryd, bxctl | 1574 | 6 | 1, 1 |
|
||||||
|
| 11 | A failed heartbeat turn still sets `last_heartbeat` and pushes `Heartbeat` | loopd | 1575 | 1 | 1, 1 |
|
||||||
|
|
||||||
After task 10, the design model runs the M5d-2 device check (spec 7) on straylight: a heartbeat
|
After task 11, the design model runs the M5d-2 device check (spec 7) on straylight: a heartbeat
|
||||||
turn on the background slot leaves a following main-slot turn's `cache_n` near its previous
|
turn on the background slot leaves a following main-slot turn's `cache_n` near its previous
|
||||||
total; a `jobs/tier.toml` for the check promotes only a trusted note.
|
total; a `jobs/tier.toml` for the check promotes only a trusted note.
|
||||||
|
|
||||||
@@ -90,6 +91,8 @@ happy-path tests will miss them.
|
|||||||
(task 08).
|
(task 08).
|
||||||
- A secret `ToolSummary` is skipped when the index is private (task 09).
|
- A secret `ToolSummary` is skipped when the index is private (task 09).
|
||||||
- Untrusted notes appear in `Tiers` and are marked; they cannot be omitted (task 10).
|
- Untrusted notes appear in `Tiers` and are marked; they cannot be omitted (task 10).
|
||||||
|
- A failed heartbeat turn still advances `last_heartbeat`; the next tick before `every_s` does
|
||||||
|
not retry (task 11).
|
||||||
|
|
||||||
## Known limits
|
## Known limits
|
||||||
|
|
||||||
@@ -129,4 +132,13 @@ OPENCODE_MODEL=llama.cpp/ornith-1.5-35b-a3b RESUME=1 \
|
|||||||
tools/run-plan.sh docs/plans/M5d2 01
|
tools/run-plan.sh docs/plans/M5d2 01
|
||||||
```
|
```
|
||||||
|
|
||||||
After task 10, the gate count is 1574. Then the device check above.
|
After task 11, the gate count is 1575. Then the device check above.
|
||||||
|
|
||||||
|
Task 11 is the M5d-2 review follow-up (finding 1). From the repository root, with the OpenCode
|
||||||
|
TUI closed:
|
||||||
|
|
||||||
|
```sh
|
||||||
|
OPENCODE_MODEL=llama.cpp/ornith-1.5-35b-a3b RESUME=1 \
|
||||||
|
SLOTS_URL='http://straylight:11434/slots?model=ornith-1.5-35b-a3b' \
|
||||||
|
tools/run-plan.sh docs/plans/M5d2 11
|
||||||
|
```
|
||||||
|
|||||||
@@ -0,0 +1,50 @@
|
|||||||
|
//! A failed heartbeat turn still advances `last_heartbeat`. Do not edit.
|
||||||
|
|
||||||
|
mod support;
|
||||||
|
|
||||||
|
use bxcore::Timestamp;
|
||||||
|
use loopd::channel::Context;
|
||||||
|
use loopd::llama::Client;
|
||||||
|
use loopd::memory::NoMemory;
|
||||||
|
use loopd::scheduler::{Scheduler, TickWork};
|
||||||
|
use loopd::tools::Registry;
|
||||||
|
use std::sync::Arc;
|
||||||
|
use support::{FakeServer, Home, Reply, ScriptedPort};
|
||||||
|
|
||||||
|
const CHAT: &str = "/v1/chat/completions";
|
||||||
|
const PROMPT: &str = "Look around.\n";
|
||||||
|
|
||||||
|
fn ts(s: &str) -> Timestamp {
|
||||||
|
Timestamp::parse(s).unwrap()
|
||||||
|
}
|
||||||
|
|
||||||
|
fn world() -> (Home, FakeServer, Arc<Context>) {
|
||||||
|
let home = Home::new();
|
||||||
|
home.write("heartbeat.md", PROMPT);
|
||||||
|
home.write("compact.md", "Summarise.\n");
|
||||||
|
let server = FakeServer::start();
|
||||||
|
let mut cfg = home.config(&server.socket);
|
||||||
|
cfg.heartbeat.every_s = 60;
|
||||||
|
cfg.heartbeat.prompt = home.dir.join("heartbeat.md");
|
||||||
|
server.route(CHAT, vec![Reply::fixture("bad_request")]);
|
||||||
|
let ctx = Arc::new(Context::new(
|
||||||
|
cfg.clone(),
|
||||||
|
Client::new(cfg),
|
||||||
|
Box::new(ScriptedPort::new(vec![])),
|
||||||
|
Registry::m2b(),
|
||||||
|
Box::new(NoMemory),
|
||||||
|
));
|
||||||
|
(home, server, ctx)
|
||||||
|
}
|
||||||
|
|
||||||
|
#[test]
|
||||||
|
fn a_failed_heartbeat_turn_still_advances_last_heartbeat() {
|
||||||
|
let (_home, server, ctx) = world();
|
||||||
|
let mut sched = Scheduler::new(ts("2026-10-03T07:00:00.000Z"));
|
||||||
|
let work = sched.tick(&ctx, ts("2026-10-03T07:01:00.000Z"));
|
||||||
|
assert_eq!(work, vec![TickWork::Heartbeat]);
|
||||||
|
assert_eq!(server.requests_to(CHAT).len(), 1);
|
||||||
|
let work = sched.tick(&ctx, ts("2026-10-03T07:01:30.000Z"));
|
||||||
|
assert!(work.is_empty(), "{work:?}");
|
||||||
|
assert_eq!(server.requests_to(CHAT).len(), 1);
|
||||||
|
}
|
||||||
Reference in New Issue
Block a user