Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
651 lines
30 KiB
Markdown
651 lines
30 KiB
Markdown
# M5c-1 task 09: `memoryd` decides a reported turn, after `loopd` has its answer
|
|
|
|
**Branch:** `m5c1` (run `git switch m5c1`; `git status --short` must be empty, otherwise stop)
|
|
**Commit subject:** `memoryd: a reported turn decided after loopd has its answer, from brokerd and the log`
|
|
**Log row:** the Task cell is `M5c1/09-memoryd-honcho-turns`.
|
|
|
|
## Goal
|
|
|
|
When a turn ends, `loopd` sends `memoryd` a `SessionUpdated { session, turn }` (task 02). From
|
|
this task on, with a `[honcho]` section, `memoryd` then decides where that turn goes in Honcho
|
|
(task 08's `decide`) and appends the decision to the ledger (task 07) at once (M5c spec, section
|
|
5). Nothing is sent to Honcho yet: that is task 11.
|
|
|
|
- **Answer first.** `loopd` waits for `memoryd`'s answer to `SessionUpdated` before it sends the
|
|
turn's reply to the person, and a question to `brokerd` may take seconds. So `memoryd` answers
|
|
`loopd` exactly as today, closes the connection, and only then, on the same connection's
|
|
thread, decides. The Honcho state has a lock of its own, never the index lock, so a slow
|
|
`brokerd` holds up neither `loopd` nor a memory call.
|
|
- **Every waiting turn.** `memoryd` keeps, in memory, each session with undecided turns and the
|
|
highest turn `loopd` reported for it. At each report it walks every such session, in the order
|
|
of their ids: it reads the session's log, and decides every undecided turn from 1 to the
|
|
smaller of the reported turn and the log's turns. So a turn left undecided while `brokerd` was
|
|
down is decided at the next report of **any** session (spec 5, step 3).
|
|
- **The log's last turn waits for its reply.** A log's last turn without a final answer may still
|
|
be running: it is not decided (and never skipped) until it has one. This only matters when a
|
|
report names a turn past the log's end; `loopd` never sends one, but the number comes over a
|
|
socket.
|
|
- **One question to `brokerd` per session per walk**, on its `state.sock` (task 05), and none for
|
|
a session with nothing undecided. The answer is the session's class and trust; an error frame,
|
|
no listener or 5 s without a byte means no answer.
|
|
- **`brokerd` does not answer:** nothing is decided, the walk stops there (the next sessions
|
|
would fail the same way), and the outage is logged once, with the new runbook entry
|
|
`broker-state-unavailable`. The first answer after it is logged once too.
|
|
- **A log that does not read** (a damaged complete line): nothing is decided for that session,
|
|
logged once with `session-log-damaged`; the walk goes on with the next session. A last line
|
|
without its newline is one `loopd` is still writing, and is left out, silently.
|
|
- **The ledger at start.** `memoryd` reads `<home>/memory/honcho.jsonl` through before anything
|
|
else in `serve::start`. A damaged or unreadable ledger stops `memoryd`: exit 1, one line with
|
|
the pointer `honcho-ledger-damaged`, no socket bound (as task 06 does for
|
|
`honcho-config-invalid`). A cut-off last line is dropped by the ledger (task 07) and logged
|
|
once. A ledger that cannot be **written** while `memoryd` runs is logged with the same pointer;
|
|
the walk stops, and the turn is decided at a later report.
|
|
- **Without `[honcho]`** nothing changes: no ledger, no question, no line. Every existing test
|
|
passes unchanged.
|
|
|
|
Epochs are out of scope: `loopd` writes only `sessions/<id>/0.jsonl` and numbers turns in it, and
|
|
so does this task. (A later milestone that adds compaction must settle how turns are numbered
|
|
across epochs.)
|
|
|
|
## Interfaces
|
|
|
|
In `crates/memoryd/src/honcho/state.rs` (the module `memoryd::honcho::state`):
|
|
|
|
```rust
|
|
pub const WAIT: Duration = Duration::from_secs(5);
|
|
pub const RUNBOOK: &str = "see docs/runbook.md#broker-state-unavailable";
|
|
pub fn ask(socket: &Path, session: &SessionId, wait: Duration) -> Result<SessionStateReply, String>;
|
|
```
|
|
|
|
In `crates/memoryd/src/honcho/turns.rs` (the module `memoryd::honcho::turns`):
|
|
|
|
```rust
|
|
pub const LEDGER: &str = "honcho.jsonl";
|
|
pub const LOG_RUNBOOK: &str = "see docs/runbook.md#session-log-damaged";
|
|
pub type Ask<'a> = &'a dyn Fn(&SessionId) -> Result<SessionStateReply, String>;
|
|
pub fn log_path(sessions: &Path, session: &SessionId) -> PathBuf;
|
|
pub fn read_log(path: &Path) -> Result<Vec<LogRecord>, String>;
|
|
pub fn finished(turn: &[LogRecord]) -> bool;
|
|
pub fn decidable(records: &[LogRecord], reported: u64) -> (u64, bool);
|
|
pub struct TurnSync { /* private */ }
|
|
impl TurnSync {
|
|
pub fn new(ledger: Ledger, agent: &str, sessions: &Path) -> TurnSync;
|
|
pub fn ledger(&self) -> &Ledger;
|
|
pub fn waiting(&self) -> Vec<(SessionId, u64)>;
|
|
pub fn updated(&mut self, session: &SessionId, turn: u64, ask: Ask<'_>, now: Timestamp,
|
|
log: &dyn Fn(&str));
|
|
}
|
|
```
|
|
|
|
In `crates/memoryd/src/serve.rs`: `pub enum StartError { Failed(String), Ledger(LedgerError) }`
|
|
(Debug, Display, Error, `From<String>`), and `start` returns `Result<Running, StartError>`.
|
|
Task 11 uses `TurnSync`, `decidable` and `waiting` when it rebuilds what a restart forgets.
|
|
|
|
## The lines `memoryd` logs
|
|
|
|
| When | The line |
|
|
|---|---|
|
|
| the ledger's last line was cut off (at start) | `memoryd: <home>/memory/honcho.jsonl: a last line cut off by a crash was dropped` |
|
|
| `brokerd` gives no answer, the first time | `memoryd: <reason>; turns wait undecided; see docs/runbook.md#broker-state-unavailable` |
|
|
| `brokerd` answers again | `memoryd: brokerd answers on its state socket again; turns are decided` |
|
|
| a session's log does not read, the first time | `memoryd: <path>:<line>: <reason>; see docs/runbook.md#session-log-damaged` |
|
|
| the ledger cannot be written | `memoryd: <path>: <error>; see docs/runbook.md#honcho-ledger-damaged` |
|
|
|
|
`<reason>` for `brokerd` is `ask`'s error: `<socket path>: <error>` (connect, timeout, frame) or
|
|
`brokerd: <detail>` for an error frame. At start, a damaged ledger is `memoryd: <path>:<line>:
|
|
<why>; see docs/runbook.md#honcho-ledger-damaged` on stderr, and an unreadable one `memoryd:
|
|
<path>: <error>; see docs/runbook.md#honcho-ledger-damaged`: one line each, then exit 1.
|
|
|
|
## Every place that changes
|
|
|
|
This is the complete list.
|
|
|
|
| Place | What happens there |
|
|
|---|---|
|
|
| `crates/memoryd/src/honcho/mod.rs` | `pub mod state;` and `pub mod turns;` (edit H1) |
|
|
| `crates/memoryd/src/honcho/state.rs` | new: a skeleton you copy, then fill 1 body |
|
|
| `crates/memoryd/src/honcho/turns.rs` | new: a skeleton you copy, then fill 6 bodies |
|
|
| `crates/memoryd/src/serve.rs` | module comment, imports, `Memoryd.honcho`, `StartError`, `start`, `connection`, `decide_turns` (edits S1 to S7) |
|
|
| `crates/memoryd/src/main.rs` | a ledger error printed with its own pointer (edit M1) |
|
|
| `docs/runbook.md` | `session-log-damaged` and `honcho-ledger-damaged` changed (edits R1 to R4); `broker-state-unavailable` appended |
|
|
| everything else (`decide.rs`, `ledger.rs`, `config.rs`, other crates) | **no change** |
|
|
|
|
## Files
|
|
|
|
- Copy, from `docs/plans/M5c1/files/` (the same path under it):
|
|
- new tests: `crates/memoryd/tests/honcho_state.rs`, `crates/memoryd/tests/honcho_turns.rs`,
|
|
and `crates/memoryd/tests/honcho_serve.rs` from `honcho_serve.rs-task09` (this task's
|
|
version; task 11 gives a later one)
|
|
- new test support: `crates/memoryd/tests/support/brokerd.rs` (a fake `brokerd` state
|
|
socket), `crates/memoryd/tests/support/turns.rs` (logs of turns with speakers)
|
|
- the skeletons `crates/memoryd/src/honcho/state.rs` and `crates/memoryd/src/honcho/turns.rs`
|
|
- Append, from `docs/plans/M5c1/files/`, to the end of `docs/runbook.md`, with one blank line
|
|
before it: `runbook-broker-state-unavailable.md` (step 7 gives the command).
|
|
- Modify: `crates/memoryd/src/honcho/mod.rs`, `crates/memoryd/src/serve.rs`,
|
|
`crates/memoryd/src/main.rs`, `docs/runbook.md`, `docs/implementer-log.md`.
|
|
- Do not touch: any other source file, any other test or support file (`support/logs.rs` and
|
|
`support/tmp.rs` in particular, which the new tests use), any file of another crate, any
|
|
`Cargo.toml`, and in the skeletons anything but the 7 `todo!()` bodies.
|
|
|
|
## Facts you need
|
|
|
|
- **The socket.** `brokerd` serves `state.sock` (task 05): one frame in, one final frame out, the
|
|
connection then closed. The question is `Envelope { v: PROTOCOL_VERSION, id: 1, r#final: true,
|
|
msg: Message::SessionState(SessionState { session }) }`; the answer
|
|
`Message::SessionStateReply(SessionStateReply { class, untrusted })`, or `Message::Error` (for
|
|
example a damaged state file, `broker-state-damaged` in `brokerd`'s own log). Frames are
|
|
`bxcore::write_frame` and `read_frame`, as on every socket. `Config::state_socket()` (task 06)
|
|
is the path.
|
|
- **Timeouts.** The read and the write timeouts are 5 s (`WAIT`): "no byte for 5 s", as `loopd`'s
|
|
`MemoryPort` does it with `set_read_timeout` and `set_write_timeout`. There is no connect
|
|
timeout: `std::os::unix::net::UnixStream` has none, and a connect fails at once when nothing
|
|
listens. A read that times out is `frame I/O error: Resource temporarily unavailable (os error
|
|
11)` on Linux.
|
|
- **The log.** `sessions/<id>/0.jsonl` under `[paths] home` (`Config::sessions_dir()`), one
|
|
`LogRecord` per line. `loopd` writes a record and its newline in two writes, so `memoryd` can
|
|
read a last line without its newline: it is left out. A turn is finished when its last
|
|
`Assistant` record asks for no tool (`finished`); it need not have text (task 08 skips an empty
|
|
reply as `the turn has no reply`).
|
|
- **The walk** is `TurnSync::updated`, given whole with `walk`: it calls your `records`,
|
|
`decidable`, `state` and task 08's `decide`. Read both before you fill the bodies. A session
|
|
leaves the map when every turn up to its reported one (as far as its log goes) is decided; it
|
|
stays while a turn waits (for `brokerd`, a readable log, or a reply).
|
|
- **The ledger's time** is the walk's `now` (`Timestamp::now()` in `serve.rs`), so a decision
|
|
made late carries the time it was made.
|
|
- **What a restart forgets.** The map lives in memory. After a restart a waiting session is
|
|
walked again only at its next report. Rebuilding the map at start (from the ledger and the
|
|
logs, with `decidable` so that a running turn is never decided) is task 11's.
|
|
- **The given tests.**
|
|
- `honcho_state.rs` (5 tests): the question as `brokerd` reads it and its answer; an error
|
|
frame; no listener (an error at once); silence (an error after the wait, here 300 ms).
|
|
- `honcho_turns.rs` (16 tests): `TurnSync` with `brokerd` as a closure: every undecided turn up
|
|
to the report and the log; the running last turn; one question per session, none when
|
|
nothing waits; the outage and the return logged once; the walk stopped at the first failure;
|
|
an earlier report never lowering the turn; a damaged log and a line being written; a ledger
|
|
that cannot be written; `read_log`, `finished`, `decidable`.
|
|
- `honcho_serve.rs` (6 tests): `serve::start` on a thread with a fake `brokerd`: a turn decided
|
|
after `loopd` has its answer (and a report on the wrong socket deciding nothing); a silent
|
|
`brokerd` holding up neither a second report nor a memory call (each answered within 1 s),
|
|
then logged once after 5 s; nothing asked or written without `[honcho]`; a damaged or
|
|
unreadable ledger stopping `start` before any socket is bound; a cut-off last line logged;
|
|
and the `memoryd` process exiting with exactly the ledger's line. It takes about 6 s.
|
|
- The calls the bodies use:
|
|
|
|
| Call | Signature, and what it gives |
|
|
|---|---|
|
|
| `UnixStream::connect` | `UnixStream::connect(socket) -> io::Result<UnixStream>` |
|
|
| `set_read_timeout`, `set_write_timeout` | `stream.set_read_timeout(Some(wait)) -> io::Result<()>` |
|
|
| `write_frame`, `read_frame` | `write_frame(&mut stream, &env) -> Result<(), FrameError>`; `read_frame(&mut stream) -> Result<Envelope, FrameError>` |
|
|
| `std::fs::read` | `std::fs::read(path) -> io::Result<Vec<u8>>` |
|
|
| `Iterator::rposition` | `text.iter().rposition(\|b\| *b == b'\n') -> Option<usize>` |
|
|
| `slice::split_inclusive` | `bytes.split_inclusive(\|b\| *b == b'\n')`: each line with its newline, no empty last piece |
|
|
| `serde_json::from_slice` | `serde_json::from_slice::<LogRecord>(line) -> serde_json::Result<LogRecord>` |
|
|
| `Path::join` | `sessions.join(session.as_str()).join("0.jsonl")` |
|
|
| `BTreeSet::insert`, `remove` | `self.unreadable.insert(s) -> bool` (true when it was not there); `remove(s: &str)` |
|
|
| `turns`, `turn_records` | task 08's: `turns(records) -> u64`; `turn_records(records, turn) -> Option<&[LogRecord]>` |
|
|
| `Option::is_some_and` | `turn_records(records, limit).is_some_and(finished)` |
|
|
| `matches!` with a guard | `matches!(last, Some(LogRecord::Assistant { tool_calls, .. }) if tool_calls.is_empty())` |
|
|
|
|
## The edits, exactly
|
|
|
|
Nothing else in these files changes. Each quoted "before" block occurs exactly once in its file.
|
|
|
|
### `crates/memoryd/src/honcho/mod.rs`
|
|
|
|
**H1.** The line
|
|
|
|
```rust
|
|
pub mod ledger;
|
|
```
|
|
|
|
becomes
|
|
|
|
```rust
|
|
pub mod ledger;
|
|
pub mod state;
|
|
pub mod turns;
|
|
```
|
|
|
|
### `crates/memoryd/src/serve.rs`
|
|
|
|
**S1.** The lines
|
|
|
|
```rust
|
|
//! Serving: three sockets, one request per connection, each answered by one frame (M5a spec,
|
|
//! section 7). One lock covers the files and the index, so writes never interleave.
|
|
```
|
|
|
|
become
|
|
|
|
```rust
|
|
//! Serving: three sockets, one request per connection, each answered by one frame (M5a spec,
|
|
//! section 7). One lock covers the files and the index, so writes never interleave. With
|
|
//! `[honcho]`, a reported turn is decided after `loopd` has its answer, under a lock of its own
|
|
//! (M5c spec, section 5).
|
|
```
|
|
|
|
**S2.** The lines
|
|
|
|
```rust
|
|
MemoryCall, MemoryCommand, Message, PROTOCOL_VERSION, RecallArgs, Reindexed, Timestamp,
|
|
ToolResponse, WireError, read_frame, write_frame,
|
|
};
|
|
```
|
|
|
|
become
|
|
|
|
```rust
|
|
MemoryCall, MemoryCommand, Message, PROTOCOL_VERSION, RecallArgs, Reindexed, SessionId,
|
|
SessionUpdated, Timestamp, ToolResponse, WireError, read_frame, write_frame,
|
|
};
|
|
```
|
|
|
|
**S3.** The line
|
|
|
|
```rust
|
|
use crate::files::Store;
|
|
```
|
|
|
|
becomes
|
|
|
|
```rust
|
|
use crate::files::Store;
|
|
use crate::honcho::ledger::{self, Ledger, LedgerError};
|
|
use crate::honcho::state;
|
|
use crate::honcho::turns::{self, TurnSync};
|
|
```
|
|
|
|
**S4.** In `pub struct Memoryd`, the lines
|
|
|
|
```rust
|
|
state: Mutex<State>,
|
|
log: Log,
|
|
}
|
|
```
|
|
|
|
become
|
|
|
|
```rust
|
|
state: Mutex<State>,
|
|
/// With `[honcho]`: the turns reported and not yet decided, and the ledger. Never taken
|
|
/// while `state` is held, and never held by a request that waits for its answer.
|
|
honcho: Option<Mutex<TurnSync>>,
|
|
log: Log,
|
|
}
|
|
|
|
/// Why `start` failed. `main` prints a start failure with the pointer `memoryd-start-failed`, and
|
|
/// a ledger that does not read through with its own.
|
|
#[derive(Debug)]
|
|
pub enum StartError {
|
|
Failed(String),
|
|
Ledger(LedgerError),
|
|
}
|
|
|
|
impl std::fmt::Display for StartError {
|
|
fn fmt(&self, f: &mut std::fmt::Formatter<'_>) -> std::fmt::Result {
|
|
match self {
|
|
StartError::Failed(why) => write!(f, "{why}"),
|
|
// `Damaged` names its pointer itself.
|
|
StartError::Ledger(e @ LedgerError::Damaged { .. }) => write!(f, "{e}"),
|
|
StartError::Ledger(e) => write!(f, "{e}; {}", ledger::RUNBOOK),
|
|
}
|
|
}
|
|
}
|
|
|
|
impl std::error::Error for StartError {}
|
|
|
|
impl From<String> for StartError {
|
|
fn from(why: String) -> StartError {
|
|
StartError::Failed(why)
|
|
}
|
|
}
|
|
```
|
|
|
|
**S5.** The lines
|
|
|
|
```rust
|
|
/// Create what is missing under `memory/`, open the index and catch it up, then bind the three
|
|
/// sockets. The error names what failed; the caller points at `memoryd-start-failed`.
|
|
pub fn start(config: Config, log: Log) -> Result<Running, String> {
|
|
let store = Store::new(&config.memory_dir());
|
|
```
|
|
|
|
become
|
|
|
|
```rust
|
|
/// With `[honcho]`, read the ledger through first. Then create what is missing under `memory/`,
|
|
/// open the index and catch it up, and bind the three sockets. The error names what failed.
|
|
pub fn start(config: Config, log: Log) -> Result<Running, StartError> {
|
|
let honcho = match config.honcho_config()? {
|
|
Some(honcho) => {
|
|
let path = config.memory_dir().join(turns::LEDGER);
|
|
let ledger = Ledger::open(&path).map_err(StartError::Ledger)?;
|
|
if ledger.repaired() {
|
|
(log)(
|
|
format!(
|
|
"memoryd: {}: a last line cut off by a crash was dropped",
|
|
path.display()
|
|
)
|
|
.as_str(),
|
|
);
|
|
}
|
|
Some(Mutex::new(TurnSync::new(
|
|
ledger,
|
|
&honcho.agent,
|
|
&config.sessions_dir(),
|
|
)))
|
|
}
|
|
None => None,
|
|
};
|
|
let store = Store::new(&config.memory_dir());
|
|
```
|
|
|
|
**S6.** In `start`, where `Memoryd` is built, the lines
|
|
|
|
```rust
|
|
state: Mutex::new(State { index }),
|
|
log,
|
|
```
|
|
|
|
become
|
|
|
|
```rust
|
|
state: Mutex::new(State { index }),
|
|
honcho,
|
|
log,
|
|
```
|
|
|
|
**S7.** In `Memoryd::connection`, the lines
|
|
|
|
```rust
|
|
let reply = self.handle(request.msg, which);
|
|
let env = Envelope {
|
|
v: PROTOCOL_VERSION,
|
|
id: request.id,
|
|
r#final: true,
|
|
msg: reply,
|
|
};
|
|
let _ = write_frame(&mut stream, &env);
|
|
}
|
|
```
|
|
|
|
become
|
|
|
|
```rust
|
|
let update = match (which, &request.msg) {
|
|
(Which::Loop, Message::SessionUpdated(update)) => Some(update.clone()),
|
|
_ => None,
|
|
};
|
|
let reply = self.handle(request.msg, which);
|
|
let env = Envelope {
|
|
v: PROTOCOL_VERSION,
|
|
id: request.id,
|
|
r#final: true,
|
|
msg: reply,
|
|
};
|
|
let _ = write_frame(&mut stream, &env);
|
|
// `loopd` has its answer before `brokerd` is asked: it sends the turn's reply only after
|
|
// this frame, and `brokerd` may take seconds.
|
|
drop(stream);
|
|
if let Some(update) = update {
|
|
self.decide_turns(&update);
|
|
}
|
|
}
|
|
|
|
/// With `[honcho]`: decide the reported turn, and every other that waits (`TurnSync`).
|
|
fn decide_turns(&self, update: &SessionUpdated) {
|
|
let Some(honcho) = &self.honcho else {
|
|
return;
|
|
};
|
|
let socket = self.config.state_socket();
|
|
let ask = |session: &SessionId| state::ask(&socket, session, state::WAIT);
|
|
let mut sync = honcho.lock().unwrap_or_else(|p| p.into_inner());
|
|
sync.updated(
|
|
&update.session,
|
|
update.turn,
|
|
&ask,
|
|
Timestamp::now(),
|
|
&*self.log,
|
|
);
|
|
}
|
|
```
|
|
|
|
### `crates/memoryd/src/main.rs`
|
|
|
|
**M1.** The lines
|
|
|
|
```rust
|
|
Ok(running) => running,
|
|
Err(e) => {
|
|
eprintln!("memoryd: {e}\n{START_FAILED}");
|
|
```
|
|
|
|
become
|
|
|
|
```rust
|
|
Ok(running) => running,
|
|
// The ledger's line names its own pointer, `honcho-ledger-damaged`.
|
|
Err(e @ serve::StartError::Ledger(_)) => {
|
|
eprintln!("memoryd: {e}");
|
|
return ExitCode::from(1);
|
|
}
|
|
Err(e) => {
|
|
eprintln!("memoryd: {e}\n{START_FAILED}");
|
|
```
|
|
|
|
### `docs/runbook.md`
|
|
|
|
**R1.** In `session-log-damaged`, the lines
|
|
|
|
```text
|
|
**What you see.** A turn on an existing session fails with `<file>:<line>: <reason>` naming
|
|
`sessions/<id>/<epoch>.jsonl`, and this entry. Other sessions work.
|
|
```
|
|
|
|
become
|
|
|
|
```text
|
|
**What you see.** A turn on an existing session fails with `<file>:<line>: <reason>` naming
|
|
`sessions/<id>/<epoch>.jsonl`, and this entry. Other sessions work. With `[honcho]`, `memoryd`
|
|
logs the same line once, starting `memoryd: `, and that session's turns wait undecided for Honcho
|
|
until the log reads again.
|
|
```
|
|
|
|
**R2.** In `honcho-ledger-damaged` (the entry task 07 appended), the lines
|
|
|
|
````text
|
|
**What you see.** `memoryd` logs a line naming `memory/honcho.jsonl`, a line number and a reason,
|
|
then this entry:
|
|
|
|
```text
|
|
/srv/bx/memory/honcho.jsonl:12: turn 3 of session chat-1: decided already; see docs/runbook.md#honcho-ledger-damaged
|
|
```
|
|
````
|
|
|
|
become
|
|
|
|
````text
|
|
**What you see.** `memoryd` exits 1 at start, before it binds its sockets, with one line naming
|
|
`memory/honcho.jsonl`, a line number and a reason, then this entry:
|
|
|
|
```text
|
|
memoryd: /srv/bx/memory/honcho.jsonl:12: turn 3 of session chat-1: decided already; see docs/runbook.md#honcho-ledger-damaged
|
|
```
|
|
|
|
Or, while it runs, `memoryd: <path>: <error>; see docs/runbook.md#honcho-ledger-damaged` when the
|
|
ledger cannot be written (a full disk, a wrong owner). The turn waits undecided, and is decided at
|
|
a later turn's end once the ledger can be written.
|
|
````
|
|
|
|
**R3.** In `honcho-ledger-damaged`, the lines
|
|
|
|
```text
|
|
does not guess: it does not start its Honcho sync, and nothing is decided or sent to Honcho until
|
|
the ledger reads through. The rules: an item is decided once (`decided already`), and it is sent
|
|
or refused once, only after it is decided and only when it was not decided to go nowhere (`not
|
|
waiting to be sent`). A last line cut off by a crash is not damage: `memoryd` drops it and says so
|
|
in its log.
|
|
```
|
|
|
|
become
|
|
|
|
```text
|
|
does not guess: it does not start. While it is stopped no memory is served either, and `loopd`
|
|
cannot start a session (`memory-unavailable`). The rules: an item is decided once (`decided
|
|
already`), and it is sent or refused once, only after it is decided and only when it was not
|
|
decided to go nowhere (`not waiting to be sent`). A last line cut off by a crash is not damage:
|
|
`memoryd` drops it and says so in its log.
|
|
```
|
|
|
|
**R4.** In `honcho-ledger-damaged`, the lines
|
|
|
|
```text
|
|
**Fix.** Keep a copy first: `cp -a "$f" "$f.damaged-$(date +%s)"`. The ledger is written only by
|
|
`memoryd`, so a damaged line was changed by hand or by the disk. If you know what the line should
|
|
say, correct it. Otherwise remove it (`sed -i '<line>d' "$f"`), knowing what that costs: a removed
|
|
`sent` or `failed` line means the item is sent again (a turn's messages appear twice in Honcho, a
|
|
conclusion is made twice); a removed `decided` line means the item is decided again, and any
|
|
later `sent` or `failed` line of the same item then breaks the rules, so remove those too. Honcho
|
|
cannot delete a message, so a turn sent twice stays twice.
|
|
```
|
|
|
|
become
|
|
|
|
```text
|
|
**Fix.** If the ledger cannot be written, free space or correct the owner of `memory/`
|
|
(`df -h`, `ls -l "$f"`); nothing else is needed. For a damaged line, keep a copy first:
|
|
`cp -a "$f" "$f.damaged-$(date +%s)"`. The ledger is written only by `memoryd`, so a damaged line
|
|
was changed by hand or by the disk. If you know what the line should say, correct it. Otherwise
|
|
remove it (`sed -i '<line>d' "$f"`), knowing what that costs: a removed `sent` or `failed` line
|
|
means the item is sent again (a turn's messages appear twice in Honcho, a conclusion is made
|
|
twice); a removed `decided` line means the item is decided again, and any later `sent` or
|
|
`failed` line of the same item then breaks the rules, so remove those too. Honcho cannot delete a
|
|
message, so a turn sent twice stays twice.
|
|
```
|
|
|
|
## Steps
|
|
|
|
- [ ] **1. Copy the given tests.** From the repository root, five commands:
|
|
|
|
```sh
|
|
cp docs/plans/M5c1/files/crates/memoryd/tests/honcho_state.rs crates/memoryd/tests/
|
|
cp docs/plans/M5c1/files/crates/memoryd/tests/honcho_turns.rs crates/memoryd/tests/
|
|
cp docs/plans/M5c1/files/crates/memoryd/tests/honcho_serve.rs-task09 crates/memoryd/tests/honcho_serve.rs
|
|
cp docs/plans/M5c1/files/crates/memoryd/tests/support/brokerd.rs crates/memoryd/tests/support/
|
|
cp docs/plans/M5c1/files/crates/memoryd/tests/support/turns.rs crates/memoryd/tests/support/
|
|
```
|
|
|
|
- [ ] **2. See them fail.** `cargo build --tests --keep-going -p memoryd`, with a shell timeout
|
|
of 600000 ms: three test targets do not compile, one `E0432` each:
|
|
`honcho_state` (``unresolved import `memoryd::honcho::state` ``), `honcho_turns`
|
|
(``unresolved import `memoryd::honcho::turns` ``) and `honcho_serve` (``unresolved import
|
|
`memoryd::serve::StartError` ``). Every other target compiles.
|
|
- [ ] **3. The modules.** Make edit H1, then copy the two skeletons:
|
|
|
|
```sh
|
|
cp docs/plans/M5c1/files/crates/memoryd/src/honcho/state.rs crates/memoryd/src/honcho/
|
|
cp docs/plans/M5c1/files/crates/memoryd/src/honcho/turns.rs crates/memoryd/src/honcho/
|
|
```
|
|
|
|
`cargo check -p memoryd`: no error, and 19 warnings, all from the unfilled bodies: four
|
|
`unused import` lines (`std::os::unix::net::UnixStream`; `Envelope`, `Message`,
|
|
`PROTOCOL_VERSION`, `SessionState`, `read_frame` and `write_frame`; `crate::honcho::state`;
|
|
`turn_records` and `turns`), fourteen `unused variable` (`session` four times, `log` twice,
|
|
and `socket`, `wait`, `path`, `sessions`, `records`, `reported`, `turn`, `ask` once each), and
|
|
``fields `sessions`, `broker_down`, and `unreadable` are never read``. They go as you fill the bodies.
|
|
- [ ] **4. Fill the seven bodies, one at a time**, each from its comment, and run
|
|
`cargo check -p memoryd` after each: `ask` in `state.rs`, then in `turns.rs` `log_path`,
|
|
`read_log`, `finished`, `records`, `state`, `decidable`. When a body is filled, delete its
|
|
instruction comment (the `//` lines inside the body) and keep the function's doc comment
|
|
(`///`) as it is. Change nothing else in the two files: `TurnSync`, `new`, `waiting`,
|
|
`updated` and `walk` are given. After the seventh, `cargo check -p memoryd` prints no warning,
|
|
and this command passes all its tests (5 and 16):
|
|
|
|
```sh
|
|
cargo test -p memoryd --test honcho_state --test honcho_turns
|
|
```
|
|
- [ ] **5. Serving.** Make edits S1 to S7 in `serve.rs`, then M1 in `main.rs`.
|
|
`cargo check -p memoryd`: no error and no warning.
|
|
- [ ] **6. The tests.** This command, with a shell timeout of 600000 ms, and every test passes
|
|
(6, 16, 5, and the existing `serve` and `honcho_start` tests unchanged):
|
|
|
|
```sh
|
|
cargo test -p memoryd --test honcho_serve --test honcho_turns --test honcho_state \
|
|
--test serve --test honcho_start
|
|
```
|
|
- [ ] **7. The runbook.** Make edits R1 to R4 in `docs/runbook.md`, then from the repository
|
|
root:
|
|
|
|
```sh
|
|
{ echo; cat docs/plans/M5c1/files/runbook-broker-state-unavailable.md; } >> docs/runbook.md
|
|
```
|
|
|
|
Then `grep -c '^## broker-state-unavailable$' docs/runbook.md` prints `1`, and
|
|
`grep -c 'does not start its Honcho sync' docs/runbook.md` prints `0`.
|
|
- [ ] **8. Check.** Three commands:
|
|
- `grep -n 'decide_turns\|let _ = write_frame(&mut stream, &env);' crates/memoryd/src/serve.rs`
|
|
prints three lines, in this order: the `write_frame` line, the call
|
|
`self.decide_turns(&update);`, and `fn decide_turns(`. `loopd` has its answer first.
|
|
- `grep -rn 'runbook.md#broker-state-unavailable' crates/memoryd/src` prints exactly one line,
|
|
the `RUNBOOK` constant in `honcho/state.rs`.
|
|
- `git diff --stat -- crates/bxcore crates/loopd crates/brokerd crates/gatewayd crates/bxctl
|
|
Cargo.toml crates/memoryd/Cargo.toml crates/memoryd/src/honcho/decide.rs
|
|
crates/memoryd/src/honcho/ledger.rs crates/memoryd/src/config.rs` prints nothing.
|
|
- [ ] **9. Run the gate.** `cargo fmt --all`, then `make gate`, with a shell timeout of 600000 ms.
|
|
Expected last line: `gate: ok`, with 1257 tests: 1230 before this task, plus 27 new ones (5 in
|
|
`honcho_state.rs`, 16 in `honcho_turns.rs`, 6 in `honcho_serve.rs`).
|
|
- [ ] **10. Log and commit.** Add your row to `docs/implementer-log.md`; its Task cell is
|
|
`M5c1/09-memoryd-honcho-turns`. `git status --short` lists `honcho/mod.rs`, `serve.rs`,
|
|
`main.rs`, `docs/runbook.md` and your log as modified, and `honcho/state.rs`, `honcho/turns.rs`,
|
|
the three test files and the two support files as new. Stage with
|
|
`git add crates/memoryd docs/runbook.md docs/implementer-log.md Cargo.lock`, then commit as
|
|
`AGENTS.md` says. `Cargo.lock` does not change in this task; the `git add` is harmless.
|
|
|
|
## Done when
|
|
|
|
- `make gate` prints `gate: ok` with 1257 tests.
|
|
- `git diff HEAD~1 --stat` lists only the files of step 10.
|
|
|
|
## Stop and report if
|
|
|
|
- A "before" block of H1, S1 to S7, M1 or R1 to R4 is not in its file exactly as shown, or is
|
|
there more than once.
|
|
- Step 2 or 3 shows a different result: other test targets, errors of another kind, or other
|
|
warnings.
|
|
- `cargo check` reports an error after a body you filled or an edit you made, and the comment or
|
|
the edit does not tell you how to fix it. Quote the error, and stop.
|
|
- A comment in a skeleton does not fit the code under it, or disagrees with this task's text or
|
|
"The lines `memoryd` logs". Quote both and stop.
|
|
- After step 4 or 6 a test fails although each body follows its comment and each edit is as
|
|
shown. Do not change a test, a support file or the skeletons' given code: stop, and give the
|
|
failing tests' names and their messages.
|
|
- The fix seems to need a change anywhere else: `decide.rs`, `ledger.rs`, `config.rs`, another
|
|
crate, a new thread, or a lock held while `brokerd` is asked other than `honcho`'s. It does
|
|
not; stop and say why you think so.
|
|
- A test outside the new files fails in the gate: check once whether it fails without your
|
|
change, then stop and report both results.
|
|
- The gate passes with a count other than 1257.
|
|
|
|
## What the given tests pin
|
|
|
|
Each of these changes to the finished code makes at least one given test fail (42 in all): the
|
|
map keeping the lowest or the latest report instead of the highest; walking only the reported
|
|
session; keeping a finished session, or dropping one whose last turn waits; going on after a
|
|
silent `brokerd`, or after a ledger write failed; ignoring the log's length or the report; deciding
|
|
a running last turn; deciding a turn twice; asking with nothing undecided; logging the outage, the
|
|
return or a damaged log more than once, or not at all, or without its pointer; deciding a
|
|
session whose log does not read; parsing a line being written; line numbers from 0; a missing log
|
|
taken as an error; any `Assistant` record, or the first, ending a turn; another epoch's file;
|
|
deciding before `loopd` has its answer, or under the index lock, or not at all, or for a report on
|
|
another socket; the repaired ledger not logged; a damaged ledger ignored, or read after a socket
|
|
is bound; the ledger's error printed without its pointer, with it twice, or with the start
|
|
pointer too; no read timeout; an error frame taken for a clean session; the wrong session asked;
|
|
another frame id.
|
|
|
|
## After this task
|
|
|
|
- A restart forgets the sessions that wait. Task 11 rebuilds them at start (from the logs and the
|
|
ledger, with `decidable`, so that a running turn is never decided).
|
|
- Whether turns from before `[honcho]` was turned on are decided at all is the owner's open
|
|
question; if not, a later task adds a cutoff as a new skip row in `decide`.
|