memoryd: the Honcho ledger, decided, sent and failed, read through at start

Co-Authored-By: ornith-1.5-35b-a3b <agent@wntrmute.dev>

docs/implementer-log.md

@@ -6,6 +6,7 @@ reviewer adds findings under "Reviews" once per milestone.
 
 | Task | Date | Status | Gate runs | First gate | Deviations | Notes | Model |
 |---|---|---|---|---|---|---|---|
+| M5c1/07-memoryd-honcho-ledger | 2026-09-28 | done | 1 | pass | none | Copied the three given files from `docs/plans/M5c1/files/`: the tests `crates/memoryd/tests/honcho_ledger.rs`, `honcho_ledger_file.rs` and the support `crates/memoryd/tests/support/ledger.rs`. Step 2 showed exactly the named failures: two test targets (`honcho_ledger` and `honcho_ledger_file`) with two E0432 each (`unresolved import memoryd::honcho::ledger`), one in each test file and one in `support/ledger.rs`; every other target compiled. Made H1 (`pub mod ledger;` after `pub mod config;` in `honcho/mod.rs`) and copied the skeleton `crates/memoryd/src/honcho/ledger.rs`; `cargo check` gave 20 warnings from the unfilled bodies. Step 3 `cargo test -p memoryd --test honcho_ledger` gave 2 pass (`each_record_is_one_line_exactly_as_given`, `an_unknown_field_or_value_is_refused`) and 10 fail with `not yet implemented`. Filled its twelve bodies in the order of the file, each from its comment and `cargo check` after: `is_decided` (`contains_key`); `outbox` (walk `order` with `.iter().filter_map`, keeping `(item, action)` when `state == Waiting`); `sent_ids` (match `State::Sent(ids)` -> `ids.as_slice()`); `conclusion` (walk `sends` in order, pushing a sent `Conclude`'s first id and popping on a matching sent `Unconclude`, returning `live.pop()` so only workspace and text match and a `Sent` with no ids pushes nothing); `counts` (one per entry, matching `&entry.state` since `State` is not `Copy`); `check` (`AlreadyDecided` for a second `Decided`, `NotWaiting` unless a `Sent`/`Failed` finds its entry `Waiting`); `apply` (`Skip` -> `Skipped` else `Waiting`, push to `order` and `entries`; `Sent` sets `Sent(ids)` and pushes to `sends`, `Failed` sets `Failed` without pushing); `write_line` (append, create, mode 0600, one `write_all` then `sync_data`); then `read` (NotFound -> None, else Io), `complete` (rposition of `\n`), `parse` (serde error -> Damaged), `cut` (`set_len` then `sync_data`). Removed the `mut` in `cut` that clippy flagged (`set_len` and `sync_data` take `&self`); after the twelfth `cargo check` was clean and `cargo fmt` left the file. Both test files passed 12 and 12; appended `runbook-honcho-ledger-damaged.md` (`grep -c '^## honcho-ledger-damaged$'` printed 1). Step 7: `sync_data` 2, `write_all` 1, one runbook pointer line in `src`, no `create_dir`, no diff in the other crates. `make gate` printed `gate: ok` with 1212 tests (1211 in `cargo test --workspace`, plus the ignored `end_to_end` the gate runs with `BOXMAKER_BROKERD` set). No new dependency, so `Cargo.lock` is unchanged. | ornith-1.5-35b-a3b |
 | M5c1/06-memoryd-honcho-config | 2026-09-28 | done | 1 | pass | none | Copied the three given tests from `docs/plans/M5c1/files/crates/memoryd/tests/`: `honcho_config.rs`, `honcho_reasons.rs`, `honcho_start.rs`. Step 2 showed exactly the named failures: `honcho_config` with eight (E0433 cannot find `honcho` in `memoryd`, E0432 unresolved import `bxcore::secrets`, two E0599 for `honcho_config`, three for `state_socket`, one for the `Honcho` variant) and `honcho_reasons` with two (the same E0433 and E0432); `honcho_start` compiled. Made K1 (`bxcore` gains `features = ["secrets"]`) and L1 (`pub mod honcho;` after `pub mod files;`), created `crates/memoryd/src/honcho/mod.rs` with the given one-line module, and copied the skeleton `crates/memoryd/src/honcho/config.rs`. Filled its seven bodies in the given order, each from its comment and `cargo check` after: `is_name` (not empty, byte length `<= MAX_NAME`, every byte `is_ascii_alphanumeric()` or `_` or `-`, bytes not chars so `é` and U+0660 fail); `check_url` (strip `https://` then `http://`, `split_once(':')` into host and port, a `host_ok` helper for the non-empty ASCII-alnum/`.`/`-` host, a port that is non-empty, all digits and parses to `u16` `> 0` via `ok().is_some_and`, the `URL_REASON` which never quotes the URL); `check_own_workspaces` (the three pairs in order, first equal pair is R5 with its name); `trust_of` (match on `as_deref`, `None`/`Some("untrusted")` is `Untrusted`, `Some("trusted")` is `Trusted`, else R9 with the written text); `class_of` (same shape, `Private`/`Public`/`Secret`, else R10); `read_workspace` (check agent then workspace names, R6 the agent is this agent, R7 the agent is listed in `earlier`, R8 the workspace is one of the three, then the `ReadWorkspace` with `trust_of`/`class_of`); `check_keys` (a name other than the three is a read key only when `strip_prefix(READ_KEY)` names an agent in `read`, else R11 with the suffix or R12 without the prefix, then `spec.source()` with R13). Made C1-C8 in `config.rs` (module comment, imports with `BTreeMap`/`SecretSpec`/`honcho`, `Sockets.state`, `Config.honcho` and `.secrets`, `ConfigError::Honcho` and its Display arm, `load` calls `honcho_config` before returning, `honcho_config` and `state_socket`), and M1-M2 in `main.rs` (imports and the pointer matched by error kind on the `Config::load` arm); the pointer edit first landed on the `instance::lock` arm and I moved it to the `Config::load` arm, the diff being exactly M1-M2. Appended `runbook-honcho-config-invalid.md` to `docs/runbook.md` (`grep -c '^## honcho-config-invalid$'` printed 1) and made D1 in `docs/dependencies.md`. Step 7: `cargo build -p memoryd` builds, the runbook grep prints one line, the `secrets::load`/`read_checked` grep prints nothing, the other-crates diff is empty. `cargo test -p memoryd` every suite passes (config 4, honcho_config 8, honcho_reasons 5, honcho_start 4). `make gate` printed `gate: ok` with 1188 tests (1187 in `cargo test --workspace` plus the ignored `end_to_end` the gate runs separately with `BOXMAKER_BROKERD`; 1171 before this task plus the 17 new). No new dependency, so `Cargo.lock` is unchanged. | ornith-1.5-35b-a3b |
 | M5c1/05-brokerd-state-socket | 2026-09-28 | done | 2 | pass | none | Copied five files from `docs/plans/M5c1/files/`: `state_socket.rs` and `serve_state.rs` (new) and new versions of `config.rs`, `serve.rs` and `tests/fixtures/config/full.toml` (all now set and check `state`). Step 2 showed exactly the named failures: `config` with four E0599 (`no method named state_socket`) and `state_socket` with one E0432 (`unresolved import brokerd::state_socket`); the new `serve.rs` compiled. `cargo build --tests` reported those two targets only. Made the edits exactly as the task gives: C1 `Sockets` gains `state: PathBuf` with the doc comment, C2 `Config::state_socket` after `admin_socket` returning the default `run/memory-broker/state.sock` when empty; L1 `pub mod state_socket;` after `pub mod state;`. Created `crates/brokerd/src/state_socket.rs` with the given body: `read_request`, match `Message::SessionState(question)` (no `SessionState` import), `StateStore::new(&broker.cfg().state_dir())`, `Ok` -> `SessionStateReply { class: state.taint, untrusted: state.untrusted }`, `Err` -> log `brokerd: {e}` and send `internal` with `e.to_string()`, `forbid` for any other kind; `send` the final frame. V1-V11 in `serve.rs`: doc comment, `use crate::state_socket`, `Started` gains `state: UnixListener`, `state.sock` bound first then tool and admin, `Which::State`, the `run` destructure, the second `accept_loop` with `tx` moved, the `accept-state` name, and `state_socket::handle` in `serve_one`; M1-M3 in `main.rs`: doc comment, `state_path = cfg.state_socket()`, the `serving session state on <path>` notice. R1-R3 in `docs/runbook.md`: `accept-broker (or accept-admin, accept-state)`, the `state.sock` line in `socket-forbidden`, and `pgrep -a 'loopd|bxctl|memoryd'`. After the edits `cargo check -p brokerd` is clean with no warning; `config` 7, `state_socket` 8, `serve_state` 5, `instance` 2, `serve` 8, `serve_pointers` 10 pass. Step 7: `grep -n 'listen(&cfg'` prints three lines with `state_socket` first, `grep -n '^use' state_socket.rs` prints four and the `bxcore` import carries `ErrorCode`, `Message`, `SessionStateReply`, `WireError` with no `SessionState`, `git diff --stat -- crates/brokerd/src` lists exactly config.rs/lib.rs/main.rs/serve.rs, the other-crates diff is empty. `make gate` printed `gate: ok` with 1171 tests (1158 before this task plus the 13 new: eight in `state_socket.rs`, five in `serve_state.rs`; the new `config.rs` and `serve.rs` kept their counts, 7 and 8). No new dependency, so `Cargo.lock` is unchanged. | ornith-1.5-35b-a3b |
 | M5c1/04-bxctl-speaker | 2026-09-28 | done | 1 | pass | none | Copied four files from `docs/plans/M5c1/files/`: `chat_speaker.rs` (new) and new versions of `chat.rs`, `chat_approvals.rs` and `cli.rs` (did not copy `chat.rs-task01`). Step 2 showed exactly the named failures: four non-compiling test targets (`chat`, `chat_approvals`, `chat_speaker`, `cli`) with E0061 for `run_turn` (5 args) and `stream_turn` (6 args), E0282 on the closure in `chat.rs`, and E0560/E0609 for the missing `speaker` field. Made the edits exactly as the task gives: C1 the `--speaker NAME` flag added to the chat line of `USAGE` with one explaining line (six-space indent); C2 `ChatOptions` gains `speaker: Option<String>`; C3 `default` sets `speaker: None`; C4 the `--speaker` arm in `parse_chat` refuses a second one and falls through to `i += 2`; C5 `parse_speaker` (empty, or any whitespace/control char -> `UsageError`, else the value as typed). R1 `run_turn` gains `speaker: Option<&str>` after `resume` with the doc comment, R2 `speaker.map(str::to_string)` on the `Turn`, R3 `stream_turn` gains the parameter and passes it. M1-M3 all three `stream_turn` calls in `main.rs` pass `opts.speaker.as_deref()`. `main` unchanged (a `UsageError` already prints the usage text and exits 2). After the edits `cargo check -p bxctl` is clean with no warning; `cli` 15 passed, `chat_speaker` 7 passed, every bxctl suite passes. Step 7: `grep -rn 'speaker: None' crates/bxctl/src` prints one line (the default), `git diff --stat -- crates/bxctl/src` lists exactly chat.rs/cli.rs/main.rs, the other-crates diff is empty. `make gate` printed `gate: ok` with 1158 tests (1148 before plus the 10 new: three in `cli.rs`, seven in `chat_speaker.rs`; `chat.rs` and `chat_approvals.rs` kept their counts). No new dependency, so `Cargo.lock` is unchanged. | ornith-1.5-35b-a3b |

docs/runbook.md

@@ -923,3 +923,41 @@ the same in every agent's file. To turn Honcho off, remove `[honcho]`, its `[[ho
 tables and every `[secrets.honcho_…]`.
 
 **Check.** `memoryd serve --config <path>` prints `memoryd: serving`.
+
+## honcho-ledger-damaged
+
+**What you see.** `memoryd` logs a line naming `memory/honcho.jsonl`, a line number and a reason,
+then this entry:
+
+```text
+/srv/bx/memory/honcho.jsonl:12: turn 3 of session chat-1: decided already; see docs/runbook.md#honcho-ledger-damaged
+```
+
+**Why.** The ledger is `memoryd`'s record of what it sends to Honcho: one line when a turn or a
+memory write is decided (where it goes, or that it goes nowhere), and one when it is sent or
+refused (M5c spec, section 5). What is decided and not yet sent is the outbox. `memoryd` reads the
+ledger through at start so that it sends nothing twice and nothing to a workspace it was not
+decided for. A line it cannot read, or one that breaks the ledger's rules, could do either, so it
+does not guess: it does not start its Honcho sync, and nothing is decided or sent to Honcho until
+the ledger reads through. The rules: an item is decided once (`decided already`), and it is sent
+or refused once, only after it is decided and only when it was not decided to go nowhere (`not
+waiting to be sent`). A last line cut off by a crash is not damage: `memoryd` drops it and says so
+in its log.
+
+**Confirm.** Look at the line named, and at every line of the same item:
+
+```sh
+f="$BOXMAKER_HOME/memory/honcho.jsonl"
+sed -n '<line>p' "$f"
+grep -n '"session":"<id>","turn":<n>}' "$f"      # a turn; for an edit: '"id":"<id>","part":<n>}'
+```
+
+**Fix.** Keep a copy first: `cp -a "$f" "$f.damaged-$(date +%s)"`. The ledger is written only by
+`memoryd`, so a damaged line was changed by hand or by the disk. If you know what the line should
+say, correct it. Otherwise remove it (`sed -i '<line>d' "$f"`), knowing what that costs: a removed
+`sent` or `failed` line means the item is sent again (a turn's messages appear twice in Honcho, a
+conclusion is made twice); a removed `decided` line means the item is decided again, and any
+later `sent` or `failed` line of the same item then breaks the rules, so remove those too. Honcho
+cannot delete a message, so a turn sent twice stays twice.
+
+**Check.** `memoryd` starts, and its log has no line with this entry.