# M5c-1 task 07: `memoryd`'s Honcho ledger, `memory/honcho.jsonl` **Branch:** `m5c1` (run `git switch m5c1`; `git status --short` must be empty, otherwise stop) **Commit subject:** `memoryd: the Honcho ledger, decided, sent and failed, read through at start` **Log row:** the Task cell is `M5c1/07-memoryd-honcho-ledger`. ## Goal From M5c on, `memoryd` sends finished turns and memory writes to Honcho (M5c spec, section 5). Each one is first **decided** (where it goes, or that it goes nowhere), then **sent** or **failed** (Honcho refused it). This task adds the record of that: the ledger, an append-only file of JSON lines, `/memory/honcho.jsonl`. What is decided and neither sent nor failed is the **outbox**, the work still to do; there is no other queue. `memoryd` reads the ledger through when it starts, so nothing is decided twice or sent twice. This task is the module `memoryd::honcho::ledger` alone. Nothing calls it yet: task 09 opens it at start and decides turns into it, later tasks send from its outbox. Its path is passed in. - **Items.** A turn of a session (`Item::Turn { session, turn }`), or one part of a journalled memory edit (`Item::Edit { id, part }`; the id is the journal's, `2026-09-28-0001`). - **Actions** (what a `Decided` record says): the turn's messages to a workspace; a conclusion to create or delete; a message, or its retraction, in the fleet workspace; or `Skip`, nothing to send, and why. A turn's text is never stored here: the sender reads it from the session log. - **The rules.** An item has exactly one `Decided` record. It has at most one `Sent` or `Failed` record, and only while it is **waiting**: decided, not `Skip`, and neither sent nor failed yet. `append` refuses a record that breaks a rule, and writes nothing for it. - **Durable before believed.** `append` writes the line, syncs it to disk, and only then takes it into the ledger's state in memory, as `loopd` does with its session log. A failed write leaves the state as it was, so the ledger never says an item is decided or sent when the file does not. - **Reading through.** `open` reads the whole file and replays it. A missing file is an empty ledger. A last line with no newline was cut off by a crash in the middle of an append: it is dropped and cut from the file, and `repaired()` says so. Any other line that does not parse, or that breaks a rule when replayed in order, stops the ledger: `LedgerError::Damaged` with its line number and the runbook pointer `honcho-ledger-damaged`, and the file is left as it is. ## Interfaces Later tasks use these exactly as written. They are in the skeleton you copy (step 3); only 12 private or small function bodies are yours to fill. In `crates/memoryd/src/honcho/ledger.rs` (the module `memoryd::honcho::ledger`): ```rust pub const RUNBOOK: &str = "see docs/runbook.md#honcho-ledger-damaged"; pub enum Workspace { Trusted, Untrusted, Fleet } pub enum Item { Turn { session: SessionId, turn: u64 }, Edit { id: String, part: u32 } } pub enum Action { Turn { workspace: Workspace, speaker: String }, Conclude { workspace: Workspace, peer: String, session: SessionId, text: String }, Unconclude { workspace: Workspace, text: String }, Fleet { session: SessionId, text: String }, Retract { session: SessionId, text: String }, Skip { why: String }, } pub enum Record { Decided { time: Timestamp, item: Item, action: Action }, Sent { time: Timestamp, item: Item, ids: Vec }, Failed { time: Timestamp, item: Item, status: Option, kind: String }, } pub struct Counts { pub decided: usize, pub sent: usize, pub waiting: usize, pub failed: usize, pub skipped: usize } pub enum LedgerError { Io(PathBuf, std::io::Error), Damaged { path: PathBuf, line: usize, why: String }, Encode(serde_json::Error), AlreadyDecided(Item), NotWaiting(Item), } pub struct Ledger { /* private */ } impl Ledger { pub fn open(path: &Path) -> Result; pub fn repaired(&self) -> bool; pub fn append(&mut self, record: Record) -> Result<(), LedgerError>; pub fn is_decided(&self, item: &Item) -> bool; pub fn outbox(&self) -> Vec<(Item, Action)>; pub fn sent_ids(&self, item: &Item) -> Option<&[String]>; pub fn conclusion(&self, workspace: Workspace, text: &str) -> Option<(Item, String)>; pub fn counts(&self) -> Counts; } ``` `SessionId` and `Timestamp` are `bxcore`'s. The derives and serde attributes are in the skeleton; do not change them (see "The lines"). ## What each public function gives | Function | Gives | |---|---| | `open` | the ledger read through; see "Reading through" above | | `repaired` | `true` when `open` dropped a cut-off last line | | `append` | the record checked, written with its newline in one write, synced, then taken into the state; a refused or failed append changes nothing | | `is_decided` | whether the item has its `Decided` record (a `Skip` counts) | | `outbox` | every waiting item with its action, in the order of the `Decided` records | | `sent_ids` | the ids of the item's `Sent` record; `None` when it has none (waiting, failed, skipped, unknown) | | `conclusion` | the conclusion to delete for an undo: see below | | `counts` | how many items are decided (every one, `Skip` included), sent, waiting, failed and skipped; `decided` is always the sum of the other four | **`conclusion(workspace, text)`.** An undo must delete the conclusion that the undone write created. Honcho gave its id in the `Sent` record of the `Conclude` (a `Conclude`'s `Sent` has exactly one id; one with no ids is ignored). Replay the `Sent` records in the order they were appended: a sent `Conclude` of this workspace and text puts `(item, its id)` on top of a stack, and a sent `Unconclude` of the same workspace and text takes the top one off. The answer is the top of the stack, or `None`. So it is the latest one sent that is not undone yet; another workspace's, or one decided but not sent, is never the answer. **The errors' text** (`Display`, given in the skeleton): | Error | Text | |---|---| | `Io(path, e)` | `: ` | | `Damaged { path, line, why }` | `:: ; see docs/runbook.md#honcho-ledger-damaged` (the line counted from 1) | | `Encode(e)` | `a ledger record could not be encoded: ` | | `AlreadyDecided(item)` | `turn 3 of session chat-1: decided already`, or `part 0 of edit 2026-09-28-0001: decided already` | | `NotWaiting(item)` | `turn 3 of session chat-1: not waiting to be sent` (the same naming) | When a replayed line breaks a rule, `why` is the text of `AlreadyDecided` or `NotWaiting`; when it does not parse, `why` is serde's error text. ## The lines One record per line, as `serde_json::to_string` writes it, then `\n`. Every enum except `Workspace` is internally tagged: its variant is the field `"type"`, in snake case, first; and every one refuses unknown fields. `Workspace` is a plain string. A `Timestamp` is RFC 3339 with milliseconds; a `SessionId` a string. For example (each is one line in the file): ```text {"type":"decided","time":"2026-09-28T10:00:00.000Z","item":{"type":"turn","session":"chat-1","turn":3},"action":{"type":"turn","workspace":"trusted","speaker":"kyle"}} {"type":"decided","time":"2026-09-28T10:00:00.000Z","item":{"type":"edit","id":"2026-09-28-0001","part":0},"action":{"type":"conclude","workspace":"trusted","peer":"kyle","session":"chat-1","text":"Kyle takes his coffee black."}} {"type":"sent","time":"2026-09-28T10:00:00.000Z","item":{"type":"turn","session":"chat-1","turn":3},"ids":["msg-a1","msg-a2"]} {"type":"failed","time":"2026-09-28T10:00:00.000Z","item":{"type":"turn","session":"chat-2","turn":1},"status":null,"kind":"refused"} ``` This is our own format, so unknown fields are errors: `#[serde(tag = "type", rename_all = "snake_case", deny_unknown_fields)]` on `Item`, `Action` and `Record`. serde honours `deny_unknown_fields` on internally tagged enums, at every level (a given test adds a field to the record, to its item and to its action, and expects an error each time). The skeleton has all of this already. ## Every place that changes This is the complete list. | Place | What happens there | |---|---| | `crates/memoryd/src/honcho/mod.rs` | `pub mod ledger;` (edit H1) | | `crates/memoryd/src/honcho/ledger.rs` | new: a skeleton you copy, then fill 12 bodies (step 4) | | `docs/runbook.md` | the entry `honcho-ledger-damaged`, appended (step 6) | | everything else in `memoryd` (`config.rs`, `serve.rs`, `main.rs`, `honcho/config.rs`, …) | **no change** | ## Files (A copy command names `-taskNN` where a later task gives a newer version of that file: this task's version is the one named.) - Copy, from `docs/plans/M5c1/files/` (the same path under it): - new tests: `crates/memoryd/tests/honcho_ledger.rs`, `crates/memoryd/tests/honcho_ledger_file.rs` - new test support: `crates/memoryd/tests/support/ledger.rs` - the skeleton `crates/memoryd/src/honcho/ledger.rs` (you fill 12 bodies in it) - Append, from `docs/plans/M5c1/files/`, to the end of `docs/runbook.md`, with one blank line before it: `runbook-honcho-ledger-damaged.md` (step 6 gives the command). - Modify: `crates/memoryd/src/honcho/mod.rs`, `docs/implementer-log.md`. - Do not touch: any other source file, any file of another crate, any `Cargo.toml`, any other test, `crates/memoryd/tests/support/tmp.rs`, and in the skeleton anything but the 12 `todo!()` bodies. ## Facts you need - **The state in memory.** The skeleton gives it: `order` (every decided item, in the order of its `Decided` record), `entries` (a `HashMap` from each item to its `Entry { action, state }`), `sends` (every sent item, in the order of its `Sent` record), and `repaired`. An entry's `State` is `Skipped` (decided `Skip`), `Waiting`, `Sent(ids)` or `Failed`. Only `apply` changes them, and only for a record `check` accepted. - **`open` and `append` are given**, as glue over your functions. `open` checks every complete line before it cuts a cut-off tail, so a damaged file is never changed. `append` calls `check`, encodes, calls `write_line`, and only then `apply`. Read both before you start. - **One write per line.** `write_line` builds the line and its newline as one string and writes it with one `write_all`, then `sync_data`: a crash leaves either the whole line or a cut-off line without its newline, which `open` drops. Two writes, or no sync, would pass every test and still be wrong; step 7 checks for them. - **A missing file creates nothing.** `open` only reads. `write_line` creates the file on the first append, at mode `0o600`, and never creates the directory: a missing `memory/` is an `Io` error. The file's mode is given only when the file is created (`OpenOptionsExt::mode`, and the process's umask can only take bits away). - **Bytes, not text.** `open` reads the file as bytes (`std::fs::read`), because a cut-off last line can end inside a multi-byte character. A complete line is parsed with `serde_json::from_slice`, which accepts the trailing newline. - **The given tests.** - `honcho_ledger.rs` (12 tests): each kind of line exactly; `append` writes the line and a newline; unknown fields and values refused; the outbox (what is in it, and its order); a second decision refused and nothing written; `Sent` or `Failed` refused before the decision, after an outcome and for a `Skip`; `sent_ids`; `conclusion` (the latest not undone, only its workspace and text, only once sent); the counts add up. - `honcho_ledger_file.rs` (12 tests): a missing and an empty file; reopening gives the same state; a cut-off last line (in the middle, one byte short, inside a character, the only line) dropped and cut; a damaged line and a line that breaks a rule, with their numbers, the text and the file unchanged; mode 0600; a missing directory; a failed write changes nothing (the test turns the ledger's path into a directory); an unreadable ledger. - `support/ledger.rs`: record builders (`decided`, `sent`, `failed`, `turn`, `edit`, …) and `Home`, a ledger path in a directory removed on drop. - The calls the bodies use: | Call | Signature, and what it gives | |---|---| | `HashMap::contains_key` | `self.entries.contains_key(item) -> bool` | | `HashMap::get`, `get_mut` | `self.entries.get(item) -> Option<&Entry>`; `get_mut(&item) -> Option<&mut Entry>` | | `HashMap::insert` | `self.entries.insert(item, Entry { action, state })` (the old value, ignored) | | `HashMap::values` | `self.entries.values()`: every `&Entry` | | `Iterator::filter_map` | `.iter().filter_map(\|item\| { ..; Some(..) }).collect()` | | `bool::then` | `(cond).then(\|\| value) -> Option`: `Some(value)` when `cond` | | `slice::first` | `ids.first() -> Option<&String>` | | `Vec::push`, `Vec::pop` | `live.push(x)`; `live.pop() -> Option`: takes the last one off | | `Vec::as_slice` | `ids.as_slice() -> &[String]` | | `OpenOptions` | `OpenOptions::new().append(true).create(true).mode(0o600).open(&path) -> io::Result`; `.mode` is `std::os::unix::fs::OpenOptionsExt` (imported) | | `Write::write_all` | `file.write_all(bytes: &[u8]) -> io::Result<()>` (`std::io::Write`, imported) | | `File::sync_data` | `file.sync_data() -> io::Result<()>`: the data on disk, not necessarily the metadata | | `File::set_len` | `file.set_len(size: u64) -> io::Result<()>`: shrinks the file to `size` bytes; the file needs `write(true)` | | `std::fs::read` | `std::fs::read(path) -> io::Result>` | | `io::Error::kind` | `e.kind() == std::io::ErrorKind::NotFound` | | `io::Error::other` | `std::io::Error::other(e)`: wraps any error as an `io::Error` | | `u64::try_from` | `u64::try_from(keep: usize) -> Result` | | `Iterator::rposition` | `bytes.iter().rposition(\|b\| *b == b'\n') -> Option`: the index of the last newline | | `serde_json::from_slice` | `serde_json::from_slice::(line: &[u8]) -> serde_json::Result` | ## The edit, exactly ### `crates/memoryd/src/honcho/mod.rs` **H1.** The line ```rust pub mod config; ``` becomes ```rust pub mod config; pub mod ledger; ``` ## Steps - [ ] **1. Copy the given tests.** From the repository root, three commands: ```sh cp docs/plans/M5c1/files/crates/memoryd/tests/honcho_ledger.rs-task07 crates/memoryd/tests/honcho_ledger.rs cp docs/plans/M5c1/files/crates/memoryd/tests/honcho_ledger_file.rs-task07 crates/memoryd/tests/honcho_ledger_file.rs cp docs/plans/M5c1/files/crates/memoryd/tests/support/ledger.rs-task07 crates/memoryd/tests/support/ledger.rs ``` - [ ] **2. See them fail.** `cargo build --tests --keep-going -p memoryd`, with a shell timeout of 600000 ms: two test targets do not compile, `honcho_ledger` and `honcho_ledger_file`, with 2 errors each, both `E0432` (``unresolved import `memoryd::honcho::ledger` ``): one in the test file, one in `support/ledger.rs`. Every other target compiles. - [ ] **3. The module.** Make edit H1, then copy the skeleton: ```sh cp docs/plans/M5c1/files/crates/memoryd/src/honcho/ledger.rs crates/memoryd/src/honcho/ ``` `cargo check -p memoryd`: no error, and 20 warnings, all from the unfilled bodies: three `unused import` (`std::fs::OpenOptions`, `std::io::Write`, `std::os::unix::fs::OpenOptionsExt`); fourteen `unused variable` (`path` three times, `item`, `line` and `record` twice each, `workspace`, `text`, `bytes`, `number`, `keep`); ``fields `action` and `state` are never read``; ``fields `path`, `order`, `entries`, and `sends` are never read``; and ``variants `Skipped`, `Waiting`, `Sent`, and `Failed` are never constructed``. They go as you fill the bodies. Then `cargo test -p memoryd --test honcho_ledger`: 2 tests pass (`each_record_is_one_line_exactly_as_given` and `an_unknown_field_or_value_is_refused`: the types are given) and 10 fail with `not yet implemented`. - [ ] **4. Fill the twelve bodies of `honcho/ledger.rs`, one at a time, in the order of the file**, each from its comment, and run `cargo check -p memoryd` after each: `is_decided`, `outbox`, `sent_ids`, `conclusion`, `counts`, `check`, `apply`, `write_line`, then the free functions `read`, `complete`, `parse`, `cut`. When a body is filled, delete its instruction comment (the `//` lines inside the body) and keep the function's doc comment (`///`) as it is. Change nothing else in the file. After the twelfth, `cargo check -p memoryd` prints no warning. - [ ] **5. The tests.** This command, and every test passes (12 and 12): ```sh cargo test -p memoryd --test honcho_ledger --test honcho_ledger_file ``` - [ ] **6. The runbook.** From the repository root: ```sh { echo; cat docs/plans/M5c1/files/runbook-honcho-ledger-damaged.md; } >> docs/runbook.md ``` Then `grep -c '^## honcho-ledger-damaged$' docs/runbook.md` prints `1`. - [ ] **7. Check.** Four commands: - `grep -c 'sync_data' crates/memoryd/src/honcho/ledger.rs` prints `2` (in `write_line` and in `cut`), and `grep -c 'write_all' crates/memoryd/src/honcho/ledger.rs` prints `1`. - `grep -rn 'runbook.md#honcho-ledger-damaged' crates/memoryd/src` prints exactly one line, the `RUNBOOK` constant in `honcho/ledger.rs`. - `grep -n 'create_dir' crates/memoryd/src/honcho/ledger.rs` prints nothing: the ledger never makes its directory. - `git diff --stat -- crates/bxcore crates/loopd crates/brokerd crates/gatewayd crates/bxctl Cargo.toml crates/memoryd/Cargo.toml` prints nothing. - [ ] **8. Run the gate.** `cargo fmt --all`, then `make gate`, with a shell timeout of 600000 ms. Expected last line: `gate: ok`, with 1212 tests: 1188 before this task, plus 24 new ones (12 in `honcho_ledger.rs`, 12 in `honcho_ledger_file.rs`). - [ ] **9. Log and commit.** Add your row to `docs/implementer-log.md`; its Task cell is `M5c1/07-memoryd-honcho-ledger`. `git status --short` lists `crates/memoryd/src/honcho/mod.rs`, `docs/runbook.md` and your log as modified, and `crates/memoryd/src/honcho/ledger.rs`, the two test files and `crates/memoryd/tests/support/ledger.rs` as new. Stage with `git add crates/memoryd docs/runbook.md docs/implementer-log.md Cargo.lock`, then commit as `AGENTS.md` says. `Cargo.lock` does not change in this task; the `git add` is harmless. ## Done when - `make gate` prints `gate: ok` with 1212 tests. - `git diff HEAD~1 --stat` lists only `honcho/mod.rs`, `honcho/ledger.rs`, the two new test files, `tests/support/ledger.rs`, `docs/runbook.md` and `docs/implementer-log.md`. ## Stop and report if - The line of edit H1 is not in `crates/memoryd/src/honcho/mod.rs` exactly as shown, or is there more than once, or `crates/memoryd/tests/support/ledger.rs` exists before step 1. - Step 2 or 3 shows a different result: other test targets, errors of another kind, other warnings, or other tests passing or failing. - `cargo check` reports an error after a body you filled, and the comment does not tell you how to fix it. Quote the error, and stop. - A comment in the skeleton does not fit the code under it, or disagrees with this task's text. Quote both and stop. - After step 4 a test fails although each body follows its comment. Do not change a test, the support file or the skeleton's given code (`open`, `append`, the types, `Display`): stop, and give the failing tests' names and their messages. - The fix seems to need a change anywhere else: another `memoryd` file, another crate, a dependency, or a change to a derive or a serde attribute. It does not; stop and say why you think so. - A test outside these two files fails in the gate: check once whether it fails without your change, then stop and report both results. - The gate passes with a count other than 1212. ## What the given tests pin Each of these changes to a finished `ledger.rs` makes at least one given test fail: the outbox taking in a skipped, sent or failed item, or losing the decision order; `conclusion` ignoring an `Unconclude`, the workspace, the order of the sends, or a `Sent` with no ids, or answering the oldest; a second decision, or a `Sent` or `Failed` before the decision, after an outcome or for a `Skip`, accepted; the state taken in before the write; the directory made; the mode; a cut-off line kept, not cut, cut before the other lines are checked, or not reported by `repaired`; line numbers from 0; replay without the rules; a missing file an error or an unreadable one empty; any error text; the counts; a serde attribute removed. Two writes for one line, and a missing `sync_data`, pass the tests; step 7 checks those.