Files
boxmaker/docs/plans/M5c1/14-bxctl-memory-sync.md
2026-09-28 09:58:26 -07:00

19 KiB

M5c-1 task 14: bxctl memory sync shows what memoryd sends to Honcho

Branch: m5c1 (run git switch m5c1; git status --short must be empty, otherwise stop) Commit subject: bxctl memory sync: the Honcho ledger's counts and failed items Log row: the Task cell is M5c1/14-bxctl-memory-sync.

Goal

The owner's view of the Honcho ledger (M5c spec, section 5): bxctl memory sync asks memoryd on its admin socket, as memory log, undo, pending, accept and reject do, and prints the answer as it comes.

  • The command. A new bxcore::MemoryCommand::Sync ("sync" on the wire, with an empty arg), sent in the MemoryAdmin message those commands use, and answered, like them, with a MemoryAdminReply { text } that memoryd writes and bxctl prints followed by a newline.

  • The report, exactly, from the ledger: one line of counts, then, if any item failed, a heading and one line per failed item, oldest first (the order of the Failed records), all of them:

    honcho sync: 7 decided: 2 sent, 2 waiting, 2 failed, 1 skipped
    failed, oldest first:
      2026-09-28T10:02:00.000Z  turn 4 of session mm-a  log
      2026-09-28T10:03:00.000Z  part 0 of edit 2026-09-28-0001  status (HTTP 422)
    

    The first line is format!("honcho sync: {} decided: {} sent, {} waiting, {} failed, {} skipped", c.decided, c.sent, c.waiting, c.failed, c.skipped); each failed item is two spaces, its record's time (Timestamp::to_rfc3339), two spaces, its label (turn N of session S, part N of edit ID), two spaces, its kind, and (HTTP <status>) when it has a status. No item's text is ever printed.

  • Without [honcho]: honcho sync: not configured (memoryd.toml has no [honcho]). Not an error: bxctl prints it and exits 0.

  • memoryd down: as for every memory command, cannot reach memoryd at <socket>: <error> on stderr, exit 1 (bxctl's existing run).

The report is read under the ledger's own lock (task 12), inside the admin call's index lock: index → ledger, the one order. The worker's state (a workspace whose key was refused, an outage in progress) is its own, behind no lock the report can take, so it is not in the report; the log has both.

Interfaces

  • bxcore::MemoryCommand gains Sync (edit X1).
  • In crates/memoryd/src/honcho/ledger.rs: pub struct Failure { pub item: Item, pub time: Timestamp, pub status: Option<u16>, pub kind: String } (Debug, Clone, PartialEq, Eq); Ledger::failures(&self) -> Vec<Failure>, oldest first; pub fn label(item: &Item) -> String (the old private describe, renamed and made public). The private State::Failed keeps the record's time, status and kind (edits L1 to L9).
  • In crates/memoryd/src/honcho/report.rs (new): pub const NOT_CONFIGURED: &str and pub fn report(ledger: &Ledger) -> String.
  • Mirror::report(&self) -> String (edits R1, R2), and memoryd's admin answers Sync with it, or with NOT_CONFIGURED (edits S1, S2).
  • bxctl: memory sync parses to Ask::Admin(MemoryCommand::Sync, String::new()) (edits B1 to B3); the usage text names it (edit U1).

Every place that changes

This is the complete list.

Place What happens there
crates/bxcore/src/memory.rs MemoryCommand::Sync (edit X1)
crates/bxctl/src/memory.rs, cli.rs memory sync parsed; the usage text (edits B1 to B3, U1)
crates/memoryd/src/honcho/ledger.rs failures kept, in order; Failure, failures, label (edits L1 to L9)
crates/memoryd/src/honcho/report.rs new: a skeleton you copy, then fill 1 body
crates/memoryd/src/honcho/mirror.rs, mod.rs, serve.rs Mirror::report, pub mod report;, the Sync answer (edits R1, R2, H1, S1, S2)
everything else no change

Files

(A copy command names <file>-taskNN where a later task gives a newer version of that file: this task's version is the one named.)

  • Copy, from docs/plans/M5c1/files/ (the same path under it): the new tests crates/memoryd/tests/honcho_sync.rs and crates/bxctl/tests/cli_memory_sync.rs, and the skeleton crates/memoryd/src/honcho/report.rs.
  • Modify: the files of the edits, docs/implementer-log.md.
  • Do not touch: any other source file, any other test, any Cargo.toml, and in the skeleton anything but its todo!() body.

Facts you need

  • A new MemoryCommand variant changes no existing message: MemoryAdmin's fixture uses undo, and the wire test that names each command loops over a list. Every match on MemoryCommand is in memoryd's admin, which edit S2 extends.
  • Ledger::failures reads the new fails: Vec<Item> (each failed item, in the order of its Failed record) and the time, status and kind each entry's state now keeps; a restart reads the same back from the file.
  • The given tests.
    • honcho_sync.rs (4 tests, memoryd): the report of a ledger with every state, exactly, and of an empty one; the failures in order, and the same after a reopen; label; memoryd answering Sync (the counts before and after a memory write) and, without [honcho], NOT_CONFIGURED.
    • cli_memory_sync.rs (4 tests, bxctl): parsing (and memory sync now refused), the usage line; the wire name sync; the answer printed as it came, with what was sent; memoryd down.
  • The calls the body uses: ledger.counts() -> Counts; ledger.failures() -> Vec<Failure>; Timestamp::to_rfc3339(&self) -> String; label(&Item) -> String; String::push_str.

The edits, exactly

Nothing else in these files changes. Each quoted "before" block occurs exactly once in its file when its turn comes (make them in order).

crates/bxcore/src/memory.rs

X1. The lines

    Accept,
    Reject,
}

become

    Accept,
    Reject,
    /// What the Honcho ledger holds: its counts and failed items (M5c spec, section 5).
    Sync,
}

crates/bxctl/src/memory.rs

B1. The lines

//! The owner's memory commands over `memoryd`'s owner socket: `reindex`, and `memory log`,
//! `undo`, `pending`, `accept`, `reject` (M5a spec, section 10).

use std::io::Write;

become

//! The owner's memory commands over `memoryd`'s owner socket: `reindex`, and `memory log`,
//! `undo`, `pending`, `accept`, `reject` (M5a spec, section 10), and `sync` (M5c spec, section 5).

use std::io::Write;

B2. The lines

}

/// `bxctl memory log [N] | undo <id> | pending | accept <id> | reject <id>`, each with an optional
/// `--memory-socket PATH`.
pub fn parse_memory(args: &[String], home: &Path) -> Result<Command, UsageError> {
    let (socket, rest) = socket_flag(args, home)?;

become

}

/// `bxctl memory log [N] | undo <id> | pending | accept <id> | reject <id> | sync`, each with an
/// optional `--memory-socket PATH`.
pub fn parse_memory(args: &[String], home: &Path) -> Result<Command, UsageError> {
    let (socket, rest) = socket_flag(args, home)?;

B3. The lines

        ["accept", id] => Ask::Admin(MemoryCommand::Accept, id.to_string()),
        ["reject", id] => Ask::Admin(MemoryCommand::Reject, id.to_string()),
        _ => return Err(UsageError),
    };

become

        ["accept", id] => Ask::Admin(MemoryCommand::Accept, id.to_string()),
        ["reject", id] => Ask::Admin(MemoryCommand::Reject, id.to_string()),
        ["sync"] => Ask::Admin(MemoryCommand::Sync, String::new()),
        _ => return Err(UsageError),
    };

crates/bxctl/src/cli.rs

U1. The lines

      Rebuild memoryd's index from the session logs and notes.

  bxctl memory log [N] | undo <id> | pending | accept <id> | reject <id> [--memory-socket PATH]
      List memory edits, undo the latest edit of a file, or answer writes waiting for review.

  bxctl sanitize check [TEXT]

become

      Rebuild memoryd's index from the session logs and notes.

  bxctl memory log [N] | undo <id> | pending | accept <id> | reject <id> | sync [--memory-socket PATH]
      List memory edits, undo the latest edit of a file, or answer writes waiting for review.
      sync shows what memoryd sends to Honcho: counts, and each failed item by its label.

  bxctl sanitize check [TEXT]

crates/memoryd/src/honcho/ledger.rs

L1. The lines

}

#[derive(Debug)]
pub enum LedgerError {

become

}

/// A failed item, as its `Failed` record says.
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Failure {
    pub item: Item,
    pub time: Timestamp,
    pub status: Option<u16>,
    pub kind: String,
}

#[derive(Debug)]
pub enum LedgerError {

L2. The lines

            }
            LedgerError::Encode(err) => write!(f, "a ledger record could not be encoded: {err}"),
            LedgerError::AlreadyDecided(item) => write!(f, "{}: decided already", describe(item)),
            LedgerError::NotWaiting(item) => {
                write!(f, "{}: not waiting to be sent", describe(item))
            }
        }

become

            }
            LedgerError::Encode(err) => write!(f, "a ledger record could not be encoded: {err}"),
            LedgerError::AlreadyDecided(item) => write!(f, "{}: decided already", label(item)),
            LedgerError::NotWaiting(item) => {
                write!(f, "{}: not waiting to be sent", label(item))
            }
        }

L3. The lines

    Waiting,
    Sent(Vec<String>),
    Failed,
}

become

    Waiting,
    Sent(Vec<String>),
    Failed {
        time: Timestamp,
        status: Option<u16>,
        kind: String,
    },
}

L4. The lines

    /// Every sent item, in the order of its `Sent` record.
    sends: Vec<Item>,
    repaired: bool,
}

become

    /// Every sent item, in the order of its `Sent` record.
    sends: Vec<Item>,
    /// Every failed item, in the order of its `Failed` record.
    fails: Vec<Item>,
    repaired: bool,
}

L5. The lines

            entries: HashMap::new(),
            sends: Vec::new(),
            repaired: false,
        };

become

            entries: HashMap::new(),
            sends: Vec::new(),
            fails: Vec::new(),
            repaired: false,
        };

L6. The lines

    }

    /// How many items are in each state.
    pub fn counts(&self) -> Counts {

become

    }

    /// Every failed item, oldest first (the order of the `Failed` records), with its record's
    /// time, status and kind.
    pub fn failures(&self) -> Vec<Failure> {
        self.fails
            .iter()
            .filter_map(|item| match &self.entries.get(item)?.state {
                State::Failed { time, status, kind } => Some(Failure {
                    item: item.clone(),
                    time: *time,
                    status: *status,
                    kind: kind.clone(),
                }),
                _ => None,
            })
            .collect()
    }

    /// How many items are in each state.
    pub fn counts(&self) -> Counts {

L7. The lines

                State::Waiting => counts.waiting += 1,
                State::Sent(_) => counts.sent += 1,
                State::Failed => counts.failed += 1,
            }
        }

become

                State::Waiting => counts.waiting += 1,
                State::Sent(_) => counts.sent += 1,
                State::Failed { .. } => counts.failed += 1,
            }
        }

L8. The lines

                self.sends.push(item);
            }
            Record::Failed { item, .. } => {
                if let Some(entry) = self.entries.get_mut(&item) {
                    entry.state = State::Failed;
                }
            }
        }

become

                self.sends.push(item);
            }
            Record::Failed {
                time,
                item,
                status,
                kind,
            } => {
                if let Some(entry) = self.entries.get_mut(&item) {
                    entry.state = State::Failed { time, status, kind };
                }
                self.fails.push(item);
            }
        }

L9. The lines

}

/// An item as the error messages name it.
fn describe(item: &Item) -> String {
    match item {
        Item::Turn { session, turn } => format!("turn {turn} of session {}", session.as_str()),

become

}

/// An item as the error messages and `bxctl memory sync` name it: `turn 3 of session mm-a`,
/// `part 0 of edit 2026-09-28-0001`.
pub fn label(item: &Item) -> String {
    match item {
        Item::Turn { session, turn } => format!("turn {turn} of session {}", session.as_str()),

crates/memoryd/src/honcho/mirror.rs

R1. The lines

use crate::files::Store;
use crate::honcho::ledger::{self, Action, Item, Ledger, Record, Workspace};
use crate::journal;
use crate::notes::{self, Kind};

become

use crate::files::Store;
use crate::honcho::ledger::{self, Action, Item, Ledger, Record, Workspace};
use crate::honcho::report;
use crate::journal;
use crate::notes::{self, Kind};

R2. The lines

    }

    /// Decide the journalled edit `id` (the latest when `None`), called under the index lock
    /// right after it was journalled, so the ledger has the journal's order. Its parts are

become

    }

    /// `bxctl memory sync`'s report of the ledger, under the ledger's lock alone.
    pub fn report(&self) -> String {
        report::report(&self.ledger.lock().unwrap_or_else(|p| p.into_inner()))
    }

    /// Decide the journalled edit `id` (the latest when `None`), called under the index lock
    /// right after it was journalled, so the ledger has the journal's order. Its parts are

crates/memoryd/src/honcho/mod.rs

H1. The lines

pub mod mirror;
pub mod naming;
pub mod start;
pub mod state;

become

pub mod mirror;
pub mod naming;
pub mod report;
pub mod start;
pub mod state;

crates/memoryd/src/serve.rs

S1. The lines

use crate::honcho::ledger::{self, LedgerError};
use crate::honcho::mirror::Mirror;
use crate::honcho::state;
use crate::honcho::turns::TurnSync;

become

use crate::honcho::ledger::{self, LedgerError};
use crate::honcho::mirror::Mirror;
use crate::honcho::report;
use crate::honcho::state;
use crate::honcho::turns::TurnSync;

S2. The lines

            MemoryCommand::Accept => review::accept(&self.store, arg, now),
            MemoryCommand::Reject => review::reject(&self.store, arg, now),
        };
        if result.is_ok() && matches!(command, MemoryCommand::Undo | MemoryCommand::Accept) {

become

            MemoryCommand::Accept => review::accept(&self.store, arg, now),
            MemoryCommand::Reject => review::reject(&self.store, arg, now),
            MemoryCommand::Sync => Ok(match &self.mirror {
                Some(mirror) => mirror.report(),
                None => report::NOT_CONFIGURED.to_string(),
            }),
        };
        if result.is_ok() && matches!(command, MemoryCommand::Undo | MemoryCommand::Accept) {

Steps

  • 1. Copy the given tests. From the repository root, two commands:

    cp docs/plans/M5c1/files/crates/memoryd/tests/honcho_sync.rs-task14 crates/memoryd/tests/honcho_sync.rs
    cp docs/plans/M5c1/files/crates/bxctl/tests/cli_memory_sync.rs crates/bxctl/tests/
    
  • 2. See them fail. cargo build --tests --keep-going -p memoryd -p bxctl, with a shell timeout of 600000 ms: two test targets do not compile. honcho_sync, 5 errors: E0432 (unresolved import `memoryd::honcho::report` ), E0432 (unresolved imports `memoryd::honcho::ledger::Failure`, `memoryd::honcho::ledger::label` ), two E0599 (no method named `failures` ) and one E0599 (no variant, associated function, or constant named `Sync` found for enum `MemoryCommand` ). cli_memory_sync, 6 errors, all that last E0599. Every other target compiles.

  • 3. The edits. Make every edit below, in order (X1, B1 to B3, U1, L1 to L9, R1, R2, H1, S1, S2), then copy the skeleton:

    cp docs/plans/M5c1/files/crates/memoryd/src/honcho/report.rs crates/memoryd/src/honcho/
    

    cargo check -p memoryd -p bxctl: no error, and 2 warnings, from the unfilled body: unused import: `label` and unused variable: `ledger` . And cargo test -p bxctl --test cli_memory_sync passes its 4 tests.

  • 4. Fill the body of report from its comment and "The report" above. Delete its instruction comment and keep the doc comment. cargo check -p memoryd prints no warning.

  • 5. The tests. With a shell timeout of 600000 ms, every test passes (4 and 4, and the others unchanged):

    cargo test -p memoryd --test honcho_sync --test honcho_ledger --test honcho_ledger_file \
      --test honcho_worker && cargo test -p bxctl --test cli_memory_sync --test cli_memory --test cli
    
  • 6. Check. Two commands:

    • grep -n 'MemoryCommand::Sync' crates/memoryd/src/serve.rs crates/bxctl/src/memory.rs prints two lines.
    • git diff --stat -- crates/loopd crates/brokerd crates/gatewayd Cargo.toml Cargo.lock prints nothing.
  • 7. Run the gate. cargo fmt --all, then make gate, with a shell timeout of 600000 ms. Expected last line: gate: ok, with 1332 tests: 1324 before this task, plus 8 new ones (4 in honcho_sync.rs, 4 in cli_memory_sync.rs).

  • 8. Log and commit. Add your row to docs/implementer-log.md; its Task cell is M5c1/14-bxctl-memory-sync. git status --short lists the seven edited files and your log as modified, and report.rs and the two test files as new. Stage with git add crates/bxcore crates/bxctl crates/memoryd docs/implementer-log.md Cargo.lock, then commit as AGENTS.md says. Cargo.lock does not change in this task; the git add is harmless.

Done when

  • make gate prints gate: ok with 1332 tests.
  • git diff HEAD~1 --stat lists only the files of step 8.

Stop and report if

  • A "before" block is not in its file exactly as shown when its turn comes, or is there more than once.
  • Step 2 or 3 shows a different result: other test targets, errors of another kind, other warnings.
  • cargo check reports an error after an edit or the body, and the edit or the comment does not tell you how to fix it. Quote the error, and stop.
  • After step 4 a test fails although the body follows its comment and each edit is as shown. Do not change a test: stop, and give the failing tests' names and their messages.
  • The fix seems to need a change anywhere else: another crate, the worker, a new message type. It does not; stop and say why you think so.
  • A test outside the new files fails in the gate: check once whether it fails without your change, then stop and report both results. (gatewayd's gateway_sock test brokerd_closing_first_ends_it_as_no_longer_pending fails now and then on a loaded machine, without this task's change; if it is the one, run the gate once more.)
  • The gate passes with a count other than 1332.

What the given tests pin

Each of these changes to the finished code makes at least one given test fail (14 tried): the counts in another order; the heading with no failure; the status or the kind left out; the failures newest first, or in the order of decision; a failure's time or status lost, or not recorded at all; an edit's label; "not configured" with [honcho], or sync answered as an error; memory sync not parsed, or sent as log.