kyle/boxmaker
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity
Files
m5c1
boxmaker/docs/plans/M5c1/15-memoryd-honcho-began.md
T
kyleandClaude Opus 5.5 d8952ca77d M5c-1 plan: task 15, sync from when Honcho is on, conclusions matched by peer 
Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
2026-09-28 09:58:26 -07:00

786 lines
25 KiB
Markdown
Raw Permalink Blame History

# M5c-1 task 15: sync begins when Honcho is turned on, and conclusions are matched by peer
**Branch:** `m5c1` (run `git switch m5c1`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `memoryd: sync begins when Honcho is turned on, and conclusions are matched by peer`
**Log row:** the Task cell is `M5c1/15-memoryd-honcho-began`.
## Goal
Two changes to what tasks 07 to 14 built (M5c spec, section 5, as of 2026-09-28).
**1. Sync starts when Honcho is turned on** (the owner's decision): turning on `[honcho]` never
sends earlier conversations.
- The ledger gains a record, `Record::Began { time }`: when sync began. `memoryd` writes it at a
start with `[honcho]` when the ledger has none, so the first time: after `memory/` is made and
before any socket is bound, appended and synced like every record. It is always the ledger's
first line.
- A ledger read back that has records but does not start with `Began`, or has a second `Began`, is
`Damaged` (`the first line is not when sync began`, `sync began, but not on the first line`),
as task 07's other rule breaks are. Nothing is deployed yet, so there are no older ledgers
without the line to allow for. `Ledger::began() -> Option<Timestamp>` gives the time.
- `decide` (task 08) gains a skip, `the turn is from before sync began` (`BEFORE_SYNC`), for a turn
whose `User` record's time is before `began`. It is the **first** check: it is unconditional
and hides nothing sensitive. The order is now:
| # | When | The action |
|---|---|---|
| 0 | the log has no such turn | `Err(NoSuchTurn)` |
| 1 | the turn's `User` record is from before sync began | `Skip`, `BEFORE_SYNC` |
| 2 to 8 | as task 08: secret class; no speaker; not a peer name; the agent's name; no reply; a secret in the text; untrusted or trusted | as task 08 |
`decide` takes `began: Option<Timestamp>` (`None`: no cutoff); the walk (task 09) passes the
ledger's. The first turn after enabling is always after `Began`: it is written at start, before
any turn can be reported.
- **The cost:** the first report of a session that ran before Honcho was turned on decides one
`Skip` per earlier turn (task 09 walks every turn up to the reported one). `bxctl memory sync`'s
"skipped" count shows them all; that is expected.
**2. Conclusions are matched by peer too.** The same text can be a `user.md` entry (about the
owner) and a `memory.md` entry (about the agent). `Action::Unconclude` gains `peer`, set as its
`Conclude` would be (the owner for `user.md`, the agent for `memory.md`), and
`Ledger::conclusion(workspace, peer, text)` matches all three. Removing the `user.md` entry
deletes only the owner's conclusion. The ledger's line for an `Unconclude` changes (it gains
`"peer"`); nothing is deployed, so nothing old needs reading.
## Every place that changes
This is the complete list.
| Place | What happens there |
|---|---|
| `crates/memoryd/src/honcho/ledger.rs` | `Unconclude.peer`, `Record::Began`, two errors, `began`, the first-line rule, `conclusion` by peer (edits L1 to L13) |
| `crates/memoryd/src/honcho/decide.rs` | `BEFORE_SYNC`, the `began` parameter and check (edits D1 to D5) |
| `crates/memoryd/src/honcho/turns.rs` | the walk passes the ledger's `began` (edit T1) |
| `crates/memoryd/src/honcho/mirror.rs` | `Unconclude`'s peer (edit M1) |
| `crates/memoryd/src/honcho/worker.rs` | `conclusion` by peer (edit W1) |
| `crates/memoryd/src/honcho/start.rs` | `Opened::begin` (edits A1, A2) |
| `crates/memoryd/src/serve.rs` | `begin` after `memory/` is made (edits S1, S2) |
| tests | new: `honcho_ledger_began.rs`, `honcho_decide_began.rs`, `honcho_began_serve.rs`; new versions of every earlier test that builds an `Unconclude`, calls `decide` or `conclusion`, or reads a ledger back (see Files) |
## Files
- Copy, from `docs/plans/M5c1/files/` (the same path under it), into `crates/memoryd/tests/`:
- new: `honcho_ledger_began.rs` (5 tests), `honcho_decide_began.rs` (4), `honcho_began_serve.rs`
(2);
- **new versions** (they replace the files in the tree; the `-taskNN` files beside them are the
older ones, which earlier tasks copied: do not copy those): `honcho_ledger.rs`,
`honcho_ledger_file.rs`, `honcho_decide.rs`, `honcho_turns.rs`, `honcho_naming.rs`,
`honcho_serve.rs`, `honcho_mirror.rs`, `honcho_mirror_serve.rs`, `honcho_edits.rs`,
`honcho_worker_more.rs`, `honcho_sync.rs`, and `support/ledger.rs`, `support/turns.rs`,
`support/worker.rs`.
- Modify: the files of the edits, `docs/implementer-log.md`.
- Do not touch: any other source or test file, any `Cargo.toml`.
What the new versions change, so that you can check them: `Unconclude` built with a peer and
`conclusion` called with one; `decide` called with `None` for `began`; every ledger that a test
reads back begun first (`support/ledger.rs`'s `Home::begun` and `began_line`, `support/worker.rs`'s
`Home::new`, which begins at `BEGAN`, and whose `records` leaves the `Began` out); the line numbers
of the damaged-ledger tests one higher; `support/turns.rs`'s `spoken` at `Timestamp::MAX`, after
any time sync began (`spoken_at` for another); `honcho_serve.rs`'s ledgers with their `began`
line; `honcho_edits.rs` (a new test: the same text about two peers) and `honcho_mirror.rs` (a new
test: each file's peer on a deletion).
## Facts you need
- **`LogRecord::User` has a `time`** (`bxcore`, M2b): the time `loopd` logged the turn's message.
`decide`'s check compares it with `began` (`Timestamp` is `Ord`), for the turn's first record,
which is its `User` record.
- **Why `began` is written after `memory/` is made.** `Ledger::open` only reads, and task 09 opens
the ledger before anything under the home is made, so that a damaged one stops `memoryd` first.
The ledger's directory is `memory/`, made by `Store::init`; `begin` runs right after it, and
before the sockets are bound.
- **`Ledger`'s new fields**: `began: Option<Timestamp>` (set by `apply` for a `Began`) and
`records: usize` (every record applied, counted in `apply`); `check` refuses a `Began` when
`records > 0`, and `open` refuses a first line that is not a `Began`.
- **The given tests** of this task:
- `honcho_ledger_began.rs` (5): the line; `began` kept and read back; a `Began` only first and
once; a ledger without it, or with two, damaged at the right line; a conclusion found by peer,
and one peer's undone while the other's stays.
- `honcho_decide_began.rs` (4): the text; before, at and after `began`, and `None`; the first
check, before the secret class; each turn by its own time.
- `honcho_began_serve.rs` (2): the first start writes one `Began`, before `start` returns, and a
second leaves it; a turn logged before it and reported after is skipped.
## The edits, exactly
Nothing else in these files changes. Each quoted "before" block occurs exactly once in its file
when its turn comes: make them in order. Several change code you wrote from comments in earlier
tasks (`check`, `apply` and `conclusion` in `ledger.rs`, task 07; `identity_actions` in
`mirror.rs`, task 12): if yours differs from a "before" block, stop and report.
### `crates/memoryd/src/honcho/ledger.rs`
**L1.** The lines
```rust
},
/// Delete the conclusion that a sent `Conclude` with this text created.
Unconclude { workspace: Workspace, text: String },
/// A message from the agent's peer in the fleet workspace.
Fleet { session: SessionId, text: String },
```
become
```rust
},
/// Delete the conclusion that a sent `Conclude` with this text created.
Unconclude {
workspace: Workspace,
peer: String,
text: String,
},
/// A message from the agent's peer in the fleet workspace.
Fleet { session: SessionId, text: String },
```
**L2.** The lines
```rust
#[serde(tag = "type", rename_all = "snake_case", deny_unknown_fields)]
pub enum Record {
Decided {
time: Timestamp,
```
become
```rust
#[serde(tag = "type", rename_all = "snake_case", deny_unknown_fields)]
pub enum Record {
/// When sync began: always the ledger's first line, written once, by `memoryd` at its first
/// start with `[honcho]`. A turn from before it is not synced.
Began { time: Timestamp },
Decided {
time: Timestamp,
```
**L3.** The lines
```rust
AlreadyDecided(Item),
NotWaiting(Item),
}
```
become
```rust
AlreadyDecided(Item),
NotWaiting(Item),
/// A `Began` after other records, or a second one.
BeganNotFirst,
/// A ledger whose first line is not its `Began`.
NotBegun,
}
```
**L4.** The lines
```rust
write!(f, "{}: not waiting to be sent", label(item))
}
}
}
```
become
```rust
write!(f, "{}: not waiting to be sent", label(item))
}
LedgerError::BeganNotFirst => write!(f, "sync began, but not on the first line"),
LedgerError::NotBegun => write!(f, "the first line is not when sync began"),
}
}
```
**L5.** The lines
```rust
/// Every failed item, in the order of its `Failed` record.
fails: Vec<Item>,
repaired: bool,
}
```
become
```rust
/// Every failed item, in the order of its `Failed` record.
fails: Vec<Item>,
/// When sync began: the `Began` record's time.
began: Option<Timestamp>,
/// How many records the ledger holds.
records: usize,
repaired: bool,
}
```
**L6.** The lines
```rust
sends: Vec::new(),
fails: Vec::new(),
repaired: false,
};
```
become
```rust
sends: Vec::new(),
fails: Vec::new(),
began: None,
records: 0,
repaired: false,
};
```
**L7.** The lines
```rust
for (index, line) in lines.split_inclusive(|b| *b == b'\n').enumerate() {
let record = parse(path, index + 1, line)?;
if let Err(e) = ledger.check(&record) {
return Err(LedgerError::Damaged {
path: path.to_path_buf(),
```
become
```rust
for (index, line) in lines.split_inclusive(|b| *b == b'\n').enumerate() {
let record = parse(path, index + 1, line)?;
let first_not_began = index == 0 && !matches!(record, Record::Began { .. });
let checked = if first_not_began {
Err(LedgerError::NotBegun)
} else {
ledger.check(&record)
};
if let Err(e) = checked {
return Err(LedgerError::Damaged {
path: path.to_path_buf(),
```
**L8.** The lines
```rust
self.apply(record);
Ok(())
}
```
become
```rust
self.apply(record);
Ok(())
}
/// When sync began, once the ledger has its `Began` record.
pub fn began(&self) -> Option<Timestamp> {
self.began
}
```
**L9.** The lines
```rust
}
/// The latest sent `Conclude` of `text` in `workspace` that no sent `Unconclude` has undone:
/// its item and the conclusion's id.
pub fn conclusion(&self, workspace: Workspace, text: &str) -> Option<(Item, String)> {
let mut live: Vec<(Item, String)> = Vec::new();
for item in &self.sends {
```
become
```rust
}
/// The latest sent `Conclude` of `text` about `peer` in `workspace` that no sent `Unconclude`
/// of the same has undone: its item and the conclusion's id.
pub fn conclusion(
&self,
workspace: Workspace,
peer: &str,
text: &str,
) -> Option<(Item, String)> {
let mut live: Vec<(Item, String)> = Vec::new();
for item in &self.sends {
```
**L10.** The lines
```rust
Action::Conclude {
workspace: w,
text: t,
..
} if *w == workspace && t == text => {
if let Some(id) = ids.first() {
live.push((item.clone(), id.clone()));
```
become
```rust
Action::Conclude {
workspace: w,
peer: p,
text: t,
..
} if *w == workspace && p == peer && t == text => {
if let Some(id) = ids.first() {
live.push((item.clone(), id.clone()));
```
**L11.** The lines
```rust
Action::Unconclude {
workspace: w,
text: t,
} if *w == workspace && t == text => {
live.pop();
}
```
become
```rust
Action::Unconclude {
workspace: w,
peer: p,
text: t,
} if *w == workspace && p == peer && t == text => {
live.pop();
}
```
**L12.** The lines
```rust
fn check(&self, record: &Record) -> Result<(), LedgerError> {
match record {
Record::Decided { item, .. } => {
if self.entries.contains_key(item) {
```
become
```rust
fn check(&self, record: &Record) -> Result<(), LedgerError> {
match record {
Record::Began { .. } => {
if self.records > 0 {
Err(LedgerError::BeganNotFirst)
} else {
Ok(())
}
}
Record::Decided { item, .. } => {
if self.entries.contains_key(item) {
```
**L13.** The lines
```rust
/// Take a checked record into the state.
fn apply(&mut self, record: Record) {
match record {
Record::Decided { item, action, .. } => {
let state = match action {
```
become
```rust
/// Take a checked record into the state.
fn apply(&mut self, record: Record) {
self.records += 1;
match record {
Record::Began { time } => self.began = Some(time),
Record::Decided { item, action, .. } => {
let state = match action {
```
### `crates/memoryd/src/honcho/decide.rs`
**D1.** The lines
```rust
use bxcore::sanitize::refuse;
use bxcore::{DataClass, LogRecord, SessionStateReply};
use crate::honcho::config::is_name;
```
become
```rust
use bxcore::sanitize::refuse;
use bxcore::{DataClass, LogRecord, SessionStateReply, Timestamp};
use crate::honcho::config::is_name;
```
**D2.** The lines
```rust
/// Why a turn goes nowhere: the `why` of its `Action::Skip`.
pub const SECRET_CLASS: &str = "the session is secret";
pub const NO_SPEAKER: &str = "the turn has no single speaker";
```
become
```rust
/// Why a turn goes nowhere: the `why` of its `Action::Skip`.
pub const BEFORE_SYNC: &str = "the turn is from before sync began";
pub const SECRET_CLASS: &str = "the session is secret";
pub const NO_SPEAKER: &str = "the turn has no single speaker";
```
**D3.** The lines
```rust
/// Where turn `turn` of the session whose log is `records` goes. `state` is `brokerd`'s answer
/// for the session; `agent` is `[honcho] agent`. The checks run in this order, and the first that
/// holds decides: the class `secret`; no speaker; a speaker that is not a peer name, or is the
/// agent's; no reply; a secret in the user's text or the reply. Otherwise the turn goes to the
/// untrusted workspace if `brokerd` or any tool result up to the turn's end says untrusted, else
```
become
```rust
/// Where turn `turn` of the session whose log is `records` goes. `state` is `brokerd`'s answer
/// for the session; `agent` is `[honcho] agent`; `began` is when sync began (the ledger's
/// `Began`). The checks run in this order, and the first that holds decides: the turn is from
/// before `began`; the class `secret`; no speaker; a speaker that is not a peer name, or is the
/// agent's; no reply; a secret in the user's text or the reply. Otherwise the turn goes to the
/// untrusted workspace if `brokerd` or any tool result up to the turn's end says untrusted, else
```
**D4.** The lines
```rust
turn: u64,
agent: &str,
) -> Result<Action, NoSuchTurn> {
let missing = NoSuchTurn {
```
become
```rust
turn: u64,
agent: &str,
began: Option<Timestamp>,
) -> Result<Action, NoSuchTurn> {
let missing = NoSuchTurn {
```
**D5.** The lines
```rust
};
if state.class == DataClass::Secret {
return skip(SECRET_CLASS);
```
become
```rust
};
if let (Some(began), Some(LogRecord::User { time, .. })) = (began, this.first())
&& *time < began
{
return skip(BEFORE_SYNC);
}
if state.class == DataClass::Secret {
return skip(SECRET_CLASS);
```
### `crates/memoryd/src/honcho/turns.rs`
**T1.** The lines
```rust
};
for turn in undecided {
let Ok(action) = decide(&state, &records, turn, &self.agent) else {
continue;
};
```
become
```rust
};
for turn in undecided {
let began = self.ledger().began();
let Ok(action) = decide(&state, &records, turn, &self.agent, began) else {
continue;
};
```
### `crates/memoryd/src/honcho/mirror.rs`
**M1.** The lines
```rust
.map(|text| Action::Unconclude {
workspace: Workspace::Trusted,
text,
})
```
become
```rust
.map(|text| Action::Unconclude {
workspace: Workspace::Trusted,
peer: peer.to_string(),
text,
})
```
### `crates/memoryd/src/honcho/worker.rs`
**W1.** The lines
```rust
},
),
(Item::Edit { .. }, Action::Unconclude { workspace, text }) => (
workspace,
Job::Unconclude {
id: ledger.conclusion(workspace, &text).map(|(_, id)| id),
},
),
```
become
```rust
},
),
(
Item::Edit { .. },
Action::Unconclude {
workspace,
peer,
text,
},
) => (
workspace,
Job::Unconclude {
id: ledger.conclusion(workspace, &peer, &text).map(|(_, id)| id),
},
),
```
### `crates/memoryd/src/honcho/start.rs`
**A1.** The lines
```rust
use std::sync::{Arc, Mutex};
use bxcore::SessionId;
use crate::config::Config;
use crate::honcho::client::{Clients, Timeouts};
use crate::honcho::config::HonchoConfig;
use crate::honcho::ledger::Ledger;
use crate::honcho::mirror::Mirror;
use crate::honcho::state;
```
become
```rust
use std::sync::{Arc, Mutex};
use bxcore::{SessionId, Timestamp};
use crate::config::Config;
use crate::honcho::client::{Clients, Timeouts};
use crate::honcho::config::HonchoConfig;
use crate::honcho::ledger::{Ledger, Record};
use crate::honcho::mirror::Mirror;
use crate::honcho::state;
```
**A2.** The lines
```rust
impl Opened {
/// The memory writes' side of the ledger (task 12).
pub fn mirror(&self) -> Mirror {
```
become
```rust
impl Opened {
/// The first time: sync begins now. The `Began` line is written and synced once `memory/`
/// exists and before any socket is bound, so every turn reported later is after it.
pub fn begin(&self) -> Result<(), StartError> {
let sync = self.sync.lock().unwrap_or_else(|p| p.into_inner());
let mut ledger = sync.ledger();
if ledger.began().is_none() {
let began = Record::Began {
time: Timestamp::now(),
};
ledger.append(began).map_err(StartError::Ledger)?;
}
Ok(())
}
/// The memory writes' side of the ledger (task 12).
pub fn mirror(&self) -> Mirror {
```
### `crates/memoryd/src/serve.rs`
**S1.** The lines
```rust
/// With `[honcho]`, read the ledger through and the keys first. Then create what is missing
/// under `memory/`, open the index and catch it up, bind the three sockets, and start the Honcho
/// worker. The error names what failed.
pub fn start(config: Config, log: Log) -> Result<Running, StartError> {
let opened = match config.honcho_config()? {
```
become
```rust
/// With `[honcho]`, read the ledger through and the keys first. Then create what is missing
/// under `memory/` (and, the first time, write when sync began), open the index and catch it up,
/// bind the three sockets, and start the Honcho worker. The error names what failed.
pub fn start(config: Config, log: Log) -> Result<Running, StartError> {
let opened = match config.honcho_config()? {
```
**S2.** The lines
```rust
let store = Store::new(&config.memory_dir());
store.init(&config.memory.soul_starter)?;
let index = Index::open(&config.index_path(), config.index.max_class)?;
let (_counts, problems) = index.catch_up(&config.sessions_dir(), &store.path("notes.md"))?;
```
become
```rust
let store = Store::new(&config.memory_dir());
store.init(&config.memory.soul_starter)?;
if let Some(opened) = &opened {
opened.begin()?;
}
let index = Index::open(&config.index_path(), config.index.max_class)?;
let (_counts, problems) = index.catch_up(&config.sessions_dir(), &store.path("notes.md"))?;
```
## Steps
- [ ] **1. Copy the given tests.** From the repository root:
```sh
cd docs/plans/M5c1/files/crates/memoryd/tests
cp honcho_ledger_began.rs honcho_decide_began.rs honcho_began_serve.rs \
honcho_ledger.rs honcho_ledger_file.rs honcho_decide.rs honcho_turns.rs honcho_naming.rs \
honcho_serve.rs honcho_mirror.rs honcho_mirror_serve.rs honcho_edits.rs \
honcho_worker_more.rs honcho_sync.rs ../../../../../../../crates/memoryd/tests/
cp support/ledger.rs support/turns.rs support/worker.rs \
../../../../../../../crates/memoryd/tests/support/
cd -
```
Then `git status --short crates/memoryd/tests` lists 14 modified and 3 new files.
- [ ] **2. See them fail.** `cargo build --tests --keep-going -p memoryd`, with a shell timeout
of 600000 ms: 14 test targets do not compile (`honcho_began_serve`, `honcho_decide`,
`honcho_decide_began`, `honcho_edits`, `honcho_ledger`, `honcho_ledger_began`,
`honcho_ledger_file`, `honcho_mirror`, `honcho_mirror_serve`, `honcho_naming`, `honcho_sync`,
`honcho_turns`, `honcho_worker`, `honcho_worker_more`), with errors of these kinds only:
`E0061` (`decide` takes 4 arguments, `conclusion` 2), `E0432` (`BEFORE_SYNC`), `E0559`
(`Unconclude` has no `peer`), `E0599` (no `began`, `begin`, `BeganNotFirst` or `Began`). Every
other target compiles.
- [ ] **3. The ledger.** Make edits L1 to L13. `cargo check -p memoryd`: three errors, one in
`mirror.rs` (`E0063`, ``missing field `peer` ``) and two in `worker.rs` (`E0027`, the
`Unconclude` pattern, and `E0061`, `conclusion`'s arguments): the next edits.
- [ ] **4. The rest.** Make edits D1 to D5, T1, M1, W1, A1, A2, S1 and S2. `cargo check -p
memoryd`: no error and no warning.
- [ ] **5. The tests.** With a shell timeout of 600000 ms, `cargo test -p memoryd`: every test
passes.
- [ ] **6. Check.** Two commands:
- `grep -n 'fn begin\|opened.begin()' crates/memoryd/src/honcho/start.rs crates/memoryd/src/serve.rs`
prints two lines, and in `serve.rs` the call comes after `store.init` and before the sockets
are bound (`crate::socket::listen`).
- `git diff --stat -- crates/bxcore crates/loopd crates/brokerd crates/gatewayd crates/bxctl
Cargo.toml Cargo.lock` prints nothing.
- [ ] **7. Run the gate.** `cargo fmt --all`, then `make gate`, with a shell timeout of 600000 ms.
Expected last line: `gate: ok`, with 1345 tests: 1332 before this task, plus 13 new ones (11 in
the three new files, one more in `honcho_mirror.rs`, one more in `honcho_edits.rs`).
- [ ] **8. Log and commit.** Add your row to `docs/implementer-log.md`; its Task cell is
`M5c1/15-memoryd-honcho-began`. Stage with `git add crates/memoryd docs/implementer-log.md
Cargo.lock`, then commit as `AGENTS.md` says. `Cargo.lock` does not change in this task; the
`git add` is harmless.
## Done when
- `make gate` prints `gate: ok` with 1345 tests.
- `git diff HEAD~1 --stat` lists only the seven source files, the test files of step 1, and
`docs/implementer-log.md`.
## Stop and report if
- A "before" block is not in its file exactly as shown when its turn comes, or is there more than
once.
- Step 2 or 3 shows a different result: other test targets, errors of another kind.
- `cargo check` reports an error after step 4, and the edits do not tell you how to fix it. Quote
the error, and stop.
- A test fails although each edit is as shown. Do not change a test: stop, and give the failing
tests' names and their messages.
- The fix seems to need a change anywhere else: another crate, the journal, `loopd`'s log. It
does not; stop and say why you think so.
- A test outside the given files fails in the gate: check once whether it fails without your
change, then stop and report both results. (`gatewayd`'s `gateway_sock` test
`brokerd_closing_first_ends_it_as_no_longer_pending` fails now and then on a loaded machine,
without this task's change; if it is the one, run the gate once more.)
- The gate passes with a count other than 1345.
## What the given tests pin
Each of these changes to the finished code makes at least one given test fail (15 tried): a
`Began` allowed anywhere; the first line not checked; the records not counted; `began` not kept;
`conclusion` ignoring the peer of a `Conclude` or of an `Unconclude`; the before-sync check off,
inclusive of `began` itself, or after the secret-class check; the walk passing no `began`; the
`Began` line not written, or written at every start; `begin` not called; an `Unconclude`'s peer
always the agent; the worker looking up the owner's conclusion whatever the peer.
Reference in New Issue View Git Blame Copy Permalink