boxmaker/docs/plans/M5c1/12-memoryd-honcho-mirror.md

# M5c-1 task 12: `memoryd` decides memory writes for Honcho as they are journalled

**Branch:** `m5c1` (run `git switch m5c1`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `memoryd: memory writes decided into the Honcho ledger as they are journalled`
**Log row:** the Task cell is `M5c1/12-memoryd-honcho-mirror`.

## Goal

Every memory write is mirrored in Honcho the moment it is journalled (M5c spec, section 4), so
that "remember this" reaches Honcho at once, through the memory tool and its checks. This task
**decides** each journalled edit into the ledger (task 07); the worker sends them in task 13.
With `[honcho]` only: without it nothing changes.

- **When.** Right after `journal::record` (the memory tool, and a reviewed write when the owner
  accepts it) and after `journal::undo` (`bxctl memory undo`), in the same call, under the index
  lock those calls already hold. So edits are decided in the journal's order.
- **Items.** `Item::Edit { id: <the journal id>, part }`, one part per Honcho action, numbered
  from 0 in the order they must be sent. The outbox's order is then the send order.
- **The mapping** (spec section 4's table, as journal records show it):

  | The journal record | The actions, in order |
  |---|---|
  | an identity edit (`file` is `memory.md` or `user.md`): add, replace, remove, consolidate, and an undo | each entry dropped: `Unconclude`; then each entry added: `Conclude` about the owner (`user.md`) or the agent (`memory.md`) |
  | entries `consolidate` moved to notes (`§ moved` notes of the record) | nothing: not counted as dropped |
  | a write saved as notes (`file` is none) with a flag rule (`flag` set) | one `Skip { why: "flagged: <rule>" }` |
  | the same, untrusted (`untrusted`), not flagged | each `noted` note: `Fleet` |
  | the same, trusted, not flagged | each `noted` note: `Conclude` about the agent |
  | anything else (a rejection) | nothing |

  "Dropped" and "added" are the multiset difference of the entries before and after, by text: a
  replace is one of each (part 0 the `Unconclude` of the old, part 1 the `Conclude` of the new),
  a reorder is none, and one of two equal entries removed is one dropped. An undo's `before` is
  the file as it was, its `after` what the undo restored, so the same difference is its inverse:
  the conclusions its edit made are deleted, and any it deleted are made again.
- **The session.** A `Conclude` or `Fleet` carries the Boxmaker session id of the write, as
  task 07's schema has it (which Honcho session that is sent into is task 13's). The owner's undo
  is journalled with the session `owner`: its conclusions carry the undone edit's session instead,
  following `undoes` back (through an undo of an undo). With no session at all, a part that needs
  one is a `Skip { why: "the edit has no Boxmaker session" }`; a deletion needs none.
- **Text.** Kept as journalled; the worker neutralises it when it sends (task 13).
- **Nothing to send:** an edit whose mapping is empty gets no line. A flagged write gets its
  `Skip` line, so that `bxctl memory sync` (task 14) can show that it stayed local.
- **Retractions.** An undo never touches a note (notes are append-only: `journal::undo` refuses
  them), so nothing here ever decides `Action::Retract`; it stays in the ledger's schema unused.

**The locks.** A walk (task 09) holds `TurnSync`'s lock while it asks `brokerd`, up to 5 s. A
memory write must not wait for that: it is the model's tool call. So this task gives the ledger a
lock of its own, held only to read or append (edits T1 to T8): the walk takes it briefly for each
check and append, the worker for each read and append, and a memory write for its appends. The
order is always index → ledger, or walk → ledger, and the ledger's lock is never held while
another is taken. A given test holds a walk on a silent `brokerd` and makes a memory call.

## Interfaces

In `crates/memoryd/src/honcho/mirror.rs` (the module `memoryd::honcho::mirror`):

```rust
pub const OWNER: &str = "owner";
pub const NO_SESSION: &str = "the edit has no Boxmaker session";
pub fn diff(before: &[String], after: &[String]) -> (Vec<String>, Vec<String>); // (dropped, added)
pub fn session_of(record: &journal::Record, records: &[journal::Record]) -> Option<SessionId>;
pub fn actions(record: &journal::Record, session: Option<&SessionId>, agent: &str, owner: &str)
    -> Vec<Action>;
pub struct Mirror { /* private */ }
impl Mirror {
    pub fn new(ledger: Arc<Mutex<Ledger>>, agent: &str, owner: &str) -> Mirror;
    pub fn journalled(&self, store: &Store, id: Option<&str>, now: Timestamp, log: &dyn Fn(&str));
}
```

`TurnSync` (task 09): its ledger is an `Arc<Mutex<Ledger>>`; `ledger()` returns the locked
ledger (a `MutexGuard`, which the existing calls use as before), and `shared_ledger()` the lock
itself. `worker::Opened` gains `mirror()`, and `Memoryd` a `mirror: Option<Mirror>` and a
private `mirror(&self, id)`.

## Every place that changes

This is the complete list.

| Place | What happens there |
|---|---|
| `crates/memoryd/src/honcho/mod.rs` | `pub mod mirror;` (edit H1) |
| `crates/memoryd/src/honcho/mirror.rs` | new: a skeleton you copy, then fill 7 bodies |
| `crates/memoryd/src/honcho/turns.rs` | the ledger's own lock (edits T1 to T8) |
| `crates/memoryd/src/honcho/worker.rs` | the worker reads and appends under the ledger's lock; `Opened::mirror` (edits W1 to W9) |
| `crates/memoryd/src/serve.rs` | `Memoryd.mirror`, and the two calls (edits S1 to S7) |
| everything else (`journal.rs`, `edit.rs`, `review.rs`, `ledger.rs`, tests of earlier tasks) | **no change** |

## Files

- Copy, from `docs/plans/M5c1/files/` (the same path under it): the new tests
  `crates/memoryd/tests/honcho_mirror.rs` and `crates/memoryd/tests/honcho_mirror_serve.rs`, and
  the skeleton `crates/memoryd/src/honcho/mirror.rs`.
- Modify: the files of edits H1 to S7, `docs/implementer-log.md`.
- Do not touch: any other source file, any other test or support file, any `Cargo.toml`, and in
  the skeleton anything but the 7 `todo!()` bodies.

## Facts you need

- **A journal record** (`memoryd::journal::Record`, M5a): `id`, `session`, `action`, `file` (the
  identity file's name, or `None` when the write was saved as notes), `before` and `after` (the
  file's entries), `notes` (each note appended, rendered: a header line and its text), `untrusted`,
  `flag` (the flag rule a write matched), `undoes` (for an undo, the edit it reverted). An undo is
  journalled with the session `owner`.
- **Notes.** `memoryd::notes::parse(text: &str) -> Vec<Note>` reads rendered notes back;
  `Note { kind, time, session, text }`, and `kind` is `Kind::Noted { class, untrusted }`,
  `Kind::Moved(file)` (`consolidate` took it out of the prompt: still held true) or
  `Kind::Dropped(file)` (no longer held true; it is also missing from `after`).
- **Which records come here.** `Memoryd::memory` journals a write (the id is returned); with
  review on, an identity write from a clean session is staged instead and journalled only when
  the owner accepts it (`review::accept`); a rejection is journalled too, and asks nothing. The
  given `mirror` in `serve.rs` passes the id for the memory tool, and `None` (the latest record,
  the one just journalled: the index lock is held) for an undo and an accept.
- **What the skeleton gives whole:** the constants, `actions` (which picks
  `identity_actions` or `note_actions`), `Mirror` with `new` and `journalled` (reads the journal,
  finds the record and its session, appends each part not yet decided, and logs a failure with
  the ledger's pointer: the write itself stands).
- **The given tests.**
  - `honcho_mirror.rs` (8 tests): the multiset diff (add, replace, remove, reorder only, duplicates,
    order); each table row from journal records; the undo of an add, a replace and a remove;
    the session of an owner's undo, an undo of an undo, and none.
  - `honcho_mirror_serve.rs` (6 tests): `memoryd` serving: the memory tool's writes decided in
    the journal's order (ids and parts); a flagged write's `Skip`; the owner's undo with the undone
    edit's session; a reviewed write decided at accept, a rejection not at all; nothing without
    `[honcho]`; a memory call answered within 1 s, and decided at once, while a walk waits for a
    silent `brokerd` (the fake from task 09).
- The calls the bodies use:

  | Call | Signature, and what it gives |
  |---|---|
  | `slice::iter().position` | `left.iter().position(\|t\| t == text) -> Option<usize>` |
  | `Vec::swap_remove` | `left.swap_remove(at)`: removes one element, order not kept (fine: a multiset) |
  | `Iterator::filter`, `cloned`, `collect` | `a.iter().filter(..).cloned().collect::<Vec<String>>()` |
  | `Iterator::flat_map` | `record.notes.iter().flat_map(\|n\| notes::parse(n))` |
  | `Option::as_deref`, `?` | `current.undoes.as_deref()?`: the `&str`, or return `None` |
  | `SessionId::new` | `SessionId::new(&s).ok() -> Option<SessionId>` |
  | `matches!` | `matches!(n.kind, Kind::Moved(_))` |

## The edits, exactly

Nothing else in these files changes. Each quoted "before" block occurs exactly once in its file.
Edits W5 and W7 change code you wrote in task 11 from its comments: if yours differs from the
"before" block, stop and report.

### `crates/memoryd/src/honcho/mod.rs`

**H1.** The line

```rust
pub mod ledger;
```

becomes

```rust
pub mod ledger;
pub mod mirror;
```

### `crates/memoryd/src/honcho/turns.rs`

**T1.** The line

```rust
use std::path::{Path, PathBuf};
```

becomes

```rust
use std::path::{Path, PathBuf};
use std::sync::{Arc, Mutex, MutexGuard};
```

**T2.** In `turns.rs`, the lines

```rust
/// The turns reported and not yet decided, and the ledger their decisions go to.
pub struct TurnSync {
    ledger: Ledger,
```

become

```rust
/// The turns reported and not yet decided, and the ledger their decisions go to. The ledger has a
/// lock of its own, held only to read or append: a walk holds `TurnSync`'s lock while it asks
/// `brokerd`, and a memory write, decided under the index lock, takes only the ledger's.
pub struct TurnSync {
    ledger: Arc<Mutex<Ledger>>,
```

**T3.** In `TurnSync::new`, the lines

```rust
        TurnSync {
            ledger,
```

become

```rust
        TurnSync {
            ledger: Arc::new(Mutex::new(ledger)),
```

**T4.** The lines

```rust
    pub fn ledger(&self) -> &Ledger {
        &self.ledger
    }
```

become

```rust
    /// The ledger, locked. Never hold it while taking another lock.
    pub fn ledger(&self) -> MutexGuard<'_, Ledger> {
        self.ledger.lock().unwrap_or_else(|p| p.into_inner())
    }

    /// The ledger's lock itself, for the worker and for memory writes.
    pub fn shared_ledger(&self) -> Arc<Mutex<Ledger>> {
        Arc::clone(&self.ledger)
    }
```

**T5.** In `resume`, the line

```rust
        for (item, _) in self.ledger.decisions() {
```

becomes

```rust
        for (item, _) in self.ledger().decisions() {
```

**T6.** In `append`, the line

```rust
        self.ledger.append(record)
```

becomes

```rust
        self.ledger().append(record)
```

**T7.** In `walk`, the line

```rust
            .filter(|turn| !self.ledger.is_decided(&item(*turn)))
```

becomes

```rust
            .filter(|turn| !self.ledger().is_decided(&item(*turn)))
```

**T8.** In `walk`, the line

```rust
            if let Err(e) = self.ledger.append(record) {
```

becomes

```rust
            let appended = self.ledger().append(record);
            if let Err(e) = appended {
```

### `crates/memoryd/src/honcho/worker.rs`

**W1.** The line

```rust
use std::path::Path;
```

becomes

```rust
use std::path::{Path, PathBuf};
```

**W2.** The line

```rust
use crate::honcho::naming::{HonchoSession, honcho_session};
```

becomes

```rust
use crate::honcho::mirror::Mirror;
use crate::honcho::naming::{HonchoSession, honcho_session};
```

**W3.** The lines

```rust
pub struct Worker {
    sync: Arc<Mutex<TurnSync>>,
```

become

```rust
pub struct Worker {
    sync: Arc<Mutex<TurnSync>>,
    /// The ledger's own lock (task 12): taken alone, briefly, never with `sync`'s.
    ledger: Arc<Mutex<Ledger>>,
    sessions: PathBuf,
```

**W4.** In `Worker::new`, the lines

```rust
    ) -> Worker {
        Worker {
            sync,
```

become

```rust
    ) -> Worker {
        let (ledger, sessions) = {
            let sync = sync.lock().unwrap_or_else(|p| p.into_inner());
            (sync.shared_ledger(), sync.sessions().to_path_buf())
        };
        Worker {
            sync,
            ledger,
            sessions,
```

**W5.** In `next`, the lines

```rust
        let sync = self.sync.lock().unwrap_or_else(|p| p.into_inner());
        let ledger = sync.ledger();
```

become

```rust
        let ledger = self.ledger.lock().unwrap_or_else(|p| p.into_inner());
```

**W6.** In `send`, the lines

```rust
        let sessions = self
            .sync
            .lock()
            .unwrap_or_else(|p| p.into_inner())
            .sessions()
            .to_path_buf();
        let (text, reply) = read_turn(&log_path(&sessions, &next.session), next.turn)
```

become

```rust
        let (text, reply) = read_turn(&log_path(&self.sessions, &next.session), next.turn)
```

**W7.** In `record`, the lines

```rust
        let result = self
            .sync
            .lock()
```

become

```rust
        let result = self
            .ledger
            .lock()
```

**W8.** In `pub struct Opened`, the lines

```rust
    clients: Clients,
    agent: String,
}
```

become

```rust
    clients: Clients,
    agent: String,
    owner: String,
}

impl Opened {
    /// The memory writes' side of the ledger (task 12).
    pub fn mirror(&self) -> Mirror {
        let ledger = self
            .sync
            .lock()
            .unwrap_or_else(|p| p.into_inner())
            .shared_ledger();
        Mirror::new(ledger, &self.agent, &self.owner)
    }
}
```

**W9.** In `open`, the lines

```rust
        agent: honcho.agent.clone(),
    })
```

become

```rust
        agent: honcho.agent.clone(),
        owner: honcho.owner.clone(),
    })
```

### `crates/memoryd/src/serve.rs`

**S1.** The line

```rust
use crate::honcho::ledger::{self, LedgerError};
```

becomes

```rust
use crate::honcho::ledger::{self, LedgerError};
use crate::honcho::mirror::Mirror;
```

**S2.** In `pub struct Memoryd`, the lines

```rust
    wake: Arc<Wake>,
    log: Log,
}
```

become

```rust
    wake: Arc<Wake>,
    /// With `[honcho]`: decides memory writes into the ledger, under the ledger's own lock.
    mirror: Option<Mirror>,
    log: Log,
}
```

**S3.** In `start`, the lines

```rust
    let honcho = match opened {
        Some(opened) => {
            let sync = Arc::clone(&opened.sync);
            worker::spawn(opened, &config, Arc::clone(&wake), Arc::clone(&log))?;
            Some(sync)
        }
        None => None,
    };
```

become

```rust
    let (honcho, mirror) = match opened {
        Some(opened) => {
            let sync = Arc::clone(&opened.sync);
            let mirror = opened.mirror();
            worker::spawn(opened, &config, Arc::clone(&wake), Arc::clone(&log))?;
            (Some(sync), Some(mirror))
        }
        None => (None, None),
    };
```

**S4.** In `start`, where `Memoryd` is built, the lines

```rust
            honcho,
            wake,
            log,
```

become

```rust
            honcho,
            wake,
            mirror,
            log,
```

**S5.** The line

```rust
    /// With `[honcho]`: decide the reported turn, and every other that waits (`TurnSync`).
```

becomes

```rust
    /// With `[honcho]`: decide journalled edit `id` (the latest when `None`) into the ledger. Called
    /// under the index lock, so edits are decided in the journal's order (M5c spec, section 4).
    fn mirror(&self, id: Option<&str>) {
        if let Some(mirror) = &self.mirror {
            mirror.journalled(&self.store, id, Timestamp::now(), &*self.log);
        }
    }

    /// With `[honcho]`: decide the reported turn, and every other that waits (`TurnSync`).
```

**S6.** In `memory`, the line

```rust
            let id = journal::record(&self.store, &ctx, &outcome)?;
```

becomes

```rust
            let id = journal::record(&self.store, &ctx, &outcome)?;
            self.mirror(Some(&id));
```

**S7.** In `admin`, the lines

```rust
            MemoryCommand::Reject => review::reject(&self.store, arg, now),
        };
```

become

```rust
            MemoryCommand::Reject => review::reject(&self.store, arg, now),
        };
        if result.is_ok() && matches!(command, MemoryCommand::Undo | MemoryCommand::Accept) {
            self.mirror(None);
        }
```

## Steps

- [ ] **1. Copy the given tests.** From the repository root, two commands:

  ```sh
  cp docs/plans/M5c1/files/crates/memoryd/tests/honcho_mirror.rs crates/memoryd/tests/
  cp docs/plans/M5c1/files/crates/memoryd/tests/honcho_mirror_serve.rs crates/memoryd/tests/
  ```

- [ ] **2. See them fail.** With a shell timeout of 600000 ms:
  - `cargo build --tests --keep-going -p memoryd`: one test target does not compile,
    `honcho_mirror`, with one error, `E0432` (``unresolved import `memoryd::honcho::mirror` ``).
    Every other target compiles.
  - `cargo test -p memoryd --test honcho_mirror_serve`: 1 test passes
    (`without_honcho_nothing_is_decided`) and 5 fail: nothing is decided yet.
- [ ] **3. The module.** Make edit H1, then copy the skeleton:

  ```sh
  cp docs/plans/M5c1/files/crates/memoryd/src/honcho/mirror.rs crates/memoryd/src/honcho/
  ```

  `cargo check -p memoryd`: no error, and 24 warnings, all from the unfilled bodies:
  ``function `conclusion` is never used``, ``function `minus` is never used``, ``unused import:
  `Kind` ``, ``unused import: `Workspace` ``, and twenty `unused variable` (`record` four times,
  `session` three times, `agent` and `notes` twice each, and `a`, `after`, `b`, `before`,
  `file`, `owner`, `peer`, `records`, `text` once each).
- [ ] **4. Fill the seven bodies of `mirror.rs`, one at a time, in the order of the file**, each
  from its comment, and run `cargo check -p memoryd` after each: `diff`, `minus`, `notes_of`,
  `session_of`, `conclusion`, `identity_actions`, `note_actions`. When a body is filled, delete
  its instruction comment (the `//` lines inside the body) and keep the function's doc comment
  (`///`) as it is. Change nothing else. Then `cargo check -p memoryd` prints no warning, and
  `cargo test -p memoryd --test honcho_mirror` passes its 8 tests.
- [ ] **5. The locks and the calls.** Make edits T1 to T8, W1 to W9 and S1 to S7.
  `cargo check -p memoryd`: no error and no warning.
- [ ] **6. The tests.** With a shell timeout of 600000 ms, and every test passes (8, 6, and the
  earlier ones unchanged):

  ```sh
  cargo test -p memoryd --test honcho_mirror --test honcho_mirror_serve --test honcho_turns \
    --test honcho_worker --test honcho_worker_more --test honcho_serve --test serve
  ```
- [ ] **7. Check.** Three commands:
  - `grep -n '\.ledger()' crates/memoryd/src/honcho/worker.rs` prints nothing: the worker takes
    the ledger's own lock, never through `TurnSync::ledger`.
  - `grep -n 'self.mirror(' crates/memoryd/src/serve.rs` prints two lines: after
    `journal::record` in `memory`, and in `admin`.
  - `git diff --stat -- crates/bxcore crates/loopd crates/brokerd crates/gatewayd crates/bxctl
    Cargo.toml Cargo.lock crates/memoryd/Cargo.toml crates/memoryd/src/journal.rs
    crates/memoryd/src/edit.rs crates/memoryd/src/honcho/ledger.rs` prints nothing.
- [ ] **8. Run the gate.** `cargo fmt --all`, then `make gate`, with a shell timeout of 600000 ms.
  Expected last line: `gate: ok`, with 1314 tests: 1300 before this task, plus 14 new ones (8 in
  `honcho_mirror.rs`, 6 in `honcho_mirror_serve.rs`).
- [ ] **9. Log and commit.** Add your row to `docs/implementer-log.md`; its Task cell is
  `M5c1/12-memoryd-honcho-mirror`. `git status --short` lists `honcho/mod.rs`, `turns.rs`,
  `worker.rs`, `serve.rs` and your log as modified, and `mirror.rs` and the two test files as
  new. Stage with `git add crates/memoryd docs/implementer-log.md Cargo.lock`, then commit as
  `AGENTS.md` says. `Cargo.lock` does not change in this task; the `git add` is harmless.

## Done when

- `make gate` prints `gate: ok` with 1314 tests.
- `git diff HEAD~1 --stat` lists only the files of step 9.

## Stop and report if

- A "before" block of H1, T1 to T8, W1 to W9 or S1 to S7 is not in its file exactly as shown, or
  is there more than once (W5 and W7 are code you wrote in task 11).
- Step 2 or 3 shows a different result: other test targets, errors of another kind, other
  warnings, or other tests passing or failing.
- `cargo check` reports an error after a body you filled or an edit you made, and the comment or
  the edit does not tell you how to fix it. Quote the error, and stop.
- A comment in the skeleton does not fit the code under it, or disagrees with this task's table.
  Quote both and stop.
- After step 4 or 6 a test fails although each body follows its comment and each edit is as
  shown. Do not change a test or the skeleton's given code: stop, and give the failing tests'
  names and their messages.
- The fix seems to need a change anywhere else: `journal.rs`, `edit.rs`, `review.rs`,
  `ledger.rs`, another crate, or a lock taken in another order. It does not; stop and say why you
  think so.
- A test outside the new files fails in the gate: check once whether it fails without your
  change, then stop and report both results.
- The gate passes with a count other than 1314.

## What the given tests pin

Each of these changes to the finished code makes at least one given test fail (21 tried; four
cannot change behaviour, see below): the diff taken as sets, or its sides swapped; moved entries
counted as dropped, or dropped notes left out; the wrong peer for either file; the new before the
old; a flag ignored; an untrusted note concluded, or a trusted one sent to the fleet; the owner's
session kept, or the undo chain cut short; a conclusion without a session sent anyway; the parts
numbered from 1; the memory tool's write, or the undo and accept, not decided; and a walk that
holds the ledger's lock while it asks `brokerd`. Not caught, since they cannot change what is
decided: moved notes treated as `noted` ones (only identity edits have them), the latest record
taken instead of the id (under the index lock they are the same), and the check against deciding
an edit twice (nothing calls it twice).