boxmaker/docs/specs/2026-09-27-m5c-honcho.md

# M5c design: Honcho

Status: approved by the owner, 2026-09-27, with section 15's answers. Designed with the owner the
same day (the decisions are marked "owner"). Supersedes P23 where they differ (section 14).

## 1. What M5c proves

The agent's experience outlives a session and is shared across the fleet, without anything a
session read from outside reaching memory the fleet treats as trusted. Finished turns go to
Honcho, whose own extraction replaces a flush job of ours; every trusted memory write is mirrored
there at once; `recall` searches the fleet as well as the local index; a new `ask_memory` tool asks
Honcho's dialectic.

The milestone is checked by:

- a turn from a session that read untrusted content never reaches a trusted workspace, and a turn
  from a `secret` session reaches no workspace;
- a result from an untrusted workspace taints the session that reads it;
- `recall` still answers from the local index when Honcho is down, and says what it could not
  reach;
- a memory write the agent is asked to keep ("remember that …") is in Honcho at once, and undoing
  it removes it.

Honcho is not rebuilt from the logs (owner, 2026-09-27). It accumulates; what it holds wrongly is
corrected by writing a contrary fact, which Honcho's dreaming reconciles, or, for a conclusion
`memoryd` wrote itself, by deleting that conclusion.

## 2. Two halves

One spec, two plans, each run and device-checked on its own (owner):

1. **Write path (M5c-1)**: the `brokerd` session-state socket, the speaker on each turn, the
   ledger, the sync worker, mirrored memory writes, configuration, `bxctl memory sync`, runbook.
2. **Read path (M5c-2)**: `recall` over the fleet, the `ask_memory` tool, trust labels on other
   agents' workspaces.

Projects in memory stay out of M5c (open row in `docs/decisions.md`).

## 3. Workspaces and peers

**Workspaces** (owner, 2026-09-27). Each agent writes to three:

| Workspace | Holds | Read by | Trust | Class |
|---|---|---|---|---|
| its **trusted** one | turns from clean sessions; conclusions mirroring its trusted memory writes | the agent (through `ask_memory`); other agents, as the owner labels it | trusted | `private` |
| its **untrusted** one | turns from sessions that read untrusted content | the agent only (through `ask_memory`) | untrusted, fixed | `private` |
| the **fleet** one, shared by every agent | untrusted notes, as messages from each agent's own peer | every agent | untrusted, fixed in code | `private` |

A workspace's class is also its ceiling for writes: nothing above `private` is written to Honcho
(owner). A `public` turn is written as `private`, since `public` records where a result came
from, not a permission. A session that reached `secret` by a turn's end is never synced, to any
workspace (owner).

**Peers** (owner):

- the owner: one peer, the same name in every workspace of the fleet (`[honcho] owner`); observed;
- other people who speak in a channel the agent answers: a peer each, named from their
  Mattermost username (a `.` becomes `_`: Honcho allows `[a-zA-Z0-9_-]`); observed;
- every peer in the fleet workspace is a member of the fleet, whether or not this agent's
  configuration names it (owner);
- the agent: a peer under its own name (`[honcho] agent`), **not observed** (`observe_me = false`).
  Its replies are written as messages and serve as context when Honcho reasons about the people
  it talks to, but Honcho draws no conclusions about the agent from them: they are the agent's
  own output and can restate what the session read. What the agent knows enters Honcho only as
  conclusions `memoryd` writes (section 5).

A Honcho session's peers, and who observes whom, are fixed when it is created (`roncho`'s blocking
client cannot change them). A Boxmaker session maps to one Honcho session per workspace, named after
it (in the fleet workspace too, where it holds only the agent's peer and the session's untrusted
notes); when a speaker who is not yet in it speaks, `memoryd` starts the next Honcho session for it,
the same name with `-2`, `-3` and so on, holding every peer so far.

## 4. What goes where

**Turns.** A finished turn is two messages: the speaker's message (the turn's text as `loopd`
logged it) from the speaker's peer, and the agent's final reply from the agent's peer (P23).
Tool calls, tool results and reasoning are not written. A text over 25,000 characters (Honcho's
limit) is written as several messages of at most 25,000, in order.

| The turn's session at its end | Goes to |
|---|---|
| clean, class `public` or `private` | the trusted workspace |
| untrusted (read untrusted content), class `public` or `private` | the untrusted workspace |
| class `secret` | nowhere |
| no speaker, or incomplete speaker information (a burst joined from several people) | nowhere (owner) |
| the text holds a known-prefix secret (`sanitize::refuse`'s secret rules) | nowhere |

**Memory writes.** Every write is mirrored as a conclusion the moment it is journalled (owner),
so "remember this" reaches Honcho at once, through the memory tool and its checks, never by
spotting the phrase:

| Journalled edit | In Honcho |
|---|---|
| identity `add` (memory.md or user.md) | a conclusion in the trusted workspace |
| identity `replace` | the old entry's conclusion deleted, the new one created |
| an entry dropped from identity (`remove`, or dropped by `consolidate`) | its conclusion deleted |
| an entry moved out of identity by `consolidate` | nothing: its conclusion is already there |
| a trusted note | a conclusion in the trusted workspace |
| an untrusted note | a message in the fleet workspace, from the agent's peer |
| a flagged note (a sanitizer flag rule matched) | nothing: it stays local (owner) |
| `bxctl memory undo` of an edit whose conclusions were written | those conclusions deleted; any it had deleted, written again |

A conclusion's observer and observed are the same peer, which is Honcho's own picture of that
peer: a `user.md` entry is the owner's (it is about the owner); a `memory.md` entry or a note is
the agent's. Each goes into its Boxmaker session's memory session in Honcho, `<id>_memory`, which
holds the agent's peer and the owner's and is created on first use, whether or not any turn of the
session is synced (a write is made during a turn, before that turn's session exists; owner,
2026-09-28). An undo's conclusions carry the undone edit's session. Every conclusion has a session
(Honcho leaves session-less conclusions out of session-filtered queries, and its automatic
dreaming skips a peer whose newest conclusion has none). `memoryd` never deletes a Honcho session:
that would delete the conclusions stamped with it.

The session logs and `memory/` stay the source of truth for everything local; Honcho is not
rebuilt from them.

## 5. The write path

1. **A turn ends.** `loopd` already sends `SessionUpdated` to `memoryd`. It gains the turn's
   number in the session (M5c-1 wire change, section 11).
2. **Decide.** `memoryd` asks `brokerd` for the session's state (section 7) and reads the turn's
   records in `loopd`'s log. The turn is untrusted if either says so (`brokerd`'s `untrusted`, or
   any `ToolResult` so far labelled untrusted); its class is `brokerd`'s. It decides by the table
   of section 4 and appends the decision to the ledger **at once**, so a Honcho outage later
   cannot change where the turn goes.
3. **If `brokerd` cannot answer**, nothing is decided. The turn is decided at the next
   `SessionUpdated` of any session, or by the worker's retry, from the session's state then; a
   late decision can only move a turn towards untrusted or unsynced (P23).
4. **Memory writes** are decided when journalled: the table of section 4, recorded in the same
   ledger.
5. **Send.** One worker thread in `memoryd` sends what is decided and not yet sent, oldest first:
   for a turn it creates the Honcho workspace, peers and session on first use (section 6), then
   adds the two messages as one batch; for a memory write it creates or deletes the conclusion,
   or, for an untrusted note, adds the message to the fleet workspace.
   Each send is recorded in the ledger with Honcho's ids.
6. **Honcho down** (a connection error, a timeout, a 5xx): the worker waits and tries again,
   doubling the wait from 5 s to 10 minutes. It logs the outage once, when it starts, and once
   when it ends, not on every try (runbook `honcho-unreachable`).
7. **Honcho refuses** (a 4xx other than 409 on create, which means "exists"): the entry is marked
   failed in the ledger, logged with the status and without the text (runbook
   `honcho-send-refused`), and the worker moves on. `bxctl memory sync` lists failed entries.

**The ledger**, `memory/honcho.jsonl`, is append-only JSON lines, one per event: `decided` (the
item, and the workspace it goes to or `skip` and why), `sent` (the item and Honcho's ids), `failed`
(the item and the status). What is decided and not sent is the outbox; there is no separate queue.
The worker checks for an item's `sent` line before sending it, and on start `memoryd` reads the
ledger through, so an item is sent at least once and a second time only if a crash falls between
the send and its `sent` line, or the owner removes that line (Honcho cannot delete a message).
Items are a turn (`session`, `turn`) or an edit (its journal id).

**Sync starts when Honcho is turned on.** The ledger records when sync began, and a turn that
started before then is skipped, so turning on `[honcho]` never sends earlier conversations.

Everything written to Honcho is neutralised (`sanitize::neutralise`); a reply is the model's own
output and was logged as it came, so it is neutralised here, when it leaves.

**The guarantee.** Nothing reaches a trusted workspace unless `brokerd` and `loopd`'s labels both
say the session was clean, and `brokerd` says its class was at most `private`, when the turn
ended.

## 6. Honcho, through `roncho`

`memoryd` uses `roncho` 1.0.0 from crates.io with `default-features = false, features =
["tls"]`, which includes the blocking client, so an HTTPS URL works as well as plain HTTP (task 00). It adds one crate to the lock file; `cargo deny`
passes (checked 2026-09-27).

- **Every client is built with `.without_config_file()`.** `roncho` otherwise reads
  `~/.config/roncho/roncho.toml` and takes an `api_key` from it when none is set.
- One client per workspace (a key is per workspace), shared by the worker and the read path.
- Timeouts: connect 5 s; read ("no bytes") 30 s for writes and searches, 180 s for chat (rift's
  dialectic took up to 120 s, and a non-streamed chat sends nothing until it is done).
- `roncho`'s `Status` and `Decode` errors carry up to 4 KiB of the response, which can be memory
  text: `memoryd` logs only the error's kind and status, never its `Display`.
- On creation, `memoryd` configures each workspace: the trusted and untrusted ones with reasoning,
  summaries and dreaming on (Honcho's defaults); the fleet one with `reasoning`, `summary` and
  `dream` all `enabled = false`, and every agent's peer in it unobserved, so no model reads the
  fleet's notes to derive anything (Honcho enqueues summaries before it checks reasoning, and
  schedules dreams by their own setting, so all three are set).
- Honcho ids allow `[a-zA-Z0-9_-]`, at most 512 characters. Boxmaker session ids fit; names from
  configuration are checked at start.

## 7. `brokerd`'s session state

A new socket, `run/memory-broker/state.sock`, served by `brokerd`, connected to only by
`memoryd` (directory mode 0700, owned by the user both run as, as the other sockets). One message,
`SessionState { session }`, answered with `SessionStateReply { class, untrusted }`. It changes
nothing. A session `brokerd` has not seen answers `private`, trusted: it has made no tool call.

## 8. `recall`

One tool, as today, searching (owner):

| Source | Searched with | Tag on each hit | Class |
|---|---|---|---|
| the local index | FTS5, as today | as today (`identity`, `memory`, `memory-tainted`) | as recorded |
| each other agent's trusted workspace in `[[honcho.read]]` | message search, and a conclusion query for that agent's peer and the owner's | `memory` if the owner labelled it trusted, else `memory-tainted`; the agent's name | the workspace's `class` |
| the fleet workspace | message search over the whole workspace | `memory-tainted`; the writing peer's name | `private` |

The agent's own workspaces are not searched: the local index holds every turn and note of this
agent with exact labels, and would return the same text twice. Each source runs on its own thread
with its own limit (the local index as today; 5 s for each Honcho source); `recall` answers with
what arrived, and a source that did not answer is one line, `<source>: unavailable`. Hits are
ranked within their source and interleaved to `k`, the local index first. Text from Honcho is
neutralised on arrival and size-capped like every recall snippet. Any `memory-tainted` hit marks
the session untrusted, and every hit raises the session's class to its label, as local hits do.

Honcho's search endpoints find messages only, and the mirrored memory writes in another agent's
trusted workspace are conclusions: so for those `recall` also queries conclusions
(`POST /conclusions/query`, which needs the observer and the observed; for a conclusion `memoryd`
wrote, both are the same peer). The fleet workspace holds messages, so one search finds the notes
of every peer in it, including agents this one was never told about (owner).

## 9. `ask_memory`

A new core tool, `ask_memory { question, workspaces?, reasoning? }`:

- `workspaces`: names from `[honcho]`: `own` (default), `untrusted`, `fleet`, or another agent's
  name from `[[honcho.read]]`;
- `reasoning`: `minimal` (default: one search, then the answer) or `low`;
- `question`: at most 2,000 characters.

One workspace chat (`POST /workspaces/{w}/chat`) per named workspace, in parallel, at most two at a
time; workspace chat searches every conclusion and message of the workspace without the model
choosing a peer (it averaged 13 s on rift, over only three calls; peer chat, 32 s over 318). Each
has 150 s. The answer lists each workspace's reply, neutralised and capped, tagged like a `recall`
hit (`memory` for the agent's own trusted workspace and for others the owner labelled trusted,
`memory-tainted` otherwise), or `<workspace>: unavailable`. Labels taint the session as `recall`'s
do. `brokerd`'s wait for this tool is 170 s and `loopd`'s wait on `brokerd` for it 180 s (section
11).

Adding a tool changes the `tools` array: a new epoch for every session when M5c-2 is deployed,
and the baseline is measured again against the 3,000-token budget.

## 10. Configuration and secrets

In `memoryd.toml`; without a `[honcho]` section, `memoryd` behaves exactly as after M5b:

```toml
[honcho]
url = "http://rift.scylla-hammerhead.ts.net:8000"
agent = "boxmaker0"
owner = "kyle"
trusted = "boxmaker0-trusted"
untrusted = "boxmaker0-untrusted"
fleet = "fleet"

[secrets.honcho_trusted]      # optional, one per workspace: honcho_trusted, honcho_untrusted,
file = "…"                    # honcho_fleet, honcho_read_<agent>; sent only when configured

[[honcho.read]]
agent = "boxmaker1"
workspace = "boxmaker1-trusted"
trust = "untrusted"           # the default; "trusted" must be written out
class = "private"
```

**API keys** (owner: the design must support them): each workspace this agent writes or reads has
its key, `honcho_trusted`, `honcho_untrusted`, `honcho_fleet` and `honcho_read_<agent>`, from
`bxcore::secrets` (M5b), and `memoryd` sends it as a bearer token on every call to that workspace
and never logs it. A workspace without a key configured is called without one; rift has auth off
today (2026-09-27), and turning it on needs only the keys. A refused key (401 or 403) stops the
sends to that workspace, is logged once with the workspace's name (runbook `honcho-key-refused`),
and is shown by `bxctl memory sync`; the others go on. An invalid `[honcho]` section stops
`memoryd` at start with the reason (runbook `honcho-config-invalid`). `docs/egress.md` gains rift:
`memoryd` → the configured URL, for writes, searches and chat.

## 11. Links between the roles

M5c's changes to the table of the M5b spec, section 10:

| Link | Serves | Connects | Path or address | Carries | When the server is down |
|---|---|---|---|---|---|
| Session state (new) | `brokerd` | `memoryd` | `run/memory-broker/state.sock` | `SessionState` → `SessionStateReply` | turns wait undecided (`broker-state-unavailable`) |
| Honcho (new) | Honcho | `memoryd` | `[honcho] url` | `roncho` calls | sends wait (`honcho-unreachable`); `recall` answers from the local index |
| Memory for turns | `memoryd` | `loopd` | as before | `SessionUpdated` gains `turn` | as before |
| Turns | `loopd` | `gatewayd`, `bxctl chat` | as before | `Turn` gains `speaker` | as before |

Wire changes (fixtures change): `Turn` gains `speaker: Option<String>`, which `gatewayd` sets to
the Mattermost username when every post of the burst is from one person, and `bxctl chat` sets
to the owner's name from its configuration; `loopd` logs it on the `User` record (a new optional
field: older logs read as `None`); `SessionUpdated` gains `turn: u64`. A turn whose speaker is
`None`, or not a valid peer name, is not synced (owner).

## 12. Runbook

New entries, given as fragments: `honcho-unreachable`, `honcho-send-refused`, `honcho-key-refused`,
`broker-state-unavailable`, `honcho-config-invalid`. Each names what `bxctl memory sync` shows.

## 13. Testing

Given tests, first, against a fake Honcho server (a small HTTP server in `memoryd`'s test support
that records requests and answers from fixtures taken from rift's OpenAPI document, like
`loopd`'s fake llama server):

- **Decisions**: clean → trusted; untrusted by `brokerd` or by the log → untrusted; `secret` →
  none; no single speaker → none; a key in the text → none; `brokerd` down → undecided, then
  decided from the later state, never towards trusted.
- **The ledger**: a restart sends nothing twice; an outage sends everything once it ends, in
  order; a refused entry is marked failed and the rest go on.
- **Memory writes**: each row of section 4's table, including undo.
- **Peers and sessions**: the agent's peer not observed; a new speaker starts the next Honcho
  session; names fit Honcho's pattern.
- **`roncho`**: a `roncho.toml` with an `api_key` in the test's `HOME` is not read.
- **Keys**: a configured key is sent as a bearer token to its workspace only, and appears in no
  log line or error; without a key, no `Authorization` header; a 401 stops that workspace's sends
  only, once logged.
- **The fleet**: an untrusted note is a message from the agent's peer; the fleet workspace is
  created with reasoning, summaries and dreaming off; a note
  from a peer this agent's configuration does not name is found by `recall`.
- **`recall`**: the fleet sources with their tags and classes; a source that times out is one
  `unavailable` line and the rest still answer; the agent's own workspaces are not searched.
- **`ask_memory`**: two at a time; the per-workspace limit; labels; unknown workspace names.
- **No `[honcho]` section**: M5b's behaviour, every M5b test unchanged.
- **Attack tests (T37)**: a turn from a session that fetched a page never reaches the trusted
  workspace, even when `brokerd` answers late; a fleet hit taints the reading session.

**On the device** (straylight, rift, the owner's Mattermost), for each half:

- M5c-1: a clean turn appears in the trusted workspace, an untrusted one in the untrusted
  workspace, a `secret` session's in neither; "remember that …" appears as a conclusion at once,
  and `bxctl memory undo` removes it; rift stopped (or unreachable) → the outbox waits and drains
  once it is back, sending nothing twice; `bxctl memory sync` shows each state.
- M5c-2: `recall` returns a fleet note written by a second agent (a second `memoryd` with its own
  name), tagged `memory-tainted`, and the session turns untrusted; `ask_memory` answers from the
  agent's own trusted workspace within its limit; with rift down, `recall` answers from the local
  index with `unavailable` lines.

## 14. Changes to the brief and decisions

With this spec, as their own commits:

- `docs/design.md`, Memory items 4 and 5: two workspaces per agent and the fleet workspace; no
  rebuild ("corrected by writing a contrary fact"); mirrored memory writes; the peers.
- `docs/decisions.md`: P23 applied as amended here (the fleet workspace for untrusted notes; no
  rebuild); rows for the class ceiling, the peers, mirroring, the ledger, and `recall`'s sources.
- `docs/milestones.md`, M5c: its check "Honcho's contents are rebuilt from the logs" goes.

## 15. The owner's answers (2026-09-27)

1. **The speaker**: a turn with no speaker, or incomplete speaker information, is not sent
   (sections 4 and 11).
2. **The fleet**: every peer in the fleet workspace is a member of the fleet. Fleet notes are
   therefore messages, which one search finds whoever wrote them; an undone note gets a
   retraction message, since Honcho cannot delete a message (sections 3, 4, 8). (2026-09-28:
   `bxctl memory undo` cannot undo a note, since notes are append-only, so no retraction is ever
   sent and the row went from section 4; owner.)
3. **For `roncho`'s agent**, accepted (found 2026-09-27, none blocking): make the config file
   opt-in (it supplies an `api_key` today; the README's "reads no environment variables" is
   wrong); evidence decoding requires `observer_id` and `observed_id`, which rift's schema lacks;
   a message's `configuration` (reasoning per message) is not exposed; the tag `v1.0.0` names
   `b4e7864` on `origin` and `1f5ea42` on crates.io.
4. **API keys**: the design supports them per workspace (section 10); rift has auth off today.