review: a session log that cannot be read now is waited out

The worker marked a turn failed on any error reading its log. A missing log,
a damaged line or a turn not found still fail it; any other I/O error is now
waited out with the outage's doubling wait, logged once when it starts and
once when the log reads again (pointer session-log-damaged, whose runbook
entry now covers it), shown by bxctl memory sync, and the turn stays waiting.

read_records tells an unreadable file from a damaged line (read_log is
unchanged). The message helpers move from worker.rs to honcho/messages.rs,
re-exported, to keep worker.rs under 500 lines.

Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
2026-09-28 14:15:04 -07:00
co-authored by Claude Opus 5.5
parent bd4cf9fb7f
commit 5e57cd69a3
8 changed files with 306 additions and 69 deletions
+19 -1
View File
@@ -372,6 +372,21 @@ that does not parse tool calls, or whose cache does not work.
logs the same line once, starting `memoryd: `, and that session's turns wait undecided for Honcho
until the log reads again.
With `[honcho]`, `memoryd`'s Honcho worker also reads a decided turn again from the log to send
it. If the log is missing, has a damaged line, or no longer holds the turn, the turn is marked
failed (`memoryd: turn 3 of session mm-chat-1 cannot be sent (log); it is marked failed; …`), and
`bxctl memory sync` lists it under `failed, oldest first:` with the kind `log`. If the file is
there but cannot be read now (a permission, an I/O error, a directory in its place), the worker
logs once
```text
memoryd: turn 3 of session mm-chat-1 cannot be read now (/srv/bx/sessions/mm-chat-1/0.jsonl: Permission denied (os error 13)); sends wait and are retried; see docs/runbook.md#session-log-damaged
```
and tries again with the doubling wait of `honcho-unreachable`, sending nothing else meanwhile;
`bxctl memory sync` shows `session log unreadable since <time>: sends wait and are retried`. When
the log reads, it logs `memoryd: <path> reads again; waiting sends go on`, and the turn is sent.
**Why.** `loopd` replays the log exactly to rebuild the conversation, so a line it cannot read
would change what the model sees. It refuses rather than guess. `last line does not end in a
newline` means a write was cut off: `loopd` writes a record only after the request it describes
@@ -396,7 +411,10 @@ Anything else (a damaged line in the middle, an unknown record type) means the f
by something other than `loopd`. Start a new session and keep the old log for inspection.
**Check.** The next turn on the session succeeds. The first request re-reads the conversation if
the cache was lost, which `loopd` logs as a cache loss.
the cache was lost, which `loopd` logs as a cache loss. With `[honcho]`, a log that could not be
read is fixed by making it readable again (`ls -l "$f"`; the owner and mode `loopd` gave it):
`memoryd` logs that it reads again, and `bxctl memory sync` no longer shows the `session log
unreadable` line.
## memory-unavailable