boxmaker/docs/plans/M3a/03-proto-chain-verifier.md

M3a task 03: the audit chain verifier

Branch: m3a (run git switch m3a; git status --short must be empty, otherwise stop) Commit subject: Add the audit chain verifier

Goal

proto::ChainVerifier checks the audit log's hash chain. It is a pure state machine: no files, no clock. The caller feeds it lines; it returns a report. brokerd's startup and bxctl audit verify both use it, so they cannot disagree.

Files

• Copy: crates/proto/tests/chain.rs, and the directory crates/proto/tests/fixtures/audit/ (30 small logs; their hashes are real)
• Create: crates/proto/src/chain.rs
• Modify: crates/proto/src/lib.rs, crates/proto/src/audit.rs, docs/implementer-log.md

lib.rs gains pub mod chain; and pub use chain::{ChainFailure, ChainReport, ChainVerifier, Location, TornTail};. audit.rs gains the same pub use crate::chain::{…}; line, so proto::audit::ChainVerifier works too.

Interfaces

#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Location { pub file: String, pub line: u64 }          // line is 1-based

#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ChainFailure {
    pub file: String, pub line: u64, pub what: String,
    pub last_good: Hash32,    // hash of the last line that verified before the failure
    pub break_prev: Hash32,   // hash of the last line fed
    pub break_seq: u64,       // the seq an AcceptedBreak appended now must carry (rule 7)
    pub tail_torn: bool,      // the last line fed had no newline
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct TornTail {
    pub at: Location, pub has_newline: bool, pub bytes: u64, pub sha256: Hash32,
    pub recovery_prev: Hash32,   // hash of the last line that verified
    pub recovery_seq: u64,       // the seq a Recovery appended now must carry
}
#[derive(Debug, Clone, PartialEq, Eq)]
pub struct ChainReport {
    pub records: u64, pub head: Option<Hash32>, pub next_seq: u64,
    pub failure: Option<ChainFailure>,
    pub recoveries: Vec<Location>, pub accepted_breaks: Vec<Location>,
    pub abandoned: Vec<u64>, pub unfinished: Vec<u64>,
    pub clock_warnings: Vec<Location>, pub torn_tail: Option<TornTail>,
}
pub struct ChainVerifier { /* private */ }
impl ChainVerifier {
    pub fn new() -> Self;                                   // expects seq 0 and prev all zeros
    pub fn resume(next_seq: u64, prev: Hash32) -> Self;    // continue after a known line
    pub fn file(&mut self, name: &str);                     // starts the next file; line 1 is next
    pub fn line(&mut self, bytes: &[u8], has_newline: bool);    // bytes exclude the newline
    pub fn feed(&mut self, name: &str, content: &[u8]);     // file(name), then each line
    pub fn finish(self) -> ChainReport;
}

Also impl Default (it calls new). feed splits content at every \n; bytes after the last \n are one more line with has_newline false; an empty content is no lines.

How it judges

The hash of a line is proto::sha256 of its bytes without the newline. The state is: the seq expected next, prev (hash of the last line that verified; zeros or the resume value at the start), the hash of the last line fed whether it verified or not, and the first failure if any.

Hold each line back until the next one arrives, and judge it then, or in finish. The reason is rule 2: whether a line is a record can depend on the line after it.

1. A verified record. The line parses as AuditRecord, its seq is the expected one and its prev equals prev. Then: records += 1, head and prev become its hash, next_seq is its seq + 1. If its time is earlier than the previous verified record's, push its location to clock_warnings. Track what it opens and closes: Decision with Allowed {} opens a run under its own seq; with Ask {} opens an ask under its own seq. Approval { decision, outcome } closes the ask decision, and if outcome is Allowed {} opens a run under decision. Result { decision } closes that run. At finish, open asks are abandoned and open runs are unfinished, both in ascending order.
2. A recovered line. Before judging a held line, look at the next line. If the next line parses as a record whose event is Recovery { torn_bytes, torn_sha256 }, with torn_bytes the held line's length, torn_sha256 its hash, prev equal to the current prev and seq equal to the expected seq, the held line is not a record: push its location to recoveries and change nothing else. It makes no difference whether the held line parses. A crash can cut a record exactly before its newline, leaving complete JSON. The Recovery line is then judged in its turn by rule 1 and takes that seq.
3. A Recovery record that reaches rule 1 without having recovered the line before it fails: a recovery record that does not describe the line before it.
4. Failures, in this order, with exactly these texts as what: does not parse as an audit record; seq is {got}, expected {want}; for line 1 of any file but the first one ever, does not chain from the last line of the file before, otherwise prev is not the hash of the line before. (For a resumed verifier line 1 of its first file gets the "file before" text too.) Only the first failure is kept. On a failure nothing is updated: last_good is the current prev. If sha256 returns an error for a line, that line fails with the line is too long to hash.
5. After a failure no line is checked. Count them: the failing line is 1. Each held line is only tested for being the break record: it parses, its event is AcceptedBreak { file, line, last_good }, its prev is the hash of the line fed just before it, file and line name the failure, last_good equals the failure's, and its seq equals the failure's expected seq plus the count so far (failure at line 7, break at line 10: plus 3). If so: clear the failure, push the break's location to accepted_breaks, and treat it as a verified record (rule 1), so next_seq is its seq + 1. If not, add 1 to the count. Use checked_add; a line in the region may claim seq 18446744073709551615.
6. A break with no failure before it. A line that parses with an AcceptedBreak event and is not in a failed region skips rule 1's checks and comes here. It is accepted in one case only: the verifier was made with resume, and the break's file sorts before the first file this verifier was given (file < first, as strings). Then check only prev, accept it as in rule 5 and continue from its seq. The same exception applies inside a failed region in place of the four checks on file, line, last_good and seq. Otherwise it fails: an accepted break with no failure before it.
7. finish. If a line is still held and there is no failure: when it has no newline, or has one but does not parse, it is a torn tail. Fill torn_tail and judge it no further. Otherwise judge it with no next line. If there is a failure, the held line is one more line of the region (rule 5). break_seq is the failure's expected seq plus the final count; break_prev is the hash of the last line fed.

Steps

• 1. Copy. cp docs/plans/M3a/files/crates/proto/tests/chain.rs crates/proto/tests/ and cp -r docs/plans/M3a/files/crates/proto/tests/fixtures/audit crates/proto/tests/fixtures/
• 2. See the test fail. cargo test -p proto --test chain. Expected: it does not compile.
• 3. Write chain.rs, add the mod and use lines. Run cargo fmt --all.
• 4. See the tests pass. cargo test -p proto --test chain. Expected: 13 passed.
• 5. Run the gate. make gate. Expected last line: gate: ok.
• 6. Log and commit. git add crates/proto docs/implementer-log.md && git commit

Done when

• cargo test -p proto --test chain reports 13 passed; make gate prints gate: ok.

Stop and report if

• A fixture seems wrong. Read the comment beside its case in chain.rs first: each directory name says what was done to the log, and several are meant to verify.
• chain.rs would pass 500 lines.