Hand over the M3a plan: 22 tasks, their files, and the check record
Task files, the files they copy in (byte-identical to the reference on m3a-ref), each area's check record, and a README with the per-task table of what each check exposed. The handoff note is done with. Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
This commit is contained in:
Claude Opus 5
@@ -0,0 +1,138 @@
|
||||
# M3a task 03: the audit chain verifier
|
||||
|
||||
**Branch:** `m3a` (run `git switch m3a`; `git status --short` must be empty, otherwise stop)
|
||||
**Commit subject:** `Add the audit chain verifier`
|
||||
|
||||
## Goal
|
||||
|
||||
`proto::ChainVerifier` checks the audit log's hash chain. It is a pure state machine: no files,
|
||||
no clock. The caller feeds it lines; it returns a report. `brokerd`'s startup and
|
||||
`bxctl audit verify` both use it, so they cannot disagree.
|
||||
|
||||
## Files
|
||||
|
||||
- Copy: `crates/proto/tests/chain.rs`, and the directory `crates/proto/tests/fixtures/audit/`
|
||||
(30 small logs; their hashes are real)
|
||||
- Create: `crates/proto/src/chain.rs`
|
||||
- Modify: `crates/proto/src/lib.rs`, `crates/proto/src/audit.rs`, `docs/implementer-log.md`
|
||||
|
||||
`lib.rs` gains `pub mod chain;` and
|
||||
`pub use chain::{ChainFailure, ChainReport, ChainVerifier, Location, TornTail};`.
|
||||
`audit.rs` gains the same `pub use crate::chain::{…};` line, so `proto::audit::ChainVerifier`
|
||||
works too.
|
||||
|
||||
## Interfaces
|
||||
|
||||
```rust
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct Location { pub file: String, pub line: u64 } // line is 1-based
|
||||
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct ChainFailure {
|
||||
pub file: String, pub line: u64, pub what: String,
|
||||
pub last_good: Hash32, // hash of the last line that verified before the failure
|
||||
pub break_prev: Hash32, // hash of the last line fed
|
||||
pub break_seq: u64, // the seq an AcceptedBreak appended now must carry (rule 7)
|
||||
pub tail_torn: bool, // the last line fed had no newline
|
||||
}
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct TornTail {
|
||||
pub at: Location, pub has_newline: bool, pub bytes: u64, pub sha256: Hash32,
|
||||
pub recovery_prev: Hash32, // hash of the last line that verified
|
||||
pub recovery_seq: u64, // the seq a Recovery appended now must carry
|
||||
}
|
||||
#[derive(Debug, Clone, PartialEq, Eq)]
|
||||
pub struct ChainReport {
|
||||
pub records: u64, pub head: Option<Hash32>, pub next_seq: u64,
|
||||
pub failure: Option<ChainFailure>,
|
||||
pub recoveries: Vec<Location>, pub accepted_breaks: Vec<Location>,
|
||||
pub abandoned: Vec<u64>, pub unfinished: Vec<u64>,
|
||||
pub clock_warnings: Vec<Location>, pub torn_tail: Option<TornTail>,
|
||||
}
|
||||
pub struct ChainVerifier { /* private */ }
|
||||
impl ChainVerifier {
|
||||
pub fn new() -> Self; // expects seq 0 and prev all zeros
|
||||
pub fn resume(next_seq: u64, prev: Hash32) -> Self; // continue after a known line
|
||||
pub fn file(&mut self, name: &str); // starts the next file; line 1 is next
|
||||
pub fn line(&mut self, bytes: &[u8], has_newline: bool); // bytes exclude the newline
|
||||
pub fn feed(&mut self, name: &str, content: &[u8]); // file(name), then each line
|
||||
pub fn finish(self) -> ChainReport;
|
||||
}
|
||||
```
|
||||
|
||||
Also `impl Default` (it calls `new`). `feed` splits `content` at every `\n`; bytes after the
|
||||
last `\n` are one more line with `has_newline` false; an empty `content` is no lines.
|
||||
|
||||
## How it judges
|
||||
|
||||
The hash of a line is `proto::sha256` of its bytes without the newline. The state is: the `seq`
|
||||
expected next, `prev` (hash of the last line that verified; zeros or the `resume` value at the
|
||||
start), the hash of the last line fed whether it verified or not, and the first failure if any.
|
||||
|
||||
**Hold each line back until the next one arrives**, and judge it then, or in `finish`. The reason
|
||||
is rule 2: whether a line is a record can depend on the line after it.
|
||||
|
||||
1. **A verified record.** The line parses as `AuditRecord`, its `seq` is the expected one and its
|
||||
`prev` equals `prev`. Then: `records += 1`, `head` and `prev` become its hash, `next_seq` is
|
||||
its `seq + 1`. If its `time` is earlier than the previous verified record's, push its location
|
||||
to `clock_warnings`. Track what it opens and closes:
|
||||
`Decision` with `Allowed {}` opens a run under its own `seq`; with `Ask {}` opens an ask under
|
||||
its own `seq`. `Approval { decision, outcome }` closes the ask `decision`, and if `outcome` is
|
||||
`Allowed {}` opens a run under `decision`. `Result { decision }` closes that run. At `finish`,
|
||||
open asks are `abandoned` and open runs are `unfinished`, both in ascending order.
|
||||
2. **A recovered line.** Before judging a held line, look at the next line. If the next line
|
||||
parses as a record whose event is `Recovery { torn_bytes, torn_sha256 }`, with `torn_bytes`
|
||||
the held line's length, `torn_sha256` its hash, `prev` equal to the current `prev` and `seq`
|
||||
equal to the expected `seq`, the held line is not a record: push its location to `recoveries`
|
||||
and change nothing else. **It makes no difference whether the held line parses.** A crash can
|
||||
cut a record exactly before its newline, leaving complete JSON. The `Recovery` line is then
|
||||
judged in its turn by rule 1 and takes that `seq`.
|
||||
3. A `Recovery` record that reaches rule 1 without having recovered the line before it fails:
|
||||
`a recovery record that does not describe the line before it`.
|
||||
4. **Failures**, in this order, with exactly these texts as `what`:
|
||||
`does not parse as an audit record`; `seq is {got}, expected {want}`;
|
||||
for line 1 of any file but the first one ever, `does not chain from the last line of the file
|
||||
before`, otherwise `prev is not the hash of the line before`. (For a resumed verifier line 1
|
||||
of its first file gets the "file before" text too.) Only the first failure is kept. On a
|
||||
failure nothing is updated: `last_good` is the current `prev`. If `sha256` returns an error
|
||||
for a line, that line fails with `the line is too long to hash`.
|
||||
5. **After a failure** no line is checked. Count them: the failing line is 1. Each held line is
|
||||
only tested for being the break record: it parses, its event is
|
||||
`AcceptedBreak { file, line, last_good }`, its `prev` is the hash of the line fed just before
|
||||
it, `file` and `line` name the failure, `last_good` equals the failure's, and its `seq` equals
|
||||
the failure's expected `seq` plus the count so far (failure at line 7, break at line 10: plus
|
||||
3). If so: clear the failure, push the break's location to `accepted_breaks`, and treat it as
|
||||
a verified record (rule 1), so `next_seq` is its `seq + 1`. If not, add 1 to the count.
|
||||
Use `checked_add`; a line in the region may claim `seq` 18446744073709551615.
|
||||
6. **A break with no failure before it.** A line that parses with an `AcceptedBreak` event and
|
||||
is not in a failed region skips rule 1's checks and comes here. It is accepted in one case only: the
|
||||
verifier was made with `resume`, and the break's `file` sorts before the first file this
|
||||
verifier was given (`file < first`, as strings). Then check only `prev`, accept it as in rule
|
||||
5 and continue from its `seq`. The same exception applies inside a failed region in place of
|
||||
the four checks on `file`, `line`, `last_good` and `seq`. Otherwise it fails:
|
||||
`an accepted break with no failure before it`.
|
||||
7. **`finish`.** If a line is still held and there is no failure: when it has no newline, or has
|
||||
one but does not parse, it is a torn tail. Fill `torn_tail` and judge it no further. Otherwise
|
||||
judge it with no next line. If there is a failure, the held line is one more line of the
|
||||
region (rule 5). `break_seq` is the failure's expected `seq` plus the final count;
|
||||
`break_prev` is the hash of the last line fed.
|
||||
|
||||
## Steps
|
||||
|
||||
- [ ] **1. Copy.** `cp docs/plans/M3a/files/crates/proto/tests/chain.rs crates/proto/tests/` and
|
||||
`cp -r docs/plans/M3a/files/crates/proto/tests/fixtures/audit crates/proto/tests/fixtures/`
|
||||
- [ ] **2. See the test fail.** `cargo test -p proto --test chain`. Expected: it does not compile.
|
||||
- [ ] **3. Write `chain.rs`**, add the `mod` and `use` lines. Run `cargo fmt --all`.
|
||||
- [ ] **4. See the tests pass.** `cargo test -p proto --test chain`. Expected: `13 passed`.
|
||||
- [ ] **5. Run the gate.** `make gate`. Expected last line: `gate: ok`.
|
||||
- [ ] **6. Log and commit.** `git add crates/proto docs/implementer-log.md && git commit`
|
||||
|
||||
## Done when
|
||||
|
||||
- `cargo test -p proto --test chain` reports 13 passed; `make gate` prints `gate: ok`.
|
||||
|
||||
## Stop and report if
|
||||
|
||||
- A fixture seems wrong. Read the comment beside its case in `chain.rs` first: each directory
|
||||
name says what was done to the log, and several are meant to verify.
|
||||
- `chain.rs` would pass 500 lines.
|
||||
Reference in New Issue
Block a user