The store opens before trusted time. Observations already in the log are quarantined (not summarised, offered, believed or age-trimmed; addressed ones report released) until LogStore::set_wall_clock runs the downtime fold. Trusted time comes from the robot's RTC/GPS, else a peer's observed_at + age_ms (smallest age wins), else the robot's own fail-safe. Supersedes "don't open the store until trusted time". Plan: Task 10 rewritten (quarantine, set_wall_clock, FoldReport, derived marker wall time); new Task 10b (host trusted-time state machine, time from peers, observed_at). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
383 lines
40 KiB
Markdown
383 lines
40 KiB
Markdown
# tines-core 1.0 Implementation Plan
|
||
|
||
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
|
||
|
||
**Goal:** Ship `tines-core` 1.0.0: a versioned wire format, a frozen minimal public API, a settled `Store` trait, a reserved signature slot, and evidence (fuzzing, pinned vectors, MSRV, semver-checks) that the untrusted-input and compatibility promises hold.
|
||
|
||
**Architecture:** 1.0 adds nothing to the exchange's behaviour; it removes, hides, pins, documents and reserves. Two wire-format changes (a version byte, then the `Observation` reshape) plus a `Store` trait extension and four core-hardening fixes, each landed as an independently green, committed task on top of `master` (`aedb802`).
|
||
|
||
**Tech Stack:** Rust 2021, `#![no_std]` + optional `alloc`, `postcard`, `heapless`, `blake2`, `serde`, `proptest`, cargo-fuzz (nightly), cargo-msrv, cargo-semver-checks.
|
||
|
||
**Spec:** `docs/superpowers/specs/2026-10-18-tines-1-0-design.md` (approved; decisions F and G added 2026-10-20, G amended after review) — the plan argues from the spec, so the spec travels with it; executors read both. Review: `HANDOFF.md`.
|
||
|
||
## Global Constraints
|
||
|
||
Every task's requirements implicitly include this section.
|
||
|
||
- `tines-core` stays `#![no_std]`; `alloc` an optional default-on feature; `--no-default-features` builds for `thumbv7em-none-eabihf`. No async/threads/I/O/panics on untrusted input; wire frames are untrusted, malformed bytes become `Err`/a counted drop, never a panic.
|
||
- `Addr` is never mapped to `NodeId` by core.
|
||
- **Identity hash never changes**: `id()` reads `dest.map(|d| d.to)` and is byte-for-byte the 0.x hash; the pinned vector (below) carries over unchanged. `sig` is excluded from the id.
|
||
- Exact values: `MAX_TOPIC=64`, `MAX_PAYLOAD=1024`, `MAX_PEERS=8`, `SUMMARY_BYTES=512`, `BLOOM_K=3`, `SUMMARY_INTERVAL_MS=2000`, `REASSEMBLY_TIMEOUT_MS=5000`, `SEND_QUEUE_CAP=256`, `MAX_FRAME=1500`, `ACK_IDS_CAP=90`, `EVENT_QUEUE_CAP=64`, `BCAST_SENT_CAP=256`, `ACK_BUFFER_CAP=256`, `MAX_SIG=64`, `MAX_FRAGMENTS=64` (pub(crate)), `FRAG_OVERHEAD` (currently 64, becomes 65 in Task 1, private in Task 5), `SUMMARY_OVERHEAD` (private, currently 24, becomes 25 in Task 1).
|
||
- Decisions already made (do not reopen): D1 reserve signature slot; D2 `Dest` replaces `to`+`lifetime_ms`; D3 drop-and-report; D4 only `tines-core` goes 1.0; D5 no store migration (old logs refused, wiped); Q1 `MAX_SIG=64`; Q2 coverage = whole observation + `version`+`algo`; Q3 malformed reported like version mismatch; R1 drop reports carry transport `Addr`; R2 drops counted by `(Addr, reason)`, not queued; R3 `MAX_FRAME=1500` stays; R4 `observed_at` = observer's wall-clock ms since Unix epoch; F `Store::flush` + `Node::flush(now)` replaces `store_mut`; G age across downtime via a wall-clock marker, **as amended twice on 2026-10-20**: the store opens *before* trusted time (`LogStore::open(path, now)`, unchanged), with everything already in the log **quarantined** until `LogStore::set_wall_clock(wall_now: u64, now)` runs the downtime fold. Trusted time comes from the robot's own RTC/GPS, else from a peer's observation (`observed_at + age_ms`, smallest `age_ms` wins), else the robot's own fail-safe (outside tines). The marker's wall time is derived (`wall_at_set + (now − tick_at_set)`). This supersedes "open only once trusted time exists".
|
||
- `tines-ros2` is not in the default gate (its `ros2` feature is off); verify any change touching `tines-ros2` with `docker build -f Dockerfile.ros2 -t tines-ros2-check .` (cached image exists).
|
||
- **Gate:** run `sh scripts/ci.sh` verbatim and require it green at every commit — it runs `cargo fmt --all -- --check`, then clippy/doc/test for the whole workspace, then the no-`alloc` clippy/doc/test matrix for `tines-core` (`sh scripts/check-no-std.sh`). Do not substitute the individual steps for the script.
|
||
- House rule: primary branch is `master`; land by rebase + fast-forward. Commit messages use the conventional prefix shown in each task; license Apache-2.0, copyright "K. Isom".
|
||
- Execution: local `ornith-1.5-35b-a3b` subagents, reviews run sequentially (single-GPU llama-server). When a task touches `tines-ros2`, its final step is the Docker build, not a plain `cargo check`.
|
||
|
||
## Review Focus
|
||
|
||
The six failure modes the spec implies but no single task's tests exercise, most likely to bite first:
|
||
|
||
1. **A frame with the wrong version byte** (un-updated or foreign peer) must be dropped *and reported* as `VersionMismatch { version }`, never silently swallowed and never folded into `Malformed`. An *empty* frame is `Malformed` (there is no version to mismatch). → pinned in Task 2.
|
||
2. **A frame whose `from` changes within one contact** must be dropped as `IdentityChanged` and the peer's addressed mail must *not* be handed over to the new identity. → pinned in Task 9.
|
||
3. **A signature with an unknown algorithm id, or the wrong length for a known one** — the first must be carried and forwarded untouched; the second (and `algo = 0` and `version = 0`) is `Malformed`, including when the observation is rebuilt from fragments. → pinned in Task 4.
|
||
4. **Held data must not leak before the fold, and unknown downtime must take the safe path.**
|
||
- Before `set_wall_clock`, a quarantined observation must not appear in a summary, a custody offer or a belief, and a quarantined addressed duplicate must not be acked.
|
||
- When the downtime is unknown (no marker, a marker without wall time, or a wall clock that went backwards), held addressed messages expire (a directive that may have outlived `lifetime_ms` is never delivered), and held broadcasts are floored at `unknown_downtime_age_ms`. Neither is delivered as-is.
|
||
- → pinned in Task 10.
|
||
5. **A valid frame with trailing bytes** must be rejected as `Malformed` (canonical decode), and a frame whose decode finds the `from` field missing must not panic. → pinned in Task 9.
|
||
6. **Every frame must fit `mtu()`** after the +1 version byte and the +67-byte signature — swept across MTUs from the BLE floor (`MIN_MTU = 23`) to `MAX_FRAME`, with an oversized observation given up cleanly as `TooLarge`, never sent truncated. → pinned in Task 4.
|
||
|
||
---
|
||
|
||
## File Map (decomposition locked in)
|
||
|
||
- `tines-core/src/wire.rs` — envelope encode/decode, `frame_sender`; `Frame`/`WireError` become private in Task 5.
|
||
- `tines-core/src/observation.rs` — `Observation`, `Dest`, `Signature`, `SigAlgo`, `NodeId`, `ObservationId`, `Tick`, `Topic`, `Payload`.
|
||
- `tines-core/src/consts.rs` — `WIRE_VERSION` + the wire/local split.
|
||
- `tines-core/src/exchange.rs` — drop counters, `NodeEvent`, peer pinning, hardening. Stays private (`pub(crate)` from Task 5).
|
||
- `tines-core/src/node.rs` — `Node` facade: `flush(now)`, `drain_events -> NodeEvent`.
|
||
- `tines-core/src/store.rs` — `Store` trait: `is_empty`, `flush`; the `store_conformance` module.
|
||
- `tines-core/src/lib.rs` — the 1.0 public surface + `#![deny(missing_docs)]`.
|
||
- `tines-core/tests/vectors.rs` — pinned identity + golden wire vectors.
|
||
- `tines-store/src/log.rs` — format v2 + wall-clock marker + quarantine + `set_wall_clock` downtime fold.
|
||
- `tines-sim/src/capped.rs` — the `CappedStore` wrapper (every `Store` change goes through it).
|
||
- `tines/src/ipc.rs`, `tines-ros2/src/node.rs` — host consumers of `NodeEvent`/`flush`/`observed_at`.
|
||
- `tines-bt/src/transport.rs` — `frame_sender` import site.
|
||
- `fuzz/` — cargo-fuzz target (outside the workspace).
|
||
|
||
---
|
||
|
||
### Task 1: Wire envelope — `WIRE_VERSION` byte
|
||
|
||
**Files:**
|
||
- Modify: `tines-core/src/consts.rs` (add `WIRE_VERSION`)
|
||
- Modify: `tines-core/src/wire.rs` (encode/decode prepend+check the byte; `frame_sender` becomes version-aware; update the golden Summary vector; keep the pre-change Summary bytes as a 0.x-rejection vector)
|
||
- Modify: `tines-core/src/exchange.rs` (`SUMMARY_OVERHEAD` 24 → 25)
|
||
- Modify: `tines-core/src/frag.rs` (`FRAG_OVERHEAD` 64 → 65)
|
||
- Modify: `tines-bt/tests/transport.rs`, `tines-bt/tests/adv.rs` (the `frame(from, payload)` helpers prepend `WIRE_VERSION`)
|
||
|
||
**Interfaces:**
|
||
- Produces: `pub const WIRE_VERSION: u8 = 4;` (decided: 4 is the first value in 4–0x7F, a single-byte postcard tag, ≥ the 0.x tags 0–3); `WireError` gains `VersionMismatch { version: u8 }`; `frame_sender(bytes: &[u8]) -> Option<NodeId>` now skips *two* leading bytes (version + variant tag) and returns `None` when `bytes[0] != WIRE_VERSION`.
|
||
|
||
- [ ] **Step 1: Write the failing tests**
|
||
|
||
In `tines-core/src/wire.rs` (mod tests): `encode_prepends_version_byte` (encode a Summary, assert `buf[0] == WIRE_VERSION` *and* `Frame::decode(&buf[..n])` round-trips — do **not** decode `&buf[1..]`, which is missing the version byte); `decode_wrong_version_is_mismatch` (a frame with `buf[0] = WIRE_VERSION + 1` returns `Err(WireError::VersionMismatch { version })`); `decode_empty_is_malformed` (`Frame::decode(&[])` returns `Err(WireError::Malformed)`, not a panic and not `VersionMismatch { version: 0 }`); `rejects_0x_frame` (the pre-change 0.x Summary golden bytes — captured from the current `summary_golden_vector` literal before editing it — are rejected as `VersionMismatch { version: 0 }`); `frame_sender_skips_version_and_tag` (bytes `[WIRE_VERSION, 0, from..]` yields `Some(from)`); `frame_sender_wrong_version_is_none` (`[WIRE_VERSION+1, 0, from..]` → `None`).
|
||
|
||
- [ ] **Step 2: Run, expect failure** — `cargo test -p tines-core wire` fails on the new names/behaviour.
|
||
|
||
- [ ] **Step 3: Implement** — add `WIRE_VERSION`; in `Frame::encode` write `out[0] = WIRE_VERSION` then postcard into `out[1..]` (return `1 + len`, `TooLarge` if `out.len() < 1 + len`); in `Frame::decode` check `bytes.is_empty()` → `Malformed`, then `bytes[0] != WIRE_VERSION` → `Err(VersionMismatch { version: bytes[0] })`, then postcard the remainder (the empty-remainder/canonical check is *not* yet enforced — that is Task 9); rewrite `frame_sender` to `ENVELOPE_OVERHEAD = 2`.
|
||
|
||
- [ ] **Step 4: Run, expect pass** — `cargo test -p tines-core` green; then update the tines-bt test helpers and re-run `cargo test -p tines-bt`.
|
||
|
||
- [ ] **Step 5: Commit** — `git commit -m "feat(core): WIRE_VERSION byte on every frame"`.
|
||
|
||
### Task 2: Drop reporting — `NodeEvent` and the drop counter table (§1, §3 events)
|
||
|
||
**Files:**
|
||
- Modify: `tines-core/src/exchange.rs` (rename `ConfidenceEvent` → `NodeEvent<A>`; add `DropReason`, the counter table, the overflow counter; report drops in `on_frame`; `drain_events` returns `NodeEvent`)
|
||
- Modify: `tines-core/src/node.rs` (`drain_events` signature)
|
||
- Modify: `tines-core/src/lib.rs` (export `NodeEvent`, `DropReason`; drop `ConfidenceEvent`)
|
||
- Modify: `tines-core/src/consts.rs` (`DROP_TABLE_CAP`)
|
||
- Modify: `tines/src/ipc.rs` (map `NodeEvent::Delivered` into `EventOut`; log `NodeEvent::Dropped` to stderr)
|
||
- Modify: `tines-ros2/src/node.rs` + `lib.rs` (match `NodeEvent`; fix the doc-comment links to `ConfidenceEvent`)
|
||
|
||
**Interfaces:**
|
||
- Consumes: `WireError::VersionMismatch` (Task 1).
|
||
- Produces: `#[non_exhaustive] pub enum NodeEvent<A> { Delivered { peer: NodeId, id: ObservationId }, Dropped { addr: Option<A>, reason: DropReason, count: u32 } }`; `#[non_exhaustive] pub enum DropReason { VersionMismatch { version: u8 }, Malformed }` (`IdentityChanged` arrives in Task 9); `pub const DROP_TABLE_CAP: usize = 32;` (local const); `Node::drain_events(&mut self) -> heapless::Vec<NodeEvent<T::Addr>, { EVENT_QUEUE_CAP + DROP_TABLE_CAP + 1 }>`. Decided: the return buffer is sized `EVENT_QUEUE_CAP + DROP_TABLE_CAP + 1` (64 Delivered + 32 counted Drops + 1 overflow slot) so drops always have room; the table itself is `DROP_TABLE_CAP`, so a drop counter that would exceed the buffer is impossible.
|
||
|
||
- [ ] **Step 1: Write the failing tests** — in `exchange.rs` (mod tests): `version_mismatch_is_counted` (drive one frame with a bad version byte from `addr`, drain, assert exactly one `Dropped { addr: Some(addr), reason: VersionMismatch{..}, count: 1 }`); `malformed_is_counted` (garbage bytes → `Dropped { reason: Malformed }`); `same_drop_increments_count` (two bad frames → one `Dropped` with `count: 2`, drained once); `drops_do_not_evict_delivered` (fill the event queue with `Delivered`, add a `Dropped`, assert all `Delivered` still present and the `Dropped` is reported); `overflow_counter_has_no_addr` (drive > `DROP_TABLE_CAP` distinct `(addr, reason)` pairs, assert the last is reported with `addr: None`); `expired_addressed_is_not_a_drop` (an addressed observation that arrives already past its `lifetime_ms` produces *no* `Dropped`).
|
||
|
||
- [ ] **Step 2: Run, expect failure** — `cargo test -p tines-core exchange` fails on `NodeEvent`/`DropReason` names.
|
||
|
||
- [ ] **Step 3: Implement** — rename to `NodeEvent<A>` (two variants above); add the counter table (fixed `heapless::Vec<(A, DropReason, u32), DROP_TABLE_CAP>` or equivalent) + one `Option<(DropReason, u32)>` overflow slot; in `on_frame`, replace the `Err(_) => return None` arm with a version-mismatch vs malformed branch that bumps the table; `drain_events` returns queued `Delivered` first, then one `Dropped` per nonzero counter (resetting it). Expired addressed observations are dropped by policy in `accept`/`reject`, *not* counted.
|
||
|
||
- [ ] **Step 4: Run, expect pass** — `cargo test -p tines-core`, then `cargo test -p tines`, then `docker build -f Dockerfile.ros2 -t tines-ros2-check .` (tines-ros2 consumers).
|
||
|
||
- [ ] **Step 5: Commit** — `git commit -m "feat(core): NodeEvent with counted drop reporting"`.
|
||
|
||
### Task 3: `Dest` — the addressing reshape (§2) + the log format bump (D5)
|
||
|
||
**Files:**
|
||
- Modify: `tines-core/src/observation.rs` (`Dest`; `Observation.dest` replaces `to`/`lifetime_ms`; `id()`; `aged()`; constructors `broadcast`/`addressed`/`with_observed_at`; the `age_ms` lower-bound and `observed_at` rustdoc; update the module's own tests)
|
||
- Modify: `tines-core/src/node.rs` (`observe`/`observe_to` build via `dest`; `belief` reads `dest`), `store.rs` (docs), `mem_store.rs`, `exchange.rs` (every `obs.to`/`obs.lifetime_ms` → `obs.dest`; `reject`), `wire.rs` (`Obs` variant + tests)
|
||
- Modify: `tines-store/src/log.rs` (bump `format_version`; refuse the old version on open)
|
||
- Modify: `tines-store/tests/persistence.rs` (move `old_format_is_refused` here from Task 10)
|
||
- Modify: `tines-sim/src/capped.rs` (re-verify it compiles), `tines-sim/src/sim.rs`/`world.rs` (any `to:`/`lifetime_ms:` construction), `tines/src/ipc.rs`, `tines-ros2/src/node.rs`
|
||
|
||
**Interfaces:**
|
||
- Consumes: nothing new.
|
||
- Produces: `#[non_exhaustive] pub struct Dest { pub to: NodeId, pub lifetime_ms: u64 }`; `Observation { pub observer, pub seq, pub age_ms, pub observed_at, pub topic, pub payload, pub dest: Option<Dest> }` (`sig` is added by Task 4 — this task's `Observation` has no `sig` field yet); `Observation::broadcast(observer, seq, topic, payload) -> Observation`, `Observation::addressed(observer, seq, topic, payload, to, lifetime_ms) -> Observation`, `Observation::with_observed_at(mut self, at: u64) -> Observation`. `tines-store`'s `format_version` is bumped here (D5: old logs are refused, not migrated).
|
||
|
||
- [ ] **Step 1: Write the failing tests** — in `observation.rs`: `dest_invariant_by_type` (an addressed observation *always* has `dest: Some`); `id_reads_dest_to` (`obs(7,0,None,Some([2;16]),Some(100)).id() == obs(7,0,None,Some([2;16]),Some(200)).id()` — lifetime still not part of identity — and the pinned `id_known_vector` value is unchanged); `constructors_set_age_zero` (`broadcast`/`addressed` produce `age_ms == 0` and the right `dest`); `with_observed_at_sets_field`. In `tines-store/tests/persistence.rs`: `old_format_is_refused` (a v1 log → `Err` naming the fix; moved from Task 10, since the encoding changes here).
|
||
|
||
- [ ] **Step 2: Run, expect failure** — `cargo test -p tines-core observation` fails on `dest`.
|
||
|
||
- [ ] **Step 3: Implement** — mechanical: `dest: Option<Dest>` replaces `to: Option<NodeId>` + `lifetime_ms: Option<u64>`; `id()` folds `self.dest.map(|d| d.to)` (marker bytes unchanged); `reject` becomes `obs.dest.is_some_and(|d| obs.age_ms > d.lifetime_ms)` (the malformed-pair branch and its tests are deleted); write the two rustdoc items now — `age_ms` is a lower bound (it counts time on running clocks only; power-off time is recovered on store open, crash time is bounded by the host flush cadence, in-flight time is negligible) and points to `observed_at`, and `observed_at` is the observer's wall-clock time in ms since the Unix epoch (UTC), `None` if the observer has no trusted clock, never changed by relays. Bump `tines-store`'s `format_version` and make `open` refuse any older version. Update every downstream read/construct; `cargo check --workspace` and fix all fallout.
|
||
|
||
- [ ] **Step 4: Run, expect pass** — `cargo test --workspace` green; `cargo build --workspace` green.
|
||
|
||
- [ ] **Step 5: Commit** — `git commit -m "feat(core): Dest type replaces to + lifetime_ms"`.
|
||
|
||
### Task 4: `Signature` + `SigAlgo` — the reserved slot (§2a)
|
||
|
||
**Files:**
|
||
- Modify: `tines-core/src/observation.rs` (`Signature`, `SigAlgo`, `Observation.sig`, `MAX_SIG`; the "signing and verification arrive in 1.1" rustdoc)
|
||
- Modify: `tines-core/src/consts.rs` (`MAX_SIG = 64` in the wire group)
|
||
- Modify: `tines-core/src/wire.rs` (`Obs`/`Frag` encode/decode of `sig`; the golden Obs vector)
|
||
- Modify: `tines-core/tests/vectors.rs` (golden Obs vector, both the Ed25519-shaped and unknown-algo cases)
|
||
|
||
**Interfaces:**
|
||
- Consumes: `Observation` (Task 3).
|
||
- Produces: `pub const MAX_SIG: usize = 64;`; `#[non_exhaustive] pub struct Signature { pub version: u8, pub algo: SigAlgo, pub bytes: heapless::Vec<u8, MAX_SIG> }`; `#[non_exhaustive] #[repr(u8)] pub enum SigAlgo { Ed25519 = 1, Unknown(u8) }` (decided: catch-all is `Unknown(u8)`); `algo` serializes as a raw `u8` on the wire, *not* a postcard enum tag. The rustdoc for `Signature`, `SigAlgo` and `Observation::sig` says "signing and verification arrive in 1.1" in those words.
|
||
|
||
- [ ] **Step 1: Write the failing tests** — in `wire.rs`/`vectors.rs`: `obs_roundtrip_with_ed25519_signature` (an `Observation` carrying `Signature { version: 1, algo: Ed25519, bytes: 64 bytes }` decodes equal); `obs_roundtrip_with_unknown_algo` (`algo: Unknown(7)`, short bytes, decodes and is *forwarded*, not rejected); `algo_zero_is_malformed` (`algo: Unknown(0)` → `Malformed`); `version_zero_is_malformed` (`version: 0` → `Malformed`); `raw_one_decodes_to_ed25519` (a wire `algo` byte `1` decodes to `SigAlgo::Ed25519`, never `Unknown(1)`); `ed25519_wrong_length_is_malformed` (`algo: Ed25519`, 63 bytes → `Malformed`); `sig_excluded_from_id` (two observations identical except `sig` share one `id()`); `reassembled_bad_signature_is_malformed` (an observation whose fragments reassemble to a bad-signature `Observation` is rejected as `Malformed`, not accepted); `fragmentation_fits_mtu_sweep` (Review Focus #6 — see below).
|
||
|
||
`fragmentation_fits_mtu_sweep`: for `mtu` in `MIN_MTU(=23)..=MAX_FRAME`, a maximum-size signed observation (full `MAX_PAYLOAD` + 64-byte signature) must either fragment into ≤ `MAX_FRAGMENTS` (64) frames each ≤ `mtu`, or — below the threshold (~85 bytes) — be given up cleanly (`SendError::TooLarge` / `ObserveError::TooLarge`), never emitted oversized.
|
||
|
||
- [ ] **Step 2: Run, expect failure** — tests fail on missing `Signature`/`SigAlgo`.
|
||
|
||
- [ ] **Step 3: Implement** — the types above; `sig: Option<Signature>` on `Observation` (no public setter in 1.0; core's decode sets it); wire: `algo` as raw `u8` (custom `Serialize`/`Deserialize` or a `#[serde(with = "…")]` module that writes the `repr(u8)` discriminant; `Unknown(n)` writes `n`); `version = 0` **and** `algo = 0` and a wrong length for `Ed25519` are `Malformed`; `id()` ignores `sig`; the same decode-level checks run on observations rebuilt from fragments.
|
||
|
||
- [ ] **Step 4: Run, expect pass** — `cargo test -p tines-core` green; pin the golden Obs vector.
|
||
|
||
- [ ] **Step 5: Commit** — `git commit -m "feat(core): reserved Signature slot on Observation"`.
|
||
|
||
### Task 5: Public API narrowing (§3)
|
||
|
||
**Files:**
|
||
- Modify: `tines-core/src/lib.rs` (the 1.0 export list; `mod wire;` private, `pub use wire::frame_sender;`)
|
||
- Modify: `tines-core/src/wire.rs`, `exchange.rs`, `frag.rs`, `bloom.rs` (mark the non-API items `pub(crate)`)
|
||
- Modify: `tines-core/src/observation.rs`, `store.rs`, `transport.rs`, `node.rs` (add `#[non_exhaustive]` where §3 requires)
|
||
- Modify: `tines-sim/src/world.rs` (fix the `[`Exchange`]` doc link), `tines-bt/src/transport.rs` (import `tines_core::frame_sender`)
|
||
- Modify: `tines-bt/tests/transport.rs`, `adv.rs` (import path)
|
||
|
||
**Interfaces:**
|
||
- Consumes: all prior types.
|
||
- Produces: the exact §3 public list. `Frame`, `WireError`, `Exchange`, `Peer`, `InFlight`, `Bloom` (and its `salt` field), `Reassembler`, `fragment`, `FRAG_OVERHEAD` are `pub(crate)`. `#[non_exhaustive]` on `ObserveError`, `StoreError`, `SendError`, `Event`, `NodeEvent`, `DropReason`, `SigAlgo`, `Observation`, `Dest`, `Signature`, `TrimPolicy`. `pub use wire::frame_sender;` at crate root (decided over keeping `wire` public).
|
||
|
||
- [ ] **Step 1: Write the failing tests** — a compile-time doc/export check: in `tines-core/tests/public_api.rs` (or the `conformance` feature), a `pub use`-only probe that `use`s each item on the §3 public list and asserts nothing; the *failing* signal is `cargo check -p tines-core` (private items are no longer importable — existing external callers break, which the plan fixes in this task).
|
||
|
||
- [ ] **Step 2: Run, expect failure** — `cargo check --workspace` fails on `tines-sim`/`tines-bt` importing now-private items.
|
||
|
||
- [ ] **Step 3: Implement** — apply `pub(crate)`/`pub` and `#[non_exhaustive]` exactly per §3; fix the two external doc-link/import sites; run `cargo doc --workspace --no-deps` to confirm no broken private-link warnings.
|
||
|
||
- [ ] **Step 4: Run, expect pass** — `cargo check --workspace` + `cargo test --workspace` green.
|
||
|
||
- [ ] **Step 5: Commit** — `git commit -m "refactor(core): narrow the 1.0 public API"`.
|
||
|
||
### Task 6: The `Store` trait, settled (§4)
|
||
|
||
**Files:**
|
||
- Modify: `tines-core/src/store.rs` (`is_empty` default, `flush` default, release docs, new `store_conformance` module behind a `conformance` feature)
|
||
- Modify: `tines-core/src/node.rs` (`flush(now)` replaces `store_mut`)
|
||
- Modify: `tines-core/Cargo.toml` (`conformance` feature)
|
||
- Modify: `tines-store/src/log.rs` (implement `flush`), `tines-sim/src/capped.rs` (delegate/implement `is_empty`/`flush`), `tines/src/ipc.rs` (call `node.flush(now)` in `poll_exchange`, around line 214)
|
||
|
||
**Interfaces:**
|
||
- Consumes: nothing new.
|
||
- Produces: `Store::is_empty(&self) -> bool` (default `self.len() == 0`; drop the `len_without_is_empty` allow); `Store::flush(&mut self, now: Tick) -> Result<(), StoreError>` (default `Ok(())`); `Node::flush(&mut self, now: Tick) -> Result<(), StoreError>`; `Node::store_mut` removed. A `conformance` feature gating `tines_core::store_conformance` — a `no_std`-clean module of functions that take a backend factory (not a `tines-core/tests/` file), so they can build observations with `sig` set from inside the crate; each function exercises one property (dedupe, age lineage, saturating ticks, release-surviving-`get`, `sig` round-trip, trim order, no tombstones, idempotent `flush`). `MemStore`, `LogStore` (a fresh file per case) and `CappedStore` call the module from their own crates.
|
||
|
||
- [ ] **Step 1: Write the failing tests** — the `store_conformance` module itself, with one function per property; then wire `MemStore`, `tines_store::LogStore` and `tines_sim::CappedStore` to it from their own crates.
|
||
|
||
- [ ] **Step 2: Run, expect failure** — `cargo test -p tines-core --features conformance` fails on `flush`/`is_empty`/`store_conformance`.
|
||
|
||
- [ ] **Step 3: Implement** — trait defaults; `Node::flush(now)` delegating to `Store::flush`; `LogStore::flush(&mut self, now)` writing a `Record::Clock` with the *caller's* `now` (fixing F18's stale-marker tick); delete `store_mut`; update `poll_exchange` in `tines/src/ipc.rs` to `node.flush(clock.now())`.
|
||
|
||
- [ ] **Step 4: Run, expect pass** — `cargo test --workspace` + `docker build -f Dockerfile.ros2 -t tines-ros2-check .`.
|
||
|
||
- [ ] **Step 5: Commit** — `git commit -m "feat(core): settle the Store trait (flush, is_empty, conformance suite)"`.
|
||
|
||
### Task 7: Constants — wire vs local split (§5)
|
||
|
||
**Files:**
|
||
- Modify: `tines-core/src/consts.rs` (re-group + document each const as wire or local)
|
||
- Modify: `AGENTS.md` (the constant list shows the split)
|
||
|
||
**Interfaces:**
|
||
- Produces: `consts.rs` split into two documented groups — wire (frozen for v1: `MAX_TOPIC`, `MAX_PAYLOAD`, `MAX_FRAME`, `SUMMARY_BYTES`, `ACK_IDS_CAP`, `MAX_SIG`, `BLOOM_K`, and the private `MAX_FRAGMENTS`) vs local (`MAX_PEERS`, `SEND_QUEUE_CAP`, `ACK_BUFFER_CAP`, `BCAST_SENT_CAP`, `EVENT_QUEUE_CAP`, `SUMMARY_INTERVAL_MS`, `REASSEMBLY_TIMEOUT_MS`, `DROP_TABLE_CAP`).
|
||
|
||
- [ ] **Step 1: No new tests** — this task only re-documents; the test is `cargo doc --workspace --no-deps` staying clean and the AGENTS.md list matching `consts.rs`.
|
||
|
||
- [ ] **Step 2: Implement** — move + document per §5's rule ("wire if changing it changes how a v1 peer decodes or interprets a frame").
|
||
|
||
- [ ] **Step 3: Verify** — `cargo test --workspace` green (no behaviour change).
|
||
|
||
- [ ] **Step 4: Commit** — `git commit -m "docs(core): split wire vs local constants"`.
|
||
|
||
### Task 8: Bounce-table overflow, pinned (§6)
|
||
|
||
**Files:**
|
||
- Modify: `tines-sim/src/sim.rs`, `tines-sim/tests/props.rs` (a property driving the table past `SEND_QUEUE_CAP`)
|
||
|
||
**Interfaces:**
|
||
- Consumes: the exchange's bounce table (unchanged).
|
||
- Produces: a proptest that floods > `SEND_QUEUE_CAP` distinct observations across `MAX_PEERS` peers and asserts the "always ≥ 1 holder" invariant and that the custody cursor still gives every candidate a turn.
|
||
|
||
- [ ] **Step 1: Write the failing test** — `tines-sim/tests/props.rs::bounce_overflow_keeps_a_holder` (drive custody past capacity; assert ≥1 holder for each id and no candidate starved).
|
||
|
||
- [ ] **Step 2: Run, expect pass or a real finding** — this pins behaviour; if it fails, the finding is the deliverable (do not "fix" by weakening the invariant).
|
||
|
||
- [ ] **Step 3: Implement any needed fix** — only if the property exposes a genuine ≥1-holder violation; §6 says the current drop-oldest design is already safe, so expect pass-as-is.
|
||
|
||
- [ ] **Step 4: Commit** — `git commit -m "test(sim): pin bounce-table overflow behaviour"`.
|
||
|
||
### Task 9: Core hardening — F6, F8, F9, F13 (§10)
|
||
|
||
**Files:**
|
||
- Modify: `tines-core/src/exchange.rs` (peer `id` pinning; idle eviction; summary replay limit; drop-oldest events; pending-ack purge; canonical decode)
|
||
- Modify: `tines-core/src/consts.rs` (`PEER_IDLE_MS = 30000`, local)
|
||
|
||
**Interfaces:**
|
||
- Consumes: `NodeEvent`, `DropReason` (Task 2).
|
||
- Produces: `DropReason::IdentityChanged`; `pub const PEER_IDLE_MS: u64 = 30_000;`.
|
||
|
||
- [ ] **Step 1: Write the failing tests** — `id_changes_are_dropped_as_identity_changed` (frame 1 sets `peer.id = A`, frame 2 claims `from = B` → dropped, counted `IdentityChanged`, and B receives no addressed mail); `idle_peer_is_evicted` (a peer with no inbound frame for > `PEER_IDLE_MS` is removed as if `Lost`); `replayed_summary_is_ignored` (two summaries from one peer within `SUMMARY_INTERVAL_MS` → the second is not acted on and produces no re-send); `events_drop_oldest` (overflow the event queue → the *oldest* `Delivered` is gone, newest retained); `lost_peer_purges_pending_acks` (pending acks to a removed peer are cleared); `trailing_bytes_is_malformed` (valid frame + one trailing byte → `Malformed`, Review Focus #5).
|
||
|
||
- [ ] **Step 2: Run, expect failure** — each test fails against current behaviour.
|
||
|
||
- [ ] **Step 3: Implement** — F6: first decoded frame pins `peer.id`, later `from` mismatch → `IdentityChanged` drop (reset on `Contact`/`Lost`); F8: track last-inbound tick, evict at `PEER_IDLE_MS`; F9: at most one summary per peer per `SUMMARY_INTERVAL_MS`, and skip already-sent ids on rebuild; F13: `events.push` drops the oldest (remove(0)), purge acks on `Lost`/evict, `take_from_bytes` + empty-remainder check.
|
||
|
||
- [ ] **Step 4: Run, expect pass** — `cargo test -p tines-core` + full `sh scripts/ci.sh`.
|
||
|
||
- [ ] **Step 5: Commit** — `git commit -m "fix(core): peer pinning, idle eviction, replay and queue hardening"`.
|
||
|
||
### Task 10: tines-store quarantine, wall-clock marker and downtime fold (§9, decision G as amended)
|
||
|
||
**Files:**
|
||
- Modify: `tines-store/src/log.rs` (quarantine of held observations; `set_wall_clock`; wall-clock marker with derived wall time; downtime fold; addressed expiry; broadcast floor; fold report)
|
||
- Modify: `tines-store/tests/persistence.rs` (quarantine and downtime tests)
|
||
|
||
**Interfaces:**
|
||
- Consumes: `Observation` with `observed_at` (Task 3), `Store::flush(now)` (Task 6), the `format_version` bump (Task 3).
|
||
- Produces:
|
||
- `LogStore::open(path: &Path, now: Tick)` keeps its signature. Every observation already in the log is quarantined.
|
||
- While quarantined, an observation is:
|
||
- left out of `for_each_broadcast_id`, `for_each_addressed_id` and `for_each_topic`;
|
||
- reported `true` by `is_released` if addressed;
|
||
- not trimmed on its age;
|
||
- still reported by `contains`, so dedupe works.
|
||
- `LogStore::set_wall_clock(&mut self, wall_now: u64, now: Tick) -> FoldReport`. It runs once (a second call is a no-op), folds the downtime, lifts the quarantine, and starts wall time for markers.
|
||
- `FoldReport { downtime_ms: Option<u64>, expired_addressed: usize, floored_broadcasts: usize }`. It's also printed once on stderr and readable afterwards, like `skipped_records`.
|
||
- `Record::Clock` gains `wall: Option<u64>`. After `set_wall_clock` it is `wall_at_set + (now − tick_at_set)`; before, `None`.
|
||
- A `LogStore` option `unknown_downtime_age_ms`.
|
||
- **Downtime known:** the last marker has a wall time, and `wall_now` ≥ it. Every quarantined observation's age gains the wall time elapsed since that marker, minus the ticks already counted since this open (saturating).
|
||
- **Downtime unknown:** no marker, a marker without wall time, or a marker ahead of `wall_now`. Quarantined addressed observations get an age past `lifetime_ms`, and quarantined broadcasts get `max(age, unknown_downtime_age_ms)`.
|
||
|
||
- [ ] **Step 1: Write the failing tests**
|
||
- `held_is_quarantined_until_wall_clock`: reopen a log holding a broadcast and an addressed observation. Before `set_wall_clock`, neither is visited by `for_each_broadcast_id`, `for_each_addressed_id` or `for_each_topic`; the addressed one is `is_released`; both are `contains`. After it, they're all visible.
|
||
- `quarantined_addressed_duplicate_not_acked`: an exchange-level test with `LogStore` (or in `tines-sim` through a quarantining wrapper). A peer offers a duplicate of a quarantined addressed observation, and no `Ack` comes back.
|
||
- `new_inserts_not_quarantined`: an observation inserted after `open` is visible at once and isn't changed by the fold.
|
||
- `downtime_known_advances_ages`: marker at wall 1000; reopen; `set_wall_clock(2000, …)` adds 1000 to every held age (saturating), less any ticks already counted since open.
|
||
- `downtime_unknown_expires_addressed` (no marker) and `marker_without_wall_is_unknown`: held addressed observations are never delivered, and the report counts them.
|
||
- `downtime_unknown_floors_broadcasts`: a held broadcast's age is raised to `unknown_downtime_age_ms`.
|
||
- `clock_went_backwards_is_unknown`: marker wall time ahead of `wall_now`.
|
||
- `marker_wall_time_is_derived`: `set_wall_clock(W, T)`, then `flush(T+5000)` writes a marker carrying W+5000. A flush before `set_wall_clock` writes `wall: None`.
|
||
- `set_wall_clock_runs_once`: a second call changes nothing.
|
||
- `fold_reported_once`: the `FoldReport` is printed once, and readable afterwards.
|
||
|
||
- [ ] **Step 2: Run, expect failure** — `cargo test -p tines-store`.
|
||
|
||
- [ ] **Step 3: Implement** — per §9 as amended. The quarantine is a per-entry flag set at load and cleared by the fold; it isn't persisted. Whether quarantined entries count towards `trim`'s `max_count` is the builder's call, with one rule: untrusted ages must never cause fresher data to be trimmed in their place.
|
||
|
||
- [ ] **Step 4: Run, expect pass** — `cargo test -p tines-store`, then `sh scripts/ci.sh`.
|
||
|
||
- [ ] **Step 5: Commit** — `git commit -m "feat(store): quarantine held observations until a wall-clock downtime fold"`.
|
||
|
||
### Task 10b: Hosts — trusted time, time from peers, `observed_at` (§9, decision G as amended)
|
||
|
||
**Files:**
|
||
- Modify: `tines/src/clock.rs` (`HostClock` gains trusted wall time), `tines/src/main.rs`, `tines/src/ipc.rs`
|
||
- Modify: `tines-ros2/src/node.rs`
|
||
|
||
**Interfaces:**
|
||
- Consumes: `LogStore::set_wall_clock` (Task 10).
|
||
- Produces, in both hosts:
|
||
- **Open at once.** `LogStore::open(path, now)` at startup, as today. The robot runs tines immediately.
|
||
- **Trusted time, in this order:**
|
||
1. A local source. A config flag (`trusted_clock: "system"`) says the system clock is trusted, meaning an RTC or a GPS/NTP-disciplined clock. This is config, not detection, and the docs say so.
|
||
2. Otherwise, a peer. After each `poll`, scan the observations stored *since this open* for the one carrying `observed_at` with the smallest `age_ms`. The estimate is `observed_at + age_ms + (now − tick it was stored)`, which is never later than the true time, because age is a lower bound. Quarantined data is invisible to the scan, so only this boot's observations count.
|
||
3. Otherwise, stay quarantined. The robot's own fail-safe is outside tines: log once that tines is waiting for trusted time, and keep running.
|
||
- The first trusted time calls `set_wall_clock(wall, now)` once and anchors `HostClock`'s wall time: wall at set + ticks since.
|
||
- From then on, every observation the host makes carries `observed_at = Some(HostClock wall ms)`. Before trusted time it stays `None` (R4).
|
||
|
||
- [ ] **Step 1: Write the failing tests** (in `tines`, over the existing in-memory transport)
|
||
- `observe_has_observed_at_after_trusted_time`: with `trusted_clock` set, an IPC `Observe` produces an observation with `observed_at = Some(_)`.
|
||
- `observe_has_no_observed_at_before_trusted_time`: without it, `None`.
|
||
- `time_from_peer`: instance A has trusted time and observes; instance B has none and a log holding a quarantined observation. After an exchange, B's `set_wall_clock` has run with an estimate within one poll interval of A's wall time (A's `observed_at + age_ms`), and B's held observation is visible.
|
||
- `smallest_age_wins`: given two received observations with `observed_at`, the estimate comes from the one with the smaller `age_ms`.
|
||
|
||
- [ ] **Step 2: Run, expect failure.**
|
||
|
||
- [ ] **Step 3: Implement** — `HostClock` wall anchor; the trusted-time state machine (waiting → set) in the service's poll loop and in `TinesNode::poll`; the `trusted_clock` config field; `observed_at` on every observe once trusted. `unknown_downtime_age_ms` is a host config field passed to `LogStore`. It must be below the trim `max_age_ms`, but the trim defaults are still empty (host hardening), so note in the host docs that this check waits for host hardening.
|
||
|
||
- [ ] **Step 4: Run, expect pass** — `cargo test -p tines`, `sh scripts/ci.sh`, then `docker build -f Dockerfile.ros2 -t tines-ros2-check .`.
|
||
|
||
- [ ] **Step 5: Commit** — `git commit -m "feat(hosts): trusted wall time from the system clock or a peer; set observed_at"`.
|
||
|
||
### Task 11: Evidence — pinned vectors + MSRV (§7)
|
||
|
||
**Files:**
|
||
- Create: `tines-core/tests/vectors.rs` (consolidate `id_known_vector` + the golden wire vectors)
|
||
- Modify: `tines-core/Cargo.toml` (`rust-version` on `tines-core` only), `scripts/ci.sh` (MSRV job building only `tines-core`)
|
||
|
||
**Interfaces:**
|
||
- Consumes: final wire encodings (Tasks 1–4).
|
||
- Produces: `tines-core/tests/vectors.rs` with the identity vector (unchanged value) + one golden encoding per `Frame` variant including the version byte + the Ed25519/unknown-algo Obs vectors + the 0.x-shaped frame rejected as version mismatch; `rust-version` set on `tines-core` only (the host crates' deps — bluer, tokio, rclrs — need newer Rust; do **not** claim tines-core's MSRV for them); a CI job building/testing `tines-core` alone on that MSRV.
|
||
|
||
- [ ] **Step 1: Write the failing tests** — move the existing vectors in (the `id_known_vector` and the golden encodings); add the missing golden encodings and the 0.x-rejection vector.
|
||
|
||
- [ ] **Step 2: Run, expect pass** — `cargo test -p tines-core vectors` green.
|
||
|
||
- [ ] **Step 3: Implement** — find MSRV with `cargo msrv` searching up to the pinned toolchain (1.99; a `repr(u8)` enum with a data-carrying variant like `SigAlgo` needs ≥ 1.66, and dependencies set their own floors — don't guess the range); set `rust-version`; add the CI job.
|
||
|
||
- [ ] **Step 4: Commit** — `git commit -m "test(core): pinned vectors + MSRV gate"`.
|
||
|
||
### Task 12: Evidence — fuzz harness + proptest twin (§7)
|
||
|
||
**Files:**
|
||
- Create: `fuzz/` (cargo-fuzz, outside the workspace, nightly), `fuzz/Cargo.toml`, `fuzz/fuzz_targets/node.rs`
|
||
- Modify: `tines-core/tests/` (the proptest stable twin of the same harness)
|
||
- Modify: `.github/workflows/` (scheduled fuzz job)
|
||
|
||
**Interfaces:**
|
||
- Consumes: `Node`, `MemStore`, a scripted `Transport`.
|
||
- Produces: a fuzz target whose input decodes into `Contact`/`Lost`/`Frame(bytes)`/`Tick(delta, may go backwards)`/`observe` steps; an adversarial-peer mode building well-formed frames (chosen summary bits, Obs/Frag ids/observers/ages/lifetimes, acks for sent/unsent ids, conflicting fragments, changing `from`) from many addresses beyond `MAX_PEERS`; post-step checks (no panic, ≤ `mtu()`, no addressed stored past lifetime, drop counts reconcile, signatures round-trip). A proptest twin with a bounded case count runs in the normal suite.
|
||
|
||
Adversarial mode must *also* check the §10 properties: the peer table recovers after a flood from more than `MAX_PEERS` addresses; output per received summary is bounded and repeating a summary does not re-send; a NodeId never changes within a contact; the event queue drops the oldest; an ack is sent only for an observation that was stored (F1).
|
||
|
||
- [ ] **Step 1: Write the harness** — the twin first (it runs in CI); then the fuzz target wrapping it.
|
||
|
||
- [ ] **Step 2: Run** — `cargo test -p tines-core` (twin green); a short `cargo fuzz run node -- -max_total_time=60` sanity run (full 24 CPU-hour budget is release-time, §"Done means").
|
||
|
||
- [ ] **Step 3: Commit** — `git commit -m "test(core): fuzz harness and proptest twin"` (committing the corpus too).
|
||
|
||
### Task 13: Docs, compatibility policy, and the 1.0.0 release (§8, "Done means")
|
||
|
||
**Files:**
|
||
- Modify: `README.md` (a "Stability" section with §8; switch the `TrimPolicy` example to `TrimPolicy::default()` + field assignment, since the struct is now `#[non_exhaustive]`), `CHANGELOG.md` (1.0.0 entry), `AGENTS.md` (spec list + trait extension rule + wire/local split), `tines-core/src/lib.rs` (`#![deny(missing_docs)]` + document the remaining items), `THREAT_MODEL.md` (mark F6/F8/F9/F13 fixed with SHAs; update F18 — Task 6 fixed its stale clock-marker tick; add a finding for peer-derived time: unauthenticated until 1.1, a forged time ahead only expires early, behind takes the unknown path, but the robot's own `observed_at` stamps are wrong until a local source corrects them), `Cargo.toml` + dependent crates (`tines-core` → 1.0.0, hosts depend on `"1"`)
|
||
|
||
**Interfaces:**
|
||
- Consumes: everything above.
|
||
- Produces: README §8; CHANGELOG 1.0.0 listing every 0.3 break; `#![deny(missing_docs)]` with zero missing items (recount — the spec's "26" predates the fleet fixes); `THREAT_MODEL.md` marks F6/F8/F9/F13 fixed (and updates F18); `tines-core` at 1.0.0 with `tines-store`/`tines-bt`/`tines-ros2`/`tines` depending on `"1"`.
|
||
|
||
- [ ] **Step 1: Write the failing test** — `cargo doc --workspace --no-deps` under `#![deny(missing_docs)]` (it fails until every public item is documented).
|
||
|
||
- [ ] **Step 2: Implement** — the docs; the release bump.
|
||
|
||
- [ ] **Step 3: Verify** — `sh scripts/ci.sh` green; `THREAT_MODEL.md` marks F6/F8/F9/F13 fixed with commit SHAs.
|
||
|
||
- [ ] **Step 4: Commit** — `git commit -m "release(core): tines-core 1.0.0"`.
|
||
|
||
**Publishing is Kyle's step** (it is outward-facing). The `cargo-semver-checks` gate is *not* added here: before 1.0.0 is published the only baseline is 0.2.1, and the check fails on every intended break. It is a follow-up commit *after* publishing, turning `cargo-semver-checks` against the published 1.0.0 on in `scripts/ci.sh`.
|