kyle/tines
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: sweep README, CHANGELOG, THREAT_MODEL and AGENTS after the 1.0 review fixes

Browse Source 
- CHANGELOG: tines-core 1.0.0 and the hosts' 0.4.0 are separate,
  unreleased sections. Every core API break is listed (the nine
  pub(crate) items, the private modules and the frame_sender path, the
  local constants per decision K, non_exhaustive types, store_mut). The
  new names (WallClockStore, Node::set_wall_clock, peer_time_estimate,
  flush, IdentityChanged) are called out. The host section covers
  tines::trust, the config changes, decision J, snapshots, and the
  review fixes.
- THREAT_MODEL: F6, F8, F9 and F13 are fixed with their commits. F18 is
  partly fixed: the store side and the flush on first trusted time are
  done, but timed and shutdown flushes are still open. F19 now records
  decision J and the plausibility bounds, and what remains. §9, §10 and
  §11 are updated to match.
- README: Dest and sig in the observation table; flush and is_empty in
  the store table; only the public wire constants are listed; a new
  "Trusted time" section with the config keys and the systemd ordering;
  the MSRV and fuzz CI jobs; frame_sender's new path.
- AGENTS: 1.0 is implemented but not yet published. The constants list
  shows the pub(crate) split, PEER_IDLE_MS, MIN_MTU's role and where
  MAX_FRAGMENTS lives. Notes the hidden conformance module.
- tines-bt README: frame_sender's path, and its own MAX_PEERS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
kyle
2026-10-22 08:44:09 -07:00
co-authored by Claude Opus 5.5
parent a0dcc0ad89
commit 0cb1ad5360
5 changed files with 648 additions and 209 deletions
Download Patch File Download Diff File Expand all files Collapse all files
AGENTS.md
+21 -8
View File
@@ -19,7 +19,7 @@ architecture in the **spec first**:
- **Addressed delivery (authoritative):** `docs/superpowers/specs/2026-10-18-tines-addressed-delivery.md`
- **Fleet deployment v1 (authoritative):** `docs/superpowers/specs/2026-10-18-tines-fleet-deployment-design.md` — supersedes the addressed-delivery spec's epidemic forwarding (custody chain with handoff receipts).
- **Fleet fixes (done):** `docs/superpowers/specs/2026-10-19-tines-fleet-fixes.md` — security-assessment findings F1–F5 and F17.
- **tines-core 1.0 (done — shipped as tines-core 1.0.0, 2026-10-21):** `docs/superpowers/specs/2026-10-18-tines-1-0-design.md` — wire version, public API, signature slot, core hardening (§10).
- **tines-core 1.0 (implemented on `feat/tines-core-1-0`; publish pending):** `docs/superpowers/specs/2026-10-18-tines-1-0-design.md` — wire version, public API, signature slot, core hardening (§10).
- **Host hardening (next, hosts):** `docs/superpowers/specs/2026-10-19-tines-host-hardening.md` — findings F10–F12.
- **Implementation plans:** `docs/superpowers/plans/`
@@ -111,15 +111,28 @@ Split into **wire** (frozen for wire v1 — changing one changes how a v1 peer
decodes or interprets a frame) and **local** (never crosses the wire; may
change in a minor release):
- **Wire:** `WIRE_VERSION=4`, `MAX_TOPIC=64`, `MAX_PAYLOAD=1024`,
- **Wire** (public): `WIRE_VERSION=4`, `MAX_TOPIC=64`, `MAX_PAYLOAD=1024`,
`MAX_FRAME=1500`, `SUMMARY_BYTES=512`, `ACK_IDS_CAP=90`, `MAX_SIG=64`,
`BLOOM_K=3`, plus the private `MAX_FRAGMENTS=64`.
- **Local:** `MAX_PEERS=8`, `SUMMARY_INTERVAL_MS=2000`,
`REASSEMBLY_TIMEOUT_MS=5000`, `SEND_QUEUE_CAP=256`, `MIN_MTU`,
`EVENT_QUEUE_CAP=64`, `DROP_TABLE_CAP=32`, `BCAST_SENT_CAP=256`,
`ACK_BUFFER_CAP=256`.
`BLOOM_K=3`. Also wire, but private: `MAX_FRAGMENTS=64`, which lives in
`tines-core/src/frag.rs`, not `consts.rs`.
- **Local** (`pub(crate)`, spec decision K: defaults that a later 1.x makes
configurable): `MAX_PEERS=8`, `SUMMARY_INTERVAL_MS=2000`,
`REASSEMBLY_TIMEOUT_MS=5000`, `SEND_QUEUE_CAP=256`, `EVENT_QUEUE_CAP=64`,
`DROP_TABLE_CAP=32`, `BCAST_SENT_CAP=256`, `ACK_BUFFER_CAP=256`,
`PEER_IDLE_MS=30_000`, and `MIN_MTU=23`. `MIN_MTU` is the lower bound of the
fragmentation MTU sweep in tests (BLE ATT's default), not the smallest
usable MTU; `Transport::mtu`'s rustdoc states those floors (85 bytes for
every valid observation).
Match the exact values above.
Match the exact values above. A crate outside core that needs a matching
number keeps its own constant (as `tines-bt` does for `MAX_PEERS` and
`MIN_MTU`).
The `conformance` feature also compiles a `#[doc(hidden)]`
`tines_core::conformance` module: byte-level frame encoders and decoders, and
mirrors of the local limits, for the fuzz harness and tests outside core. It
isn't supported public API and isn't covered by semver. Don't build on it
outside the workspace and `fuzz/`.
## Crate layout
CHANGELOG.md
+305 -90
View File
@@ -1,119 +1,314 @@
# Changelog
All notable changes to the tines crates. Versions follow
[Semantic Versioning](https://semver.org/); before 1.0, a minor bump may break
the API.
[Semantic Versioning](https://semver.org/). `tines-core` is 1.x and keeps the
compatibility promise in README's "Stability" section. The host crates
(`tines`, `tines-store`, `tines-bt`, `tines-ros2`, `tines-sim`) stay 0.x,
where a minor bump may break the API.
## 1.0.0 — 2026-10-21
## tines-core 1.0.0 — Unreleased
tines-core 1.0: the wire format is frozen (wire v1), the public API is
narrowed and documented, and the core hardening from the 1.0 spec is in.
This release ships the fleet-deployment milestone (handoff receipts) and the
remaining fleet fixes (F1–F5, F6, F8, F9, F13, F17; see `THREAT_MODEL.md`).
Implemented on `feat/tines-core-1-0`; not yet tagged or published. The wire
format is frozen as wire v1, the public API is narrowed and documented
(`#![deny(missing_docs)]`), and the core hardening from the 1.0 spec
(`docs/superpowers/specs/2026-10-18-tines-1-0-design.md` §10) is in. It also
carries the fleet-deployment milestone (handoff receipts) and the fleet fixes,
which were never released on their own. Findings F1, F6, F8, F9 and F13 are
fixed in core; see `THREAT_MODEL.md`.
### Breaking (from 0.3.0)
- **Wire v1** (tines-core). Every frame now leads with a `WIRE_VERSION` byte
(4). 0.x-shaped frames are rejected as `WireError::VersionMismatch`.
- **`Dest` replaces `to`/`lifetime_ms`** (tines-core). Addressed delivery now
takes `dest: Option<Dest>` where `Dest { to, lifetime_ms }`; the lifetime is
excluded from observation identity.
- **Signature slot** (tines-core). `Observation` carries a reserved
`sig: Option<Signature>` (algorithm + up to 64 bytes). 1.0 enforces the
wire shape but does not sign or verify (1.1).
- **Public API narrowing** (tines-core). Nine items moved `pub(crate)`,
eleven types are `#[non_exhaustive]` (build via constructors, not struct
literals). `Frame` and `WireError` are now *public*: the wire format is the
1.x compatibility contract.
- **`Store` trait settled** (tines-core). `flush(&mut self, now)` and
`is_empty` joined the trait; `Node::store_mut` is gone. A new opt-in
`WallClockStore` trait carries the trusted-time operations
(`set_wall_clock`, `peer_time_estimate`) that only `LogStore` implements.
- **`NodeEvent` + counted drops** (tines-core). Exchange outcomes surface as
`NodeEvent` (`Delivered`, `Dropped { reason, count }`) through
`Node::drain_events`; `DropReason` is `VersionMismatch`, `Malformed`, or
`IdentityChanged`.
- **`Node::poll` is the only pump** (tines-core). The separate custody
re-offer pass folded into `poll`.
- **Log format v2** (tines-store). `FORMAT_VERSION` bumped for the new
`Observation` encoding; opening an older log is a hard error that names
the fix (delete the log).
- **Quarantine + wall-clock fold** (tines-store). `LogStore::open(path, now)`
takes no wall time; observations held across a power-off are quarantined
until the host feeds trusted time via `set_wall_clock`.
- **Trusted wall time in the hosts** (tines). A `trusted_clock = "system"`
config flag or a peer-derived estimate (smallest `observed_at + age_ms`)
anchors the wall clock once; observations carry `observed_at: None` until
then.
- **MSRV** (tines-core). `rust-version = "1.82"`; CI builds and tests
tines-core alone on it.
- **Wire v1.**
- Every frame now starts with a `WIRE_VERSION` byte (4). A 0.x-shaped frame
is dropped and counted as `DropReason::VersionMismatch`.
- The new `Ack` frame is the last `Frame` variant (see Added).
- Frames must be canonical. Trailing bytes after a complete frame, or after
a reassembled observation, are rejected and counted as `Malformed`.
- **`Dest` replaces `to`/`lifetime_ms`.**
- An addressed observation carries `dest: Option<Dest>`, where
`Dest { to, lifetime_ms }`.
- The identity hash covers `dest.to`. It still excludes `lifetime_ms`,
`age_ms` and `observed_at`.
- **Signature slot.**
- `Observation` carries `sig: Option<Signature>` (version, `SigAlgo`, and up
to `MAX_SIG` = 64 bytes).
- 1.0 checks the wire shape and carries signatures verbatim. It neither
signs nor verifies; that's 1.1. `sig` is outside the identity hash.
- **Nine items are now `pub(crate)`:**
- `Exchange` (and with it `Exchange::on_send_error`; drive the exchange
through `Node`);
- `Peer`;
- `InFlight`;
- `Bloom` (and its public `salt` field);
- `Reassembler`;
- `fragment`;
- `Frame`;
- `WireError`;
- `FRAG_OVERHEAD`.
- **The `bloom`, `exchange`, `frag` and `wire` modules are private**, so paths
through them are gone:
- `tines_core::wire::frame_sender` is now `tines_core::frame_sender`;
- `DropReason` and `NodeEvent` are reached at the crate root.
- **Local constants are `pub(crate)`** (spec decision K). They are defaults
that a later 1.x release will make configurable:
- `MAX_PEERS`, `SUMMARY_INTERVAL_MS`, `REASSEMBLY_TIMEOUT_MS`,
`SEND_QUEUE_CAP`, `EVENT_QUEUE_CAP`, `BCAST_SENT_CAP` and `ACK_BUFFER_CAP`
are no longer in `tines_core::consts`;
- the new local constants, `PEER_IDLE_MS`, `DROP_TABLE_CAP` and `MIN_MTU`,
were never public;
- only the wire constants stay public (see Added).
- `Transport::mtu`'s rustdoc now states the smallest MTU core can use (85
bytes to carry every valid observation).
- **Eleven types are `#[non_exhaustive]`.**
- Enums (match with a wildcard): `ObserveError`, `StoreError`, `SendError`,
`Event`, `NodeEvent`, `DropReason`, `SigAlgo`.
- Structs (no struct literals outside core): `Observation`, `Dest`,
`Signature`, `TrimPolicy`.
- Build an `Observation` with `Observation::broadcast` or
`Observation::addressed`, and a `TrimPolicy` with
`TrimPolicy::default()` plus field assignment.
- **`Node::drain_events` returns `NodeEvent<T::Addr>`.** This replaces the fleet
milestone's unreleased `ConfidenceEvent`.
- **`Node::store_mut` is removed.** It was added by the unreleased fleet fixes
so the service could reach `LogStore::flush`. Use `Node::flush`, and
`Node::set_wall_clock` / `Node::peer_time_estimate` for a `WallClockStore`.
Read access through `Node::store` stays.
- **MSRV.** `rust-version = "1.82.0"`. CI builds tines-core on 1.82, with and
without default features and for `thumbv7em-none-eabihf`. It runs no tests
there, because a dev-dependency needs edition 2024. From now on an MSRV bump
is a minor release.
### Added
- **Handoff receipts** (tines-core). The new `Ack` frame (the last `Frame`
variant) carries up to `ACK_IDS_CAP` (90) ids. Acks are sent as the
exchange runs and are accepted only from a peer the id was sent to during
the current contact.
- **Handoff receipts.** The new `Ack` frame carries up to `ACK_IDS_CAP` (90)
ids. Acks are sent as the exchange runs, and are accepted only from a peer
the id was sent to during the current contact.
- An ack for an addressed observation moves custody: the sender releases
its copy (`Store::release`) and never offers it again.
- An ack for a broadcast is confidence only. It surfaces as a
`NodeEvent<A>::Delivered { peer, id }` from the new `Node::drain_events`.
- Broadcast confirmations and counted drops both come through
`Node::drain_events`; a confirmed broadcast arrives as
`NodeEvent::Delivered`, and a frame the peer dropped arrives as
`NodeEvent::Dropped { reason, count }` (§3).
- **Counted drop reporting** (tines-core). A frame the peer could not use is no
dropped silently: the exchange counts it under a `DropReason` (`VersionMismatch`
or `Malformed`) keyed by the peer's transport address and reports the count
through `Node::drain_events` as `NodeEvent::Dropped`. Counters are held in a
bounded table (`consts::DROP_TABLE_CAP`) with a single overflow slot for the
case where the table is full, so an adversarial peer can't spin forever — it
just reports `addr: None`. Expired addressed observations dropped by policy are
*not* counted.
- New constants: `ACK_IDS_CAP`, `EVENT_QUEUE_CAP`, `BCAST_SENT_CAP`,
`ACK_BUFFER_CAP`.
- `wire::frame_sender(bytes) -> Option<NodeId>`: a frame's claimed sender,
for transports that bind a connection to an identity.
- `Node::store_mut`, so a host can call backend methods such as
`LogStore::flush`.
- **New host crates:**
- `tines-store`: `LogStore`, a persistent append-only backend.
- `tines-bt`: a BLE transport over BlueZ.
- `tines`: a standalone service with Unix-socket JSON IPC.
- `tines-ros2`: a ROS2 node; rclrs is behind the `ros2` feature.
- An ack for a broadcast is confidence only. It surfaces as
`NodeEvent::Delivered { peer, id }` from `Node::drain_events`.
- **Counted drop reporting.** A frame this node can't use isn't dropped
silently.
- The exchange counts it under a `DropReason`: `VersionMismatch`,
`Malformed`, or `IdentityChanged` (F6). The count is keyed by the
sender's transport address.
- `Node::drain_events` reports each counter as
`NodeEvent::Dropped { addr, reason, count }`, after the queued
`Delivered` events.
- The counter table is bounded. When it's full, further drops go to one
overflow counter, reported with `addr: None` and the reason of the first
drop that overflowed since the last drain. Draining empties the table,
keys included.
- An expired addressed observation that's refused by policy is *not*
counted.
- **`Store::flush(now) -> Result<(), StoreError>`** and **`Store::is_empty`**,
both with safe defaults, plus **`Node::flush(now)`**. `flush` is the store's
durability point and its time anchor for ages after a restart. Flushing
twice is harmless, and an earlier `now` is clamped. Core never flushes on
its own.
- **`WallClockStore`** (spec decision H). An opt-in supertrait of `Store`, for
backends that hold data across power-off. It has an associated
`FoldReport`, `set_wall_clock(wall_now, now)` and `peer_time_estimate(now)`.
`LogStore` implements it; `MemStore` doesn't.
- **`Node::set_wall_clock`** and **`Node::peer_time_estimate`**. They exist
when `S: WallClockStore` and delegate to it, so no `&mut Store` escapes the
facade. Core forwards the wall time and never reads it.
- **`Dest`, `Signature` and `SigAlgo`**, and the constructors
`Observation::broadcast`, `Observation::addressed` and
`Observation::with_observed_at`. The two constructors panic on an
oversized topic or payload; `Node::observe` and `Node::observe_to` return
`ObserveError::TooLarge` instead.
- **`tines_core::frame_sender(bytes) -> Option<NodeId>`**: a frame's claimed
sender, for transports that bind a connection to an identity.
- **Public wire constants:** `WIRE_VERSION`, `MAX_TOPIC`, `MAX_PAYLOAD`,
`MAX_FRAME`, `SUMMARY_BYTES`, `ACK_IDS_CAP`, `MAX_SIG` and `BLOOM_K`.
- **`tines_core::store_conformance`**, behind the `conformance` feature and
`std`-free. A suite that lets any backend prove the §4 `Store` contract.
`MemStore` and `LogStore` both run it.
- The `conformance` feature also enables a `#[doc(hidden)]` `conformance`
module of frame encoders and local limits for the fuzz harness. It isn't
part of the supported API.
### Changed
- **Addressed delivery is a custody chain.** It used to be epidemic
forwarding. A message has one custodian at a time, which hands it on and
releases it when the next holder acks. A lost ack can leave two
custodians for a while; a message is never dropped by the protocol.
- Bounce records carry a direction, so only records of an actual send
validate an ack.
- **`LogStore` log format v1** (incompatible; there's no migration, so delete
old logs):
- The file starts with a `"TINESLOG"` header and a format version.
- Each record has a sync marker, a length and a CRC-32C.
- Ids are recomputed on load, not stored.
forwarding.
- A message has one custodian at a time, which hands it on and releases it
when the next holder acks.
- A lost ack can leave two custodians for a while; the protocol never drops
a message.
- Bounce records carry a direction, so only a record of an actual send
validates an ack.
- **Core never trusts a backend's filtering.** It re-checks `dest` and
`is_released` on every custody candidate. It re-checks `dest` when building
a summary, rebuilding a broadcast diff, and sending a broadcast.
- **Core hardening** (spec §10):
- **F6:** a peer's `NodeId` is pinned by its first frame in a contact. A
later frame with another `from` is dropped and counted as
`IdentityChanged`. The pin resets on `Lost`, and on a repeat `Contact`
for a known peer.
- **F8:** a peer with no inbound frame for `PEER_IDLE_MS` (30 s) is evicted
as if `Lost`. That includes a peer that only ever sent `Contact`. Idle
peers are swept before a peer is added and at the start of every
`next_frame`, and a peer that resumes later is a new implicit contact.
- **F9:** at most one received Summary per peer is acted on per
`SUMMARY_INTERVAL_MS`. A summary is accepted once at least half the
interval has passed, to allow for poll jitter (spec decision L). The
send-queue rebuild skips ids the peer's filter says it holds
(`Bloom::might_contain`).
- **F13:**
- the event queue drops the oldest event when full;
- a removed peer's pending acks are purged, and only that peer's;
- frames, and reassembled observations, must be canonical.
- **Receipts are scoped to the contact.** Broadcast-send records for an
address are dropped when its contact starts or ends. A broadcast whose send
fails has its record undone, so its ack isn't accepted.
### Fixed
- A receiver whose store returned `Err` (for example `Full`) still acked, so
the sender released custody of a message nobody kept. Now nothing is acked
unless it was stored, addressed or broadcast (F1).
- From the 1.0 review:
- Silent and contact-only peers were never evicted, so eight spoofed
addresses could still lock out every newcomer (S1, F8).
- The drop table never freed its keys. After 32 distinct
`(addr, reason)` pairs, every drop went to the overflow slot until reboot
(S7).
- Losing one peer purged every peer's pending acks (S9).
- Broadcast receipts weren't limited to this contact, and a failed send
could still be confirmed (S10).
- A store whose broadcast iterator yielded addressed ids could leak them
into a summary or forward them as broadcasts (S11). A test store that
overrides no filtering now pins the re-checks (B8).
- A repeat `Contact` didn't reset the F6 pin.
- A reassembled observation was decoded with trailing bytes allowed.
- The summary rate limit could ignore an on-time summary measured a
millisecond early.
- `DropReason::IdentityChanged` carried a variant-level `#[non_exhaustive]`,
so code outside the crate couldn't match it.
## Hosts 0.4.0 — Unreleased
`tines`, `tines-store`, `tines-bt`, `tines-ros2` and `tines-sim` move to 0.4.0
and depend on `tines-core = "1"`. Like tines-core 1.0.0, nothing here is
tagged or published yet.
The 0.3.0 release shipped only `tines-core` and `tines-sim`. The four other
host crates were added afterwards, during the fleet milestone, and carried the
workspace version 0.3.0 in the tree without being released. The breaks below
are measured against that in-tree 0.3.0. This release also carries the fleet
milestone's host work (F2–F5, F17).
### Breaking
- **Log format v2** (tines-store).
- The file starts with a `"TINESLOG"` header and a format version (now 2,
for tines-core 1.0's `Observation` encoding).
- Each record has a sync marker, a length and a CRC-32C, and ids are
recomputed on load.
- Opening a log of any other version is an error that names the fix
(delete the log). There's no migration.
- **`LogStore::flush` is the `Store::flush(now)` trait method** (tines-store).
The inherent `flush()` is gone. Call `Node::flush(now)`; the marker now
uses the caller's tick.
- **`LogStore::open(path, now)` quarantines what it loads** (tines-store).
Observations already in the log are held back until the host feeds
trusted wall time (see Added).
- **`tines::serve`** takes a fifth argument, `peer_time_window_ms`.
**`tines::handle`** takes `observed_at: Option<u64>` before `now`.
- **Service config** (tines):
- unknown keys are rejected (`deny_unknown_fields`);
- `trusted_clock` accepts only `"system"`; any other value is a config
error;
- the `first_seq` field is gone (seqs come from the store's boot counter).
### Added
- **Trusted wall time in the hosts** (spec §9, decisions G and J). A host
opens its store before it knows the time, then adopts a wall time once per
run.
- **Local source first.** The service's `trusted_clock = "system"`, or
tines-ros2's `trust_system_clock` / `trust_wall_clock`.
- **Otherwise a peer, by decision J.**
- The host collects `Node::peer_time_estimate` for `peer_time_window_ms`
(default 10 000) from the first plausible estimate.
- It then adopts the latest estimate.
- The store only offers estimates within its
`LogStoreBuilder::plausible_wall_ms` range, so a forged one can't hide
the honest ones.
- **Plausible range.** A wall time is adopted only if it's between
2026-01-01Z (`tines::trust::WALL_FLOOR_MS`) and about 2126
(`WALL_CEILING_MS`). That applies to the system clock and to peers.
- **Flush on trust.** Adopting the time flushes the store at once, so a
marker carrying the wall time reaches disk.
- Until trust, observations carry `observed_at: None`; after it, the
host's wall time.
- **`tines::trust`**: `WALL_FLOOR_MS`, `WALL_CEILING_MS`,
`PLAUSIBLE_WALL_MS`, `DEFAULT_PEER_TIME_WINDOW_MS`, `is_plausible_wall_ms`,
`system_wall_ms`, `adopt_wall_clock` and `PeerTimeWindow`. `HostClock`
gains `anchor_wall`, `wall_ms` and `is_trusted`.
- **Service config keys** (tines):
- `trusted_clock`;
- `peer_time_window_ms` (default 10000);
- `unknown_downtime_age_ms` (default: the LogStore default, 24 h).
- **`TinesNode` trust methods** (tines-ros2): `trust_system_clock`,
`trust_wall_clock`, `set_peer_time_window_ms` and `has_trusted_time`. They
give a ROS2-only fleet a local trusted-time source.
- **LogStore quarantine and fold** (tines-store).
- `LogStore` implements `WallClockStore`.
- `set_wall_clock` folds the downtime into held ages, once, and returns a
`FoldReport`, which `fold_report` reads back later.
- **Downtime known:** held ages gain the wall-clock downtime.
- **Downtime unknown:** held addressed observations expire, and held
broadcasts are floored at `unknown_downtime_age_ms`.
- Quarantined entries are exempt from every trim rule (spec decision L).
- `peer_time_estimate` returns the latest estimate among observations
stored since this open.
- **`LogStoreBuilder`** (tines-store):
- `set_unknown_downtime_age_ms` (default 24 h);
- `plausible_wall_ms` (default unbounded; both hosts pass
`tines::trust::PLAUSIBLE_WALL_MS`).
- **Snapshots** (tines-store, spec decision I).
- `open`, after its tick fold, rewrites the log as a snapshot. So does
`set_wall_clock`, after its fold. A snapshot holds every live observation
at its current age, a `Release` for every released id still held, and a
closing clock marker.
- The snapshot goes to `<log>.snapshot-tmp` in the same directory. That
file is fsynced, locked, renamed over the log, and then the directory is
fsynced. A leftover temp file is removed on open.
- The fold's results and age lineage then survive any number of reboots.
Each snapshot also compacts the log.
- **New host crates** (first released in this line, see above):
- `tines-store`: `LogStore`, a persistent backend;
- `tines-bt`: a BLE transport over BlueZ;
- `tines`: a standalone service with Unix-socket JSON IPC;
- `tines-ros2`: a ROS2 node, with rclrs behind the `ros2` feature.
### Changed
- `LogStore` persists a boot counter, and `first_seq()` returns
`boot_count << 32`. Both hosts seed `Node::new` from it, and the
`first_seq` config field is gone.
- `LogStore::release` writes a `Release` record immediately. `flush` writes
a clock marker, so ages carry across a restart.
`boot_count << 32`. Both hosts seed `Node::new` from it.
- `LogStore::release` writes a `Release` record immediately.
- `LogStore::open` takes an exclusive lock and creates the file `0600`.
- `LogStore::flush` and `set_wall_clock` clamp an earlier `now` to the store's
time anchor, so `stored_at` and clock markers never move backwards.
- A log whose first header was torn while it was being created (shorter than
the header, starting with the magic) opens as a new log. Any other short
file is still refused.
- tines-bt rewrite:
- L2CAP SeqPacket channels, one frame per packet.
- Every node advertises and scans; the smaller `NodeId` dials and the
larger accepts.
- Connections are bound to the advertised `NodeId`, and frames whose
claimed sender differs are dropped.
claimed sender differs (`tines_core::frame_sender`) are dropped.
- Connections emit `Contact` and `Lost`.
- Live connections are capped by tines-bt's own `MAX_PEERS` (8).
- tines-sim's property tests run 64 cases again, and the bounce-overflow
property checks that the table really overflowed.
### Fixed
- A receiver whose store returned `Err` (for example `Full`) still acked, so
the sender released custody of a message nobody kept. Now nothing is
acked unless it was stored, addressed or broadcast (F1).
- The standalone service never polled its `Node`, so it never exchanged
anything. It now runs the exchange on a poll loop (F2).
- tines-bt had no listener, lost frame boundaries on a stream socket, and
@@ -125,13 +320,33 @@ remaining fleet fixes (F1–F5, F6, F8, F9, F13, F17; see `THREAT_MODEL.md`).
fsync of the file and its directory (F5).
- The service's tick was wall-clock *seconds*. Both hosts now use one
monotonic millisecond clock (F17).
- From the 1.0 review:
- `LogStore` didn't implement `Store::flush`, so `Node::flush` wrote no
marker and fsynced nothing (B1).
- Fold results and age lineage lived only in memory. An expired message
came back after the next reboot, and ages reset on a third boot (B2).
Snapshots fix both.
- A forged `observed_at = u64::MAX` overflowed: a panic in debug, a time
near 1970 in release (B3). The arithmetic saturates, and the hosts
bound every wall time they adopt.
- A flush between open and the fold made the fold count that time twice
(S2).
- `flush` didn't clamp an earlier `now` (S3).
- tines-ros2 had no local trusted-time source (S4).
- A quiet reboot lost everything, because nothing flushed after trust
was set (S5).
- The service trusted a 1970 system clock and ignored config typos (S6).
### Known issues
- The service talks to exactly one configured `peer`. The multi-peer
allowlist the fleet-fixes spec describes isn't wired up (F3).
- Neither host flushes on a timer, and tines-ros2 never flushes or trims
(F18).
- Neither host flushes on a timer or at shutdown (F18).
- The service trims and flushes only after 64 poll rounds that each stored
something new.
- tines-ros2 flushes only when it adopts trusted time, and never trims.
- Peer time is unauthenticated until 1.1. An adopted time isn't corrected
while the host runs (F19).
## 0.3.0 — 2026-10-18
README.md
+155 -49
View File
@@ -29,6 +29,7 @@ hear each other.
- [Running on a robot](#running-on-a-robot)
- [Crate layout](#crate-layout)
- [Building and CI](#building-and-ci)
- [Stability](#stability)
- [Security](#security)
- [Where to read more](#where-to-read-more)
@@ -88,14 +89,18 @@ An `Observation` is the unit of storage and exchange:
| `seq` | the observer's sequence number | ✓ |
| `topic` | what it is about, ≤ 64 bytes (`b"door/3"`) | ✓ |
| `payload` | opaque bytes, ≤ 1024 | ✓ |
| `to` | `None` for broadcast, or a recipient `NodeId` | ✓ |
| `age_ms` | how old the observation is | |
| `observed_at` | optional absolute timestamp, if the robot has a clock | |
| `lifetime_ms` | validity window, present exactly when `to` is | |
| `dest` | `None` for broadcast, or `Some(Dest { to, lifetime_ms })` for an addressed observation | `to` only |
| `age_ms` | how old the observation is: a lower bound that every holder advances | |
| `observed_at` | the observer's wall-clock time (ms since the Unix epoch), or `None` if it had no trusted clock | |
| `sig` | a reserved `Signature` slot (algorithm + up to 64 bytes); carried verbatim, signed and verified from 1.1 | |
The **content id** is the first 16 bytes of Blake2s-256 over the ✓ fields.
Because age and timestamps are excluded, the same fact arriving by two
different paths, at two different ages, dedupes to one entry.
The **content id** is the first 16 bytes of Blake2s-256 over the ✓ fields
(`dest.to`, but not `dest.lifetime_ms`). Because age, timestamps, lifetimes
and signatures are excluded, the same fact arriving by two different paths, at
two different ages, dedupes to one entry.
`Observation` is `#[non_exhaustive]`: build one with `Observation::broadcast`
or `Observation::addressed`, or let `Node::observe` / `Node::observe_to` do it.
### Time is relative
@@ -154,9 +159,11 @@ sequenceDiagram
### Wire frames
Frames are [postcard](https://docs.rs/postcard)-encoded and each carries the
Every frame starts with a `WIRE_VERSION` byte (4 for wire v1), followed by
the [postcard](https://docs.rs/postcard)-encoded frame. Each frame carries the
sender's `NodeId`, so the core never needs to map transport addresses to
robots.
robots. A frame with another version byte, trailing bytes, or anything else it
can't decode is dropped and counted (see [Receipts](#receipts)).
```mermaid
classDiagram
@@ -183,13 +190,14 @@ classDiagram
```
A transport that binds a connection to a peer's identity can check a frame's
claimed sender with `tines_core::wire::frame_sender`, without decoding the
claimed sender with `tines_core::frame_sender`, without decoding the
whole frame.
### Fragmentation
An observation that doesn't fit the transport's MTU is split into up to 64
`Frag` frames (64 bytes of each frame are reserved for the header). Each peer
`Frag` frames (65 bytes of each frame are reserved for the version byte and
the header). Each peer
has one stream in flight at a time, and it is finished before anything else
starts for that peer. All fragments of a stream are cut from one encoding, so
the receiver can always reassemble them. A partial observation is never
@@ -295,12 +303,15 @@ it has already released, since that would hand custody back to a robot that
no longer carries the message.
Broadcast confirmations surface as `NodeEvent<A>::Delivered { peer, id }` from
`Node::drain_events`. Frames the peer could not use are counted under a
`DropReason` (`VersionMismatch` or `Malformed`) and surface as
`NodeEvent::Dropped { reason, count }` (see the `drop` reporting in the `tines`
service and the ROS2 node). The event queue is bounded (64) plus a small
counted-drop table, so drops are always reported even under a chatty peer —
events are telemetry, and losing one never loses data.
`Node::drain_events`. Frames this node received but couldn't use are counted
under a `DropReason` (`VersionMismatch`, `Malformed`, or `IdentityChanged` for
a peer whose `from` changed mid-contact) and surface as
`NodeEvent::Dropped { addr, reason, count }`, keyed by the sender's transport
address (see the `drop` reporting in the `tines` service and the ROS2 node).
The event queue is bounded (64, dropping the oldest), and drops go to a small
counted table with one overflow counter (`addr: None`), so drops are always
reported even under a chatty peer. Events are telemetry: losing one never
loses data.
## Usage
@@ -347,6 +358,7 @@ for event in node.drain_events() {
tines_core::NodeEvent::Dropped { addr, reason, count } => {
println!("dropped {count} frame(s) from {addr:?}: {reason:?}");
}
_ => {} // NodeEvent is #[non_exhaustive]
}
}
@@ -407,10 +419,11 @@ pub trait Transport {
- **Report back-pressure honestly.** Return `SendError::Busy` when the radio
can't take a frame right now; the exchange will offer the same frame again
on the next poll. `TooLarge` tells the exchange to give that frame up.
- **Driving `Exchange` without `Node`?** `Exchange::next_frame` assumes the
frame it returns gets sent. If `send` fails, call
`Exchange::on_send_error(&err)` before the next `next_frame`, or the frame
is treated as delivered. `Node::poll` does this for you.
- **Mind the MTU.** `mtu()` must be at most 1500. Core can carry every valid
observation from 85 bytes up; below that it never sends an oversized frame,
but it sends less (see `Transport::mtu`'s docs for each floor).
- **Binding identities?** A transport that binds a connection to a `NodeId`
can check each frame's claimed sender with `tines_core::frame_sender`.
## Writing a store
@@ -424,12 +437,14 @@ flash or SQLite backend implements the `Store` trait:
| `get` | the observation with `age_ms` advanced by `now − stored_at`, saturating if `now` is earlier |
| `len` | number of stored observations |
| `for_each_id` | every stored id, any order |
| `for_each_broadcast_id` | ids with `to == None`; these feed the summary |
| `for_each_addressed_id` | ids with `to == Some(_)`; scanned for custody on every poll |
| `for_each_broadcast_id` | ids with `dest == None`; these feed the summary |
| `for_each_addressed_id` | ids with `dest == Some(_)`; scanned for custody on every poll |
| `release` | mark an id as handed off, so it's never offered again; a no-op if the id isn't stored |
| `is_released` | whether an id has been released; a re-inserted observation starts un-released |
| `for_each_topic` | age-adjusted observations whose topic starts with a prefix |
| `trim` | apply a `TrimPolicy` (per-topic, then age, then count); return how many were removed |
| `is_empty` | `len() == 0` (default provided) |
| `flush` | the durability point and the time anchor for ages after a restart; harmless twice, an earlier `now` is clamped; returns `Err(StoreError::Backend)` on failure and leaves the store usable (default: no-op `Ok(())`) |
The store must remember when each observation was inserted (`stored_at`, in
the caller's `Tick`s) so it can report current ages. `insert` may return
@@ -439,7 +454,19 @@ custody.
`for_each_addressed_id` has a default implementation that visits every id,
which is correct but makes the exchange read every broadcast observation on
each poll just to find the few addressed ones. Override it if your backend can
filter by `to` cheaply, e.g. with an index or a separate table.
filter by `dest` cheaply, e.g. with an index or a separate table. The
filtering is only an optimisation: core re-checks `dest` and `is_released` on
every id a backend yields.
A store that keeps `sig` must return it verbatim. Core never calls `flush`;
the application decides how often, because on flash every flush costs wear.
A backend that holds data across power-off can also implement the opt-in
`WallClockStore` trait, which lets the host pass in trusted wall time through
`Node::set_wall_clock` (see [Trusted time](#trusted-time)).
With the `conformance` feature, `tines_core::store_conformance` checks a
backend against the trait contract (dedupe, age lineage, saturating ticks,
release, `sig` round-trip, trim order, no tombstones, `flush`).
## The simulator
@@ -508,13 +535,21 @@ flowchart LR
A robot runs either the service or the ROS2 node, not both: each owns its
`Node` and its log.
- **`tines-store`**: `LogStore`, an append-only log file.
- **`tines-store`**: `LogStore`, an append-only log file (format v2; an
older log is refused with an error that says to delete it).
- Each record is checksummed (CRC-32C). Recovery skips a corrupt record and
truncates only a torn tail.
- Releases are written as they happen.
- A boot counter provides `first_seq()`.
- `flush()` writes a clock marker, so ages carry across a restart, and
fsyncs the log.
- `Store::flush(now)`, reached through `Node::flush`, writes a clock marker
at `now` and fsyncs the log, so ages carry across a restart. An earlier
`now` is clamped.
- What the log held at open is quarantined until the host has trusted
wall time (see [Trusted time](#trusted-time)).
- The log is rewritten as a snapshot at open and after the wall-clock
fold, through a `<log>.snapshot-tmp` file that's fsynced and renamed
over it. That keeps the fold's results and ages across any number of
reboots, and compacts the log.
- The file is created `0600` and locked, so only one process can open it.
- **`tines-bt`**: BLE over BlueZ, using L2CAP SeqPacket channels (one
frame per packet).
@@ -527,8 +562,9 @@ A robot runs either the service or the ROS2 node, not both: each owns its
- **`tines`**: the standalone service, for robots without ROS2.
- It runs the exchange on a poll loop and answers `Observe`, `Belief`,
`Events` and `Status` requests on a Unix socket.
- It's configured by a JSON file: `node_id`, `peer`, `log_path`,
`socket_path` and `poll_interval_ms`.
- It's configured by a JSON file (below). Library users call
`tines::serve(node, socket_path, clock, poll_interval_ms,
peer_time_window_ms)`.
- **`tines-ros2`**: a ROS2 node.
- It publishes each tines topic under `/tines/` and confirmed deliveries on
`/tines/receipts`.
@@ -536,12 +572,73 @@ A robot runs either the service or the ROS2 node, not both: each owns its
- rclrs is behind the `ros2` feature, so the crate builds without a ROS2
toolchain.
The service's config file is a JSON object. An unknown key is an error, so a
typo can't silently fall back to a default.
| Key | Default | Meaning |
|---------------------------|--------------|---------|
| `node_id` | required | this robot's `NodeId`, 32 hex chars |
| `peer` | required | the one peer's `NodeId`, 32 hex chars |
| `log_path` | required | the `LogStore` log file |
| `socket_path` | required | the IPC Unix socket |
| `poll_interval_ms` | 20 | exchange poll cadence |
| `trusted_clock` | unset | `"system"` to trust the system clock; the only accepted value |
| `peer_time_window_ms` | 10000 | how long to collect peer time estimates before adopting the latest |
| `unknown_downtime_age_ms` | 24 h | age floor for held broadcasts when the downtime is unknown |
The hosts use a monotonic millisecond clock that starts at 0 when the
process does. Neither host sets `observed_at` yet.
process does.
### Trusted time
Ages are relative, but a robot that was powered off can't tell from ticks how
long it was off. So the hosts open the store before they know the wall time,
quarantine what it held (not offered, delivered or acked, but still deduped),
and adopt a trusted wall time **once per run**:
1. **A local source.** The service adopts the system clock at startup when
`trusted_clock = "system"`. tines-ros2's `TinesNode` has
`trust_system_clock(now)`, and `trust_wall_clock(wall_ms, now)` for an RTC
or GPS reading; call one right after construction.
2. **Otherwise, peers.** From the first plausible estimate a peer's
observations give (`observed_at + age_ms`), the host collects estimates
for `peer_time_window_ms` (10 s by default; `set_peer_time_window_ms` in
tines-ros2), then adopts the latest.
3. **Otherwise** the quarantine stays. The robot's own fail-safe is outside
tines.
Only a wall time between 2026-01-01 and about 2126 is adopted
(`tines::trust::WALL_FLOOR_MS` and `WALL_CEILING_MS`), from a peer or from
the system clock. A board that booted before NTP and reads 1970 is logged and
ignored, and the host falls back to peers. Adopting a time runs the store's
fold and flushes at once. If the log's last clock marker carries a wall time
no later than the adopted one, the fold adds the downtime to held ages.
Otherwise the downtime is unknown: the fold expires held addressed messages
and floors held broadcasts at `unknown_downtime_age_ms`. From then on
the host stamps its observations' `observed_at`; before, they carry `None`.
`TinesNode::has_trusted_time` reports whether a time was adopted.
The system clock is read once, at startup. Under systemd, order the service
after clock sync:
```ini
[Unit]
Wants=time-sync.target
After=time-sync.target
```
`time-sync.target` only waits for a synced clock when the sync service
supports it (`systemd-timesyncd` with `systemd-time-wait-sync.service`
enabled, or chrony's equivalent).
Peer time is unauthenticated until 1.1, and an adopted time isn't corrected
while the host runs; see F19 in [`THREAT_MODEL.md`](THREAT_MODEL.md).
**Current limits** (tracked in [`THREAT_MODEL.md`](THREAT_MODEL.md)):
- The service talks to one configured `peer`.
- The hosts don't yet flush on a timer.
- The hosts don't flush on a timer or at shutdown. The service flushes after
64 productive poll rounds, and both hosts flush when they adopt trusted
time.
- The host-hardening work (trim defaults, bounded channels, socket
permissions, ROS2 isolation) hasn't landed.
@@ -564,23 +661,24 @@ tines/
tines-ros2/ std: the ROS2 node (rclrs behind the `ros2` feature).
```
Fixed limits, all compile-time constants in `tines-core/src/consts.rs`:
The public constants, in `tines_core::consts`, are the wire constants. They
are frozen for wire v1: changing one is a `WIRE_VERSION` bump.
| Constant | Value | Meaning |
|-------------------------|--------|------------------------------------------|
| `MAX_TOPIC` | 64 | topic bytes |
| `MAX_PAYLOAD` | 1024 | payload bytes |
| `MAX_PEERS` | 8 | peers tracked at once |
| `SUMMARY_BYTES` | 512 | full Bloom summary size |
| `BLOOM_K` | 3 | hash functions per id |
| `SUMMARY_INTERVAL_MS` | 2000 | re-summary period per peer |
| `REASSEMBLY_TIMEOUT_MS` | 5000 | partial observation discard time |
| `SEND_QUEUE_CAP` | 256 | ids queued per peer |
| `MAX_FRAME` | 1500 | largest frame (and MTU) supported |
| `ACK_IDS_CAP` | 90 | ids in one `Ack` frame |
| `EVENT_QUEUE_CAP` | 64 | confirmed deliveries awaiting `drain_events` |
| `BCAST_SENT_CAP` | 256 | broadcast sends remembered per peer, to validate acks |
| `ACK_BUFFER_CAP` | 256 | received ids waiting to be acked, per kind |
| Constant | Value | Meaning |
|-----------------|-------|------------------------------------------|
| `WIRE_VERSION` | 4 | the first byte of every frame |
| `MAX_TOPIC` | 64 | topic bytes |
| `MAX_PAYLOAD` | 1024 | payload bytes |
| `MAX_FRAME` | 1500 | largest frame (and MTU) supported |
| `SUMMARY_BYTES` | 512 | full Bloom summary size |
| `ACK_IDS_CAP` | 90 | ids in one `Ack` frame |
| `MAX_SIG` | 64 | signature bytes |
| `BLOOM_K` | 3 | hash functions per id |
Local capacities and timings (peers tracked at once, the re-summary interval,
queue sizes, the idle-eviction window, and so on) are internal defaults, not
public API. They can change in a minor release, and a later 1.x release will
make them configurable.
## Building and CI
@@ -593,7 +691,15 @@ sh scripts/ci.sh
```
GitHub Actions runs the same script on every push to `master` and on every
pull request.
pull request. Two more jobs run there:
- **MSRV:** builds `tines-core` on Rust 1.82 (its `rust-version`), with and
without default features and for `thumbv7em-none-eabihf`. It builds only,
because a test dependency needs a newer compiler.
- **Fuzz:** a scheduled nightly job runs `cargo-fuzz` on the `fuzz/` crate's
`node` target, in both its random-bytes and adversarial-peer modes, and
keeps any crash reproducer. A bounded proptest twin of the same harness runs
in the normal test suite.
## Stability
@@ -637,7 +743,7 @@ The specs are authoritative; the plans argue from them.
- **Fleet deployment:** `docs/superpowers/specs/2026-10-18-tines-fleet-deployment-design.md`
- **Fleet fixes:** `docs/superpowers/specs/2026-10-19-tines-fleet-fixes.md`
- **Host hardening (next for the hosts):** `docs/superpowers/specs/2026-10-19-tines-host-hardening.md`
- **tines-core 1.0 (next for the core):** `docs/superpowers/specs/2026-10-18-tines-1-0-design.md`
- **tines-core 1.0 (implemented on `feat/tines-core-1-0`; publish pending):** `docs/superpowers/specs/2026-10-18-tines-1-0-design.md`
- **Threat model:** [`THREAT_MODEL.md`](THREAT_MODEL.md)
- **Plans:** `docs/superpowers/plans/`
- **Changes:** [`CHANGELOG.md`](CHANGELOG.md)
THREAT_MODEL.md
+162 -57
View File
@@ -3,10 +3,13 @@
- **Assessed:** `master` at `3b1a644` (tines-core / tines-store / tines-bt /
tines-ros2 / tines 0.3.0, fleet-deployment milestone mostly landed)
- **Date:** 2026-10-19
- **Updated:** 2026-10-20 at `89041ea`, after the fleet fixes landed. The
findings keep their original text; each fixed one ends with a **Fixed**
or **Partly fixed** line, and §6 adds F18, found in the re-check.
- **Status:** living document.It's not a spec: update it when the code or the
- **Updated:** 2026-10-21 at `411daa0` (`feat/tines-core-1-0`), after the
tines-core 1.0 review fixes landed. Earlier update: 2026-10-20 at
`89041ea`, after the fleet fixes. The findings keep their original text;
each fixed one ends with a **Fixed** or **Partly fixed** line. §6 adds
F18, found in the fleet re-check, and F19, found with the 1.0 trusted-time
work.
- **Status:** living document. It's not a spec: update it when the code or the
trust assumptions change.
- **Related:** the 1.0 spec (`docs/superpowers/specs/2026-10-18-tines-1-0-design.md`),
especially §2a (signature slot) and §2b (age and the 1.1 threat model).
@@ -57,22 +60,24 @@ threat questions (§8) before designing signing.
| F5 | Fixed | `4718b6f`, `24b5a78` |
| F17 | Fixed | `6e2dc6f` |
**1.0 fixes, 2026-10-21** (through `941a7ce`):
**1.0 fixes, 2026-10-21** (through `411daa0`, on `feat/tines-core-1-0`; not
yet merged or published):
| # | Status | Commit |
|---|---|---|
| F6 | Fixed | `fb71810` |
| F8 | Fixed | `fb71810` |
| F9 | Fixed | `fb71810`, `a180ec2` |
| F13 | Fixed | `fb71810` |
| F18 | **Partly fixed**: the stale clock-marker tick is fixed (`Store::flush(now)` takes the caller's `now`); hosts still don't flush on a schedule | `c42f5de` |
| F19 | **Open** (new, 2026-10-21): peer-derived trusted time is unauthenticated | — |
| F6 | Fixed | `fb71810`, `506d880` (the pin resets on a repeat `Contact`) |
| F8 | Fixed | `fb71810`, `84c79fa` (silent and contact-only peers are evicted) |
| F9 | Fixed | `fb71810`, `a180ec2`, `53f592f` (rate-limit slack) |
| F13 | Fixed (Reassembler displacement accepted, per spec) | `fb71810`, `506d880`, `9362e9f` |
| F18 | **Partly fixed**: the store side is done (flush through the `Store` trait, the caller's tick as the marker, clamping), and both hosts flush right after they first adopt trusted time. Hosts still don't flush on a timer or at shutdown | `c42f5de`, `06a0577`, `039fbf7`, `e4544bf` |
| F19 | **Partly mitigated** (new, 2026-10-21): peer time is chosen by decision J and bounded to a plausible range, but it stays unauthenticated until 1.1 | `039fbf7`, `e4544bf`, `4690bf9`, `03b75f2` |
Still open:
- F10–F12: the host-hardening spec.
- F7 and F14: 1.1.
- F18 (remaining): scheduled flushes, host work (§10).
- F19: 1.1 (signatures), or a local clock source before peers (§6).
- F18 (remaining): timed and shutdown flushes, host work (§10).
- F19 (remaining): 1.1 (signatures); until then, a local clock source
wherever one exists (§6).
## 2. System and trust boundaries
@@ -145,10 +150,10 @@ the fleet fixes.
| Malformed frames dropped | Hard constraint | **Holds** in core (wire.rs:64-69). The 1.0 spec adds reporting. | Unchanged; 1.0 adds reporting. |
| Bounded memory in core | Fixed-capacity buffers | **Holds** for the exchange (§6 table). **Doesn't hold for the system**: MemStore and LogStore are unbounded without `trim`, and no host calls `trim` (F10). | Unchanged in practice: the service calls `trim` with an empty default policy, tines-ros2 never does (F10, F18). |
| Custody: ≥ 1 holder (no carrier failure) | Fleet spec, sim property | **Breaks without an attacker** when a store is full (F1). Breaks trivially under A1 (F7). | Holds without an attacker (F1 fixed, with a sim property). Still breaks under A1 (F7). |
| Frames bound to the advertised NodeId | Fleet spec ("frames whose `from` differs … are dropped at the transport") | **Doesn't hold** (F3, F6). | **Holds at the transport** for tines-bt (F3), against unauthenticated adverts. Core still re-reads `from` per frame (F6). |
| Frames bound to the advertised NodeId | Fleet spec ("frames whose `from` differs … are dropped at the transport") | **Doesn't hold** (F3, F6). | **Holds at the transport** for tines-bt (F3), against unauthenticated adverts. Since 1.0 (`fb71810`, `506d880`) core also pins a peer's NodeId to its first frame in a contact and drops, and counts, a frame whose `from` differs (F6). |
| Released messages not re-forwarded after reboot | Fleet spec | **Doesn't hold**: release flags are never flushed (F4). Safe direction (duplicate custodian), but wasteful. | **Holds**: `Release` records are appended at once (not fsynced until the next flush, so a power cut can still lose one). |
| `next_seq` persisted across reboots | Core contract | **Doesn't hold** in either host (F4). | **Holds**: a persisted boot counter seeds `first_seq` in both hosts. |
| Age continues across reboot | Fleet spec | Partly. Hosts open the store at tick 0, and the service's tick is wall-clock *seconds* (F17). | Partly. Monotonic ms (F17 fixed) and clock markers fold age across a restart, but only up to the last flush, which may be long ago or never (F18). |
| Age continues across reboot | Fleet spec | Partly. Hosts open the store at tick 0, and the service's tick is wall-clock *seconds* (F17). | Partly. Monotonic ms (F17 fixed) and clock markers fold age across a restart, but only up to the last flush, which may be long ago or never (F18). Since 1.0 the downtime itself is folded in once the host has trusted wall time; without it, held addressed messages expire and broadcasts are floored (F18, F19). |
## 6. Findings
@@ -312,6 +317,10 @@ handshake.
the transport, so this is F3's consequence. Still, core could cheaply
refuse a NodeId *change* for an existing peer within a contact, and
should, once the transport binds identities.
- **Fixed in `fb71810`, `506d880`:** the first decoded frame in a contact
pins `peer.id`. A later frame with another `from` is dropped and counted as
`DropReason::IdentityChanged`. The pin resets on `Lost` and on a repeat
`Contact`.
### F7 — Custody can be taken by anyone the carrier sends to (Medium; by design v1)
@@ -339,6 +348,12 @@ Eight spoofed addresses lock a robot out of exchange until restart.
**Fix:** idle-timeout eviction in core (a local constant), and real
`Lost` events from tines-bt.
- **Fixed in `84c79fa`** (first attempt `fb71810` evicted only the peer that
had just sent): a peer with no inbound frame for `PEER_IDLE_MS` (30 s),
counted from its contact if it never sent, is removed as if `Lost`. Idle
peers are swept before a peer is added and at the start of every
`next_frame`, so eight silent or contact-only addresses can't hold the
table.
### F9 — Summary amplification (Medium, A1)
@@ -351,9 +366,18 @@ re-queues what was already sent.
- Each rebuild also scans every broadcast and clones payloads, which costs
CPU and allocation per summary, with no rate limit.
- **Fix:** rate-limit summary processing per peer (honour at most one per
`SUMMARY_INTERVAL_MS`), and remember what was already sent this contact
(`bcast_sent` already exists) so a repeated summary doesn't re-queue it.
- **Fix** (as amended in spec §10, 2026-10-20 and decision L):
- Rate-limit summary processing per peer: act on at most one summary per
`SUMMARY_INTERVAL_MS`. To allow for poll jitter, a summary is accepted
once at least `SUMMARY_INTERVAL_MS / 2` has passed since the last one
acted on, which still bounds replay to two per interval.
- The send-queue rebuild skips ids the peer's filter says it holds
(`Bloom::might_contain`). Output per received summary stays bounded by
`SEND_QUEUE_CAP`.
- It doesn't skip ids already sent this contact. "Sent" doesn't mean
"received", so a peer that lost an id must get it again on its next
summary.
- **Fixed in `fb71810`, `a180ec2`, `53f592f`.**
### F10 — Unbounded growth (Medium, A0/A1/A4)
@@ -364,6 +388,9 @@ re-queues what was already sent.
which removes nothing; tines-ros2 never calls it. Releases are now one
small record each rather than a growing list, but the log still never
compacts.
- *At `411daa0`:* `LogStore` rewrites the log as a snapshot at every open
and after the wall-clock fold (`06a0577`), which compacts it. Between
those points it only grows, and the trim policy is still empty.
- Unbounded channels: tines-bt's inbound and outbound `mpsc`
(transport.rs:207-208), and tines-ros2's inbound `mpsc` (node.rs:152).
- The IPC server allocates up to 1 MiB per request frame.
@@ -435,6 +462,15 @@ re-queues what was already sent.
- **Trailing bytes:** `postcard::from_bytes` accepts trailing bytes after a
valid frame. That's harmless now, but the 1.0 vectors should pin
rejection so the wire format is canonical.
- **Fixed in `fb71810`, `506d880`, `9362e9f`**, with Reassembler
displacement accepted, per spec §10:
- the event queue drops the oldest event;
- removing a peer purges that peer's pending acks, and only its own
(`506d880`; the first fix purged every peer's);
- a frame, and a reassembled observation (`9362e9f`), must decode with
no trailing bytes, or it's counted as `Malformed`;
- displacement stays as it is: it's same-address only, and the transport
binding (F3) plus the F6 pin limit who can do it.
### F14 — Belief policies trust attacker-controlled fields (Medium; by design v1)
@@ -489,8 +525,8 @@ marker that carries age across a restart, and it's the only `fsync` for
inserts and trims (log.rs module docs, "Durability contract").
- The service runs `trim` and `flush` only after `POLL_CAP` (64) polls that
each stored something new (tines/src/ipc.rs:199-217). The counter
advances only while observations arrive, so a robot that is mostly
each stored something new (tines/src/ipc.rs:269-279 at `411daa0`). The
counter advances only while observations arrive, so a robot that is mostly
offline, the normal case, may never flush.
- tines-ros2 never calls `flush` or `trim`.
- Even when it runs, `flush` wrote `LogStore`'s own tick as the clock
@@ -514,30 +550,81 @@ inserts and trims (log.rs module docs, "Durability contract").
`flush_interval_ms`, from the host's monotonic clock) in both hosts, and
on clean shutdown. The fleet-fixes spec asked for "a scheduled flush". The
trim defaults belong to host hardening (F10).
- **Partly fixed in `c42f5de`, `06a0577`, `039fbf7`, `e4544bf`.**
- **Store side, done.** `Store::flush(now)` takes the caller's tick
(`c42f5de`). `LogStore` implements it as the trait method, so
`Node::flush` really writes the marker and fsyncs; before `06a0577` it
hit the trait's no-op default. An earlier `now` is clamped to the
store's time anchor.
- **Flush after first trusted time, done.** Both hosts adopt trusted time
through `tines::trust::adopt_wall_clock` (tines/src/trust.rs:90-108),
which flushes right after the fold. So the first marker with a wall time
reaches disk at once.
- **Snapshots.** `LogStore` also rewrites the log at open and after the
fold (decision I), so the fold's results survive a reboot.
- **Still open (host work):**
- The service's scheduled trim and flush are still tied to `POLL_CAP`
productive polls (tines/src/ipc.rs:269-279).
- tines-ros2 flushes only when it adopts trusted time, and never trims.
- Neither host flushes on a timer or at clean shutdown. A quiet robot
that loses power long after its last flush still under-counts the
time since that flush.
### F19 — Peer-derived trusted time is unauthenticated (Medium, A0)
### F19 — Peer-derived trusted time is unauthenticated (Medium, A0/A1)
*Added 2026-10-21, when the hosts gained the peer-time path (Task 10b).*
When a host has no `trusted_clock = "system"` source, it estimates wall time
from a peer's observation: `observed_at + age_ms` from the youngest such
observation, then folds it into the store once. Nothing authenticates that
estimate.
When a host has no local trusted clock, it estimates wall time from peers'
observations (`observed_at + age_ms`, advanced by local ticks since the
observation arrived), then folds it into the store once. Nothing
authenticates that estimate.
- **Effect:**
- A forged time **ahead** of the true time only makes observations expire
early: the fold ages everything faster, addressed messages die sooner,
and `observed_at` stamps run fast. Availability loss, not integrity loss.
- A forged time **behind** the true time takes the "downtime unknown"
path (or under-folds), leaving the store quarantined longer or ages
younger — the same effect as F17/F18 by another route.
- Worst case: the robot's own `observed_at` stamps are wrong until a
local source corrects them, so every observation it publishes between
the forged fold and the correction carries a bad wall time.
- **Mitigation:** prefer a local source whenever one exists (the config
flag is exactly that); otherwise accept that a peer's time is a
lower-bound hint, not a trusted clock. Signing and verification (1.1)
closes it; a monotonic-host sanity check (reject a wall time behind the
last fold) would contain the "behind" case today.
- A time too far **ahead** makes the fold over-count downtime: addressed
messages expire early, facts look stale, and `observed_at` stamps run
fast. That's availability loss, not integrity loss.
- A time **behind** the last clock marker reads as "clock went backwards",
which takes the unknown-downtime path: held addressed messages expire
and broadcasts are floored. Also the safe direction.
- A time behind the truth but still ahead of the marker under-counts the
downtime, so held ages come out too young. That's the dangerous
direction (spec §2b, §9). Honest estimates can land here, because age
is a lower bound.
- Before the 1.0 review, a forged `observed_at = u64::MAX` overflowed
`wall + elapsed` in the hosts and the store: a panic in debug, a time
near 1970 in release (review B3).
- **Mitigated since `039fbf7`, `e4544bf`, `4690bf9`, `03b75f2`:**
- **Decision J: the latest estimate in a window.** From the first
plausible estimate, the host collects for `peer_time_window_ms`
(default 10 s) and then adopts the latest one, so an honest undershoot
loses to a better estimate. `LogStore::peer_time_estimate` returns the
latest estimate among observations stored since this open. A forger
gains nothing over the old smallest-age rule.
- **Plausibility bounds (review B3).**
- The hosts adopt only wall times between `WALL_FLOOR_MS`
(2026-01-01Z) and `WALL_CEILING_MS` (floor + 100 years, about 2126).
That applies to peers and to the system clock (`tines::trust`).
- The store drops estimates outside
`LogStoreBuilder::plausible_wall_ms` before choosing the latest, so a
forged far-future estimate can't hide the honest ones. Both hosts set
it.
- The wall-time arithmetic saturates.
- **Local sources first.** The service's `trusted_clock = "system"`, and
tines-ros2's new `trust_system_clock` / `trust_wall_clock`, both take
priority over peers. A ROS2-only fleet now has a local source.
- **Still open:**
- Peer time is unauthenticated until 1.1. Within the plausible range, a
forger can still choose the time, and the window takes the latest
estimate, forged or not.
- A wrong-but-plausible time spreads. The robot stamps its own
observations' `observed_at` from it, and peers without a trusted clock
can adopt those stamps in turn.
- The adopted time is corrected only at the next restart, or when a
local trusted source arrives. Nothing corrects it while the host runs
(decision J), and on the peer path it's adopted once per run.
- **Mitigation for deployments:** give every robot a local trusted source
(RTC, GPS, or an NTP-synced clock with the service ordered after
`time-sync.target`). Signing and verification in 1.1 close the forged
case.
## 7. Threats by STRIDE, mapped to findings
@@ -582,22 +669,34 @@ The sim covers honest-node custody properties and random garbage frames
(`tests/props.rs:281`, `tests/relay.rs`). It doesn't cover:
- well-formed malicious frames: forged acks, spoofed `from`/`observer`,
NodeId changes mid-contact;
NodeId changes mid-contact (since `37ca04d` the fuzz harness generates
all three, but in the core harness only; the sim and the hosts don't);
- ~~a store returning `Full` (F1)~~ (covered since `2738de4`: a unit test
and the full-store custody sim property);
- summary floods and peer-table exhaustion (F8, F9);
- reassembly abuse;
- ~~summary floods and peer-table exhaustion (F8, F9)~~ (covered since
`37ca04d`, `8ba1e86`: the fuzz harness's adversarial mode replays
summaries and floods the peer table from 3 × `MAX_PEERS` addresses, and
the proptest twin checks recovery after `PEER_IDLE_MS`);
- reassembly abuse (now partly covered: the adversarial mode sends
conflicting, repeated and out-of-order fragment streams);
- a full disk (`LogStore` corruption in the middle, oversized records,
concurrent openers, and restart and seq continuity are covered since
`848721a` and `4718b6f`);
- a quiet host restarting without having flushed (F18);
- a quiet host restarting without having flushed (F18). `tines-store`
now tests flushing through the trait, opening three times, and clamping
(`06a0577`), but no host-level test covers a long quiet run;
- forged peer time beyond overflow and range checks: nothing tests a
wrong-but-plausible estimate spreading through `observed_at` (F19);
- most of the hosts: IPC abuse, ROS2 injection and self-echo (F11, F12).
The service now has a two-instance exchange test (`d98895b`), and
tines-bt has mock-radio tests for framing, binding and contact events
(`7df9f41`).
The 1.0 fuzz harness (spec §7) should take an *adversarial peer* mode, not
just random bytes: structured frames with chosen ids and identities.
The 1.0 fuzz harness (spec §7) has an *adversarial peer* mode since
`37ca04d`, as well as random bytes: structured frames with chosen ids and
identities, checked against the §7 and §10 properties. It runs nightly in CI
(`.github/workflows/fuzz.yml`), with a proptest twin in the PR gate. The
agreed fuzz budget hasn't been run and recorded yet.
## 10. Where each finding is being fixed
@@ -605,17 +704,18 @@ just random bytes: structured frames with chosen ids and identities.
|---|---|---|---|
| F1, F2, F5, F17 | Fleet milestone (M3) | `docs/superpowers/specs/2026-10-19-tines-fleet-fixes.md` | Fixed (§1) |
| F3, F4 | Fleet milestone (M3) | same | Partly fixed; the rest has no spec yet |
| F18 | Host work, with or before host hardening | none yet | Open |
| F18 | Store side: tines-core 1.0 and tines-store; timed and shutdown flushes: host work, with or before host hardening | 1.0 spec §4, §9; none yet for the host part | Partly fixed (§1) |
| F10–F12 | Host hardening, before a robot leaves the lab | `docs/superpowers/specs/2026-10-19-tines-host-hardening.md` | Open |
| F6, F8, F9, F13 | tines-core 1.0 | `docs/superpowers/specs/2026-10-18-tines-1-0-design.md` §10 | Open |
| F6, F8, F9, F13 | tines-core 1.0 | `docs/superpowers/specs/2026-10-18-tines-1-0-design.md` §10 | Fixed (§1); F13's Reassembler displacement accepted |
| F19 | Hosts and tines-store (decision J, plausibility bounds); authentication in 1.1 | 1.0 spec §9, decision J; §2a | Partly mitigated (§1) |
| F7, F14, §8 | 1.1 (threat model, link authentication, signing) | 1.0 spec §2a, §2b | Open |
| F15, F16 | Accepted for now | — | — |
Mark each finding fixed here, with its commit, when it lands.
The remaining F3 work (one configured peer) and F18 are host-only. Neither
blocks the tines-core 1.0 work, but both block a fleet of more than two
robots.
blocks tines-core 1.0, but both block a fleet of more than two robots. F19's
remainder waits for 1.1.
## 11. Recommendations, in order
@@ -629,22 +729,27 @@ robots.
locking).
- F17 (monotonic ms clock).
- Each with a regression test.
2. **Host follow-ups:** the F3 multi-peer allowlist and F18 scheduled
flush. Small, and needed before more than two robots share a fleet.
2. **Host follow-ups:** the F3 multi-peer allowlist and the rest of F18
(timed and shutdown flushes in both hosts, and trim in tines-ros2).
Small, and needed before more than two robots share a fleet. The F18
store side and the flush after first trusted time are done.
3. **Host hardening before any robot leaves the lab:**
- F10 (trim, bounded channels, compaction).
- F11 (socket permissions, peer creds, timeouts).
- F12 (ignore own publications, SROS2 guidance, rate limits).
4. **Core hardening (rides with 1.0):**
- F8 (idle eviction).
- F9 (summary rate limit, no re-queue).
- F13 (event drop-oldest, purge acks on peer loss, reject trailing
bytes).
4. **Core hardening (rides with 1.0):** done on `feat/tines-core-1-0`
(2026-10-21), publish pending.
- F8 (idle eviction, including silent and contact-only peers).
- F9 (summary rate limit with slack; the rebuild skips what the peer's
filter holds).
- F13 (event drop-oldest, purge only the lost peer's acks, reject
trailing bytes; displacement accepted).
- F6 (refuse NodeId change within a contact).
5. **1.1:**
- Write the full threat model from §4/§8.
- Choose link authentication.
- Add signing per spec §2a (F14, F7).
- Add signing per spec §2a (F14, F7), and authenticate peer time
(F19).
- Add "replace if better signed" dedupe.
6. **Later:** confidentiality and rotating rendezvous identifiers (F15),
only if the deployment needs them.
tines-bt/README.md
+5 -5
View File
@@ -37,7 +37,7 @@ the peer's advertised [`NodeId`] instead:
hard-coded one.
A frame's declared sender is then checked against the bound [`NodeId`]: the
transport peeks the `from` field with [`wire::frame_sender`] and drops any frame
transport peeks the `from` field with [`frame_sender`] and drops any frame
whose declared sender differs from the bound peer (an address-spoofing
violation).
@@ -49,7 +49,8 @@ violation).
## Multi-peer
Up to [`MAX_PEERS`] connections may be live at once, one per bound [`NodeId`];
Up to `MAX_PEERS` (8, tines-bt's own cap, equal to tines-core's peer table)
connections may be live at once, one per bound [`NodeId`];
a second connection to the same [`NodeId`] replaces the first.
**Current limit:** `BluerRadio::new` and `BtTransport::new` still take a single
@@ -75,12 +76,11 @@ BlueZ version here, along with whether SeqPacket worked or the length-prefix
fallback was needed.
[`Transport`]: https://docs.rs/tines-core/latest/tines_core/transport/trait.Transport.html
[`NodeId`]: https://docs.rs/tines-core/latest/tines_core/observation/struct.NodeId.html
[`NodeId`]: https://docs.rs/tines-core/latest/tines_core/observation/type.NodeId.html
[`Radio`]: crate::Radio
[`BluerRadio`]: crate::BluerRadio
[`adv::decode_ad_node_id`]: crate::adv::decode_ad_node_id
[`wire::frame_sender`]: https://docs.rs/tines-core/latest/tines_core/wire/fn.frame_sender.html
[`frame_sender`]: https://docs.rs/tines-core/latest/tines_core/fn.frame_sender.html
[`MAX_FRAME`]: https://docs.rs/tines-core/latest/tines_core/consts/constant.MAX_FRAME.html
[`MAX_PEERS`]: https://docs.rs/tines-core/latest/tines_core/consts/constant.MAX_PEERS.html
[`l2cap::SeqPacket`]: https://docs.rs/bluer/latest/bluer/l2cap/struct.SeqPacket.html
[`SeqPacketListener`]: https://docs.rs/bluer/latest/bluer/l2cap/struct.SeqPacketListener.html