kyle/tines
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: sweep README, CHANGELOG, THREAT_MODEL and AGENTS after the 1.0 review fixes

Browse Source 
- CHANGELOG: tines-core 1.0.0 and the hosts' 0.4.0 are separate,
  unreleased sections. Every core API break is listed (the nine
  pub(crate) items, the private modules and the frame_sender path, the
  local constants per decision K, non_exhaustive types, store_mut). The
  new names (WallClockStore, Node::set_wall_clock, peer_time_estimate,
  flush, IdentityChanged) are called out. The host section covers
  tines::trust, the config changes, decision J, snapshots, and the
  review fixes.
- THREAT_MODEL: F6, F8, F9 and F13 are fixed with their commits. F18 is
  partly fixed: the store side and the flush on first trusted time are
  done, but timed and shutdown flushes are still open. F19 now records
  decision J and the plausibility bounds, and what remains. §9, §10 and
  §11 are updated to match.
- README: Dest and sig in the observation table; flush and is_empty in
  the store table; only the public wire constants are listed; a new
  "Trusted time" section with the config keys and the systemd ordering;
  the MSRV and fuzz CI jobs; frame_sender's new path.
- AGENTS: 1.0 is implemented but not yet published. The constants list
  shows the pub(crate) split, PEER_IDLE_MS, MIN_MTU's role and where
  MAX_FRAGMENTS lives. Notes the hidden conformance module.
- tines-bt README: frame_sender's path, and its own MAX_PEERS.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
kyle
2026-10-22 08:44:09 -07:00
co-authored by Claude Opus 5.5
parent a0dcc0ad89
commit 0cb1ad5360
5 changed files with 648 additions and 209 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+155 -49
View File
@@ -29,6 +29,7 @@ hear each other.
- [Running on a robot](#running-on-a-robot)
- [Crate layout](#crate-layout)
- [Building and CI](#building-and-ci)
- [Stability](#stability)
- [Security](#security)
- [Where to read more](#where-to-read-more)
@@ -88,14 +89,18 @@ An `Observation` is the unit of storage and exchange:
| `seq` | the observer's sequence number | ✓ |
| `topic` | what it is about, ≤ 64 bytes (`b"door/3"`) | ✓ |
| `payload` | opaque bytes, ≤ 1024 | ✓ |
| `to` | `None` for broadcast, or a recipient `NodeId` | ✓ |
| `age_ms` | how old the observation is | |
| `observed_at` | optional absolute timestamp, if the robot has a clock | |
| `lifetime_ms` | validity window, present exactly when `to` is | |
| `dest` | `None` for broadcast, or `Some(Dest { to, lifetime_ms })` for an addressed observation | `to` only |
| `age_ms` | how old the observation is: a lower bound that every holder advances | |
| `observed_at` | the observer's wall-clock time (ms since the Unix epoch), or `None` if it had no trusted clock | |
| `sig` | a reserved `Signature` slot (algorithm + up to 64 bytes); carried verbatim, signed and verified from 1.1 | |
The **content id** is the first 16 bytes of Blake2s-256 over the ✓ fields.
Because age and timestamps are excluded, the same fact arriving by two
different paths, at two different ages, dedupes to one entry.
The **content id** is the first 16 bytes of Blake2s-256 over the ✓ fields
(`dest.to`, but not `dest.lifetime_ms`). Because age, timestamps, lifetimes
and signatures are excluded, the same fact arriving by two different paths, at
two different ages, dedupes to one entry.
`Observation` is `#[non_exhaustive]`: build one with `Observation::broadcast`
or `Observation::addressed`, or let `Node::observe` / `Node::observe_to` do it.
### Time is relative
@@ -154,9 +159,11 @@ sequenceDiagram
### Wire frames
Frames are [postcard](https://docs.rs/postcard)-encoded and each carries the
Every frame starts with a `WIRE_VERSION` byte (4 for wire v1), followed by
the [postcard](https://docs.rs/postcard)-encoded frame. Each frame carries the
sender's `NodeId`, so the core never needs to map transport addresses to
robots.
robots. A frame with another version byte, trailing bytes, or anything else it
can't decode is dropped and counted (see [Receipts](#receipts)).
```mermaid
classDiagram
@@ -183,13 +190,14 @@ classDiagram
```
A transport that binds a connection to a peer's identity can check a frame's
claimed sender with `tines_core::wire::frame_sender`, without decoding the
claimed sender with `tines_core::frame_sender`, without decoding the
whole frame.
### Fragmentation
An observation that doesn't fit the transport's MTU is split into up to 64
`Frag` frames (64 bytes of each frame are reserved for the header). Each peer
`Frag` frames (65 bytes of each frame are reserved for the version byte and
the header). Each peer
has one stream in flight at a time, and it is finished before anything else
starts for that peer. All fragments of a stream are cut from one encoding, so
the receiver can always reassemble them. A partial observation is never
@@ -295,12 +303,15 @@ it has already released, since that would hand custody back to a robot that
no longer carries the message.
Broadcast confirmations surface as `NodeEvent<A>::Delivered { peer, id }` from
`Node::drain_events`. Frames the peer could not use are counted under a
`DropReason` (`VersionMismatch` or `Malformed`) and surface as
`NodeEvent::Dropped { reason, count }` (see the `drop` reporting in the `tines`
service and the ROS2 node). The event queue is bounded (64) plus a small
counted-drop table, so drops are always reported even under a chatty peer —
events are telemetry, and losing one never loses data.
`Node::drain_events`. Frames this node received but couldn't use are counted
under a `DropReason` (`VersionMismatch`, `Malformed`, or `IdentityChanged` for
a peer whose `from` changed mid-contact) and surface as
`NodeEvent::Dropped { addr, reason, count }`, keyed by the sender's transport
address (see the `drop` reporting in the `tines` service and the ROS2 node).
The event queue is bounded (64, dropping the oldest), and drops go to a small
counted table with one overflow counter (`addr: None`), so drops are always
reported even under a chatty peer. Events are telemetry: losing one never
loses data.
## Usage
@@ -347,6 +358,7 @@ for event in node.drain_events() {
tines_core::NodeEvent::Dropped { addr, reason, count } => {
println!("dropped {count} frame(s) from {addr:?}: {reason:?}");
}
_ => {} // NodeEvent is #[non_exhaustive]
}
}
@@ -407,10 +419,11 @@ pub trait Transport {
- **Report back-pressure honestly.** Return `SendError::Busy` when the radio
can't take a frame right now; the exchange will offer the same frame again
on the next poll. `TooLarge` tells the exchange to give that frame up.
- **Driving `Exchange` without `Node`?** `Exchange::next_frame` assumes the
frame it returns gets sent. If `send` fails, call
`Exchange::on_send_error(&err)` before the next `next_frame`, or the frame
is treated as delivered. `Node::poll` does this for you.
- **Mind the MTU.** `mtu()` must be at most 1500. Core can carry every valid
observation from 85 bytes up; below that it never sends an oversized frame,
but it sends less (see `Transport::mtu`'s docs for each floor).
- **Binding identities?** A transport that binds a connection to a `NodeId`
can check each frame's claimed sender with `tines_core::frame_sender`.
## Writing a store
@@ -424,12 +437,14 @@ flash or SQLite backend implements the `Store` trait:
| `get` | the observation with `age_ms` advanced by `now − stored_at`, saturating if `now` is earlier |
| `len` | number of stored observations |
| `for_each_id` | every stored id, any order |
| `for_each_broadcast_id` | ids with `to == None`; these feed the summary |
| `for_each_addressed_id` | ids with `to == Some(_)`; scanned for custody on every poll |
| `for_each_broadcast_id` | ids with `dest == None`; these feed the summary |
| `for_each_addressed_id` | ids with `dest == Some(_)`; scanned for custody on every poll |
| `release` | mark an id as handed off, so it's never offered again; a no-op if the id isn't stored |
| `is_released` | whether an id has been released; a re-inserted observation starts un-released |
| `for_each_topic` | age-adjusted observations whose topic starts with a prefix |
| `trim` | apply a `TrimPolicy` (per-topic, then age, then count); return how many were removed |
| `is_empty` | `len() == 0` (default provided) |
| `flush` | the durability point and the time anchor for ages after a restart; harmless twice, an earlier `now` is clamped; returns `Err(StoreError::Backend)` on failure and leaves the store usable (default: no-op `Ok(())`) |
The store must remember when each observation was inserted (`stored_at`, in
the caller's `Tick`s) so it can report current ages. `insert` may return
@@ -439,7 +454,19 @@ custody.
`for_each_addressed_id` has a default implementation that visits every id,
which is correct but makes the exchange read every broadcast observation on
each poll just to find the few addressed ones. Override it if your backend can
filter by `to` cheaply, e.g. with an index or a separate table.
filter by `dest` cheaply, e.g. with an index or a separate table. The
filtering is only an optimisation: core re-checks `dest` and `is_released` on
every id a backend yields.
A store that keeps `sig` must return it verbatim. Core never calls `flush`;
the application decides how often, because on flash every flush costs wear.
A backend that holds data across power-off can also implement the opt-in
`WallClockStore` trait, which lets the host pass in trusted wall time through
`Node::set_wall_clock` (see [Trusted time](#trusted-time)).
With the `conformance` feature, `tines_core::store_conformance` checks a
backend against the trait contract (dedupe, age lineage, saturating ticks,
release, `sig` round-trip, trim order, no tombstones, `flush`).
## The simulator
@@ -508,13 +535,21 @@ flowchart LR
A robot runs either the service or the ROS2 node, not both: each owns its
`Node` and its log.
- **`tines-store`**: `LogStore`, an append-only log file.
- **`tines-store`**: `LogStore`, an append-only log file (format v2; an
older log is refused with an error that says to delete it).
- Each record is checksummed (CRC-32C). Recovery skips a corrupt record and
truncates only a torn tail.
- Releases are written as they happen.
- A boot counter provides `first_seq()`.
- `flush()` writes a clock marker, so ages carry across a restart, and
fsyncs the log.
- `Store::flush(now)`, reached through `Node::flush`, writes a clock marker
at `now` and fsyncs the log, so ages carry across a restart. An earlier
`now` is clamped.
- What the log held at open is quarantined until the host has trusted
wall time (see [Trusted time](#trusted-time)).
- The log is rewritten as a snapshot at open and after the wall-clock
fold, through a `<log>.snapshot-tmp` file that's fsynced and renamed
over it. That keeps the fold's results and ages across any number of
reboots, and compacts the log.
- The file is created `0600` and locked, so only one process can open it.
- **`tines-bt`**: BLE over BlueZ, using L2CAP SeqPacket channels (one
frame per packet).
@@ -527,8 +562,9 @@ A robot runs either the service or the ROS2 node, not both: each owns its
- **`tines`**: the standalone service, for robots without ROS2.
- It runs the exchange on a poll loop and answers `Observe`, `Belief`,
`Events` and `Status` requests on a Unix socket.
- It's configured by a JSON file: `node_id`, `peer`, `log_path`,
`socket_path` and `poll_interval_ms`.
- It's configured by a JSON file (below). Library users call
`tines::serve(node, socket_path, clock, poll_interval_ms,
peer_time_window_ms)`.
- **`tines-ros2`**: a ROS2 node.
- It publishes each tines topic under `/tines/` and confirmed deliveries on
`/tines/receipts`.
@@ -536,12 +572,73 @@ A robot runs either the service or the ROS2 node, not both: each owns its
- rclrs is behind the `ros2` feature, so the crate builds without a ROS2
toolchain.
The service's config file is a JSON object. An unknown key is an error, so a
typo can't silently fall back to a default.
| Key | Default | Meaning |
|---------------------------|--------------|---------|
| `node_id` | required | this robot's `NodeId`, 32 hex chars |
| `peer` | required | the one peer's `NodeId`, 32 hex chars |
| `log_path` | required | the `LogStore` log file |
| `socket_path` | required | the IPC Unix socket |
| `poll_interval_ms` | 20 | exchange poll cadence |
| `trusted_clock` | unset | `"system"` to trust the system clock; the only accepted value |
| `peer_time_window_ms` | 10000 | how long to collect peer time estimates before adopting the latest |
| `unknown_downtime_age_ms` | 24 h | age floor for held broadcasts when the downtime is unknown |
The hosts use a monotonic millisecond clock that starts at 0 when the
process does. Neither host sets `observed_at` yet.
process does.
### Trusted time
Ages are relative, but a robot that was powered off can't tell from ticks how
long it was off. So the hosts open the store before they know the wall time,
quarantine what it held (not offered, delivered or acked, but still deduped),
and adopt a trusted wall time **once per run**:
1. **A local source.** The service adopts the system clock at startup when
`trusted_clock = "system"`. tines-ros2's `TinesNode` has
`trust_system_clock(now)`, and `trust_wall_clock(wall_ms, now)` for an RTC
or GPS reading; call one right after construction.
2. **Otherwise, peers.** From the first plausible estimate a peer's
observations give (`observed_at + age_ms`), the host collects estimates
for `peer_time_window_ms` (10 s by default; `set_peer_time_window_ms` in
tines-ros2), then adopts the latest.
3. **Otherwise** the quarantine stays. The robot's own fail-safe is outside
tines.
Only a wall time between 2026-01-01 and about 2126 is adopted
(`tines::trust::WALL_FLOOR_MS` and `WALL_CEILING_MS`), from a peer or from
the system clock. A board that booted before NTP and reads 1970 is logged and
ignored, and the host falls back to peers. Adopting a time runs the store's
fold and flushes at once. If the log's last clock marker carries a wall time
no later than the adopted one, the fold adds the downtime to held ages.
Otherwise the downtime is unknown: the fold expires held addressed messages
and floors held broadcasts at `unknown_downtime_age_ms`. From then on
the host stamps its observations' `observed_at`; before, they carry `None`.
`TinesNode::has_trusted_time` reports whether a time was adopted.
The system clock is read once, at startup. Under systemd, order the service
after clock sync:
```ini
[Unit]
Wants=time-sync.target
After=time-sync.target
```
`time-sync.target` only waits for a synced clock when the sync service
supports it (`systemd-timesyncd` with `systemd-time-wait-sync.service`
enabled, or chrony's equivalent).
Peer time is unauthenticated until 1.1, and an adopted time isn't corrected
while the host runs; see F19 in [`THREAT_MODEL.md`](THREAT_MODEL.md).
**Current limits** (tracked in [`THREAT_MODEL.md`](THREAT_MODEL.md)):
- The service talks to one configured `peer`.
- The hosts don't yet flush on a timer.
- The hosts don't flush on a timer or at shutdown. The service flushes after
64 productive poll rounds, and both hosts flush when they adopt trusted
time.
- The host-hardening work (trim defaults, bounded channels, socket
permissions, ROS2 isolation) hasn't landed.
@@ -564,23 +661,24 @@ tines/
tines-ros2/ std: the ROS2 node (rclrs behind the `ros2` feature).
```
Fixed limits, all compile-time constants in `tines-core/src/consts.rs`:
The public constants, in `tines_core::consts`, are the wire constants. They
are frozen for wire v1: changing one is a `WIRE_VERSION` bump.
| Constant | Value | Meaning |
|-------------------------|--------|------------------------------------------|
| `MAX_TOPIC` | 64 | topic bytes |
| `MAX_PAYLOAD` | 1024 | payload bytes |
| `MAX_PEERS` | 8 | peers tracked at once |
| `SUMMARY_BYTES` | 512 | full Bloom summary size |
| `BLOOM_K` | 3 | hash functions per id |
| `SUMMARY_INTERVAL_MS` | 2000 | re-summary period per peer |
| `REASSEMBLY_TIMEOUT_MS` | 5000 | partial observation discard time |
| `SEND_QUEUE_CAP` | 256 | ids queued per peer |
| `MAX_FRAME` | 1500 | largest frame (and MTU) supported |
| `ACK_IDS_CAP` | 90 | ids in one `Ack` frame |
| `EVENT_QUEUE_CAP` | 64 | confirmed deliveries awaiting `drain_events` |
| `BCAST_SENT_CAP` | 256 | broadcast sends remembered per peer, to validate acks |
| `ACK_BUFFER_CAP` | 256 | received ids waiting to be acked, per kind |
| Constant | Value | Meaning |
|-----------------|-------|------------------------------------------|
| `WIRE_VERSION` | 4 | the first byte of every frame |
| `MAX_TOPIC` | 64 | topic bytes |
| `MAX_PAYLOAD` | 1024 | payload bytes |
| `MAX_FRAME` | 1500 | largest frame (and MTU) supported |
| `SUMMARY_BYTES` | 512 | full Bloom summary size |
| `ACK_IDS_CAP` | 90 | ids in one `Ack` frame |
| `MAX_SIG` | 64 | signature bytes |
| `BLOOM_K` | 3 | hash functions per id |
Local capacities and timings (peers tracked at once, the re-summary interval,
queue sizes, the idle-eviction window, and so on) are internal defaults, not
public API. They can change in a minor release, and a later 1.x release will
make them configurable.
## Building and CI
@@ -593,7 +691,15 @@ sh scripts/ci.sh
```
GitHub Actions runs the same script on every push to `master` and on every
pull request.
pull request. Two more jobs run there:
- **MSRV:** builds `tines-core` on Rust 1.82 (its `rust-version`), with and
without default features and for `thumbv7em-none-eabihf`. It builds only,
because a test dependency needs a newer compiler.
- **Fuzz:** a scheduled nightly job runs `cargo-fuzz` on the `fuzz/` crate's
`node` target, in both its random-bytes and adversarial-peer modes, and
keeps any crash reproducer. A bounded proptest twin of the same harness runs
in the normal test suite.
## Stability
@@ -637,7 +743,7 @@ The specs are authoritative; the plans argue from them.
- **Fleet deployment:** `docs/superpowers/specs/2026-10-18-tines-fleet-deployment-design.md`
- **Fleet fixes:** `docs/superpowers/specs/2026-10-19-tines-fleet-fixes.md`
- **Host hardening (next for the hosts):** `docs/superpowers/specs/2026-10-19-tines-host-hardening.md`
- **tines-core 1.0 (next for the core):** `docs/superpowers/specs/2026-10-18-tines-1-0-design.md`
- **tines-core 1.0 (implemented on `feat/tines-core-1-0`; publish pending):** `docs/superpowers/specs/2026-10-18-tines-1-0-design.md`
- **Threat model:** [`THREAT_MODEL.md`](THREAT_MODEL.md)
- **Plans:** `docs/superpowers/plans/`
- **Changes:** [`CHANGELOG.md`](CHANGELOG.md)