diff --git a/AGENTS.md b/AGENTS.md index 88f4fdc..4c3aa21 100644 --- a/AGENTS.md +++ b/AGENTS.md @@ -15,11 +15,13 @@ observations, a belief layer derived from them, and a delay-tolerant exchange protocol over any datagram link. Read the full intent, goals, anti-goals, and architecture in the **spec first**: -- **Spec (authoritative):** `docs/superpowers/specs/2026-10-16-tines-design.md` -- **Implementation plan (how to build it):** `docs/superpowers/plans/2026-10-16-tines-core-and-sim.md` +- **Core design (authoritative):** `docs/superpowers/specs/2026-10-16-tines-design.md` +- **Addressed delivery (authoritative):** `docs/superpowers/specs/2026-10-18-tines-addressed-delivery.md` +- **Implementation plans:** `docs/superpowers/plans/` -The plan argues from the spec. Where the plan is silent, the **spec rules**. -Do not edit the spec or the plan — they are design history, not source. +The plans argue from the specs. Where a plan is silent, the **spec rules**. +Do not edit the specs or the plans — they are design history, not source. +`README.md` is the user-facing overview and should track the code. ## Architecture (three layers) @@ -36,20 +38,15 @@ details. ## Workflow -This project is built by working the implementation **task-by-task, in order**. -Each task is TDD: write the failing test → see it fail → implement → see it -pass → commit. Do not skip ahead — later tasks depend on the exact names and -signatures earlier tasks produce. Tick the task checkboxes as you go. +Changes are TDD: write the failing test → see it fail → implement → see it +pass → commit. -For each task: - -1. Read its section in the plan. -2. Implement to the interfaces the plan pins; don't invent new public names. -3. Before committing, run: - - `cargo test --workspace` - - `cargo clippy --workspace -- -D warnings` - - `sh scripts/check-no-std.sh` (for any core change) -4. Commit with the **exact** message the plan gives for that task. +1. Keep the public API small. New public names in `tines-core` (types, trait + methods, functions) need a reason; call them out when you add them. +2. Before committing, run the full gate, which is what CI runs: + `sh scripts/ci.sh` (fmt, clippy and rustdoc with warnings as errors, tests + with and without `alloc`, and the `thumbv7em-none-eabihf` build). +3. Use conventional commit messages (`fix(core): …`, `ci: …`, `docs: …`). The `Node` facade is core, not application glue: an application on any target should only need to construct a `Node` with its `NodeId`, store, and transport, @@ -65,21 +62,25 @@ call `poll()` in its loop, and `observe()` when it sees something. unwrapped. - Core never maps a transport `Addr` to a `NodeId`; every frame carries the sender's `NodeId`. -- Identity hash covers `(observer, seq, topic, payload)` **only**. `age_ms` and - `observed_at` are excluded. +- Identity hash covers `(observer, seq, topic, payload, to)` **only**. + `age_ms`, `observed_at` and `lifetime_ms` are excluded. +- Addressed observations (`to = Some`) never appear in a summary Bloom + filter, and an addressed observation past its `lifetime_ms` is never + stored or forwarded. - `Node::next_seq()` must be persisted by the application across reboots (`boot_count << 32 | counter`); a repeated seq after reboot collides in the ID hash and must dedupe silently (`insert` returns `Ok(false)`). - `NodeId = [u8; 16]`, `ObservationId = [u8; 16]`, `Tick = u64` (local monotonic ms, supplied on every call). -### Review Focus (each pinned to a task with a test) +### Review Focus (each covered by a test; keep it that way) -1. Reused `seq` after reboot → store dedupes, `insert` returns `Ok(false)`. (Task 3) -2. Oversized `topic`/`payload` at `observe()` → `Err(TooLarge)`, never truncate/panic. (Task 10) -3. Garbage frames → dropped, no panic, no store change. (Tasks 2, 6, 7, 13) -4. Frame from an address with no prior `Contact` → treated as implicit contact. (Task 7) -5. Tick going backwards → age saturates at stored value, never underflow. (Task 3) +1. Reused `seq` after reboot → store dedupes, `insert` returns `Ok(false)`. +2. Oversized `topic`/`payload` at `observe()` → `Err(TooLarge)`, never truncate/panic. +3. Garbage frames → dropped, no panic, no store change. +4. Frame from an address with no prior `Contact` → treated as implicit contact. +5. Tick going backwards → age saturates at stored value, never underflow. +6. Every emitted frame fits the transport's `mtu()`, including summaries. ### Compile-time constants (`tines-core/src/consts.rs`) @@ -94,11 +95,11 @@ tines/ tines-core/ no_std core: Observation, Store/Transport/BeliefPolicy traits, exchange state machine, Node facade, in-memory store, reference policies - tines-sim/ std: simulator, contact traces, convergence property tests + tines-sim/ std: simulator, contact traces, convergence and relay + property tests ``` -## Not in scope for this plan +## Not started -`tines-udp`, `tines-sqlite`, `tines-cli`, and `tines-serial` belong to a second -plan. Do not start them here. The README lives in Task 14 of the plan — it will -be produced by the implementer; do not author it separately. +`tines-udp`, `tines-sqlite`, `tines-cli`, and `tines-serial` (from the core +spec's crate layout) belong to a future plan. Don't start them unless asked. diff --git a/README.md b/README.md index e86c69c..d336fa1 100644 --- a/README.md +++ b/README.md @@ -14,39 +14,249 @@ protocol that runs over whatever datagram link the robot has. The name is from Vinge's *A Fire Upon the Deep*: the Tines only form a mind when close enough to hear each other. +## Contents + +- [Design at a glance](#design-at-a-glance) +- [Architecture](#architecture) +- [Observations](#observations) +- [The exchange protocol](#the-exchange-protocol) +- [Addressed delivery](#addressed-delivery) +- [Usage](#usage) +- [Writing a transport](#writing-a-transport) +- [The simulator](#the-simulator) +- [Crate layout](#crate-layout) +- [Building and CI](#building-and-ci) +- [Where to read more](#where-to-read-more) + +## Design at a glance + +- **Sharing is opportunistic, never load-bearing.** A contact may be cut off + at any frame. Two nodes' stores converge given enough cumulative contact + time; no single contact promises anything. +- **Order doesn't matter.** Storage is synced by set union over content-derived + ids, so a node ends up with the same store whether it met B or C first. +- **The MCU is a first-class target.** `tines-core` is `#![no_std]`, has no + async, threads, or I/O, and builds without an allocator for + `thumbv7em-none-eabihf`. Every buffer is fixed-size. +- **Untrusted input never panics.** Every frame off the wire is decoded + defensively; malformed bytes are dropped. +- **The application sets the pace.** `Node::poll` emits at most one frame per + call, so duty cycle and airtime are the application's call. + ## Architecture Three layers, each a separate concern, top to bottom: -``` - ┌──────────────────────────────────────────┐ - │ memory — "my priors": beliefs per │ derived, local, never synced - │ topic, computed from │ - │ storage under a policy │ - ├──────────────────────────────────────────┤ - │ storage — append-only log of raw │ synced by set-union - │ observations; local │ - │ trimming policy │ - ├──────────────────────────────────────────┤ - │ comms — contact-driven exchange │ PHY-agnostic; transport - │ of observations │ trait + adapters - └──────────────────────────────────────────┘ +```mermaid +flowchart TB + subgraph node["Node<S, T> — the application facade"] + direction TB + memory["memory
beliefs per topic, computed
from storage under a BeliefPolicy"] + storage["storage — Store trait
append-only set of observations,
keyed by content id; local trimming"] + comms["comms — Exchange
contact-driven summary / diff /
send over a Transport"] + memory -- "derived from (pure function)" --> storage + comms -- "inserts received / reads to send" --> storage + end + app(["application"]) -- "observe() / belief()" --> node + app -- "poll()" --> node + comms <-- "frames" --> transport[["Transport
(UDP, LoRa, serial, sim…)"]] ``` +| Layer | Holds | Crosses the wire? | +|---------|------------------------------------|-------------------| +| memory | beliefs, recomputed on demand | never | +| storage | raw observations | yes, by set union | +| comms | per-peer exchange state | carries the frames | + Only observations cross the wire. Memory is a pure function of storage plus a -policy and is never transmitted; storage is synced by set-union (order- and -path-independent); comms is a contact-driven exchange of observations over a -`Transport` trait. +policy and is never transmitted, so two robots can hold different beliefs about +the same facts — that is expected, not a bug. + +## Observations + +An `Observation` is the unit of storage and exchange: + +| Field | Meaning | In the id? | +|---------------|----------------------------------------------------------------|:----------:| +| `observer` | `NodeId` (`[u8; 16]`) of the robot that saw it | ✓ | +| `seq` | the observer's sequence number | ✓ | +| `topic` | what it is about, ≤ 64 bytes (`b"door/3"`) | ✓ | +| `payload` | opaque bytes, ≤ 1024 | ✓ | +| `to` | `None` for broadcast, or a recipient `NodeId` | ✓ | +| `age_ms` | how old the observation is | | +| `observed_at` | optional absolute timestamp, if the robot has a clock | | +| `lifetime_ms` | validity window, present exactly when `to` is | | + +The **content id** is the first 16 bytes of Blake2s-256 over the ✓ fields. +Because age and timestamps are excluded, the same fact arriving by two +different paths, at two different ages, dedupes to one entry. + +### Time is relative + +Robots don't share a clock, so the protocol carries *age*, not time. Each call +takes a local monotonic `Tick` (milliseconds), and a store reports an +observation's current age as: + +``` +current_age = age_ms_at_insert + (now − stored_at) // saturating +``` + +When an observation is sent, it goes out with its current age, so age keeps +accumulating across hops. A tick that goes backwards (after a reboot, say) +saturates rather than underflowing. + +### Sequence numbers must survive reboots + +`seq` is part of the id, so a robot must not reuse one for different content. +Persist `Node::next_seq()` and pass it back as `first_seq`, typically as +`boot_count << 32 | counter`. A repeated `seq` with identical content simply +dedupes (`insert` returns `Ok(false)`). + +## The exchange protocol + +When two robots come into range, both run the same symmetric procedure; +neither is the initiator. + +```mermaid +sequenceDiagram + autonumber + participant A as Robot A + participant B as Robot B + Note over A,B: Transport reports Contact on both sides + A->>B: Summary (salted Bloom filter of A's broadcast ids) + B->>A: Summary (salted Bloom filter of B's broadcast ids) + Note over A: diff: queue what B appears to lack, youngest first + Note over B: diff: queue what A appears to lack, youngest first + A->>B: Obs (or Frag × n) + B->>A: Obs (or Frag × n) + A->>B: Obs … + Note over A,B: every 2 s: re-summarise with a new salt,
so ids learned mid-contact propagate too + A-->>B: Summary (new salt) + Note over A,B: Lost — or the contact simply ends mid-frame +``` + +- **Summaries** are 512-byte salted Bloom filters (k = 3). On a narrow link + the filter is folded down to the largest power of two that fits the MTU. A + false "peer has it" only delays propagation; the salt changes every round, so + a different set of ids is affected each time. +- **Diffs** are sent youngest-first, on the theory that short contacts should + carry the freshest data. Each peer's queue holds up to 256 ids. +- **Every frame is self-contained and idempotent.** Duplicates, reordering and + loss are all harmless; there is no session state to corrupt. + +### Wire frames + +Frames are [postcard](https://docs.rs/postcard)-encoded and each carries the +sender's `NodeId`, so the core never needs to map transport addresses to +robots. + +```mermaid +classDiagram + class Summary { + from: NodeId + salt: u32 + bits: ≤512 bytes + } + class Obs { + from: NodeId + obs: Observation + } + class Frag { + from: NodeId + id: ObservationId + index: u8 + count: u8 + data: bytes + } +``` + +### Fragmentation + +An observation that doesn't fit the transport's MTU is split into up to 64 +`Frag` frames (64 bytes of each frame are reserved for the header). Each peer +has one stream in flight at a time, and it is finished before anything else +starts for that peer. All fragments of a stream are cut from one encoding, so +the receiver can always reassemble them. A partial observation is never +stored; the receiver discards it after 5 s. + +```mermaid +flowchart LR + obs["Observation
(e.g. 1 KB payload)"] --> enc["postcard encode
once per stream"] + enc --> f0["Frag 0/8"] & f1["Frag 1/8"] & fd["…"] & f7["Frag 7/8"] + f0 & f1 & fd & f7 -. "lossy link,
any order" .-> re["Reassembler
(per peer)"] + re -- "all 8 present,
id checks out" --> st[("store")] + re -- "timeout / bad id" --> drop(["discarded"]) +``` + +### What one `poll()` does + +`Node::poll` drains inbound transport events, then emits **at most one** +frame: + +```mermaid +flowchart TD + start(["poll(now)"]) --> drain["Drain transport events:
Contact → add peer
Lost → drop peer
Frame → decode, store, update peer"] + drain --> due{"Summary due
for some peer?"} + due -- yes --> sum["Send Summary"] + due -- no --> rr["Pick next peer, round-robin"] + rr --> inflight{"Fragment stream
in flight?"} + inflight -- yes --> frag["Send next fragment"] + inflight -- no --> turn{"Whose turn?"} + turn -- custody --> addr["Start an addressed message
(deliver, else forward)"] + turn -- broadcast --> diff["Start the front of the
broadcast diff"] + addr -. "nothing to send" .-> diff + diff -. "nothing to send" .-> addr + sum & frag & addr & diff --> send["transport.send()"] + send -- "Busy / Unreachable" --> undo["Roll back:
same frame next poll"] + send -- ok --> done([return]) + undo --> done +``` + +## Addressed delivery + +Most observations are broadcasts: everyone gets everything. An observation can +also be **addressed** to one robot with `Node::observe_to`, e.g. a tasking from +a base station. Addressed messages travel by store-and-forward: any robot that +meets a carrier becomes a carrier too, until the message reaches its recipient +or its lifetime runs out. + +```mermaid +sequenceDiagram + participant Base + participant R1 as Robot 1 + participant R2 as Robot 2 + participant R3 as Robot 3 (recipient) + Base->>Base: observe_to(R3, lifetime 60 s, "task", …) + Note over Base,R1: contact + Base->>R1: Obs to=R3 (epidemic forward) + Note over R1,R2: later contact + R1->>R2: Obs to=R3 (epidemic forward) + Note over R2,R3: later contact + R2->>R3: Obs to=R3 (delivery) + Note over R3: shows up in R3's belief() + Note over R1,R2: carriers drop it once age > lifetime +``` + +- Addressed messages **never appear in a summary**, so their existence doesn't + leak to unrelated robots through gossip. +- **Delivery** (the peer is the recipient) is preferred over **forwarding**. + Custody and broadcast take turns for each peer, so neither can starve the + other. +- **Bounce suppression:** within one contact, a robot never offers a message + back to the peer it came from, or to a peer that already has it. +- **Expiry is local and silent.** A message older than its `lifetime_ms` is + never forwarded and never stored. There are no acks and no cancellation + messages. +- `belief()` sees broadcasts and messages addressed *to me*. Messages carried + for others exist only to be forwarded. ## Usage `tines-core` exposes a single application-facing facade, `Node`, generic -over any `Store` and `Transport`. Persist `Node::next_seq()` across reboots -(`boot_count << 32 | counter`) and pass it back as the initial `first_seq`; a -repeated `seq` after reboot collides in the content-id hash and dedupes silently. +over any `Store` and `Transport`: ```rust -use tines_core::{LatestWins, MemStore, Node, NodeId}; +use tines_core::{LatestWins, MemStore, Node, NodeId, TrimPolicy}; let me: NodeId = [7; 16]; let boot_count = 0u64; @@ -68,6 +278,10 @@ let id = node .observe(b"door/3", b"open", Some(1_700_000_000_000), 0) .expect("observation fits the size limits"); +// Address a tasking to one robot; it is carried until delivered or 60 s old. +node.observe_to([9; 16], 60_000, b"task", b"inspect door/3", None, 0) + .expect("observation fits the size limits"); + // Drive the exchange in your loop. Returns how many new observations were // stored this round. let stored = node.poll(500); @@ -75,32 +289,159 @@ let stored = node.poll(500); // Ask what you believe about a topic prefix, under a belief policy // (e.g. the youngest observation wins). let belief = node.belief(b"door", &LatestWins, 500); + +// Keep the store bounded. +node.trim( + &TrimPolicy { + keep_per_topic: Some(4), + max_age_ms: Some(3_600_000), + max_count: Some(10_000), + }, + 500, +); ``` +`observe` rejects an oversized topic or payload with `ObserveError::TooLarge`; +nothing is ever truncated. + +### Belief policies + +A `BeliefPolicy` folds the matching observations, one at a time and in no +particular order, into a single belief. Nothing is buffered, which matters +with 1 KB payloads on an MCU. Two policies ship: + +| Policy | Picks | +|--------------|-------------------------------------------------------------------------| +| `LatestWins` | the youngest observation; ties broken by `(observer, seq)` | +| `SelfFirst` | my own youngest observation if I have one, otherwise like `LatestWins` | + +Write your own by implementing `BeliefPolicy::step` (for example, a majority +vote or a weighted average over payloads). Because beliefs are recomputed from +storage, changing policy never requires touching stored data. + +### Trimming + +`Store::trim` applies a `TrimPolicy` locally. In order, it keeps the N youngest +observations per topic, drops anything older than `max_age_ms`, then drops the +oldest observations until `max_count` remain. Trimmed ids aren't tombstoned: +if a peer offers one again later, it's accepted. + +## Writing a transport + +A transport is the boundary with the physical link. It only has to move +datagrams and report who is in range: + +```rust +pub trait Transport { + type Addr: Clone + PartialEq + core::fmt::Debug; + fn poll(&mut self) -> Option>; // Contact | Lost | Frame + fn send(&mut self, to: &Self::Addr, frame: &[u8]) -> Result<(), SendError>; + fn mtu(&self) -> usize; // ≤ 1500 +} +``` + +- **Datagram semantics.** Frames may be lost, duplicated or reordered; the + protocol assumes nothing more. +- **Contact detection is the transport's job:** a beacon heard, a link + coming up, or the simulator saying "in range". A frame from an address with + no prior `Contact` is treated as an implicit contact. +- **`Addr` is only a routing handle.** It can change mid-contact (DHCP, + roaming) without harm. +- **Report back-pressure honestly.** Return `SendError::Busy` when the radio + can't take a frame right now; the exchange will offer the same frame again + on the next poll. `TooLarge` tells the exchange to give that frame up. + +## The simulator + +`tines-sim` runs N nodes over a scripted contact trace with seeded loss and +duplication, and is the protocol's test harness: + +```rust +use tines_sim::{Contact, ContactTrace, Sim, WorldConfig}; + +let mut trace = ContactTrace::default(); +trace.contacts.push(Contact { start_ms: 0, end_ms: 5_000, a: 0, b: 1 }); +trace.contacts.push(Contact { start_ms: 10_000, end_ms: 15_000, a: 1, b: 2 }); + +let cfg = WorldConfig { nodes: 3, mtu: 512, loss: 0.1, dup: 0.05, seed: 42 }; +let mut sim = Sim::new(cfg, trace.clone()); +sim.observe(0, b"door/3", b"open"); +sim.run_until(trace.end()); + +assert!(sim.converged(&[0, 1, 2])); // 0 and 2 never met, but 1 carried it +``` + +```mermaid +gantt + title The contact trace above + dateFormat x + axisFormat %S s + section 0 ↔ 1 + in range :0, 5000 + section 1 ↔ 2 + in range :10000, 15000 +``` + +The property tests (`tines-sim/tests/props.rs`) generate random traces and +check that: + +- every pair of nodes that was ever transitively connected ends up with the + same broadcast store; +- no store depends on the order of contacts; +- a store only ever grows during a run (union is monotone); +- garbage frames never panic and never change a store; +- addressed messages reach their recipient along a connected path. + +`tines-sim/tests/relay.rs` adds scripted relay scenarios, including a message +that expires before it reaches its recipient and is never stored. + ## Crate layout ``` tines/ tines-core/ no_std core: Observation, Store / Transport / BeliefPolicy - traits, the exchange state machine, and the `Node` facade. - `std` is optional; `tines-core` builds for `thumbv7em-none-eabihf` - with `--no-default-features`. + traits, the exchange state machine, the `Node` facade, the + in-memory store, and the reference belief policies. + `alloc` is a default feature; without it, tines-core builds + for thumbv7em-none-eabihf (only MemStore needs alloc). tines-sim/ std: the simulator, contact traces, and the convergence / - order-independence property tests. + order-independence / relay property tests. ``` -## CI +Fixed limits, all compile-time constants in `tines-core/src/consts.rs`: -`scripts/ci.sh` runs the full gate — formatting, clippy, the test suite, and the -`no_std` build — and is safe to drop into a CI config: +| Constant | Value | Meaning | +|-------------------------|--------|------------------------------------------| +| `MAX_TOPIC` | 64 | topic bytes | +| `MAX_PAYLOAD` | 1024 | payload bytes | +| `MAX_PEERS` | 8 | peers tracked at once | +| `SUMMARY_BYTES` | 512 | full Bloom summary size | +| `BLOOM_K` | 3 | hash functions per id | +| `SUMMARY_INTERVAL_MS` | 2000 | re-summary period per peer | +| `REASSEMBLY_TIMEOUT_MS` | 5000 | partial observation discard time | +| `SEND_QUEUE_CAP` | 256 | ids queued per peer | +| `MAX_FRAME` | 1500 | largest frame (and MTU) supported | + +## Building and CI + +The toolchain is pinned in `rust-toolchain.toml`. `scripts/ci.sh` runs the full +gate — formatting, clippy and docs (warnings are errors), the test suite with +and without `alloc`, and the `no_std` build for `thumbv7em-none-eabihf`: ```sh sh scripts/ci.sh ``` +GitHub Actions runs the same script on every push to `master` and on every +pull request. + ## Where to read more -The spec (authoritative) and the implementation plan that argues from it: +The specs are authoritative; the plans argue from them. -- **Spec:** `docs/superpowers/specs/2026-10-16-tines-design.md` -- **Plan:** `docs/superpowers/plans/2026-10-16-tines-core-and-sim.md` +- **Core design:** `docs/superpowers/specs/2026-10-16-tines-design.md` +- **Addressed delivery:** `docs/superpowers/specs/2026-10-18-tines-addressed-delivery.md` +- **Plans:** `docs/superpowers/plans/` + +Not yet built (a second plan): `tines-udp`, `tines-sqlite`, `tines-cli`, and +`tines-serial`.