Design doc: tines opportunistic shared memory

2026-10-16 14:43:28 -07:00
commit f73f9247ed
1 changed files with 292 additions and 0 deletions
@@ -0,0 +1,292 @@
+# tines — opportunistic shared memory for mostly-offline robots
+
+Design document. 2026-10-16.
+
+## Intent
+
+Small robots doing their work collect facts about the world. They are
+mostly offline. When two of them happen to be in range, they should swap
+what they know so that each robot's picture of the world improves over
+time. No robot *depends* on receiving anything — it can do its job on its
+own observations — but there is no reason not to share.
+
+`tines` is the library that does this: a store of timestamped
+observations, a belief layer derived from them, and a delay-tolerant
+exchange protocol that runs over whatever radio or link the robot has.
+
+The name is from Vinge's *A Fire Upon the Deep*: the Tines only form a
+mind when close enough to hear each other.
+
+## Goals
+
+- A robot's knowledge is a monotonically growing set of observations;
+  meeting another robot can only add to it.
+- Any two robots holding the same set of observations have identical
+  storage, regardless of the order or path by which they got them.
+- Exchange is opportunistic, session-free, and tolerant of being cut off
+  at any byte. A half-finished exchange leaves both sides valid.
+- The same core code runs on a Jetson-class Linux robot, an MCU-class
+  robot, and inside a simulator, with only the transport and storage
+  adapters differing.
+- The semantics of an observation's payload are opaque to storage and
+  transport. Only the memory layer interprets payloads, and the
+  interpretation can change without touching stored data.
+
+## Non-goals
+
+- Guaranteed or timely delivery to any particular node. There is no
+  "destination"; every node is a sink.
+- Routing, custody transfer, or any multi-hop forwarding logic beyond
+  "I carry what I have and give it to whoever I meet". Classic Bundle
+  Protocol (RFC 9171) routing and custody are explicitly not in scope.
+- Consensus. Robots may hold different beliefs about the same topic;
+  that is acceptable and expected.
+- Security, authentication, or encryption in v1. Observer IDs are
+  asserted, not proven. (Noted as future work; the ID scheme should not
+  preclude signing later.)
+- Defining the observation payload schema. That is the application's
+  problem; `tines` ships a byte-string payload and a topic key.
+
+## Anti-goals
+
+Things that would make this a failure even if it technically works:
+
+- Sync behaviour that depends on contact order (robot A ends up with a
+  different store depending on whether it met B or C first).
+- A core crate that silently requires an allocator, `std`, threads, or a
+  filesystem, making the MCU target an afterthought.
+- A memory layer that must be synced, or whose state cannot be rebuilt
+  from storage alone.
+- Exchange that requires a reliable, ordered, long-lived connection.
+
+## Constraints (decided)
+
+- **Language:** Rust.
+- **Targets:** Jetson-class Linux (primary), MCU-class (`no_std`, likely
+  no allocator), and an in-process simulator for development and test.
+- **Core crate is `no_std`.** `alloc` is an optional feature. Storage and
+  transport are traits; the core contains no I/O.
+- **PHY-agnostic.** The core never sees a radio. It sees a transport trait
+  (below). Adapters exist outside the core.
+- **Observations are opaque, timestamped, observer-attributed, and keyed
+  by topic.** The payload shape is deliberately undecided (one candidate
+  is a timestamped RSO observation; it does not matter to this design).
+
+## Architecture
+
+Three layers, each a separate concern, top to bottom:
+
+```
+ ┌──────────────────────────────────────────┐
+ │ memory      — "my priors": beliefs per   │  derived, local, never synced
+ │               topic, computed from       │
+ │               storage under a policy     │
+ ├──────────────────────────────────────────┤
+ │ storage     — append-only log of raw     │  synced by set-union
+ │               observations; local        │
+ │               trimming policy            │
+ ├──────────────────────────────────────────┤
+ │ comms       — contact-driven exchange    │  PHY-agnostic; transport
+ │               of observations            │  trait + adapters
+ └──────────────────────────────────────────┘
+```
+
+Only observations cross the wire. Memory is a pure function of storage
+plus a policy.
+
+### Observation (the unit of everything)
+
+```
+Observation {
+    id:        ObservationId   // content-derived hash of the fields below
+    observer:  NodeId          // who saw it
+    time:      Timestamp       // when they saw it (observer's clock)
+    topic:     Topic           // key for grouping/trimming, e.g. "door/3"
+    payload:   bytes           // opaque to tines
+}
+```
+
+Decisions:
+
+- `id` is a hash over `(observer, time, topic, payload)`. Two robots that
+  observe the same thing produce two observations; de-duplication is
+  by identity, not by meaning. Union is therefore idempotent and
+  order-independent.
+- `time` is the observer's local clock. `tines` does not attempt clock
+  sync. Cross-observer recency comparison is a memory-layer policy
+  question and must tolerate skew.
+- `topic` is a short byte string. Its structure (hierarchy, separators)
+  is application-defined but `tines` supports prefix matching for
+  trimming and queries.
+- Observations are immutable. There is no update or delete at the
+  protocol level — only local trimming.
+- Fixed upper bounds on `topic` and `payload` length are a core
+  configuration constant so the MCU target can use fixed-size buffers.
+  *(My call: 64 bytes topic, 1 KiB payload default; configurable at
+  compile time.)*
+
+### Storage layer
+
+Responsibilities: hold observations, answer "do you have X", enumerate
+what's held, enforce a local trimming policy.
+
+```
+trait Store {
+    fn insert(&mut self, obs: &Observation) -> Result<Inserted, StoreError>;
+    fn contains(&self, id: &ObservationId) -> bool;
+    fn get(&self, id: &ObservationId) -> Option<Observation>;
+    fn ids(&self) -> impl Iterator<Item = ObservationId>;
+    fn by_topic(&self, prefix: &[u8]) -> impl Iterator<Item = Observation>;
+    fn trim(&mut self, policy: &TrimPolicy) -> usize;
+}
+```
+
+Decisions:
+
+- Storage is append-only from the protocol's point of view. `trim` is
+  the only removal and is purely local.
+- Trimming is policy, not semantics: keep the last N per topic, drop
+  older than T, stay under B bytes. Trimming never merges or rewrites
+  observations.
+- A trimmed observation may be re-received from a peer later. *(My call:
+  accept it; the trim policy will drop it again. Keeping a tombstone set
+  to refuse re-receipt is a possible later optimisation, bounded in size,
+  and must not be required for correctness.)*
+- Implementations: SQLite-backed (Jetson), flash ring buffer or
+  fixed-size table (MCU), in-memory (simulator and tests). The core
+  ships the in-memory one.
+
+### Memory layer
+
+Responsibilities: answer "what do I believe about topic X" by folding the
+relevant observations under a policy.
+
+```
+trait BeliefPolicy<B> {
+    fn fold(&self, me: NodeId, obs: impl Iterator<Item = Observation>) -> B;
+}
+```
+
+Decisions:
+
+- Memory is derived. It can be a cache, but it must be rebuildable from
+  storage at any time and is never transmitted.
+- Policies are pluggable. Examples the application might implement:
+  most recent wins; own observations outrank others'; per-observer trust
+  weights; observation-count thresholds; a confidence that decays with
+  age. `tines` ships a `LatestWins` and a `SelfFirst` policy as
+  references; anything that needs to parse the payload belongs to the
+  application.
+- Memory is where disagreement between observations is handled. Storage
+  never resolves conflicts.
+
+### Communications layer
+
+Responsibilities: notice a peer, find out what the peer has that we lack
+(and vice versa), transfer it, survive being interrupted.
+
+#### Transport trait (the PHY boundary)
+
+```
+trait Transport {
+    type Addr;
+    fn poll(&mut self) -> Option<Event<Self::Addr>>;   // Contact(addr) | Lost(addr) | Frame(addr, bytes)
+    fn send(&mut self, to: &Self::Addr, frame: &[u8]) -> Result<(), SendError>;
+    fn mtu(&self) -> usize;
+}
+```
+
+Decisions:
+
+- Datagram semantics, unreliable, unordered, bounded size. Everything
+  above assumes frames can be lost or duplicated.
+- The core is driven by polling (`poll` / `send`) so it fits both a
+  bare-metal superloop and an async runtime via a thin shim. *(My call:
+  no async in the core.)*
+- Contact detection is the transport's job (beacon heard, link up, sim
+  says "in range"). The core reacts to `Contact` and `Lost`.
+- Adapters planned: UDP broadcast/unicast over ad-hoc Wi-Fi (Jetson),
+  serial-attached LoRa/packet radio, in-process simulator.
+
+#### Exchange protocol
+
+On `Contact(peer)`, both sides run the same symmetric procedure; neither
+is the initiator in any semantic sense.
+
+1. **Summarise.** Send a compact summary of held observation IDs.
+2. **Diff.** On receiving the peer's summary, compute which held IDs the
+   peer appears to lack and send those observations.
+3. **Repeat** while the contact lasts, re-summarising periodically so
+   observations received from a third party mid-contact also propagate.
+
+Decisions:
+
+- Every message is self-contained and idempotent. Receiving the same
+  observation twice is a no-op; receiving a summary out of order is
+  harmless. There is no session state that must survive to keep the
+  store consistent.
+- Summaries must fit in a small number of frames even when the store is
+  large. *(My call: an inverted Bloom filter / IBLT-style structure or a
+  per-topic-prefix hash tree; the builder picks, with the requirement
+  that a false "peer has it" costs only a delayed propagation, never an
+  inconsistency.)*
+- Observations are sent smallest-first or oldest-first? *(My call:
+  prioritise the peer's probable interest if the application supplies a
+  priority function; otherwise newest-first, on the theory that short
+  contacts should carry the freshest data.)*
+- Fragmentation: observations may exceed MTU on narrow radios. The core
+  fragments and reassembles; a partially received observation is
+  discarded, never stored.
+- Rate: the core emits at most one frame per `poll` so the application
+  controls pacing and duty cycle.
+
+#### Simulator
+
+A `std` crate wrapping the core with:
+
+- N nodes, each with its own store and clock skew.
+- A contact model: positions and ranges over time, or an explicit
+  contact-trace file.
+- A loss/duplication model per link.
+- Assertions: after the trace, every pair of nodes that were ever
+  transitively connected hold the same store (modulo trimming); no node's
+  store depends on contact order.
+
+The simulator is also the test harness for the protocol.
+
+## Crate layout (my call)
+
+```
+tines/
+  tines-core/      no_std; Observation, Store trait, Transport trait,
+                   BeliefPolicy trait, exchange state machine,
+                   in-memory Store, reference policies
+  tines-sim/       std; simulator, contact traces, property tests
+  tines-udp/       std; UDP adapter (Jetson)
+  tines-serial/    no_std-compatible; framed serial adapter
+  tines-sqlite/    std; SQLite Store
+  tines-cli/       std; inspect a store, replay a trace, run a node
+```
+
+## Left to the builder
+
+- Hash function for `ObservationId` (should be cheap on MCU; BLAKE2s or
+  SipHash-128 are reasonable).
+- Summary data structure and its parameters.
+- Wire encoding (postcard or a hand-rolled fixed layout; must be
+  `no_std`).
+- Fragment header layout and reassembly timeout.
+- SQLite schema.
+- MCU flash store layout (can be a stub in v1 as long as the trait
+  boundary is honoured).
+
+## Open questions (not blocking)
+
+- Payload schema. Deliberately out of scope; an RSO observation is one
+  candidate.
+- Whether topics should carry a schema/version tag so the memory layer
+  can skip payloads it cannot parse. *(Leaning yes, as a convention in
+  the topic string rather than a protocol field.)*
+- Observer authentication (signing). Out of scope for v1; the ID hash
+  should be chosen so a signature can be appended without changing the
+  ID.