# Current state

Last updated 2026-10-04. `HANDOFF.md` is the brief; this file tracks where the work stands.

## Hardware on hand

Raspberry Pi 4 (8 GB), BrainCraft HAT, Arducam UC-346 camera, and nothing else: **no servos, no
fan, no speakers**, no USB-C splitter, no printed enclosure (the camera hangs loose on its ribbon and is
held by hand). The Pi runs at about 62 C open-air with everything on and no fan; an enclosure will be
hotter, so fit the fan before enclosing it. Playback and the servos are untested. Work until those arrive is limited to what the Pi, HAT and camera can do.

## The Pi (`hermes-bot`)

- Ubuntu 24.04.5 server, kernel `6.8.0-1065-raspi` (fully upgraded), flashed from `talos`. User
  `hermes` (key-only; passwordless sudo), SSH keys for `kyle@vade` (talos) and `kyle@straylight`.
- On Wi-Fi at `192.168.88.152` (MAC `d8:3a:dd:0d:66:f5`); `eth0` is not cabled. The owner says it has
  a static IP on the rack switch; that address is not recorded here yet. **USB link: `10.55.0.2`**
  (see package 0b below); `ssh hermes@10.55.0.2` works from talos. `avahi-daemon` is
  installed, but mDNS is off in `systemd-resolved` on talos, so `hermes-bot.local` doesn't resolve
  there; use the IP.
- **cloud-init applies `network-config` and `user-data` on the first boot only.** To change Wi-Fi
  later, edit `/etc/netplan/50-cloud-init.yaml` on the Pi, or bump `instance_id` in `meta-data` on
  `system-boot`. `pi/50-cloud-init.yaml` is a template for the Wi-Fi block; it is gitignored
  because it holds credentials.
- Installed: ROS 2 Jazzy (`ros-base`, `ros-dev-tools`, colcon, `robot_state_publisher`, `xacro`,
  `camera_ros`); `python3-numpy/pil/spidev/lgpio/rpi-lgpio`; a venv `~/venv` (system site packages
  visible) with `adafruit-circuitpython-rgb-display` and Blinka. Workspace: `~/ros_ws`, source synced
  from `ros/` with `rsync -a --delete --exclude __pycache__ ros/ hermes@<ip>:~/ros_ws/src/`, then
  `colcon build --symlink-install`.
- **One unit runs the whole robot:** `robot.service` runs `ros2 launch hermes_bringup robot.launch.py`
  as `hermes` (face, camera, face detection, audio levels), so everything is ROS and the face is up
  whenever the robot is powered; each node is its own process, respawned by the launch file. Install
  with `pi/install-robot-service.sh`. `RuntimeDirectory=robot` gives lgpio a writable working
  directory (it drops a pipe in its cwd; systemd's default `/` is not writable). The face starts
  first and the camera, vision and audio nodes follow `start_delay` (6 s) later, so the Python
  processes do not fight over the CPU. Parameters are in `ros/hermes_bringup/config/robot.yaml`.
  `robot-cpu-performance.service` holds the governor at `performance` (the face costs about half the
  CPU at full clock). `pi/trim-boot.sh` disables snapd and ModemManager (unused; ~9 s of boot CPU).
- **Boot timing** (reboot-tested, everything returns by itself): the face draws at about 28 s on the
  boot clock (kernel 6.9 s, `robot.service` at ~10 s into userspace, then `ros2 launch` takes ~10 s
  to start on a cold Pi, then the face ~3 s); camera, audio and vision are up by ~37 s. The owner is
  fine with up to a minute, and prefers everything in ROS to a faster non-ROS face; a split
  `robot-face.service` that skipped the launch framework only saved 3.7 s and was dropped.
- **Steady-state CPU** with everything running (4 cores): face ~76%, camera ~57% (15 fps, from 30
  at ~40% alone), detector ~40% (was 131%), audio ~12%; about half the machine idle, 62 C, no
  throttling, no fan fitted. Camera `FrameDurationLimits` is the lever (15 fps ~20% alone, 10 fps
  ~14%); JPEG quality barely matters. OpenCV's DNN spins every core and burns 3-4x its elapsed
  time in CPU, so the detector node pins it to one thread (YuNet 86 ms, SFace 161 ms of CPU), decodes
  the JPEG at half size (`IMREAD_REDUCED_COLOR_2`, boxes scaled back to full-size pixels) and
  re-identifies a tracked face at most once a second (`IdentityCache`).
- `ros2 node list` with its default short discovery window often shows only some of the nodes; use
  `ros2 node list --no-daemon --spin-time 6` (all four appear), or check topics.
- **ROS never leaves the Pi** (owner's decision): `ROS_AUTOMATIC_DISCOVERY_RANGE=LOCALHOST` (set in
  `pi/robot-env.sh`, which `source`s everything) and the firewall drops domain 42's DDS ports
  (17900-18149) on every interface but loopback. `hermes_mcp` is the only bridge to the outside; a
  host that wants data from the robot gets it through an MCP tool, not DDS. This supersedes
  HANDOFF.md's `host/ros-shell` talking to the robot. (ROS's setup script exports `SUBNET`, so the
  variable must be assigned, not defaulted.)
- **PipeWire** (1.0.5, WirePlumber 0.4.17) runs as `hermes`'s user services (`loginctl enable-linger`).
  `~/.config/pipewire/pipewire-pulse.conf.d/50-network-server.conf` (from `pi/pipewire/`) makes
  pipewire-pulse listen on TCP 4713 on all interfaces; the firewall is the protection. Defaults are
  the WM8960 source and sink (`alsa_input/output.platform-soc_sound.stereo-fallback`). Verified from
  talos: `pactl -s tcp:10.55.0.2:4713 info` works; the same on the Wi-Fi address times out.

## Package 0a (platform proof on Ubuntu 24.04)

- Step 1 done: `host/check-pi-platform.sh` passes.
- Step 2 done: flashed, booted, upgraded; ROS installed; `ros2 topic list` works.
- **Step 5 (display) done.** ST7789 works on `lgpio` through Blinka (`rpi-lgpio`, no `RPi.GPIO`).
  Two gotchas: pass `cs=None` (the kernel drives CE0; claiming GPIO8 raises "GPIO busy"), and the
  240x240 glass sits at `y_offset=80` in the controller's RAM, for every rotation (the driver
  rotates in software and leaves the scan direction fixed). Mounted pose (microSD end up, USB-C edge
  on the left) needs content turned 90 degrees clockwise: `rotation=270`. Full-frame push is
  ~23 fps through the driver; the face pushes raw RGB565 with `_block` and reaches ~27-28 fps.
- **Camera: works with the Raspberry Pi libcamera fork, not with the stock stacks.** The OV5647 is
  detected (`ov5647 10-0036`) and Ubuntu's libcamera 0.2.0 lists it once `libcamera-ipa` is
  installed, but capture dies in the Pi IPA (`assertion "it != buffers_.end()" in prepareIsp()`),
  and ROS Jazzy's libcamera 0.7.2 dies with `FATAL … A list of V4L2 controls requires a
  ControlInfoMap` right after `configure`. Both are deterministic whatever the stream format or
  size. Fix: build the fork (`github.com/raspberrypi/libcamera`, commit `6c1dd9d`, 2026-08-17) with
  `-Dpipelines=rpi/vc4 -Dipas=rpi/vc4 -Dcam=enabled -Dv4l2=false …` into `/opt/libcamera-rpi`; its
  `cam` then captures 1296x972 RGB888 at ~16 fps (dim room). `pi/capture_still.py` takes a still
  with it. **`camera_ros` still has to be built against this fork** (the apt one uses ROS's broken
  libcamera). Frames are padded (stride 3936) and B,G,R in memory.
- **`camera_ros` 0.7.0 is built from source against the fork** in `~/camera_ws` (overlay on ROS; at
  runtime the fork must come first in `LD_LIBRARY_PATH`, which `source ~/robot-env.sh` does; copy
  from `pi/robot-env.sh`). Verified: `ros2 run camera_ros camera_node --ros-args -p width:=1296 -p
  height:=972 -p format:=RGB888` publishes `/camera/image_raw`, `/camera/image_raw/compressed` (~26
  Hz) and `/camera/camera_info`. It costs ~140% of a core with raw and compressed both on; lower the
  frame rate or size before running everything together. No calibration file yet.
- **`ros/hermes_vision`**: `hermes_vision.detect` (YuNet, Haar fallback; no ROS imports) and a node
  `face_detector` that subscribes to the compressed image and publishes `/vision/faces`
  (`vision_msgs/Detection2DArray`, bbox centre/size in pixels, `class_id: face`, score), at `rate_hz`
  (default 4, ~47% of a core on a Pi 4); `publish_debug:=true` adds an annotated image. CLI:
  `python -m hermes_vision.detect_cli IMAGE`. YuNet 2023mar needs OpenCV 4.8+, so `~/venv` has
  `opencv-python-headless==4.10.0.84` with `numpy<2` (system NumPy stays 1.26.4 for ROS), and
  `robot-env.sh` puts the venv first on `PYTHONPATH`; YuNet also needs input sides that are multiples
  of 32. Tested on a sample image and on a live frame (one face found, upright, no flip needed).
- **Audio capture works (verified 2026-10-04).** The stock `wm8960-soundcard` overlay fails every
  stream with `wm8960: No MCLK configured` on Ubuntu 24.04: it puts the 12.288 MHz fixed clock on the
  sound card's codec link, but the codec driver reads `mclk` from its own node.
  `pi/overlays/hermes-wm8960.dts` is the stock overlay plus `clocks`/`clock-names` on the codec node;
  `pi/install-audio-overlay.sh` compiles it to `/boot/firmware/overlays/hermes-wm8960.dtbo`, sets
  `dtparam=audio=off` and `dtoverlay=hermes-wm8960`. Capture then opens at 8-48 kHz. Address the
  card by name (`hw:CARD=wm8960soundcard,DEV=0`): its index changes (it was card 1, then 0).
- **The HAT's hardware Audio ON/OFF slide switch (right edge, by the "Audio" label) must be ON.**
  With it off, the codec still answers on I2C and every software check looks right, but all
  recordings are exactly zero. That cost an hour of debugging; check it first.
- Mixer: `Left/Right Input Mixer Boost Switch` default to off and must be on. Settings in use:
  Capture 47/47 (+18 dB), `Left/Right Input Boost Mixer LINPUT1/RINPUT1` volume 1, input mixer boost
  on. Singing at arm's length: RMS -34 to -49 dBFS, peaks -18 dBFS, no clipping, channels within
  1-2 dB. Saved with `alsactl store` and exported as `pi/wm8960_asound.state`; restoring at boot is
  not yet verified. No speakers on hand, so playback is untested.
- Not done: PWM (waits for servos). Ask before rebooting the Pi.

## Package 0b (USB link): Pi side and talos side working

- Pi: `dtoverlay=dwc2,dr_mode=peripheral`; `robot-usb-gadget.service` builds a configfs gadget
  (NCM `usb0` + ACM `ttyGS0`, host MAC `02:68:65:72:6d:01`, robot MAC `02:68:65:72:6d:02`) before
  networking; `serial-getty@ttyGS0` is enabled; `/etc/netplan/60-usb0.yaml` gives `usb0`
  `10.55.0.2/30` with no gateway, so the default route stays on Wi-Fi. Installed by
  `pi/install-usb-gadget.sh` (a backup of `config.txt` is `config.txt.bak-usbgadget`). NCM works on
  the Pi 4 with kernel 1065; ECM was not needed.
- **netplan gotcha:** match the gadget interface by `name: usb0`. A `macaddress:` match becomes
  `PermanentMACAddress=` in networkd, which a gadget interface lacks, so it stays unmanaged.
- talos (Omarchy, NetworkManager): `host/omarchy/robot-usb-link.sh` (needs sudo) adds profile
  `hermes-robot-usb`: `10.55.0.1/30` on the interface with MAC `02:68:65:72:6d:01`, priority 100,
  never-default. Verified from talos: ping ~2.4 ms, SSH over the link.
- **Firewall:** `robot-firewall.service` loads `pi/nftables-robot.nft` before networking at every
  boot (`pi/install-firewall.sh`; it must be loaded before the PipeWire network server is enabled).
  It accepts 4713/8765/8766 only on `usb0` from `10.55.0.1`, drops them elsewhere, and drops DDS on
  every interface but loopback. Verified: from talos, 4713 connects over `usb0` and times out over
  Wi-Fi, while an unused port on the Pi's Wi-Fi address refuses instantly (so only 4713 is blocked).
  SSH is deliberately left open on all interfaces (key-only), for V2; the owner's call.
- Not yet verified: serial login prompt (talos's `/dev/ttyACM0` is `root:uucp`; use
  `sudo screen /dev/ttyACM0 115200`). The `hermes` account has no password, so the console shows a
  login prompt but cannot log in until the owner sets one; no autologin by design. Also not done:
  the Pi booting with Wi-Fi off, replug and second-host recovery,
  `get_throttled` after a servo sweep on the splitter, the NixOS snippet, and straylight/other-host
  testing. Currently the Pi is powered from talos's USB port (bench test, no servos).

## Face (`ros/hermes_face`)

- `render.py`: numpy port of `docs/face-prototype.html`, **flat face only** (head and lowpoly not
  ported). Phosphor and vector styles. States: `sleeping idle listening thinking looking speaking`.
  `sleeping` (new) is the "no agent connected" pose: eyes closed to thin lines, mouth short, dim and
  slowly breathing. `looking` returns to the state it interrupted. Tone mapping is a lookup table
  (RGB and RGB565 tables), glow is computed at half resolution, rotation is a free `np.rot90`.
- `display.py`: ST7789 driver plus an async pusher (render overlaps the SPI transfer).
- `face_node.py`: node `hermes_face`; subscribes `/hermes/state` (`std_msgs/String`) and
  `/joint_states` (`pan_joint`, `tilt_joint`, radians); starts in `initial_state` (`sleeping`), so it
  runs with nothing else up. Parameters: `rotation` (270), `x_offset` (0), `y_offset` (80),
  `baudrate`, `fps`, `output` (`st7789` or `none`), `phosphor` (P7), `style`, `persistence_ms`,
  `focus`, `jitter`, `intensity`, `initial_state`.
- `demo.py`: no ROS. `python -m hermes_face.demo --sequence` cycles the states on the panel;
  `--png-dir DIR` writes a PNG per state without hardware.
- Tests: `python3 -m pytest test` in `ros/hermes_face` (25 pass on the Pi). Renderer parameters
  (`focus`, `persistence_ms`, `intensity`, `jitter`) are clamped; `fps` is clamped to 1-60.
- Measured on the Pi 4: ~9 ms per frame in isolation. With the CPU at full clock: 24% of a core at
  20 fps and 40% at 30 fps (target was under 40% at 20 fps or more). Under `ondemand` the same
  load reads 40% at 20 fps and ~67% at 30 fps. No C++ rewrite was needed.
- **Gaze:** the node subscribes to `/vision/faces` (and `/camera/camera_info` for the image size)
  and the eyes glance toward the best face: a recognised one first, then the biggest. Image x flips
  on the display (a person on the image's right is on the robot's right, the viewer's left); the
  glance is smoothed, held 2.5 s, then drifts back to centre. Sleeping ignores it. The pan sign in
  the renderer was wrong (carried over from the prototype) and is fixed: positive pan = the robot's
  left = the viewer's right, positive tilt = looks up, as the MCP `turn_head` tool documents.
  **Confirmed by the owner on the real display (2026-10-04):** the glance directions look right.
  An earlier run showed nothing because the one-off wake message was lost (volatile QoS drops a
  sample published before discovery matches); test publishers must republish, as `hermes_mcp`
  will. The node logs every state change (`state: idle (agent=..., audio=...)`), so behaviour can be
  checked from `journalctl -u robot-face` without watching the panel.
- **Agent heartbeat:** `/hermes/state` doubles as the agent's heartbeat. `hermes_mcp` must republish
  its state about once a second; after `agent_timeout_s` (default 10, 0 disables) with none, the
  face logs `agent gone, sleeping` and sleeps, so "sleeping" means "no agent connected". Verified:
  woke on messages, slept 10 s after the last.
- Not yet: audio levels for the mouth (`/audio/levels`), `hermes_mcp` publishing states, launch
  file in `hermes_bringup`, head/lowpoly shapes, DotStar LEDs, button mute.
- Open: IPS black level. The backlight is on at full brightness, so black reads dark gray on
  camera. Dimming it needs a software PWM on GPIO26 (not a hardware PWM pin).

## Audio levels and the literal voice trace (`ros/hermes_audio`, face integration)

- `hermes_audio` node `audio_state` taps the default source and the default sink's monitor with
  `parec` (16 kHz mono) and publishes `/audio/levels` ([mic_dbfs, speaker_dbfs]), `/audio/activity`
  (`speaking` | `listening` | `quiet`) and trigger-locked waveform windows `/audio/scope/{mic,speaker}`
  (256 samples). Audio itself never goes through ROS. `hermes_audio.levels.Activity` debounces and
  keeps an adaptive noise floor: a 3 s start-up calibration, then it drops fast and rises slowly, and
  anything 'listening' without a gap for 8 s is absorbed as ambient noise. The microphone is ignored
  while speaking (it would hear the robot).
- The face arbitrates (`hermes_face.arbiter`): sleeping (agent side only) > speaking > listening >
  thinking > idle. **Room sound never wakes a sleeping face**: sleeping means "no agent connected".
  `looking` stays a timed event owned by the renderer. While fresh (< 0.35 s) scope data exists, the
  mouth draws the real waveform (speaker when speaking, mic when listening), auto-scaled.
- **WirePlumber resets the codec's capture gain** to 100% (+30 dB) when it takes the device,
  overriding `alsactl`'s saved level. Fix: `pactl set-source-volume @DEFAULT_SOURCE@ 63%` (PipeWire's
  scale is cubic: 63% is -12 dB from the +30 dB maximum, so ALSA `Capture` lands at 46, about +17
  dB). WirePlumber remembers it; recheck after a reboot. The quiet-room floor at this gain is about
  -43 dBFS (preamp and mic noise).
- **Confirmed by the owner:** the mouth draws their live voice and its size follows how close and
  loud they are (proportional). The activity detector was retuned after the log showed the face
  flickering between listening and idle about twice a second: margin 7 dB (quiet-room noise peaks
  sit at about +3 dB over the floor), hold 1.2 s so pauses between words do not drop it. The
  flicker fix is covered by unit tests but not yet re-confirmed live (the retest window was silent).
- Verified on the Pi: the `speaking` path (a tone played to the sink, with no speaker attached, shows
  on the monitor at -13.5 dBFS, `speaking` for its length, mic ignored); levels and scope publish;
  49 face tests, 10 audio tests. **Not yet verified:** the audio node as a service (it was run as a
  transient unit).

## The MCP bridge (`ros/hermes_mcp`)

The only way in or out of the ROS graph. Serves MCP (SDK 2.3, streamable HTTP) on
`http://10.55.0.2:8765/mcp`, the USB address only (retrying the bind if the link is not up yet), with
a bearer token. **Tool names are the stable contract with Hermes**: `look()`, `faces()`, `list_people()`, and `turn_head` /
`head_position` only when a head is fitted. With no servos those two are not registered, by design
(`head: false`; the node refuses `head: true` until `hermes_head` exists). `look()` waits for the next
camera frame taken after the call (never a stale one) and passes the camera's JPEG through (1296x972,
~155 KB); it also publishes `looking` on `/hermes/state` so the face squints and flashes.
While an authenticated client is connected (in flight, or heard within `connected_grace_s` = 30) it
republishes `idle` on `/hermes/state` once a second: the agent heartbeat that wakes the face. Verified
end to end from talos over the USB link with a real MCP client: connects, lists `look`, `look()`
returns a JPEG in 0.08 s, a wrong token is refused, the Wi-Fi path is dropped by the firewall, and
the face went `looking` -> `idle` on connect and `agent gone, sleeping` about 40 s after the client
left. The token is in `/etc/robot-mcp.env` (root:hermes, mode 640), generated by
`pi/install-robot-service.sh` and never printed; read it with `sudo cat`. The MCP SDK is in `~/venv`
(`pip install "mcp>=2"`). The old Pi Zero files (`pi/robot_mcp_server.py`, `pi/robot-mcp.service`,
`pi/requirements.txt`, the old `pi/install.sh`, and `tools/`) were deleted (git has them). **Not yet verified:** a real Hermes client, and
whether Hermes passes MCP image content to the model (HANDOFF package 6).

**Hermes on talos is wired to it (2026-10-05, with the owner's authorisation).** Hermes Agent v0.21.5 is
installed on talos (`~/.hermes`; hosted main model `grok-4.6` via xai-oauth; vision is auxiliary and
runs locally, `qwen3-vl-8b-abliterated` on straylight; the `ornith-*` models are the local ones on
straylight/dixie). `~/.hermes/config.yaml` gained an `mcp_servers.robot` block (URL
`http://10.55.0.2:8765/mcp`, `Authorization: Bearer ${ROBOT_MCP_TOKEN}`) and `~/.hermes/.env` gained
`ROBOT_MCP_TOKEN`; backups are `config.yaml.bak-20261005T002659-robot-mcp` and
`.env.bak-20261005T002659-robot-mcp`. `hermes mcp list` shows it enabled and **`hermes mcp test
robot` passes: connected in 789 ms, 1 tool (`look`)**, which is the HANDOFF acceptance check. A fresh
`HERMES_HOME` hangs on first-run setup, so test against the real one.

**A real agent turn works, entirely on local models (2026-10-05):** `hermes chat -q "Call the look tool
once ... what you see" --oneshot -Q --provider custom:llama-router -m ornith-1.5-35b-a3b --max-turns 4`
(straylight's router; no hosted provider, so the image stayed on the LAN). Hermes called
`mcp__robot__look` (0.07 s), the model described the live photo correctly (blurry close-up of a board
with blue LEDs, a red label, camouflage fabric), and the robot's face went `looking` -> `idle`.
**Hermes does not hand MCP image content to the main model.** It saves the JPEG to
`~/.hermes/cache/images/img_<hash>.jpg` on talos (157,147 bytes, the same as the robot sent) and
returns a 72-character reference; the model then calls Hermes's `vision_analyze`, which ran on the local
vision model (`qwen3-vl-8b-abliterated` on straylight) in 29 s. That is HANDOFF package 6's fallback and
Hermes does it itself, so `look()` needs no change. Two consequences: each look() leaves a photo of the
room in talos's Hermes image cache (Hermes's retention, not the robot's), and a look-and-describe turn
takes ~70 s on the local models, mostly the vision call.

### Hermes's feedback on the bridge (2026-10-05) and what was done

Hermes (on talos, from `~/.hermes/cache/scratch/hermes-bot-v0-face-identity.md`) asked for recognitions
instead of raw photos. **Built:** `faces()` returns the faces in view as structured JSON with no photo
(`track_id`, pixel and normalised `bbox`, detection `confidence`, `seen_for_s`, and `identity`
`{name, similarity}` or `null`); `look(send_image="always"|"never")` now returns that same JSON as a
text block plus the photo (default kept as before: the image, so scene description still works);
`list_people()` gives names and sample counts; a `robot://status` resource (hardware present or
absent, what is detected, enrolled count, tools) and prompts `who_is_here` and `look_around` fill the
previously empty `list_resources` / `list_prompts`; `/vision/faces` carries a persistent track id
(`hermes_vision.tracking.FaceTracker`, matching by box overlap, identity re-checked once a second;
`Detection2D.id` is the track id, `results[0]` the face, `results[1]` the identity if recognised).
Verified with a real Hermes turn on the local `ornith` model: it called `faces()` (0.03 s, 593
characters, no image, no vision call) and answered that Kyle was there at 0.78 similarity.
**Declined, deliberately:** embeddings in responses (biometric templates, meaningless to an LLM,
and the registry stays on the robot); age/gender/hat attributes (no model for them here, and guessing
age and gender from faces is unreliable and sensitive); auto-updating a person's face data from new
captures (it lets a stranger slowly become "Kyle" with nobody deciding that); a remote `enroll` tool
(a prompt could make the robot call anyone "Kyle": enrolment stays the owner's step on the robot,
`python3 -m hermes_vision.people enroll NAME`; a gated enrol tool, off by default, is possible if the
owner wants it); detecting people, objects or motion (only faces exist, and `robot://status` says so).
`look()`'s default was not flipped to JSON-only, because the image is what lets Hermes describe a
scene; that default is the owner's call.

### Hermes bug report: "faces and list_people are not registered" (2026-10-05) — not a bridge bug

Checked, not assumed: a raw MCP client lists `faces`, `look`, `list_people`; `hermes mcp test robot`
(fresh connection) discovers all three; there is no tool filter in `~/.hermes/config.yaml`. Hermes's own
`agent.log` explains the report: my robot restart at 01:17 ended the MCP session (`keepalive failed ...
Session not found`), Hermes reconnected and logged `registered 7 tool(s): mcp__robot__faces, look,
list_people, ...` at 01:18:46, and a session that started afterwards (`...011845...`) called
`mcp__robot__faces` fine. The reporting session (`...003532...`) started at 00:35, before `faces`
existed, and keeps its start-time snapshot of the tool table: it still reaches live things
(`list_prompts`, `read_resource`) but gets `'mcp__robot__faces' is not a known tool name`. Fix on the
owner's side: start a new Hermes session after the robot's tool set changes. For Hermes: refresh a
session's tool table (and its deferred-tool index) when the MCP server is re-registered. We keep the
bridge's stateful sessions on purpose: the `Session not found` after a restart is what makes Hermes
reconnect and re-register (stateless HTTP would hide a changed tool set from a connected client).

## Face recognition (opt-in)

`hermes_vision.recognize` / `people`: SFace embeddings on YuNet's landmarks. Nothing is stored until
someone is enrolled (`python -m hermes_vision.people enroll NAME`, live from the camera topic, or
`enroll-images`); only 128-number embeddings are kept, in `~/.local/share/hermes/faces.json` (mode
0600, directory 0700, written atomically), never images. `list`, `forget NAME` and `check IMAGE`
manage and test it. The detector node puts a recognised name in `Detection2D.id` and picks up
database changes without a restart; its log prints a count, not names. Checked with a throwaway
database: 8 of 9 unenrolled frames of the same person were recognised, down to side profiles
(similarity 0.52-0.66), the miss being a head tipped back (0.34 against the 0.363 threshold); an
unrelated photo scored 0.16. **Nobody is enrolled yet**; the owner enrols themselves live.

## Next

1. **Camera calibration: tool built and tested, not yet run.** `python3 -m hermes_vision.calibrate
   --cols 9 --rows 6 --square-mm <measured>` (after `source ~/robot-env.sh`, robot running) collects 20
   varied views of a checkerboard (`docs/checkerboard-9x6.png` can be shown on a screen; measure a
   square), solves with OpenCV and writes the camera_info YAML camera_ros looks for
   (`~/.ros/camera_info/ov5647__base_soc_i2c0mux_i2c_1_ov5647_36_1296x972.yaml`: the name includes the
   resolution). The solver recovers a known synthetic camera to within 1%. It needs the owner holding a
   board in front of the camera, and the lens focus set first (the lens turns if not glued; earlier
   close-range frames were blurry). V2's perception needs the result.
2. Waiting on hardware: the servos (see "Arrival day" above), the speakers
   (playback, echo cancel, the `robot_mic` naming on the host, Hermes talking aloud), the fan (before
   any enclosure), the splitter, the printed enclosure (measure first, see `enclosure/README.md`).
3. Still open: replug and second-host recovery of the USB link, the serial console login, an
   `agent_timeout_s` check against a long idle Hermes session, the retuned listening detector live
   (unit-tested only), a second person in frame, running `pi/install.sh` on a blank card, and the
   NixOS host config (skipped for now by the owner).
4. A new Omarchy host with a GPU (it runs the local `ornith` model) is arriving: it needs only
   `host/omarchy/robot-usb-link.sh` and its SSH key added to the Pi.

## Arrival day: the neck (servos), planned and deliberately not built ahead of the hardware

The servos are expected Thursday or Friday at the earliest. The real unknowns cannot be faked (which
`pwmchip` the overlay creates, its permissions on Ubuntu, each servo's real pulse range, whether the
servos brown out the Pi), so `hermes_head` waits for the hardware. The order, with what to verify at
each step:

1. **Wire it:** pan servo to the HAT's GPIO12 port, tilt to GPIO13; the "3pin Volt" jumper is already
   verified at 5 V. Power matters: servo spikes on a host USB port may brown the Pi out, so use the
   5.1 V 3 A supply through the splitter if it has arrived. Horns off for the first moves.
2. **Overlay (needs a reboot, ask first):** add `dtoverlay=pwm-2chan,pin=12,func=4,pin2=13,func2=4` under
   `[all]` in `/boot/firmware/config.txt` (`dtparam=audio=off` is already set; the Pi's headphone audio
   shares PWM hardware). Add it to `pi/install-*.sh`, not by hand.
3. **sysfs:** find the chip (`/sys/class/pwm/pwmchip*`; it was not assumed to be 0), export channels 0
   and 1, `period=20000000`. Add a udev rule that gives a group (not root) write access, as Ubuntu's
   defaults differ from Raspberry Pi OS.
4. **Measure each servo:** centre (1.5 ms), then find the safe min and max pulse with the horn off; set
   `ROBOT_SERVO_MIN_US` / `MAX_US` per servo (start 500 / 2400), the pan and tilt trims and invert flags.
5. **Write `hermes_head`:** `PwmHead` on sysfs, a ~0.3 s interpolated move, PWM off after ~1 s idle (stops
   hold-jitter), limits (pan 80, tilt 45), `/joint_states` for `pan_joint` / `tilt_joint` in radians.
   Convention (the MCP contract): positive pan = the robot's left, positive tilt = up. Unit-test the
   logic against a fake sysfs tree, then verify on the Pi: `vcgencmd get_throttled` must stay `0x0`
   through a full sweep while the face is running.
6. **Wire it in:** the bridge refuses `head: true` today. Give it a head backend that calls
   `hermes_head`, set `head: true` in `robot.yaml`, and check `turn_head` / `head_position` appear and
   that a new Hermes session sees them. Check the face's eyes follow the real joints the right way
   (the sign was fixed in software, never seen with a moving head).
7. **URDF:** `hermes_description` from the enclosure STLs' `pose_*` parts, frames `base_link`,
   `pan_link`, `tilt_link`, `camera_link`, `camera_optical_frame`, so `/tf` follows the head.
8. **Mounted camera:** set the flip (`orientation` parameter) for how it is actually mounted, re-check
   the focus (a changed focus invalidates `pi/calibration/`), and set `ROBOT_FACE_ROTATION` if the
   display ends up mounted differently from the portrait pose.

## Repository

Branch `master`. `.gitignore` covers `.remember/`, caches, `.venv/`, `*.orig`, `*.rej` and
`pi/50-cloud-init.yaml`. Layout: `ros/` (five packages), `pi/` (everything installed on the Pi),
`host/` (the owner's machines: `host/omarchy/` has the USB link script, Hermes and PipeWire config and
the launcher; `host/check-pi-platform.sh`), `enclosure/`, `docs/face-prototype.html`. `README.md` is
rewritten for this build. **`pi/install.sh` is the one installer** (re-runnable: run twice on the live
robot, the second run changed nothing, exit 0): it installs ROS, builds the libcamera fork (pinned
commit) and `camera_ros`, sets up the venv, models, workspace, USB gadget, audio overlay, firewall,
PipeWire, mixer and services. It has not been run on a blank card (no spare card). HANDOFF.md is the
original brief and has drifted; where they differ, this file wins.

## Security notes (all resolved)

The automated review of the initial commit flagged three items, all from the Pi Zero design:
`ROBOT_MCP_BIND` defaulting to `0.0.0.0` (now the USB address only, with a bearer token), PipeWire on
TCP 4713 on every interface (now protected by the boot-time firewall, which loads first), and the
single-IP any-interface nftables file (rewritten: usb0 from 10.55.0.1 only). Later reviews added:
DDS is blocked on every interface and discovery is local-only; recognised names are not logged; node
parameters are clamped. SSH stays open on all interfaces (key-only) by the owner's choice, for V2.