hermes-bot/HANDOFF.md

# Hermes robot: Pi 4 + BrainCraft build — implementation handoff

This document is the brief for finishing the robot. It is written to be worked through by
Claude Code with the owner at hand for the physical steps. Read it in full before changing
anything; then work the packages in order, since later ones depend on earlier ones.

**Platform decision:** the robot's software is a **ROS 2 Jazzy** system from V1 onward, running
natively on **Ubuntu 24.04** on the Pi, so that the mobile V2 (section 10) grows out of it rather
than replacing it. The owner has used ROS extensively and wants it. Package 0a proves the
hardware on Ubuntu before anything else is built.

## 1. What this is

A small desk robot that acts as the eyes, ears, voice and face of **Hermes Agent** (Nous
Research's open-source agent), which runs on the owner's host machine (**NixOS** or **Omarchy**,
an Arch + Hyprland setup). The robot does no reasoning. It:

- serves photos and head movement to Hermes as MCP tools, over a USB-C cable that carries both
  power and a private network link (Wi-Fi remains as a fallback);
- exposes its microphones and speakers to the host as PipeWire audio devices;
- shows an animated oscilloscope-style face on its display, reflecting what the agent is doing.

The repository already contains a working design for an earlier variant (Pi Zero 2 W, separate
I²S mic and amp boards, PCA9685 servo board). That code is the starting point. **This document
supersedes `README.md` wherever they differ**; `README.md` describes the Zero build and should be
rewritten for this hardware at the end (package 7).

## 2. Hardware actually on hand

| Part | Details that matter |
|---|---|
| Raspberry Pi 4 | In a case. **Ubuntu Server 24.04 (arm64) with ROS 2 Jazzy installed natively.** Fallback if package 0a fails: Raspberry Pi OS with ROS in a container. |
| Adafruit BrainCraft HAT | Display: 1.54″ ST7789, 240×240, SPI CE0, DC = GPIO25, backlight = GPIO26. Audio: **WM8960** codec at I²C `0x1A`, I²S on GPIO 18/19/20/21, stereo mics (L/R), stereo 8 Ω 1 W speaker JSTs, headphone jack, hardware audio on/off switch. 3-pin JST ports on **GPIO12** and **GPIO13**, with a 3V/5V solder jumper on the back ("3pin Volt"). STEMMA QT (I²C). Fan header on GPIO4. 5-way joystick, one button, three DotStar LEDs. |
| Arducam UC-346 | OV5647 5 MP CSI camera, fixed focus, 15-pin ribbon. Natively supported by libcamera. |
| Pan/tilt | Two **MG90S** metal-gear servos. Pan turns the whole cabinet on a 6810 bearing turntable (servo in the base); tilt moves only the camera, in a yoke on top. See `enclosure/`. |
| Speakers | 8 Ω, ≤1 W, need JST PH 2-pin plugs for the HAT. |
| USB-C power/data splitter | GODIYMODULES 30 × 25 mm board, PiKVM-style (ports RPI4 / USB / PWR, φ2.75 mm holes): power from the 5.1 V 3 A supply, data only to the host. Mounted in the base's connector block; a ~30 cm USB-C cable (right-angle at the Pi end) links it to the Pi. |
| Host computers | Either a **Framework Desktop (Strix Halo)** or an **ASUS ROG Flow Z13**, each running **NixOS or Omarchy (Arch)**. The robot must work with whichever it's plugged into. Hermes runs natively on the host; **any ROS software on a host runs in a podman container** (owner's rule, so both distros behave the same). |

### Physical steps for the owner (not code)

1. ~~Set the "3pin Volt" jumper to 5 V~~ **Verified by the owner:** the jumper is factory-set to
   5 V (continuity: middle pad to 5V pad only). The GPIO12/13 ports already supply 5 V.
2. Pan servo (in the base) → GPIO12 port, tilt servo (camera yoke) → GPIO13 port.
3. The camera ribbon drops from the yoke straight into the Pi–HAT gap; 15–20 cm should do. Only the
   tilt flexes it, so leave a gentle loop.
4. Check whether the camera lens is glued; if not, it can be turned to focus at desk distance.
5. Power the robot from a **5.1 V 3 A USB-C supply through the power/data splitter**; servos share
   the Pi's 5 V rail. Fit a fan to the HAT's fan header.
6. Connect the splitter's data end to whichever host is in use. The link uses fixed addresses
   (package 0b), so no DHCP reservation or Wi-Fi network is needed.

## 3. Architecture

```
Host: NixOS or Omarchy                            Raspberry Pi 4 + BrainCraft HAT (Ubuntu 24.04)
──────────────────────                            ──────────────────────────────────────────────
hermes-robot launcher (sets default audio devices)
Hermes voice mode / wake word                     PipeWire (not ROS)
  mic  ◄── pulse-tunnel source ◄── TCP 4713 ───── pipewire-pulse ◄── HAT mics (WM8960, in-kernel)
  spkr ──► pulse-tunnel sink   ─── TCP 4713 ────► pipewire-pulse ──► HAT speakers
                                                         │ levels
                                                  ROS 2 Jazzy graph                ▼
MCP client ── HTTP :8765/mcp (bearer) ──────────► hermes_mcp ──action──► hermes_head ─► PWM GPIO12/13
                                                     │  └─ latest image ◄── camera_ros (OV5647)
                                                     └─ /hermes/state ─► hermes_face ─► ST7789 (SPI)
                                                  hermes_audio_state ─► /audio/levels ─┘
                                                  robot_state_publisher (URDF) ─► /tf
(optional) ROS tools in podman ◄── ROS traffic ──► (RViz, ros2 CLI, rosbag; Nav2 in V2)
```

All of the traffic above runs over **USB gadget Ethernet** on the Pi's USB-C port: the host is
`10.55.0.1`, the robot `10.55.0.2` (a /30 used for nothing else). Wi-Fi stays configured as a
fallback. On the host the only changes are the robot's address in the configs and a small
network config for the USB interface (package 0b).

### ROS 2 layout

One colcon workspace, `ros/` in this repository, built on the Pi. Packages:

| Package | Role |
|---|---|
| `hermes_description` | URDF/xacro of the robot, with visual meshes exported from `enclosure/` (base, turntable, cabinet, yoke, camera). Frames: `base_link`, `pan_link`, `tilt_link`, `camera_link`, `camera_optical_frame`. Joints `pan_joint`, `tilt_joint`. |
| `hermes_head` | Servo driver node (package 2). Accepts `control_msgs/action/FollowJointTrajectory` (or a small custom `LookAt` action if that proves simpler), publishes `/joint_states`. |
| `camera_ros` (upstream) | Publishes `/camera/image_raw`, `/camera/image_raw/compressed`, `/camera/camera_info`. Released as a binary for Jazzy. |
| `hermes_audio_state` | Reads PipeWire's speaker monitor and mic levels; publishes `/audio/levels` and speaking/listening flags. Audio itself never goes through ROS. |
| `hermes_face` | The face renderer (package 4) as a node; subscribes to `/hermes/state`, `/joint_states`, `/audio/levels`. |
| `hermes_mcp` | The MCP server (existing code, MCP SDK 2.x) wrapped in an `rclpy` node. `look()` returns the latest image; `turn_head()` calls the head action; it publishes `/hermes/state` (e.g. `looking`). **The MCP tool names are the stable contract with Hermes** and must not change when the internals do. |
| `hermes_bringup` | Launch files and parameters; one systemd unit runs `ros2 launch hermes_bringup robot.launch.py`. |

Conventions, so V2 slots in without renames: standard message types; REP 103 units and axes;
REP 105 frames (`base_link`, later `odom` and `map`); every node its own process, so one crash
can't take the others down (respawn in the launch file). Parameters live in YAML in
`hermes_bringup`, replacing most of `/etc/robot-mcp.env` (secrets such as the MCP token stay in
an env file readable only by the service user).

**Middleware.** Start with Jazzy's default (Fast DDS), with a fixed `ROS_DOMAIN_ID` and
`ROS_AUTOMATIC_DISCOVERY_RANGE=SUBNET`, which is simple on a two-machine /30. Evaluate
`rmw_zenoh` (router on the Pi) before V2, when Wi-Fi and a second robot computer arrive;
**verify** its maturity in Jazzy at that point.

**ROS on the hosts (podman only).** Hermes needs no ROS: it talks MCP to `hermes_mcp`. ROS on a
host is for tools (RViz, `ros2` CLI, rosbag) now and Nav2 later. Provide `host/ros-shell`, a script
that runs `docker.io/osrf/ros:jazzy-desktop` (or a small Containerfile built on it) under podman
with `--network=host` (needed for discovery), `--ipc=host`, `/dev/dri` passed through for RViz
rendering, and the Wayland/X11 sockets for display (Hyprland on Omarchy; whatever compositor the
NixOS host runs). distrobox over podman is an acceptable alternative for interactive use.
**Verify** RViz renders under both hosts' compositors. Same `ROS_DOMAIN_ID` and RMW settings as
the Pi.

## 4. Existing code (read these first)

| Path | What it is | Status for this build |
|---|---|---|
| `pi/robot_mcp_server.py` | MCP server (MCP Python SDK **2.x**, `MCPServer`, streamable HTTP, bearer auth). Tools: `look`, `turn_head`, `head_position`. | Becomes the `hermes_mcp` ROS node: keep the MCP layer and tool names; replace Picamera2 and the servo code with ROS calls. Tested against mcp 2.3 with stubbed hardware only. |
| `pi/install.sh` | One-shot Pi setup, written for Raspberry Pi OS. | Rewrite for Ubuntu 24.04 + ROS 2 Jazzy (package 5). |
| `pi/pipewire/50-network-server.conf` | Exposes pipewire-pulse on TCP 4713. | Keep as is. |
| `pi/pipewire/60-echo-cancel.conf.optional` | WebRTC echo cancellation. | Recommended **on** for this build: the HAT's mics sit next to its speakers, and a Pi 4 has CPU to spare. |
| `pi/nftables-robot.nft` | Limits 4713/8765 to the host's IP. | Change to: accept 4713/8765 (and 8766 if built) only on the USB interface from `10.55.0.1`; drop them on every other interface, Wi-Fi included. |
| `pi/robot-mcp.service` | systemd unit for the MCP server. | Replace with one unit that runs the ROS launch file. |
| `omarchy/*` | PipeWire tunnel config, `hermes-robot` launcher, Hermes config snippet. | Set `ROBOT_IP` to `10.55.0.2`; add the USB interface config (package 0b). Rename the folder `host/` and add NixOS equivalents alongside. |
| `host/check-pi-platform.sh` | Checks, from either host and without the Pi, that Ubuntu 24.04's Pi kernel ships every driver and overlay the robot needs. Tested only against a mock mirror. | Run it at the start of package 0a. |
| `docs/face-prototype.html` | Browser prototype of the face. **The reference implementation for package 4.** | Port, do not ship. |
| `enclosure/` | OpenSCAD model, STLs, renders, print and assembly guide. | Refine with real measurements (package 6b). |

## 5. Work packages

Each package ends with acceptance checks. Do not mark one done until its checks pass on the real
Pi. Where something is marked **verify**, it was not confirmed when this brief was written; check
the current upstream docs or the running system before relying on it.

### Package 0a: Platform proof on Ubuntu 24.04 (do first)

Before writing any robot code, prove the hardware works on the chosen platform.

**Already established off the Pi** (October 2026; recheck if much time has passed):
- The BrainCraft's audio is a **WM8960** codec at I²C `0x1A`. The seeed-voicecard "2-mic" overlay
  Adafruit's guide installs is functionally identical to the **`wm8960-soundcard` overlay that
  ships in the Raspberry Pi kernel itself**: same codec and address, same 12.288 MHz MCLK, same
  `simple-audio-card` routing. So **no out-of-tree driver should be needed**, and kernel upgrades
  should no longer break audio.
- Ubuntu 24.04's Pi kernel (`6.8.0-1065-raspi` at the time) ships `wm8960-soundcard.dtbo` and
  `pwm-2chan.dtbo`. The Raspberry Pi kernel's config builds the WM8960 codec, simple card and Pi
  I²S drivers as modules; that Ubuntu's build includes those modules was **not** confirmed.
- Ubuntu 24.04's `libcamera-ipa` includes the Pi 4 pipeline (`ipa_rpi_vc4.so`) and an `ov5647.json`
  tuning file, so the stock libcamera should drive this camera; the Raspberry Pi libcamera fork
  should not be needed on a Pi 4. `camera_ros` 0.7.0 is released for Jazzy.

**Steps**
1. On a host, run `host/check-pi-platform.sh`. It downloads the current Ubuntu Pi kernel package
   and reports any missing driver or overlay for audio, camera, PWM, SPI and the USB gadget.
   Stop and tell the owner if anything is missing.
2. Flash Ubuntu Server 24.04 (arm64) for the Pi; install ROS 2 Jazzy (`ros-jazzy-ros-base` plus
   the packages the workspace needs) from the official apt repository.
3. **Audio:** `dtoverlay=wm8960-soundcard` and `dtparam=audio=off` in `/boot/firmware/config.txt`.
   The mainline driver's default mixer settings may leave the speaker or mic paths muted: set the
   needed controls with `amixer` (seeed's `wm8960_asound.state` in the HinTak/seeed-voicecard repo
   is a useful reference for which ones), then `alsactl store`. Record the final settings in the
   repo.
4. **Camera:** `camera_auto_detect=1` (or `dtoverlay=ov5647`); check with `cam -l`, then
   `ros2 run camera_ros camera_node`.
5. **Display:** SPI on; draw a test pattern on the ST7789 from Python. **Verify** which GPIO
   backend Adafruit Blinka needs on Ubuntu 24.04 (likely `lgpio`/`rpi-lgpio`, not `RPi.GPIO`).
6. **PWM:** `dtoverlay=pwm-2chan,pin=12,func=4,pin2=13,func2=4`; move one servo through sysfs.

**Fallback.** If audio or the camera can't be made to work on Ubuntu within a reasonable effort,
stop and ask the owner. The fallback is Raspberry Pi OS (where Adafruit and Raspberry Pi support
these parts directly) with ROS 2 Jazzy in a container that has the devices passed through.

**Accept when:** the check script passes; on the Pi, `aplay -l` shows the WM8960 card, a test tone
plays and both mics record; `cam -l` lists the OV5647 and `camera_ros` publishes images; the
display shows a test pattern; a servo moves on PWM; `ros2 topic list` works.

### Package 0b: The USB link (power and data over one cable)

Do this next: every later package's networking assumes it.

**Pi side**
- In `config.txt`, under `[all]`: `dtoverlay=dwc2,dr_mode=peripheral`.
- Build a composite gadget with configfs (`libcomposite`), started by a small script and a
  `robot-usb-gadget.service` that runs before networking. Two functions:
  - **NCM** Ethernet (Linux hosts handle it natively via `cdc_ncm`). Set fixed `host_addr` and
    `dev_addr` MAC addresses so the host can match the interface reliably.
  - **ACM serial**, with a getty on `ttyGS0`: a rescue console if networking breaks
    (`screen /dev/ttyACM0 115200` from the host).
- Give `usb0` the static address `10.55.0.2/30` with **no gateway**, so the robot never routes its
  internet traffic through the host by accident. On Ubuntu Server this means netplan (with
  systemd-networkd); **verify** the gadget interface's name.
- Leave Wi-Fi configured. It is still how the Pi reaches the internet for `apt` and `pip`, and
  the owner may want it as a fallback. Ask before disabling it.
- **Verify** that NCM gadget mode works on the Pi 4 with the current kernel. Fall back to ECM if
  NCM misbehaves, and record which was used.

**Host side (NixOS or Omarchy, on either machine)**
- Give the gadget's interface `10.55.0.1/30`, matched by the fixed MAC address so it works on
  whichever USB port and whichever machine it's plugged into.
  - **Omarchy:** **verify** its network stack (e.g. NetworkManager, or iwd with systemd-networkd)
    and write a config file for it that lives in `host/omarchy/`.
  - **NixOS:** a snippet for `configuration.nix` in `host/nixos/` (e.g. `systemd.network` with a
    `matchConfig.MACAddress` for the gadget). Declarative, so it survives rebuilds.
- Point the configs at the robot: `pulse.server.address = "tcp:10.55.0.2:4713"` in
  `50-robot-tunnel.conf`, and `url: "http://10.55.0.2:8765/mcp"` in the Hermes snippet.
- The PipeWire tunnels already retry every 5 s and Hermes retries MCP servers on a cooldown, so
  unplugging, sleeping the host or switching machines should recover without intervention.
  Confirm this.

**Power.** The Pi 4 can't negotiate USB Power Delivery; it needs plain 5 V at up to 3 A, with
spikes when the servos move.
- **Default:** a USB-C power/data splitter (PiKVM-style). Power comes from the 5.1 V 3 A supply
  and only the data lines go to the host. Pimoroni noted that its (now discontinued) version was
  designed around the official Raspberry Pi supply; **verify** the owner's splitter and supply
  together.
- Powering straight from the **Framework Desktop** is acceptable only if
  `vcgencmd get_throttled` stays `0x0` through a full pan/tilt sweep while speaking.
- **Never power the robot from the ROG Flow Z13.** Its ports may not supply 3 A, and the robot
  would drain the tablet's battery at roughly 10 W. Use the splitter with that machine.

**Firewall.** Rewrite `nftables-robot.nft`: accept 4713 and 8765 (and 8766 if built) only when
arriving on the USB interface from `10.55.0.1`, and drop those ports on every other interface.
SSH policy is the owner's call; ask.

**Accept when:**
- With the robot on the splitter and plugged into either host, the interface comes up and
  `ping 10.55.0.2` answers within about 30 s of the Pi booting.
- `ssh` works over the link, and the serial console gives a login prompt.
- `hermes mcp test robot` passes and the audio tunnel devices appear, with no Wi-Fi involved
  (test with the Pi's Wi-Fi off).
- Unplugging and replugging, or moving the cable to the other host, recovers on its own.
- `vcgencmd get_throttled` reads `0x0` after a servo sweep on the splitter.

### Package 1: Audio through the BrainCraft HAT

- Use the in-kernel `wm8960-soundcard` overlay proven in package 0a, with the stored mixer
  settings. Only if that fails, fall back to Adafruit's documented route (the `HinTak/seeed-voicecard`
  fork, branch matching `uname -r`), and then warn the owner that kernel upgrades can break audio.
- Remove `dtparam=i2s=on` / `dtoverlay=googlevoicehat-soundcard` from the install path.
- Keep `dtparam=audio=off` (the Pi's own headphone audio shares PWM hardware with package 2).
- Add the `hermes_audio_state` node: levels and speaking/listening flags from PipeWire's sink
  monitor and source, published on `/audio/levels`.
- Make WirePlumber's default source and sink the HAT's codec, and enable the echo-cancel config,
  exposing `robot_ec_source` / `robot_ec_sink`; then uncomment the `pulse.device` lines in
  `omarchy/50-robot-tunnel.conf` (and tell the owner to update their copy).
- **Accept when:** `arecord -l` lists the codec; `pw-record` captures both mics; a test tone plays
  from both speakers; from the host, `pactl list short sources` shows `robot_mic`, and
  `hermes-robot` with `/voice on` completes one spoken exchange without the robot's own speech
  triggering barge-in.

### Package 2: Servos on hardware PWM (`hermes_head` node)

- Add `dtoverlay=pwm-2chan,pin=12,func=4,pin2=13,func2=4` to `config.txt` (GPIO12 = PWM0,
  GPIO13 = PWM1 on the Pi 4).
- Write `hermes_head` as a ROS node: it accepts head targets as an action and publishes
  `/joint_states` for `pan_joint` and `tilt_joint` (radians, per REP 103), so
  `robot_state_publisher` keeps the camera frame correct in `/tf`. Also write
  `hermes_description`'s URDF now, with meshes from the enclosure STLs.
- Inside the node, keep the backend interface with two implementations, selected by the
  `servo_backend` parameter (`pwm` or `pca9685`, default `pwm`):
  - `PwmHead`: drives `/sys/class/pwm/pwmchip0/pwm{0,1}` directly (export, `period=20000000`,
    `duty_cycle` in ns, `enable=1`). No third-party library needed. **Verify** the chip number
    and that the service user can write there (add a udev rule granting a group rather than
    running as root; Ubuntu's defaults differ from Raspberry Pi OS here).
  - `Pca9685Head`: the existing code, kept for anyone using a servo board.
- Pulse range is per-servo: expose `ROBOT_SERVO_MIN_US` / `ROBOT_SERVO_MAX_US` (start at 500 and
  2400) and centre trims `ROBOT_PAN_TRIM_DEG` / `ROBOT_TILT_TRIM_DEG`. Keep the existing limits
  and invert flags.
- Move gently: interpolate to the target over ~0.3 s instead of jumping, which also limits the
  current spike on the shared 5 V rail. Disable PWM output after ~1 s idle to stop holding-jitter
  and buzz, and re-enable on the next move.
- **Accept when:** `turn_head(0,0)`, `(±60,0)`, `(0,±30)` reach those positions with no jitter at
  rest, and the Pi does not brown out (`vcgencmd get_throttled` stays `0x0`) during moves; RViz
  in the host's ROS container shows the robot model's head following.

### Package 3: Camera

- Run upstream `camera_ros` at `1296x972` (the OV5647's full-field binned mode, so the whole field
  of view is kept), publishing raw and compressed images and `camera_info`.
- Calibrate once with the `camera_calibration` package and store the result in
  `hermes_bringup`; V2's perception needs it.
- `hermes_mcp`'s `look()` returns the most recent frame as JPEG (encode from raw, or pass the
  compressed image through), and publishes `looking` on `/hermes/state`.
- **Accept when:** `camera_ros` publishes at a steady rate, `look()` returns a sharp-enough image of the
  desk in ordinary room light, and orientation is right on the mounted head (use the HFLIP/VFLIP
  settings).

### Package 4: The face (`hermes_face` node)

Its own node and process (respawned by the launch file), so a face crash never takes down the
MCP server or audio.

**Renderer.** Port `docs/face-prototype.html` to Python with NumPy. Keep its model:

- a 240×240 float32 energy buffer, multiplied each frame by `exp(-dt / persistence)`;
- the beam traced at constant speed, depositing energy per sample with bilinear splatting
  (`np.add.at`), so slow parts of the path burn brighter and retraces leave faint streaks;
- two passes of a separable box blur for glow; tone mapping `1 - exp(-1.6v)` tinted with the
  phosphor colour, overdriven spots whitening;
- the three shapes from the prototype: `flat`, `head` (smooth procedural wireframe with contour
  lines, depth cueing and back-face dimming) and `lowpoly` (coarse faceted mesh with alternating
  diagonals). `ROBOT_FACE_MODE` selects; **default `flat`** unless the owner says otherwise.
- both render styles: `phosphor` (afterglow, beam-speed brightness, retrace streaks) and `vector`
  (buffer cleared each frame, lines saturated to even weight, gaps filled, retrace blanked, a
  fading overlay for the shutter flash). `ROBOT_FACE_STYLE` selects; default `phosphor`.
- **Owner's choice: flat face, phosphor render, P7.** Other defaults as the prototype:
  persistence 70 ms, focus 0.35, beam noise 0.3 px, 30 fps target.

**Display output.** Use Adafruit's `adafruit-circuitpython-rgb-display` ST7789 driver (CE0,
DC = GPIO25, backlight = GPIO26, 240×240, SPI baud ≥ 48 MHz; set rotation for the mounting).
Convert to RGB565 with NumPy and push whole frames. **Measure** the achieved frame rate; if it is
under ~20 fps, try the kernel's ST7789 framebuffer/DRM overlay and write to the framebuffer
instead, and note what was chosen.

**State inputs**, in priority order (highest wins):

| State | Source |
|---|---|
| `speaking` | RMS of the speaker output, read from the sink's monitor (`pw-record --target <sink>.monitor`, or `parec -d <sink>.monitor`, 16 kHz mono s16). Above threshold for >80 ms. **The mouth draws these actual samples**, trigger-locked on a rising zero crossing so the waveform stands still. |
| `looking` | Event from the MCP server when `look()` starts (shutter flash and squint), held ~1.2 s. |
| `listening` | Mic RMS above threshold while not speaking. Draw the mic waveform, dimmer. |
| `thinking` | **Fallback heuristic first:** from the end of a listening stretch until speaking starts, capped at 30 s. Then, as an upgrade, Hermes webhooks (below). |
| `idle` | Otherwise. Random blinks every 2–6 s. |

Head pose comes from `/joint_states`, so the eyes (flat mode) or the whole wireframe (head mode)
follow the real servos.

**Inputs are ROS topics** (this replaces the unix-socket design in earlier drafts):
`/hermes/state` (from `hermes_mcp`: `looking`, later `thinking` and V2 states such as `driving`),
`/joint_states`, and `/audio/levels`. For the mouth's actual waveform, the node may read
PipeWire's sink monitor directly rather than sending audio samples over ROS.

**Hermes webhooks (optional, last).** Hermes Agent can send signed webhooks for session activity,
completed turns and tool events. **Verify** in the current Hermes docs: the event names, payload
shape, and signature scheme. If they give a reliable "turn started" signal, add a small HTTP
listener on port 8766 that verifies the HMAC and sets `thinking`. Show tool calls as brief
flickers. Add the port to the nftables rules. If the docs don't support this well, keep the
heuristic and say so.

**Extras, if time allows:** mirror the state on the three DotStar LEDs (dim), and make the
HAT button toggle a mute that also shows on the face.

**Accept when:** the face runs at ≥20 fps with CPU under ~40% of one core; in a real exchange
it goes idle → listening while the owner speaks → thinking → speaking, with the mouth tracing
the actual voice; `look()` flashes it; `turn_head` moves it.

### Package 5: Installer and services

- Rewrite `pi/install.sh` for Ubuntu 24.04, keeping it re-runnable: ROS 2 Jazzy apt repository and
  packages (`ros-jazzy-ros-base`, `camera_ros`, `robot_state_publisher`, `xacro`, and what the
  nodes need); `config.txt` lines (`wm8960-soundcard`, PWM overlay, `audio=off`, SPI on, `dwc2`);
  the stored ALSA mixer state; Python dependencies for the nodes (`mcp` 2.x, `numpy`,
  `adafruit-circuitpython-rgb-display`, the Blinka GPIO backend) installed so `rclpy` nodes can
  import them; `colcon build` of `ros/`; one systemd unit for the launch file; the PipeWire
  configs including echo cancel; nftables; lingering for the user's PipeWire.
- `--servos` flag no longer needed; servos are standard on this build.
- Print the MCP token at the end, as now.
- Add `host/ros-shell` (the podman wrapper from section 3).

### Package 6: End-to-end check with Hermes

- On the host: `hermes mcp test robot`, then a voice session through `hermes-robot`.
- **Verify the one open question from the first build:** that the image returned by `look()`
  actually reaches the model. If Hermes drops MCP image content, change `look()` to also save
  the JPEG and return its path for Hermes's own vision tool, and document which path is used.

### Package 6b: Enclosure

`enclosure/hermes_robot.scad` is a parametric OpenSCAD model (cabinet, back cover, turntable plate,
base, camera yoke and cradle, plus four fit-test pieces), with STLs in `enclosure/stl/` and a
person-facing guide in `enclosure/README.md`. It was modelled from photos and datasheets, not
measurements.

- Work with the owner: they measure the `MEASURE` values with calipers and print the `test_*`
  pieces; you update the parameters and re-export. Don't print the large parts until
  `test_front` seats correctly on the real stack.
- Export with `openscad -D 'PART="<part>"' -o stl/<part>.stl hermes_robot.scad`, then check each
  STL is watertight and a single body (e.g. with `trimesh`).
- **Power and data path:** a 30 × 25 mm USB/PWR splitter (PiKVM-style; ports RPI4, USB, PWR) sits
  in a connector block on the back of the base. USB and PWR face out through the back wall; the
  RPI4 socket faces a tunnel into the base, and a ~30 cm USB-C cable (right-angle at the Pi end)
  runs up through the bearing to the Pi. The splitter's socket positions (`spl_*`) are estimates
  from a product photo; confirm with `test_pod`.
- **URDF meshes:** export the moving parts in their assembled positions for `hermes_description`
  (the model already has `pose_*` parts for this; see `enclosure/poster/README.md`).
- **Software implications:** pan now rotates the whole cabinet, so `turn_head` pan should move
  smoothly (the interpolation in package 2 matters more), and the face's eyes should follow tilt
  and small glances only. Set `ROBOT_FACE_ROTATION` for the portrait-mounted display.

### Package 7: Documentation

Rewrite `README.md` for this build: parts, the owner's physical steps from section 2, setup,
tuning, troubleshooting (including the kernel/driver coupling from package 1). Keep it a guide for
a person; this file stays as the engineering brief.

## 6. Design principles for the face

Proposed during design discussion; the owner may revise them. Until then, use them for
judgement calls:

1. **Show only real states.** The face signals what the agent is actually doing: listening,
   thinking, speaking, looking. It does not perform emotions it has no basis for.
2. **The voice trace is literal.** The mouth is the real outgoing audio, not an animation timed
   to it.
3. **It is plainly a machine.** Oscilloscope phosphor, vector lines, no attempt at a realistic
   human face. Head mode is acceptable because it is clearly a wireframe instrument display, but
   flat is the default.
4. **Gaze is honest.** Where the face looks matches where the camera points.

## 7. Configuration reference (target)

Network (package 0b): host `10.55.0.1/30`, robot `10.55.0.2/30`, gadget functions NCM + ACM,
fixed MAC addresses for both ends.

ROS: Jazzy, Fast DDS to start, fixed `ROS_DOMAIN_ID`, `ROS_AUTOMATIC_DISCOVERY_RANGE=SUBNET`.

The values below become ROS parameters in `hermes_bringup` (names may change to ROS style, e.g.
`face.mode`); only the MCP token stays in an env file. Current targets:

```
ROBOT_MCP_TOKEN=<hex>            ROBOT_MCP_PORT=8765
ROBOT_CAM_SIZE=1296x972          ROBOT_CAM_HFLIP=0   ROBOT_CAM_VFLIP=0
ROBOT_SERVO_BACKEND=pwm          ROBOT_SERVO_MIN_US=500   ROBOT_SERVO_MAX_US=2400
ROBOT_PAN_LIMIT=80               ROBOT_TILT_LIMIT=45
ROBOT_PAN_INVERT=0               ROBOT_TILT_INVERT=0
ROBOT_PAN_TRIM_DEG=0             ROBOT_TILT_TRIM_DEG=0
ROBOT_FACE_MODE=flat             ROBOT_FACE_STYLE=phosphor
ROBOT_FACE_PHOSPHOR=P7           ROBOT_FACE_FPS=30
ROBOT_FACE_ROTATION=0            ROBOT_FACE_WEBHOOK_SECRET=   (empty = webhooks off)
```

## 8. Unverified items (check before relying on them)

| Item | Why it's uncertain |
|---|---|
| WM8960 modules in Ubuntu's Pi kernel | Overlays confirmed; modules confirmed only in Raspberry Pi's kernel config. `host/check-pi-platform.sh` settles it. |
| WM8960 default mixer state on the BrainCraft | The in-kernel driver may start with paths muted; set once and store. |
| Adafruit Blinka on Ubuntu 24.04 | GPIO backend differs from Raspberry Pi OS. |
| RViz in podman on both hosts' compositors | Display and GPU passthrough untested. |
| `rmw_zenoh` on Jazzy | Only matters from V2; check maturity then. |
| PWM sysfs chip number and permissions | Differs by Pi model and OS release. |
| Servo pulse range | Varies between servo makes; calibrate. |
| ST7789 frame rate from Python | Untested on this HAT; may need the kernel driver. |
| Hermes webhook events and signatures | Seen in release coverage, not checked in docs. |
| Hermes passing MCP images to the model | Not documented when the first build was written. |
| `pipewire-pulse` TCP config merge behaviour | Written from docs, untested on the Pi. |
| NCM gadget mode on the Pi 4's current kernel | Widely used, but not tested on this build; ECM is the fallback. |
| Omarchy's network stack | Not checked; the USB interface config must match it. |
| NixOS host config for the gadget interface | Needs writing and testing on the owner's NixOS machine. |
| USB-C power output of the Framework Desktop's ports | Unknown; only matters if powering without the splitter. |
| Splitter compatibility with the owner's supply | Pimoroni's version was designed for the official Pi supply. |

## 9. Ground rules for the implementer

- Test on the Pi itself; the camera, codec, display and PWM cannot be meaningfully mocked.
- Keep the MCP server, audio, and face independent: any one failing must not stop the others.
- Never widen network exposure beyond the USB link (`10.55.0.1` on the gadget interface) without asking.
- Ask the owner before anything that reboots the Pi, upgrades the kernel, or holds packages.
- ROS on the hosts runs only in podman containers; don't install ROS natively on NixOS or Omarchy.
- Keep the MCP tool names stable; they are Hermes's contract with the robot.
- Keep audio out of ROS, and keep any future motor safety (V2) out of ROS too: it lives in
  microcontroller firmware.

## 10. V2 direction: design for it, don't build it

The owner intends a mobile V2. V1 should make no choice that blocks it. Current thinking:

- **Platform:** the cabinet, yoke, tilt servo, face and audio move onto a differential-drive
  chassis. Turning in place replaces the pan turntable, so `pan_joint` disappears from the URDF
  and the base gains `odom` → `base_link` from wheel odometry. The USB link gives way to Wi-Fi
  (firewalled to the host, ideally over Tailscale); a Wi-Fi drop must stop the motors.
- **Three layers of control**, each able to veto the one above and to carry on without it:
  1. **Reflex**, on a Raspberry Pi Pico: motor control, encoders, bumper and proximity stops,
     speed limits, a watchdog that stops the motors if the Pi goes quiet, battery cut-off. It may
     speak micro-ROS, but its safety rules must not depend on ROS or the Pi.
  2. **Autopilot**, on the Pi 4 plus an added **Jetson Orin Nano Super** (8 GB): perception
     (free floor, people, doorways), mapping (slam_toolbox or Isaac ROS visual SLAM), Nav2, and
     skills: go to a place, follow, turn toward a voice, find an object, dock. A recent guide
     reports JetPack 7.2 runs Ubuntu 24.04 + Jazzy on the Orin Nano, with Isaac ROS 4.6 targeting
     that stack; **verify** with NVIDIA before buying. The owner's older Jetson Nano (Ubuntu 18.04)
     is fine for experiments only.
  3. **Mind**, Hermes on the host: goals in, summaries out. It never sends motor commands or
     receives raw video.
- **MCP contract for V2:** skills as long-running tools that return outcomes (`go_to(place)`,
  `follow()`, `dock()`, `status()`), plus a way for Hermes to ask what has happened. How events
  reach Hermes unprompted is an open question.
- **Environment:** no stairs and no pets, so cliff sensors are optional; front bumper and a few
  forward distance sensors are required.
- **Owner decisions still open:** rooms the robot may not enter; a visible light whenever camera or
  mic is live; whether Hermes may drive on its own initiative (e.g. from a scheduled task).