hermes-bot/HANDOFF.md

Hermes robot: Pi 4 + BrainCraft build — implementation handoff

Status note (2026-10-05): this is the original brief, kept as the record of intent. It has drifted. Read CONTEXT.md for where things really stand. The main differences:

• ROS never leaves the Pi. There is no host/ros-shell talking to the robot over DDS; hermes_mcp is the only bridge, and the firewall blocks DDS on every interface (section 3, package 0b, package 5).
• Stock libcamera does not work on this kernel; the camera needs the Raspberry Pi libcamera fork, with camera_ros built against it (package 0a, package 3). The WM8960 overlay needed a fix too (package 1), and the HAT's Audio switch must be ON.
• MCP tools are look, faces, list_people (and turn_head / head_position only when a head is fitted), not just look / turn_head. Hermes does not pass MCP images to the model; it saves the photo and the model calls its own vision tool, which is package 6's fallback, handled by Hermes.
• One launch file and one unit run the whole robot; pi/install.sh is the single installer; the old Pi Zero files in section 4 are deleted; omarchy/ became host/omarchy/.
• Done since: the face (flat shape only), recognition (opt-in), the thermal guard, camera calibration.
• Not built yet because the hardware has not arrived: hermes_head and the servos, speakers and echo cancel, the fan, the enclosure. The NixOS host config is skipped for now.

This document is the brief for finishing the robot. It is written to be worked through by Claude Code with the owner at hand for the physical steps. Read it in full before changing anything; then work the packages in order, since later ones depend on earlier ones.

Platform decision: the robot's software is a ROS 2 Jazzy system from V1 onward, running natively on Ubuntu 24.04 on the Pi, so that the mobile V2 (section 10) grows out of it rather than replacing it. The owner has used ROS extensively and wants it. Package 0a proves the hardware on Ubuntu before anything else is built.

1. What this is

A small desk robot that acts as the eyes, ears, voice and face of Hermes Agent (Nous Research's open-source agent), which runs on the owner's host machine (NixOS or Omarchy, an Arch + Hyprland setup). The robot does no reasoning. It:

• serves photos and head movement to Hermes as MCP tools, over a USB-C cable that carries both power and a private network link (Wi-Fi remains as a fallback);
• exposes its microphones and speakers to the host as PipeWire audio devices;
• shows an animated oscilloscope-style face on its display, reflecting what the agent is doing.

The repository already contains a working design for an earlier variant (Pi Zero 2 W, separate I²S mic and amp boards, PCA9685 servo board). That code is the starting point. This document supersedes README.md wherever they differ; README.md describes the Zero build and should be rewritten for this hardware at the end (package 7).

2. Hardware actually on hand

Part	Details that matter
Raspberry Pi 4	In a case. Ubuntu Server 24.04 (arm64) with ROS 2 Jazzy installed natively. Fallback if package 0a fails: Raspberry Pi OS with ROS in a container.
Adafruit BrainCraft HAT	Display: 1.54″ ST7789, 240×240, SPI CE0, DC = GPIO25, backlight = GPIO26. Audio: WM8960 codec at I²C 0x1A, I²S on GPIO 18/19/20/21, stereo mics (L/R), stereo 8 Ω 1 W speaker JSTs, headphone jack, hardware audio on/off switch. 3-pin JST ports on GPIO12 and GPIO13, with a 3V/5V solder jumper on the back ("3pin Volt"). STEMMA QT (I²C). Fan header on GPIO4. 5-way joystick, one button, three DotStar LEDs.
Arducam UC-346	OV5647 5 MP CSI camera, fixed focus, 15-pin ribbon. Natively supported by libcamera.
Pan/tilt	Two MG90S metal-gear servos. Pan turns the whole cabinet on a 6810 bearing turntable (servo in the base); tilt moves only the camera, in a yoke on top. See enclosure/.
Speakers	8 Ω, ≤1 W, need JST PH 2-pin plugs for the HAT.
USB-C power/data splitter	GODIYMODULES 30 × 25 mm board, PiKVM-style (ports RPI4 / USB / PWR, φ2.75 mm holes): power from the 5.1 V 3 A supply, data only to the host. Mounted in the base's connector block; a ~30 cm USB-C cable (right-angle at the Pi end) links it to the Pi.
Host computers	Either a Framework Desktop (Strix Halo) or an ASUS ROG Flow Z13, each running NixOS or Omarchy (Arch). The robot must work with whichever it's plugged into. Hermes runs natively on the host; any ROS software on a host runs in a podman container (owner's rule, so both distros behave the same).

Physical steps for the owner (not code)

1. Set the "3pin Volt" jumper to 5 V Verified by the owner: the jumper is factory-set to 5 V (continuity: middle pad to 5V pad only). The GPIO12/13 ports already supply 5 V.
2. Pan servo (in the base) → GPIO12 port, tilt servo (camera yoke) → GPIO13 port.
3. The camera ribbon drops from the yoke straight into the Pi–HAT gap; 15–20 cm should do. Only the tilt flexes it, so leave a gentle loop.
4. Check whether the camera lens is glued; if not, it can be turned to focus at desk distance.
5. Power the robot from a 5.1 V 3 A USB-C supply through the power/data splitter; servos share the Pi's 5 V rail. Fit a fan to the HAT's fan header.
6. Connect the splitter's data end to whichever host is in use. The link uses fixed addresses (package 0b), so no DHCP reservation or Wi-Fi network is needed.

3. Architecture

Host: NixOS or Omarchy                            Raspberry Pi 4 + BrainCraft HAT (Ubuntu 24.04)
──────────────────────                            ──────────────────────────────────────────────
hermes-robot launcher (sets default audio devices)
Hermes voice mode / wake word                     PipeWire (not ROS)
  mic  ◄── pulse-tunnel source ◄── TCP 4713 ───── pipewire-pulse ◄── HAT mics (WM8960, in-kernel)
  spkr ──► pulse-tunnel sink   ─── TCP 4713 ────► pipewire-pulse ──► HAT speakers
                                                         │ levels
                                                  ROS 2 Jazzy graph                ▼
MCP client ── HTTP :8765/mcp (bearer) ──────────► hermes_mcp ──action──► hermes_head ─► PWM GPIO12/13
                                                     │  └─ latest image ◄── camera_ros (OV5647)
                                                     └─ /hermes/state ─► hermes_face ─► ST7789 (SPI)
                                                  hermes_audio_state ─► /audio/levels ─┘
                                                  robot_state_publisher (URDF) ─► /tf
(optional) ROS tools in podman ◄── ROS traffic ──► (RViz, ros2 CLI, rosbag; Nav2 in V2)

All of the traffic above runs over USB gadget Ethernet on the Pi's USB-C port: the host is 10.55.0.1, the robot 10.55.0.2 (a /30 used for nothing else). Wi-Fi stays configured as a fallback. On the host the only changes are the robot's address in the configs and a small network config for the USB interface (package 0b).

ROS 2 layout

One colcon workspace, ros/ in this repository, built on the Pi. Packages:

Package	Role
hermes_description	URDF/xacro of the robot, with visual meshes exported from enclosure/ (base, turntable, cabinet, yoke, camera). Frames: base_link, pan_link, tilt_link, camera_link, camera_optical_frame. Joints pan_joint, tilt_joint.
hermes_head	Servo driver node (package 2). Accepts control_msgs/action/FollowJointTrajectory (or a small custom LookAt action if that proves simpler), publishes /joint_states.
camera_ros (upstream)	Publishes /camera/image_raw, /camera/image_raw/compressed, /camera/camera_info. Released as a binary for Jazzy.
hermes_audio_state	Reads PipeWire's speaker monitor and mic levels; publishes /audio/levels and speaking/listening flags. Audio itself never goes through ROS.
hermes_face	The face renderer (package 4) as a node; subscribes to /hermes/state, /joint_states, /audio/levels.
hermes_mcp	The MCP server (existing code, MCP SDK 2.x) wrapped in an rclpy node. look() returns the latest image; turn_head() calls the head action; it publishes /hermes/state (e.g. looking). The MCP tool names are the stable contract with Hermes and must not change when the internals do.
hermes_bringup	Launch files and parameters; one systemd unit runs ros2 launch hermes_bringup robot.launch.py.

Conventions, so V2 slots in without renames: standard message types; REP 103 units and axes; REP 105 frames (base_link, later odom and map); every node its own process, so one crash can't take the others down (respawn in the launch file). Parameters live in YAML in hermes_bringup, replacing most of /etc/robot-mcp.env (secrets such as the MCP token stay in an env file readable only by the service user).

Middleware. Start with Jazzy's default (Fast DDS), with a fixed ROS_DOMAIN_ID and ROS_AUTOMATIC_DISCOVERY_RANGE=SUBNET, which is simple on a two-machine /30. Evaluate rmw_zenoh (router on the Pi) before V2, when Wi-Fi and a second robot computer arrive; verify its maturity in Jazzy at that point.

ROS on the hosts (podman only). Hermes needs no ROS: it talks MCP to hermes_mcp. ROS on a host is for tools (RViz, ros2 CLI, rosbag) now and Nav2 later. Provide host/ros-shell, a script that runs docker.io/osrf/ros:jazzy-desktop (or a small Containerfile built on it) under podman with --network=host (needed for discovery), --ipc=host, /dev/dri passed through for RViz rendering, and the Wayland/X11 sockets for display (Hyprland on Omarchy; whatever compositor the NixOS host runs). distrobox over podman is an acceptable alternative for interactive use. Verify RViz renders under both hosts' compositors. Same ROS_DOMAIN_ID and RMW settings as the Pi.

4. Existing code (read these first)

Path	What it is	Status for this build
pi/robot_mcp_server.py	MCP server (MCP Python SDK 2.x, MCPServer, streamable HTTP, bearer auth). Tools: look, turn_head, head_position.	Becomes the hermes_mcp ROS node: keep the MCP layer and tool names; replace Picamera2 and the servo code with ROS calls. Tested against mcp 2.3 with stubbed hardware only.
pi/install.sh	One-shot Pi setup, written for Raspberry Pi OS.	Rewrite for Ubuntu 24.04 + ROS 2 Jazzy (package 5).
pi/pipewire/50-network-server.conf	Exposes pipewire-pulse on TCP 4713.	Keep as is.
pi/pipewire/60-echo-cancel.conf.optional	WebRTC echo cancellation.	Recommended on for this build: the HAT's mics sit next to its speakers, and a Pi 4 has CPU to spare.
pi/nftables-robot.nft	Limits 4713/8765 to the host's IP.	Change to: accept 4713/8765 (and 8766 if built) only on the USB interface from 10.55.0.1; drop them on every other interface, Wi-Fi included.
pi/robot-mcp.service	systemd unit for the MCP server.	Replace with one unit that runs the ROS launch file.
omarchy/*	PipeWire tunnel config, hermes-robot launcher, Hermes config snippet.	Set ROBOT_IP to 10.55.0.2; add the USB interface config (package 0b). Rename the folder host/ and add NixOS equivalents alongside.
host/check-pi-platform.sh	Checks, from either host and without the Pi, that Ubuntu 24.04's Pi kernel ships every driver and overlay the robot needs. Tested only against a mock mirror.	Run it at the start of package 0a.
docs/face-prototype.html	Browser prototype of the face. The reference implementation for package 4.	Port, do not ship.
enclosure/	OpenSCAD model, STLs, renders, print and assembly guide.	Refine with real measurements (package 6b).

5. Work packages

Each package ends with acceptance checks. Do not mark one done until its checks pass on the real Pi. Where something is marked verify, it was not confirmed when this brief was written; check the current upstream docs or the running system before relying on it.

Package 0a: Platform proof on Ubuntu 24.04 (do first)

Before writing any robot code, prove the hardware works on the chosen platform.

Already established off the Pi (October 2026; recheck if much time has passed):

• The BrainCraft's audio is a WM8960 codec at I²C 0x1A. The seeed-voicecard "2-mic" overlay Adafruit's guide installs is functionally identical to the wm8960-soundcard overlay that ships in the Raspberry Pi kernel itself: same codec and address, same 12.288 MHz MCLK, same simple-audio-card routing. So no out-of-tree driver should be needed, and kernel upgrades should no longer break audio.
• Ubuntu 24.04's Pi kernel (6.8.0-1065-raspi at the time) ships wm8960-soundcard.dtbo and pwm-2chan.dtbo. The Raspberry Pi kernel's config builds the WM8960 codec, simple card and Pi I²S drivers as modules; that Ubuntu's build includes those modules was not confirmed.
• Ubuntu 24.04's libcamera-ipa includes the Pi 4 pipeline (ipa_rpi_vc4.so) and an ov5647.json tuning file, so the stock libcamera should drive this camera; the Raspberry Pi libcamera fork should not be needed on a Pi 4. camera_ros 0.7.0 is released for Jazzy.

Steps

1. On a host, run host/check-pi-platform.sh. It downloads the current Ubuntu Pi kernel package and reports any missing driver or overlay for audio, camera, PWM, SPI and the USB gadget. Stop and tell the owner if anything is missing.
2. Flash Ubuntu Server 24.04 (arm64) for the Pi; install ROS 2 Jazzy (ros-jazzy-ros-base plus the packages the workspace needs) from the official apt repository.
3. Audio: dtoverlay=wm8960-soundcard and dtparam=audio=off in /boot/firmware/config.txt. The mainline driver's default mixer settings may leave the speaker or mic paths muted: set the needed controls with amixer (seeed's wm8960_asound.state in the HinTak/seeed-voicecard repo is a useful reference for which ones), then alsactl store. Record the final settings in the repo.
4. Camera: camera_auto_detect=1 (or dtoverlay=ov5647); check with cam -l, then ros2 run camera_ros camera_node.
5. Display: SPI on; draw a test pattern on the ST7789 from Python. Verify which GPIO backend Adafruit Blinka needs on Ubuntu 24.04 (likely lgpio/rpi-lgpio, not RPi.GPIO).
6. PWM: dtoverlay=pwm-2chan,pin=12,func=4,pin2=13,func2=4; move one servo through sysfs.

Fallback. If audio or the camera can't be made to work on Ubuntu within a reasonable effort, stop and ask the owner. The fallback is Raspberry Pi OS (where Adafruit and Raspberry Pi support these parts directly) with ROS 2 Jazzy in a container that has the devices passed through.

Accept when: the check script passes; on the Pi, aplay -l shows the WM8960 card, a test tone plays and both mics record; cam -l lists the OV5647 and camera_ros publishes images; the display shows a test pattern; a servo moves on PWM; ros2 topic list works.

Package 0b: The USB link (power and data over one cable)

Do this next: every later package's networking assumes it.

Pi side

• In config.txt, under [all]: dtoverlay=dwc2,dr_mode=peripheral.
• Build a composite gadget with configfs (libcomposite), started by a small script and a robot-usb-gadget.service that runs before networking. Two functions:
  • NCM Ethernet (Linux hosts handle it natively via cdc_ncm). Set fixed host_addr and dev_addr MAC addresses so the host can match the interface reliably.
  • ACM serial, with a getty on ttyGS0: a rescue console if networking breaks (screen /dev/ttyACM0 115200 from the host).
• Give usb0 the static address 10.55.0.2/30 with no gateway, so the robot never routes its internet traffic through the host by accident. On Ubuntu Server this means netplan (with systemd-networkd); verify the gadget interface's name.
• Leave Wi-Fi configured. It is still how the Pi reaches the internet for apt and pip, and the owner may want it as a fallback. Ask before disabling it.
• Verify that NCM gadget mode works on the Pi 4 with the current kernel. Fall back to ECM if NCM misbehaves, and record which was used.

Host side (NixOS or Omarchy, on either machine)

• Give the gadget's interface 10.55.0.1/30, matched by the fixed MAC address so it works on whichever USB port and whichever machine it's plugged into.
  • Omarchy: verify its network stack (e.g. NetworkManager, or iwd with systemd-networkd) and write a config file for it that lives in host/omarchy/.
  • NixOS: a snippet for configuration.nix in host/nixos/ (e.g. systemd.network with a matchConfig.MACAddress for the gadget). Declarative, so it survives rebuilds.
• Point the configs at the robot: pulse.server.address = "tcp:10.55.0.2:4713" in 50-robot-tunnel.conf, and url: "http://10.55.0.2:8765/mcp" in the Hermes snippet.
• The PipeWire tunnels already retry every 5 s and Hermes retries MCP servers on a cooldown, so unplugging, sleeping the host or switching machines should recover without intervention. Confirm this.

Power. The Pi 4 can't negotiate USB Power Delivery; it needs plain 5 V at up to 3 A, with spikes when the servos move.

• Default: a USB-C power/data splitter (PiKVM-style). Power comes from the 5.1 V 3 A supply and only the data lines go to the host. Pimoroni noted that its (now discontinued) version was designed around the official Raspberry Pi supply; verify the owner's splitter and supply together.
• Powering straight from the Framework Desktop is acceptable only if vcgencmd get_throttled stays 0x0 through a full pan/tilt sweep while speaking.
• Never power the robot from the ROG Flow Z13. Its ports may not supply 3 A, and the robot would drain the tablet's battery at roughly 10 W. Use the splitter with that machine.

Firewall. Rewrite nftables-robot.nft: accept 4713 and 8765 (and 8766 if built) only when arriving on the USB interface from 10.55.0.1, and drop those ports on every other interface. SSH policy is the owner's call; ask.

Accept when:

• With the robot on the splitter and plugged into either host, the interface comes up and ping 10.55.0.2 answers within about 30 s of the Pi booting.
• ssh works over the link, and the serial console gives a login prompt.
• hermes mcp test robot passes and the audio tunnel devices appear, with no Wi-Fi involved (test with the Pi's Wi-Fi off).
• Unplugging and replugging, or moving the cable to the other host, recovers on its own.
• vcgencmd get_throttled reads 0x0 after a servo sweep on the splitter.

Package 1: Audio through the BrainCraft HAT

• Use the in-kernel wm8960-soundcard overlay proven in package 0a, with the stored mixer settings. Only if that fails, fall back to Adafruit's documented route (the HinTak/seeed-voicecard fork, branch matching uname -r), and then warn the owner that kernel upgrades can break audio.
• Remove dtparam=i2s=on / dtoverlay=googlevoicehat-soundcard from the install path.
• Keep dtparam=audio=off (the Pi's own headphone audio shares PWM hardware with package 2).
• Add the hermes_audio_state node: levels and speaking/listening flags from PipeWire's sink monitor and source, published on /audio/levels.
• Make WirePlumber's default source and sink the HAT's codec, and enable the echo-cancel config, exposing robot_ec_source / robot_ec_sink; then uncomment the pulse.device lines in omarchy/50-robot-tunnel.conf (and tell the owner to update their copy).
• Accept when: arecord -l lists the codec; pw-record captures both mics; a test tone plays from both speakers; from the host, pactl list short sources shows robot_mic, and hermes-robot with /voice on completes one spoken exchange without the robot's own speech triggering barge-in.

Package 2: Servos on hardware PWM (hermes_head node)

• Add dtoverlay=pwm-2chan,pin=12,func=4,pin2=13,func2=4 to config.txt (GPIO12 = PWM0, GPIO13 = PWM1 on the Pi 4).
• Write hermes_head as a ROS node: it accepts head targets as an action and publishes /joint_states for pan_joint and tilt_joint (radians, per REP 103), so robot_state_publisher keeps the camera frame correct in /tf. Also write hermes_description's URDF now, with meshes from the enclosure STLs.
• Inside the node, keep the backend interface with two implementations, selected by the servo_backend parameter (pwm or pca9685, default pwm):
  • PwmHead: drives /sys/class/pwm/pwmchip0/pwm{0,1} directly (export, period=20000000, duty_cycle in ns, enable=1). No third-party library needed. Verify the chip number and that the service user can write there (add a udev rule granting a group rather than running as root; Ubuntu's defaults differ from Raspberry Pi OS here).
  • Pca9685Head: the existing code, kept for anyone using a servo board.
• Pulse range is per-servo: expose ROBOT_SERVO_MIN_US / ROBOT_SERVO_MAX_US (start at 500 and 2400) and centre trims ROBOT_PAN_TRIM_DEG / ROBOT_TILT_TRIM_DEG. Keep the existing limits and invert flags.
• Move gently: interpolate to the target over ~0.3 s instead of jumping, which also limits the current spike on the shared 5 V rail. Disable PWM output after ~1 s idle to stop holding-jitter and buzz, and re-enable on the next move.
• Accept when: turn_head(0,0), (±60,0), (0,±30) reach those positions with no jitter at rest, and the Pi does not brown out (vcgencmd get_throttled stays 0x0) during moves; RViz in the host's ROS container shows the robot model's head following.

Package 3: Camera

• Run upstream camera_ros at 1296x972 (the OV5647's full-field binned mode, so the whole field of view is kept), publishing raw and compressed images and camera_info.
• Calibrate once with the camera_calibration package and store the result in hermes_bringup; V2's perception needs it.
• hermes_mcp's look() returns the most recent frame as JPEG (encode from raw, or pass the compressed image through), and publishes looking on /hermes/state.
• Accept when: camera_ros publishes at a steady rate, look() returns a sharp-enough image of the desk in ordinary room light, and orientation is right on the mounted head (use the HFLIP/VFLIP settings).

Package 4: The face (hermes_face node)

Its own node and process (respawned by the launch file), so a face crash never takes down the MCP server or audio.

Renderer. Port docs/face-prototype.html to Python with NumPy. Keep its model:

• a 240×240 float32 energy buffer, multiplied each frame by exp(-dt / persistence);
• the beam traced at constant speed, depositing energy per sample with bilinear splatting (np.add.at), so slow parts of the path burn brighter and retraces leave faint streaks;
• two passes of a separable box blur for glow; tone mapping 1 - exp(-1.6v) tinted with the phosphor colour, overdriven spots whitening;
• the three shapes from the prototype: flat, head (smooth procedural wireframe with contour lines, depth cueing and back-face dimming) and lowpoly (coarse faceted mesh with alternating diagonals). ROBOT_FACE_MODE selects; default flat unless the owner says otherwise.
• both render styles: phosphor (afterglow, beam-speed brightness, retrace streaks) and vector (buffer cleared each frame, lines saturated to even weight, gaps filled, retrace blanked, a fading overlay for the shutter flash). ROBOT_FACE_STYLE selects; default phosphor.
• Owner's choice: flat face, phosphor render, P7. Other defaults as the prototype: persistence 70 ms, focus 0.35, beam noise 0.3 px, 30 fps target.

Display output. Use Adafruit's adafruit-circuitpython-rgb-display ST7789 driver (CE0, DC = GPIO25, backlight = GPIO26, 240×240, SPI baud ≥ 48 MHz; set rotation for the mounting). Convert to RGB565 with NumPy and push whole frames. Measure the achieved frame rate; if it is under ~20 fps, try the kernel's ST7789 framebuffer/DRM overlay and write to the framebuffer instead, and note what was chosen.

State inputs, in priority order (highest wins):

State	Source
speaking	RMS of the speaker output, read from the sink's monitor (pw-record --target <sink>.monitor, or parec -d <sink>.monitor, 16 kHz mono s16). Above threshold for >80 ms. The mouth draws these actual samples, trigger-locked on a rising zero crossing so the waveform stands still.
looking	Event from the MCP server when look() starts (shutter flash and squint), held ~1.2 s.
listening	Mic RMS above threshold while not speaking. Draw the mic waveform, dimmer.
thinking	Fallback heuristic first: from the end of a listening stretch until speaking starts, capped at 30 s. Then, as an upgrade, Hermes webhooks (below).
idle	Otherwise. Random blinks every 2–6 s.

Head pose comes from /joint_states, so the eyes (flat mode) or the whole wireframe (head mode) follow the real servos.

Inputs are ROS topics (this replaces the unix-socket design in earlier drafts): /hermes/state (from hermes_mcp: looking, later thinking and V2 states such as driving), /joint_states, and /audio/levels. For the mouth's actual waveform, the node may read PipeWire's sink monitor directly rather than sending audio samples over ROS.

Hermes webhooks (optional, last). Hermes Agent can send signed webhooks for session activity, completed turns and tool events. Verify in the current Hermes docs: the event names, payload shape, and signature scheme. If they give a reliable "turn started" signal, add a small HTTP listener on port 8766 that verifies the HMAC and sets thinking. Show tool calls as brief flickers. Add the port to the nftables rules. If the docs don't support this well, keep the heuristic and say so.

Extras, if time allows: mirror the state on the three DotStar LEDs (dim), and make the HAT button toggle a mute that also shows on the face.

Accept when: the face runs at ≥20 fps with CPU under ~40% of one core; in a real exchange it goes idle → listening while the owner speaks → thinking → speaking, with the mouth tracing the actual voice; look() flashes it; turn_head moves it.

Package 5: Installer and services

• Rewrite pi/install.sh for Ubuntu 24.04, keeping it re-runnable: ROS 2 Jazzy apt repository and packages (ros-jazzy-ros-base, camera_ros, robot_state_publisher, xacro, and what the nodes need); config.txt lines (wm8960-soundcard, PWM overlay, audio=off, SPI on, dwc2); the stored ALSA mixer state; Python dependencies for the nodes (mcp 2.x, numpy, adafruit-circuitpython-rgb-display, the Blinka GPIO backend) installed so rclpy nodes can import them; colcon build of ros/; one systemd unit for the launch file; the PipeWire configs including echo cancel; nftables; lingering for the user's PipeWire.
• --servos flag no longer needed; servos are standard on this build.
• Print the MCP token at the end, as now.
• Add host/ros-shell (the podman wrapper from section 3).

Package 6: End-to-end check with Hermes

• On the host: hermes mcp test robot, then a voice session through hermes-robot.
• Verify the one open question from the first build: that the image returned by look() actually reaches the model. If Hermes drops MCP image content, change look() to also save the JPEG and return its path for Hermes's own vision tool, and document which path is used.

Package 6b: Enclosure

enclosure/hermes_robot.scad is a parametric OpenSCAD model (cabinet, back cover, turntable plate, base, camera yoke and cradle, plus four fit-test pieces), with STLs in enclosure/stl/ and a person-facing guide in enclosure/README.md. It was modelled from photos and datasheets, not measurements.

• Work with the owner: they measure the MEASURE values with calipers and print the test_* pieces; you update the parameters and re-export. Don't print the large parts until test_front seats correctly on the real stack.
• Export with openscad -D 'PART="<part>"' -o stl/<part>.stl hermes_robot.scad, then check each STL is watertight and a single body (e.g. with trimesh).
• Power and data path: a 30 × 25 mm USB/PWR splitter (PiKVM-style; ports RPI4, USB, PWR) sits in a connector block on the back of the base. USB and PWR face out through the back wall; the RPI4 socket faces a tunnel into the base, and a ~30 cm USB-C cable (right-angle at the Pi end) runs up through the bearing to the Pi. The splitter's socket positions (spl_*) are estimates from a product photo; confirm with test_pod.
• URDF meshes: export the moving parts in their assembled positions for hermes_description (the model already has pose_* parts for this; see enclosure/poster/README.md).
• Software implications: pan now rotates the whole cabinet, so turn_head pan should move smoothly (the interpolation in package 2 matters more), and the face's eyes should follow tilt and small glances only. Set ROBOT_FACE_ROTATION for the portrait-mounted display.

Package 7: Documentation

Rewrite README.md for this build: parts, the owner's physical steps from section 2, setup, tuning, troubleshooting (including the kernel/driver coupling from package 1). Keep it a guide for a person; this file stays as the engineering brief.

6. Design principles for the face

Proposed during design discussion; the owner may revise them. Until then, use them for judgement calls:

1. Show only real states. The face signals what the agent is actually doing: listening, thinking, speaking, looking. It does not perform emotions it has no basis for.
2. The voice trace is literal. The mouth is the real outgoing audio, not an animation timed to it.
3. It is plainly a machine. Oscilloscope phosphor, vector lines, no attempt at a realistic human face. Head mode is acceptable because it is clearly a wireframe instrument display, but flat is the default.
4. Gaze is honest. Where the face looks matches where the camera points.

7. Configuration reference (target)

Network (package 0b): host 10.55.0.1/30, robot 10.55.0.2/30, gadget functions NCM + ACM, fixed MAC addresses for both ends.

ROS: Jazzy, Fast DDS to start, fixed ROS_DOMAIN_ID, ROS_AUTOMATIC_DISCOVERY_RANGE=SUBNET.

The values below become ROS parameters in hermes_bringup (names may change to ROS style, e.g. face.mode); only the MCP token stays in an env file. Current targets:

ROBOT_MCP_TOKEN=<hex>            ROBOT_MCP_PORT=8765
ROBOT_CAM_SIZE=1296x972          ROBOT_CAM_HFLIP=0   ROBOT_CAM_VFLIP=0
ROBOT_SERVO_BACKEND=pwm          ROBOT_SERVO_MIN_US=500   ROBOT_SERVO_MAX_US=2400
ROBOT_PAN_LIMIT=80               ROBOT_TILT_LIMIT=45
ROBOT_PAN_INVERT=0               ROBOT_TILT_INVERT=0
ROBOT_PAN_TRIM_DEG=0             ROBOT_TILT_TRIM_DEG=0
ROBOT_FACE_MODE=flat             ROBOT_FACE_STYLE=phosphor
ROBOT_FACE_PHOSPHOR=P7           ROBOT_FACE_FPS=30
ROBOT_FACE_ROTATION=0            ROBOT_FACE_WEBHOOK_SECRET=   (empty = webhooks off)

8. Unverified items (check before relying on them)

Item	Why it's uncertain
WM8960 modules in Ubuntu's Pi kernel	Overlays confirmed; modules confirmed only in Raspberry Pi's kernel config. host/check-pi-platform.sh settles it.
WM8960 default mixer state on the BrainCraft	The in-kernel driver may start with paths muted; set once and store.
Adafruit Blinka on Ubuntu 24.04	GPIO backend differs from Raspberry Pi OS.
RViz in podman on both hosts' compositors	Display and GPU passthrough untested.
rmw_zenoh on Jazzy	Only matters from V2; check maturity then.
PWM sysfs chip number and permissions	Differs by Pi model and OS release.
Servo pulse range	Varies between servo makes; calibrate.
ST7789 frame rate from Python	Untested on this HAT; may need the kernel driver.
Hermes webhook events and signatures	Seen in release coverage, not checked in docs.
Hermes passing MCP images to the model	Not documented when the first build was written.
pipewire-pulse TCP config merge behaviour	Written from docs, untested on the Pi.
NCM gadget mode on the Pi 4's current kernel	Widely used, but not tested on this build; ECM is the fallback.
Omarchy's network stack	Not checked; the USB interface config must match it.
NixOS host config for the gadget interface	Needs writing and testing on the owner's NixOS machine.
USB-C power output of the Framework Desktop's ports	Unknown; only matters if powering without the splitter.
Splitter compatibility with the owner's supply	Pimoroni's version was designed for the official Pi supply.

9. Ground rules for the implementer

• Test on the Pi itself; the camera, codec, display and PWM cannot be meaningfully mocked.
• Keep the MCP server, audio, and face independent: any one failing must not stop the others.
• Never widen network exposure beyond the USB link (10.55.0.1 on the gadget interface) without asking.
• Ask the owner before anything that reboots the Pi, upgrades the kernel, or holds packages.
• ROS on the hosts runs only in podman containers; don't install ROS natively on NixOS or Omarchy.
• Keep the MCP tool names stable; they are Hermes's contract with the robot.
• Keep audio out of ROS, and keep any future motor safety (V2) out of ROS too: it lives in microcontroller firmware.

10. V2 direction: design for it, don't build it

The owner intends a mobile V2. V1 should make no choice that blocks it. Current thinking:

• Platform: the cabinet, yoke, tilt servo, face and audio move onto a differential-drive chassis. Turning in place replaces the pan turntable, so pan_joint disappears from the URDF and the base gains odom → base_link from wheel odometry. The USB link gives way to Wi-Fi (firewalled to the host, ideally over Tailscale); a Wi-Fi drop must stop the motors.
• Three layers of control, each able to veto the one above and to carry on without it:
  1. Reflex, on a Raspberry Pi Pico: motor control, encoders, bumper and proximity stops, speed limits, a watchdog that stops the motors if the Pi goes quiet, battery cut-off. It may speak micro-ROS, but its safety rules must not depend on ROS or the Pi.
  2. Autopilot, on the Pi 4 plus an added Jetson Orin Nano Super (8 GB): perception (free floor, people, doorways), mapping (slam_toolbox or Isaac ROS visual SLAM), Nav2, and skills: go to a place, follow, turn toward a voice, find an object, dock. A recent guide reports JetPack 7.2 runs Ubuntu 24.04 + Jazzy on the Orin Nano, with Isaac ROS 4.6 targeting that stack; verify with NVIDIA before buying. The owner's older Jetson Nano (Ubuntu 18.04) is fine for experiments only.
  3. Mind, Hermes on the host: goals in, summaries out. It never sends motor commands or receives raw video.
• MCP contract for V2: skills as long-running tools that return outcomes (go_to(place), follow(), dock(), status()), plus a way for Hermes to ask what has happened. How events reach Hermes unprompted is an open question.
• Environment: no stairs and no pets, so cliff sensors are optional; front bumper and a few forward distance sensors are required.
• Owner decisions still open: rooms the robot may not enter; a visible light whenever camera or mic is live; whether Hermes may drive on its own initiative (e.g. from a scheduled task).