Files
kyleandClaude Sonnet 5.5 720a7cfc34 Bring every doc up to date, and give the repo a Wi-Fi template that exists
CONTEXT.md had grown by appending and contradicted itself (all four nodes vs six, a retracted CPU
figure, "no calibration", "nobody enrolled", "audio untested as a service", the Wi-Fi-off boot as
not done, a class the tracker replaced). Rewritten as one current-state document organised by
subsystem, with fresh measurements, the owner's decisions, the gotchas, the neck's arrival-day plan
and what is genuinely still open.

README: status, speakers not fitted, new-session note for Hermes, the calibration section (shipped
calibration, coverage hints, outlier dropping, --from-views), running the tests, the docs row.
HANDOFF.md gets a status banner listing where the original brief has drifted. CLAUDE.md starts at
CONTEXT.md. The echo-cancel config no longer talks about a Pi Zero, and the enclosure guide no longer
tells you to centre a servo with a tool that does not exist yet.

pi/network-config.example is a tracked, credentials-free first-boot template: the README pointed at
pi/50-cloud-init.yaml, which is gitignored and so missing from a fresh clone. The dead IdentityCache
(replaced by the tracker) is removed. Docstrings corrected: audio scope publishing, the MCP health
topics, and what the calibration tool does.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AFN8nRRCiDiMN7ChBpHJhs
2026-10-05 08:38:32 -07:00

317 lines
25 KiB
Markdown

# Current state
Last updated 2026-10-05. This file says where the work really stands. `HANDOFF.md` is the original
brief and has drifted (see its banner); where they differ, this file wins. `README.md` is the guide for a
person using the robot.
## Hardware on hand
Raspberry Pi 4 (8 GB), BrainCraft HAT, Arducam UC-346 camera, and **nothing else: no servos, no fan, no
speakers**, no USB-C splitter, no printed enclosure. The camera hangs loose on its ribbon and is held by
hand. The Pi is powered from talos's USB port (a bench setup, fine without servos). It runs at about
60 °C open-air with everything on; an enclosure will be hotter, so fit the fan before enclosing it.
Playback and the servos are untested because they cannot be.
## What works (verified on the real robot)
- **Face** on the display, started at power-up, with a `sleeping` pose that means "no agent connected".
The eyes glance toward detected faces; the mouth draws the live voice. Gaze direction and the voice
trace were confirmed by the owner on the real display.
- **Camera**, **face detection**, **opt-in recognition** (the owner is enrolled and recognised live).
- **MCP bridge**: `hermes mcp test robot` passes; real Hermes turns on the local `ornith` model call
`look()` and `faces()`.
- **USB network link**, **firewall**, and the whole robot **boots and works with Wi-Fi off**.
- **Thermal guard**, and a reboot-tested **single installer**, `pi/install.sh` (re-run on the live robot,
it changed nothing; **not yet run on a blank card**).
- 122 unit tests on the Pi: face 50, vision 30, audio 10, MCP 16, thermal 12, bringup 4.
## The Pi (`hermes-bot`)
- Ubuntu 24.04.5 server, kernel `6.8.0-1065-raspi`, flashed from talos. User `hermes` (key-only login,
passwordless sudo; no password is set). SSH keys for `kyle@vade` (talos) and `kyle@straylight`.
- Network: USB link `10.55.0.2` (host `10.55.0.1`); Wi-Fi `192.168.88.152` (MAC `d8:3a:dd:0d:66:f5`);
`eth0` is not cabled (the owner mentioned a static address on the rack switch; not recorded here).
`avahi-daemon` is installed, but mDNS is off in `systemd-resolved` on talos, so use the IP. `rfkill`
is installed.
- **cloud-init applies `network-config` and `user-data` on the first boot only.** To change Wi-Fi later,
edit `/etc/netplan/50-cloud-init.yaml` on the Pi, or bump `instance_id` in `meta-data` on
`system-boot`. `pi/network-config.example` is a credentials-free template for the Wi-Fi block (`pi/50-cloud-init.yaml`
is gitignored, for a filled copy).
- **Install:** `rsync` this repo to the Pi and `sudo ./pi/install.sh`. It installs ROS 2 Jazzy, builds
the Raspberry Pi libcamera fork (pinned commit) into `/opt/libcamera-rpi` and `camera_ros` against it
(`~/camera_ws`), makes `~/venv` (display libraries, OpenCV 4.10, the MCP SDK), downloads the face
models to `~/models`, restores the camera calibration, builds `~/ros_ws`, sets up the USB gadget, the
audio overlay and mixer, the firewall (before anything listens), PipeWire and the services, and
creates the MCP token. Update just the code with `rsync -a --delete --exclude __pycache__ ros/
hermes@<ip>:~/ros_ws/src/`, `colcon build --symlink-install` in `~/ros_ws`, `sudo systemctl restart
robot`. Any shell that runs a node by hand must `source ~/robot-env.sh` first.
- **Services:** `robot.service` runs `ros2 launch hermes_bringup robot.launch.py` as `hermes`: the face,
camera, face detector, audio levels, MCP bridge and thermal guard, each its own process respawned by
the launch file. The face starts first; the others follow `start_delay` (6 s). Parameters:
`ros/hermes_bringup/config/robot.yaml`. `RuntimeDirectory=robot` gives lgpio a writable working
directory (it drops a pipe in its cwd; systemd's default `/` is not writable). Also
`robot-cpu-performance.service` (governor `performance`; the face costs about half the CPU at full
clock), `robot-firewall.service`, `robot-usb-gadget.service`. `pi/trim-boot.sh` disables snapd and
ModemManager (unused; ~9 s of boot CPU).
- **Boot:** the face draws about 28 s after the kernel starts (kernel ~7 s, `robot.service` ~10 s into
userspace, `ros2 launch` itself ~10 s on a cold Pi, the face ~3 s); the other nodes are up by ~37 s.
The owner is fine with up to a minute and prefers everything in ROS: a split face unit that skipped
`ros2 launch` saved only 3.7 s and was dropped. Reboot-tested: everything returns by itself.
- **CPU**, full stack, 4 cores: face ~84%, camera ~63% (15 fps; 30 fps alone is ~40%, 15 is ~20%),
detector ~45%, audio ~15%, MCP ~15%, PipeWire ~12%; about 40% of the machine idle, 60 °C, no
throttling. OpenCV's DNN spins every core and burns 3-4x its elapsed time in CPU, so the detector pins
it to one thread (YuNet 86 ms, SFace 161 ms of CPU), decodes the JPEG at half size and re-identifies a
tracked face at most once a second.
- **ROS never leaves the Pi** (owner's decision): `ROS_AUTOMATIC_DISCOVERY_RANGE=LOCALHOST` (ROS's
setup script exports `SUBNET`, so `pi/robot-env.sh` assigns it) and the firewall drops domain 42's DDS
ports (17900-18149) on every interface but loopback. `hermes_mcp` is the only bridge out. This
supersedes HANDOFF's `host/ros-shell`.
- **Firewall** (`pi/nftables-robot.nft`, loaded at boot before networking by `robot-firewall.service`):
accepts 4713 (audio), 8765 (MCP) and 8766 (reserved, nothing listens) only on `usb0` from `10.55.0.1`
and drops them elsewhere. SSH is deliberately open on all interfaces (key-only), for V2: the owner's call.
- **PipeWire** 1.0.5 / WirePlumber 0.4.17 run as `hermes`'s user services (`loginctl enable-linger`).
`pi/pipewire/50-network-server.conf` makes pipewire-pulse listen on TCP 4713 on all interfaces; the
firewall is the protection. Defaults are the WM8960 source and sink.
## Display and the face (`ros/hermes_face`)
- **Display:** the ST7789 works on `lgpio` through Blinka (`rpi-lgpio`, not `RPi.GPIO`). Pass `cs=None`
(the kernel drives CE0; claiming GPIO8 raises "GPIO busy"). The 240x240 glass sits 80 rows into the
controller's RAM, `y_offset=80`, for every rotation (the driver rotates in software and leaves the
scan direction fixed). The mounted pose (microSD end up, USB-C edge on the left) needs the content
turned 90° clockwise: `rotation=270`. The face pushes raw RGB565 with `_block` at ~27-28 fps.
- **Renderer** (`render.py`, no ROS imports): a numpy port of `docs/face-prototype.html`, flat face only
(head and low-poly shapes are not ported). Phosphor and vector styles; tone map is a lookup table,
glow at half resolution, rotation a free `np.rot90`; parameters are clamped. States: `sleeping idle
listening thinking looking speaking`; `looking` returns to the state it interrupted.
- **Node** (`face_node.py`): subscribes `/hermes/state`, `/joint_states`, `/vision/faces`,
`/camera/camera_info`, `/audio/activity`, `/audio/scope/{mic,speaker}`; starts in `sleeping`; logs
every state change (`journalctl -u robot`), so behaviour can be checked without watching the panel.
`fps` can be changed live (the thermal guard does).
- **State arbitration** (`arbiter.py`): sleeping (agent side only) > speaking > listening > thinking >
idle. **Room sound never wakes a sleeping face.**
- **Agent heartbeat:** `/hermes/state` doubles as the heartbeat. After `agent_timeout_s` (10) without
it the face logs `agent gone, sleeping` and sleeps.
- **Gaze:** eyes glance toward the best face (a named one first, then the biggest), smoothed, held
2.5 s, then drift back; sleeping ignores it. Image x flips on the display (a person on the image's
right is on the robot's right, the viewer's left). The renderer's pan sign was wrong (inherited from
the prototype) and is fixed: positive pan = the robot's left = the viewer's right; positive tilt = up,
as the MCP `turn_head` documents. Never seen with a moving head.
- **Voice trace:** while fresh scope data exists, the mouth draws the real waveform (speaker when
speaking, mic when listening), auto-scaled; the owner confirmed it follows how close and loud they are.
- Not done: head/low-poly shapes, DotStar LEDs, a button mute. Open: the IPS panel's black reads dark
gray on camera at full backlight; dimming needs a software PWM on GPIO26.
## Camera and vision (`ros/hermes_vision`, `camera_ros`)
- **Camera: only the Raspberry Pi libcamera fork works.** The OV5647 is detected (`ov5647 10-0036`), but
Ubuntu's libcamera 0.2.0 dies in the Pi IPA (`assertion "it != buffers_.end()" in prepareIsp()`) and
ROS Jazzy's 0.7.2 with `FATAL … A list of V4L2 controls requires a ControlInfoMap`, deterministically
for every format and size. The fork (commit `6c1dd9d`, built with `-Dpipelines=rpi/vc4 -Dipas=rpi/vc4
-Dv4l2=false …` into `/opt/libcamera-rpi`) captures; `camera_ros` 0.7.0 is built from source against
it, and the fork must come first in `LD_LIBRARY_PATH` (`robot-env.sh` does it). Stream: 1296x972 at
15 fps (`FrameDurationLimits` in `robot.yaml`); `/camera/image_raw`, `…/compressed`, `camera_info`.
`pi/capture_still.py` takes a still with the fork's `cam`. Frames are padded and B,G,R in memory.
- **Calibration done** (`pi/calibration/…_1296x972.yaml`, restored by the installer): 18 of 24 views,
reprojection error 0.296 px, focal length ~1153 px, centre (653, 463), small distortion with `k3` held
at zero. The first pass fit 0.41 px but with cancelling k1/k2/k3 because the board never reached the
edges; the tool now fixes `k3`, reports coverage, drops blurred outlier views and can re-solve saved
views (`--from-views`). **The top-left and bottom-right corners were never reached**, so the model is
least certain there. The result is valid for the current focus and image size; a changed focus
invalidates it. Tool: `python3 -m hermes_vision.calibrate --cols 9 --rows 6 --square-mm <mm>`
(`docs/checkerboard-9x6.png` can be shown on a screen; the square size only scales distance, not the
intrinsics).
- **Detection** (`detect.py`, no ROS imports): OpenCV YuNet 2023mar (Haar fallback). It needs OpenCV
4.8+ (apt has 4.6), so `~/venv` carries `opencv-python-headless==4.10.0.84` with `numpy<2` (the system
NumPy stays 1.26.4 for ROS), and the venv is first on `PYTHONPATH`; input sides must be multiples of 32.
- **Tracking** (`tracking.py`): faces are matched across frames by box overlap, so a person keeps one
track id; identity is re-resolved at most once a second.
- **Node** `face_detector` publishes `/vision/faces` (`vision_msgs/Detection2DArray`): boxes in
full-size camera pixels, `Detection2D.id` = the track id, `results[0]` the face, `results[1]` the
identity if recognised (class_id = name, score = cosine similarity, not a probability). An unknown
face has no `results[1]`: a name is never made up. `rate_hz` (4) can be changed live.
- **Recognition is opt-in** (`recognize.py`, `people.py`): SFace embeddings on YuNet's landmarks. Nothing
is stored until someone is enrolled with `python3 -m hermes_vision.people enroll NAME`; only
128-number embeddings are kept, in `~/.local/share/hermes/faces.json` (mode 0600, dir 0700, written
atomically), never images. `list`, `forget`, `check`, `enroll-images` too. **The owner is enrolled
(10 samples) and was recognised in 29 of 31 detections live (similarity ~0.9).** A throwaway-database
test recognised 8 of 9 unenrolled frames of the same person, down to side profiles; an unrelated photo
scored 0.16. Not tested: a second person live. Names are never logged (a count is).
## Audio (`ros/hermes_audio`, `pi/overlays`, PipeWire)
- **The stock `wm8960-soundcard` overlay fails on Ubuntu 24.04** (`wm8960: No MCLK configured`): it puts
the 12.288 MHz fixed clock on the card's codec link, but the codec driver reads `mclk` from its own
node. `pi/overlays/hermes-wm8960.dts` adds it there; `pi/install-audio-overlay.sh` compiles it, sets
`dtparam=audio=off` and `dtoverlay=hermes-wm8960`. Capture then opens at 8-48 kHz. Address the card
by name (`hw:CARD=wm8960soundcard,DEV=0`): its index changes.
- **The HAT's hardware Audio ON/OFF switch (right edge) must be ON.** With it off, the codec still
answers on I²C and every software check looks right, but all recordings are exactly zero.
- **Mixer:** `Left/Right Input Mixer Boost` default to off and must be on. `pi/wm8960_asound.state` holds
the settings (capture 47, boost 1). **WirePlumber resets the codec's capture gain to +30 dB** when it
takes the device, so the installer sets PipeWire's source volume to 63% (its scale is cubic: -12 dB,
ALSA `Capture` 46, about +17 dB), which WirePlumber remembers; survives reboots (checked). Quiet-room
floor at this gain: about -43 dBFS; singing at arm's length peaks around -18 dBFS.
- **Node** `audio_state` taps the default source and sink monitor with `parec` (16 kHz mono) and
publishes `/audio/levels`, `/audio/activity` (speaking | listening | quiet) and trigger-locked
waveform windows `/audio/scope/{mic,speaker}` (only while listening or speaking). Audio itself never
goes through ROS. `Activity`: 3 s start-up calibration of an adaptive noise floor, margin 7 dB, 1.2 s
hold so pauses between words do not flicker, steady noise is absorbed after 8 s, the mic is ignored
while speaking. **The flicker retune is unit-tested but not re-confirmed live.**
- Verified: the `speaking` path (a tone played to the sink, no speaker attached, shows on the monitor);
the mic path with the owner's voice. Playback to a real speaker, echo cancel (`pi/pipewire/
60-echo-cancel.conf.optional`) and the host's `robot_mic` naming wait for the speakers.
## The MCP bridge and Hermes (`ros/hermes_mcp`)
- Serves MCP (SDK 2.3, streamable HTTP) on `http://10.55.0.2:8765/mcp`: the USB address only (it retries
the bind until the link is up), bearer token in `/etc/robot-mcp.env` (root:hermes, mode 640, made by
the installer, never printed; read it with `sudo cat`). **Tool names are the stable contract with
Hermes.** Tools: `look(send_image="always"|"never")` (the faces summary as JSON text plus the next
fresh JPEG, never a stale frame; it also makes the face squint), `faces()` (the faces in view, no
photo: `track_id`, `bbox`, `confidence`, `seen_for_s`, `identity` `{name, similarity}` or `null`),
`list_people()` (names and sample counts only), and `turn_head` / `head_position` **only when a head
is fitted** (`head: false` now; the node refuses `head: true` until `hermes_head` exists). Resource
`robot://status` (hardware present or absent, what is detected, enrolled count, tools, and health:
temperature, thermal level, throttle flags now and since boot) and prompts `who_is_here`,
`look_around` make it discoverable. Sessions are stateful on purpose (see the Hermes note below).
- **Heartbeat:** while an authenticated client is connected (in flight, or heard within
`connected_grace_s` = 30) it republishes `idle` on `/hermes/state` once a second; the face wakes on
connect and sleeps about 40 s after the client leaves.
- **Hermes on talos is wired to it** (with the owner's authorisation). Hermes Agent v0.21.5 is in
`~/.hermes` (hosted main model `grok-4.6` via xai-oauth; vision is auxiliary and local,
`qwen3-vl-8b-abliterated` on straylight; the `ornith-*` models are the local ones on straylight and
dixie). `config.yaml` has an `mcp_servers.robot` block and `.env` has `ROBOT_MCP_TOKEN`; backups:
`config.yaml.bak-20261005T002659-robot-mcp`, `.env.bak-20261005T002659-robot-mcp`. `hermes mcp test
robot` passes. A fresh `HERMES_HOME` hangs on first-run setup, so test against the real one.
- **Real agent turns, entirely on local models:** `hermes chat -q "..." --oneshot -Q --provider
custom:llama-router -m ornith-1.5-35b-a3b --max-turns 5`. `look()` works; **Hermes does not hand MCP
images to the main model**: it saves the JPEG to `~/.hermes/cache/images/` on talos and returns a short
reference, and the model calls `vision_analyze` (local vision model, ~29 s). So a look-and-describe turn
takes ~70 s, and each look leaves a photo in talos's Hermes cache (Hermes's retention). `faces()` is
the cheap path: 0.03 s, no image, and Hermes answered "Kyle is at the desk, similarity 0.78".
- **Hermes's feedback (`~/.hermes/cache/scratch/hermes-bot-v0-face-identity.md`).** Built: `faces()`,
`look` returning that JSON, `list_people`, `robot://status` and prompts (previously empty), track ids.
**Declined, deliberately:** embeddings in responses (biometric templates, meaningless to an LLM; the
registry stays on the robot); age/gender/hat (no model, and unreliable and sensitive); auto-updating
face data from new captures (lets a stranger slowly become "Kyle"); a remote `enroll` tool (a prompt
could make the robot call anyone "Kyle": enrolment stays the owner's step; a gated tool, off by
default, is possible if wanted); detecting people, objects or motion (only faces exist). `look()`'s
default stays photo-plus-faces, because the photo is what lets Hermes describe a scene: **flipping it
to JSON-only is the owner's call.**
- **Hermes bug report "faces and list_people are not registered" was not a bridge bug.** A raw client and a
fresh `hermes mcp test` list all three; Hermes's own `agent.log` shows it registering all seven tools
at 01:18:46 after a robot restart, and a later session calling `faces()` fine. The reporting session
started at 00:35, before `faces` existed, and keeps a start-time snapshot of its tool table (live calls
like `list_prompts` still work, which matches the report). Fix: start a new Hermes session after the
robot's tool set changes; Hermes should refresh a session's tool table on re-registration. The bridge
keeps stateful sessions on purpose: the `Session not found` after a restart is what makes Hermes
reconnect and re-register.
## Thermal guard (`ros/hermes_thermal`)
The Pi has no fan. `thermal_guard` reads the CPU temperature and `vcgencmd get_throttled`, publishes
`/health/temperature`, `/health/thermal_level` (ok | warm | hot | critical) and `/health/throttled`, and
sheds load through the target nodes' live parameters: the camera's `FrameDurationLimits`, the face's `fps`,
the detector's `rate_hz`. Levels enter at 72 / 77 / 82 °C (the Pi throttles at ~80); heating is acted on
at once, cooling goes one level at a time only below the threshold minus 3 °C and after a 20 s dwell.
Rates scale by 1.0 / 0.7 / 0.45 / 0.2 with floors (camera 2 fps, face 8, detector 1 Hz). Its "normal"
rates are its own parameters, and a bringup test asserts they equal the real ones in `robot.yaml`, so
restoring is exact and survives a restart. `fake_temp_c` (`ros2 param set /hermes_thermal fake_temp_c
85.0`, -1 to stop) forces a temperature. **Verified live** by forcing 73 / 78 / 85 °C and back: measured
camera rate 16.5 -> 11.5 -> 7.2 -> 3.5 Hz and back, cooling stepped down one dwell apart, same process
throughout. **A live bug the unit tests missed:** one log line that warned when heating and informed when
cooling raised rclpy's `Logger severity cannot be changed between calls` and killed the guard on its
first cooling step (the respawn hid it). Fixed (separate call sites), the tick now survives any error, and
a test drives the real node through heat-then-cool; a mutation check confirmed it fails on the original
bug. Not yet tested under a real thermal load.
## USB link (HANDOFF package 0b)
- Pi: `dtoverlay=dwc2,dr_mode=peripheral`; `robot-usb-gadget.service` builds a configfs gadget (NCM `usb0`
+ ACM `ttyGS0`; host MAC `02:68:65:72:6d:01`, robot MAC `…:02`) before networking;
`/etc/netplan/60-usb0.yaml` gives `usb0` `10.55.0.2/30` with no gateway (the default route stays on
Wi-Fi). NCM works on the Pi 4 (ECM was not needed). **netplan must match `usb0` by name, not MAC:** a
`macaddress:` match becomes `PermanentMACAddress=`, which a gadget interface lacks, so it stays unmanaged.
- Host (Omarchy, NetworkManager): `sudo host/omarchy/robot-usb-link.sh` adds profile `hermes-robot-usb`
(`10.55.0.1/30`, matched by the MAC, never-default). Ping ~2 ms.
- **Wi-Fi-off acceptance passed:** with Wi-Fi blocked (`rfkill block wifi`, remembered across reboots),
live and after a reboot, the Pi came up on the USB link alone and everything worked. With no internet
there is no NTP, so the clock is unsynchronised (roughly right).
- Not yet verified: replug and second-host recovery, and the serial console login (talos's `/dev/ttyACM0`
is `root:uucp`; `hermes` has no password, so the console shows a prompt but cannot log in; no autologin
by design).
## Owner's decisions
ROS never leaves the Pi (`hermes_mcp` is the only bridge); SSH stays open for V2; NixOS host config is
skipped for now; no remote `enroll`; `look()`'s default is open; the face may take up to a minute to
appear at boot; everything should be ROS. A new Omarchy host with a GPU (it runs the local `ornith`
model) is arriving: it needs only `host/omarchy/robot-usb-link.sh` and its SSH key on the Pi.
## Gotchas
- `ros2 node list` with its default short window shows only some nodes: use `--no-daemon --spin-time 6`.
- A one-off message published before ROS discovery matches is lost (volatile QoS): republish, as
`hermes_mcp` does every second. Anything that sets `PYTHONPATH` by hand must prepend, not replace.
- A shell that sourced `robot-env.sh` before a build cannot import the newly built package: re-source.
- rclpy needs each logging call site to keep one severity.
## Arrival day: the neck (servos), planned and deliberately not built ahead of the hardware
Expected Thursday or Friday at the earliest. What matters cannot be faked (which `pwmchip` the overlay
creates, its permissions on Ubuntu, each servo's real pulse range, whether they brown out the Pi), so
`hermes_head` waits. Order, with what to verify:
1. **Wire it:** pan servo to the HAT's GPIO12 port, tilt to GPIO13; the "3pin Volt" jumper is already
verified at 5 V. Servo spikes on a host USB port may brown the Pi out: use the 5.1 V 3 A supply
through the splitter if it has arrived. Horns off for the first moves.
2. **Overlay (needs a reboot, ask first):** add `dtoverlay=pwm-2chan,pin=12,func=4,pin2=13,func2=4` under
`[all]` in `config.txt` (`dtparam=audio=off` is already set: the Pi's headphone audio shares PWM
hardware). Add it to `pi/install-*.sh`, not by hand.
3. **sysfs:** find the chip (`/sys/class/pwm/pwmchip*`; do not assume 0), export channels 0 and 1,
`period=20000000`. A udev rule should give a group, not root, write access.
4. **Measure each servo:** centre (1.5 ms), then the safe min and max pulse with the horn off; set
`ROBOT_SERVO_MIN_US` / `MAX_US` per servo (start 500 / 2400), the trims and invert flags.
5. **Write `hermes_head`:** `PwmHead` on sysfs, a ~0.3 s interpolated move, PWM off after ~1 s idle,
limits (pan 80, tilt 45), `/joint_states` for `pan_joint` / `tilt_joint` in radians (positive pan =
the robot's left, positive tilt = up). Unit-test against a fake sysfs tree, then verify on the Pi:
`vcgencmd get_throttled` stays `0x0` through a full sweep with the face running; watch the
under-voltage flag in `robot://status`.
6. **Wire it in:** give the bridge a head backend that calls `hermes_head`, set `head: true`, check
`turn_head` / `head_position` appear to a new Hermes session, and that the face's eyes follow the real
joints the right way.
7. **URDF:** `hermes_description` from the enclosure STLs' `pose_*` parts, frames `base_link`, `pan_link`,
`tilt_link`, `camera_link`, `camera_optical_frame`.
8. **Mounted camera:** set the flip (`orientation`) for how it is mounted, re-check the focus (a changed
focus invalidates `pi/calibration/`), and the display rotation if it is mounted differently.
## Open items
1. Waiting on hardware: servos (above), speakers (playback, echo cancel, Hermes talking aloud, the host's
`robot_mic` naming), the fan, the splitter, the printed enclosure (measure first: `enclosure/README.md`).
2. Unconfirmed: the retuned listening detector live; a second person in frame; replug and second-host
recovery; the serial console; `agent_timeout_s` over a long idle Hermes session; the calibration's
uncovered corners (retake with the board in the top-left and bottom-right); `pi/install.sh` on a
blank card (back up the working SD card first: it holds all of this).
3. Optional: a gated `enroll` tool, if the owner wants one.
## Repository
Branch `master`. `.gitignore` covers `.remember/`, caches, `.venv/`, `*.orig`, `*.rej` and
`pi/50-cloud-init.yaml`. `ros/` has `hermes_face`, `hermes_vision`, `hermes_audio`, `hermes_mcp`,
`hermes_thermal` and `hermes_bringup`; `pi/` is everything installed on the Pi; `host/` is the owner's
machines (`host/omarchy/`: the USB link script, Hermes and PipeWire config, the launcher;
`host/check-pi-platform.sh`); `enclosure/`; `docs/` (`face-prototype.html`, `checkerboard-9x6.png`).
Run a package's tests on the Pi with `python3 -m pytest test` in its directory after `source
~/robot-env.sh` (the face suite takes about a minute).
## Security notes (all resolved)
The automated review of the initial commit flagged `ROBOT_MCP_BIND` defaulting to `0.0.0.0` (now the USB
address only, with a bearer token), PipeWire on TCP 4713 on every interface (now behind the boot-time
firewall, which loads first), and a single-IP any-interface nftables file (rewritten). Later reviews led
to: DDS blocked on every interface and local-only discovery; recognised names never logged; node
parameters clamped. The token and the owner's Wi-Fi password never go through the repo or logs (one
`netplan get` once printed the Wi-Fi password into a session; it was flagged to the owner).