hermes-bot/CONTEXT.md

# Current state

Last updated 2026-10-04. `HANDOFF.md` is the brief; this file tracks where the work stands.

## Hardware on hand

Raspberry Pi 4 (8 GB), BrainCraft HAT, Arducam UC-346 camera. Not yet: servos, USB-C splitter,
printed enclosure, speakers. Work until those arrive is limited to what the Pi, HAT and camera can do.

## The Pi (`hermes-bot`)

- Ubuntu 24.04.5 server, kernel `6.8.0-1065-raspi` (fully upgraded), flashed from `talos`. User
  `hermes` (key-only; passwordless sudo), SSH keys for `kyle@vade` (talos) and `kyle@straylight`.
- On Wi-Fi at `192.168.88.152` (MAC `d8:3a:dd:0d:66:f5`); `eth0` is not cabled. The owner says it has
  a static IP on the rack switch; that address is not recorded here yet. `avahi-daemon` is
  installed, but mDNS is off in `systemd-resolved` on talos, so `hermes-bot.local` doesn't resolve
  there; use the IP.
- **cloud-init applies `network-config` and `user-data` on the first boot only.** To change Wi-Fi
  later, edit `/etc/netplan/50-cloud-init.yaml` on the Pi, or bump `instance_id` in `meta-data` on
  `system-boot`. `pi/50-cloud-init.yaml` is a template for the Wi-Fi block; it is gitignored
  because it holds credentials.
- Installed: ROS 2 Jazzy (`ros-base`, `ros-dev-tools`, colcon, `robot_state_publisher`, `xacro`,
  `camera_ros`); `python3-numpy/pil/spidev/lgpio/rpi-lgpio`; a venv `~/venv` (system site packages
  visible) with `adafruit-circuitpython-rgb-display` and Blinka. Workspace: `~/ros_ws`, source synced
  from `ros/` with `rsync -a --delete --exclude __pycache__ ros/ hermes@<ip>:~/ros_ws/src/`, then
  `colcon build --symlink-install`.
- CPU governor is `ondemand`; at full clock the face costs about half as much CPU. Set
  `performance` in the installer (package 5). It is not persistent yet.

## Package 0a (platform proof on Ubuntu 24.04)

- Step 1 done: `host/check-pi-platform.sh` passes.
- Step 2 done: flashed, booted, upgraded; ROS installed; `ros2 topic list` works.
- **Step 5 (display) done.** ST7789 works on `lgpio` through Blinka (`rpi-lgpio`, no `RPi.GPIO`).
  Two gotchas: pass `cs=None` (the kernel drives CE0; claiming GPIO8 raises "GPIO busy"), and the
  240x240 glass sits at `y_offset=80` in the controller's RAM, for every rotation (the driver
  rotates in software and leaves the scan direction fixed). Mounted pose (microSD end up, USB-C edge
  on the left) needs content turned 90 degrees clockwise: `rotation=270`. Full-frame push is
  ~23 fps through the driver; the face pushes raw RGB565 with `_block` and reaches ~27-28 fps.
- Not done: audio (`wm8960-soundcard` overlay + mixer), camera (`cam -l`, `camera_ros`), PWM
  (waits for servos). Ask before rebooting the Pi.

## Face (`ros/hermes_face`)

- `render.py`: numpy port of `docs/face-prototype.html`, **flat face only** (head and lowpoly not
  ported). Phosphor and vector styles. States: `sleeping idle listening thinking looking speaking`.
  `sleeping` (new) is the "no agent connected" pose: eyes closed to thin lines, mouth short, dim and
  slowly breathing. `looking` returns to the state it interrupted. Tone mapping is a lookup table
  (RGB and RGB565 tables), glow is computed at half resolution, rotation is a free `np.rot90`.
- `display.py`: ST7789 driver plus an async pusher (render overlaps the SPI transfer).
- `face_node.py`: node `hermes_face`; subscribes `/hermes/state` (`std_msgs/String`) and
  `/joint_states` (`pan_joint`, `tilt_joint`, radians); starts in `initial_state` (`sleeping`), so it
  runs with nothing else up. Parameters: `rotation` (270), `x_offset` (0), `y_offset` (80),
  `baudrate`, `fps`, `output` (`st7789` or `none`), `phosphor` (P7), `style`, `persistence_ms`,
  `focus`, `jitter`, `intensity`, `initial_state`.
- `demo.py`: no ROS. `python -m hermes_face.demo --sequence` cycles the states on the panel;
  `--png-dir DIR` writes a PNG per state without hardware.
- Tests: `python3 -m pytest test` in `ros/hermes_face` (23 pass on the Pi).
- Measured on the Pi 4: ~9 ms per frame in isolation. With the CPU at full clock: 24% of a core at
  20 fps and 40% at 30 fps (target was under 40% at 20 fps or more). Under `ondemand` the same
  load reads 40% at 20 fps and ~67% at 30 fps. No C++ rewrite was needed.
- Not yet: audio levels for the mouth (`/audio/levels`), `hermes_mcp` publishing states, launch
  file in `hermes_bringup`, head/lowpoly shapes, DotStar LEDs, button mute.
- Open: IPS black level. The backlight is on at full brightness, so black reads dark gray on
  camera. Dimming it needs a software PWM on GPIO26 (not a hardware PWM pin).

## Next

1. Package 0b: USB gadget Ethernet (`dwc2` peripheral, NCM + ACM, `usb0` at `10.55.0.2/30`). Needs a
   reboot and a USB-C data cable to a host; the splitter has not arrived, so a bench test would
   power the Pi from the host's port (fine without servos). Keep Wi-Fi up throughout.
2. Audio (package 1), camera (package 3), then the rest of the packages in `HANDOFF.md`.

## Repository

Branch `master`. `.gitignore` covers `.remember/`, caches, `.venv/`, `*.orig`, `*.rej` and
`pi/50-cloud-init.yaml`. `tools/check-pi-platform.sh` duplicates `host/check-pi-platform.sh` and can
probably go.

## Security review notes (parked until the USB link is up)

An automated review of the initial commit flagged three items. All stem from the code predating
the USB-gadget design (HANDOFF.md package 0b); none is exposed until the Pi runs the services.

- `pi/robot_mcp_server.py`: plain HTTP, `ROBOT_MCP_BIND` defaults to `0.0.0.0`. Plan: default to
  `10.55.0.2`, start after `usb0`.
- `pi/pipewire/50-network-server.conf`: TCP 4713 on all interfaces, unrestricted. Plan: keep,
  comment that the firewall is the protection.
- `pi/nftables-robot.nft`: still the Pi Zero version (single host IP, any interface). Package 0b
  rewrites it to accept only on the USB interface from `10.55.0.1`.
- `pi/install.sh` permissions looked fine (`/etc/robot-mcp.env` is `root:$USER`, 640).

The owner wants to confirm the real IPs on the link before changing any of this. If the USB
link doesn't work, revisit: TLS or a Tailscale-only bind for MCP and audio.