Add hermes_face ROS package and verify the display on the Pi
The Pi (hermes-bot) is flashed, upgraded and running ROS 2 Jazzy. The ST7789 works through Blinka on lgpio: pass cs=None (the kernel owns CE0), and the glass sits at y_offset=80 in the controller RAM for every rotation, since the driver rotates in software. The mounted pose needs rotation=270. hermes_face is a numpy port of the flat face from docs/face-prototype.html, with phosphor and vector styles, plus a new `sleeping` pose for "no agent connected". The renderer has no ROS or hardware imports. The tone map is a lookup table and the glow runs at half resolution, so the face reaches ~28 fps on the panel at 24% of a core (20 fps) to 40% (30 fps) with the CPU at full clock. CONTEXT.md now records the Pi, the cloud-init first-boot-only gotcha, and what is left. pi/50-cloud-init.yaml is a gitignored Wi-Fi template. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AFN8nRRCiDiMN7ChBpHJhs
This commit is contained in:
Claude Sonnet 5.5
+67
-27
@@ -4,43 +4,83 @@ Last updated 2026-10-04. `HANDOFF.md` is the brief; this file tracks where the w
|
||||
|
||||
## Hardware on hand
|
||||
|
||||
Raspberry Pi 4, BrainCraft HAT, Arducam UC-346 camera. Not yet: servos, USB-C splitter, printed
|
||||
enclosure, speakers. Work until those arrive is limited to what the Pi, HAT and camera can do.
|
||||
Raspberry Pi 4 (8 GB), BrainCraft HAT, Arducam UC-346 camera. Not yet: servos, USB-C splitter,
|
||||
printed enclosure, speakers. Work until those arrive is limited to what the Pi, HAT and camera can do.
|
||||
|
||||
## The Pi (`hermes-bot`)
|
||||
|
||||
- Ubuntu 24.04.5 server, kernel `6.8.0-1065-raspi` (fully upgraded), flashed from `talos`. User
|
||||
`hermes` (key-only; passwordless sudo), SSH keys for `kyle@vade` (talos) and `kyle@straylight`.
|
||||
- On Wi-Fi at `192.168.88.152` (MAC `d8:3a:dd:0d:66:f5`); `eth0` is not cabled. The owner says it has
|
||||
a static IP on the rack switch; that address is not recorded here yet. `avahi-daemon` is
|
||||
installed, but mDNS is off in `systemd-resolved` on talos, so `hermes-bot.local` doesn't resolve
|
||||
there; use the IP.
|
||||
- **cloud-init applies `network-config` and `user-data` on the first boot only.** To change Wi-Fi
|
||||
later, edit `/etc/netplan/50-cloud-init.yaml` on the Pi, or bump `instance_id` in `meta-data` on
|
||||
`system-boot`. `pi/50-cloud-init.yaml` is a template for the Wi-Fi block; it is gitignored
|
||||
because it holds credentials.
|
||||
- Installed: ROS 2 Jazzy (`ros-base`, `ros-dev-tools`, colcon, `robot_state_publisher`, `xacro`,
|
||||
`camera_ros`); `python3-numpy/pil/spidev/lgpio/rpi-lgpio`; a venv `~/venv` (system site packages
|
||||
visible) with `adafruit-circuitpython-rgb-display` and Blinka. Workspace: `~/ros_ws`, source synced
|
||||
from `ros/` with `rsync -a --delete --exclude __pycache__ ros/ hermes@<ip>:~/ros_ws/src/`, then
|
||||
`colcon build --symlink-install`.
|
||||
- CPU governor is `ondemand`; at full clock the face costs about half as much CPU. Set
|
||||
`performance` in the installer (package 5). It is not persistent yet.
|
||||
|
||||
## Package 0a (platform proof on Ubuntu 24.04)
|
||||
|
||||
- **Step 1 done.** `host/check-pi-platform.sh` passes: Ubuntu's `6.8.0-1065-raspi` kernel ships
|
||||
every driver and overlay needed (WM8960 audio, OV5647, Unicam, PWM, SPI, dwc2 gadget with
|
||||
NCM/ECM/ACM). The script needed a fix for uncompressed `data.tar` (applied to `host/` and the
|
||||
duplicate in `tools/`; committed).
|
||||
- **Step 2 in progress: flashing the SD card.** Image chosen:
|
||||
`ubuntu-24.04.5-preinstalled-server-arm64+raspi.img.xz`, SHA256
|
||||
`b23371a5c8d02f612c26e7c18cacb81c5a48f968c7c66a2645391320aa94351e` (from
|
||||
cdimage.ubuntu.com/releases/24.04/release/SHA256SUMS). It was downloaded to the session
|
||||
scratchpad under `/tmp`, which a reboot likely cleared; re-download and re-verify.
|
||||
- The Pi is not running yet. No card has been written. `straylight` (NixOS; has `xz`, `dd`,
|
||||
`udisksctl`, no `rpi-imager`) never detected the Framework SD expansion card: no kernel event
|
||||
in two bays, so the flashing moves to **`talos`**, where the owner confirmed the card mounts.
|
||||
Continue the session there: clone the repo, read this file, re-download and re-verify the
|
||||
image (the copy on straylight is in its `/tmp`). Check talos's OS and available tools first.
|
||||
- **Open questions for the owner before flashing:** Wi-Fi credentials or Ethernet for first boot;
|
||||
hostname (proposed `hermes-robot`); username; SSH key (default `~/.ssh/id_ed25519.pub`).
|
||||
Plan: write the image after the owner confirms the device from `lsblk`, then edit cloud-init
|
||||
on the `system-boot` partition (user, key, hostname, SSH, network).
|
||||
- Remaining 0a steps after boot: ROS 2 Jazzy, audio (`wm8960-soundcard` overlay + mixer),
|
||||
camera (`cam -l`, `camera_ros`), display test pattern. PWM test waits for the servos. Ask
|
||||
before rebooting the Pi.
|
||||
- Step 1 done: `host/check-pi-platform.sh` passes.
|
||||
- Step 2 done: flashed, booted, upgraded; ROS installed; `ros2 topic list` works.
|
||||
- **Step 5 (display) done.** ST7789 works on `lgpio` through Blinka (`rpi-lgpio`, no `RPi.GPIO`).
|
||||
Two gotchas: pass `cs=None` (the kernel drives CE0; claiming GPIO8 raises "GPIO busy"), and the
|
||||
240x240 glass sits at `y_offset=80` in the controller's RAM, for every rotation (the driver
|
||||
rotates in software and leaves the scan direction fixed). Mounted pose (microSD end up, USB-C edge
|
||||
on the left) needs content turned 90 degrees clockwise: `rotation=270`. Full-frame push is
|
||||
~23 fps through the driver; the face pushes raw RGB565 with `_block` and reaches ~27-28 fps.
|
||||
- Not done: audio (`wm8960-soundcard` overlay + mixer), camera (`cam -l`, `camera_ros`), PWM
|
||||
(waits for servos). Ask before rebooting the Pi.
|
||||
|
||||
## Face (`ros/hermes_face`)
|
||||
|
||||
- `render.py`: numpy port of `docs/face-prototype.html`, **flat face only** (head and lowpoly not
|
||||
ported). Phosphor and vector styles. States: `sleeping idle listening thinking looking speaking`.
|
||||
`sleeping` (new) is the "no agent connected" pose: eyes closed to thin lines, mouth short, dim and
|
||||
slowly breathing. `looking` returns to the state it interrupted. Tone mapping is a lookup table
|
||||
(RGB and RGB565 tables), glow is computed at half resolution, rotation is a free `np.rot90`.
|
||||
- `display.py`: ST7789 driver plus an async pusher (render overlaps the SPI transfer).
|
||||
- `face_node.py`: node `hermes_face`; subscribes `/hermes/state` (`std_msgs/String`) and
|
||||
`/joint_states` (`pan_joint`, `tilt_joint`, radians); starts in `initial_state` (`sleeping`), so it
|
||||
runs with nothing else up. Parameters: `rotation` (270), `x_offset` (0), `y_offset` (80),
|
||||
`baudrate`, `fps`, `output` (`st7789` or `none`), `phosphor` (P7), `style`, `persistence_ms`,
|
||||
`focus`, `jitter`, `intensity`, `initial_state`.
|
||||
- `demo.py`: no ROS. `python -m hermes_face.demo --sequence` cycles the states on the panel;
|
||||
`--png-dir DIR` writes a PNG per state without hardware.
|
||||
- Tests: `python3 -m pytest test` in `ros/hermes_face` (23 pass on the Pi).
|
||||
- Measured on the Pi 4: ~9 ms per frame in isolation. With the CPU at full clock: 24% of a core at
|
||||
20 fps and 40% at 30 fps (target was under 40% at 20 fps or more). Under `ondemand` the same
|
||||
load reads 40% at 20 fps and ~67% at 30 fps. No C++ rewrite was needed.
|
||||
- Not yet: audio levels for the mouth (`/audio/levels`), `hermes_mcp` publishing states, launch
|
||||
file in `hermes_bringup`, head/lowpoly shapes, DotStar LEDs, button mute.
|
||||
- Open: IPS black level. The backlight is on at full brightness, so black reads dark gray on
|
||||
camera. Dimming it needs a software PWM on GPIO26 (not a hardware PWM pin).
|
||||
|
||||
## Next
|
||||
|
||||
1. Package 0b: USB gadget Ethernet (`dwc2` peripheral, NCM + ACM, `usb0` at `10.55.0.2/30`). Needs a
|
||||
reboot and a USB-C data cable to a host; the splitter has not arrived, so a bench test would
|
||||
power the Pi from the host's port (fine without servos). Keep Wi-Fi up throughout.
|
||||
2. Audio (package 1), camera (package 3), then the rest of the packages in `HANDOFF.md`.
|
||||
|
||||
## Repository
|
||||
|
||||
Git repo initialised on `main` with one commit ("Initial commit"). `.gitignore` covers
|
||||
`.remember/`, caches, `.venv/`, `*.orig`, `*.rej`. The earlier zip and patch files were deleted.
|
||||
`tools/check-pi-platform.sh` duplicates `host/check-pi-platform.sh` and can probably go.
|
||||
Branch `master`. `.gitignore` covers `.remember/`, caches, `.venv/`, `*.orig`, `*.rej` and
|
||||
`pi/50-cloud-init.yaml`. `tools/check-pi-platform.sh` duplicates `host/check-pi-platform.sh` and can
|
||||
probably go.
|
||||
|
||||
## Security review notes (parked until the USB link is up)
|
||||
|
||||
An automated review of the initial commit flagged three items. All stem from the code predating
|
||||
the USB-gadget design (HANDOFF.md package 0b); none is exposed until the Pi runs.
|
||||
the USB-gadget design (HANDOFF.md package 0b); none is exposed until the Pi runs the services.
|
||||
|
||||
- `pi/robot_mcp_server.py`: plain HTTP, `ROBOT_MCP_BIND` defaults to `0.0.0.0`. Plan: default to
|
||||
`10.55.0.2`, start after `usb0`.
|
||||
|
||||
Reference in New Issue
Block a user