Add hermes_vision, build camera_ros on the fork, and bring up WM8960 audio

hermes_vision wraps OpenCV YuNet (Haar fallback) as a plain module plus a node
that publishes /vision/faces as vision_msgs/Detection2DArray. It replaces the
standalone detect_faces.py. pi/robot-env.sh puts the Raspberry Pi libcamera fork
ahead of ROS's own copy, the venv's OpenCV 4.10 ahead of the apt one, and fixes
the ROS domain and subnet-only discovery. camera_ros 0.7.0 now runs against the
fork and publishes raw and compressed images.

The stock wm8960-soundcard overlay fails on Ubuntu 24.04 with "No MCLK
configured": it hangs the clock on the sound card's codec link, but the codec
driver reads mclk from its own node. hermes-wm8960.dts adds it there, and
capture opens at 8-48 kHz. The microphones still read digital zeros; see
CONTEXT.md for what has been ruled out.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AFN8nRRCiDiMN7ChBpHJhs
This commit is contained in:
2026-10-04 22:55:53 -07:00
co-authored by Claude Sonnet 5.5
parent 5e48b6fa03
commit 8d5033d0a2
14 changed files with 440 additions and 107 deletions
+29 -6
View File
@@ -48,12 +48,35 @@ printed enclosure, speakers. Work until those arrive is limited to what the Pi,
`cam` then captures 1296x972 RGB888 at ~16 fps (dim room). `pi/capture_still.py` takes a still
with it. **`camera_ros` still has to be built against this fork** (the apt one uses ROS's broken
libcamera). Frames are padded (stride 3936) and B,G,R in memory.
- Face detection: `pi/detect_faces.py` (OpenCV YuNet, Haar fallback). YuNet 2023mar needs OpenCV
4.8+, so `~/venv` has `opencv-python-headless==4.10.0.84` with `numpy<2` (the system NumPy stays
1.26.4 for ROS); YuNet also needs input sides in multiples of 32. Verified on a sample image.
First camera shot was nearly black (mean 5/255): the lens was facing something close and dim.
- Not done: audio (`wm8960-soundcard` overlay + mixer; `alsa-utils` is installed, no card yet), PWM
(waits for servos). Ask before rebooting the Pi.
- **`camera_ros` 0.7.0 is built from source against the fork** in `~/camera_ws` (overlay on ROS; at
runtime the fork must come first in `LD_LIBRARY_PATH`, which `source ~/robot-env.sh` does; copy
from `pi/robot-env.sh`). Verified: `ros2 run camera_ros camera_node --ros-args -p width:=1296 -p
height:=972 -p format:=RGB888` publishes `/camera/image_raw`, `/camera/image_raw/compressed` (~26
Hz) and `/camera/camera_info`. It costs ~140% of a core with raw and compressed both on; lower the
frame rate or size before running everything together. No calibration file yet.
- **`ros/hermes_vision`**: `hermes_vision.detect` (YuNet, Haar fallback; no ROS imports) and a node
`face_detector` that subscribes to the compressed image and publishes `/vision/faces`
(`vision_msgs/Detection2DArray`, bbox centre/size in pixels, `class_id: face`, score), at `rate_hz`
(default 4, ~47% of a core on a Pi 4); `publish_debug:=true` adds an annotated image. CLI:
`python -m hermes_vision.detect_cli IMAGE`. YuNet 2023mar needs OpenCV 4.8+, so `~/venv` has
`opencv-python-headless==4.10.0.84` with `numpy<2` (system NumPy stays 1.26.4 for ROS), and
`robot-env.sh` puts the venv first on `PYTHONPATH`; YuNet also needs input sides that are multiples
of 32. Tested on a sample image and on a live frame (one face found, upright, no flip needed).
- **Audio: card works, microphones read digital silence (open).** The stock `wm8960-soundcard`
overlay fails every stream with `wm8960: No MCLK configured` on Ubuntu 24.04: it puts the 12.288 MHz
fixed clock on the sound card's codec link, but the codec driver reads `mclk` from its own node.
`pi/overlays/hermes-wm8960.dts` is the stock overlay plus `clocks`/`clock-names` on the codec node;
`pi/install-audio-overlay.sh` compiles it to `/boot/firmware/overlays/hermes-wm8960.dtbo`, sets
`dtparam=audio=off` and `dtoverlay=hermes-wm8960`. With it, capture opens at 8-48 kHz. Address
the card by name (`hw:CARD=wm8960soundcard,DEV=0`): its index changes (it was card 1, then 0).
But every recording is exactly zero (no noise floor), with every mixer path on and gains maxed.
Checked: DAPM shows ADCs, input mixers, boost mixers and LINPUT1/RINPUT1 on; registers
`R25=0xfc`, `R47=0x30`, `R32/33=0x108`; MCLK 12.288 MHz and BCLK 1.536 MHz running; GPIO18-21 in
`alt0` for the I2S; setting `ALRCGPIO` (R9 bit 6) over I2C made no difference. Not yet checked:
the HAT's hardware **Audio ON/OFF slide switch** position, MICBIAS, a DAC-to-ADC loopback.
`Left/Right Input Mixer Boost Switch` default to off and must be turned on for the mic path.
No speakers on hand, so playback is untested.
- Not done: PWM (waits for servos). Ask before rebooting the Pi.
## Package 0b (USB link): Pi side and talos side working