Add hermes_vision, build camera_ros on the fork, and bring up WM8960 audio
hermes_vision wraps OpenCV YuNet (Haar fallback) as a plain module plus a node that publishes /vision/faces as vision_msgs/Detection2DArray. It replaces the standalone detect_faces.py. pi/robot-env.sh puts the Raspberry Pi libcamera fork ahead of ROS's own copy, the venv's OpenCV 4.10 ahead of the apt one, and fixes the ROS domain and subnet-only discovery. camera_ros 0.7.0 now runs against the fork and publishes raw and compressed images. The stock wm8960-soundcard overlay fails on Ubuntu 24.04 with "No MCLK configured": it hangs the clock on the sound card's codec link, but the codec driver reads mclk from its own node. hermes-wm8960.dts adds it there, and capture opens at 8-48 kHz. The microphones still read digital zeros; see CONTEXT.md for what has been ruled out. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AFN8nRRCiDiMN7ChBpHJhs
This commit is contained in:
+29
-6
@@ -48,12 +48,35 @@ printed enclosure, speakers. Work until those arrive is limited to what the Pi,
|
||||
`cam` then captures 1296x972 RGB888 at ~16 fps (dim room). `pi/capture_still.py` takes a still
|
||||
with it. **`camera_ros` still has to be built against this fork** (the apt one uses ROS's broken
|
||||
libcamera). Frames are padded (stride 3936) and B,G,R in memory.
|
||||
- Face detection: `pi/detect_faces.py` (OpenCV YuNet, Haar fallback). YuNet 2023mar needs OpenCV
|
||||
4.8+, so `~/venv` has `opencv-python-headless==4.10.0.84` with `numpy<2` (the system NumPy stays
|
||||
1.26.4 for ROS); YuNet also needs input sides in multiples of 32. Verified on a sample image.
|
||||
First camera shot was nearly black (mean 5/255): the lens was facing something close and dim.
|
||||
- Not done: audio (`wm8960-soundcard` overlay + mixer; `alsa-utils` is installed, no card yet), PWM
|
||||
(waits for servos). Ask before rebooting the Pi.
|
||||
- **`camera_ros` 0.7.0 is built from source against the fork** in `~/camera_ws` (overlay on ROS; at
|
||||
runtime the fork must come first in `LD_LIBRARY_PATH`, which `source ~/robot-env.sh` does; copy
|
||||
from `pi/robot-env.sh`). Verified: `ros2 run camera_ros camera_node --ros-args -p width:=1296 -p
|
||||
height:=972 -p format:=RGB888` publishes `/camera/image_raw`, `/camera/image_raw/compressed` (~26
|
||||
Hz) and `/camera/camera_info`. It costs ~140% of a core with raw and compressed both on; lower the
|
||||
frame rate or size before running everything together. No calibration file yet.
|
||||
- **`ros/hermes_vision`**: `hermes_vision.detect` (YuNet, Haar fallback; no ROS imports) and a node
|
||||
`face_detector` that subscribes to the compressed image and publishes `/vision/faces`
|
||||
(`vision_msgs/Detection2DArray`, bbox centre/size in pixels, `class_id: face`, score), at `rate_hz`
|
||||
(default 4, ~47% of a core on a Pi 4); `publish_debug:=true` adds an annotated image. CLI:
|
||||
`python -m hermes_vision.detect_cli IMAGE`. YuNet 2023mar needs OpenCV 4.8+, so `~/venv` has
|
||||
`opencv-python-headless==4.10.0.84` with `numpy<2` (system NumPy stays 1.26.4 for ROS), and
|
||||
`robot-env.sh` puts the venv first on `PYTHONPATH`; YuNet also needs input sides that are multiples
|
||||
of 32. Tested on a sample image and on a live frame (one face found, upright, no flip needed).
|
||||
- **Audio: card works, microphones read digital silence (open).** The stock `wm8960-soundcard`
|
||||
overlay fails every stream with `wm8960: No MCLK configured` on Ubuntu 24.04: it puts the 12.288 MHz
|
||||
fixed clock on the sound card's codec link, but the codec driver reads `mclk` from its own node.
|
||||
`pi/overlays/hermes-wm8960.dts` is the stock overlay plus `clocks`/`clock-names` on the codec node;
|
||||
`pi/install-audio-overlay.sh` compiles it to `/boot/firmware/overlays/hermes-wm8960.dtbo`, sets
|
||||
`dtparam=audio=off` and `dtoverlay=hermes-wm8960`. With it, capture opens at 8-48 kHz. Address
|
||||
the card by name (`hw:CARD=wm8960soundcard,DEV=0`): its index changes (it was card 1, then 0).
|
||||
But every recording is exactly zero (no noise floor), with every mixer path on and gains maxed.
|
||||
Checked: DAPM shows ADCs, input mixers, boost mixers and LINPUT1/RINPUT1 on; registers
|
||||
`R25=0xfc`, `R47=0x30`, `R32/33=0x108`; MCLK 12.288 MHz and BCLK 1.536 MHz running; GPIO18-21 in
|
||||
`alt0` for the I2S; setting `ALRCGPIO` (R9 bit 6) over I2C made no difference. Not yet checked:
|
||||
the HAT's hardware **Audio ON/OFF slide switch** position, MICBIAS, a DAC-to-ADC loopback.
|
||||
`Left/Right Input Mixer Boost Switch` default to off and must be turned on for the mic path.
|
||||
No speakers on hand, so playback is untested.
|
||||
- Not done: PWM (waits for servos). Ask before rebooting the Pi.
|
||||
|
||||
## Package 0b (USB link): Pi side and talos side working
|
||||
|
||||
|
||||
Reference in New Issue
Block a user