kyle/hermes-bot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Get the camera working with the Raspberry Pi libcamera fork; add face detection

Browse Source 
Stock libcamera 0.2.0 (Ubuntu) and ROS Jazzy's 0.7.2 both detect the OV5647 but
crash in the Pi IPA when capturing, for every stream format and size. The
Raspberry Pi fork, built into /opt/libcamera-rpi, captures 1296x972 frames.
capture_still.py takes a still with it, and detect_faces.py runs OpenCV YuNet
(Haar fallback) on images. YuNet needs OpenCV 4.8+ and input sides in multiples
of 32, so the venv carries opencv-python-headless 4.10 pinned with numpy<2.

camera_ros still has to be built against the fork.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AFN8nRRCiDiMN7ChBpHJhs
This commit is contained in:
kyle
2026-10-04 21:32:32 -07:00
co-authored by Claude Sonnet 5.5
parent 2535344f3d
commit 5e48b6fa03
3 changed files with 159 additions and 1 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CONTEXT.md
+15 -1
View File
@@ -38,7 +38,21 @@ printed enclosure, speakers. Work until those arrive is limited to what the Pi,
rotates in software and leaves the scan direction fixed). Mounted pose (microSD end up, USB-C edge
on the left) needs content turned 90 degrees clockwise: `rotation=270`. Full-frame push is
~23 fps through the driver; the face pushes raw RGB565 with `_block` and reaches ~27-28 fps.
- Not done: audio (`wm8960-soundcard` overlay + mixer), camera (`cam -l`, `camera_ros`), PWM
- **Camera: works with the Raspberry Pi libcamera fork, not with the stock stacks.** The OV5647 is
detected (`ov5647 10-0036`) and Ubuntu's libcamera 0.2.0 lists it once `libcamera-ipa` is
installed, but capture dies in the Pi IPA (`assertion "it != buffers_.end()" in prepareIsp()`),
and ROS Jazzy's libcamera 0.7.2 dies with `FATAL … A list of V4L2 controls requires a
ControlInfoMap` right after `configure`. Both are deterministic whatever the stream format or
size. Fix: build the fork (`github.com/raspberrypi/libcamera`, commit `6c1dd9d`, 2026-08-17) with
`-Dpipelines=rpi/vc4 -Dipas=rpi/vc4 -Dcam=enabled -Dv4l2=false …` into `/opt/libcamera-rpi`; its
`cam` then captures 1296x972 RGB888 at ~16 fps (dim room). `pi/capture_still.py` takes a still
with it. **`camera_ros` still has to be built against this fork** (the apt one uses ROS's broken
libcamera). Frames are padded (stride 3936) and B,G,R in memory.
- Face detection: `pi/detect_faces.py` (OpenCV YuNet, Haar fallback). YuNet 2023mar needs OpenCV
4.8+, so `~/venv` has `opencv-python-headless==4.10.0.84` with `numpy<2` (the system NumPy stays
1.26.4 for ROS); YuNet also needs input sides in multiples of 32. Verified on a sample image.
First camera shot was nearly black (mean 5/255): the lens was facing something close and dim.
- Not done: audio (`wm8960-soundcard` overlay + mixer; `alsa-utils` is installed, no card yet), PWM
(waits for servos). Ask before rebooting the Pi.
## Package 0b (USB link): Pi side and talos side working