Give Hermes recognitions, not just photos: faces(), tracking ids, discovery
Hermes asked for structured detections instead of a bare JPEG. faces() returns the
faces in view with no photo: a persistent track id, bbox, detection confidence, how
long the person has been in view, and identity {name, similarity} or null, never a
made-up name. look() returns that same JSON as text plus the photo (send_image can
skip it); list_people() gives names and counts; robot://status and two prompts make
the server discoverable, where list_resources and list_prompts were empty. The
detector now tracks faces across frames (FaceTracker) and publishes the track id and
the identity as a second hypothesis.
Deliberately not built: embeddings in responses, age/gender attributes, auto-updating
face data, and a remote enroll tool (a prompt could make the robot call anyone "Kyle").
Checked end to end: a real Hermes turn on the local ornith model called faces() (0.03 s,
no image) and answered that Kyle was there at 0.78 similarity. Also: the camera is
calibrated (18 of 24 views, 0.296 px, outlier rejection added) and the calibration is
kept in pi/calibration and restored by the installer.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AFN8nRRCiDiMN7ChBpHJhs
This commit is contained in:
@@ -0,0 +1,20 @@
|
||||
image_width: 1296
|
||||
image_height: 972
|
||||
camera_name: ov5647__base_soc_i2c0mux_i2c_1_ov5647_36_1296x972
|
||||
camera_matrix:
|
||||
rows: 3
|
||||
cols: 3
|
||||
data: [1152.96331, 0, 652.952458, 0, 1151.52195, 463.351689, 0, 0, 1]
|
||||
distortion_model: plumb_bob
|
||||
distortion_coefficients:
|
||||
rows: 1
|
||||
cols: 5
|
||||
data: [0.0727701258, -0.232756953, 0.00468014288, -0.00488550855, 0]
|
||||
rectification_matrix:
|
||||
rows: 3
|
||||
cols: 3
|
||||
data: [1, 0, 0, 0, 1, 0, 0, 0, 1]
|
||||
projection_matrix:
|
||||
rows: 3
|
||||
cols: 4
|
||||
data: [1152.96331, 0, 652.952458, 0, 0, 1151.52195, 463.351689, 0, 0, 0, 1, 0]
|
||||
@@ -116,6 +116,13 @@ chown -R "$ROBOT_USER:$ROBOT_USER" "$HOME_DIR/ros_ws/src"
|
||||
as_user bash -c "source /opt/ros/jazzy/setup.bash && source '$HOME_DIR/camera_ws/install/setup.bash' \
|
||||
&& cd '$HOME_DIR/ros_ws' && colcon build --symlink-install"
|
||||
|
||||
step "Camera calibration (this camera's, from pi/calibration; an existing file is never overwritten)"
|
||||
as_user mkdir -p "$HOME_DIR/.ros/camera_info"
|
||||
for f in "$PI_DIR"/calibration/*.yaml; do
|
||||
[ -e "$f" ] || continue
|
||||
[ -e "$HOME_DIR/.ros/camera_info/$(basename "$f")" ] || install -o "$ROBOT_USER" -g "$ROBOT_USER" -m 644 "$f" "$HOME_DIR/.ros/camera_info/"
|
||||
done
|
||||
|
||||
step "USB gadget (NCM Ethernet + ACM serial) and the audio codec overlay"
|
||||
before="$(cfg_sum)"
|
||||
bash "$PI_DIR/install-usb-gadget.sh"
|
||||
|
||||
Reference in New Issue
Block a user