Give Hermes recognitions, not just photos: faces(), tracking ids, discovery
Hermes asked for structured detections instead of a bare JPEG. faces() returns the
faces in view with no photo: a persistent track id, bbox, detection confidence, how
long the person has been in view, and identity {name, similarity} or null, never a
made-up name. look() returns that same JSON as text plus the photo (send_image can
skip it); list_people() gives names and counts; robot://status and two prompts make
the server discoverable, where list_resources and list_prompts were empty. The
detector now tracks faces across frames (FaceTracker) and publishes the track id and
the identity as a second hypothesis.
Deliberately not built: embeddings in responses, age/gender attributes, auto-updating
face data, and a remote enroll tool (a prompt could make the robot call anyone "Kyle").
Checked end to end: a real Hermes turn on the local ornith model called faces() (0.03 s,
no image) and answered that Kyle was there at 0.78 similarity. Also: the camera is
calibrated (18 of 24 views, 0.296 px, outlier rejection added) and the calibration is
kept in pi/calibration and restored by the installer.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AFN8nRRCiDiMN7ChBpHJhs
This commit is contained in:
@@ -3,7 +3,8 @@
|
||||
Subscribes: /hermes/state (std_msgs/String): sleeping idle listening thinking looking speaking
|
||||
/joint_states (sensor_msgs/JointState): pan_joint, tilt_joint in radians. The sign
|
||||
convention is the MCP tool's: positive pan = the robot's left, positive tilt = up.
|
||||
/vision/faces (vision_msgs/Detection2DArray): the eyes glance toward the best face
|
||||
/vision/faces (vision_msgs/Detection2DArray): the eyes glance toward the best face (a named one
|
||||
first: the name is results[1].class_id)
|
||||
/camera/camera_info (sensor_msgs/CameraInfo): the image size for that, if published
|
||||
/audio/activity (std_msgs/String): speaking | listening | quiet, from hermes_audio
|
||||
/audio/scope/{mic,speaker} (std_msgs/Float32MultiArray): the real waveform for the mouth
|
||||
@@ -115,7 +116,8 @@ class FaceNode(Node):
|
||||
def _on_faces(self, msg: Detection2DArray) -> None:
|
||||
target = pick_target([
|
||||
Target(d.bbox.center.position.x, d.bbox.center.position.y, d.bbox.size_x, d.bbox.size_y,
|
||||
d.results[0].hypothesis.score if d.results else 1.0, d.id)
|
||||
d.results[0].hypothesis.score if d.results else 1.0,
|
||||
d.results[1].hypothesis.class_id if len(d.results) > 1 else '') # the name, if recognised
|
||||
for d in msg.detections])
|
||||
if target:
|
||||
gx, gy = gaze_from_pixel(target.cx, target.cy, *self.image_size)
|
||||
|
||||
Reference in New Issue
Block a user