Hermes asked for structured detections instead of a bare JPEG. faces() returns the
faces in view with no photo: a persistent track id, bbox, detection confidence, how
long the person has been in view, and identity {name, similarity} or null, never a
made-up name. look() returns that same JSON as text plus the photo (send_image can
skip it); list_people() gives names and counts; robot://status and two prompts make
the server discoverable, where list_resources and list_prompts were empty. The
detector now tracks faces across frames (FaceTracker) and publishes the track id and
the identity as a second hypothesis.
Deliberately not built: embeddings in responses, age/gender attributes, auto-updating
face data, and a remote enroll tool (a prompt could make the robot call anyone "Kyle").
Checked end to end: a real Hermes turn on the local ornith model called faces() (0.03 s,
no image) and answered that Kyle was there at 0.78 similarity. Also: the camera is
calibrated (18 of 24 views, 0.296 px, outlier rejection added) and the calibration is
kept in pi/calibration and restored by the installer.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AFN8nRRCiDiMN7ChBpHJhs
The MCP server is now a ROS node and the only way in or out, per the owner's
decision that ROS never leaves the robot. It serves MCP on the USB address only
with a bearer token, keeps the stable tool name look() (which waits for the next
camera frame, so it is never stale, and passes the camera's JPEG through), and
registers turn_head/head_position only when a head is fitted, which today it is
not. While an authenticated client is connected it republishes the agent
heartbeat on /hermes/state, so the face wakes on connect and sleeps about 40 s
after the client leaves; look() also makes the face squint.
The token is generated on the Pi into /etc/robot-mcp.env (root:hermes, 640) and
never printed. Verified end to end from talos with a real MCP client over the USB
link: tools listed, a 157 KB JPEG back in 0.08 s, a wrong token refused, the Wi-Fi
path dropped by the firewall. Host config snippets now point at 10.55.0.2.
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AFN8nRRCiDiMN7ChBpHJhs
kyleandClaude Sonnet 5.5