kyle/hermes-bot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Give Hermes recognitions, not just photos: faces(), tracking ids, discovery

Browse Source 
Hermes asked for structured detections instead of a bare JPEG. faces() returns the
faces in view with no photo: a persistent track id, bbox, detection confidence, how
long the person has been in view, and identity {name, similarity} or null, never a
made-up name. look() returns that same JSON as text plus the photo (send_image can
skip it); list_people() gives names and counts; robot://status and two prompts make
the server discoverable, where list_resources and list_prompts were empty. The
detector now tracks faces across frames (FaceTracker) and publishes the track id and
the identity as a second hypothesis.

Deliberately not built: embeddings in responses, age/gender attributes, auto-updating
face data, and a remote enroll tool (a prompt could make the robot call anyone "Kyle").

Checked end to end: a real Hermes turn on the local ornith model called faces() (0.03 s,
no image) and answered that Kyle was there at 0.78 similarity. Also: the camera is
calibrated (18 of 24 views, 0.296 px, outlier rejection added) and the calibration is
kept in pi/calibration and restored by the installer.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AFN8nRRCiDiMN7ChBpHJhs
This commit is contained in:
kyle
2026-10-05 01:21:32 -07:00
co-authored by Claude Sonnet 5.5
parent 497f026dc9
commit 3b65697f1e
16 changed files with 599 additions and 44 deletions
Download Patch File Download Diff File Expand all files Collapse all files
README.md
+9 -2
View File
@@ -100,10 +100,17 @@ door.
```
ping 10.55.0.2
hermes mcp test robot # connects, finds the look tool
hermes chat -q "Use the look tool and tell me what you see."
hermes mcp test robot # connects, lists the tools
hermes chat -q "Who is at the robot's desk right now?" # calls faces(): instant, no photo
hermes chat -q "Use the look tool and tell me what you see." # sends a photo
```
What Hermes can use: `faces()` (who is in view: names, positions, tracking ids; instant, no photo),
`look()` (the same faces summary plus a current photo), `list_people()`, and, once a head is fitted,
`turn_head` and `head_position`. `robot://status` says what hardware this robot actually has. Only faces
are detected. Face data never leaves the robot: Hermes gets names and similarity scores, and an
unrecognised face is `null`, never a guess.
The robot's face wakes when Hermes connects and falls asleep about 40 seconds after it leaves. A
look-and-describe turn takes a minute or so on local models: `look()` itself is instant. Hermes does not
give MCP images to the main model: it saves the photo to its image cache and the model calls