Add hermes_mcp, the bridge between Hermes and the ROS graph

The MCP server is now a ROS node and the only way in or out, per the owner's
decision that ROS never leaves the robot. It serves MCP on the USB address only
with a bearer token, keeps the stable tool name look() (which waits for the next
camera frame, so it is never stale, and passes the camera's JPEG through), and
registers turn_head/head_position only when a head is fitted, which today it is
not. While an authenticated client is connected it republishes the agent
heartbeat on /hermes/state, so the face wakes on connect and sleeps about 40 s
after the client leaves; look() also makes the face squint.

The token is generated on the Pi into /etc/robot-mcp.env (root:hermes, 640) and
never printed. Verified end to end from talos with a real MCP client over the USB
link: tools listed, a 157 KB JPEG back in 0.08 s, a wrong token refused, the Wi-Fi
path dropped by the firewall. Host config snippets now point at 10.55.0.2.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AFN8nRRCiDiMN7ChBpHJhs
This commit is contained in:
2026-10-05 00:20:58 -07:00
co-authored by Claude Sonnet 5.5
parent d8db0a48f5
commit 167f53ceeb
17 changed files with 462 additions and 24 deletions
+27 -7
View File
@@ -4,8 +4,10 @@ Last updated 2026-10-04. `HANDOFF.md` is the brief; this file tracks where the w
## Hardware on hand
Raspberry Pi 4 (8 GB), BrainCraft HAT, Arducam UC-346 camera. Not yet: servos, USB-C splitter,
printed enclosure, speakers. Work until those arrive is limited to what the Pi, HAT and camera can do.
Raspberry Pi 4 (8 GB), BrainCraft HAT, Arducam UC-346 camera, and nothing else: **no servos, no
fan, no speakers**, no USB-C splitter, no printed enclosure (the camera hangs loose on its ribbon and is
held by hand). The Pi runs at about 62 C open-air with everything on and no fan; an enclosure will be
hotter, so fit the fan before enclosing it. Playback and the servos are untested. Work until those arrive is limited to what the Pi, HAT and camera can do.
## The Pi (`hermes-bot`)
@@ -205,6 +207,26 @@ printed enclosure, speakers. Work until those arrive is limited to what the Pi,
49 face tests, 10 audio tests. **Not yet verified:** the audio node as a service (it was run as a
transient unit).
## The MCP bridge (`ros/hermes_mcp`)
The only way in or out of the ROS graph. Serves MCP (SDK 2.3, streamable HTTP) on
`http://10.55.0.2:8765/mcp`, the USB address only (retrying the bind if the link is not up yet), with
a bearer token. **Tool names are the stable contract with Hermes**: `look()`, and `turn_head` /
`head_position` only when a head is fitted. With no servos only `look` is registered, by design
(`head: false`; the node refuses `head: true` until `hermes_head` exists). `look()` waits for the next
camera frame taken after the call (never a stale one) and passes the camera's JPEG through (1296x972,
~155 KB); it also publishes `looking` on `/hermes/state` so the face squints and flashes.
While an authenticated client is connected (in flight, or heard within `connected_grace_s` = 30) it
republishes `idle` on `/hermes/state` once a second: the agent heartbeat that wakes the face. Verified
end to end from talos over the USB link with a real MCP client: connects, lists `look`, `look()`
returns a JPEG in 0.08 s, a wrong token is refused, the Wi-Fi path is dropped by the firewall, and
the face went `looking` -> `idle` on connect and `agent gone, sleeping` about 40 s after the client
left. The token is in `/etc/robot-mcp.env` (root:hermes, mode 640), generated by
`pi/install-robot-service.sh` and never printed; read it with `sudo cat`. The MCP SDK is in `~/venv`
(`pip install "mcp>=2"`). The old `pi/robot_mcp_server.py`, `pi/robot-mcp.service` and `pi/install.sh`
are superseded (package 5 rewrites the installer). **Not yet verified:** a real Hermes client, and
whether Hermes passes MCP image content to the model (HANDOFF package 6).
## Face recognition (opt-in)
`hermes_vision.recognize` / `people`: SFace embeddings on YuNet's landmarks. Nothing is stored until
@@ -223,11 +245,9 @@ unrelated photo scored 0.16. **Nobody is enrolled yet**; the owner enrols themse
`host/nixos/`. A new Omarchy host with a GPU (runs the local
model "ornith") arrives this week: it needs only `host/omarchy/robot-usb-link.sh` and its SSH key
added to the Pi.
2. `hermes_mcp`: the MCP server as an `rclpy` node (stable tool names; `look()` returns the latest
camera image, `turn_head()` calls the head action), the only bridge to the outside; it must
republish `/hermes/state` about once a second as the agent heartbeat. Then `hermes_head` and
the servos when they arrive, echo cancel and the `robot_mic` naming once speakers arrive, the
Wi-Fi-off boot test, and the rest of `HANDOFF.md`.
2. `hermes_head` and the servos when they arrive (then `head: true`), echo cancel and the `robot_mic`
naming once speakers arrive, a fan before any enclosure, the Wi-Fi-off boot test, a real Hermes
client against `hermes_mcp`, and the rest of `HANDOFF.md`.
3. Confirm the retuned listening detector live (the flicker fix is unit-tested only); a second person
in frame for the recognition test (not possible on 2026-10-04).