Add hermes_mcp, the bridge between Hermes and the ROS graph
The MCP server is now a ROS node and the only way in or out, per the owner's decision that ROS never leaves the robot. It serves MCP on the USB address only with a bearer token, keeps the stable tool name look() (which waits for the next camera frame, so it is never stale, and passes the camera's JPEG through), and registers turn_head/head_position only when a head is fitted, which today it is not. While an authenticated client is connected it republishes the agent heartbeat on /hermes/state, so the face wakes on connect and sleeps about 40 s after the client leaves; look() also makes the face squint. The token is generated on the Pi into /etc/robot-mcp.env (root:hermes, 640) and never printed. Verified end to end from talos with a real MCP client over the USB link: tools listed, a 157 KB JPEG back in 0.08 s, a wrong token refused, the Wi-Fi path dropped by the firewall. Host config snippets now point at 10.55.0.2. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AFN8nRRCiDiMN7ChBpHJhs
This commit is contained in:
+27
-7
@@ -4,8 +4,10 @@ Last updated 2026-10-04. `HANDOFF.md` is the brief; this file tracks where the w
|
||||
|
||||
## Hardware on hand
|
||||
|
||||
Raspberry Pi 4 (8 GB), BrainCraft HAT, Arducam UC-346 camera. Not yet: servos, USB-C splitter,
|
||||
printed enclosure, speakers. Work until those arrive is limited to what the Pi, HAT and camera can do.
|
||||
Raspberry Pi 4 (8 GB), BrainCraft HAT, Arducam UC-346 camera, and nothing else: **no servos, no
|
||||
fan, no speakers**, no USB-C splitter, no printed enclosure (the camera hangs loose on its ribbon and is
|
||||
held by hand). The Pi runs at about 62 C open-air with everything on and no fan; an enclosure will be
|
||||
hotter, so fit the fan before enclosing it. Playback and the servos are untested. Work until those arrive is limited to what the Pi, HAT and camera can do.
|
||||
|
||||
## The Pi (`hermes-bot`)
|
||||
|
||||
@@ -205,6 +207,26 @@ printed enclosure, speakers. Work until those arrive is limited to what the Pi,
|
||||
49 face tests, 10 audio tests. **Not yet verified:** the audio node as a service (it was run as a
|
||||
transient unit).
|
||||
|
||||
## The MCP bridge (`ros/hermes_mcp`)
|
||||
|
||||
The only way in or out of the ROS graph. Serves MCP (SDK 2.3, streamable HTTP) on
|
||||
`http://10.55.0.2:8765/mcp`, the USB address only (retrying the bind if the link is not up yet), with
|
||||
a bearer token. **Tool names are the stable contract with Hermes**: `look()`, and `turn_head` /
|
||||
`head_position` only when a head is fitted. With no servos only `look` is registered, by design
|
||||
(`head: false`; the node refuses `head: true` until `hermes_head` exists). `look()` waits for the next
|
||||
camera frame taken after the call (never a stale one) and passes the camera's JPEG through (1296x972,
|
||||
~155 KB); it also publishes `looking` on `/hermes/state` so the face squints and flashes.
|
||||
While an authenticated client is connected (in flight, or heard within `connected_grace_s` = 30) it
|
||||
republishes `idle` on `/hermes/state` once a second: the agent heartbeat that wakes the face. Verified
|
||||
end to end from talos over the USB link with a real MCP client: connects, lists `look`, `look()`
|
||||
returns a JPEG in 0.08 s, a wrong token is refused, the Wi-Fi path is dropped by the firewall, and
|
||||
the face went `looking` -> `idle` on connect and `agent gone, sleeping` about 40 s after the client
|
||||
left. The token is in `/etc/robot-mcp.env` (root:hermes, mode 640), generated by
|
||||
`pi/install-robot-service.sh` and never printed; read it with `sudo cat`. The MCP SDK is in `~/venv`
|
||||
(`pip install "mcp>=2"`). The old `pi/robot_mcp_server.py`, `pi/robot-mcp.service` and `pi/install.sh`
|
||||
are superseded (package 5 rewrites the installer). **Not yet verified:** a real Hermes client, and
|
||||
whether Hermes passes MCP image content to the model (HANDOFF package 6).
|
||||
|
||||
## Face recognition (opt-in)
|
||||
|
||||
`hermes_vision.recognize` / `people`: SFace embeddings on YuNet's landmarks. Nothing is stored until
|
||||
@@ -223,11 +245,9 @@ unrelated photo scored 0.16. **Nobody is enrolled yet**; the owner enrols themse
|
||||
`host/nixos/`. A new Omarchy host with a GPU (runs the local
|
||||
model "ornith") arrives this week: it needs only `host/omarchy/robot-usb-link.sh` and its SSH key
|
||||
added to the Pi.
|
||||
2. `hermes_mcp`: the MCP server as an `rclpy` node (stable tool names; `look()` returns the latest
|
||||
camera image, `turn_head()` calls the head action), the only bridge to the outside; it must
|
||||
republish `/hermes/state` about once a second as the agent heartbeat. Then `hermes_head` and
|
||||
the servos when they arrive, echo cancel and the `robot_mic` naming once speakers arrive, the
|
||||
Wi-Fi-off boot test, and the rest of `HANDOFF.md`.
|
||||
2. `hermes_head` and the servos when they arrive (then `head: true`), echo cancel and the `robot_mic`
|
||||
naming once speakers arrive, a fan before any enclosure, the Wi-Fi-off boot test, a real Hermes
|
||||
client against `hermes_mcp`, and the rest of `HANDOFF.md`.
|
||||
3. Confirm the retuned listening detector live (the flicker fix is unit-tested only); a second person
|
||||
in frame for the recognition test (not possible on 2026-10-04).
|
||||
|
||||
|
||||
Reference in New Issue
Block a user