kyle/hermes-bot
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

Record a real Hermes turn against the robot on local models

Browse Source 
hermes chat on talos, pinned to the local ornith-1.5-35b-a3b on straylight, called
the robot's look() and described the live frame correctly, with nothing sent to a
hosted provider. Hermes does not pass MCP images to the main model: it saves the
JPEG to ~/.hermes/cache/images and returns a short reference, and the model then
calls vision_analyze, which ran on the local qwen3-vl. That is the HANDOFF package 6
fallback, handled by Hermes itself, so look() needs no change.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AFN8nRRCiDiMN7ChBpHJhs
This commit is contained in:
kyle
2026-10-05 00:33:00 -07:00
co-authored by Claude Sonnet 5.5
parent 0527113968
commit 841ddc3573
1 changed files with 15 additions and 4 deletions
Download Patch File Download Diff File Expand all files Collapse all files
CONTEXT.md
+15 -4
View File
@@ -234,10 +234,21 @@ straylight/dixie). `~/.hermes/config.yaml` gained an `mcp_servers.robot` block (
`http://10.55.0.2:8765/mcp`, `Authorization: Bearer ${ROBOT_MCP_TOKEN}`) and `~/.hermes/.env` gained
`ROBOT_MCP_TOKEN`; backups are `config.yaml.bak-20261005T002659-robot-mcp` and
`.env.bak-20261005T002659-robot-mcp`. `hermes mcp list` shows it enabled and **`hermes mcp test
robot` passes: connected in 789 ms, 1 tool (`look`)**, which is the HANDOFF acceptance check. Not done
yet: a real agent turn that calls `look()` (it would send a camera image to the hosted model, so it
needs the owner's say-so), which is also what settles whether Hermes hands MCP images to the model.
A fresh `HERMES_HOME` hangs on first-run setup, so test against the real one.
robot` passes: connected in 789 ms, 1 tool (`look`)**, which is the HANDOFF acceptance check. A fresh
`HERMES_HOME` hangs on first-run setup, so test against the real one.
**A real agent turn works, entirely on local models (2026-10-05):** `hermes chat -q "Call the look tool
once ... what you see" --oneshot -Q --provider custom:llama-router -m ornith-1.5-35b-a3b --max-turns 4`
(straylight's router; no hosted provider, so the image stayed on the LAN). Hermes called
`mcp__robot__look` (0.07 s), the model described the live photo correctly (blurry close-up of a board
with blue LEDs, a red label, camouflage fabric), and the robot's face went `looking` -> `idle`.
**Hermes does not hand MCP image content to the main model.** It saves the JPEG to
`~/.hermes/cache/images/img_<hash>.jpg` on talos (157,147 bytes, the same as the robot sent) and
returns a 72-character reference; the model then calls Hermes's `vision_analyze`, which ran on the local
vision model (`qwen3-vl-8b-abliterated` on straylight) in 29 s. That is HANDOFF package 6's fallback and
Hermes does it itself, so `look()` needs no change. Two consequences: each look() leaves a photo of the
room in talos's Hermes image cache (Hermes's retention, not the robot's), and a look-and-describe turn
takes ~70 s on the local models, mostly the vision call.
## Face recognition (opt-in)