A small desk robot that gives Hermes Agent (Nous Research's open-source agent) eyes, ears, a voice and a face. It does no thinking of its own: Hermes runs on your computer and uses the robot through MCP.

Eyes: a camera. Hermes calls look() and gets a current photo.
Face: an oscilloscope-style display that shows what the agent is really doing: asleep (no agent connected), idle, listening, thinking, looking, speaking. The mouth draws the actual voice, and the eyes glance toward whoever the camera sees.
Ears and voice: the HAT's two microphones and speakers, shared with your computer as ordinary audio devices (the speakers are not fitted yet; the microphones work).
Neck: pan and tilt servos (turn_head), once they are fitted.
One cable: USB-C carries power and a private network link to your computer. Wi-Fi is a fallback.

It runs ROS 2 Jazzy on Ubuntu 24.04, so a mobile version can grow out of it. Hermes needs no ROS: it only speaks MCP.

Status

Works and is tested	Waiting on hardware
Face on the display, always on at power-up, with a sleeping pose	Servos: `turn_head`, `head_position`, the pan/tilt URDF
Camera (calibrated), face detection, opt-in recognition of people you enrol	Speakers: playback, echo cancel, Hermes talking out loud
Eyes follow faces; mouth draws your live voice	Fan (the Pi runs at about 60 °C open-air without one)
MCP bridge: `hermes mcp test robot` passes, real Hermes turns call `look()` and `faces()`	USB-C splitter (the Pi is powered from your computer's port for now)
Thermal guard: slows the camera, face and detector before the Pi throttles
USB network link, firewall, boots and works with Wi-Fi off	Printed enclosure (`enclosure/`)

How it fits together

Your computer (Omarchy)                          Raspberry Pi 4 + BrainCraft HAT (Ubuntu 24.04)
  Hermes Agent ── MCP over HTTP, USB link ────►  hermes_mcp  (the only way in or out of ROS)
  PipeWire     ◄── audio, TCP 4713 ──────────►  PipeWire (not ROS)
                                                   │ ROS 2 graph, this machine only
                      camera_ros ► hermes_face_detector ► /vision/faces ─┐
                      hermes_audio_state ► levels, speaking/listening ───┤
                      hermes_mcp ► /hermes/state (the agent heartbeat) ──┴► hermes_face ► display

Every node is its own process, started by one launch file (ros/hermes_bringup) and restarted if it dies. ROS never leaves the Pi. The firewall blocks it on every interface, so hermes_mcp is the one door.

Parts

Part	Notes
Raspberry Pi 4	8 GB here. Ubuntu Server 24.04 arm64.
Adafruit BrainCraft HAT	1.54″ ST7789 display, WM8960 audio codec with two mics, speaker and servo ports.
Arducam UC-346	OV5647 camera, 15-pin ribbon, fixed focus.
2 × MG90S servos	Pan (in the base) on the HAT's GPIO12 port, tilt (camera yoke) on GPIO13.
2 × speakers	8 Ω, up to 1 W, JST PH 2-pin.
USB-C power/data splitter	Power from a 5.1 V 3 A supply, data only to your computer. A Pi 4 cannot negotiate USB PD.
Fan	On the HAT's fan header.
Enclosure	`enclosure/`: printed, with a turntable.

Physical steps

The HAT's "3pin Volt" jumper must be on 5 V, which is the factory setting. Check it with a meter.
Pan servo into the GPIO12 port, tilt servo into GPIO13.
Camera ribbon: leave a gentle loop, because only the tilt flexes it.
Slide the HAT's "Audio" switch (right edge) to ON. With it off, everything looks configured and every recording is exactly silent. This cost an hour once.
Power the Pi from the splitter, not straight from a port that cannot give 3 A. Never power it from a laptop or tablet you care about the battery of.

Setting up the Pi

Flash Ubuntu Server 24.04 for the Pi (arm64, preinstalled image) and check its SHA256.
Configure first boot on the card's system-boot partition. In user-data, create the user hermes with key-only login and the hostname hermes-bot. For Wi-Fi, put the access point in network-config (copy pi/network-config.example and fill it in; keep a filled copy out of the repo: pi/50-cloud-init.yaml is gitignored for that). cloud-init applies these on the first boot only: to change Wi-Fi later, edit /etc/netplan/50-cloud-init.yaml on the Pi.
Boot, then copy this repo to the Pi and run the installer:
```
rsync -a ./ hermes@<pi>:~/hermes-bot/
ssh hermes@<pi> 'cd hermes-bot && sudo ./pi/install.sh'
```
It installs ROS, builds the camera driver against the Raspberry Pi libcamera fork, builds the workspace, sets up audio, the USB link, the firewall and the services. It can be run again safely. The first run takes a while (the libcamera build alone is about 15 minutes). Reboot afterwards.
Plug the USB-C data cable into your computer.

Setting up your computer (Omarchy)

Network link: sudo host/omarchy/robot-usb-link.sh gives your end of the link 10.55.0.1/30, matched by the gadget's fixed MAC, so any port and any machine works. The robot is 10.55.0.2.
Hermes: the token is in /etc/robot-mcp.env on the Pi (ssh hermes@10.55.0.2 sudo cat /etc/robot-mcp.env). Put it in ~/.hermes/.env as ROBOT_MCP_TOKEN=, and merge host/omarchy/hermes-config-snippet.yaml into ~/.hermes/config.yaml. Start a new Hermes session afterwards: a session keeps the tool list it started with.
Audio (optional, needs the speakers): host/omarchy/50-robot-tunnel.conf makes the robot's microphone and speaker appear as devices; host/omarchy/hermes-robot starts Hermes with them as the defaults.

First test

ping 10.55.0.2
hermes mcp test robot          # connects, lists the tools
hermes chat -q "Who is at the robot's desk right now?"          # calls faces(): instant, no photo
hermes chat -q "Use the look tool and tell me what you see."    # sends a photo

What Hermes can use: faces() (who is in view: names, positions, tracking ids; instant, no photo), look() (the same faces summary plus a current photo), list_people(), and, once a head is fitted, turn_head and head_position. robot://status says what hardware this robot actually has. Only faces are detected. Face data never leaves the robot: Hermes gets names and similarity scores, and an unrecognised face is null, never a guess.

The robot's face wakes when Hermes connects and falls asleep about 40 seconds after it leaves. A look-and-describe turn takes a minute or so on local models: look() itself is instant. Hermes does not give MCP images to the main model: it saves the photo to its image cache and the model calls vision_analyze on it.

Day to day

State	Face	Comes from
sleeping	eyes closed, dim, slowly breathing	no agent connected (no heartbeat)
idle	blinks, smiles	the agent is connected
listening	mouth draws your voice, dimmer	the microphone (never wakes a sleeping face)
thinking	eyes trace looping curves	the agent
looking	squint, then a flash	`look()`
speaking	mouth draws the outgoing audio	the speaker

Heat: hermes_thermal watches the CPU temperature and, at 72, 77 and 82 °C (the Pi starts throttling at about 80), scales the camera, face and detector rates down, then restores them as it cools (one step at a time, so it does not flap). The temperature and the Pi's throttle and under-voltage flags are in robot://status and on /health/*. To test it without heating the Pi, on the Pi: ros2 param set /hermes_thermal fake_temp_c 85.0 (and -1 to stop).
Logs and control: journalctl -u robot -f (the face logs every state change), sudo systemctl restart robot.
Enrol yourself so the eyes prefer you (numbers only are stored, never photos, in ~/.local/share/hermes/faces.json): on the Pi, with the robot running and after source ~/robot-env.sh, python3 -m hermes_vision.people enroll YOURNAME. list shows who is enrolled; forget YOURNAME removes you. Enrolling is deliberately not an MCP tool: a prompt must not be able to teach the robot a face.
Update the code: rsync -a --delete --exclude __pycache__ ros/ hermes@<pi>:~/ros_ws/src/, then colcon build --symlink-install in ~/ros_ws, then sudo systemctl restart robot.
Run a node by hand: source ~/robot-env.sh first: it puts the libcamera fork and the right OpenCV on the path, and keeps ROS local.
Run the tests: on the Pi, after source ~/robot-env.sh, python3 -m pytest test in each package under ~/ros_ws/src/ (the face suite takes about a minute). A shell that sourced the environment before a package was built cannot import it: source again.

Calibrating the camera

This camera's calibration ships in pi/calibration/ and the installer restores it, so a fresh install is already calibrated. Redo it if you change the lens focus, the image size or the camera: a changed focus invalidates it. Set the focus first (the lens turns if it is not glued: focus at the distance you will use), then show docs/checkerboard-9x6.png on a screen (or print it) and measure one square with a ruler. With the robot running, on the Pi, after source ~/robot-env.sh:

python3 -m hermes_vision.calibrate --cols 9 --rows 6 --square-mm <measured>

Hold the camera 25 to 60 cm from the board and vary distance, tilt and position, including the edges and corners of the camera's view (the tool says which regions are still uncovered), and hold still: blurred views are dropped as outliers. It collects 24 different views, solves with OpenCV (holding the k3 distortion term at zero, which a lens this mild does not need) and writes ~/.ros/camera_info/ov5647__base_soc_i2c0mux_i2c_1_ov5647_36_1296x972.yaml, plus the collected views so --from-views FILE can re-solve them without recapturing. A reprojection error under about 0.5 px is good. The square size only scales distances, not the lens model, so an approximate measurement is fine. sudo systemctl restart robot to load it, and copy the YAML into pi/calibration/ to keep it.

Tuning and troubleshooting

Recordings are exactly silent: the Audio switch is off (see above).
Microphone too noisy or too quiet: WirePlumber resets the codec's capture gain to +30 dB. The installer sets PipeWire's source volume to 63% (about +17 dB); change it with pactl set-source-volume @DEFAULT_SOURCE@ 63%. Remember that PipeWire's scale is cubic.
The face flickers between listening and idle: raise hold_s in ros/hermes_audio/hermes_audio/levels.py.
Stock libcamera crashes when capturing: that is expected on this kernel. The installer builds the Raspberry Pi fork into /opt/libcamera-rpi and builds camera_ros against it. Anything that uses the camera must run after source ~/robot-env.sh.
Display shows noise or a band: the 240×240 glass sits 80 rows into the controller's memory (y_offset: 80), and rotation: 270 is the mounted pose. Both are in ros/hermes_bringup/config/robot.yaml.
GPIO busy or a crash at start from systemd: the display's GPIO backend needs a writable working directory; the unit sets one.
The USB interface is unmanaged: netplan must match usb0 by name, not by MAC.
ros2 node list shows only some nodes: use ros2 node list --no-daemon --spin-time 6.
The Pi's clock is wrong with Wi-Fi off: there is no internet to sync from; it stays roughly right but unsynchronised.
Hermes says faces (or another new tool) is "not a known tool name": a Hermes session takes a snapshot of its tool table when it starts. If it began before the robot offered the tool (or before a robot restart that added it), it never sees it, even though robot://status, list_prompts and a fresh hermes mcp test robot all show it. Start a new Hermes session. (Hermes's log shows the reconnect and "registered N tool(s)" for the whole process; only that old session keeps its stale table.)
A one-off message to the face is ignored: ROS drops a message published before the connection matches. Republish it, as hermes_mcp does every second.

Security

The robot's ports are open only over the USB link, only to 10.55.0.1: audio (4713), MCP (8765) and a reserved webhook port (8766). Everything else of those is dropped on Wi-Fi.
ROS is blocked on every interface, including USB. It has no authentication and carries the camera and recognised names.
MCP needs a bearer token, kept in /etc/robot-mcp.env (root and the service user only).
SSH is open on all interfaces (key-only), by choice, for the mobile version.
Face recognition is opt-in and stores only numbers. Hermes keeps each look() photo in its image cache on your computer.
Never power the robot from a device whose battery you care about.

Files

Path	What
`ros/`	The ROS 2 workspace: `hermes_face`, `hermes_vision`, `hermes_audio`, `hermes_mcp`, `hermes_thermal`, `hermes_bringup`.
`pi/`	Everything installed on the Pi: `install.sh` and its parts, services, firewall, overlays.
`host/`	Your computer's side: the USB link script, Hermes and PipeWire config, a platform check.
`enclosure/`	OpenSCAD model, STLs and the print guide. Built from estimates: measure before printing.
`docs/`	`face-prototype.html` (the browser prototype the face was ported from) and `checkerboard-9x6.png` (for calibration).
`HANDOFF.md`	The original implementation brief. `CONTEXT.md` tracks where things really stand.

README.md Unescape Escape

Hermes robot