Initial commit
Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
This commit is contained in:
Claude Sonnet 5.5
@@ -0,0 +1,205 @@
|
||||
# Hermes robot
|
||||
|
||||
A small Wi-Fi robot that gives Hermes Agent eyes, ears, a voice and (optionally) a
|
||||
neck. All thinking happens on your Omarchy machine. The robot only takes pictures,
|
||||
moves its head, and carries audio.
|
||||
|
||||
```
|
||||
Omarchy machine Robot (Pi Zero 2 W)
|
||||
─────────────── ───────────────────
|
||||
hermes-robot ─► Hermes Agent
|
||||
│ voice mode / wake word
|
||||
├── default mic = "Robot microphone" ◄─┐
|
||||
├── default sink = "Robot speaker" ─────┤ PipeWire pulse tunnel, TCP 4713
|
||||
│ └─► pipewire-pulse ─► I²S mic(s) + amp
|
||||
└── MCP tools: look(), turn_head() ──────► robot_mcp_server.py, HTTP 8765
|
||||
├─ Camera Module 3
|
||||
└─ PCA9685 ─► 2 servos (optional)
|
||||
```
|
||||
|
||||
## Parts
|
||||
|
||||
Prices are rough single-unit figures and will vary by vendor.
|
||||
|
||||
| Part | Notes | ≈ USD |
|
||||
|---|---|---|
|
||||
| Raspberry Pi Zero 2 W (the "WH" has headers soldered) | 2.4 GHz Wi-Fi only | 15–18 |
|
||||
| microSD card, 16–32 GB | | 8 |
|
||||
| Raspberry Pi Camera Module 3 | Autofocus; the "Wide" variant suits a desk robot | 25 |
|
||||
| **Pi Zero camera cable** (22-pin to 15-pin) | The camera ships with the wrong cable for a Zero | 3 |
|
||||
| INMP441 I²S MEMS microphone breakout, ×2 | One works; two give a stereo pair and a better signal | 3 each |
|
||||
| MAX98357A I²S amplifier breakout | | 2–6 |
|
||||
| Speaker, 4–8 Ω, 2–3 W, ~40 mm | | 3 |
|
||||
| 5 V 3 A USB power supply | | 10 |
|
||||
| *Optional head:* PCA9685 16-channel servo board | Clean hardware PWM over I²C | 6 |
|
||||
| *Optional head:* 2× SG90 servos + pan/tilt bracket | | 8 |
|
||||
| *Optional head:* 1000 µF electrolytic capacitor | Across the servo supply, against brown-outs | 1 |
|
||||
| Dupont jumper wires, a bit of perfboard | | 5 |
|
||||
|
||||
Without the head, the robot is roughly $70; with it, about $85.
|
||||
|
||||
## Wiring
|
||||
|
||||
Pin numbers are **physical header pins** (1–40), not GPIO numbers. The microphone(s)
|
||||
and amplifier share the I²S clock lines; that's expected.
|
||||
|
||||
**Microphones (INMP441)**
|
||||
|
||||
| INMP441 | Pi pin | |
|
||||
|---|---|---|
|
||||
| VDD | 1 (3.3 V) | **3.3 V only**, never 5 V |
|
||||
| GND | 6 (GND) | |
|
||||
| SCK | 12 (GPIO18, I²S BCLK) | |
|
||||
| WS | 35 (GPIO19, I²S LRCLK) | |
|
||||
| SD | 38 (GPIO20, I²S DIN) | |
|
||||
| L/R | GND for mic A; 3.3 V for mic B | Selects left or right channel |
|
||||
|
||||
With two microphones, wire both identically except L/R, so their SD lines share pin 38.
|
||||
|
||||
**Amplifier (MAX98357A)**
|
||||
|
||||
| MAX98357A | Pi pin | |
|
||||
|---|---|---|
|
||||
| Vin | 2 (5 V) | |
|
||||
| GND | 9 (GND) | |
|
||||
| BCLK | 12 (GPIO18) | Shared with the mics |
|
||||
| LRC | 35 (GPIO19) | Shared with the mics |
|
||||
| DIN | 40 (GPIO21, I²S DOUT) | |
|
||||
| SD, GAIN | leave unconnected | Defaults: enabled, 9 dB gain |
|
||||
| Speaker + / − | to the speaker | |
|
||||
|
||||
**Optional head (PCA9685)**
|
||||
|
||||
| PCA9685 | Connects to | |
|
||||
|---|---|---|
|
||||
| VCC | Pi pin 17 (3.3 V) | Logic supply |
|
||||
| GND | Pi pin 14 (GND) | |
|
||||
| SDA | Pi pin 3 (GPIO2) | |
|
||||
| SCL | Pi pin 5 (GPIO3) | |
|
||||
| V+ terminal | **5 V from the power supply**, not from the Pi's header | Servos' power; add the capacitor here |
|
||||
| Channel 0 | pan servo | |
|
||||
| Channel 1 | tilt servo | |
|
||||
|
||||
Servos draw current spikes that will brown out the Pi if they share its header pins. Feed
|
||||
the supply's 5 V to both the Pi (micro-USB) and the PCA9685's V+ terminal, with a common ground.
|
||||
|
||||
**Camera:** the Zero cable's narrow end goes in the Pi, contacts facing the board.
|
||||
|
||||
The `googlevoicehat-soundcard` overlay is the standard trick for this exact mic + amp pairing.
|
||||
Google's AIY Voice HAT used the same chips, so its driver drives both in one full-duplex sound card.
|
||||
|
||||
## Setting up the Pi
|
||||
|
||||
1. Flash **Raspberry Pi OS Lite (64-bit)** with Raspberry Pi Imager. In its settings, set the
|
||||
hostname (e.g. `robot`), your user, your Wi-Fi, and enable SSH.
|
||||
2. Boot, find the Pi's IP, and give it a **DHCP reservation** on your router.
|
||||
3. Copy the `pi/` folder over and run the installer, passing your Omarchy machine's IP:
|
||||
|
||||
```bash
|
||||
scp -r pi robot.local:~/robot-setup
|
||||
ssh robot.local
|
||||
cd ~/robot-setup && chmod +x install.sh
|
||||
./install.sh 192.168.1.20 # add --servos if you built the head
|
||||
sudo reboot
|
||||
```
|
||||
|
||||
The installer prints a token at the end. Keep it for the Hermes step.
|
||||
4. After the reboot, check things on the Pi:
|
||||
|
||||
```bash
|
||||
arecord -l # a "googlevoicehat" card should be listed
|
||||
wpctl status # its source and sink should appear under Audio
|
||||
rpicam-still -o test.jpg # camera works
|
||||
systemctl status robot-mcp # MCP server running
|
||||
```
|
||||
|
||||
5. If the camera is mounted upside down, set both `ROBOT_CAM_HFLIP=1` and `ROBOT_CAM_VFLIP=1`
|
||||
in `/etc/robot-mcp.env`, then run `sudo systemctl restart robot-mcp`.
|
||||
|
||||
## Setting up Omarchy
|
||||
|
||||
1. **Audio tunnel.** Copy `omarchy/50-robot-tunnel.conf` to
|
||||
`~/.config/pipewire/pipewire.conf.d/`, replace both `ROBOT_IP`s, then:
|
||||
|
||||
```bash
|
||||
systemctl --user restart pipewire pipewire-pulse wireplumber
|
||||
pactl list short sources | grep robot_mic # should show the robot
|
||||
```
|
||||
|
||||
Omarchy's audio mixer (`wiremix`) will now list "Robot speaker" and "Robot microphone".
|
||||
If `pactl` is missing, install it with `sudo pacman -S libpulse`.
|
||||
|
||||
2. **Hermes.** Add `ROBOT_MCP_TOKEN=<token>` to `~/.hermes/.env`. Merge
|
||||
`omarchy/hermes-config-snippet.yaml` into `~/.hermes/config.yaml`, with `ROBOT_IP` replaced.
|
||||
Then check the connection:
|
||||
|
||||
```bash
|
||||
hermes mcp test robot
|
||||
```
|
||||
|
||||
The tools will appear in Hermes as `mcp_robot_look`, `mcp_robot_turn_head` and so on.
|
||||
|
||||
3. **Launcher.** Install the wrapper, which switches your default audio to the robot for the
|
||||
length of the session and restores it afterwards:
|
||||
|
||||
```bash
|
||||
install -Dm755 omarchy/hermes-robot ~/.local/bin/hermes-robot
|
||||
hermes-robot
|
||||
```
|
||||
|
||||
Inside Hermes, `/voice on` starts the conversation. Set up the wake word as described in the
|
||||
Hermes docs ("Wake Word") so you never need the keyboard.
|
||||
|
||||
## First test
|
||||
|
||||
Say or type: *"Look around and tell me what's on my desk."* Hermes should call `look`, then
|
||||
describe the scene.
|
||||
|
||||
**One thing to confirm early:** that the image actually reaches the model. The server returns a
|
||||
standard MCP image block, and the test client sees it correctly. If Hermes answers as though it
|
||||
saw nothing, check `~/.hermes/logs/agent.log`, and make sure your model supports vision. If
|
||||
Hermes turns out to drop image blocks from MCP results, the fallback is a small change to
|
||||
`look()` to save the JPEG and return a path for Hermes's own vision tool. Tell me if that happens.
|
||||
|
||||
## Tuning and troubleshooting
|
||||
|
||||
- **Mic too quiet** (the level bar barely moves): on the Pi, run
|
||||
`wpctl set-volume @DEFAULT_AUDIO_SOURCE@ 2.5`. Alternatively, lower `voice.silence_threshold`
|
||||
in Hermes. A single mic is about 6 dB quieter than a pair, because the unused channel is
|
||||
averaged in.
|
||||
- **Hermes interrupts itself:** this is the robot hearing its own speaker. Raise
|
||||
`voice.barge_in_threshold_multiplier`, point the speaker away from the mics, or enable
|
||||
`pi/pipewire/60-echo-cancel.conf.optional` and uncomment the `pulse.device` lines in the
|
||||
tunnel config.
|
||||
- **Audio crackles:** this means weak Wi-Fi. Raise `pulse.latency` to 300–400. The Zero 2 W has
|
||||
only 2.4 GHz Wi-Fi and a small antenna, so keep it within reasonable range of the router.
|
||||
- **Robot devices vanish after the Pi reboots:** the tunnels reconnect every 5 s by themselves.
|
||||
Start `hermes-robot` again once they reappear.
|
||||
- **Head turns the wrong way:** set `ROBOT_PAN_INVERT=1` or `ROBOT_TILT_INVERT=1` in
|
||||
`/etc/robot-mcp.env`, then restart `robot-mcp`.
|
||||
|
||||
## Security
|
||||
|
||||
- The microphone stream (port 4713) and the camera server (port 8765) are open only to your
|
||||
Omarchy machine's IP, via nftables on the Pi.
|
||||
- The camera additionally requires the bearer token.
|
||||
- Traffic is unencrypted on your LAN. If that matters to you, put both machines on Tailscale and
|
||||
use the Pi's tailnet address in both configs.
|
||||
- If your Omarchy machine's IP changes, re-run `install.sh` with the new one.
|
||||
|
||||
## Files
|
||||
|
||||
```
|
||||
pi/
|
||||
install.sh one-shot Pi setup
|
||||
robot_mcp_server.py MCP server: look, turn_head, head_position
|
||||
requirements.txt MCP Python SDK 2.x
|
||||
robot-mcp.service systemd unit
|
||||
nftables-robot.nft port restriction
|
||||
pipewire/50-network-server.conf exposes audio to the network
|
||||
pipewire/60-echo-cancel.conf.optional
|
||||
omarchy/
|
||||
50-robot-tunnel.conf robot audio as local devices
|
||||
hermes-config-snippet.yaml MCP and voice settings for Hermes
|
||||
hermes-robot launcher that swaps default audio devices
|
||||
```
|
||||
Reference in New Issue
Block a user