hermes-bot/README.md

# Hermes robot

A small Wi-Fi robot that gives Hermes Agent eyes, ears, a voice and (optionally) a
neck. All thinking happens on your Omarchy machine. The robot only takes pictures,
moves its head, and carries audio.

```
 Omarchy machine                                   Robot (Pi Zero 2 W)
 ───────────────                                   ───────────────────
 hermes-robot ─► Hermes Agent
                   │  voice mode / wake word
                   ├── default mic  = "Robot microphone" ◄─┐
                   ├── default sink = "Robot speaker" ─────┤  PipeWire pulse tunnel, TCP 4713
                   │                                       └─► pipewire-pulse ─► I²S mic(s) + amp
                   └── MCP tools: look(), turn_head() ──────► robot_mcp_server.py, HTTP 8765
                                                               ├─ Camera Module 3
                                                               └─ PCA9685 ─► 2 servos (optional)
```

## Parts

Prices are rough single-unit figures and will vary by vendor.

| Part | Notes | ≈ USD |
|---|---|---|
| Raspberry Pi Zero 2 W (the "WH" has headers soldered) | 2.4 GHz Wi-Fi only | 15–18 |
| microSD card, 16–32 GB | | 8 |
| Raspberry Pi Camera Module 3 | Autofocus; the "Wide" variant suits a desk robot | 25 |
| **Pi Zero camera cable** (22-pin to 15-pin) | The camera ships with the wrong cable for a Zero | 3 |
| INMP441 I²S MEMS microphone breakout, ×2 | One works; two give a stereo pair and a better signal | 3 each |
| MAX98357A I²S amplifier breakout | | 2–6 |
| Speaker, 4–8 Ω, 2–3 W, ~40 mm | | 3 |
| 5 V 3 A USB power supply | | 10 |
| *Optional head:* PCA9685 16-channel servo board | Clean hardware PWM over I²C | 6 |
| *Optional head:* 2× SG90 servos + pan/tilt bracket | | 8 |
| *Optional head:* 1000 µF electrolytic capacitor | Across the servo supply, against brown-outs | 1 |
| Dupont jumper wires, a bit of perfboard | | 5 |

Without the head, the robot is roughly $70; with it, about $85.

## Wiring

Pin numbers are **physical header pins** (1–40), not GPIO numbers. The microphone(s)
and amplifier share the I²S clock lines; that's expected.

**Microphones (INMP441)**

| INMP441 | Pi pin | |
|---|---|---|
| VDD | 1 (3.3 V) | **3.3 V only**, never 5 V |
| GND | 6 (GND) | |
| SCK | 12 (GPIO18, I²S BCLK) | |
| WS | 35 (GPIO19, I²S LRCLK) | |
| SD | 38 (GPIO20, I²S DIN) | |
| L/R | GND for mic A; 3.3 V for mic B | Selects left or right channel |

With two microphones, wire both identically except L/R, so their SD lines share pin 38.

**Amplifier (MAX98357A)**

| MAX98357A | Pi pin | |
|---|---|---|
| Vin | 2 (5 V) | |
| GND | 9 (GND) | |
| BCLK | 12 (GPIO18) | Shared with the mics |
| LRC | 35 (GPIO19) | Shared with the mics |
| DIN | 40 (GPIO21, I²S DOUT) | |
| SD, GAIN | leave unconnected | Defaults: enabled, 9 dB gain |
| Speaker + / − | to the speaker | |

**Optional head (PCA9685)**

| PCA9685 | Connects to | |
|---|---|---|
| VCC | Pi pin 17 (3.3 V) | Logic supply |
| GND | Pi pin 14 (GND) | |
| SDA | Pi pin 3 (GPIO2) | |
| SCL | Pi pin 5 (GPIO3) | |
| V+ terminal | **5 V from the power supply**, not from the Pi's header | Servos' power; add the capacitor here |
| Channel 0 | pan servo | |
| Channel 1 | tilt servo | |

Servos draw current spikes that will brown out the Pi if they share its header pins. Feed
the supply's 5 V to both the Pi (micro-USB) and the PCA9685's V+ terminal, with a common ground.

**Camera:** the Zero cable's narrow end goes in the Pi, contacts facing the board.

The `googlevoicehat-soundcard` overlay is the standard trick for this exact mic + amp pairing.
Google's AIY Voice HAT used the same chips, so its driver drives both in one full-duplex sound card.

## Setting up the Pi

1. Flash **Raspberry Pi OS Lite (64-bit)** with Raspberry Pi Imager. In its settings, set the
   hostname (e.g. `robot`), your user, your Wi-Fi, and enable SSH.
2. Boot, find the Pi's IP, and give it a **DHCP reservation** on your router.
3. Copy the `pi/` folder over and run the installer, passing your Omarchy machine's IP:

   ```bash
   scp -r pi robot.local:~/robot-setup
   ssh robot.local
   cd ~/robot-setup && chmod +x install.sh
   ./install.sh 192.168.1.20            # add --servos if you built the head
   sudo reboot
   ```

   The installer prints a token at the end. Keep it for the Hermes step.
4. After the reboot, check things on the Pi:

   ```bash
   arecord -l                            # a "googlevoicehat" card should be listed
   wpctl status                          # its source and sink should appear under Audio
   rpicam-still -o test.jpg              # camera works
   systemctl status robot-mcp            # MCP server running
   ```

5. If the camera is mounted upside down, set both `ROBOT_CAM_HFLIP=1` and `ROBOT_CAM_VFLIP=1`
   in `/etc/robot-mcp.env`, then run `sudo systemctl restart robot-mcp`.

## Setting up Omarchy

1. **Audio tunnel.** Copy `omarchy/50-robot-tunnel.conf` to
   `~/.config/pipewire/pipewire.conf.d/`, replace both `ROBOT_IP`s, then:

   ```bash
   systemctl --user restart pipewire pipewire-pulse wireplumber
   pactl list short sources | grep robot_mic          # should show the robot
   ```

   Omarchy's audio mixer (`wiremix`) will now list "Robot speaker" and "Robot microphone".
   If `pactl` is missing, install it with `sudo pacman -S libpulse`.

2. **Hermes.** Add `ROBOT_MCP_TOKEN=<token>` to `~/.hermes/.env`. Merge
   `omarchy/hermes-config-snippet.yaml` into `~/.hermes/config.yaml`, with `ROBOT_IP` replaced.
   Then check the connection:

   ```bash
   hermes mcp test robot
   ```

   The tools will appear in Hermes as `mcp_robot_look`, `mcp_robot_turn_head` and so on.

3. **Launcher.** Install the wrapper, which switches your default audio to the robot for the
   length of the session and restores it afterwards:

   ```bash
   install -Dm755 omarchy/hermes-robot ~/.local/bin/hermes-robot
   hermes-robot
   ```

   Inside Hermes, `/voice on` starts the conversation. Set up the wake word as described in the
   Hermes docs ("Wake Word") so you never need the keyboard.

## First test

Say or type: *"Look around and tell me what's on my desk."* Hermes should call `look`, then
describe the scene.

**One thing to confirm early:** that the image actually reaches the model. The server returns a
standard MCP image block, and the test client sees it correctly. If Hermes answers as though it
saw nothing, check `~/.hermes/logs/agent.log`, and make sure your model supports vision. If
Hermes turns out to drop image blocks from MCP results, the fallback is a small change to
`look()` to save the JPEG and return a path for Hermes's own vision tool. Tell me if that happens.

## Tuning and troubleshooting

- **Mic too quiet** (the level bar barely moves): on the Pi, run
  `wpctl set-volume @DEFAULT_AUDIO_SOURCE@ 2.5`. Alternatively, lower `voice.silence_threshold`
  in Hermes. A single mic is about 6 dB quieter than a pair, because the unused channel is
  averaged in.
- **Hermes interrupts itself:** this is the robot hearing its own speaker. Raise
  `voice.barge_in_threshold_multiplier`, point the speaker away from the mics, or enable
  `pi/pipewire/60-echo-cancel.conf.optional` and uncomment the `pulse.device` lines in the
  tunnel config.
- **Audio crackles:** this means weak Wi-Fi. Raise `pulse.latency` to 300–400. The Zero 2 W has
  only 2.4 GHz Wi-Fi and a small antenna, so keep it within reasonable range of the router.
- **Robot devices vanish after the Pi reboots:** the tunnels reconnect every 5 s by themselves.
  Start `hermes-robot` again once they reappear.
- **Head turns the wrong way:** set `ROBOT_PAN_INVERT=1` or `ROBOT_TILT_INVERT=1` in
  `/etc/robot-mcp.env`, then restart `robot-mcp`.

## Security

- The microphone stream (port 4713) and the camera server (port 8765) are open only to your
  Omarchy machine's IP, via nftables on the Pi.
- The camera additionally requires the bearer token.
- Traffic is unencrypted on your LAN. If that matters to you, put both machines on Tailscale and
  use the Pi's tailnet address in both configs.
- If your Omarchy machine's IP changes, re-run `install.sh` with the new one.

## Files

```
pi/
  install.sh                         one-shot Pi setup
  robot_mcp_server.py                MCP server: look, turn_head, head_position
  requirements.txt                   MCP Python SDK 2.x
  robot-mcp.service                  systemd unit
  nftables-robot.nft                 port restriction
  pipewire/50-network-server.conf    exposes audio to the network
  pipewire/60-echo-cancel.conf.optional
omarchy/
  50-robot-tunnel.conf               robot audio as local devices
  hermes-config-snippet.yaml         MCP and voice settings for Hermes
  hermes-robot                       launcher that swaps default audio devices
```