diff --git a/CLAUDE.md b/CLAUDE.md index 59c43da..f59490e 100644 --- a/CLAUDE.md +++ b/CLAUDE.md @@ -8,11 +8,13 @@ the Pi; on the hosts, ROS runs only in podman containers. **Start with `HANDOFF.md`.** It is the implementation brief: hardware facts, what exists, the work packages in order, acceptance checks, and the list of unverified assumptions. -- `pi/` and `ros/` run on the robot; `omarchy/` (to become `host/`) runs on the owner's machines. +- `pi/` and `ros/` run on the robot; `host/` runs on the owner's machines. `CONTEXT.md` tracks where + the work stands; `HANDOFF.md` is the original brief and has drifted (ROS stays on the Pi, and + `hermes_mcp` is the only bridge out; see `CONTEXT.md`). - Package 0a (platform proof) comes before everything; start with `host/check-pi-platform.sh`. - `docs/face-prototype.html` is the reference for the face renderer; port it, don't ship it. - `enclosure/` holds the OpenSCAD model and STLs. It was built from estimates; refine it from the owner's caliper measurements and test prints before anyone prints the large parts. -- `README.md` still describes the earlier Pi Zero build until package 7 rewrites it. +- `README.md` is the guide for this build; keep it a guide for a person, and keep `CONTEXT.md` honest. - The MCP server uses the MCP Python SDK 2.x (`mcp.server.mcpserver.MCPServer`), not 1.x `FastMCP`. - Ask before rebooting the Pi, upgrading or holding kernel packages, or opening network ports. diff --git a/CONTEXT.md b/CONTEXT.md index 5c82307..5bb5a85 100644 --- a/CONTEXT.md +++ b/CONTEXT.md @@ -223,8 +223,8 @@ returns a JPEG in 0.08 s, a wrong token is refused, the Wi-Fi path is dropped by the face went `looking` -> `idle` on connect and `agent gone, sleeping` about 40 s after the client left. The token is in `/etc/robot-mcp.env` (root:hermes, mode 640), generated by `pi/install-robot-service.sh` and never printed; read it with `sudo cat`. The MCP SDK is in `~/venv` -(`pip install "mcp>=2"`). The old `pi/robot_mcp_server.py`, `pi/robot-mcp.service` and `pi/install.sh` -are superseded (package 5 rewrites the installer). **Not yet verified:** a real Hermes client, and +(`pip install "mcp>=2"`). The old Pi Zero files (`pi/robot_mcp_server.py`, `pi/robot-mcp.service`, +`pi/requirements.txt`, the old `pi/install.sh`, and `tools/`) were deleted (git has them). **Not yet verified:** a real Hermes client, and whether Hermes passes MCP image content to the model (HANDOFF package 6). **Hermes on talos is wired to it (2026-10-05, with the owner's authorisation).** Hermes Agent v0.21.5 is @@ -264,34 +264,36 @@ unrelated photo scored 0.16. **Nobody is enrolled yet**; the owner enrols themse ## Next -1. Finish package 0b: Wi-Fi-off boot test, replug recovery, serial login check, NixOS snippet in - `host/nixos/`. A new Omarchy host with a GPU (runs the local - model "ornith") arrives this week: it needs only `host/omarchy/robot-usb-link.sh` and its SSH key - added to the Pi. -2. `hermes_head` and the servos when they arrive (then `head: true`), echo cancel and the `robot_mic` - naming once speakers arrive, a fan before any enclosure, the Wi-Fi-off boot test, a real Hermes - client against `hermes_mcp`, and the rest of `HANDOFF.md`. -3. Confirm the retuned listening detector live (the flicker fix is unit-tested only); a second person - in frame for the recognition test (not possible on 2026-10-04). +1. **Camera calibration** (needs the owner and a flat checkerboard, printed or on a screen, with the + square size measured): collect ~20 views at different poses, solve with OpenCV, write the + camera_info YAML so `camera_ros` stops logging the missing-file error. V2's perception needs it. +2. Waiting on hardware: the servos (`hermes_head`, the pan/tilt URDF, then `head: true`), the speakers + (playback, echo cancel, the `robot_mic` naming on the host, Hermes talking aloud), the fan (before + any enclosure), the splitter, the printed enclosure (measure first, see `enclosure/README.md`). +3. Still open: replug and second-host recovery of the USB link, the serial console login, an + `agent_timeout_s` check against a long idle Hermes session, the retuned listening detector live + (unit-tested only), a second person in frame, running `pi/install.sh` on a blank card, and the + NixOS host config (skipped for now by the owner). +4. A new Omarchy host with a GPU (it runs the local `ornith` model) is arriving: it needs only + `host/omarchy/robot-usb-link.sh` and its SSH key added to the Pi. ## Repository Branch `master`. `.gitignore` covers `.remember/`, caches, `.venv/`, `*.orig`, `*.rej` and -`pi/50-cloud-init.yaml`. `tools/check-pi-platform.sh` duplicates `host/check-pi-platform.sh` and can -probably go. +`pi/50-cloud-init.yaml`. Layout: `ros/` (five packages), `pi/` (everything installed on the Pi), +`host/` (the owner's machines: `host/omarchy/` has the USB link script, Hermes and PipeWire config and +the launcher; `host/check-pi-platform.sh`), `enclosure/`, `docs/face-prototype.html`. `README.md` is +rewritten for this build. **`pi/install.sh` is the one installer** (re-runnable: run twice on the live +robot, the second run changed nothing, exit 0): it installs ROS, builds the libcamera fork (pinned +commit) and `camera_ros`, sets up the venv, models, workspace, USB gadget, audio overlay, firewall, +PipeWire, mixer and services. It has not been run on a blank card (no spare card). HANDOFF.md is the +original brief and has drifted; where they differ, this file wins. -## Security review notes (parked until the USB link is up) +## Security notes (all resolved) -An automated review of the initial commit flagged three items. All stem from the code predating -the USB-gadget design (HANDOFF.md package 0b); none is exposed until the Pi runs the services. - -- `pi/robot_mcp_server.py`: plain HTTP, `ROBOT_MCP_BIND` defaults to `0.0.0.0`. Plan: default to - `10.55.0.2`, start after `usb0`. -- `pi/pipewire/50-network-server.conf`: TCP 4713 on all interfaces, unrestricted. Plan: keep, - comment that the firewall is the protection. -- `pi/nftables-robot.nft`: still the Pi Zero version (single host IP, any interface). Package 0b - rewrites it to accept only on the USB interface from `10.55.0.1`. -- `pi/install.sh` permissions looked fine (`/etc/robot-mcp.env` is `root:$USER`, 640). - -The owner wants to confirm the real IPs on the link before changing any of this. If the USB -link doesn't work, revisit: TLS or a Tailscale-only bind for MCP and audio. +The automated review of the initial commit flagged three items, all from the Pi Zero design: +`ROBOT_MCP_BIND` defaulting to `0.0.0.0` (now the USB address only, with a bearer token), PipeWire on +TCP 4713 on every interface (now protected by the boot-time firewall, which loads first), and the +single-IP any-interface nftables file (rewritten: usb0 from 10.55.0.1 only). Later reviews added: +DDS is blocked on every interface and discovery is local-only; recognised names are not logged; node +parameters are clamped. SSH stays open on all interfaces (key-only) by the owner's choice, for V2. diff --git a/README.md b/README.md index a9e55a5..28e7075 100644 --- a/README.md +++ b/README.md @@ -1,205 +1,175 @@ # Hermes robot -A small Wi-Fi robot that gives Hermes Agent eyes, ears, a voice and (optionally) a -neck. All thinking happens on your Omarchy machine. The robot only takes pictures, -moves its head, and carries audio. +A small desk robot that gives Hermes Agent (Nous Research's open-source agent) eyes, ears, +a voice and a face. It does no thinking of its own: Hermes runs on your computer and uses the robot +through MCP. + +- **Eyes:** a camera. Hermes calls `look()` and gets a current photo. +- **Face:** an oscilloscope-style display that shows what the agent is really doing: asleep (no agent + connected), idle, listening, thinking, looking, speaking. The mouth draws the actual voice, and the + eyes glance toward whoever the camera sees. +- **Ears and voice:** the HAT's two microphones and speakers, shared with your computer as ordinary + audio devices. +- **Neck:** pan and tilt servos (`turn_head`), once they are fitted. +- **One cable:** USB-C carries power and a private network link to your computer. Wi-Fi is a fallback. + +It runs ROS 2 Jazzy on Ubuntu 24.04, so a mobile version can grow out of it. Hermes needs no ROS: it +only speaks MCP. + +## Status + +| Works and is tested | Waiting on hardware | +|---|---| +| Face on the display, always on at power-up, with a sleeping pose | Servos: `turn_head`, `head_position`, the pan/tilt URDF | +| Camera, face detection, opt-in recognition of people you enrol | Speakers: playback, echo cancel, Hermes talking out loud | +| Eyes follow faces; mouth draws your live voice | Fan (the Pi runs at about 62 °C open-air without one) | +| MCP bridge: `hermes mcp test robot` passes, a real Hermes turn calls `look()` | USB-C splitter (the Pi is powered from your computer's port for now) | +| USB network link, firewall, boots and works with Wi-Fi off | Printed enclosure (`enclosure/`) | + +## How it fits together ``` - Omarchy machine Robot (Pi Zero 2 W) - ─────────────── ─────────────────── - hermes-robot ─► Hermes Agent - │ voice mode / wake word - ├── default mic = "Robot microphone" ◄─┐ - ├── default sink = "Robot speaker" ─────┤ PipeWire pulse tunnel, TCP 4713 - │ └─► pipewire-pulse ─► I²S mic(s) + amp - └── MCP tools: look(), turn_head() ──────► robot_mcp_server.py, HTTP 8765 - ├─ Camera Module 3 - └─ PCA9685 ─► 2 servos (optional) +Your computer (Omarchy) Raspberry Pi 4 + BrainCraft HAT (Ubuntu 24.04) + Hermes Agent ── MCP over HTTP, USB link ────► hermes_mcp (the only way in or out of ROS) + PipeWire ◄── audio, TCP 4713 ──────────► PipeWire (not ROS) + │ ROS 2 graph, this machine only + camera_ros ► hermes_face_detector ► /vision/faces ─┐ + hermes_audio_state ► levels, speaking/listening ───┤ + hermes_mcp ► /hermes/state (the agent heartbeat) ──┴► hermes_face ► display ``` +Every node is its own process, started by one launch file (`ros/hermes_bringup`) and restarted if it +dies. ROS never leaves the Pi. The firewall blocks it on every interface, so `hermes_mcp` is the one +door. + ## Parts -Prices are rough single-unit figures and will vary by vendor. +| Part | Notes | +|---|---| +| Raspberry Pi 4 | 8 GB here. Ubuntu Server 24.04 arm64. | +| Adafruit BrainCraft HAT | 1.54″ ST7789 display, WM8960 audio codec with two mics, speaker and servo ports. | +| Arducam UC-346 | OV5647 camera, 15-pin ribbon, fixed focus. | +| 2 × MG90S servos | Pan (in the base) on the HAT's GPIO12 port, tilt (camera yoke) on GPIO13. | +| 2 × speakers | 8 Ω, up to 1 W, JST PH 2-pin. | +| USB-C power/data splitter | Power from a 5.1 V 3 A supply, data only to your computer. A Pi 4 cannot negotiate USB PD. | +| Fan | On the HAT's fan header. | +| Enclosure | `enclosure/`: printed, with a turntable. | -| Part | Notes | ≈ USD | -|---|---|---| -| Raspberry Pi Zero 2 W (the "WH" has headers soldered) | 2.4 GHz Wi-Fi only | 15–18 | -| microSD card, 16–32 GB | | 8 | -| Raspberry Pi Camera Module 3 | Autofocus; the "Wide" variant suits a desk robot | 25 | -| **Pi Zero camera cable** (22-pin to 15-pin) | The camera ships with the wrong cable for a Zero | 3 | -| INMP441 I²S MEMS microphone breakout, ×2 | One works; two give a stereo pair and a better signal | 3 each | -| MAX98357A I²S amplifier breakout | | 2–6 | -| Speaker, 4–8 Ω, 2–3 W, ~40 mm | | 3 | -| 5 V 3 A USB power supply | | 10 | -| *Optional head:* PCA9685 16-channel servo board | Clean hardware PWM over I²C | 6 | -| *Optional head:* 2× SG90 servos + pan/tilt bracket | | 8 | -| *Optional head:* 1000 µF electrolytic capacitor | Across the servo supply, against brown-outs | 1 | -| Dupont jumper wires, a bit of perfboard | | 5 | +## Physical steps -Without the head, the robot is roughly $70; with it, about $85. - -## Wiring - -Pin numbers are **physical header pins** (1–40), not GPIO numbers. The microphone(s) -and amplifier share the I²S clock lines; that's expected. - -**Microphones (INMP441)** - -| INMP441 | Pi pin | | -|---|---|---| -| VDD | 1 (3.3 V) | **3.3 V only**, never 5 V | -| GND | 6 (GND) | | -| SCK | 12 (GPIO18, I²S BCLK) | | -| WS | 35 (GPIO19, I²S LRCLK) | | -| SD | 38 (GPIO20, I²S DIN) | | -| L/R | GND for mic A; 3.3 V for mic B | Selects left or right channel | - -With two microphones, wire both identically except L/R, so their SD lines share pin 38. - -**Amplifier (MAX98357A)** - -| MAX98357A | Pi pin | | -|---|---|---| -| Vin | 2 (5 V) | | -| GND | 9 (GND) | | -| BCLK | 12 (GPIO18) | Shared with the mics | -| LRC | 35 (GPIO19) | Shared with the mics | -| DIN | 40 (GPIO21, I²S DOUT) | | -| SD, GAIN | leave unconnected | Defaults: enabled, 9 dB gain | -| Speaker + / − | to the speaker | | - -**Optional head (PCA9685)** - -| PCA9685 | Connects to | | -|---|---|---| -| VCC | Pi pin 17 (3.3 V) | Logic supply | -| GND | Pi pin 14 (GND) | | -| SDA | Pi pin 3 (GPIO2) | | -| SCL | Pi pin 5 (GPIO3) | | -| V+ terminal | **5 V from the power supply**, not from the Pi's header | Servos' power; add the capacitor here | -| Channel 0 | pan servo | | -| Channel 1 | tilt servo | | - -Servos draw current spikes that will brown out the Pi if they share its header pins. Feed -the supply's 5 V to both the Pi (micro-USB) and the PCA9685's V+ terminal, with a common ground. - -**Camera:** the Zero cable's narrow end goes in the Pi, contacts facing the board. - -The `googlevoicehat-soundcard` overlay is the standard trick for this exact mic + amp pairing. -Google's AIY Voice HAT used the same chips, so its driver drives both in one full-duplex sound card. +1. The HAT's "3pin Volt" jumper must be on 5 V, which is the factory setting. Check it with a meter. +2. Pan servo into the GPIO12 port, tilt servo into GPIO13. +3. Camera ribbon: leave a gentle loop, because only the tilt flexes it. +4. **Slide the HAT's "Audio" switch (right edge) to ON.** With it off, everything looks configured and + every recording is exactly silent. This cost an hour once. +5. Power the Pi from the splitter, not straight from a port that cannot give 3 A. Never power it from a + laptop or tablet you care about the battery of. ## Setting up the Pi -1. Flash **Raspberry Pi OS Lite (64-bit)** with Raspberry Pi Imager. In its settings, set the - hostname (e.g. `robot`), your user, your Wi-Fi, and enable SSH. -2. Boot, find the Pi's IP, and give it a **DHCP reservation** on your router. -3. Copy the `pi/` folder over and run the installer, passing your Omarchy machine's IP: +1. **Flash** Ubuntu Server 24.04 for the Pi (arm64, preinstalled image) and check its SHA256. +2. **Configure first boot** on the card's `system-boot` partition. In `user-data`, create the user + `hermes` with key-only login and the hostname `hermes-bot`. For Wi-Fi, put the access point in + `network-config` (a template is `pi/50-cloud-init.yaml`; it holds your password, so it is + gitignored). cloud-init applies these on the *first* boot only: to change Wi-Fi later, edit + `/etc/netplan/50-cloud-init.yaml` on the Pi. +3. **Boot, then copy this repo to the Pi and run the installer:** - ```bash - scp -r pi robot.local:~/robot-setup - ssh robot.local - cd ~/robot-setup && chmod +x install.sh - ./install.sh 192.168.1.20 # add --servos if you built the head - sudo reboot + ``` + rsync -a ./ hermes@:~/hermes-bot/ + ssh hermes@ 'cd hermes-bot && sudo ./pi/install.sh' ``` - The installer prints a token at the end. Keep it for the Hermes step. -4. After the reboot, check things on the Pi: + It installs ROS, builds the camera driver against the Raspberry Pi libcamera fork, builds the + workspace, sets up audio, the USB link, the firewall and the services. It can be run again safely. + The first run takes a while (the libcamera build alone is about 15 minutes). **Reboot afterwards.** +4. Plug the USB-C data cable into your computer. - ```bash - arecord -l # a "googlevoicehat" card should be listed - wpctl status # its source and sink should appear under Audio - rpicam-still -o test.jpg # camera works - systemctl status robot-mcp # MCP server running - ``` +## Setting up your computer (Omarchy) -5. If the camera is mounted upside down, set both `ROBOT_CAM_HFLIP=1` and `ROBOT_CAM_VFLIP=1` - in `/etc/robot-mcp.env`, then run `sudo systemctl restart robot-mcp`. - -## Setting up Omarchy - -1. **Audio tunnel.** Copy `omarchy/50-robot-tunnel.conf` to - `~/.config/pipewire/pipewire.conf.d/`, replace both `ROBOT_IP`s, then: - - ```bash - systemctl --user restart pipewire pipewire-pulse wireplumber - pactl list short sources | grep robot_mic # should show the robot - ``` - - Omarchy's audio mixer (`wiremix`) will now list "Robot speaker" and "Robot microphone". - If `pactl` is missing, install it with `sudo pacman -S libpulse`. - -2. **Hermes.** Add `ROBOT_MCP_TOKEN=` to `~/.hermes/.env`. Merge - `omarchy/hermes-config-snippet.yaml` into `~/.hermes/config.yaml`, with `ROBOT_IP` replaced. - Then check the connection: - - ```bash - hermes mcp test robot - ``` - - The tools will appear in Hermes as `mcp_robot_look`, `mcp_robot_turn_head` and so on. - -3. **Launcher.** Install the wrapper, which switches your default audio to the robot for the - length of the session and restores it afterwards: - - ```bash - install -Dm755 omarchy/hermes-robot ~/.local/bin/hermes-robot - hermes-robot - ``` - - Inside Hermes, `/voice on` starts the conversation. Set up the wake word as described in the - Hermes docs ("Wake Word") so you never need the keyboard. +1. **Network link:** `sudo host/omarchy/robot-usb-link.sh` gives your end of the link `10.55.0.1/30`, + matched by the gadget's fixed MAC, so any port and any machine works. The robot is `10.55.0.2`. +2. **Hermes:** the token is in `/etc/robot-mcp.env` on the Pi. Put it in `~/.hermes/.env` as + `ROBOT_MCP_TOKEN=`, and merge `host/omarchy/hermes-config-snippet.yaml` into + `~/.hermes/config.yaml`. +3. **Audio (optional, needs the speakers):** `host/omarchy/50-robot-tunnel.conf` makes the robot's + microphone and speaker appear as devices; `host/omarchy/hermes-robot` starts Hermes with them as + the defaults. ## First test -Say or type: *"Look around and tell me what's on my desk."* Hermes should call `look`, then -describe the scene. +``` +ping 10.55.0.2 +hermes mcp test robot # connects, finds the look tool +hermes chat -q "Use the look tool and tell me what you see." +``` -**One thing to confirm early:** that the image actually reaches the model. The server returns a -standard MCP image block, and the test client sees it correctly. If Hermes answers as though it -saw nothing, check `~/.hermes/logs/agent.log`, and make sure your model supports vision. If -Hermes turns out to drop image blocks from MCP results, the fallback is a small change to -`look()` to save the JPEG and return a path for Hermes's own vision tool. Tell me if that happens. +The robot's face wakes when Hermes connects and falls asleep about 40 seconds after it leaves. A +look-and-describe turn takes a minute or so on local models: `look()` itself is instant. Hermes does not +give MCP images to the main model: it saves the photo to its image cache and the model calls +`vision_analyze` on it. + +## Day to day + +| State | Face | Comes from | +|---|---|---| +| sleeping | eyes closed, dim, slowly breathing | no agent connected (no heartbeat) | +| idle | blinks, smiles | the agent is connected | +| listening | mouth draws your voice, dimmer | the microphone (never wakes a sleeping face) | +| thinking | eyes trace looping curves | the agent | +| looking | squint, then a flash | `look()` | +| speaking | mouth draws the outgoing audio | the speaker | + +- **Logs and control:** `journalctl -u robot -f` (the face logs every state change), `sudo systemctl + restart robot`. +- **Enrol yourself** so the eyes prefer you (numbers only are stored, never photos, in + `~/.local/share/hermes/faces.json`): on the Pi, with the robot running and after `source + ~/robot-env.sh`, `python3 -m hermes_vision.people enroll YOURNAME`. `list` shows who is enrolled, `forget YOURNAME` removes you. +- **Update the code:** `rsync -a --delete --exclude __pycache__ ros/ hermes@:~/ros_ws/src/`, then + `colcon build --symlink-install` in `~/ros_ws`, then `sudo systemctl restart robot`. +- **Run a node by hand:** `source ~/robot-env.sh` first: it puts the libcamera fork and the right + OpenCV on the path, and keeps ROS local. ## Tuning and troubleshooting -- **Mic too quiet** (the level bar barely moves): on the Pi, run - `wpctl set-volume @DEFAULT_AUDIO_SOURCE@ 2.5`. Alternatively, lower `voice.silence_threshold` - in Hermes. A single mic is about 6 dB quieter than a pair, because the unused channel is - averaged in. -- **Hermes interrupts itself:** this is the robot hearing its own speaker. Raise - `voice.barge_in_threshold_multiplier`, point the speaker away from the mics, or enable - `pi/pipewire/60-echo-cancel.conf.optional` and uncomment the `pulse.device` lines in the - tunnel config. -- **Audio crackles:** this means weak Wi-Fi. Raise `pulse.latency` to 300–400. The Zero 2 W has - only 2.4 GHz Wi-Fi and a small antenna, so keep it within reasonable range of the router. -- **Robot devices vanish after the Pi reboots:** the tunnels reconnect every 5 s by themselves. - Start `hermes-robot` again once they reappear. -- **Head turns the wrong way:** set `ROBOT_PAN_INVERT=1` or `ROBOT_TILT_INVERT=1` in - `/etc/robot-mcp.env`, then restart `robot-mcp`. +- **Recordings are exactly silent:** the Audio switch is off (see above). +- **Microphone too noisy or too quiet:** WirePlumber resets the codec's capture gain to +30 dB. The + installer sets PipeWire's source volume to 63% (about +17 dB); change it with `pactl + set-source-volume @DEFAULT_SOURCE@ 63%`. Remember that PipeWire's scale is cubic. +- **The face flickers between listening and idle:** raise `hold_s` in `ros/hermes_audio/hermes_audio/levels.py`. +- **Stock libcamera crashes when capturing:** that is expected on this kernel. The installer builds the + Raspberry Pi fork into `/opt/libcamera-rpi` and builds `camera_ros` against it. Anything that uses + the camera must run after `source ~/robot-env.sh`. +- **Display shows noise or a band:** the 240×240 glass sits 80 rows into the controller's memory + (`y_offset: 80`), and `rotation: 270` is the mounted pose. Both are in `ros/hermes_bringup/config/robot.yaml`. +- **`GPIO busy` or a crash at start from systemd:** the display's GPIO backend needs a writable working + directory; the unit sets one. +- **The USB interface is unmanaged:** netplan must match `usb0` by *name*, not by MAC. +- **`ros2 node list` shows only some nodes:** use `ros2 node list --no-daemon --spin-time 6`. +- **The Pi's clock is wrong with Wi-Fi off:** there is no internet to sync from; it stays roughly right + but unsynchronised. +- **A one-off message to the face is ignored:** ROS drops a message published before the connection + matches. Republish it, as `hermes_mcp` does every second. ## Security -- The microphone stream (port 4713) and the camera server (port 8765) are open only to your - Omarchy machine's IP, via nftables on the Pi. -- The camera additionally requires the bearer token. -- Traffic is unencrypted on your LAN. If that matters to you, put both machines on Tailscale and - use the Pi's tailnet address in both configs. -- If your Omarchy machine's IP changes, re-run `install.sh` with the new one. +- The robot's ports are open **only over the USB link, only to `10.55.0.1`**: audio (4713), MCP (8765) + and a reserved webhook port (8766). Everything else of those is dropped on Wi-Fi. +- **ROS is blocked on every interface**, including USB. It has no authentication and carries the camera + and recognised names. +- MCP needs a bearer token, kept in `/etc/robot-mcp.env` (root and the service user only). +- **SSH is open on all interfaces** (key-only), by choice, for the mobile version. +- Face recognition is opt-in and stores only numbers. Hermes keeps each `look()` photo in its image + cache on your computer. +- Never power the robot from a device whose battery you care about. ## Files -``` -pi/ - install.sh one-shot Pi setup - robot_mcp_server.py MCP server: look, turn_head, head_position - requirements.txt MCP Python SDK 2.x - robot-mcp.service systemd unit - nftables-robot.nft port restriction - pipewire/50-network-server.conf exposes audio to the network - pipewire/60-echo-cancel.conf.optional -omarchy/ - 50-robot-tunnel.conf robot audio as local devices - hermes-config-snippet.yaml MCP and voice settings for Hermes - hermes-robot launcher that swaps default audio devices -``` +| Path | What | +|---|---| +| `ros/` | The ROS 2 workspace: `hermes_face`, `hermes_vision`, `hermes_audio`, `hermes_mcp`, `hermes_bringup`. | +| `pi/` | Everything installed on the Pi: `install.sh` and its parts, services, firewall, overlays. | +| `host/` | Your computer's side: the USB link script, Hermes and PipeWire config, a platform check. | +| `enclosure/` | OpenSCAD model, STLs and the print guide. Built from estimates: measure before printing. | +| `docs/face-prototype.html` | The browser prototype the face was ported from. | +| `HANDOFF.md` | The original implementation brief. `CONTEXT.md` tracks where things really stand. | diff --git a/omarchy/50-robot-tunnel.conf b/host/omarchy/50-robot-tunnel.conf similarity index 100% rename from omarchy/50-robot-tunnel.conf rename to host/omarchy/50-robot-tunnel.conf diff --git a/omarchy/hermes-config-snippet.yaml b/host/omarchy/hermes-config-snippet.yaml similarity index 100% rename from omarchy/hermes-config-snippet.yaml rename to host/omarchy/hermes-config-snippet.yaml diff --git a/omarchy/hermes-robot b/host/omarchy/hermes-robot similarity index 88% rename from omarchy/hermes-robot rename to host/omarchy/hermes-robot index 53ac533..d5c454b 100755 --- a/omarchy/hermes-robot +++ b/host/omarchy/hermes-robot @@ -12,7 +12,7 @@ SOURCE=robot_mic has() { pactl list short "$1" | awk '{print $2}' | grep -qxF "$2"; } if ! has sinks "$SINK" || ! has sources "$SOURCE"; then echo "hermes-robot: the robot's audio devices aren't present." >&2 - echo " Is the robot powered and on Wi-Fi? Check with: pactl list short sources" >&2 + echo " Is the robot powered and plugged in over USB (ping 10.55.0.2)? Check with: pactl list short sources" >&2 exit 1 fi diff --git a/pi/install.sh b/pi/install.sh index a8316e8..71a7882 100755 --- a/pi/install.sh +++ b/pi/install.sh @@ -1,96 +1,165 @@ -#!/usr/bin/env bash -# Robot setup for Raspberry Pi OS Lite (64-bit) on a Pi Zero 2 W. -# Run as your normal user (not root) from inside this "pi" folder: +#!/bin/bash +# Set up the robot's Raspberry Pi 4 from a fresh Ubuntu Server 24.04 (arm64) image. # -# ./install.sh [--servos] +# rsync -a ./ hermes@:~/hermes-bot/ +# ssh hermes@ 'cd hermes-bot && sudo ./pi/install.sh' # -# Safe to re-run. Reboot afterwards so the I2S sound card overlay takes effect. +# Re-runnable: every step checks before it changes anything, and the heavy builds skip themselves if +# they are already done. Needs the internet (apt, git, pip, model downloads). The first run takes a +# while (the libcamera build alone is about 15 minutes). Reboot afterwards: it changes /boot/firmware/ +# config.txt for the USB gadget and the audio codec. +# +# The user (default "hermes", set ROBOT_USER) must already exist: create it in cloud-init user-data. +# Order matters: the firewall goes in before PipeWire's network server, which listens on every +# interface and is protected only by that firewall. set -euo pipefail -HOST_IP="${1:-}" -SERVOS=0 -[[ "${2:-}" == "--servos" ]] && SERVOS=1 +ROBOT_USER="${ROBOT_USER:-hermes}" +PI_DIR="$(cd "$(dirname "$0")" && pwd)" +REPO="$(dirname "$PI_DIR")" +# Raspberry Pi's fork of libcamera, at a commit known to capture on this kernel. Upstream libcamera, +# Ubuntu's 0.2.0 and ROS's 0.7.2 all crash in the Pi image-processing plugin when capturing. +LIBCAMERA_REF=6c1dd9d55573010f710c9e190a73e7e76f0d9432 +CAMERA_ROS_TAG=0.7.0 +MODEL_BASE=https://github.com/opencv/opencv_zoo/raw/main/models # a plain raw URL returns a Git LFS pointer -if [[ -z "$HOST_IP" || ! "$HOST_IP" =~ ^[0-9]{1,3}(\.[0-9]{1,3}){3}$ ]]; then - echo "usage: $0 [--servos]" >&2 - exit 1 -fi -if [[ $EUID -eq 0 ]]; then - echo "Run this as your normal user; it calls sudo where needed." >&2 - exit 1 +export DEBIAN_FRONTEND=noninteractive NEEDRESTART_MODE=a +REBOOT=0 + +[ "$(id -u)" = 0 ] || { echo "run with sudo" >&2; exit 1; } +HOME_DIR="$(getent passwd "$ROBOT_USER" | cut -d: -f6)" +[ -n "$HOME_DIR" ] || { echo "no user '$ROBOT_USER': create it in cloud-init user-data first" >&2; exit 1; } +grep -qs "Ubuntu 24.04" /etc/os-release || echo "warning: this is written for Ubuntu 24.04" >&2 + +step() { printf '\n==> %s\n' "$*"; } +as_user() { sudo -u "$ROBOT_USER" -H "$@"; } +cfg_sum() { sha256sum /boot/firmware/config.txt | cut -d' ' -f1; } +USER_UID="$(id -u "$ROBOT_USER")" + +step "Base packages" +apt-get update -q +apt-get -y -q install git curl rsync software-properties-common openssl rfkill device-tree-compiler i2c-tools \ + alsa-utils python3-numpy python3-pil python3-spidev python3-lgpio python3-rpi-lgpio python3-pip \ + python3-venv pipewire pipewire-pulse wireplumber libspa-0.2-modules pulseaudio-utils \ + meson ninja-build pkg-config python3-yaml python3-ply python3-jinja2 libyaml-dev libgnutls28-dev \ + libssl-dev libudev-dev libevent-dev libdrm-dev libjpeg-dev libpng-dev libtiff-dev libboost-dev v4l-utils + +step "Groups for hardware access" +for g in video audio i2c spi gpio dialout plugdev; do + getent group "$g" >/dev/null && usermod -aG "$g" "$ROBOT_USER" || echo " (no group $g on this image)" +done + +step "ROS 2 Jazzy" +if ! dpkg -s ros-jazzy-ros-base >/dev/null 2>&1; then + add-apt-repository -y universe + . /etc/os-release + V="$(curl -s https://api.github.com/repos/ros-infrastructure/ros-apt-source/releases/latest | grep -F tag_name | awk -F\" '{print $4}')" + curl -fL -o /tmp/ros2-apt-source.deb \ + "https://github.com/ros-infrastructure/ros-apt-source/releases/download/${V}/ros2-apt-source_${V}.${UBUNTU_CODENAME:-$VERSION_CODENAME}_all.deb" + dpkg -i /tmp/ros2-apt-source.deb + apt-get update -q fi +# ros-jazzy-camera-ros is installed for its dependencies; the camera driver actually used is built below +apt-get -y -q install ros-jazzy-ros-base ros-dev-tools python3-colcon-common-extensions \ + ros-jazzy-robot-state-publisher ros-jazzy-xacro ros-jazzy-camera-ros ros-jazzy-vision-msgs -HERE="$(cd "$(dirname "$0")" && pwd)" -ME="$(id -un)" -CONFIG_TXT=/boot/firmware/config.txt -[[ -f $CONFIG_TXT ]] || CONFIG_TXT=/boot/config.txt - -echo "==> Installing packages" -sudo apt-get update -sudo apt-get install -y --no-install-recommends \ - python3-picamera2 python3-venv \ - pipewire pipewire-pulse pipewire-alsa wireplumber libspa-0.2-modules \ - pulseaudio-utils alsa-utils nftables openssl - -echo "==> Enabling the I2S microphone/amplifier overlay in $CONFIG_TXT" -add_line() { grep -qxF "$1" "$CONFIG_TXT" || echo "$1" | sudo tee -a "$CONFIG_TXT" >/dev/null; } -add_line "dtparam=i2s=on" -add_line "dtoverlay=googlevoicehat-soundcard" -if [[ $SERVOS -eq 1 ]]; then - add_line "dtparam=i2c_arm=on" -fi - -echo "==> Installing the MCP server to /opt/robot-mcp" -sudo mkdir -p /opt/robot-mcp -sudo chown "$ME:$ME" /opt/robot-mcp -cp "$HERE/robot_mcp_server.py" /opt/robot-mcp/ -# --system-site-packages so the venv can see the apt-installed picamera2/libcamera -[[ -d /opt/robot-mcp/venv ]] || python3 -m venv --system-site-packages /opt/robot-mcp/venv -/opt/robot-mcp/venv/bin/pip install --upgrade pip >/dev/null -/opt/robot-mcp/venv/bin/pip install -r "$HERE/requirements.txt" -if [[ $SERVOS -eq 1 ]]; then - /opt/robot-mcp/venv/bin/pip install adafruit-circuitpython-servokit -fi - -echo "==> Writing /etc/robot-mcp.env" -if ! sudo test -f /etc/robot-mcp.env; then - TOKEN="$(openssl rand -hex 32)" - sudo tee /etc/robot-mcp.env >/dev/null < /opt/libcamera-rpi" +if [ -x /opt/libcamera-rpi/bin/cam ]; then + echo " already built" else - sudo sed -i "s/^ROBOT_SERVOS=.*/ROBOT_SERVOS=$SERVOS/" /etc/robot-mcp.env + rm -rf /var/tmp/libcamera-rpi + git clone -q --no-checkout https://github.com/raspberrypi/libcamera.git /var/tmp/libcamera-rpi + git -C /var/tmp/libcamera-rpi fetch -q --depth 1 origin "$LIBCAMERA_REF" + git -C /var/tmp/libcamera-rpi checkout -q FETCH_HEAD + (cd /var/tmp/libcamera-rpi \ + && meson setup build --buildtype=release -Dprefix=/opt/libcamera-rpi -Dpipelines=rpi/vc4 -Dipas=rpi/vc4 \ + -Dcam=enabled -Dqcam=disabled -Dgstreamer=disabled -Dpycamera=disabled -Dlc-compliance=disabled \ + -Dtest=false -Ddocumentation=disabled -Dv4l2=false \ + && ninja -C build -j4 && ninja -C build install) fi -echo "==> Installing the systemd service" -sed "s/__USER__/$ME/g" "$HERE/robot-mcp.service" | sudo tee /etc/systemd/system/robot-mcp.service >/dev/null -getent group i2c >/dev/null || sudo groupadd --system i2c -sudo systemctl daemon-reload -sudo systemctl enable robot-mcp.service +step "Python environment (~/venv: display, OpenCV 4.10 for YuNet, MCP SDK)" +# OpenCV's DNN needs 4.8+ for the YuNet model and apt has 4.6; numpy stays below 2 because the ROS +# Python packages are built against 1.x. +[ -d "$HOME_DIR/venv" ] || as_user python3 -m venv --system-site-packages "$HOME_DIR/venv" +as_user "$HOME_DIR/venv/bin/pip" install -q adafruit-circuitpython-rgb-display adafruit-blinka \ + "opencv-python-headless==4.10.0.84" "numpy<2" "mcp>=2.3,<3" -echo "==> Configuring PipeWire to serve audio to $HOST_IP" -mkdir -p ~/.config/pipewire/pipewire-pulse.conf.d -cp "$HERE/pipewire/50-network-server.conf" ~/.config/pipewire/pipewire-pulse.conf.d/ -# Run the audio stack at boot without anyone logging in. -sudo loginctl enable-linger "$ME" -systemctl --user enable pipewire.socket pipewire-pulse.socket wireplumber.service 2>/dev/null || true +step "Face models (~/models)" +as_user mkdir -p "$HOME_DIR/models" +fetch_model() { # name, path in the zoo + local dest="$HOME_DIR/models/$1" + if [ "$(stat -c %s "$dest" 2>/dev/null || echo 0)" -lt 100000 ]; then + as_user curl -fsSL -o "$dest" "$MODEL_BASE/$2/$1" + fi + [ "$(stat -c %s "$dest")" -gt 100000 ] || { echo " $1 looks like an LFS pointer, not the model" >&2; exit 1; } +} +fetch_model face_detection_yunet_2023mar.onnx face_detection_yunet +fetch_model face_recognition_sface_2021dec.onnx face_recognition_sface -echo "==> Restricting ports 4713 and 8765 to $HOST_IP" -sed "s/__HOST_IP__/$HOST_IP/g" "$HERE/nftables-robot.nft" | sudo tee /etc/nftables-robot.nft >/dev/null -grep -qxF 'include "/etc/nftables-robot.nft"' /etc/nftables.conf \ - || echo 'include "/etc/nftables-robot.nft"' | sudo tee -a /etc/nftables.conf >/dev/null -sudo systemctl enable nftables.service -sudo systemctl restart nftables.service +step "camera_ros $CAMERA_ROS_TAG, built against the libcamera fork (~/camera_ws)" +if [ -d "$HOME_DIR/camera_ws/install/camera_ros" ]; then + echo " already built" +else + as_user mkdir -p "$HOME_DIR/camera_ws/src" + [ -d "$HOME_DIR/camera_ws/src/camera_ros" ] || as_user git clone -q --depth 1 --branch "$CAMERA_ROS_TAG" \ + https://github.com/christianrauch/camera_ros.git "$HOME_DIR/camera_ws/src/camera_ros" + as_user bash -c "source /opt/ros/jazzy/setup.bash && export PKG_CONFIG_PATH=/opt/libcamera-rpi/lib/aarch64-linux-gnu/pkgconfig \ + && cd '$HOME_DIR/camera_ws' && colcon build --symlink-install --cmake-args -DCMAKE_BUILD_TYPE=Release" +fi -echo -echo "Done. Reboot now: sudo reboot" -echo -echo "Your MCP token (put this in ~/.hermes/.env on the Omarchy machine as ROBOT_MCP_TOKEN):" -sudo grep '^ROBOT_MCP_TOKEN=' /etc/robot-mcp.env | cut -d= -f2 +step "Environment script and the robot's ROS workspace (~/ros_ws)" +install -o "$ROBOT_USER" -g "$ROBOT_USER" -m 644 "$PI_DIR/robot-env.sh" "$HOME_DIR/robot-env.sh" +as_user mkdir -p "$HOME_DIR/ros_ws/src" +rsync -a --delete --exclude __pycache__ "$REPO/ros/" "$HOME_DIR/ros_ws/src/" +chown -R "$ROBOT_USER:$ROBOT_USER" "$HOME_DIR/ros_ws/src" +as_user bash -c "source /opt/ros/jazzy/setup.bash && source '$HOME_DIR/camera_ws/install/setup.bash' \ + && cd '$HOME_DIR/ros_ws' && colcon build --symlink-install" + +step "USB gadget (NCM Ethernet + ACM serial) and the audio codec overlay" +before="$(cfg_sum)" +bash "$PI_DIR/install-usb-gadget.sh" +bash "$PI_DIR/install-audio-overlay.sh" +[ "$before" = "$(cfg_sum)" ] || REBOOT=1 + +step "Firewall (before anything listens on the network)" +bash "$PI_DIR/install-firewall.sh" + +step "PipeWire for $ROBOT_USER (audio server, reachable only over the USB link)" +loginctl enable-linger "$ROBOT_USER" +as_user mkdir -p "$HOME_DIR/.config/pipewire/pipewire-pulse.conf.d" +install -o "$ROBOT_USER" -g "$ROBOT_USER" -m 644 "$PI_DIR/pipewire/50-network-server.conf" \ + "$HOME_DIR/.config/pipewire/pipewire-pulse.conf.d/50-network-server.conf" +for _ in $(seq 20); do [ -S "/run/user/$USER_UID/bus" ] && break; sleep 1; done +as_user env XDG_RUNTIME_DIR="/run/user/$USER_UID" systemctl --user enable --now \ + pipewire.socket pipewire-pulse.socket wireplumber.service \ + || echo " (the user session is not up yet; PipeWire starts at the next boot)" + +step "Audio codec mixer and microphone gain" +if aplay -l 2>/dev/null | grep -q wm8960; then + alsactl -f "$PI_DIR/wm8960_asound.state" restore wm8960soundcard && alsactl store wm8960soundcard + # WirePlumber resets the codec's capture gain to +30 dB when it takes the device; 63% on PipeWire's + # cubic scale is -12 dB from that, about +17 dB. + as_user env XDG_RUNTIME_DIR="/run/user/$USER_UID" pactl set-source-volume @DEFAULT_SOURCE@ 63% \ + || echo " (could not set the source volume: run pactl set-source-volume @DEFAULT_SOURCE@ 63% later)" +else + echo " no WM8960 card yet: reboot, then run this script again" + REBOOT=1 +fi + +step "Services: the robot (face, camera, vision, audio, MCP bridge), the CPU governor, trimmed boot" +bash "$PI_DIR/install-robot-service.sh" +bash "$PI_DIR/trim-boot.sh" + +cat <=2.3,<3 diff --git a/pi/robot-mcp.service b/pi/robot-mcp.service deleted file mode 100644 index d1e0124..0000000 --- a/pi/robot-mcp.service +++ /dev/null @@ -1,17 +0,0 @@ -[Unit] -Description=Robot MCP server (camera and head for Hermes Agent) -Wants=network-online.target -After=network-online.target - -[Service] -Type=simple -User=__USER__ -Group=__USER__ -SupplementaryGroups=video i2c -EnvironmentFile=/etc/robot-mcp.env -ExecStart=/opt/robot-mcp/venv/bin/python /opt/robot-mcp/robot_mcp_server.py -Restart=on-failure -RestartSec=3 - -[Install] -WantedBy=multi-user.target diff --git a/pi/robot_mcp_server.py b/pi/robot_mcp_server.py deleted file mode 100644 index ccd7ce0..0000000 --- a/pi/robot_mcp_server.py +++ /dev/null @@ -1,229 +0,0 @@ -#!/usr/bin/env python3 -""" -Robot MCP server: runs on the Raspberry Pi and gives Hermes Agent the robot's eyes -(and, optionally, a neck). - -Tools exposed - look() -> a JPEG still from the camera - turn_head(pan, tilt) -> aim the pan/tilt servos (only if ROBOT_SERVOS=1) - head_position() -> current pan/tilt (only if ROBOT_SERVOS=1) - -Transport: MCP streamable HTTP at http://:/mcp, guarded by a bearer token. -Audio is NOT handled here; it travels over PipeWire (see README). - -Configuration is by environment variable (normally set in /etc/robot-mcp.env): - ROBOT_MCP_TOKEN required shared secret; Hermes sends "Authorization: Bearer " - ROBOT_MCP_PORT 8765 - ROBOT_MCP_BIND 0.0.0.0 - ROBOT_CAM_SIZE 1280x720 still resolution handed to the model - ROBOT_CAM_QUALITY 85 JPEG quality - ROBOT_CAM_HFLIP 0 set 1 if the camera is mounted mirrored - ROBOT_CAM_VFLIP 0 set 1 (with HFLIP=1) if mounted upside down - ROBOT_SERVOS 0 set 1 to enable turn_head via a PCA9685 board - ROBOT_PAN_CHANNEL 0 - ROBOT_TILT_CHANNEL 1 - ROBOT_PAN_LIMIT 80 degrees either side of centre - ROBOT_TILT_LIMIT 45 - ROBOT_PAN_INVERT 0 set 1 if "positive = left" comes out backwards - ROBOT_TILT_INVERT 0 -""" - -from __future__ import annotations - -import hmac -import io -import logging -import os -import sys -import threading -import time - -import uvicorn -from mcp.server.mcpserver import Image, MCPServer -from mcp.server.transport_security import TransportSecuritySettings - -log = logging.getLogger("robot-mcp") -logging.basicConfig(level=logging.INFO, format="%(asctime)s %(levelname)s %(name)s: %(message)s") - - -def env_flag(name: str, default: str = "0") -> bool: - return os.environ.get(name, default).strip().lower() in ("1", "true", "yes", "on") - - -def env_int(name: str, default: int) -> int: - return int(os.environ.get(name, str(default))) - - -TOKEN = os.environ.get("ROBOT_MCP_TOKEN", "").strip() -PORT = env_int("ROBOT_MCP_PORT", 8765) -BIND = os.environ.get("ROBOT_MCP_BIND", "0.0.0.0") -CAM_W, CAM_H = (int(v) for v in os.environ.get("ROBOT_CAM_SIZE", "1280x720").lower().split("x")) -CAM_QUALITY = env_int("ROBOT_CAM_QUALITY", 85) - - -# --------------------------------------------------------------------------- camera - -class Camera: - """Keeps Picamera2 running so a still is ready in well under a second.""" - - def __init__(self) -> None: - from libcamera import Transform # type: ignore[import-not-found] - from picamera2 import Picamera2 # type: ignore[import-not-found] - - self._lock = threading.Lock() - self.cam = Picamera2() - config = self.cam.create_still_configuration( - main={"size": (CAM_W, CAM_H)}, - transform=Transform(hflip=env_flag("ROBOT_CAM_HFLIP"), vflip=env_flag("ROBOT_CAM_VFLIP")), - buffer_count=2, - ) - self.cam.configure(config) - self.cam.options["quality"] = CAM_QUALITY - self.has_autofocus = "AfMode" in self.cam.camera_controls - self.cam.start() - time.sleep(1.0) # let auto-exposure settle once at startup - log.info("camera ready at %dx%d (autofocus: %s)", CAM_W, CAM_H, self.has_autofocus) - - def snapshot(self) -> bytes: - with self._lock: - if self.has_autofocus: - # Camera Module 3: run one focus sweep so whatever is in front is sharp. - self.cam.autofocus_cycle() - buf = io.BytesIO() - self.cam.capture_file(buf, format="jpeg") - return buf.getvalue() - - -# --------------------------------------------------------------------------- servos - -class Head: - """Pan/tilt on a PCA9685 servo board. Angles are degrees from centre.""" - - def __init__(self) -> None: - from adafruit_servokit import ServoKit # type: ignore[import-not-found] - - self._lock = threading.Lock() - self.kit = ServoKit(channels=16) - self.pan_ch = env_int("ROBOT_PAN_CHANNEL", 0) - self.tilt_ch = env_int("ROBOT_TILT_CHANNEL", 1) - self.pan_limit = env_int("ROBOT_PAN_LIMIT", 80) - self.tilt_limit = env_int("ROBOT_TILT_LIMIT", 45) - self.pan_sign = -1 if env_flag("ROBOT_PAN_INVERT") else 1 - self.tilt_sign = -1 if env_flag("ROBOT_TILT_INVERT") else 1 - self.pan = 0.0 - self.tilt = 0.0 - self.move(0.0, 0.0) - log.info("servos ready (pan ch %d, tilt ch %d)", self.pan_ch, self.tilt_ch) - - def move(self, pan: float, tilt: float) -> tuple[float, float]: - pan = max(-self.pan_limit, min(self.pan_limit, pan)) - tilt = max(-self.tilt_limit, min(self.tilt_limit, tilt)) - with self._lock: - self.kit.servo[self.pan_ch].angle = 90 + self.pan_sign * pan - self.kit.servo[self.tilt_ch].angle = 90 + self.tilt_sign * tilt - self.pan, self.tilt = pan, tilt - time.sleep(0.4) # let the servos arrive and stop jittering before any photo - return pan, tilt - - -# --------------------------------------------------------------------------- MCP server - -mcp = MCPServer( - "robot", - instructions=( - "This server is a small physical robot on the user's desk. " - "Call look() whenever you need to see what is in front of it; the image is current. " - "If turn_head is available, aim first and then look." - ), -) - -camera: Camera | None = None -head: Head | None = None - - -@mcp.tool() -def look() -> Image: - """Take a photograph with the robot's camera and return it. Use this to see the room, - the desk, the person, or anything the user asks you to look at.""" - if camera is None: - raise RuntimeError("camera is not available on the robot") - t0 = time.monotonic() - data = camera.snapshot() - log.info("look(): %d bytes in %.2fs", len(data), time.monotonic() - t0) - return Image(data=data, format="jpeg") - - -def register_head_tools() -> None: - @mcp.tool() - def turn_head(pan: float = 0.0, tilt: float = 0.0) -> str: - """Aim the robot's head (and so its camera). Angles are in degrees from straight ahead. - pan: positive turns to the robot's left, negative to its right. - tilt: positive looks up, negative looks down. - Values beyond the mechanical limits are clamped. Call look() afterwards to see.""" - assert head is not None - p, t = head.move(pan, tilt) - return f"head now at pan {p:+.0f} degrees, tilt {t:+.0f} degrees" - - @mcp.tool() - def head_position() -> str: - """Report where the robot's head is currently pointing.""" - assert head is not None - return f"pan {head.pan:+.0f} degrees, tilt {head.tilt:+.0f} degrees" - - -# --------------------------------------------------------------------------- auth wrapper - -class BearerAuth: - """Minimal ASGI middleware: reject any HTTP request without the right bearer token. - Lifespan events pass straight through so the MCP session manager starts properly.""" - - def __init__(self, app, token: str) -> None: - self.app = app - self.expected = f"Bearer {token}".encode() - - async def __call__(self, scope, receive, send): - if scope["type"] == "http": - supplied = dict(scope.get("headers") or []).get(b"authorization", b"") - if not hmac.compare_digest(supplied, self.expected): - await send({"type": "http.response.start", "status": 401, - "headers": [(b"content-type", b"text/plain"), - (b"www-authenticate", b"Bearer")]}) - await send({"type": "http.response.body", "body": b"unauthorized\n"}) - return - await self.app(scope, receive, send) - - -def build_app(): - app = mcp.streamable_http_app( - host=BIND, - # Bearer auth already stops drive-by requests; DNS-rebinding checks would only - # get in the way of reaching the Pi by IP or by robot.local. - transport_security=TransportSecuritySettings(enable_dns_rebinding_protection=False), - ) - return BearerAuth(app, TOKEN) - - -def main() -> None: - global camera, head - - if len(TOKEN) < 16: - sys.exit("ROBOT_MCP_TOKEN is missing or too short (generate one with: openssl rand -hex 32)") - - try: - camera = Camera() - except Exception as exc: # keep serving servos even if the camera ribbon is loose - log.error("camera failed to start: %s", exc) - - if env_flag("ROBOT_SERVOS"): - try: - head = Head() - register_head_tools() - except Exception as exc: - log.error("servos failed to start, turn_head disabled: %s", exc) - - log.info("serving MCP on http://%s:%d/mcp", BIND, PORT) - uvicorn.run(build_app(), host=BIND, port=PORT, log_level="warning") - - -if __name__ == "__main__": - main() diff --git a/tools/check-pi-platform.sh b/tools/check-pi-platform.sh deleted file mode 100644 index 2a86198..0000000 --- a/tools/check-pi-platform.sh +++ /dev/null @@ -1,109 +0,0 @@ -#!/usr/bin/env bash -# check-pi-platform.sh: confirm, without the Pi, that Ubuntu 24.04's Raspberry Pi kernel ships the -# drivers and overlays the robot needs. It downloads the current linux-modules-*-raspi package for -# arm64 (about 50 MB) and inspects it. Nothing is installed; everything happens in a temp folder. -# -# Needs: curl, gzip, awk, ar (binutils), zstd, tar -# NixOS: nix-shell -p curl binutils zstd gnutar gawk --run ./check-pi-platform.sh -# Omarchy: sudo pacman -S --needed binutils zstd && ./check-pi-platform.sh -# -# Usage: ./check-pi-platform.sh [suite] (default suite: noble-updates) -# MIRROR= overrides http://ports.ubuntu.com/ubuntu-ports -set -euo pipefail - -SUITE="${1:-noble-updates}" -MIRROR="${MIRROR:-http://ports.ubuntu.com/ubuntu-ports}" - -for tool in curl gzip awk ar zstd tar; do - command -v "$tool" >/dev/null || { echo "Missing tool: $tool (see the header of this script)" >&2; exit 2; } -done - -work="$(mktemp -d)" -trap 'rm -rf "$work"' EXIT -cd "$work" - -echo "==> Reading the arm64 package index for $SUITE" -curl -fsSL "$MIRROR/dists/$SUITE/main/binary-arm64/Packages.gz" | gzip -dc > Packages - -# field : the value of Field in that package's stanza (last stanza wins) -field() { - awk -v p="$1" -v f="$2" ' - /^Package: / { inpkg = ($2 == p) } - inpkg && index($0, f ": ") == 1 { v = substr($0, length(f) + 3) } - END { if (v != "") print v }' Packages -} - -meta_deps="$(field linux-image-raspi Depends)" -[[ -n "$meta_deps" ]] || { echo "linux-image-raspi not found in $SUITE" >&2; exit 1; } -abi="$(grep -oE 'linux-image-[0-9][^ ,(]*-raspi' <<<"$meta_deps" | head -1 | sed 's/^linux-image-//')" -modpkg="linux-modules-$abi" -debpath="$(field "$modpkg" Filename)" -[[ -n "$debpath" ]] || { echo "$modpkg not found in $SUITE" >&2; exit 1; } -echo "==> Current Pi kernel for Ubuntu 24.04: $abi" -echo "==> Downloading $modpkg" -curl -fsSL "$MIRROR/$debpath" -o modules.deb - -member="$(ar t modules.deb | grep '^data\.tar' | head -1)" -case "$member" in - *.zst) decomp=(zstd -dc) ;; - *.xz) decomp=(xz -dc) ;; - *.gz) decomp=(gzip -dc) ;; - *.tar) decomp=(cat) ;; - *) echo "Unexpected package layout: $member" >&2; exit 1 ;; -esac -ar p modules.deb "$member" | "${decomp[@]}" | tar -t > files -ar p modules.deb "$member" | "${decomp[@]}" | tar -xO "./boot/config-$abi" > config 2>/dev/null || : > config - -missing=0 -# check