Block all DDS off the Pi and bring up PipeWire behind the firewall
ROS never leaves the robot, by the owner's decision: the firewall drops domain 42's DDS ports on every interface but loopback (usb0 included), discovery is pinned to LOCALHOST, and hermes_mcp is the only bridge to the outside. PipeWire 1.0.5 runs as hermes's user services with its network server on TCP 4713; the firewall, loaded at boot before it, is the protection. Verified from talos: 4713 connects over usb0 and times out over Wi-Fi. CONTEXT.md brought up to date: the face service, gaze, recognition, the firewall, and what is left. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AFN8nRRCiDiMN7ChBpHJhs
This commit is contained in:
Claude Sonnet 5.5
+47
-6
@@ -25,8 +25,22 @@ printed enclosure, speakers. Work until those arrive is limited to what the Pi,
|
||||
visible) with `adafruit-circuitpython-rgb-display` and Blinka. Workspace: `~/ros_ws`, source synced
|
||||
from `ros/` with `rsync -a --delete --exclude __pycache__ ros/ hermes@<ip>:~/ros_ws/src/`, then
|
||||
`colcon build --symlink-install`.
|
||||
- CPU governor is `ondemand`; at full clock the face costs about half as much CPU. Set
|
||||
`performance` in the installer (package 5). It is not persistent yet.
|
||||
- CPU governor is held at `performance` by `robot-cpu-performance.service` (the face costs about
|
||||
half as much CPU at full clock). The face starts at boot: `robot-face.service` runs
|
||||
`ros2 run hermes_face face_node` as `hermes` and restarts it on failure (it needs a writable
|
||||
working directory, `RuntimeDirectory=robot-face`, because lgpio drops a pipe in its cwd). It comes
|
||||
up in the `sleeping` pose. Install with `pi/install-face-service.sh`.
|
||||
- **ROS never leaves the Pi** (owner's decision): `ROS_AUTOMATIC_DISCOVERY_RANGE=LOCALHOST` (set in
|
||||
`pi/robot-env.sh`, which `source`s everything) and the firewall drops domain 42's DDS ports
|
||||
(17900-18149) on every interface but loopback. `hermes_mcp` is the only bridge to the outside; a
|
||||
host that wants data from the robot gets it through an MCP tool, not DDS. This supersedes
|
||||
HANDOFF.md's `host/ros-shell` talking to the robot. (ROS's setup script exports `SUBNET`, so the
|
||||
variable must be assigned, not defaulted.)
|
||||
- **PipeWire** (1.0.5, WirePlumber 0.4.17) runs as `hermes`'s user services (`loginctl enable-linger`).
|
||||
`~/.config/pipewire/pipewire-pulse.conf.d/50-network-server.conf` (from `pi/pipewire/`) makes
|
||||
pipewire-pulse listen on TCP 4713 on all interfaces; the firewall is the protection. Defaults are
|
||||
the WM8960 source and sink (`alsa_input/output.platform-soc_sound.stereo-fallback`). Verified from
|
||||
talos: `pactl -s tcp:10.55.0.2:4713 info` works; the same on the Wi-Fi address times out.
|
||||
|
||||
## Package 0a (platform proof on Ubuntu 24.04)
|
||||
|
||||
@@ -92,10 +106,16 @@ printed enclosure, speakers. Work until those arrive is limited to what the Pi,
|
||||
- talos (Omarchy, NetworkManager): `host/omarchy/robot-usb-link.sh` (needs sudo) adds profile
|
||||
`hermes-robot-usb`: `10.55.0.1/30` on the interface with MAC `02:68:65:72:6d:01`, priority 100,
|
||||
never-default. Verified from talos: ping ~2.4 ms, SSH over the link.
|
||||
- **Firewall:** `robot-firewall.service` loads `pi/nftables-robot.nft` before networking at every
|
||||
boot (`pi/install-firewall.sh`; it must be loaded before the PipeWire network server is enabled).
|
||||
It accepts 4713/8765/8766 only on `usb0` from `10.55.0.1`, drops them elsewhere, and drops DDS on
|
||||
every interface but loopback. Verified: from talos, 4713 connects over `usb0` and times out over
|
||||
Wi-Fi, while an unused port on the Pi's Wi-Fi address refuses instantly (so only 4713 is blocked).
|
||||
SSH is deliberately left open on all interfaces (key-only), for V2; the owner's call.
|
||||
- Not yet verified: serial login prompt (talos's `/dev/ttyACM0` is `root:uucp`; use
|
||||
`sudo screen /dev/ttyACM0 115200`). The `hermes` account has no password, so the console shows a
|
||||
login prompt but cannot log in until the owner sets one; no autologin by design. Also not done:
|
||||
the nftables rewrite, the Pi booting with Wi-Fi off, replug and second-host recovery,
|
||||
the Pi booting with Wi-Fi off, replug and second-host recovery,
|
||||
`get_throttled` after a servo sweep on the splitter, the NixOS snippet, and straylight/other-host
|
||||
testing. Currently the Pi is powered from talos's USB port (bench test, no servos).
|
||||
|
||||
@@ -119,19 +139,40 @@ printed enclosure, speakers. Work until those arrive is limited to what the Pi,
|
||||
- Measured on the Pi 4: ~9 ms per frame in isolation. With the CPU at full clock: 24% of a core at
|
||||
20 fps and 40% at 30 fps (target was under 40% at 20 fps or more). Under `ondemand` the same
|
||||
load reads 40% at 20 fps and ~67% at 30 fps. No C++ rewrite was needed.
|
||||
- **Gaze:** the node subscribes to `/vision/faces` (and `/camera/camera_info` for the image size)
|
||||
and the eyes glance toward the best face: a recognised one first, then the biggest. Image x flips
|
||||
on the display (a person on the image's right is on the robot's right, the viewer's left); the
|
||||
glance is smoothed, held 2.5 s, then drifts back to centre. Sleeping ignores it. The pan sign in
|
||||
the renderer was wrong (carried over from the prototype) and is fixed: positive pan = the robot's
|
||||
left = the viewer's right, positive tilt = looks up, as the MCP `turn_head` tool documents.
|
||||
Not yet confirmed by eye on the real display.
|
||||
- Not yet: audio levels for the mouth (`/audio/levels`), `hermes_mcp` publishing states, launch
|
||||
file in `hermes_bringup`, head/lowpoly shapes, DotStar LEDs, button mute.
|
||||
- Open: IPS black level. The backlight is on at full brightness, so black reads dark gray on
|
||||
camera. Dimming it needs a software PWM on GPIO26 (not a hardware PWM pin).
|
||||
|
||||
## Face recognition (opt-in)
|
||||
|
||||
`hermes_vision.recognize` / `people`: SFace embeddings on YuNet's landmarks. Nothing is stored until
|
||||
someone is enrolled (`python -m hermes_vision.people enroll NAME`, live from the camera topic, or
|
||||
`enroll-images`); only 128-number embeddings are kept, in `~/.local/share/hermes/faces.json` (mode
|
||||
0600, directory 0700, written atomically), never images. `list`, `forget NAME` and `check IMAGE`
|
||||
manage and test it. The detector node puts a recognised name in `Detection2D.id` and picks up
|
||||
database changes without a restart; its log prints a count, not names. Checked with a throwaway
|
||||
database: 8 of 9 unenrolled frames of the same person were recognised, down to side profiles
|
||||
(similarity 0.52-0.66), the miss being a head tipped back (0.34 against the 0.363 threshold); an
|
||||
unrelated photo scored 0.16. **Nobody is enrolled yet**; the owner enrols themselves live.
|
||||
|
||||
## Next
|
||||
|
||||
1. Finish package 0b: Wi-Fi-off boot test, replug recovery, serial login check, NixOS snippet in
|
||||
`host/nixos/`, and make the firewall load at boot. A new Omarchy host with a GPU (runs the local
|
||||
`host/nixos/`. A new Omarchy host with a GPU (runs the local
|
||||
model "ornith") arrives this week: it needs only `host/omarchy/robot-usb-link.sh` and its SSH key
|
||||
added to the Pi.
|
||||
2. Audio (package 1; needs the `wm8960-soundcard` overlay and a reboot), camera (package 3), then
|
||||
the rest of the packages in `HANDOFF.md`.
|
||||
2. Audio: the `hermes_audio_state` node (levels, speaking/listening, a trigger-locked scope for the
|
||||
literal voice trace), echo cancel and `robot_mic` naming once speakers arrive. Then a launch file
|
||||
that brings up camera, vision, face and audio together, and CPU tuning (camera ~140% of a core).
|
||||
3. Second-person detection test; enrol the owner live; the rest of `HANDOFF.md`.
|
||||
|
||||
## Repository
|
||||
|
||||
|
||||
@@ -1,9 +1,10 @@
|
||||
#!/usr/sbin/nft -f
|
||||
# The robot's audio (4713), MCP (8765) and face webhook (8766) ports are reachable only over the
|
||||
# USB link, and only from the host at 10.55.0.1. Every other interface, Wi-Fi included, is dropped.
|
||||
# ROS 2's DDS has no authentication, and /camera/* and /vision/faces carry video and recognised
|
||||
# names, so its ports (domain 42: 17900-18149 with Fast DDS's default offsets) are dropped on every
|
||||
# interface but usb0 and loopback too.
|
||||
# ROS 2 never leaves the robot: its DDS has no authentication, and /camera/* and /vision/faces
|
||||
# carry video and recognised names. Domain 42's ports (17900-18149 with Fast DDS's default offsets)
|
||||
# are dropped on every interface but loopback, usb0 included. The only way in or out is hermes_mcp,
|
||||
# which talks to the ROS graph locally and to the host over MCP.
|
||||
# SSH and everything else are left alone: that policy is the owner's call.
|
||||
|
||||
table inet robot
|
||||
@@ -15,6 +16,6 @@ table inet robot {
|
||||
iif "lo" accept
|
||||
iifname "usb0" ip saddr 10.55.0.1 tcp dport { 4713, 8765, 8766 } accept
|
||||
tcp dport { 4713, 8765, 8766 } drop
|
||||
iifname != { "lo", "usb0" } udp dport 17900-18149 drop
|
||||
iifname != "lo" udp dport 17900-18149 drop
|
||||
}
|
||||
}
|
||||
|
||||
@@ -1,11 +1,13 @@
|
||||
# ~/.config/pipewire/pipewire-pulse.conf.d/50-network-server.conf (on the Pi)
|
||||
#
|
||||
# Lets your Omarchy machine reach the robot's microphone and speaker over the
|
||||
# network, using the PulseAudio protocol that pipewire-pulse speaks.
|
||||
# Lets the host reach the robot's microphone and speaker over the USB link, using the
|
||||
# PulseAudio protocol that pipewire-pulse speaks.
|
||||
#
|
||||
# This replaces the default address list, so "unix:native" must stay in it.
|
||||
# Anyone on the LAN who can reach port 4713 could listen to the microphone;
|
||||
# the nftables rules installed alongside restrict it to your machine's IP.
|
||||
# It listens on every interface and is unauthenticated: anyone who can reach port 4713 could
|
||||
# listen to the microphone. The protection is the firewall (pi/nftables-robot.nft, loaded at boot
|
||||
# by robot-firewall.service), which accepts 4713 only on usb0 from 10.55.0.1 and drops it on every
|
||||
# other interface, Wi-Fi included. Do not enable this without that firewall loaded.
|
||||
|
||||
pulse.properties = {
|
||||
server.address = [
|
||||
|
||||
+5
-5
@@ -3,14 +3,14 @@
|
||||
# - the Raspberry Pi libcamera fork ahead of ROS's own copy (same soname, but ROS's crashes in the
|
||||
# Pi image plugin: see CONTEXT.md)
|
||||
# - ~/venv's site-packages first: Blinka/Adafruit for the display, OpenCV 4.10 for YuNet
|
||||
# - a fixed ROS domain (42), discovery on this machine only. DDS has no authentication and the
|
||||
# camera and recognised names travel on it: set ROBOT_ROS_DISCOVERY=SUBNET only to
|
||||
# reach it from a host over the USB link (the firewall drops DDS ports on every other interface)
|
||||
# - a fixed ROS domain (42), discovery on this machine only. ROS never leaves the robot: DDS has
|
||||
# no authentication and carries the camera and recognised names, so the firewall drops its ports
|
||||
# on every interface but loopback. hermes_mcp is the only bridge to the outside world.
|
||||
source /opt/ros/jazzy/setup.bash
|
||||
[ -f "$HOME/camera_ws/install/setup.bash" ] && source "$HOME/camera_ws/install/setup.bash"
|
||||
[ -f "$HOME/ros_ws/install/setup.bash" ] && source "$HOME/ros_ws/install/setup.bash"
|
||||
export LD_LIBRARY_PATH="/opt/libcamera-rpi/lib/aarch64-linux-gnu:${LD_LIBRARY_PATH:-}"
|
||||
export PYTHONPATH="$HOME/venv/lib/python3.12/site-packages:${PYTHONPATH:-}"
|
||||
export ROS_DOMAIN_ID="${ROS_DOMAIN_ID:-42}"
|
||||
# ROS's own setup script exports SUBNET, so a ${VAR:-default} on the ROS variable would never apply
|
||||
export ROS_AUTOMATIC_DISCOVERY_RANGE="${ROBOT_ROS_DISCOVERY:-LOCALHOST}"
|
||||
# ROS's own setup script exports SUBNET, so this must be assigned, not defaulted
|
||||
export ROS_AUTOMATIC_DISCOVERY_RANGE=LOCALHOST
|
||||
|
||||
Reference in New Issue
Block a user