Add hermes_audio and draw the mouth from the real waveform

audio_state taps the microphone and the speaker monitor with parec and publishes
levels, a speaking/listening/quiet activity, and trigger-locked waveform windows;
audio itself stays out of ROS. The activity detector calibrates its noise floor
at start-up and absorbs steady noise, so a room at -43 dBFS does not read as
"listening" forever, and it ignores the mic while the robot speaks.

The face arbitrates the state it shows: sleeping (agent side only, so room sound
never wakes a face that means "no agent connected") > speaking > listening >
thinking > idle, and draws the mouth from the real samples while fresh scope data
exists, auto-scaled so a quiet voice is still visible.

Checked: a tone played to the sink shows as `speaking` with the mic ignored. The
microphone-to-mouth path with a live voice is not yet confirmed.

Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01AFN8nRRCiDiMN7ChBpHJhs
This commit is contained in:
2026-10-04 23:26:01 -07:00
co-authored by Claude Sonnet 5.5
parent 880bdd3f44
commit ac4e09bf35
15 changed files with 534 additions and 4 deletions
+24
View File
@@ -151,6 +151,30 @@ printed enclosure, speakers. Work until those arrive is limited to what the Pi,
- Open: IPS black level. The backlight is on at full brightness, so black reads dark gray on
camera. Dimming it needs a software PWM on GPIO26 (not a hardware PWM pin).
## Audio levels and the literal voice trace (`ros/hermes_audio`, face integration)
- `hermes_audio` node `audio_state` taps the default source and the default sink's monitor with
`parec` (16 kHz mono) and publishes `/audio/levels` ([mic_dbfs, speaker_dbfs]), `/audio/activity`
(`speaking` | `listening` | `quiet`) and trigger-locked waveform windows `/audio/scope/{mic,speaker}`
(256 samples). Audio itself never goes through ROS. `hermes_audio.levels.Activity` debounces and
keeps an adaptive noise floor: a 3 s start-up calibration, then it drops fast and rises slowly, and
anything 'listening' without a gap for 8 s is absorbed as ambient noise. The microphone is ignored
while speaking (it would hear the robot).
- The face arbitrates (`hermes_face.arbiter`): sleeping (agent side only) > speaking > listening >
thinking > idle. **Room sound never wakes a sleeping face**: sleeping means "no agent connected".
`looking` stays a timed event owned by the renderer. While fresh (< 0.35 s) scope data exists, the
mouth draws the real waveform (speaker when speaking, mic when listening), auto-scaled.
- **WirePlumber resets the codec's capture gain** to 100% (+30 dB) when it takes the device,
overriding `alsactl`'s saved level. Fix: `pactl set-source-volume @DEFAULT_SOURCE@ 63%` (PipeWire's
scale is cubic: 63% is -12 dB from the +30 dB maximum, so ALSA `Capture` lands at 46, about +17
dB). WirePlumber remembers it; recheck after a reboot. The quiet-room floor at this gain is about
-43 dBFS (preamp and mic noise).
- Verified on the Pi: the `speaking` path (a tone played to the sink, with no speaker attached, shows
on the monitor at -13.5 dBFS, `speaking` for its length, mic ignored); levels and scope publish;
47 face tests, 8 audio tests. **Not yet verified:** the microphone-to-mouth path with a real voice
(one live attempt saw no speech: the cue and the talking did not overlap), and the audio node as a
service (it was run as a transient unit).
## Face recognition (opt-in)
`hermes_vision.recognize` / `people`: SFace embeddings on YuNet's landmarks. Nothing is stored until