Add hermes_audio and draw the mouth from the real waveform
audio_state taps the microphone and the speaker monitor with parec and publishes levels, a speaking/listening/quiet activity, and trigger-locked waveform windows; audio itself stays out of ROS. The activity detector calibrates its noise floor at start-up and absorbs steady noise, so a room at -43 dBFS does not read as "listening" forever, and it ignores the mic while the robot speaks. The face arbitrates the state it shows: sleeping (agent side only, so room sound never wakes a face that means "no agent connected") > speaking > listening > thinking > idle, and draws the mouth from the real samples while fresh scope data exists, auto-scaled so a quiet voice is still visible. Checked: a tone played to the sink shows as `speaking` with the mic ignored. The microphone-to-mouth path with a live voice is not yet confirmed. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AFN8nRRCiDiMN7ChBpHJhs
This commit is contained in:
+24
@@ -151,6 +151,30 @@ printed enclosure, speakers. Work until those arrive is limited to what the Pi,
|
||||
- Open: IPS black level. The backlight is on at full brightness, so black reads dark gray on
|
||||
camera. Dimming it needs a software PWM on GPIO26 (not a hardware PWM pin).
|
||||
|
||||
## Audio levels and the literal voice trace (`ros/hermes_audio`, face integration)
|
||||
|
||||
- `hermes_audio` node `audio_state` taps the default source and the default sink's monitor with
|
||||
`parec` (16 kHz mono) and publishes `/audio/levels` ([mic_dbfs, speaker_dbfs]), `/audio/activity`
|
||||
(`speaking` | `listening` | `quiet`) and trigger-locked waveform windows `/audio/scope/{mic,speaker}`
|
||||
(256 samples). Audio itself never goes through ROS. `hermes_audio.levels.Activity` debounces and
|
||||
keeps an adaptive noise floor: a 3 s start-up calibration, then it drops fast and rises slowly, and
|
||||
anything 'listening' without a gap for 8 s is absorbed as ambient noise. The microphone is ignored
|
||||
while speaking (it would hear the robot).
|
||||
- The face arbitrates (`hermes_face.arbiter`): sleeping (agent side only) > speaking > listening >
|
||||
thinking > idle. **Room sound never wakes a sleeping face**: sleeping means "no agent connected".
|
||||
`looking` stays a timed event owned by the renderer. While fresh (< 0.35 s) scope data exists, the
|
||||
mouth draws the real waveform (speaker when speaking, mic when listening), auto-scaled.
|
||||
- **WirePlumber resets the codec's capture gain** to 100% (+30 dB) when it takes the device,
|
||||
overriding `alsactl`'s saved level. Fix: `pactl set-source-volume @DEFAULT_SOURCE@ 63%` (PipeWire's
|
||||
scale is cubic: 63% is -12 dB from the +30 dB maximum, so ALSA `Capture` lands at 46, about +17
|
||||
dB). WirePlumber remembers it; recheck after a reboot. The quiet-room floor at this gain is about
|
||||
-43 dBFS (preamp and mic noise).
|
||||
- Verified on the Pi: the `speaking` path (a tone played to the sink, with no speaker attached, shows
|
||||
on the monitor at -13.5 dBFS, `speaking` for its length, mic ignored); levels and scope publish;
|
||||
47 face tests, 8 audio tests. **Not yet verified:** the microphone-to-mouth path with a real voice
|
||||
(one live attempt saw no speech: the cue and the talking did not overlap), and the audio node as a
|
||||
service (it was run as a transient unit).
|
||||
|
||||
## Face recognition (opt-in)
|
||||
|
||||
`hermes_vision.recognize` / `people`: SFace embeddings on YuNet's landmarks. Nothing is stored until
|
||||
|
||||
Reference in New Issue
Block a user