Add a thermal guard that slows the robot before the Pi throttles
hermes_thermal reads the CPU temperature and throttle flags, publishes /health/*, and at 72 / 77 / 82 C scales the camera, face and detector rates down through their live parameters, restoring them as it cools one level at a time with a margin and a dwell so it does not flap. The face and detector gained live parameter callbacks for this, the bridge's robot://status carries a health block (temperature, level, throttle flags now and since boot), and a bringup test asserts the guard's normal rates equal the real ones. Live testing on the robot (forcing 73 / 78 / 85 C and back) caught a bug the unit tests missed: one logging line whose severity changed between warn and info raised in rclpy and killed the guard on its first cooling step, and the respawn hid it. Fixed with separate call sites; the tick now survives any error; a full heat-then-cool test drives the real node, and a mutation check confirmed it fails on the original bug. Co-Authored-By: Claude Sonnet 5.5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01AFN8nRRCiDiMN7ChBpHJhs
This commit is contained in:
+25
@@ -321,6 +321,31 @@ unrelated photo scored 0.16. **Nobody is enrolled yet**; the owner enrols themse
|
||||
4. A new Omarchy host with a GPU (it runs the local `ornith` model) is arriving: it needs only
|
||||
`host/omarchy/robot-usb-link.sh` and its SSH key added to the Pi.
|
||||
|
||||
## Thermal guard (`ros/hermes_thermal`)
|
||||
|
||||
Built because the Pi has no fan (about 60 C open-air with everything running) and an enclosure will be
|
||||
hotter. `thermal_guard` reads the CPU temperature (`/sys/class/thermal`) and `vcgencmd get_throttled`,
|
||||
publishes `/health/temperature`, `/health/thermal_level` (ok | warm | hot | critical) and
|
||||
`/health/throttled`, and sheds load through the target nodes' own live parameters: the camera's
|
||||
`FrameDurationLimits`, the face's `fps` and the detector's `rate_hz` (the face and detector gained live
|
||||
parameter callbacks for this). Levels enter at 72 / 77 / 82 C (the Pi throttles at about 80); heating is
|
||||
acted on at once, cooling goes one level at a time, only below the threshold minus 3 C and after a 20 s
|
||||
dwell. Rates scale by 1.0 / 0.7 / 0.45 / 0.2 with floors (camera 2 fps, face 8, detector 1 Hz). The
|
||||
"normal" rates are parameters of the guard and a bringup test asserts they equal the real ones in
|
||||
`robot.yaml`, so restoring is exact and survives the guard restarting. `fake_temp_c` forces a
|
||||
temperature for testing. `robot://status` now carries a `health` block (temperature, level, throttle
|
||||
flags now and since boot, which doubles as a power-supply check).
|
||||
**Verified live on the robot** by forcing 73 / 78 / 85 C and back: the camera, face and detector rates
|
||||
followed (measured camera 16.5 -> 11.5 -> 7.2 -> 3.5 Hz and back to 16.5), cooling stepped down critical
|
||||
-> hot -> warm -> ok exactly one dwell apart, and the guard stayed the same process throughout.
|
||||
**A live bug the unit tests missed:** one logging call that warned when heating and informed when
|
||||
cooling raised `ValueError: Logger severity cannot be changed between calls` in rclpy, killing the
|
||||
guard on its first cooling step (the launch file respawned it, which only looked like a recovery). Fixed
|
||||
with separate call sites, the tick now catches and logs any error instead of dying, and a test drives
|
||||
the real node through a full heat-then-cool run; a mutation check (the old logging in a throwaway
|
||||
copy) confirmed that test fails on exactly that error. Not yet tested under a real thermal load.
|
||||
When the servos arrive, watch the under-voltage flag in `robot://status` during a sweep.
|
||||
|
||||
## Arrival day: the neck (servos), planned and deliberately not built ahead of the hardware
|
||||
|
||||
The servos are expected Thursday or Friday at the earliest. The real unknowns cannot be faked (which
|
||||
|
||||
Reference in New Issue
Block a user