brokerd: recover a torn line in place, real dates only, bounded ttl, EMFILE
From the independent review of task 23. A torn last line followed by an empty later file had its recovery written into the later file, which broke the chain for good; the line is now ended in its own file. The log-name rule takes months 01 to 12 and days 01 to 31 only. [approvals] ttl_ms is limited to a day, the longest loopd waits after a pending frame. Running out of file descriptors or memory pauses the listener instead of stopping brokerd (the errors the previous fix skipped do not occur on Linux). args.rs's doc fixed. Co-Authored-By: Claude Opus 5.5 (1M context) <noreply@anthropic.com>
This commit is contained in:
+25
-11
@@ -197,9 +197,10 @@ impl Started {
|
||||
}
|
||||
}
|
||||
|
||||
/// Accept streams from one listener forever, starting a handler thread per stream. An error that
|
||||
/// only concerns one connection (the peer aborted it, a signal interrupted `accept`) is skipped;
|
||||
/// any other stops the daemon through `tx`.
|
||||
/// Accept streams from one listener forever, starting a handler thread per stream. When the
|
||||
/// system is out of file descriptors or memory, the listener pauses and tries again instead of
|
||||
/// stopping: otherwise a peer holding many idle connections could stop the daemon, and the other
|
||||
/// socket with it. Any other error stops the daemon through `tx`.
|
||||
fn accept_loop(
|
||||
listener: UnixListener,
|
||||
broker: Arc<Broker>,
|
||||
@@ -213,10 +214,22 @@ fn accept_loop(
|
||||
std::thread::Builder::new()
|
||||
.name(name.to_string())
|
||||
.spawn(move || {
|
||||
let mut reported = false;
|
||||
for stream in listener.incoming() {
|
||||
match stream {
|
||||
Ok(stream) => serve_one(stream, &broker, which),
|
||||
Err(e) if one_connection_only(&e) => {}
|
||||
Ok(stream) => {
|
||||
reported = false;
|
||||
serve_one(stream, &broker, which);
|
||||
}
|
||||
Err(e) if out_of_resources(&e) => {
|
||||
if !reported {
|
||||
broker.log(&format!(
|
||||
"brokerd: cannot accept on {name} for now, retrying: {e}\nsee docs/runbook.md#brokerd-listener-lost"
|
||||
));
|
||||
reported = true;
|
||||
}
|
||||
std::thread::sleep(Duration::from_millis(200));
|
||||
}
|
||||
Err(e) => {
|
||||
// Once `run` has returned, nobody receives; dropping this is correct.
|
||||
let _ = tx.send(e);
|
||||
@@ -243,10 +256,11 @@ fn serve_one(stream: UnixStream, broker: &Arc<Broker>, which: Which) {
|
||||
}
|
||||
}
|
||||
|
||||
/// Errors from `accept` that end one connection, not the listener.
|
||||
fn one_connection_only(e: &io::Error) -> bool {
|
||||
matches!(
|
||||
e.kind(),
|
||||
io::ErrorKind::ConnectionAborted | io::ErrorKind::Interrupted | io::ErrorKind::WouldBlock
|
||||
)
|
||||
/// Errors from `accept` that mean "not now" rather than "never": out of file descriptors
|
||||
/// (`EMFILE`, `ENFILE`, the same numbers on Linux and macOS) or out of memory. `accept` already
|
||||
/// retries `EINTR` itself.
|
||||
fn out_of_resources(e: &io::Error) -> bool {
|
||||
const ENFILE: i32 = 23;
|
||||
const EMFILE: i32 = 24;
|
||||
matches!(e.raw_os_error(), Some(ENFILE | EMFILE)) || e.kind() == io::ErrorKind::OutOfMemory
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user