Add the per-slot gate and chat_with_retry

Implemented SlotGate (one request in flight per server slot, FIFO queue,
bounded) and chat_with_retry (retry only server-gone errors, jittered
backoff within a window and attempt budget). chat acquires the gate and
maps GateFull to Busy; Client gained a gate field.

Implemented-By: OpenCode session (model recorded in docs/implementer-log.md)
This commit is contained in:
2026-09-18 14:45:45 -07:00
parent a07c341cbb
commit 59cf89e7ba
6 changed files with 597 additions and 2 deletions
+7 -1
View File
@@ -2,8 +2,10 @@
pub mod assemble;
pub mod chat;
pub mod gate;
pub mod info;
pub mod request;
pub mod retry;
#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ChatMessage {
@@ -136,11 +138,15 @@ impl std::error::Error for InferError {}
pub struct Client {
pub(crate) cfg: crate::config::Config,
pub(crate) gate: gate::SlotGate,
}
impl Client {
pub fn new(cfg: crate::config::Config) -> Self {
Self { cfg }
Self {
cfg,
gate: gate::SlotGate::new(),
}
}
pub fn config(&self) -> &crate::config::Config {
&self.cfg