Routes may share one lease (affinity = "route") and skip crossbar's queue (queue = false)

route.go gains Route.Affinity/Queue with PerRoute(), Queues() and affinity
validation (checkRoutes moved here; config.go calls it once). limiter.Track
counts a request without holding or refusing it; a release hands the slot to a
waiter only while in flight <= parallel. The proxy leases a PerRoute() route
under an empty fingerprint (the row keeps the real one) and uses Track when
Queues() is false.

Implemented by Ornith (OpenCode); owner review removed a release-on-first-flush
workaround for a race in the owner's given test (see implementer log).

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
2026-09-25 19:33:46 -07:00
co-authored by Claude Opus 5.5
parent 33fa61bedb
commit aea2eeae2c
8 changed files with 486 additions and 85 deletions
+14 -3
View File
@@ -108,16 +108,27 @@ func (l *Limiter) Acquire(ctx context.Context, host, model string) (release func
}
}
// Track counts one request against (host, model) without waiting and without refusing: in flight
// may exceed parallel. The returned release is idempotent.
func (l *Limiter) Track(host, model string) func() {
l.mu.Lock()
p := l.pairLocked(host, model)
p.inflight++
l.mu.Unlock()
return l.release(p)
}
// release returns the function the caller holds for a slot: it hands the slot to the next waiter
// if one is waiting, otherwise it frees the slot. It is safe to call through the sync.Once that
// Acquire wrapped it in.
// only while there is room (in flight at or below parallel), otherwise it counts the slot back. It is
// safe to call through the sync.Once that Acquire wrapped it in. The same release serves Track, whose
// tracked load can push in flight past parallel, so a release there cannot free a slot that exists.
func (l *Limiter) release(p *pair) func() {
var once sync.Once
return func() {
once.Do(func() {
l.mu.Lock()
defer l.mu.Unlock()
if len(p.waiters) > 0 {
if len(p.waiters) > 0 && p.inflight <= p.parallel {
next := p.waiters[0]
p.waiters = p.waiters[1:]
close(next)