Routes may share one lease (affinity = "route") and skip crossbar's queue (queue = false)
route.go gains Route.Affinity/Queue with PerRoute(), Queues() and affinity validation (checkRoutes moved here; config.go calls it once). limiter.Track counts a request without holding or refusing it; a release hands the slot to a waiter only while in flight <= parallel. The proxy leases a PerRoute() route under an empty fingerprint (the row keeps the real one) and uses Track when Queues() is false. Implemented by Ornith (OpenCode); owner review removed a release-on-first-flush workaround for a race in the owner's given test (see implementer log). Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
@@ -108,16 +108,27 @@ func (l *Limiter) Acquire(ctx context.Context, host, model string) (release func
|
||||
}
|
||||
}
|
||||
|
||||
// Track counts one request against (host, model) without waiting and without refusing: in flight
|
||||
// may exceed parallel. The returned release is idempotent.
|
||||
func (l *Limiter) Track(host, model string) func() {
|
||||
l.mu.Lock()
|
||||
p := l.pairLocked(host, model)
|
||||
p.inflight++
|
||||
l.mu.Unlock()
|
||||
return l.release(p)
|
||||
}
|
||||
|
||||
// release returns the function the caller holds for a slot: it hands the slot to the next waiter
|
||||
// if one is waiting, otherwise it frees the slot. It is safe to call through the sync.Once that
|
||||
// Acquire wrapped it in.
|
||||
// only while there is room (in flight at or below parallel), otherwise it counts the slot back. It is
|
||||
// safe to call through the sync.Once that Acquire wrapped it in. The same release serves Track, whose
|
||||
// tracked load can push in flight past parallel, so a release there cannot free a slot that exists.
|
||||
func (l *Limiter) release(p *pair) func() {
|
||||
var once sync.Once
|
||||
return func() {
|
||||
once.Do(func() {
|
||||
l.mu.Lock()
|
||||
defer l.mu.Unlock()
|
||||
if len(p.waiters) > 0 {
|
||||
if len(p.waiters) > 0 && p.inflight <= p.parallel {
|
||||
next := p.waiters[0]
|
||||
p.waiters = p.waiters[1:]
|
||||
close(next)
|
||||
|
||||
Reference in New Issue
Block a user