P6 review: runner shutdown, cancel-while-busy, concurrent submits, batch-off 503, transactional store, CLI 202
Owner review of p6 (Ornith tasks 01-09), with an independent reviewer pass whose findings I
confirmed against the code. Every fix has a regression test that fails on the old code.
internal/batch
- Close let the workers keep draining every live job under the cancelled context, failing
every target; they now stop at once, store nothing on a failure during shutdown, and leave
the job live for MarkInFlightFailed("restart").
- A job cancelled while its in-flight target came back busy stayed in the rotation forever
(depth gauge stuck, job held in memory); it now leaves. The task 03 design had this gap: mine.
- Submit's idempotency lookup, caps and create ran unlocked: 20 concurrent submits by one
caller made 8 jobs. Serialised with submitMu.
- A failed target insert left a queued job no worker would run (holding the caller's cap);
it is marked failed.
- A second concurrent cancel returned true and counted twice.
- Config: zero OR negative values take the defaults (negative Workers panicked the
WaitGroup; a negative TTL or byte budget would sweep or evict everything).
- Retry timing used the pre-fetch time (a 5 s busy answer made RetryWait moot).
- The rotation cursor is kept within [0, len(live)] (it was absolute and drifted on removal).
- The unused in-memory callers map is gone (Submit racing Close wrote to a nil map).
internal/dcache
- PutBatchTargets, SweepBatch and eviction are transactional; the sweep never removes a
queued or running job.
internal/api
- Every /v1/batch endpoint answers 503 when batch is off; POST panicked on the nil runner.
- GET /{id}/{idx} bounds idx by the rows actually returned.
- batchFetch copies a *fetch.Error instead of mutating one the PDF pool may share.
internal/config
- ParseSize no longer overflows to a negative size.
cmd/sift
- sift batch treated 202 Accepted (submit) as an error and exited 1 with no message.
Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
This commit is contained in:
+12
-6
@@ -66,14 +66,16 @@ func (s *Server) batchFetch(ctx context.Context, c *auth.Caller, t batch.Target)
|
||||
case 429, 503:
|
||||
return batch.Result{}, fmt.Errorf("%s: %w", fe.Msg, batch.ErrRetryLater)
|
||||
}
|
||||
if fe.Status == 502 {
|
||||
fe.Status = 500
|
||||
fe.Msg = "internal error"
|
||||
// Copy, never mutate: the PDF pool's single-flight may share this error with an
|
||||
// interactive request.
|
||||
status, msg := fe.Status, fe.Msg
|
||||
if status == 502 {
|
||||
status, msg = 500, "internal error"
|
||||
}
|
||||
if fe.Status == 403 {
|
||||
if status == 403 {
|
||||
s.blockN.Add(1)
|
||||
}
|
||||
return batch.Result{}, &batch.TargetError{Status: fe.Status, Msg: fe.Msg, Hint: fe.Hint}
|
||||
return batch.Result{}, &batch.TargetError{Status: status, Msg: msg, Hint: fe.Hint}
|
||||
}
|
||||
s.Log.Error("internal", "caller", c.Name, "host", "-", "err", s.logErr(c, err))
|
||||
return batch.Result{}, &batch.TargetError{Status: 500, Msg: "internal error"}
|
||||
@@ -158,6 +160,10 @@ func (s *Server) batchSubmit(w http.ResponseWriter, r *http.Request) {
|
||||
if c == nil {
|
||||
return
|
||||
}
|
||||
if s.Batch == nil { // [batch] disabled, or the store did not open
|
||||
writeErr(w, 503, "batch disabled", "")
|
||||
return
|
||||
}
|
||||
var req struct {
|
||||
Targets []v1Req `json:"targets"`
|
||||
}
|
||||
@@ -264,7 +270,7 @@ func (s *Server) batchResult(w http.ResponseWriter, r *http.Request) {
|
||||
return
|
||||
}
|
||||
idx, err := strconv.Atoi(r.PathValue("idx"))
|
||||
if err != nil || idx < 0 || idx >= j.TargetsTotal {
|
||||
if err != nil || idx < 0 || idx >= j.TargetsTotal || idx >= len(ts) {
|
||||
writeErr(w, 404, "not found", "")
|
||||
return
|
||||
}
|
||||
|
||||
Reference in New Issue
Block a user