kyleandornith-1.5-35b-a3b
f6644c9278
Cached chunk embeddings change what a long document answers with: BM25 first, embedding uncached chunks (or the whole document) in the background so the next question on the same doc scores hybrid. selectText now returns its mode and the number of chunks that had a vector at answer-build time; the digest cache stores only hybrid answers, never a provisional bm25 one. The background worker batches through a Dixie gate and WaitBackground polls for true idle. Co-Authored-By: ornith-1.5-35b-a3b <ornith-1.5-35b-a3b@llama.cpp.invalid>
sift
LLM-oriented fetch and search for the Hermes fleet: fetch a resource and get it back cleaned, compact, and with useful context attached — using local models (Dixie), SearXNG and Brave on orion, and Honcho for retrieval.
Status: implemented through P5a: P1 clean fetch (/v2/scrape, /v1/fetch, CLI, auth), P2
digest, fleet context and recover, P3 search, P2.7 Prometheus metrics, P5a PDF documents. The design
is docs/PLAN.md; each phase has its own doc (docs/P1.md … docs/P5a.md).
API
All endpoints except /healthz take Authorization: Bearer <key> (per caller; see sift keys).
The server binds 127.0.0.1:8890 and is published on the tailnet by tailscale serve at
https://orion.scylla-hammerhead.ts.net:8443.
| Endpoint | What |
|---|---|
POST /v2/scrape |
Firecrawl-compatible fetch (url, formats) — what Hermes web_extract calls |
POST /v2/search |
Firecrawl-compatible search (query, limit) — what Hermes web_search calls |
POST /v1/fetch |
native fetch: url, mode (clean/raw/digest), question, jq, context, recover, and for PDFs pages, grep |
POST /v1/search |
native search: query, question, k, rerank, digest |
GET /metrics, /metrics.json |
Prometheus text or JSON; open on loopback, bearer elsewhere |
GET /healthz |
liveness |
Every outbound connection goes through the SSRF guard (internal/guard): private, tailnet and
configured deny_prefixes addresses, and the host's own WAN address ([self_ip]), are refused
with blocked: ….
CLI
sift serve [-listen addr] [-config /etc/sift/config.toml] [-keys /var/lib/sift/keys.toml] [-deny-prefix CIDR]...
sift fetch <url> [--question q] [--mode clean|raw|digest] [--jq filter] [--pages a-b] [--grep re]
[--no-context] [--recover] [--json]
sift search "<query>" [--k n] [--json]
sift keys add --caller name [--workspace ws] [--out file] | list | revoke --caller name
sift version
sift fetch and sift search are clients: they read url and key_file from client.toml
($SIFT_CLIENT, else $HERMES_HOME/sift/client.toml, else ~/.config/sift/client.toml); the key
file must be mode 0600. The agent-facing guide is skill/SKILL.md.
Configuration
/etc/sift/config.toml ($SIFT_CONFIG); every key is shown in
dist/sift.toml.example. Paid-search keys (BRAVE_API_KEY,
TAVILY_API_KEY) live on the box in /var/lib/sift/secret.env ($SIFT_SECRET), never in git or
the Nix store. On NixOS use nixosModules.sift (services.sift), which writes the config and
the hardened unit.
Build
Makefile is the source of truth for flags (CGO_ENABLED=0, -trimpath, -s -w -X main.version=).
make build # bin/sift
make test # go test -race ./...
make lint # go vet, and staticcheck if installed
make install # PREFIX=/usr/local; example config in /etc/sift/
nix build # packages.sift, same ldflags
nix flake check # make test lint
The service does not assume Nix store paths. Non-Nix hosts: make install plus dist/sift.service.
PDF documents
A resource whose content type (or magic bytes) look like a PDF is extracted instead of cleaned.
Extraction runs in a child process — the sift pdftext subcommand of this binary, built on the
go-pdfkit/extract v0.3.0 engine, chosen by the bake-off in docs/P5a.md §3 — so a
malicious PDF can't take down the server: the child is isolated (empty environment, native syscalls
only), killed at a deadline, and its output capped.
Limits: a PDF may be 100 MB and take up to 120 s to download; extraction is capped at
60 s and 2,000 pages, with child output bounded to 64 MiB. Extraction runs under a pool
of 2 slots, 4 waiting, a 20 s wait cap — a caller that can't get a slot gets 503 busy,
and a caller with a job already in flight gets 429. A PDF already extracted is answered from the
cache for 24 h, then revalidated with a conditional GET (a 304 reuses the stored text).
Answers (through /v1/fetch or /v2/scrape):
- whole text when it fits the output cap, each page prefixed with
--- page N ---; - otherwise a map — the outline (with page numbers) and the first pages;
--pages 41-60for a range,--grep 'pattern'for matching lines with page context;--question "…"for a digest that, on a PDF, cites each fact as(p. N).
A range returns at most 100 pages; grep shows at most 100 matches and takes a pattern of at most
200 bytes. pages and grep on anything but a PDF are a 400. A question over a document too long
to digest (more than 200 chunks) is refused with
422 document too long for question (N chunks); use grep or pages.
Scanned PDFs have no text layer and return no text layer (scanned PDF?); point them at an OCR
source rather than fixing them here.
Errors (all 422 unless noted — a PDF never yields a 500):
| cause | Status | Message | Hint | Retry-After |
|---|---|---|---|---|
no text layer (ErrNoText) |
422 | no text layer (scanned PDF?) |
ocr |
— |
encrypted (ErrEncrypted) |
422 | encrypted PDF (password required) |
— | — |
unreadable (ErrUnreadable) |
422 | unreadable PDF |
— | — |
timeout (ErrTimeout) |
422 | pdf extraction timed out |
— | — |
too much text (ErrTooMuchText) |
422 | pdf text over the output limit |
— | — |
pool full (ErrBusy) |
503 | pdf workers busy |
— | 10 |
caller busy (ErrCallerBusy) |
429 | busy |
— | — |
| anything else (e.g. a recovered panic) | 422 | unreadable PDF |
— | — |
Metrics (scraped at http://127.0.0.1:8890/metrics): sift_pdf_extract_total{outcome}
(ok, no_text, encrypted, unreadable, timeout, too_large, busy, or other),
sift_pdf_extract_seconds, sift_pdf_pages_total, sift_pdf_pool_wait_seconds, and
sift_pdf_cache_hits_total.