From 14eb7808ce9e331772aa2b3e40bbfd9a44b9b773 Mon Sep 17 00:00:00 2001 From: Kyle Isom Date: Wed, 23 Sep 2026 06:00:50 -0700 Subject: [PATCH] docs: skill matches shipped P2 (no search, recover, fallback cap) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit sift search is P3 and not built — keep web_search until then. Recover ladder is Wayback then archive.today. digest.fallback means clean text capped at ~6k tokens. --- skill/SKILL.md | 23 ++++++++++++----------- 1 file changed, 12 insertions(+), 11 deletions(-) diff --git a/skill/SKILL.md b/skill/SKILL.md index 70fc298..ff71832 100644 --- a/skill/SKILL.md +++ b/skill/SKILL.md @@ -1,30 +1,33 @@ --- name: sift -description: "Use FIRST whenever you need content from a URL or a web search: sift fetch/search returns cleaned, compact, sourced content with fleet context — never curl | python/jq." +description: "Use FIRST whenever you need content from a URL: sift fetch returns cleaned, compact, sourced content with fleet context — never curl | python/jq. For web search, use web_search until sift search ships in P3." version: 0.1.0 author: Claude (for Kyle's fleet) license: MIT platforms: [linux, macos] metadata: hermes: - tags: [Web, Fetch, Search, Research, Extraction] + tags: [Web, Fetch, Research, Extraction] related_skills: [blocked-page-recovery, grounded-citations, external-claim-verification] prerequisites: commands: [sift] --- -# sift — fetch and search, the fleet way +# sift — fetch, the fleet way -`sift` is the fleet's fetch/search service on orion (repo `git.wntrmute.dev:kyle/sift`). +`sift` is the fleet's fetch service on orion (repo `git.wntrmute.dev:kyle/sift`). The `sift` CLI on this host talks to it. It returns the resource **cleaned** (HTML → markdown, JSON → JSON), optionally **digested** for your question by a local model, with **what the fleet already knows** attached from Honcho — so you read 1k tokens, not 20k. +`sift search` is **coming in P3** and is not built. Until then, use `web_search` for +searches, then `sift fetch` on the URLs you need. + ## Rule: sift is the explicit first step -Whenever you need the content of a URL or a web search: +Whenever you need the content of a URL: -1. **Use `sift`.** Not `curl`, not `wget`, not `curl … | python -c`, not `curl … | jq`. +1. **Use `sift fetch`.** Not `curl`, not `wget`, not `curl … | python -c`, not `curl … | jq`. Shell fetches trip the security scanner and put an approval prompt in front of Kyle; `sift` commands don't. 2. If `sift` says the page is **blocked**, follow its `hint` (below). Do not improvise @@ -43,10 +46,6 @@ sift fetch "https://example.com/article" --mode clean # JSON APIs: filter server-side with jq syntax (replaces `curl … | jq`) sift fetch "https://huggingface.co/api/datasets/owner/name" --jq '.siblings[].rfilename' - -# Search: SearXNG + Brave LLM Context, deduped, reranked; --question adds digests -sift search "DGX Spark price availability" --k 5 -sift search "SK Hynix HBM wafer allocation 2026" --question "what share of DRAM wafers go to HBM?" ``` Useful flags: `--mode digest|clean|raw` (default `digest` when `--question` is given, @@ -67,13 +66,15 @@ for blocked pages), `--json` (full envelope: content, source, context, sizes, wa re-researching; don't present it as coming from the page. - **`warnings`** matter: e.g. `digest fell back to clean: dixie timeout` means you got the full cleaned page, not a digest. +- When `digest.fallback` is set, **`content` is clean text capped at ~6k tokens**, not a + digest. Read it as a truncated page, not as an answer. ## When sift reports a problem | Result | Meaning | Do this | |---|---|---| | `blocked: private address …` | URL resolves into the tailnet/LAN | Stop. That's the SSRF guard working. Don't work around it. Ask Kyle if you really need an internal resource. | -| `upstream 403/429 (likely bot-blocked)`, `hint: recover` | Site blocks non-browser clients | `sift fetch URL --recover` (Wayback → archive.today → API/RSS pivot, server-side). Cite snapshots **with their date**. | +| `upstream 403/429 (likely bot-blocked)`, `hint: recover` | Site blocks non-browser clients | `sift fetch URL --recover` (Wayback → archive.today, server-side). Cite snapshots **with their date**. | | `hint: browser` | Needs JS rendering or a real browser | Use the browser tool. | | `415 unsupported type` | e.g. PDF in v1 | Browser tool, or ask. | | connection refused / timeout to sift itself | sift is down | Fallback below, and tell Kyle sift is down. |