kyle/sift
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

docs: skill matches shipped P2 (no search, recover, fallback cap)

Browse Source 
sift search is P3 and not built — keep web_search until then.
Recover ladder is Wayback then archive.today. digest.fallback
means clean text capped at ~6k tokens.
This commit is contained in:
kyle
2026-09-23 06:00:50 -07:00
parent 767c12c1af
commit 14eb7808ce
1 changed files with 12 additions and 11 deletions
Download Patch File Download Diff File Expand all files Collapse all files
skill/SKILL.md
+12 -11
View File
@@ -1,30 +1,33 @@
--- ---
name: sift name: sift
description: "Use FIRST whenever you need content from a URL or a web search: sift fetch/search returns cleaned, compact, sourced content with fleet context — never curl | python/jq." description: "Use FIRST whenever you need content from a URL: sift fetch returns cleaned, compact, sourced content with fleet context — never curl | python/jq. For web search, use web_search until sift search ships in P3."
version: 0.1.0 version: 0.1.0
author: Claude (for Kyle's fleet) author: Claude (for Kyle's fleet)
license: MIT license: MIT
platforms: [linux, macos] platforms: [linux, macos]
metadata: metadata:
hermes: hermes:
tags: [Web, Fetch, Search, Research, Extraction] tags: [Web, Fetch, Research, Extraction]
related_skills: [blocked-page-recovery, grounded-citations, external-claim-verification] related_skills: [blocked-page-recovery, grounded-citations, external-claim-verification]
prerequisites: prerequisites:
commands: [sift] commands: [sift]
--- ---
# sift — fetch and search, the fleet way # sift — fetch, the fleet way
`sift` is the fleet's fetch/search service on orion (repo `git.wntrmute.dev:kyle/sift`). `sift` is the fleet's fetch service on orion (repo `git.wntrmute.dev:kyle/sift`).
The `sift` CLI on this host talks to it. It returns the resource **cleaned** (HTML → The `sift` CLI on this host talks to it. It returns the resource **cleaned** (HTML →
markdown, JSON → JSON), optionally **digested** for your question by a local model, with markdown, JSON → JSON), optionally **digested** for your question by a local model, with
**what the fleet already knows** attached from Honcho — so you read 1k tokens, not 20k. **what the fleet already knows** attached from Honcho — so you read 1k tokens, not 20k.
`sift search` is **coming in P3** and is not built. Until then, use `web_search` for
searches, then `sift fetch` on the URLs you need.
## Rule: sift is the explicit first step ## Rule: sift is the explicit first step
Whenever you need the content of a URL or a web search: Whenever you need the content of a URL:
1. **Use `sift`.** Not `curl`, not `wget`, not `curl … | python -c`, not `curl … | jq`. 1. **Use `sift fetch`.** Not `curl`, not `wget`, not `curl … | python -c`, not `curl … | jq`.
Shell fetches trip the security scanner and put an approval prompt in front of Kyle; Shell fetches trip the security scanner and put an approval prompt in front of Kyle;
`sift` commands don't. `sift` commands don't.
2. If `sift` says the page is **blocked**, follow its `hint` (below). Do not improvise 2. If `sift` says the page is **blocked**, follow its `hint` (below). Do not improvise
@@ -43,10 +46,6 @@ sift fetch "https://example.com/article" --mode clean
# JSON APIs: filter server-side with jq syntax (replaces `curl … | jq`) # JSON APIs: filter server-side with jq syntax (replaces `curl … | jq`)
sift fetch "https://huggingface.co/api/datasets/owner/name" --jq '.siblings[].rfilename' sift fetch "https://huggingface.co/api/datasets/owner/name" --jq '.siblings[].rfilename'
# Search: SearXNG + Brave LLM Context, deduped, reranked; --question adds digests
sift search "DGX Spark price availability" --k 5
sift search "SK Hynix HBM wafer allocation 2026" --question "what share of DRAM wafers go to HBM?"
``` ```
Useful flags: `--mode digest|clean|raw` (default `digest` when `--question` is given, Useful flags: `--mode digest|clean|raw` (default `digest` when `--question` is given,
@@ -67,13 +66,15 @@ for blocked pages), `--json` (full envelope: content, source, context, sizes, wa
re-researching; don't present it as coming from the page. re-researching; don't present it as coming from the page.
- **`warnings`** matter: e.g. `digest fell back to clean: dixie timeout` means you got the - **`warnings`** matter: e.g. `digest fell back to clean: dixie timeout` means you got the
full cleaned page, not a digest. full cleaned page, not a digest.
- When `digest.fallback` is set, **`content` is clean text capped at ~6k tokens**, not a
digest. Read it as a truncated page, not as an answer.
## When sift reports a problem ## When sift reports a problem
| Result | Meaning | Do this | | Result | Meaning | Do this |
|---|---|---| |---|---|---|
| `blocked: private address …` | URL resolves into the tailnet/LAN | Stop. That's the SSRF guard working. Don't work around it. Ask Kyle if you really need an internal resource. | | `blocked: private address …` | URL resolves into the tailnet/LAN | Stop. That's the SSRF guard working. Don't work around it. Ask Kyle if you really need an internal resource. |
| `upstream 403/429 (likely bot-blocked)`, `hint: recover` | Site blocks non-browser clients | `sift fetch URL --recover` (Wayback → archive.today → API/RSS pivot, server-side). Cite snapshots **with their date**. | | `upstream 403/429 (likely bot-blocked)`, `hint: recover` | Site blocks non-browser clients | `sift fetch URL --recover` (Wayback → archive.today, server-side). Cite snapshots **with their date**. |
| `hint: browser` | Needs JS rendering or a real browser | Use the browser tool. | | `hint: browser` | Needs JS rendering or a real browser | Use the browser tool. |
| `415 unsupported type` | e.g. PDF in v1 | Browser tool, or ask. | | `415 unsupported type` | e.g. PDF in v1 | Browser tool, or ask. |
| connection refused / timeout to sift itself | sift is down | Fallback below, and tell Kyle sift is down. | | connection refused / timeout to sift itself | sift is down | Fallback below, and tell Kyle sift is down. |