docs: skill matches shipped P2 (no search, recover, fallback cap)
sift search is P3 and not built — keep web_search until then. Recover ladder is Wayback then archive.today. digest.fallback means clean text capped at ~6k tokens.
This commit is contained in:
+12
-11
@@ -1,30 +1,33 @@
|
||||
---
|
||||
name: sift
|
||||
description: "Use FIRST whenever you need content from a URL or a web search: sift fetch/search returns cleaned, compact, sourced content with fleet context — never curl | python/jq."
|
||||
description: "Use FIRST whenever you need content from a URL: sift fetch returns cleaned, compact, sourced content with fleet context — never curl | python/jq. For web search, use web_search until sift search ships in P3."
|
||||
version: 0.1.0
|
||||
author: Claude (for Kyle's fleet)
|
||||
license: MIT
|
||||
platforms: [linux, macos]
|
||||
metadata:
|
||||
hermes:
|
||||
tags: [Web, Fetch, Search, Research, Extraction]
|
||||
tags: [Web, Fetch, Research, Extraction]
|
||||
related_skills: [blocked-page-recovery, grounded-citations, external-claim-verification]
|
||||
prerequisites:
|
||||
commands: [sift]
|
||||
---
|
||||
|
||||
# sift — fetch and search, the fleet way
|
||||
# sift — fetch, the fleet way
|
||||
|
||||
`sift` is the fleet's fetch/search service on orion (repo `git.wntrmute.dev:kyle/sift`).
|
||||
`sift` is the fleet's fetch service on orion (repo `git.wntrmute.dev:kyle/sift`).
|
||||
The `sift` CLI on this host talks to it. It returns the resource **cleaned** (HTML →
|
||||
markdown, JSON → JSON), optionally **digested** for your question by a local model, with
|
||||
**what the fleet already knows** attached from Honcho — so you read 1k tokens, not 20k.
|
||||
|
||||
`sift search` is **coming in P3** and is not built. Until then, use `web_search` for
|
||||
searches, then `sift fetch` on the URLs you need.
|
||||
|
||||
## Rule: sift is the explicit first step
|
||||
|
||||
Whenever you need the content of a URL or a web search:
|
||||
Whenever you need the content of a URL:
|
||||
|
||||
1. **Use `sift`.** Not `curl`, not `wget`, not `curl … | python -c`, not `curl … | jq`.
|
||||
1. **Use `sift fetch`.** Not `curl`, not `wget`, not `curl … | python -c`, not `curl … | jq`.
|
||||
Shell fetches trip the security scanner and put an approval prompt in front of Kyle;
|
||||
`sift` commands don't.
|
||||
2. If `sift` says the page is **blocked**, follow its `hint` (below). Do not improvise
|
||||
@@ -43,10 +46,6 @@ sift fetch "https://example.com/article" --mode clean
|
||||
|
||||
# JSON APIs: filter server-side with jq syntax (replaces `curl … | jq`)
|
||||
sift fetch "https://huggingface.co/api/datasets/owner/name" --jq '.siblings[].rfilename'
|
||||
|
||||
# Search: SearXNG + Brave LLM Context, deduped, reranked; --question adds digests
|
||||
sift search "DGX Spark price availability" --k 5
|
||||
sift search "SK Hynix HBM wafer allocation 2026" --question "what share of DRAM wafers go to HBM?"
|
||||
```
|
||||
|
||||
Useful flags: `--mode digest|clean|raw` (default `digest` when `--question` is given,
|
||||
@@ -67,13 +66,15 @@ for blocked pages), `--json` (full envelope: content, source, context, sizes, wa
|
||||
re-researching; don't present it as coming from the page.
|
||||
- **`warnings`** matter: e.g. `digest fell back to clean: dixie timeout` means you got the
|
||||
full cleaned page, not a digest.
|
||||
- When `digest.fallback` is set, **`content` is clean text capped at ~6k tokens**, not a
|
||||
digest. Read it as a truncated page, not as an answer.
|
||||
|
||||
## When sift reports a problem
|
||||
|
||||
| Result | Meaning | Do this |
|
||||
|---|---|---|
|
||||
| `blocked: private address …` | URL resolves into the tailnet/LAN | Stop. That's the SSRF guard working. Don't work around it. Ask Kyle if you really need an internal resource. |
|
||||
| `upstream 403/429 (likely bot-blocked)`, `hint: recover` | Site blocks non-browser clients | `sift fetch URL --recover` (Wayback → archive.today → API/RSS pivot, server-side). Cite snapshots **with their date**. |
|
||||
| `upstream 403/429 (likely bot-blocked)`, `hint: recover` | Site blocks non-browser clients | `sift fetch URL --recover` (Wayback → archive.today, server-side). Cite snapshots **with their date**. |
|
||||
| `hint: browser` | Needs JS rendering or a real browser | Use the browser tool. |
|
||||
| `415 unsupported type` | e.g. PDF in v1 | Browser tool, or ask. |
|
||||
| connection refused / timeout to sift itself | sift is down | Fallback below, and tell Kyle sift is down. |
|
||||
|
||||
Reference in New Issue
Block a user