From 3b41d6ffbc909a05b1d07961df362a7d6a9d529b Mon Sep 17 00:00:00 2001 From: Kyle Isom Date: Tue, 22 Sep 2026 21:28:04 -0700 Subject: [PATCH] =?UTF-8?q?plan:=20UA=20decided=20=E2=80=94=20default=20si?= =?UTF-8?q?ft/,=20config=20override,=20measure=20bot-blocks?= MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit Co-Authored-By: Claude Opus 5.5 (1M context) --- docs/PLAN.md | 7 ++++--- 1 file changed, 4 insertions(+), 3 deletions(-) diff --git a/docs/PLAN.md b/docs/PLAN.md index 1ae42d7..9398bad 100644 --- a/docs/PLAN.md +++ b/docs/PLAN.md @@ -173,9 +173,10 @@ approval prompt. Hence: same host), with that skill's fake-success detection, returning provenance `snapshot` + timestamp. Every hop still goes through the SSRF guard. This moves the ladder's `curl` calls out of agent shells. -- **User-Agent policy — Kyle to decide**: (a) honest `sift/` UA only, bot-walled sites - go to recover/browser; or (b) one retry with a browser UA. (b) removes more prompts but is - UA spoofing. +- **User-Agent policy — decided (Kyle, 2026-09-22)**: default UA is `sift/`. A config + setting (`user_agent`) overrides it for the whole service; no per-request UA and no + automatic browser-UA retry. Revisit only if bot walls turn out to be common — measure it: + count `upstream 403/429 (likely bot-blocked)` results per host in `/metrics`. Phase placement: CLI `fetch --mode clean` + skill in **P1**; digest/context flags in P2; `search` in P3; `--recover` in P2 (it is what removes the titan-style prompts).