kyle/sift
1
0
Fork 0
Code Issues Pull Requests Actions Packages Projects Releases Wiki Activity

p3: bake-off collector that refuses a mislabelled dump

Browse Source 
run.py records one named config at a time. Sift dumps fail if
sizes.sources_ok does not match --config. Hermes Tavily uses the
live-tool request shape, not sift. Raw dumps stay gitignored.
This commit is contained in:
kyle
2026-09-23 15:29:46 -07:00
parent 35e2356541
commit 7558eca46d
3 changed files with 300 additions and 0 deletions
Download Patch File Download Diff File Expand all files Collapse all files
tests/testdata/BAKEOFF.md
Vendored
+13
View File
@@ -38,6 +38,19 @@ tests/testdata/bakeoff-out/{A,B,C,D}.json
Shuffle with `tests/bakeoff/blind.py` after the four raw dumps exist.
Collect a dump after orion is set to that configuration:
```
python3 tests/bakeoff/run.py --config searxng
python3 tests/bakeoff/run.py --config searxng_brave
python3 tests/bakeoff/run.py --config searxng_brave_tavily
TAVILY_API_KEY=… python3 tests/bakeoff/run.py --config hermes_tavily
python3 tests/bakeoff/blind.py --raw-dir tests/testdata/bakeoff-raw
```
`run.py` refuses to stamp a sift dump if `sizes.sources_ok` does not match `--config`.
Titan does not enable sources or edit nixos.
Judges: **talos or Claude**. Score each query: would you `sift fetch` the
correct URL from this list? Against `search_targets.txt` when it is not `open`.