kyleandornith-1.5-35b-a3b
f57aaef59e
The parser is pure Go but a goroutine cannot be killed, so extraction runs in a child process: the binary re-executed as `sift pdftext`, reading the PDF on stdin and writing JSON Lines on stdout, killed at a deadline by the parent. pdftext.go holds the types, the five errors, Extract (reader.Open with ErrWrongPassword -> ErrEncrypted, per-page recover, trailer /Info /Title, and rule-3 UTF-16BE/Latin-1 text decode) and ChildMain (soft 768 MiB memory limit, -no env read, no network, exit codes 0/3/4/1). outline.go walks /Outlines -> /First -> /Next with /First children at level+1, resolving /Dest and /GoTo /A dests to 1-based pages, bounded at 10,000 items and never visiting an object twice. runner.go is the Runner: env-isolated child (nil env is an empty environment), 60 s deadline, 64 MiB output cap, 200 MiB stdin cap, stderr capped at 4 KiB, exit-code to error mapping, and Doc parsing with ErrNoText. cmd/sift/main.go adds the unlisted `pdftext` case. tools.go is removed; the go-pdfkit modules stay vendored. Co-Authored-By: ornith-1.5-35b-a3b <ornith-1.5-35b-a3b@llama.cpp.invalid>
53 lines
998 B
Go
53 lines
998 B
Go
package main
|
|
|
|
import (
|
|
"fmt"
|
|
"os"
|
|
|
|
"git.wntrmute.dev/kyle/sift/internal/pdftext"
|
|
)
|
|
|
|
// version is set by ldflags: -X main.version=…
|
|
var version = "dev"
|
|
|
|
func main() {
|
|
if len(os.Args) < 2 {
|
|
usage()
|
|
os.Exit(2)
|
|
}
|
|
cmd := os.Args[1]
|
|
os.Args = append([]string{os.Args[0]}, os.Args[2:]...)
|
|
switch cmd {
|
|
case "serve":
|
|
serveMain()
|
|
case "fetch":
|
|
fetchMain()
|
|
case "search":
|
|
searchMain()
|
|
case "keys":
|
|
keysMain()
|
|
case "pdftext":
|
|
os.Exit(pdftext.ChildMain(os.Args[1:], os.Stdin, os.Stdout))
|
|
case "version", "-version", "--version":
|
|
fmt.Println(version)
|
|
case "-h", "--help", "help":
|
|
usage()
|
|
default:
|
|
fmt.Fprintf(os.Stderr, "sift: unknown command %q\n", cmd)
|
|
usage()
|
|
os.Exit(2)
|
|
}
|
|
}
|
|
|
|
func usage() {
|
|
fmt.Fprintf(os.Stderr, `sift %s — LLM-oriented fetch
|
|
|
|
Commands:
|
|
sift serve [flags] run the service
|
|
sift fetch <url> fetch via the service
|
|
sift search "<query>" search via the service
|
|
sift keys add|list|revoke
|
|
sift version
|
|
`, version)
|
|
}
|