boxmaker/docs/plans/M2a/02-inferproxy.md

M2a task 02: inferproxy

Branch: m2a (run git switch m2a; git status --short must be empty, otherwise stop) Commit subject: Add the inferproxy byte forwarder with its two limits

Goal

loopd will run with no network at all. inferproxy is how it still reaches the model: it listens on a Unix socket and forwards every byte to one fixed TCP address, and back.

Context

From the design brief: "inferproxy is a forwarder of about 100 lines: one Unix socket in, one fixed upstream out." and "it forwards bytes and logs nothing". From the spec: it has two limits, which exist to stop accidents (a retry storm, a loop that spins), and "it never parses what it forwards".

One exchange looks like this: the client connects, sends a request, and reads until the server closes. Some clients also shut down their sending side when the request is sent. Both must work.

Files

• Copy: crates/inferproxy/tests/bucket.rs, crates/inferproxy/tests/forward.rs
• Modify: crates/inferproxy/src/lib.rs, crates/inferproxy/src/main.rs, docs/implementer-log.md

Interfaces

Produces, in crates/inferproxy/src/lib.rs:

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Limits { pub max_connections: usize, pub burst: u32, pub per_second: u32 }
impl Default for Limits { /* max_connections 8, burst 10, per_second 2 */ }

/// Holds at most `burst` tokens and gains `per_second` tokens each second.
pub struct TokenBucket { /* private */ }
impl TokenBucket {
    pub fn new(burst: u32, per_second: u32, now: std::time::Instant) -> Self;   // starts full
    pub fn take(&mut self, now: std::time::Instant) -> bool;                    // one token, if any
}

/// Accepts connections forever. Returns only if `accept` itself fails.
pub fn serve(listener: std::os::unix::net::UnixListener, upstream: String, limits: Limits) -> std::io::Result<()>;

Rules:

1. TokenBucket takes the time as an argument and never reads the clock, so it can be tested without sleeping. Use integer arithmetic (count thousandths of a token); partial refills must add up, so ten checks 100 ms apart at 2 per second grant two tokens. A now earlier than the last one must not panic or underflow (Instant::saturating_duration_since).
2. serve, for each accepted connection, in this order: take a token, else close it at once; check that fewer than max_connections are open, else close it at once; otherwise count it as open and handle it on a new thread. Print one line to stderr for each refusal, saying which limit. Print nothing else about connections, ever: not addresses, not byte counts, not errors.
3. Forwarding. Connect to upstream over TCP. If that fails, close the client. Copy client to server on one thread and server to client on another.
   • When the client stops sending (its read returns 0), shut down the write side of the server connection only, and keep copying the server's answer back.
   • When the server stops sending, the exchange is over: close both.
4. Release the place before closing the client. Decrease the open count before the client's connection is shut down. A client that has read to the end of the answer may connect again at once, and must not be refused because the old connection is still being counted. Do this with a small guard type whose Drop decreases the count, so that every path releases it, including the path where the upstream connection fails.
5. main parses --listen <socket path> and --upstream <host:port> (both required, either order; anything else prints a usage line and exits 2). It removes a socket file left by an earlier run, binds, sets the socket file's mode to 0600, prints one line (inferproxy: <path> -> <upstream>), and calls serve with Limits::default().

API notes (std)

• UnixStream::try_clone() and TcpStream::try_clone() give a second handle for the other direction. stream.shutdown(std::net::Shutdown::Write) sends the half-close.
• std::os::unix::fs::PermissionsExt::from_mode(0o600) with std::fs::set_permissions.
• std::sync::atomic::AtomicUsize in an Arc is enough for the open count.

Steps

• 1. Copy the tests.

git switch m2a
mkdir -p crates/inferproxy/tests
cp docs/plans/M2a/files/crates/inferproxy/tests/*.rs crates/inferproxy/tests/

Read forward.rs. Its two fake upstreams show the two behaviours rule 3 must handle.

• 2. See the tests fail. cargo test -p inferproxy. Expected: they do not compile.
• 3. Write lib.rs and main.rs. Keep the crate's one-line doc comment at the top of lib.rs. Run cargo fmt --all.
• 4. See the tests pass. cargo test -p inferproxy. Expected: bucket 6 passed, forward 6 passed. Run cargo test -p inferproxy --test forward ten times in a row: it must pass every time. If serves_connections_one_after_another fails now and then, rule 4 is not met.
• 5. Try it by hand. In one terminal: cargo run -p inferproxy -- --listen /tmp/ip.sock --upstream straylight:11434. In another: curl --unix-socket /tmp/ip.sock http://localhost/health. Expected: {"status":"ok"}. Stop the proxy with Ctrl-C and remove /tmp/ip.sock. If straylight cannot be reached, note it in your log row and carry on; the tests do not need it.
• 6. Run the gate. make gate. Expected last line: gate: ok.
• 7. Log and commit.

git add crates/inferproxy docs/implementer-log.md
git commit

Done when

• cargo test -p inferproxy passes 12 tests, ten runs in a row; make gate prints gate: ok.
• grep -n "eprintln\|println" crates/inferproxy/src/lib.rs shows only the two refusal messages.

Stop and report if

• A test in forward.rs cannot pass without parsing the bytes being forwarded.