boxmaker/docs/plans/M2a/02-inferproxy.md

# M2a task 02: `inferproxy`

**Branch:** `m2a` (run `git switch m2a`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `Add the inferproxy byte forwarder with its two limits`

## Goal

`loopd` will run with no network at all. `inferproxy` is how it still reaches the model: it
listens on a Unix socket and forwards every byte to one fixed TCP address, and back.

## Context

From the design brief: "`inferproxy` is a forwarder of about 100 lines: one Unix socket in, one
fixed upstream out." and "it forwards bytes and logs nothing". From the spec: it has two limits,
which exist to stop accidents (a retry storm, a loop that spins), and "it never parses what it
forwards".

One exchange looks like this: the client connects, sends a request, and reads until the server
closes. Some clients also shut down their sending side when the request is sent. Both must work.

## Files

- Copy: `crates/inferproxy/tests/bucket.rs`, `crates/inferproxy/tests/forward.rs`
- Modify: `crates/inferproxy/src/lib.rs`, `crates/inferproxy/src/main.rs`,
  `docs/implementer-log.md`

## Interfaces

Produces, in `crates/inferproxy/src/lib.rs`:

```rust
#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub struct Limits { pub max_connections: usize, pub burst: u32, pub per_second: u32 }
impl Default for Limits { /* max_connections 8, burst 10, per_second 2 */ }

/// Holds at most `burst` tokens and gains `per_second` tokens each second.
pub struct TokenBucket { /* private */ }
impl TokenBucket {
    pub fn new(burst: u32, per_second: u32, now: std::time::Instant) -> Self;   // starts full
    pub fn take(&mut self, now: std::time::Instant) -> bool;                    // one token, if any
}

/// Accepts connections forever. Returns only if `accept` itself fails.
pub fn serve(listener: std::os::unix::net::UnixListener, upstream: String, limits: Limits) -> std::io::Result<()>;
```

Rules:

1. **`TokenBucket`** takes the time as an argument and never reads the clock, so it can be tested
   without sleeping. Use integer arithmetic (count thousandths of a token); partial refills must
   add up, so ten checks 100 ms apart at 2 per second grant two tokens. A `now` earlier than the
   last one must not panic or underflow (`Instant::saturating_duration_since`).
2. **`serve`**, for each accepted connection, in this order: take a token, else close it at once;
   check that fewer than `max_connections` are open, else close it at once; otherwise count it as
   open and handle it on a new thread. Print one line to stderr for each refusal, saying which
   limit. Print nothing else about connections, ever: not addresses, not byte counts, not errors.
3. **Forwarding.** Connect to `upstream` over TCP. If that fails, close the client. Copy client to
   server on one thread and server to client on another.
   - When the client stops sending (its read returns 0), shut down the **write** side of the
     server connection only, and keep copying the server's answer back.
   - When the server stops sending, the exchange is over: close both.
4. **Release the place before closing the client.** Decrease the open count *before* the client's
   connection is shut down. A client that has read to the end of the answer may connect again at
   once, and must not be refused because the old connection is still being counted. Do this with a
   small guard type whose `Drop` decreases the count, so that every path releases it, including
   the path where the upstream connection fails.
5. **`main`** parses `--listen <socket path>` and `--upstream <host:port>` (both required, either
   order; anything else prints a usage line and exits 2). It removes a socket file left by an
   earlier run, binds, sets the socket file's mode to 0600, prints one line
   (`inferproxy: <path> -> <upstream>`), and calls `serve` with `Limits::default()`.

## API notes (std)

- `UnixStream::try_clone()` and `TcpStream::try_clone()` give a second handle for the other
  direction. `stream.shutdown(std::net::Shutdown::Write)` sends the half-close.
- `std::os::unix::fs::PermissionsExt::from_mode(0o600)` with `std::fs::set_permissions`.
- `std::sync::atomic::AtomicUsize` in an `Arc` is enough for the open count.

## Steps

- [ ] **1. Copy the tests.**

```sh
git switch m2a
mkdir -p crates/inferproxy/tests
cp docs/plans/M2a/files/crates/inferproxy/tests/*.rs crates/inferproxy/tests/
```

Read `forward.rs`. Its two fake upstreams show the two behaviours rule 3 must handle.

- [ ] **2. See the tests fail.** `cargo test -p inferproxy`. Expected: they do not compile.
- [ ] **3. Write `lib.rs` and `main.rs`.** Keep the crate's one-line doc comment at the top of
  `lib.rs`. Run `cargo fmt --all`.
- [ ] **4. See the tests pass.** `cargo test -p inferproxy`. Expected: `bucket` 6 passed, `forward`
  6 passed. Run `cargo test -p inferproxy --test forward` ten times in a row: it must pass every
  time. If `serves_connections_one_after_another` fails now and then, rule 4 is not met.
- [ ] **5. Try it by hand.** In one terminal: `cargo run -p inferproxy -- --listen /tmp/ip.sock
  --upstream straylight:11434`. In another:
  `curl --unix-socket /tmp/ip.sock http://localhost/health`. Expected: `{"status":"ok"}`. Stop the
  proxy with Ctrl-C and remove `/tmp/ip.sock`. If straylight cannot be reached, note it in your log
  row and carry on; the tests do not need it.
- [ ] **6. Run the gate.** `make gate`. Expected last line: `gate: ok`.
- [ ] **7. Log and commit.**

```sh
git add crates/inferproxy docs/implementer-log.md
git commit
```

## Done when

- `cargo test -p inferproxy` passes 12 tests, ten runs in a row; `make gate` prints `gate: ok`.
- `grep -n "eprintln\|println" crates/inferproxy/src/lib.rs` shows only the two refusal messages.

## Stop and report if

- A test in `forward.rs` cannot pass without parsing the bytes being forwarded.