Brief P12: inferproxy is required
Approved 2026-09-17. Evidence is in docs/decisions.md. Co-Authored-By: Claude Fable 5.1 <noreply@anthropic.com>
This commit is contained in:
+1
-1
@@ -36,6 +36,7 @@ Approved by the owner on 2026-09-17. One commit each.
|
|||||||
| P9 | Inference contract 5: slots are pinned with `id_slot` but not reserved. Another client can evict a harness slot, and the router can unload Ornith. Cache loss is an expected event: `loopd` detects it (`cache_n` far below the previous request's total), logs it in the session log, and carries on. It is never an error. A session always uses the same slot; threads may share the main slot. | Shared router decision. M0 (d): evicted sessions came back from the host-RAM prompt cache in under a second on the same slot, and cost a full re-read on another slot. |
|
| P9 | Inference contract 5: slots are pinned with `id_slot` but not reserved. Another client can evict a harness slot, and the router can unload Ornith. Cache loss is an expected event: `loopd` detects it (`cache_n` far below the previous request's total), logs it in the session log, and carries on. It is never an error. A session always uses the same slot; threads may share the main slot. | Shared router decision. M0 (d): evicted sessions came back from the host-RAM prompt cache in under a second on the same slot, and cost a full re-read on another slot. |
|
||||||
| P10 | Inference contract 7: a pinned request can queue behind another client's request on the same slot, and the router can spend tens of seconds reloading the model. The liveness timer starts at the first byte received, and a separate, longer "waiting for slot or model" limit covers the time before it. | Shared router decision. Needs a measurement of what the stream sends while queued. |
|
| P10 | Inference contract 7: a pinned request can queue behind another client's request on the same slot, and the router can spend tens of seconds reloading the model. The liveness timer starts at the first byte received, and a separate, longer "waiting for slot or model" limit covers the time before it. | Shared router decision. Needs a measurement of what the stream sends while queued. |
|
||||||
| P11 | Inference contract 10: the serving flags live in `~/src/nixos`, not here. This repo records the expected values (template hash, per-slot `n_ctx`, slot count, KV type, sampling) and the startup self-test compares them with `/props` and `/slots`. KV type is recorded as q8_0. | Shared router decision. |
|
| P11 | Inference contract 10: the serving flags live in `~/src/nixos`, not here. This repo records the expected values (template hash, per-slot `n_ctx`, slot count, KV type, sampling) and the startup self-test compares them with `/props` and `/slots`. KV type is recorded as q8_0. | Shared router decision. |
|
||||||
|
| P12 | Network isolation 2: `inferproxy` is kept. Remove the conditional about dropping it. Its upstream is the router's TCP listener on the host. | Shared router decision; M0 (g). |
|
||||||
|
|
||||||
## Proposed changes to the design brief
|
## Proposed changes to the design brief
|
||||||
|
|
||||||
@@ -43,4 +44,3 @@ From M0 and the kickoff review. None is applied yet. Each lands as its own commi
|
|||||||
|
|
||||||
| # | Change | Evidence |
|
| # | Change | Evidence |
|
||||||
|---|---|---|
|
|---|---|---|
|
||||||
| P12 | Network isolation 2: `inferproxy` is kept. Remove the conditional about dropping it. Its upstream is the router's TCP listener on the host. | Shared router decision; M0 (g). |
|
|
||||||
|
|||||||
+3
-3
@@ -61,9 +61,9 @@ ordinary outbound networking can reach the whole tailnet and the internet as the
|
|||||||
|
|
||||||
1. `loopd` runs with no network namespace interfaces except loopback (`--network=none`). Its only
|
1. `loopd` runs with no network namespace interfaces except loopback (`--network=none`). Its only
|
||||||
I/O is three Unix sockets on a shared volume.
|
I/O is three Unix sockets on a shared volume.
|
||||||
2. `inferproxy` is a forwarder of about 100 lines: one Unix socket in, one fixed upstream out. If
|
2. `inferproxy` is a forwarder of about 100 lines: one Unix socket in, one fixed upstream out. The
|
||||||
M0 shows `llama-server` can listen on a Unix socket and it runs on the same host, drop
|
upstream is the shared router's TCP listener on the host. It is required: `loopd` has no
|
||||||
`inferproxy` and mount that socket instead.
|
network, and the router stays on TCP for its other clients.
|
||||||
3. `gatewayd` has outbound network only. It opens no listening TCP port. v0 approvals use replies
|
3. `gatewayd` has outbound network only. It opens no listening TCP port. v0 approvals use replies
|
||||||
or emoji reactions, which arrive over the Mattermost WebSocket, so no callback URL is needed.
|
or emoji reactions, which arrive over the Mattermost WebSocket, so no callback URL is needed.
|
||||||
4. Tool containers get no network unless the grant names hosts. Granted network access goes through
|
4. Tool containers get no network unless the grant names hosts. Granted network access goes through
|
||||||
|
|||||||
Reference in New Issue
Block a user