# M2a task 08: small server calls and the cache-loss check

**Branch:** `m2a` (run `git switch m2a`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `Add props, slots, tokenize and the cache-loss check to the llama client`

## Goal

Three small calls to the server that are not streams, and one pure function that says whether the
server's prompt cache was lost between two requests.

## Context

The server is a router for several models. For GET calls the model goes in the query
(`/props?model=<id>`); for POST calls it goes in the JSON body. Responses are the server's format:
**ignore unknown fields**. `/props` alone has dozens.

From the measurements: after each request the slot holds `cache_n + prompt_n + predicted_n`
tokens. If the next request of the same conversation reuses about that many (`cache_n`), the cache
held. A normal turn loses one to four tokens at the boundary. If it reuses far fewer, the cache was
lost: another client took the slot, the model was reloaded, or the prompt did not repeat exactly.

## Files

- Copy: `crates/loopd/tests/info.rs`
- Create: `crates/loopd/src/llama/info.rs`
- Modify: `crates/loopd/src/llama/mod.rs` (add `pub mod info;`), `docs/implementer-log.md`

## Interfaces

Consumes: `loopd::http::{Connection, Request, HttpError, read_capped}`, `Client`, `InferError`,
`Timings`.

Produces, in `crates/loopd/src/llama/info.rs`:

```rust
pub const MAX_BODY: usize = 4 * 1024 * 1024;   // bodies that are not streams
pub const MAX_ERROR_BODY: usize = 4 * 1024;    // how much of an error body is kept
pub const CACHE_TOLERANCE: u64 = 64;

#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Props { pub chat_template: String, pub n_ctx: u64, pub total_slots: u32 }

#[derive(Debug, Clone, Copy, PartialEq, Eq, serde::Deserialize)]
pub struct SlotInfo { pub id: u32, pub is_processing: bool }

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum CacheOutcome { Hit, Loss { expected: u64, got: u64 } }

pub fn cache_outcome(previous: &Timings, current: &Timings) -> CacheOutcome;

impl Client {
    pub fn props(&self) -> Result<Props, InferError>;         // GET /props?model=<id>
    pub fn slots(&self) -> Result<Vec<SlotInfo>, InferError>; // GET /slots?model=<id>
    pub fn tokenize(&self, text: &str) -> Result<usize, InferError>;   // POST /tokenize; the token count
}

// Used again by the next tasks, so make them pub(crate):
pub(crate) fn map_http(e: HttpError) -> InferError;
pub(crate) fn error_text(bytes: &[u8]) -> String;
impl Client { pub(crate) fn call(&self, method: &str, path: &str, body: Option<&[u8]>) -> Result<Vec<u8>, InferError>; }
```

Rules the tests check:

1. `call` is one whole exchange: open `cfg.infer.socket`, set the read timeout to
   `cfg.limits.liveness_ms`, send, read the head, read the body with `read_capped(…, MAX_BODY)`.
   A status other than 200 is `InferError::Http { status, body: error_text(&bytes) }`.
2. `map_http`: `Connect(e)` to `Connect(e)`; `Timeout` to `Stalled`; `Closed` to
   `StreamClosedEarly`; everything else to `Protocol` with the error's message.
3. `error_text` keeps at most `MAX_ERROR_BODY` bytes and cuts on a character boundary (use
   `String::from_utf8_lossy`, then `is_char_boundary`). It must not panic on a body whose 4,096th
   byte is in the middle of a character; one test sends exactly that.
4. `props` reads three things from the response: `chat_template`, `total_slots`, and
   `default_generation_settings.n_ctx` (the context of one slot).
5. `tokenize` sends `{"model": <id>, "content": <text>}` and returns the length of `tokens`.
6. A 200 response that does not have the expected shape is `Protocol`.
7. `cache_outcome`: `expected` is `previous.cache_n + previous.prompt_n + previous.predicted_n`.
   It is a `Hit` when `current.cache_n + CACHE_TOLERANCE >= expected`, else
   `Loss { expected, got: current.cache_n }`. 64 short is a hit, 65 short is a loss.

## Steps

- [ ] **1. Copy.** `git switch m2a`, then
  `cp docs/plans/M2a/files/crates/loopd/tests/info.rs crates/loopd/tests/`
- [ ] **2. See the test fail.** `cargo test -p loopd --test info`. Expected: it does not compile.
- [ ] **3. Write `info.rs`** and register the module. Run `cargo fmt --all`.
- [ ] **4. See the test pass.** `cargo test -p loopd --test info`. Expected: `7 passed`.
- [ ] **5. Run the gate.** `make gate`. Expected last line: `gate: ok`. If clippy reports
  `map_http`, `error_text` or `call` as unused, check that `props` really goes through `call`.
- [ ] **6. Log and commit.** `git add crates/loopd docs/implementer-log.md && git commit`

## Done when

- `cargo test -p loopd --test info` reports 7 passed; `make gate` prints `gate: ok`.

## Stop and report if

- The recorded `/props` response has no `default_generation_settings.n_ctx`.