boxmaker/docs/plans/M2a/06-llama-request.md

# M2a task 06: the llama client's types, and the request body

**Branch:** `m2a` (run `git switch m2a`; `git status --short` must be empty, otherwise stop)
**Commit subject:** `Add the llama client types and the chat request builder`

## Goal

Define what goes into a chat request and what comes back, and build the JSON body the server
expects. The streaming and the waiting come in later tasks; this one has no I/O.

## Context

The server speaks the OpenAI chat-completions format with some fields of its own. Two facts from
the project's measurements shape this task:

- The server caches the prompt. A request only gets that cache if it repeats the earlier messages
  exactly, so an assistant message is sent back with the same three fields it arrived with.
- `tools` is rendered at the very top of the prompt. It is sent only when there are tools.

## Files

- Copy: `crates/loopd/tests/request.rs`
- Create: `crates/loopd/src/llama/mod.rs`, `crates/loopd/src/llama/request.rs`
- Modify: `crates/loopd/src/lib.rs`, `docs/implementer-log.md`

## Interfaces

Produces, in `crates/loopd/src/llama/mod.rs`. Copy these definitions as they are; later tasks and
their tests depend on every name.

```rust
pub mod request;

#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ChatMessage {
    System { content: String },
    User { content: String },
    Assistant { content: Option<String>, reasoning_content: Option<String>, tool_calls: Vec<proto::ToolCall> },
    Tool { tool_call_id: String, content: String },
}

#[derive(Debug, Clone, PartialEq)]
pub struct ToolSchema { pub name: String, pub description: String, pub parameters: serde_json::Value }

#[derive(Debug, Clone, PartialEq)]
pub struct ChatRequest { pub slot: u32, pub messages: Vec<ChatMessage>, pub tools: Vec<ToolSchema>, pub thinking: bool }

#[derive(Debug, Clone, PartialEq, Eq)]
pub enum ChatEvent {
    Queued { ahead: usize },
    Waiting { slot_busy: bool },
    Progress { total: u64, cache: u64, processed: u64 },
    Reasoning(String),
    Content(String),
    ToolCallDelta { index: u32, id: Option<String>, name: Option<String>, arguments: String },
    ThinkingCapped { tokens: u64 },
    Retrying { attempt: u32, after_ms: u64, error: String },
}

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum FinishReason { Stop, ToolCalls, Length }

/// The three counts loopd uses. The server sends more fields beside them; they are ignored.
#[derive(Debug, Clone, Copy, PartialEq, Eq, Default, serde::Deserialize)]
pub struct Timings { pub cache_n: u64, pub prompt_n: u64, pub predicted_n: u64 }

#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Completion {
    pub id: String,
    pub content: Option<String>,
    pub reasoning_content: Option<String>,
    pub tool_calls: Vec<proto::ToolCall>,
    pub finish_reason: FinishReason,
    pub timings: Timings,
    pub reasoning_tokens: u64,
    pub thinking_capped: bool,
}

#[derive(Debug)]
pub enum InferError {
    Busy, Connect(std::io::Error), WaitTimeout, LoadTimeout, Stalled, StreamClosedEarly,
    ThinkingOverrun, Http { status: u16, body: String }, Protocol(String),
}   // Display (one short sentence per variant) and std::error::Error

pub struct Client { pub(crate) cfg: crate::config::Config }
impl Client {
    pub fn new(cfg: crate::config::Config) -> Self;
    pub fn config(&self) -> &crate::config::Config;
}
```

`Timings` parses **the server's** JSON, so it must **not** have `deny_unknown_fields`: the server
sends six more timing fields beside these three.

And in `crates/loopd/src/llama/request.rs`:

```rust
/// The request body as JSON text. The same inputs always give the same bytes.
pub fn build_body(cfg: &crate::config::Config, req: &super::ChatRequest) -> Result<String, serde_json::Error>;
```

The body, for a request with one tool and thinking on:

```json
{"model":"<cfg.infer.model>","messages":[…],"tools":[…],"id_slot":<req.slot>,"cache_prompt":true,
 "stream":true,"return_progress":true,"timings_per_token":true,"reasoning_control":true,
 "max_tokens":<cfg.limits.max_tokens>,"temperature":…,"top_p":…,"top_k":…,
 "chat_template_kwargs":{"enable_thinking":true}}
```

One example of each kind of message, because each is rendered differently:

| `ChatMessage` | JSON |
|---|---|
| `System { content }` | `{"role":"system","content":"…"}` |
| `User { content }` | `{"role":"user","content":"…"}` |
| `Tool { tool_call_id, content }` | `{"role":"tool","tool_call_id":"…","content":"…"}` |
| `Assistant` with text only | `{"role":"assistant","content":"…"}` |
| `Assistant` with reasoning and a tool call, no text | `{"role":"assistant","content":"","reasoning_content":"…","tool_calls":[{"id":"…","type":"function","function":{"name":"…","arguments":"…"}}]}` |

Rules: an assistant `content` of `None` is sent as `""` (the template needs a string).
`reasoning_content` is left out when `None`. `tool_calls` is left out when empty. The top-level
`tools` is left out when empty; each tool is
`{"type":"function","function":{"name","description","parameters"}}`. `arguments` stays the string
it is; do not parse it. The tests compare bodies as JSON values, so key order is yours to choose,
but it must be the same every time: build the body from structs with `#[derive(Serialize)]`, not
from a map.

## API notes (serde 1.0, verified)

`#[serde(skip_serializing_if = "Option::is_none")]` and `#[serde(skip_serializing_if =
"Vec::is_empty")]` leave a field out. A field named `r#type` serializes as `type`.

## Steps

- [ ] **1. Copy.** `git switch m2a`, then
  `cp docs/plans/M2a/files/crates/loopd/tests/request.rs crates/loopd/tests/`
- [ ] **2. See the test fail.** `cargo test -p loopd --test request`. Expected: it does not compile.
- [ ] **3. Write `llama/mod.rs` and `llama/request.rs`,** and add `pub mod llama;` to `lib.rs`.
  Run `cargo fmt --all`.
- [ ] **4. See the test pass.** `cargo test -p loopd --test request`. Expected: `5 passed`.
- [ ] **5. Run the gate.** `make gate`. Expected last line: `gate: ok`.
- [ ] **6. Log and commit.** `git add crates/loopd docs/implementer-log.md && git commit`

## Done when

- `cargo test -p loopd --test request` reports 5 passed; `make gate` prints `gate: ok`.

## Stop and report if

- A name or a derive above cannot be written as given.