boxmaker/docs/plans/M3b/04-toolkit-files.md

M3b task 04: the toolkit program, read_file and write_file

Branch: m3b (run git switch m3b; git status --short must be empty, otherwise stop) Commit subject: toolkit: the tool program, read_file and write_file

Goal

toolkit is the one binary inside a tool container. brokerd will run toolkit <tool> with the tool's arguments as JSON on standard input (the proto::tools structs from task 01). toolkit prints what the model should see on standard output and exits 0 (done), 1 (the tool could not do it, for a reason the model should read) or 2 (it was run wrongly). It decides nothing: whether the call may run was decided before the container existed. Spec section 4.

This task makes the program and its first two tools. shell and http_fetch come in tasks 05 and 06; until then they are unknown tools (exit 2).

Files

• Copy: crates/toolkit/tests/support/mod.rs, crates/toolkit/tests/files.rs
• Create: crates/toolkit/src/input.rs, crates/toolkit/src/files.rs
• Replace: crates/toolkit/src/lib.rs, crates/toolkit/src/main.rs (both are stubs today)
• Modify: crates/toolkit/Cargo.toml, docs/dependencies.md, docs/implementer-log.md

Cargo.toml: under [dependencies], after proto.workspace = true, add serde.workspace = true and serde_json.workspace = true, each on its own line. docs/dependencies.md: in the serde and serde_json rows, the "Used by" column becomes `proto`, `brokerd`, `toolkit`.

Interfaces

lib.rs (module doc: the programs that run inside tool containers):

pub mod files;
pub mod input;

#[derive(Debug, Clone, Copy, PartialEq, Eq)]
pub enum Exit { Done, ToolError, Misuse }

impl Exit {
    /// 0, 1, 2. No `as` cast: a `match`.
    pub fn code(self) -> u8;
}

#[derive(Debug, Clone, PartialEq, Eq)]
pub struct Outcome { pub exit: Exit, pub stdout: String, pub stderr: String }

impl Outcome {
    pub fn done(stdout: String) -> Outcome;        // Done, stderr empty
    pub fn tool_error(stdout: String) -> Outcome;  // ToolError, stderr empty
    pub fn misuse(stderr: String) -> Outcome;      // Misuse, stdout EMPTY: the model sees nothing
}

/// Run tool `name` with the arguments on `stdin`.
pub fn run(name: &str, stdin: &mut dyn std::io::Read) -> Outcome;

run:

1. input::read_input(stdin); an error e → Outcome::misuse(format!("toolkit: {e}")).
2. By name: "read_file" → input::parse::<ReadFileArgs>(name, &text) then files::read_file(&args); "write_file" → the same with WriteFileArgs and files::write_file. A parse error e (already a whole line) → Outcome::misuse(e).
3. Any other name → Outcome::misuse(format!("toolkit: unknown tool {name:?}")).

The input is read before the name is looked at, for every name.

input.rs:

pub const MAX_INPUT: usize = 2 * 1024 * 1024;

#[derive(Debug)]
pub enum InputError { TooLarge, NotUtf8, Io(std::io::Error) }
// Display, by hand:
//   TooLarge → "the arguments are larger than 2097152 bytes"   (write it with {MAX_INPUT})
//   NotUtf8  → "the arguments are not UTF-8"
//   Io(e)    → "cannot read the arguments: {e}"

/// All of `stdin`, at most MAX_INPUT bytes, as UTF-8. Read at most MAX_INPUT + 1 bytes
/// (`Read::take`); more than MAX_INPUT is TooLarge.
pub fn read_input(stdin: &mut dyn std::io::Read) -> Result<String, InputError>;

/// Parse `text` as the arguments of tool `name`; the error is the whole line for stderr:
/// "toolkit: {name}: the arguments do not parse: {serde's error}".
pub fn parse<T: serde::de::DeserializeOwned>(name: &str, text: &str) -> Result<T, String>;

files.rs:

pub const MAX_READ: usize = 1024 * 1024;
pub fn read_file(args: &ReadFileArgs) -> Outcome;
pub fn write_file(args: &WriteFileArgs) -> Outcome;

read_file: every exit

Every failure is Outcome::tool_error(format!("read_file: {path}: {why}")), one line, no newline at the end.

1. std::fs::metadata(path) fails with ErrorKind::NotFound → why = no such file.
2. It fails any other way → why = the error's Display text.
3. It is a directory → why = is a directory.
4. Opening fails → why = the error's text.
5. Read at most MAX_READ + 1 bytes (take). Reading fails → the error's text. More than MAX_READ bytes → why = larger than 1048576 bytes (write it with {MAX_READ}). Exactly MAX_READ is fine.
6. Not UTF-8 → why = not UTF-8 text.
7. Otherwise Outcome::done(text): the content exactly, nothing added.

write_file: every exit

Every failure is Outcome::tool_error(format!("write_file: {path}: {why}")).

1. The path's parent is not an existing directory (Path::parent is None, or not is_dir()) → why = the directory does not exist. No directory is ever created.
2. The path is a directory → why = is a directory.
3. std::fs::write(path, content) fails → why = the error's text.
4. Otherwise Outcome::done(format!("wrote {} bytes to {path}", content.len())) (bytes, not characters).

main.rs

Read the arguments with std::env::args_os() (never args(), which panics on non-UTF-8). Skip the program name. If there is exactly one argument, it is the tool name (to_str(), or "" if it is not UTF-8); otherwise the name is "", which run answers as an unknown tool. Call toolkit::run(name, &mut std::io::stdin().lock()), write stdout to standard output and stderr to standard error with write_all (ignore their errors), and return ExitCode::from(outcome.exit.code()).

Steps

• 1. Copy. git switch m3b, then mkdir -p crates/toolkit/tests/support and cp docs/plans/M3b/files/crates/toolkit/tests/support/mod.rs crates/toolkit/tests/support/ and cp docs/plans/M3b/files/crates/toolkit/tests/files.rs crates/toolkit/tests/
• 2. See it fail. cargo test -p toolkit --test files. Expected: it does not compile.
• 3. Write the code and the two manifest and doc changes. Run cargo fmt --all.
• 4. See it pass. cargo test -p toolkit --test files. Expected: 7 passed. One of them returns early and passes if you are root; you should not be.
• 5. Walk the exits. Point at the line of your code for each numbered exit above.
• 6. Run the gate. make gate. Expected last line: gate: ok.
• 7. Log and commit. git add crates/toolkit docs/dependencies.md docs/implementer-log.md Cargo.lock && git commit

Done when

• cargo test -p toolkit --test files reports 7 passed; make gate prints gate: ok.

Stop and report if

• A test wants a misuse to print anything on standard output.
• A test wants write_file to create a directory.