eng-pad/docs/SERVER_DESIGN.md

# eng-pad Server Design

## Overview

The eng-pad server (`eng-pad-server`) receives notebook data from the
Android app via gRPC, stores it, and serves read-only views via a web
UI. Users authenticate with password or FIDO2/U2F keys. Shareable links
allow unauthenticated read-only access to specific notebooks.

This is a standalone service. It may integrate with MCIAS in the future
but does not depend on it initially.

## Architecture

```
Android App                    Server
+-----------+    gRPC/TLS     +------------------+
| eng-pad   |  ------------> | eng-pad-server   |
| (writer)  |   SyncNotebook | (read-only store)|
+-----------+                 +------------------+
                                    |
                              +-----+------+
                              |            |
                          REST API    Web UI (htmx)
                          (JSON)      (SVG rendering)
```

The app is the single writer. The server receives whatever the app
pushes and serves it read-only. No conflict resolution needed.

## Data Model

### SQLite Schema

```sql
-- Users (password + optional FIDO2 keys)
CREATE TABLE users (
    id            INTEGER PRIMARY KEY AUTOINCREMENT,
    username      TEXT NOT NULL UNIQUE,
    password_hash TEXT NOT NULL,       -- Argon2id
    created_at    INTEGER NOT NULL,
    updated_at    INTEGER NOT NULL
);

-- FIDO2/U2F credentials (one user can have many keys)
CREATE TABLE webauthn_credentials (
    id              INTEGER PRIMARY KEY AUTOINCREMENT,
    user_id         INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
    credential_id   BLOB NOT NULL UNIQUE,
    public_key      BLOB NOT NULL,
    name            TEXT NOT NULL,      -- User-assigned label ("YubiKey 5")
    sign_count      INTEGER NOT NULL DEFAULT 0,
    created_at      INTEGER NOT NULL
);

-- Synced notebooks
CREATE TABLE notebooks (
    id            INTEGER PRIMARY KEY AUTOINCREMENT,
    user_id       INTEGER NOT NULL REFERENCES users(id) ON DELETE CASCADE,
    remote_id     INTEGER NOT NULL,    -- ID from the Android app's DB
    title         TEXT NOT NULL,
    page_size     TEXT NOT NULL,        -- "REGULAR" or "LARGE"
    synced_at     INTEGER NOT NULL,
    UNIQUE(user_id, remote_id)
);

-- Synced pages
CREATE TABLE pages (
    id            INTEGER PRIMARY KEY AUTOINCREMENT,
    notebook_id   INTEGER NOT NULL REFERENCES notebooks(id) ON DELETE CASCADE,
    remote_id     INTEGER NOT NULL,    -- ID from the Android app's DB
    page_number   INTEGER NOT NULL,
    UNIQUE(notebook_id, remote_id)
);

-- Synced strokes
CREATE TABLE strokes (
    id            INTEGER PRIMARY KEY AUTOINCREMENT,
    page_id       INTEGER NOT NULL REFERENCES pages(id) ON DELETE CASCADE,
    pen_size      REAL NOT NULL,
    color         INTEGER NOT NULL,
    style         TEXT NOT NULL DEFAULT 'plain',
    point_data    BLOB NOT NULL,       -- Packed float array, same format as app
    stroke_order  INTEGER NOT NULL
);

-- Shareable links
CREATE TABLE share_links (
    id            INTEGER PRIMARY KEY AUTOINCREMENT,
    notebook_id   INTEGER NOT NULL REFERENCES notebooks(id) ON DELETE CASCADE,
    token         TEXT NOT NULL UNIQUE, -- Random URL-safe token
    expires_at    INTEGER,             -- NULL = never expires
    created_at    INTEGER NOT NULL
);

CREATE INDEX idx_notebooks_user ON notebooks(user_id);
CREATE INDEX idx_pages_notebook ON pages(notebook_id);
CREATE INDEX idx_strokes_page ON strokes(page_id);
CREATE INDEX idx_share_links_token ON share_links(token);
CREATE INDEX idx_webauthn_user ON webauthn_credentials(user_id);
```

## gRPC API (Sync)

Proto definitions in `proto/engpad/v1/`.

```protobuf
syntax = "proto3";
package engpad.v1;

import "google/protobuf/timestamp.proto";

service EngPadSync {
    // Push a complete notebook (pages + strokes). Replaces any
    // existing data for this notebook on the server.
    rpc SyncNotebook(SyncNotebookRequest) returns (SyncNotebookResponse);

    // Delete a notebook from the server.
    rpc DeleteNotebook(DeleteNotebookRequest) returns (DeleteNotebookResponse);

    // List notebooks synced by the authenticated user.
    rpc ListNotebooks(ListNotebooksRequest) returns (ListNotebooksResponse);

    // Generate a shareable link for a notebook.
    rpc CreateShareLink(CreateShareLinkRequest) returns (CreateShareLinkResponse);

    // Revoke a shareable link.
    rpc RevokeShareLink(RevokeShareLinkRequest) returns (RevokeShareLinkResponse);

    // List active share links for a notebook.
    rpc ListShareLinks(ListShareLinksRequest) returns (ListShareLinksResponse);
}

message SyncNotebookRequest {
    int64 notebook_id = 1;             // App-side notebook ID
    string title = 2;
    string page_size = 3;              // "REGULAR" or "LARGE"
    repeated PageData pages = 4;
}

message PageData {
    int64 page_id = 1;                 // App-side page ID
    int32 page_number = 2;
    repeated StrokeData strokes = 3;
}

message StrokeData {
    float pen_size = 1;
    int32 color = 2;
    string style = 3;                  // "plain", "dashed", "arrow", "double_arrow"
    bytes point_data = 4;              // Packed little-endian floats
    int32 stroke_order = 5;
}

message SyncNotebookResponse {
    int64 server_notebook_id = 1;
    google.protobuf.Timestamp synced_at = 2;
}

message DeleteNotebookRequest {
    int64 notebook_id = 1;             // App-side notebook ID
}

message DeleteNotebookResponse {}

message ListNotebooksRequest {}

message ListNotebooksResponse {
    repeated NotebookSummary notebooks = 1;
}

message NotebookSummary {
    int64 server_id = 1;
    int64 remote_id = 2;
    string title = 3;
    string page_size = 4;
    int32 page_count = 5;
    google.protobuf.Timestamp synced_at = 6;
}

message CreateShareLinkRequest {
    int64 notebook_id = 1;             // App-side notebook ID
    int64 expires_in_seconds = 2;      // 0 = never expires
}

message CreateShareLinkResponse {
    string token = 1;
    string url = 2;                    // Full shareable URL
    google.protobuf.Timestamp expires_at = 3;  // Not set if never expires
}

message RevokeShareLinkRequest {
    string token = 1;
}

message RevokeShareLinkResponse {}

message ListShareLinksRequest {
    int64 notebook_id = 1;
}

message ListShareLinksResponse {
    repeated ShareLinkInfo links = 1;
}

message ShareLinkInfo {
    string token = 1;
    string url = 2;
    google.protobuf.Timestamp created_at = 3;
    google.protobuf.Timestamp expires_at = 4;
}
```

### Sync Semantics

`SyncNotebook` is an upsert: the server replaces all data for the
given `notebook_id` (keyed by user + app-side ID). The entire notebook
is sent each time — no incremental sync. This is simple and correct
since the app is the sole writer.

For a notebook with ~100 pages and ~500 strokes per page, the payload
is roughly 5-10 MB — manageable for manual sync over WiFi.

## REST API (Web Viewing)

| Method | Path | Auth | Description |
|--------|------|------|-------------|
| `POST` | `/v1/auth/login` | None | Password login, returns bearer token |
| `POST` | `/v1/auth/webauthn/register/begin` | Bearer | Start FIDO2 registration |
| `POST` | `/v1/auth/webauthn/register/finish` | Bearer | Complete FIDO2 registration |
| `POST` | `/v1/auth/webauthn/login/begin` | None | Start FIDO2 login |
| `POST` | `/v1/auth/webauthn/login/finish` | None | Complete FIDO2 login |
| `GET` | `/v1/notebooks` | Bearer | List user's synced notebooks |
| `GET` | `/v1/notebooks/:id` | Bearer | Notebook metadata + page list |
| `GET` | `/v1/notebooks/:id/pages/:num` | Bearer | Page metadata + stroke count |
| `GET` | `/v1/notebooks/:id/pages/:num/svg` | Bearer | Page rendered as SVG |
| `GET` | `/v1/notebooks/:id/pages/:num/jpg` | Bearer | Page rendered as JPG (300 DPI) |
| `GET` | `/v1/notebooks/:id/pdf` | Bearer | Full notebook as PDF |
| `GET` | `/s/:token` | None | Shareable link → notebook view |
| `GET` | `/s/:token/pages/:num/svg` | None | Shareable link → page SVG |
| `GET` | `/s/:token/pages/:num/jpg` | None | Shareable link → page JPG |
| `GET` | `/s/:token/pdf` | None | Shareable link → notebook PDF |

### SVG Rendering

Each page is rendered as an SVG document:

```xml
<svg xmlns="http://www.w3.org/2000/svg"
     viewBox="0 0 612 792"
     width="612" height="792">
  <!-- Coordinates scaled from 300 DPI to 72 DPI (×0.24) for SVG -->
  <path d="M x0 y0 L x1 y1 L x2 y2 ..."
        stroke="black" stroke-width="1.08"
        stroke-linecap="round" stroke-linejoin="round"
        fill="none" />
  <!-- Dashed strokes -->
  <path d="M x0 y0 L x1 y1"
        stroke="black" stroke-width="1.08"
        stroke-dasharray="7.2 4.8"
        fill="none" />
  <!-- Arrow heads as separate paths -->
</svg>
```

SVG coordinates use 72 DPI (PDF points) for consistency with PDF export.
The `viewBox` maps to physical page dimensions. Browsers handle
zoom/pan natively via scroll and pinch.

### JPG and PDF Rendering

Server-side rendering using Go's `image` package (JPG) and a PDF
library like `go-pdf` or `jung-kurt/gofpdf` (PDF). Same rendering
logic as the Android app — scale coordinates by 72/300 for PDF,
render at 300 DPI for JPG.

## Authentication

### gRPC Auth (Android App)

The app sends `username` and `password` in gRPC metadata on every
sync RPC. No separate login step, no token management on the app side.

- TLS required — no plaintext gRPC connections accepted
- A unary interceptor extracts credentials from metadata, verifies
  the password against Argon2id hash in the DB
- If invalid, returns `UNAUTHENTICATED`
- Password stored on the Android device in EncryptedSharedPreferences
  (hardware-backed Android Keystore)

This is simpler than a token-based flow and acceptable because:
- Sync is manual and infrequent
- The password travels over TLS
- The app stores the password in hardware-backed encrypted storage

### Web Auth (Password + Bearer Token)

The web UI uses a traditional login flow:

- `POST /v1/auth/login` with username/password → returns bearer token
- Token stored as `HttpOnly`, `Secure`, `SameSite=Strict` session cookie
- Token validated via SHA-256 keyed lookup with short TTL cache
- Argon2id hashing (memory-hard, tuned to hardware)

### FIDO2/U2F (WebAuthn) — Web UI Only

Users can register multiple security keys after initial password login.
Once registered, keys can be used as an alternative login method for
the web UI. This does not apply to the gRPC sync path.

Implementation via the `go-webauthn/webauthn` library:

1. **Registration flow**:
   - User logs in with password via web UI
   - Calls `POST /v1/auth/webauthn/register/begin` → gets challenge
   - Browser signs challenge with security key
   - Calls `POST /v1/auth/webauthn/register/finish` with attestation
   - Server stores credential in `webauthn_credentials` table

2. **Login flow**:
   - User calls `POST /v1/auth/webauthn/login/begin` with username → gets challenge
   - Browser signs challenge with registered key
   - Calls `POST /v1/auth/webauthn/login/finish` with assertion
   - Server validates, returns bearer token

A user can have any number of registered keys. Each key has a
user-assigned name for identification.

## Shareable Links

Tokens are 32-byte random values encoded as URL-safe base64 (43 chars).
Generated via `crypto/rand`.

- Default expiry: never (0 = no expiry)
- Optional expiry set at creation time
- Expired links return 410 Gone
- Links can be revoked via the gRPC API or web UI
- Each link is scoped to a single notebook

The shareable URL format: `https://pad.metacircular.net/s/{token}`

The web view for shared links shows the notebook title, page navigation,
and export options (SVG view, JPG download, PDF download) — identical
to the authenticated view but without the notebook list or share
management.

## Android App Changes

### Sync Integration

New files:
- `internal/sync/SyncClient.kt` — gRPC client for `EngPadSync`
- `internal/sync/SyncManager.kt` — orchestrates notebook serialization
  and sync

### UI Changes

- Notebook overflow menu: add "Sync to server" option
- Library toolbar: add "Sync all" button
- Settings screen: server URL, credentials
- Sync status indicator on notebook cards (synced/unsynced/error)

### Configuration

Server URL, username, and password stored in Android
EncryptedSharedPreferences (hardware-backed Keystore). Configured
via a Sync settings screen accessible from the library view.
First sync prompts for server URL and credentials if not configured.

## Server Repository Layout

```
eng-pad-server/
├── cmd/
│   └── eng-pad-server/          CLI entry point
├── internal/
│   ├── auth/                    Password + WebAuthn auth
│   ├── config/                  TOML config
│   ├── db/                      SQLite setup, migrations
│   ├── grpcserver/              gRPC sync service
│   ├── server/                  REST API routes
│   ├── render/                  SVG, JPG, PDF rendering
│   ├── share/                   Share link management
│   └── webserver/               htmx web UI
├── proto/engpad/v1/             Protobuf definitions
├── gen/engpad/v1/               Generated Go code
├── web/
│   ├── templates/               HTML templates
│   └── static/                  CSS, htmx
├── deploy/
│   ├── docker/
│   ├── systemd/
│   └── scripts/
├── Makefile
├── buf.yaml
├── CLAUDE.md
├── ARCHITECTURE.md
└── eng-pad-server.toml.example
```

## Configuration

```toml
[server]
listen_addr = ":8443"
grpc_addr   = ":9443"
tls_cert    = "/srv/eng-pad-server/certs/cert.pem"
tls_key     = "/srv/eng-pad-server/certs/key.pem"

[database]
path = "/srv/eng-pad-server/eng-pad-server.db"

[web]
listen_addr   = ":8080"
base_url      = "https://pad.metacircular.net"

[auth]
token_ttl     = "24h"
argon2_memory = 65536
argon2_time   = 3
argon2_threads = 4

[webauthn]
rp_display_name = "Engineering Pad"
rp_id           = "pad.metacircular.net"
rp_origins      = ["https://pad.metacircular.net"]

[log]
level = "info"
```

## Design Decisions

1. **No grid in web view.** The grid is a writing aid on the tablet
   only. SVG, JPG, and PDF renders are all gridless.

2. **No per-page share links for now.** The URL structure supports
   per-page access (`/s/:token/pages/:num`), so adding it later is
   trivial. For now, shared links always scope to the whole notebook.

3. **No server-side page deletion.** The server is a mirror of the
   tablet. `SyncNotebook` replaces all data for a notebook — if a
   page was deleted on the tablet, it disappears from the server on
   the next sync. The server never modifies notebook content.