lots of documentation updates

README.md

@@ -3,3 +3,14 @@
 This is my setup for my
 [computeblade](https://publish.obsidian.md/ai6ua/Projects/Computing/ComputeBlade)
 cluster, which will theoretically be here around September 2023.
+
+See `docs/` for the documentation. The docs are written using Sphinx, so either build
+the docs using the provided Makefile or see the documentation link below. The
+`install-dependencies` target in the Makefile can install Sphinx.
+
+### Links
+
+- [Docs](https://bladerunner-docs.wntrmute.dev/)
+- [Source](https://git.wntrmute.dev/kyle/bladerunner) ([Github mirror](https://github.com/kisom/bladerunner))
+- [Second brain project page](https://publish.obsidian.md/ai6ua/Projects/Computing/ComputeBlade)
+

TODO.md

@@ -5,10 +5,16 @@
 - bazel setup
 - tooling setup
 
+## Dockerfiles
+
+- move to an OCI setup
+
 ## Provisioning
 
 - packer image building
 - ansible deployment
+- initrd/initramfs for auto FDE
+  - using dracut?
 
 ## Secure computing
 
@@ -21,7 +27,4 @@
 
 ## Documentation
 
-- meta
-  - sphinx docs?
-    - with bazel integration?
 - functional specs for everything

ansible/roles/base/tasks/main.yml

@@ -0,0 +1,8 @@
+- name: set up TPM tooling
+  become: true
+  ansible.builtin.apt: name={{ item }} state=present
+  with_items:
+    - clevis-tpm2
+    - tpm-tools
+    - tpm2-tools
+    - tss2

docs/Makefile

@@ -14,6 +14,9 @@ help:
 
 .PHONY: help Makefile
 
+install-dependencies:
+	/usr/bin/env python3 -m pip install -r requirements.txt
+
 # Catch-all target: route all unknown targets to Sphinx using the new
 # "make mode" option.  $(O) is meant as a shortcut for $(SPHINXOPTS).
 %: Makefile

docs/intro.rst

@@ -11,6 +11,8 @@ There are some assumptions made:
    build system here will assume this. It may work on non-Ubuntu apt-based
    systems. For non-Debian systems, I've also been working on including
    container builds that may work.
+2. The primary target for this setup is Ubuntu 22.04. This needs to be 
+   validated still.
 
 There are three types of systems:
 
@@ -57,10 +59,34 @@ Below is a diagram of the planned system.
     }
 
 
-The hardware isn't slated to arrive until September at the earliest.
-
 Hardware
 --------
 
-Blades:
+The hardware isn't slated to arrive until September at the earliest. I am
+leaning towards having the 1TB NVMe drives go with the AI modules, and use
+the gateway system as the storage machine if needed.
+
++----------------------------+----------+----------------------------------------+
+| Item                       | Quantity | Notes                                  |
++----------------------------+----------+----------------------------------------+
+| TPM blade                  | 5        | TPM 2.0                                |
++----------------------------+----------+----------------------------------------+
+| DEV blade                  | 6        | TPM 2.0, µSD, nRPIBOOT                 |
++----------------------------+----------+----------------------------------------+
+| CM4                        | 10       | 8GB RAM, no eMMC/WiFi/BT               |
++----------------------------+----------+----------------------------------------+
+| CM4                        | 2        | 8 GB RAM, eMMC/WiFi/BT (gw, dev blade) |
++----------------------------+----------+----------------------------------------+
+| SAMSUNG 970 EVO Plus 500GB | 4/7      | 2280                                   |
++----------------------------+----------+----------------------------------------+
+| SAMSUNG 970 EVO Plus 1 TB  | 2/4      | 2280 (1 allocated to gw)               |
++----------------------------+----------+----------------------------------------+
+| RTC module                 | 10       | DS3231                                 |
++----------------------------+----------+----------------------------------------+
+| AI module                  | 3        | 2x Coral TPU                           |
++----------------------------+----------+----------------------------------------+
+| CM4 carrier board          | 1        | Dual-homed, NVMe slot, Zymbit 4i       |
++----------------------------+----------+----------------------------------------+
+| Netgear GS316PP            | 1        | 16-port PoE+ (183W)                    |
++----------------------------+----------+----------------------------------------+
 

docs/packer.rst

@@ -12,6 +12,9 @@ the workflow looks like:
 1. ``install-packer.sh``
 2. ``build-image.sh``
 
+Alternatively, you can run ``run-docker.sh`` to optionally build a container
+from the provided Dockerfile and then run it.
+
 ``install-packer.sh``
 ^^^^^^^^^^^^^^^^^^^^^^
 
@@ -34,6 +37,75 @@ The dependencies required to build images with ``packer`` are:
 Go will also need to be installed; there is a script provided in the :doc:`tools`
 directory.
 
+If the ``FORCE_DEPENDENCY_INSTALL`` environment variable is set to ``"yes"``, it
+will attempt to install the dependencies even if ``git`` is installed.
+
+``build-image.sh``
+^^^^^^^^^^^^^^^^^^
+
+``build-image.sh`` will attempt to build a packer image. It requires that
+``packer`` and ``packer-builder-arm`` have been installed, e.g. via
+``install-packer.sh``. It will use a board file (see below) to build this
+image. If the board file contains both a remote file URL and a local file path,
+it will attempt to download the remote file to the local path to cache it. If
+the environment variable ``SKIP_LOCAL_CACHE=yes``, it will skip doing this. It
+will also skip caching if the local file exists, though it will print a command
+to remove the file to force redownloading.
+
 Board files
 ------------
 
+A board file is a JSON [#]_ file describing the image that packer should build.
+There are a lot of examples in the packer-builder-arm boards_ directory.
+
+.. [#] Hashicorp would like you to use their HCL, but I haven't switched
+   over yet.
+
+``ubuntu-board-gen``
+--------------------
+
+A Go program is provided to generate an Ubuntu-based Packer board file from a
+YAML file description. It is a single-minded tool to solve an exact problem;
+for more control, or to handle edge case, the board JSON file may be
+handwritten or another generator written.
+
+The YAML board specification has the following format:
+
+.. yaml ::
+
+    boards:
+      - version: 22.04.2
+        size: 32G
+        name: cm4-cluster-ubuntu-22.04.2.img
+        scripts:
+          - scripts/install-base.sh
+
+It will set up a board file pointing to the preinstalled Ubuntu server image.
+The size parameter should be one of "4G", "8G", "16G", "32G", or "64G". The
+example above is only using a shell provisioner, but there are many different
+provisioners available. A longer example would look like
+
+.. yaml ::
+
+    boards:
+      - version: 22.04.2
+        size: 4G
+        name: cm4-cluster-ubuntu-22.04.2.img
+        local-scripts:
+          - scripts/generate-auth-keys
+          - scripts/template-that-one-file
+        files:
+          - source: build/privkey.pem 
+            destination: /etc/myservice/privkey.pem
+          - source: build/cert.pem 
+            destination: /etc/myservice/cert.pem
+          - source: build/that-one-file
+            destination: /etc/that-one-file
+        scripts:
+          - scripts/set-auth-key-permissions.sh
+          - scripts/install-base-platform.sh      
+
+The order of precedence for provisioners is local scripts (which might be used
+to generate files), files, and then scripts.
+
+.. _boards: https://github.com/mkaczanowski/packer-builder-arm/tree/master/boards

docs/tools.rst

@@ -1,4 +1,39 @@
 tools
 =====
 
-The ``tools`` directory contains various helper scripts.
+The ``tools`` directory contains various helper scripts for building systems.
+
+- ``install.sh`` will run all of the scripts in the expected order.
+
+- ``install-dependencies.sh`` installs all of the dependencies required by the
+  various parts of the build system.
+- ``install-go.sh`` attempts to install Go using godeb_. It requires some of
+  the dependencies that would be installed by ``install-dependencies.sh``.
+- ``install-bazel.sh`` installs bazelisk_ and buildifier_. It requires Go in
+  addition to some of the dependencies that would be installed by
+  ``install-depdencies.sh``.
+
+Dependencies
+------------
+
+Optional dependencies are marked with a *?*.
+
++-----------------------------+----------------------+------------------------------------------------+
+| Script                      | Dependencies         | Solved by                                      |
++-----------------------------+----------------------+------------------------------------------------+
+| ``install-dependencies.sh`` | apt, sudo?           |                                                |
++-----------------------------+----------------------+------------------------------------------------+
+| ``install-go.sh``           | curl, sudo, tar      | ``install-dependencies.sh``                    |
++-----------------------------+----------------------+------------------------------------------------+
+| ``install-bazel.sh``        | curl, git, go, sudo? | ``install-dependencies.sh``, ``install-go.sh`` |
++-----------------------------+----------------------+------------------------------------------------+
+
+Dockerfile
+----------
+
+The Dockerfile sets up an Ubuntu container and runs the install scripts. Its
+entrypoint is ``bash``.
+
+.. _bazelisk: https://github.com/bazelbuild/bazelisk
+.. _buildifier: https://github.com/bazelbuild/buildtools
+.. _godeb: https://github.com/niemeyer/godeb

packer/build-image.sh

@@ -4,17 +4,30 @@ set -euxo pipefail
 
 IMAGE_TYPE="${1:-ubuntu}"
 PACKER_BUILD_FILE="${2:-}"
+SKIP_LOCAL_CACHE="${SKIP_LOCAL_CACHE:-no}"
 
 errmsg () {
     echo "$@" > /dev/stderr
 }
 
+IMAGE_TYPE="${1:-ubuntu}"
+
 preflight () {
     case "${IMAGE_TYPE}" in
-        ubuntu) PACKER_BUILD_FILE="boards/cm4-cluster-ubuntu-22.04.2.json" ;;
+        ubuntu) 
-        ## TODO(kyle): look into building a Raspbian version if needed.
+            PACKER_BUILD_FILE="boards/pi-cm4-ubuntu-22.04.2.json" ;;
-        # raspbian) PACKER_BUILD_FILE="boards/raspberry-pi/raspios-lite-arm.json" ;;
+            if [ "${SKIP_LOCAL_CACHE}" != "yes" ]
+            then
+                REMOTE_IMAGE_URL="$(jq '.builders[0].file_urls' boards/pi-cm4-ubuntu-22.04.2.json | grep https | tr -d ' \"')"
+            fi
+
         custom)
+            PACKER_BUILD_FILE="${2:-}"
+            if [ "${SKIP_LOCAL_CACHE}" != "yes" ]
+            then
+                REMOTE_IMAGE_URL="$(jq '.builders[0].file_urls' ${PACKER_BUILD_FILE} | grep https | tr -d ' \"')"
+            fi
+
             if [ -z "${PACKER_BUILD_FILE}" ]
             then
                 errmsg "[!] custom board requires a board file path"
@@ -24,7 +37,6 @@ preflight () {
         *)
             errmsg "[!] invalid image type ${IMAGE_TYPE}."
             errmsg "[!] valid image types are"
-            # errmsg "    - raspbian"
             errmsg "    - ubuntu"
             errmsg "    - custom path/to/board/file"
             exit 1
@@ -32,10 +44,43 @@ preflight () {
     esac
 }
 
+cache_remote_url () {
+    if [ "${SKIP_LOCAL_CACHE}" != "yes" ]
+    then
+        echo "[+] skipping fetch of remote file: SKIP_LOCAL_CACHE=yes"
+        return 0
+    fi
+
+    local CACHED_FILE="$(jq '.builders[0].file_urls' boards/pi-cm4-ubuntu-22.04.2.json | grep -v https | tr -d ' \"')"
+    if [ -z "${CACHED_FILE}" ]
+    then
+        echo "[+] skipping fetch of remote file: no local file provided"
+        return 0
+    fi
+    
+    if [ -z "${REMOTE_URL}" ]
+    then
+        echo "[+] skipping fetch of remote file: no remote file provided"
+        return 0
+    fi
+
+    if [ -s "${CACHED_FILE}" ]
+    then
+        echo "[+] skipping fetch of remote file: file already exists. To force redownloading,"
+        echo "    run the following:"
+        echo ""
+        echo "        rm ${CACHED_FILE}"
+        return 0
+    fi
+
+    curl -fsSL -o "${CACHED_FILE}" "${REMOTE_URL}" 
+}
+
 build_image () {
     sudo packer build ${PACKER_BUILD_FILE}
 }
 
 preflight
+cache_remote_url
 build_image
 

packer/install-packer.sh

@@ -9,12 +9,12 @@ PACKER_FILE=packer_${PACKER_VERSION}_linux_${ARCH}
 UPSTREAM="https://github.com/mkaczanowski/packer-builder-arm"
 UPGRADE="false"
 BUILD_DIR="$(pwd)/build"
-FORCE_DEPENDENCY_INSTALL="no"
+FORCE_DEPENDENCY_INSTALL="${FORCE_DEPENDENCY_INSTALL:-no}"
 
 prep () {
     if [ -z "$(command -v git)" -o "${FORCE_DEPENDENCY_INSTALL}" = "yes" ]
     then
-        sudo apt-get update && sudo apt-get -y install git unzip qemu-user-static e2fsprogs dosfstools libarchive-tools
+        sudo apt-get update && sudo apt-get -y install git unzip qemu-user-static e2fsprogs dosfstools libarchive-tools xz-utils jq
     fi
     mkdir -p ${BUILD_DIR}
     pushd ${BUILD_DIR}

packer/run-docker.sh

@@ -3,9 +3,10 @@
 set -euxo pipefail
 
 PACKER_IMAGE_NAME="bladerunner/packer:latest"
+FORCE_DOCKER_BUILD="${FORCE_DOCKER_BUILD:-no}"
 
 preflight () {
-    if [ -z "$(docker image ls -q ${PACKER_IMAGE_NAME})" ]
+    if [ "${FORCE_DOCKER_BUILD}" = "yes" -o -z "$(docker image ls -q ${PACKER_IMAGE_NAME})" ]
     then
         docker image build -t "${PACKER_IMAGE_NAME}" .
     fi

packer/setup-env.sh

@@ -1,34 +0,0 @@
-#!/usr/bin/env bash
-
-set -euxo pipefail
-
-IMAGE_TYPE="${1:-ubuntu}"
-
-select_image () {
-    case "${IMAGE_TYPE}" in
-        ubuntu) 
-            PACKER_BUILD_FILE="boards/pi-cm4-ubuntu-22.04.2.json" ;;
-            REMOTE_IMAGE_URL="$(jq '.builders[0].file_urls' boards/pi-cm4-ubuntu-22.04.2.json | grep https | tr -d ' \"')"
-
-        raspbian) PACKER_BUILD_FILE="boards/raspberry-pi/raspios-lite-arm.json" ;;
-            PACKER_BUILD_FILE="boards/pi-cm4-ubuntu-22.04.2.json" ;;
-            REMOTE_IMAGE_URL="$(jq '.builders[0].file_urls' boards/pi-cm4-ubuntu-22.04.2.json | grep https | tr -d ' \"')"
-
-        custom)
-            PACKER_BUILD_FILE="${2:-}"
-            if [ -z "${PACKER_BUILD_FILE}" ]
-            then
-                errmsg "[!] custom board requires a board file path"
-                exit 1
-            fi
-            ;;
-        *)
-            errmsg "[!] invalid image type ${IMAGE_TYPE}."
-            errmsg "[!] valid image types are"
-            errmsg "    - raspbian"
-            errmsg "    - ubuntu"
-            errmsg "    - custom path/to/board/file"
-            exit 1
-            ;;
-    esac
-}

tools/install-dependencies.sh

@@ -14,7 +14,9 @@ preflight () {
 
 apt_packages () {
     $SUDO apt-get update
-    $SUDO apt-get -y install git bash curl sudo build-essential
+    $SUDO apt-get -y install git bash curl sudo build-essential unzip \
+                             qemu-user-static e2fsprogs dosfstools    \
+                             libarchive-tools xz-utils
 }
 
 preflight