diff --git a/README.md b/README.md index 6334d93..03a1bd3 100644 --- a/README.md +++ b/README.md @@ -3,3 +3,14 @@ This is my setup for my [computeblade](https://publish.obsidian.md/ai6ua/Projects/Computing/ComputeBlade) cluster, which will theoretically be here around September 2023. + +See `docs/` for the documentation. The docs are written using Sphinx, so either build +the docs using the provided Makefile or see the documentation link below. The +`install-dependencies` target in the Makefile can install Sphinx. + +### Links + +- [Docs](https://bladerunner-docs.wntrmute.dev/) +- [Source](https://git.wntrmute.dev/kyle/bladerunner) ([Github mirror](https://github.com/kisom/bladerunner)) +- [Second brain project page](https://publish.obsidian.md/ai6ua/Projects/Computing/ComputeBlade) + diff --git a/TODO.md b/TODO.md index 6a4a525..ed169e4 100644 --- a/TODO.md +++ b/TODO.md @@ -5,10 +5,16 @@ - bazel setup - tooling setup +## Dockerfiles + +- move to an OCI setup + ## Provisioning - packer image building - ansible deployment +- initrd/initramfs for auto FDE + - using dracut? ## Secure computing @@ -21,7 +27,4 @@ ## Documentation -- meta - - sphinx docs? - - with bazel integration? - functional specs for everything diff --git a/ansible/roles/base/tasks/main.yml b/ansible/roles/base/tasks/main.yml index e69de29..be082b0 100644 --- a/ansible/roles/base/tasks/main.yml +++ b/ansible/roles/base/tasks/main.yml @@ -0,0 +1,8 @@ +- name: set up TPM tooling + become: true + ansible.builtin.apt: name={{ item }} state=present + with_items: + - clevis-tpm2 + - tpm-tools + - tpm2-tools + - tss2 \ No newline at end of file diff --git a/docs/Makefile b/docs/Makefile index d4bb2cb..a11d659 100644 --- a/docs/Makefile +++ b/docs/Makefile @@ -14,6 +14,9 @@ help: .PHONY: help Makefile +install-dependencies: + /usr/bin/env python3 -m pip install -r requirements.txt + # Catch-all target: route all unknown targets to Sphinx using the new # "make mode" option. $(O) is meant as a shortcut for $(SPHINXOPTS). %: Makefile diff --git a/docs/intro.rst b/docs/intro.rst index 4af7c7c..6c59bd6 100644 --- a/docs/intro.rst +++ b/docs/intro.rst @@ -11,6 +11,8 @@ There are some assumptions made: build system here will assume this. It may work on non-Ubuntu apt-based systems. For non-Debian systems, I've also been working on including container builds that may work. +2. The primary target for this setup is Ubuntu 22.04. This needs to be + validated still. There are three types of systems: @@ -57,10 +59,34 @@ Below is a diagram of the planned system. } -The hardware isn't slated to arrive until September at the earliest. - Hardware -------- -Blades: +The hardware isn't slated to arrive until September at the earliest. I am +leaning towards having the 1TB NVMe drives go with the AI modules, and use +the gateway system as the storage machine if needed. + ++----------------------------+----------+----------------------------------------+ +| Item | Quantity | Notes | ++----------------------------+----------+----------------------------------------+ +| TPM blade | 5 | TPM 2.0 | ++----------------------------+----------+----------------------------------------+ +| DEV blade | 6 | TPM 2.0, µSD, nRPIBOOT | ++----------------------------+----------+----------------------------------------+ +| CM4 | 10 | 8GB RAM, no eMMC/WiFi/BT | ++----------------------------+----------+----------------------------------------+ +| CM4 | 2 | 8 GB RAM, eMMC/WiFi/BT (gw, dev blade) | ++----------------------------+----------+----------------------------------------+ +| SAMSUNG 970 EVO Plus 500GB | 4/7 | 2280 | ++----------------------------+----------+----------------------------------------+ +| SAMSUNG 970 EVO Plus 1 TB | 2/4 | 2280 (1 allocated to gw) | ++----------------------------+----------+----------------------------------------+ +| RTC module | 10 | DS3231 | ++----------------------------+----------+----------------------------------------+ +| AI module | 3 | 2x Coral TPU | ++----------------------------+----------+----------------------------------------+ +| CM4 carrier board | 1 | Dual-homed, NVMe slot, Zymbit 4i | ++----------------------------+----------+----------------------------------------+ +| Netgear GS316PP | 1 | 16-port PoE+ (183W) | ++----------------------------+----------+----------------------------------------+ diff --git a/docs/packer.rst b/docs/packer.rst index 7343845..9b4171d 100644 --- a/docs/packer.rst +++ b/docs/packer.rst @@ -12,6 +12,9 @@ the workflow looks like: 1. ``install-packer.sh`` 2. ``build-image.sh`` +Alternatively, you can run ``run-docker.sh`` to optionally build a container +from the provided Dockerfile and then run it. + ``install-packer.sh`` ^^^^^^^^^^^^^^^^^^^^^^ @@ -34,6 +37,75 @@ The dependencies required to build images with ``packer`` are: Go will also need to be installed; there is a script provided in the :doc:`tools` directory. +If the ``FORCE_DEPENDENCY_INSTALL`` environment variable is set to ``"yes"``, it +will attempt to install the dependencies even if ``git`` is installed. + +``build-image.sh`` +^^^^^^^^^^^^^^^^^^ + +``build-image.sh`` will attempt to build a packer image. It requires that +``packer`` and ``packer-builder-arm`` have been installed, e.g. via +``install-packer.sh``. It will use a board file (see below) to build this +image. If the board file contains both a remote file URL and a local file path, +it will attempt to download the remote file to the local path to cache it. If +the environment variable ``SKIP_LOCAL_CACHE=yes``, it will skip doing this. It +will also skip caching if the local file exists, though it will print a command +to remove the file to force redownloading. + Board files ------------ +A board file is a JSON [#]_ file describing the image that packer should build. +There are a lot of examples in the packer-builder-arm boards_ directory. + +.. [#] Hashicorp would like you to use their HCL, but I haven't switched + over yet. + +``ubuntu-board-gen`` +-------------------- + +A Go program is provided to generate an Ubuntu-based Packer board file from a +YAML file description. It is a single-minded tool to solve an exact problem; +for more control, or to handle edge case, the board JSON file may be +handwritten or another generator written. + +The YAML board specification has the following format: + +.. yaml :: + + boards: + - version: 22.04.2 + size: 32G + name: cm4-cluster-ubuntu-22.04.2.img + scripts: + - scripts/install-base.sh + +It will set up a board file pointing to the preinstalled Ubuntu server image. +The size parameter should be one of "4G", "8G", "16G", "32G", or "64G". The +example above is only using a shell provisioner, but there are many different +provisioners available. A longer example would look like + +.. yaml :: + + boards: + - version: 22.04.2 + size: 4G + name: cm4-cluster-ubuntu-22.04.2.img + local-scripts: + - scripts/generate-auth-keys + - scripts/template-that-one-file + files: + - source: build/privkey.pem + destination: /etc/myservice/privkey.pem + - source: build/cert.pem + destination: /etc/myservice/cert.pem + - source: build/that-one-file + destination: /etc/that-one-file + scripts: + - scripts/set-auth-key-permissions.sh + - scripts/install-base-platform.sh + +The order of precedence for provisioners is local scripts (which might be used +to generate files), files, and then scripts. + +.. _boards: https://github.com/mkaczanowski/packer-builder-arm/tree/master/boards \ No newline at end of file diff --git a/docs/tools.rst b/docs/tools.rst index 0c0fc1b..2ce4133 100644 --- a/docs/tools.rst +++ b/docs/tools.rst @@ -1,4 +1,39 @@ tools ===== -The ``tools`` directory contains various helper scripts. \ No newline at end of file +The ``tools`` directory contains various helper scripts for building systems. + +- ``install.sh`` will run all of the scripts in the expected order. + +- ``install-dependencies.sh`` installs all of the dependencies required by the + various parts of the build system. +- ``install-go.sh`` attempts to install Go using godeb_. It requires some of + the dependencies that would be installed by ``install-dependencies.sh``. +- ``install-bazel.sh`` installs bazelisk_ and buildifier_. It requires Go in + addition to some of the dependencies that would be installed by + ``install-depdencies.sh``. + +Dependencies +------------ + +Optional dependencies are marked with a *?*. + ++-----------------------------+----------------------+------------------------------------------------+ +| Script | Dependencies | Solved by | ++-----------------------------+----------------------+------------------------------------------------+ +| ``install-dependencies.sh`` | apt, sudo? | | ++-----------------------------+----------------------+------------------------------------------------+ +| ``install-go.sh`` | curl, sudo, tar | ``install-dependencies.sh`` | ++-----------------------------+----------------------+------------------------------------------------+ +| ``install-bazel.sh`` | curl, git, go, sudo? | ``install-dependencies.sh``, ``install-go.sh`` | ++-----------------------------+----------------------+------------------------------------------------+ + +Dockerfile +---------- + +The Dockerfile sets up an Ubuntu container and runs the install scripts. Its +entrypoint is ``bash``. + +.. _bazelisk: https://github.com/bazelbuild/bazelisk +.. _buildifier: https://github.com/bazelbuild/buildtools +.. _godeb: https://github.com/niemeyer/godeb diff --git a/packer/build-image.sh b/packer/build-image.sh index dbf6d1b..8307042 100755 --- a/packer/build-image.sh +++ b/packer/build-image.sh @@ -4,17 +4,30 @@ set -euxo pipefail IMAGE_TYPE="${1:-ubuntu}" PACKER_BUILD_FILE="${2:-}" +SKIP_LOCAL_CACHE="${SKIP_LOCAL_CACHE:-no}" errmsg () { echo "$@" > /dev/stderr } +IMAGE_TYPE="${1:-ubuntu}" + preflight () { case "${IMAGE_TYPE}" in - ubuntu) PACKER_BUILD_FILE="boards/cm4-cluster-ubuntu-22.04.2.json" ;; - ## TODO(kyle): look into building a Raspbian version if needed. - # raspbian) PACKER_BUILD_FILE="boards/raspberry-pi/raspios-lite-arm.json" ;; + ubuntu) + PACKER_BUILD_FILE="boards/pi-cm4-ubuntu-22.04.2.json" ;; + if [ "${SKIP_LOCAL_CACHE}" != "yes" ] + then + REMOTE_IMAGE_URL="$(jq '.builders[0].file_urls' boards/pi-cm4-ubuntu-22.04.2.json | grep https | tr -d ' \"')" + fi + custom) + PACKER_BUILD_FILE="${2:-}" + if [ "${SKIP_LOCAL_CACHE}" != "yes" ] + then + REMOTE_IMAGE_URL="$(jq '.builders[0].file_urls' ${PACKER_BUILD_FILE} | grep https | tr -d ' \"')" + fi + if [ -z "${PACKER_BUILD_FILE}" ] then errmsg "[!] custom board requires a board file path" @@ -24,7 +37,6 @@ preflight () { *) errmsg "[!] invalid image type ${IMAGE_TYPE}." errmsg "[!] valid image types are" - # errmsg " - raspbian" errmsg " - ubuntu" errmsg " - custom path/to/board/file" exit 1 @@ -32,10 +44,43 @@ preflight () { esac } +cache_remote_url () { + if [ "${SKIP_LOCAL_CACHE}" != "yes" ] + then + echo "[+] skipping fetch of remote file: SKIP_LOCAL_CACHE=yes" + return 0 + fi + + local CACHED_FILE="$(jq '.builders[0].file_urls' boards/pi-cm4-ubuntu-22.04.2.json | grep -v https | tr -d ' \"')" + if [ -z "${CACHED_FILE}" ] + then + echo "[+] skipping fetch of remote file: no local file provided" + return 0 + fi + + if [ -z "${REMOTE_URL}" ] + then + echo "[+] skipping fetch of remote file: no remote file provided" + return 0 + fi + + if [ -s "${CACHED_FILE}" ] + then + echo "[+] skipping fetch of remote file: file already exists. To force redownloading," + echo " run the following:" + echo "" + echo " rm ${CACHED_FILE}" + return 0 + fi + + curl -fsSL -o "${CACHED_FILE}" "${REMOTE_URL}" +} + build_image () { sudo packer build ${PACKER_BUILD_FILE} } preflight +cache_remote_url build_image diff --git a/packer/install-packer.sh b/packer/install-packer.sh index 918c7a9..be989e3 100755 --- a/packer/install-packer.sh +++ b/packer/install-packer.sh @@ -9,12 +9,12 @@ PACKER_FILE=packer_${PACKER_VERSION}_linux_${ARCH} UPSTREAM="https://github.com/mkaczanowski/packer-builder-arm" UPGRADE="false" BUILD_DIR="$(pwd)/build" -FORCE_DEPENDENCY_INSTALL="no" +FORCE_DEPENDENCY_INSTALL="${FORCE_DEPENDENCY_INSTALL:-no}" prep () { if [ -z "$(command -v git)" -o "${FORCE_DEPENDENCY_INSTALL}" = "yes" ] then - sudo apt-get update && sudo apt-get -y install git unzip qemu-user-static e2fsprogs dosfstools libarchive-tools + sudo apt-get update && sudo apt-get -y install git unzip qemu-user-static e2fsprogs dosfstools libarchive-tools xz-utils jq fi mkdir -p ${BUILD_DIR} pushd ${BUILD_DIR} diff --git a/packer/run-docker.sh b/packer/run-docker.sh index 9438c48..e3c040b 100755 --- a/packer/run-docker.sh +++ b/packer/run-docker.sh @@ -3,9 +3,10 @@ set -euxo pipefail PACKER_IMAGE_NAME="bladerunner/packer:latest" +FORCE_DOCKER_BUILD="${FORCE_DOCKER_BUILD:-no}" preflight () { - if [ -z "$(docker image ls -q ${PACKER_IMAGE_NAME})" ] + if [ "${FORCE_DOCKER_BUILD}" = "yes" -o -z "$(docker image ls -q ${PACKER_IMAGE_NAME})" ] then docker image build -t "${PACKER_IMAGE_NAME}" . fi diff --git a/packer/setup-env.sh b/packer/setup-env.sh deleted file mode 100644 index 08bcd4e..0000000 --- a/packer/setup-env.sh +++ /dev/null @@ -1,34 +0,0 @@ -#!/usr/bin/env bash - -set -euxo pipefail - -IMAGE_TYPE="${1:-ubuntu}" - -select_image () { - case "${IMAGE_TYPE}" in - ubuntu) - PACKER_BUILD_FILE="boards/pi-cm4-ubuntu-22.04.2.json" ;; - REMOTE_IMAGE_URL="$(jq '.builders[0].file_urls' boards/pi-cm4-ubuntu-22.04.2.json | grep https | tr -d ' \"')" - - raspbian) PACKER_BUILD_FILE="boards/raspberry-pi/raspios-lite-arm.json" ;; - PACKER_BUILD_FILE="boards/pi-cm4-ubuntu-22.04.2.json" ;; - REMOTE_IMAGE_URL="$(jq '.builders[0].file_urls' boards/pi-cm4-ubuntu-22.04.2.json | grep https | tr -d ' \"')" - - custom) - PACKER_BUILD_FILE="${2:-}" - if [ -z "${PACKER_BUILD_FILE}" ] - then - errmsg "[!] custom board requires a board file path" - exit 1 - fi - ;; - *) - errmsg "[!] invalid image type ${IMAGE_TYPE}." - errmsg "[!] valid image types are" - errmsg " - raspbian" - errmsg " - ubuntu" - errmsg " - custom path/to/board/file" - exit 1 - ;; - esac -} \ No newline at end of file diff --git a/tools/install-dependencies.sh b/tools/install-dependencies.sh index 45685d8..59b9184 100755 --- a/tools/install-dependencies.sh +++ b/tools/install-dependencies.sh @@ -14,7 +14,9 @@ preflight () { apt_packages () { $SUDO apt-get update - $SUDO apt-get -y install git bash curl sudo build-essential + $SUDO apt-get -y install git bash curl sudo build-essential unzip \ + qemu-user-static e2fsprogs dosfstools \ + libarchive-tools xz-utils } preflight