From b44ac1afd0a50d0db9dab8bbdb4b9bd0b9b54d33 Mon Sep 17 00:00:00 2001 From: Kyle Isom Date: Mon, 28 Feb 2022 21:45:54 -0800 Subject: [PATCH] Stub out drafts for the next two sets of stream notes. --- content/posts/stream-0x03.md | 30 ++++++++++++++++++++++++++++++ content/posts/stream-0x04.md.md | 13 +++++++++++++ 2 files changed, 43 insertions(+) create mode 100644 content/posts/stream-0x03.md create mode 100644 content/posts/stream-0x04.md.md diff --git a/content/posts/stream-0x03.md b/content/posts/stream-0x03.md new file mode 100644 index 0000000..bb0f24f --- /dev/null +++ b/content/posts/stream-0x03.md @@ -0,0 +1,30 @@ +Title: Stream 0x03 +Slug: stream-0x03 +Date: 2022-03-01 +Modified: 2022-03-01 +Category: +Tags: stream +Authors: kyle +Summary: Stream notes for tonight's stream. +Status: draft + +Tonight's work focused on adding in a mirror between local storage and +a remote S3 (Minio) instance. The basic flow goes something like: + +1. Fetch the object described +2. Store the object locally +3. Throw the object ID onto the work queue +4. The work queue pulls an item off the queue. +5. Load the object from local storage. +6. If there's no error, go look for the next item. +7. If the job doesn't have a backoff set, initialize a new backoff. +8. Launch a goroutine that waits as long as the backoff says before + putting the item back on the work queue. + +Next stream (2021/03/03), we'll look at designing the artifact, maybe +working on some of the protobuf definitions. + +### References + +* [cloudflare/backoff](https://github.com/cloudflare/backoff/) - software + I wrote at a previous job. diff --git a/content/posts/stream-0x04.md.md b/content/posts/stream-0x04.md.md new file mode 100644 index 0000000..9c7906e --- /dev/null +++ b/content/posts/stream-0x04.md.md @@ -0,0 +1,13 @@ +Title: stream-0x04.md +Slug: stream-0x04.md +Date: 2022-03-03 +Modified: 2022-03-03 +Category: +Tags: stream +Authors: kyle +Summary: +Status: draft + +In tonight's stream, we looked at building out the artifact data +type. This started with reviewing the artifact spec, then translating +that into protobufs. This also required the SQL schema to be developed.