Title: Stream 0x03
Slug: stream-0x03
Date: 2022-03-01
Modified: 2022-03-01
Category: 
Tags: stream
Authors: kyle
Summary: Stream notes for tonight's stream.
Status: draft

Tonight's work focused on adding in a mirror between local storage and
a remote S3 (Minio) instance. The basic flow goes something like:

1. Fetch the object described
2. Store the object locally
3. Throw the object ID onto the work queue
4. The work queue pulls an item off the queue.
5. Load the object from local storage.
6. If there's no error, go look for the next item.
7. If the job doesn't have a backoff set, initialize a new backoff.
8. Launch a goroutine that waits as long as the backoff says before
   putting the item back on the work queue.

Next stream (2021/03/03), we'll look at designing the artifact, maybe
working on some of the protobuf definitions.

### References

* [cloudflare/backoff](https://github.com/cloudflare/backoff/) - software
  I wrote at a previous job.
