Skip to content
jzbzPublic

About

Independent, mountable, encrypted Blu-ray backup discs: SquashFS + age + par2, one self-contained image per disc

Resources

Stars

0 stars

Watchers

0 watching

Forks

Repository files navigation

brb — Blu-ray Backup

CI

Independent, mountable, encrypted backup discs.

brb bin-packs a directory tree into disc-sized groups, builds one self-contained SquashFS image per disc, encrypts it with age, computes par2 recovery data over the ciphertext, and wraps each disc in an ISO ready to burn to BD-R, BD-R DL or BDXL — including M-DISC, which is what you want if the discs are meant to outlive the decade.

bin-pack the tree into disc-sized groups
  -> mksquashfs   (one self-contained image per disc)
    -> age        (encrypt the image)
      -> par2     (recovery data over the CIPHERTEXT)
        -> xorriso (one ISO per disc)
          -> BD-R / BD-R DL / BDXL

Every disc is independent. Losing disc 7 loses exactly the files on disc 7; every other disc still restores on its own. The last step of a restore is

mount -o loop,ro,nosuid,nodev disc07.squashfs /mnt

which needs nothing but the Linux kernel — no brb, no Python, no bespoke archive format.

Two programs, one format

brb ships as two implementations with a deliberate split:

Go brb-linux-amd64 / brb-linux-aarch64 writes and reads. Packing, mksquashfs, encryption, par2, ISOs, burning, resume — and every restore command too
bash brb.sh reads only. doctor, ingest, restore, mount, list, index, verify-disc

The Go build is the tool you use day to day. The bash script exists for one reason: a restore fifteen years from now, by someone holding a disc and no particular reason to trust an 8 MB binary. It is one file — wc -l brb.sh on the disc is the authority on how long, because a number written into prose drifts and this one did twice before the number came out — and it can be read end to end in an afternoon, so the answer to "what is this going to do to my bytes" is available without running anything.

brb.sh refuses backup, plan, burn, iso and init-key by name, and says where they went, rather than failing as an unknown command.

The two are held to the same on-disc format by xcompat-test.sh, which builds a set with the Go build and reads it with both, and asserts they produce identical answers — down to the encrypted index and the byte-for-byte restored tree.


Requirements

The distinction that matters is not between the two programs but between writing discs and reading them. Writing needs the most tooling and you need it today. Reading is what someone needs in fifteen years, and it is deliberately tiny: age, par2, unsquashfs — and for the final step, nothing but the Linux kernel. Every member of that set is a small, specified, widely-implemented format that outlives whoever wrote this.

Run doctor on either program at any time; each checks the dependencies it actually uses and reports what it found.

What each dependency is for

The Go build, which writes and reads:

Binary Package (Fedora) Needed for
mksquashfs squashfs-tools ≥ 4.5 building each disc image — backup only
unsquashfs squashfs-tools extracting and listing an image — restore
par2 par2cmdline parity over the ciphertext, and repair — both
xorriso xorriso building and burning ISOs — backup and burn

It does not need the age binary: encryption uses filippo.io/age as a library, so the format is exactly the same and one fewer thing has to be installed.

The bash reader (brb.sh), which restores:

Binary Package (Fedora) Needed for
age age decrypting an image
par2 par2cmdline repairing a rotted disc
unsquashfs squashfs-tools extracting and listing
sha512sum, stat, cp, cut, sort, tr, dd, df, readlink coreutils hashing, accounting, and the destination symlink check
gzip / gunzip gzip reading the encrypted index
find, awk, sed findutils, gawk, sed locating and formatting
bash ≥ 4.2 bash the script itself. The newest constructs in it are mapfile -t (4.0) and declare -g (4.2); it uses no mapfile -d, no ${var@Q}, no namerefs or negative subscripts, and every expansion of an array that may be empty is guarded, so set -u on a bash older than 4.4 does not trip it either

No python3 for either — the bin-packer that once needed it is Go code now.

And to build the Go program from source: Go 1.26 or newer. Nothing else needs it, and a restore never does.

Optional, but worth having:

Binary Package (Fedora) Why
ddrescue ddrescue salvages partially readable discs; cp stops at the first I/O error, ddrescue does not — this is what makes par2 usable on a scratched disc
udisksctl udisks2 lets verify-disc and ingest mount the drive for you
eject, findmnt eject, util-linux disc swapping during ingest
flock util-linux lets brb.sh tell whether another brb is already using the staging directory; the Go build uses flock(2) directly and needs no program

age-keygen is deliberately not on that list. brb init-key generates both the primary and the rescue keypair with the age library it already links, so neither implementation ever execs it.

Install

Fedora / RHEL / CentOS

sudo dnf install age squashfs-tools par2cmdline xorriso findutils gawk coreutils gzip
sudo dnf install ddrescue udisks2 eject util-linux

Debian / Ubuntu

sudo apt install age squashfs-tools par2 xorriso findutils gawk coreutils gzip
sudo apt install gddrescue udisks2 eject util-linux

Note the package name differences from Fedora: par2cmdline is par2, and ddrescue is gddrescue (the ddrescue package on Debian is a different, unrelated program).

Arch

sudo pacman -S age squashfs-tools par2cmdline libisoburn gawk coreutils gzip
sudo pacman -S ddrescue udisks2 util-linux

xorriso is provided by libisoburn on Arch.

Version requirements worth checking

  • squashfs-tools must be 4.5 or newer. brb feeds mksquashfs a NUL-delimited file list via -cpiostyle0, which does not exist in 4.4. Fedora 36+, Debian 12+ and Ubuntu 22.04+ ship 4.5 or newer; Debian 11 and Ubuntu 20.04 ship 4.4 and will not work. brb doctor tests for the flag directly rather than parsing a version string, so trust it over this table.
  • age is packaged on Fedora, Debian 12+, Ubuntu 22.04+ and Arch. If your distribution does not carry it, take a static binary from the releases page — it is a single file with no runtime dependencies.
  • Go 1.26+ only if you build the Go program from source. A prebuilt static binary from any disc needs nothing at all.

Install brb itself

The Go build is the one to install — it is the only one that can write a set:

cd go && go build -o ~/.local/bin/brb ./cmd/brb

go install does not work, and no spelling of it does: go/go.mod declares the module path as github.com/jzbz/brb while the module itself lives in the repository's go/ subdirectory, so the proxy either resolves the module and finds no package there, or resolves .../brb/go/cmd/brb and rejects the contradictory declaration. Build from a checkout as above, or take a binary.

Or take a prebuilt static binary from any disc (brb-linux-amd64 / brb-linux-aarch64, matching uname -m) — they need no libc, no interpreter and no shared libraries.

The bash reader is a single self-contained script with nothing to build:

install -Dm755 brb.sh ~/.local/bin/brb.sh

Install it under a distinct name. Both name themselves after however they were invoked, so installing both as brb would make their help text and every suggested command ambiguous — and one of them cannot back up.

Verifying a download

A release carries the two static binaries, the bash script and the source tarball, a SHA256SUMS listing all four, and a SHA256SUMS.asc signing that list. Fetch the signing key once, from GitHub:

curl -sL https://github.com/jzbz.gpg | gpg --import

or from a keyserver, which is the better of the two — it does not come from the same host as the release:

gpg --locate-keys jz@jz.bz

Either way the fingerprint below is what to trust, not where you got it. Then check the signature before the files:

gpg --verify SHA256SUMS.asc SHA256SUMS && sha256sum -c --ignore-missing SHA256SUMS

gpg --verify has to report a Good signature from 252B 901C 8885 3CF9 F939 2559 2497 38C8 641C 3359; any other key, or none, and the rest is meaningless. --ignore-missing checks whichever of the four you actually downloaded and stays quiet about the rest.

A freshly imported key also draws "WARNING: This key is not certified with a trusted signature". That is expected and is not a failed check: it says the key carries no web-of-trust path from anything you already trust, which a key you just fetched never does. The signature is still good. Compare the fingerprint gpg prints against the one above and move on, or sign the key locally (gpg --lsign-key jz@jz.bz) to silence it on later releases.

The order is the whole point. SHA256SUMS sits in the same release as the files it describes, so on its own it catches a truncated download and nothing else — anyone able to replace a binary could replace the list beside it just as easily. The signature is what turns it into a check, and it is made by hand: the key never goes near CI, so a compromised workflow can publish a binary but cannot sign for one.

None of this is the SHA512SUMS a disc carries. That file is written at backup time over the files of that one disc, is what verify-disc and ingest check a disc's files against (and sha512sum -c by hand), and nothing signs it — a disc is in your hand, and the question it answers is whether the plastic still reads back, not who published it.


Building the disc payload

Every disc carries the tool itself, three ways: the bash script, a static Go binary for each of the two architectures worth caring about, and the complete source. A restorer in fifteen years needs none of it — the manual restore path is four standard commands — but having it costs about 10 MiB out of a 100 MiB reserve, so there is no reason not to.

./build-dist.sh

That cross-compiles brb-linux-amd64 and brb-linux-aarch64 (static, CGO_ENABLED=0, so no libc and no shared libraries), vendors every Go dependency, and packs brb-src.tar.gz. It writes to a directory you name, deliberately outside the working tree — build output does not belong in the source:

./build-dist.sh /path/to/dist

With no argument it uses $BRB_DIST_OUT, falling back to ~/brb-dist.

Then point brb at the result, either way:

export BRB_DIST_DIR=/path/to/dist
ln -sfn /path/to/dist ./dist

brb also looks in /usr/local/share/brb and /usr/share/brb. If it finds nothing it warns and burns discs carrying only a copy of the binary that is running (brb-linux-amd64 on x86-64) — a missing payload never fails a backup. brb doctor reports exactly what it found.

Building the payload needs a Go toolchain (1.26+); nothing else in brb does, and a restore never does.

The source tarball vendors its dependencies, so it rebuilds with no network:

tar xzf brb-src.tar.gz && cd brb-*/go && go build -mod=vendor ./cmd/brb

That leaves the binary at go/brb, inside the tree, and build-dist.sh will not run from that tree until it is moved out. The script copies go/ whole into brb-src.tar.gz, so it refuses to run while go/ holds anything that is not source — a stray binary, profile or key there would otherwise ride along in plaintext on every disc. In a git checkout that means any file git does not track, ignored ones included, apart from the go/vendor the script rewrites itself; in an unpacked tarball, anything but *.go, *.tmpl, go.mod, go.sum and go/vendor. It names each file it refuses.

Quick start

Everything here is the Go build — it is the one that writes. brb.sh can run the last two steps (ingest, restore) and nothing before them.

brb doctor
brb init-key

init-key writes ~/.config/brb/identity.txt (mode 0400) and appends its public key to ~/.config/brb/recipients.txt. Back that identity file up somewhere that is not these discs — a password manager, and printed on paper. Losing it means losing every backup, permanently and irrecoverably.

Optionally, give yourself a second way in:

brb init-key --rescue-key

That adds a second recipient whose identity is kept encrypted under a passphrase, so the set restores with either the key file or something you remember. See Keys, and the rescue key for what it does and why it is built the way it is. It can be run later, on an archive that already exists — discs built from that point on carry both keys. It asks for the passphrase on the terminal, so it refuses to run under --yes.

brb plan

plan scans and reports how many discs the archive needs, without building anything. Use it to sanity-check before committing to a multi-hour run.

brb backup
brb burn all
brb verify-disc 1
brb restore /tmp/testrestore

Do that last step at least once, before you trust the set with anything. A backup you have never restored is a hypothesis, not a backup.

Then wipe staging, which held plaintext:

rm -rf /var/tmp/brb

Commands

Writing a set — the Go build only. brb.sh refuses these by name and points here.

Command What it does
brb init-key generate an age keypair and recipients file
brb init-key --rescue-key additionally add a second recipient whose identity is passphrase-protected. It asks for the passphrase on the terminal, so it is refused under --yes
brb plan scan and show the disc layout without building anything
brb backup build, encrypt, protect and image every disc. --verify-roundtrip decrypts each image back and compares hashes before the plaintext is deleted — it needs a readable, unencrypted AGE_IDENTITY and refuses to start without one
brb burn <n|n-m|n-|all> burn ISOs, confirming before each; builds each ISO if it is missing and removes it again after a successful burn, unless there is no disc directory to rebuild it from, when it is kept
brb iso <n|n-m|n-|all> build ISO images without burning, for burning elsewhere

Reading a set — either implementation. Substitute brb.sh for brb below and a set restores identically: same files, same bytes, same refusals. That is what xcompat-test.sh proves, and it is the only equivalence claimed here.

The two are separate programs, and they differ around the edges. Everything in this table has been checked by running both; it is what is known to differ, not a promise that nothing else does.

Where brb (Go) brb.sh
version a command not a command — exits 1, "unknown command"
usage errors exit 2, including <command> --help exit 1
doctor checks backup readiness; exits 1 with an identity but no recipients file checks restore readiness; exits 0 on the same config
flags after -- left alone: brb index -- -y searches for -y -y and -c are stripped from anywhere on the line, so the same command searches for something else
$'...', $"...", $-, $[...] in a config refused, naming the line: they cannot be expanded faithfully outside a shell, and $- not at all expanded, because the file is sourced — so a config using any of them is one the Go build will not load
a backslash-newline in a config straight after a $, or one that runs $HOME on into a longer name refused, naming the line the lines are joined first: $ and then HOME on the next line is $HOME, and $HOME and then X is the variable HOMEX, fatal under set -u when it is unset
KEY= value in a config — a space after the = refused, naming the line an empty KEY, and value run as a command: usually an error, but STAGING= : quietly keeps the default
~ in a config other than alone or before a /: ~user, ~"...", ~#, or after a : in a value refused, naming the line the shell's rules: ~user is that user's home if there is one, a quote stops the expansion, an unquoted : starts another
a leading ~ in a config with HOME unset refused, naming the line resolved from the password database — but only when BRB_CONFIG and AGE_RECIPIENTS_FILE are both exported; otherwise brb.sh dies on HOME: unbound variable before it reads the file, -c or not
HOME set but empty treated as unset: $HOME and a leading ~ in a config are refused, naming the line, and so is a run that needs a home-derived default (the config path, AGE_RECIPIENTS_FILE) the empty string: $HOME/brb is /brb, and the defaults are looked for under /.config/brb
text glued to an array's ), as in PRUNE_DIRS=( a )#x refused, naming the line the string (a)#x; only the Go build reads these keys
a config path that stat cannot answer for any reason but a missing file: a directory that cannot be searched, a component that is a regular file, a symlink loop refused, with the error: "permission denied", "not a directory", "too many levels of symbolic links" -c: "config file not found"; BRB_CONFIG: "no config file", and the defaults are used — test -e cannot tell any of these from a missing file
help, or no command at all, with a config brb.sh cannot load: a -c that names nothing, a path that is not a regular file, a bash syntax error prints help on the built-in defaults, as for any config it cannot load stops with the config error
ingest with no terminal runs unattended prompts on /dev/tty between discs even with a mount path and --yes, and fails outright without one
a disc ingest refuses — another set's index or published key, a missing data/, or one with no file at its top level refused before anything from it is staged; interactively the run offers another disc and exits non-zero at the end, under --yes it exits non-zero at once refused before anything from it is staged; the run ends there
a disc whose own copy of the index cannot be read back stages the files that sort before the index — its image — then fails that disc at the index refused as another set's, with nothing staged, when nothing on the disc records the index's hash; when something does, the copy is taken for rot and the rest of the disc is ingested
verify-disc, a SHA512SUMS naming an absolute path or a .. component reported as a mismatch, with the par2 hint refused outright as a checksum file brb did not write; the exit status is the same
verify-disc with no ARCHIVE_NAME in the config or the environment compares the disc's archive name with the one it derives for a backup, <basename of SOURCE_DIR>-<today>, so it warns that nearly every real disc "belongs to archive …" — advice only, the exit status is unaffected says nothing: it compares only an ARCHIVE_NAME that is set
ingest, a disc whose SHA512SUMS has a line that does not parse (a mangled digest, a BSD-style line, leading whitespace) no copy is checked against it, with a warning that line is skipped and the lines that do parse are still used, so every other copy is still checked
--only repeat the flag per path as documented below
case in an index pattern folded by Unicode rules: ÉTÉ finds Été.txt in any locale folded by grep -i in the current locale: ASCII only under C/POSIX (and, going by its source, on busybox in any locale), so ÉTÉ finds nothing there, and under UTF-8 not quite Go's either (longss finds longſs.txt). Both match the pattern as a literal substring, not a regular expression

KEEP_IMAGES and ASSUME_YES used to belong on that list and no longer do: both readers now take them from the config file, so one file drives both.

Command What it does
brb doctor check dependencies and report which key a restore would use
brb ingest [mount] copy discs back onto disk, prompting for each, any order
brb restore <dest> [opts] repair, decrypt and extract, overwriting what is already in <dest> — see below
brb mount <n> <mountpoint> decrypt one disc's image and mount it read-only
brb list <n> list the contents of one disc's image
brb index [pattern] which disc holds a given path
brb verify-disc <n> [mount] read a burned disc back and check every hash
brb help full help text

Global flags, before the command: --yes / -y to skip confirmations and -c CONFIG to point at an alternate config file, on both; --no-color on the Go build only. -c naming a file that does not exist is an error on both, not a silent fall-through to the defaults — a typo in the path must not quietly run the command against a different staging directory. Per-command flags go after the command, and are recognised only where they are documented — brb --resume backup is a usage error, brb backup --resume is the resume.

restore takes --only <path-in-archive> to extract a single path, and --disc <n> to restore only one disc. --only is repeatable on the Go build and every path given is extracted; brb.sh takes one path and a second --only silently replaces the first. The Go build additionally takes --keep-images, which is KEEP_IMAGES=1 for one run.

Two details of restore are worth knowing on both implementations. If unsquashfs finishes with exit status 2 — it extracted everything but hit non-fatal errors, typically an xattr or a special file the destination filesystem will not take — the restore warns, keeps that image's unsquashfs log under $STAGING/restore, and continues with the next disc; exit 1 still stops it. And a decrypted image already sitting in $STAGING/restore (left by KEEP_IMAGES=1, or by an interrupted run) is not trusted on sight: it is verified against the recorded plaintext hash before it is reused, and a copy whose hash disagrees is not reused — the image is decrypted again from the ciphertext instead.

restore overwrites the destination

restore runs unsquashfs -f for every image, because discs 2..N extract into a tree disc 1 already populated. Into a directory that has contents of its own that means existing files are replaced by the backup's versions, mode and mtime included. Both implementations guard it the same way, and both guards run for --only and --disc too:

  • A <dest> that is not empty is named, its first few entries listed, and the overwrite confirmed before anything is extracted. --yes answers that confirmation, so a --yes restore into a live directory overwrites it without asking. Answering no aborts with "restore into an empty directory and merge by hand".
  • A <dest> that holds a symlink resolving to a directory — at any depth — is refused outright, because unsquashfs -f follows such a link and writes the backup's files outside the destination. --yes cannot wave this one through; remove the link, or restore somewhere empty. A symlink to a file is harmless and is left alone.

Resuming an interrupted backup

A twenty-disc set takes days to build. backup records its progress in $STAGING/state.json after every completed disc, so an interruption — a reboot, a full disk, a Ctrl-C — costs only the disc that was in flight:

brb backup --resume

--resume is a flag of backup, so it goes after the command; brb --resume backup is rejected as an unknown global flag.

It picks up at the disc after the last complete one. The state file lists every path already written to a disc, and the learned pack ratios every finished disc measured, so the resumed run re-scans the source tree and then skips exactly what is already on a disc — files added since the run started land on later discs, files deleted since are simply absent, and the run warns when the tree's measured size has changed. ARCHIVE_NAME and SOURCE_DIR must match the interrupted run, or the resume stops rather than write two different trees into one set. Without --resume, a backup that finds finished discs in staging refuses to start rather than overwrite days of work.

One change to the tree is refused: a path a finished disc holds that has since changed kind — a directory moved elsewhere and symlinked back to free space, a file replaced by a directory. The finished discs cannot be rewritten, so the set would hold that path as one kind on them and as another on every later disc, and a restore cannot reliably lay one over the other. The resume names each path with what it was and what it is now; put them back, or start the set over. A resume with nothing left to write, interrupted after its last disc, only warns, because no further disc carries the change — unless the path is now a file, which would itself need one.

The resume also refuses anything that would make the finished discs and the ones still to come disagree about what set they belong to. The state file records the recipients the first disc was encrypted to, the disc geometry (DISC_TYPE, the capacity, RESERVE_BYTES, PAR2_REDUNDANCY) and whether the set is a public archive, and a --resume whose configuration differs in any of those stops with a message naming the setting rather than continuing. Each of those would otherwise produce a set that looks whole and is not: a changed recipients file yields later discs your key cannot open, a changed geometry yields images planned for one medium and burned to another, and a public archive that turns ordinary mid-set — or the reverse — yields discs that disagree about whether the key is on them. Change the setting back, or clear staging and start the set again.

ISOs

By default (ISO_MODE=ondemand) no ISOs are built during backup. burn images each disc at the moment it goes into the drive and deletes the ISO once it is written, which keeps staging near the size of the compressed set instead of roughly 2.2x it for the whole length of a burn campaign — an ISO is a full second copy of its disc directory. Set KEEP_ISOS=1 to keep them, or ISO_MODE=eager for the old behaviour of building every ISO before the first burn. To materialise them as files without burning — to take to another machine, or a different burner — use brb iso all. On that other machine, with ISOs and no disc directories, burn keeps each ISO after burning it whatever KEEP_ISOS says, since there is nothing to rebuild it from.


Configuration

Config lives at ~/.config/brb/config (override with BRB_CONFIG). It is a plain list of assignments, and every setting can equally be given as an environment variable.

Nearly everything here is a writer setting, read only by the Go build. How a set was built — source tree, disc geometry, compression, pack ratio, par2 parameters, ISO mode — was decided when it was written and is recorded in MANIFEST.txt on every disc. The bash reader ignores all of it and uses only STAGING, AGE_IDENTITY, AGE_RECIPIENTS_FILE, BURNER, KEEP_IMAGES and ASSUME_YES (and, if a writer config happens to set ARCHIVE_NAME, verify-disc warns when a disc names a different archive).

brb.sh sources the config file — it executes it as bash. Anything in it runs with your privileges, not just KEY=value lines: a config containing rm -rf ~ deletes your home directory the moment any brb.sh command loads it. So trust the file exactly as much as you trust brb.sh itself. Never point -c or BRB_CONFIG at a file you did not write — least of all one carried on a disc, which is data from wherever that disc has been. The script's own header and brb.sh help say the same thing.

The Go build does not source anything: it parses the file, accepts only KEY=value (and the two array settings), and reports anything else as a syntax error naming the line. Pointing it at the same hostile file is an error message, not an execution.

One config can drive both readers, with care. Every key brb.sh reads is one the Go build knows, and both readers take KEEP_IMAGES and ASSUME_YES from the file. So mind what ASSUME_YES=1 in a shared config does: it answers every confirmation, from either reader, including the one in front of a restore that overwrites its destination. And anything in the file the Go build cannot accept still stops it where brb.sh carries on. The Go build validates the whole file, writer keys included, on every command but help and version — an index or a restore as much as a backup — and refuses to run at all, naming the line. That covers a key it does not know: brb.sh sources the file, so an unknown key is simply a variable nobody reads. It covers a value it cannot read for a key it does know, even a writer key brb.sh never looks at: PACK_RATIO=abc or DISC_TYPE=floppy is invisible to brb.sh and fatal to the Go build. And it covers shell beyond a plain assignment — any command, and the shell-only syntax listed in the table of reader differences under Commands — which brb.sh runs or expands. Booleans, on the other hand, do travel: both readers take 1/0, true/false, yes/no and on/off, in any case, so a value written for one is not misread by the other. A spelling neither of them can read as a boolean stops the Go build at load, naming the key and the line, rather than being quietly taken as false.

SOURCE_DIR=/home/you
STAGING=/var/tmp/brb
DISC_TYPE=bd25                  # bd25 | bd50 | bdxl100 | bdxl128 (M-DISC uses the same value)
DISC_CAPACITY_BYTES=            # override for unusual media
COMPRESSION=zstd                # zstd | xz | gzip | lz4 | lzo | none
COMPRESSION_LEVEL=19            # zstd 1-22, gzip/lzo 1-9; xz, lz4 and none ignore it
BLOCK_SIZE=1M
PACK_RATIO=1.00                 # expected compressed/raw; lower = fuller discs
PAR2_REDUNDANCY=10
BURNER=/dev/sr0
BURN_SPEED=4
LABEL_PREFIX=BACKUP
ISO_MODE=ondemand               # ondemand | eager — see "ISOs" above
KEEP_ISOS=0                     # 1 keeps each ISO after a successful burn
AGE_RECIPIENTS_FILE=~/.config/brb/recipients.txt
AGE_IDENTITY=~/.config/brb/identity.txt      # restore only; see below
DIST_DIR=                       # copies of brb for every disc; empty = auto-locate

KEEP_IMAGES=0                   # reader: 1 keeps each decrypted image

PRUNE_DIRS=( ".cache" ".local/share/Trash" "snap" )   # paths relative to SOURCE_DIR
EXCLUDE_MASKS=( "*.pyc" "core.[0-9]*" )               # filename patterns

That is the useful subset, not the whole set of keys. Since the Go build rejects a key it does not know, take the authoritative list from brb help, which prints every setting with the value actually in force.

A few rules hold on both readers. KEY= with nothing after it means "not set" and leaves the default in force: that is how the sample above leaves DISC_CAPACITY_BYTES and DIST_DIR alone, and STAGING= is /var/tmp/brb to both. (The two lists are the exception: an empty PRUNE_DIRS or EXCLUDE_MASKS switches its defaults off.) -c and BRB_CONFIG must name a regular file or a symlink to one; a directory, a FIFO, a device or a <(...) pipe is refused rather than read or skipped (by every command but the Go build's help, which prints on the built-in defaults instead, and its version, which reads no config). And an environment value that carries a literal ~/ or $HOME — from a systemd Environment= line or an env file, where no shell expanded it — has it expanded when HOME is set, so STAGING=~/brb there names the same directory to both.

Note the mask is core.[0-9]*, not core. A bare core matches every directory named core/ as well as the dump files, and a matching directory is pruned whole — its contents vanish from the backup without appearing even in the disc's directory skeleton, so a restored tree shows no sign anything is missing. Any Go, Drupal or kernel checkout would lose its core/. This was a real bug; the default now matches the dumps glibc actually writes and leaves directories alone. Directory-shaped exclusions belong in PRUNE_DIRS.

DIST_DIR is the disc payload described under Building the disc payload. In the environment it is spelled BRB_DIST_DIR; left empty, brb looks beside itself for a dist directory, then in /usr/local/share/brb and /usr/share/brb. Setting it to a directory that is not there is reported rather than ignored, and a payload that cannot be found never fails a backup — the discs simply carry fewer copies of the tool.

AGE_IDENTITY is only read by the restore side (restore, mount, list, index), by doctor's round-trip check, and by backup --verify-roundtrip. A plain backup never touches it: it encrypts to public keys and needs no secret at all. Left empty, brb looks next to the recipients file for identity.txt, then identity.txt.age, then rescue-identity.txt.age, and uses the first one it finds — so a machine whose plaintext identity has been shredded, or lost, keeps working with no configuration change. An age-encrypted identity works anywhere a plain one does; brb asks for the passphrase once per command, not once per disc.

Setting PRUNE_DIRS or EXCLUDE_MASKS in the config replaces the built-in defaults rather than adding to them. The defaults prune the usual regenerable caches: .cache, Trash, Steam, .var/app, snap, .npm/_cacache, .cargo/registry, .rustup/toolchains, .gradle/caches, .m2/repository, go/pkg/mod, container and Docker storage, and Vagrant boxes.

Disc capacities

DISC_TYPE Media Raw capacity Usable image budget
bd25 BD-R single layer 23.31 GiB 20.49 GiB
bd50 BD-R DL dual layer 46.61 GiB 41.07 GiB
bdxl100 BDXL triple layer 93.23 GiB 82.22 GiB
bdxl128 BDXL quad layer 119.21 GiB 105.16 GiB

The image budget is what is left after reserving 2% for ISO 9660 overhead, RESERVE_BYTES (100 MiB by default) for the plaintext files carried on every disc, and room for the par2 recovery data. Budgets above assume the default PAR2_REDUNDANCY=10; raising redundancy shrinks the budget proportionally.

M-DISC, and what media to actually buy

If the point of this exercise is long-term storage, buy M-DISC BD-R.

Ordinary recordable Blu-ray writes to an organic dye layer, which is a consumable: it degrades with heat, humidity and light, and the disc's lifetime is the dye's lifetime. M-DISC records into an inorganic, heat-resistant layer instead — the bit pattern is physically etched rather than chemically stained, so there is no dye left to fade. That is the whole difference, and it is a real one.

It needs no special support from brb or from your drive. M-DISC BD-R is ordinary BD-R as far as the format is concerned, so any Blu-ray writer burns it and any Blu-ray reader reads it. (This is unlike M-DISC DVD, which needed an "M-Ready" burner — a distinction that still confuses people shopping for media.) Nothing in this tool changes: use the same DISC_TYPE as the equivalent conventional disc.

DISC_TYPE M-DISC available?
bd25 yes — M-DISC BD-R 25 GB
bd50 yes — M-DISC BD-R DL 50 GB
bdxl100 yes — M-DISC BDXL 100 GB
bdxl128 no — quad-layer 128 GB is conventional BD-R XL only

On the longevity numbers, be skeptical. You will see "1,000 years" quoted. That figure is an extrapolation from accelerated-aging tests, not an observation — nobody has had one of these for a thousand years, and independent testing has been less enthusiastic than the marketing. What is defensible is the physical argument: an inorganic recording layer has no organic dye to break down, and that removes the failure mode that kills ordinary BD-R sitting in a warm cupboard. Treat M-DISC as clearly better than conventional BD-R for archival use, and treat any specific century count as a number somebody wanted to sell you.

Two practical notes:

  • Burn slower than you think. BURN_SPEED=4 is the default here and is a reasonable ceiling for M-DISC; 2x is a defensible choice for a set you intend to keep. Write speed is not where you want to economise on a disc you are buying for its lifetime.
  • It costs more per gigabyte. That is the trade. For a set you plan to re-cut every few years, conventional BD-R is fine; for the set you want to still read in thirty years, the media is the cheapest part of the exercise.

None of this changes the honest caveat that the on-disc README already makes to whoever finds these later: the realistic failure mode is drive availability, not disc decay. M-DISC buys you a disc that outlasts the drives, which makes "can I still source a Blu-ray reader" the question that actually decides whether the archive survives. Plan for that too — and test your restores.

About PACK_RATIO

Discs are packed by uncompressed size, so brb has to guess how well the content will compress before it compresses it. PACK_RATIO is that guess, expressed as compressed ÷ raw.

The default of 1.00 assumes no compression at all. That is always safe but leaves discs partly empty whenever the content actually is compressible. If your first run reports images compressing to, say, 0.62 of raw, set PACK_RATIO=0.65 and re-run for fuller discs.

You are not required to get this right. If an image overshoots its budget, brb measures the real ratio, re-packs that disc with it, and continues on its own — up to MAX_SHRINK_ATTEMPTS (4) times. A bad guess costs rebuild time, not correctness.

You rarely have to tune it at all: PACK_RATIO_ADAPT=1 is the default, so every finished disc feeds its measured ratio back and the next disc is planned from the worst of the last PACK_RATIO_WINDOW (3) discs times PACK_RATIO_MARGIN (1.05). The estimate moves in both directions, so a stretch of incompressible files raises it again rather than packing the rest of the set to a ratio only the early discs achieved. PACK_RATIO is then the starting guess for disc 1.

Set PACK_RATIO_ADAPT=0 to stop learning the ratio from finished discs and plan from the configured value instead. That does not hold the value fixed. A disc that overshoots is still measured and re-packed as above, and the ratio that re-pack used stays in force for every later disc, and through --resume: dropping back would have the next disc overshoot the same way and rebuild its whole image again, once per disc. So with adaptation off the ratio only ever moves up, and only on an overshoot — and one disc of incompressible files then packs the rest of the set as if it were incompressible too, even where the configured value would have fitted. That is the case the adaptive window exists for; leave it on unless you have a reason not to.

Why two compressors

brb uses zstd for the disc images and gzip for the index and the source tarball. That looks inconsistent. It is deliberate, and the deciding question is not the compression ratio — it is who has to decompress it, and what they need installed at the time.

Artifact Compressor Size Decompressed by
discNN.squashfs zstd hundreds of GB the kernel, on mount -o loop
index.tsv.gz.age gzip KB to tens of MB a person, with a userspace tool
brb-src.tar.gz gzip ~1–2 MB a person, with tar xzf

zstd costs nothing on the images, because SquashFS-zstd is decompressed by the Linux kernel itself. The promise that mount -o loop,ro,nosuid,nodev disc07.squashfs /mnt needs no userspace tooling survives intact, and across hundreds of gigabytes the ratio is worth real money in media.

The index inverts that calculus. It is the artifact you reach for when a disc has been lost — you are asking "what was on it?", quite possibly from a rescue USB or a borrowed machine. gunzip has been on every Unix since 1992 and is in busybox and every base install; zstd dates from 2016 and is not guaranteed on a minimal or elderly system. What you would gain is a rounding error: even at two million files the index is perhaps 40 MB gzipped against 30 MB with zstd, on a 23 GiB disc. That is not a trade worth making in the one code path that runs when things have already gone wrong. The source tarball follows the same reasoning.

So the rule is: zstd where the kernel decompresses it and the data is large, gzip where a person needs a userspace tool in a degraded situation and the data is small.

One tradeoff runs the other way, and it is worth knowing. SquashFS-zstd requires kernel 4.14 or newer (2017). A restorer booting a genuinely ancient rescue kernel would find mount fails on a zstd image where a gzip one would have worked; they would fall back to unsquashfs, which still works but is no longer "nothing but the kernel". If you value maximum mountability decades out over space, set COMPRESSION=gzip; for maximum compression instead, COMPRESSION=xz. zstd is the right default, not a free lunch.


What ends up on each disc

README.md                       restore instructions, written for a stranger
MANIFEST.txt                    the whole set, and the exact tool versions used
SHA512SUMS                      hashes of every file on this disc
brb.sh                          the bash reader — restores this disc, and is short
                                enough to read before you trust it
brb-linux-amd64                 static Go binary, 64-bit Intel/AMD
brb-linux-aarch64               static Go binary, 64-bit ARM
brb-src.tar.gz                  complete source for both, dependencies vendored
data/
  discNN.squashfs.age           the filesystem image, encrypted
  discNN.squashfs.age.sha512    hash of the encrypted image
  discNN.squashfs.sha512        hash of the image AFTER decryption
  discNN.squashfs.age.par2      par2 index
  discNN.squashfs.age.vol*.par2 recovery data
  index.tsv.gz.age              encrypted map of which disc holds which file
  index.tsv.gz.age.sha512       hash of the encrypted index
  sidecars.par2                 par2 index over the small .sha512 files
  sidecars.vol*.par2            recovery data for them

The .sha512 sidecars get their own parity because they are tiny and a single rotted byte in one of them would otherwise condemn an image that is perfectly intact. If a sidecar fails to verify but par2 says the image is whole, it is the sidecar that is corrupt: par2 repair -- sidecars.par2 from the disc's data/ directory puts it back.

Every disc carries the full directory skeleton of the original tree — directories, symlinks, device nodes — so mounting any single disc shows you the whole shape of the backup, with the files that live on that disc present and the rest absent. Skeleton entries carry no data, so replicating them across discs is nearly free.

Parity is computed over the encrypted bytes, so it protects exactly what is physically on the disc.


Restoring without brb

This is the point of the design, so it is worth stating plainly. Given one disc and your age identity:

cp /mnt/data/disc07.squashfs.age* .

The glob is deliberate: it brings the .sha512 sidecar and the .par2 files along, which the next line needs.

sha512sum -c disc07.squashfs.age.sha512 || par2 repair -- disc07.squashfs.age.par2
age -d -i /path/to/identity.txt -o disc07.squashfs disc07.squashfs.age
mkdir -p image && sudo mount -o loop,ro,nosuid,nodev disc07.squashfs image

Not /mnt — the disc is mounted there, and mounting the image over it hides the data/ directory the first line reads from for every remaining disc. And nosuid,nodev because ro alone leaves setuid bits and device nodes working: a setuid program or a device node in the backup — an old sudo from a backup of /, say — would otherwise act on the machine reading it, for anyone who can reach the mount. brb mount uses the same options.

That is the whole restore path. unsquashfs -d /dest disc07.squashfs extracts instead of mounting; run it as root if you want original ownership back, and add -user-xattrs if you are not root — the images carry extended attributes, and an unprivileged unsquashfs exits non-zero on them after extracting every file correctly.

If a disc will not read cleanly, pull it off with ddrescue first — it fills unreadable regions with zeros and keeps going, which is exactly what par2 needs:

ddrescue -d -r3 /mnt/data/disc07.squashfs.age ./disc07.squashfs.age ./disc07.mapfile
cp /mnt/data/disc07.squashfs.age.* .    # the sidecar and the parity — .age.* so cp skips the damaged image
par2 repair -- disc07.squashfs.age.par2

ddrescue fetches only the image, and par2 can only use recovery files it can see, so the .par2 set has to be copied beside it before the repair; the on-disc README spells the same three steps out.


Keys, and the rescue key

Backup needs public keys only. Every line of the recipients file is an age recipient, every image is encrypted to all of them, and any single one of the matching identities restores the whole set on its own. No secret has to sit on the backup machine for the days or weeks a set takes to build, so compromising that machine yields nothing that outlives the compromise.

The price of that design is a single point of failure in the other direction: lose identity.txt and the archive is gone. brb init-key --rescue-key is the answer.

brb init-key --rescue-key

This is the Go build. brb.sh has no init-key at all — it reads disc sets and refuses every writing command by name — so a rescue key is minted with the Go binary (or by hand, below) and read by both.

It mints a second keypair and appends its public key to the recipients file, so every disc from then on is encrypted to both. Discs already burned are not, and still need the key they were built with. What makes it a rescue key is how the private half is stored — the shape is this:

age-keygen | age -p -o ~/.config/brb/rescue-identity.txt.age

The identity goes from age-keygen's stdout straight into age -p's stdin, never through a file. brb does the same thing in-process with the age library it already links, so it needs no age-keygen binary and the plaintext identity is never written anywhere: it is generated in memory and handed straight to the scrypt container. That matters because there would be nothing useful to shred afterwards — shred cannot promise anything on a copy-on-write, compressed or flash-translated filesystem, and the plaintext would be in the page cache regardless. What lands on disk is a ~400-byte file, mode 0400, encrypted under a passphrase you chose and typed twice. Copy it to a USB stick, a cloud drive, a relative's machine: it is inert without the passphrase.

The path is not configurable: rescue-identity.txt.age, beside the recipients file, is the last place both readers look for an identity. An existing one is never overwritten — move it aside first — and an existing identity.txt is left untouched, which is what makes --rescue-key safe to run on an archive that already exists.

Restoring with it needs no special flags, on either implementation. Once identity.txt is gone, brb finds rescue-identity.txt.age, asks for the passphrase once, and carries on:

brb restore /path/to/destination
# or, explicitly:
AGE_IDENTITY=/media/usb/rescue-identity.txt.age brb restore /path/to/destination

Do not add --yes to that: the Go build refuses to unlock an encrypted identity under it, since a run that promised to be unattended has nobody to type a passphrase. Run it without, and answer the one prompt.

Without brb, age takes the encrypted identity wherever a plain one goes:

age -d -i rescue-identity.txt.age -o disc07.squashfs disc07.squashfs.age

Why not just encrypt the discs with a passphrase?

The obvious request is "let me use a passphrase instead of a key file". brb deliberately does not do that, for three reasons:

  1. age will not express "my key OR my passphrase" in one file. A passphrase in age is an scrypt stanza, and age refuses to mix an scrypt stanza with recipient stanzas. Encrypting with a passphrase therefore means giving up recipients entirely — no multi-key sets, no rescue key, no third party who can restore for your estate.
  2. A stolen disc is ciphertext forever. Whoever holds the disc holds the attack surface, on their hardware, with unlimited time. A passphrase a human can memorise loses that race over the decades this medium is meant to last; an X25519 key does not. Passphrase-encrypting the images would move the secret onto the thing you lose control of.
  3. Backup would need a secret on the machine. Today it needs only public keys. Passphrase encryption would mean the passphrase — and so the ability to decrypt every disc — is present on the backup host for the whole run.

The rescue key inverts all three. The discs stay encrypted to public keys, and the passphrase guards one small file the thief does not have. Guessing at it requires stealing that file first, and if they have stolen it, you can generate a new key and rebuild — which you cannot do with discs already in the wild.

The corollary: never put the rescue file on these discs. A file stored next to its own passphrase, or on the media it unlocks, is one secret, not two.

brb doctor reports whether a rescue key is present, and which identity a restore on this machine would use.

A public archive, with no secret at all

brb backup --public-archive

Encryption is a second way to lose an archive. Media that outlives its key is still landfill, and for a set meant to be mounted by a stranger in forty years — a family photo archive, a public record, anything where there is nobody left to ask for the key — that risk can outweigh confidentiality entirely.

--public-archive (or PUBLIC_ARCHIVE=1) makes a set that keeps no secret. brb mints a keypair for that archive, encrypts to it exactly as usual, and writes the secret key onto every disc as identity.txt. The set opens with nothing but the disc in hand:

age -d -i /mnt/identity.txt -o disc01.squashfs /mnt/data/disc01.squashfs.age

Two things are worth being explicit about.

It is not encryption. Shipping the key alongside the ciphertext is, in cryptographic terms, no protection at all, and that is the intent. What it buys is that nothing else about the format changes: same age container, same par2 over the same ciphertext, same SHA512SUMS, same two readers. A public set is an ordinary set that happens to carry its own key, not a second on-disc format — so the manual recipe on the disc (age -d -i /mnt/identity.txt …) needs nothing new. Nor do the two readers need telling: ingest, on both, copies the disc's identity.txt into staging as enc/identity.txt alongside the images, and every command that decrypts (restore, mount, list, index) tries that key in addition to whatever AGE_IDENTITY, or the identity found beside the recipients file, supplies. So brb ingest followed by brb restore /dest opens a public set with nothing configured at all, and the on-disc README's worked example says exactly that. A configured AGE_IDENTITY is not displaced by it — the disc's key is added, not substituted — so a machine set up for your own ordinary sets reads a public one without any change.

The key is always freshly generated. AGE_RECIPIENTS_FILE is not consulted and neither is AGE_IDENTITY, deliberately: publishing a key you already use would retroactively expose every other archive encrypted to it. One flag must never be able to disclose unrelated backups, so the published key belongs to that one archive and to nothing else.

The key is written three times per disc — identity.txt, MANIFEST.txt and README.md — because it is not in sidecars.par2 (par2 will not reach above its own working directory, and the key sits at the disc root where both readers look for it). An age secret key is 74 bech32 characters with a checksum, so if one copy rots another can be retyped and will be either accepted or rejected outright, never silently wrong.

Each disc's README says plainly, at the top, that the archive is not confidential. An ordinary set is completely unaffected: no identity.txt, no key in its documents, and its README still says the key is not on the disc and never will be.


Security notes

  • Staging holds unencrypted images while a backup runs. Each image is deleted as soon as it is encrypted, but at any moment one full disc's worth of your plaintext is sitting in $STAGING. Put staging on an encrypted volume, or wipe it afterwards. brb warns about this and asks for confirmation before starting.
  • $STAGING/restore holds decrypted images — plaintext, mode 0700. restore deletes each one as soon as its contents are on disk, so by default nothing is left there but what a failed run dropped. KEEP_IMAGES=1 (or --keep-images) keeps them all, and list and mount leave the image they decrypted behind by design. Remove them when you are done: rm -rf "$STAGING/restore".
  • $STAGING must never have been writable by another account. Each run makes every staging directory it uses mode 0700 and refuses one that another account owns, so nothing can be planted there once brb has secured it. What it cannot vouch for is what was planted before that: in a staging directory you made group- or world-writable by hand, or adopted from another account with chown -R, which leaves everything inside it in place. The writer trusts little of what it finds — each disc directory is emptied and laid out afresh on every run (one that is a mount point, or holds one, is refused instead, since emptying it would empty what is mounted there; a directory bind-mounted from the filesystem $STAGING/discs is on is recognised only on Linux with /proc mounted), and MANIFEST.txt and the documents beside it are written under a new name and renamed into place, so none of them follows a link left at the name — but the resume state ($STAGING/state.json), the plaintext index in work/ and the ciphertext in enc/ are the set, read back as they are, and a file planted there cannot be told from one brb wrote. If staging was ever shared, remove it and start the set over.
  • The recipients file contains public keys only and is harmless. The identity file is the secret, and it is never written to a disc — the one exception being a public archive, whose freshly minted key is on every disc by design.
  • Encrypting to multiple recipients is supported: append more age1... public keys to the recipients file and every image becomes decryptable by any of them. brb init-key --rescue-key is that mechanism applied to your own second key — see Keys, and the rescue key. The ceiling is 1024 keys: age will encrypt a file to more, but no age from 1.3.2 on will decrypt it, so backup refuses a recipients file that long and init-key will not add a key past it.
  • An identity can be passphrase-protected (age -p) and used exactly like a plain one on the restore side; brb unlocks it once per command. The passphrase is read from /dev/tty and nowhere else, so it is the one prompt --yes cannot answer: with no terminal, brb says so and stops rather than hanging, and an empty passphrase is reported as an empty passphrase rather than as a missing terminal. The Go build refuses --yes outright when the only identity it can find is encrypted — a run that promised to be unattended should not stop to ask — and tells you to run without it; brb.sh prompts anyway. An unattended backup is unaffected either way — it encrypts to public keys and reads no identity at all. The exception is backup --verify-roundtrip, which decrypts every image back to prove the set readable: that needs an unencrypted AGE_IDENTITY and refuses to start without one, rather than stopping for a passphrase hours into a run.
  • Run brb backup as root if you want to capture files you cannot read as your own user, and to record real ownership. As a non-root user, unreadable files are skipped and ownership is recorded as yours.
  • brb mount is a browse mount, not this machine's system. Both readers mount the image nosuid,nodev, so a setuid binary or a device node in the backup does nothing on the host; copying out with cp -a, tar or rsync still keeps both. What it does not change is who can read the tree: any local account that can reach the mount point reads the decrypted files under their archived permissions, and an archived uid may be somebody else here. The directories brb mount creates are 0700; on a shared machine, mount somewhere only you can enter.
  • Back up a tree other accounts can change from a snapshot. brb scans SOURCE_DIR and then hands the names it recorded to mksquashfs, once per disc and hours later on a long set. mksquashfs, like the scan itself, resolves each path again. Neither follows a symlink it finds inside the tree, but either will follow a directory that was renamed and replaced by a symlink in the meantime. An account that can rename SOURCE_DIR, a directory above it or a directory inside it can therefore have files from elsewhere archived under the recorded names, read with the backing-up user's privileges and with no warning. Run as root, that can put files the account could never read onto the discs. The images are encrypted to your keys, so this discloses something only to whoever can read the set, which for a public archive is anyone. As on the restore side (see Limitations), brb cannot close the window: no descriptor survives into a subprocess that takes a path. Back such a tree up from a read-only snapshot (LVM, btrfs, ZFS) mounted where only root can write, and make a public archive of it no other way.

Limitations

Known and deliberate, but you should hear them before you rely on this:

  • Not incremental. Every run is a full backup of the whole tree. A completed set cannot be updated in place; the next backup builds a new one.

  • Resuming is deliberate, not automatic. An interrupted run continues with brb backup --resume, but a plain backup refuses to start on top of one rather than silently throwing the work away. The resumed run must agree with the interrupted one on ARCHIVE_NAME and SOURCE_DIR, on the recipients, the disc geometry and public-or-ordinary mode (see Resuming an interrupted backup), and needs the state file in $STAGING; if staging has been cleared, the set is built again from scratch. A resume re-scans the source and skips what is already on a disc, so files added since the run started are picked up on later discs and a set can span two points in time — it warns when the tree's measured size has changed. It refuses a path a finished disc holds that has changed kind since, such as a directory moved away and symlinked back, while discs remain to be written.

  • One filesystem at a time. The scan does not descend into a mounted directory under SOURCE_DIR: the mount point is reported at plan and backup time, appears on the discs as an empty directory, and nothing mounted there is backed up. Back a second filesystem up as its own set, or point SOURCE_DIR at it. This is what keeps a home directory backup from quietly swallowing an external drive or a network share that happened to be mounted under it that day.

    The boundary is enforced at directories, which is what find -xdev, tar --one-file-system, rsync -x and cp -x all mean by it. A single file bind-mounted from another filesystem — /etc/resolv.conf inside a container, say — is an ordinary entry and is backed up, contents and all. It is one file and cannot pull a subtree in behind it, and leaving out something plainly present in ls is the worse way for a backup to surprise you.

  • Files you cannot read are left out, and said so. A file the scanning user has no permission to open is reported by the scan and excluded from the set — it is not written as an empty file, which would restore as a silent truncation. Run backup as root to capture them, and to record real ownership; see Security notes.

  • A single file larger than one disc cannot be stored. brb detects this during plan/backup and stops rather than silently dropping it. Exclude it, use larger media, or split it yourself.

  • A disc is checked against its set, but the set is not signed. age encrypts to a public key, and MANIFEST.txt on every disc names it — so anyone holding one disc can build another that decrypts, matches its own hashes and par2, and extracts. Two checks make that hard rather than trivial: the index staged from the first disc is pinned, and a later disc carrying a different one is refused; and each image must hold exactly the files the index says that disc holds. Together they force a forger to ship the genuine index (which they cannot read) and then match it (which they therefore cannot do). Be clear about the limits. This detects that the discs of a set disagree with each other — it cannot say which one is lying, and if the forgery is the first disc you ingest, the genuine discs are what gets refused. Nothing bounds how many rows that pinned index may hold, either, so a forgery carrying a large enough one exhausts memory on a later restore rather than being refused at all. It gives nothing when a single self-consistent forged disc is all you have. And a filename containing a newline cannot survive the line-based listing the check reads, so for a set holding one the check says out loud that it could not be made exact, and does not refuse. The same happens when the index fails the hash recorded beside it, in index.tsv.gz.age.sha512 — and that file, unlike the index, is not pinned: a later disc carrying the genuine index and a different .sha512 replaces the staged one, and the check is then skipped, with that warning, for every disc of the set. A forger gains nothing by it that a newline in a filename on their own disc does not already give them. It is not a signature; the format has no signing key.

    The same listing limits what restore --only can prove. A requested path whose name holds a newline (or a carriage return) cannot be looked up in an image's listing at all, and an ordinary one can be found there falsely, in a piece of a newline-named entry — --only foo against an image holding $'foo\nbar', whose listing prints a line reading foo. In both cases the two readers take the destination as the proof: the path counts as restored if it is there once unsquashfs has run. If the destination already held that path, the proof proves nothing — a disc that never had it still ends in "restore complete", with the old copy left in place. With a readable index and no --disc, both readers refuse a path the index does not list before opening any image, so this needs one of three things, each the same on both readers. A missing or unreadable index, which both warn about and search past. A --disc N naming a disc that lacks the path when the index does not list the path at all: both search disc N anyway, because --disc names it, and refuse only when the index puts the path on another disc. Or an image that disagrees with its index. Restore such a path into an empty directory, or move the existing copy aside first.

  • A restore overwrites its destination. unsquashfs -f replaces existing files with the backup's versions, mode and mtime included, and --yes answers the confirmation that would otherwise have stopped it. A destination holding a symlink to a directory is refused outright, --yes or not. See restore overwrites the destination.

  • The destination must not be writable by others while a restore runs. The symlink refusal above is a check followed by an act: both readers walk the destination, refuse what they find, and then hand the path to unsquashfs, which resolves every name again itself. A local account that can write inside the destination can plant a symlink in the gap between those two steps and have files written through it, outside the destination, with the restoring user's privileges. Nothing in either reader can close that window — no descriptor survives into a subprocess that takes a path — so restore somewhere only you can write. Staging is better defended, though not unconditionally, and the two readers differ in how far they can go: the Go build opens $STAGING with O_NOFOLLOW and applies the mode and ownership checks through the handle it opened, never by name a second time, while brb.sh has no way to hold a descriptor and tests the name with [[ -L ]] before creating the directory. Neither reader examines the path above $STAGING: the parents are created with mkdir -p and trusted. So point staging somewhere whose parent directories are yours or root's — the default /var/tmp/brb qualifies, because /var/tmp is root-owned and sticky and no other account can put anything in brb's way.

  • The same holds for a mount point. Both readers refuse a brb mount point that is itself a symlink, since mount(8) would mount the image over whatever it points at — which also refuses a root-owned link such as /mnt on Fedora Atomic desktops; pass the real directory (/var/mnt). But they trust every directory above it, and hand mount(8) a path it resolves again, so a link planted in between is followed. Mount under a directory only root can write, never inside a group-writable directory or another user's.

  • A restore needs room for the extracted tree plus one decrypted image. Each image is removed as soon as its contents are on disk, so only one exists at a time. KEEP_IMAGES=1 (or --keep-images on the Go build) keeps them all for repeated restores, and then you do need room for the whole archive twice.

  • ISOs are ISO 9660 level 3 only, not UDF. Level 3 multi-extent is what allows the >4 GiB images. Any Linux system reads these; some appliances that expect UDF on Blu-ray may not.

  • COMPRESSION_LEVEL applies to zstd, gzip and lzo only. Those are the compressors mksquashfs takes -Xcompression-level for (1–22 for zstd, 1–9 for the other two, and a value outside the range is a config error). xz and lz4 are tuned through different flags and run at their own defaults; a non-zero COMPRESSION_LEVEL alongside one of them — and the default is 19 — is called out in a warning by doctor and at the start of backup rather than dropped silently. COMPRESSION_LEVEL=0 quiets it.


Testing

Two suites, with no overlap. Both build real disc sets with the real tools — mksquashfs, age, par2, xorriso — and restore them with the real tools too. The only stand-ins are in a few xcompat-test.sh checks, which put one first on PATH to reach a condition a well-formed set cannot produce: a mount(8) that records its arguments, and wrappers that make unsquashfs or sha512sum fail.

./go-e2e-test.sh

The writer. Backs up a multi-disc set, kills it mid-set with kill -9 on the process group, resumes it, and asserts the discs finished before the kill come back byte-identical and the completed set restores byte-identical to its source. Also covers ISO modes and that a finished run leaves no resume state behind.

./xcompat-test.sh

The format contract. Builds a set with the Go build and reads it with both implementations, asserting they agree — the restored trees, the encrypted index, the disc inventory, list, --only, KEEP_IMAGES, and the staging lock that stops a second run writing into a tree a backup has not finished. It also runs the recipe printed on the disc itself, with neither implementation involved, and damages a set on purpose: a rotted .sha512 sidecar must not condemn an image par2 proves is whole, while an image par2 cannot repair must still be refused rather than decrypted — refused and leaving the destination empty, since a reader that decrypted first and failed later would satisfy the exit code alone.

It also plants symlinks in the destination — the direction a restore is attacked from — and holds both readers to the same two refusals: a link that resolves to a directory, which unsquashfs -f would follow to write the archive's files outside the destination, and a link that resolves to a file sitting where the image holds a directory, whose target would otherwise be given the archive's directory mode, owner and times. Each refusal is checked alongside the cases that must still restore, so neither guard can pass by refusing everything.

mount needs root to do for real, so it is held to what can be checked without it: both readers refuse a mount point that is a symlink, in every spelling the kernel would resolve through and before either touches staging, and both hand mount(8) the same -o loop,ro,nosuid,nodev -- list and create the same 0700 directories. That last check runs the readers as root of a user namespace (unshare -r) with a stand-in mount; where user namespaces are refused it skips, and only the script's mount line is checked, by reading it. CI does not let it skip (see Continuous integration).

Where the two differ in a way that ought to be fixed, the check is written the way it ought to pass and marked XFAIL with the divergence named. An XFAIL that starts passing is reported as XPASS and counted as a failure, so a fixed divergence gets promoted to a real assertion instead of sitting in the ledger forever. Nothing known-broken is quietly omitted.

That ledger and the command-line table above are separate records and mean different things. The table holds differences that are deliberate and are not going to converge: version is a command in one reader and not the other because only one of them writes discs, doctor answers a writer's question in the Go build and a reader's in the script. Writing those the way they "ought to pass" would be writing a test for something nobody intends to do. The XFAIL ledger holds the other kind — a difference the project means to close — and it is deliberately small. It was empty for a long time, and the suite reported "0 known divergence(s) remain" with the machinery never once executed, which read as agreement rather than as an unused ledger.

Plus the Go unit tests:

cd go && go test ./...

Continuous integration

.github/workflows/ci.yml runs all of the above on every push and pull request, weekly on a schedule, and on demand. Both suites and build-dist.sh are invoked there with no arguments and no environment, which is the only arrangement that catches a machine-specific path getting baked into a default again. For build-dist.sh that is taken literally: no output argument, BRB_DIST_OUT unset, and HOME pointed at a fresh, empty directory for that one step, so the script's own ~/brb-dist fallback is what decides where the payload lands, and the checks that follow look only there. (One later step runs it twice more with a throwaway output directory, on purpose: those runs exist to be refused, as described below, and must not land anywhere the payload checks look.)

Every third-party action the workflow uses is pinned to a 40-character commit SHA, with the release tag it came from in a trailing comment. A @v5-style tag is movable by whoever owns that repository, and the job holding it is the one that builds the payload burned onto the discs; the same reasoning that checksums the shellcheck download applies to it. The header of ci.yml records how to resolve and verify a SHA when one is moved forward.

The four shell scripts are linted with a pinned, checksummed shellcheck 0.11.0 at its most inclusive severity, and all four are clean at it. The version is pinned deliberately: Ubuntu ships 0.9.0, which does not report the same set, and a lint gate that changes its mind when the runner image is rebuilt is worse than no gate. The two places where a deliberate idiom trips a check are disabled at the site, with the reason written next to them, rather than switched off across the repo.

One step in it is load-bearing and worth knowing about: CI verifies every external tool is present before running anything, and it checks for more than the suites themselves require. The quiet failures are the reason. A tool-dependent Go test calls t.Skip when detection fails, so a runner missing par2 would report a green go test ./... in which the integration tests never ran at all; and xcompat-test.sh treats script(1), flock(1), an en_US.UTF-8 locale and a working unshare -r as optional, skipping the pty, staging-lock, locale and mount(8)-argument checks and still exiting 0 without them — so CI checks for those four by name. Both are right on a laptop and dangerously quiet on a build runner. A tool the suites do require is loud already — xcompat-test.sh exits 77 and go-e2e-test.sh exits 2, either of which fails the step — so for those the check only buys a named tool up front instead of a dead suite forty minutes in. The unshare -r check has a step of its own before it: Ubuntu 24.04 refuses unprivileged user namespaces through an AppArmor setting, which CI switches off on its throwaway runner.

The release job rebuilds brb-src.tar.gz with GOPROXY=off, so the offline rebuild the on-disc README promises is re-proven on every run rather than resting on the last time somebody tried it by hand. It also checks the two cross-compiled binaries are genuinely x86-64 and aarch64 and both static, and that go.mod and go.sum are already tidy — build-dist.sh runs go mod tidy itself, so any drift there means the committed module no longer describes what ships on a disc.

The release assets are checked as the set a releaser will upload: the SHA256SUMS that gets signed must name exactly the four assets, no more and no fewer, since sha256sum -c checks only the lines that are there; the release directory must hold those four and SHA256SUMS and nothing else; and each asset must be the payload file it was renamed from, byte for byte. And build-dist.sh's refusal to pack a non-source file under go/ is tripped on every run, in both of its modes: an untracked file in the checkout, and the go/brb the offline rebuild recipe leaves in an unpacked tarball. Each run must fail by naming that file and nothing else, so a build broken for some other reason cannot pass for the refusal, and a new kind of source file the tarball allowlist does not know fails CI here rather than build-dist.sh in every unpacked tarball afterwards.


License

MIT — see LICENSE. The full notice is also embedded in the header of brb.sh itself, so the copy of the script carried on every disc stays properly licensed even when separated from this repository.

No warranty. Test your restores.

About

Independent, mountable, encrypted Blu-ray backup discs: SquashFS + age + par2, one self-contained image per disc

Resources

Stars

0 stars

Watchers

0 watching

Forks

Releases

Contributors

Languages