---
tags: [restic]
---

How I Use Restic to Back up My Home Folders to Backblaze B2
===========================================================

<p class="lead" markdown="1">
My guide to using the open source backup program
[restic](https://restic.net/)
to back up your home folders to 
[Backblaze B2](https://www.backblaze.com/cloud-storage)
cloud storage.
</p>

<div class="seealso" markdown="1">
**See also:**

* [restic's documentation](https://restic.readthedocs.io/en/stable/), including
  the [section on B2](https://restic.readthedocs.io/en/latest/030_preparing_a_new_repo.html#backblaze-b2)
* [Backblaze B2's own restic guide](https://help.backblaze.com/hc/en-us/articles/4403944998811-Quickstart-Guide-for-Restic-and-Backblaze-B2-Cloud-Storage)
</div>

* This unordered list will be replaced with a table of contents.
{:toc}

Backing up a new machine for the first time
-----------------------------------------------

### 1. Create a B2 bucket

All files in B2 are contained in
[buckets](https://www.backblaze.com/b2/docs/buckets.html), which are like the
top-level folders in your B2 account. Before you can back anything up to B2 you
need to create a bucket:
[Sign in to the Backblaze website](https://secure.backblaze.com/user_signin.htm),
navigate to the [B2 Cloud Storage Buckets page](https://secure.backblaze.com/b2_buckets.htm),
and click the <kbd>Create a Bucket</kbd> button:

<figure markdown="1">
  ![Creating a B2 bucket.]({{ "/assets/images/restic/create-bucket.png" | relative_url }} "Creating a B2 bucket.")
  <figcaption markdown="1">
  Creating a B2 bucket.
  </figcaption>
</figure>

Bucket names must be unique across all of B2, including the names of other user's buckets!
Hence the random numbers at the end of the bucket name.

Make sure the bucket is set to **Private**.

You can leave B2's default encryption disabled: restic will encrypt your files for you.

<details markdown="1">
<summary><b>One-to-one mapping between buckets and repositories</b></summary>
You'll see below that
I create one restic repository in each B2 bucket, at the root of the bucket.
That's why my `RESTIC_REPOSITORY` environment variable contains just a bucket
name with no path (`b2:restic-seanh-laptop-79539`).

Alternatively you could create the restic repository in a subdirectory within the bucket.
Just add a path to the end of your `RESTIC_REPOSITORY` like this: `b2:bucketname:path/to/repo`.
This would allow a single B2 bucket to contain multiple restic repositories at
different paths, and I *think* B2 application keys can be given access restricted
to subpaths of buckets so different hosts don't have access to each other's backups.

I find it simpler to create a separate bucket for each restic repo.
A B2 account is limited to 100 buckets so if you have a lot of repos you might
want to try sharing buckets.
</details>

<details markdown="1">
<summary><b>Shared versus separate repos</b></summary>
When backing up multiple computers they can all back up to the same restic repo
or each computer can back up to its own separate repo.

Most of the time I create a separate B2 bucket and restic repo for each machine
because it's more secure: when machines share a restic repo they have
full read write access to each other's backups. Using separate repos also
avoids locking: machines can back up to the same repo simultaneously but
if one machine is running certain maintenance commands like `restic prune` or
`restic check` then other machines can't back up to (or run maintenance
commands on) the same repo at the same time.

My main desktop and laptop do share the same repo however. This is more storage
efficient because restic will deduplicate files across the two different hosts:
if the desktop and laptop both contain copies of the same file only one copy of
that data will actually be stored in the restic repository.  In this case I
think the security risk is acceptable: these machines have a lot of files in
common so the deduplication might be significant. There's already all sorts of
shared sensitive files that I have on both of these machines anyway (my GPG and
SSH keys, log ins to my Bitwarden account, etc). Both machines use full disk
encryption. To avoid locking I [run all my maintenance commands from one host](#accessing-another-hosts-backups).

When sharing a single restic repo between multiple hosts I create a separate
application key (in Backblaze) and a separate `RESTIC_PASSWORD` (using the
[`restic key` command](https://restic.readthedocs.io/en/stable/070_encryption.html),
see `restic help key`) for each machine. This means each machine's access can
be revoked individually if that machine is compromised or is no longer using the repo.
</details>

### 2. Create an application key for the B2 bucket

Restic needs a B2
[application key](https://www.backblaze.com/b2/docs/application_keys.html)
to read and write to the bucket.
Navigate to the [Application Keys page](https://secure.backblaze.com/app_keys.htm)
and click the <kbd>Add a New Application Key</kbd> button:

<figure markdown="1">
  ![Creating an application key.]({{ "/assets/images/restic/create-key.png" | relative_url }} "Creating an application key.")
  <figcaption markdown="1">
  Creating an application key.
  </figcaption>
</figure>

In the <samp>Allow access to Bucket(s)</samp> dropdown select only the bucket you just created.

<samp>Type of Access</samp> must be set to <samp>Read and Write</samp>.

Copy the `applicationKey` that's shown after you click <kbd>Create New Key</kbd>, you'll need it later:

<figure markdown="1">
  ![Copying the application key.]({{ "/assets/images/restic/key.png" | relative_url }} "Copying the application key.")
  <figcaption markdown="1">
  Copy the application key for later.
  </figcaption>
</figure>

### 3. Install pass

We're going to use [pass](https://www.passwordstore.org/) to save encrypted
copies of the application key and other restic settings on the host machine
that's going to be backed up. On the host machine:

1. Install pass.

   On Ubuntu:

   ```terminal
   $ sudo apt install pass
   ```

   On macOS install [Homebrew](https://brew.sh/) and then run:
   
   ```terminal
   $ brew install pass
   ```

2. Create a GPG keypair for pass to encrypt things with:

   ```terminal
   $ gpg --full-generate-key
   ...
   public and secret key created and signed.

   pub   rsa3072 2022-04-03 [SC]
         6346C69761B1155E228D488F8C23BC2681C72FBA
   uid                      Sean Hammond (GPG key for `pass` on seanh-laptop)
   sub   rsa3072 2022-04-03 [E]
   ```

   Take note of the key's UID (the part beginning `6346...` in my example), you'll need it for the next step.
   (You can always retrieve it with `gpg --list-secret-keys`).

3. Initialize the password store with the GPG key:

   ```terminal
   $ pass init <GPG_KEY_ID>
   ```

### 4. Insert restic settings into pass

Insert the B2 bucket name, application key ID, and application key value into pass:

```terminal
$ pass insert "$(hostname)"/RESTIC_REPOSITORY
<When prompted enter "b2:" followed by the name of the B2 bucket you created.>
<Example: b2:restic-seanh-laptop-79539>

$ pass insert "$(hostname)"/B2_ACCOUNT_ID
<When prompted paste in the keyID of the application key that you created.>

$ pass insert "$(hostname)"/B2_ACCOUNT_KEY
<When prompted paste in the secret value of the application key you created.>
```

We're inserting the settings into a password store subfolder named after your
hostname so that the password store can potentially be extended to hold
settings for multiple restic repos, which we'll
be doing later.

### 5. Generate a `RESTIC_PASSWORD` and save it in Bitwarden and pass

restic also needs a `RESTIC_PASSWORD` setting which is the password it'll use
to encrypt your backups. I use the cloud password manager
[Bitwarden](https://bitwarden.com/) to back up the `RESTIC_PASSWORD`:

1. Use pass to generate a password and save it in the local password store:

   ```terminal
   $ pass generate "$(hostname)"/RESTIC_PASSWORD
   ```

2. Also copy-paste the password into Bitwarden as a backup.
   pass can copy the password to the clipboard for you:

   ```terminal
   $ pass show --clip "$(hostname)"/RESTIC_PASSWORD
```

<div class="warning" markdown="1">
**Don't lose your `RESTIC_PASSWORD`!**

`RESTIC_PASSWORD` is a restic thing, Backblaze doesn't know anything about it.
While `RESTIC_REPOSITORY`, `B2_ACCOUNT_ID` and `B2_ACCOUNT_KEY` can all be
retrieved or regenerated from the Backblaze website, `RESTIC_PASSWORD` can't
be. And since `RESTIC_PASSWORD` is what's used to encrypt your backups, if you
lose your `RESTIC_PASSWORD` you've lost your backups. This is why I save a copy of each
`RESTIC_PASSWORD` in Bitwarden as well as in pass on the host machine.
</div>


### 6. Create an env script

Create a `~/.restic` folder (`mkdir -p ~/.restic`) and create a
`~/.restic/env.bash` script to load the restic settings from pass into
environment variables:

```bash
#!/bin/bash
set -euo pipefail

RESTIC_HOST="$(hostname)"
RESTIC_PATH="$HOME"
RESTIC_REPOSITORY="$(pass "$RESTIC_HOST"/RESTIC_REPOSITORY)"
B2_ACCOUNT_ID="$(pass "$RESTIC_HOST"/B2_ACCOUNT_ID)"
B2_ACCOUNT_KEY="$(pass "$RESTIC_HOST"/B2_ACCOUNT_KEY)"
RESTIC_PASSWORD_COMMAND=pass\ "$RESTIC_HOST"/RESTIC_PASSWORD

export RESTIC_HOST RESTIC_PATH RESTIC_REPOSITORY B2_ACCOUNT_ID B2_ACCOUNT_KEY RESTIC_PASSWORD_COMMAND
```

<details markdown="1">
<summary><b>RESTIC_HOST and RESTIC_PATH</b></summary>
`RESTIC_HOST` and `RESTIC_PATH` aren't environment variables recognised by
restic itself. We're exporting them because we'll use them in our backup and
maintenance scripts later on.
</details>

<details markdown="1">
<summary><b>fish-shell</b></summary>
If you use fish-shell you might also want to create a `~/.restic/env.fish`
script that you can `source` from fish:

```fish
#!/usr/bin/fish

set -x RESTIC_HOST (hostname)
set -x RESTIC_PATH $HOME
set -x RESTIC_REPOSITORY (pass $RESTIC_HOST/RESTIC_REPOSITORY)
set -x B2_ACCOUNT_ID (pass $RESTIC_HOST/B2_ACCOUNT_ID)
set -x B2_ACCOUNT_KEY (pass $RESTIC_HOST/B2_ACCOUNT_KEY)
set -x RESTIC_PASSWORD_COMMAND pass $RESTIC_HOST/RESTIC_PASSWORD
```
</details>

### 7. Source the env script

Now you can `source` the `env.bash` script from a bash shell and any subsequent
restic commands in that shell session will use your B2 bucket, application key,
and restic password without having to pass them as options to each command.

**The rest of the commands in this post will assume you've sourced the `env.bash` script**,
so do that now:

```terminal
$ source ~/.restic/env.bash
```

The script can also be sourced from other bash scripts, which we'll do below.

### 8. Install restic

To install restic on Ubuntu:

```terminal
$ sudo apt install restic
$ sudo restic self-update

$ # Install bash autocompletions for restic subcommands and options.
$ mkdir -p ~/.local/share/bash-completion
$ restic generate --bash-completion ~/.local/share/bash-completion/restic
```

The autocompletions enable you to press <kbd>Tab</kbd> after typing `restic` to
autocomplete restic subcommands, or press <kbd>Tab</kbd> after `--` to
autocomplete restic options.

On macOS install [Homebrew](https://brew.sh/) and then run:
   
```terminal
$ brew install restic
```

<details markdown="1">
<summary><b>fish-shell</b></summary>
If you use fish-shell you might also want to install fish-shell
autocompletions:

```terminal
$ mkdir -p ~/.config/fish/completions
$ restic generate --fish-completion ~/.config/fish/completions/restic.fish
```
</details>

### 9. Initialize a restic repo in the B2 bucket

Initialize a restic repository (repo) in the B2 bucket:

```terminal
$ restic init
created restic repository 191d720a00 at b2:restic-seanh-laptop-79539

Please note that knowledge of your password is required to access
the repository. Losing your password means that your data is
irrecoverably lost.
```

If you browse your bucket in Backblaze you should see that restic has created
some housekeeping files in it:

![Files created by `restic init`.]({{ "/assets/images/restic/files.png" | relative_url }} "Files created by `restic init`.")

### 10. Create a backup script

Create a `~/.restic/backup` script that runs a `restic backup` command to back
up your home directory.

You'll want to pass some command line options to `restic backup`, consider:

`--compression max`
: Increase compression to save storage space on your backup location, at the cost of using more CPU.

`--exclude-caches`
: Excludes directories that are marked as caches by having a `CACHEDIR.TAG` file.

`--one-file-system`
: Excludes any other file systems that're mounted within your home directory.

`--cleanup-cache`
: Removes old cache directories after running.

`--exclude <FILE>`
: Excludes unwanted files from the backup.
  `--exclude` can be given multiple times, and `<FILE>` can be a pattern.
  There are other options for excluding or including files as well,
  see [Excluding Files](https://restic.readthedocs.io/en/stable/040_backup.html#excluding-files)
  in the restic docs for details.
  You might want to use Disk Usage Analyzer (`baobab`) to scan your home directory for large files
  or folders that you don't want to back up.

See `restic help backup` for the rest of the available options.

Here's an example `backup` script with lots of `--exclude`'s for Linux and macOS directories that you probably don't want to back up:

```bash
#!/bin/bash
set -euo pipefail

source ~/.restic/env.bash

restic backup \
    --compression max \
    --exclude-caches \
    --one-file-system \
    --cleanup-cache \
    --exclude "$HOME/Downloads" \
    --exclude "$HOME/Library" \
    --exclude "$HOME/snap" \
    --exclude "$HOME/.Trash" \
    --exclude "$HOME/.bundle" \
    --exclude "$HOME/.cache" \
    --exclude "$HOME/.dbus" \
    --exclude "$HOME/.dropbox" \
    --exclude "$HOME/.dropbox-dist" \
    --exclude "$HOME/.local/pipx" \
    --exclude "$HOME/.local/share/Trash" \
    --exclude "$HOME/.npm" \
    --exclude "$HOME/.pyenv" \
    --exclude "$HOME/.thumbnails" \
    --exclude "$HOME/.virtualenvs" \
    --exclude "node_modules" \
    --exclude ".tox" \
    "$RESTIC_PATH"
```

### 11. Make the backup script executable

```terminal
$ chmod u+x ~/.restic/backup
```

### 12. Do the initial backup

Finally, to backup your home directory to B2 just run:

```terminal
$ ~/.restic/backup
```

This will take a long time to run. Once it's finished you'll see more files in
your B2 bucket:

![Files created by `restic backup`.]({{ "/assets/images/restic/finished-files.png" | relative_url }} "Files created by `restic backup`.")

Updating a backup
-----------------

To incrementally update a machine's backup, uploading only new or modified files,
just re-run the same backup script:

```terminal
$ ~/.restic/backup
```

Interrupting and resuming a backup
----------------------------------

You can interrupt a backup with <kbd>Ctrl</kbd> + <kbd>c</kbd> and later resume
it by re-running the same `~/.restic/backup` command.
See [Will restic resume an interrupted backup?](https://restic.readthedocs.io/en/stable/faq.html#will-restic-resume-an-interrupted-backup)
in restic's FAQ.

Limiting bandwidth usage
------------------------

You can limit restic's speed so it doesn't hog all your bandwidth when you're
updating a backup, just add the `--limit-download` and `--limit-upload`
options. You might not want to do this for your initial backup, but once
your first backup has completed consider adding these to your backup script:

```terminal
$ restic backup --limit-download 5600 --limit-upload 1800 ...
```

The values are the maximum rates in KiB/s.

Other commands like `restic check` (for verifying a backup) and `restic restore` (for restoring a backup)
also support the the `--limit-download` and `--limit-upload` options.


Backing up just one file to the same repo
-----------------------------------------

You can run a one-off command to quickly back up one or more specific files or
directories to the same repo, without doing an incremental backup of the
whole machine. Just pass the file path(s) to `restic backup`.
This while create a separate snapshot that contains only the given file(s):

```terminal
$ restic backup --verbose=3 foo.txt
open repository
repository d59c3d15 opened successfully, password is correct
lock repository
load index files
no parent snapshot found, will read all files
start scan on [foo.txt]
start backup on [foo.txt]
scan finished in 28.574s: 1 files, 4 B
new       /foo.txt, saved in 0.004s (4 B added)

Files:           1 new,     0 changed,     0 unmodified
Dirs:            0 new,     0 changed,     0 unmodified
Data Blobs:      1 new
Tree Blobs:      1 new
Added to the repo: 384 B

processed 1 files, 4 B in 0:35
snapshot c6d56f86 saved
```

Doing a dry run with `restic backup --dry-run`
----------------------------------------------

You can always test what a `restic backup` command would do by passing the
`--dry-run` and `--verbose` arguments.
This can be good for testing whether your exclude options will do what you expect,
or checking how much data will be added to the backup before actually running it:

```terminal
$ restic backup --dry-run --verbose=3 foo.txt
open repository
repository d59c3d15 opened successfully, password is correct
lock repository
load index files
no parent snapshot found, will read all files
start scan on [foo.txt]
start backup on [foo.txt]
scan finished in 8.994s: 1 files, 4 B
new       /foo.txt, saved in 0.003s (0 B added)

Files:           1 new,     0 changed,     0 unmodified
Dirs:            0 new,     0 changed,     0 unmodified
Data Blobs:      0 new
Tree Blobs:      1 new
Would add to the repo: 380 B

processed 1 files, 4 B in 0:09
```

Comparing two snapshots with `restic diff`
------------------------------------------

You can use `restic diff` to compare the contents of two snapshots.
Comparing a snapshot to the previous snapshot can be like a
retrospective `--dry-run`, showing you what a `restic backup` did.

First use [`restic snapshots`](#listing-snapshots-with-restic-snapshots) to get
the IDs of two snapshots that you want to compare, then pass them to `restic
diff`:

```terminal
$ restic diff 9eb356c4 b072e1ac
-    foo.txt
+    bar.txt
M    gar.txt

Files:           1 new, 1 removed,     1 changed
Dirs:            0 new, 0 removed
Others:          0 new,    0 removed
Data Blobs:      1 new, 1 removed
Tree Blobs:      1 new, 1 removed
  Added:   384 B
  Removed: 51.780 GiB
```

In the list of files `+` denotes a file that was added,
`-` is a file that was removed,
`M` means the file's contents were modified,
`U` means the file's metadata was changed
and `T` mean's the file's type was changed (for example it changed from a file
to a symlink).

Restoring files from your backup
--------------------------------

### Listing snapshots with `restic snapshots`

`restic snapshots` prints out a list of all the snapshots in the repo (each
completed `restic backup` command creates a new snapshot).
This is mostly useful for getting snapshot IDs to pass to commands like
`restic restore` (see below).


```terminal
$ restic snapshots
repository d59c3d15 opened successfully, password is correct
ID        Time                 Host          Tags        Paths
--------------------------------------------------------------------
1f5cfe20  2020-04-28 09:04:47  beatsworking              /home/seanh
81754433  2020-12-24 17:43:24  chamlis                   /home/seanh
27ca05c7  2021-04-29 18:20:00  chamlis                   /home/seanh
f362f638  2021-05-28 14:13:38  chamlis                   /home/seanh
ad4bf831  2021-06-10 18:32:55  chamlis                   /home/seanh
1fa5c822  2021-07-29 17:26:37  chamlis                   /home/seanh
50d3f0c2  2021-08-28 18:10:00  chamlis                   /home/seanh
2a62a5ed  2021-09-23 17:18:26  chamlis                   /home/seanh
9ae436fb  2021-10-28 17:01:38  chamlis                   /home/seanh
09f6ec02  2021-11-25 17:17:52  chamlis                   /home/seanh
053973eb  2021-12-25 19:43:58  chamlis                   /home/seanh
7939dbe7  2022-01-27 19:49:08  chamlis                   /home/seanh
43b899df  2022-02-26 16:55:29  chamlis                   /home/seanh
e377529b  2022-04-01 19:16:42  chamlis                   /home/seanh
9eb356c4  2022-04-01 20:37:36  beatsworking              /home/seanh
22a390d4  2022-04-02 23:41:36  beatsworking              /home/seanh
--------------------------------------------------------------------
16 snapshots
```

### Listing files in snapshots with `restic ls`

`restic ls` lists files within snapshots.
This is mostly useful for finding files to restore with `restic restore` (see
below).

```terminal
$ # List all files in the latest snapshot from any host.
$ restic ls latest
repository d59c3d15 opened successfully, password is correct
snapshot 7518e078 of [/home/seanh/foo.txt] filtered by [] at 2022-04-02 22:23:21.053983381 +0100 BST):
/foo.txt

$ # List all files in the latest snapshot from the current host.
$ restic ls --host "$(hostname)" latest
repository d59c3d15 opened successfully, password is correct
snapshot 7518e078 of [/home/seanh/foo.txt] filtered by [] at 2022-04-02 22:23:21.053983381 +0100 BST):
/foo.txt

$ # List all files in the latest snapshot of your home directory from the
$ # current host.
$ restic ls --host "$(hostname)" --path "$HOME" latest
repository d59c3d15 opened successfully, password is correct
snapshot 7518e078 of [/home/seanh/foo.txt] filtered by [] at 2022-04-02 22:23:21.053983381 +0100 BST):
/foo.txt

$ # List all files in a specific snapshot by snapshot ID.
$ restic ls 7518e078
repository d59c3d15 opened successfully, password is correct
snapshot 7518e078 of [/home/seanh/foo.txt] filtered by [] at 2022-04-02 22:23:21.053983381 +0100 BST):
/foo.txt

$ # List all files in the $HOME/Mail directory in the latest snapshot.
$ # When one or more directory arguments are given it only lists top-level files
$ # and folders in the given folder(s) unless you also give --recursive to tell
$ # it to recurse into subfolders.
$ restic ls --recursive latest $HOME/Mail
...
```

### Restoring files from a snapshot with `restic restore`

Use [`restic snapshots`](#listing-snapshots-with-restic-snapshots) to find the ID of the snapshot that you want to restore from.
(`f11c451b` in the examples below).
Then to restore the entire contents of the snapshot to a `/tmp/restored` directory:

```terminal
$ mkdir /tmp/restored
$ restic restore f11c451b --target /tmp/restored
```

Add `--verify` to verify the contents of the files after restoring them.

Use `--include` to restore just a single file or folder from the snapshot.
First use [`restic ls`](#listing-files-in-snapshots-with-restic-ls) to find the paths of the files you want to restore
and then run a `restore` command like:

```terminal
$ restic restore f11c451b --target /tmp/restore-work --include /foo.txt
```

`--include` can be given multiple times to restore multiple specific files or
folders at once and patterns can be used to restore all files that match a
pattern.
There's also `--exclude` to exclude specific or matching files from the
restore, restoring everything that _isn't_ excluded.
See [Restoring from backup](https://restic.readthedocs.io/en/stable/050_restore.html)
in the restic docs or `restic help restore` for details.

#### Restoring from the "latest" snapshot

You can use `latest` instead of a snapshot ID to restore from the most recent snapshot:

```terminal
$ restic restore latest --target /tmp/restored
```

`latest` on its own means the most recent snapshot **from any host** and **of any path**.

If you have multiple hosts backing up to the same repo and you want to restore
the latest snapshot from the current host use `--host "$(hostname)"` with
`latest`.

If you back up multiple paths to the same repo and you want to restore the
latest snapshot of your home directory use `--path "$HOME"` with `latest`.

Here's an example combining both:

```terminal
$ restic restore --host "$(hostname)" --path "$HOME" latest --target /tmp/restored
```

Maintaining your backup
-----------------------

<div class="warning" markdown="1">
If your backup is large the `restic prune` and `restic check` commands can
download a lot of data (potentially gigabytes) because they need to load all
your indexes and snapshots. This can take a long time and cost you a lot of
money in B2 download fees. For this reason you might not want to run `prune`
and `check` too frequently. I run the `maint` script below once a year.
</div>

### Deleting snapshots with `restic forget` and `restic prune`

You can delete old snapshots to save storage space.
To delete a specific snapshot or snapshots get the snapshot ID(s) from
[`restic snapshots`](#listing-snapshots-with-restic-snapshots) and pass them to `restic forget`. For example:

```terminal
$ restic forget f11c451b bea27147 f7e72151
repository d59c3d15 opened successfully, password is correct
[0:02] 100.00%  3 / 3 files deleted...
```

Just forgetting snapshots doesn't actually save any space.
You then have to run `restic prune` to find and delete the data that's no
longer used by any of the remaining snapshots:

```terminal
$ restic prune
repository d59c3d15 opened successfully, password is correct
loading indexes...
loading all snapshots...
finding data that is still in use for 17 snapshots
[2:37] 100.00%  17 / 17 snapshots...
searching used packs...
collecting packs for deletion and repacking
[1:01] 100.00%  181618 / 181618 packs processed...

to repack:             0 blobs / 0 B
this removes:          0 blobs / 0 B
to delete:             5 blobs / 1.986 KiB
total prune:           5 blobs / 1.986 KiB
remaining:       5877282 blobs / 862.804 GiB
unused size after prune: 16.160 GiB (1.87% of remaining size)

rebuilding index
[7:50] 100.00%  181616 / 181616 packs processed...
deleting obsolete index files
[0:17] 100.00%  120 / 120 files deleted...
removing 2 old packs
[0:00] 100.00%  2 / 2 files deleted...
done
```

Restic's docs advise you to run `restic check` (see below) after doing a `restic prune`.

To automatically select which snapshots to delete use `restic forget` with one
or more `--keep-*` options. There are lots of `--keep-*` options, like
`--keep-last n` to keep the last `n` snapshots or `--keep-hourly n` to keep only
one snapshot per hour for the last `n` hours that have snapshots.
See [Removing snapshots according to a policy](https://restic.readthedocs.io/en/stable/060_forget.html#removing-snapshots-according-to-a-policy)
in restic's docs for all the details. Any snapshots that don't match one of the
given `--keep-*` options will be deleted.

<div class="warning" markdown="1">
By default `restic forget` will delete snapshots **from all hosts** and **of any path**.

If you have multiple hosts backing up to the same repo add
`--host "$(hostname)"` to limit restic to only deleting snapshots from the current
host.

If you back up multiple paths to the same repo add `--path "$HOME"`
to limit restic to only deleting snapshots of your home directory.
</div>

Use `--dry-run` to see which snapshots would be deleted before actually
deleting them.

`--tag <TAG>` will limit restic to only deleting snapshots with the given tag(s).
You can use `--tag` multiple times to delete snapshots with _any_ of the given tags,
for example `restic forget --tag foo --tag bar ...` will consider backups with _either_
the tag `foo` or the tag `bar` for deletion.
Or you can pass a comma-separated list of tags to `--tag` to only delete snapshots with _all_ of the given tags,
for example `restic forget --tag foo,bar` will only consider snapshots with _both_ the tags `foo` and `bar` for deletion.

Use `--tag ''` to limit restic to only deleting _untagged_ snapshots.
This way you can protect certain snapshots from deletion by tagging them.
Either use `restic tag --add foo <SNAPSHOT_ID>` to add a tag to an existing
snapshot or pass `--tag foo` to the `restic backup` command when you first
create the snapshot. For example to add the tag `foo` to the latest snapshot:

```terminal
$ restic tag --add foo latest
```

See `restic help tag` for more documentation on managing tags.
You can use [`restic snapshots`](#listing-snapshots-with-restic-snapshots) to see which snapshots have which tags.

Here's an example `restic forget` command using some of the `--keep-*` options:

```terminal
$ restic forget --dry-run \
                --host "$(hostname)" \
                --path "$HOME" \
                --tag '' \
                --keep-within-daily 7d \
                --keep-within-weekly 1m \
                --keep-within-monthly 1y \
                --keep-within-yearly 100y
```

This will keep daily snapshots for the last week,
weekly snapshots for the last month,
monthly snapshots for the last year,
and yearly snapshots for the last century.
"Last week/month/year/century" are relative to the date of the latest snapshot,
not to the current date.
In addition it will keep all snapshots that are not from the current host,
all snapshots that are not of your home directory,
and all tagged snapshots.
All other snapshots will be deleted (if the command is re-run without the
`--dry-run`).

Again, you'd then need to run `restic prune` to actually free up the space.

### Verifying a backup with `restic check`

The `restic check` command verifies the integrity of your restic repository:

```terminal
$ restic check
using temporary cache in /tmp/restic-check-cache-423433534
repository d59c3d15 opened successfully, password is correct
created new cache in /tmp/restic-check-cache-423433534
create exclusive lock for repository
load indexes
check all packs
check snapshots, trees and blobs
[2:02:10] 100.00%  130 / 130 snapshots
no errors were found
```

`restic check` just checks the "structural integrity and consistency" of the
backup. If you want to check that the actual backed up files are correct you
can use `restic check --read-data` but this will download all the files in
the repository which can take a long time and add a lot of money to your
Backblaze bill. Alternatively you can use `restic check --read-data-subset=1G`
to check just a random one gigabyte subset of the data:

```terminal
$ restic check --read-data-subset=1G
using temporary cache in /tmp/restic-check-cache-844156038
repository d59c3d15 opened successfully, password is correct
created new cache in /tmp/restic-check-cache-844156038
create exclusive lock for repository
load indexes
check all packs
check snapshots, trees and blobs
[2:11:01] 100.00%  130 / 130 snapshots
read 1G of data packs
[0:06] 100.00%  1 / 1 packs
no errors were found
```

`restic check` with `--read-data` or `--read-data-subset` first checks the
repo's structural integrity and consistency (everything that a plain
`restic check` would do) and _then_ downloads and checks the files.

There are other ways to specify the subset to check as well,
see [Checking integrity and consistency](https://restic.readthedocs.io/en/stable/045_working_with_repos.html#checking-integrity-and-consistency)
in the restic docs.

### Create a maintenance script

To make maintaining your backup easy create a `~/.restic/maint` script that runs
`restic forget` with some `--keep-*` options, then `restic prune`, then `restic check`.
Here's an example:

```bash
#!/bin/bash
set -euo pipefail

source ~/.restic/env.bash

restic forget \
    --host "$RESTIC_HOST" \
    --path "$RESTIC_PATH" \
    --tag '' \
    --keep-within-daily 7d \
    --keep-within-weekly 1m \
    --keep-within-monthly 1y \
    --keep-within-yearly 100y

restic prune

restic check --read-data-subset=1G
```

Make the `maint` script executable:

```terminal
$ chmod u+x ~/.restic/maint
```

Now to weed out old snapshots and verify your backup you can just run:

```terminal
$ ~/.restic/maint
```

Accessing another host's backups
--------------------------------

Sometimes you might want to access one host's backups from a different host, for
example to recover a different host's files or to run maintenance
commands for another host's backups. I like to run the maintenance commands for
all my backups from my main desktop machine, it saves having to run them on
each laptop separately and saves the laptops from being occupied by long
running commands.

An easy way to do this is to create a separate `env.HOSTNAME.bash` file for
each host whose backups you want to access:

```terminal
$ ls ~/.restic/*.bash
/home/seanh/.restic/env.bash
/home/seanh/.restic/env.beatsworking.bash
/home/seanh/.restic/env.xenocrat.bash
```

The `env.bash` file is for this host's backups, the `env.beatsworking.bash` and
`env.xenocrat.bash` files are for `beatsworking` and `xenocrat`'s backups (the
hostnames of two laptops that I back up).
If you use fish-shell you'll want to create `env.HOSTNAME.fish` files for each host as well.

The contents of each `env.HOSTNAME.bash` file should be the same as the `env.bash`
file but hard-coding the appropriate hostname and home directory instead of
using `$(hostname)` and `$HOME`. Here's an example:

```bash
#!/bin/bash
set -euo pipefail

RESTIC_HOST=xenocrat
RESTIC_PATH=/home/shan
RESTIC_REPOSITORY="$(pass "$RESTIC_HOST"/RESTIC_REPOSITORY)"
B2_ACCOUNT_ID="$(pass "$RESTIC_HOST"/B2_ACCOUNT_ID)"
B2_ACCOUNT_KEY="$(pass "$RESTIC_HOST"/B2_ACCOUNT_KEY)"
RESTIC_PASSWORD_COMMAND=pass\ "$RESTIC_HOST"/RESTIC_PASSWORD

export RESTIC_HOST RESTIC_PATH RESTIC_REPOSITORY B2_ACCOUNT_ID B2_ACCOUNT_KEY RESTIC_PASSWORD_COMMAND
```

For the `env.HOSTNAME.bash` scripts to work you'll need to add a
`RESTIC_REPOSITORY`, `B2_ACCOUNT_ID`, `B2_ACCOUNT_KEY` and `RESTIC_PASSWORD`
for each host for the password store:

```terminal
~> pass ls
Password Store
├── beatsworking
│   ├── B2_ACCOUNT_ID
│   ├── B2_ACCOUNT_KEY
│   ├── RESTIC_PASSWORD
│   └── RESTIC_REPOSITORY
├── chamlis
│   ├── B2_ACCOUNT_ID
│   ├── B2_ACCOUNT_KEY
│   ├── RESTIC_PASSWORD
│   └── RESTIC_REPOSITORY
└── xenocrat
    ├── B2_ACCOUNT_ID
    ├── B2_ACCOUNT_KEY
    ├── RESTIC_PASSWORD
    └── RESTIC_REPOSITORY
```

When I do this I create a new application key in Backblaze and add a second
`RESTIC_PASSWORD` to the repo using the `restic key` command (remembering to
back up the `RESTIC_PASSWORD` in Bitwarden) rather than copying the same key
and password to two machines. This just means that each machine's key and
password can be revoked separately.

Now you can easily move between backups by just sourcing the env files and then
running restic commands:

```terminal
$ source ~/.restic/env.bash
$ restic check
...
$ source ~/.restic/env.beatsworking.bash
$ restic check
...
$ source ~/.restic/env.xenocrat.bash
$ restic check
...
```

I extended my annual maintenance script to source each env file in turn and run
my maintenance commands on each repo:

```bash
#!/bin/bash
set -euo pipefail

function maint {
    # Print this so you can tell which backup restic's output belongs to.
    echo "maint $1"

    source "$1"

    restic forget \
        --host "$RESTIC_HOST" \
        --path "$RESTIC_PATH" \
        --tag '' \
        --keep-within-daily 7d \
        --keep-within-weekly 1m \
        --keep-within-monthly 1y \
        --keep-within-yearly 100y

    restic prune

    restic check --read-data-subset=1G
}

for env_file in ~/.restic/env*.bash; do
    maint "$env_file"
done
```

restic's built-in help
----------------------

restic has built-in reference documentation for all its subcommands and options.
For a list of restic's subcommands and global flags run `restic help` or
`restic --help`:

```terminal
$ restic help
```

For documentation of a subcommand and its options run `restic help <SUBCOMMAND>`
or `restic <SUBCOMMAND> --help`, for example:

```terminal
$ restic help backup
```

Troubleshooting
---------------

### "Please specify repository location" error

If you get this error:

    Fatal: Please specify repository location (-r or --repository-file)

It probably means you forgot to source your env file. Run:

```terminal
$ source ~/.restic/env.bash
```

And then re-run your original command.

### "unable to create lock in backend" error

If you get this error:

    unable to create lock in backend: repository is already locked exclusively by PID ...
    ...
    lock was created at ...
    storage ID ...
    the `unlock` command can be used to remove stale locks

Just run the `restic unlock` command, as the error message says:

```terminal
$ restic unlock
```

And then re-run your original command.

Stale locks can happen if a restic command crashes or if a computer that was running
a restic command crashes or loses power etc.

TODO
----

### `restic find`

There's a `restic find` command for finding files that match a pattern,
searching across all snapshots at once which could be useful. It's poorly
documented and didn't seem to behave as I expected it to. Just grepping the
output of `restic ls` can probably get you pretty far.

### `restic mount`

There's a `restic mount` command for mounting your snapshots as a browseable FUSE filesystem.
You can restore files by just copying them normally.

```terminal
$ mkdir /tmp/restic
$ restic mount /tmp/restic
repository d59c3d15 opened successfully, password is correct
Now serving the repository at /tmp/restic/
Use another terminal or tool to browse the contents of this folder.
When finished, quit with Ctrl-c here or umount the mountpoint.
```

I found `restic mount` to be buggy when I tried it.
It worked once, but after that the mounted filesystem kept appearing to be empty.

### What if `restic check` finds a problem?

What should you do if `restic check` reports a problem with your backup?
I've never had this happen so I don't know how to handle it.
Will restic offer to fix the problem or suggest a solution?
Should you create a new repo and start again from scratch?