speak of the dedup

We all have our own robust and reliable backup solutions so there’s no point in me telling you how important backups are, or how more important a working restore is.

The /home partition on my main Linux system is a 3 disk RAID5 array giving me 3TB of storage with some resilience. I use dedup for backing up important files on a daily basis, combined with rclone to my cloud storage giving me an offsite backup. Every so often I will do a full backup to an external 5TB encrypted drive, which lives in my bug out bag.

To use dedup you have to first initialise a repository

dup-init <repo>

Encryption is always a good idea so instead you can generate a secret key, then use the key to initialise a repo

dup-keygen <keyfile>
dub-init -k <keyfile> -E XChaCha20-Poly1305 <repo>

Make sure you keep that key safe and secure, it is required for restoring.

It is best to use tar for backing up directories as dedup only handles single files. To backup a directory into the new repo incant

tar -cf - /path/to/dir | dup-pack -k <keyfile> -r <repo> <snapshot_name>

The snapshot_name can be anything you want, I like to use the date.

Restoring a backup is straightforward, using tar again to extract the archive

dup-unpack -k <keyfile> -r <repo> <snapshot_name> | tar -xf -

My daily cronjob runs a script to backup a list of important files and directories then sync the backups with my cloud storage using rclone.

Once rclone is authenticated with your cloud storage the command to copy is literally that

rclone copy /local/path/to/backups <storage_name>:/remote/path

My script uses ntfy to alert me if a backup fails (see my previous post on ntfy).

My full backup is not automated as it requires unlocking my external drive, so I don’t do this as often as I would like. I usually pick a weekend every couple of months when I will get the drive out and back up all my devices.

I try to run a practice restore every few weeks to ensure everything is still in good form and the process becomes habit. Panicking over lost data is never fun, so at least if I can perform a restore calmly I won’t be completely losing my mind.

This system has been working well for me for a long time. I try not to overcomplicate things, K.I.S.S. as they say.