Photography backup and archiving – plan for disaster

Protecting your work with a successful photography backup plan

Photography Backup - MemeI’ve been asked how I protect my artwork, images and related information with a dedicated photography backup plan. I take a belt and suspenders approach with multiple copies on and off site using both free and subscription services. When approaching a backup solution or plan, first you really need to think about what you are backing up, what your loss acceptance is, and budgets. You really can build an overkill photography backup solution that will cost far more than necessary, you can roll the dice and have no backup for no cost, or anywhere in the middle of these two extremes.

While I have posted about frustrations with Apple recently, I am still a Macaddict and parts of this would be applicable to Mac or Linux workstations, and may or may not be viable for Windows. I do know recent Windows OS updates have made a Linux subsystem available, and perhaps this will work for those systems – I have not had a chance to test so YMMV.

Step 1: Acceptance is the First Step to (Data) Recovery

Photography Backup? Hopefully not using this ancient tech

Photocredit: Wikipedia Commons: https://goo.gl/x8cMie

Let’s start by first accepting the simple hard fact, hardware fails, software fails, malware and ransomware happens and data corrupts. This is a simple fact that you need to accept and plan for. Start from the expectation you are powerless to prevent the majority of this, and plan your photography backup for successful recovery.

What files can you live without? Which files would cause the end of the world and existential crisis should they disappear into the ether? Which files do you need to support the later? What the hell is with all these questions?

Ok, as this is targeted towards my photography backup planning, we should start there. Think about your workflow and what can be restored from install media, and what cannot. I use Lightroom, On1 RAW, AuroraHDR and to a lesser degree Photoshop. Reinstalling applications is straightforward, but reconfiguring them would be a royal PITA so I’ll skip backing up the Apps, but I do need to backup their configuration files. I also want to backup all my Lightroom catalogs, and the original RAW files, working versions, and final images themselves. Versioning of backups is also desired and has saved my butt more than I want to admit.

Step 2: Acknowledge “stuff” will happen, and compensate for it

My Lightroom catalogs, images and RAW files are all stored and worked on a rugged external drive that I travel with. I keep this drive limited to what I am working on, and images taken while traveling. I use an external RAID attached storage for long-term storage and working on older photos. To keep things in sync, I use rsync commands to move data from my traveling device to the RAID box.

Another thing to keep in mind. I never delete images off my camera’s storage card until I get home and complete this first sync. If I lose my external drive, or somehow dies, I still have all the originals. I also have a cloud based backup solution that I can use if I have decent connectivity, but that will be covered later…

All of my images are stored on the portable/rugged drive in /Volumes/ExternalDrive001/LightRoom Catalogs and in a similar file structure on the RAID enclosure. I have a bash script in the root of my Lightroom directory “syncFromTravelToRAID.sh” containing the following:

echo "STARTING SYNC"
rsync -arhv --exclude '*.sh' "." "/Volumes/ExternalLaCieBoxes/Lightroom Catalogs"
echo "----DONE-----"

I can execute this as part of my workflow. What this does is tell my Mac to copy anything that has changed (but not deletions) except files ending in .sh (shell scripts) from my traveling drive to my desktop RAID box. I can then delete anything I don’t need on the travel drive to free up space. Once I confirm the photography backup worked I also clean up my camera’s storage cards. 

More Belts and Suspenders?

Now I have essentially one copy as the travel drive is ephemeral and I delete from it regularly. Luckily I have a home 4 Disk Synology NAS device with around 14TB of space for music, movies, and yes, photography backups. I use a similar script to make this happen, but I do this via the network using SSH and not direct mounted drives as in the example above.  This script “syncToNAS.sh looks like this:

echo "STARTING SYNC"
rsync -arhv "." <IP OF NAS>:/volume1/homes/jhoke/rsync_files/LaCieExtDriveLightroomImages/
echo "----DONE-----"

This will archive everything that has changed on the external RAID device is sent via rsync over ssh to my NAS. You may note there is nothing in this script for password authentication. I have setup SSHKey Authentication for unattended authentication for this purpose. I won’t go into the specifics of this here, but there are numerous tutorials on the internet on how to get this going – something I strongly recommend.

Now I have two copies of my photography backup sets, but I haven’t touched on the various configuration files, library files, etc. that I mentioned above in Step 1. I use two methods for this as well – remember belt and suspenders?

Step 3 (to the right): Let’s do the Time-Warp Again!

It’s just a jump to the left..

The Synology NAS natively supports Apple TimeMachine backups, and I back my MacBookPro up to the TimeMachine service on the NAS. I do not include my external drives in this backup, but it does grab all the files in ~/Library and /Library among others. This ensures that if my laptop dies, corrupts or otherwise goes sideways, I can restore to a working version. If I needed to, I could use CloudSync on the Synology to backup the TimeMachine files to Dropbox, Amazon Glacier, or other services. I take a different route for offsite storage, so this is overkill (even for the paranoid like me).

Step 4: Prepare for local catastrophe

Photography Backup via CrashPlanThe one flaw in my photography backup solution up to now is that everything is stored in my home. In the event of a local catastrophe, fire, flood, zombie apocalypse or what-have-you, I would be screwed if my RAID and NAS both were impacted. This is where cloud backup comes into play.

From my Mac I backup the various files that are important as indicated in Step 1 above to a CrashPlan subscription. I use this on all my family’s Macs and Windows devices’ critical files. 

I configured CrashPlan to automatically backup my Mac’s Library files, as well as folders my external devices that I wanted to have an offsite copy of. This protects me from a catastrophic event at the house, or if I need to retrieve a version of a file while on the road. You may wonder why if I do this to the cloud do I also keep two backups at the house. The answer is that this provides me some resilience to file corruption, as well as the fact that restoring from CrashPlan takes a LONG time. If I can restore from a faster, local storage location so much the better. CrashPlan becomes my photography backup of last resort but well worth the $100 or so a year I pay for our computers. 

Backup planning and execution takes time and resources, but this process has saved me from data loss and worse a number of times. Like I said, drives fail, sectors corrupt, humans screw up, but a well designed and executed backup and restore plan can save you from the nightmare that is losing all your hard work.

Step 5: What about Restore? Did you forget that part?

Photography Backup - Restoring is the whole point!One thing I have to stress – test your backup plan! The best plan on paper may have a fatal flaw, so test restoring files often! There is nothing worse than thinking you have solved world hunger with your amazing backup plan only to find out that it cannot restore your data correctly!

Up till now, I concentrated on photography backups, the reality is that files are files and this could be adapted with little effort to any backup need. I hope this helps the folks who asked about how I do this, and if there are questions about this please do leave a comment below!

Share Some Love!


This entry was posted in Computing, Photography and tagged , , , , , , .