Here's what we do today:
The most important part is that we have a dedupe appliance, a DD620 to be specific. We have plans to keep a dedupe appliance, though it may not always be a Data Domain… We take weekly fulls with nightly incrementals. This ensures that we have regular full backups without the overhead of synthetic transformations, which (this is important later) I’ve read really don’t like to behave with dedupe targets because of their random, 1-read+1-write IO needs. Right now our backups are taken from production and put on the DD where our retention policy keeps them around for a couple months. They then get magically cleaned up, one week at a time, by Veeam doing its thing.
Here’s what I want to do tomorrow:
Based on the Ultimate Backup Architecture blog post and follow-up webcast, I want to introduce a new primary storage target that’s fast, small, and cheap. From there the backups will move to a dedupe appliance for retention. Primary backups would be kept a week or two while deduped backups would be retained a few months or more. We’d be able to maintain RPO, improve RTO, lower TCO, and shrink our backup window. All the important buzzwords are achieved.
Here’s the problem:
I can’t figure out how I’m supposed to do this. I know that it can be done, but how do I do it efficiently? Backups should be captured during their window and the copies should be made during their window. The first solution is a group of Backup Copy Jobs. The second is a custom PowerShell script.
Pros of using Backup Copy
- Veeam handles the movement of data from primary to secondary storage
Veeam handles the two different retention policies for each “type” of media
Veeam moves only new or changed blocks from primary to secondary regardless of whether the source file is full or incremental, which means data should move faster
Veeam is my backup platform and all my backup tasks stay inside Veeam
- Veeam moves only new or changed blocks from primary to secondary regardless of whether the source file is full or incremental, which means transforms have to be performed to get synthetic fulls
Veeam documents and forums recommend against using and synthetic operations with dedupe appliances because of the heavy, random IO workload
Veeam Backup Copy Jobs are literally always running and like to log failures when the server reboots (this is more of an annoyance…I personally would prefer if the jobs, which we’d only allow to run when we’re not doing production backups anyway, had a true scheduled start time and ran like a normal backup job)
- I can do exactly what I want without any changes to Veeam and on my backup window schedule
- I have to write and maintain the whole script
I might be duplicating features (that I can’t figure out) and wasting time and resources
I must move every single new backup file, which could be slower than intelligent Backup Copy Jobs
(Note: I did review a forum post at http://forums.veeam.com/veeam-backup-re ... t1860.html from a couple of years ago where someone was asking about simultaneous targets. The point Gostev made was that it might be doable [in that particular situation] but the overall backup job would only work at the slower of the two targets. So while read-once-write-twice would be neat it still leaves a potential bottleneck. But, maybe this is still an option if enough customers have two targets that can both keep up with the single read from production storage. We currently fit our backups into our backup window, so we’d be able to take advantage of it even if it’s not the FASTEST option for us. Of course, then I’m looking to the future where we might be outside our backup window and need the speed WITH a dedupe appliance still in the picture…it’s a vicious cycle when you have to save money. Otherwise I’d just own a bunch of pods like https://www.backblaze.com/blog/cloud-storage-hardware/.)
So then my distilled questions:
Can I make Backup Copy Jobs play nice with dedupe storage? Am I overthinking it and everything will actually be fine (besides those quirks of always-running jobs that bug me)? Or do I need my own script that lets me do what I want but not within Veeam itself?
Any and all thoughts are much appreciated.
Thanks!
Alan