Backup to rotated discs, best practice big picture

[christianstauffer]

Hi,

I'm sorry if that's a FAQ, but I didn't find an answer. We are currently backing up all our data with a combination of Backup Exec, VMExplorer, Acronis and Robocopy to a tape and disc destination. We want to consolidate all of that with Veeam on a new Backup machine. I'm currently in the process of putting together a shopping list for our hardware. I don't need detailed instructions how to set up the backup, but I'm wondering what's the general strategy when backing up "large" amounts of data to external disks daily.

We have about 5TB of data we need backed up. This consists of about 50-60 VMs of all sorts, Oracle databases, file servers, SQL servers, etc. ppp.

Target will be 8-10TB external hard disks, the fastest ones I can get my hands on (so 7200 rpm probably). My problem is that even our current backup volume of 5TB, at a very optimistic 100MB/s, will take 14 hours. Now I know that Veeam is quite smart with the synthetic full backups etc, but I'm not sure. What's the quickest way to get the data out onto the external disks?

From the top of my head, not knowing exactly how Veeam handles things, I would have though of the following options:
a) Backing up directly into a repository of rotated disks
b) Backing up into a local repository on the backup server (RAID5 of 10k or even SSDs), then "mirroring/exporting" this backup to the external drive

In both cases I assume Veeam will not suffer from performance loss due to small files being written. But what's the fastest way to export this data? We have a 18h window, ultimately that's the hard limit. Is this even realistic?

Thanks a lot, Chris

[Adrian1980]

Hello Christian,

to variant a.) No, don't do that, you not will be happy with this approach.
Variant b.) looks the better way, you do a backup to local storage und then do Backup Copy Jobs to external disks.

What you mean with this 18h limit? the time for backup AND export must have finished in this time?

You write, that you don't need exakt instructiuons, but let me say:
Veeam looks like easy, bit in the detail it is complex offer room for bad designs, especaially as a beginner in Veeam, because Veeam handles things different than other software.

Don't unterestimate veeam!

[Adrian1980]

For the questions regarding the time limits, it is usefull to have more information about your environment and what veeam design you planned, espacially for the proxys and the way how you planned to get the data out of your VMware.

[christianstauffer]

Hi Adrian,

Thanks for the reply. By "I don't need detailed instructions" I'm talking about right now. I'll kneel into the docs and the forum and I'm sure I'll be back with many detailed questions.

By 18 hours I mean that we'll start the whole backup process at ~20:00 and it should be done by 14:00 the next day so we can change the disks and take them off site. Maybe 16:00, so 20 hours in total tops. Any more than that and the scenario is not workable since we want to be able to rotate the disks daily. That's why I'm asking: If Veeam performs a full backup to the external disk, the schedule is going to be very tight. If Veeam has a way of performing an incremental backup to the external disk, even though the last backup on that disk is 30 days old, then that means there is probably enough time. Same if Veeam manages to compress/deduplicate "average" files when writing them into the repository.

We haven't planned a Veeam design yet, that's the stage I'm at at the moment. We have a VMWare cluster of 3 servers, managed by a virtual VCenter. That's the bulk of the data. We need to have object level restore for 2 Active Directories, we don't need "agents" for anything else.

I'm currently thinking of buying a DL380 G10 with an 8 core Xeon, 32GB RAM and 8 2.4TB drives, configured to one RAID5 of 7 disks plus a hot spare. This would mean about 13 TiB of net storage to hold the OS (separate partition) and the main repository, and then "exporting" that to the rotated external disks.

Thanks, Chris

[Adrian1980]

Hi Christian,

if you do backup to Repo and from them to rotaable drives, in case of a good job design, the copy to rotate can start working while backup from vmware is still in progress.
I think, with the info you provide, you don't need so much time, in a proper environment.

Did you plan the Backup from the VBR-Server via direct-san-access?
Or did you take backup via proxy placed as VM in Vmware? Or VBR as proxy with network access to VMware?
How allt this things are connected on network level?

[Adrian1980]

The Veeam Backup Copy Job works always as forever incremental.

The behavior is like this:

Day 1, Disk 1: Active Full A1
Day 2, Disk 2: Active Full B1
Day 3, Disk 1: Inc A2 based on A1
Day 4, Disk 2: Inc B2 based on B1
Day 5, Disk 1: Inc A3 based on A2-A1
Day 6, Disk 2: Inc B3 based on B2-B1

You see, Veeam is aware of the rotated drive and ensure, on each disk a separate chain exists.
What consumes time here, is the process of merging incs into the full, this is an real big impact on USB, but it seems you use something faster.
You change disks daily and veeam does forever inc, so the diff between the changes are not so big.
Theoretically you can archive your goal of 18 hours easy, but at this stage this is theory.

[christianstauffer]

Ok, thank you, that's the info I was looking for. So it seems the actual volume copied to the external disk ins't "too" big.
Do these incremental backups grow and grow and eventually fill the rotated disks, though?

As for what way we back up the VMs, I think I'll have to look into that. I naively assumed that Veeam would connect to our Vcenter and get the data from there, but if there's a quicker way ("direct-san-access" sounds quicker ), I'll have to look into it. We have HPE Storevirtual SAN, I'm not sure if that supports anything like that. Anyway, you've given me a few very good pointers for things to look into, thank you.

[Adrian1980]

The part who doe the backup is the Veeam Proxy.
The proxy can do backup by san, hotadd and ndb.

As you identified, direct-san is the fastest way and always used with physical proxys.
The next way and often used for smaller environments, you implement one or more VM as proxy in you’re environment to backup.
This adds vmdk to the proxy and reads the vmdk.

ndb is the slowest, you only use it, if you have no other choice.


That’s what I mean, as I said you should not underestimate veeam and it’s design.
You have to care about design aspect, before coming to the shopping list. You have to investigate this Questions.
I think it is possible, that one or two virtual proxy fits your needs properly, if you have 10gbe between virtual-proxy and Repository-Server.


Greets,
Adrian

[Adrian1980]

As another option, you can check if your HPE storevirtual san has the ability to use storage snapshots with veeam for your backup, that would be even faster then direct-San.