Our organization have been running Veeam 2.0 for about half a year, and we were excited to see 3.0 with many of the features we have requested. Our organization has about 200 servers (95% virtual) and 2000 clients. We have been through several organizational changes during the last few years, buying several companies and joining together with others. This of course brings a lot of challenges, especially in data management. Each company has it's own systems and it's own history of data, often stretching back more than 10 years.
The claim is that data throughput and performance is not an issue with Veeam Backup 3.0. It's a pretty strong claim to make, especially from a customer.
Our VMWare setup is as follows
- 13 ESX servers, Dell PowerEdge R900 servers. Each with 4x quad-core CPUs and 64GB of RAM, totalling 208x 2.93Ghz cores and 832GB of RAM. CPUs are Intel Xeon X7350.
- Two Veeam Backup servers, Dell PowerEdge 2950, with 1x Intel Xeon E5335 CPU and 4GB of RAM. Each has 2x Qlogic 4Gbps FC cards.
- 3 SANs: 2x Dell|EMC CX4-240 (production) + 1x Dell|EMC CX3-40 (backup/archive). All connected via 4Gb/s Fibrechannel.
The two SANs are mirrored and the ESX servers are split into two clusters: Cluster1 and Cluster2. Each of the clusters have their datastores on different CX4-240's.
- Cluster1 has its datastores on EMCPRI (CX4-240 #1). The datastores are then mirrored over to the other CX4-240 (EMCSEC) via EMC MirrorView/Sync.
- Cluster2 has its datastores on EMCSEC (CX4-240 #2). The datastores are then mirrored over to the other CX4-240 (EMCPRI) via EMC MirrorView/Sync.
The LUNs are configured as such:
- Each CX4-240 has 11 shelves of 15 disks each (165 disks per CX4-240). 3 shelves (45 drives) are 450GB FC4 drives. The other 8 shelves are 300GB FC2 and FC4 drives (about half of each).
- Each 300GB shelf is configured as a single Raid 5 Raidgroup of 14 disks, while the 15th disk is configured as a hot-spare.
- These are then split into 3 LUNs that span the 14 drives: 2x 1TB generic LUNs (1MB blocksize, max 256GB vmdk) and 1x 1,7TB large LUN (4MB blocksize, max 1024GB vmdk).
- The setup for the 450GB-shelves are pretty much similar, but there are 2x 1TB + 2x 2TB.
This makes a total of 36 datastores.
The Dell|EMC CX3-40 backup SAN consists of (for Veeam):
- 3 shelves of 15 disks each. 2 shelves (30 disks) of 146GB FC2 drives, 1 shelf (15 disks) of 300GB FC2 drives.
- The two 146GB shelves are grouped together 14+14 (Raid5) and joined together into one big LUN of 2,97GB and is gives to the one Veeam backup server.
- The 300GB shelf is grouped together 14 disks (Raid5) of 3,4TB.
- These are the backup targets for the two Veeam Backup servers.
Now for the interesting bits...
The Veeam Backup setup
- Each of the two backup servers have several jobs configured.
- Backupserver1 has 8 jobs, and Backupserver2 has 10 jobs.
- All jobs are scheduled to start at 7pm each night, 1 minute after each other (7:01, 7:02, 7:03, etc).
- Compression is left at "optimal" and VSS is enabled for all servers (Many fail VSS snapshot, many work. Mostly due to multiple domains. Set to backup even without successful VSS.)
- BackupServer1 backs up the following: Fileservers (5 individual jobs, one per server), Generic VMs, Archive/History, SQL Servers. Those are the actual jobs.
- All jobs are SAN VCB jobs, except for SQL Servers which is network based vmdk 0:0 only (system drive only).
- Total data to be backed up: 2547GB. 62 VMs.
- BackupServer2 backs up the following: Citrixservers, Emailservers, ERP, Monitoring/NOC, Portal, Central Servers/AD, Development, Webservers, Templates and Testservers. Those are the actual jobs.
- All jobs are SAN VCB jobs, except for Templates which is network based.
- Total data to be backed up: 3098GB. 79 VMs.
- Total data to be backed up daily: 5645GB.
Results
- All backup jobs started at scheduled times, and no problems seen so far with corrupt .vbk datastores (seen in 2.0.1) or other errors.
- With Veeam 2.0.1 the last backup jobs usually finished about midday (or even later) on the following day. They were usually started at 6pm, instead of 7pm as now. However, we were not able to run more than 2 or 3 jobs at a time with 2.0.1, as more simultaneous jobs usually caused failures (often with .vbk corruption).
- With Veeam 3.0 the last backup job on BackupServer1 finishes at 01:33am, and the last backup job on BackupServer2 finishes at 00:37am.
Average data speeds based on these (incremental) jobs are as follows:
- Backupserver1: 110MB/s
- Backupserver2: 157MB/s
- Combined: 267MB/s
- Peak combined speed was recorded at around 500-600MB/s.
File size on disk is as follows:
- Backupserver1: Data was 2547GB, one (full) backup is 1669GB. This is 65% of initial size (35% compression/dedup).
- 3 days worth of backup (1 full + 2 historical) results in a total filesize of 1955GB. This means the backup increases by 143GB per extra day, or 5.6% of initial data size.
- BackupServer2: Data was 3098GB, one (full) backup is 1598GB. This is 52% of initial size (48% compression/dedup).
- 3 days worth of backup (1 full + 2 historical) results in a total filesize of 1895GB. This means the backup increases by 148GB per extra day, or 4.8% of initial data size.
- Combined: Data was 5645GB, one full is 3267GB or 58% of intial size; a compression/dedup factor of 42%.
- 3 days worth of backup results in a growth of 291GB per day or 5.2% of datasize.
We have a total of 6.4TB of backup space available to backup 5.5TB (a factor of 116%), which means we could hold up to 10 days of historical data.
Or in other words, the full backup takes up 58% of the intial data size, and each incremental takes up another 5.2%.
By using these numbers I can calculate that to keep 30 days historical backups I need at least 214% of the intial data size (or in our case, I would need 12080GB (11,8TB) of available backup space.
There you have it..
Is this a scientific research report? No, not in any way! However, it does show one customer's experience with real life data and a real life setup. You might not end up near these numbers, or you might surpass them (good for you!), but I merely decided to share this with you all because then you have an independent source for reference. There might be typos above, however I have double checked the numbers to ensure they should be right. However, something might have slipped past me.
I want to stress that I am not paid by Veeam in any way shape or form, but I am a happy customer that was delighted to see a real VMWare backup solution appear. Veeam Backup 3.0 will allow me to sleep better at night.
I feel I must send one stern warning to Veeam though: Please do not forget where you came from. I see nothing stopping you from becoming the market leader in VMWare backup, so when you one day reach that peak, please remember that what made the product awesome was listening closely to the customers and delivering! Now... there I saved you a whole "back to basics restructuring" in 5-7 years!
"When you're green you grow, when you're ripe you rot". Veeam is definetly a green company
Kind regards,
Even Glemmestad
IT Engineer
Felleskjopet Agri BA, Norway
