-
- Enthusiast
- Posts: 35
- Liked: never
- Joined: Jan 01, 2006 1:01 am
- Contact:
3.0 Some Speed and Compression numbers
Here is some stats from my first run of 3.0. I have to say I am pretty happy with it but it does take lots and lots of processing power for high compression. Compression algorithm does multi-threading very well - all four cores were busy 100%
Here is a mix of typical win2003 servers - 2 oracle servers, version control server which compresses files, several of development servers (build server and QA server ...) . Very average because it has well compressable things like database files, lots of binaries because many our dev servers do not hold much data but windows itself, compressed data (java repositories, version control server) etc
I used High Compression. Optimal compression gives very little benefit as far as compression is concerned but you might have to use it to decrease your time
Number of VMs - 9
Data size: 310GB
Backup Size 66GB
De-dup Ratio: 44%
Compression Ratio: 48%
Here is 4 win2008 servers (ms exchange pretty full disks not much free space dedup, sharepoint, domain controller and a file server)
Data size: 200GB
Backup Size 50GB
De-dup Ratio: 56%
Compression Ratio: 44%
Speed on 2 core two processor Dell 2950:
It is processor bound (speed of compression) when high compression is used
before dedup kicks-in (actual compression performance) I was getting 30-35MB/s
once dedup kicks in average goes up to about 50-55Mb/s which is a very stable average for all FULL backups I done last night. Now the incremental portion will be much less backup CPU bound (much less data to compress) and more ESX/VCB proxy bound where changes are extracted so I expect big speed increase there and if you do network backup rather than VCB you can do 2-3 hosts in parallel and will double your speed again
Here is a mix of typical win2003 servers - 2 oracle servers, version control server which compresses files, several of development servers (build server and QA server ...) . Very average because it has well compressable things like database files, lots of binaries because many our dev servers do not hold much data but windows itself, compressed data (java repositories, version control server) etc
I used High Compression. Optimal compression gives very little benefit as far as compression is concerned but you might have to use it to decrease your time
Number of VMs - 9
Data size: 310GB
Backup Size 66GB
De-dup Ratio: 44%
Compression Ratio: 48%
Here is 4 win2008 servers (ms exchange pretty full disks not much free space dedup, sharepoint, domain controller and a file server)
Data size: 200GB
Backup Size 50GB
De-dup Ratio: 56%
Compression Ratio: 44%
Speed on 2 core two processor Dell 2950:
It is processor bound (speed of compression) when high compression is used
before dedup kicks-in (actual compression performance) I was getting 30-35MB/s
once dedup kicks in average goes up to about 50-55Mb/s which is a very stable average for all FULL backups I done last night. Now the incremental portion will be much less backup CPU bound (much less data to compress) and more ESX/VCB proxy bound where changes are extracted so I expect big speed increase there and if you do network backup rather than VCB you can do 2-3 hosts in parallel and will double your speed again
-
- Chief Product Officer
- Posts: 31632
- Liked: 7129 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: 3.0 Some Speed and Compression numbers
Thank you very much for your time Alex, I really appreciate you providing us with real-life stats.
-
- Enthusiast
- Posts: 35
- Liked: never
- Joined: Jan 01, 2006 1:01 am
- Contact:
Re: 3.0 Some Speed and Compression numbers
When doing subsequent backups in network mode with high compression, average speed goes to 100MB/s. It puts more load on ESX to do dedup, delivers much less data to backup and not any more backup processor bound. When running backups against two or free ESX hosts simultaneously we hit over 200MB/S
-
- Chief Product Officer
- Posts: 31632
- Liked: 7129 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: 3.0 Some Speed and Compression numbers
If you do backup in off-hours when ESX hosts load is minimal, this might be good solution indeed. It is true that deduping directly on source ESX server (and this is what happens during network backup) reduces the amount of data the actual backup server has to deal with significantly.
Both dedupe and light traffic compression happening on source ESX host during the network backup puts very little load on ESX (the code was specifically designed to run on ESX server and spend least CPU cycles possible). Still, this may be a concern for customers with 24/7 heavily utilized ESX servers. In these cases I definitely recommend to use VCB in order to completely offload all backup activities from production ESX host.
As your testing shows, given the high data throughput Veeam Backup is able to provide, using the high compression loads all 4 CPU cores of your backup server easily and makes the CPU processing power a bottleneck. Now, imagine this same activity happening on the production ESX server (either in a service console agent, or inside a backup virtual appliance - like some competitive solutions are doing). This would surely bring your ESX servers along with all VMs to knees. This is perfect demonstration why VCB + backup proxy approach is the only right way to perform backup in larger and heavy utilized environments. Offloading backup activities to a powerful physical backup proxy is definitely a must.
Both dedupe and light traffic compression happening on source ESX host during the network backup puts very little load on ESX (the code was specifically designed to run on ESX server and spend least CPU cycles possible). Still, this may be a concern for customers with 24/7 heavily utilized ESX servers. In these cases I definitely recommend to use VCB in order to completely offload all backup activities from production ESX host.
As your testing shows, given the high data throughput Veeam Backup is able to provide, using the high compression loads all 4 CPU cores of your backup server easily and makes the CPU processing power a bottleneck. Now, imagine this same activity happening on the production ESX server (either in a service console agent, or inside a backup virtual appliance - like some competitive solutions are doing). This would surely bring your ESX servers along with all VMs to knees. This is perfect demonstration why VCB + backup proxy approach is the only right way to perform backup in larger and heavy utilized environments. Offloading backup activities to a powerful physical backup proxy is definitely a must.
-
- Enthusiast
- Posts: 35
- Liked: never
- Joined: Jan 01, 2006 1:01 am
- Contact:
Re: 3.0 Some Speed and Compression numbers
Yes, certainly people should pick what's the best for their environment. Since we do not run 24/7 but would like to minimize backup window, running network mode backups in parallel is great. My only wish is to be able to order VM backup execution within a job. Right now it is some strange semi-random order (or at least I cannot figure the order and have no control over it) of execution so it is not possible to ensure that parallel jobs do not hit the same ESX server at the same time.
I can imagine a big company or a government agency which have large number of users VMs and tons of data but not running 24/7. For such organizations, being able to distribute all the backup work to powerful ESX servers, parallelize backups to fit them into nightly backup window could be a priority
Anyway - strategy choice and ease of use make your product attractive for big and small setups. keep it up!
I can imagine a big company or a government agency which have large number of users VMs and tons of data but not running 24/7. For such organizations, being able to distribute all the backup work to powerful ESX servers, parallelize backups to fit them into nightly backup window could be a priority
Anyway - strategy choice and ease of use make your product attractive for big and small setups. keep it up!
-
- Veteran
- Posts: 357
- Liked: 17 times
- Joined: Feb 13, 2009 10:13 am
- Full Name: Trevor Bell
- Location: Worcester UK
- Contact:
Re: 3.0 Some Speed and Compression numbers
Guys,
Here are some stats for 19 Vm`s all Windows 2003R2 Servers.
Impressive!
Here are some stats for 19 Vm`s all Windows 2003R2 Servers.
Code: Select all
Date Size Backup Size De-dupe Ratio Compression Ratio
490.18GB 70.88GB 38% 37%
-
- Chief Product Officer
- Posts: 31632
- Liked: 7129 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: 3.0 Some Speed and Compression numbers
Just to clarify numbers and values (this is commonly asked question). These percent values are provided in comparison to the original size. First, Veeam Backup performs deduplications, then compression. In this case, deduplication resulted in backup file size reduced to 186GB (38% of original data size). Then, compression resulted in backup file size reduced to 70GB (37% of deduped data size).
-
- Influencer
- Posts: 12
- Liked: never
- Joined: Apr 27, 2009 8:42 pm
- Contact:
Re: 3.0 Some Speed and Compression numbers
So, you are saying that the lower the percentage number is, the better the dedupe/compression is?
-
- Chief Product Officer
- Posts: 31632
- Liked: 7129 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: 3.0 Some Speed and Compression numbers
That is correct. The number reflect the size of backup comparing to source data size, which is 100%.
-
- Influencer
- Posts: 12
- Liked: never
- Joined: Apr 27, 2009 8:42 pm
- Contact:
Re: 3.0 Some Speed and Compression numbers
Well, I have a couple of backups with compression ratios over 100% (using Optimal compression). This would mean that after compression the file got larger. How is this possible?
-
- Chief Product Officer
- Posts: 31632
- Liked: 7129 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: 3.0 Some Speed and Compression numbers
This needs to be investigated before I can answer, please open support case with our support.
-
- Influencer
- Posts: 11
- Liked: 1 time
- Joined: Apr 16, 2009 1:09 pm
- Full Name: Duane Haas
- Contact:
Re: 3.0 Some Speed and Compression numbers
Where are you actually getting the breakdown of dedup vs compression percentages???
-
- Expert
- Posts: 155
- Liked: 40 times
- Joined: Jan 01, 2006 1:01 am
- Contact:
Re: 3.0 Some Speed and Compression numbers
Duane - Click on the backups pane in the file explorer view in Veeam Backup, and right click any backup and select "properties". This will populate this data for a given backup job.
Who is online
Users browsing this forum: droegech, gmajestix, Proban, r.horowski, Semrush [Bot] and 106 guests