Optimizing Configuration Setup

[bob.baird.xedo]

Hi All,

I'm looking for some recommendations on the best configuration for my environment.

I have 3 x VMware ESXi 5.0 hosts all connected via FC to a IBM DS3500 SAN. I have 14 Datastores created on the SAN.

Between the 3 hosts, I have approx 36 running VM's.

We are moving from Symantec Backup Exec to Veeam B&R.

I have installed a Synology DS1812+ NAS with 8 x 3TB disks. This has been configured as RAID6, so I have 18TB available storage on the NAS. The NAS is connected to the same Gigabit switch as my 3 hosts.

I have created 2 x VM's - one as my backup server (where I have installed the B&R software) and another as a proxy server.

I have created a single job to save 13 of my 36 VM's - the backups are incremental with a synthetic full being done every Saturday.

This has been configured with the following settings:

Deduplication = Enabled
Compression level = Optimal
Storage optimization = LAN target
Transport mode on the proxy = Virtual Appliance
Max Current Tasks = 4 (as I have assigned 8 vCPU's to my proxy server)

My daily backup jobs are taking just over 1:10 hours to complete, with the backup size being on average 8.0GB of data.

I would appreciate some help or advice on optimizing the configuration for my environment before I start moving more of my VM's off the Symantec backup.


Best Regards,
Bob

[veremin]

For more thoughtful piece of advice regarding how backup performance can be increased it’s necessary to have bottleneck statistics for these jobs. Can you provide some? Thanks.

[bob.baird.xedo]

Hi,

From last nights backup, the bottleneck statistics are showing as follows:

Source 97% > Proxy 28% > Network 3% > Target 3%


Thanks,
Bob

[veremin]

Should have mentioned earlier, but can you also provide statistics regarding amount of read/processed/transferred data, since they might be helpful in understanding whether or not CBT mechanism was used?

Additionally, have you considered deploying physical proxy and specifying it in Direct SAN mode; might give a better performance rate? Thanks.

[bob.baird.xedo]

Hi,




Also, we were hoping to avoid having to have a physical server acting as the proxy server, but if it were to provide significant performance increase, then it's an option we could look at.


Thanks,
Bob

[bob.baird.xedo]

Should have mentioned, looking at the detailed statistics of the 13 VM's, each state that CBT was used.

[foggy]

Bob, what transport mode is being effectively used to retrieve VMs data from the source storage? You should see the corresponding tag ([nbd], [hotadd]) right next to the proxy server name selected for processing in the job statistics window if you select the particular VM to the left.

Also, what is the processing rate for the full job run?

[bob.baird.xedo]

Hi Foggy,

Transport mode is showing as [hotadd].

Also, the processing rate is showing as 495MB/s.

Thanks,
Bob

[veremin]

Also, the processing rate is showing as 495MB/s.

This is for full backup run, not for an incremental one (statistics of which you’ve provided above), right? Thanks.

[bob.baird.xedo]

Hi,

The 495MB/s relates to the Processing Rate for my incremental backup from last night (for all 13 VM's)

My synthetic full backup takes place at 22:00 on a Saturday evening. Not sure how I retrieve the processing rate for this job.

Thanks,
Bob

[foggy]

You can list back to the full run in the job stats window using the Back arrow key.

[veremin]

The question has to do with an active full backup rather than with a synthetic one. So, apart from synthetic full do you use active full, as well? If so, kindly find its last session (as mentioned by Alexander) and see what numbers are there. Otherwise, just find the very first backup session and get required information from there. Thanks.

[bob.baird.xedo]

Hi,

Never knew you could do that!

The processing rate for my full backup is only showing as 79MB/s.

Thanks.

[foggy]

Pretty fair speed, not sure you could get better. Bottleneck source means that data cannot be provided by your production SAN to ESXi host any faster.

[bob.baird.xedo]

The question has to do with an active full backup rather than with a synthetic one. So, apart from synthetic full do you use active full, as well? If so, kindly find its last session (as mentioned by Alexander) and see what numbers are there. Otherwise, just find the very first backup session and get required information from there. Thanks.

Hi,

We only use incremental with synthetic full - no active full backups.

The difficulty in trying to find the very first backup statistics is that I have added VM's to my original job i.e. I started off with just a single VM and have been adding additional VM's to the list, so I don't have a single point where I have a full backup of all 13 VM's.


Thanks.

[bob.baird.xedo]

Pretty fair speed, not sure you could get better. Bottleneck source means that data cannot be provided by your production SAN to ESXi host any faster.

So if I had a physical server acting as a proxy, and attached this directly to my SAN, I should notice considerable improvements in performance?

[veremin]

I wouldn't be sure about "considerable", but the usage of Direct SAN method might result in increased performance rates; so, it's worth giving a shot.

Thanks.

[foggy]

You could notice some improvement, depending on the configuration (typically direct SAN and hotadd modes provide quite similar performance).

[bob.baird.xedo]

Okay, I may just leave things the way they are for now then, especially if my stats aren't showing anything glaringly wrong.

I appreciate all your replies.

Best Regards,
Bob

[foggy]

Yep, there's definitely nothing to worry about, especially if you meet your backup window with the current setup.

[yizhar]

Also, we were hoping to avoid having to have a physical server acting as the proxy server, but if it were to provide significant performance increase, then it's an option we could look at.

Hi.
Regarding a physical backup server - the point is not only about best backup performance, but also looking to restore scenarios for several cases.
Most ofter you need to restore specific VM or data within (files/email/db etc).
But in worst case you might need to restore the whole system - such as a human/hardware error causing the SAN to be down.
In such case - the main advantage of physical backup server is that it is independent on the production systems (VMware, san fabric, san storage, etc). Allowing you immediate access to backups in such situation.

About backup performance of physical server in SAN mode vs VM in hotadd mode, I assume that it will be similar. However SAN mode also reduces load from esxi hosts (cpu/ram/disk io) and only SAN storage and backup server + repository are working during backup.

You can also consider a hybrid combination of both physical backup server with additional Veeam proxy running on VM in hotadd mode, which allows you flexibility in testing and choosing what best fits for you.

Please also read the following discussion about it:
Community Forums • View topic - Moving Veeam to Physical Server - Options
http://forums.veeam.com/viewtopic.php?f=2&t=17484

Please note that Veeam running as VM is also a great option like you have now, just the main disadvantage is that it depends on the same infrastructure as the production systems and in some situations it might be an obstacle.

Yizhar

[yizhar]

Hi.

I would also suggest that you plan how to divide your VMs into Veeam backup jobs.
In general I'm favoring more jobs with less VMs - vs single job with all of them.
For example if you have 36 VMs, You can setup about 5-10 jobs with 1-8 VMs per job.
If you are running a multitenant hosting system (or multi department) you can group VMs by tenant/department.

You can group and select VMs to backup in several ways, such as:
individual VM selection.
by VMware folders.
by VMware datastores.

Take a look here - regarding planning VM placement in single vs multiple jobs:
Community Forums • View topic - Optimizing backups for dedup
http://forums.veeam.com/viewtopic.php?t=17169

[bob.baird.xedo]

Okay, thanks for your input.

That was actually going to be my next question - how best to add the 36 VM's i.e as a single job, or 4 separate jobs (with 9 VM's in each).

As previously mentioned, my proxy has been configured with 8 x vCPU's which allows me to run 4 concurrent tasks.

So, would I have my 4 backup jobs scheduled to start at the same time?

Thanks,
Bob

[foggy]

General recommendation is to group similar VMs (i.e. created from the same templates, for example) to get better dedupe rates. You can then schedule jobs to start at the same time (or with short intervals) and they will just queue up and start automatically according to the number of allowed concurrent tasks.

[bob.baird.xedo]

Okay, thanks Foggy.