How do I get better processing rate?

[braman1]

I've been using Veeam for a few months. I'm having problems getting my backups to process at a faster rate. I am running a full back up of 75 GB and it's taking just over 2 hours to complete.

Background:
I have three Hosts. Veeam and my VCenter server are loaded on the same server residing on Host3.
On Host2, I have 10 VM's. One of the ten VM's is my DC which I am trying to backup. The VM is about 75 GB.

I have a backup proxy on Host2 and Host3 set the transport mode using only virtural applicance. When review the job statistics, the process rate is 10 +/- MB/sec.

Any thoughts?

[veremin]

What are the bottleneck statistics of the backup jobs? Does VM in question have more than one virtual disk? If so, you can start utilizing new parallel processing option. With the introduction of this option, VB&R can process multiple VMs and VM disks in parallel, this’s supposed to reduce significantly backup window.

Thanks.

[braman1]

Thanks for the reply v.Eremin.

What are the bottleneck statistics of the backup jobs?

9/16/2013 12:50:15 PM :: Load: Source 78% > Proxy 45% > Network 30% > Target 34%

VMware Backup job: DC Test (Full)
Created by administrator at 9/16/2013 10:29:33 AM. Success
1 of 1 VMs processed
Monday, September 16, 2013 10:44:33 AM
Success 1 Start time 10:44:33 AM Total size 75.0 GB Backup size 16.6 GB
Warning 0 End time 12:50:15 PM Data read 71.0 GB Dedupe 1.4x
Error 0 Duration 2:05:42 Transferred 16.5 GB Compression 3.2x
Details
Name Status Start time End time Size Read Transferred Duration Details
DC Success 10:45:18 AM 12:50:08 PM 75.0 GB 71.0 GB 16.5 GB 2:04:50

Does VM in question have more than one virtual disk?

No...the VM is on one virtual disk.

I look forward to your response.

Matt

[Vitaliy S.]

What about other VMs? What is their processing rate?

[braman1]

Process rate is about the same on all my VM's.

Any suggestions?

[tsightler]

Do you jobs show that they are using hotadd mode? Are you running multiple tasks at one time?

[braman1]

Do you jobs show that they are using hotadd mode?

Yes, Tom. The job shows using hotadd mode.

Are you running multiple tasks at one time?

During the regular back up schedule, I do sometimes run multiple tasks but for problem solving the processing rate, I am only running one job.

[tsightler]

How many vCPUs does your Veeam server have? That fact that you're seeing 45% CPU usage at such a slow processing rate is surprising. Is this with V7? What compression level are you using? You might want to open a support case for proper investigation.

[Butha]

I see the initial job posting shows (FULL) in brackets - which indicates an "active full" job was run. It processed 71 GB - which at 10MB/s gives you roughly - 36GB/hour. So the total time seems correct (2 hours)

There are a few things to check, please ignore the ones that might seem obvious.

Storage:
1. What storage are you using? (Internal/San/Nas?) - what is "normal" transfer speeds when copying files between vms on the same or on different hosts?

Vmware
1. Are you running up to date vmtools and VMXnet3 drivers for the nics?
2. CBT enabled (change block tracking)
3. Which version of ESXi are you running?
4. What is your physical switch connectivity between the hosts? (gig?)

Veeam:
1. What type of incrementals are you using? - edit backup job - click storage- could be reverse increment or increment
2. What Storage settings are used? - edit job - storage - 2nd tab "Storage"- check compression and dedupe settings.
3. CBT settings - edit job - storage - 4th tab - "Vmware" - there are 2 blocks to enable CBT at the bottom.

If CBT is enabled and you run the job a second time - it should only process changes - and the time should be a lot less. Also found that using network mode in some instances works a bit faster. You change this for the proxy itself - backup infrastructure - backup proxies - right click - properties - transport mode.

It might be a start to try the few things listed.

[Butha]

Any reason to run FULL on every job?

[braman1]

How many vCPUs does your Veeam server have?

I have two CPUs on the Veeam server. I do use this server as my vCenter server too. Maybe I need to increase the CPU's?

That fact that you're seeing 45% CPU usage at such a slow processing rate is surprising. Is this with V7?

I'm running version 6.5.

What compression level are you using? You might want to open a support case for proper investigation.

I have the compression level set to "optimal" on all my backups.

Any reason to run FULL on every job?

This was a test job to so I can "tweak" it to see any improvements in the processing rate. No, normal jobs are incremental during the week and I have "full" jobs running during the weekends. One of the issues I have is backing up my Exchange server. At this processing rate, it takes almost the entire weekend to backup just the Exchange server.

[tsightler]

It's certainly not considered best practice to run Veeam on the vCenter server as Veeam itself needs significant resources. I'd suggest that you consider installing Veeam on a dedicated VM with at least 2 vCPUs and try the new V7 which uses a new compression method that has much less CPU overhead.

[braman1]

It's certainly not considered best practice to run Veeam on the vCenter server as Veeam itself needs significant resources.

Who follows best practice? I will give this a shot. Thanks Tom.

[tsightler]

Almost no one...but everyone always wants to know what they are.