Very different backup speed on VM's

[rfn]

Hi,

We're using Veeam B&R 5.0.1 in VA mode. Generally we have very good backup performance but a few VM's take quite a long while to backup compared to most of the others. One is a Windows Server 2003 with Symantec Endpoint Protection Manager and a few other services installed, but none that should generate at lot of disk changes that could cause a lot of backup activity. The other is our Terminal Server which isn't very busy.

Most VM's are processed at around 300 MB/s but the Windows Server 2003 VM is typically processed at 15-20 MB/s.

We're also replicating with B&R and it's the same thing there... Most VM's are processed at 500-700 MB/s but the troublesome VM is procesed at around 40-50 MB/s.

Besides a lot of changed blocks is there something that could cause B&R to process a VM slowly?

[Gostev]

Hi,

Primary reason is for incremental run slowness is CBT not being leveraged for some reason, which would be reflected in the session results.

On top of that, processing rate counter is affected by the following:
- A lot of changed blocks to process
- Time it take to perform application-aware processing (primarily VSS freeze time)
- VMware snapshot creation/deletion time (primarily deletion time)
- Slowness of specific datastore hosting the corresponding VMDKs
- Slowness of target (if these VMs are processed in a separate job)

Typically, it is easy to see from the log files what operations take too much time, thus affecting processing rate counter.

The smaller VM size is, the more processing rate will be affected obviously, because processing rate is calculated by dividing total VM size by total processing time.

Thanks!

[rfn]

Is there anyway that I can check if it's the amount of changed blocks that is causing this, without looking at Veeam logs?

The VM is located on the the same datastores and is backed up to the same destination as the others, so it has to something else. Is this something that I could have Veeam support look at?

[Gostev]

Incremental backup size for the corresponging run can provide guesstimate on the amount of changed data.

[rfn]

The VM is backed up with other VM's in one job, so how do I find the amount of backed up data? For the replication part it's easy and the last restore point is 8 MB, but it still took nearly 20 minutes to replicate at 43 MB/s. The VMDK is 50 GB.

I guess that I will have to find a way to see how long it takes to freeze the filesystem and if snapshotting is particulary slow on this VM.

[Gostev]

According to rollback size for replication, daily changes for this VM are minimal, so yes you need to be looking at some other bullets I listed. By the way, you don't have to research this yourself (unless you want to)... as long as you have active maintenance agreement in force, our support will be more than happy to assist and do this for you. Thanks!

[rfn]

I'll send this off to support next week... Thank you for your assistance!

[dekkar]

Hello, I am running Veeam backup on both my exchange server and File server.

The exchange server races through at 76MB/s and complete 180gb in 40 mins.... Amazing performance.
The file server gets it done at 10-15MB/s and completes 519Gb in around 13 hrs.


The method of backup is identical. (SAN, Reverse Incremental, CBT)

Is it because the file server is full of millions smaller documents? I was hopeing to get similar performance out of both VMs.


Thanks,
Dekkar

[dekkar]

Hi,.... what you are saying makes a lot of sense..... each exchange incremental is around 9gb... each file server incremental is around 35gb....

[mattiasd]

Hello,

I'm running Veeam5 to backup our virtual machines, about 120 in total.

I've noticed a great difference in speeds of backups of Windows vs. Linux machines where the Windows machines will actually be orders of magnitude faster for backup?
Im using network mode with CBT and on the Windows machines i see atleast ~60Mb/s and up to 200+Mb/s.
Whereas for the Linux machines i rarely get above 10Mb/s.

Is there some explanation for this and is it to be expected or might there be something strange with my configuration?

Regards,
Mattias

[Rybakovas]

I have the same problem,

I have some big VMs, the average size is around 9TB, they have a big number of disks as well around 20 hard disks per VM.
I'm trying to run an Active Full on them and the job still running after 24 horas.
I saw some disks over the process (task) very fast (200gb) but the big ones (2tb p/disk) take many hours.
On backup status the bottleneck is target.

Any idea How can I improve this performance?


Wish the Best
Victor Rybakovas

[foggy]

Hi Victor, it is expected that larger disks take more time to process. Do you mean the read speed for these disks is slower than for smaller ones? You can see the hard disk read speed in the VM session log. If the speed between disks is comparable and the overall bottleneck reported by the job is target, then you need to pay attention to your target storage. Thanks!