-
- Expert
- Posts: 141
- Liked: 5 times
- Joined: Jan 27, 2010 9:43 am
- Full Name: René Frej Nielsen
- Contact:
Very different backup speed on VM's
Hi,
We're using Veeam B&R 5.0.1 in VA mode. Generally we have very good backup performance but a few VM's take quite a long while to backup compared to most of the others. One is a Windows Server 2003 with Symantec Endpoint Protection Manager and a few other services installed, but none that should generate at lot of disk changes that could cause a lot of backup activity. The other is our Terminal Server which isn't very busy.
Most VM's are processed at around 300 MB/s but the Windows Server 2003 VM is typically processed at 15-20 MB/s.
We're also replicating with B&R and it's the same thing there... Most VM's are processed at 500-700 MB/s but the troublesome VM is procesed at around 40-50 MB/s.
Besides a lot of changed blocks is there something that could cause B&R to process a VM slowly?
We're using Veeam B&R 5.0.1 in VA mode. Generally we have very good backup performance but a few VM's take quite a long while to backup compared to most of the others. One is a Windows Server 2003 with Symantec Endpoint Protection Manager and a few other services installed, but none that should generate at lot of disk changes that could cause a lot of backup activity. The other is our Terminal Server which isn't very busy.
Most VM's are processed at around 300 MB/s but the Windows Server 2003 VM is typically processed at 15-20 MB/s.
We're also replicating with B&R and it's the same thing there... Most VM's are processed at 500-700 MB/s but the troublesome VM is procesed at around 40-50 MB/s.
Besides a lot of changed blocks is there something that could cause B&R to process a VM slowly?
-
- Chief Product Officer
- Posts: 31766
- Liked: 7266 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Very different backup speed on VM's
Hi,
Primary reason is for incremental run slowness is CBT not being leveraged for some reason, which would be reflected in the session results.
On top of that, processing rate counter is affected by the following:
- A lot of changed blocks to process
- Time it take to perform application-aware processing (primarily VSS freeze time)
- VMware snapshot creation/deletion time (primarily deletion time)
- Slowness of specific datastore hosting the corresponding VMDKs
- Slowness of target (if these VMs are processed in a separate job)
Typically, it is easy to see from the log files what operations take too much time, thus affecting processing rate counter.
The smaller VM size is, the more processing rate will be affected obviously, because processing rate is calculated by dividing total VM size by total processing time.
Thanks!
Primary reason is for incremental run slowness is CBT not being leveraged for some reason, which would be reflected in the session results.
On top of that, processing rate counter is affected by the following:
- A lot of changed blocks to process
- Time it take to perform application-aware processing (primarily VSS freeze time)
- VMware snapshot creation/deletion time (primarily deletion time)
- Slowness of specific datastore hosting the corresponding VMDKs
- Slowness of target (if these VMs are processed in a separate job)
Typically, it is easy to see from the log files what operations take too much time, thus affecting processing rate counter.
The smaller VM size is, the more processing rate will be affected obviously, because processing rate is calculated by dividing total VM size by total processing time.
Thanks!
-
- Expert
- Posts: 141
- Liked: 5 times
- Joined: Jan 27, 2010 9:43 am
- Full Name: René Frej Nielsen
- Contact:
Re: Very different backup speed on VM's
Is there anyway that I can check if it's the amount of changed blocks that is causing this, without looking at Veeam logs?
The VM is located on the the same datastores and is backed up to the same destination as the others, so it has to something else. Is this something that I could have Veeam support look at?
The VM is located on the the same datastores and is backed up to the same destination as the others, so it has to something else. Is this something that I could have Veeam support look at?
-
- Chief Product Officer
- Posts: 31766
- Liked: 7266 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Very different backup speed on VM's
Incremental backup size for the corresponging run can provide guesstimate on the amount of changed data.
-
- Expert
- Posts: 141
- Liked: 5 times
- Joined: Jan 27, 2010 9:43 am
- Full Name: René Frej Nielsen
- Contact:
Re: Very different backup speed on VM's
The VM is backed up with other VM's in one job, so how do I find the amount of backed up data? For the replication part it's easy and the last restore point is 8 MB, but it still took nearly 20 minutes to replicate at 43 MB/s. The VMDK is 50 GB.
I guess that I will have to find a way to see how long it takes to freeze the filesystem and if snapshotting is particulary slow on this VM.
I guess that I will have to find a way to see how long it takes to freeze the filesystem and if snapshotting is particulary slow on this VM.
-
- Chief Product Officer
- Posts: 31766
- Liked: 7266 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Very different backup speed on VM's
According to rollback size for replication, daily changes for this VM are minimal, so yes you need to be looking at some other bullets I listed. By the way, you don't have to research this yourself (unless you want to)... as long as you have active maintenance agreement in force, our support will be more than happy to assist and do this for you. Thanks!
-
- Expert
- Posts: 141
- Liked: 5 times
- Joined: Jan 27, 2010 9:43 am
- Full Name: René Frej Nielsen
- Contact:
Re: Very different backup speed on VM's
I'll send this off to support next week... Thank you for your assistance!
-
- Enthusiast
- Posts: 38
- Liked: 1 time
- Joined: Jun 17, 2010 12:21 am
- Full Name: Nathan Tong
- Contact:
[MERGED] Different VMs give different results
Hello, I am running Veeam backup on both my exchange server and File server.
The exchange server races through at 76MB/s and complete 180gb in 40 mins.... Amazing performance.
The file server gets it done at 10-15MB/s and completes 519Gb in around 13 hrs.
The method of backup is identical. (SAN, Reverse Incremental, CBT)
Is it because the file server is full of millions smaller documents? I was hopeing to get similar performance out of both VMs.
Thanks,
Dekkar
The exchange server races through at 76MB/s and complete 180gb in 40 mins.... Amazing performance.
The file server gets it done at 10-15MB/s and completes 519Gb in around 13 hrs.
The method of backup is identical. (SAN, Reverse Incremental, CBT)
Is it because the file server is full of millions smaller documents? I was hopeing to get similar performance out of both VMs.
Thanks,
Dekkar
-
- Enthusiast
- Posts: 38
- Liked: 1 time
- Joined: Jun 17, 2010 12:21 am
- Full Name: Nathan Tong
- Contact:
Re: Very different backup speed on VM's
Hi,.... what you are saying makes a lot of sense..... each exchange incremental is around 9gb... each file server incremental is around 35gb....
-
- Lurker
- Posts: 2
- Liked: never
- Joined: Nov 15, 2011 10:22 am
- Contact:
[MERGED] Backup speeds: Windows vs. Linux?
Hello,
I'm running Veeam5 to backup our virtual machines, about 120 in total.
I've noticed a great difference in speeds of backups of Windows vs. Linux machines where the Windows machines will actually be orders of magnitude faster for backup?
Im using network mode with CBT and on the Windows machines i see atleast ~60Mb/s and up to 200+Mb/s.
Whereas for the Linux machines i rarely get above 10Mb/s.
Is there some explanation for this and is it to be expected or might there be something strange with my configuration?
Regards,
Mattias
I'm running Veeam5 to backup our virtual machines, about 120 in total.
I've noticed a great difference in speeds of backups of Windows vs. Linux machines where the Windows machines will actually be orders of magnitude faster for backup?
Im using network mode with CBT and on the Windows machines i see atleast ~60Mb/s and up to 200+Mb/s.
Whereas for the Linux machines i rarely get above 10Mb/s.
Is there some explanation for this and is it to be expected or might there be something strange with my configuration?
Regards,
Mattias
-
- Novice
- Posts: 3
- Liked: never
- Joined: Mar 13, 2020 1:04 pm
- Full Name: Victor Rybakovas
- Contact:
Re: Very different backup speed on VM's
I have the same problem,
I have some big VMs, the average size is around 9TB, they have a big number of disks as well around 20 hard disks per VM.
I'm trying to run an Active Full on them and the job still running after 24 horas.
I saw some disks over the process (task) very fast (200gb) but the big ones (2tb p/disk) take many hours.
On backup status the bottleneck is target.
Any idea How can I improve this performance?
Wish the Best
Victor Rybakovas
I have some big VMs, the average size is around 9TB, they have a big number of disks as well around 20 hard disks per VM.
I'm trying to run an Active Full on them and the job still running after 24 horas.
I saw some disks over the process (task) very fast (200gb) but the big ones (2tb p/disk) take many hours.
On backup status the bottleneck is target.
Any idea How can I improve this performance?
Wish the Best
Victor Rybakovas
Wish the Best,
Rybakovas
Rybakovas
-
- Veeam Software
- Posts: 21133
- Liked: 2140 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Very different backup speed on VM's
Hi Victor, it is expected that larger disks take more time to process. Do you mean the read speed for these disks is slower than for smaller ones? You can see the hard disk read speed in the VM session log. If the speed between disks is comparable and the overall bottleneck reported by the job is target, then you need to pay attention to your target storage. Thanks!
Who is online
Users browsing this forum: No registered users and 39 guests