Discussions specific to the VMware vSphere hypervisor
Post Reply
sosborne
Enthusiast
Posts: 54
Liked: 4 times
Joined: Jan 30, 2018 12:06 pm
Full Name: Simon Osborne
Contact:

How do I improve backup performance

Post by sosborne » Jul 01, 2019 12:16 pm

The backup window on our system has been exceeded when we have our month end backups (Active Fulls) which are in addition to the standard incrementals. We only get by due to some reordering of backups so the less time sensitive ones overrun.

The backups are defaulting to network mode. The VM infrastructure is connected to the SAN by 10GB iSCSI and to the Core network at 10GB. The Dell Backup Server is connected to the same Core network at 10GB. It is a 2 x 8 Core CPU with 64GB of RAM and it has direct attached storage both internally and externally over two separate RAID controllers. The server is also the backup proxy which is limited to 16 concurrent tasks. There are three different repositories being used (with two combined as a a SOBR) all limited to 4 concurrent tasks (each repository has 12 disks RAID 6). The Active Full backups are split across all repositories for capacity reasons as two of the jobs are large and are on the SOBR (3.5 and 2.5 TB respectively per restore point). All of the jobs use per VM backup files. But I am not overly familiar with the RAW capabilities of the PERC H730 or PERC H840.

I assume this would effectively give me 8 concurrent tasks (4 internal and 4 external). So 4 tasks would handle the two large jobs and 4 tasks the 5 smaller jobs. Is this correct?

I am looking at reordering VMs within jobs, rescheduling jobs and anything else I can think of. But I thought it was worth double checking the config/setup to see if anything could be tweaked there. And confirming that my understanding is right on the setup. Plus perhaps get an idea of what the performance hit could be when the large jobs kick off on the external array to see if they could be scheduled earlier and run in tandem with the smaller jobs on the internal array?

sosborne
Enthusiast
Posts: 54
Liked: 4 times
Joined: Jan 30, 2018 12:06 pm
Full Name: Simon Osborne
Contact:

Re: How do I improve backup performance

Post by sosborne » Jul 01, 2019 2:50 pm

I will partially answer myself after having discovered the RTS.ResourcesUsage.log (from c:\ProgramData\Veeam\Backup) which seems to give a periodic update on what system resources are being used with regards to both Proxy and Repository. The concurrent tasks per repository seem to be potentially capable of max-ing out even across both the SOBR extents. So any job going through the repository backed by the internal RAID is limited to 4 tasks, but the external can potentially go to 8 tasks with jobs being allocated to both repositories (although 1 extent is low on space). The proxy at its peak has 13 tasks when the standard incrementals kick off for the last few hours of the window.

Obviously there is the question of whether both the proxy (cores/RAM) and more to the point the repositories (RAID/disks) can properly serve the load?

And if the 99% load at source can be eased through a change to configuration/infrastructure to allow us to meet the window, or whether this is expected with CBT load and dealing with the file/folder exceptions especially on the large jobs?

sosborne
Enthusiast
Posts: 54
Liked: 4 times
Joined: Jan 30, 2018 12:06 pm
Full Name: Simon Osborne
Contact:

Re: How do I improve backup performance

Post by sosborne » Jul 01, 2019 2:58 pm

BTW there is a 4th repository which only serves the standard incrementals but it is the effectively the same physical disk as the month end jobs (I just chose to try to simplify the initial description).

Post Reply

Who is online

Users browsing this forum: Baidu [Spider], ccouchman, tblime and 24 guests