One Proxy, two repositories: Concurrent Jobs

[doggatas]

We have a fairly basic setup of one backup server with Veeam 7 (latest patch) installed. The box has a bunch of local sas drives installed (RAID 50 - not configured by me) which is configured in windows as one large volume (6TB). The box is also has a HBA fibre card installed and is connected to a VNX5300 which also has a 5TB lun presented to the box, again one large volume. The box has 2 quad core cpus and 24gb RAM.

I have setup two repositories (one to each volume) and limited them to 2 concurrent jobs each and I have configured the "Backup Proxy" to allow for 4 concurrent jobs. I have setup 4 jobs and split them across the 2 repositories. I expected 1 job per repository to run at the same time. This would mean efficient use of the multiple core CPUs (allows me to up the concurrent jobs to 8 based on cores) and the as much memory as it needs to run both jobs and the disk writes (jobs are in incremental mode) split across the two different lots of spindles(repositories) resulting in zero bottleneck and fast backup.

However what happens is the one job processes two vms at the same time and the other job is waiting for resources. CPU sitting at 40-50%(its super fast like this but I expect the total job time to be less if I can get the above working)

Do I need to turn parallel processing off to achieve what I have described above?

Apologies if this has been covered in previous threads (yes, I did search, lots of results but nothing quite like what i was asking)

Regards
David

[dellock6]

David, if the backup server is the only proxy with 8 cores, parallel processing can backup 8 vmdks at the same time, and for sure it's better to have it enabled. There are however some things you should check:
- what backup mode is being used? There are some limits to parallel execution based on backup mode (directsan, hotadd, network mode)
- how many VMDKs are processed at the same time by all running jobs?

PS: I do hope the VNX is not the same storage also storing production VMs, it's not good practice to save backups inside the same storage you are protecting...

[doggatas]

Hey

Thanks for the reply.

I haven't actually set the limit on the proxy to 8 just yet, but at 4, it was processing 4 vmdks at once. However it was processing all 4 vmdks from the same job, while the other job ( backing up to different repository) was waiting for resources to become free. However processing the 4 vmdks from the same job and writing to the one repository meant the write speed was hitting the limits.

With 2 jobs running at the same time writing to different repositories would be ideal, i.e processing 2 vmdks per job or if I find the storage has the headroom to take more ill up the concurrent processing to 6 or even 8.

If I could limit the number of vmdks processed per job, I believe I could achieve what I am after. So with my example of 4 vmdks processed at once, with 2 jobs running at the same time it would be 2 vmdks per job, writing (vbk or vibs) to 2 different repositories. This would allow the most out of the 2 quad CPUs and 24gb memory. If I'm only writing to one repository at a time the storage is bottleneck.

Transport mode is SAN.

unfortunately the vnx does have the productions vms. We inherited the environment and they had BE 2010 with dedupe folder. The amount of errors and failed backups has to be seen to be believed. Got sick of trying to fix up backup exec and I know how good veeam is. It is only temporary until we can consolidate data centres(which veeam will help with replica jobs, thank you Veeam). As soon as the veeam backup has finished it goes straight to tape. I'd rather have a veeam backups on prod storage than unreliable backup exec jobs. Unfortunately BE is still needed for some physical boxes, so at this stage vbk, vib and vbm are put to tape using BE.

Once again thanks for the reply. Cheers

[dellock6]

Well, the answer is in your configuration: if the first job gets all the 4 available slots, obviously the other ones do not have free slots to start and they wait. What if you increase the proxy slots to 8, keep parallel processing on, and configure each repository to accept 1 job at the time? Then the second job /again, if there are still free execution slots) should start one of the jobs targeting the other repository. I don't have a chance to test this, can you?

They way you think it works (or it should work) is not how it works. What you configure is the start time, but then you also need enough slots to effectively run it. The main goal for separating VMs between jobs is to have them backed up together (consistency, application groups...), so a job should finish as soon as possible. Evenly spreading slots between multiple jobs imho keeps jobs open too much; but I see your need here.

[doggatas]

Hi Luca

I have managed to achieve the outcome we were after.

I have set the "Backup Proxy" to allow for 6 concurrent task and each "Backup Repository" for 3 concurrent tasks. Kicked the jobs off together and 3 vmdks per job are processed at the same time and written to their respective repository. Excellent. I think my problem was a configuration error on my part. (perhaps I forgot to edit the original repository, it was set to 4 concurrent tasks to align with the original proxy setting)

I will monitor performance of the CPU/Mem/HDD and up it to 8 tasks for proxy and 4 per repository if the box will handle it.

The VMs are grouped into a job based on OS, application, size. The hardware we have inherited is quite good, we didn't think this to begin with as everything was running slow. SO perhaps application (exchange, ad, sql, etc) config is not so great.

The biggest VM being backed up is in its own job and is 1.9TB (multiple VMDKs). The job transferred 200Gb of changes and snapshot was there for about 1hr and 5 mins and took 59 seconds to consolidate.

Thanks for you replies, much appreciated.

David

[dellock6]

Glad to hear!
If you have any other questions, feel free to open another thread.

Luca.