Comprehensive data protection for all workloads
Post Reply
frankive
Service Provider
Posts: 1092
Liked: 134 times
Joined: May 14, 2013 8:35 pm
Full Name: Frank Iversen
Location: Norway
Contact:

Merging of backup files

Post by frankive »

We have a customer running 20 7200 RPM SATA drives in Storages Spaces 2012 R2 mirror. The IO is pretty OK.
They have a forever incrementeal job. FUll backupfil about 700 GB and daily incremental of the 8 VMs at about 40 GB.
What takes a pretty long time during their jobs (they have multiple jobs) is the merging of backup files.
Again, that is not critical since it is a offloaded task.

But, when we use the schedule "run job after "navme of another job" this will create a significant long backupwindow since we are "wasting" manay hour while merging files when the job could have be done.

I think their is two possibilites here:

1. To use SSD on the backup job and use regular SATA for the GFS job for retention. This should make the mergeprocess a lot faster.
2. To have the backup job to start running when the MERGE FILES appear at the jon in front.

Any suggestions here?

Edit: I can perhaps change backup method to regular incremental? That will not merged files?
chrisdearden
Veteran
Posts: 1531
Liked: 226 times
Joined: Jul 21, 2010 9:47 am
Full Name: Chris Dearden
Contact:

Re: Merging of backup files

Post by chrisdearden »

if you where to do a fwd incremental with active full , you would avoid any synthetic activity , as long as you have the time and capacity to hold more than 1 full backup.
frankive
Service Provider
Posts: 1092
Liked: 134 times
Joined: May 14, 2013 8:35 pm
Full Name: Frank Iversen
Location: Norway
Contact:

Re: Merging of backup files

Post by frankive »

I think the active backup will also be to long for a standard backup window in this case
chrisdearden
Veteran
Posts: 1531
Liked: 226 times
Joined: Jul 21, 2010 9:47 am
Full Name: Chris Dearden
Contact:

Re: Merging of backup files

Post by chrisdearden »

whats the backup window ? you should be able to take a full backup of 8Vm's in a reasonable time.
frankive
Service Provider
Posts: 1092
Liked: 134 times
Joined: May 14, 2013 8:35 pm
Full Name: Frank Iversen
Location: Norway
Contact:

Re: Merging of backup files

Post by frankive »

The backup Windows i 10 P.M to 07 A M.
They have multiple jobs with 8 VMs. (I thing about 50-60 Vms.
the reason behind the 8 VM pr.job is the suggested design from Veeam when multiple VM shared same hyper-v host and CSV to benefit one snapshot for the job through the hardwaare vss provider.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Merging of backup files

Post by foggy »

Any particular reason of using chained jobs? How about starting all the jobs simultaneously and limit the number of concurrent tasks on proxy/repository servers? This will allow to avoid waste of time during merge operations.
frankive
Service Provider
Posts: 1092
Liked: 134 times
Joined: May 14, 2013 8:35 pm
Full Name: Frank Iversen
Location: Norway
Contact:

Re: Merging of backup files

Post by frankive »

Hmm.. not a bad idea!

No, there is no reason for not doing that.
I want just to have one snapshot on our san simoutansely, I think that is the reason we have configured it this way.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Merging of backup files

Post by foggy »

We do not typically recommend chaining jobs due to the number of undesired consequences it might have (including the one you've encountered).
frankive
Service Provider
Posts: 1092
Liked: 134 times
Joined: May 14, 2013 8:35 pm
Full Name: Frank Iversen
Location: Norway
Contact:

Re: Merging of backup files

Post by frankive »

We have 2 separate SANs. I want to have only 1 snapshot at the same time for each.
How would you recommended to accomplish that?
I know I can create 2 separate repositories and just set 1 on that, but on the ssame time i want to leverage parallel processing of disc on the snapshots.
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Merging of backup files

Post by foggy »

Latency control could help here. Btw, what is the reason for this "1 SAN/1 snapshot" limitation?
frankive
Service Provider
Posts: 1092
Liked: 134 times
Joined: May 14, 2013 8:35 pm
Full Name: Frank Iversen
Location: Norway
Contact:

Re: Merging of backup files

Post by frankive »

When I run all the job at the same time, 1 one of the job takes all the proxy resources so it will not do snapshot on our other SAN.
I was hoping we could take a snapshit at each san at the same time for most effiecent backup window.

OR; will it not go faster anyway since I am using the proxy (4 core - 4 processes) to the max anyway?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Merging of backup files

Post by foggy »

Since the first job takes all the available slots anyway, there's no gain from the second parallel snapshot. If you have dedicated (explicitly selected) proxies for different jobs, they will be able to take snapshot concurrently.
frankive
Service Provider
Posts: 1092
Liked: 134 times
Joined: May 14, 2013 8:35 pm
Full Name: Frank Iversen
Location: Norway
Contact:

Re: Merging of backup files

Post by frankive »

When runing all the job at the same, the jobs sure will take much less time.
I am seeing though, that perhaps 1 of 8 Vms in the job can fail. Works at nex retry.
Error message: Unable to retrieve next block transmission.

Could i be a kind of timeout that occurs and I should perhaps decrease from 8 to 7 or 6 VMs pr. job?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Merging of backup files

Post by foggy »

Hard to say, the error is too general to judge on the actual reasons. Better involve support here.
Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 286 guests