-
- Service Provider
- Posts: 1092
- Liked: 134 times
- Joined: May 14, 2013 8:35 pm
- Full Name: Frank Iversen
- Location: Norway
- Contact:
Merging of backup files
We have a customer running 20 7200 RPM SATA drives in Storages Spaces 2012 R2 mirror. The IO is pretty OK.
They have a forever incrementeal job. FUll backupfil about 700 GB and daily incremental of the 8 VMs at about 40 GB.
What takes a pretty long time during their jobs (they have multiple jobs) is the merging of backup files.
Again, that is not critical since it is a offloaded task.
But, when we use the schedule "run job after "navme of another job" this will create a significant long backupwindow since we are "wasting" manay hour while merging files when the job could have be done.
I think their is two possibilites here:
1. To use SSD on the backup job and use regular SATA for the GFS job for retention. This should make the mergeprocess a lot faster.
2. To have the backup job to start running when the MERGE FILES appear at the jon in front.
Any suggestions here?
Edit: I can perhaps change backup method to regular incremental? That will not merged files?
They have a forever incrementeal job. FUll backupfil about 700 GB and daily incremental of the 8 VMs at about 40 GB.
What takes a pretty long time during their jobs (they have multiple jobs) is the merging of backup files.
Again, that is not critical since it is a offloaded task.
But, when we use the schedule "run job after "navme of another job" this will create a significant long backupwindow since we are "wasting" manay hour while merging files when the job could have be done.
I think their is two possibilites here:
1. To use SSD on the backup job and use regular SATA for the GFS job for retention. This should make the mergeprocess a lot faster.
2. To have the backup job to start running when the MERGE FILES appear at the jon in front.
Any suggestions here?
Edit: I can perhaps change backup method to regular incremental? That will not merged files?
-
- Veteran
- Posts: 1531
- Liked: 226 times
- Joined: Jul 21, 2010 9:47 am
- Full Name: Chris Dearden
- Contact:
Re: Merging of backup files
if you where to do a fwd incremental with active full , you would avoid any synthetic activity , as long as you have the time and capacity to hold more than 1 full backup.
-
- Service Provider
- Posts: 1092
- Liked: 134 times
- Joined: May 14, 2013 8:35 pm
- Full Name: Frank Iversen
- Location: Norway
- Contact:
Re: Merging of backup files
I think the active backup will also be to long for a standard backup window in this case
-
- Veteran
- Posts: 1531
- Liked: 226 times
- Joined: Jul 21, 2010 9:47 am
- Full Name: Chris Dearden
- Contact:
Re: Merging of backup files
whats the backup window ? you should be able to take a full backup of 8Vm's in a reasonable time.
-
- Service Provider
- Posts: 1092
- Liked: 134 times
- Joined: May 14, 2013 8:35 pm
- Full Name: Frank Iversen
- Location: Norway
- Contact:
Re: Merging of backup files
The backup Windows i 10 P.M to 07 A M.
They have multiple jobs with 8 VMs. (I thing about 50-60 Vms.
the reason behind the 8 VM pr.job is the suggested design from Veeam when multiple VM shared same hyper-v host and CSV to benefit one snapshot for the job through the hardwaare vss provider.
They have multiple jobs with 8 VMs. (I thing about 50-60 Vms.
the reason behind the 8 VM pr.job is the suggested design from Veeam when multiple VM shared same hyper-v host and CSV to benefit one snapshot for the job through the hardwaare vss provider.
-
- Veeam Software
- Posts: 21069
- Liked: 2115 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Merging of backup files
Any particular reason of using chained jobs? How about starting all the jobs simultaneously and limit the number of concurrent tasks on proxy/repository servers? This will allow to avoid waste of time during merge operations.
-
- Service Provider
- Posts: 1092
- Liked: 134 times
- Joined: May 14, 2013 8:35 pm
- Full Name: Frank Iversen
- Location: Norway
- Contact:
Re: Merging of backup files
Hmm.. not a bad idea!
No, there is no reason for not doing that.
I want just to have one snapshot on our san simoutansely, I think that is the reason we have configured it this way.
No, there is no reason for not doing that.
I want just to have one snapshot on our san simoutansely, I think that is the reason we have configured it this way.
-
- Veeam Software
- Posts: 21069
- Liked: 2115 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Merging of backup files
We do not typically recommend chaining jobs due to the number of undesired consequences it might have (including the one you've encountered).
-
- Service Provider
- Posts: 1092
- Liked: 134 times
- Joined: May 14, 2013 8:35 pm
- Full Name: Frank Iversen
- Location: Norway
- Contact:
Re: Merging of backup files
We have 2 separate SANs. I want to have only 1 snapshot at the same time for each.
How would you recommended to accomplish that?
I know I can create 2 separate repositories and just set 1 on that, but on the ssame time i want to leverage parallel processing of disc on the snapshots.
How would you recommended to accomplish that?
I know I can create 2 separate repositories and just set 1 on that, but on the ssame time i want to leverage parallel processing of disc on the snapshots.
-
- Veeam Software
- Posts: 21069
- Liked: 2115 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Merging of backup files
Latency control could help here. Btw, what is the reason for this "1 SAN/1 snapshot" limitation?
-
- Service Provider
- Posts: 1092
- Liked: 134 times
- Joined: May 14, 2013 8:35 pm
- Full Name: Frank Iversen
- Location: Norway
- Contact:
Re: Merging of backup files
When I run all the job at the same time, 1 one of the job takes all the proxy resources so it will not do snapshot on our other SAN.
I was hoping we could take a snapshit at each san at the same time for most effiecent backup window.
OR; will it not go faster anyway since I am using the proxy (4 core - 4 processes) to the max anyway?
I was hoping we could take a snapshit at each san at the same time for most effiecent backup window.
OR; will it not go faster anyway since I am using the proxy (4 core - 4 processes) to the max anyway?
-
- Veeam Software
- Posts: 21069
- Liked: 2115 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Merging of backup files
Since the first job takes all the available slots anyway, there's no gain from the second parallel snapshot. If you have dedicated (explicitly selected) proxies for different jobs, they will be able to take snapshot concurrently.
-
- Service Provider
- Posts: 1092
- Liked: 134 times
- Joined: May 14, 2013 8:35 pm
- Full Name: Frank Iversen
- Location: Norway
- Contact:
Re: Merging of backup files
When runing all the job at the same, the jobs sure will take much less time.
I am seeing though, that perhaps 1 of 8 Vms in the job can fail. Works at nex retry.
Error message: Unable to retrieve next block transmission.
Could i be a kind of timeout that occurs and I should perhaps decrease from 8 to 7 or 6 VMs pr. job?
I am seeing though, that perhaps 1 of 8 Vms in the job can fail. Works at nex retry.
Error message: Unable to retrieve next block transmission.
Could i be a kind of timeout that occurs and I should perhaps decrease from 8 to 7 or 6 VMs pr. job?
-
- Veeam Software
- Posts: 21069
- Liked: 2115 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Merging of backup files
Hard to say, the error is too general to judge on the actual reasons. Better involve support here.
Who is online
Users browsing this forum: Bing [Bot] and 286 guests