What is "Merge" and why it takes so long to finish?

Backup agent for Microsoft Windows servers and workstations (formerly Veeam Endpoint Backup FREE)

What is "Merge" and why it takes so long to finish?

Veeam Logoby JChris » Fri May 26, 2017 9:41 pm

I've been using VEB for some months now, but just recently I noticed that VEB started adding one more step to my routine backups, something called "Merge". This thing increased my backup times considerably. What's it and how can I "fix" this?

VEB Version 1.5.0.306. I didn't upgrade to Veeam Agent for Windows yet because it's on beta.

Image
JChris
Enthusiast
 
Posts: 27
Liked: 1 time
Joined: Fri Sep 16, 2016 3:51 am
Full Name: Juan C.

Re: What is "Merge" and why it takes so long to finish?

Veeam Logoby vClintWyckoff » Fri May 26, 2017 10:23 pm

What's your retention set for?

https://helpcenter.veeam.com/endpoint/15/retention.html

Also, FYI - VAW 2.0 went GA 2 weeks ago.
vClintWyckoff
Veeam Software
 
Posts: 429
Liked: 95 times
Joined: Sat Oct 27, 2012 1:22 am
Location: Technical Evangelist
Full Name: Clint Wyckoff

Re: What is "Merge" and why it takes so long to finish?

Veeam Logoby JChris » Fri May 26, 2017 11:41 pm

180 days, why? Ok, so I read the link and I believe it has something to do with the rebuild of backups, but it takes way to long, every new backup triggers this "merge" and does the injecting to move the VBK forward, is there a way to make it faster? I did a backup today and by doing a sort on my backup files by modified date I have this (only last items, in total I have more than 250 items):

2017-05-26 is today, as of this message :p

Image
JChris
Enthusiast
 
Posts: 27
Liked: 1 time
Joined: Fri Sep 16, 2016 3:51 am
Full Name: Juan C.

Re: What is "Merge" and why it takes so long to finish?

Veeam Logoby rekun » Sat May 27, 2017 10:49 am

Hi
The merge is the process that happens when it deletes the oldest recovery points. It has to make a copy of whatever in that file is still required to restore the newer recovery points, so it will take some time. There is normal for the merge to take about 15 min, so nothing wrong here.

If you want to speed up the merge process, a simple solution (if you are running the newest update of Windows 10) would be to upgrade to Veeam Agent 2 (it is not beta anymore), and then format your backup target with REFS (remember to use 64kb blocksize). Then Veeam would utilize its fastclone technology, which should make it way faster.
rekun
Service Provider
 
Posts: 18
Liked: 1 time
Joined: Tue Jun 02, 2015 7:24 pm
Full Name: Rene Knudsen

Re: What is "Merge" and why it takes so long to finish?

Veeam Logoby JChris » Sat May 27, 2017 4:04 pm

Thanks for letting me know it wasn't in Beta anymore. I'm now using Veeam Agent 2.0.0.700 Free. Sadly I can't format my NAS to REFS, it would be a headache as I have tons and tons of files there and not only backups. :cry:
JChris
Enthusiast
 
Posts: 27
Liked: 1 time
Joined: Fri Sep 16, 2016 3:51 am
Full Name: Juan C.

Re: What is "Merge" and why it takes so long to finish?

Veeam Logoby JChris » Sat May 27, 2017 5:44 pm

UPDATE: I just ran my first backup job after the upgrade to Veeam Agent and this time there wasn't any info regarding "merge" and the backup took the normal 7~8 minutes to finish.

Image
JChris
Enthusiast
 
Posts: 27
Liked: 1 time
Joined: Fri Sep 16, 2016 3:51 am
Full Name: Juan C.

Re: What is "Merge" and why it takes so long to finish?

Veeam Logoby Technogod » Sat May 27, 2017 9:26 pm

I did some research on Fast Clone and it seems to only be available in B&R 9.5 . Can somebody from Veeam verify this?
Technogod
Enthusiast
 
Posts: 79
Liked: 13 times
Joined: Tue Apr 11, 2017 3:52 am

Re: What is "Merge" and why it takes so long to finish?

Veeam Logoby vClintWyckoff » Sat May 27, 2017 10:33 pm

Fast Clone is available in VAW 2.0 on ReFS v3.1 formatted disk targets.
vClintWyckoff
Veeam Software
 
Posts: 429
Liked: 95 times
Joined: Sat Oct 27, 2012 1:22 am
Location: Technical Evangelist
Full Name: Clint Wyckoff

Re: What is "Merge" and why it takes so long to finish?

Veeam Logoby Technogod » Sun May 28, 2017 12:05 am

I stand corrected.
Technogod
Enthusiast
 
Posts: 79
Liked: 13 times
Joined: Tue Apr 11, 2017 3:52 am

Re: What is "Merge" and why it takes so long to finish?

Veeam Logoby JChris » Sun May 28, 2017 6:02 pm

New day, same issue :(

Image

It seems there's no way to stop this merge process. My retention is 180 days, a number that I passed already, and every new backup job adds 1 day to it, forcing it to delete the oldest backup and merge. All my backups from now on will take this long to finish (20+ min) compared to the default 5~7 min I had, this is depressing. I thought the retention policy would simply delete the oldest backups and be done with it, but this merge process take quite a lot of time and system resources of my NAS.

Is there anything else I can do to make it faster other than formatting as REFS? :roll:
JChris
Enthusiast
 
Posts: 27
Liked: 1 time
Joined: Fri Sep 16, 2016 3:51 am
Full Name: Juan C.

Re: What is "Merge" and why it takes so long to finish?

Veeam Logoby vClintWyckoff » Sun May 28, 2017 6:44 pm

TBF, its not really an issue - its the product working as it should and as expected :D The reason why it takes so long is that we're merging the full VBK which is the last restore point with the next incremental to form a VBK and drop the incremental. So this is IO intensive on the NAS device which isn't exactly a high performance SAN...
vClintWyckoff
Veeam Software
 
Posts: 429
Liked: 95 times
Joined: Sat Oct 27, 2012 1:22 am
Location: Technical Evangelist
Full Name: Clint Wyckoff

Re: What is "Merge" and why it takes so long to finish?

Veeam Logoby Technogod » Sun May 28, 2017 7:51 pm 1 person likes this post

Under configure backup local storage tab select advanced. On the backup tab check create active full backups periodically. I'd create a new backup once a month. That way it doesn't have to go all the back to 180 days to create your next incremental. The most it would have to go back is 31 days. Since your retention is set to 180 days it will still be 180 days before the oldest backup is deleted.
Technogod
Enthusiast
 
Posts: 79
Liked: 13 times
Joined: Tue Apr 11, 2017 3:52 am

Re: What is "Merge" and why it takes so long to finish?

Veeam Logoby vClintWyckoff » Mon May 29, 2017 3:12 am

This is correct - Since its a chain, creating an active full resets the backup chain. So you'd have to wait an additional 180 restore points, using capacity but would remove your merge question. Pros and Cons that you should weigh out.

https://helpcenter.veeam.com/docs/agentforwindows/userguide/active_full_backup.html?ver=20#retention
vClintWyckoff
Veeam Software
 
Posts: 429
Liked: 95 times
Joined: Sat Oct 27, 2012 1:22 am
Location: Technical Evangelist
Full Name: Clint Wyckoff

Re: What is "Merge" and why it takes so long to finish?

Veeam Logoby JChris » Mon May 29, 2017 10:03 pm

Technogod wrote:Under configure backup local storage tab select advanced. On the backup tab check create active full backups periodically. I'd create a new backup once a month. That way it doesn't have to go all the back to 180 days to create your next incremental. The most it would have to go back is 31 days. Since your retention is set to 180 days it will still be 180 days before the oldest backup is deleted.


That is AMAZING. I'll do that as storage isn't the issue here with more than 10TB of free space on my NAS :D
JChris
Enthusiast
 
Posts: 27
Liked: 1 time
Joined: Fri Sep 16, 2016 3:51 am
Full Name: Juan C.

Re: What is "Merge" and why it takes so long to finish?

Veeam Logoby Anguel » Wed Jul 26, 2017 3:40 pm

Technogod wrote:Under configure backup local storage tab select advanced. On the backup tab check create active full backups periodically. I'd create a new backup once a month.

I wonder if with an active full every month the merge start will not only be postponed by one month but then again happen every day?

That way it doesn't have to go all the back to 180 days to create your next incremental. The most it would have to go back is 31 days. Since your retention is set to 180 days it will still be 180 days before the oldest backup is deleted.


I thought that the full backup (which is always the latest file, e.g. Sunday) is always merged with the NEXT incremental (i.e. Monday), so why would Veeam have to go back that many days at all?
Anguel
Enthusiast
 
Posts: 79
Liked: 6 times
Joined: Thu Apr 16, 2015 9:01 am

Next

Return to Veeam Agent for Windows



Who is online

Users browsing this forum: No registered users and 33 guests