Standalone backup agent for Microsoft Windows servers and workstations (formerly Veeam Endpoint Backup FREE)
Post Reply
JChris
Enthusiast
Posts: 31
Liked: 1 time
Joined: Sep 16, 2016 3:51 am
Full Name: Juan C.
Contact:

What is "Merge" and why it takes so long to finish?

Post by JChris »

I've been using VEB for some months now, but just recently I noticed that VEB started adding one more step to my routine backups, something called "Merge". This thing increased my backup times considerably. What's it and how can I "fix" this?

VEB Version 1.5.0.306. I didn't upgrade to Veeam Agent for Windows yet because it's on beta.

Image
vClintWyckoff
Veteran
Posts: 500
Liked: 109 times
Joined: Oct 27, 2012 1:22 am
Full Name: Clint Wyckoff
Location: Technical Evangelist
Contact:

Re: What is "Merge" and why it takes so long to finish?

Post by vClintWyckoff »

What's your retention set for?

https://helpcenter.veeam.com/endpoint/15/retention.html

Also, FYI - VAW 2.0 went GA 2 weeks ago.
JChris
Enthusiast
Posts: 31
Liked: 1 time
Joined: Sep 16, 2016 3:51 am
Full Name: Juan C.
Contact:

Re: What is "Merge" and why it takes so long to finish?

Post by JChris »

180 days, why? Ok, so I read the link and I believe it has something to do with the rebuild of backups, but it takes way to long, every new backup triggers this "merge" and does the injecting to move the VBK forward, is there a way to make it faster? I did a backup today and by doing a sort on my backup files by modified date I have this (only last items, in total I have more than 250 items):

2017-05-26 is today, as of this message :p

Image
rekun
Service Provider
Posts: 18
Liked: 1 time
Joined: Jun 02, 2015 7:24 pm
Full Name: Rene Knudsen
Contact:

Re: What is "Merge" and why it takes so long to finish?

Post by rekun »

Hi
The merge is the process that happens when it deletes the oldest recovery points. It has to make a copy of whatever in that file is still required to restore the newer recovery points, so it will take some time. There is normal for the merge to take about 15 min, so nothing wrong here.

If you want to speed up the merge process, a simple solution (if you are running the newest update of Windows 10) would be to upgrade to Veeam Agent 2 (it is not beta anymore), and then format your backup target with REFS (remember to use 64kb blocksize). Then Veeam would utilize its fastclone technology, which should make it way faster.
JChris
Enthusiast
Posts: 31
Liked: 1 time
Joined: Sep 16, 2016 3:51 am
Full Name: Juan C.
Contact:

Re: What is "Merge" and why it takes so long to finish?

Post by JChris »

Thanks for letting me know it wasn't in Beta anymore. I'm now using Veeam Agent 2.0.0.700 Free. Sadly I can't format my NAS to REFS, it would be a headache as I have tons and tons of files there and not only backups. :cry:
JChris
Enthusiast
Posts: 31
Liked: 1 time
Joined: Sep 16, 2016 3:51 am
Full Name: Juan C.
Contact:

Re: What is "Merge" and why it takes so long to finish?

Post by JChris »

UPDATE: I just ran my first backup job after the upgrade to Veeam Agent and this time there wasn't any info regarding "merge" and the backup took the normal 7~8 minutes to finish.

Image
Technogod
Expert
Posts: 150
Liked: 26 times
Joined: Apr 11, 2017 3:52 am
Contact:

Re: What is "Merge" and why it takes so long to finish?

Post by Technogod »

I did some research on Fast Clone and it seems to only be available in B&R 9.5 . Can somebody from Veeam verify this?
vClintWyckoff
Veteran
Posts: 500
Liked: 109 times
Joined: Oct 27, 2012 1:22 am
Full Name: Clint Wyckoff
Location: Technical Evangelist
Contact:

Re: What is "Merge" and why it takes so long to finish?

Post by vClintWyckoff »

Fast Clone is available in VAW 2.0 on ReFS v3.1 formatted disk targets.
Technogod
Expert
Posts: 150
Liked: 26 times
Joined: Apr 11, 2017 3:52 am
Contact:

Re: What is "Merge" and why it takes so long to finish?

Post by Technogod »

I stand corrected.
JChris
Enthusiast
Posts: 31
Liked: 1 time
Joined: Sep 16, 2016 3:51 am
Full Name: Juan C.
Contact:

Re: What is "Merge" and why it takes so long to finish?

Post by JChris »

New day, same issue :(

Image

It seems there's no way to stop this merge process. My retention is 180 days, a number that I passed already, and every new backup job adds 1 day to it, forcing it to delete the oldest backup and merge. All my backups from now on will take this long to finish (20+ min) compared to the default 5~7 min I had, this is depressing. I thought the retention policy would simply delete the oldest backups and be done with it, but this merge process take quite a lot of time and system resources of my NAS.

Is there anything else I can do to make it faster other than formatting as REFS? :roll:
vClintWyckoff
Veteran
Posts: 500
Liked: 109 times
Joined: Oct 27, 2012 1:22 am
Full Name: Clint Wyckoff
Location: Technical Evangelist
Contact:

Re: What is "Merge" and why it takes so long to finish?

Post by vClintWyckoff »

TBF, its not really an issue - its the product working as it should and as expected :D The reason why it takes so long is that we're merging the full VBK which is the last restore point with the next incremental to form a VBK and drop the incremental. So this is IO intensive on the NAS device which isn't exactly a high performance SAN...
Technogod
Expert
Posts: 150
Liked: 26 times
Joined: Apr 11, 2017 3:52 am
Contact:

Re: What is "Merge" and why it takes so long to finish?

Post by Technogod » 1 person likes this post

Under configure backup local storage tab select advanced. On the backup tab check create active full backups periodically. I'd create a new backup once a month. That way it doesn't have to go all the back to 180 days to create your next incremental. The most it would have to go back is 31 days. Since your retention is set to 180 days it will still be 180 days before the oldest backup is deleted.
vClintWyckoff
Veteran
Posts: 500
Liked: 109 times
Joined: Oct 27, 2012 1:22 am
Full Name: Clint Wyckoff
Location: Technical Evangelist
Contact:

Re: What is "Merge" and why it takes so long to finish?

Post by vClintWyckoff »

This is correct - Since its a chain, creating an active full resets the backup chain. So you'd have to wait an additional 180 restore points, using capacity but would remove your merge question. Pros and Cons that you should weigh out.

https://helpcenter.veeam.com/docs/agent ... #retention
JChris
Enthusiast
Posts: 31
Liked: 1 time
Joined: Sep 16, 2016 3:51 am
Full Name: Juan C.
Contact:

Re: What is "Merge" and why it takes so long to finish?

Post by JChris »

Technogod wrote:Under configure backup local storage tab select advanced. On the backup tab check create active full backups periodically. I'd create a new backup once a month. That way it doesn't have to go all the back to 180 days to create your next incremental. The most it would have to go back is 31 days. Since your retention is set to 180 days it will still be 180 days before the oldest backup is deleted.
That is AMAZING. I'll do that as storage isn't the issue here with more than 10TB of free space on my NAS :D
Anguel
Expert
Posts: 193
Liked: 18 times
Joined: Apr 16, 2015 9:01 am
Location: Germany / Bulgaria
Contact:

Re: What is "Merge" and why it takes so long to finish?

Post by Anguel »

Technogod wrote:Under configure backup local storage tab select advanced. On the backup tab check create active full backups periodically. I'd create a new backup once a month.
I wonder if with an active full every month the merge start will not only be postponed by one month but then again happen every day?
That way it doesn't have to go all the back to 180 days to create your next incremental. The most it would have to go back is 31 days. Since your retention is set to 180 days it will still be 180 days before the oldest backup is deleted.
I thought that the full backup (which is always the latest file, e.g. Sunday) is always merged with the NEXT incremental (i.e. Monday), so why would Veeam have to go back that many days at all?
Dima P.
Product Manager
Posts: 14396
Liked: 1568 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: What is "Merge" and why it takes so long to finish?

Post by Dima P. »

with an active full every month the merge start will not only be postponed by one month but then again happen every day
There is not merge when periodic full backup is enabled. Instead the ‘old’ part of backup chain remains till the last incremental backup is retired. Then the full chain gets deleted.
Anguel
Expert
Posts: 193
Liked: 18 times
Joined: Apr 16, 2015 9:01 am
Location: Germany / Bulgaria
Contact:

Re: What is "Merge" and why it takes so long to finish?

Post by Anguel »

Thanks Dima, now I see that this is shown in the docs:
https://helpcenter.veeam.com/docs/agent ... #retention
Post Reply

Who is online

Users browsing this forum: No registered users and 24 guests