What is "Merge" and why it takes so long to finish?

[JChris]

I've been using VEB for some months now, but just recently I noticed that VEB started adding one more step to my routine backups, something called "Merge". This thing increased my backup times considerably. What's it and how can I "fix" this?

VEB Version 1.5.0.306. I didn't upgrade to Veeam Agent for Windows yet because it's on beta.

[vClintWyckoff]

What's your retention set for?

https://helpcenter.veeam.com/endpoint/15/retention.html

Also, FYI - VAW 2.0 went GA 2 weeks ago.

[JChris]

180 days, why? Ok, so I read the link and I believe it has something to do with the rebuild of backups, but it takes way to long, every new backup triggers this "merge" and does the injecting to move the VBK forward, is there a way to make it faster? I did a backup today and by doing a sort on my backup files by modified date I have this (only last items, in total I have more than 250 items):

2017-05-26 is today, as of this message :p

[rekun]

Hi
The merge is the process that happens when it deletes the oldest recovery points. It has to make a copy of whatever in that file is still required to restore the newer recovery points, so it will take some time. There is normal for the merge to take about 15 min, so nothing wrong here.

If you want to speed up the merge process, a simple solution (if you are running the newest update of Windows 10) would be to upgrade to Veeam Agent 2 (it is not beta anymore), and then format your backup target with REFS (remember to use 64kb blocksize). Then Veeam would utilize its fastclone technology, which should make it way faster.

[JChris]

Thanks for letting me know it wasn't in Beta anymore. I'm now using Veeam Agent 2.0.0.700 Free. Sadly I can't format my NAS to REFS, it would be a headache as I have tons and tons of files there and not only backups.

[JChris]

UPDATE: I just ran my first backup job after the upgrade to Veeam Agent and this time there wasn't any info regarding "merge" and the backup took the normal 7~8 minutes to finish.

[Technogod]

I did some research on Fast Clone and it seems to only be available in B&R 9.5 . Can somebody from Veeam verify this?

[vClintWyckoff]

Fast Clone is available in VAW 2.0 on ReFS v3.1 formatted disk targets.

[Technogod]

I stand corrected.

[JChris]

New day, same issue



It seems there's no way to stop this merge process. My retention is 180 days, a number that I passed already, and every new backup job adds 1 day to it, forcing it to delete the oldest backup and merge. All my backups from now on will take this long to finish (20+ min) compared to the default 5~7 min I had, this is depressing. I thought the retention policy would simply delete the oldest backups and be done with it, but this merge process take quite a lot of time and system resources of my NAS.

Is there anything else I can do to make it faster other than formatting as REFS?

[vClintWyckoff]

TBF, its not really an issue - its the product working as it should and as expected The reason why it takes so long is that we're merging the full VBK which is the last restore point with the next incremental to form a VBK and drop the incremental. So this is IO intensive on the NAS device which isn't exactly a high performance SAN...

[Technogod]

Under configure backup local storage tab select advanced. On the backup tab check create active full backups periodically. I'd create a new backup once a month. That way it doesn't have to go all the back to 180 days to create your next incremental. The most it would have to go back is 31 days. Since your retention is set to 180 days it will still be 180 days before the oldest backup is deleted.

[vClintWyckoff]

This is correct - Since its a chain, creating an active full resets the backup chain. So you'd have to wait an additional 180 restore points, using capacity but would remove your merge question. Pros and Cons that you should weigh out.

https://helpcenter.veeam.com/docs/agent ... #retention

[JChris]

Under configure backup local storage tab select advanced. On the backup tab check create active full backups periodically. I'd create a new backup once a month. That way it doesn't have to go all the back to 180 days to create your next incremental. The most it would have to go back is 31 days. Since your retention is set to 180 days it will still be 180 days before the oldest backup is deleted.

That is AMAZING. I'll do that as storage isn't the issue here with more than 10TB of free space on my NAS

[Anguel]

Under configure backup local storage tab select advanced. On the backup tab check create active full backups periodically. I'd create a new backup once a month.

I wonder if with an active full every month the merge start will not only be postponed by one month but then again happen every day?

That way it doesn't have to go all the back to 180 days to create your next incremental. The most it would have to go back is 31 days. Since your retention is set to 180 days it will still be 180 days before the oldest backup is deleted.

I thought that the full backup (which is always the latest file, e.g. Sunday) is always merged with the NEXT incremental (i.e. Monday), so why would Veeam have to go back that many days at all?

[Dima P.]

with an active full every month the merge start will not only be postponed by one month but then again happen every day

There is not merge when periodic full backup is enabled. Instead the ‘old’ part of backup chain remains till the last incremental backup is retired. Then the full chain gets deleted.

[Anguel]

Thanks Dima, now I see that this is shown in the docs:
https://helpcenter.veeam.com/docs/agent ... #retention