Merge operations very slow

Availability for the Always-On Enterprise

Merge operations very slow

Veeam Logoby odruard » Fri Sep 15, 2017 7:35 am


We are facing slowness during merge operations.
Same results with all jobs, proportional to the size of the job.

We use a physical proxy/repository connected to a SAN.
- Proxy reads vmdk files in Direct Storage Access mode.
- Repository uses SAN Storage allowing more than 250 MB/s

Jobs are configured as Forever Incremental, with 30 restore points kept.

Let's have an example of a job result :
- Full VBK file is 2,5 To large
- Incremental VIB file has an average of 185 GB
- Incremental backup itself (without merge) needs 70 minutes
- The Merge operation needs 400 minutes, 5 times more than the backup itself.

I tested a VIB file copy on same volume (generating both read and write I/O), and it took less than 10 minutes.
Then it is not due to the storage performaces.

How could we explain such slowness during merge ?

I have detected 2 amelioration points :
- Reformat the storage volumes with a 64 K cluster (currently 4 K)
- Enable the "Defragment an compact full backup file" option
First point will be complicated, as we need to find space to move backup files during the operation.
However I have no idea about the expected savings : 5% (which won't solve our problem) or 80% (which will reduce the merge duration to the backup duration) ?

Any idea about the cause of such slowness and the way to reduce is is welcome.

Olivier Druard
Posts: 34
Liked: 2 times
Joined: Tue Jan 25, 2011 2:12 pm
Full Name: Olivier Druard

Re: Merge operations very slow

Veeam Logoby aceit » Sat Sep 16, 2017 6:10 pm

I've a setup almost similar to yours.
Processing time of backup job in the reports are 260MB/s and speed 1.3GB/s.
The merge operation is indeed the longest operation in the backup job, in my case 2hours. Usually 100GB daily deltas.
yep, it is technically kind of slow... still I don't care so much because all the activity is done by a dedicated physical backup server and fits in the time window.
All repos are NTFS and 64K cluster.

It improve the situation if you are able to keep two repos, one with full and one with incrementals (you may need to have the license for that) via scale out repos.
Posts: 19
Liked: 8 times
Joined: Tue Jun 20, 2017 3:17 pm

Re: Merge operations very slow

Veeam Logoby foggy » Tue Oct 31, 2017 1:58 pm

odruard wrote:I tested a VIB file copy on same volume (generating both read and write I/O), and it took less than 10 minutes.
Then it is not due to the storage performaces.

In fact it is. Merge is a heavy random I/O so a (most likely) sequential file copy test cannot be used for comparison. Try to switch to simple forward incremental mode, which doesn't require merge operations on the repository.
Veeam Software
Posts: 15589
Liked: 1161 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Return to Veeam Backup & Replication

Who is online

Users browsing this forum: Bing [Bot], cb1ocker, KFlynn, Shestakov, SyNtAxx, Unigas and 1 guest