Availability for the Always-On Enterprise
Post Reply
odruard
Enthusiast
Posts: 39
Liked: 4 times
Joined: Jan 25, 2011 2:12 pm
Full Name: Olivier Druard
Contact:

Merge operations very slow

Post by odruard » Sep 15, 2017 7:35 am

Hello,

We are facing slowness during merge operations.
Same results with all jobs, proportional to the size of the job.

We use a physical proxy/repository connected to a SAN.
- Proxy reads vmdk files in Direct Storage Access mode.
- Repository uses SAN Storage allowing more than 250 MB/s

Jobs are configured as Forever Incremental, with 30 restore points kept.

Let's have an example of a job result :
- Full VBK file is 2,5 To large
- Incremental VIB file has an average of 185 GB
- Incremental backup itself (without merge) needs 70 minutes
- The Merge operation needs 400 minutes, 5 times more than the backup itself.

I tested a VIB file copy on same volume (generating both read and write I/O), and it took less than 10 minutes.
Then it is not due to the storage performaces.

How could we explain such slowness during merge ?

I have detected 2 amelioration points :
- Reformat the storage volumes with a 64 K cluster (currently 4 K)
- Enable the "Defragment an compact full backup file" option
First point will be complicated, as we need to find space to move backup files during the operation.
However I have no idea about the expected savings : 5% (which won't solve our problem) or 80% (which will reduce the merge duration to the backup duration) ?

Any idea about the cause of such slowness and the way to reduce is is welcome.

Thanks
Olivier Druard

aceit
Influencer
Posts: 22
Liked: 8 times
Joined: Jun 20, 2017 3:17 pm
Contact:

Re: Merge operations very slow

Post by aceit » Sep 16, 2017 6:10 pm

I've a setup almost similar to yours.
Processing time of backup job in the reports are 260MB/s and speed 1.3GB/s.
The merge operation is indeed the longest operation in the backup job, in my case 2hours. Usually 100GB daily deltas.
yep, it is technically kind of slow... still I don't care so much because all the activity is done by a dedicated physical backup server and fits in the time window.
All repos are NTFS and 64K cluster.

It improve the situation if you are able to keep two repos, one with full and one with incrementals (you may need to have the license for that) via scale out repos.

foggy
Veeam Software
Posts: 16837
Liked: 1361 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Merge operations very slow

Post by foggy » Oct 31, 2017 1:58 pm

odruard wrote:I tested a VIB file copy on same volume (generating both read and write I/O), and it took less than 10 minutes.
Then it is not due to the storage performaces.
In fact it is. Merge is a heavy random I/O so a (most likely) sequential file copy test cannot be used for comparison. Try to switch to simple forward incremental mode, which doesn't require merge operations on the repository.

Post Reply

Who is online

Users browsing this forum: Mig29777, raymix and 63 guests