Host-based backup of VMware vSphere VMs.
Post Reply
Manu80
Service Provider
Posts: 12
Liked: 1 time
Joined: Mar 10, 2020 9:05 pm
Full Name: Manu80
Contact:

Merge oldest incremental very slow

Post by Manu80 »

Hello,
I have Veeam Backup & Replication 9.5 U4 installed on Windows Server 2019.
Veeam Server is a VM. 4vCPU and 16 Gb Ram.

I have one backup job with 43 VM in a vSphere Cluster (Windows and Linux Guest OS).
Backup settings are daily with 30 restore points, backup mode is Incremental.
VM Size into vSphere are about 7.5 Tb.

Every day the Data Read from Veeam are 1.6 Tb and Transferred are about 400/500 Gb.

Load from Veeam are about this: Source 62% > Proxy 60% > Network 46% > Target 31%

The problem is that every day the Full backup file merge require about 10/12 hours.

VBK file are 5 Tb, every vib are abount 400/500 Gb.

Target repository are CIFS share on NAS with 12 HDD (7.2Krpm) with RAID 6.
Connectivity betweend vSphere, Veaam Server and target repository are 10 Gb/s.

I never seen the merge so slow.

Do you recommend me in this situation to avoid slow merge ?
What is the bottleneck?

I would divide the current single job with 43 VM into 4 jobs of 10 VM ... could it help?


Thanks
Manu80
HannesK
Product Manager
Posts: 14287
Liked: 2877 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: Merge oldest incremental very slow

Post by HannesK »

Hello,
Target repository are CIFS share on NAS with 12 HDD (7.2Krpm) with RAID 6.
that looks like expected for me.

Just to clarify:
1) are you using weekly synthetic fulls or forward incremental forever?
2) are you using per-vm backup files?

There are two ways to solve the issue
1) use active full to avoid merges
2) use REFS (or XFS if you have Linux experience) on iSCSI instead of SMB protocol

Best regards,
Hannes
Manu80
Service Provider
Posts: 12
Liked: 1 time
Joined: Mar 10, 2020 9:05 pm
Full Name: Manu80
Contact:

Re: Merge oldest incremental very slow

Post by Manu80 »

I use only incremental forever, no weekly synthetic fulls.
I not using per-vm backup files.

Manu80
Manu80
Service Provider
Posts: 12
Liked: 1 time
Joined: Mar 10, 2020 9:05 pm
Full Name: Manu80
Contact:

Re: Merge oldest incremental very slow

Post by Manu80 »

HannesK wrote: May 22, 2020 5:32 am There are two ways to solve the issue
1) use active full to avoid merges
2) use REFS (or XFS if you have Linux experience) on iSCSI instead of SMB protocol

Best regards,
Hannes
You suggest to
- move backup files to new target repository with ISCSI and ReFS;
- enable weekly synthetic full (mantain actual unique job with 43 VM and 30 restore points)

?


Thanks
Manu80
HannesK
Product Manager
Posts: 14287
Liked: 2877 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: Merge oldest incremental very slow

Post by HannesK »

Hello,
hmm okay, then I would expect around 6h for the merge (500GB / (2MByte/s per disk x 12). 10-12 sounds a little bit slow for me. But well, NAS boxes are hard to predict.

My suggestion is an OR. So you both suggestions should solve your problem.

No, I did not suggest synthetic full. I suggested active full :-)


Per-VM backup chains also usually improve performance.

Best regards,
Hannes
Manu80
Service Provider
Posts: 12
Liked: 1 time
Joined: Mar 10, 2020 9:05 pm
Full Name: Manu80
Contact:

Re: Merge oldest incremental very slow

Post by Manu80 »

Active Full ... but with Active Full I need more more space on repository ...
With synthetic full I save more space ...
HannesK
Product Manager
Posts: 14287
Liked: 2877 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: Merge oldest incremental very slow

Post by HannesK »

active / synthetic full that makes no difference on SMB storage from a disk space usage perspective

With REFS you don't need to change the backup mode.
Manu80
Service Provider
Posts: 12
Liked: 1 time
Joined: Mar 10, 2020 9:05 pm
Full Name: Manu80
Contact:

Re: Merge oldest incremental very slow

Post by Manu80 »

But, if I change repository from SMB to ISCSI and I format volume with ReFS, it's good in this situation if I use Synthetic Full instead Active Full ?
If I have incremental forever backup daily, with Active full weekly with 30 restore points, I have 6 VBK from 5 TB each.
With synthetic full I saving abount 20 Tb.

Finally ... for daily backup with 30 restore point, 43 VM in unique job, Backup mode incremental forever with synthetic full weekly is good and have fast merge and space saving ... is the best solution ?
Or you not reccomend this ?

Very thanks
Manu80
HannesK
Product Manager
Posts: 14287
Liked: 2877 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: Merge oldest incremental very slow

Post by HannesK »

With REFS you don't need to change the backup mode.
but yes, you can also use synthetic full. please check the animation in the blog post I posted earlier.

also check out the restore point simulator to understand the backup modes. the "manual run" button helps to understand every step. It also shows REFS disk space savings if you check the checkbox https://rps.dewin.me/

everything is good on REFS :-)
Manu80
Service Provider
Posts: 12
Liked: 1 time
Joined: Mar 10, 2020 9:05 pm
Full Name: Manu80
Contact:

Re: Merge oldest incremental very slow

Post by Manu80 »

Very Thanks Hannes ;)
Post Reply

Who is online

Users browsing this forum: No registered users and 71 guests