-
- Service Provider
- Posts: 15
- Liked: 1 time
- Joined: Mar 10, 2020 9:05 pm
- Full Name: Manu80
- Contact:
Merge oldest incremental very slow
Hello,
I have Veeam Backup & Replication 9.5 U4 installed on Windows Server 2019.
Veeam Server is a VM. 4vCPU and 16 Gb Ram.
I have one backup job with 43 VM in a vSphere Cluster (Windows and Linux Guest OS).
Backup settings are daily with 30 restore points, backup mode is Incremental.
VM Size into vSphere are about 7.5 Tb.
Every day the Data Read from Veeam are 1.6 Tb and Transferred are about 400/500 Gb.
Load from Veeam are about this: Source 62% > Proxy 60% > Network 46% > Target 31%
The problem is that every day the Full backup file merge require about 10/12 hours.
VBK file are 5 Tb, every vib are abount 400/500 Gb.
Target repository are CIFS share on NAS with 12 HDD (7.2Krpm) with RAID 6.
Connectivity betweend vSphere, Veaam Server and target repository are 10 Gb/s.
I never seen the merge so slow.
Do you recommend me in this situation to avoid slow merge ?
What is the bottleneck?
I would divide the current single job with 43 VM into 4 jobs of 10 VM ... could it help?
Thanks
Manu80
I have Veeam Backup & Replication 9.5 U4 installed on Windows Server 2019.
Veeam Server is a VM. 4vCPU and 16 Gb Ram.
I have one backup job with 43 VM in a vSphere Cluster (Windows and Linux Guest OS).
Backup settings are daily with 30 restore points, backup mode is Incremental.
VM Size into vSphere are about 7.5 Tb.
Every day the Data Read from Veeam are 1.6 Tb and Transferred are about 400/500 Gb.
Load from Veeam are about this: Source 62% > Proxy 60% > Network 46% > Target 31%
The problem is that every day the Full backup file merge require about 10/12 hours.
VBK file are 5 Tb, every vib are abount 400/500 Gb.
Target repository are CIFS share on NAS with 12 HDD (7.2Krpm) with RAID 6.
Connectivity betweend vSphere, Veaam Server and target repository are 10 Gb/s.
I never seen the merge so slow.
Do you recommend me in this situation to avoid slow merge ?
What is the bottleneck?
I would divide the current single job with 43 VM into 4 jobs of 10 VM ... could it help?
Thanks
Manu80
-
- Product Manager
- Posts: 14840
- Liked: 3086 times
- Joined: Sep 01, 2014 11:46 am
- Full Name: Hannes Kasparick
- Location: Austria
- Contact:
Re: Merge oldest incremental very slow
Hello,
Just to clarify:
1) are you using weekly synthetic fulls or forward incremental forever?
2) are you using per-vm backup files?
There are two ways to solve the issue
1) use active full to avoid merges
2) use REFS (or XFS if you have Linux experience) on iSCSI instead of SMB protocol
Best regards,
Hannes
that looks like expected for me.Target repository are CIFS share on NAS with 12 HDD (7.2Krpm) with RAID 6.
Just to clarify:
1) are you using weekly synthetic fulls or forward incremental forever?
2) are you using per-vm backup files?
There are two ways to solve the issue
1) use active full to avoid merges
2) use REFS (or XFS if you have Linux experience) on iSCSI instead of SMB protocol
Best regards,
Hannes
-
- Service Provider
- Posts: 15
- Liked: 1 time
- Joined: Mar 10, 2020 9:05 pm
- Full Name: Manu80
- Contact:
Re: Merge oldest incremental very slow
I use only incremental forever, no weekly synthetic fulls.
I not using per-vm backup files.
Manu80
I not using per-vm backup files.
Manu80
-
- Service Provider
- Posts: 15
- Liked: 1 time
- Joined: Mar 10, 2020 9:05 pm
- Full Name: Manu80
- Contact:
Re: Merge oldest incremental very slow
You suggest to
- move backup files to new target repository with ISCSI and ReFS;
- enable weekly synthetic full (mantain actual unique job with 43 VM and 30 restore points)
?
Thanks
Manu80
-
- Product Manager
- Posts: 14840
- Liked: 3086 times
- Joined: Sep 01, 2014 11:46 am
- Full Name: Hannes Kasparick
- Location: Austria
- Contact:
Re: Merge oldest incremental very slow
Hello,
hmm okay, then I would expect around 6h for the merge (500GB / (2MByte/s per disk x 12). 10-12 sounds a little bit slow for me. But well, NAS boxes are hard to predict.
My suggestion is an OR. So you both suggestions should solve your problem.
No, I did not suggest synthetic full. I suggested active full
Per-VM backup chains also usually improve performance.
Best regards,
Hannes
hmm okay, then I would expect around 6h for the merge (500GB / (2MByte/s per disk x 12). 10-12 sounds a little bit slow for me. But well, NAS boxes are hard to predict.
My suggestion is an OR. So you both suggestions should solve your problem.
No, I did not suggest synthetic full. I suggested active full
Per-VM backup chains also usually improve performance.
Best regards,
Hannes
-
- Service Provider
- Posts: 15
- Liked: 1 time
- Joined: Mar 10, 2020 9:05 pm
- Full Name: Manu80
- Contact:
Re: Merge oldest incremental very slow
Active Full ... but with Active Full I need more more space on repository ...
With synthetic full I save more space ...
With synthetic full I save more space ...
-
- Product Manager
- Posts: 14840
- Liked: 3086 times
- Joined: Sep 01, 2014 11:46 am
- Full Name: Hannes Kasparick
- Location: Austria
- Contact:
Re: Merge oldest incremental very slow
active / synthetic full that makes no difference on SMB storage from a disk space usage perspective
With REFS you don't need to change the backup mode.
With REFS you don't need to change the backup mode.
-
- Service Provider
- Posts: 15
- Liked: 1 time
- Joined: Mar 10, 2020 9:05 pm
- Full Name: Manu80
- Contact:
Re: Merge oldest incremental very slow
But, if I change repository from SMB to ISCSI and I format volume with ReFS, it's good in this situation if I use Synthetic Full instead Active Full ?
If I have incremental forever backup daily, with Active full weekly with 30 restore points, I have 6 VBK from 5 TB each.
With synthetic full I saving abount 20 Tb.
Finally ... for daily backup with 30 restore point, 43 VM in unique job, Backup mode incremental forever with synthetic full weekly is good and have fast merge and space saving ... is the best solution ?
Or you not reccomend this ?
Very thanks
Manu80
If I have incremental forever backup daily, with Active full weekly with 30 restore points, I have 6 VBK from 5 TB each.
With synthetic full I saving abount 20 Tb.
Finally ... for daily backup with 30 restore point, 43 VM in unique job, Backup mode incremental forever with synthetic full weekly is good and have fast merge and space saving ... is the best solution ?
Or you not reccomend this ?
Very thanks
Manu80
-
- Product Manager
- Posts: 14840
- Liked: 3086 times
- Joined: Sep 01, 2014 11:46 am
- Full Name: Hannes Kasparick
- Location: Austria
- Contact:
Re: Merge oldest incremental very slow
but yes, you can also use synthetic full. please check the animation in the blog post I posted earlier.With REFS you don't need to change the backup mode.
also check out the restore point simulator to understand the backup modes. the "manual run" button helps to understand every step. It also shows REFS disk space savings if you check the checkbox https://rps.dewin.me/
everything is good on REFS
-
- Service Provider
- Posts: 15
- Liked: 1 time
- Joined: Mar 10, 2020 9:05 pm
- Full Name: Manu80
- Contact:
Re: Merge oldest incremental very slow
Very Thanks Hannes
Who is online
Users browsing this forum: No registered users and 61 guests