B&R 9 : Health check of VM Backup Files needs so long time

VMware specific discussions

Re: B&R 9 : Health check of VM Backup Files needs so long ti

Veeam Logoby PTide » Tue Feb 16, 2016 10:09 am

Can you explain how fragmentation would not be reduced by having only one VM processed at a time? With 4 VMs all writing to the target at the same time their data blocks will be interleaved with each other as the target allocates free space to write to. Simultaneous writes is a classic cause of file fragmentation.
My mistake. I was sure that something should had been invented in order to reduce fragmentation during parallel writes. My apologies.

If I enable compact full, and all 4 vms start compacting their fulls at the same time, I'll have the same problem ... simultaneous writes to the same data store = fragmented files.
Your logic is correct, that's why there is no parallel processing for compact operations.

after reconfiguring veeam to use the good interface, I'm now seeing processing rate around 35MB/sec. yeah!
Good to hear that. Feel free to contact us if any isuues arise.

Thank you.
PTide
Veeam Software
 
Posts: 3230
Liked: 269 times
Joined: Tue May 19, 2015 1:46 pm

Re: B&R 9 : Health check of VM Backup Files needs so long ti

Veeam Logoby bkc » Wed Feb 17, 2016 2:45 am

so the copy job has now been running for 28 hours and about 26 of those hours it's been at 99% complete state.

it seems that after the vms are copied over there's a bunch of house-keeping to perform, creating some kind of fulls or something and now it seems to be creating a GFS restore point.

I think it would be very useful if these house-keeping steps appeared after the named list of vms with clear details of what's happening, what % is completed and the I/O performance the system sees such as read or write performance of the target backup repository.

likewise during the monthly health check it would be good to see more details about what is happening, how far along it is and what the I/O performance is.

showing only 99% complete for 26 hours w/o any other performance info isn't very informative..

thanks
bkc
Influencer
 
Posts: 20
Liked: 3 times
Joined: Tue Dec 21, 2010 10:31 pm
Full Name: brad clements

Re: B&R 9 : Health check of VM Backup Files needs so long ti

Veeam Logoby bkc » Wed Feb 17, 2016 2:28 pm

Job has been running 40 hours, still creating a GFS restore point, still at 99%.

I really think more feedback would be helpful..
bkc
Influencer
 
Posts: 20
Liked: 3 times
Joined: Tue Dec 21, 2010 10:31 pm
Full Name: brad clements

[MERGED] Health check for copy jobs

Veeam Logoby andriktr » Mon Feb 29, 2016 8:43 am

Hello,
Recently migrated our backup copy jobs from StoreOnce CIFS shares to StoreOnce catalyst stores. As was recommended I cloned old BC jobs and configured them with new catalyst repository. That means absolutely new backup chain started (GFS retention used). Jobs performance few times better than it was with shares, but one thing still look strange for me. We have enabled health check for these jobs ( being executed once per month) and this procedure takes a lot of time ~ 5-8 hours for job. I would like to mark that it's a pretty new jobs configured few days ago and each VM have only 2-3 recovery points. StoreOnce catalyst have a requirement to use per-VM backup files that means we will have much more .vbk files in repository. Can this will be a reason for such long health check process. Also what else can impact health check performance?
Thank you.
andriktr
Enthusiast
 
Posts: 35
Liked: 1 time
Joined: Mon Mar 02, 2015 11:53 am
Full Name: Andrej

Re: B&R 9 : Health check of VM Backup Files needs so long ti

Veeam Logoby foggy » Mon Feb 29, 2016 2:52 pm

Depending on the backup size, this might be expected (see above). Are you saying it used to complete faster before migration to Catalyst?
foggy
Veeam Software
 
Posts: 15272
Liked: 1131 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: B&R 9 : Health check of VM Backup Files needs so long ti

Veeam Logoby andriktr » Mon Feb 29, 2016 3:18 pm

No, on CIFS shares it wasn't faster also needed many time to complete. I expected that after migration to catalyst this time will be some how minimized.
Also hope that completion time amount will not grow up hugely in future due to a largest amount of backup files.
andriktr
Enthusiast
 
Posts: 35
Liked: 1 time
Joined: Mon Mar 02, 2015 11:53 am
Full Name: Andrej

Re: B&R 9 : Health check of VM Backup Files needs so long ti

Veeam Logoby lando_uk » Mon Feb 29, 2016 5:21 pm

On this subject - If a health check of a copy job is fine, could one presume the last restore point of the main backup job that it was sourced from is also fine? Or could the same restore point be knackered on the backup job, but ok on the copy job?
lando_uk
Expert
 
Posts: 247
Liked: 18 times
Joined: Thu Oct 17, 2013 10:02 am
Location: UK
Full Name: Mark

Re: B&R 9 : Health check of VM Backup Files needs so long ti

Veeam Logoby PTide » Mon Feb 29, 2016 6:01 pm

Hi

On this subject - If a health check of a copy job is fine, could one presume the last restore point of the main backup job that it was sourced from is also fine?
No.

Or could the same restore point be knackered on the backup job, but ok on the copy job?
Yes. For example your primary storage corruption can occur after the backup copy sync has been completed. In this case your backup copy restore point will be ok whereas your primary backup will not.

Please refer to helpcenter:

When a new synchronization interval starts, Veeam Backup & Replication performs a health check for the most recent restore point in the backup chain. Veeam Backup & Replication calculates checksums for data blocks in the backup file on the target backup repository and compares them with the checksums that are already stored in the backup file.


Your backup copy job is an independent set of files so Backup Copy health check makes sure that your secondary restore point is not corrupted.

Thank you.
PTide
Veeam Software
 
Posts: 3230
Liked: 269 times
Joined: Tue May 19, 2015 1:46 pm

Re: B&R 9 : Health check of VM Backup Files needs so long ti

Veeam Logoby foggy » Tue Mar 01, 2016 12:54 pm

andriktr wrote:Also hope that completion time amount will not grow up hugely in future due to a largest amount of backup files.

Per-VM backup chains option should not affect health check performance (actually, it could even be faster with per-VM, since less metadata is kept within each backup file and they are less fragmented).
foggy
Veeam Software
 
Posts: 15272
Liked: 1131 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: B&R 9 : Health check of VM Backup Files needs so long ti

Veeam Logoby andriktr » Mon Mar 07, 2016 7:53 am

Are there any recommendations for StoreOnce catalyst repository maximum concurrent tasks?
andriktr
Enthusiast
 
Posts: 35
Liked: 1 time
Joined: Mon Mar 02, 2015 11:53 am
Full Name: Andrej

Re: B&R 9 : Health check of VM Backup Files needs so long ti

Veeam Logoby Gostev » Mon Mar 07, 2016 1:52 pm

Kindly please do not hijack this topic. This question is best directed into Catalyst support topic (and the number depends on Catalyst model anyway). Thanks!
Gostev
Veeam Software
 
Posts: 21603
Liked: 2405 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

[MERGED] Healthcheck Offside

Veeam Logoby robvs » Mon Aug 07, 2017 7:01 am

Hello,

I got some questions about the Veeam health check on the copy out jobs.

We got a VMWare enviroment with 120 VM's the total backup size is around 10 TB, these vm's are separated in 11 Veeam jobs (forever incremental).
All these veeam jobs have copy out jobs to a offside location.
The connection between these location is around 200Mbit.
The current problem we have now is that a health check on the offside backup could take around 1 week for some jobs, this is causing problems because the copyout job will not run untill the healthcheck is finished. When it acctually starts it will have to copy such a big amount of data that that will also take a lot of time to complete.

Is there a way to offload the health checks to a offside server because this would eliminate the bandwith problem that is causing the health checks taking a lot of time.
If not what would be a goot solution to this for speeding it up ?


Kind regards
Rob
robvs
Lurker
 
Posts: 1
Liked: never
Joined: Mon Aug 07, 2017 6:52 am
Full Name: Rob

Re: Healthcheck Offside

Veeam Logoby DGrinev » Mon Aug 07, 2017 1:18 pm

Hi Rob and welcome to the community!

Health-check performance depends on reading speed of the storage and the backup file size. It cannot be delegated to the remote server as it's part of the backup/copy job.
As an alternative you can to set up Surebackup job for the source backups as it's best approach for the recovery verification.
Please review this thread for additional information. Thanks!
DGrinev
Veeam Software
 
Posts: 608
Liked: 71 times
Joined: Thu Dec 01, 2016 3:49 pm
Full Name: Dmitry Grinev

[MERGED] Health Check Long Duration

Veeam Logoby egrutman » Fri Oct 27, 2017 3:08 pm

Hello,

I have backups running and the health check takes days. We are backing up about 85 machines with average size of 125 GB. So total size is 10 TB of data we are backing up. The backups are stored on DD2200. Is there anyway to speed the process of the healthcheck? 12 hours have gone by and I am only at 5%. At this rate it will take 10 days to run a health check. This is not acceptable for critical system production machines to be without backups and I am trying to find anyway possible to increase resources to complete the job at a faster pace.
egrutman
Novice
 
Posts: 4
Liked: never
Joined: Tue May 31, 2016 1:22 pm
Full Name: Yevgeniy Grutman

Re: Health Check Long Duration

Veeam Logoby DGrinev » Fri Oct 27, 2017 3:30 pm

Hi Yevgeniy,

Please review this discussion about the health check performance as it contains plenty of useful considerations. Thanks!
DGrinev
Veeam Software
 
Posts: 608
Liked: 71 times
Joined: Thu Dec 01, 2016 3:49 pm
Full Name: Dmitry Grinev

PreviousNext

Return to VMware vSphere



Who is online

Users browsing this forum: No registered users and 5 guests