Manual health check?

[andyg]

feature request - offline compacting

It would be great if we could schedule the Defrag/compacting of a job outside the backup window. Our backup storage is slow, so the compacting really hurts backup performance (lots of reading and writting to make new VBK file) and also make other jobs run longer due to high IO on backup storage. It could easily be run outside the backup window and not impact backups. I realise this will also mean higher storage requirements.

I realise Win 2016 and ReFS will mostly nullify this request (Fast clone etc)

[RGijsen]

Hi,
maybe this is related slightly to vmware-vsphere-f24/health-check-on-copy ... 22409.html.

Since we moved our remote-backup to ReFS we get quite good data reduction and we are rather pleased with it! But of course the previously sequential data now gets more and more fragmented on the disk because of the block-cloning. While we have a dedicated SAN on the main site, the remote site has to do with a rather simple Synology 1524 NAS, with only a 1GB conenction in our setup and a bunch of relatively slow 5400RPM 2TB spindles. We now do a weekly health-check on our jobs, spread out over as many days as we have jobs, so only one job does health check on a given day, to prevent the IO from filling up. Still it takes a bit longer than I'd like. As health-check runs BEFORE the actual copy-job, and the amount of data growing, in our setup that means the health check sometimes runs for several hours.

Wouldn't it be nice to first do the copy-job and THEN do the health-check? I think Veeams primary job is to get the data in a safe place as quickly as possible. Given that TCP SHOULD cover for transmission errors, it's rather unlike that corruption would occur often anyway. And even if errors were found, which I've never seen in our setup so far, the regular repair-functionality could kick in.

[RGijsen]

It's about 4 months, no reply. The request still applies though. I should place requests here, right?

[ferrus]

+1 here.
As a poster on the original thread, we've had to abandon health checks because of the performance, but better scheduling could help.
There are many admin tasks that I believe would be better run concurrently to other jobs, rather than preventing them from starting. Merges, Compacts, Health Checks.

Having Health Checks that are restart-able between backups - rather than disrupt them, would be great (although I imagine this would be tricky if the parent job uses merges).

[Fredo]

I have a feature request: we want to be able to schedule the health check as a separate job.
We monthly perform a backup files health check (we have 19 disk backup jobs).
It looks like, the health check is slowing down our GFS tape backup.
It would be convenient if we can create separate health check jobs and be able to schedule the exact starting moment.

[PTide]

Hi,

You feature request is noted. For now you might want to use this script to perform validation of your backups separately from the backup job.

Thanks

[RGijsen]

Could anyone from Veeam reply, or tell me where to properly raise a feature request?

[foggy]

You've already done this by posting your suggestion on forums.

[LarsWulf]

Hi everybody,
I am having the need to compact a single VM (using per VM Backup Files) out of a job with multiple VMs.
Is there a possibility to do such thing with a powershell script or an application I am missing?
I am think about something like the validator.exe for Backup validation.

If there is no other way to compact & defrag Backupfiles then using the regular maintenance job settings it would be nice to get such option like with the validator.exe or with a dedicated powershell cmdlet.

Regards
Lars

[foggy]

Hi Lars, please review this thread, discussing similar questions. Thanks.

[acatic]

+1 for being able to manually run health checks/compactions. Our repository craps out with all that IO while backups run as well. I already have my backups running sequentially, one VM at a time - together with Backup Copy and SureBackup. Would loooove to squeeze in health check/compaction during low-IO periods.

[reaperhammer]

+1 for running manual compact jobs on Backup and Backup Copy Jobs, whether its via powershell or the GUI, which ever gets it implemented first would be fine.

[billcouper]

Hello!

I am having slow performance of health check, which then causes linked jobs to not process during backup window. The primary backup has completed, but then health check starts, so copy jobs and tape jobs cannot run until it completes.

Is it possible to let health check run simultaneous to copy/tape jobs?

Or, can the health check be moved to a job that ONLY runs health check? That way I could schedule health check to start at 6am and not run during backup window (6pm-6am).

Would the health-check have less work to do if it was scheduled on a day when a synthetic full has been created? Rather than on a day that might include multiple incremental in the chain.
Which Veeam component processes the health check? I have a completely split component architecture with backup servers, proxy servers, repository servers, etc. The repo server has to read the data and transport it to..... which server?

[foggy]

Is it possible to let health check run simultaneous to copy/tape jobs?

Or, can the health check be moved to a job that ONLY runs health check?

This is not possible.

Would the health-check have less work to do if it was scheduled on a day when a synthetic full has been created? Rather than on a day that might include multiple incremental in the chain.

No, it would still need to calculate checksums for all the blocks in the latest restore point.

Which Veeam component processes the health check? I have a completely split component architecture with backup servers, proxy servers, repository servers, etc. The repo server has to read the data and transport it to..... which server?

Health check is performed by the data mover installed on the repository server (or the gateway, in case of CIFS or dedupe type repository).

[owen.wright]

+1 for scheduling compacting as a seperate job without requiring backups.
I have dozens of customers asking about this...

[Avi Milbaum]

Hello

Running the jobs one after the other is problematic because I have quite a few jobs on a daily and weekly basis.
The maintenance process greatly changes the timing.
It is very important to have the possibility of running maintenance separately, not in the backup frame

This way I can run the maintenance in the middle of the day and does not conflict with backup jobs

Thanks
Avi

[Vitaliy S.]

Hi Avi,

Can you please clarify what maintenance task are you referring to? Do you possibly talk about this functionality? Does enabling maintenance tasks greatly increase job duration? If yes, what are your jobs bottleneck stats?

Thanks!

[RGijsen]

Hi,
what's the status on this? Now we moved to ReFS, and over time out volumes obviously get more and more fragmented because of the block-cloning, health checking gets a bigger problem every time we run it. I'd really like to FIRST do the backup, and then the health check, rather than the other way around. For our fileservers for example, we schedule that at 0:00 AM at night. So that backup is usually done between 0:00 and 1:00. However when healthcheck runs first, it's done literally any point in the day, which actually makes us breach our SLA in which we commit to doing backups between 0:00am and 6:00am.

We are using a dedicated 8Gbps SAN with enough spindles to make it fast, but still these health checks are a real pain in the .... Our SAN's do media scrubbing by them selves, so I'm not even sure if I really need these Veeam checks, but it's a very safe idea of course. So please, what's the status / roughly ETA of this request?

[foggy]

Are you talking about backup copy? Since for regular backups, health check is performed at the end of the job.

[RGijsen]

That's for copies indeed. I lately was in the understanding that regular backup jobs behaved the same with health checks, but it turns out that was the extreme ReFS slowdown as of lately that 'hung' the jobs for too long, confusing me. But indeed, this is for copies, and that's what we've asked before and 'was put in a request'. Howeve I've never heard from it before. Currently, due to the health checks, and probably because of the slow ReFS, when a copy job triggers the health check, chances are big that the copy job just doesn't make it in time. I'd rather have the copy be done first, and then the health check, regardless of how quick a health check can or should be. I think it's better to have the data copied as soon as possible. The chances of data actually being corrupted is pretty small anyway, and missing a copy in the chain is not cool.

[edit]
Maybe it's nice to have a choice, some people might want the check BEFORE the copy, some might want it AFTER the copy. Or, as I've read somewhere is being worked on, Veeam should make the health checks as a job, so we can schedule them to run after a certain job.

[RGijsen]

I now remember what another issue was, as it occured today again. We've set our jobs to run after each other. But the first job got its health check today. So first the backup runs, then the health check for about 8 hours, and only when THAT'S finished, the copy job of the first job starts, as well as the backup jobs scheduled to run after the first.
To work around that I've now configured the jobs to run as 0:00, 0:02, 0:04 and so on, so they don't have to wait on each other, but just wait for free resources. Still, again, I'd really like to see more options here. Having to wait for health check before getting the copy to offsite is a no-go in my opinion.

[foggy]

To work around that I've now configured the jobs to run as 0:00, 0:02, 0:04 and so on, so they don't have to wait on each other, but just wait for free resources.

And this is the recommended approach anyway. Jobs chaining is not the best practice.

[CPCaveman]

+1 for this request, it would be useful to process such tasks as this and others outside of the time for backups, regardless of normal backup or BCJ.
The more flexibility to schedule the individual tasks at the appropriate times or orders the easier it is for people to ensure the workloads on the servers are balanced and completed on time.

The Online (SLA tied, generally backups/off-siting) processes should be able to be split from the "offline, Non SLA" tasks, these "offline" tasks which involve just the backup infrastructure (health checks, synthetics Tranforms, etc) should be a separate task or at the very least an option to make it fully configurable when required.

[Fredo]

+1 for this request. I made my request already in September 2017, but it would still be very helpful if health checks could be scheduled to run on their own (and not bound to backup jobs).

[Avi Milbaum]

The separation between jobs and maintenance will allow for example to run the maintenance on the weekend or in the morning not during the backup time.

[Voiper]

+1 for this request! We also want to run health-checks when backup and backup copy jobs are done. I mean periods of low activity. Hope this can be implemented soon!

[cseeliger]

+1 for this request.
We have multiple customers who would like to schedule maintenance and compact tasks seperately.

[Tecbil]

+1 for running manual health-checks on Backup and Backup Copy Jobs

[4ljames]

+1

These should be configurable on when to run or the ability to schedule them as part of a separate maintenace job. Makes no sense to hold up production backups on this maintenance task. Primary priority in a backup window is to get the data off fast. Let us configure the health checks outside of our backup windows or specific to the environment.

Could you please confirm if this is on a roadmap to be implemented? The original request is quite old now.

[Stephan23]

+1 for manual/independent health check/compact schedule for reasons mentioned above.

Another issue I observed is that, whether the health check runs is determined by the time the backup finishes, not when it began.
So, if a health check is set up to run every Thursday and the backup that started Wednesday finishes at 1 am Thursday, the health check would run. I can't imagine, this is intended.