Availability for the Always-On Enterprise
Post Reply
Zew
Expert
Posts: 206
Liked: 42 times
Joined: Mar 17, 2015 9:50 pm
Full Name: Aemilianus Kehler
Contact:

Progress Information for certain tasks

Post by Zew » Nov 08, 2017 5:57 pm

Hey all,

Been a long time since I posted, guess that's a good thing meaning things are working well.

I had started doing air-gapped backups since the introduction of possilby getting your backups encrypted or destroyed via malware so doing my usual monthly backup I noticed its taking FOREVER to complete a backup files health check.

I use a USB hard drive connected via my hypervisor host and do USB pass-through to my Veeam VM. The USB HDD enclosure is pretty new, and the host is a HP G7, so I'm assuming USB 2 speeds here.

also the size is 3 TB and my backup job is about 2 TB.

Last time backup jobs themselves ran at about 44 MB/s (Not to bad and generally good enough). and One can see the transfer rate and all that fun stuff when a job is completing these types of tasks.

Now the thing is its been over 36 hours and checking backup health is reporting only 80% completed, so is there anyway I can get more information on the rate of this check?

As you can expect the rate is showing @ 0 KB/s

foggy
Veeam Software
Posts: 16815
Liked: 1358 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Progress Information for certain tasks

Post by foggy » Nov 09, 2017 3:29 pm

Health check performance is mostly defined by the backup size and by what the storage can provide in terms of the read speed. In your case, both factors play the role, since the backup is pretty large and the storage is not top grade in terms of IOPS. Besides, pass-through itself is not optimal in terms of performance, attaching the USB drive to some physical repository would be more effective.

Zew
Expert
Posts: 206
Liked: 42 times
Joined: Mar 17, 2015 9:50 pm
Full Name: Aemilianus Kehler
Contact:

Re: Progress Information for certain tasks

Post by Zew » Nov 10, 2017 8:36 pm

That's a ridiculous suggestion for a air gapped backup solution. I hope you guys implement something easier..

Maybe I'll Fry the Data before running the job in hopes it simply recreates the dataset. That be faster than the health check...

foggy
Veeam Software
Posts: 16815
Liked: 1358 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Progress Information for certain tasks

Post by foggy » Nov 12, 2017 1:56 pm

Zew wrote:That's a ridiculous suggestion for a air gapped backup solution.
Why? You still copy backups to a detachable drive, it is just presented to Veeam B&R differently.

Zew
Expert
Posts: 206
Liked: 42 times
Joined: Mar 17, 2015 9:50 pm
Full Name: Aemilianus Kehler
Contact:

Re: Progress Information for certain tasks

Post by Zew » Nov 22, 2017 5:53 pm

What difference would the I/O be if it was directly connected to the B&R server VS another physical repository? the bottleneck remains the same no?

foggy
Veeam Software
Posts: 16815
Liked: 1358 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Progress Information for certain tasks

Post by foggy » Nov 23, 2017 11:55 am

My point was to avoid pass-through.

Zew
Expert
Posts: 206
Liked: 42 times
Joined: Mar 17, 2015 9:50 pm
Full Name: Aemilianus Kehler
Contact:

Re: Progress Information for certain tasks

Post by Zew » Dec 07, 2017 3:15 pm

I see, I don't think that is where the issue resided... I believe the bottleneck was the USB drive I/O in general, but it be nice to be able to see this info, but the only real way to see this stuff is via Resource Monitor.

I was doing a restore to another old USB passthough HDD, I'll spin up a VM on the same host and enable hardware passthrough for the these two different USB enclosures and drives I have and run FIO testes against them and report back my findings. Thanks for the help and suggestions so far.

FYI I noticed the job stated completed but resource manager still showed I/O to the USB disk for almost a minute afterwards. Always a good idea to ensure all I/O is done before removing a USB HDD. (Usually why they tell you to use the safely remove drive option) ;)

Zew
Expert
Posts: 206
Liked: 42 times
Joined: Mar 17, 2015 9:50 pm
Full Name: Aemilianus Kehler
Contact:

Re: Progress Information for certain tasks

Post by Zew » Dec 07, 2017 5:02 pm 1 person likes this post

I think you maybe right foggy. I haven't run direct I/O testing but even simply from using this USB drive on my local workstation VS the hardware pass-through to my Veeam VM. Indeed it shows totally different results. What's really funny is Windows file transfer (regular file explorer copy n paste) at first showed a really impressive throughput (80MB/s) watching resource manager showed the true write speed which was @ 6 MB/s. so while it'll state completing in 2 minutes (Average 6 Gig file), once it reaches 75% you see the rate on teh windows file transfer start to drop and the time remaining slowly increases... it will eventually report back the proper write speed, but it takes a while for that to happen. I'll def look into other methods to move data to this device. Thanks Alex!

foggy
Veeam Software
Posts: 16815
Liked: 1358 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Progress Information for certain tasks

Post by foggy » Dec 07, 2017 5:39 pm

Take any available physical box and make it a repository instead.

Post Reply

Who is online

Users browsing this forum: AndrewPBG, daniel.triplehorn, ManOrs and 58 guests