Backup Very Slow

[JaxIsland7575]

Last night I installed the Windows updates that were published this week. Rebooted the machines. I have 3 Hyper-V hosts in a cluster and each host machine I had to cancel the backup job as it was running over 6+ hours holding up my remaining VBR jobs.
I am not sure if there is a windows update that caused this or the reboot but here is my case#: 01694533

Cheers!

[Dima P.]

Hi Ryan,

We will investigate - I'll follow up shortly.

[JaxIsland7575]

Thanks Dmitry!
I just uploaded Application and System logs to the FTP site. There is no A/V on these host servers having the issues. Not quite sure what's going on with them.

Cheers!

[Dima P.]

Thanks Ryan! Yes, we do see the perfomance decrees in your logs. Could you servers be heavily loaded at that time? The reason why I am asking – VEB processes are suppressed due the throttling. QA is still investigating.

[JaxIsland7575]

I removed all the guest servers from the host machine and tried the backup, same result.

Let me know if you find anything else!

[JaxIsland7575]

I was able to do another reboot (3rd one) and ran the jobs again with no guests on them and the backups worked. Moved guests back onto them and it worked again. I am not sure what finally changed but the backups seem to be working again. I would be curious what QA would find in the logs!

Cheers!

[Dima P.]

Thanks for sharing the news. I'll let team know - they may ask for additional logs thru the support team.

[JaxIsland7575]

Absolutely, I get nervous when there are windows updates and then something doesn't work after the reboot. No problem though, let me know if they need anything else!

[Dima P.]

Hi Ryan,

VEB process has lowest (background) priority to have zero impact on running apps, thus backup was sitting in the idle state. According to the logs 99% of backup time VEB was waiting for the disks to respond, while it was busy with other operations. Unfortunately, we cannot identify the root cause - might be some resource consuming operating with higher process priority.

[JaxIsland7575]

Yeah it seems that since the reboots there hasn't been an issue. Have to move this to the "one time freak event" thing, even though I don't like doing that I cant reproduce and haven't seen it again.

Thanks for taking a look!

Cheers!

[Dima P.]

Out of curiosity - do your know the ids for the applied updates? I am mostly thinking updates caused it, but still wonder how.

[JaxIsland7575]

I agree that it was most likely updates, here is the list


*Let me know if that's too tough to read Ill repost larger if it helps.

[Dima P.]

Good enough, thanks. I’ll let you know if I find anything.