Discussions specific to the VMware vSphere hypervisor
Post Reply
perseusco
Influencer
Posts: 16
Liked: never
Joined: May 08, 2018 8:17 pm
Full Name: Perseus Coronado
Contact:

Surebackup VM's not being consistent on booting

Post by perseusco » Sep 27, 2018 5:46 pm

Hi,
I'd like to know what has been the results of anyone who has used Surebackup as a way of testing backups. I have been playing with this part of VBR for months and my biggest challenge has been the inconsistency on the booting times of the restored VM's while running the same job several times, using the same restore point, booting for a VM, a DC for example goes from not booting in 2hrs to booting on the next attempt in as low as 20 mins, and by boot I mean, VBR logic detecting the restored VM as being up and running through the VLab logic, in some cases I can login just fine to the restored VM while VBR wont detect it as being up. I have worked with Veeam Engineers and gone through my lab setup, submitted logs, etc. however i have not been able to get it right and consistent.
Any help would be appreciated.


thanks

kayvonp@re-wa.org
Influencer
Posts: 14
Liked: 1 time
Joined: Aug 13, 2018 1:52 pm
Full Name: KTHP
Contact:

Re: Surebackup VM's not being consistent on booting

Post by kayvonp@re-wa.org » Sep 27, 2018 6:40 pm

I've never seen something quite that severe, but have seen inconsistencies. Most of the time, I just bump up the amount of time allotted for the machine to start and it will fix it. I have relatively slow hardware, so I always assumed that was the issue.

I do have one tip. When I made my first SB groups, I thought they all had to be listed in the Application Group. Eventually I learned you could load entire jobs of VMs. If you use the linked jobs, you can control how many boot at one time. On some jobs, I have it set to one so the VMs don't compete. On some, I have it set to two and I will put the slow VM near the top so it can load and the job can keep going through the list while it waits for the slow VM. Keeping this number low also means the VMs later in the list don't have to load with all of the others idling; might help with performance.

Hope that makes sense. I have found SB to be A LOT of trial and error and 90% of my problems are fixed by just adding more time to the boot time or application time out.

perseusco
Influencer
Posts: 16
Liked: never
Joined: May 08, 2018 8:17 pm
Full Name: Perseus Coronado
Contact:

Re: Surebackup VM's not being consistent on booting

Post by perseusco » Sep 27, 2018 9:20 pm

hi,
I have bumped the max boot time to 3 hrs. on all, I started with 1 hour but seeing how in some cases and specially my DC takes close to 2 hrs. to boot. I have only been working with Application groups and process one at a time, typically my DC will be the first one on all App groups, initially I thought it was an issue related to my DC only, but I have seen the long boot times/not booting at all affect my SQL VM's and File server VM's as well so I think its a general illness. I have also tried running my jobs outside of any backup, backup copy activity and I did not see any difference. When I review my Surebackup logs, I see the section where the VM detection logic is performed and ultimately what determines if its successfully detected as "Powered on" by detecting the IP. My networks are setup right on the Lab side.

I have seen though, that in some cases the VMTools on the restored VM do not start or take a long time to start, i know the VLab depends on this to detect the IP, etc. I dont kow why it would not start.
I am going to include the section of my Surebackup log where the detection takes place, perhaps someone can compare.

thanks

foggy
Veeam Software
Posts: 18789
Liked: 1650 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Surebackup VM's not being consistent on booting

Post by foggy » Sep 28, 2018 4:59 pm

I'd also look at the storage, SureBackup does random I/O and typically the bottleneck for its performance is the backup storage.

perseusco
Influencer
Posts: 16
Liked: never
Joined: May 08, 2018 8:17 pm
Full Name: Perseus Coronado
Contact:

Re: Surebackup VM's not being consistent on booting

Post by perseusco » Sep 28, 2018 8:09 pm

I looked at the surebackup logs more closely and the IP detection relies on VMTools, I noticed that as soon as the service starts running on the restored VM the IP gets picked up by the Surebackup logic, and in the cases where it does not get picked up and the boot process ends up failing is because the VMTools service is not running (why? I don't know), more intriguing is that in some cases and looking at this screenshot https://1drv.ms/u/s!AuonoLCSdVZmhQg3jHdqwfrHmaSM the vmtools service is detected running, then it seems that it stops, then it restarts, etc.
Has someone seen this behavior in their setups?
I was able to confirm this just yesterday while my VM was 1 hour and 54 minutes trying to boot, I logged in to the restored VM and started VMTools services, the Surebackup log shows "IP being detected at 1:54" at the same time stamp as the time when I started the service.

Post Reply

Who is online

Users browsing this forum: No registered users and 16 guests