Discussions specific to the Microsoft Hyper-V hypervisor
mvalpreda
Enthusiast
Posts: 48
Liked: 1 time
Joined: May 06, 2015 10:57 pm
Full Name: Mark Valpreda
Contact:

v9 replication jobs getting stuck on 'Building VMs list'

Post by mvalpreda » Feb 08, 2016 3:57 pm

Case # 01690842

2x Windows 2012 R2 Hyper-V servers on the same subnet. Veeam running on the host with production VMs. Second Hyper-V server is old server and now being used as a replication target.

Upgraded from v8 to v9 and had an issue with replication jobs. Futzed around and wound up re-creating the replication job. Got one replication job to run fine and when the next replication job goes to run, it gets stuck at 'Building VMs list' forever. It usually takes <10 seconds to build the VM list and then run the job. If I kill all the Veeam services, run a backup job, then a replication job it will run. If I kill the services and run a replication job, it gets stuck on 'Building VMs list'.

This worked flawlessly in v8. Can I roll back to v8?

Gostev
SVP, Product Management
Posts: 24797
Liked: 3557 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by Gostev » Feb 08, 2016 4:07 pm

There is no rollback for the existing installations, but you can deploy the new instance of v8. But I would first confirm with support that this is indeed a v9-specific issue, and not some other change in the environment. Thanks!

mvalpreda
Enthusiast
Posts: 48
Liked: 1 time
Joined: May 06, 2015 10:57 pm
Full Name: Mark Valpreda
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by mvalpreda » Feb 09, 2016 2:53 pm

No other changes. In-place upgrade to v9.

foggy
Veeam Software
Posts: 18276
Liked: 1564 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by foggy » Feb 09, 2016 3:27 pm

Job logs should tell what it is doing while it is stuck, so please working with support on this.

ptoro
Influencer
Posts: 21
Liked: 5 times
Joined: Jul 24, 2015 4:41 pm
Contact:

[MERGED] Building VMs list = SLOW

Post by ptoro » Mar 19, 2016 7:50 am

I am on version 9 of Veeam and loving it (coming from v8)!

One thing I have noticed though is that the building VMs list has increased dramatically (maybe I just wasn't paying attention before, but nonetheless).

HyperV environment (clusters) going back to scale out file servers that serve up SMB3 shares (no SCVMM).

What is this "building VMs list" step actually doing? Any info would be greatly appreciated.

I have multiple jobs that just focus on the VMs that reside on that shares CSV (volume) and at the most I am backing up 17 VMs per job. That one job with 17VMs takes almost 2 hours just to build this VMs list!!!!!!!!! (insane). And then it starts to backup the VMs..... It's crazy town 8)

All my jobs are now using per-vm backup chains (I checked the repo and the individual files have been created). I'm not really concerned about the backup time when it is actually backing up the vms but of the time to build these lists.

I just created a support ticket but wanted to know if anyone else was experiencing these issues. From what i've read, doesn't appear to be an issue with VMware/vCenter. I'm hoping it's something dumb on my end and not a norm with hyperV in cluster not controlled by SCVMM.

Gostev
SVP, Product Management
Posts: 24797
Liked: 3557 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Building VMs list = SLOW

Post by Gostev » Mar 19, 2016 7:34 pm

ptoro wrote:one job with 17VMs takes almost 2 hours just to build this VMs list
This is not normal.
ptoro wrote:I just created a support ticket
What is the support case ID?

ptoro
Influencer
Posts: 21
Liked: 5 times
Joined: Jul 24, 2015 4:41 pm
Contact:

Re: Building VMs list = SLOW

Post by ptoro » Mar 19, 2016 8:23 pm

Veeam Support - Case # 01736028 - Building VMs List....

ptoro
Influencer
Posts: 21
Liked: 5 times
Joined: Jul 24, 2015 4:41 pm
Contact:

Re: Building VMs list = SLOW

Post by ptoro » Mar 26, 2016 3:23 pm 1 person likes this post

So still working with support on this. quick recap of what has been done:

- enabled creating the VM list from the actual hosts and not from veeam server (through reg key)
- disabled guest processing

no difference what so ever.

Just sent over another set of logs, will keep you guys posted. From what Veeam tech said, building the VM list should only be taking a couple of minutes. So there is something definitely going on...

FUBIT
Lurker
Posts: 1
Liked: never
Joined: Mar 29, 2016 5:05 pm
Full Name: FUB IT
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by FUBIT » Mar 29, 2016 5:10 pm

I am having the exacty same issue but I even have identical servers and the jobs get stuck at building VMs list. It works so well when it runs but is so unreliable now.

Vitaliy S.
Product Manager
Posts: 22997
Liked: 1556 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by Vitaliy S. » Mar 29, 2016 7:27 pm

I cannot find any resolution in the OP case as it just started to work, so please contact our technical team for troubleshooting directly.

mkaec
Expert
Posts: 322
Liked: 72 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by mkaec » Apr 08, 2016 3:11 pm

I have experienced a similar problem 3 times with backup copy jobs in v9. The job would get stuck at "Building VMs list" and sit there indefinitely. I would recover by rebooting the backup server. Support case 01705801.

If it occurs again after installing update 1, I will revisit with support.

mbrinkho
Enthusiast
Posts: 28
Liked: 2 times
Joined: Jan 03, 2014 5:14 pm
Full Name: M Brinks
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by mbrinkho » Apr 15, 2016 12:31 pm

Having the same issue here but with a standard backup. Veeam 8 was great, 9 has given us some major issues.

Aside from rebooting the server is there anything else I can do for this?

foggy
Veeam Software
Posts: 18276
Liked: 1564 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by foggy » Apr 15, 2016 12:37 pm

Do you man reboot doesn't help or rebooting the server is not an option for you?

mbrinkho
Enthusiast
Posts: 28
Liked: 2 times
Joined: Jan 03, 2014 5:14 pm
Full Name: M Brinks
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by mbrinkho » Apr 15, 2016 12:45 pm

Rebooting makes the job run for a few days and then it gets stuck at building VMs list again. If I try to cancel the job it never cancels.

foggy
Veeam Software
Posts: 18276
Liked: 1564 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by foggy » Apr 15, 2016 12:58 pm

Then I suggest letting our engineers review your setup and logs.

djesko
Novice
Posts: 4
Liked: never
Joined: Jan 14, 2015 9:28 am
Full Name: Niklas Engberg
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by djesko » Apr 15, 2016 6:19 pm

Same problem here , did you get a fix ?

EDIT : did see now that the respons was from today :)

Did you create a case ?

Br Niklas

foggy
Veeam Software
Posts: 18276
Liked: 1564 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by foggy » Apr 17, 2016 8:39 pm

Niklas, it is always recommended to open your own case, since the reasons of this behavior can vary from one environment to another.

MarkWatson
Lurker
Posts: 1
Liked: never
Joined: Apr 20, 2016 4:44 pm
Full Name: Mark Watson
Contact:

Re: Building VMs list = SLOW

Post by MarkWatson » Apr 20, 2016 5:07 pm

Did you get anywhere with this? I'm facing a similar but worse issue where the job doesn't get past the "Building VMs list", I'm working with Veeam suport on this, case # 01767317.

mbrinkho
Enthusiast
Posts: 28
Liked: 2 times
Joined: Jan 03, 2014 5:14 pm
Full Name: M Brinks
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by mbrinkho » Apr 20, 2016 6:02 pm

I'm still psyching myself up to put a call in to support. :(

jazzoberoi
Enthusiast
Posts: 79
Liked: 17 times
Joined: Oct 08, 2014 9:07 am
Full Name: Jazz Oberoi
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by jazzoberoi » Apr 21, 2016 4:51 am

Same thing is happening to us. Works beautifully for a few days and sometimes just sits at building the VM's list till the server is restarted.

Vitaliy S.
Product Manager
Posts: 22997
Liked: 1556 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by Vitaliy S. » Apr 21, 2016 10:30 am

Jazz, in order to troubleshoot this issue quicker, please open a support case.

mbrinkho
Enthusiast
Posts: 28
Liked: 2 times
Joined: Jan 03, 2014 5:14 pm
Full Name: M Brinks
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by mbrinkho » Apr 21, 2016 1:00 pm

Opened a case, 01771011.

peter_cosa
Lurker
Posts: 1
Liked: never
Joined: Apr 06, 2016 12:11 am
Full Name: Peter
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by peter_cosa » Apr 25, 2016 9:54 pm

any fix for this?

bcann
Novice
Posts: 6
Liked: never
Joined: Oct 30, 2015 3:28 am
Full Name: Brad Cann
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by bcann » Apr 26, 2016 5:13 am

I found that if you backed up via selecting the host rather then the individual VM's, AND you had application aware on, it would happen a lot more often. when I selected the individual VM's and only the application aware items on the specific servers it seemed to disappear.

mbrinkho
Enthusiast
Posts: 28
Liked: 2 times
Joined: Jan 03, 2014 5:14 pm
Full Name: M Brinks
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by mbrinkho » Apr 26, 2016 12:42 pm

Installing Update 1 fixed this for us.

kamiwa
Novice
Posts: 3
Liked: 3 times
Joined: Apr 30, 2016 3:10 pm
Full Name: Kai Michael Wadsack
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by kamiwa » Apr 30, 2016 3:14 pm

Installing Update 1 fixed this for us.
Installing Update 1 introduced this bug for me.

As general advice seems to open a support ticket with Veeam:
How do I open a support ticket?

Vitaliy S.
Product Manager
Posts: 22997
Liked: 1556 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by Vitaliy S. » May 01, 2016 6:53 pm

You can do that via our website > https://www.veeam.com/support.html

kamiwa
Novice
Posts: 3
Liked: 3 times
Joined: Apr 30, 2016 3:10 pm
Full Name: Kai Michael Wadsack
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by kamiwa » May 01, 2016 8:16 pm 2 people like this post

Thanks. Already found out by myself and support got back to me with a hotfix that I'm currently testing.

mkaec
Expert
Posts: 322
Liked: 72 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by mkaec » May 02, 2016 4:21 pm

I had this happen twice this weekend. This was the first time it happened with backup jobs. Previously, it had only happened with BCJs. It happened both times with the same job.

Vitaliy S.
Product Manager
Posts: 22997
Liked: 1556 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: v9 replication jobs getting stuck on 'Building VMs list'

Post by Vitaliy S. » May 02, 2016 4:32 pm

Marc, if you still experience this issue, then please reach out to your support team for the hotfix.

Post Reply

Who is online

Users browsing this forum: No registered users and 8 guests