SureBackup selecting VM

[Fiskepudding]

I have question about sure backup and selection of VMs.

Lets say we have configured a backupjob on "host level"/Selecting the entire HOST.
Lets call the Job backup job (BackupHOST2)
this job/host contains these VMs and is included in the backup:
HOST2:
-DomainController2
-LicenseServer1
-AntiVirServer1

I have 1 other Job (BackupDomainController1) that pics up just 1 VM from another host and thats DomainController1



As I understand, a logical way to set up a verification job would be to create a ApplicationGroup containg both domian controllers.

Then link the job BackupHOST2 to the job.

How does Veeam handle this?
I already put DomainController2 in the application group.So it boots that up before the linked jobs (BackupHOST2).
Eventually it will start booting VMs in the linked backup job (BackupHost2) witch also contains DomianController2.

Is Veeam/SureBackup smart enought to understand that this VMs is already running (in the application group) and will simply skipp that VM when running the linked job?
Because I find no way to exclude VMs that are linked (where the job is an entire host)

[Cokovic]

Hi Espen,

just add them to the application group too and select the roles you need. That way you will prevent any conflict.

[Fiskepudding]

Why wont there be a conflict?
DomainController2 is part of the application group (and is started first)
DomainController2 is ALSO part of the BackupHOST2 job and wil probabaly start again?
I dont se how adding roles to these will make any differnece. Both will have DC role, but will they start twice is actually my question.

Thanx

[Cokovic]

What i ment was to add both DCs, Antivirus server and License server all together in one application group and select there the roles you want to test instead linking the whole job to it.

And i'm not saying there won't be a conflict when trying to start the same VM twice in one SureBackup job. Never tried it to do that. But that way you won't start it twice.

[Fiskepudding]

As far as I can se there is no way to accoplish what you are saying.
I can not run EVERY single server in the application group, there is not enogh resources for that. Thats not what application grups are made for either.
The good thing about linked jobs is that it power down the VM after testing, freeing up resources for the next VM...
(Obvisually the above is an example, I have about 20 VMs that I want to test, and having all of them in the application group would be, well, madness )

I will try explain again.
My backup job of an entire HOST contains DomainController2.

This VM (DomainController2) is set up in the application group (because it is needed for all other server that are going to be linked).
(In the application group it is possible to select 1 or more VMs from a backup job. I find no such options in when I link a backup job. Thats part of the problem)

However DomianController2 is part of a Job that is named BackupHOST2 (witch is an ESX host containing many VM's including DomianController2)
And I want all VMs in this job tested with SureBackup, so I link this job....

So that means DomainController2 is part of the application gorup,
and when SureBackup loops threw the linked backup jobs (incluing BackupHost2) it wil eventually start DomianController2.
(Unless SureBacakup is aware that DomainController2 is allready running in the application gruop of the surebackup job, and decides to skip to start DomianControler2 from the linked backup job)

As I stated before, in the linked job I find no way to exclude a VM that is part of a backup job.

Maybe my explination was/is bad..but your reply did not answer my question.

[Fiskepudding]

A workaround for this would to simply make 1 backup job for each VM on "HOST2" Then i can link all other bakcup jobs then "DomainCtontroller2" to the sure backup job.
However, when I asked aobut VM selections for backup jobs I got the udnerstanding that Selecting entire HOSTS was common practice:
Considerations, selecting entire ESX host in backup jobs?

[Cokovic]

Ah ok. I referred to your example above. If you want to test about 20 VMs then you are right this is not the way to go.

Why not just seperate the DCs in their own job and exclude DC2 from the entire host job?

Right now i'm testing what will happen if the same VM is in the application group and a linked job.

[foggy]

Espen, nothing to worry about, actually. SureBackup is smart enough and if a VM is contained both in application group and linked backup job, it will be fired up just once (from the application group).

[Fiskepudding]

Rigth on

Thanx for clearing that up.

[Cokovic]

Tested it and it works like foggy stated. The only issue i had was that the lab was still running after all VMs were processed (i didn't checked the box to keep the application group running).

Maybe you could give it a try Espen and report back your observations on this.

[foggy]

The only issue i had was that the lab was still running after all VMs were processed (i didn't checked the box to keep the application group running).

Yes, this is a known issue that will be addressed in v6.1.

[ThomasMc]

Yes, this is a known issue that will be addressed in v6.1.

Woohoo

[Fiskepudding]

Yea I have the same behaviour. The lab stays running (even tho "keep the app. group running" is not checked)

I also have an issue where some VMs fails tests when they are linked, but passes the same test IF they simply put in the application group....(the order of boot i the same). But thats a different discussion. I opened a ticet yesterday 5186999.

[Cokovic]

Hi Espen,

thats interesting. Last year i've tested alot with SureBackup in v5. And there i had the same issue with SureBackup like you now. The linked job always failed on different VMs. Always one or two VMs from ten in the linked job hang and didn't start correctly. I've opened a case too but we never found a solution.

The only thing that worked for me was to reduce the amount of VMs that get processed concurrently to 1. By default it's set to 3. You can find this setting in the same window where you link the job. Maybe you will give it a try.

[Fiskepudding]

hmm..

I assumed that myself.
I started out with giving 25% RAM to the all VMS in the job and running 3 at a time as default.
Because there were issues, I upped the RAM back to 100% still errors.
Finally set the count to current VMss to 1. Did not help ehither.
We have problems with about 30% of the VMs.
Always the same that fails when linked.

If I look at the VM booting on the console in vCenter, it never gets to the logon screen, it hangs on "Applying computer settings" for 10 minutes.(default 600sec).
HOWEVER it passes the Heartbeat test.
I have tried with longer timout. But 10 minutes should be more then enough.
When i run it in the application group it boots in well under 2 min.
(Same lab, same HW, running from the same backupjob, just not linked)

If I understand you correctly it helped to set the numbers of concurrent vms to 1?...
It does not for me, so maybe our problem is not related. We use Veeam 6.0.0.181

Thanx for the info tho.

[Fiskepudding]

Ah ok. I referred to your example above. If you want to test about 20 VMs then you are right this is not the way to go.

Why not just seperate the DCs in their own job and exclude DC2 from the entire host job?

Right now i'm testing what will happen if the same VM is in the application group and a linked job.

Forgot to anwswer your question Cokovic.

I thought of this, but actually wanted to know how this works. And my tests also shows that the Veeam is "smart" and wont satart the VM twice.
So I did not have to change the job.

I think it's also more comprehensible NOT to exlude VMs when managing the jobs.

[Cokovic]

If I look at the VM booting on the console in vCenter, it never gets to the logon screen, it hangs on "Applying computer settings" for 10 minutes.(default 600sec).
HOWEVER it passes the Heartbeat test.

In my case the VM never came to the logon screen. My screen was just......well......black

If I understand you correctly it helped to set the numbers of concurrent vms to 1?...
It does not for me, so maybe our problem is not related. We use Veeam 6.0.0.181

Yes thats right. It helped to set this to 1. But then the VMs get processed sequential which took a long time to proof 10-20 VMs. And i've tested this only under Veeam v5. Not v6. But i will give it a try these days and report back what happened.

I think it's also more comprehensible NOT to exlude VMs when managing the jobs.

How about managing your jobs with VMWare folders? No need to exclude VMs and every new VM in the folder would be backed up automatically.

[Fiskepudding]

Folders could be a solution. But as it turns out, it won’t process the VM twice, so the selection can stay as is

I can easy live with processing 1 VM at a time, it only takes a total of 5 minutes to process a normal VM on my system.
And if needed it the SureBackup job can run into production time without any issues.

I just want it to work

It’s impossible to run ALL my VMS in one application group and test them all (there is not enough RAM and CPU)...
My second option is to have MANY application groups with corresponding jobs, which in total will test all my backups in turn.
However this will require to boot DCs many times. I am also not sure how I would schedule this, making one job start after the other is finished.
Bring in "linked jobs" hehe.


I will post back here as soon as I get a response from support.
Somehow it is good to know I am not alone

[Fiskepudding]

I just got a reply from support...

"The VM has to be added either via application group or via linked backup jobs, it's a common limitation."

Thats it, nothing else. Not very helpful...

We just confirmed in this post that it CAN indeed be placed in the app group and as a linked job, even in the same surebackup job.
It just wont process that VM swice (it is not even listed twice in the job monitor). In other words it should boot normally..

Support must have misunderstood my question.

Hoping for an update soon.

[Fiskepudding]

Veeam support nailed this one.

It was firewall related. All "File and printer sharing" inbound rules needed to be enabled.
In some cases it was not enough to just disable the firewall alltogether.

Works fine now.

[averylarry]

Bump -- my Surebackup jobs don't shutdown the application group. I thought that was supposed to be fixed in 6.1 . . ?

[foggy]

App. group is supposed to be shutdown normally in case where it does not contain the same VMs set as the linked backup job (which is not considered to be a reasonable scenario).

[averylarry]

Clarifications to be affirmed or corrected:

1) Application group only OR linked backup job only will shutdown correctly.
2) Application group AND linked backup job will shutdown ONLY if NO VMs are referenced more than once (no VMs in both app. group and backup job).
3) "not ... reasonable" means a lot of places will have to change their backup structure (basically create a new backup job for the application group VMs so they are not part of a linked backup job) to make Surebackup shutdown correctly.

Is this expected to be changed?

Considering how unbearable it is to try and startup a Windows server when DNS isn't working (hence the need for a domain controller in an application group), is changing the backup structure the only way to allow for an application group and linked backup job to co-exist?

[Gostev]

Yep, does not make sense to me either,specifically because of 3 - in many cases, ALL of my VMs will be in the same backup job! Extremely common scenario? I'll let foggy comment and explain this statement further, since clearly he talked to QC or R&D before answering - and I am currently out of office.

[foggy]

Well, seems the phrasing was not clear enough, sorry for that. I meant the case where both the app.group and the linked backup job contain the identical set of VMs - this situation is not handled correctly at the moment (the app. group is not shut down after the SureBackup job completes). In all cases where the whole set of VMs is different (including those where some VMs belong both to the app. group and the linked backup job - such VMs will be started only once, from the app.group), the app.group (according to QC) should shut down correctly after processing all the VMs from the linked job. If it does not, further investigation with debug logs review is required.

[averylarry]

Well I'm confused. If I understand foggy -- the application group should shutdown as long as it has at least 1 VM to process from the linked backup job. The 'not reasonable' scenario referenced is if the linked backup job is entirely superfluous since ALL of its VMs are already in the application group.

That's not my scenario. And that's not what tech support told me.

Case #5202448. To clarify:
1) I have an application group with 2 VMs in it -- both domain controllers (for 2 separate forests/subnets).
2) I have 2 backup jobs linked. Both backup jobs contain multiple VMs. No VM is in both backup jobs. The application group VMs are included in one or the other linked backup job. (So each linked backup job contains VMs that are not in the application group.)
3) The application group does not shutdown. Subsequent backup jobs fail with the standard (paraphrased) error 'backup file is locked by surebackup job'.

[tsightler]

Effectively the rule for right now with 6.x is that any VMs in an application group must not also exist in the linked backup jobs. If you create a separate job for your two DC's and remove them from their current backup jobs, it should work. I believe they are looking to fix this in the next patch for 6.1.

For the interim, you can shutdown the SB job via PowerShell using a scheduled task in Windows if you don't want to change up your jobs.

[foggy]

As always, Tom is spot on. After talking with support guys it is clear now that the issue shows up not in the case of identical set of VMs in the AG and linked job only. This is being now reproduced by QC and will be logged to be fixed in the patch.