Jobs deteled and Veeam Server booting as another VM

[rgs3]

I had my Veeam VA server reboot over the weekend (unexpectedly) and this morning I find 2 things very interesting:
1) Most of my jobs are gone, all I see is a single backup job that was created a while ago. All my other backup and replication jobs simply disappeared. This has happed several times before, and extremely frustrating.
2) The Veeam VA server, must have rebooted during a backup job of one of our file servers. This morning when I came in the office the Veeam server was booted AS THE FILE SERVER! It was complaining about duplicate name exists on the network, and there were 2 instances of my file server running, the actual file server VM and the Veeam backup VM with the drives still attached. For some reason the backup servers actual C drive was "Hard Disk 2" and the file servers C drive was "Hard Disk 1". Is this common behavior? I can only imagine how bad this can really screw up my production VM's. Luckily everything seemed to be back to normal after I shutdown the Veeam backup server and manually removed the hard disks which were attached from the file server.

Thanks,
Richard

[Gostev]

1) Most of my jobs are gone, all I see is a single backup job that was created a while ago. All my other backup and replication jobs simply disappeared. This has happed several times before, and extremely frustrating.

This could only happen when Veeam Backup VM disks are configured in independent, non-persistent mode. Please check the VM disk settings for Veeam Backup VM in the VMware Infrastructure Client.

[rgs3]

Thats weird, your right Gostev, it was in independent non persistent mode, but I've never set that. Can that happen by itself?

[Gostev]

I heard once from another customer that the same thing happened to him after he tried to use Veeam Backup VM to backup itself in the Virtual Appliance mode (which is not supported as per the release notes). Basically this results in Veeam Backup VM having duplicate disks, and this probably makes ESX go crazy.

[rgs3]

ok, that makes sense. It is very possible that the Veeam Backup VM was inadvertently set backup itself.
Thanks

[rgs3]

Ok, this happened again. My backup VM was booted as my file server, complaining about duplicate names on the network. I immediately shut the VM down, removed the attached hard disks from our file server, and booted back into the Veeam backup VM. I looked at the logs of the backup VM and it looks like the windows update rebooted the VM (i've changed the settings to prevent this)

This seems scarily dangerous! Is this normal? How do I prevent this? And most importantly, is there risk for my backup VM doing this in the future and having 2 VM's online at the same time!???

Regards,
Richard

[Gostev]

Richard, if the job is terminated abnormally (for example, with forced reboot), this obviously does not give Veeam Backup a chance to dismount and remove hot added disks of the backed up VM from Veeam Backup VM configuration. Then when the VM is rebooted, it still has disks from other VMs added to it.

For some reason though, these disks take priority boot order in you case, and VM is booted from those disks. I suspect this is specific to your VM configuration. I would try to create new VM from scratch using default VM creation wizard settings. This should make VM disk be the first in the configuration, and Veeam Backup VM will always boot from this disk even if there are other disks still added. Frankly, while the virtual appliance mode is arguably most commonly used backup mode by our customers, the issue you are describing is pretty unique (so again, must be connected with unique VM disk configuration).

Thanks!

[rgs3]

I think I have found the issue. For some reason the Veeam backup VM hard disk is connected to SCSI Controller 0:1 and not 0:0, so when other disks are added to the VM during a backup, they take the lowest scsi number 0:0, during a reboot anything connected to 0:0 i suspect will be the booted drive.
Cant verify or change the SCSI settings while its powered on and doing a backup, but as soon as my backup finishes I will verify this.

Thanks for your help Gostev.

[Gostev]

OK, all makes perfect sense now. Thanks for the update!

[Kevin_TAN]

I've had Veeam change it's own harddisk to nonpersistent mode on 3 separate installations now. This time it also changed the SCSI Virtual Device Node to (0:1). I think it is caused when you backup your Veeam VM with "virtual appliance" mode.

[Gostev]

Hello Kevin, correct - this is caused by configuring Veeam Backup VM to backup itself (which is not supported, as documented in the Release Notes).

[Kevin_TAN]

How do you recommend that we backup the Veeam Backup server?

[Gostev]

Kevin, there are a number of existing topics discussing the same question which you can refer to for more information on this. Thanks!

Here is the most recent one from last week:
http://www.veeam.com/forums/viewtopic.p ... eam+server

[Bunce]

Wouldn't it be useful for the VBR server to check if its installed in a VM and cancel out before this happens?

Its quite easy to do - the VBR VM gets moved into a folder (that is part of a backup job) by accident etc.

We had to rebuild our VBR server after this happened as it hosed the VM..

[Gostev]

This feature is scheduled for v5, however I am hearing that it is hard to reliably identify that Veeam Backup is installed in a VM in 100% of cases. I am still insisting on implementing this though, even if it works in 90% of cases only