Problem with hotadd in appliance mode

[ecaspers]

I have an issue with backing up using a proxy in applinace mode.
Its an older verzion of Veeam B & R, (6.5.0 144), upgrading is not an option at this moment.

infrasturcture:
VMWare cluster with backup servers
VMWare cluster with prod servers
proxy vm in cluster with prod servers
Storage is netapp with nfs datatsores


I have reviewed http://www.veeam.com/kb1054

If the Veeam Proxy being used to process the source VM is on a VMFS 3 datastore it must be formatted with proper block size to be able to mount the largest virtual disk of the hot added VMs.
o 1MB block size – 256GB maximum file size
o 2MB block size – 512GB maximum file size
o 4MB block size – 1024GB maximum file size
o 8MB block size – 2048GB maximum file sizeThe Proxy being used to back up the VM must be a Virtual Machine on a host that is able to access the datastore where the source VM’s disks are stored.
largest disk is 200gb

In versions 5.1 and older of vSphere, the maximum supported VMDK size is 1.98 TB.
Not an issue

The disks that are to be hotadd must be SCSI. IDE drives are not compatible with hotadd.
all disks are scsi

VMware Tools must be installed and up-to-date on the Veeam server and any Veeam proxies
checked

Datastore needs sufficient space for a VM snapshot
snapshots can be created

Known issues are not effective.

Troubleshooting:
Test to see if the disk of the VM to be backed up can be manually hotadded to the proxy. See: http://www.veeam.com/kb1184
Checked, this works
Check all Veeam Proxies for old hotadded disks and release them manually. See: http://www.veeam.com/kb1775
Checked, no old disks
Disable automount on the backup proxy, as documented in http://www.veeam.com/kb1882.
Done

From the backup log i get the following information :

Code: Select all

[25.08.2015 16:30:02] <07> Info           [AP] (d453) output: 2015-08-25T16:30:01.942+02:00 [01440 info 'Default'] Establishing NFC connection to host xxx.domain.local on port 902, service vpxa-nfc\n
[25.08.2015 16:30:02] <07> Info           [AP] (d453) output: 2015-08-25T16:30:02.504+02:00 [01440 info 'transport'] Established NFC session for metadata and disk signature access.\n
[25.08.2015 16:30:04] <07> Info           [AP] (d453) output: 2015-08-25T16:30:04.032+02:00 [01440 info 'Default'] Establishing NFC connection to host xxx.domain.local on port 902, service vpxa-nfc\n
[25.08.2015 16:30:04] <11> Info           [AP] (d453) output: 2015-08-25T16:30:04.703+02:00 [01440 verbose 'transport'] Mounting VM vim.VirtualMachine:vm-183058 using transport hotadd failed : Could not open disk for disk signature retrieval..\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:05.951+02:00 [01440 verbose 'transport'] Cannot use mode hotadd to access [datastore_001] bck0000tst01/bck0000tst01.vmdk: Cannot mount using this method.\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:05.967+02:00 [01440 info 'transport'] Creating mount directory C:\Windows\TEMP\VeeamBackup\VeeamAgent-336\vmware-SYSTEM\42335bc5-97b0-6ecb-799f-0bdf4209027f-vm-183058\nbd.\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:05.998+02:00 [01440 info 'transport'] Starting to write unmount info.\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:06.029+02:00 [01440 verbose 'transport'] Found a device: vim.vm.device.VirtualIDEController\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:06.029+02:00 [01440 verbose 'transport'] Found a device: vim.vm.device.VirtualIDEController\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:06.045+02:00 [01440 verbose 'transport'] Found a device: vim.vm.device.VirtualPS2Controller\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:06.061+02:00 [01440 verbose 'transport'] Found a device: vim.vm.device.VirtualPCIController\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:06.061+02:00 [01440 verbose 'transport'] Found a device: vim.vm.device.VirtualSIOController\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:06.061+02:00 [01440 verbose 'transport'] Found a device: vim.vm.device.VirtualKeyboard\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:06.076+02:00 [01440 verbose 'transport'] Found a device: vim.vm.device.VirtualPointingDevice\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:06.092+02:00 [01440 verbose 'transport'] Found a device: vim.vm.device.VirtualVideoCard\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:06.107+02:00 [01440 verbose 'transport'] Found a device: vim.vm.device.VirtualVMCIDevice\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:06.107+02:00 [01440 verbose 'transport'] Found a device: vim.vm.device.VirtualLsiLogicSASController\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:06.123+02:00 [01440 verbose 'transport'] Found a device: vim.vm.device.VirtualFloppy\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:06.123+02:00 [01440 verbose 'transport'] Found a device: vim.vm.device.VirtualCdrom\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:06.139+02:00 [01440 verbose 'transport'] Found a device: vim.vm.device.VirtualDisk\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:06.154+02:00 [01440 verbose 'transport'] Found a device: vim.vm.device.VirtualVmxnet3\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:06.154+02:00 [01440 verbose 'transport'] Found a device: vim.vm.device.VirtualVmxnet3\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:06.185+02:00 [01440 info 'transport'] Mount operation obtained backup info.\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:06.185+02:00 [01440 info 'transport'] Successfully mounted snapshot ssid:snapshot-183059 on mount point C:\Windows\TEMP\VeeamBackup\VeeamAgent-336\vmware-SYSTEM\42335bc5-97b0-6ecb-799f-0bdf4209027f-vm-183058\nbd\n
[25.08.2015 16:30:06] <11> Info           [AP] (d453) output: 2015-08-25T16:30:06.232+02:00 [01440 verbose 'transport'] Got credentials to access disk [datastore_001] bck0000tst01/bck0000tst01.vmdk using mode nbd.\n
[25.08.2015 16:30:08] <11> Info           [AP] (d453) output: --wn:Hot add is not supported for this disk, failing over to network mode...\n
[25.08.2015 16:30:08] <11> Info           [AP] (d453) warning: Hot add is not supported for this disk, failing over to network mode...
[25.08.2015 16:30:08] <11> Info     Hot add is not supported for this disk, failing over to network mode...
[25.08.2015 16:30:08] <11> Info     Hot add is not supported for this disk, failing over to network mode...\n

The problem seems to be :
Mounting VM vim.VirtualMachine:vm-183058 using transport hotadd failed : Could not open disk for disk signature retrieval..\n
I haven't been able to find any information on this problem.

any suggestions ?

[Shestakov]

Hello Eelco, please provide support case number as required when creating topics about technical errors.
Thanks!

[ecaspers]

Hello Shestakov

The customer hasn't got an active support contract as they are evaluation the current backup vs possible other products.
So I'm not able to provide a support case number.
I was trying to show them the benefits of veeam and hot add wich they haven't been using...
I was hoping for some leniency

[Shestakov]

Even customers on evaluation period can get an assistance from the technical support team.

I`m a bit confused about the product version you mentioned (6.5.0 144), while you said the customer is evaluating the product.
You can get free trial with all functionality included and if you have troubles, open a support case. There were 2 big releases since v6.5, so showing its benefits while we are all about releasing v9 might be not the best strategy.

Thanks!

[PTide]

Hi,

Do you have the same problem with all VMs or with just some of them?

Thank you.

[ecaspers]

shestakov,

The customer had an active support subscription, but that expired last year. they have been using the product for some time now, however they are evaluating if they should continue to use it. or if they switch to another product. So yeah its a really old version.

Ptide,
All vm's have this problem.

[Shestakov]

The customer had an active support subscription, but that expired last year. they have been using the product for some time now, however they are evaluating if they should continue to use it. or if they switch to another product. So yeah its a really old version.

Thanks for the clarification! In this case I would definitely suggest to evaluate v8 trial, instead of comparing other products with our old version.
There is a huge number of new features since v6.5(v7 what`s new, v8 what`s new), please take a look.
Thanks!

[ecaspers]

shestakov,

I am aware of the new features, however I would like to show them a working proxy in appliance mode with hot add feature without having to build a complete new backup infrasturture.

[foggy]

Make sure the correct proxy server is being selected for processing. Little we can do without logs review here, so firing up a new Veeam B&R v8 console for demonstration might be a better/faster way.

[flaris]

Hello,

I'm facing the same issue. Please, let me know if you got any workaround to solve it.

Thanks in advance,
Flaris.

[veremin]

Have you already tried to reach our support team for closer logs investigation? Thanks.