Hot add is not supported for this disk - 6.0.0.153

[micwe]

Hello
I have upgraded my entire ESXI host to last build ESXI 5.0
And Install new Veeam backup on VM win2008 r2 with last veeam build - version 6.0.0.153 (clean install not upgrade)

After setting up new copy and replica job in veeam - all job finish with Warning - Hot add is not supported for this disk, failing over to network mode...
In my old veeam backup V5 and esxi4.0 host environment all job completed Successful

I search on this forum about this error but didn't come up with reference to veeam build - version 6.0.0.153
I came across this KB ID: 1442 that reference to my problem
http://www.veeam.com/kb_articles.html/KB1442
But after applying error still comes up "Hot add is not supported for this disk, failing over to network mode..."
How can I solve this issue and use the Virtual Appliance option

Thanks
Micwe

[foggy]

Micwe, there are a number of limitations (put by VMware vStorage API) that prevent hot add mode to be used in some cases. Please refer to this KB article
for complete list (this is covered in our release notes, btw). Hope this helps. If you cannot find any matching reasons, please open a support case for investigation. Thanks!

[lobo519]

Not to hijack this thread but:

I do have this issue one of my replication jobs and I believe it is due to this reason:

"With HotAdd transport, full backup of a virtual machine fails when multiple disks have the same name, as they could if they are on different datastores."

I have a second replication job that is configured the same way and also has the above naming problem but does not give the Hot add failing to network warning.

Is it using hot add or is it actually failing to network mode??

I found some documentation on VMware's site that shows how to rename a vmdk file - will that break the replication job??

[Gostev]

No failover warning means that it is using hot add.
Renaming disk will not "brake" the job, but that disk will likely need to be replicated over entirely.

[lobo519]

Any thoughts as to why sometimes it works?

[Gostev]

No idea - this is not our code...

[spalmer]

I had this exact same issue and it is now resolved - the root cause was the target proxy was on a datastore with a 1MB block size, but it was trying to hotadd a disk larger than 256 GB. By moving the target proxy (storage vmotion) to a datastore with an 8 MB block size the warnings dissappeared (note: hotadd would fail for all disks of the VM, even those smaller than 256GB just because there was at least 1 disk that was larger than the block size limitation). Hope this helps.

[Vitaliy S.]

Yes, this (block size mismatch) is one of the limitations for Virtual Appliance (aka Hot Add) mode, upgrading to vmfs 5 file system should solve this issue too since it has the same block size no matter of datastore size.

[Gostev]

Pretty sure that "upgrading" to VMFS5 will not change block sizes, instead you need to create new VMFS5 LUN and migrate your VMs there. There are a few VMFS5 capabilities that you will lose when upgrading existing datastores to VMFS5.

[jeancharles]

Hi,
If you want to see the differences between VMFS3 upgraded to 5, and new VMFS5:
http://blogs.vmware.com/vsphere/2011/07 ... mfs-5.html

[loren10]

I solved this issue with hotfix "veeam_backup_6.0.0.158.hf1_setup.exe"
Regards

[tomas.olsen]

Hello everyone, I have just upgraded a few esx hosts from vsphere 4.1 to vsphere 5.0.
I upgraded vmware tools and hardware of all my vm's as well. except the virtual center server which was a virtual machine.
At the same time, I also upgraded Veeam from 5.0.230 to 6.0.0.153.
everything worked like a charm. All my vm's backed up just fine.

Today I upgraded my virtual center server with vmware tools and new virtual hardware.
The first time my scheduled backup ran, all my vms failed with the same error as specified here.

I tried the patch, but it doesnt seem to work for me. My datastores is still vmfs 3.46.

[tomas.olsen]

Hello again,

Problem solved for now.
The trick was to migrate the Veeam virtual machine configuration files and the virtual machine system disk to a datastore that was formatted with VMFS 5. I ran the backup again, and all machines was successfull. In my environment the veeam management server is also the proxy.

I suspect that the same thing applies if you have deployed multiple proxies since the hot-add will be done to the proxy and not the management server. If you just use dedicated proxies for backup it might not be important that the management server is located on vmfs5 datastores but this is just my guess and have not been testet.

So in my experience this has to do with when the virtual center is upgraded with new virtual hardware and vmware tools, you need to make sure that the virtual veeam server configuration files and system disk need to be located on a vmfs5 datastore.

[Gostev]

Hi Tomas - this is a great finding, thank you very much for taking time to share this with the community!

[tomas.olsen]

You are welcome Gostev,

Let me know if you need further info in case you want to investigate this more thoroughly.

Happy new year to all of you !!

[mr.bigbear]

Hello everyone,

we run the version 6 on a german localized windows server 2008 r2, upgraded from version 5!
Patch 2 of beckup & replication 6 is installed...
Veeam Backup Server and Proxy migrated to a datastore with vmfs 5...

But the "hot add..." problem still alive!

In Version 5 all works fine, with Version 6 we are now in trouble with this issue!

Anybody such idea???

Thx & best regards,
Robert

[foggy]

Robert, logging of this event was not enabled in v5, the engine simply would not notify you about failover to network mode. There are a number of limitations (put by VMware vStorage API) that prevent hot add mode to be used in some cases (please refer to this KB article). If you can identify and remove the reason, then hotadd should start working. Otherwise, please open a support case for investigation. Thanks!

[mr.bigbear]

@foggy

Thx for this Information...
In Version 5 it worked very well! We noticed this at first on the amount of time for backup (at min. two times more as in version 5!!!!!) and in version 5 the configuration of the virtual backup server successfully added the disk for backup....

Thx,
Robert

[jp34]

Hello, same issue, but nothing matches incompatibility list and hotfix 1 has been installed.
I also run Veeam on a localized Windows Server (French).

Additional Details: v6.0.0.164 (issue 1,2,3 patched)
VMDK file is smaller than 1TB (139GB)
Veeam proxy resides on a proper block size datastore
VMDK is neither an independant disk nor a IDE disk
No other vmdk file has same name (even in other datastore)
VBR server has neither been cloned nor restored
Veeam proxy is installed in a VM running Windows Server 2008 French (know issue but fixed by patch) and access the source VM though a VCenter server (itself: veeam proxy and VCenter server in the same VM)
Backup server is installed on a physical server.

@Mr Bigbear:
Do you run Veeam Backup directly on the VCenter server ? Do you use Veeam Proxy ? are they installed in a VM ?
Please can you describe your architecture ?

[foggy]

Guys, the fastest way to resolution will be definitely to log a support case and let our technical guys dig into your log files.

[jp34]

Already done, but no resolution/explaination from support for 1 week... I hope they will find something...
I try to find similarities with other users to help support...

[mr.bigbear]

@jp34

I dont't run the Veeam-Backup server on vCenter... I have only a version 5 to 6 upgraded virtual solution running on my esx 5 cluster.
That means Veeam Backup-Server and Proxy are on the same VM.

And i have the issue on various vm's, independing of the vmdk size..... No backup works without the hot-add warning after backup!

Did this answer your question???

Robert

[murdocmk]

Recently I ran into this problem where my Veeam v6 VM backup proxy (the default out-of-the-box backup proxy) could not hot add any of the disks in backup jobs. Other backup proxies were working normally. I checked and re-checked the list of reasons why hot add might not be working but couldn't find any of those issues in my situation.

I created a test job containing just one of the VMs that I was having problems with (so I could run a quicker backup for testing). I tested that job and it gave the hot add warning. I migrated the Veeam VM (just the configuration and OS disk, not the virtual disk containing the backups) to another datastore. I ran a job after that and it succeeded. Then I migrated the Veeam VM back to the original datastore and ran the test job again, and it succeeded hot adding the disks again. Then I re-ran my production jobs and they also succeeded.

I don't really know what was resolved by migrating the VM but it seemed to clean up something in the configuration/VMDK files that resolved my problem. I just thought I'd pass that along in case others haven't tried it.

[jp34]

My issue has been fixed by the support:
- The value "UUIDHost" of the key [HKEY_LOCAL_MACHINE\SOFTWARE\VMware, Inc.\VMware Workstation\volatile] has been removed (VMWare Wordkstation incompatible with Veeam HotAdd ?)
- Account of Proxy services on Backup Server and Backup Proxy VM have been changed from LocalSystem to Administrator accounts

HotAdd is running fine now.
Thank to the Veeam Support.

[itdirector]

I went through everything in KB1054; but our setup does not have any of the described issues.
The only thing I can think of is if "Hot Add disk" does not work with a VM (in this case a VBR 6.x (latest patch) that has a passthrough SAS card (Perc 6E). Can anyone verify?

Thx

[pauguin]

[merged]

Hello,

My configuration is the following :

Vmware ESXI 4.1.0348481, ISCSI datastore VMFS 3.46
Veeam 6.0.0.181 on Windows 2008 R2 server SP1 Build 7601 ( all English)
Veeam is a Vm on a datastore with 4Mb block size and 1.19 Tb free

All VMs are backing up correctly except one machine.

This machine has 3 disks all on the same datastore with different names.
All 3 disks are mounted in the veeam machine after the snapshot.
The 1st disk is backing up OK through hot add, the 2 others give me the following error :
Hot add is not supported for this disk, failing over to network mode.

Can someone help, I have allready checked all hints I have found on the forum

[mmonroe]

Our Veeam box is 6.0.0.181. It will run along for days or even weeks and then the "Hot add" error will appear for every single VM being backed up. It will continue showing up until I reboot the Veeam server. Once rebooted, the error will go away for days or weeks and eventually the "Hot add" error will pop up again.

Mark

[Oletho]

Our Veeam box is 6.0.0.181. It will run along for days or even weeks and then the "Hot add" error will appear for every single VM being backed up. It will continue showing up until I reboot the Veeam server. Once rebooted, the error will go away for days or weeks and eventually the "Hot add" error will pop up again.

Thanks for the reboot hint.

I have a system that gave the hotadd error on random VMs, the same VM could succeed in the backup and fail in the replication shortly after or vice versa.

Upon rebooting the Veeam server all jobs tonight were without errors.

[Jfmoots]

I would submit logs to support and let them have a look if it happens again. You should not need to reboot to keep hotadd jobs running.

[Gostev]

You should not need to reboot to keep hotadd jobs running.

Unless we are talking about the default backup proxy (the one on the backup management server), and you do some file level restores there... then this makes it a known issue.