Host-based backup of VMware vSphere VMs.
natalron
Novice
Posts: 4
Liked: never
Joined: Nov 29, 2013 6:12 am
Full Name: Ronnie
Contact:

hotadd disks visible in Disk Manager on Proxy

Post by natalron »

Hi All,

I have an issue in that I am using a Veeam backup proxy for hotadd backup.

But some of my target VMs are visible in the proxy servers disk manager as mounted drives after the hotadd.

This then triggers alerts in my monitoring systems for disk threshold breeches. etc.

Most VMs disks appear in the proxy server as "offline" disks, but some actually are mounted and assigned drive letters.

Trying to determine why this is the case ...

Thanks in Advance,

Nat
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: hotadd disks visible in Disk Manager on Proxy

Post by foggy »

Ronnie, having disks of the backed up VM attached (I believe, in read-only mode) to the proxy server is completely expected behavior. This is just how hotadd mode works. Are you saying that those disks remain attached to the proxy after the backup completes?
natalron
Novice
Posts: 4
Liked: never
Joined: Nov 29, 2013 6:12 am
Full Name: Ronnie
Contact:

Re: hotadd disks visible in Disk Manager on Proxy

Post by natalron »

Hi Alexander,

Thanks for the reply. What I mean is .... that always the disks of the backed up VM attach to the proxy server, which I understand is normal.

But mainly the drives are shown in the proxy server disk manager as "offline" disks ... but for some VMs they attach and mount with a drive letter .. then my monitoring software detects the drive added and sends a disk threshold alert.

Does that make sense?

Thank you.

Cheers,

Nat
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: hotadd disks visible in Disk Manager on Proxy

Post by veremin »

I believe that you'd better contact VMware team with this issue, since the Hot Add functionality is orchestrated solely by VMware, while, VB&R just appropriates it in order to backup VMs. Thanks.
natalron
Novice
Posts: 4
Liked: never
Joined: Nov 29, 2013 6:12 am
Full Name: Ronnie
Contact:

Re: hotadd disks visible in Disk Manager on Proxy

Post by natalron »

Thanks. I might try logging a case with VMware.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: hotadd disks visible in Disk Manager on Proxy

Post by Vitaliy S. »

Ronnie, can you please update this topic with the reason why some disks are online and some are offline during hotadd process? Meanwhile, I will try to reproduce it my labs. Thanks!
natalron
Novice
Posts: 4
Liked: never
Joined: Nov 29, 2013 6:12 am
Full Name: Ronnie
Contact:

Re: hotadd disks visible in Disk Manager on Proxy

Post by natalron »

Yes. I can add more details after I have spoken to VMware and determined why some disks are offline and some online during the backup process. Cheers.
arsprod
Enthusiast
Posts: 86
Liked: never
Joined: Jan 21, 2011 6:09 pm
Full Name: Aaron Spiegel
Contact:

[MERGED] veeam reconfiguring vm

Post by arsprod »

I've been having one vm show up now daily with multiple hard drives attached that shouldn't be. The vcenter db looks fine as does everything else so I suspected veeam might be involved. Sure enough, as soon as one of my replication jobs starts this vm (not the one replicating) reconfigures itself and adds hard drives. The vm that adds drives is a veeam proxy. Anyone else see this bizarre behavior?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: veeam reconfiguring vm

Post by foggy »

Aaron, this bizarre behavior is nothing but hotadd transport mode in action.
arsprod
Enthusiast
Posts: 86
Liked: never
Joined: Jan 21, 2011 6:09 pm
Full Name: Aaron Spiegel
Contact:

Re: veeam reconfiguring vm

Post by arsprod »

Oh... well don't I feel stupid! I'm assuming this is new to 7 since I've not seen it before? Do the disks stay there or supposed to be removed once the job is finished? Thanks
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: veeam reconfiguring vm

Post by Vitaliy S. »

HotAdd mode is not new to v7 and disks are not supposed to be left on the proxy server after job finishes. If you see otherwise, I would recommend investigating this issue with the support team.
arsprod
Enthusiast
Posts: 86
Liked: never
Joined: Jan 21, 2011 6:09 pm
Full Name: Aaron Spiegel
Contact:

Re: veeam reconfiguring vm

Post by arsprod »

Thanks!
Guido
Enthusiast
Posts: 27
Liked: never
Joined: Jul 17, 2012 10:56 am
Full Name: Guido
Contact:

[MERGED] Disks remain connected to backup proxy

Post by Guido »

There are 11 disks connected to my proxy backup server. No backups are running at this time. Some disks belong to other production vm's and some from replica's.
How can this happen and how to resolve?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: veeam reconfiguring vm

Post by foggy »

Guido, most likely you're backing up in hotadd mode, so disks attachment to the proxy server is totally expected (see above for details). However what is not expected is that the attached disks are not removed after the backup, so I suggest contacting support to investigate that. Thanks.
Peejay62
Expert
Posts: 235
Liked: 37 times
Joined: Aug 06, 2013 10:40 am
Full Name: Peter Jansen
Contact:

[MERGED] Hotadd disks not released.

Post by Peejay62 »

Every now and then I notice that disks processed with hotadd are not released from a proxy. Is this something to live with or should this always work, releasing the disks after the backup completes?
Just as a workaround we made a script to check the proxys if they have "unwanted disks" mounted and release them if appropriate. I have been digging in some logfiles to see if there are any messages on not releasing disk, cannot find anything though.
SO any experience or suggestions on this?

thanks, Peter
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: hotadd disks visible in Disk Manager on Proxy

Post by foggy »

Peter, this is definitely not expected, so please contact support for investigation. Thanks.
Peejay62
Expert
Posts: 235
Liked: 37 times
Joined: Aug 06, 2013 10:40 am
Full Name: Peter Jansen
Contact:

Re: hotadd disks visible in Disk Manager on Proxy

Post by Peejay62 »

Ok, thanks, will do that.

btw, shouldn't I receive a message when a subject is merged or a reply is given? my post was merged into this one but took me some time to figure that out (did I post it???, put me in doubt..), didn't receive a notification or somethng..
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: hotadd disks visible in Disk Manager on Proxy

Post by foggy »

You should receive a notification on the new post in the thread in case you're subscribed to it.
iwik
Influencer
Posts: 20
Liked: 4 times
Joined: Apr 02, 2014 11:10 am
Contact:

Re: hotadd disks visible in Disk Manager on Proxy

Post by iwik »

natalron wrote:Hi All,

I have an issue in that I am using a Veeam backup proxy for hotadd backup.

But some of my target VMs are visible in the proxy servers disk manager as mounted drives after the hotadd.

This then triggers alerts in my monitoring systems for disk threshold breeches. etc.

Most VMs disks appear in the proxy server as "offline" disks, but some actually are mounted and assigned drive letters.

Trying to determine why this is the case ...

Thanks in Advance,

Nat
Hi,

today same thing happened to us. During backup our monitoring discovered new drives on veeam proxy server. This drives had assigned letters and according to monitoring, windows was able to detect occupied space on this disks.
I know, that is OK that disks are added and visible in disk manager, but I'm not sure about that driver assigment. Even worse, when I check the event viewer:

Code: Select all

Log Name:      System
Source:        Ntfs
Description:
The file system structure on the disk is corrupt and unusable. Please run the chkdsk utility on the volume F:.
Logs shows, this error is not new and it is repeating. Can veeam confirm this is "feature"?
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: hotadd disks visible in Disk Manager on Proxy

Post by Vitaliy S. »

Can you please tell us what configuration is set for automount and SAN POLICY settings on the proxy server?
iwik
Influencer
Posts: 20
Liked: 4 times
Joined: Apr 02, 2014 11:10 am
Contact:

Re: hotadd disks visible in Disk Manager on Proxy

Post by iwik »

Ok, got it:

Code: Select all

DISKPART> SAN
SAN Policy  : Offline Shared
DISKPART> automount
Automatic mounting of new volumes enabled.
DISKPART>
I guess I need to disable automount, right?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: hotadd disks visible in Disk Manager on Proxy

Post by foggy »

Yes, you can disable automount (unless the proxy is also a DC) to avoid this in future.
pidthepiper
Enthusiast
Posts: 80
Liked: 7 times
Joined: Aug 11, 2015 9:10 am
Full Name: Bilal AHmed
Contact:

[MERGED] Proxy is not releasing vmdks once jobs are complete

Post by pidthepiper »

Hi there,
#01822452 case ID

So I have come across this issue, the jobs complete successfully but when I look at the VM properties of the proxy, it still has multiple vmdks attached, which means the source vms are running on snapshots.

I noticed it the other day, and noticed it today. I am having to manually remove the vmdks from the proxy myself.

Has anyone had this issue before?

Veeam v8 Update 3 and ESXi 5.5u3b EP10 (latest)
widmerkarl
Expert
Posts: 122
Liked: 29 times
Joined: Jan 06, 2015 10:03 am
Full Name: Karl Widmer
Location: Switzerland
Contact:

Re: Proxy is not releasing vmdks once jobs are complete

Post by widmerkarl »

Hi,

As stated in KB 1775 (https://www.veeam.com/kb1775) the cause can be the VMware API call to remove the VMDK which was not received or did not complete properly.

Did support respond in the meantime?

How about updating Veeam to latest version 9 update 1?
Best regards,
Karl

-----------------------
vExpert 2017-2024
VMware VCP-DCV 2023 / VCA6-DCV / VCA5-DCV / VCA5-Cloud
Former Veeam Vanguard / VMCE v9 / VMTSP v9 / VMSP v9
Personal blog: https://www.driftar.ch
Twitter: @widmerkarl
pidthepiper
Enthusiast
Posts: 80
Liked: 7 times
Joined: Aug 11, 2015 9:10 am
Full Name: Bilal AHmed
Contact:

Re: Proxy is not releasing vmdks once jobs are complete

Post by pidthepiper »

Yep Support have given me that KB and https://www.veeam.com/kb1882

My question is why would I disable automount?

v9 is on the cards, but there is no real reason for me to upgrade just yet, as v8 is fully supported
widmerkarl
Expert
Posts: 122
Liked: 29 times
Joined: Jan 06, 2015 10:03 am
Full Name: Karl Widmer
Location: Switzerland
Contact:

Re: Proxy is not releasing vmdks once jobs are complete

Post by widmerkarl »

Have a look at this Microsoft article regarding AutoMount:

https://technet.microsoft.com/en-us/lib ... s.11).aspx
When enabled (the default), Windows automatically mounts the file system for a new basic or dynamic volume when it is added to the system, and then assigns a drive letter to the volume.
The proxy with hot-add does not need to mount the filesystem as far as i know. The proxy just adds the VMDK to itself to read the block based data out of it, for backing it up to the repository.
Best regards,
Karl

-----------------------
vExpert 2017-2024
VMware VCP-DCV 2023 / VCA6-DCV / VCA5-DCV / VCA5-Cloud
Former Veeam Vanguard / VMCE v9 / VMTSP v9 / VMSP v9
Personal blog: https://www.driftar.ch
Twitter: @widmerkarl
pidthepiper
Enthusiast
Posts: 80
Liked: 7 times
Joined: Aug 11, 2015 9:10 am
Full Name: Bilal AHmed
Contact:

Re: Proxy is not releasing vmdks once jobs are complete

Post by pidthepiper »

The proxy is a 2012 r2 vm

I will disable automout and see what happens.

You are my new hero kawiMTF! thank you for your quick response
widmerkarl
Expert
Posts: 122
Liked: 29 times
Joined: Jan 06, 2015 10:03 am
Full Name: Karl Widmer
Location: Switzerland
Contact:

Re: Proxy is not releasing vmdks once jobs are complete

Post by widmerkarl »

Never mind. I help where I can :)
Best regards,
Karl

-----------------------
vExpert 2017-2024
VMware VCP-DCV 2023 / VCA6-DCV / VCA5-DCV / VCA5-Cloud
Former Veeam Vanguard / VMCE v9 / VMTSP v9 / VMSP v9
Personal blog: https://www.driftar.ch
Twitter: @widmerkarl
pidthepiper
Enthusiast
Posts: 80
Liked: 7 times
Joined: Aug 11, 2015 9:10 am
Full Name: Bilal AHmed
Contact:

Re: Proxy is not releasing vmdks once jobs are complete

Post by pidthepiper »

So I disabled auto mount and the issue still persists

I have notified support of this, but I am wondering is there anything else I can do?
pidthepiper
Enthusiast
Posts: 80
Liked: 7 times
Joined: Aug 11, 2015 9:10 am
Full Name: Bilal AHmed
Contact:

Re: [MERGED] Hotadd disks not released.

Post by pidthepiper »

Peejay62 wrote:Every now and then I notice that disks processed with hotadd are not released from a proxy. Is this something to live with or should this always work, releasing the disks after the backup completes?
Just as a workaround we made a script to check the proxys if they have "unwanted disks" mounted and release them if appropriate. I have been digging in some logfiles to see if there are any messages on not releasing disk, cannot find anything though.
SO any experience or suggestions on this?

thanks, Peter
Could you share this script. My Main Veeam VM doesn't suffer from this issue, but my proxies do, every morning they had disks still attached and I have to un attach them from the proxy and then run disk consolidation on the actual VMs that were backed up. This shouldn't be happening but it is, but only on the proxies!

I have created a new proxy but I get the same issue, I find it odd that only the proxies suffer from this and not the main Veeam VM. I am running v8 update 3, and I am going to upgrade to v9 now to see if that helps at all. I set the proxies to nbd mode which solved the issue, but now there is an issue with nbd mode: vmware-vsphere-f24/repeted-vddk-error-1 ... ml#p197816 so I am back to trying to fix this issue
Post Reply

Who is online

Users browsing this forum: bct44 and 59 guests