Discussions specific to the VMware vSphere hypervisor
Post Reply
stuartmacgreen
Expert
Posts: 109
Liked: 20 times
Joined: May 01, 2012 11:56 am
Full Name: Stuart Green
Contact:

Deleting/Adding VMDK's affects IR and FLR

Post by stuartmacgreen » Aug 29, 2012 8:24 am

Have a VM where it had 10 VMDK's.
Deleted 1 of the VMDK's and added a new 1 while the VM/system was online.
(The disk was number 6 disk in the VM that was removed.)

Thereafter SureBackup, Instant Recovery and FLR failed to work as expected.

FLR failed with a message:

Failed to power on FLR appliance Error: Unable to access file [VeeamBackup_vproxy.tld] VeeamFLR_BARNEY_6a8f1f0a-5a35-49e2-b8d7-a36caa10d07a/BARNEY.vmdk since it is locked
Unknown failure Error: Unable to access file [VeeamBackup_vproxy.tld] VeeamFLR_BARNEY_6a8f1f0a-5a35-49e2-b8d7-a36caa10d07a/BARNEY.vmdk since it is locked

SureBackup and Instant Recovery failed in respect that one of its disks was being seen as a duplicate of another one (this verified using fdisk and checking diskid), this means that we lost access to the production data on one of these disks.

All of the synthetic/incremental backups since disk addition/removal show this issue.

Veeam B&R 6.1
VM Guest OS: SLES 11 SP1 X64
VMDK's for this VM are covered by SDRS
vsphere 5.0

We however, have performed an Active Full which works as expected under Instant Recovery & SureBackup.

Support Case: 5213026

--------
Is this a bug or documented somewhere that you should run an active full backup after disk addition/removal.?

Thanks
Backup Infrastructure: Veeam B&R 9.5 U4 • vSphere 6.5 u2 • HPE MSA 2040 [FC] (Backup Target) • HPE MSL6480 LTO-6 (Tape Storage)
(Veeam Agent for Windows protecting Veeam Proxy / Tape / Repository servers)

foggy
Veeam Software
Posts: 18295
Liked: 1569 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Deleting/Adding VMDK's affects IR and FLR

Post by foggy » Aug 29, 2012 10:14 am

Neither bug nor documented, please continue working with support on this. Should be investigated closely.

stuartmacgreen
Expert
Posts: 109
Liked: 20 times
Joined: May 01, 2012 11:56 am
Full Name: Stuart Green
Contact:

Re: Deleting/Adding VMDK's affects IR and FLR

Post by stuartmacgreen » Sep 13, 2012 2:02 pm

The resolution from Veeam support and I assume must be a best practice by all means...
the VM configuration has been modified in the midle of an incremental chain, which is not recommended. If you want to modify a VM in the middle of the forward incremental chain please either create a full backup after the changes have been made, or recreate the job mapping the existing backup files to a newly created job.
Will be following this from now on.
Thanks
Backup Infrastructure: Veeam B&R 9.5 U4 • vSphere 6.5 u2 • HPE MSA 2040 [FC] (Backup Target) • HPE MSL6480 LTO-6 (Tape Storage)
(Veeam Agent for Windows protecting Veeam Proxy / Tape / Repository servers)

foggy
Veeam Software
Posts: 18295
Liked: 1569 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Deleting/Adding VMDK's affects IR and FLR

Post by foggy » Sep 13, 2012 2:07 pm

Thanks for getting back with this, Stuart!

chrisdearden
Expert
Posts: 1530
Liked: 225 times
Joined: Jul 21, 2010 9:47 am
Full Name: Chris Dearden
Contact:

Re: Deleting/Adding VMDK's affects IR and FLR

Post by chrisdearden » Sep 13, 2012 2:13 pm

Of course VM admins dont always tell the backup admins this. Its a good use case for a periodic active full backup even when you are using reverse incremental.

tsightler
VP, Product Management
Posts: 5425
Liked: 2246 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Deleting/Adding VMDK's affects IR and FLR

Post by tsightler » Sep 13, 2012 3:52 pm 1 person likes this post

This still seems like a bug to me. VMs change, disks are added and removed, that's the nature of a virtual environment. To expect customers to know that they need to run a full backup in this case isn't very realistic, what if there are dozens of VMs in the job? Interestingly, I have never run into this issue, and I've added and removed disks to many VMs over the years. I'm wondering if there is some unique circumstance that causes this issue to be exhibited.

Berniebgf
Service Provider
Posts: 94
Liked: 9 times
Joined: Sep 01, 2010 11:36 pm
Full Name: Bernard Tyers
Contact:

Re: Deleting/Adding VMDK's affects IR and FLR

Post by Berniebgf » Sep 14, 2012 4:04 am

+1, this seems a little flaw'd to me.

What is an acceptable change and what is going to break the chain?

Add memory / CPU = OK?
Add disk = OK?
Delete disk = Not OK?
Add SCSi controller????

Do we have a list of do's and don't for VM property changes in active chains?

Bernie

Gostev
SVP, Product Management
Posts: 24825
Liked: 3578 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Deleting/Adding VMDK's affects IR and FLR

Post by Gostev » Sep 14, 2012 9:59 am

I agree with Tom, I didn't believe we have any known dont's. If there is a bug, it needs to be fixed. Do you still have your support case open, are they still investigating this?

stuartmacgreen
Expert
Posts: 109
Liked: 20 times
Joined: May 01, 2012 11:56 am
Full Name: Stuart Green
Contact:

Re: Deleting/Adding VMDK's affects IR and FLR

Post by stuartmacgreen » Sep 14, 2012 10:39 am

I am resorting to the fact that I must perform a Active Full, when this kind of activity is performed.

Fortunately for me. Myself and 1 other are the both the backup and vm admin for our environment. So it wasnt discord from two different departments. Just a learning situation for future.

I havent had a closed case email response so I guess its still open. But going no further forward.

NOTE: I did try to replicate the same scenario with another VM and multiple disks, multiple SCSI controllers and same type of backup job, removing/adding disks etc But could not repeat the scenario. FLR and IR worked as expected without error.

Only thing to add to the mix is that our VM is in a SDRS cluster. But I dont expect that to be an issue.

I just love the fact we can find this problem with the backup chain with the features of Veeam. And can understand how to work with the product, around system admin tasks.

Stuart
Backup Infrastructure: Veeam B&R 9.5 U4 • vSphere 6.5 u2 • HPE MSA 2040 [FC] (Backup Target) • HPE MSL6480 LTO-6 (Tape Storage)
(Veeam Agent for Windows protecting Veeam Proxy / Tape / Repository servers)

Post Reply

Who is online

Users browsing this forum: tblime and 26 guests