Comprehensive data protection for all workloads
Vitaliy S.
Product Manager
Posts: 22289
Liked: 1413 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Hot add didn't work for one server

Post by Vitaliy S. » Jan 15, 2010 5:08 pm

Darhl,

Do you see anything in common between those disks? Are you able to reproduce the issue by removing all those disks, then running jobs again and checking whether you see those disks appearing once again or not, anyway please keep up working with our support team, I'm sure together we will be able to find the root cause for your issue.

Thank you!

withanh
Expert
Posts: 262
Liked: never
Joined: Jul 21, 2009 3:19 pm
Full Name: Darhl
Location: Pacific Northwest
Contact:

Re: Hot add didn't work for one server

Post by withanh » Jan 15, 2010 6:22 pm

The disks are not the same .VMDK name, they are numbered but not sequentially, it doesn't make any sense. I will forward the info to support.

I will also do as you suggest and remove the duplicate drives. One thing that is weird is HD1 is the boot drive, but was moved to SCSI 0:3. It's just weird. I'll probably move that back to 0:0 where it belongs.
For every expert there is an equal and opposite expert - Arthur C Clarke's Fourth Law

ceebee
Influencer
Posts: 17
Liked: never
Joined: Dec 24, 2009 2:39 am
Full Name: Chet
Contact:

Re: Hot add didn't work for one server

Post by ceebee » Jan 18, 2010 4:29 am

I have the same exact issue and i am being told to get in touch with VMWare as this is not a Veeam issue! Fantastic!
I am working with Ben M. on this issue but this email was from Stanislav.
This is getting bloody ridiculous!!! :oops: :oops: :oops: :oops: :oops: :oops: :oops: :oops: :oops: :oops: :oops: :oops: :oops: :oops: :oops: :oops: :oops: :oops: :oops: :oops:
Not only the product is buggy and does not work as touted but the the icing on the cake is the support. If someone wants to write a book on how to setup a complete cluster F***** of a support then they need to call Veeam.
I am about done with my patience.

Gostev
SVP, Product Management
Posts: 24017
Liked: 3254 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Hot add didn't work for one server

Post by Gostev » Jan 18, 2010 11:31 am

ceebee wrote:I have the same exact issue
Hello Chet, what issue are you talking about specifically? The original issue discussed in this thread were indeed caused by some known vStorage API limitations (I've posted a quote from VMware documentation on the previous page).

withanh
Expert
Posts: 262
Liked: never
Joined: Jul 21, 2009 3:19 pm
Full Name: Darhl
Location: Pacific Northwest
Contact:

Re: Hot add didn't work for one server

Post by withanh » Jan 18, 2010 1:54 pm

Chet,

Were your backup issues caused by some corruption in a P2V conversion? All details point to that as the problem here. That is VMware's job to resolve, not Veeam's job to resolve.

Anton, I worked with Stanislav over the weekend and we've got most of this straightened out. The last piece that is missing is the part I just mentioned above. When I am finished with VMware today, I will post back what their resolution was for the P2V conversions.

h
For every expert there is an equal and opposite expert - Arthur C Clarke's Fourth Law

Gostev
SVP, Product Management
Posts: 24017
Liked: 3254 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Hot add didn't work for one server

Post by Gostev » Jan 19, 2010 8:32 am

Darhl, do you have any updates? I am travelling today, and it is hard for me to check with our support directly.

withanh
Expert
Posts: 262
Liked: never
Joined: Jul 21, 2009 3:19 pm
Full Name: Darhl
Location: Pacific Northwest
Contact:

Re: Hot add didn't work for one server

Post by withanh » Jan 27, 2010 2:56 pm

The reason the HOTADD/Virtual Appliance mode was not working for some VMs is because I have the disks on different data stores. This results in the .VMDKs having the same name. When Veeam then tries to HOTADD the disks, it fails because VMware does not allow disks with the same name.

The filename of disk 1 is \\datastore1\servername\diskname.vmdk
The filename of disk 2 is \\datastore2\servername\diskname.vmdk

It seems that it does not see the datastore name as part of the path for the file, so it's looking at the \servername\diskname.vmdk and they certainly are the same at that point.

h
For every expert there is an equal and opposite expert - Arthur C Clarke's Fourth Law

sbrumlwy
Novice
Posts: 5
Liked: never
Joined: Feb 02, 2010 10:55 am
Full Name: Steve Brumley
Contact:

Re: Hot add didn't work for one server

Post by sbrumlwy » Feb 04, 2010 3:40 am

I had this same issue and none of the above suggestions worked. Everything checked out ok. I solved the problem by migrating the Veeam host vm to a data store with a 4mb block size. The VM that was failing to backup had a 500GB .vmdk disk. The datastore that housed the Veeam VM had a 1 MB block size.

Why? I have no idea. But maybe it helps someone else. And for the record, VMware Data recovery choked on the same VM, in the same way.

Steven

Vitaliy S.
Product Manager
Posts: 22289
Liked: 1413 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Hot add didn't work for one server

Post by Vitaliy S. » Feb 04, 2010 10:51 am

Hello Steve,

The issue seems to be related to VMFS block size formatting and maximum file size for each type:

• 1MB block size – 256GB maximum file size
• 2MB block size – 512GB maximum file size
• 4MB block size – 1024GB maximum file size
• 8MB block size – 2048GB maximum file size

That explains why you were successfull after moving Veeam Backup and Replication VM to another datastore with 4MB block size of VMFS, because the size of the Hot-Added virtual disk was bigger than maximum file size supported for 1MB block formatted datastore.

sbrumlwy
Novice
Posts: 5
Liked: never
Joined: Feb 02, 2010 10:55 am
Full Name: Steve Brumley
Contact:

Re: Hot add didn't work for one server

Post by sbrumlwy » Feb 04, 2010 2:47 pm

Yes Vitaliy, thanks for your response. I guess my confusion was that I assumed the hot added disk never actually lives on the data store with the smaller block size, so I didn't think that would be a problem. But evidently something happens during hot add that makes the disk think it on the smaller data store (which actually doesn't have enough space to hold it, even if it had the right block size). The data store with the Veeam VM is only a 200GB data store, so I never considered a larger block size for it. I guess it would be a good practice to go ahead and use larger block sizes for data stores no matter what. I am new to this and learning.

Gostev
SVP, Product Management
Posts: 24017
Liked: 3254 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Hot add didn't work for one server

Post by Gostev » Feb 04, 2010 3:09 pm

Now when you mention it is 200GB datastore and 500GB VMDK, I recognize one known issue we have documented in the release notes :
• If ESX datastore hosting Veeam Backup VM has less free space than biggest virtual disk of backed up VM, Virtual Appliance mode fails with the insufficient disk space error
So looks like this issue has nothing to do with VMFS block size... :D

withanh
Expert
Posts: 262
Liked: never
Joined: Jul 21, 2009 3:19 pm
Full Name: Darhl
Location: Pacific Northwest
Contact:

Re: Hot add didn't work for one server

Post by withanh » Feb 04, 2010 3:27 pm

I have noticed a different issue with snapshots as well. I had a VM that had a 25GiB OS drive and a 250GiB Data drive. It was running on a VMFS partition with 1MB blocks (256GiB max file size). It kept failing during the snapshot operation which didn't make sense to me because I stayed under the limits of the max file size. After I created a new partition with 2MB blocks (512GiB max file size) the snapshots (and subsequent backup operation) started working properly. The only explanation I could find is because the total machine size was 275GiB which was over the 256GiB max.

h
For every expert there is an equal and opposite expert - Arthur C Clarke's Fourth Law

Gostev
SVP, Product Management
Posts: 24017
Liked: 3254 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Hot add didn't work for one server

Post by Gostev » Feb 04, 2010 3:38 pm

Darhl, yep this issue with snapshots was reported before by some customers after they upgraded to vSphere, see this thread for example.

sbrumlwy
Novice
Posts: 5
Liked: never
Joined: Feb 02, 2010 10:55 am
Full Name: Steve Brumley
Contact:

Re: Hot add didn't work for one server

Post by sbrumlwy » Feb 04, 2010 3:44 pm

That sounds reasonable Gostev. But the error msg and logs did not reference "insufficient disk space" in any way. It just said it was unable to power on the scsi disk, and that the hotadd transport was not available. An "insufficent disk space" error may have saved me some time! :lol:
Gostev wrote:Now when you mention it is 200GB datastore and 500GB VMDK, I recognize one known issue we have documented in the release notes : So looks like this issue has nothing to do with VMFS block size... :D

withanh
Expert
Posts: 262
Liked: never
Joined: Jul 21, 2009 3:19 pm
Full Name: Darhl
Location: Pacific Northwest
Contact:

Re: Hot add didn't work for one server

Post by withanh » Feb 04, 2010 3:44 pm

Yeah, I found that out way early on when we first installed vSphere and Veeam, it hasn't been an issue for me :)
For every expert there is an equal and opposite expert - Arthur C Clarke's Fourth Law

Post Reply

Who is online

Users browsing this forum: Karinne and 14 guests