-
- Veteran
- Posts: 264
- Liked: 30 times
- Joined: Mar 22, 2011 7:43 pm
- Full Name: Ted
- Contact:
Max virtual disk size that allows hotadd
Case # 00154561
I'm expecting to hear back from Tier 2 (or above) as to the technical "why", but I thought maybe I could get a faster answer here.
I'm being told that hotadd has a smaller virtual disk size limit than VMware snapshot.
From here:
http://kb.vmware.com/selfservice/micros ... Id=1012384
the maximum file size of vmfs5 is 2TB-512B. But (and I found this out the hard way), you cannot snapshot a virtual disk that big. There is overhead associated with a snapshot. To almost be precise, it is a maximum overhead of about 16GB for a 2TB-512B virtual disk.
So if you want to snapshot a virtual disk, the actual maximum size is 2032GB. Straight from the chart referenced aboved. I created a 2030GB virtual disk, but Veeam tech support says it is too big for hotadd, even though I can manually take a snapshot and hotadd it.
So -- I would like to know the actual technical limit so I can (eventually, since I don't have 2TB of free space until I can buy a new SAN in January) create an appropriate virtual disk that Veeam can successfully hotadd.
I'm expecting to hear back from Tier 2 (or above) as to the technical "why", but I thought maybe I could get a faster answer here.
I'm being told that hotadd has a smaller virtual disk size limit than VMware snapshot.
From here:
http://kb.vmware.com/selfservice/micros ... Id=1012384
the maximum file size of vmfs5 is 2TB-512B. But (and I found this out the hard way), you cannot snapshot a virtual disk that big. There is overhead associated with a snapshot. To almost be precise, it is a maximum overhead of about 16GB for a 2TB-512B virtual disk.
So if you want to snapshot a virtual disk, the actual maximum size is 2032GB. Straight from the chart referenced aboved. I created a 2030GB virtual disk, but Veeam tech support says it is too big for hotadd, even though I can manually take a snapshot and hotadd it.
So -- I would like to know the actual technical limit so I can (eventually, since I don't have 2TB of free space until I can buy a new SAN in January) create an appropriate virtual disk that Veeam can successfully hotadd.
-
- Veeam Software
- Posts: 21139
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Max virtual disk size that allows hotadd
Ted, have you tried to run Veeam jobs for this disk? I'm pretty sure that if you can hotadd it manually, Veeam can do that either.
The actual technical limit is referenced in the same article - it is 2,181,972,430,848 bytes for VMFS5 (which is ~2032GB).
The actual technical limit is referenced in the same article - it is 2,181,972,430,848 bytes for VMFS5 (which is ~2032GB).
-
- VP, Product Management
- Posts: 6035
- Liked: 2860 times
- Joined: Jun 05, 2009 12:57 pm
- Full Name: Tom Sightler
- Contact:
Re: Max virtual disk size that allows hotadd
Yes, I agree. There were some technical limits with hotadd regarding size with earlier versions of the VDDK, however, with current versions you should be able to backup a VMDK of 2032GB.
However, it is very important to remember that, with vSphere 5.0 and above, by default, the snapshot delta's are stored on the same disk with the VMDK file. That means you need to have enough space not just for the initial snapshot overhead (16GB), but also enough to contain any changes that might be written to the disk while the snapshot is open. You can redirect the delta files to another disk, but this has some impact on features like sVmotion and storage DRS.
However, it is very important to remember that, with vSphere 5.0 and above, by default, the snapshot delta's are stored on the same disk with the VMDK file. That means you need to have enough space not just for the initial snapshot overhead (16GB), but also enough to contain any changes that might be written to the disk while the snapshot is open. You can redirect the delta files to another disk, but this has some impact on features like sVmotion and storage DRS.
-
- Veteran
- Posts: 264
- Liked: 30 times
- Joined: Mar 22, 2011 7:43 pm
- Full Name: Ted
- Contact:
Re: Max virtual disk size that allows hotadd
Very well, I will wait and see what tier 2 tech support says.
-
- Enthusiast
- Posts: 38
- Liked: 2 times
- Joined: Dec 07, 2011 5:21 pm
- Full Name: Serge Adam
- Contact:
Re: Max virtual disk size that allows hotadd
For practical purposes, I create all my lage disks at 1.9TB. You do need enough free space to hold the snapshot.
-
- Veteran
- Posts: 264
- Liked: 30 times
- Joined: Mar 22, 2011 7:43 pm
- Full Name: Ted
- Contact:
Re: Max virtual disk size that allows hotadd
This is my reply from tech support:
I would consider this important enough to add to the documentation as a limitation of hotadd.
Further, as an enhancement request, it would be nice if hotadd wouldn't fail on ALL disks just because one of the disks is too big. Additionally, even if I exclude my 1.983 Tb disk from the backup job, the job still fails to hotadd (though it is implied that it would not fail if the disk was independent).
They also said direct san mode does not have this limitation (it's a limitation of the 32 bit vddk supplied by VMware).We have concluded our tests. The limitation is 1.98 TB even.
1.98TB even is 2027.52 GB which worked successfully.
Once we incremented the VMDK to 2028 GB it failed with the same error.
This should be a concrete answer for the VDDK 5.1 limitation so you can use this when creating new disks you intend to backup using hot-add. Let me know if you need anything else or if you have further questions regarding this.
I would consider this important enough to add to the documentation as a limitation of hotadd.
Further, as an enhancement request, it would be nice if hotadd wouldn't fail on ALL disks just because one of the disks is too big. Additionally, even if I exclude my 1.983 Tb disk from the backup job, the job still fails to hotadd (though it is implied that it would not fail if the disk was independent).
-
- Veteran
- Posts: 264
- Liked: 30 times
- Joined: Mar 22, 2011 7:43 pm
- Full Name: Ted
- Contact:
Re: Max virtual disk size that allows hotadd
PS -- I don't really think I should have been merged since this has nothing to do with snapshots, but with a separate vddk/hotadd limitation that is more restrictive than vmware snapshot limitations. The snapshot limitation is a red herring
-
- Veeam Software
- Posts: 21139
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Max virtual disk size that allows hotadd
Ted, thanks for posting back the testing results. I'll ask support guys and technical writers to update the corresponding sources.averylarry wrote:I would consider this important enough to add to the documentation as a limitation of hotadd.
This is a known limitation of vStorage API: it is not possible to hotadd VM disks partially, all disks of the backed up VM are hotadded during its processing.averylarry wrote:Further, as an enhancement request, it would be nice if hotadd wouldn't fail on ALL disks just because one of the disks is too big. Additionally, even if I exclude my 1.983 Tb disk from the backup job, the job still fails to hotadd (though it is implied that it would not fail if the disk was independent).
-
- Veteran
- Posts: 264
- Liked: 30 times
- Joined: Mar 22, 2011 7:43 pm
- Full Name: Ted
- Contact:
Re: Max virtual disk size that allows hotadd
Thank you. Now I just have to wait until next year when I buy a new SAN and then create a new (slightly smaller) virtual disk and manually copy all the data (using multiple passes of robocopy) and then carry my disaster recovery server back to my main office to re-sync the (almost) 2Tb of data.
Or maybe I'll try to squeeze a direct san proxy onto my not-enough-ports SAN switch somehow. Oh well, at least all the jobs are working well enough failing back to network mode.
Or maybe I'll try to squeeze a direct san proxy onto my not-enough-ports SAN switch somehow. Oh well, at least all the jobs are working well enough failing back to network mode.
-
- VP, Product Management
- Posts: 6035
- Liked: 2860 times
- Joined: Jun 05, 2009 12:57 pm
- Full Name: Tom Sightler
- Contact:
Re: Max virtual disk size that allows hotadd
If your SAN supports iSCSI connectivity you could configure a virtual proxy to use Direct SAN access with the Microsoft iSCSI initiator. It's usually still faster than using network mode.
-
- Veteran
- Posts: 264
- Liked: 30 times
- Joined: Mar 22, 2011 7:43 pm
- Full Name: Ted
- Contact:
Re: Max virtual disk size that allows hotadd
Yes indeed. But I still need physical network cards, physical switch ports, and proper virtual networking across my cluster OR a physical server. I don't really have any of those things, and I don't want to try and piggy-back onto my software iSCSI networking in VMware.tsightler wrote:If your SAN supports iSCSI connectivity you could configure a virtual proxy to use Direct SAN access with the Microsoft iSCSI initiator. It's usually still faster than using network mode.
Eventually I'll probably configure direct san mode -- but it'll likely have to wait until I get 10Gb SAN networking up and running first.
Who is online
Users browsing this forum: No registered users and 18 guests