Comprehensive data protection for all workloads
Post Reply
egroeg
Enthusiast
Posts: 55
Liked: never
Joined: Sep 23, 2010 2:36 pm
Full Name: George Kenny
Contact:

Volume size over 2TB

Post by egroeg »

Hi all.

I need to create a 3TB volume to a Win2008 R2 Virtual Machine.

It cannot be an RDM or direct iSCSI as I need it to be backed up with Veeam.

VMWare have a native ceiling cap of 2TB for volume sizes (I've extented my datastore to be 7TB, but it's the VOLUME level we're talking about here guys).

Does anyone have an recommendations for manners I can create a +2TB VMFS datastore?

Storage will be on an iSCSI EMC
Gostev
Chief Product Officer
Posts: 31455
Liked: 6646 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Volume size over 2TB

Post by Gostev »

Hi, I suppose you could use spanned volume in Windows that is based on two 1.5TB VMDKs? The only gotcha is that file level restores will not work for this. Thanks.
TaylorB
Enthusiast
Posts: 92
Liked: 14 times
Joined: Jan 28, 2011 4:40 pm
Full Name: Taylor B.
Contact:

Re: Volume size over 2TB

Post by TaylorB »

Create two LUNs smaller than 2TB each. Initialize and format the first one as a VMFS datastore within Vmware. Then go into the properties for that datastore and click to "increase size". This will allow you to add what is called an extent, which extends the space across another LUN.

It's basically the VMware equivalent of the spanned disk, but your Windows VM won't know it isn't a single disk.
Gostev
Chief Product Officer
Posts: 31455
Liked: 6646 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Volume size over 2TB

Post by Gostev »

He already did that, please read the first post carefully...
rfn
Expert
Posts: 141
Liked: 5 times
Joined: Jan 27, 2010 9:43 am
Full Name: René Frej Nielsen
Contact:

How to best create a big file server and back it up

Post by rfn »

[merged]

Hi,

I'm going to migrate an Apple Xserve based file server onto a new VM with Windows Server 2008 R2. There's around 9 TB of data and that will keep growing at a steady pace.

How can I do this best so that I'm able to back up the VM with Veeam Backup? We're using vSphere 5 and VBR 6. The SAN is a HDS HUS 110 (new model) that we will receive in about a month.

I can't find a way to create a really big LUN (fx. 10 TB) and present it to the VM in a way that Veeam can backup. RDM's can be big now (with VMFS5) but vRDM's are still limited to 2 TB. LUN's connected with iSCSI inside the VM are also a no go.

I could create several 1 or 2 TB LUN's with big VMDK's and divide the data on several shares, but we would really like to present it as one big share.

Regards,
René Frej Nielsen
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Volume size over 2TB

Post by dellock6 »

Windows 2008 has mount points as linux has, you can map different vmdk as folder of the same base drive letter. Do not know however how Veeam will recognize this, but the underlying vmdk are there and are ntfs-formatted. Worth a try?
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
Choodee
Influencer
Posts: 21
Liked: 3 times
Joined: Mar 03, 2012 4:30 pm
Full Name: Sandee Dela Cruz
Contact:

Re: Volume size over 2TB

Post by Choodee »

Rene

I'm in the same boat as you (discontinue xServe with 9TB of data to a windows backend). These are the different scenario's I played with.

Hardware
x1 Poweredge R510 /w 12 900GB SAS drives configured as RAID5 (Production vSphere Server)
x1 Poweredge 2950 /w 6 2TB SATA drives configured as RAID5 (Veeam Replication Target)
no shared storage

Scenario 1
- x1 9TB LUN
- x4 2TB VMDK
- Windows 2008 VM

Setup: Setup all vmdks as separate drive in windows and create multiple shares. Present the shares as 1 logical hierarchy using Microsoft DFS.
Pro:
- Works well in windows environments.
- Don't have to worry about windows soft disk spanning/striping
- Full support from VEEAM including Instant FLR
Con:
- Inefficient use of space. eg some drives will have 50% free space while other would have 20%. This will prove problematic in the future as certain folders require more space.
- OSX doesn't like DFS very much. I get around it using by using an application such as ExtremeZIP but certain versions of OSX ie Lion poses issues with microsoft DFS.
- Had to breakup large folders to multiple subfolders creating folder sprawls and broken workflows.

Scenario 2
- x1 9TB LUN
- x4 2TB VMDK (Setup as a single 8TB windows spanned disk)
- Windows 2008 VM

Setup: Setup windows disk spanning and configure the x4 2TB drives as a single 8TB disk (Important: Use disk spanning and not disk striping - Spanning will allow you to add more disks in the future to expand the logical drives).
Pro:
- Single large datastore for storing all data.
- Preserve existing folder structure.
- Share a single root folder
- No need for DFS
- Works well for Mac and Windows environment
Con:
- Disk Spanning. A lot of people will lose sleep over dynamic disks and software raid but with a proper backup/replication the plan the risks are lower.
- Dynamic Disks. Veeam does not support Instant FLR from spanned/stripped dynamic disks.

In the end I chose Scenario 2 (this is after I spent two weeks on Scenario 1... live). The benefits outweighed the potential to lose the spanned disk due to a failed VMDK. However, I made sure that Veeam replication was working on my backup Poweredge 2950 and that I can failover easily if necessary. With Veeam replication, I can get fire up the replica in less than 10 minutes and be up and running in 15minutes. Furthermore, I am able to replicate only the delta changes from my backup replica to my live production after recovery (which is necessary since a full replication takes 30 hours on 8TB of data).

I've been running this setup for a month now without any problems. My MAC's and PC's don't have any issues accessing their data and without any noticeable speed issues. In the future, I will be able to scale up easily by adding more 2TB VMDK disks in the spanned volume. I'm planning the second part of this DR with offsite tape backup storage but so far i'm very happy with my decision.

PS I tested the resiliency of my spanned disk by taking one of the vmdk's offline. As expected, upon bootup Windows gave an error that the disk was broken which prevented any access to the disk. However, once I added the missing disk and restarted the server the logical spanned disk came back online without any issues. I think the probability of losing a single vmdk due to corruption is actually lower since I don't have the complexity of iSCSI/shared storage. My VMDK's are all local running on VMFS datastores.

My 2 cents
Sandee
Cokovic
Veteran
Posts: 295
Liked: 59 times
Joined: Sep 06, 2011 8:45 am
Full Name: Haris Cokovic
Contact:

Re: Volume size over 2TB

Post by Cokovic »

You could setup a SureBackup job or use Other OS FLR to access the dynamic disk from your backup.
Choodee
Influencer
Posts: 21
Liked: 3 times
Joined: Mar 03, 2012 4:30 pm
Full Name: Sandee Dela Cruz
Contact:

Re: Volume size over 2TB

Post by Choodee »

Cokovic wrote:You could setup a SureBackup job or use Other OS FLR to access the dynamic disk from your backup.
I've tried using Other OS FLR to restore a file in my spanned volume and it doesn't work. Veeam will mount the drives as individual vmdk's but won't span the disks to create the volume. The only option that works is to power up the replica and copy the data manually while the VM is running. This obviously has it's own issues with duplicate IP's and being in the same network but there are ways around that.

Sandee
signal_lost
Lurker
Posts: 1
Liked: never
Joined: Mar 14, 2012 4:19 am
Contact:

Re: Volume size over 2TB

Post by signal_lost »

Honestly I don't trust over 2TB to NTFS as chkdsks take FOREVER, and NTFS gets unhappy with the millions of tiny files that generally go with that type of workload.
Honestly if your talking that kind of workload and already buying a HUS, you should talk to your HDS partner about getting a HNAS to front he HUS to handle your file services. It can handle external tiering, backup, replication, NDMP and all the other goodies you need for keeping your file's protected.

If your using FC on the HUS you could always do pRDM's and use something with NDMP to move the data off to a VTL.

(Disclaimer I work for a HDS partner, and am glad the NDA is over for the HUS).
Cokovic
Veteran
Posts: 295
Liked: 59 times
Joined: Sep 06, 2011 8:45 am
Full Name: Haris Cokovic
Contact:

Re: Volume size over 2TB

Post by Cokovic »

Hmm ok. And what about a SureBackup Job? There you should be able to access the spanned volume.
rfn
Expert
Posts: 141
Liked: 5 times
Joined: Jan 27, 2010 9:43 am
Full Name: René Frej Nielsen
Contact:

Re: Volume size over 2TB

Post by rfn »

signal_lost wrote:Honestly I don't trust over 2TB to NTFS as chkdsks take FOREVER, and NTFS gets unhappy with the millions of tiny files that generally go with that type of workload.
Honestly if your talking that kind of workload and already buying a HUS, you should talk to your HDS partner about getting a HNAS to front he HUS to handle your file services. It can handle external tiering, backup, replication, NDMP and all the other goodies you need for keeping your file's protected.

If your using FC on the HUS you could always do pRDM's and use something with NDMP to move the data off to a VTL.

(Disclaimer I work for a HDS partner, and am glad the NDA is over for the HUS).
Since we need to support our Mac users then we need to do the filesharing from Windows Server with ExtremeZ-IP (AppleShare server), so we're unable to use a NAS frontend. I'm also stretching our budget already by buying a new SAN and new servers so there's no money for a NAS box.

We're going to use iSCSI since we already do that for our EqualLogic SAN and buying FC equipment is just to expensive.

So if we're not going to use a NAS do you then suggest that we create many disks and share them one by one? That would be a disk per letter in the alphabet, and even when using thin provisioning it will take up a lot of space over time because of blocks that have been written once. I now that the HUS is supposed to do space reclamation but I don't know how effective it is.
rfn
Expert
Posts: 141
Liked: 5 times
Joined: Jan 27, 2010 9:43 am
Full Name: René Frej Nielsen
Contact:

Re: Volume size over 2TB

Post by rfn »

Choodee wrote:Rene

I'm in the same boat as you (discontinue xServe with 9TB of data to a windows backend). These are the different scenario's I played with.

...

PS I tested the resiliency of my spanned disk by taking one of the vmdk's offline. As expected, upon bootup Windows gave an error that the disk was broken which prevented any access to the disk. However, once I added the missing disk and restarted the server the logical spanned disk came back online without any issues. I think the probability of losing a single vmdk due to corruption is actually lower since I don't have the complexity of iSCSI/shared storage. My VMDK's are all local running on VMFS datastores.
Hmm... I've been advised against this like you and would really like to avoid doing something that other wize guys is telling me not to... Our volume can easily grow to 15-20 TB in a couple of years so it's important that we're not down for a day if Windows needs to check the filesystem for some reason.

I'm almost considering doing a physical RDM and running another kind of backup solution on that server or even to create a physical server with a big LUN on the SAN. It's not what I really want but it might be the best solution?
umenz
Enthusiast
Posts: 52
Liked: never
Joined: Jun 15, 2011 9:24 am
Full Name: Ulrich Menz
Contact:

Re: Volume size over 2TB

Post by umenz »

Gostev wrote:Hi, I suppose you could use spanned volume in Windows that is based on two 1.5TB VMDKs? The only gotcha is that file level restores will not work for this. Thanks.
Is this right today ? - how we can provide 30 TB Volumes in ESXi 5.1 without splitting in 2 TB Drives and than span in Windows ? :shock:

Filelevel Restores are mandatory for me
Gostev
Chief Product Officer
Posts: 31455
Liked: 6646 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Volume size over 2TB

Post by Gostev »

Today, you should simply upgrade to ESXi 5.5 that support 64TB VMDK.
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Volume size over 2TB

Post by dellock6 »

62 to be correct. But still plenty of space to create a 30TB volume.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
averylarry
Veteran
Posts: 264
Liked: 30 times
Joined: Mar 22, 2011 7:43 pm
Full Name: Ted
Contact:

Re: Volume size over 2TB

Post by averylarry »

Isn't there still something you can't do in Veeam with a > 2Tb virtual disk? Like a snapshot or maybe it's hotadd . . .
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Volume size over 2TB

Post by foggy »

Everything that is allowed by vSphere, is possible.
averylarry
Veteran
Posts: 264
Liked: 30 times
Joined: Mar 22, 2011 7:43 pm
Full Name: Ted
Contact:

Re: Volume size over 2TB

Post by averylarry »

Huh. Wonder when the 1.98Tb limitation for hotadd went away.
tsightler
VP, Product Management
Posts: 6009
Liked: 2842 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Volume size over 2TB

Post by tsightler »

With ESXi 5.5 when VMware finally broke the 2TB VMDK limit. Prior to that it was impossible to snapshot a VMDK that was larger than approximately 1.98TB. It wasn't really a hotadd limit (there were other limits around hotadd and block sizes mainly with older VMFS versions).
Jahara
Lurker
Posts: 1
Liked: never
Joined: Feb 07, 2018 6:14 am
Full Name: Jahara Limbaga
Contact:

Re: Volume size over 2TB

Post by Jahara »

Upgraded to VMFS5 but veeam is still seeing the 2TB cap on his VMDK which shouldn't be the case as it's no longer of VMFS3
but the vmdk has inheritied this limitation in the new file system (VMFS5)
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Volume size over 2TB

Post by Vitaliy S. »

Are you able to create a disk larger than 2 TB manually via vSphere Client?
Post Reply

Who is online

Users browsing this forum: legil.miguel and 195 guests