FLR and Instant Recovery fail with 24 TB vbk

[jkoop]

Hi,
I hope I don't spam this forum too much, but I don't run out of problems with a very large VM (24TB).
Regarding Veeam 7, Patch 3, build 839.

I managed to backup the VM after some trouble, but now i can't get a FLR or Instant Recovery to work.
The error I get is always:

Poweron failed: the destination file system does not support large files

An Instant Recovery job starts up all-right, presents the VM correctly, but in vSphere (using latest 5.5 U1) I get the same error, as soon as I try to power on the VM.
In the inventory the VM only uses the PowerNFS store. The underlying filesystem of the PowerNFS store is the same as that hosting the 24 TB vbk file. NFS cluster size: 32K, Windows 8.

I'm starting to wonder if I'm the only one backing up such large VMs...
Jan

[foggy]

Jan, please check whether you have redirected virtual disk updates while configuring Instant Recovery to some other datastore. Probably the block size for the specified datastore is too small to host redo log for such a big disk.

[jkoop]

The redirect checkbox is not set. According to the info in the wizard, virtual disk updates should go to the PowerNFS store.

[tsightler]

Curious, do you have a single VMDK that is >16TB? You mentioned both Instant Recovery and FLR, is this a Windows machine as the two processes are very different so I would expect a very different method from FLR, although vPower NFS is used for FLR if the host is Linux. The best option is to open a support case. Certainly I've seen customers backup and restore large VMs that are of the sizes you're attempting to cover, however, not until ESXi 5.5 has VMware supported VMDK sizes larger than 2TB so in most cases these VMs had many, smaller VMDK files and were spanned in the OS. Even now I'm really not seeing a lot of customers creating single, large VMDK files yet, although I do run across them now and again. It's certainly possible we're running across some limits that have never previously been seen/reported and the only way this gets resolved is by opening a support case for log investigation. There could also be several other possible causes of the error as well.

[Gostev]

I am going to guess here that file system of C:\ drive on backup server does not support large files.

[jkoop]

Tom,
well, we never went for multiple 2 TB vmdks with spanned volumes in the OS and didn't virtualize these large instances until vsphere 5.5 came around the corner and enabled us to do so without spanning.
We have a support case open regarding another problem which has been solved, but appended this problem now since it is kind of related.

Anton,
I will investigate that and change the backup server to an instance that has a large enough (thin-provisioned ) C:\ drive with a corresponding cluster size.

[jkoop]

Well, I'm extremely disappointed with the on-going of our support case.
The case is open for 2 weeks now and we are sent back to numerous KB articles which all have proven to be unrelated.
I really HATE this kind of support ping-pong.

Isn't there anyone who has VMDKs that big that can tell me "Yes, I have a 30 TB VMDK full of data and it works over here!" ?

[Vitaliy S.]

What is your support case number, so we could take a look at this too?

[veremin]

Also, you can always ask to escalate your ticket to another engineer, if you're not satisfied with the level of support currently provided. Thanks.

[jkoop]

Hi Vitaly,
our support case number is: 00570665.

Vladimir, we requested an escalation.

I noticed one thing, when I tried an instant recovery again:
The mounted vPower NFS store has a maximum file size of 16 TB, while any other NFS data store shows me 64 TB max file size as it should be
(vmware KB article 2058287).
The vbk file is on an NTFS under windows 8, 256TB max file size and is about 19 TB in size.
Disk updates are not redirected and point to the vPower NFS Store which resides on the same NTFS.
So it looks to me as if the the vPowerNFS service reports its file system, which must be the data structure inside the .vbk file or dynamically emulated, supports 16 TB max.
Here's a screen shot:

Thank you for any help.
Jan

[dellock6]

Hi Jan,
just a note, the inner limit of NFS v3 is 64 bit, but the real limit (at least on unix system) is determined by the underlying file system in use, NFS is only the network protocol even if the name says "file system". Interested too in better understand this scenario.

Luca.

[jkoop]

Hi Luca,
Yes, I can see larger possible maximum file sizes on other mounted NFS v3 data stores.
For instance a datastore residing on a Linux machine with an underlying XFS gives me 64 TB max file size using the same interface as used above.
I can only repeat, the underlying NTFS of the vPower NFS store we are trying to use, supports a maximum of 256 TB in file size.
We can confirm that it's working, since the backup itself creates a .vbk file larger 16 TB without a problem.
We tried under Windows 8 and Server 2012R2 with the same outcome.

Still puzzled,
Jan