Discussions specific to the VMware vSphere hypervisor
Post Reply
jkoop
Influencer
Posts: 12
Liked: 1 time
Joined: May 30, 2014 7:19 am
Contact:

FLR and Instant Recovery fail with 24 TB vbk

Post by jkoop » Jun 04, 2014 12:06 pm

Hi,
I hope I don't spam this forum too much, but I don't run out of problems with a very large VM (24TB).
Regarding Veeam 7, Patch 3, build 839.

I managed to backup the VM after some trouble, but now i can't get a FLR or Instant Recovery to work.
The error I get is always:
Poweron failed: the destination file system does not support large files
An Instant Recovery job starts up all-right, presents the VM correctly, but in vSphere (using latest 5.5 U1) I get the same error, as soon as I try to power on the VM.
In the inventory the VM only uses the PowerNFS store. The underlying filesystem of the PowerNFS store is the same as that hosting the 24 TB vbk file. NFS cluster size: 32K, Windows 8.

I'm starting to wonder if I'm the only one backing up such large VMs...
Jan

foggy
Veeam Software
Posts: 18278
Liked: 1564 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: FLR and Instant Recovery fail with 24 TB vbk

Post by foggy » Jun 04, 2014 1:17 pm

Jan, please check whether you have redirected virtual disk updates while configuring Instant Recovery to some other datastore. Probably the block size for the specified datastore is too small to host redo log for such a big disk.

jkoop
Influencer
Posts: 12
Liked: 1 time
Joined: May 30, 2014 7:19 am
Contact:

Re: FLR and Instant Recovery fail with 24 TB vbk

Post by jkoop » Jun 04, 2014 2:15 pm

The redirect checkbox is not set. According to the info in the wizard, virtual disk updates should go to the PowerNFS store.

tsightler
VP, Product Management
Posts: 5421
Liked: 2243 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: FLR and Instant Recovery fail with 24 TB vbk

Post by tsightler » Jun 04, 2014 2:34 pm

Curious, do you have a single VMDK that is >16TB? You mentioned both Instant Recovery and FLR, is this a Windows machine as the two processes are very different so I would expect a very different method from FLR, although vPower NFS is used for FLR if the host is Linux. The best option is to open a support case. Certainly I've seen customers backup and restore large VMs that are of the sizes you're attempting to cover, however, not until ESXi 5.5 has VMware supported VMDK sizes larger than 2TB so in most cases these VMs had many, smaller VMDK files and were spanned in the OS. Even now I'm really not seeing a lot of customers creating single, large VMDK files yet, although I do run across them now and again. It's certainly possible we're running across some limits that have never previously been seen/reported and the only way this gets resolved is by opening a support case for log investigation. There could also be several other possible causes of the error as well.

Gostev
SVP, Product Management
Posts: 24807
Liked: 3567 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: FLR and Instant Recovery fail with 24 TB vbk

Post by Gostev » Jun 04, 2014 6:55 pm

I am going to guess here that file system of C:\ drive on backup server does not support large files.

jkoop
Influencer
Posts: 12
Liked: 1 time
Joined: May 30, 2014 7:19 am
Contact:

Re: FLR and Instant Recovery fail with 24 TB vbk

Post by jkoop » Jun 05, 2014 11:44 am

Tom,
well, we never went for multiple 2 TB vmdks with spanned volumes in the OS and didn't virtualize these large instances until vsphere 5.5 came around the corner and enabled us to do so without spanning.
We have a support case open regarding another problem which has been solved, but appended this problem now since it is kind of related.

Anton,
I will investigate that and change the backup server to an instance that has a large enough (thin-provisioned ;) ) C:\ drive with a corresponding cluster size.

jkoop
Influencer
Posts: 12
Liked: 1 time
Joined: May 30, 2014 7:19 am
Contact:

Re: FLR and Instant Recovery fail with 24 TB vbk

Post by jkoop » Jun 12, 2014 2:45 pm

Well, I'm extremely disappointed with the on-going of our support case.
The case is open for 2 weeks now and we are sent back to numerous KB articles which all have proven to be unrelated.
I really HATE this kind of support ping-pong.

Isn't there anyone who has VMDKs that big that can tell me "Yes, I have a 30 TB VMDK full of data and it works over here!" ?

Vitaliy S.
Product Manager
Posts: 23001
Liked: 1557 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: FLR and Instant Recovery fail with 24 TB vbk

Post by Vitaliy S. » Jun 12, 2014 4:00 pm

What is your support case number, so we could take a look at this too?

veremin
Product Manager
Posts: 16901
Liked: 1439 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: FLR and Instant Recovery fail with 24 TB vbk

Post by veremin » Jun 13, 2014 10:52 am

Also, you can always ask to escalate your ticket to another engineer, if you're not satisfied with the level of support currently provided. Thanks.

jkoop
Influencer
Posts: 12
Liked: 1 time
Joined: May 30, 2014 7:19 am
Contact:

Re: FLR and Instant Recovery fail with 24 TB vbk

Post by jkoop » Jun 13, 2014 7:32 pm

Hi Vitaly,
our support case number is: 00570665.

Vladimir, we requested an escalation.

I noticed one thing, when I tried an instant recovery again:
The mounted vPower NFS store has a maximum file size of 16 TB, while any other NFS data store shows me 64 TB max file size as it should be
(vmware KB article 2058287).
The vbk file is on an NTFS under windows 8, 256TB max file size and is about 19 TB in size.
Disk updates are not redirected and point to the vPower NFS Store which resides on the same NTFS.
So it looks to me as if the the vPowerNFS service reports its file system, which must be the data structure inside the .vbk file or dynamically emulated, supports 16 TB max.
Here's a screen shot:
Image
Thank you for any help.
Jan

dellock6
Veeam Software
Posts: 5734
Liked: 1625 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: FLR and Instant Recovery fail with 24 TB vbk

Post by dellock6 » Jun 15, 2014 1:05 am

Hi Jan,
just a note, the inner limit of NFS v3 is 64 bit, but the real limit (at least on unix system) is determined by the underlying file system in use, NFS is only the network protocol even if the name says "file system". Interested too in better understand this scenario.

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2019
Veeam VMCE #1

jkoop
Influencer
Posts: 12
Liked: 1 time
Joined: May 30, 2014 7:19 am
Contact:

Re: FLR and Instant Recovery fail with 24 TB vbk

Post by jkoop » Jun 17, 2014 10:12 am

Hi Luca,
Yes, I can see larger possible maximum file sizes on other mounted NFS v3 data stores.
For instance a datastore residing on a Linux machine with an underlying XFS gives me 64 TB max file size using the same interface as used above.
I can only repeat, the underlying NTFS of the vPower NFS store we are trying to use, supports a maximum of 256 TB in file size.
We can confirm that it's working, since the backup itself creates a .vbk file larger 16 TB without a problem.
We tried under Windows 8 and Server 2012R2 with the same outcome.

Still puzzled,
Jan

Post Reply

Who is online

Users browsing this forum: Baidu [Spider], Google [Bot], Mandietha and 10 guests