16TB+ vbk files on NTFS/exFAT

[jkoop]

Hi there,
We have several VMs that are using up to the maximum of 62 TB VMDKs.
Not all of that data is allocated, but we have 2 Jobs that need to back up approximately 24 TB of data each. Even with compression and dedup that produces vbk files larger than 16 TB. Theoretically!
I just cannot get past 16.0 TB in my jobs!
Here's the setup:
- Veeam 7, Patch 3, build 839
- The proxy + repo server is a VM that runs Windows Server 2008 R2 Datacenter Edition.
- The repo is an iSCSI device connected via iSCSI initiator inside the VM and is 72 TB in size.
- The repo used to be formatted with NTFS, but since there is a 16 TB (16 TiB - 64 KiB = exactly 17592185978880 Bytes ) limitation in 2008 R2's implementation of NTFS, i switched to exFAT
- I wasn't able to create files larger than 17592185978880 Bytes on NTFS (even 17592185978881 bytes failed), but on exFAT it works (created a 27 TB file sucessfully)
- Job fails at 16.0 TB data transferred
- Job: compression on, dedup on, optimization for 16TB+ vbk files

Has anyone successfully seen a vbk larger than 16TB? What can I do to make this work?

Cheers,
Jan

[veremin]

Hi, Jan,

According to the QA team, we don't have any issues with creating backup files larger than 16TB, as long as underlying OS allows such operation. So, I'd recommend you opening a ticket with our support team and letting them confirm your environment. Once opened, don't forget to post the ticket number here, so that, we can follow the case and update the topic, once resolution is found.

Thanks.

[tsightler]

Can you upgrade the host to 2012? I've certainly never tried exFAT, and I wouldn't consider it a suitable filesystem for storing backups as it is not a journaled filesystem and keeps only a single copy of the FAT so any time of hard crash or other failure could leave your filesystem in a world of hurt and your backups unrecoverable.

In Windows 2012 and above properly formatted NTFS should support file sizes of 256TB (well, 256TB - 64K), or you could use ReFS which I believe has a maximum size of 16EB. As NTFS is a well proven filesystem with solid journalling and recovery functions I'd probably go that route first, but ReFS seems to be well designed, and has worked well in the testing scenarios I've seen with it, but I just don't have enough experience with it in the field to feel 100% comfortable recommending it. Only time can prove a filesystem reliable. There are also plenty of options for >16TB file support when using a Linux repository.

[jkoop]

Hi,
Thanks for the replies.
just to make this post complete, with a repo server running Windows 8 with its implementation of NTFS the job succeeded.
I abandoned the thought of using exFAT in favor of stability. ReFS is very tempting, with the downside of only being supported by the very latest Microsoft OSes, so I think I'll stick to NTFS.
Since I like to keep the repo and the proxy in one machine, I won't take the Linux options (namely xfs as option #1), even though most of our machines run Linux.
So maybe this is of use to someone else, problem is [solved]
Cheers,
Jan