Comprehensive data protection for all workloads
deepak.irani
Novice
Posts: 4
Liked: never
Joined: Aug 04, 2018 8:02 am
Full Name: Deepak R
Contact:

[MERGED] Options to split huge VBK files

Post by deepak.irani » Aug 10, 2018 10:20 am

HI All,

We have backup of a single drive which is 8 TB in size. For secondary storage option we are using azure blobs. Limitation with blob is to have 4.6 TB of single file. We can not use per VM vbk as only one server is the object for backup here.

DGrinev
Veeam Software
Posts: 1687
Liked: 211 times
Joined: Dec 01, 2016 3:49 pm
Full Name: Dmitry Grinev
Location: St.Petersburg
Contact:

Re: Options to split huge VBK files

Post by DGrinev » Aug 10, 2018 10:57 am

Hi DeepakR,

There is no tool to split large VBK in the current version of the product.
However, your post is counted as a feature request, thank you!

mclemen2
Novice
Posts: 3
Liked: never
Joined: Aug 15, 2012 11:18 pm
Full Name: Michael Clemente
Contact:

Re: feature request: split vbk

Post by mclemen2 » Aug 13, 2018 4:58 pm

Another +1.

I have several hosts (some VM some Veeam Agent for Windows hosts) that have up to 25TB disks. I'd like to split them up. Especially since when backing up to SMB repos where Kerberos has a 10hour timeout. If separate files were written out, the ticket would be updated within the window.

fika
Influencer
Posts: 15
Liked: 1 time
Joined: Sep 20, 2017 8:10 am
Full Name: fika

Re: feature request: split vbk

Post by fika » Nov 27, 2018 9:44 pm

Any news regarding this feature request ?
Thank you very much Veeam dev team !

foggy
Veeam Software
Posts: 18029
Liked: 1533 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: feature request: split vbk

Post by foggy » Nov 28, 2018 4:29 pm

Hi, what is your use case for splitting VBK file?

AlexK977
Novice
Posts: 6
Liked: never
Joined: Mar 10, 2016 10:43 am
Contact:

Re: feature request: split vbk

Post by AlexK977 » Dec 05, 2018 11:22 am

Another +1 for this.
We're hitting a 4TB max file size problem on our Isilon.

@DGrinev: Is there a list online where we could see, if Feature Requests will be/have been tackled ?
Thx

DGrinev
Veeam Software
Posts: 1687
Liked: 211 times
Joined: Dec 01, 2016 3:49 pm
Full Name: Dmitry Grinev
Location: St.Petersburg
Contact:

Re: feature request: split vbk

Post by DGrinev » Dec 05, 2018 11:37 am

Hey,

There is no way to track FRs online, since they are noted in the internal system which is available only for Veeamers. Thanks!

Gostev
SVP, Product Management
Posts: 24450
Liked: 3412 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: feature request: split vbk

Post by Gostev » Dec 05, 2018 5:15 pm

Question: does Isilon support S3 interface?

AlexK977
Novice
Posts: 6
Liked: never
Joined: Mar 10, 2016 10:43 am
Contact:

Re: feature request: split vbk

Post by AlexK977 » Dec 10, 2018 9:33 am

@Gostev: I don't think so. There is only a thing called CloudPool, where you could connect the Isilon to S3 storage in the cloud. I think the Dell EMC ECS storage has it's own S3 interface.

madmadmaddog
Lurker
Posts: 2
Liked: never
Joined: Dec 13, 2017 10:10 pm
Full Name: Matt
Contact:

Re: feature request: split vbk

Post by madmadmaddog » Dec 12, 2018 7:20 pm

We asked for this feature 6 months ago. Tired of waiting with no idea if it would happen we had to move to a competitor. What ever happened to v10?

-Matt.

mkaec
Expert
Posts: 318
Liked: 72 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: feature request: split vbk

Post by mkaec » Dec 12, 2018 7:45 pm

I don't think they ever committed to implementing the feature. Sadly, the common destination for most customer requests (with most software vendors, not just Veeam) is eternal hold.

Gostev
SVP, Product Management
Posts: 24450
Liked: 3412 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: feature request: split vbk

Post by Gostev » Dec 13, 2018 1:52 pm

That is a fair thing to say. With hundreds of pending feature requests, 95% of them will remain in "eternal hold" - with only the most pressing ones ever getting a chance to be implemented.

Andreas Neufert
Veeam Software
Posts: 3711
Liked: 656 times
Joined: May 04, 2011 8:36 am
Full Name: Andreas Neufert
Location: Germany
Contact:

Re: feature request: split vbk

Post by Andreas Neufert » Dec 13, 2018 2:30 pm

Maybe already too late. But let me add here that there is dsmISI Load Balancer Software that split our workload in a way that all Isilon nodes work on a single Veeam write/read stream in parallel and as well split the backup file. So performance plus and no file sice limitation anymore.

lmbutault
Service Provider
Posts: 14
Liked: 1 time
Joined: Oct 21, 2011 2:11 pm
Full Name: Louis-Marie BUTAULT
Contact:

Re: feature request: split vbk

Post by lmbutault » Jan 21, 2019 2:40 pm

any news of this feature request in update 4 ?

please add me to this feature request please

veremin
Product Manager
Posts: 16691
Liked: 1393 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: feature request: split vbk

Post by veremin » Jan 21, 2019 3:15 pm

This feature is not available in Update 4. Thanks!

cmcgee
Lurker
Posts: 1
Liked: never
Joined: Apr 24, 2019 2:28 pm
Full Name: Clint McGee
Contact:

Re: feature request: split vbk

Post by cmcgee » Apr 24, 2019 2:29 pm

+1 for this - huge files are very difficult to manage, single VMs option hurts the dedupe ratio, and is still not a viable alternative for large file and database servers, etc.

bhnelson
Influencer
Posts: 10
Liked: 4 times
Joined: Oct 01, 2018 5:32 pm
Full Name: Brian Nelson
Contact:

Re: feature request: split vbk

Post by bhnelson » May 21, 2019 5:31 pm

Another +1 here.

We're trying to mirror our Veeam backup repo to AWS S3, since Veeam can't do that natively. 5TB per-file limit and we have a couple VMs over that size for a full backup.

This is another 'how does a backup product even exist without this feature' kind of feature. And seeing forum requests for it going back to at least 2014 doesn't bode well for ever seeing it implemented. :roll:

veremin
Product Manager
Posts: 16691
Liked: 1393 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: feature request: split vbk

Post by veremin » May 21, 2019 7:10 pm

It seems you're trying submit a feature request for one feature, while in reality you need a different one - copy backups to object storage as soon as they are created .This feature is already in development. Thanks!

bhnelson
Influencer
Posts: 10
Liked: 4 times
Joined: Oct 01, 2018 5:32 pm
Full Name: Brian Nelson
Contact:

Re: feature request: split vbk

Post by bhnelson » May 21, 2019 8:59 pm

v.eremin wrote:
May 21, 2019 7:10 pm
It seems you're trying submit a feature request for one feature, while in reality you need a different one - copy backups to object storage as soon as they are created .This feature is already in development. Thanks!
Yes, that would be great. However after all the hype at VMware World last year, we expected that functionality to come in U4... and were disappointed. Now we're expecting it to come in U5, maybe this year, but who knows? Not gonna hold my breath. We're trying to do it now and can't wait around for Veeam's development cycle.

It'd be nice to at least see basic functionality in lieu of advanced... This feature should be SIGNIFICANTLY less work to implement than direct S3 integration. And it's been requested a slew of times over at least the last 5 years. And there are numerous use cases, as others have mentioned. Mine is just one more on the pile. Please don't tell me I'm asking for the wrong feature.

Incidentally, how do you handle S3's 5TB limit for the scale-out repos today? And how will you handle it with the 'copy to S3' option in the future? Seems like you'd need the 'split vbk' feature for those tasks anyways! I'd wager the functionality is already in the code somewhere, just not exposed to the user. Or maybe Veeam just flat out fails in such situations today. I can't say I've tested it.

jmmarton
Veeam Software
Posts: 1595
Liked: 209 times
Joined: Nov 17, 2015 2:38 am
Full Name: Joe Marton
Location: Chicago, IL
Contact:

Re: feature request: split vbk

Post by jmmarton » May 21, 2019 9:43 pm 1 person likes this post

Keep in mind backups sent to object storage are stored as objects, not as files and a single restore point could be made up of multiple objects. This means the splitting up of VBKs shouldn't be necessary when storing in S3. Here's an example from my home lab where I take a daily active full of a VM and then move those backups to Minio, which provides S3-storage. You'll see a bunch of objects from May 1 even though there's just a single RP from that day.

Image

Ultimately within a backup chain there's a bit of source-side dedupe going on as well when sending to object storage which minimizes the data sent and stored in the first place. The large VBK problem is more of an issue with file-based storage.

Joe

LBegnaud
Service Provider
Posts: 19
Liked: 7 times
Joined: Jan 24, 2018 12:08 am
Contact:

Re: feature request: split vbk

Post by LBegnaud » Jun 17, 2019 6:31 pm

+1 for this feature

Azure SMB is a pretty perfect way to create an SoBR for a BR server in azure... except that they have a 1TiB filesize limitation.

controlfreak
Enthusiast
Posts: 46
Liked: 5 times
Joined: Jan 12, 2018 11:20 pm
Full Name: Control
Location: Washington, USA
Contact:

Re: feature request: split vbk

Post by controlfreak » Jun 25, 2019 12:00 am

+1 for this feature request.

This affects our managed agent backups more than our per VM backups, because the agent backups are larger. We are talking about 2TB+ file server backups (this is a single server in this job).

This also affects my VM backup copy jobs, which are 1TB+. Having one master backup copy job is great from a management perspective, but it doesn't work with huge VBK files. I have about 6 backup copy jobs to reduce the VBK size. I would prefer to have a single backup copy job.

To support the 2 TB+ VBKs, I'm having to reformat my 20TB file server data partitions with "Format /L /A:64K" due to file system limitations (https://www.veeam.com/kb2023). I can now take larger backups, until we hit the new NTFS limits. We can't expect our data volumes to support VBKs that perpetually increase in size. These huge files are also bad for data dedup and for multithreading.

Please add a Max VBK size or a VBK split option for VM and Agent based Backups and Backup Copy Jobs.

Thanks!

Control Freak

jmmarton
Veeam Software
Posts: 1595
Liked: 209 times
Joined: Nov 17, 2015 2:38 am
Full Name: Joe Marton
Location: Chicago, IL
Contact:

Re: feature request: split vbk

Post by jmmarton » Jun 26, 2019 2:38 pm

In the meantime, for your situation you may wish to consider ReFS rather than NTFS with deduplication. Windows dedupe typically doesn't work with files > 1 TB, and for large file servers it's quite possible for a single disk to exceed that size so splitting the VBK still wouldn't help. With ReFS at least merge/transform should be much quicker thanks to block cloning, plus ReFS supports larger file sizes than NTFS.

https://docs.microsoft.com/en-us/window ... s-overview

Joe

controlfreak
Enthusiast
Posts: 46
Liked: 5 times
Joined: Jan 12, 2018 11:20 pm
Full Name: Control
Location: Washington, USA
Contact:

Re: feature request: split vbk

Post by controlfreak » Jun 26, 2019 4:20 pm

Thanks for the ReFS recommendation Joe. We will look into it as an alternative to NTFS.

-Control

mkaec
Expert
Posts: 318
Liked: 72 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: feature request: split vbk

Post by mkaec » Jun 26, 2019 8:37 pm 2 people like this post

I'm starting to think there is a serious architectural limitation that is causing the product team to not want to implement this. The reasons given for wanting to be able to split the VBK files are reasonable and logical.

Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 39 guests