Comprehensive data protection for all workloads
deepak.irani
Novice
Posts: 4
Liked: never
Joined: Aug 04, 2018 8:02 am
Full Name: Deepak R
Contact:

[MERGED] Options to split huge VBK files

Post by deepak.irani »

HI All,

We have backup of a single drive which is 8 TB in size. For secondary storage option we are using azure blobs. Limitation with blob is to have 4.6 TB of single file. We can not use per VM vbk as only one server is the object for backup here.
DGrinev
Veteran
Posts: 1943
Liked: 247 times
Joined: Dec 01, 2016 3:49 pm
Full Name: Dmitry Grinev
Location: St.Petersburg
Contact:

Re: Options to split huge VBK files

Post by DGrinev »

Hi DeepakR,

There is no tool to split large VBK in the current version of the product.
However, your post is counted as a feature request, thank you!
mclemen2
Novice
Posts: 3
Liked: never
Joined: Aug 15, 2012 11:18 pm
Full Name: Michael Clemente
Contact:

Re: feature request: split vbk

Post by mclemen2 »

Another +1.

I have several hosts (some VM some Veeam Agent for Windows hosts) that have up to 25TB disks. I'd like to split them up. Especially since when backing up to SMB repos where Kerberos has a 10hour timeout. If separate files were written out, the ticket would be updated within the window.
fika
Influencer
Posts: 15
Liked: 1 time
Joined: Sep 20, 2017 8:10 am
Full Name: fika

Re: feature request: split vbk

Post by fika »

Any news regarding this feature request ?
Thank you very much Veeam dev team !
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: feature request: split vbk

Post by foggy »

Hi, what is your use case for splitting VBK file?
AlexK977
Novice
Posts: 8
Liked: never
Joined: Mar 10, 2016 10:43 am
Contact:

Re: feature request: split vbk

Post by AlexK977 »

Another +1 for this.
We're hitting a 4TB max file size problem on our Isilon.

@DGrinev: Is there a list online where we could see, if Feature Requests will be/have been tackled ?
Thx
DGrinev
Veteran
Posts: 1943
Liked: 247 times
Joined: Dec 01, 2016 3:49 pm
Full Name: Dmitry Grinev
Location: St.Petersburg
Contact:

Re: feature request: split vbk

Post by DGrinev »

Hey,

There is no way to track FRs online, since they are noted in the internal system which is available only for Veeamers. Thanks!
Gostev
Chief Product Officer
Posts: 31459
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: feature request: split vbk

Post by Gostev »

Question: does Isilon support S3 interface?
AlexK977
Novice
Posts: 8
Liked: never
Joined: Mar 10, 2016 10:43 am
Contact:

Re: feature request: split vbk

Post by AlexK977 »

@Gostev: I don't think so. There is only a thing called CloudPool, where you could connect the Isilon to S3 storage in the cloud. I think the Dell EMC ECS storage has it's own S3 interface.
madmadmaddog
Lurker
Posts: 2
Liked: never
Joined: Dec 13, 2017 10:10 pm
Full Name: Matt
Contact:

Re: feature request: split vbk

Post by madmadmaddog »

We asked for this feature 6 months ago. Tired of waiting with no idea if it would happen we had to move to a competitor. What ever happened to v10?

-Matt.
mkaec
Veteran
Posts: 462
Liked: 133 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: feature request: split vbk

Post by mkaec »

I don't think they ever committed to implementing the feature. Sadly, the common destination for most customer requests (with most software vendors, not just Veeam) is eternal hold.
Gostev
Chief Product Officer
Posts: 31459
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: feature request: split vbk

Post by Gostev »

That is a fair thing to say. With hundreds of pending feature requests, 95% of them will remain in "eternal hold" - with only the most pressing ones ever getting a chance to be implemented.
Andreas Neufert
VP, Product Management
Posts: 6707
Liked: 1401 times
Joined: May 04, 2011 8:36 am
Full Name: Andreas Neufert
Location: Germany
Contact:

Re: feature request: split vbk

Post by Andreas Neufert »

Maybe already too late. But let me add here that there is dsmISI Load Balancer Software that split our workload in a way that all Isilon nodes work on a single Veeam write/read stream in parallel and as well split the backup file. So performance plus and no file sice limitation anymore.
lmbutault
Service Provider
Posts: 31
Liked: 6 times
Joined: Oct 21, 2011 2:11 pm
Full Name: Louis-Marie BUTAULT
Contact:

Re: feature request: split vbk

Post by lmbutault »

any news of this feature request in update 4 ?

please add me to this feature request please
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: feature request: split vbk

Post by veremin »

This feature is not available in Update 4. Thanks!
cmcgee
Lurker
Posts: 2
Liked: never
Joined: Apr 24, 2019 2:28 pm
Contact:

Re: feature request: split vbk

Post by cmcgee »

+1 for this - huge files are very difficult to manage, single VMs option hurts the dedupe ratio, and is still not a viable alternative for large file and database servers, etc.
bhnelson
Enthusiast
Posts: 34
Liked: 9 times
Joined: Oct 01, 2018 5:32 pm
Full Name: Brian Nelson
Contact:

Re: feature request: split vbk

Post by bhnelson »

Another +1 here.

We're trying to mirror our Veeam backup repo to AWS S3, since Veeam can't do that natively. 5TB per-file limit and we have a couple VMs over that size for a full backup.

This is another 'how does a backup product even exist without this feature' kind of feature. And seeing forum requests for it going back to at least 2014 doesn't bode well for ever seeing it implemented. :roll:
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: feature request: split vbk

Post by veremin »

It seems you're trying submit a feature request for one feature, while in reality you need a different one - copy backups to object storage as soon as they are created .This feature is already in development. Thanks!
bhnelson
Enthusiast
Posts: 34
Liked: 9 times
Joined: Oct 01, 2018 5:32 pm
Full Name: Brian Nelson
Contact:

Re: feature request: split vbk

Post by bhnelson »

v.eremin wrote: May 21, 2019 7:10 pm It seems you're trying submit a feature request for one feature, while in reality you need a different one - copy backups to object storage as soon as they are created .This feature is already in development. Thanks!
Yes, that would be great. However after all the hype at VMware World last year, we expected that functionality to come in U4... and were disappointed. Now we're expecting it to come in U5, maybe this year, but who knows? Not gonna hold my breath. We're trying to do it now and can't wait around for Veeam's development cycle.

It'd be nice to at least see basic functionality in lieu of advanced... This feature should be SIGNIFICANTLY less work to implement than direct S3 integration. And it's been requested a slew of times over at least the last 5 years. And there are numerous use cases, as others have mentioned. Mine is just one more on the pile. Please don't tell me I'm asking for the wrong feature.

Incidentally, how do you handle S3's 5TB limit for the scale-out repos today? And how will you handle it with the 'copy to S3' option in the future? Seems like you'd need the 'split vbk' feature for those tasks anyways! I'd wager the functionality is already in the code somewhere, just not exposed to the user. Or maybe Veeam just flat out fails in such situations today. I can't say I've tested it.
jmmarton
Veeam Software
Posts: 2092
Liked: 309 times
Joined: Nov 17, 2015 2:38 am
Full Name: Joe Marton
Location: Chicago, IL
Contact:

Re: feature request: split vbk

Post by jmmarton » 1 person likes this post

Keep in mind backups sent to object storage are stored as objects, not as files and a single restore point could be made up of multiple objects. This means the splitting up of VBKs shouldn't be necessary when storing in S3. Here's an example from my home lab where I take a daily active full of a VM and then move those backups to Minio, which provides S3-storage. You'll see a bunch of objects from May 1 even though there's just a single RP from that day.

Image

Ultimately within a backup chain there's a bit of source-side dedupe going on as well when sending to object storage which minimizes the data sent and stored in the first place. The large VBK problem is more of an issue with file-based storage.

Joe
LBegnaud
Service Provider
Posts: 19
Liked: 7 times
Joined: Jan 24, 2018 12:08 am
Contact:

Re: feature request: split vbk

Post by LBegnaud »

+1 for this feature

Azure SMB is a pretty perfect way to create an SoBR for a BR server in azure... except that they have a 1TiB filesize limitation.
controlfreak
Enthusiast
Posts: 59
Liked: 12 times
Joined: Jan 12, 2018 11:20 pm
Full Name: Control
Location: Washington, USA
Contact:

Re: feature request: split vbk

Post by controlfreak »

+1 for this feature request.

This affects our managed agent backups more than our per VM backups, because the agent backups are larger. We are talking about 2TB+ file server backups (this is a single server in this job).

This also affects my VM backup copy jobs, which are 1TB+. Having one master backup copy job is great from a management perspective, but it doesn't work with huge VBK files. I have about 6 backup copy jobs to reduce the VBK size. I would prefer to have a single backup copy job.

To support the 2 TB+ VBKs, I'm having to reformat my 20TB file server data partitions with "Format /L /A:64K" due to file system limitations (https://www.veeam.com/kb2023). I can now take larger backups, until we hit the new NTFS limits. We can't expect our data volumes to support VBKs that perpetually increase in size. These huge files are also bad for data dedup and for multithreading.

Please add a Max VBK size or a VBK split option for VM and Agent based Backups and Backup Copy Jobs.

Thanks!

Control Freak
jmmarton
Veeam Software
Posts: 2092
Liked: 309 times
Joined: Nov 17, 2015 2:38 am
Full Name: Joe Marton
Location: Chicago, IL
Contact:

Re: feature request: split vbk

Post by jmmarton »

In the meantime, for your situation you may wish to consider ReFS rather than NTFS with deduplication. Windows dedupe typically doesn't work with files > 1 TB, and for large file servers it's quite possible for a single disk to exceed that size so splitting the VBK still wouldn't help. With ReFS at least merge/transform should be much quicker thanks to block cloning, plus ReFS supports larger file sizes than NTFS.

https://docs.microsoft.com/en-us/window ... s-overview

Joe
controlfreak
Enthusiast
Posts: 59
Liked: 12 times
Joined: Jan 12, 2018 11:20 pm
Full Name: Control
Location: Washington, USA
Contact:

Re: feature request: split vbk

Post by controlfreak »

Thanks for the ReFS recommendation Joe. We will look into it as an alternative to NTFS.

-Control
mkaec
Veteran
Posts: 462
Liked: 133 times
Joined: Jul 16, 2015 1:31 pm
Full Name: Marc K
Contact:

Re: feature request: split vbk

Post by mkaec » 3 people like this post

I'm starting to think there is a serious architectural limitation that is causing the product team to not want to implement this. The reasons given for wanting to be able to split the VBK files are reasonable and logical.
jrbaumann
Enthusiast
Posts: 41
Liked: 2 times
Joined: Oct 18, 2018 12:37 pm
Full Name: Jeff Baumann
Contact:

Re: feature request: split vbk

Post by jrbaumann »

+ for this feature

Anyone know if this is being worked on?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: feature request: split vbk

Post by foggy »

Hi Jeff, so far it didn't get high enough in the priority list.
jplopper
Novice
Posts: 3
Liked: never
Joined: Apr 11, 2016 5:23 pm
Full Name: Josiah Plopper
Contact:

Re: feature request: split vbk

Post by jplopper »

+1 For this. NetApp has a single file size of 16TB. When largers CIFS or NFS shares are backed up to a VBK, we're probably going to hit this limit.
edingc
Influencer
Posts: 15
Liked: 2 times
Joined: Jun 26, 2018 12:51 pm
Full Name: Cody Eding
Contact:

Re: feature request: split vbk

Post by edingc »

+1 for this request.
PHBG
Influencer
Posts: 10
Liked: 1 time
Joined: May 30, 2011 8:50 am
Full Name: Petar Havezov
Contact:

Re: feature request: split vbk

Post by PHBG »

+1 for this request
I'm an old Veeam fan and I remember their story. The first versions of Veeam Backup were design to be simple, tiny specialized for virtualization only. Very good products but not covering many of big company needs (for example backup physical server and not only windows but more AIX, Linux). With every new version they become more and more enterprise products. For better or worse some of the limitations they have now, are due to
their legacy in the past design.
I think sooner or later they will implement splitting vbk. The VMs and severs become bigger every year. We have some monster VMs in our company. One of them is 150TB and still not possible to backup with VBR due to such limitation. One 150TB file? What about 1 PB server - 1 PB vbk file?
Post Reply

Who is online

Users browsing this forum: Google [Bot], sergiosergio and 254 guests