Is there a way to SPAN large VBK files ??

[jkeegan]

Hello all:

Some of my Veeam Backups are of extremely large servers. Sometimes the backups fail on larger servers with the message "Backup failed due to the following error: The connection to the storage was forcibly closed by the remote host".

When I see this error, I can say for sure that the NAS was under very high utilization. When a 600GB file is being written, there is a lot of time for that connection to be closed / cancelled / time out, BUT if the files being written were smaller, say ... 60 x 10GB files rather than 1 x 600GB file... there is less of an opportunity for the job to fail due to the NAS storage timing out.

What are the communities thoughts on this? Is there a way to force the files to be multiple smaller files?

[nikpolini]

I had this issues with 12TB file servers backing up to an SMB share.
Ended up splitting the file servers up into 3TB size and using DFS so users don't see it. Also ended up attaching 100TB JBOD directly to the backup server.
Splitting large VBKs would be a great feature!

[veremin]

How many VMs are included in a backup job? Or is it just one VM amounting for a big backup file? Thanks.

[Vitaliy S.]

I believe using per-VM backup chains should help in this situation. Each VM will have its own backup files, assuming that you have multiple VMs in the job.

[Delo123]

Nope, doesn't help with big vm's. We also would like this (have ask for this featute in other threads already some time ago), splitting vbk's has much advantages, a few weeks ago i was swearing again when i had to copy a 4TB vbk (single vm) and we had a network glitch at 98%, would these have been 40 100GB files i would still be smiling today, also this would make things much easier for things like windows dedupe. Not being able to split vbk's in chunks is the ONLY reason we are still using acronis on some stupid machines (checkmark split fulls in xxx chunks)

[dellock6]

Guido, I know this is not the answer, but at least for vbk management, are you using regular filesytem tools to copy them around? Because many copy tools have multi-stream and resume capability, that missing 2% would have been copied upon retry after the network was re-established. And by the way it's the default mechanism with backup copy job

Just out of curiosity, what would be the desired file size? Or totally configurable in the UI?

[Delo123]

Hi Luca,

i agree there are other and better tools to handle this, but as so often this was some midnight action to "quickly" copy something away. I don't know, if you would implement an option like this i would say let the customer decide. But i guess something like 500GB would be nice, especially for windows dedupe which at least "we" still love for veeam The REfs announcment sound great and i guess we will start using it on our primary repositories, but i still think the archival box will stay on deduped storage spaces. One on our repositores is 270TB, 166Tb free but 471TB's saved by dedupe, which is quite amazing...

Ps. i feel veeam copy jobs could also run faster if they could run multi-stream, or do they??

[foggy]

Ps. i feel veeam copy jobs could also run faster if they could run multi-stream, or do they??

With per-VM backup chains, yes.

[Delo123]

Ok, so for single large VBK's (which obviously run much longer than all the rest) no...

[reaperhammer]

I suggest splitting large servers up by making one job per disk and have the jobs run linked to each other. This has worked for me with some 10TB high change file servers, also snapshots are around for less time which is good.

[Delo123]

Sounds creative but totally unneccesary if possible to split up vbk's. Doing seperate disk backups of fileservers with dynamic disks and dedupe enabled in the vm's sound very scary to me (impossible, will not work)

[foggy]

This approach also has some other downsides.

[bg.ranken]

I second this request. Though with the way the backup files and incrementals work, I'm assuming it wouldn't be easy since Veeam needs to modify multiple sections of the backup file when rolling in incremental chains.

But until that time perhaps having a per-vmdk option would be easier to implement? Would that help processing speeds for large 10+TB servers with lots of disks? I know for me I have two servers with 12+ disks and my proxy and repository is only allowed to have 12 concurrent activities. I believe this means I only get one stream to the repository for a bit and lose performance compared to when I have other jobs with per-vm running concurrently.

[gosnells]

+1
per-VM backup chains only goes so far with really large VMs.

[thefold]

+1 for a split vbk feature

I have a 40+TB VM that I keep getting the same error as OP on.

If I already had a full backup the incrementals would be fine, but as I had an issue with the previous full I had to initiate an active full and expire the previous backup chain. I'm currently looking at ways to increase the backup speed by installing an additional 4-port NIC which can be dedicated to backups (my current bottleneck is the network), but any advice for the time being would be extremely helpful!

[Delo123]

I'm currently looking at ways to increase the backup speed by installing an additional 4-port NIC which can be dedicated to backups

You could of course test that, what are you looking to add? 4x1GB? The problem with multiple nics is you either have to distribute the jobs over them which wouldn't help since your doing a single vm backup. Also fancy aggregation of nics wouldn't be very easy here as source destination are always the same. Maybe you can add 10GBit nics? Maybe direct connection? SAN Direct connection might also be an option...

[thefold]

My plan is to team the NICs up to increase the available bandwidth. Certainly when I've done this for the 2 internal ports before I've managed to double the throughput. It would be a 4x1GB card I'm adding in.

[Delo123]

What are you using? LACP? so would still need to have mupltiple sources or multiple destionations but you could give it a try No options for 10Gbit? Not that expensive anymore, and maybe you can do without switches if your environment isn't very big...