Comprehensive data protection for all workloads
Post Reply
ekisner
Expert
Posts: 202
Liked: 34 times
Joined: Jul 26, 2012 8:04 pm
Full Name: Erik Kisner
Contact:

Feature Request: Multiple VBK files

Post by ekisner »

Hi there,

So I got to thinking that it would be nice to have each virtual disk on a server go to it's own VBK file.

Why? If a tape job is running, it might be able to squeeze something into that last 30gb left on a tape or if the tape job fails, part of the VM may be on the tape and a retry will then take less time as it won't have to start the whole VBK again.

It wouldn't be a ground-breaking improvement, and it might make the folder structure a bit more complex inside the repository, but I could see it offering minor performance improvements.

Compression would need to be enabled on the volume, rather than on the VBK, to ensure optimal compression. But that's kind of the case already.
Cragdoo
Veeam Vanguard
Posts: 628
Liked: 251 times
Joined: Sep 27, 2011 12:17 pm
Full Name: Craig Dalrymple
Location: Scotland
Contact:

Re: Feature Request: Multiple VBK files

Post by Cragdoo »

This very feature , per-vm backup chain, will be available in v9 :)
nielsengelen
Product Manager
Posts: 5619
Liked: 1177 times
Joined: Jul 15, 2013 11:09 am
Full Name: Niels Engelen
Contact:

Re: Feature Request: Multiple VBK files

Post by nielsengelen »

As mentioned this is coming within v9, have a look at http://go.veeam.com/v9#backup-storage (final point: New features for high-performance backups with ANY deduplication appliance: Get up to 10x faster backup performance with the new per-VM backup file chains option, which enables multiple write streams by leveraging parallel VM processing. Improve local backup copy performance and reduce the load on deduplication appliances by eliminating data rehydration with new support for active full backups with backup copy jobs.) for more information.
Personal blog: https://foonet.be
GitHub: https://github.com/nielsengelen
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Feature Request: Multiple VBK files

Post by dellock6 »

Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Feature Request: Multiple VBK files

Post by foggy »

Just to avoid confusion, v9 will feature per-VM backup chain, but not per-disk Erik is talking about. So each VM will have its own own VBK file, containing all its virtual disks.
ekisner
Expert
Posts: 202
Liked: 34 times
Joined: Jul 26, 2012 8:04 pm
Full Name: Erik Kisner
Contact:

Re: Feature Request: Multiple VBK files

Post by ekisner »

Thank you Foggy.

I presently use 1 backup job per VM to simulate this feature already, the next progression for me is indeed per-disk backup chains.

I do appreciate the answers.
PHBG
Influencer
Posts: 10
Liked: 1 time
Joined: May 30, 2011 8:50 am
Full Name: Petar Havezov
Contact:

[MERGED] Feature Request > per-Disk chain

Post by PHBG »

Reading about new features in v9 – Scale Out Repository and Per-VM backup chains I didn’t find resolution of challenges backuping one big VM .
What is the problem of backuping one big VM, let me try an example of our environment:
Our Data Warehouse is 40 TB at the moment with multiple vDisks. We design VM storage layout properly (many vDisk balancing through 4x8 Gbps FC HBA adapter) so the VM is able to generate 4 GB/s (32 Gbps). It’s average disk rate is around 1,5 GB/s with many pick up to 3-4 GB/s.
I think at the current Veeam design it is not possible to backup with speed more than 400-500 MB/s per one VM because many Veeam Proxy generate traffic to one backup file which become a bottleneck. Same situation like Per-VM backup chains – more speed if VMs are separated.
So my feature request is to go one step further – Per-Disk backup chains. One BIG VM with multiple disk to be able to split to multiple vbk files and multiple repository.

Some math:
Speed --> Backup Time (40 TB)
250 MB/s --> 46,6 h
500 MB/s --> 23,3 h
1 GB/s --> 11,7 h
2 GB/s --> 5,8 h
4 GB/s --> 2,9 h
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Feature Request > per-Disk chain

Post by Vitaliy S. »

Processing rate depends not only on the way you backup your VM, but also on the hardware (source and target). Upgrading target storage hardware (to support more IOPs) should bump up the performance of the backup job. Thanks for the feedback anyway!
Gostev
Chief Product Officer
Posts: 31457
Liked: 6647 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Feature Request > per-Disk chain

Post by Gostev »

Your math is only applicable to full backups though. And I am sure that with 40TB VM, you don't do periodic fulls, right?
PHBG
Influencer
Posts: 10
Liked: 1 time
Joined: May 30, 2011 8:50 am
Full Name: Petar Havezov
Contact:

Re: Feature Request > per-Disk chain

Post by PHBG »

Vitaliy S. wrote:Processing rate depends not only on the way you backup your VM, but also on the hardware (source and target). Upgrading target storage hardware (to support more IOPs) should bump up the performance of the backup job. Thanks for the feedback anyway!
I am 100% agree that hardware (source/proxy/target) is the most important component of processing rate. But I am 100% sure that there is no company that backup storage is more powerful than the primary storage. Why EMC Data Domain, HP StorOnce or other Dedup solution used NL-SAS/SATA disk (7200)?

Our primary storage is mix of SSD, SAS-10k, NL-SAS disks with Auto Tiering where 40-50% storage space is from capacity disk (NL-SAS). We have two instance of Data Warehouse – new on VMware (x86) - 40 TB mention above, and old one on AIX (Power7) – 50 TB. AIX instance is entirely on NL-SAS storage at the moment because of retirement. Now we are implementing HP StorOnce and make some backup test with AIX (SAN to VTL Backup LAN-free) and the speed was 1,5 GB/s due to parallel reading different file system and writing to different drivers. One VM (AIX) on a slow storage but backup speed was nice 1,5 GB/s to StorOnce due to parallelism.

So if you have a big VM on not very fast storage (from IOPS perspective), but able to generate enough disk rate/throughput you will not invest in more expensive storage for backup. So my point is that in equal situation where performance of primary storage = backup storage, you can read parallel from different vDisk/Datastore but not able to write to different veeam file/repository. So there are some limits in veeam backup software in this situation.
Gostev wrote:Your math is only applicable to full backups though. And I am sure that with 40TB VM, you don't do periodic fulls, right?
40 TB is a challenge to every backup software. We don’t use Veeam to backup this VM because this is linux oracle DB server and recently jump from AIX. So at moment we use old way to backup it – with backup agent integrated with Oracle RMAN, DB in archive mode - once per months full backup, everyday increments. But with new version of Veeam with Oracle support I am thinking about alternatives with pros and cons.
Zew
Veteran
Posts: 365
Liked: 80 times
Joined: Mar 17, 2015 9:50 pm
Full Name: Aemilianus Kehler
Contact:

Re: Feature Request: Multiple VBK files

Post by Zew »

Nice! I currently achieve this by having a seperate job per VM... it really makes things painful as I have so many jobs I have to manually configure. Glad to see this coming in V9.
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Feature Request > per-Disk chain

Post by dellock6 »

PHBG wrote:I am 100% agree that hardware (source/proxy/target) is the most important component of processing rate. But I am 100% sure that there is no company that backup storage is more powerful than the primary storage. Why EMC Data Domain, HP StorOnce or other Dedup solution used NL-SAS/SATA disk (7200)?
Because for these solutions, price per GB is usually the main acquisition factor, and it's why they also add deduplication. If you are talking about performance instead, you should look at some non-dedupe solution that can fully leverage the speed of the underlying disk system, and also consider possible ways to leverage Scale-out backup repository. Sorry for another shameless plug, I just published an example today: http://www.virtualtothecore.com/en/an-e ... x-servers/
PHBG wrote:Our primary storage is mix of SSD, SAS-10k, NL-SAS disks with Auto Tiering where 40-50% storage space is from capacity disk (NL-SAS). We have two instance of Data Warehouse – new on VMware (x86) - 40 TB mention above, and old one on AIX (Power7) – 50 TB. AIX instance is entirely on NL-SAS storage at the moment because of retirement. Now we are implementing HP StorOnce and make some backup test with AIX (SAN to VTL Backup LAN-free) and the speed was 1,5 GB/s due to parallel reading different file system and writing to different drivers. One VM (AIX) on a slow storage but backup speed was nice 1,5 GB/s to StorOnce due to parallelism.
So if you have a big VM on not very fast storage (from IOPS perspective), but able to generate enough disk rate/throughput you will not invest in more expensive storage for backup. So my point is that in equal situation where performance of primary storage = backup storage, you can read parallel from different vDisk/Datastore but not able to write to different veeam file/repository. So there are some limits in veeam backup software in this situation.
Right because the storage is hybrid, you should use forever forward incremental, so data are retrieved entirely only on day1. On following days, the incremental is going tipically to come out from the fast tier as the changed blocks "should" be the active working set. I say should because it heavily depends on the specific vendor and model, but usually this is a suggested practice. So the 40TB backup will happen only in the first day.

Indeed parallel processing is a source activity and not a target one, but with v9 you should see additional increase in target performance (so back to your point) thanks to Catalyst integration.

Your scenario is once I'd like to see with v8 vs v9 benchmarks right to see what improvements it can have.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
Post Reply

Who is online

Users browsing this forum: Google [Bot], srlarsen and 195 guests