run multiple simultaneous backup jobs (i.e. volume backups) against a single host

[cferraro]

is this possible? my field experience indicates it's not. if that's true, why isn't this possible? how do folks back up systems with large amounts of data (~20TB) daily?

-----------------------------------

my real issue is that, in my opinion, SOBR technology isn't really ready to provide RPO of less than a day at present if you don't have underlying storage resiliency (i work for a small firm, our primary file server is at our HQ with no recovery frame elsewhere). what i'm trying to do (in parallel to, idk, fixing the root storage problem) is backup a file server residing in the HQ site in the interim and achieve site resiliency and an RPO of 1 day using s3 as my capacity tier in the SOBR.

[HannesK]

Hello,
and welcome to the forums. I'm not sure how I can really help you. I mean I have several customers backing up 20TB VMs (one VM) or hundreds of TB with thousands of VMs. Of course that needs proper hardware.

Just to clarify because you talk about a primary file server. Are you talking about Veeam Agent for Windows backup or "normal" VMware VM backup?

Could you describe your hardware, configuration and speed you achieve?

Thanks,
Hannes

[cferraro]

the file server i am trying to back up is a Windows server running file resource manager. essentially a NAS on the cheap. the backup is an agent-based backup as the server is a physical one. what other information can i provide that's helpful here?

the B&R server is a Hype-V VM with 8vCPU and 32GB RAM, running 2012 R2.

the file server is a physical server running Windows 2008 R2 with 8CPU and 12GB RAM

i use a SOBR with primary storage on a local synology NAS with 1 day replication to object storage in AWS.

[HannesK]

Hello,
Thanks for the details. It helps in some parts. I moved your question to the "Veeam Agent for Windows" forum.

Which backup mode do you use (Entire System / Volume based / File based)?
The amount of data is 20TB?
Which speed do you get during backup to the Synology NAS? (this one is the most important question)
Is synthetic full backup configured in the backup job? (storage -> advanced -> backup -> create sythetic full)

Best regards,
Hannes

[cferraro]

i've tried entire system (wiating on a restore to load for over 3+ days now) and volume backup (run in serial vs parallel) ... neither works. backup to synology is ~100 MB/s. i am trying active fulls over synthetic but have used both methodologies.

[HannesK]

Hello,
I don't know how the Synology replication to AWS works, but that probably breaks the restore (like with any other external S3 gateway).

Entire Computer / volume backup configuration is fine, but does not run in parallel today. We are looking into enhance that in future.

Just to make sure: everything is on 10 GBit or more? Otherwise the 100MByte/s are the normal limit for for 1GBit SMB connections.

Best regards,
Hannes

[cferraro]

i'm using SOBR to send to the object store. so, no synology to s3 replication outside of what sobr uses. and not having the ability to back up multiple volumes on the same system is criminal.

[HannesK]

i use a SOBR with primary storage on a local synology NAS with 1 day replication to object storage in AWS.

we don't support this configuration today. You can only move old backups. No copy / replication today to S3. The only way you can move backups daily to S3 is reverse incremental backup mode. Reverse incremental backup mode is not available for agents. So I can only guess what your real configuration is.

running parallel jobs / volumes will not make it faster than 100MByte/s on 1Gbit links (you did not answer my question, so I assume that you have 1GBit)

How fast can you write with windows to your synology?

[cferraro]

what are you saying you don't support? i believe you are saying you don't support backup-copy jobs to object storage (check, i'm clear on that)

amy configuration:
- performance tier synology nas
- capacity tier s3
- scale out backup repository settings - keep 7 copies on disk, move copies older than 1 day to capacity tier

my belief is you certainly support that configuration. if that is not true, please let me know.

the network infra is 1Gb at present, with an upgrade to 10Gb in progress.

the fact is the veeam product handles backups slightly different than most leading brands w/r/t concurrent tasks on the same host. why is that?

[HannesK]

what are you saying you don't support?

we don't do any "copy" or "replication" to capacity tier. It is a "move" of inactive backup chains. The point is "inactive" See user guide validation process step one

Verifies whether data that is about to be offloaded belongs to an inactive backup chain.

Depending on your configuration (number of restore points) backups will be moved eventually. But not after one day, because your chain is not inactive after one day.


For your performance question: so the bottleneck is the network, not the software as far as I can see.