Standalone backup agents for Linux and Unix workloads on-premises or in the public cloud
Post Reply
ph801
Novice
Posts: 4
Liked: never
Joined: Nov 30, 2017 10:07 am
Contact:

Multiple backup jobs in parallel?

Post by ph801 »

Hi,

Running Linux agent veeam-1.0.1.364-1.el7.x86_64 on CentOS 7, backing up to a VB&R server. It seems that I can only run one agent backup job at a time on the client. Is this a configurable option? I'd like to run multiple backup jobs in parallel.

Thanks.

PTide
Product Manager
Posts: 5650
Liked: 553 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Multiple backup jobs in parallel?

Post by PTide »

Hi,

That is by design - you cannot run multiple jobs in parallel for the same linux host. Would you describe your use case please?

Thanks

ph801
Novice
Posts: 4
Liked: never
Joined: Nov 30, 2017 10:07 am
Contact:

Re: Multiple backup jobs in parallel?

Post by ph801 »

Hi,

Sure, this is a server that receives database backups from multiple database servers. Each database backup is received in a separate directory that belongs to it's own LVM LV, eg:

/backups/dbserver1 (/dev/backupvg/lvdbserver1)
/backups/dbserver2 (/dev/backupvg/lvdbserver2)
/backups/dbserver3 (/dev/backupvg/lvdbserver3)
/backups/dbserver4 (/dev/backupvg/lvdbserver4)
[..and so on..]

We then want to backup this data into VB&R.

I set up each mount point as its own "volume level backup" job in the Veeam client with different schedules because each backup is received at different times of day. However because these dumps are quite large it's not possible to schedule daily backups of all the database data within a 24hr window without some kind of overlap.

This server has plenty of CPU and network so overlapping backup jobs would not cause a problem there - it's just the Veeam client that is stopping it.

Thanks.

PTide
Product Manager
Posts: 5650
Liked: 553 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Multiple backup jobs in parallel?

Post by PTide »

Can it happen so that the first bits of, say, DB2backup hit the disk but DB1backup transfer is still in progress? How big are the dumps? I assume all of them are full-dumps, not binary logs backups, is that correct? And another side question - what kind of database do you backup, I mean is it MySQL, MariaDB, Postgres?

Thanks

ph801
Novice
Posts: 4
Liked: never
Joined: Nov 30, 2017 10:07 am
Contact:

Re: Multiple backup jobs in parallel?

Post by ph801 »

Unfortunately I don't have much control of when the incoming DB dumps happen - this is often down to DB loads.

DB dumps vary in size from 50GB (smallest) to 1.8TB (largest), yes full dumps. Databases are a mixture of Oracle and HANA.

PTide
Product Manager
Posts: 5650
Liked: 553 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Multiple backup jobs in parallel?

Post by PTide »

I think that I understand your use case, but I'd like to know how the dumps are delivered (scp, rsync, smth else) ? Unfortunately, the current version of VAL is unable to meet your requirement out of the box, so I consider this as a feature request.

Since you've mentioned that you don't have much control over the dumps, I'd like to add one more thing - it is important to ensure that every backup job starts not sooner than the dump file has completely arrived to the host. Otherwise, you may end up with one half of dump being in one backup file, and the other half of dump being in the other. If you already perform such check, please tell us how you do it. You could accomplish that by using custom crontab script that would perfrom the check and trigger a job to start, that is, you should not use Veeam scheduling wizard. Another option would be to use a pre-freeze script that would perform the same check before taking a snapshot.

Thanks

ph801
Novice
Posts: 4
Liked: never
Joined: Nov 30, 2017 10:07 am
Contact:

Re: Multiple backup jobs in parallel?

Post by ph801 »

Hi,

Yes sorry when I said about not having much control over the timings we still have a generally predictable window where we know we have a fully complete dump on disk. This is checked manually on a regular basis just by logging the completion times of the DB backups. The backups are sent over NFS, each of the mounts described above are NFS-exported.

If there is a "wish list" it would be good to have this as an option, thank you :)

PTide
Product Manager
Posts: 5650
Liked: 553 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Multiple backup jobs in parallel?

Post by PTide »

Actually, the forum itself is some sort of a wishlist :) For uses cases like yours it seems that a continuous file-level backup job that checks if there are new files, waits for them to finish being written to, and takes them to the destination would be a better match. Running several jobs in parallel might be really tough on the IO subsystem.

Thanks

mamosorre84
Service Provider
Posts: 186
Liked: 15 times
Joined: Oct 24, 2016 3:56 pm
Full Name: Marco Sorrentino
Location: Ancona - Italy
Contact:

Re: Multiple backup jobs in parallel?

Post by mamosorre84 »

Hi all,

I've discovered this post because I have a similar scenario:

- 1 linux agent
- 2 nfs mounts from a qnap share (oracle export and oracle rman)
- 2 backup jobs to a cloud connect repository

I have to run 1 backup job daily, and one backup job every 3 days.

The problem is that this jobs can overlap..do I have to deploy another VM with VAL? Is there no way to force multiple jobs run in veeam.ini config?

Thanks

Marco S.

mamosorre84
Service Provider
Posts: 186
Liked: 15 times
Joined: Oct 24, 2016 3:56 pm
Full Name: Marco Sorrentino
Location: Ancona - Italy
Contact:

[MERGED] VAL - concurrent jobs

Post by mamosorre84 »

Hi,

is it possible to modify VAL settings (i.e. veeam.ini) in order to run two or more jobs at the same time?

Thanks

Marco S.

Vitaliy S.
Product Manager
Posts: 24329
Liked: 1880 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Multiple backup jobs in parallel?

Post by Vitaliy S. »

Hi Marco, this functionality is not available today, even with the config file. Thanks!

mamosorre84
Service Provider
Posts: 186
Liked: 15 times
Joined: Oct 24, 2016 3:56 pm
Full Name: Marco Sorrentino
Location: Ancona - Italy
Contact:

Re: Multiple backup jobs in parallel?

Post by mamosorre84 »

Hi Vitaliy,

thank you for the reply.

Marco S.

Post Reply

Who is online

Users browsing this forum: No registered users and 2 guests