Backup agent for Linux servers and workstations on-premises or in the public cloud
Post Reply
ph801
Novice
Posts: 4
Liked: never
Joined: Nov 30, 2017 10:07 am
Contact:

Multiple backup jobs in parallel?

Post by ph801 » Nov 30, 2017 10:14 am

Hi,

Running Linux agent veeam-1.0.1.364-1.el7.x86_64 on CentOS 7, backing up to a VB&R server. It seems that I can only run one agent backup job at a time on the client. Is this a configurable option? I'd like to run multiple backup jobs in parallel.

Thanks.

PTide
Veeam Software
Posts: 4320
Liked: 355 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Multiple backup jobs in parallel?

Post by PTide » Nov 30, 2017 12:11 pm

Hi,

That is by design - you cannot run multiple jobs in parallel for the same linux host. Would you describe your use case please?

Thanks

ph801
Novice
Posts: 4
Liked: never
Joined: Nov 30, 2017 10:07 am
Contact:

Re: Multiple backup jobs in parallel?

Post by ph801 » Nov 30, 2017 1:55 pm

Hi,

Sure, this is a server that receives database backups from multiple database servers. Each database backup is received in a separate directory that belongs to it's own LVM LV, eg:

/backups/dbserver1 (/dev/backupvg/lvdbserver1)
/backups/dbserver2 (/dev/backupvg/lvdbserver2)
/backups/dbserver3 (/dev/backupvg/lvdbserver3)
/backups/dbserver4 (/dev/backupvg/lvdbserver4)
[..and so on..]

We then want to backup this data into VB&R.

I set up each mount point as its own "volume level backup" job in the Veeam client with different schedules because each backup is received at different times of day. However because these dumps are quite large it's not possible to schedule daily backups of all the database data within a 24hr window without some kind of overlap.

This server has plenty of CPU and network so overlapping backup jobs would not cause a problem there - it's just the Veeam client that is stopping it.

Thanks.

PTide
Veeam Software
Posts: 4320
Liked: 355 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Multiple backup jobs in parallel?

Post by PTide » Nov 30, 2017 2:18 pm

Can it happen so that the first bits of, say, DB2backup hit the disk but DB1backup transfer is still in progress? How big are the dumps? I assume all of them are full-dumps, not binary logs backups, is that correct? And another side question - what kind of database do you backup, I mean is it MySQL, MariaDB, Postgres?

Thanks

ph801
Novice
Posts: 4
Liked: never
Joined: Nov 30, 2017 10:07 am
Contact:

Re: Multiple backup jobs in parallel?

Post by ph801 » Nov 30, 2017 3:01 pm

Unfortunately I don't have much control of when the incoming DB dumps happen - this is often down to DB loads.

DB dumps vary in size from 50GB (smallest) to 1.8TB (largest), yes full dumps. Databases are a mixture of Oracle and HANA.

PTide
Veeam Software
Posts: 4320
Liked: 355 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Multiple backup jobs in parallel?

Post by PTide » Dec 09, 2017 2:47 pm

I think that I understand your use case, but I'd like to know how the dumps are delivered (scp, rsync, smth else) ? Unfortunately, the current version of VAL is unable to meet your requirement out of the box, so I consider this as a feature request.

Since you've mentioned that you don't have much control over the dumps, I'd like to add one more thing - it is important to ensure that every backup job starts not sooner than the dump file has completely arrived to the host. Otherwise, you may end up with one half of dump being in one backup file, and the other half of dump being in the other. If you already perform such check, please tell us how you do it. You could accomplish that by using custom crontab script that would perfrom the check and trigger a job to start, that is, you should not use Veeam scheduling wizard. Another option would be to use a pre-freeze script that would perform the same check before taking a snapshot.

Thanks

ph801
Novice
Posts: 4
Liked: never
Joined: Nov 30, 2017 10:07 am
Contact:

Re: Multiple backup jobs in parallel?

Post by ph801 » Dec 11, 2017 1:31 pm

Hi,

Yes sorry when I said about not having much control over the timings we still have a generally predictable window where we know we have a fully complete dump on disk. This is checked manually on a regular basis just by logging the completion times of the DB backups. The backups are sent over NFS, each of the mounts described above are NFS-exported.

If there is a "wish list" it would be good to have this as an option, thank you :)

PTide
Veeam Software
Posts: 4320
Liked: 355 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Multiple backup jobs in parallel?

Post by PTide » Dec 12, 2017 11:49 am

Actually, the forum itself is some sort of a wishlist :) For uses cases like yours it seems that a continuous file-level backup job that checks if there are new files, waits for them to finish being written to, and takes them to the destination would be a better match. Running several jobs in parallel might be really tough on the IO subsystem.

Thanks

Post Reply

Who is online

Users browsing this forum: No registered users and 3 guests