Order backup jobs before backing up to tape

[UT2015]

Hi there,

we have a backup to tape job, which backups a full backup of several source jobs to tape once a week. Our tape library has two drives. Of course we want Veeam to use both drives parallel to speed up the backup to tape.

Veeam does actually use both drives, but not in the best way. The source jobs have different sizes, there are smaller source jobs (<1GB) and bigger ones (>1TB). Currently Veeam processes those source jobs alphabetically. This leads to both drives processing some small jobs first. After that drive A starts processing a big job while B starts processing a medium one. Drive B finishes the medium job and starts backing up the biggest job after that. Drive A has nothing to do for several hours.

It would be *much* better if Veeam would calculate the best sequence of the source jobs when using multiple drives. Because Veeam actually knows the sizes of the source jobs (the backups are already on disk) it should be no problem at all to calculate a good sequence, e.g. one drive starting with the biggest job, so the other drive can handle all small and medium jobs. Is it possible to realize that feature?

If not it would at least be good to be able to change the sequence of source jobs manually. This is actually currently not possible. I cannnot rename the source jobs and I cannot change the sequence when editing the tape job. I could actually create a brand new tape job with a different sequence, but this would probably destroy my backup chain and would start with a full backup again

Any ideas?

Best regards,
UT2015

[lyapkost]

Hi.

Thanks for the requests. Calculating backup size is not always possible because of synthetic fulls and such smart jogic also requires changes in current mechanism of tape jobs so it is a tricky feature. Tape jobs' order change makes more sense and will be introduced in future releases. For now as a workaround try splitting the tape job into two different containing approximately equal backups.

[UT2015]

I tried splitting the tape job into two different jobs containing approximately equal backups. Because I don't want the increments to be split on two tapes (they fit on one tape), I now have:

- Media Pool Full Backups, parallel processing for tape jobs enabled
- Media Pool Increments, parallel processing disabled

I could actually combine both media pools again, because now actually two tape jobs are processed parallel (and not two source jobs as before), but I like having full backups in a separate pool.

I now have the following jobs:
- Full Backup to Tape #1 (some source jobs), incremental backup disabled, full backup scheduled for Saturday
- Full Backup to Tape #2 (some source jobs), incremental backup disabled, full backup scheduled for Saturday
- Incremental Backup to Tape (all source jobs), incremental backup enabled, full backup weekly on *no* day (will that work?)

The first thing that did not work: Both Full Backup-Jobs are scheduled to do the full backup only on Saturday. But when they started on Tuesday, they tried to do a full backup to tape! Why is that? The log file states "Backup doesn't contain any backupsets. Forcing full pass." - But this does not make sense. Incremental backup is disabled, full backup is scheduled for Saturdays!

Another strange thing/bug: When editing properties of a media pool and selecting "automatically create new media set daily at" the value "00:00" leads to the time being changed to 12:00 which is definately wrong/buggy.

So now I am forced to let the full backup jobs being disabled. I will enable them Friday evening so that they can actually perform the full backup Saturday morning and hope everything will work as expected...

Best regards,
UT2015

[lyapkost]

Hi.

One shouldn't confuse synthetic (or virtual) fulls on tape and full backups. It's possible to disable synthetic full backups by just setting source job to run with periodic fulls (forward incremental mode) but it's impossible to create backup to tape job which will copy only incremental backups skipping full ones because such backup on tape will be useless due to the impossibility of restore.

So in your case I suggest to create two jobs: the first one has fulls pointed to full_meida_pool1, incrementals to incremental_meida_pool, the second one to full_media_pool2 and again incremental_media_pool.

full backup weekly on *no* day (will that work?)

The option 'no' day shouldn't be available, that issue appeared in U1 and will be fixed in future releases. But jobs should work in that configuration.

Another strange thing/bug: When editing properties of a media pool and selecting "automatically create new media set daily at" the value "00:00" leads to the time being changed to 12:00 which is definately wrong/buggy.

We have time settings in 12-hour format in the product so 00:00 is 12:00 AM.

[UT2015]

We have time settings in 12-hour format in the product so 00:00 is 12:00 AM.

This is not true. The mentioned field has 24-hour format. When I set the field value to 23:59 and store it, still 23:59 is displayed. When I set the field to 00:00, 12:00 is displayed, which is actually a bug.

And what about that?

Both Full Backup-Jobs are scheduled to do the full backup only on Saturday. But when they started on Tuesday, they tried to do a full backup to tape! Why is that? The log file states "Backup doesn't contain any backupsets. Forcing full pass." - But this does not make sense. Incremental backup is disabled, full backup is scheduled for Saturdays!

[lyapkost]

This is not true.

We're again talking about different versions. Which version are you at?

And what about that?

More information is needed to answer the question. By 'they tried to do a full backup to tape' do you mean synthetic full or just copying vbk-file from repository to tape? What files were in repository and on tape at the moment when the tape job started?

[UT2015]

We're again talking about different versions. Which version are you at?

I am using B&R 9.5 U1. Mentioned field is editable in 24-hours-format. Field is also displayed in 24-hours-format except when editing 00:00, value 12:00 is displayed after saving. So you are saying in U2 24-hours-format was replaced by 12-hours-format?!

By 'they tried to do a full backup to tape' do you mean synthetic full or just copying vbk-file from repository to tape?

Both. To-tape-job should perform synthetic full of Endpoint backups and active full of vCenter environment. Both is scheduled for Saturdays and was performed on Tuesday.

What files were in repository and on tape at the moment when the tape job started?

On tape was nothing because the job was created and started for the first time. On disk were both vbks and vbis for the Endpoint backups and for the vCenter environment.



We have another problem: The full backup on weekend actually did not work as expected. Veeam actually needed twice the number of tapes than usual. When analyzing a logfile of the tape job, the following line seems suspect:

Code: Select all

[27.05.2017 11:59:50] <21> Info     [CTapeSourceBackupJob]Backing up vbk Backup Job DC012017-05-15T211018.vbk:88475858-0409-4ea7-a946-8f91a9c6b445:15.05.2017 21:10:18:15.05.2017 21:10:18 again because it's a first time backup!

At this time there is a 2017-05-15 vbk and several vbis (up to 26.05.). Veeam should actually do a synthetic full of 26.05., but actually stored vbk of 15.05. *and* synthetic full of 26.05...

Best regards,
UT2015

[lyapkost]

Field is also displayed in 24-hours-format except when editing 00:00, value 12:00 is displayed after saving.

Ok, I've confirmed it, will be fixed in future releases. Thank you. Time format in that field depends on your system settings and it was 12-hours in my lab that's why I didn't reproduced it for the first time.

Both is scheduled for Saturdays and was performed on Tuesday.

On tape was nothing because the job was created and started for the first time. On disk were both vbks and vbis for the Endpoint backups and for the vCenter environment.

If there's no full backup on tape than a job will first copy full and process increments afterwards.

We have another problem

To avoid copying previous backup chains which is happened in your case you need to check Process latest full backup chain only check box in job's Advanced settings.

[UT2015]

If there's no full backup on tape than a job will first copy full and process increments afterwards.

But don't you agree that it is unlogical, if a job is configured to perform full backup once a week on Saturdays and incrementals are disabled for the job, if the job is running on Tuesday and performs the full backup? I cannot see why this is a good behavior. It would be more logical if the job would do nothing on Tuesday and would do the full backup on Saturday.

To avoid copying previous backup chains which is happened in your case you need to check Process latest full backup chain only check box in job's Advanced settings.

You are right, the box is not checked. But it is also not checked in our "old" Backup-to-Tape-job. The files on disk did not change so it is strange that the new job seems to behave different to the old one. And I don't understand why this has influence on synthetic full backups. There is always one old full backup on disk and the to-Tape-job actually creates a new synthetic full...

Best regards,
UT2015

[lyapkost]

Again, tape job doesn't create synthetic full on Tuesday. But it started and found a full backup in repository which is not copied on tape yet. The job copies it. That's how backup to tape job works. Also notice that if there was no run on synthetic date for any reason the job will create synthetic full next time it runs (see user guide).

New job is not connected with an old one so it will copy backups to tape even if they're already copied by another job.

[UT2015]

So in your case I suggest to create two jobs: the first one has fulls pointed to full_meida_pool1, incrementals to incremental_meida_pool, the second one to full_media_pool2 and again incremental_media_pool.

I have those two jobs pointing to one incremental pool. That pool is configured to automatically create new media set daily at 00:01. Those jobs copy only increments, which is very little and which fits on a single tape (for both jobs). Sometimes those jobs are using the same tape, but sometimes each job uses a tape on its own. How can I force those jobs using one tape together?

[lyapkost]

If both tape jobs start between 00:02 and 23:59 they should place backups into the same media set and if tape size is more than sum of jobs' incremental backups' size (+ about 10% reserve) the both incrementals should be placed to the same tape. Are both these conditions fulfilled? If yes, do different tapes used by jobs which run on the same day have the same media set or different ones?

[UT2015]

Hi,

yes, both conditions are fullfilled. Both tape jobs start automatically after the To-Disk-Job has finished, which is at approx 00:20. Both incrementals fit onto one tape easily.

The media sets of the two jobs have the same media id in case one tape is used and different ones in case two tapes are used.

Tonight both jobs used one tape (as wanted). In that case the first tape job finishes with a warning: "Failed to eject some tapes. Skipping tape media xxxxxx eject because lock failed", which is logical, because the other tape job is still using this tape.

[lyapkost]

Hi. Is 'export' option also enabled for the first job? If yes, that's the cause. If not, only the eject shouldn't cause creating new media set but if I were you I would disable eject option in the first job anyway.

[UT2015]

Hi,

the "Export current media set upon job completion" option is disabled for both Jobs.

The "Eject media upon job completion" option is enabled for both jobs, because the GUI states it is best practice.

If not, only the eject shouldn't cause creating new media set

Well it does some days and some days not, which is very strange.

but if I were you I would disable eject option in the first job anyway

I do not know, which of both jobs is started first, because both are depending on a Backup-To-Disk-Job and start after this job finished. I cannot create a dependency between both tape-jobs, because I created those jobs to be executed in parallel (to use two tape drives and to store the disk backups in a certain order to tape, as described above).

[lyapkost]

I forgot about paralleling, the initial reason of creation two jobs. Then it's clear: if both jobs are simultaneously writing incremental backups, they use two drives and two tapes. Usually jobs start with several seconds delay and if backups are very small and have written within these several seconds by the first job the second one uses the same tape. So sometimes it happens and sometimes not. Try adding the following powershell (.ps1) pre-script to one of the jobs (you may vary the sleep time):

Code: Select all

Start-Sleep -Seconds 300

This will make one job to wait a bit before another finishes copying incrementals to tape and use the same tape afterwards.

[UT2015]

Then it's clear: if both jobs are simultaneously writing incremental backups, they use two drives and two tapes.

Why is that clear? I disabled option "Enable parallel processing for tape jobs using this media pool" in the incremental pool, therefore I would assume they would automatically be executed one after another (only for the incremental pool!).

And why is it sometimes working? E.g. last night:

- Tape Job #1: Start time 00:26:55, End time 00:45:43, Duration 0:18:48
- Tape Job #2: Start time 00:26:55, End time 00:49:16, Duration 0:22:20
- Both jobs have written the incrementals to the same tape!

...and some days ago:
- Tape Job #1: Start time 00:25:11, End time 00:44:17, Duration 0:19:06
- Tape Job #2: Start time 00:25:11, End time 00:34:02, Duration 0:08:51
- The two jobs have written the incrementals to two different tapes!

According to your explanation I should delay one job for approx. 30 minutes to be sure that it starts after the first one finished, correct? I assume the "Eject media" option may stay enabled.

[lyapkost]

Ok, not so clear as I thought. Tried but cannot confirm it in my lab (9.5 upd2). Try to set different backup jobs in tape jobs' 'start after' schedule or to set delay as mentioned (I think, 5 minutes is enough and 30 minutes to be sure). If it still won't help, please, open a support ticket for logs need to be investigated to find the reason of such behavior.