Use the last syntetic full instead of synthesizing (GFS)

[mkretzer]

Hello,

we create Synthetic Fulls every week with the REFS integration.

Furthermore, we use a GFS tape pool.

With GFS backup most of the time the backups can not keep the tape streaming. With normal backups (non-GFS) this is no problem and we get 2-3x the write speed - even with the REFS synthesized fulls.

Why is there no option to just use the last full on disk instead of creating a "tape synthetic full"? The speed difference is there even if there is only 1-2 incrementals on disk...

Markus

[veremin]

The idea here is that GFS point should be synthesized on the exact day (usually controlled by internal or external strict policy). If there is a full backup created on that day, it will be archived to tapes as is (without any synthetic logic involved). Thanks.

[mkretzer]

But i think this is not that important for many customers.

Perhaps when we finished our move to REFS repo we can do all the merges the day before the tape backup.....

[veremin]

But i think this is not that important for many customers.

I wouldn't say so, preliminary investigation showed that quite often GFS dates are implied by strict polices. So, we had to implement mechanisms guaranteeing policy abidance.

Anyway, tweaking dates so that day selected for GFS coincides with full backup day should allow you to achieve your goal.

Thanks.

[mkretzer]

If you have a storage with limited performance (non-SSD), but a fast tape drive the current logic just makes things harder for the storage.

Our idea was to do all synthetics on one day so we have the synthetics all ready for the GFS backup the following night. But that also does not work because GFS wants to have the files from the *same* day as the GFS backup is scheduled (which always starts at 0:00 on that day). That means we would have to schedule the GFS backup for the day the synthetics are generated. But this way we are putting load on the backup storage which it needs to create synthitics. There is no way to win...

Markus

[veremin]

I might have missed that, but why not to schedule both synthetic full and GFS tape job on the same day? Being executed at 00:00, GFS job will sit and wait for a new restore point to appear. As soon as synthetic fulls are created, they will be archived to tapes by the GFS job. Thanks.

[mkretzer]

Because that means even more load for the storage.

We have + 20 backups. All get backed up in one GFS tape job.

Creating the synthetic fulls take 7 hours with REFS so in theory this could run in one day (we will test that). Before with NTFS it took a total of 106 hours so only with REFS this is an option. But still on that day the backups will take 12-14 hours to get the incrementals and then creating the synthetics.

Now if at the same time our tape drive will start reading from the storage (after the first backup finished) the load might be so high that in total the backups will take more than 24 hours...

If we had SSD in our backup storage this would be no issue but with any kind of HDD this will be a problem for the time beeing. So we need to solve that. And the simplest way to do this is just writing the last full to tape even with GFS!

This feature should be quite easy to implement will it not?

Markus

[veremin]

I think I can propose a workaround for you.

There is a regkey controlling timeout after which GFS tape job decides to look at previous day for restore point candidate.

TapeGFSBackupWaitTimeout <DWORD> (value should be set in minutes)

- Add a key
- Set a small value, say, 1 minute or so
- Schedule the tape job to take place one day after source jobs create full backup

This way, the tape job will be executed on the given day, wait for 1 minute and starts looking at previous day for point to copy. Then, it will discover a full backup and archive it to tape accordingly.

Thanks.

[mkretzer]

Very nice. Do we need a restart for that? What is the default?

[mkretzer]

I thought about this and i am not sure that will help. It might help for the first object in the backup but what about the next object when there is a new incremental avaiable?

[veremin]

Do we need a restart for that?

No.

What is the default?

24 hours.

It might help for the first object in the backup but what about the next object when there is a new incremental avaiable?

If no increments appear within one minute interval, the job will be copying previous fulls, paying no attention to newly appeared source backups.