Feature Request: Always copy the latest Full Bakcup to Tape

Everything about backing up to tape

Feature Request: Always copy the latest Full Bakcup to Tape

Veeam Logoby staskorz » Wed Aug 09, 2017 4:27 pm

Hello,

This is a follow-up topic for Support Case ID 02210537.

TL;DR;

We need a way to make Veeam Backup and Replication to be able to ALWAYS copy full VM backups to tapes. Currently, full backups are copied to tapes ONLY IF disk backup job starts and completes the same day, before 00:00. The Support Representative said it’s by design…

Full Version:

At first glance there was nothing special about our backup requirements, it's just a plain Backup-to-Disk-to-Tape scenario. We've created a backup job to backup some VMware VMs, and a Tape job to copy the latest full VM backups to tape. The Tape job is set to run after the source VM-to-Disk job completes.

The thing is we've noticed sometimes VMs were not copied to tapes, with a log message saying there were no new backups to process, which of course was not the case. Furthermore, the job didn’t fail, it completed successfully, but without actually copying anything to tapes. Opening a support case at Veeam resulted in a surprising reply saying it’s by design – and here is the explanation we’ve got: the tape job will copy full VM backup ONLY IF they were taken during the same daily cycle, which starts at 00:00.

There are several issues with such an approach:
1.Backups are usually scheduled to off-hours – and 00:00 is about in the middle of the off-hours period
2.It’s impossible to predict the duration of a backup job, as it can be affected by numerous factors, such as unexpected load on underlying production or backup infrastructure, a CBT reset (say after changing VM disk size) resulting in a full disk re-read, etc.
3.Global organizations might have different daily cycle boundary preference than the local 00:00 time of their backup server

Our current workaround, which is far from perfect, is to split the jobs so those starting before 00:00 will, with high probability (???), finish the same day. The thing is, one would expect a far more deterministic approach to her backup setup… I mean restoring from a backup is your last hope to gain your systems and data back would anything go seriously wrong – and as such it MUST be rock-solid, not depending on uncontrolled external factors. Another issue with our workaround is the backup duration – as described, we had to split the jobs: those that have high probability to finish before 00:00 are scheduled to start at evenings, leaving the longer ones to start after 00:00. As a result, the backup now takes far longer to complete and does not finish on time.

Here are the possible solutions I’ve proposed to the support representative:
1.Add an option to configure the daily cycle boundary, defaulting to 00:00 – this way nothing will change for those not interested
2.Add an option to force the tape copy job to ignore the daily cycle boundary, so it will always copy the latest full backup to tapes – same here, leaving this option off by default means nothing will change for those not interested

The second proposed solution seems to be better than the first one, because it IS deterministic, while the first one IS NOT. The thing is no one can tell for sure a job won’t take more than 24 hours to complete – say due to an unusual load on the underlying infrastructure or a planned change to sizes of several large VM disks – an operation causing CBT reset followed by a full disk re-read – and those are just two of many possible causes for a backup job to take way longer than usual to complete.


Thanks and Best Regards,

Stas.
staskorz
Novice
 
Posts: 6
Liked: never
Joined: Sun Aug 06, 2017 10:12 am
Full Name: Stas Korzovsky

Re: Feature Request: Always copy the latest Full Bakcup to T

Veeam Logoby v.Eremin » Wed Aug 09, 2017 4:58 pm

Basically, you want to archive latest available full backup on everyday basis, regardless of whether it's been archived previously or not? What mode does a source backup job use? Reversed incremental, forward incremental with periodic full or forward forever incremental (which I believe is not the case, since you mentioned periodic full backup, but it's still worth double checking)? Thanks.
v.Eremin
Veeam Software
 
Posts: 13734
Liked: 1027 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Feature Request: Always copy the latest Full Bakcup to T

Veeam Logoby staskorz » Wed Aug 09, 2017 7:59 pm

As far as I understand, when Veeam needs to write a "Full Backup" to a tape, the tape copy job synthetically produces a full backup from the the latest incremental available in a backup chain (please correct me if I'm wrong) - which is exactly what we need.

Regarding the backup mode: we use forever forward incremental without periodic fulls. By "copy full backup to tape" I meant "synthetically produce a full backup and copy it to tape".

Thanks!
staskorz
Novice
 
Posts: 6
Liked: never
Joined: Sun Aug 06, 2017 10:12 am
Full Name: Stas Korzovsky

Re: Feature Request: Always copy the latest Full Bakcup to T

Veeam Logoby v.Eremin » Thu Aug 10, 2017 9:46 am

Got it, in this case all you need to do is to select all week days as dates for virtual full backup ("Schedule" button on "Full backup" settings page).

With such configuration whenever a backup to tape job runs, it will synthetically create a full backup directly on tapes.
v.Eremin
Veeam Software
 
Posts: 13734
Liked: 1027 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Feature Request: Always copy the latest Full Bakcup to T

Veeam Logoby staskorz » Thu Aug 10, 2017 1:11 pm

The thing it ONLY works IF the Backup-to-Disk job completes the same day (before 00:00). If, say, the job starts at 22:00 and completes at 00:05 it doesn't work - the Tape job says there are now new backups to process. You can take a look at the logs attached to Case ID 02210537. Thanks!
staskorz
Novice
 
Posts: 6
Liked: never
Joined: Sun Aug 06, 2017 10:12 am
Full Name: Stas Korzovsky

Re: Feature Request: Always copy the latest Full Bakcup to T

Veeam Logoby v.Eremin » Thu Aug 10, 2017 3:25 pm

In this case why not to make backup to tape job run during the day (say, 3 pm or something) , instead of scheduling it after the source backup job? This way, whenever executed the backup to tape job will create virtual full backup without skipping anything. Thanks.
v.Eremin
Veeam Software
 
Posts: 13734
Liked: 1027 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Feature Request: Always copy the latest Full Bakcup to T

Veeam Logoby staskorz » Thu Aug 10, 2017 5:21 pm

The requirement is to take the Backup-to-Disk after 19:00 (after the day shift ends), and finish copying to tapes around noon, so a courier could pick up the backup tapes and ship them to an offsite location.

In any case, even if we do it as you say and start the backup job during the day (which does not meet our requirements), no one can tell for sure the backup-to-disk job will actually finish before 00:00 - say due to a CBT reset or because of additional load on underlying infrastructure.

Thanks.
staskorz
Novice
 
Posts: 6
Liked: never
Joined: Sun Aug 06, 2017 10:12 am
Full Name: Stas Korzovsky

Re: Feature Request: Always copy the latest Full Bakcup to T

Veeam Logoby v.Eremin » Fri Aug 11, 2017 11:43 am

I was talking about starting tape job during the day, not the backup one. Once executed, it will find restore point created on previous day (or on current day at 00-something - doesn't matter) and will use it as a source for virtual full backup.

The desired scenario will be achieved - one virtual full backup a day.

Thanks.
v.Eremin
Veeam Software
 
Posts: 13734
Liked: 1027 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Feature Request: Always copy the latest Full Bakcup to T

Veeam Logoby staskorz » Fri Aug 11, 2017 12:18 pm

As I said, the backup tape media is taken offsite. The courier comes to pick up the tapes before noon. The obvious goal is to handle the courier the freshest backup copy possible - if we start copying to tapes the day before, then the data on tapes sent to an offsite location is already over 30 hours old.

But generally speaking, I don't really get it: I mean Veeam Backup and Replication is a backup software, isn't it?? So it's supposed to backup. To ALWAYS backup. Not to think whether it should backup or not, but to always backup when instructed so. You guys try to find workarounds for a ridiculously simple classic scenario - just plain Backup-to-Disk-to-Tape. Why does the source backup start/completion times or duration can at all affect Veeam's decision whether to copy to tapes or not??? Sorry, but it's just ridiculous!

Really, it's not just me, when I tell other IT guys working for various organizations about this ridiculous limitation, their first reaction is shock - the second is laughter.

Veeam Backup and Replication is a great backup software, I just don't get it how the Product Management allowed such a limitation to slip in.

Sorry, it just frustrates me a lot.

Thanks.
staskorz
Novice
 
Posts: 6
Liked: never
Joined: Sun Aug 06, 2017 10:12 am
Full Name: Stas Korzovsky

Re: Feature Request: Always copy the latest Full Bakcup to T

Veeam Logoby v.Eremin » Fri Aug 11, 2017 12:40 pm

As I said, the backup tape media is taken offsite. The courier comes to pick up the tapes before noon.

Then, start a backup to tape job before noon, say, at 5 am. The virtual full will be created, based on the latest available restore point.

Virtual full backup is created once a day, not frequently - this is currently by design behaviour. If don't you find that applicable to your scenario, consider switching backup mode.

However, we do appreciate your feedback, and will think how current experience might be improved in one of the next product versions.

Thanks.
v.Eremin
Veeam Software
 
Posts: 13734
Liked: 1027 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Feature Request: Always copy the latest Full Bakcup to T

Veeam Logoby staskorz » Fri Aug 11, 2017 1:45 pm

Then, start a backup to tape job before noon, say, at 5 am. The virtual full will be created, based on the latest available restore point.


Not enough time to complete copying everything to tapes before noon.

Thanks.
staskorz
Novice
 
Posts: 6
Liked: never
Joined: Sun Aug 06, 2017 10:12 am
Full Name: Stas Korzovsky

Re: Feature Request: Always copy the latest Full Bakcup to T

Veeam Logoby Absalom » Wed Aug 16, 2017 9:19 pm 1 person likes this post

Just want to say "me too". We only do monthly backups to tape but sometimes they fail because of a bad tape etc and it's really frustrating that I can't just tell the backup to run again because there are no new backups to process. To compound the issue, we don't copy to tape directly from backup - we do it from the backup copy. The backup copy only runs every few days so I have to wait a few days before retrying the backup. Would be really nice to have a tick box or something to force it to copy the latest backup to tape.

-Aaron
Absalom
Lurker
 
Posts: 2
Liked: 1 time
Joined: Wed Aug 16, 2017 8:36 pm
Full Name: Aaron Dalgleish

Re: Feature Request: Always copy the latest Full Bakcup to T

Veeam Logoby v.Eremin » Thu Aug 17, 2017 3:08 pm

Monthly backups? Are you using GFS backup to tape job for that? If so, you can always make it create a gfs restore point immediately (right-click - active full - monthly). Thanks.
v.Eremin
Veeam Software
 
Posts: 13734
Liked: 1027 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Feature Request: Always copy the latest Full Bakcup to T

Veeam Logoby Absalom » Thu Aug 17, 2017 9:15 pm

Yes, we are using GFS. Wasn't aware of that feature so thanks for that :)
Absalom
Lurker
 
Posts: 2
Liked: 1 time
Joined: Wed Aug 16, 2017 8:36 pm
Full Name: Aaron Dalgleish

Re: Feature Request: Always copy the latest Full Bakcup to T

Veeam Logoby v.Eremin » Mon Aug 21, 2017 4:06 pm

You're welcome. If this tasks needs to be automated, you can leverage our PowerShell snap-in for that:

Code: Select all
asnp VeeamPssnapin
Sync-VBRBackupToTapeJob -Job (Get-VBRTapeJob -Name "Name of your tape job") -Monthly

Thanks.
v.Eremin
Veeam Software
 
Posts: 13734
Liked: 1027 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Next

Return to Tape



Who is online

Users browsing this forum: No registered users and 1 guest