Backup Copy Job Questions

[JeffCloss]

Hi All,

We've been using Veeam for a little while for our School District. Currently, we are using B&R 9.5 update 4. We have a SAN in our datacenter with a couple of NAS's on it, and a second NAS at our offsite location. So far we have physical servers in each of our schools and are doing daily backups from the Agents to the Repository, and that is going fine. (In the future the physical servers will each be replaced with one or more VM's.)

However, for archival purposes, we want to keep Monthly backups and yearly backups. I've created backup copy jobs of all the agent backups to a second repository (we called it Long Term) and then Backup Copy Jobs from there to our offsite NAS repository (which we called Offsite, cleverly enough.)

Our retention policy looks like this:

Daily Backups with 7 restore points, forward incremental. One active full weekly on Saturdays.
Monthly Backups (1 per month) and 1 yearly backup.

(I understand this to mean that each Saturday a full backup will be created and the previous chain will be pruned, starting a new chain until the next active full)

We do realize this creates up to a 3 week period of non-recovery. Currently, that works for us, but as our budget allows us to expand our SAN we would like to have daily backups with a retention policy of up to 30 restore points. (ie Daily for a Month, Monthly for a year, Yearly for 2 years)

My main question has to do with the backup copy jobs screen. On the Target page we have ticked "Keep the following restore points for archival storage" and have filled in 12 monthly and 1 yearly. However, in the section above it says "Restore Points to Keep". That will not let me select 0 (as we only want the restore points specified in the boxes below- as I read the screen it appears to be an "or" statement) it will only let me go down as far as 2.

This has become an issue as our long term repository has run out of space (we are in the process of obtaining more disk). When I browse the file system there appear to be some .vbk files (which I'm assuming are our monthly restore points) and a bunch of .vib files (these seem to correspond to our previous settings which specified "Restore Points to Keep as 6) I'm not sure the purpose of the incremental .vib files.

I expect Veeam will auto-prune those .vib files down to 2 when the jobs can run successfully, but I want to make sure that I'm reading the settings correctly. I'm worried that by having set "Restore Points to Keep" to 2, it will apply to the checkboxes below and will only keep 2 monthly backups. Also, I'm not sure why there are incremental jobs being retained when all we want to do is keep up to 12 monthly full backups and 1 yearly backup from the previous year. Can anyone explain this page to me in more detail? I haven't been able to find a full explanation of those settings.

Not helping to clarify my questions/problems is that we haven't had much in the way of successful monthly backups as we have had problems with storage (we're using far more than the storage calculator lead us to expect) and so I only have 2 successful monthly backups per job (I'm assuming from how I'm looking at it) to go by.

Questions? Please ask away - this is quickly becoming my most prioritized project as the start of school next week looms over us.

[ejenner]

I guess you're talking about this screen:



It could be confusing, but I don't think it is.

If the 'Restore points to keep:' is set to 2 then that means the copy job can only be restored to yesterday or the day before. That's assuming you're creating one restore point every 24 hours.

Any Saturday, any First Sunday of the month, any First Sunday of the quarter and any First Sunday of the year will have either the full or the incremental of those 2 restore points available on disk at any time. So when it wants to retain an a restore point for archival purposes it will either take the newly created full if that has happened on the same day or it will have to combine the full and the incremental if it happens to fall on that day instead.

The advantage of your configuration is that there won't be many incrementals taking up space on copy repository. The disadvantages are that you'll be using a lot of processor for recombining incrementals and fulls every couple of days (perhaps that's a tradeoff you can easily accommodate?) but the more worrying disadvantage is that if your primary backup is unavailable and you have to go back more than one day (perhaps a disaster occurred on Friday and you don't rectify until Monday?) then your copy repository may not hold the copy you want. You'll have to back to the most recent archival point, if you have one.

I think the simplest way to look at it is that the 'Restore Points to Keep' box and the next 4 boxes for the archival settings could be thought of as separate.

[JeffCloss]

OK, that's exactly the screen I was thinking of.

(I would use bbcode, but my borrowed french/english keyboard is garbage and I don't have square brackets right now....)

Quote:
The advantage of your configuration is that there won't be many incrementals taking up space on copy repository. The disadvantages are that you'll be using a lot of processor for recombining incrementals and fulls every couple of days (perhaps that's a tradeoff you can easily accommodate?) but the more worrying disadvantage is that if your primary backup is unavailable and you have to go back more than one day (perhaps a disaster occurred on Friday and you don't rectify until Monday?) then your copy repository may not hold the copy you want. You'll have to back to the most recent archival point, if you have one.
End Quote.

That pretty much makes sense to me - in fact, ideally, in our situation, we would *only* have archival restore points, and not incrementals from the copy job. My understanding is that each archival copy is not created from previous backup copies, but synthesized separately from the original backups.

Our intention with the backup copy jobs is to have restore points (1 per month) in case we ever have to go back that far. Regularly we will have 7 days retention (eventually 30), but if a teacher approaches me and says "last month I had a file and I can't find it again...." I can go to the archival copies and restore from there. With our previous tape backup system we very rarely, never in the last 6 years, had to go back more than 7 days - but it could happen. We're willing to lose up to 4 weeks of intermediate data (if we had a disaster on the last day of the month and our last 7 days backup did not work, we would have an archival one from the end of the last month. (These archival points are also duplicated on our offsite NAS, where it might make sense to have more restore points in case of disaster at the primary site, using those restore points - as that NAS isn't as over subscribed)

Do you know if there is a way to make it so that the backup copy job does not constantly make copies of the current backup, but only synthesizes the archival backups? That way I wouldn't be wasting processor time on creating a backup that isn't part of our plan.

I was most worried that the two sections *were not* separate, even though we had assumed them to be, and that by lowering the number of restore points it would not be able to keep the specified number of archival points.

[ejenner]

I'd say the processing for the copy isn't going to be reprocessing the entire backup but rather recombining increments with fulls to keep it going. But if you have only 2 copies it will have to recombine the files every other day (assuming a 24-hour schedule)

The biggest flaw in your plan is the number of days which are recoverable. You've got to try and relate to likely real world scenarios. For instance, people will often take a holiday which lasts 2 weeks. If they made some big mistake shortly before going on holiday and want to be able to rectify the problem a day or two after returning to the office you may not be able to help. They may have to begin again from scratch. Similarly, if you patch systems on a weekly cycle and your backup has only 7 days worth of restore points you may end up with a problem which occurred where you must be able to recover from more than 7 days ago. Or if corruption occurred in a database and wasn't picked up soon enough... You'd be better off having more resource dedicated to more current backups and less for monthly archives. But it's up to you.

[foggy]

That pretty much makes sense to me - in fact, ideally, in our situation, we would *only* have archival restore points, and not incrementals from the copy job.

Due to its synthetic nature, backup copy job doesn't allow setting less than 2 restore points for the 'regular' part of the chain. This doesn't affect your GFS settings (weekly, monthly, yearly), they have their own retention.

My understanding is that each archival copy is not created from previous backup copies, but synthesized separately from the original backups.

Actually, the opposite - by default backups are synthesized unless you enable the 'Read from source...' check box.