Backup files not deleted according to retention policy

[JorisK]

hi all,
had the same situation. Our problem was that the day we chose to compact the incrementals (saterday) was NOT in the schedule for the job (only mon-fri).One we added satuday problem was solved. Kind of strange however that veeam lets you create the job this way. Would be nice to have a check implemented that avoids this kind of errors.
pim

Help!

We have exactly the same backup! Where can i setup the day to compact our backups? At this moment i configured 5 restore points and backups run from mo-fri!

[Vitaliy S.]

You need to pick up any day from Monday to Friday using Advanced Settings of the backup job. See our User Guide (page 62) for additional instructions.

[JorisK]

You need to pick up any day from Monday to Friday using Advanced Settings of the backup job. See our User Guide (page 62) for additional instructions.

I See.

By default Saturday is configured as the day the backups should be compacted. My backup runs tonight at 18:00 and could be done before or after 0:00, so should i configure wensday or thursday as day to compact the backup data?

[Vitaliy S.]

If your job starts on Wednesday, then you should choose Wednesday to transform your backup chain, as only job start time matters here.

[JorisK]

If your job starts on Wednesday, then you should choose Wednesday to transform your backup chain, as only job start time matters here.

Vitaliy, is there a way to perform a 'compact' of the backup data by using commandline?

[Gostev]

Compact operation is not currently available. Thanks.

[habibalby]

[merged]

I'm still a bit confused on how to configure the retention policy. Before Veeam backup, I used to use the Backup Exec, Weekly Full, Daily incremental and Monthly Full Backup.

Now I do have Veeam as well as Backup Exec, What I want to achieve is to have Veeam to run Weekly as Full Backup and starting from Sunday towards Thursday as incremental. Backup Exec will backup the VBK and VIB files on a daily basis.

For how long shall I configure the retention policy? I want to have the full backup and daily backup of veeam to be backed up by Backup Exec for Off-Site storage and to save space as well when the retention policy over.

Thanks,

[Vitaliy S.]

Hello Hussain,

Basically, there are two backup modes available: reversed and forward incremental. In both modes you should configure retention policy to keep 7 restore points, but the number of roll backs located on the disk will vary based on the backup mode you've chosen. See the details below:

Reversed Incremental:
With retention policy set to 7, you will have 1 VBK file (the most recent backup state) + 6 VRB files (incremental changes)

Forward Incremental:
With retention policy set to 7, you will have 1 or 2 VBK files (full backups) and up to 14 VIB files in total (daily changes) during two weeks. Here is a good explanation why it works this way, check it out.

Thank you.

[habibalby]

Hello Vitaliy,

So, the Reversed Incremental turns to be a better approach when selecting Backup-To-Disk and Then Backup-to-Tapes. Because this reducing the number of files needed on disk and keep only the retention files needed on disk for Restore From Disk Purpose.

For my needs, it's the best to select this approach. I'll be doing the following;

Fri - Full Backup Veeam.
Sat Full Backup of Symantec.
Sun-Thu Revered Incremental of Veeam Backup
Sun-Thu at night, Incremental Backup of Symantec.

This will give me a 7 restore points I can go back to till it starts the backup for the next day. If the backup starts and deletes the previous day backup, I can get it only from tapes. Will this be good to go?

Thank you,

[tsightler]

With reverse incremental, the VBK is always the most recent backup, so if you're going to tape every night, you have to backup the entire VBK file to tape each not. I think this is not what you want.

[Vitaliy S.]

For tape backup I would recommend using forward incremental backup mode, that would allow you to drop only incremental backup files to tape everyday.

[habibalby]

Hi,
I have a problem in selecting the Forward Incremental in regards with disk space. I set all the jobs to;

Retention Policy:
Restore Points to keep on disk 6 Days.

VM Retention:
Deleted VMs data retention period 6 Days.

Active Full Backup:
Weekly on Friday.

This leaves with 2 VBK files and 5 VIB files for the daily incremental from Sun-Thursday. and today it will start running as Incremental till thursday which I'm already lack of disk space and I don't know what to do.

How can I configure the jobs to have?

1 VBK File of last Full Backup "Friday"
5 VIB File of Sun-Thur

I want to do this configuration because I want to run Symantec also in the same manner, tapes for daily contains VIB incremental backups of Veeam and other set of tapes for Weekly and Monthly contains VBK Files of veeam.

Would this be do achievable?

Thanks,

[habibalby]

For tape backup I would recommend using forward incremental backup mode, that would allow you to drop only incremental backup files to tape everyday.

With Reversed Incremental Backup will not allow me to drop only the VRB "Incremental Files"? Symantec Backup will run full and will Reset the Archive Bit on the files, so when Archives Bit reset from the last Full Backup, and on Sunday Symantec Backup runs as Incremental it will includes only the VRB files of the Veeam!!! Am I write? or it will include also the VRB files and VKBs?

Thanks,

[habibalby]

Hello,
I re-configured one of the job from Incremental 7 Retention Days to Reversed Incremental 5 Retention Days. I ran the job for 5 times and it deleted all the VIB files and created 5 VRB Files including 1 VBK File.

Does this imply to my logic of the number of the Retention Days on Disk?

Before Reversed Incremental:
RAD_b4_Reversed_Incremental


After Reversed Incremental:
RAD_after_Reversed_Incremental.jpg:

[Vitaliy S.]

This leaves with 2 VBK files and 5 VIB files for the daily incremental from Sun-Thursday. and today it will start running as Incremental till thursday which I'm already lack of disk space and I don't know what to do.

You need to have more space if you want to proceed with forward incremental backup mode.

How can I configure the jobs to have?
1 VBK File of last Full Backup "Friday"
5 VIB File of Sun-Thur

With forward incremental backup mode you will always have more than 1 VBK file, so this is simply impossible to achieve with the forward incremental backup mode.

With Reversed Incremental Backup will not allow me to drop only the VRB "Incremental Files"?

With reversed incremental, incremental data is rolled into the full backup (VBK) file during each job run, so you would have to offload the full backup file (VBK) to tape every day, because this is the file that will contain your latest VM state.

Does this imply to my logic of the number of the Retention Days on Disk?

VRB (incremental) files created by reversed incremental mode do not contain recent VM changes, as this data is stored in VBK file.

[habibalby]

Hi,
Then this leaves me with a big Dilemma then I can't configure Veeam to backup full on every Saturday and Daily incremental from Sun-Thursday because it will keep two VBK files of 700 GB. Nor I can't run the Reversed Incremental to run on Saturday Full and from Sun-Thursday as incremental to dump it daily on tape. Then what is the perfect solution for that Veeam to dump Backup files on tape?

What is the best logic to retain disk space and daily backup to tapes as well as backup on disk for only 5 Working Days and 1 Full backup?

[Vitaliy S.]

There is no perfect solution and it's not about Veeam, this is how incremental mode and reversed incremental backup modes work. If your backup window allows offloading VBK files each day, then I would proceed with reversed incremental. Otherwise, you need to free up more space on the target storage and use forward incremental mode, that's it, pretty simple

[habibalby]

I can see that Reversed Incremental does the job a bet perfect, but this allow me to backup whole files on the Target Directory as a whole not only allowing me to backup the last file changed from the Daily Incremental of Veeam Backup!!! Correct me if I'm wrong.

I can see that with Reversed Incremental and Active Full backup on every Saturday, can do the job But, daily full backup of huge data to tapes, it's a bit headache.

Week-1
Saturday Full: VM.VBK
Sunday Incr: VM.VRB
Monday Incr: VM.VRB
Tuesday Incr: VM.VRB
Wednesday Incr: VM.VRB
Thursday Incr: VM.VRB

Week-2
Saturday Full: VM.VBK
Sunday Incr: VM.VRB "And it will delete Sunday from Week-1"
Monday Incr: VM.VRB "And it will delete Monday from week-1"
Tuesday Incr: VM.VRB "And it it will delete Tuesday from Week-1"
Wednesday Incr: VM.VRB "And it will delete Wednesday from Week-1"
Thursday Incr: VM.VRB "And it will delete Thursday from Week-1" and It will delete VBK from week-1

This will allow me with full back from Week-2 and I can go back to week-1 provided all the contents of 1 week offloaded to tape. !!!

Thanks,

[foggy]

With reversed incremental mode, the latest restore point is always a full VBK file. Btw, did you have a chance to review the backup methods description given in VBR user guide? It provides intuitive visual explanations of both modes, which are pretty easy to understand.

[Vitaliy S.]

Not sure that I fully understand what you're saying, but, yes, you can backup the entire backup chain once a week.

[habibalby]

hi Foggy,

No haven't gone through the user guide, I would appreciate if you can give me the link or as an attachment. So, the latest restore mode is full which means, if Symantec Backup or any Backup software runs on that file it will Reset the Archive-Bit which means for next Symantec Backup the VBK file will be considered as a Full Backup File And with Forward Incremental, it will Reset the Archive-Bit for all the file that Symantec Backup last runs and any new Forward Incremental it will not ejected into the VBK File but will create a VIB File.

If Symantec Backup runs as Full and last Reset Archive-Bit on VBK File past 1 Week, during the week any Symantec Incremental Job runs will take the latest VIB files which veeam generates them on daily basis. But this keeps exceeding the 7 Days Retention Policy and 7 Days Deleted VMs on Disk.

Is there any way I can select Forward Incremental to have 5 VIB files and 1 VBK File? At least in this option I don't have a problem with Symantec when it runs as Full backup and Incremental Backup.

I believe Veeam has to do something with regards to the Reset Archive-Bit when customer considering backing up to Tape.

Thanks,

[Vitaliy S.]

Is there any way I can select Forward Incremental to have 5 VIB files and 1 VBK File? At least in this option I don't have a problem with Symantec when it runs as Full backup and Incremental Backup.

No

[habibalby]

Not sure that I fully understand what you're saying, but, yes, you can backup the entire backup chain once a week.

Hi Vitaliy,

Yes, I agree with you. With Reversed Incremental I can backup the entire backup Chain on a Weekly basis such as. Full Backup runs on Saturday, then Sun-Thursday as Incremental. which will leave me with 1 VBK File and 5 VIBs files. But this keep me at risk of not backing up on a daily basis and when something goes wrong on the target SAN Storage on last day of the week on Thursday, this will keep me to loose the entire back up of 1 week which will put me in a bad shape isn't ?

[foggy]

And here is the documentation: http://www.veeam.com/vmware-esx-backup/resources.html.

[Vitaliy S.]

...this will keep me to loose the entire back up of 1 week which will put me in a bad shape isn't ?

Yes, that is why forward incremental backup mode is the best choice for tape backups.

[habibalby]

Hi,
Is there a way to delete Particular Restore Point from the Backup? Or I have to delete it manually? Is it safe deleting restore particular restore point manually?

Veeam should introduce to delete Particular Restore point from Backups. Now in Backups, if I right-clicking the particular backup job, I have only two options, Remove from Backup and Delete From Disk. The option delete From Disk is a bit scary if I have backup chain of two weeks, for instant;

Week-1:
Sat - Full: VM.VBK
Sun - Incr: VM.VIB
Mon - Incr: VM.VIB
Tue - Incr: VM.VIB
Wed - Incr: VM.VIB
Thu - Incr: VM.VIB

Week-2:
Sat - Full: VM.VBK
Sun - Incr: VM.VIB
Mon - Incr: VM.VIB
Tue - Incr: VM.VIB
Wed - Incr: VM.VIB
Thu - Incr: VM.VIB

The option to delete from disk will delete a restore points of two weeks. What if I want to delete a VBK file which has got smaller VIBs files related to that particular VBK? All the VIBs files will be unusable, correct?

Thanks,

[foggy]

What if I want to delete a VBK file which has got smaller VIBs files related to that particular VBK? All the VIBs files will be unusable, correct

Correct. As well as deleting a VIB file in the middle of the chain will make the consequent VIBs unusable.

[habibalby]

Correct. As well as deleting a VIB file in the middle of the chain will make the consequent VIBs unusable.

So, what is the solution for this? Backup everything to tape, then delete From Disk and re-create the job as full and so on? Veeam.com this is not practical at all. Now, I have to free one of my servers which has got 1.4 TB transfer the role to another VM. Install OpenFiler/iSCSI software on it, map the LUN to Veeam Machine to get things sort it out. My GOD

[habibalby]

Veeam.com should consider Snapshot Scenarios at least. Go To particular Backup/Restore Point, Delete Particular Backup/Restore Point, and Delete All Backup/Restore Points. That's another thing Veeam.com should consider i believe.

Thanks,

[dellock6]

If your concern is the space needed for two full backups, why don't you consider using Forward Incremental with synthetic full and transform?
It creates the next full backup combining the incremental files, so you will end up having only one full backup at a time on the disk, and leaves behind a chain of reverse incremental files. Once you offloaded the previous .vib files to tape there is no need to keep them on disk.
But remember, synthetic + transform puts a high load on the backup storage, generating 4 I/O for every bit taken from production storage.

Luca.