Comprehensive data protection for all workloads
Post Reply
Daveyd
Veteran
Posts: 283
Liked: 11 times
Joined: May 20, 2010 4:17 pm
Full Name: Dave DeLollis
Contact:

Synthetic backup failure

Post by Daveyd »

What happens to the backup chain if a synthetic backup failure happens? Do the vibs not ever get deleted?

Reason I ask is because I am running into an issue where I have a forward incremental jobs that are set to a 7 day retention period. Each job creates synthetic fulls every week either on Saturday or Sunday (I had to separate the days due to the Exagrid backup target having performance issues running 3-4 synthetic backups at one time). One job in particular has 3 vbks and 21 vibs associated with it dating back to the beginning of this month. :shock: Is this to be expected? Is there a way to cleanup old backups?
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Synthetic backup failure

Post by Vitaliy S. »

Hello Dave,

If you happen to have a failed synthetic full run, then on the subsequent job run this failed VBK file will be removed from the destination storage. As regards the retention policy behavior, then none of the restore points will be deleted until the next synthetic full backup takes place.
Daveyd wrote:One job in particular has 3 vbks and 21 vibs associated with it dating back to the beginning of this month. Is this to be expected? Is there a way to cleanup old backups?
With a failed synthetic on the first week you would have 2 VBKs and 21 VIBs (assuming that you didn't have any failed VM backups during this period). On the 3rd synthetic all your previous rollbacks will be removed (except for the last 7 rollbacks defined by a retention policy).

If this is the case then I suggest waiting for the 3rd synthetic job run, and only after that please check the amount of rollbacks on the target storage.

Thanks.
Daveyd
Veteran
Posts: 283
Liked: 11 times
Joined: May 20, 2010 4:17 pm
Full Name: Dave DeLollis
Contact:

Re: Synthetic backup failure

Post by Daveyd »

Thank you for the clarification. I think my issue with the Synthetic full backups is a result of the Exagrid appliance getting hammered with I/O during full Synthetic backups. I might have to resort to using "normal" full backups 1 a week instead of synthetic. If I change my jobs from synthetic to normal, what will happen to my existing vibs from the failed synthetics once the normal full is processed?
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Synthetic backup failure

Post by Vitaliy S. »

These files will be deleted according to your retention policy settings.
Daveyd
Veteran
Posts: 283
Liked: 11 times
Joined: May 20, 2010 4:17 pm
Full Name: Dave DeLollis
Contact:

Re: Synthetic backup failure

Post by Daveyd »

So, I ran a normal full backup tonight hoping to get rid of some old files. The job created a new vbk.

Here is what the files are with the job set to 7 rollbacks:

VBK created on 5/14 - success
vib created on 5/15
vib created on 5/16
vib created on 5/17
vib created on 5/18
vib created on 5/19
vib created on 5/20
VBK created on 5/21 - synthetic full failed
vib created on 5/22
vib created on 5/23
vib created on 5/24
vib created on 5/25
vib created on 5/26
vib created on 5/27
VBK created on 5/28 - synthetic full failed
vib created on 5/29
vib created on 5/30
VBK created on 5/31 - "normal" full success

So, when do files get deleted?
Sethbartlett
Veteran
Posts: 282
Liked: 26 times
Joined: Nov 10, 2010 6:51 pm
Full Name: Seth Bartlett
Contact:

Re: Synthetic backup failure

Post by Sethbartlett »

When 5/28 becomes your 7th point past. Currently, if you count back 7 points, you will lane on "vib created on 5/24". If we were to delete this point, 5/25, 5/26 and 5/27 would all not be valid restore points. We remove restore points when the whole chain is available to be removed since the VIB files depend on each other.
Skype: Sethbartlett88 - Make sure to label who you are and why you want to add me ;)
Twitter: @sethbartlett
If my post was helpful, please like it. Sometimes twitter is quicker to hit me up if you need me.
Daveyd
Veteran
Posts: 283
Liked: 11 times
Joined: May 20, 2010 4:17 pm
Full Name: Dave DeLollis
Contact:

Re: Synthetic backup failure

Post by Daveyd »

Sethbartlett wrote:When 5/28 becomes your 7th point past. Currently, if you count back 7 points, you will lane on "vib created on 5/24". If we were to delete this point, 5/25, 5/26 and 5/27 would all not be valid restore points. We remove restore points when the whole chain is available to be removed since the VIB files depend on each other.
The vbk created on 5/14 will not be deleted until 6 days after the 5/28 backup? And the vibs created 5/15-5/20 wont be deleted until after that?
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Synthetic backup failure

Post by Vitaliy S. »

The entire backup chain starting from VBK (5/14) to VIB (5/30) will be deleted after a new successful synthetic full is created. Based on your schedule it should happen on the 6th day after VBK (5/31).

Currently the entire chain depends on the VBK created on 5/14, which cannot be removed due to retention policy settings. Hope it makes sense.
gordonhowes
Lurker
Posts: 2
Liked: never
Joined: Aug 05, 2011 8:51 am
Full Name: Gordon Howes
Contact:

Synthetic full missed

Post by gordonhowes »

[merged]

We have a backup that is set to use Incremental with synthetic fulls created once a week - so forever incremental. The last time the synthetic full was set to run something went wrong and the full was not created.

Backups have continued to increment since this without any problems however my question is what happens once the synthetic full jobs comes to create again? does it take two weeks worth of increments to create the file. I am concerned as one of the vib files appears to be locked by a failed sure backup session (a support case has been opened with Veeam) and am wondering if the synthetic full will not work because of this
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Synthetic backup failure

Post by Vitaliy S. »

There shouldn't be any issues, unless some files are still locked. By the way how do you determine that those files are locked?
gordonhowes
Lurker
Posts: 2
Liked: never
Joined: Aug 05, 2011 8:51 am
Full Name: Gordon Howes
Contact:

Re: Synthetic backup failure

Post by gordonhowes »

When the job completes it shows the following warning

Checking retention policy.
Storage 'Backup Job 22011-08-05T180048.vib' is locked by running session 'Sure Backup 2'

The 'Sure Backup 2' was a sure backup job that was linked to the job but has since been removed from the sure backup job list
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Synthetic backup failure

Post by Vitaliy S. »

Oh ok...never seen this error before. Please continue working with our support team on that.
Gotenks
Lurker
Posts: 2
Liked: never
Joined: Sep 01, 2011 2:29 am
Full Name: Gotenks

Re: Synthetic backup failure

Post by Gotenks »

Had this same error on a source for "Instant Recovery" where I was trying to back-up the source of a "Recovered" VM with the same job used before.
Created a new job and the error went away.
tfloor
Veteran
Posts: 270
Liked: 15 times
Joined: Jan 03, 2012 2:02 pm
Full Name: Tristan Floor
Contact:

job failed what happend next

Post by tfloor »

[merged]

Hi,

I've set forward incrementals on monday till saturday.
On Saturday i've set the syntethic full.

What happens if the saturday job failed (target was shutdown unexpected).
The next run will be on monday normally, but wil it create a full or going further with incrementals...
So that i miss 1 synthetic full and end with 8 incrementals?

Thanks.
tfloor
Veteran
Posts: 270
Liked: 15 times
Joined: Jan 03, 2012 2:02 pm
Full Name: Tristan Floor
Contact:

Re: job failed what happend next

Post by tfloor »

I've manual started a "perform full backup", to get the vbk file on this saturday.
But what i've see is that on the target the failed synthetic job for this day is ended as an (failed) .vib file.
So now i have 1.vib and 1.vbk for 1 day.

I'm wondering what this mean.
tfloor
Veteran
Posts: 270
Liked: 15 times
Joined: Jan 03, 2012 2:02 pm
Full Name: Tristan Floor
Contact:

Re: Synthetic backup failure

Post by tfloor »

Looks like I have an extra .vib file for the jobs that were finished before the target unexpectly shutdown.
So that files were not transformed to a synthetic. I started a manual full again. Maybe better was to try the failed job again. I have to learn from it
mschlott
Enthusiast
Posts: 84
Liked: 2 times
Joined: Jan 20, 2011 5:45 pm
Full Name: Mike Schlottman
Contact:

How does Veeam deal with hardware failures during backup?

Post by mschlott »

[merged]

I back up data from a veeam server in our data center to a Veeam Linux Target in our local office. On multiple occasions I have had the WAN go down or the AC go out in our local office causing the Veeam Linux Target to power off.

When this happens, if I have any backup problems after the failure, support tells me the best thing to do is a new full backup. Unfortunately, I don't have the space to keep multiple full backups on site. I do daily incrementals and synthetic fulls and roll up the incrementals into roll backs. In some cases the failure occurs during the incremental backup, and in others it occurs during the synthetic full.

I would imagine that all I would have to do is remove the latest incremental from a backup chain if the backup failed during the incremental backup. If the failure occurs during the synthetic full, I'm not sure what the case should be. Last night we had the WAN go down at 1AM, but the time stamp on one of my VBK files is 1:19 AM. That makes me think that the synthetic full continued to process after the failure. But, the name of the VBK file is from the previous day, and there is still a VIB from last night.

I'm hoping to find a document that states what I should do in the case of these types of failures. If not, I need to open Severity 1 support cases for each of my failures last night, because I have to get them resolved before tonights backups.

Thanks
mike
Post Reply

Who is online

Users browsing this forum: Alex Hung, Bing [Bot], Google [Bot] and 333 guests