-
- Novice
- Posts: 8
- Liked: never
- Joined: Dec 21, 2010 6:32 pm
- Full Name: Jérôme Poulin
- Contact:
Reverse Differential corruption
Hi,
I would like to know if it is normal that the vbk appears corrupted if the backup is interrupted, for example, by unplugging the USB in the middle of a backup and trying to restore from that disk read only?
Restoring single files worked from one VM but full restore and VM files restoration fails complaining about a missing VRB.
I would like to know if it is normal that the vbk appears corrupted if the backup is interrupted, for example, by unplugging the USB in the middle of a backup and trying to restore from that disk read only?
Restoring single files worked from one VM but full restore and VM files restoration fails complaining about a missing VRB.
-
- Chief Product Officer
- Posts: 31748
- Liked: 7251 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Reverse Differential corruption
Hi, sure - since the backup storage has failed during backup, the latest restore point will be "corrupted" (incomplete) until the following incremental pass "fixes" the VBK file. You will still be able to restore from earlier restore points though. Thanks.
-
- Novice
- Posts: 8
- Liked: never
- Joined: Dec 21, 2010 6:32 pm
- Full Name: Jérôme Poulin
- Contact:
Re: Reverse Differential corruption
I also tried restoring anterior versions, not the latest but 3 days ago and it failed, it is not production critical we're only testing recovery. In that case would you suggest using incremental+synthetic+reverse?
-
- Novice
- Posts: 8
- Liked: never
- Joined: Dec 21, 2010 6:32 pm
- Full Name: Jérôme Poulin
- Contact:
Re: Reverse Differential corruption
Sorry for the post spam but I just tried with a fresh copy of a reverse differential and it fails again, the file system is read only, would that affect VM files restoration?
Edit:
This backup was imported on another PC.
Edit:
This backup was imported on another PC.
-
- Chief Product Officer
- Posts: 31748
- Liked: 7251 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Reverse Differential corruption
If the backup run did not complete successfully because the backup storage was pulled out during backup, you cannot restore from the corresponding restore point. This is expected as the data is incomplete. I would suggest to not unplug your backup storage while backing up to it
-
- Novice
- Posts: 8
- Liked: never
- Joined: Dec 21, 2010 6:32 pm
- Full Name: Jérôme Poulin
- Contact:
Re: Reverse Differential corruption
Like I said, I tried with a complete backup, now I switched to Incremental so I have all my VRBs, a VBK and VIBs and I still get the same error.
I imported the backup on another PC via Veeam, the media is read only (as it is EXT4), files are readable and single file restore still work, I tried restoring a single .vmx file and it still fails.
I imported the backup on another PC via Veeam, the media is read only (as it is EXT4), files are readable and single file restore still work, I tried restoring a single .vmx file and it still fails.
-
- Chief Product Officer
- Posts: 31748
- Liked: 7251 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Reverse Differential corruption
For any technical issues it would be best to work directly with out support, as this would require full logs research. The issues might not be connected at all.
My main point here is that USB storage devices are handled in a special manner by Windows, there is good reason for that "Remove USB device safely" thing, this ensures that write cache is dumped to the storage. So generally I would not recommend pulling the USB storage when there is heavy I/O going.
Although I do like to say that life is too short to remove USB safely
My main point here is that USB storage devices are handled in a special manner by Windows, there is good reason for that "Remove USB device safely" thing, this ensures that write cache is dumped to the storage. So generally I would not recommend pulling the USB storage when there is heavy I/O going.
Although I do like to say that life is too short to remove USB safely
-
- Novice
- Posts: 7
- Liked: never
- Joined: Mar 14, 2011 3:31 pm
- Contact:
Re: Reverse Differential corruption
Hi, I'm linking to this thread, althoug my situation is different.
Also I reda this thread:
http://www.veeam.com/forums/viewtopic.p ... ion#p12828
but I think is referred to old version.
I'm working with DR, using backup to a remote site, with linux target.
I set up a reverse incremental with 1 restore point.
Due to bandwidth constraints, they take several hours.
So it is quite possible to have network problems, that take away the connection for a while. Also, we can suppose that the disaster happens in the middle of a backup.
I'm quite disapponted to discover that in this case, I'm not able to recover the VM, (neither in the previous state).
I would expect that the VBR agent keeps the old data available until the new has been received !!!
IS there any setup to fix this?
Also I reda this thread:
http://www.veeam.com/forums/viewtopic.p ... ion#p12828
but I think is referred to old version.
I'm working with DR, using backup to a remote site, with linux target.
I set up a reverse incremental with 1 restore point.
Due to bandwidth constraints, they take several hours.
So it is quite possible to have network problems, that take away the connection for a while. Also, we can suppose that the disaster happens in the middle of a backup.
I'm quite disapponted to discover that in this case, I'm not able to recover the VM, (neither in the previous state).
I would expect that the VBR agent keeps the old data available until the new has been received !!!
IS there any setup to fix this?
-
- Chief Product Officer
- Posts: 31748
- Liked: 7251 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Reverse Differential corruption
That's exactly what it does, so previous state is always recoverable.sbos wrote:I would expect that the VBR agent keeps the old data available until the new has been received !!!
-
- Novice
- Posts: 7
- Liked: never
- Joined: Mar 14, 2011 3:31 pm
- Contact:
Re: Reverse Differential corruption
Maybe I did some mistake in the setup, but can assure that this is not the case.
Also I have several situations, where this happened.
Should I have more that one restore point? enable forward incremental?
Should I open a support case?
let me know
TIA
sergio
Also I have several situations, where this happened.
Should I have more that one restore point? enable forward incremental?
Should I open a support case?
let me know
TIA
sergio
-
- Chief Product Officer
- Posts: 31748
- Liked: 7251 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Reverse Differential corruption
As long as you have good restore point produced by successful backup run, you should always be able to restore to it even if the following restore point is corrupted. Basically, full backup file is updated in transactionally, so we always know where the "good" data is. If you can produce situation when this is not the case, please open a support case and let us take a look.
Sometimes disaster can affect more than just the file being processed. For example, we just closed a support case recently were customer could not restore a single restore point after having some storage issues. It appeared that this was caused by RAID5 rebuild process, which was corrupting all backup files on the volume due to bad controller logic (this was reproducible).
Other common problem is backing up to a NAS. Certain low-end NAS devices have questionable performance improvements and do not handle flush command correctly, telling our engine that the data block was written when in fact it was not (still hanging in cache), which of course produces inconsistency and unrecoverable backup files in case of power loss.
So, usually this needs to be investigated on case-by-case basis. Actual engine of transactional VBK updates is solid (have not been changing for a few years now), so typically this comes down to corruption due to the actual storage device. But of course we want to investigate every occasion, to make sure the issue is not with our code.
Thanks!
Sometimes disaster can affect more than just the file being processed. For example, we just closed a support case recently were customer could not restore a single restore point after having some storage issues. It appeared that this was caused by RAID5 rebuild process, which was corrupting all backup files on the volume due to bad controller logic (this was reproducible).
Other common problem is backing up to a NAS. Certain low-end NAS devices have questionable performance improvements and do not handle flush command correctly, telling our engine that the data block was written when in fact it was not (still hanging in cache), which of course produces inconsistency and unrecoverable backup files in case of power loss.
So, usually this needs to be investigated on case-by-case basis. Actual engine of transactional VBK updates is solid (have not been changing for a few years now), so typically this comes down to corruption due to the actual storage device. But of course we want to investigate every occasion, to make sure the issue is not with our code.
Thanks!
-
- Novice
- Posts: 7
- Liked: never
- Joined: Mar 14, 2011 3:31 pm
- Contact:
Re: Reverse Differential corruption
Ok. Just before opening a case, let me understand how it should work.As long as you have good restore point produced by successful backup run, you should always be able to restore to it even if the following restore point is corrupted. Basically, full backup file is updated in transaction, so we always know where the "good" data is. If you can produce situation when this is not the case, please open a support case and let us take a look.
Our VBR system is backing up to a linux remote system (WAN @8mbit/s).
The backup is set up for reverse incremental, with a single restore point. (no history needed).
At this point we still did not had a disaster, but we had a network loss, and the backup finished.
So, I publish via NFS the backup and copy the VM to a different datastore.
If I go to see the status of the backup it tells me that some machine is incomplete.
If I try to restore it, the publishing on NFS is ok,
but if I try to move to a production datastore the VM , it fails.
The same happens if I stop manually the backup (the message tells me that "the backup will be left in a non consistent state").
If you tell me I will go on opening a case to the support, my doubt is that I'm missing some point in the config.
thank you.
sergio
-
- Chief Product Officer
- Posts: 31748
- Liked: 7251 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Reverse Differential corruption
Yes, basically you are trying to restore from the only restore point you have, which is inconsistent and will stay in this state until the next job pass restores it back to last know good state. You want to set the amount of restore points to keep to 2 at least. This way, you will always have consistent and restorable point available, no matter what. Thanks.
-
- Novice
- Posts: 7
- Liked: never
- Joined: Mar 14, 2011 3:31 pm
- Contact:
Re: Reverse Differential corruption
That's an hard learned lesson!!!
Will test it ASAP.
thank you
Sergio
Will test it ASAP.
thank you
Sergio
Who is online
Users browsing this forum: Bing [Bot], denispirvulescu, hobbit and 143 guests