First vib file got corrupt during merge, lost all increments

[mlabbe]

I posted this in another thread a couple days ago but since I got no feedback at all and I saw alot of activity from Veeam support team in other threads, I thought I'd post again in a new thread to make sure to catch more attention.

As mentionned in the other post, VEB somehow managed to corrupt my first vib file in the chain during the process of merging it into the vbk file. Since everything is incremental, this rendered all my other vib files completely useless (I lost a whole month of retention) and cheduled backups stopped working at this point.

I understand adding differential backups may not be an option because of the way VEB is designed, but then I think some better protection mechanism should to be added to make sure this issue doesn't happen again (e.g. add some PAR2 files or something before doing the merge so you can repair/rollback if there's an issue during the merge process).

I didn't happen to need anything from my lost backup files, but a backup software is as good as what it can restore when you need it.

I also suggest you add some more eye-catching notification (e.g. somewhat like win10 notification window every couple hours) to catch more attention when the backups are failing day after day. I've been sitting at my computer for at least a couple hours before I noticed the tiny red "x" (instead of usual clock) on VEB's tray icon. I know ALOT of non-IT people who wouldn't have noticed anything at all until it's too late.

[Dima P.]

Michel,
Thanks again for sharing your thoughts!

I've been sitting at my computer for at least a couple hours before I noticed the tiny red "x" (instead of usual clock) on VEB's tray icon.

How do you think, should it be a pop up notification or may be email notifications are enough for backup history tracking, or maybe both?

[mlabbe]

How do you think, should it be a pop up notification or may be email notifications are enough for backup history tracking, or maybe both?

Well, I'd say both would be best, but if you don't want to implement email notifications in the free version, a pop up notifications should do it.

You don't want to implement a 5-10 secs notification that the user will miss because he was afk when the pop up happened, and not something that will be instantly dismissed because it happened to pop while the user was typing on keyboard either.

[mlabbe]

For whoever may end up on this thread later don't freak out yet , the issue I faced prevented new incremental backups, but most important: the data in the corrupt backup chain was still recoverable. The follow-up has been done in the original thread.

(e.g. add some PAR2 files or something before doing the merge so you can repair/rollback if there's an issue during the merge process).

I sent some more details about these PAR2 files to Adam in the support ticket and he said it might also help that I post this on the forums so here it is for the records:

Have a look at Parchive if you haven't checked this already. I used PAR2 files often when I was a student and was grabbing files from newsgroups. My ISP's newsgroup servers were skipping posts every now and then so more than half of the binary files were incomplete or corrupt. It's very powerful: a small PAR2 file (even if it's damaged itself) could repair a dozen corrupt files, as long as you have enough healthy parity blocks to cover the number of corrupt blocks in a number of files.

See example images:
.

Parchive now seems to implement PAR3 with better support for multithreading.

You could use something like this (it's open source) to create some parity blocks before starting the merge (and maybe even add another set after completing for extra safety). Then whatever happen (even if there are real bad blocks on the disk), you'd be able to repair/rollback the vbk/vib files as long as there's enough healthy parity blocks available.

[Dima P.]

Michel,

Once again thanks for sharing! You issue report helped us a lot and we will make the needed corrections in upcoming version.

[ftcnet]

Yes, this area of VEB is still a bit weak (IMO). I lost a client due to a missing .vib file. The .vem and all .vbk files were still present. Veeam support worked on it for several hours, but apparently there's no way to recreate a missing or corrupted .vim file. It seems like using .PAR files might substantially improve our confidence in VEB's ability to successfully restore when needed.

Email notification at completion of backup jobs would be very much appreciated.

[Dima P.]

Hello Bernard,

Can you share the case ID please? Thanks.