Veeam not Following Retention Rules

[cardendave]

Case ID: 01839057

Hi Guys

Very frustrated user here. I have had 3 recent Veeam support cases and it has to be said the support hasn't been great. On all 3 I am told my backup is corrupted and of no use.This doesn't fill me with confidence selling the product going forward.

On this most recent case I have a backup that goes to UNC share. The share is a USB drive plugged into a PC. It is done in this way as it is faster than backing up direct to USB 2.0, if I install a USB 3 card and enable passthrough we cannot snapshot. I am happy with the backup speeds.

My issue is this. I have 5 x 2TB drives. I only wish to keep one retention on each drive. When this works the first backup is a full, then the next week it is an incremental and it then injects the incremental into the full. Great. A few weeks ago, this simply stopped working. Even though there was already a full backup on the drive, it created another full backup, filling the drive up and meaning future backups would not work. I logged this ticket. Veeam support told me to delete all my backup data on all of the drives, create a new backup job. Not a great solution as I have now lost all my backups, but we did it and it worked. Sadly only for a week. Last Wednesday the client forgot to plug the backup drive in, so the job failed. This is where it all went wrong again, on Thursday Veeam created a full backup next to the previous weeks backup, like my old issue before. It has then continued to do this every day, ever since the missed backup. I am guessing if I delete this backup and create a new job, it will work again. But what if my client misses a backup drive again?

Help please, support is very slow getting back to me!

Dave

[PTide]

Hi,

Just a few questions to make sure that I got it right:

- Do you want to have only Full + 1 incremental on each drive no matter how often they are rotated?
- Please describe the job settings, also please describe the repository settings that you've configured.
- What's your VBR version?

Thanks

[cardendave]

Hi

- No I just want one backup job per drive. So say when Mondays drive is plugged in on Monday it will have the previous Mondays backup on. The backup will run creating an incremental file which is then merged into the full file. This is what works for some time then stops working until I delete all the backups and recreate the job.
- Unsure what you're asking here, but I am backing up to UNC path to rotated USB. Rotated drives is set and also the force delete registry is set to 3.
- 9.0

Thanks

Dave

[foggy]

The backup will run creating an incremental file which is then merged into the full file.

It will not. If:

the force delete registry is set to 3.

it will delete the entire contents of the job's folder first and then run full backup.

[alanbolte]

The only option that will create incremental files immediately after swapping disks is Backup Copy to a Windows-type repository. It is not possible with a UNC path.

https://www.veeam.com/kb1154 has been updated recently to explain retention options for rotated media, as a supplement to the user guide.

[cardendave]

Sorry I don't think you follow.

We have 5 drives, Monday, Tuesday, Wednesday, Thursday, Friday.

Monday we put in Monday
Tuesday we put in Tuesday

and so on

On the first round, we get a full backup file on each drive, correct.
On the second round it creates and incremental file and at the end of the backup it injects the incremental into the full. Correct.
So after 2 weeks, we still have one vbk file on each drive, great.
Then one day the backup fails as the client did not put the drive in. This is where it goes wrong.
Client puts the next drive in and it no longer creates an incremental and injects it into the full, it creates another full (incorrectly). We now have 2 fulls on that drive with no space to create anything else.
Backups then start failing due to lack of space.

I call Veeam, they tell me to delete the job, delete all the backup files from every drive (not great) and start again.

I have even now changed that registry key to 5 so it should delete the entire contents of the drive, and it is not doing so. It is keeping the existing files. I rebooted the server after amending the key.

[foggy]

Dave, let's make it clear: are you talking about backup or backup copy jobs, first of all? What kind of repository does it use as the target: CIFS or Windows?

[cardendave]

Backups - I never mentioned it was a copy job. I don't think!

I just had this back from Veeam Support on my case?! Not ideal, I now have a broken product that I cannot use until the end of the year!

Hello Dave,

R&D has confirmed the issue, it will be fixed in Veeam 9.5 (release later this year).

Please let me know if you need any further assistance with this case, or I can consider it closed.


Best regards,
Sergey Shakhmatov

[Gostev]

I now have a broken product that I cannot use until the end of the year!

Dave, but according to your note above the issue only happens when the client does not put the drive in? So, "broken product" is not really a fair statement on your part. And the issue seems to be easily avoidable by ensuring that backup server is never left without rotated storage media attached?

But in any case, we are planning to release 9.5 sooner than end of the year.

[cardendave]

Totally disagree.

How can I guarantee a client puts in a drive every day?

The actual issue here is with a faulty drive which they are plugging in, but is not accessible.

The drive can be replaced but to say "your problem will be fixed at the end of the year" is totally unacceptable.

[Gostev]

Well, let's be fair here and admit that your real problem (and the source of all issues) is a faulty drive, and we can never fix this particular problem - not even at the end of the year

What we can and will do from our side to help you with this problem, is try to add "Support for faulty drives" feature into the next version of our product. But by all definition this should be classified as a new feature, and not a bug that makes it "a broken product".

All I am saying is that you have not been fair to our product in your statements. The only thing that is truly "broken" in this whole story is a faulty drive, which is what triggers the actual issue in the product. If not that faulty drive, we would not be having this discussion with you in the first place, as everything would have just worked - just as it does for tens of thousands of our users for many years now

[cardendave]

Gostev, I consider myself a reasonable person. I have to say that my frustrations with Veeam is growing. My last few calls, I have just been told "your backup is corrupt" you need to recreate it.

The problem here is indeed a faulty drive. However the issue I have, and I hope you understand is when a drive isn't plugged in OR the backup fails to see it, I am told by Veeam support that I have to delete my entire backup collection and start again. So I live on a knife edge while I backup again and get some kind of history. if it was just a case of deleting one backup then fine, but its not the case.

[foggy]

Checked with support and QA, and it appears that the primary cause is the fact that ForceDeleteBackupFiles registry value is not designed to trigger on job retry (otherwise, retry cycles would be deleting newly created backups). Seems you do not need to delete all the backups and start from scratch - you can just cleanup the disk manually, insert it and continue backup. The value should work on subsequent disk changes.

I'm however still not sure why you're mentioning incremental chain and merge process above, since it cannot occur in your configuration.