Attempt to overwrite existing non-sparse block version

[ConradGoodman]

Case #05014322 — Offload Job failing with 'Attempt to overwrite existing non-sparse block version'

Guys,

We logged a ticket for this on the 10th September, and for the past 2 weeks have been told RnD are looking at it and there is no workaround.

We rely on this technology for our off-site DR, and I'm sure there must be a work-around, like create another job for the same VM temporarily.

This is a big hole in our DR, and needs more care and attention than we are currently receiving.

[HannesK]

Hello,
sorry for the inconvenience. In my.veeam.com there is the "talk to manager" button which is the official way of escalation.

As the engineer mentioned that it could be a known issue, I will try to find the bug number and see whether it was already fixed in the latest update (11a).

Best regards,
Hannes

[HannesK]

Hello,
okay, looks like it's issue #340623 and there is currently no fix for that.

As far as I understood, it's about one VM that has the issue, right? Did you try to put that VM into a new job and see whether that works as temporary solution?

Best regards,
Hannes

[ConradGoodman]

Hey Hannes

Yeah its a known issue and unfortunately its affecting another datacentre and another veeam server now.

I'm going to move the VM into another job now and see if that is a viable work-around. It has been suggested to do similar, but awaiting full steps. Thankfully so far the VMs it is happening to are not so critical - if this creeps into my SQL backups with GFS retention I am in a world of pain.

[ConradGoodman]

So I tried to delete the local copy, but its immutable so won't even delete the local one that is trying to offload...

Will have to hang on very tightly for a fix from Veeam.

[pirx]

We just migrated to 11a and nearly all of our offload jobs have VMs with those errors. The funny thing is, support pushed us for months now to upgrade to 11 because offloading is much more stable and our v10 problems should be solved then. We had 4-5 hot/private fixes for offloading before.

Code: Select all

Error: Attempt to overwrite existing non-sparse block version. Block ID: [1100], new version: [yWO7CJS.PwdIJzJ7PpMOXHg.2l73Rv81], existing version: [qQ8xIOCn_cKv9KIFZCdUihfrge67SPfe]

[HannesK]

it looks like a hotfix should be available soon. I can only recommend to open a case to get it as soon as possible.

[pirx]

Created case #05079052

[Gostev]

We just migrated to 11a and nearly all of our offload jobs have VMs with those errors. The funny thing is, support pushed us for months now to upgrade to 11 because offloading is much more stable and our v10 problems should be solved then.

Support is not incorrect recommending V11 for object storage enhancements, however V11a was literally just released. With any software, it's always best to wait for a few weeks after GA before upgrading to the new release. Becase with any significant release, there's always risk of regressions. And someone needs to run into them before we can fix them. When you're first to upgrade, it's very likely to be you

[pirx]

You are right. But we had some issues where we did not get a hotfix for v10 anymore - as 11 was already available for months - but other issues we faced were not fixed in 11, but in 11a. So we were told now for months, to not update to 11, instead wait for 11a. We waited 2 weeks between RTM and update, I know that this is not a lot. But as we were still hitting bugs in v10, it didn't seem to make a huge difference.

[Gostev]

In any case, good change this particular issue is not specific to 11a. I seem to remember there was a private fix created for something very similar back in September. Your logs will help support to confirm if this is the same issue.

[ConradGoodman]

Not getting any kind of ETA at all from support on this fix, do we know roughly when to expect it? Last week I was told it could be that week, or this week...

[veremin]

The process was taking longer than usual, cause there were several cases that look similar to each other (in terms of resulting error) but that were caused by different problems. So it took some time for R&D team to separate one from the other.

The good news is the hotfix has been just created. Within next few days QA team will verify it and share with the support engineer.

Thank you for your understanding!

[ConradGoodman]

Thanks for the update Veremin.

[veremin]

According to the QA team, the hotfix for the 11a version has been verified and shared with the support engineer. The hotfix for the 11 P20210525 is being verified, as we speak, and should be provided to the support team any time soon. Thanks!

[veremin]

Conrad, as far as I can see, you have been provided with the hotfix. Once it's installed, let us know whether it solves the issue. Thanks!

[pirx]

I did not receive a hotfix yet, last feedback from support was 2 days ago.

[veremin]

I see that the internal description of your case was updated several minutes ago with the reference to the hotfix, which to me indicates that it will be shared with you soon.

If this does not happen today or tomorrow, let us know and we will reach your support engineer.

Thanks!

[wishr]

Hi Pirx,

I've pinged our support management to speed up the process.

Thanks

[ConradGoodman]

I received the hotfix, and in the case of 1 site it has fixed the offload!

The second problematic site however now throws a different error: Block not found.

Back to waiting for RnD, fingers crossed I can get this off my JIRA board this week.

[wishr]

Hi Conrad,

That sounds like a different issue. Could you please share your case id with us so we could keep an eye on it?

Thanks

[ConradGoodman]

Hey - it is the same case as the 1st post in this thread.

[veremin]

According to the support ticket, R&D team is already working on the issue, so let's wait and see what they suggest after their investigation. Thanks!

[spiritie]

We are also seeing this on multiple v11a installations, I'm following in on this thread. Let me know if we can be of assistance with providing logs, else I'll just keep lurking this thread until a hotfix has been released that resolved this error

We are using "S3-compatible" storage on these offloading jobs.

29-10-2021 00:43:16 :: Attempt to overwrite existing non-sparse block version. Block ID: [6], new version: [d1c954eb-979c-4ff2-93ff-13d1b472db9a], existing version: [44796cce-8e75-41cb-9b38-f02da72c358a]

Please keep updating this thread with your findings.

[pirx]

The fix for 11a worked for me, didn't see the issue again.

[veremin]

We are also seeing this on multiple v11a installations, I'm following in on this thread. Let me know if we can be of assistance with providing logs, else I'll just keep lurking this thread until a hotfix has been released that resolved this error

The hotfix has been released already, so kindly request it within open support case. Thanks!

[Mildur]

I'm not facing that issue too.
Will open a support case

[MrSpock]

I got this one too, shortly after upgrading to 11.0.1.1261.

Support case #05107678.

[DVA]

I have the same issue.
Case #05109320

[jmerfeld]

I am having this issue as well. Logs have been uploaded!

Case # 05110152