Error 13 out of nowhere

[mcdaniels]

Hi,
I suddenly got a backup-problem on one VM: Log is telling me:

Code: Select all

 ERR |Asynchronous data reader has failed.
[15.02.2019 15:23:48] < 19376>      ERR |VDDK error: 13 (You do not have access rights to this file). Value: 0x000000000000000d
[15.02.2019 15:23:48] < 19376>      >>  |--tr:Failed to read from a virtual disk. Start sector: 1894400, length: 2048
[15.02.2019 15:23:48] < 19376>      >>  |--tr:Failed to execute VDDK command 'Read'
[15.02.2019 15:23:48] < 19376>      >>  |--tr:Disk read operation has failed. Disk path: [[datastore1] SRV-DC2/SRV-DC2.vmdk]. Read offset: [969,932,800]. Requested data size: [1048576]. Disk size: [214,748,364,800]
[15.02.2019 15:23:48] < 19376>      >>  |--tr:CFsAwareDisk::Read failed. Offset: 969932800, length: 1048576
[15.02.2019 15:23:48] < 19376>      >>  |--tr:Failed to pre-read data ( first start block '925', blocks num '1'. Block size: '1048576').
[15.02.2019 15:23:48] < 19376>      >>  |--tr:Next asynchronous read request cannot be processed.
[15.02.2019 15:23:48] < 19376>      >>  |--tr:Asynchronous data reader has failed.
[15.02.2019 15:23:48] < 19376>      >>  |Unable to retrieve next block transmission command. Number of already processed blocks: [146].
[15.02.2019 15:23:48] < 19376>      >>  |--tr:Next asynchronous read request cannot be processed.

I have changed absolutly nothing concerning the ESXI or my backupinfrastructure.

Any hints are welcome.

Thanks

[veremin]

Kindly, open a support ticket and let our support team investigate this issue directly. Thanks!

[mcdaniels]

Hi, thanks for the reply.

Support ticket opened.

[mcdaniels]

Hi,
just for your information. The problem was a defective VMDK. In later tests it shows, that I even cannot copy the vmdk from the datastore, when VM was powered off. In addition to this, there was also a defective harddrive in the existing Raid10-array, which shouldn't be a problem at all.

[Gostev]

Hi, thanks a lot for coming back to share the solution!

[robg]

Hi,
just for your information. The problem was a defective VMDK. In later tests it shows, that I even cannot copy the vmdk from the datastore, when VM was powered off. In addition to this, there was also a defective harddrive in the existing Raid10-array, which shouldn't be a problem at all.

Hello, I got this recently as well. When you say "the problem was a defective VMDK" - how did you fix it?

Thanks in advance

[soncscy]

Hey Rob,

In the past when I've seen this, there's not much to do -- VMware was not able to do much (maybe someone else had better luck), so what we ended up doing was an in-guest backup with the windows agent, and scheduling a maintenance window to do a quick incremental backup then restore.

The instant disk recovery probably makes this process even better, but I've not run through a recovery scenario with this. Short point is that unless Vmware has some magic to fix the VMDK, probably there will be at least "some" downtime, but the tools are there to make this a matter of minutes.

[robg]

I'm in a bit of a pickle because I can't restore this VM from backup, nor can I active the replica (different error).

I think the safest approach is to create a new virtual disk, and clone the volume over to it, like the old-style way. Doesn't Veeam have a free product that will do this? I'm not quite familiar with the free offerings.

thanks

[soncscy]

Hey Rob,

I think you misunderstood me.

Use an in-guest Agent Backup like Veeam Agent for Windows or Veeam Agent for linux.

No snapshots, no need to replicate.

This will make a fresh backup and save you a ton of time.

Why can't you restore?

[robg]

I can't restore because I have no restore points. It's strange.. I'll give agent a shot, but I have an old version of Veeam (9.0). Hopefully it comes with it..

[mcdaniels]

Hello, I got this recently as well. When you say "the problem was a defective VMDK" - how did you fix it?

Thanks in advance

sorry for coming back that late. We had to transfer the VMs of the impacted ESXI HOST to another host. (this worked via Veeam) Then, we changed the defective HDD, "flattened" the datastore of this host and rebuilt it.

Note: We only have DAS-storage (Raid 10) in both of our hosts.

After that we had no issues with Veeam anymore.

Best regards,
Daniel