WS2019 - ID 25 Too many IO - backup failed

[AlexandreD]

Hi everyone,

I use Veeam Agent for backup a huge server (cloud repository).
I split the volume of 160 TB into 3 separate volume of 53 TB. (max size of a volume is 64 TB if we want to use vss snapshot on it).

Everything happen fine, but since few weeks, i have an error ID25 volsnap
"The shadow copies of volume F: were deleted because the shadow copy storage could not grow in time. Consider reducing the IO load on the system or choose a shadow copy storage volume that is not being a shadow copied."
After that, Agent backup will fail because snapshot doesn't exist anymore.

I tried to limit network bandwith in agent settings, to hope limit read IO speed before send on network, without success.
Each time, backup of the first data volume D:\ 53TB works fine, but ID event25 appears after 15-20 hours of working, and entire backup will fail.
I don't understand because I had already do new active full of the entire server by the past without problem (backup duration around 2 days).

I see in vssadmin, (cli below) that maybe we can preallocate space for vss "Allocated Shadow Copy Storage space", but i don't see anything in the ms technet.

I understand well, this is an microsoft issue. Maybe you've already encountered a similar problem.
Without backup copy feature in Veeam Cloud Connect Console, or specific storage (S3, Tape), this is the only way I found at this time.



PS C:\Users\Administrateur> vssadmin list shadowstorage
vssadmin 1.1 - Outil ligne de commande d’administration du service de cliché instantané de volume
(C) Copyright 2001-2013 Microsoft Corp.

Shadow Copy Storage association
For volume : (D:)\\?\Volume{5bc238a3-92d1-4ac7-9a66-3767416c4480}\
Shadow Copy Storage volume : (D:)\\?\Volume{5bc238a3-92d1-4ac7-9a66-3767416c4480}\
Used Shadow Copy Storage space : 0 octet(s) (0%).
Allocated Shadow Copy Storage space : 0 octet(s) (0%).
Maximum Shadow Copy Storage space : 8,00 To (14%)

Shadow Copy Storage association
For volume : (E:)\\?\Volume{e01de288-3a34-4d52-af2a-9d8b1b14f798}\
Shadow Copy Storage volume : (E:)\\?\Volume{e01de288-3a34-4d52-af2a-9d8b1b14f798}\
Used Shadow Copy Storage space : 0 octet(s) (0%).
Allocated Shadow Copy Storage space : 0 octet(s) (0%).
Maximum Shadow Copy Storage space : 8,00 To (14%)

Shadow Copy Storage association
For volume : (F:)\\?\Volume{5db5cc51-982b-4d9e-8222-9fd75a42f14f}\
Shadow Copy Storage volume : (F:)\\?\Volume{5db5cc51-982b-4d9e-8222-9fd75a42f14f}\
Used Shadow Copy Storage space : 0 octet(s) (0%).
Allocated Shadow Copy Storage space : 0 octet(s) (0%).
Maximum Shadow Copy Storage space : 8,00 To (14%)

Shadow Copy Storage association
For volume : (C:)\\?\Volume{7739ebeb-e9fb-4496-95b8-ec01d06f6642}\
Shadow Copy Storage volume : (C:)\\?\Volume{7739ebeb-e9fb-4496-95b8-ec01d06f6642}\
Used Shadow Copy Storage space : 0 octet(s) (0%).
Allocated Shadow Copy Storage space : 0 octet(s) (0%).
Maximum Shadow Copy Storage space : 200 Go (44%)

Thank you for your help.
Alexandre D

[PetrM]

Hi Alexandre,

Perhaps, it would be possible to track down the root cause from the Windows Event logs (Application, System) on the affected server or from the output of vsstrace tool. Also, can you please share a support case ID? The issue seems to be quite sophisticated, I don't believe we'll manage to resolve it via messages on forum.

Thanks!