Error: Collection was modified

[Frosty]

Last night I received the following error from one of my backup jobs:

Unable to allocate processing resources. Error: Collection was modified; enumeration operation may not execute.

My analysis of the failure leads me to think that it was caused by there being too many disks attached via Hot Add to my Veeam backup server.

My VBR server is a VM with one (1) HDD C: on SCSI 0:0 and three (3) HDDs on a separate controller SCSI 1:0 1:1 and 1:2

At the time of failure I had a VM with four (4) HDDs being backed up, plus another two VMs each with two (2) HDDs queued up. So I interpret that as possibly:

1 disk C: on the VBR server
4 disks via Hot Add for VM being backed up
2 disks in the queue for additional VM backup
2 disks in the queue for another VM backup

totalling 9 ... more than that 8 allowed. Is this analysis likely to be correct? Is it possible for me to add another SCSI controller to the VBR server and then force VBR to use that controller for the HotAdd tasks?

[Vitaliy S.]

You'll never know until you try it , however I would expect your job to failover to a network mode and continue with an appropriate warning message.

[Frosty]

Mmmm. True. I might chalk this one down as "just one of those things" for now. But if I see it again I will open a support case. Thanks!

[william1111]

Hi Frosty,
Did u have root cause for this problem? I have just had this problem one of my jobs and it repeated with only one VM. On my vcenter, I couldn't find out anything related.

[Gostev]

Cannot check on resolution for you since OP did not include the support case ID...

[Nax]

We get this error on our daily backup jobs:
1/28/2015 10:30:19 PM :: Error: Collection was modified; enumeration operation may not execute.

I get these on 5 out of 40 VMs with no predefined cause. However on the retry of the jobs, the VMs process successfully. Is Veeam handling too many VMs at once? Our support case ID is 00731422.

Thank you.

[foggy]

Logs should tell the reason, so please continue reviewing them with support. Do you use hotadd, btw?

[RaymondV]

Hi,

I have a question about the Veeam proxies. I noticed some backups failing with the error "Collection was modified; enumeration operation may not execute". I was able to track this down to the proxy having the maximum number of disks (15) attached to the SCSI controller. I was monitoring the proxies yesterday during the backup window and noticed the same VMDK connected to the proxy multiple times, which seems strange to me. I can add an extra SCSI controller to the proxies but I would like to know if this behaviour (adding the same disk more than once in a single backup session) is by design.

Kind regards,
Raymond

[Shestakov]

Hello Raymond,
What is the number of concurrent tasks allowed for this proxy? For the same proxy, is the transport mode set to automatically select, or is it explicitly defined as Network/NBD or Hot-Add?
Another step for you to try, just to see if we can narrow this down: For a single iteration of the job, even off-schedule, can you disable Application-Aware Image Processing and see if we get the same error on VMs?
By the way what version of the product are you at?
Thanks!

[WinstonWolf]

Hello ,

From Time to Time i become the following Warning on an Backup from differnet VMs .

"Failed to prepare VM for processing from storage snapshot, failing over to using VM snapshot. Details: Collection was modified; enumeration operation may not execute"

On next Backup all is ok again .

Thanks
Michael

[foggy]

Michael, please contact technical support if you'd like to investigate this issue and prevent it from occurring in future. Thanks.