Unable to allocate memory for storage metadata bank

[albertwt]

Hi All,

I've got some issue here with multiple backup job failed at the same time in my backup server last night.

Backup Repository Server - 32 GB memory Physical server.
Backup Server - 64 GB memory Physical server.

Here is the error message per VM in a backup job:

Code: Select all

Error: Unable to allocate memory for storage metadata bank. Failed to restore file from local backup. VFS link: [summary.xml]. Target file: [MemFs://frontend::CDataTransferCommandSet::RestoreText_{56d1935a-13e1-404b-bc14-c241d6cb655c}]. CHMOD mask: [0]. Agent failed to process method {DataTransfer.RestoreText}. 
Maximum retry count reached (5 out of 5)

Code: Select all

Processing PRODNAS01-VM Error: bad allocation Unable to retrieve next block transmission command. Number of already processed blocks: [134951]. Failed to download disk. An existing connection was forcibly closed by the remote host Failed to upload disk. Agent failed to process method {DataTransfer.SyncDisk}.

Error: bad allocation Unable to retrieve next block transmission command. Number of already processed blocks: [134951]. Failed to download disk. An existing connection was forcibly closed by the remote host Failed to upload disk. Agent failed to process method {DataTransfer.SyncDisk}.

Error: Unable to allocate memory for storage metadata bank.

Code: Select all

Error: Insufficient system resources exist to complete the requested service. Failed to read data from the file [I:\Backups\PRODNAS01-VM\NAS Servers 12016-04-05T010846.vbk]. Failed to restore file from local backup. VFS link: [summary.xml]. Target file: [MemFs://frontend::CDataTransferCommandSet::RestoreText_{86919b2c-6e1a-4856-8916-c275e515d470}]. CHMOD mask: [0]. Agent failed to process method {DataTransfer.RestoreText}.

Error: Insufficient system resources exist to complete the requested service. Failed to read data from the file [I:\Backups\PRODNAS01-VM\NAS Servers 12016-04-05T010846.vbk].

Processing DCSYNXIS07 Error: Unable to allocate memory for storage metadata bank. Failed to restore file from local backup. VFS link: [summary.xml]. Target file: [MemFs://frontend::CDataTransferCommandSet::RestoreText_{dd339f5f-209f-4289-acb7-2a79ac446b25}]. CHMOD mask: [0]. Agent failed to process method {DataTransfer.RestoreText}. 

Note: This case has been logged in Case # 01752448 but no one seems to be able to help me after few days of webex discussion and I've also uploaded the full backup logs into the FTP site as well.

Any help would be greatly appreciated.

Thanks.

[foggy]

How many tasks were running at the time of the failure? I'd also ask for the case escalation to a higher tier.

[albertwt]

Hi Foggy, I don't know how to escalate it to higher tier ?

It happened after hours, so I'm not sure to know the number of Backup Copy job that is running continuously. But one thing for sure, the Backup job that is usually running is about 2-3 jobs.

I'm still running Veeam Backup & Replication 8.0.0.2084, so do I need to upgrade to Veeam Backup 9.0 Update 1 ?

[foggy]

Updating to the latest version is always a good step, however, I'd ask your support engineer to verify memory consumption on involved servers.

[veremin]

I don't know how to escalate it to higher tier ?

Click "Talk to a Manager" button on the customer support portal. Thanks.