Comprehensive data protection for all workloads
Post Reply
Oletho
Enthusiast
Posts: 67
Liked: 2 times
Joined: Sep 17, 2010 4:37 am
Full Name: Ole Thomsen
Contact:

CheckStorage failed client error: bad allocation

Post by Oletho »

Support case ID 5212734

I recently installed Veeam B&R 6.1 patch1 at a customer site.

The initial job of 124 VMs went ok and generated a ~5TB backup file. But every job after that fails immediately with the error message mentioned in subject.

Support suggested that it is a memory problem and proposed to cut the job in smaller portions. But I would prefer a solution, not a workaround, and very much doubt that the system is memory limited. Even if the Veeam servers are freshly rebooted and there are a handful free GBs of memory the job fails instantly.

An equally large replication job runs fine.

Any suggestions?
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: CheckStorage failed client error: bad allocation

Post by dellock6 »

What kind of design are you using? Physical servers, hot add with virtual proxies.... It's hard to say what's wrong or what could be improved based on your post.
Only, please remember backup and replication are different operations, so it's not a correct assumption saying that if replication works, so it should do also backups.

Luca.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
Oletho
Enthusiast
Posts: 67
Liked: 2 times
Joined: Sep 17, 2010 4:37 am
Full Name: Ole Thomsen
Contact:

Re: CheckStorage failed client error: bad allocation

Post by Oletho »

It is running appliance mode with a couple of proxies.

Removing 17 small VMs from the job got it running again, so it probably had something to do with numbers or size. Adding memory to Veeam servers made no difference.

We decided to cut the jobs in smaller parts, as that was the only offer from support.
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: CheckStorage failed client error: bad allocation

Post by dellock6 »

How many vCPUs has any proxy? Usually proxies are more cpu hungry than memory, after 2-4 gb per VM I never had to increase memory size on proxies.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
Oletho
Enthusiast
Posts: 67
Liked: 2 times
Joined: Sep 17, 2010 4:37 am
Full Name: Ole Thomsen
Contact:

Re: CheckStorage failed client error: bad allocation

Post by Oletho »

4 vCPUs per proxy (4.1 enterprise license). The job failed before CPU activity was rising.

The only reason for increasing memory was that support stated it was a memory limitation problem.
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: CheckStorage failed client error: bad allocation

Post by Gostev »

I know clients who are running very big jobs (in terms of amount of VMs) without any issues. How big is your job?

Some things to check:
- Could it be bad memory on the proxy server?
- Could it be lack of memory on the backup repository server (if you have it separate).

I recommend you keep this case open until you have it resolved, you should not have to change your jobs to accommodate this issue.
Oletho
Enthusiast
Posts: 67
Liked: 2 times
Joined: Sep 17, 2010 4:37 am
Full Name: Ole Thomsen
Contact:

Re: CheckStorage failed client error: bad allocation

Post by Oletho »

The job had 124 VMs of totally 15 TB. Backup file was around 5TB but not all VMs were backed up when we started seeing this error.

Involved proxy and repository had plenty available memory, as the job failed right after start. All Veeam servers are Windows Server 2008 R2 with latest SP and updates.

They were migrated just to try other hosts.

The case is closed, as the client wanted me to get on with the installation and not spend time arguing with support.

And support scared the shit out of my customer by stating that we had to have a successful job, otherwise we would not be able to restore any VM. I proved that wrong to the customer, but anyway :)
Post Reply

Who is online

Users browsing this forum: Google [Bot], justin.hendren and 158 guests