-
- Service Provider
- Posts: 86
- Liked: 11 times
- Joined: Jul 17, 2017 12:37 pm
- Contact:
intermittent failures to wasabi
Hi all,
latest veeam v12 patched 2 different installs.
We have recently been hitting an issue in the last week with a few backup copy jobs direct to wasabi object storage.
Error: REST API error: 'S3 error: Account Connection Limited
Is the most common error
on prem Backup repo is xfs immutable
Wasabi side is also immutable.
Its only for some VM’s within the one job and usually retry will suceed after a few retries.
case ID 06358590
latest veeam v12 patched 2 different installs.
We have recently been hitting an issue in the last week with a few backup copy jobs direct to wasabi object storage.
Error: REST API error: 'S3 error: Account Connection Limited
Is the most common error
on prem Backup repo is xfs immutable
Wasabi side is also immutable.
Its only for some VM’s within the one job and usually retry will suceed after a few retries.
case ID 06358590
-
- Product Manager
- Posts: 20677
- Liked: 2382 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: intermittent failures to wasabi
No, this is not a known issue. So, keep working with the support team. Thanks!
-
- Novice
- Posts: 4
- Liked: never
- Joined: Apr 18, 2021 10:22 pm
- Full Name: Elage
- Contact:
Re: intermittent failures to wasabi
Yep we are seeing exactly this across 95% of our customers that use Veeam & Wasabi. Noticed it on the 12th.
Have logged to Wasabi, they are saying we are exceeding the connection limit of 200 connections per IP per minute per region, but something else must be happening here as it is virtually ALL our veeam customers and nothing has changed at our end...
Have logged to Wasabi, they are saying we are exceeding the connection limit of 200 connections per IP per minute per region, but something else must be happening here as it is virtually ALL our veeam customers and nothing has changed at our end...
-
- Service Provider
- Posts: 86
- Liked: 11 times
- Joined: Jul 17, 2017 12:37 pm
- Contact:
Re: intermittent failures to wasabi
Yeah i fixed it but lowering the concurrent connection on the bucket for this use case it will be ok.
Thanks for the input
Thanks for the input
-
- Service Provider
- Posts: 86
- Liked: 11 times
- Joined: Jul 17, 2017 12:37 pm
- Contact:
Re: intermittent failures to wasabi
forgot to add yes its has been from the 12th onwards.
IMO they have changed or are enforcing rate limits
IMO they have changed or are enforcing rate limits
-
- Lurker
- Posts: 2
- Liked: never
- Joined: Jul 03, 2019 8:05 pm
- Full Name: Matt Talaga
- Contact:
Re: intermittent failures to wasabi
We are experiencing this issue as well. Today, while troubleshooting it, I received the AccessDenied - Account Connection Limited when trying to login to the Wasabi Admin Console. Hard to troubleshoot anything when they are limiting your console access as well.
Our concurrent connection to the bucket is only set to 2, so we can't set it any lower at this point. We've had nothing but performance issues with Veeam v12 since we've upgraded to it. But we did that months ago, and now we can't go back. It seems like Veeam changed how it was communicating to S3 with the upgrade to v12 that requires more API calls?
Our concurrent connection to the bucket is only set to 2, so we can't set it any lower at this point. We've had nothing but performance issues with Veeam v12 since we've upgraded to it. But we did that months ago, and now we can't go back. It seems like Veeam changed how it was communicating to S3 with the upgrade to v12 that requires more API calls?
-
- Product Manager
- Posts: 20677
- Liked: 2382 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: intermittent failures to wasabi
Hi, Matt,
Can you provide us with the case numbers so we can verify what was the root cause of the experienced behavior?
Thanks!
Can you provide us with the case numbers so we can verify what was the root cause of the experienced behavior?
Thanks!
-
- Chief Product Officer
- Posts: 32229
- Liked: 7591 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: intermittent failures to wasabi
More API calls are certainly required with V12 as the new object storage format required for Direct to Object backup no longer leverages local cache, which we had when our object storage integration was limited to copying existing local backups to object storage. However, it's only a marginal increase comparing to how many calls are issued during actual backup, due to the sheer number of objects to be processed and each requiring own API call to upload, set/prolong immutability, etc.
Nevertheless, 12.1 does bring some further optimizations in this department, reducing those "extra" calls caused by the new V12 format as much as possible.
Nevertheless, 12.1 does bring some further optimizations in this department, reducing those "extra" calls caused by the new V12 format as much as possible.
Who is online
Users browsing this forum: No registered users and 9 guests