-
- Service Provider
- Posts: 218
- Liked: 28 times
- Joined: Jan 24, 2012 7:56 am
- Full Name: Massimiliano Rizzi
- Contact:
Re: Wasabi US-East-2 Issues
Hello Community and good day,
sorry for jumping straight into the thread.
I just opened Case # 05681023 in order to troubleshoot the warning message below that we have been observing for the last weeks as part of offloading to Wasabi EU Central 1 as well and which is similar to some of the errors reported in the OP:
==================================================
Checkpoint cleanup failed Details: Stopped by <Backup Job name>
==================================================
Although offload jobs are usually completing successfully in our scenario, they are frequently throwing the warning message above.
Thanks and Regards,
Massimiliano
sorry for jumping straight into the thread.
I just opened Case # 05681023 in order to troubleshoot the warning message below that we have been observing for the last weeks as part of offloading to Wasabi EU Central 1 as well and which is similar to some of the errors reported in the OP:
==================================================
Checkpoint cleanup failed Details: Stopped by <Backup Job name>
==================================================
Although offload jobs are usually completing successfully in our scenario, they are frequently throwing the warning message above.
Thanks and Regards,
Massimiliano
-
- Product Manager
- Posts: 20415
- Liked: 2302 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Wasabi US-East-2 Issues
Have you reached the Wasabi support team as well? As noted in this thread, the problem is not related to Veeam Backup & Replication, and the only thing our support team can provide right now is temporary workarounds. Thanks!
-
- Service Provider
- Posts: 218
- Liked: 28 times
- Joined: Jan 24, 2012 7:56 am
- Full Name: Massimiliano Rizzi
- Contact:
Re: Wasabi US-East-2 Issues
I have not but I will reach out to them in the afternoon.Have you reached the Wasabi support team as well? As noted in this thread, the problem is not related to Veeam Backup & Replication, and the only thing our support team can provide right now is temporary workarounds. Thanks!
I decided to reach out to Veeam support team as well in order to apply the temporary workarounds while waiting for Wasabi support team to provide a permanent fix for this.
Thanks !
Massimiliano
-
- Technology Partner
- Posts: 23
- Liked: 36 times
- Joined: Feb 23, 2021 3:42 pm
- Contact:
Re: Wasabi US-East-2 Issues
Hi All,
We are currently investigating the cause of this issue. If anyone is affected, please open a ticket with our support team at support@wasabi.com so we can further investigate. Our support team and engineering team are fully engaged in discovering the root cause.
Thanks,
chrisWasabi
We are currently investigating the cause of this issue. If anyone is affected, please open a ticket with our support team at support@wasabi.com so we can further investigate. Our support team and engineering team are fully engaged in discovering the root cause.
Thanks,
chrisWasabi
-
- Service Provider
- Posts: 192
- Liked: 38 times
- Joined: Oct 28, 2019 7:10 pm
- Full Name: Rob Miller
- Contact:
Re: Wasabi US-East-2 Issues
I did receive word yesterday from Wasabi that:
I stopped my copy jobs to the SOBR with Wasabi as the capacity tier and allowed it to catch up. I have noticed improved performance since receiving the above email from Wasabi yesterday. Everything caught up, I re-enabled the copy jobs, and it now seems to be humming along much better. I still have the API deletes limited to 500. It's not as fast as before, but it does seem to be improved and at least functional now. I'll keep an eye on it.We have made some adjustments to our infrastructure and see that processing times improved last night.
-
- Product Manager
- Posts: 20415
- Liked: 2302 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Wasabi US-East-2 Issues
Good to know you have received an answer from the Wasabi team. Assuming everything goes well, you can gradually increase the MultipleDeleteObjects regkey value. Thanks!
-
- Novice
- Posts: 4
- Liked: never
- Joined: Apr 18, 2021 10:22 pm
- Full Name: Elage
- Contact:
Re: Wasabi US-East-2 Issues
We're seeing exactly the same thing to ap-southeast-2 (Sydney) so definitely not isolated.
It's just an annoyance at this point but our plan was to move all our archive data into Wasabi, so if it doesn't get fixed soon that plan will certainly be going in the bin!
It's just an annoyance at this point but our plan was to move all our archive data into Wasabi, so if it doesn't get fixed soon that plan will certainly be going in the bin!
-
- Product Manager
- Posts: 20415
- Liked: 2302 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Wasabi US-East-2 Issues
Have you already contacted the Wasabi support team? The more information they have, the better. Plus, they seem to be aware of the issue and providing solutions. Thanks!
-
- Technology Partner
- Posts: 23
- Liked: 36 times
- Joined: Feb 23, 2021 3:42 pm
- Contact:
Re: Wasabi US-East-2 Issues
Hi fillerbunnie,fillerbunnie wrote: ↑Oct 26, 2022 9:24 am We're seeing exactly the same thing to ap-southeast-2 (Sydney) so definitely not isolated.
It's just an annoyance at this point but our plan was to move all our archive data into Wasabi, so if it doesn't get fixed soon that plan will certainly be going in the bin!
I checked the logs for Syndey, and we do not see any instances of this issue occurring. We are able to determine if this is happening because the specific problem is related to a particular S3 API call, the "MultiObjectDelete". I would urge you to contact Wasabi Support (support@wasabi.com) if you are seeing an issue that is preventing backups from working.
Thanks,
wasabiChris
-
- VeeaMVP
- Posts: 1007
- Liked: 314 times
- Joined: Jan 31, 2011 11:17 am
- Full Name: Max
- Contact:
Re: Wasabi US-East-2 Issues
An update from our side. We were able to solve the Access denied errors; those were neither Veeam nor Wasabi related.
Now we're seeing errors for the cleanups or when trying to delete backups:
"Failed to delete backup Error: HTTP exception: WinHttpQueryDataAvaliable: 12152: The server returned an invalid or unrecognized response, error code: 12152"
Not sure if this is still related to the problems mentioned in this topic.
I have contacted Wasabi Support and will see if they can find anything.
Now we're seeing errors for the cleanups or when trying to delete backups:
"Failed to delete backup Error: HTTP exception: WinHttpQueryDataAvaliable: 12152: The server returned an invalid or unrecognized response, error code: 12152"
Not sure if this is still related to the problems mentioned in this topic.
I have contacted Wasabi Support and will see if they can find anything.
Who is online
Users browsing this forum: No registered users and 22 guests