-
- Expert
- Posts: 127
- Liked: 22 times
- Joined: Feb 18, 2015 8:13 pm
- Full Name: Randall Kender
- Contact:
Veeam Failing "Offload to Archive" Tasks on SoBR
Veeam support case ID: 05593210
So I wanted to post here because we are having an issue getting the Archiving tier to work in our SoBR. I haven't been able to find reference to these specific errors anywhere from Veeam or Microsoft, so am hoping someone else might have some insight, or if we end up finding a solution I can update this for others that might have the same issue.
When performing the "Offload to Archive" tier the job is failing for each VM with the following error:
8/26/2022 11:59:47 AM :: Task failed. Error: One of the HTTP headers specified in the request is not supported.
RequestId:94fe3cc2-****-****-****-************
Time:2022-08-26T18:59:42.2134472Z, Details: 'HeaderName':'x-ms-access-tier'
Agent failed to process method {LongTerm.Upload}.
I have tickets opened with both Veeam and Microsoft. Currently Veeam is saying the issue may be related to something altering the header within Azure, but is not sure and is asking for an Azure tech to review the requestID. From the Azure side, the Microsoft tech is saying he detected we are using Veeam version 10 (though we are currently on the latest, 11.0.1.1261) and that the issue may be related to the version we are using or that Veeam is not using the correct header settings.
Does anyone possibly have any insight into this? We are trying to utilize the Azure West-US2 region, so am wondering if possibly this is a restriction on this specific region.
So I wanted to post here because we are having an issue getting the Archiving tier to work in our SoBR. I haven't been able to find reference to these specific errors anywhere from Veeam or Microsoft, so am hoping someone else might have some insight, or if we end up finding a solution I can update this for others that might have the same issue.
When performing the "Offload to Archive" tier the job is failing for each VM with the following error:
8/26/2022 11:59:47 AM :: Task failed. Error: One of the HTTP headers specified in the request is not supported.
RequestId:94fe3cc2-****-****-****-************
Time:2022-08-26T18:59:42.2134472Z, Details: 'HeaderName':'x-ms-access-tier'
Agent failed to process method {LongTerm.Upload}.
I have tickets opened with both Veeam and Microsoft. Currently Veeam is saying the issue may be related to something altering the header within Azure, but is not sure and is asking for an Azure tech to review the requestID. From the Azure side, the Microsoft tech is saying he detected we are using Veeam version 10 (though we are currently on the latest, 11.0.1.1261) and that the issue may be related to the version we are using or that Veeam is not using the correct header settings.
Does anyone possibly have any insight into this? We are trying to utilize the Azure West-US2 region, so am wondering if possibly this is a restriction on this specific region.
-
- Product Manager
- Posts: 14589
- Liked: 2961 times
- Joined: Sep 01, 2014 11:46 am
- Full Name: Hannes Kasparick
- Location: Austria
- Contact:
Re: Veeam Failing "Offload to Archive" Tasks on SoBR
Hello,
the error seems to be very rare (I found an one year old support case for UK region, and the customer closed the case without solution, because he realized that he is using by far less space than he expected. so he cancelled the whole archive topic).
The only similarity I found between your case an the other is, that both seem to have permission / network restrictions in place. So my guess would be something around networking or permissions.
Let's see, whether any other customer can comment on the error itself.
Best regards,
Hannes
the error seems to be very rare (I found an one year old support case for UK region, and the customer closed the case without solution, because he realized that he is using by far less space than he expected. so he cancelled the whole archive topic).
The only similarity I found between your case an the other is, that both seem to have permission / network restrictions in place. So my guess would be something around networking or permissions.
Let's see, whether any other customer can comment on the error itself.
Best regards,
Hannes
-
- Product Manager
- Posts: 20307
- Liked: 2270 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Veeam Failing "Offload to Archive" Tasks on SoBR
Cannot it be a West-US3 by any chance cause West-US3 does not support Archive Tier? Also, can you tell us what the storage account type is (using outdated or improperly upgraded accounts might result in the same issue)? Thanks!
-
- Expert
- Posts: 127
- Liked: 22 times
- Joined: Feb 18, 2015 8:13 pm
- Full Name: Randall Kender
- Contact:
Re: Veeam Failing "Offload to Archive" Tasks on SoBR
So after working with a Microsoft rep they discovered that there is official documentation from them stating that Archive Tier is not supported on ZRS, GZRS, or RA-GZRS.
https://docs.microsoft.com/en-us/azure/ ... ccess-tier
On Veeam's documentation side, I do see it listed under the documentation for Veeam Backup for Azure (https://helpcenter.veeam.com/docs/vbazu ... =40#backup and https://helpcenter.veeam.com/docs/vbazu ... positories) but don't see that limitation listed anywhere under the Veeam Backup and Replication documentation. The only references to limitations surrounding ZRS are related to restoring to Azure, or the helper appliance locations.
If it is there then I must have missed it, but I wonder if it might be possible to request that there is a warning popup somewhere in the console when either configuring the capacity or archive extents of a SoBR, similar to how the ReFS popup happens when creating a repository that is not formatted with ReFS 64KB cluster size. Luckily we don't have all our datasets uploaded, but we do have upwards of 40TB that we now have to move to a new storage account, so it is going to cost us quite a bit of time and money to reconfigure things.
https://docs.microsoft.com/en-us/azure/ ... ccess-tier
We had chosen ZRS for our storage account. Even the tech we were working with was surprised that this is the case, it only seems to be mentioned this one location in Microsoft's documentation; I haven't been able to find it anywhere else.Only storage accounts that are configured for LRS, GRS, or RA-GRS support moving blobs to the Archive tier. The Archive tier isn't supported for ZRS, GZRS, or RA-GZRS accounts.
On Veeam's documentation side, I do see it listed under the documentation for Veeam Backup for Azure (https://helpcenter.veeam.com/docs/vbazu ... =40#backup and https://helpcenter.veeam.com/docs/vbazu ... positories) but don't see that limitation listed anywhere under the Veeam Backup and Replication documentation. The only references to limitations surrounding ZRS are related to restoring to Azure, or the helper appliance locations.
If it is there then I must have missed it, but I wonder if it might be possible to request that there is a warning popup somewhere in the console when either configuring the capacity or archive extents of a SoBR, similar to how the ReFS popup happens when creating a repository that is not formatted with ReFS 64KB cluster size. Luckily we don't have all our datasets uploaded, but we do have upwards of 40TB that we now have to move to a new storage account, so it is going to cost us quite a bit of time and money to reconfigure things.
-
- Product Manager
- Posts: 20307
- Liked: 2270 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Veeam Failing "Offload to Archive" Tasks on SoBR
Correct, Archive Tier does not work with ZRS-like accounts. We cannot guarantee that warning will make it into release, but we have already reached our technical writers to add this information to our User Guide. Thanks for raising this!
-
- Expert
- Posts: 127
- Liked: 22 times
- Joined: Feb 18, 2015 8:13 pm
- Full Name: Randall Kender
- Contact:
Re: Veeam Failing "Offload to Archive" Tasks on SoBR
Thank you for that.
In terms of where it would help for the documentation, I would definitely say under "Limitations for Microsoft Azure Object Storage" https://helpcenter.veeam.com/docs/backu ... ct-storage
But you also might want to add it somewhere as a warning under one or more of the sections regarding Azure storage https://helpcenter.veeam.com/docs/backu ... ml?ver=110. Currently if you are just searching the documentation for the process it takes you straight to this section, and unless you look around you may not even know the "Considerations and Limitations" section exists, so having a warning there would be helpful. Or alternatively you could have a link in a warning box under these sections that redirects to the Azure section of the Considerations and Limitations page.
In terms of where it would help for the documentation, I would definitely say under "Limitations for Microsoft Azure Object Storage" https://helpcenter.veeam.com/docs/backu ... ct-storage
But you also might want to add it somewhere as a warning under one or more of the sections regarding Azure storage https://helpcenter.veeam.com/docs/backu ... ml?ver=110. Currently if you are just searching the documentation for the process it takes you straight to this section, and unless you look around you may not even know the "Considerations and Limitations" section exists, so having a warning there would be helpful. Or alternatively you could have a link in a warning box under these sections that redirects to the Azure section of the Considerations and Limitations page.
-
- Product Manager
- Posts: 20307
- Liked: 2270 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Veeam Failing "Offload to Archive" Tasks on SoBR
Yes, these are the places we were thinking about adding a note to. Also, we might add a warning (like you have suggested) later on in post-v12 releases. Thanks!
Who is online
Users browsing this forum: No registered users and 32 guests