SOAP header Action was not understood

[Robvil]

Hi,

Case 02271744

Starting from today many of my mailboxes gives this error:

05-08-2017 21:29:02 15 (10172) Error: SOAP header Action was not understood.
05-08-2017 21:29:02 15 (10172) Stack:
05-08-2017 21:29:02 15 (10172) at System.Web.Services.Protocols.SoapHeaderHandling.SetHeaderMembers(SoapHeaderCollection headers, Object target, SoapHeaderMapping[] mappings, SoapHeaderDirection direction, Boolean client)
at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
at EWServices.ExchangeServiceBinding.GetFolder(GetFolderType GetFolder1)
at Veeam.Ews.ExMailbox.GetFolder(GetFolderType gf)
at Veeam.Ews.ExError.<>c__DisplayClass6_0`1.<Catch>b__0()
at Veeam.Ews.ExError.Catch(Action action, String format, Object[] args)
05-08-2017 21:29:02 15 (10172) Mailbox processing failed: mail@domain.com
05-08-2017 21:29:10 19 (9032) Trying to resume processing...
05-08-2017 21:29:10 19 (9032) Error: Failed to get folder properties
05-08-2017 21:29:10 19 (9032) Stack:
05-08-2017 21:29:10 19 (9032) at Veeam.Ews.ExError.Throw(SoapException soap, String format, Object[] args)
at Veeam.Ews.ExError.Catch(Action action, String format, Object[] args)
at Veeam.Ews.ExError.Catch[T](Func`1 func, String format, Object[] args)
at Veeam.Ews.ExMailbox.GetFolder(BaseFolderIdType id, IReadOnlyDictionary`2 requestProps)
at Veeam.Archiver.Source.MailboxContextExtension.<>c__DisplayClass0_0.<GetSpecialFolder>b__0()
at Veeam.Archiver.Source.FolderRetry.<>c__DisplayClass14_0`1.<Process>b__0()
at Veeam.Archiver.Source.FolderRetry.Process(Action action)
05-08-2017 21:29:10 19 (9032) Error: SOAP header Action was not understood.
05-08-2017 21:29:10 19 (9032) Stack:
05-08-2017 21:29:10 19 (9032) at System.Web.Services.Protocols.SoapHeaderHandling.SetHeaderMembers(SoapHeaderCollection headers, Object target, SoapHeaderMapping[] mappings, SoapHeaderDirection direction, Boolean client)
at System.Web.Services.Protocols.SoapHttpClientProtocol.ReadResponse(SoapClientMessage message, WebResponse response, Stream responseStream, Boolean asyncCall)
at System.Web.Services.Protocols.SoapHttpClientProtocol.Invoke(String methodName, Object[] parameters)
at EWServices.ExchangeServiceBinding.GetFolder(GetFolderType GetFolder1)
at Veeam.Ews.ExMailbox.GetFolder(GetFolderType gf)
at Veeam.Ews.ExError.<>c__DisplayClass6_0`1.<Catch>b__0()
at Veeam.Ews.ExError.Catch(Action action, String format, Object[] args)
05-08-2017 21:29:10 19 (9032) Warning: Exchange Web Services request failed
05-08-2017 21:29:10 19 (9032) Retry count: 5
05-08-2017 21:29:10 19 (9032) No more retries left

Could this be related to the new Office 365 login portal page?

Robert

[heggers]

I'm seeing this also. Looking for resolution.

[bdp]

also same issue in recent days. Case #02271912

I'm finding:

• a similar but changing number of fails on each attempt (around 160-170 of our ~260 mailboxes)
• varying number of mailboxes actually enumerated on each job run
• no obvious pattern to servers of affected mailboxes (all server prefixes are found in success & fail lists)
• more than 60% of our mailboxes are throwing this error
• no correlation to user creation or licensing option
• no correlation to rollout of 50Gb->100Gb mailbox upgrades

I don't know if there's a better way, but I've copied the list of mailboxes & errors into a textfile, used cut & grep to produce two lists of UPNs, then run each against Get-Mailbox to find the location.

Code: Select all

# example input
6/08/2017 9:28:15 PM :: Processing mailbox: happyuser@example.com
6/08/2017 9:28:15 PM :: Processing mailbox saduser@example.com failed with error: Failed to get folder properties. SOAP header Action was not understood.

Code: Select all

# copy output into text file (bash), then produce user lists
$ cut -f7,8 -d' ' O365_out.txt | grep -v failed > o365_succeeded.list.txt
$ cut -f7,8 -d' ' O365_out.txt | grep failed | cut -d' ' -f1  > o365_failed.list.txt

Code: Select all

# powershell (fixed CRLF & added "upn" as 1st line in each file)
> $failed = Import-Csv o365_failed.list.txt
> $succeeded = Import-Csv o365_succeeded.list.txt
> $failed | ForEach-Object { Get-Mailbox -identity $_.upn | Select-Object * } | Out-GridView
> $succeeded | ForEach-Object { Get-Mailbox -identity $_.upn | Select-Object * } | Out-GridView

(Maybe this will help somebody, or somebody can help me with a more streamlined or concise powershell-only re-write?)

[compconsult]

We saw this show up on the 5th as well.

It doesn't seem to be consistent on the same mailboxes - Just random.

[davecoughlan]

Same issue here, appears to be getting worse with each backup. Logging a support ticket now

[hiltls]

Got this error also since yesterday! Any solution?

[Robvil]

I´m waiting for support to get backup to me after sending them my logs.

[Dat_Guardian]

I got the same error with my room mailboxes and group maiboxes. The normal user mailboxes work like before

[Mike Resseler]

Hi All,

If this is VBO 1.0, please log a support case. If this is VBO 1.5, please let me know through a PM or post it here that it is 1.5

In the meantime, I will escalate this as it seems to be suddenly appearing and with many users, which most probably means that MSFT has changed something and did not make us aware of it

Thanks
Mike

[Mike Resseler]

PS: To all people where I just had to approve the post... Welcome to the forums! This was the first time only (defense against bots) so feel free to post and contribute as much as you like!

Cheers
Mike

[Mike Resseler]

OK. Our QC engineers have already responded back (which is pretty quickly considering time of the day for them ). They have discovered it also this weekend and the investigation is already going on. Nevertheless, please continue to create support calls. This can only help the investigation and please your case # and follow-up here

Thanks
Mike

[JonJR]

We've had this problem with v1 & logged with support.

I thought I'd take the opportunity to try using the Beta 1.5, and whilst it is still running its first big backup, it actually works and doesn't SOAP error the same way that v1 does. YMMV & NFI

[Kostya]

Hi all, let me ask: does anyone still have these errors? It seems like different DEV and QC labs was affected this weekend by the same error, but we cannot reproduce it anymore, neither in 1.0, nor 1.5. If someone still have it, please let us know, so we ca review your logs. Thanks.

[JonJR]

Hi all, let me ask: does anyone still have these errors?.

The v1 O365 backups are now working perfectly, we've now had 2 successful backups. I guess it was a Microsoft issue!

Cheers Jon

[Mike Resseler]

Jon,

Thanks for coming back to us. Please keep us informed when this appears again and please let support know if you have a ticket.

[aich365]

03697632
Have had this issue since 1/8/2019 on Version 3.0.0.422 - looks like Microsoft changed something ...

Have uploaded logs to support

[BrunoNovo]

I try to setup Veeam backup for Office 365 and keep getting 5/08/2019 14:07:28 :: Connect to EWS: SOAP header Action was not understood.
Version is 3.0.0.422.
It is basic authentication.
Can you pleas ehelp me out? Regards.

[Polina]

Clive, Bruno,

Thank you both for the heads up on this issue. We're now seeing a few more similar cases and the R&D team is actively researching the root cause. Please continue working with support engineers.

@BrunoNovo, what is your case ID?

[BrunoNovo]

Case number is: 03697824

[Polina]

Thanks, Bruno

If possible, can you try to temporarily assign an O365 license to your service account and see if it helps you pass the verification check when adding the organization to VBO?

[BrunoNovo]

I did assign a Business Essentials licence, waited 10 minutes and tried again, but the problem remains.

[lukasholzknecht]

Same here, Case #03698038.
Tried with and without assigned license, Veeam is always giving the same error.

[Polina]

Was a mailbox created for a service account at the same time? Our lab tests show that having a mailbox resolves the issue. Until we complete the investigation, this might be considered as a temporary workaround.

[dan.kennedy]

Same issue here on v3 (3.0.0.422)
Case ref. 03690990

Looks like assigning a license to the service account did the trick. I had to wait about an hour after the mailbox was created before it worked in Veeam.
Does anyone know if it will carry on working if i remove the license now the initial setup of the org is complete?

[admcomputing]

What about if we dont have any spare 365 licenses to temporary give to the backup service account so we cant give it a mailbox?

[Polina]

Giving a mailbox to a service account should do the job, and the license can be removed afterward. If you currently don't have any spare licenses to assign, please try to create a new service account without any license and with a password that doesn't expire/require to be changed. In our lab test, this worked well.

These are, of course, temporary workarounds, and at the moment our DEV team is creating a fix, which will be then delivered via support.

[aich365]

After investigation with Veeam we can do one of two things:

1. assign Office 365 license and mailbox to the service account. (Veeam@mailaddress.co.uk)

2. create new service account without license from scratch and configure its password policy in the way that it will never ask to change password for it.
e.g. veeambackup@mailaddress.co.uk

[avmoutsourcing]

Hello,

I would like change the account subscription this day on my Veeam BAckup 365 dashboard, adn i have the same problem:
Eror SOAP header Action was not understood.
I have not available license , and i have an Office 365 account without license and without expired password but the add account wizard don't running...
i'm block for Migration
Veeam, please
THank you

[Polina]

Welcome to Veeam Forums, avmoutsourcing!

Please create a support case to get the fix and share your case ID here.

Thanks!

[aich365]

Customer has assigned a licence and mailbox and the EWS issue is no more

Any timescales on a fix?

Thanks