Public Folder backup error

[JerrySeltzer]

For the past week, backups have been failing for the public folders mailbox of one of our tenants. The mailbox is setup with a company contacts folder with 5 nested contacts folders by type and 3 independent email based folders. The errors are always occurring on one of the contacts folder (although it varies which one on different runs of the job). Below is a sample of the errors in the log (time stamps removed for easier reading).

Exchange Web Services error code: ErrorInternalServerError
85 (888) Error: Failed to synchronize item changes in folder
(888) Type: Veeam.Ews.Internal.ExServerCodeException
Stack:
at Veeam.Ews.ExError.Throw(ExServerCodeException error, String format, Object[] args)
at Veeam.Ews.ExError.Catch(Action action, String format, Object[] args)
at Veeam.Ews.ExError.Catch[T](Func`1 func, String format, Object[] args)
at Veeam.Ews.ExItemSyncer.GetChanges(String folderId, String folderName, String syncState, Int32 maxItems, IReadOnlyDictionary`2 props)
at Veeam.Archiver.Source.Exchange.SyncChanges.TryGetChanges(IMailboxContext bind, String folderId, String folderName, Byte[] cookieIn, CancellationToken cancel, TimeSpan& elapsed)
at Veeam.Archiver.Source.Exchange.SyncChanges.<>c__DisplayClass18_0.<GetChanges>b__0()
at Veeam.Ews.Retry.FolderRetry.<>c__DisplayClass29_0`1.<Process>b__0()
at Veeam.Ews.Retry.FolderRetry.Process(Action action)
85 (888) Error: An internal server error occurred. The operation failed., IdAndSession.MailboxSession: session is not IMailboxSession
85 (888) Type: Veeam.Ews.Internal.ExServerCodeException
85 (888) Stack:
85 (888) at Veeam.Ews.Internal.ResponseMessageTypeExtension.ThrowIfError(ResponseMessageType item)
at Veeam.Ews.ExItemSyncer.GetData(SyncFolderItemsType syncType)
at Veeam.Ews.ExError.<>c__DisplayClass7_0`1.<Catch>b__0()
at Veeam.Ews.ExError.Catch(Action action, String format, Object[] args)

[Polina]

Hi Jerry,

There're quite a few of the similar cases in our support now. It seems some changes occurred on the Microsoft side and our engineers are currently working on the ways to resolve it. Please open a support call and post its ID here, so that we could track this issue with accuracy and follow the investigation.

Thanks!

[agrob]

good day, is this similar to this error we receive since about 2 days on the public folder backup?

Processing mailbox publicfoldermbx@tenant.onmicrosoft.com failed with error: Failed to synchronize item changes in folder: IPM_SUBTREE.. An internal server error occurred. The operation failed., IdAndSession.MailboxSession: session is not IMailboxSession

Thanks

[Funki_sahib]

Hello,

we receive this error since Monday when the sheduled Backup for Public Folder is running. After 3 min the Job quit with an Error:
13.12.2018 10:38:43 :: Processing mailbox PublicFolderFG_xxx@company.onmicrosoft.com failed with error: Failed to synchronize item changes in folder: Eingang.. An internal server error occurred. The operation failed., IdAndSession.MailboxSession: session is not IMailboxSession :: 0:03:22

normal Backup for Mailbox, OneDrive and SharePoint is not affected.
Is this the same Problem on which you are working now?

[Polina]

Hi Andre,

Yes, it seems to be exactly the same issue. Please open a support case and share its ID here.
Thanks!

[agrob]

Hi Polina

Case ID is #03325441

[Polina]

@Funki_sahib,

Likewise, please contact our support team and share your case ID in this thread (the same issue most likely).
And welcome to the Veeam community!

[agrob]

Got first Feedback from Support. They are working on it, seems like an issue on O365 EWS which affects every 3rd party software...

[jbcap]

Hi,

We have the same problem, case: 03325135

[Funki_sahib]

Case # 03325420

[jamesm111]

We have the same issue, so commenting here to subscribe to updates.

[m.novelli]

Same to me, the issue raised for a couple of customers

Ciao

Marco

[cdlane]

Case #: 03329140

[blanchouins]

Same problem for me, case 03329184

regards,

[Polina]

Everyone,

Our QC team has confirmed that the issue is caused by some changes on the Microsoft side. We're researching the ways to fix the problem, however, at the moment there's not much that we can do. As soon as I get any updates, I'll post them here.

Thanks!

[sbk]

Same error. Case number: 03330296

[s2dt]

Same error. Case #03330806

[rockaut]

Hello there,

we have the same error with two of our tenants - case #03331146 .
One of them is missing a lot of items since the first error occurrence too. Not saying that veeam error has deleted those but maybe there's coincidence?

[Renaud]

Same here on public folders:

Processing mailbox ****@*****.onmicrosoft.com failed with error: Failed to synchronize item changes in folder: Contacts.. An internal server error occurred. The operation failed., IdAndSession.MailboxSession: session is not IMailboxSession

Case #03323944

[Polina]

Latest updates on this: We're in touch with MS and investigating the possible changes in parameters which potentially can cause the synchronization problems.
Stay tuned!

[coreypenford]

Same issue
Case #03331575

[S_Matasic]

We are seeing this with a few of our tenants as well. Case 03331885

[Renaud]

Seems to be fixed, no more error on my side.

[S_Matasic]

I had three jobs getting the error. Two seemed to be resolved when running on the 19th. Another that ran on the 18th still had the error. I am running the new job now to see if it errors.

EDIT:
I checked the job and it appears to have been resolved as it processed the public folders without error now.

[Polina]

Renaud, Steve,

Thank you for these updates. It seems that the issue is going to be completely resolved within the next few days; in the meanwhile, our engineers are continuing their research together with MS specialists.

[agrob]

It looks like MS changed something again. Last backup was successful, no errors at the moment

[coreypenford]

I'm still getting an error on our job
12/20/2018 9:05:01 AM :: Processing mailbox **************** failed with error: Failed to synchronize item changes in folder: IPM_SUBTREE.. An internal server error occurred. The operation failed., IdAndSession.MailboxSession: session is not IMailboxSession :: 0:00:57

[agrob]

maybe the changes are not deployed to all tenants / infrastructure at the moment. i would suggest to wait one or two days...

[coreypenford]

Can confirm that it is now working for us again. Cheers

[Polina]

According to our engineers, Microsoft has applied some fixes on their side and backup should work as expected without any issues.

If you still have any jobs failing with the "internal server error" though, please do let us know.