M365 Mailbox Backup increase since friday / latest August MS Patch installation

[SteelContainer]

Case #07219006

I'm in the same boat as some above where issue started around 27/3 and is still persisting.
Some clients that normally do 20GB incrementals are doing 100-500GB per day
I have also logged a case with MS as per Veeam tech instructions.
If this isn't resolved soon it is really going to dig into our ability to provide a cost effective solution to our clients.

[m.novelli]

I'm wondering if also other M365 Backup Vendors are in the same boat, like Acronis , Keepit, CodeTwo... Maybe Veeam could contact them to share this issue and do some brainstorming with / against Microsoft

Marco

[Gostev]

Yes, other backup vendors are affected as well. Our partner team at Microsoft works with other backup vendors too, this is how we know it's not just us.

Of course, it's more "publicly visible" in case of Veeam simply because we protect more M365 mailboxes than most other vendors combined.

[m.novelli]

Well done Gostev

Now I'm wondering how a Product Manager at Microsoft can give "green light" to a backend update that impact in such a HUGE way the backup ecosystem...

By the way Gostev, do you know what technology there is behind the Microsoft 365 Backup that started to appear in some Microsoft 365 tenants? https://learn.microsoft.com/en-us/micro ... -worldwide

Marco

[Gostev]

Yes, in simple terms this will allow "entire mailbox snapshots". We will integrate with this tech once released b/c it is complementary in the same way as storage-based snapshots are complementary to proper backups for VMware protection.

[NN25]

Hi,

Is there any update regarding this topic?

Backup increased now from 300GB to 1.2TB.

[Gostev]

Microsoft is on it. They are also working to provide a statement we can share publicly, but as all such statements it needs to gather many approvals first.

We already have an excellent explanation from them about what happened and how they plan to minimize similar changes in future, but these details are provided under NDA and we're not allowed to share them publicly.

[Gostev]

Microsoft provided us with the following statement that we're allowed to share with our customers:

Due to changes we are making to optimize search and compatibility functionality within our service, this action led to re-indexing for some of our customers' data.
The re-indexing is not indicative of a change to fields material to our BCDR and storage partners, so we will be working on a change to minimize the re-indexing that is prompting another backup of customer data.
We made changes over the weekend that should pause the unexpected behavior and those changes will be in place until Microsoft and our partners are able to make longer term enhancements to avoid future issues from taking place.
Thank you for your patience.

[m.novelli]

"some" data

I think they re-indexed the same data over and over again...

Marco

[VNR-LF]

"some" data

I think they re-indexed the same data over and over again...

Marco

yep, "some"...

it sure seems like some mailboxes have been re-indexed over and over again. for some of our mailboxes, three or four times the total mailbox size has been backed up over the past weeks. but looking at the past 3 days, backup sizes seem to be back to levels we've been used to before the issue started.

glad this is fixed and fingers crossed this issue will not resurface again.

[superdevo]

I am still seeing huge incrementals

[Polina]

Hi David,

Do you have a case with our support? Also, in which region/data center are you seeing the issue?

[superdevo]

I have a case going with MS.
Their reply as of this morning:

I’m continuing to work with the product team on this. I don’t yet have an update unfortunately. This might take a little bit of time as there are multiple tenants impacted as you’ve noted. I apologize for this continued inconvenience here.

Am I the only one still dealing with this?

Will post here further updates.

Thanks for checking Polina

PS: region: Americas/US