M365 Mailbox Backup increase since friday / latest August MS Patch installation

[m.novelli]

I confirm the issue is hitting hard (again) also some of my customers

Marco

[squebel]

Support came back on case 07583604 and said something cryptic: " At the same time, there are some concerns springing up on our side during the last 24 hours regarding a possible reoccurrence of the Exchange issue documented in https://www.veeam.com/kb4483." However, no one has provided any more info as to why people on the Veeam side believe the reoccurrence is happening. I have provided a new enhanced log bundle.

[Mike Resseler]

To everyone,

I confirm that we are indeed seeing this occurrence mentioned above again. I deeply apologize for it. While the root cause is always the same (see the KB in the message above me), the "field" or "property" that changes is not always the same, and some of these properties are not visible to a customer, but still very important. There are a few patterns when this happens:

1. MSFT makes a change causing a specific (mostly hidden) property to change for each item, hence, we start to see (some sort off) "full" backup. (Please read the KB to have more technical details, this is just high-level explanation)
2. It comes in waves (which is expected as MSFT rolls out in waves the changes to their M365 fleet)
3. We escalate and each time we get notifications that things have been stopped (meaning, rolling out the change to the rest of the fleet)
4. Repeat...

We are heavily investigating possibilities (together with MSFT) to find a solution to solve this, so that our solution "detects" this issue and does not consider it a newly needed download. But I am afraid it will take some time...

[squebel]

Mike, thank you for providing an official update from Veeam's perspective

What can we, your customer, do about it at this time? I see no solution provided here or in the KB article.

[doktornotor]

Considering this, I'd say the only thing you can do as customer is -- ditching the MSFT cloud.

[squebel]

"Considering this, I'd say the only thing you can do as customer is -- ditching the MSFT cloud."

The problem isn't that MSFT is changing stuff on their platform. It's their platform to do what they want with as long as they provide the service they said they would. That service probably does not include any kind of SLA around accessing the data from outside the environment to do backups. Their responsibility isn't to make sure that other people's hacked in backup software works properly. These 3rd party integrations trying to read data out of MSFT cloud sold as the way to protect the data is the problem. The real problem is that we, as a Veeam Service Provider partner, are 100% on the hook to our customer to answer to them why the backups of THEIR data isn't working correctly. Why the backups of THEIR data is taking 10x more space than we originally told them it would. And the main problem here is that Veeam (and others) seem totally helpless and can't provide good direction to their Service Providers on how to fix it because there is no way to fix it.

[doktornotor]

Their responsibility isn't to make sure that other people's hacked in backup software works properly.

As long as people are willing to pay $$$$$ for a service that keeps breaking their backups more or less on a quarterly basis, nothing will change.

[FrancWest]

Last night we again had the issue that backups of some mailboxes were full backups again. Backup size of 30GB were no exception. Backup job took 12 hours to complete. Anyone experiencing this also last night?

[Polina]

@FrancWest

Did you open a case with Veeam support on this increase? Can you provide me its ID?

[FrancWest]

Hi,

just did, case number #07609470

Franc.

[FrancWest]

according to support here are no known issues with some mailboxes doing full backups:

I couldn`t find any mention of the backups doing of full.

I had to enable peritemlogging on the proxy to further troubleshoot.

[agrob]

hi franc, i guess it has do to with, again, changes microsoft made... we had the same problem about 2 weeks ago with 2.6TB of transfared data in 3 Days (normaly arround 50GB). it is not the first time this happens as you can see in the post... you're problem might be the same if no full backups are made from veeam side

[FrancWest]

Yeah, I told support that, but there were no other cases with this issue according them. Last weekend I had one mailbox again that did a full backup (even for the second day in a row). The last 2 days the backup is incremental again.

[StephanG]

We also have an issue with Exchange (but also OneDrive) ... the backup jobs are not finishing. This also started around the 19th / 20th of this month
Case # 07615068

We did not get a warning or error - i just saw it because i connected to the console.

[squebel]

Veeam Support is telling me that this bug does not impact individual mailboxes for a tenant but would impact the entire tenant. That does not seem to be what others are experiencing.

[RobertoM]

I understand this is now somewhat of a... dead horse to beat.

In the grand scheme of things, we as a reseller / cloud provider are fortunate enough to operate where companies are very small compared to the rest of the world.
We have set aside any hope to be able to estimate storage needs and now reserve "total exchange online size times four".

This is an example of the last 365 days at a ~60 mailboxes customer with a 1-year snapshot retention.
Shows quite well what happened in the year before that:




Anyway, just to add to what squebel wrote, it is our observation that it is per-mailbox and not involving the entire tenant.
The same customer above, in the days Stephan also mentioned:

[fousey]

I am experiencing the same problem again. Increase of size, Tuesday 28GB(normal), Wednesday 300GB, Thursday 410GB.

It goes like this every month now since last year... and this is not even the worst month. Last month it started with 490GB next day 5.2TB and day after 965GB.

The backup takes really long now from 3 hours to 33+ hours and don't get me started about the Copy job.

[sbk]

Veeam Case # 07673159

Seeing the same thing in our tenant, starting on April 4th, 2025. Had the same issue last year in April as well. We have a large environment, and last time, the only "fix" was that we had to start over. Create a new repo, run a new full backup (literally takes a month running 24x7 to complete), and keep our fingers crossed once the full finally completes that; the next run will just be incremental. It's a really big challenge to have to more than double your storage at any given time and or be without backups for +/- a month... It would be nice if Veeam would figure this out and account for this known issue that has now been present for YEARS!