Teams Export API - Excessive Costs

[m.novelli]

As a Microsoft Partner I let our Customers to see the Cost + Billing blade, is your right

You can migrate the subscription with no downtime to my Company , probably you will even save money, we just invoice what we see in the Cost + Billing blade with no additional fee

Marco

[sumeet]

Hi All,

As informed earlier, we ask all our clients to configure Azure Budget alerts on the resource group.
One of our clients, where in we setup teams backup in Jan, had a similar alert when the first full backup for teams ran. Which was expected as first full backup will use the Teams export API to download all teams channel posts.

But then no alerts in Feb. And now in March first week, the alert is raised for usage of $85 (which when converted to USD and then divided by 0.00075 is a very high value). We checked the teams admin report and the teams channel posts usage for the last one month is 817. Do not see why the daily incremental backup would use so many API calls. Unless any of the latest 8.1 patches causes a full rescan or re-sync.

Have this case opened - 07630174 and getting some not so useful replies from support. Have requested to get this escalated.

As requested many times above, we need a way to know how many times VBM365 calls the API - this is important to have a control on this.

-Sumeet.

[Polina]

Hi Sumeet,

There were no changes in 8.1 patches, but a full sync of messages from private and shared channels should be expected after upgrading to 8.1:

To ensure backup consistency, a full backup of messages from private and shared Teams channels will be performed after you upgrade Veeam Backup for Microsoft 365 from v7.0 or v7a to v8.1. This may result in a one-time increase in costs for using the Teams Exports APIs. The full backup is necessary because your backup repositories may already contain some of the messages, but the date when Microsoft APIs began returning messages from private and shared channels is unknown and may vary across different channels, teams, and organizations.

Atm, I can't provide you with any ETA for the ability to control the paid requests count, but I can assure you that it is on our roadmap.

Thanks!

[sumeet]

Hi Polina,

Good to know that this is on your roadmap.

This is not a upgrade from v7x to v8.1. This is freshly v8.0 setup, with an upgrade from v8.0 to v8.1.
Based on your above message, upgrade from v8.0 to v8.1 should not require a full backup of messages.

So must be something else. We have had to disable the backup job, until more details are available.
The latest update is that the case is now escalated. Waiting for more details.

-Sumeet.

[Polina]

Let's see what support finds. I'll keep an eye on your case.

[sumeet]

Thanks Polina.

The case is slow - I have provided lot of information, but minimal response from support - not sure if all the info I have provided is being absorbed or considered.

Will be helpful if this can be escalated.

-Sumeet.

[sumeet]

Hi,

I have got an update in the case that this case is now referred to R&D team. Will wait for more updates.

-Sumeet.

[sumeet]

Hi Team,

Now more than a month and the update on the case is that it is with R&D team.
We and our clients are paying extra cost for the high usage of the API. Will appreciate if this can be looked into case 07630174

Thanks,
-Sumeet.

[Polina]

Hi Sumeet,

AFAIK, after inspecting your logs our engineers have not detected any abnormal or excessive requests that potentially could have led to bigger bills from Microsoft. They suggested to perform one more isolation test and are expecting an updates from your side (or maybe already received them - I might have missed the latest updates).

[sumeet]

Hi Polina,

I was asked to run a Fiddler tests in the last week of March which was done and logs provided. Since then the only update I have is that the issue is with R&D team.
No further update.

What happened today is that we have got Azure budget alert for our own Teams backup. When I check the cost in Azure, the cost has increased slightly since v8 upgrade, but drastically increased since we started getting an error in teams backup with a teams starting 23rd March and has kept increasing since that day onwards. The behavior is different than what we are observing at our client, where-in the cost for each day is fixed, our is increasing each day and has got to a stage of 15x since what it was with v7.

Regards,
-Sumeet.

[Polina]

Hi Sumeet,

1) Have you already shared this feeback with the support team?
2) What error specifically do you mean?

[sumeet]

Hi Polina,

Yes this feedback has been shared with support team. I have given them daily cost graph for the resource group of Teams export API and it shows gradual increase since v8 upgrade.

Day before yesterday, support team provided few config.xml and proxy.xml changes to apply, informing this should fix the problem.
We applied the changes to our backup and also to our clients. We do not see any change in the cost for our own backup, haven't checked with our client yet as it is holiday here, will check on Monday. But looking at our own cost usage, I do not expect this to make any difference. In two more weeks, it will be 60days since we have opened the case and still waiting to be resolved. With extra cost being paid each day this is taking to solve.

Error is -- Failed to process team: <team-name>. One or more dependencies failed
The response status code does not indicate success: 400 (Bad Request). {"error":{"code":"BadRequest","message":"Provided hosted content identifier is invalid.","innerError":{"date":"2025-04-24T11:38:44","request-id":"xxxx-xxx-xxx-xxx-dbfcf050ec30","client-request-id":"xxxx-xx-xxx-xxxxx-bfcf050ec30"}}}


-Sumeet.