Is there a concrete fix to the throttling issue?

[cparker4486]

Hello,

I reached out to support (#03134789) to report that the OneDrive repository for my own account was not finishing and would always end with a 500 error. After many days of troubleshooting, and back and forth, support said the problem is on my end (MS's to be specific). That MS is probably throttling my account and that Veeam is unable to detect this exactly but that it's the most likely explanation.

I came to the forum to see what I could find but I found nothing concrete (I don't claim to have found and/or read every single post on the topic). There was talk of changing some settings on the proxy (tried that didn't work) and calling MS to "unthrottle" the tenant (haven't tried yet).

What's Veeam's current best course of action for this?

[Mike Resseler]

Hi Chris,

The "unthrottling" of the tenant works for exchange online (but Microsoft decided on whether to do it or not, not sure what the parameters are for receiving a yes or no...) but not for SharePoint Online. And OneDrive for Business is part of SharePoint Online.


I tried looking at your case but I couldn't find really good information in it so I am hoping that you can help me a bit here.
1. What changes did you make with support (for example, did you change the threads in the proxy?)
2. That specific OneDrive for Business account, does it have a lot of (most probably small) files?
3. Can you give me the exact 500 error?
4. Are the logs in the support call (can't see it for some reason, might be because I am outside the network...)
Thanks
Mike

[cparker4486]

Hi Mike,

Thanks for looking into my issue. To be honest, I don't feel like support is really aware of this issue fully. It took a long time for the rep to come back with the suggestion that it may be MS doing throttling. They also didn't mention that it didn't apply to OneDrive for Business because I don't think they knew about it.

I came to the forum on my own to see if there was any other information out there. The rep didn't suggest changing threads on the proxy. I found that suggestion in the forum. I believe I changed it to 10. There was no change.

I believe the only other change I made was to increase the resources on the proxy but that didn't make any difference. We're a very small network and the proxy is 2016 Server Core that originally had 4GB of memory. I gave it 8GB.

2. Yes. Almost 39,000 with an average of ~1.5MB each.
3. "9/21/2018 9:24:25 AM :: Processing OneDrive username (https://domain-my.sharepoint.com/personal/username) finished with warning: Failed to backup item: , The remote server returned an error: (500) Internal Server Error. :: 3:29:40"

The message makes me think there might be a file that the job can't backup. But that could be a red herring.

4. Yes, I've uploaded logs a few times. I will do it again.


Thank you!

[Mike Resseler]

For 4. I was just asking, if they are there, no need to upload again. (I just can't see them but that is not important )

For the others, let me try and hear if there is already something else known for this

[Mike Resseler]

By the way, is it always the same file or files?

[cparker4486]

I can't say for certain about the file(s) but the error is always the same: "Failed to backup item: ,".

[Mike Resseler]

By the way Chris,
I see the support call is closed? Is that correct? Any reason for it?

[cparker4486]

The last thing support suggested was contrary to guidance they'd given me earlier so I didn't write back and the case closed automatically.

[Mike Resseler]

Hmmm. OK. I will ask someone to look into it. Give me a bit of time (I am at Ignite at the moment)

[cparker4486]

Much appreciated.

[Fatboy40]

I use a product, which I won't name, to sync a local folder to OneDrive for Business. It performs a nightly backup of various documents so that we have them for disaster recovery.

However recently I've been getting an increased amount of OneDrive / SharePoint throttling issues as well, causing the sync to fail. Maybe Microsoft have been altering their throttling thresholds recently?

[Polina]

Hi Clive,

It's not that I have heard of. I googled it a bit and it seems that the similar issues with SharePoint Online throttling and the corresponding 429 errors are splashing to multiple customers simultaneously from time to time.

[Mike Resseler]

@cparker4486
I talked to support and they asked if you could open a request again. Because of privacy regulations and such we have to destroy your logs after the case is closed. (Which is a good thing actually...). But because of that, our higher tier can't dive into the case anymore. Can you open again and let me know. I will then inform the teamlead of the case number so they can check if my feeling is correct...

[cparker4486]

Sorry for the late reply. Thank you, Mike. I will do it today.

[cparker4486]

The case # is 03230102.

[cparker4486]

Support responded with a CU download that has a bugfix for the error I'm getting. The job is rerunning now.

[Mike Resseler]

Thanks Chris,

I just passed the case to our support team lead

Brgds,
Mike

[mlange]

Is this download available? I'm getting the same 500 error on one user.

[Polina]

Hi Matt,

Which build are you running? Did you install the latest KB2809?

Thanks!

[mlange]

Hi Polina,

I'm on 2.0.0.594. I have not installed the latest patch as I didn't know there was one. I'll install it and see what happens. Thanks!

[Polina]

Great, I'm looking forward to getting the news from you. And please make sure prior to installing this patch that you don't have Veeam Backup & Replication 9.5 Update 3 or 3a running on the same server as your VBO.

Thanks!

[mlange]

Still having the same 500 error on one particular user. VBR and VBO are on separate boxes.

[Polina]

Matt,

Please open a support case and let our engineers investigate the logs. Also, take a minute to share your case ID here.

Thanks!

[mlange]

Case ID 03384922.

[Polina]

Hi Matt,

Did you have a chance to follow the support team recommendations? It's indeed likely that you're experiencing the issue covered in KB2745.

[mlange]

I got in contact with the user today, and while she wasn't having any issues with the file being blocked from download or opening, we have removed the file and am waiting for the next run to see what happens.