Maintain control of your Microsoft 365 data
Robvil
Expert
Posts: 172
Liked: 20 times
Joined: Oct 03, 2016 12:41 pm
Full Name: Robert
Contact:

Extremely slow reading from Office 365 cloud

Post by Robvil »

Hi,

I have opened a case, 02394834, on this.

I just noticed my O365 backup takes forever now .... as in 17B/s in transfer speed ... so a simpel incremental never completes.
It has been running fine until 10/13/17. After this date the transfer speed is nearly zero and has never completet as session.

Ohh, it´s not a local bandwidth issue.

Anyone seeing this?

Is Microsoft somehow inforcing speed transfer limits?

Regards
Robert
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by foggy »

Hi Robert, what Veeam Backup for Office 365 version are you using? Any chance this started to occur after the upgrade? Here's another topic with similar issue for more information.
Robvil
Expert
Posts: 172
Liked: 20 times
Joined: Oct 03, 2016 12:41 pm
Full Name: Robert
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by Robvil » 1 person likes this post

1.5

It helped to lower the active threads from 64 to 10.
Now 47GB data has been backup.
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by Mike Resseler »

Robert,

What is the speed that you see now?

Mike
(PS: you might want to play a bit with those active threads. It is difficult to tell you a formula what the ideal thread # is but playing a bit with it until you find the best setting for your environment is certainly my advice)
Robvil
Expert
Posts: 172
Liked: 20 times
Joined: Oct 03, 2016 12:41 pm
Full Name: Robert
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by Robvil »

well, i have a 100Mbit/s internet connection and it is using all bandwidth now, so it looks dine.
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by Mike Resseler »

Good to hear, although using all the bandwidth is maybe something you also don't want :-)

Note that in 1.5 you can set limits to your bandwidth per proxy: https://helpcenter.veeam.com/docs/vbo36 ... tml?ver=15

Cheers
Mike
Robvil
Expert
Posts: 172
Liked: 20 times
Joined: Oct 03, 2016 12:41 pm
Full Name: Robert
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by Robvil » 1 person likes this post

I know ... i limited it to 80Mbit/s after testing with the new thread count.
ketan.patel
Lurker
Posts: 2
Liked: never
Joined: Aug 21, 2018 2:52 pm
Full Name: Ket
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by ketan.patel »

I am experiencing the same issue. Veeam backup for o365 is extremely slow. its backing up hardly 1.5 MB/sec. we have a case 03138546 open with Veeam and they want me to test the bandwwith betweeen the veeam proxy and o365. I played around with threads but no luck.

has anyone beeen experiencing thee same isssue and any work around? Thanks in advance.
Polina
Veeam Software
Posts: 2939
Liked: 681 times
Joined: Oct 21, 2011 11:22 am
Full Name: Polina Vasileva
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by Polina »

Hi Ket,

If you have a minute to share the outcomes of your case investigation and tests, it'd be much appreciated. Was it because of the bandwidth?
Thanks!
ketan.patel
Lurker
Posts: 2
Liked: never
Joined: Aug 21, 2018 2:52 pm
Full Name: Ket
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by ketan.patel »

Hi Polina,

Bandwidth between EOL and Veeam proxy aappears to be fine. we have a case open and its not resolved yet. we are still running very low throughput. we have currently 7 jobs running and we are getting only 4-5MB/sec processsing rate. also confirmed with MS and EWS throtteling for our tenant is currently at rest. throttling policy has been disabled for o365 tenant by MS.
mbraak
Service Provider
Posts: 4
Liked: never
Joined: Nov 05, 2018 11:15 am
Full Name: MBR
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by mbraak »

Any update for us?
i also experience very slow backup performance (sharepoint backup)
Working with veeam and MS support but it seems that it isn't possible to speed up Office 365 Sharepoint backups. This makes Sharepoint useless for larger sites for now.

Looking forward to your reply.

MBR
-MBR-
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by Mike Resseler »

Hey MBR,

SharePoint indeed cannot be changed from the MSFT side (maybe it can but they won't). We are actively seeking to improve this and are researching different things within R&D. Unfortunately we haven't found a good solution for now. We are using all the best practices from MSFT around connecting to SharePoint Online and downloading data, but as you state yourself, sometimes it is extremely slow.

Do you already have a support case? We can use more logs to investigate and to see what is going on. From the logs we already have, it seems that there probably isn't a one-fit solution. It depends on the situation.
vipthomps
Service Provider
Posts: 60
Liked: 6 times
Joined: Dec 06, 2010 7:51 pm
Full Name: Eric Thompson
Location: Boston, MA
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by vipthomps »

Ketan,

What did you put in as a request with Microsoft to have the throttling policy disabled? Thanks!
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by Mike Resseler »

Eric,

This works only for Exchange and requires a support call. The problem we see is that MSFT does not want to do this for everyone. We first thought it was depending on your O365 size but that doesn't seem to be the case. What exact parameters they are using to allow or disallow it is not known (and they won't tell us :-))
tbeekmans
Influencer
Posts: 20
Liked: 5 times
Joined: Sep 07, 2018 12:13 pm
Full Name: Tom Beekmans
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by tbeekmans » 3 people like this post

I'm encountering the same problem, SharePoint backups are extremely slow between 30KB/s to 100KB/s, we just transferred 9 GB in 3 days, I opened a case at MS to see if there is any throttling (keep you all updated), bandwidth or resources aren't any issues on our backup server, seems more the limitation on the source or the API, I really think it's really necessary that Veeam and Microsoft get together for this problem, product isn't ready for full backup of large Office 365 SharePoint organisations yet, I tried to backup or complete SharePoint Office 365 organisation around 500GB after 2 weeks I stopped it didn't work.
Polina
Veeam Software
Posts: 2939
Liked: 681 times
Joined: Oct 21, 2011 11:22 am
Full Name: Polina Vasileva
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by Polina »

Hi Tom and welcome to the community!

Thank you for sharing this feedback. Please let us know what MS responds. Our team is researching possible options to address the challenge with SharePoint throttling and it's helpful to know the specific use cases.
Meanwhile, may I ask if you opened any cases with Veeam Support on this? and also did you try to play with the number of threads in proxy settings to improve the backup performance?

Thanks
mbraak
Service Provider
Posts: 4
Liked: never
Joined: Nov 05, 2018 11:15 am
Full Name: MBR
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by mbraak »

I had a case at Microsoft but that didn't gave me anything. Sharepoint Throttling is fixed and wont be release anywhere near future.
Veeam Support also officially stated Sharepoint Backups are currently not there best implementation and a lot of customers are complaining about it.
Development is researching this problem and are searching for improvementste be make but this wont be anytime near future also.
I do have one other contact (Veeam System Engineer) which is informing internally to see if there is anything left to improve things for us.
I hope to hear from him anywhere next week.

In advance i am evaluating an other promising Office 365 product atm of which the first performance tests are very promising.
As soon as i have more concrete information i will let you all know.
-MBR-
Mike Resseler
Product Manager
Posts: 8044
Liked: 1263 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by Mike Resseler »

Hi Mbraak,

Allow me to correct one thing.

1. Official support statement: (Please DM me your support case because it is not correct). We are using the correct implementation per Microsoft recommendation (and request) to connect to SharePoint.

2. We are indeed working on improvements and getting it faster. We are also working together with MSFT around this. Not anytime near future might be faster than you think. (Hence the reason why we ask logs from everyone who has very large document libraries and is suffering from this. Because that seems to be the root cause). I do not expect it to take 6 months to build improvements.

3. Your Veeam System Engineer will need to find me :-)

Cheers
Mike
whimark
Influencer
Posts: 18
Liked: 5 times
Joined: May 14, 2018 11:54 am
Full Name: Markus Johansson
Location: Gothenburg
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by whimark »

I was told by Microsoft support that by using several accounts could increase performance, so I would suggest that it would be possible to assign more Office365 accounts to the Veeam proxies for the same organisation like below.

Also they said that if to many request comes from the same account to sharepoint they will cut the performance for that user account

Proxy one: Veeam1@zxczc.onmicrosoft.com
Proxy two: Veeam2@zxczc.onmicrosoft.com
Proxy three: Veeam3@zxczc.onmicrosoft.com
Polina
Veeam Software
Posts: 2939
Liked: 681 times
Joined: Oct 21, 2011 11:22 am
Full Name: Polina Vasileva
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by Polina » 1 person likes this post

Hi Markus,

That's correct, as when you're using multiple accounts you can "distribute requests load" on the service and "postpone" throttling. However, there are both pros and cons to consider.
As Mike wrote earlier, we're currently working on some enhancements to speed up processing.
tbeekmans
Influencer
Posts: 20
Liked: 5 times
Joined: Sep 07, 2018 12:13 pm
Full Name: Tom Beekmans
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by tbeekmans »

Good news, a good improvement on the backup time for SharePoint backup, Veeam provided a hotfix (Fix_149892_211cc6a369) this fix is still not released officially but will be soon for more customers hopefully.
The speed isn’t extremely high however it seems that the problem more was the high amount of items in SharePoint that needed to be backup-ed at the end.
The backup did run normally 2-3 days with the fix 1 hour, big improvement.
Again Veeam R&D anticipates very fast on problems with products, thanks for that.
I also did create a case at Microsoft however this moves very slowly to get something done at the throttling and I don’t expect many improvement on this topic.
Polina
Veeam Software
Posts: 2939
Liked: 681 times
Joined: Oct 21, 2011 11:22 am
Full Name: Polina Vasileva
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by Polina »

Hi Tom,

Thanks for this feedback!
You're right, the fix is currently provided upon request only, as we would like to look more closely on the results in real-life environments. And it's great to know you're satisfied with the outcome. Could you please share your case ID here? this would help us to keep in sync with the support team
whimark
Influencer
Posts: 18
Liked: 5 times
Joined: May 14, 2018 11:54 am
Full Name: Markus Johansson
Location: Gothenburg
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by whimark »

Hi Polina,

Is it possible to send the hotfix to me?

//Markus
Polina
Veeam Software
Posts: 2939
Liked: 681 times
Joined: Oct 21, 2011 11:22 am
Full Name: Polina Vasileva
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by Polina »

Hi Markus,

Fixes can only be provided by our support team; please create a support call and post your ID in this thread, so that we could track it.
Thanks!
tbeekmans
Influencer
Posts: 20
Liked: 5 times
Joined: Sep 07, 2018 12:13 pm
Full Name: Tom Beekmans
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by tbeekmans »

The case number regarding the hotfix was, 03279696
Polina
Veeam Software
Posts: 2939
Liked: 681 times
Joined: Oct 21, 2011 11:22 am
Full Name: Polina Vasileva
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by Polina »

Thanks, Tom!
whimark
Influencer
Posts: 18
Liked: 5 times
Joined: May 14, 2018 11:54 am
Full Name: Markus Johansson
Location: Gothenburg
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by whimark »

There was a slight performance gain with the new hotfix, but in my case being able to assign different accounts to the proxies would probably solve most of the issues I have with the performance when backing up Sharepoint sites.
Polina
Veeam Software
Posts: 2939
Liked: 681 times
Joined: Oct 21, 2011 11:22 am
Full Name: Polina Vasileva
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by Polina »

Thank you for the feedback, Markus. Our R&D team continues their work on performance enhancements and all the possible options are being tested now.
MILJW002
Enthusiast
Posts: 46
Liked: 5 times
Joined: Apr 29, 2017 2:26 pm
Full Name: James Miller
Location: Adelaide, South Australia
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by MILJW002 »

Hi,

I’ve had slow backups for SharePoint and OneDrive since first installing version 2. I’ve played with reducing the thread count, but it hasn’t made a significant difference.

I’ve just logged a case as I’d love to try the fix to see if it helps.

03338555

James
Polina
Veeam Software
Posts: 2939
Liked: 681 times
Joined: Oct 21, 2011 11:22 am
Full Name: Polina Vasileva
Contact:

Re: Extremely slow reading from Office 365 cloud

Post by Polina »

Hi James,

It looks that the fix has been already provided to you. Can you post some updates on performance improvements once it's applied?

Thanks
Post Reply

Who is online

Users browsing this forum: No registered users and 16 guests