Maintain control of your Microsoft Office 365 email data
tomhkr
Enthusiast
Posts: 40
Liked: 9 times
Joined: Feb 03, 2014 7:40 am
Contact:

Select mailboxes takes long time

Post by tomhkr » Sep 28, 2016 12:07 pm 1 person likes this post

I have about 60,000 mailboxes in my Offce 365 tenant. Trying out the Veeam Office 365 backup, it stucks with selecting mailboxes to backup. "Loading mailboxes.." stays on for very long time (30+ minutes) and doesn't display any boxes.

How long should I wait?

Process "Veeam.Archiver.Service.exe" generates incoming network traffic with about 150-300 kByte/s but stop occasionally.

tomhkr
Enthusiast
Posts: 40
Liked: 9 times
Joined: Feb 03, 2014 7:40 am
Contact:

Re: Select mailboxes takes long time

Post by tomhkr » Sep 28, 2016 12:42 pm

Just noticed the process has allocated all RAM and swapping heavily. Need a beefier server for this.. it has 8 GB RAM.

Mike Resseler
Veeam Software
Posts: 4993
Liked: 528 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Select mailboxes takes long time

Post by Mike Resseler » Sep 28, 2016 12:54 pm 1 person likes this post

Hi,

This is interesting. So you are trying to select some mailboxes but it can't enumerate the list of mailboxes. Did the process finally succeeded or never ended?
In the meantime I will try to get more information on the RAM and the swapping

Thanks
Mike

tomhkr
Enthusiast
Posts: 40
Liked: 9 times
Joined: Feb 03, 2014 7:40 am
Contact:

Re: Select mailboxes takes long time

Post by tomhkr » Sep 28, 2016 1:41 pm

To clarify, these are the steps taken:
  • Add backup job
  • At "Select mailboxes to backup" i choose "Backup the following mailboxes:" and press "Select.."
  • The "Select mailboxes" box displays "Loading mailboxes...." forever. The process consumes all memory there is and nothing happened for about an hour, so I killed it.
I'm giving it a new try at a machine with 32 GB of RAM instead, and will get back in this thread with the results.

Dima P.
Veeam Software
Posts: 8783
Liked: 647 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: Select mailboxes takes long time

Post by Dima P. » Sep 28, 2016 4:45 pm 1 person likes this post

tomhkr,

Please let us know how it goes. We might need get access to the application debug logs in case of a failure.

tomhkr
Enthusiast
Posts: 40
Liked: 9 times
Joined: Feb 03, 2014 7:40 am
Contact:

Re: Select mailboxes takes long time

Post by tomhkr » Sep 29, 2016 7:23 am

So it finally succeeded in loading all the mailboxes with a beefier server.
The problem was, I think, that with 60.000 mailboxes the first server ran out of memory and began swapping. A swapping server seldomly does any good.

Next problem I ran into is the "An entry with the same key already exists" which I see there is a thread on already and fix coming.

It is still slow, though. Loading all the boxes takes 15-30 MINUTES. This on a server with 32 GB RAM and a 10 GbE internet connection.

There really ought to be some sort of filtering option when selecting mailboxes or creating jobs dynamically based on filter criterias.

This could be accomplished with one of the following:
  • Mailbox membership (possibly problematic?; mailbox can be member of multiple groups)
  • Retention policy
  • Custom attributes
Look, it became a feature request! ;)

Dima P.
Veeam Software
Posts: 8783
Liked: 647 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: Select mailboxes takes long time

Post by Dima P. » Sep 29, 2016 10:30 am

tomhkr,

Thanks - good catch! Is it possible to share the logs anyway? I bet DEV team would like to investigate your case.

tomhkr
Enthusiast
Posts: 40
Liked: 9 times
Joined: Feb 03, 2014 7:40 am
Contact:

Re: Select mailboxes takes long time

Post by tomhkr » Sep 29, 2016 11:04 am

Certainly. Where can I send them?

Dima P.
Veeam Software
Posts: 8783
Liked: 647 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: Select mailboxes takes long time

Post by Dima P. » Sep 29, 2016 12:10 pm

If you have a dropbox or googledrive account (any other cloud hosting works too), just send me the link via PM. You can also upload the logs to the google drive folder I've sent you via PM.

Kostya
Veeam Software
Posts: 104
Liked: 28 times
Joined: Jun 18, 2012 9:38 am
Full Name: Kostya Yasyuk
Contact:

Re: Select mailboxes takes long time

Post by Kostya » Oct 04, 2016 6:31 pm

Confirmed. Will be improved in the GA build.

Kostya
Veeam Software
Posts: 104
Liked: 28 times
Joined: Jun 18, 2012 9:38 am
Full Name: Kostya Yasyuk
Contact:

Re: Select mailboxes takes long time

Post by Kostya » Oct 06, 2016 4:16 pm

tomhkr wrote:It is still slow, though. Loading all the boxes takes 15-30 MINUTES. This on a server with 32 GB RAM and a 10 GbE internet connection.
Tom, sent you a link via PM to the experimental performance improvements and fixes.

JaySt
Service Provider
Posts: 98
Liked: 20 times
Joined: Jun 09, 2015 7:08 pm
Full Name: JaySt
Contact:

Re: Select mailboxes takes long time

Post by JaySt » Oct 07, 2016 8:41 pm

Tom, just curious, could you also share some findings when you're able to run a actual back-up? Performance for example?
interesting to see what the software is able to do with that volume of mailboxes.
Veeam Certified Engineer

jchillman
Service Provider
Posts: 6
Liked: never
Joined: Jan 13, 2016 8:49 pm
Full Name: James Chillman
Contact:

Re: Select mailboxes takes long time

Post by jchillman » Oct 09, 2016 7:37 pm

I would also be interested in the performance data. We have a couple of Large O365 Domains that we will be looking to backup.

Mike Resseler
Veeam Software
Posts: 4993
Liked: 528 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: Select mailboxes takes long time

Post by Mike Resseler » Oct 10, 2016 10:47 am

Hi James, without being able to give you performance data for now... When you say large... What # are you talking about?

jchillman
Service Provider
Posts: 6
Liked: never
Joined: Jan 13, 2016 8:49 pm
Full Name: James Chillman
Contact:

Re: Select mailboxes takes long time

Post by jchillman » Oct 10, 2016 12:23 pm

Hi Mike,

We have one around the 25k Seat mark and a couple around the 5-6k mark.

Post Reply

Who is online

Users browsing this forum: No registered users and 1 guest