Select mailboxes takes long time

Maintain control of your Microsoft Office 365 email data

Select mailboxes takes long time

Veeam Logoby tomhkr » Wed Sep 28, 2016 12:07 pm

I have about 60,000 mailboxes in my Offce 365 tenant. Trying out the Veeam Office 365 backup, it stucks with selecting mailboxes to backup. "Loading mailboxes.." stays on for very long time (30+ minutes) and doesn't display any boxes.

How long should I wait?

Process "Veeam.Archiver.Service.exe" generates incoming network traffic with about 150-300 kByte/s but stop occasionally.
tomhkr
Enthusiast
 
Posts: 38
Liked: 7 times
Joined: Mon Feb 03, 2014 7:40 am

Re: Select mailboxes takes long time

Veeam Logoby tomhkr » Wed Sep 28, 2016 12:42 pm

Just noticed the process has allocated all RAM and swapping heavily. Need a beefier server for this.. it has 8 GB RAM.
tomhkr
Enthusiast
 
Posts: 38
Liked: 7 times
Joined: Mon Feb 03, 2014 7:40 am

Re: Select mailboxes takes long time

Veeam Logoby Mike Resseler » Wed Sep 28, 2016 12:54 pm 1 person likes this post

Hi,

This is interesting. So you are trying to select some mailboxes but it can't enumerate the list of mailboxes. Did the process finally succeeded or never ended?
In the meantime I will try to get more information on the RAM and the swapping

Thanks
Mike
Mike Resseler
Veeam Software
 
Posts: 3382
Liked: 384 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: Select mailboxes takes long time

Veeam Logoby tomhkr » Wed Sep 28, 2016 1:41 pm

To clarify, these are the steps taken:
  • Add backup job
  • At "Select mailboxes to backup" i choose "Backup the following mailboxes:" and press "Select.."
  • The "Select mailboxes" box displays "Loading mailboxes...." forever. The process consumes all memory there is and nothing happened for about an hour, so I killed it.

I'm giving it a new try at a machine with 32 GB of RAM instead, and will get back in this thread with the results.
tomhkr
Enthusiast
 
Posts: 38
Liked: 7 times
Joined: Mon Feb 03, 2014 7:40 am

Re: Select mailboxes takes long time

Veeam Logoby Dima P. » Wed Sep 28, 2016 4:45 pm 1 person likes this post

tomhkr,

Please let us know how it goes. We might need get access to the application debug logs in case of a failure.
Dima P.
Veeam Software
 
Posts: 6698
Liked: 479 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: Select mailboxes takes long time

Veeam Logoby tomhkr » Thu Sep 29, 2016 7:23 am

So it finally succeeded in loading all the mailboxes with a beefier server.
The problem was, I think, that with 60.000 mailboxes the first server ran out of memory and began swapping. A swapping server seldomly does any good.

Next problem I ran into is the "An entry with the same key already exists" which I see there is a thread on already and fix coming.

It is still slow, though. Loading all the boxes takes 15-30 MINUTES. This on a server with 32 GB RAM and a 10 GbE internet connection.

There really ought to be some sort of filtering option when selecting mailboxes or creating jobs dynamically based on filter criterias.

This could be accomplished with one of the following:
  • Mailbox membership (possibly problematic?; mailbox can be member of multiple groups)
  • Retention policy
  • Custom attributes

Look, it became a feature request! ;)
tomhkr
Enthusiast
 
Posts: 38
Liked: 7 times
Joined: Mon Feb 03, 2014 7:40 am

Re: Select mailboxes takes long time

Veeam Logoby Dima P. » Thu Sep 29, 2016 10:30 am

tomhkr,

Thanks - good catch! Is it possible to share the logs anyway? I bet DEV team would like to investigate your case.
Dima P.
Veeam Software
 
Posts: 6698
Liked: 479 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: Select mailboxes takes long time

Veeam Logoby tomhkr » Thu Sep 29, 2016 11:04 am

Certainly. Where can I send them?
tomhkr
Enthusiast
 
Posts: 38
Liked: 7 times
Joined: Mon Feb 03, 2014 7:40 am

Re: Select mailboxes takes long time

Veeam Logoby Dima P. » Thu Sep 29, 2016 12:10 pm

If you have a dropbox or googledrive account (any other cloud hosting works too), just send me the link via PM. You can also upload the logs to the google drive folder I've sent you via PM.
Dima P.
Veeam Software
 
Posts: 6698
Liked: 479 times
Joined: Mon Feb 04, 2013 2:07 pm
Location: SPb
Full Name: Dmitry Popov

Re: Select mailboxes takes long time

Veeam Logoby Kostya » Tue Oct 04, 2016 6:31 pm

Confirmed. Will be improved in the GA build.
Kostya
Veeam Software
 
Posts: 104
Liked: 28 times
Joined: Mon Jun 18, 2012 9:38 am
Full Name: Kostya Yasyuk

Re: Select mailboxes takes long time

Veeam Logoby Kostya » Thu Oct 06, 2016 4:16 pm

tomhkr wrote:It is still slow, though. Loading all the boxes takes 15-30 MINUTES. This on a server with 32 GB RAM and a 10 GbE internet connection.

Tom, sent you a link via PM to the experimental performance improvements and fixes.
Kostya
Veeam Software
 
Posts: 104
Liked: 28 times
Joined: Mon Jun 18, 2012 9:38 am
Full Name: Kostya Yasyuk

Re: Select mailboxes takes long time

Veeam Logoby JaySt » Fri Oct 07, 2016 8:41 pm

Tom, just curious, could you also share some findings when you're able to run a actual back-up? Performance for example?
interesting to see what the software is able to do with that volume of mailboxes.
Veeam Certified Engineer
JaySt
Service Provider
 
Posts: 61
Liked: 15 times
Joined: Tue Jun 09, 2015 7:08 pm
Full Name: JaySt

Re: Select mailboxes takes long time

Veeam Logoby jchillman » Sun Oct 09, 2016 7:37 pm

I would also be interested in the performance data. We have a couple of Large O365 Domains that we will be looking to backup.
jchillman
Service Provider
 
Posts: 6
Liked: never
Joined: Wed Jan 13, 2016 8:49 pm
Full Name: James Chillman

Re: Select mailboxes takes long time

Veeam Logoby Mike Resseler » Mon Oct 10, 2016 10:47 am

Hi James, without being able to give you performance data for now... When you say large... What # are you talking about?
Mike Resseler
Veeam Software
 
Posts: 3382
Liked: 384 times
Joined: Fri Feb 08, 2013 3:08 pm
Location: Belgium, the land of the fries, the beer, the chocolate and the diamonds...
Full Name: Mike Resseler

Re: Select mailboxes takes long time

Veeam Logoby jchillman » Mon Oct 10, 2016 12:23 pm

Hi Mike,

We have one around the 25k Seat mark and a couple around the 5-6k mark.
jchillman
Service Provider
 
Posts: 6
Liked: never
Joined: Wed Jan 13, 2016 8:49 pm
Full Name: James Chillman

Next

Return to Veeam Backup for Microsoft Office 365



Who is online

Users browsing this forum: No registered users and 1 guest