-
- Influencer
- Posts: 11
- Liked: never
- Joined: Jun 28, 2019 5:39 pm
- Full Name: Jimmy Chen
- Contact:
Split backup jobs to improve performance
Support ID# 05734065
I need to confirm something.
Let say I have a Office 365 mailbox that has 10GB of data and it just finished backing up by job #1. And I create another job #2 to backup the same mailbox and it backs up to the same repository as the 1st job. Is it going to be smart enough to know it is already in the repository and not to download the whole 10GB of data again and update its metadata/pointer in the database?
I am trying figure out whether to create a new jobs or modify existing jobs to split the backup jobs to improve performance.
I need to confirm something.
Let say I have a Office 365 mailbox that has 10GB of data and it just finished backing up by job #1. And I create another job #2 to backup the same mailbox and it backs up to the same repository as the 1st job. Is it going to be smart enough to know it is already in the repository and not to download the whole 10GB of data again and update its metadata/pointer in the database?
I am trying figure out whether to create a new jobs or modify existing jobs to split the backup jobs to improve performance.
-
- Product Manager
- Posts: 14585
- Liked: 2959 times
- Joined: Sep 01, 2014 11:46 am
- Full Name: Hannes Kasparick
- Location: Austria
- Contact:
Re: Split backup jobs to improve performance
Hello,
and welcome to the forums.
As far as I see, support already answered that question: "it should sync with what is current in the repository."
Best regards,
Hannes
and welcome to the forums.
As far as I see, support already answered that question: "it should sync with what is current in the repository."
Best regards,
Hannes
-
- Product Manager
- Posts: 9327
- Liked: 2475 times
- Joined: May 13, 2017 4:51 pm
- Full Name: Fabian K.
- Location: Switzerland
- Contact:
Re: Split backup jobs to improve performance
Hi Jimmy
Yes, if you use the same repository, data for the same source object will only be downloaded once.
But the same mailbox cannot be processed simultaneously by two jobs. It can only be processed by one job at the same time. The second job will ignore the mailbox in it‘s run.
Do you see currently performance issues in downloading mailbox data? Is it the initial backup? 10GB doesn‘t sound „big“ to me. Also in Exchange the amount of items in a folder is more relevant to performance than the size. Do you have thousands of items in the folders of this mailbox?
Thanks
Fabian
Yes, if you use the same repository, data for the same source object will only be downloaded once.
But the same mailbox cannot be processed simultaneously by two jobs. It can only be processed by one job at the same time. The second job will ignore the mailbox in it‘s run.
Do you see currently performance issues in downloading mailbox data? Is it the initial backup? 10GB doesn‘t sound „big“ to me. Also in Exchange the amount of items in a folder is more relevant to performance than the size. Do you have thousands of items in the folders of this mailbox?
Thanks
Fabian
Product Management Analyst @ Veeam Software
-
- Influencer
- Posts: 11
- Liked: never
- Joined: Jun 28, 2019 5:39 pm
- Full Name: Jimmy Chen
- Contact:
Re: Split backup jobs to improve performance
Actually the there are thousands of mailboxes, 10GB is just an example I use.
The reason I ask is when I testing this on a team Backup.
My existing backup job is configured with manual selection of all team account.
My newly created 2nd backup job is configured with only "team" checkbox selected in Specific Processing Options.
Both jobs go to the same repository.
Existing backup took 53 minutes, 3.9 GB (1291 items processed), it says it was a incremental backup.
2nd backup took 51 minutes, 8.5 GB (3882 items processed) and it says it was a full backup.
1. I was wondering why the increased in size? is it just metadata it is downloading.
2. Mailbox performance issues seems to be ok. The one that seems to be taking a long time is the sharepoint. Running already for 3.5 hours, it only processed 537.5 MB (801 items processed)
If I cancel this job, for those sites that have already been processed with green checked circle, is it going to redownloaded them again if I immediately start the job again assuming there has been no changes? Is there a log somewhere I can read to find out what it is currently doing? How do I find out if it is being throttled by sharepoint online?
3. In my existing backup job, if I reconfigure my objects selection from selecting all user accounts to only a single organization with selection of only Mail, Archive, OneDrive. Is that pretty much the equivalent? Will that cause any strange issues?
Currently there are 20 Backup Applicationss.
The reason I ask is when I testing this on a team Backup.
My existing backup job is configured with manual selection of all team account.
My newly created 2nd backup job is configured with only "team" checkbox selected in Specific Processing Options.
Both jobs go to the same repository.
Existing backup took 53 minutes, 3.9 GB (1291 items processed), it says it was a incremental backup.
2nd backup took 51 minutes, 8.5 GB (3882 items processed) and it says it was a full backup.
1. I was wondering why the increased in size? is it just metadata it is downloading.
2. Mailbox performance issues seems to be ok. The one that seems to be taking a long time is the sharepoint. Running already for 3.5 hours, it only processed 537.5 MB (801 items processed)
If I cancel this job, for those sites that have already been processed with green checked circle, is it going to redownloaded them again if I immediately start the job again assuming there has been no changes? Is there a log somewhere I can read to find out what it is currently doing? How do I find out if it is being throttled by sharepoint online?
3. In my existing backup job, if I reconfigure my objects selection from selecting all user accounts to only a single organization with selection of only Mail, Archive, OneDrive. Is that pretty much the equivalent? Will that cause any strange issues?
Currently there are 20 Backup Applicationss.
-
- Product Manager
- Posts: 9327
- Liked: 2475 times
- Joined: May 13, 2017 4:51 pm
- Full Name: Fabian K.
- Location: Switzerland
- Contact:
Re: Split backup jobs to improve performance
For the SharePoint performance I suggest opening a support case.
Also, we don't recommend using too many backup application. It may happen that throttling starts earlier because too many API calls were used.
Support will be able to tell you if you were throttled by Microsoft.
If you add a teams as a source objects, teams channel settings and teams files from SharePoint will also be protected. So it makes sense that additional data was downloaded.
All Logs can be found here:
C:\ProgramData\Veeam\Backup365\Logs
Job design depends on the size of your environment. Yours sounds like a bigger environment. When protecting thousands of mailboxes (and probably thousands of SharePoint sites) the correct design of proxies and repositories is important.
Please see our best practice guide: https://bp.veeam.com/vb365/guide/design/maxconfig.html
Thanks
Fabian
Also, we don't recommend using too many backup application. It may happen that throttling starts earlier because too many API calls were used.
Support will be able to tell you if you were throttled by Microsoft.
Team Account or Microsoft Teams? I understand as Team Account you added the group account from this team? Then only mail will be backed up.My existing backup job is configured with manual selection of all team account.
My newly created 2nd backup job is configured with only "team" checkbox selected in Specific Processing Options.
1. I was wondering why the increased in size? is it just metadata it is downloading.
If you add a teams as a source objects, teams channel settings and teams files from SharePoint will also be protected. So it makes sense that additional data was downloaded.
There shouldn't be an additional download for already finished objects. Let me check that in my lab.2. Mailbox performance issues seems to be ok. The one that seems to be taking a long time is the sharepoint. Running already for 3.5 hours, it only processed 537.5 MB (801 items processed)
If I cancel this job, for those sites that have already been processed with green checked circle, is it going to redownloaded them again if I immediately start the job again assuming there has been no changes? Is there a log somewhere I can read to find out what it is currently doing? How do I find out if it is being throttled by sharepoint online?
All Logs can be found here:
C:\ProgramData\Veeam\Backup365\Logs
For smaller M365 tenants it's irrelevant. Multiple jobs are recommended to use different repositories. This is important if you want to have different retetion settings per source type (SharePoint, Mailbox, OneDrive, Teams).3. In my existing backup job, if I reconfigure my objects selection from selecting all user accounts to only a single organization with selection of only Mail, Archive, OneDrive. Is that pretty much the equivalent? Will that cause any strange issues?
Job design depends on the size of your environment. Yours sounds like a bigger environment. When protecting thousands of mailboxes (and probably thousands of SharePoint sites) the correct design of proxies and repositories is important.
Please see our best practice guide: https://bp.veeam.com/vb365/guide/design/maxconfig.html
Thanks
Fabian
Product Management Analyst @ Veeam Software
-
- Influencer
- Posts: 11
- Liked: never
- Joined: Jun 28, 2019 5:39 pm
- Full Name: Jimmy Chen
- Contact:
Re: Split backup jobs to improve performance
Thank you for the detailed explanation.
For the teams test,
1st job, selection is done by, "Back up the following objects", add, teams, select all objects
2nd job, selection is done by, "Back up the following objects', add, Organization, select the only object, edit, check only teams
Is there a documentation somewhere that details what objects will also be backed up if I select certain type of objects?
I opened a case on sharepoint as you suggested. Right now it is at 15 hours with 1.5GB (1953 items processed)
Please let me know what you found out from your lab on what happens to already finished objects if I cancel the job. Is it going to try to back them up again?
For the teams test,
1st job, selection is done by, "Back up the following objects", add, teams, select all objects
2nd job, selection is done by, "Back up the following objects', add, Organization, select the only object, edit, check only teams
Is there a documentation somewhere that details what objects will also be backed up if I select certain type of objects?
I opened a case on sharepoint as you suggested. Right now it is at 15 hours with 1.5GB (1953 items processed)
Please let me know what you found out from your lab on what happens to already finished objects if I cancel the job. Is it going to try to back them up again?
-
- Product Manager
- Posts: 9327
- Liked: 2475 times
- Joined: May 13, 2017 4:51 pm
- Full Name: Fabian K.
- Location: Switzerland
- Contact:
Re: Split backup jobs to improve performance
I run the same test now. There was download of data, but no items were processed by the second job. How long do you have waited until running the second job? Is there process running which could have created additional 4000 items? If you compare the session logs of both jobs, do you see the same teams backed up? Or does the second job have backed up additional teams?For the teams test,
1st job, selection is done by, "Back up the following objects", add, teams, select all objects
2nd job, selection is done by, "Back up the following objects', add, Organization, select the only object, edit, check only teams
Existing backup took 53 minutes, 3.9 GB (1291 items processed), it says it was a incremental backup.
2nd backup took 51 minutes, 8.5 GB (3882 items processed) and it says it was a full backup.
That sounds strange and it's worth to be analyzed by our support team. Could you maybe share your case number with me?I opened a case on SharePoint as you suggested. Right now it is at 15 hours with 1.5GB (1953 items processed)
I can confirm. Already downloaded items will not be downloaded again if you cancel the job and restart it.Please let me know what you found out from your lab on what happens to already finished objects if I cancel the job. Is it going to try to back them up again?
But the job needs to be finished once (successful or with warnings) before you can do a restore.
Let me check or follow up with a list here.Is there a documentation somewhere that details what objects will also be backed up if I select certain type of objects?
Thanks
Fabian
Product Management Analyst @ Veeam Software
-
- Influencer
- Posts: 11
- Liked: never
- Joined: Jun 28, 2019 5:39 pm
- Full Name: Jimmy Chen
- Contact:
Re: Split backup jobs to improve performance
Caser# 05739958
After doing some process of eliminations moving sites around to different jobs, I narrowed it down to one sharepoint site that has been slowing down all the other jobs.
In this job, I only have one website to download. This site is about 5.3TB total. After running about 17 hours, it has only transferred 55.4 MB (51 items processed)
Whenever this job runs, all the other jobs slows down. After a few minutes of stopping this job, all other jobs will resume normal speed.
After doing some process of eliminations moving sites around to different jobs, I narrowed it down to one sharepoint site that has been slowing down all the other jobs.
In this job, I only have one website to download. This site is about 5.3TB total. After running about 17 hours, it has only transferred 55.4 MB (51 items processed)
Whenever this job runs, all the other jobs slows down. After a few minutes of stopping this job, all other jobs will resume normal speed.
-
- Product Manager
- Posts: 9327
- Liked: 2475 times
- Joined: May 13, 2017 4:51 pm
- Full Name: Fabian K.
- Location: Switzerland
- Contact:
Re: Split backup jobs to improve performance
Thanks for the case number. Let's see what our support is able to find out.
I will follow the case from our side (PM).
I will follow the case from our side (PM).
Product Management Analyst @ Veeam Software
-
- Enthusiast
- Posts: 77
- Liked: 12 times
- Joined: Jul 17, 2019 10:06 pm
- Contact:
Re: Split backup jobs to improve performance
I'm running into the same type of issue. Usually 1 sharepoint site and 1 mailbox slowing down the jobs to a halt because of how massive they are. Should they have their own jobs or how should I handle those? Currently have 3 jobs for the same tenant (Exchange, OneDrive, SharePoint/Teams). Each has their own repo.
Who is online
Users browsing this forum: No registered users and 6 guests