Maintain control of your Microsoft Office 365 email data
Post Reply
BrunoNovo
Novice
Posts: 8
Liked: never
Joined: Aug 05, 2019 12:18 pm
Full Name: Bruno Vander Plaetse
Contact:

Retention Policy

Post by BrunoNovo » Aug 07, 2019 5:01 pm

CASE ID: 03702750
I want a backup of my whole sharepoint as it is right now.
I can put the retention time to FOREVER, but will this also clean up the backup repository from time to time?
Otherwise the repository will keep growing forever? Even deleted items will remain in the repository forever?

I saw there is an option "snapshot based", but can I restore to a full snapshot of the sharepoint at any given time with this option?
Will the sharepoint be exactly like it was on the day of the backup if I would do a full restore?

Regards,

PetrM
Veeam Software
Posts: 177
Liked: 21 times
Joined: Aug 28, 2013 8:23 am
Full Name: Petr
Location: Prague, Czech Republic
Contact:

Re: Retention Policy

Post by PetrM » Aug 07, 2019 5:22 pm

Hi Bruno!

Welcome to our forum!

Please find more info on our helpcenter and there is also animated version.

Please let us know if you have some questions.

Have a nice day!

Polina
Veeam Software
Posts: 1006
Liked: 170 times
Joined: Oct 21, 2011 11:22 am
Full Name: Polina Vasileva
Contact:

Re: Retention Policy

Post by Polina » Aug 07, 2019 6:04 pm 2 people like this post

@Petr,
Those articles apply for Veeam Backup & Replication, VBO retention is bit different :)

@Bruno,
"Keep forever" policy means exactly that all data will remain in the repository as long "forever" without any cleanup. And, yes, your repository will keep growing in size upon backing up new and changed data.

With snapshot-based retention, you'll get "snapshots" of the current state of your SharePoint on every backup job run, and will be able to recover to any restore point within the retention period.

The difference between the two retention types is nicely described in VBO documentation or in the blog.

Thanks!

BrunoNovo
Novice
Posts: 8
Liked: never
Joined: Aug 05, 2019 12:18 pm
Full Name: Bruno Vander Plaetse
Contact:

Re: Retention Policy

Post by BrunoNovo » Aug 08, 2019 7:51 am

Ok so if I take a backup every day and set the retention policy to 1 year on snapshot based, I can recover the full sharepoint to any day of that year exactly as it was that day including files that are for example 10 years old.

BrunoNovo
Novice
Posts: 8
Liked: never
Joined: Aug 05, 2019 12:18 pm
Full Name: Bruno Vander Plaetse
Contact:

Re: Retention Policy

Post by BrunoNovo » Aug 08, 2019 8:24 am

Retention processing: During clean-up, we will remove all items belonging to snapshots of mailbox/site/folder that are older than the retention period.

This is confusing. I must be able to guarantee I can restore the full state of the sharepoint as it was on a specific day during the last year, even files that are 10 years old.
If a file is in a snapshot from 2 years ago, but is still currently in the sharepoint, I don't want it to be removed from the backup.

Polina
Veeam Software
Posts: 1006
Liked: 170 times
Joined: Oct 21, 2011 11:22 am
Full Name: Polina Vasileva
Contact:

Re: Retention Policy

Post by Polina » Aug 08, 2019 11:03 am

>> if I take a backup every day and set the retention policy to 1 year on snapshot based, I can recover the full sharepoint to any day of that year exactly as it was that day including files that are for example 10 years old.

That's correct; each "snapshot" represent a current state of the system despite the age of the files. And with the Snapshot-based retention, a cleanup occurs to "snapshots" (or restore points), not individual files inside them.

hwelvaar
Novice
Posts: 7
Liked: never
Joined: Feb 08, 2018 1:58 pm
Contact:

Re: Retention Policy

Post by hwelvaar » Aug 14, 2019 6:04 am

Any insights on the diskspace used by snapshot based backups and retention processing?
I suppose first snapshot backup uses diskspace equivalent of the mailbox size. (Assume we backup just 1 mailbox)
So when next snapshot backup is run, what will be the extra diskspace Again full size? Or only delta? And is there any compression, deduplication or single-instance mechanism used?
And what happens when retention processing kicks in?

PS: there is ZERO information about this in the v.3 user manual.
Tnx for feedback.

Polina
Veeam Software
Posts: 1006
Liked: 170 times
Joined: Oct 21, 2011 11:22 am
Full Name: Polina Vasileva
Contact:

Re: Retention Policy

Post by Polina » Aug 14, 2019 9:33 am

Hi @ hwelvaar,

To estimate the first full backup size, you can use the VBO built-in PowerShell command Measure-VBOOrganizationFullBackupSize. On the next runs, VBO will only save the delta. There's no compression or deduplication in VBO, but the underlying Jet database where backups are stored natively provides some compression (about 15%).

And in this white paper, you'll find some recommendations on how to estimate the amount of daily changes using O365 portal.

pinkerton
Enthusiast
Posts: 78
Liked: 4 times
Joined: Sep 29, 2011 9:57 am
Contact:

Re: Retention Policy

Post by pinkerton » Nov 01, 2019 2:07 pm

Hi,

I'm still not completely getting how Retention Policy works with Snapshot based Retention. The Veeam article https://www.veeam.com/blog/new-features ... up-v3.html states the following:

1. First run: We collect ALL items no matter what the change date is. Thus, the first backup is an exact copy (snapshot) of an Exchange mailbox / OneDrive account / SharePoint site state as it looks at that point in time.

2. Following runs: We collect ALL new items that have been created or modified (attribute used here is the change date) since the previous run. Which means that the backup represents again an exact copy (snapshot) of the mailbox/site/folder state as it looks at that point in time.

3. Retention processing: During clean-up, we will remove all items belonging to snapshots of mailbox/site/folder that are older than the retention period.

Let's assume it's the first of January 2020, a mailbox contains items from the years 2016-2019 and we set retention time to one year.

1. During the first backup on 01.01.2020, all items from the mailbox are being backed up, which includes all items from 2016-2019
2. On each following run every new item from 2020 is being backed up. Additionally, all older items that were modified are also being backed up.
3. With retention policy set to one year, first content will be removed on 01.01.2021. But what content will actually be removed now?

a) Will all content from 2016-2019 be removed, as it was (unless modified) not being backed up after the initial run?
b) Or do the items from 2016-2019 belong to EACH following snapshot as long as they have not been deleted from the mailbox?

If B is true, Veeam must be aware about which items have been deleted from the mailbox. Then, once deleted items are belonging to snapshots that are older than the retention period, those items are deleted from the repository. That actually is what I want. I want all items that are present in the mailbox/folder/site during the backup are part of the most recent snapshot, so retention times only relates to items that are not available in the mailbox/folder/site anymore. I.e. once a message from a mailbox is deleted, we can restore it with Veeam until the retention cleanup kicks in and removes the data from the backup as well. Just like with Veeam Backup & Replication.

For example, if a user as data from 2016-2019 in its mailbox and the mailbox gets deleted, I would like to be able to recover ALL data by restoring the mailbox from the most recent backup.

What also confuses me: Polina wrote that "a cleanup occurs to "snapshots" (or restore points), not individual files inside them". But the mentioned article from above states that ITEMS belonging to snapshots are being removed. Also, if I understand the retention policy description from the use guide correctly, it seems that a restore point is only generated if a file is actually backed up:

Image

In the example from the user guide, it's stated that each item in a backup file has its own backup date. Then its stated that an item is removed from the backup repository once it exceeds the specified retention threshold. Applied to my example from above this should mean that unless the "backup date" is not updated during each backup run, older items get removed from the repository even if they are present in the source at the time of backup in Snapshot-Based Retention mode as well.

Can someone please shed some light on this?

Thanks
Michael

pinkerton
Enthusiast
Posts: 78
Liked: 4 times
Joined: Sep 29, 2011 9:57 am
Contact:

Re: Retention Policy

Post by pinkerton » Nov 01, 2019 2:43 pm

So the ultimate question I guess is: Will Snapshot-Based Retention keep an item until its latest RESTORE POINT is within the retention coverage (which should mean that it is kept as long as it is present in the source) OR will Snapshot-Based Retention keep an item until its latest BACKUP TIME is within the retention coverage (which should mean that items are kept until the last backup time exceeds the retention threshold).

nielsengelen
Veeam Software
Posts: 2772
Liked: 579 times
Joined: Jul 15, 2013 11:09 am
Full Name: Niels Engelen
Contact:

Re: Retention Policy

Post by nielsengelen » Nov 01, 2019 4:20 pm

Michael, have a look at the user guide for an explanation as well as an example of when we delete data based on the retention. A snapshot is a state of the mailbox at that time and the restore points will be deleted as they falls out of retention (latest/oldest backup time).
VCP-DCV
Veeam Certified Architect (VMCA)
http://foonet.be

pinkerton
Enthusiast
Posts: 78
Liked: 4 times
Joined: Sep 29, 2011 9:57 am
Contact:

Re: Retention Policy

Post by pinkerton » Nov 04, 2019 7:52 am

Hi Niels,

thanks, I already looked at the user guide (as written above) and it if I understand correctly, data is removed from the backup based on the backup time. This should mean that that old data is removed from the backup once backup time exceeds retention time. For example:

1. First backup runs on 01.01.2019 and contains data from 2016-2018. Retention time is set to 1 year.
2. On 01.01.2020 (after 1 year), data from 2016-2018 is removed from the backup as backup time exceeds 1 year retention time

In contrast to that, Polina wrote that with Snapshot based retention, no files from the backups are removed. Only restore points are. But if backup time relates to individual files (as stated in the user guide) - how can that work? Can someone please clarify if this actually is the case?

Thanks
Michael

pinkerton
Enthusiast
Posts: 78
Liked: 4 times
Joined: Sep 29, 2011 9:57 am
Contact:

Re: Retention Policy

Post by pinkerton » Nov 04, 2019 8:29 am

Hi all,

let me further clarify my question as my long text might be complicated to understand. The user guide on page 80 (PDF) states the following regarding Snapshot-Based retention type:

Data removal from backup repositories with the Snapshot-Based Retention type occurs every time the latest restore point of an item in a backup file goes beyond the retention coverage.

That is actually what we want. Let me give you an example again:

1. First backup runs on 01.01.2020 and includes data from 2016-2019. Retention time is set to one year.
2. Data from 2016-2019 stays permanently in the source and is NOT modified. So the only backup data is 01.01.2020.
3. Let's say its 01.06.2021 (1 1/2 years after first backup and retention time) and a mailbox gets accidentally deleted
4. I want to be able to restore ALL mailbox contents from the last backup including data from 2016-2019 as well!

That means it must be assured that with Snapshot-Based Retention NO data is removed from the backup repository as long as it is available in the source during the the most recent backup. It must only be removed from the backup repository if it gets deleted from the source and the latest restore point of it exceeds the retention time.

However, page 80 also states that each item in a backup file might have its own different version, which is also considered by the retention policy. It then is stated that each item has its own backup date and that all item versions that exceed the specified retention threshold will me removed from the backup repository. The example in the user guide lists an item with three versions and explicitly states that even the last version is removed after retention time has been reached. It's unclear if this happens only if the item has been removed from the source of it this even happens if the item is still available in the source.

Relating to my example above we don't want data from 2016-2019 to be removed from the repository as long as it is available in the source during the most recent backup, even if was not modified since the first backup date! As long as data is available in the source we consider it as important so it must always be included in the backup and restorable.

Thanks
Michael

Steve-nIP
Service Provider
Posts: 26
Liked: 4 times
Joined: Feb 06, 2018 10:08 am
Full Name: Steve
Contact:

Re: Retention Policy

Post by Steve-nIP » Nov 04, 2019 8:35 am

Snapshot is always what you want. I have no idea why Item-level is still chosen by default in the interface.

pinkerton
Enthusiast
Posts: 78
Liked: 4 times
Joined: Sep 29, 2011 9:57 am
Contact:

Re: Retention Policy

Post by pinkerton » Nov 04, 2019 8:37 am

Thanks. So can you confirm that with Snapshot based retention NO data is removed from the backup repository as long as it is available in the source (and therefore should be part of the most recent snapshots)? That's unclear based on the description in the user guide...

Dave338
Enthusiast
Posts: 38
Liked: 5 times
Joined: Jan 27, 2015 12:21 pm
Full Name: David
Contact:

Re: Retention Policy

Post by Dave338 » Nov 22, 2019 10:00 am

I have the same doubt.

After some trial testing, we want to implement the correct retention to different types of data.
For Exchange online we are using 1 year per item retention
For SharePoint and OneDrive we want a 30-day snapshot retention, but assuming that the older snapshot Will contain ALL data (present in the cloud 30 days ago). So like full+incrementals in VBR.

I'll do a 3 days retention test with snapshot mode to validate if this is true or not, but it's anoying that nobody can make a clear statement on that.

Best Regards.

pinkerton
Enthusiast
Posts: 78
Liked: 4 times
Joined: Sep 29, 2011 9:57 am
Contact:

Re: Retention Policy

Post by pinkerton » Nov 22, 2019 10:14 am

Hi Dave,

I've tested this now and also had a case open at Veeam. It works like in VBR with snapshot based retention - the snapshots always includes the state of the source as it was during backup, no data is removed from it even if it is deleted from the source. The documentation is a little misleading on this and I have asked them to amend it in order to make it more clear. If you contact Veeam on this you can refer to my case # 03844748.

btw we're now running VBO for Office 365 in production for two weeks now and are very satisfied so far.

Best Regards
Michael

Dave338
Enthusiast
Posts: 38
Liked: 5 times
Joined: Jan 27, 2015 12:21 pm
Full Name: David
Contact:

Re: Retention Policy

Post by Dave338 » Nov 28, 2019 9:15 am

Hi Pinkerton.

I've tested so far and I can confirm what you say :)

With snapshot mode, the backup always contains all data, as incrementals with VBR.
The only thing I would like to have is a report of the retention application in the repositories, si I can see what happens, the incremental merging, and deleting of unnecesary data, etc..

REgards.
David.

Post Reply

Who is online

Users browsing this forum: JonJR and 3 guests