Retention Policy

[BrunoNovo]

CASE ID: 03702750
I want a backup of my whole sharepoint as it is right now.
I can put the retention time to FOREVER, but will this also clean up the backup repository from time to time?
Otherwise the repository will keep growing forever? Even deleted items will remain in the repository forever?

I saw there is an option "snapshot based", but can I restore to a full snapshot of the sharepoint at any given time with this option?
Will the sharepoint be exactly like it was on the day of the backup if I would do a full restore?

Regards,

[PetrM]

Hi Bruno!

Welcome to our forum!

Please find more info on our helpcenter and there is also animated version.

Please let us know if you have some questions.

Have a nice day!

[Polina]

@Petr,
Those articles apply for Veeam Backup & Replication, VBO retention is bit different

@Bruno,
"Keep forever" policy means exactly that all data will remain in the repository as long "forever" without any cleanup. And, yes, your repository will keep growing in size upon backing up new and changed data.

With snapshot-based retention, you'll get "snapshots" of the current state of your SharePoint on every backup job run, and will be able to recover to any restore point within the retention period.

The difference between the two retention types is nicely described in VBO documentation or in the blog.

Thanks!

[BrunoNovo]

Ok so if I take a backup every day and set the retention policy to 1 year on snapshot based, I can recover the full sharepoint to any day of that year exactly as it was that day including files that are for example 10 years old.

[BrunoNovo]

Retention processing: During clean-up, we will remove all items belonging to snapshots of mailbox/site/folder that are older than the retention period.

This is confusing. I must be able to guarantee I can restore the full state of the sharepoint as it was on a specific day during the last year, even files that are 10 years old.
If a file is in a snapshot from 2 years ago, but is still currently in the sharepoint, I don't want it to be removed from the backup.

[Polina]

>> if I take a backup every day and set the retention policy to 1 year on snapshot based, I can recover the full sharepoint to any day of that year exactly as it was that day including files that are for example 10 years old.

That's correct; each "snapshot" represent a current state of the system despite the age of the files. And with the Snapshot-based retention, a cleanup occurs to "snapshots" (or restore points), not individual files inside them.

[hwelvaar]

Any insights on the diskspace used by snapshot based backups and retention processing?
I suppose first snapshot backup uses diskspace equivalent of the mailbox size. (Assume we backup just 1 mailbox)
So when next snapshot backup is run, what will be the extra diskspace Again full size? Or only delta? And is there any compression, deduplication or single-instance mechanism used?
And what happens when retention processing kicks in?

PS: there is ZERO information about this in the v.3 user manual.
Tnx for feedback.

[Polina]

Hi @ hwelvaar,

To estimate the first full backup size, you can use the VBO built-in PowerShell command Measure-VBOOrganizationFullBackupSize. On the next runs, VBO will only save the delta. There's no compression or deduplication in VBO, but the underlying Jet database where backups are stored natively provides some compression (about 15%).

And in this white paper, you'll find some recommendations on how to estimate the amount of daily changes using O365 portal.

[pinkerton]

Hi,

I'm still not completely getting how Retention Policy works with Snapshot based Retention. The Veeam article https://www.veeam.com/blog/new-features ... up-v3.html states the following:

1. First run: We collect ALL items no matter what the change date is. Thus, the first backup is an exact copy (snapshot) of an Exchange mailbox / OneDrive account / SharePoint site state as it looks at that point in time.

2. Following runs: We collect ALL new items that have been created or modified (attribute used here is the change date) since the previous run. Which means that the backup represents again an exact copy (snapshot) of the mailbox/site/folder state as it looks at that point in time.

3. Retention processing: During clean-up, we will remove all items belonging to snapshots of mailbox/site/folder that are older than the retention period.

Let's assume it's the first of January 2020, a mailbox contains items from the years 2016-2019 and we set retention time to one year.

1. During the first backup on 01.01.2020, all items from the mailbox are being backed up, which includes all items from 2016-2019
2. On each following run every new item from 2020 is being backed up. Additionally, all older items that were modified are also being backed up.
3. With retention policy set to one year, first content will be removed on 01.01.2021. But what content will actually be removed now?

a) Will all content from 2016-2019 be removed, as it was (unless modified) not being backed up after the initial run?
b) Or do the items from 2016-2019 belong to EACH following snapshot as long as they have not been deleted from the mailbox?

If B is true, Veeam must be aware about which items have been deleted from the mailbox. Then, once deleted items are belonging to snapshots that are older than the retention period, those items are deleted from the repository. That actually is what I want. I want all items that are present in the mailbox/folder/site during the backup are part of the most recent snapshot, so retention times only relates to items that are not available in the mailbox/folder/site anymore. I.e. once a message from a mailbox is deleted, we can restore it with Veeam until the retention cleanup kicks in and removes the data from the backup as well. Just like with Veeam Backup & Replication.

For example, if a user as data from 2016-2019 in its mailbox and the mailbox gets deleted, I would like to be able to recover ALL data by restoring the mailbox from the most recent backup.

What also confuses me: Polina wrote that "a cleanup occurs to "snapshots" (or restore points), not individual files inside them". But the mentioned article from above states that ITEMS belonging to snapshots are being removed. Also, if I understand the retention policy description from the use guide correctly, it seems that a restore point is only generated if a file is actually backed up:



In the example from the user guide, it's stated that each item in a backup file has its own backup date. Then its stated that an item is removed from the backup repository once it exceeds the specified retention threshold. Applied to my example from above this should mean that unless the "backup date" is not updated during each backup run, older items get removed from the repository even if they are present in the source at the time of backup in Snapshot-Based Retention mode as well.

Can someone please shed some light on this?

Thanks
Michael

[pinkerton]

So the ultimate question I guess is: Will Snapshot-Based Retention keep an item until its latest RESTORE POINT is within the retention coverage (which should mean that it is kept as long as it is present in the source) OR will Snapshot-Based Retention keep an item until its latest BACKUP TIME is within the retention coverage (which should mean that items are kept until the last backup time exceeds the retention threshold).

[nielsengelen]

Michael, have a look at the user guide for an explanation as well as an example of when we delete data based on the retention. A snapshot is a state of the mailbox at that time and the restore points will be deleted as they falls out of retention (latest/oldest backup time).

[pinkerton]

Hi Niels,

thanks, I already looked at the user guide (as written above) and it if I understand correctly, data is removed from the backup based on the backup time. This should mean that that old data is removed from the backup once backup time exceeds retention time. For example:

1. First backup runs on 01.01.2019 and contains data from 2016-2018. Retention time is set to 1 year.
2. On 01.01.2020 (after 1 year), data from 2016-2018 is removed from the backup as backup time exceeds 1 year retention time

In contrast to that, Polina wrote that with Snapshot based retention, no files from the backups are removed. Only restore points are. But if backup time relates to individual files (as stated in the user guide) - how can that work? Can someone please clarify if this actually is the case?

Thanks
Michael

[pinkerton]

Hi all,

let me further clarify my question as my long text might be complicated to understand. The user guide on page 80 (PDF) states the following regarding Snapshot-Based retention type:

Data removal from backup repositories with the Snapshot-Based Retention type occurs every time the latest restore point of an item in a backup file goes beyond the retention coverage.

That is actually what we want. Let me give you an example again:

1. First backup runs on 01.01.2020 and includes data from 2016-2019. Retention time is set to one year.
2. Data from 2016-2019 stays permanently in the source and is NOT modified. So the only backup data is 01.01.2020.
3. Let's say its 01.06.2021 (1 1/2 years after first backup and retention time) and a mailbox gets accidentally deleted
4. I want to be able to restore ALL mailbox contents from the last backup including data from 2016-2019 as well!

That means it must be assured that with Snapshot-Based Retention NO data is removed from the backup repository as long as it is available in the source during the the most recent backup. It must only be removed from the backup repository if it gets deleted from the source and the latest restore point of it exceeds the retention time.

However, page 80 also states that each item in a backup file might have its own different version, which is also considered by the retention policy. It then is stated that each item has its own backup date and that all item versions that exceed the specified retention threshold will me removed from the backup repository. The example in the user guide lists an item with three versions and explicitly states that even the last version is removed after retention time has been reached. It's unclear if this happens only if the item has been removed from the source of it this even happens if the item is still available in the source.

Relating to my example above we don't want data from 2016-2019 to be removed from the repository as long as it is available in the source during the most recent backup, even if was not modified since the first backup date! As long as data is available in the source we consider it as important so it must always be included in the backup and restorable.

Thanks
Michael

[Steve-nIP]

Snapshot is always what you want. I have no idea why Item-level is still chosen by default in the interface.

[pinkerton]

Thanks. So can you confirm that with Snapshot based retention NO data is removed from the backup repository as long as it is available in the source (and therefore should be part of the most recent snapshots)? That's unclear based on the description in the user guide...

[Dave338]

I have the same doubt.

After some trial testing, we want to implement the correct retention to different types of data.
For Exchange online we are using 1 year per item retention
For SharePoint and OneDrive we want a 30-day snapshot retention, but assuming that the older snapshot Will contain ALL data (present in the cloud 30 days ago). So like full+incrementals in VBR.

I'll do a 3 days retention test with snapshot mode to validate if this is true or not, but it's anoying that nobody can make a clear statement on that.

Best Regards.

[pinkerton]

Hi Dave,

I've tested this now and also had a case open at Veeam. It works like in VBR with snapshot based retention - the snapshots always includes the state of the source as it was during backup, no data is removed from it even if it is deleted from the source. The documentation is a little misleading on this and I have asked them to amend it in order to make it more clear. If you contact Veeam on this you can refer to my case # 03844748.

btw we're now running VBO for Office 365 in production for two weeks now and are very satisfied so far.

Best Regards
Michael

[Dave338]

Hi Pinkerton.

I've tested so far and I can confirm what you say

With snapshot mode, the backup always contains all data, as incrementals with VBR.
The only thing I would like to have is a report of the retention application in the repositories, si I can see what happens, the incremental merging, and deleting of unnecesary data, etc..

REgards.
David.

[warnox]

The documentation around snapshot based retention in VBO still makes no sense and seems to contradict other text, such as the blog referenced above. I'm not sure why they mention individual file versions within a snapshot based backup, if the retention is completely based on a snapshot of the whole data-set (mailbox, sharepoint site, one drive) at the time of backup.

[nielsengelen]

With snapshot-based we gather all new items or items which have changed since the previous backup. This allows us to show versioning within the Veeam Explorer and go to a previous state if required without the need of starting multiple explorers in case you don't know the exact deletion date.

[warnox]

Thanks Niel but that is just the standard snapshot backup based behavior? The following is an extract from the documentation I linked above...

Data removal from backup repositories with the Snapshot-Based Retention type occurs every time the latest restore point of an item in a backup file goes beyond the retention coverage.

To me, this suggests even snapshot-backup retention is item based.

[Polina]

@warnox

Both retention types in VBO work on the item-level. The difference is that one is triggered by the item's last modification date, and another - by item's last backup date.
Makes sense?

[warnox]

Ok, that part makes sense but I would assume that within one backup file, every item will have the same backup date? I'm probably missing something here but how is it possible that, "...each item in a backup file might have its own different version..."?

[Polina]

Backup date will be the same, correct; but item's last modification date will be different.
Imagine that a file has several versions and you do its initial backup with snapshot-based retention. All versions will have the same backup date, but their last modification date might be different.

[warnox]

If the retention for snapshot backups is based on the last backup date, as per your last post, why does the last modification date matter? Also, how can one item have multiple versions within a single backup file?

[Polina]

>>How can one item have multiple versions within a single backup file?

First, VBO writes backups to a database (as opposed to maintaining a chain of physical backup files). Each restore point is a collection of database records.
Next, imagine that each day backed up file is changed multiple times and gets several new versions. If you run daily backups, they will save all these new versions.
When applying retention, VBO will remove file versions that are out of the retention period. For snapshot-based, those would be versions with their backup date out of the retention period. With item-level, those would be versions with their modification date out of the retention period.

[warnox]

Alright, that clears it up. Thanks for the persistence

[Polina]

You're always welcome. Those were very good and valid questions )