Horrible data management in Veeam 365

[Shagma]

I'm not sure if this has been problematized before, but I'm posting this as it really grinds my gears.

Veeam 365 uses Jet database, maybe it's related to the fact that Exchange uses Jet, IDK. When you delete something from a Jet database the data is not actually removed, but you can overwrite the data with new data.

Multiple issues comes from this:

1. It is a problem when it comes to privacy regulations. When a user or customer requests to have their data deleted, we can not guarantee it has been done unless we delete the whole database.
2. When a customer terminates the service, it is almost impossible to clean up afterwards. Deleting the customer from the GUI does nothing to the data in the database. It will still be there, taking up space and putting you in conflict with privacy regulations.
3. A database grows as long as new data is put into it. If you don't pay attention, the database will occupy the whole disk. Usually the fix will be to free up space by deleting or moving data from database, but this is useless as the Jet database stays the same size anyway. In the end the only option is to expand the disk. If that is not possible you have to start over with new database(s).

Theres issues would not have been a problem if there were some good tools within Veeam 365 to:

1. Purge deleted objects from the database
2. Shrink/compact the database

I know there are ways to do this, either through own scripting or through the use of 3rd party script and general Jet database tools. This is however not good enough.
I've tried multiple times to manually delete organizations and their data. For one it takes a lot of work to create a script that deletes everything that is related to an organization as deleting the organization in itself will not delete it's corresponding data.
Second, most of the time the command to delete either fails or just doesn't do anything to the data.

Please correct me if I have misunderstood something. Nothing would be better than to hear there is a secret button to fix all this.
If not I'd like to hear your thoughts on the matter.

[nielsengelen]

In order to remove data (relevant to 1 and 2), we have powershell cmdlets (Remove-VBOEntityData) which helps you there. There are some examples in the user guide (https://helpcenter.veeam.com/docs/vbo36 ... tml?ver=50) and community scripts (for example I made a 'data manager' for this; https://github.com/nielsengelen/veeam-p ... anager.ps1). Another example can be found in the forum post to delete users (post386171.html#p386171). If you do see failures with this, you should contact support as there may be something else going wrong.

If you remove data often, we will just refill those blocks and the database won't grow like crazy. If you don't delete the data manually, we will keep the data until the retention is reached (which is a choice done for regulations the matter).

For 3, yes a database grows until it reaches 58TB and it will split. This is something you will have with any type of backup unless you decide to use Object Storage which is fully supported (like AWS S3, Azure Blob).

[karsten123]

It is btw. good practice to create min. one separate repository per tenant. So you can delete the whole directory in case of a tenant offboarding.

[Shagma]

@karsten123: that's more of a workaround which goes to prove the shortcomings of Veeam 365. Thanks for you suggestion though.

@nielsengelen

I've been using your script with various degree of success. Don't get me wrong, your script is much better than nothing, but it just doesn't scream enterprise ready when you have a product that relies on 3rd party scripts to accomplish basic tasks.

One problem is that when you delete an organization and realize that none of the data got deleted with it, you pull up the database and get a list of names. There is no way of knowing who to delete because you don't know which organization they belong(ed) to.
And why do we even have to delete users one by one when we could have just deleted them when removing the org in the GUI? Don't even get me started on how much worse this problem is when it comes to Sharepoint sites.

To get around these issues I created a script of my own to try to mass delete every user with connection to a specific organization (foreach).
However Remove-VBOEntityData is not always successful and I've had to give up on a few databases and just delete them. When that happens I'm not always successful in transferring data out of the database and some history for some accounts may be lost.

[DaStivi]

Hi, sounds like youve really multiple o365 tenants in one o365 Installation, sure in bigger entsrprises this might be the case to have more then once tenant... Most propaply only some prod and mybe an dev/test, where really is the question of having the need.. On the other hand if you run veeam o365 (like?) as service Provider and have multiple companies/Organisations (for what ever reason) you should Stick. To the vcsp rexommendatuons of having seperate Backup repository folders for each tenant... So you can easily delete the complete repo (folder Strukture) without having the Mix of the other tenants... As easy as that, at least for that partcular point..

For the other thing like deleting gdpr related stuff out of the Backup it's what it is... Further more deleting that stuff from Office 365 is quite another challenge I guess... Never need it to be honest, hopefully never will...

[Mike Resseler]

On the GDPR topic.

I am fighting this need of deleting a user from the backup for a few years now. Please note that it is absolutely NOT necessary to delete that information from the backup. The right to be forgotten is NOT absolute. If this would be the case, then we would start having gaps in organizations that previous sales won't be connected anymore to a person or legal entity, that certain actions will be done by an organization but that they are not traceable anymore.

What a user can do is demand this right so that his/ her information is forgotten on production (but even then a minimum can be kept depending on other legislations to keep your accounting/ stock/ whatever in order). What you CAN'T do is restore it back into production. That is something that you need to remember.

And last but not least, if you, as an organization are obliged to keep data for (example) 7 years, that obligation is still more important, so deleting data will get you into a bad situation.

Hope it helps