Comprehensive data protection for all workloads
Post Reply
TitaniumCoder477
Veteran
Posts: 319
Liked: 49 times
Joined: Apr 07, 2015 1:53 pm
Full Name: James Wilmoth
Location: Kannapolis, North Carolina, USA
Contact:

Veeam background retention and orphaned backup sets

Post by TitaniumCoder477 »

I have history with Veeam from version 7 through 12 that spans almost 9 1/2 years. But only the last year of that time covers v12, so I admittedly have less experience with this version than all the ones I supported through my employment. At the end of 2023, I changed employment and didn't have much to do with Veeam until this past month when I deployed it for a small client. Imagine my surprise when the "orphaned" backup set of the old server vanished after a week. Investigation led me to background retention which apparently affected these types of chains as of v12. I had no idea. Had I known, I would have created an Export of the latest point to be stored outside of any kind of retention process. What is done is done. Thankfully, we do have the old server, with data and programs, so I can take another backup of it. My reason for posting here is to understand Veeam's reasoning behind this. Let me first present my reasoning.

My understanding for the past decade has been that retention was associated with the job. That is, I configured it in the job. I was not aware of any backup chain that, apart from a job, somehow had retention associated with it. For example, in the job I configured 14 days or 30 days or 90 days. There was never any mechanism to configure retention through a chain. Therefore, logically speaking, there was a relationship between the job and retention, not the chain and retention. Of course, I am aware of the other features that came into existence related to SOBR and tiering and such, and the necessity of background retention processes that would run independently of those jobs, perhaps based on settings on the repository itself. However, as it stands now, even in v12, I do not configure retention on a simple SMB repository. I do not configure retention on the chain itself. I configure retention in the job. And historically, retention always took place after the job ran. So it really makes no sense to me at all why Veeam's background retention process would extend to include a regular VM backup job that has been deleted intentionally, resulting in an orphaned backup set. At least, this kind of behavior should not be the default because it's considered a "destructive action" that results in the loss of backups. Speaking of loss of backups, another behavior that I became accustomed to over the decade of experience was that Veeam would never automatically retention the last backup in a chain. If a VM, for example, no longer existed and therefore wasn't being backed up in a job, the retention would continue against that VM until the last point which Veeam would safeguard, requiring manual cleanup. I thought this was a very good design and touted it as much to my clients. Yet, this recent experience was the TOTAL loss of the entire orphaned backup set.

Thank you in advance for any explanation or clarification! :)
david.domask
Veeam Software
Posts: 2590
Liked: 606 times
Joined: Jun 28, 2016 12:12 pm
Contact:

Re: Veeam background retention and orphaned backup sets

Post by david.domask » 2 people like this post

Hi TitaniumCoder477,

Thank you for the detailed explanation and sorry to hear about the troubles.

Background retention was a long-standing request to allow for pretty much the opposite of what you're relying on. While I understand the points you're making, the same points can be rephrased as complaints:

- Retention being job bound means you cannot reclaim space without running the backup job and adding more backups
- Backups of machines removed from jobs (or backups that remain after their job is deleted) were hard to spot and could 'invisibly' take up space, requiring a bit of investigation to find no longer needed backups
- Retention always leaves behind the active chain, no way to phase out backups gradually as machines are replaced/decommissioned

So your points are understood, but it's best not to think of it as retention on the repository or on the job -- retention applies to backups. Similarly, one of the changes that impacts this which I didn't see you mention (maybe you missed it) was a shift from point based retention to day based retention. While point based currently is still possible, most of the new logic (GFS, retention, immutability) is day based instead of point based now, meaning Veeam "knows" when a backup file is past retention and is no longer needed (based on the set retention) instead of having to work through a calculation based on the number of backup chains (full + incremental backups) and the set retention.

Effectively what this comes down to is that retention now properly applies regardless of where the backup is. For backups that do need to be retained for longer periods, Export Backup or Copy Backup are indeed the best options.
David Domask | Product Management: Principal Analyst
tyler.jurgens
Veeam Software
Posts: 425
Liked: 251 times
Joined: Apr 11, 2023 1:18 pm
Full Name: Tyler Jurgens
Contact:

Re: Veeam background retention and orphaned backup sets

Post by tyler.jurgens » 1 person likes this post

VeeamZIP is also a great tool to utilize for that "I need this backup to exist indefinitely (or for a specific set amount of time)" kind of scenarios. I've often used it for decommissioning servers or infrastructure refresh tasks where you probably don't need that old copy, but its a helpful tool for the "just in case" scenarios.
Tyler Jurgens
Blog: https://explosive.cloud
Twitter: @Tyler_Jurgens BlueSky: @explosive.cloud
TitaniumCoder477
Veteran
Posts: 319
Liked: 49 times
Joined: Apr 07, 2015 1:53 pm
Full Name: James Wilmoth
Location: Kannapolis, North Carolina, USA
Contact:

Re: Veeam background retention and orphaned backup sets

Post by TitaniumCoder477 »

I have experienced each of those points, so I understand them. That said, I have a few concerns. First, if retention is to be logically associated with the backups themselves, then configuration of said retention needs to be associated with the backups themselves too. For example, I should have been able to right-click the "orphaned" backup set and selected "Configure Retention" which would have provided a dialog box similar to the one available through the deleted job. Frankly, based on the logic you have presented, this should be available for all backup sets going forward, i.e. retention configuration available through the job (legacy) and directly against the backup set itself. (Feature request?) Second, I think Veeam's previous policy of preservation of the last point is still the best policy (and presents the least liability) for Veeam. The online manual is the best online manual for any product I have ever seen, but even I have not read ever page of it, nor kept up to date on it. Veeam should assume users will not know precisely how retention works and should "fail safe" in any kind of destruction process, whether this is an actively used chain, an "orphaned" chain, or some other kind of chain. There can always be an option in advanced settings somewhere that reverses this decision for those users who want it. That said, is this "fail safe" policy still in place for an active backup chain? If not, I really need to know since this was a trust factor between me and the product for many years. Third, I do appreciate the trend towards day based retention over point based retention. I always had to explain how point based retention worked to clients. I experienced other minor negative oddities that I can only vaguely recall now and that were resolved by switching the jobs to day based retention. This was usually the more sensible option, though we did have a few clients who wanted intra-daily backups.
karsten123
Service Provider
Posts: 569
Liked: 140 times
Joined: Apr 03, 2019 6:53 am
Full Name: Karsten Meja
Contact:

Re: Veeam background retention and orphaned backup sets

Post by karsten123 » 1 person likes this post

I also lost backup data but unfortunately, the original was already deleted.
So, to get infinite retention with orphaned backups like it was in v11, I have to take one extra step, i.e. export or copy backup. Is that correct? And if i miss that step, Veeam is going to terminate my data? Right?
Is this behavior documented? Especially that the last full will also be deleted. This has to be written to the upgrade checklist in bold an red.
TitaniumCoder477
Veteran
Posts: 319
Liked: 49 times
Joined: Apr 07, 2015 1:53 pm
Full Name: James Wilmoth
Location: Kannapolis, North Carolina, USA
Contact:

Re: Veeam background retention and orphaned backup sets

Post by TitaniumCoder477 »

tyler.jurgens wrote: Apr 03, 2025 2:58 pm VeeamZIP is also a great tool to utilize for that "I need this backup to exist indefinitely (or for a specific set amount of time)" kind of scenarios. I've often used it for decommissioning servers or infrastructure refresh tasks where you probably don't need that old copy, but its a helpful tool for the "just in case" scenarios.
While I highly value the VeeamZIP feature, I present it as an "ad-hoc" backup feature that doesn't have anything to do with retention BUT which can include an entirely new encryption key. For example, if a client needs to send a backup copy of an infected server to their cyber security / insurance provider but wants to give them an entirely new encryption key instead of their own actively used one. Now that Veeam has the Export (from chain) option, VeeamZIP is less often used because it impacts production where Export does not. Still, both have their valid and often overlapping use cases. But in the context of this forum post which is retention or lack thereof pertaining to a chain, I don't think either is sufficiently relevant.
Post Reply

Who is online

Users browsing this forum: Amazon [Bot], Bing [Bot] and 79 guests