Forever Incremental - no full

[lobo519]

I searched around but didn't really find a thread

I'm considering doing forever incremental but the lack of a full backup at any time kinda freaks me out. I really want to increase my restore points without adding a bunch of full backups and reverse incremental is too slow.

Am I crazy ??

[Gostev]

Wow, there are easily over 9000 threads discussing this exact question (whether periodic fulls are really required). Not sure why you did not find a single one, may be forum search is messed up. Because arguably, this has been one of the most popular question on these forums in the past 10 years. Especially for the first few years of Veeam, as until v4 we did not even provide an option to do a periodic full so there were many concerns similar to yours, as you can imagine.

But to save you some time searching - no, you're not crazy at all. Forever incremental backup is the new norm, 75% of our customers are using some variation of one. And 25% are using the specific backup mode you're contemplating about - forever incremental backup with no periodic synthetic fulls.

[lobo519]

well - I couldn't believe it either and I figured i would get that response

It would be nice to have the option of using active full with forever incremental so that the previous chain/retention would roll off the same after a full and not have to wait for the retention policy to be met in the new chain.

Did that actually make any sense?

[Mike Resseler]

Lobo,

I probably misunderstand you but that is possible? You create a backup job with forever incremental and then do an active full periodically or a synthetic? (https://helpcenter.veeam.com/docs/backu ... tml?ver=95). Or do I understand wrongly what you mean by rolling off the previous chain?

[Gostev]

It would be nice to have the option of using active full with forever incremental so that the previous chain/retention would roll off the same after a full and not have to wait for the retention policy to be met in the new chain.

Did that actually make any sense?

To be honest, no - may be a diagram would help? I *suspect* you might be talking about the mode we already provide with the "transform to rollbacks" option, but that one is inherently synthetic full only.

[lobo519]

ok - I have been using Veeam since v5 but always used reverse incremental. I changed jobs and the new shop is significantly bigger and reverse incremental just takes too long. There is a chance, I am being a total noob here...



forever incremental:

Full - Inc1 - Inc2 - Inc3 ---

Inc1 gets merged into Full when the chain meets retention policy applies right? Cool

Forever Incremental with a Active Full:

Full - Inc1 - Inc2 - Inc3 - ActiveFull - Inc1 - Inc2

Now with a new active full - The first back up chain stay completely intact until the new active full meets the retention policy - right? That was I see anyway

What would be helpful is for the first full chain to continue to merge the oldest restore point in its full also.

so basically, The retention policy applied across all backup chains so I can use forever incremental with a active full backups and not have extra restore points I don't want.

[foggy]

Understandable, but not currently possible, this would be a mix of forever incremental and simple forward incremental with periodic fulls. What you could do know though, is what Anton has mentioned - enable transformation of the old part of the chain into rollbacks, which effectively results in not having excessive restore points (the oldest one will be deleted daily) and having just a single full backup.

[lobo519]

Oh I understand that its not currently possible - but it would be awesome if it was!!!! Maybe you can fit it in to version 10

The transform option would require a synthetic full - I would prefer to use Active. Get that warm fuzzy feeling - ya know?

[jcavalear]

Hi All,

I am trying to do something similar to Lobo in that my jobs are scheduled for forever forward incremental and I want to do a periodic full to refresh the backup chain. My issue is that once I schedule a periodic full backup my retention chain stops transforming the incremental into the full backpup. I have a 7 day retention policy and would like to do an active full once a month. When I schedule the job like this I'm seeing an unexpected result:

What I expected is to always see 7 retention points:

Full - inc - inc - inc - inc -inc - inc

and as I add additional incremental backups my Full will merge with the oldest incremental until the next full. I realize that after the next Active Full I need to have 6 incrementals, to equal my 7 restore points, before my previous chain will drop for a total of 14 restore points.

Full - inc - inc - inc- inc - inc - inc - Active Full - inc - inc - inc - inc - inc - inc (on the next incremental the first chain will drop)

This is how my jobs were working and everything was as expected until I turned on the periodic Active Full once a month. After turning on an Active full my jobs stopped merging after the 7th incremental and now I'm accumulating incremental restore points for the 4 weeks between active fulls. On the 7th day after my active full my previous month does drop off once the new Active Full chain hits the 7 day retention. This then grows for the rest of the month rather than keeping the string at 7 by merging the oldest incremental.

Full - inc - inc - inc - inc - inc - inc - inc - inc - inc - inc .....inc- inc - Active Full - inc - inc - inc - inc - inc - inc (after the new chain reaches 7 my previous month drops off)

Now, instead of having 7 retention points with a maximum of 14 after the new active full gets created, I have 30 plus retention points before the new active full and it's string reaches 7 allowing the previous month to drop.

I only discovered this unexpected behavior when one of my smaller repositories at a remote site filled up unexpectedly. I have this backup model on a mix of regular VM backup jobs as well as Agent jobs. I've seen in the agent manual where it says that the transforms will be disabled if you turn on periodic active full jobs but I did not expect this for my regular backup jobs.

Is it possible to keep a 7 day retention while also performing an active full once a month?

Thanks to everyone at Veeam who maintain the forum, this is a great resource to answer non critical questions.

[foggy]

The behavior is expected. Once you enable periodic full (be it active or synthetic), the job switches from forever forward to simple forward incremental mode. To achieve what you're after, consider triggering active full via scheduled monthly script instead.

[jcavalear]

I just had a discussion about this with a co-worker because when I explained what is actually happening he doesn't see the logic in it. I revisited this thread so I could show him your answer but as I read through the thread and the link you sent I guess I'm not fully understanding it still, I guess I just accepted it because you say the behaviour is expected. I re-read the link you sent and don't see where it says this behaviour is expected?

Once I enable a periodic full backup once a month and the job switches to "simple forward incremental mode" as you state above, I still don't understand why retention is not kept at 7 on the new backup string that is created by the new periodic active full?

I understand that I have a job with one .vbk and six .vib files for my seven retention points. I also understand that my periodic active full starts a new string of one .vbk and six .vib files before the previous string will delete. What I still don't understand is why is my new string continuing to grow out to 30 retention points with .vib incremental backups before the next periodic active full the next month? Logic would tell me that I keep 7 total retention points, which then have to grow to 14 because of the new string with the active full in the middle, before the old string of 7 gets deleted and puts me back at 7 for the rest of the month until I do another active full?

[markhfrederick]

Great job by all who have contributed to this article of explaining this phenomenon.

Foggy, thank you for confirming that this behavior is expected; means we're not all crazy. But, can you explain why it works this way ?

The guys before have a great point about being able to manage backup storage with predictable results, and the end result described by jcavalear is frustrating, for sure. Seems like it used to work exactly as they ave described, where you only see "too many restore points" for the amount of time it takes the *new* chain to honor the retention setting, as in jcavalear's case. IT seems none of us can understand why behavior seems to have changed.

If what you said is possible: "To achieve what you're after, consider triggering active full via scheduled monthly script instead," why not just build that functionality into the job ? (Also, if I had to use the scripted method i would, but have never scripted Veeam ops before....how would I do that part ?)

THANKS for all your great work in this Forum, and its great to see top dogs like Foggy and Gostev contributing !

[foggy]

What I still don't understand is why is my new string continuing to grow out to 30 retention points with .vib incremental backups before the next periodic active full the next month? Logic would tell me that I keep 7 total retention points, which then have to grow to 14 because of the new string with the active full in the middle, before the old string of 7 gets deleted and puts me back at 7 for the rest of the month until I do another active full?

There's no merge (like in forever forward) that would allow these 7 restore points to "move on" day by day until the next full. Instead, once the full is created, increment backups are created until the next full, as no single restore point can be deleted without breaking the chain. Once new full backup and 6 subsequent increments are created, the older part of the chain (full + 30 increments) can be deleted according to 7 restore points retention. Check out this article for visual representation of what's happening.

[foggy]

The guys before have a great point about being able to manage backup storage with predictable results, and the end result described by jcavalear is frustrating, for sure. Seems like it used to work exactly as they ave described, where you only see "too many restore points" for the amount of time it takes the *new* chain to honor the retention setting, as in jcavalear's case. IT seems none of us can understand why behavior seems to have changed.

Hi Mark, actually it never worked that way, it worked the way I described it above since early days - not touching existing backups (so no merges). What guys are requesting is the mixture of both methods, which is currently not possible.

If what you said is possible: "To achieve what you're after, consider triggering active full via scheduled monthly script instead," why not just build that functionality into the job ? (Also, if I had to use the scripted method i would, but have never scripted Veeam ops before....how would I do that part ?)

As a first step, I suggest to review the Veeam B&R PowerShell reference.

[TGacs]

Oh I understand that its not currently possible - but it would be awesome if it was!!!! Maybe you can fit it in to version 10

The transform option would require a synthetic full - I would prefer to use Active. Get that warm fuzzy feeling - ya know?

How about using two jobs:

• Forever incremental (FI)

• Backup Copy leveraging GFS

The idea is to rely on FI for a shorter period of time (say, 1 month). This provides the needed point-in-time granularity to restore a file or VM known to have failed/corrupted/etc. at a given specific time.
For older restores, rely on the less frequent GFS fulls. (In our environment, we don't need the same point-in-time granularity of FIs for older restores.)

See: https://helpcenter.veeam.com/docs/backu ... l?ver=95u4