CDP Short-Term retention explaination?

[bhagen]

Ok; maybe I just haven't had enough coffee yet this morning, but I gotta admit: I'm having a hard time wrapping my brain around the short- and long-term retention settings in the new v11 CDP policies.

I've read the docs and myriad blog posts that state:

• Recovery point objective (RPO) - the maximum acceptable data loss in case of a protected VM failure (default 15 seconds)

• Short-term retention - how many restore points of previous setting (RPO) you want to keep (default 4 hours)

• Long-term retention - additional restore points to take/keep (as default a restore point is taken every 8 hours and kept for 7 days)

But that doesn't compute with what I've been expecting out of CDP.

I thought that the goal of CDP was: if a VM fails/dies/blows up, I can restore it to, at the most, 10 minutes (or 1 minute, or 15 seconds, or whatever) before it died.

I expected that if I set an RPO for 10 minutes and then start a failover, the only choice I'd have is to failover to the last "sync", which would be - at the most - 20 minutes ago, depending on when the last 10 minute window had expired.

But with the default policy settings of 10 minute RPO, 4hr short-term retention, 8hrs / 7days long-term retention, I'm only able to failover to 3 points-in-time per day (midnight, 8am, and 4pm).

??

I don't get it.

I'd appreciate any help understanding this!

[Gostev]

With the CDP policy settings above, you should be able to perform failover to any point in time within past 4 hours, with 10 minutes steps. In other words, you would have 24 short-term restore points (60 minutes in an hour /10 minutes RPO x 4 hours) to choose from in the past 4 hours.

In addition, you would have 21 long-term restore points (24 hours in a day / 8 hours x 7 days) spanning past 7 days with 8 hours steps.

[bhagen]

Ah! I think we need to market a new coffee and/or energy drink: "Gostev Coffee...for when you just don't understand!" Maybe the logo could be somebody being smacked upside their head.

I read your reply and thought "How can that be? I only see 2 points in time to restore from so far today?" (Failover now, click the vm, click Point..., click Restore to a point in time, look confusedly at the UI, click the Restore point link at the bottom, only see midnight and 8am, click the X)

Then I was like "Wait; maybe this scroll bar is active even though it looks greyed out??"

So I clicked the left arrow in the scrollbar...and found all the restore points you mentioned.

P0WND by a new Veeam UI that I haven't yet found an explanation of! *facepalm*

Well, at least now I am fully awake! Thanks once again @Gostev.

[mcz]

I'd like to add my question to this thread:

https://helpcenter.veeam.com/docs/backu ... ml?ver=110

Short-Term Retention

2. If an outdated restore point exists, Veeam Backup & Replication commits data for the restore point from the transaction log file (.tlog or .tlog.vmdk for VSAN and VVOL) into the nearest delta or base disk file (<disk_name>-flat.vmdk or <disk_name>-<index>.vmdk).

I'm not able to picture that in my head... Long-term-retention is clear, you have one delta-disk per restore point (say every 8 hours for 2 days). But when you write/commit the changes from short-term-retention (say every 3 minutes for 4 hours) into the deltas from the long-term, you can't restore from the long-term I'd say because the short-term has overwritten the state from the long-term in the delta-disk...

It would make more sense if it said that veeam would create a new delta-disk and wrote the changes from the short-term into that delta-disk. That would make sense to me...

What's wrong here? Thanks!

[veremin]

I think there is one specific detail (necessary for understanding) that is currently missing in the User Guide, the part that explains that besides long-term delta disks there are the so-called "system" delta disks that we create to split transaction log into several smaller chunks: the short-term retention interval gets divided into four or five parts, so new delta disk is created every quarter of the interval. Additional delta disk can be created, if current transaction log reaches a certain size threshold, but for sake of simplicity we can neglect this case.

So upon retention transaction logs get committed to those "system" delta disks.

We will reach our documentation team to enhance current short-term retention policy explanation.

Thanks, Michael, for raising this and sorry that thread update took a bit longer than expected

[mcz]

Thanks Vladimir for your response and thanks for the explanation! So when I restore a CDP replica to a specific timestamp, what will happen then? The transactions (until that point in time) are written into the system disks and then the vm is started or are the changes written to the long-term-deltas?

When you stop the failover the changes have to be reverted, so I wonder how that is implemented...

Thanks!

[veremin]

During the failover process we do not commit or replay anything, instead we instruct the responsible component to read the closet delta plus blocks from specific part of transaction log, using block map. Only if you commit a failover, will these logs be committed to the delta disk. Thanks!

[mcz]

Very very interesting! Always amazing what you can do with a vm and what different APIs and mechanisms exist. Thanks very much Vladimir, that is very helpful!