Scheduled and Manual Replications, Best Practices

[cgsm]

I have a replication job scheduled for every 6 hours (after a backup job). I use the backup data as the source, not the live running VMs, to reduce the performance impact on the running VMs. This works great and I keep 12 snapshots of the replicas (3 days).

I have a secondary, manual initiated replication job using the VMs as the source. This is for times when I want to replicate data "now" and not have replica VMs with stale data based upon the last backup (could be up to 5:59 hours ago). The job runs successfully, however, the data in Veeam is a mess.

The "Home > Replicas > Ready" list shows duplicate entries for each of my replicated VMs, one for the scheduled replication job and another for the manually run job. I haven't run the manual job in about a month but I am still being show "2 Restore Points" which clearly do not exist on the VMs due to the "12 snapshots retained, replicated every 6 hours" policy. I have checked the replicas VMs and the manually run snapshots do not exists, as expected.

Is there some way to clean this list up automatically? Can it be displayed better?
Or, is there a better way to run both scheduled and manual replication jobs?

[PetrM]

Hello,

It's expected that you have duplicated entries because each job creates its own replica on target and you have an entry for every replica. However, I'm not sure that I understand why do you keep a separate job just for manual runs? You can trigger manually the main job as well.

Thanks!

[foggy]

Hard to understand what's going on via the forum posts, but having duplicate entries is expected with two jobs using the same VMs as a source. Each job has its own retention so make sure you're checking the VM replica created by the corresponding job, they should both exist and have the corresponding snapshots as restore points.

(Petr, the second job uses Iive VMs as a source, while the first one runs from the backups.)

[cgsm]

Hello,

Let me try to clarify a bit.
1) I cannot just manually start the scheduled replication because this job uses the backup files as the source, not the running VM. Therefore, if I manually start the scheduled job it will simply exit out since there is no newer backup data to replicate over. I use a separate, manually run, replication job using the running VMs as the source to bypass this issue so that the current state of the VMs is replicated.

2) I want to "replicate now" sometimes for testing, not having stale data at the replica destination due to expected downtime, etc.

******

I may misunderstand your notes, so please clarify. I am replicating the same list of VMs via the scheduled or manual job. For each VM, there is only one replica VM. Should I expect there to be two replicas, one for each job? Furthermore, the replica VMs only list the most recent 12 snapshots (my retention policy), not snapshots from both jobs; VBR shows "2 snapshots from manual job, 12 snapshots from scheduled job".

If I try to Failover in VBR by choosing one of the manual job snapshots/recovery points, the job unsurprisingly fails because the snapshot/point doesn't exist any more (removed by the 12 snapshot retention policy).

******

Maybe there is another way to do what I am trying to do. Yes, I could simply switch the scheduled replication job source from "backups" to "live VMs" and manually run the job as needed, but that impacts my production workload double (since I backup and replicate on the same 6 hour interval).

[PetrM]

I've just tried to reproduce the same scenario in my lab but I get the error saying that the replica already exists during run of the second replication job. The only way to make it work is to change replica suffix in this step of job settings. I don't understand how two different jobs work with the same replica in your case. I'd suggest to open a support request and ask our engineers to explain the behavior, please share a support case ID with us.

Thanks!

[cgsm]

Petr,
Same source VM, same replica VM, same suffix, using seed, with two separate jobs (scheduled using backups as source; manual using live VMs as source).

I re-ran the manual job today and here is what I have to report (immediately after the manual job completed):
1) Running the manual job causes a rehash of the disks.
2) The scheduled job created snapshots on the replica VM are removed. Only the two (why?) snapshots created during the manual process exist.

I let the scheduled job run again after the manual job and my notes are as follows:
1) The disks are rehashed.
2) The manually created snapshots are removed.
3) VBR shows "Replicas > Ready" for both the manual job and scheduled job even though the manual snapshots have been removed from the VMs and are unavailable to restore to.

******

I would have though a second, manual replicate job would just add a new recovery point snapshot to the replica. Clearly this is not so.

So based on these tests and my previous results, it does not look like having two jobs replicating the same VM is possible (with same replica name VM name/suffix). Therefore, the only way to manually replicate a VM off-schedule is to use the live VMs as the source and manually running the scheduled job.

I think there could be some improvement here (or at least some better documentation regarding this).

[foggy]

Having two different jobs writing to the same target is a bad idea anyway and shouldn't work, as Petr has mentioned above. Not sure why it somewhat did work in your case - need to investigate - but please use different prefixes in this case.

[cgsm]

Foggy,
I do not have the storage space to maintain two replica copies of each VM (with different prefixes). Otherwise, yes, this would be a solution, albeit a bit messy since we would have duplicate replicated VMs.

All I want to be able to do is run a scheduled replication job manually every so often where the replica data is near identical to the source live running VM (again, system maintenance, planned outage, etc.). It sounds like the only way to do this is to replicate the VMs using the live running VMs as the source. I will have to switch the source on my replication job and monitor to see if the snapshot creation/deletion on the live VM causes any issues (VM stun leading to disconnects or response issues).

Maybe there is a way to add an option to a manual run replication using the backups as the source? An option that says something like "do you want to replicate from the live VMs instead?"

[PetrM]

Hello,

Just one question: why don't run the backup job which will produce the source of data for replication job? Basically, it allows you to achieve the same goal: to trigger transfer of the actual data.

Thanks!

[cgsm]

Petr,

Yes, I understand I can trigger the backup job which will then cause the replication job to see "new" data. That is my plan going forward unless I switch replications use the the VMs as the source.

The only reason I don't want to have to run the backup job first is timing: running the backup job delays the replication job from running right away. The delay can vary based upon what the backup job has to do, how much data has changed, etc.

[PetrM]

Hello,

I understand your concerns but replication from backup has a big advantage: it allows you to respect 3-2-1 rule and minimize load on the source infrastructure by creating snapshot and transferring VM data just once. At some point, it might be even more important than to save time on manual incremental run, moreover I suppose you don't need to do it very often. Also, you may give CDP a try as it constantly replicates I/O operations and provides minimal RTO.

Thanks!