Veeam Replication - Cluster to Cluster

[L33T]

Hi,

I`m having some issues with replication from a Hyper-v cluster to a DR Hyper-v cluster. It appears that if a vm is moved from host to host in the DR-Cluster then a duplicate replica machine is created. Does veeam not check vm location in remote cluster when the replication job starts to ensure it can replicate the changes across instead of creating a new VM?

Cheers,

[veremin]

Backup server should be aware of replica VM movement, as long as cluster is added to backup server as a whole, not partially (say, one node). Thanks.

[L33T]

It has been added via SCVMM so it`s aware of the cluster and associated nodes... I will complete a seed replication and then move a vm within the cluster to see if it creates a duplicate vm then report back with findings.

Cheers,

[veremin]

Then, backup server should be able to locate VM, even if it's moved to a different cluster node. Should you experience the opposite behavior, feel free to reach our support team. Thanks.

[L33T]

Yep already got a case open. Will update this thread also with any news.

[foggy]

There are cases when VM could be "lost", such as attempt to move it between nodes right at the moment the job starts.

[L33T]

Update, moved a vm on DR cluster and veeam did not create a duplicate. Since found that veeam has duplicated all VM`s on the backup which in turn is being replicated. Despite no changes or moved on the source end veeam are looking into why this has happened, we could potentially loose 90 days worth of backups as we cannot storage another 90 along side it By the way this happened the day after 9.5 U3 was installed...

Cheers,

[foggy]

Have you contacted support with this? If yes, please share the case ID for reference.

[L33T]

Hi,

Yes case ref: 02457302

Taken a while and hasn't really been looked into yet, more log files needed apparently.

[foggy]

Thanks, please keep working with your engineer.

[L33T]

2 weeks of reviewing logs and getting no where. Has now been escalated to 2nd level... Really should not take this long to investigate surely?

Where we are:

Update 3 installed on the 6th Jan, jobs started failing
Identified GUID change somewhere despite the infrastructure having zero changes made
Asked for details on what GUID was seen before and after the change so we can see exactly what veeam has found, pretty much ignored the question
Duplicate replicated VM`s created on replication end
All 90 days of our 90 day repo now gone on the backup, something on the veeam job over night removed it, veeam to investigate.

I`ll update asap.

Cheers,

[L33T]

update:

No feedback yet as to the the 90 days missing
Escalated to second line for investigation on the GUID changes, they are attempting this in lab apparently

No other issues at the moment, fingers crossed it won`t happen again but without knowing why it did before i suspect it may.

Cheers,

[L33T]

Update and feedback from Veeam Support...

`The issue was reproduced in our enviroment. The problem was caused by a fix for another issues, which was implemented in U3.
We are currently deciding possibilities for fixing it.`


Am i to assume then that U3 is the cause of our duplicate reps and also a loss of 90 days of backups on our repo?

Awaiting a reply...

[veremin]

In the meantime we will also investigate that internally. Thanks.

[L33T]

Can someone get back to me about the 90 days disappearing, no replies to that as yet? I have had little response to questions through the case and it`s been over a month since it was initially opened so i hope you can see my frustration with communications and length of time with this.

Cheers,

[foggy]

It's hard to guess what were the reasons for that, your engineer has more information about your setup and what had actually happened, as well as the logs, so I'd ask him/her about that. If you feel the case is not treated properly, you can provide your feedback using the 'Talk to a Manager' option available on the customer portal.

[veremin]

Can someone get back to me about the 90 days disappearing, no replies to that as yet? I have had little response to questions through the case and it`s been over a month since it was initially opened so i hope you can see my frustration with communications and length of time with this.

In Update 3 we start identifying clustered VMs a bit differently (a fix to another issue). So, if you have a cluster added to backup console via SCVMM, clustered VMs (not local ones) residing on cluster nodes will get new IDs and will be treated as new ones. This will lead to new full backups and duplicated replica entities.

You had a deleted VM retention period configured. In your case the specified value was lower than selected retention period. As the result, clustered VMs with obsolete IDs that from backup server perspective were no longer processed were wiped out from backup files.

Thanks.

[L33T]

That`s fine BUT not knowing this change had been made, the engineer didn`t know either so escalated it and it took almost 2 weeks to get a `lab session` organised to then tell me that this change was made in U3. We could of avoided the 90 days being removed if that information was at hand earlier (4 weeks) and i would have removed the retention option enabling our now second repo of historical backups to catch up to the 90 days we lost and as such ensure we had resilience.

I think that`s a fair point, things could have been handled better...

Regards,

[veremin]

Agreed. But before the case got escalated to R&D, the issue (caused by one of the fixes introduced by Update 3) was unknown for us. So, the time it took us to find the root cause is explainable.

Anyway, I just wanted to share the detailed description with you as well as warn you that having deleted VM retention period lower than generic retention period is not recommended practice that might lead to multiple issues. Use it cautiously.

Thanks.