-
- Enthusiast
- Posts: 65
- Liked: 1 time
- Joined: Apr 28, 2012 9:51 pm
- Full Name: Ori Besser
- Contact:
Replicated VMs Start with bluescreen. v7 [Case # 00826751]
Hi,
In our latest DR drill, we have found out that 7 of our replicated VMs (out of 162) got bluescreen at the very start of the boot process.
By using diskpart I saw that for all of these VMs the C partition was there but it had no NTFS structure - like it's not formatted. In addition, several more VMs were starting with the chkdsk process - a behavior we have never seen before.
Re-replicating incrementally did not help, only full replication fixed it.
It's interesting that all of these VMs are Server 2003, while we have a lot of other OSs too.
We are not using the replica failover process, but manually start the VMs at the DR site.
The support answer was that manually powering on replicas is not recommended (while the "Replication" section of the FAQ by gostev clearly states otherwise), I have to admit that I expected a little deeper investigation on an issue that raises doubts on replication integrity. We will re-replicate and we will implement Sure Replica jobs but we have to know what in our environment caused it - and fix it.
I want to state that we were using Veeam extensively for the last 3 years and never had one issue.
In our latest DR drill, we have found out that 7 of our replicated VMs (out of 162) got bluescreen at the very start of the boot process.
By using diskpart I saw that for all of these VMs the C partition was there but it had no NTFS structure - like it's not formatted. In addition, several more VMs were starting with the chkdsk process - a behavior we have never seen before.
Re-replicating incrementally did not help, only full replication fixed it.
It's interesting that all of these VMs are Server 2003, while we have a lot of other OSs too.
We are not using the replica failover process, but manually start the VMs at the DR site.
The support answer was that manually powering on replicas is not recommended (while the "Replication" section of the FAQ by gostev clearly states otherwise), I have to admit that I expected a little deeper investigation on an issue that raises doubts on replication integrity. We will re-replicate and we will implement Sure Replica jobs but we have to know what in our environment caused it - and fix it.
I want to state that we were using Veeam extensively for the last 3 years and never had one issue.
-
- VP, Product Management
- Posts: 27377
- Liked: 2800 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: Replicated VMs Start with bluescreen. v7 [Case # 0082675
Do you revert to the previous state after doing all the tests, or just commit all the changes to the vmdk/snapshot via vSphere Client?
-
- Enthusiast
- Posts: 65
- Liked: 1 time
- Joined: Apr 28, 2012 9:51 pm
- Full Name: Ori Besser
- Contact:
Re: Replicated VMs Start with bluescreen. v7 [Case # 0082675
Just power off the machines and let Veeam continue running the replication jobs as usual.
-
- Enthusiast
- Posts: 88
- Liked: 2 times
- Joined: Jul 31, 2013 12:05 pm
- Full Name: Si
- Contact:
Re: Replicated VMs Start with bluescreen. v7 [Case # 0082675
That's what I do if I want to test something out on the DR VMs, because I thought Veeam automatically reverts the snapshot before it starts replicating any changes anyway so any changes I've made are lost at that point?ori wrote:Just power off the machines and let Veeam continue running the replication jobs as usual.
Btw, I had a similar issue a while back, do you use hotadd at the DR/replicated end? I don't know if that was the cause but support recommended I switch to NBD (network) mode and there isn't any performance difference anyway.
-
- Enthusiast
- Posts: 65
- Liked: 1 time
- Joined: Apr 28, 2012 9:51 pm
- Full Name: Ori Besser
- Contact:
Re: Replicated VMs Start with bluescreen. v7 [Case # 0082675
Thanks captain, I use hot-add. Did support say why to stop using hot-add?
I just need to know what happened in my environment in order to avoid it in the future.
I just need to know what happened in my environment in order to avoid it in the future.
-
- VP, Product Management
- Posts: 27377
- Liked: 2800 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: Replicated VMs Start with bluescreen. v7 [Case # 0082675
Yes, that's correct.CaptainFred wrote:That's what I do if I want to test something out on the DR VMs, because I thought Veeam automatically reverts the snapshot before it starts replicating any changes anyway so any changes I've made are lost at that point?
Both hotadd and NBD mode is fully supported to write data via target proxy, this should not cause any issues for target VM.CaptainFred wrote:Btw, I had a similar issue a while back, do you use hotadd at the DR/replicated end? I don't know if that was the cause but support recommended I switch to NBD (network) mode and there isn't any performance difference anyway.
Keep working with our support team on this and let us know the outcome.ori wrote:I just need to know what happened in my environment in order to avoid it in the future.
-
- Enthusiast
- Posts: 65
- Liked: 1 time
- Joined: Apr 28, 2012 9:51 pm
- Full Name: Ori Besser
- Contact:
Re: Replicated VMs Start with bluescreen. v7 [Case # 0082675
The support team says we had the corruption because we didn't manually take a snapshot before powering on a replica, but if Veeam automatically reverts to the restore point why is it necessary to take another snapshot?
-
- Enthusiast
- Posts: 65
- Liked: 1 time
- Joined: Apr 28, 2012 9:51 pm
- Full Name: Ori Besser
- Contact:
Re: Replicated VMs Start with bluescreen. v7 [Case # 0082675
Just to add on that, one of the VMs that had the issue (our PDC domain controller) was a fresh replica - it is a new VM that was never powered on before, this test was the first time it has been powered on.
-
- VP, Product Management
- Posts: 27377
- Liked: 2800 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: Replicated VMs Start with bluescreen. v7 [Case # 0082675
Taking snapshot manually is not required. Did you use application-aware image processing for the VM with DC role installed?
-
- Enthusiast
- Posts: 65
- Liked: 1 time
- Joined: Apr 28, 2012 9:51 pm
- Full Name: Ori Besser
- Contact:
-
- VP, Product Management
- Posts: 27377
- Liked: 2800 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: Replicated VMs Start with bluescreen. v7 [Case # 0082675
Ok, keep working with our support team then and please update this topic once you find the resolution. Thanks!
-
- Enthusiast
- Posts: 65
- Liked: 1 time
- Joined: Apr 28, 2012 9:51 pm
- Full Name: Ori Besser
- Contact:
Re: Replicated VMs Start with bluescreen. v7 [Case # 0082675
but the support team concluded with saying we should take a snapshot before testing, they don't further investigate
-
- Expert
- Posts: 224
- Liked: 25 times
- Joined: Apr 30, 2013 7:38 am
- Full Name: Vlad Valeriu Velciu
- Contact:
Re: Replicated VMs Start with bluescreen. v7 [Case # 0082675
From what I now, manually testing replicas without using surereplica or the failover process within Veeam needs some steps taken to guarantee the integrity of the replica after testing.
Every step is described here including taking a snapshot at the beginning: http://www.veeam.com/kb1073
Every step is described here including taking a snapshot at the beginning: http://www.veeam.com/kb1073
-
- Enthusiast
- Posts: 65
- Liked: 1 time
- Joined: Apr 28, 2012 9:51 pm
- Full Name: Ori Besser
- Contact:
Re: Replicated VMs Start with bluescreen. v7 [Case # 0082675
So taking into account Vitaliy's reply and the FAQ - the KB is outdated / wrong
-
- VP, Product Management
- Posts: 27377
- Liked: 2800 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: Replicated VMs Start with bluescreen. v7 [Case # 0082675
Veeam backup server triggers revert to snapshot operation before each run of the replication job, so creating snapshot manually is not required. Yes, KB is out-dated and was relevant to version 6. Sorry for the confusion, I will ask our support team to make the corresponding updates to that article.
Who is online
Users browsing this forum: No registered users and 31 guests