Replica testing and BSOD

[bobo1st]

Hi,

We have just finished setting up a DR Server where we are replicating most of our servers fine.

We have a problem with our 3 Windows 2000 Servers. When we replicate these servers for the first time, you can log into them and boot them fine. However, our problem comes when we re-replicate the servers the following evening. The Veeam replication works fine with no errors, but when we come to boot the re-replicated windows 2000 servers, the operating systems are corrupted. We have tried this with both block tracking turned on and off, but every time we get the same fault.
All other servers on all other operating systems run fine. I know Windows 2000 does not support VSS so this may be are issue. However, Veeam do state that Windows 2000 server is fully supported for replication.

Please can someone advise why this is happening. This is driving us mad. The error is a corruption in the WinNT/System32/Systemed. We have tried the standard windows repairs, but we just ended up with the dreaded Blue Screen Of Death.

Please help

[Gostev]

Hi, be sure to make a snapshot each time you start the replica VM for testing, and to roll the snapshot back before continuing the replication. By starting up the replica without protecting its disks with snapshot, you make VMDK go out of sync with what replication job expects them to be. As a result, the following incremental updates produce inconsistent VMDK (incremental updates created for certain VMDK state are applied on modified VMDK state). There are existing topics on replica testing that you may refer to for more information on correct testing procedure. Thanks.

[micwe]

Hello Gostev
I have the exactly the same problem when trying to replica windows server 2000 for second time.
When trying to boot the replica I get windows 2000 corrupt files
First replica work fine
you mention in your post

be sure to make a snapshot each time you start the replica VM for testing

But when replica job start it take a snapshot automatically
So What is the correct way to do a replica to a windows 2000 server without getting corrupt files

Thanks
Micwe

[Gostev]

Hi, I am talking about manually making snapshot on replica VM, you must do that manually before you run the replica VM for testing purposes. Replication job automatically only takes snapshots of source (production) VM. This has nothing to deal with replica VM.

Kindly please search the forum for the existing discussions on correct replica testing procedure. I believe there was even a link to step-by-step guide with screenshots posted by someone in the existing discussion.

Thanks.

[Gostev]

Also, please note that we do not officially support performing any operations on replica VM outside of product UI, so if replica corrupts due to improper testing procedure, you will be on your own. Our support will request that you re-create the replication job.

Please note that we provide all required functionality from replica testing directly from UI (click Failover, test, click Undo Failover). This is what you should be using to test your replicas. Manual testing procedure is intended for advanced users and advanced use cases only.

[micwe]

Hello Gostev
I read the Testing Veeam Backup & Replication Replica Testing Procedure
It doesn’t make any sense for a regular weekly backup
http://www.virtualizationteam.com/veeam ... edure.html

I was wondering if this is the case in replica win2000 server and roll with snap shoot so why use replica backup !
And why there is an option in replica "restore points to keep on disk" if it will never work
In this scenario when I what to backup my win200 server "VM copy" is better for this job
It will always have the latest working image
Am I correct?

[Vitaliy S.]

And why there is an option in replica "restore points to keep on disk" if it will never work

Failing over to a previous points will always work - if you use Veeam Backup UI to failover to earlier restore points (which is the only supported way to perform failover to an earlier restore point).

In this scenario when I what to backup my win200 server "VM copy" is better for this job
It will always have the latest working image
Am I correct?

Yes, but VM Copy doesn't perform incremental runs, neither it keeps earlier restore points. Please take a look at our sticky F.A.Q. for better understanding replication and copy job processes.

[Gostev]

Hello Gostev
I read the Testing Veeam Backup & Replication Replica Testing Procedure
It doesn’t make any sense for a regular weekly backup
http://www.virtualizationteam.com/veeam ... edure.html

It does not make any sense what you are saying? What weekly *backup* are you talking about? As per article's name, this process is intended for manual *replica* testing (not for backup testing). It allows to test the latest replica restore point only (not earlier restore points), and then continue the replication normally.

[micwe]

Hello Gostev and Thanks for replying
I finally got it
When I power up the win2000 replica it gets out of sync from the original stat
Then after next replica I boot the win2000 for testing the replica is corrupted
So the right way if I want to test the replica and continue the replica job is to take snapshot of the replica like it suggest in link
http://www.virtualizationteam.com/veeam ... edure.html

But After booting win2003 replication it always work fine for me
So why is this issue not effecting win2003 2008 servers?
I get corrupted only on win 2000 server?

Micwe

[Gostev]

Corruption will always happen in this case, no matter of OS used. However, depending on virtual disk layout and OS boot behavior, corruption may not be immediately noticeable. In cases when corruption affects virtual disk blocks backing critical system files, this of course would prevent OS from booting up, and thus you are immediately able to spot the issue. In other cases, corruption will not affect OS files (but some other files), and OS will boot up normally, and you will not notice any issues - even though some of your data is corrupted. This, of course, is worst-case scenario, since you will not know that your data is bad.

[micwe]

Gostev thanks for clearing things up
Your suggestion made me change my DR backup strategy
As Critical server will have an daily incremental replica job
And low priority server will have a weekly vm copy job

Thanks
Micwe

[rbrambley]

Sorry if I am "beating a dead horse", but this thread was a bit "cloudy" to me and I work for Veeam!

In short if you use the Veeam BaR GUI failover option from the Restore button / wizard, BaR will handle all snapshots of the replica VM for you and prevent both VM corruption and "hosing" the Veeam BaR replication job. Be sure to use the Undo Failover option from the Veeam BaR GUI when done. Be warned that if your source and replica locations are on the same network then you could have conflicts. BaR does not shut down the source VM.

Use the previously provided blog link for manual testing (or actual DR fail over) without Veeam. Be aware that starting the replica VM without using the Veeam GUI (with the vSphere client or powershell for example) could "hose" the replication job moving forward. If that happens you will need to reconfigure the replication job.

[micwe]

Hello rbrambley
It is perfectly clear
For production and real failed and DR it is ideal to use the failover option
But for testing replica vm the blog link is the preferred way
But it's not supported by Veeam because your bypass the replica job
This procedure can damage the vm disk

[cyberfed]

[merged]

Hey folks,
Our company has been using an eval of the latest version of Veeam Backup and Replication to see if we want to purchase.

So far everything is going well we are quite impressed with the product.
We have run into one snag though.

I have been testing the replication feature. I took one of our VM's and had it replicate it.
Good no problems, turned on the replica and it was good.
Then I turned on the orginal VM added some files to the desktop (to use as proof that injection was working) and let the scheduled replication job run overnight.

Today I looked at the jobs in Veeam and it says the replica succeeded last night. I go and turn on the replica copy, expecting to log into windows and see the files I had added to the source VM injected into the replica, instead I get a nice bluescreen.
Reboot the replica and try again, nothing, BSOD again.

Thoughts?

[cyberfed]

Just saw the possible fix. Looks like my issue was I turned on the replica without doing a snapshot and then letting the replication run again.
Will re-do my replica and try again and let you all know of the results.

[jim3cantos]

Just saw the possible fix. Looks like my issue was I turned on the replica without doing a snapshot and then letting the replication run again.
Will re-do my replica and try again and let you all know of the results.

Just to alert other people jumping to this thread. This posible reason for replica corruption doesn't hold anymore.