-
- Influencer
- Posts: 13
- Liked: never
- Joined: May 20, 2009 5:59 pm
Need feedback on this migration approach please
Hey folks,
Ok, here is my scenario.
Source Network/Details:
-- 2 Windows Server 2008/64bit Virtual Servers at a Datacenter in the US
-------- 1 Guest is Exchange Server 2007 with 3 large databases around 20GB each (total size of data is around 110GB)
-------- 1 Guests is Windows SharePoint Services 3.0 (with WID/SQL database) – data is about 30GB
-- VMWare host is ESXi (no VCB)
-- Internet throughput is decent
Destination Network/Details:
-- VMWare host is ESXi (but could be ESX and VCB if necessary)
-- Internet up is 3MB, down is about 6MB
The Reason for Asking:
I no longer trust the hosting facility/platform. We’ve had reason to want to move for some time due to various issues, one of which is that we can’t ‘physically’ *touch* these servers and we are at the mercy of the datacenter response time when problems occur. I need these servers running locally
The Challenges:
-- I only have a 24 hour window in which these servers can be *down* to make this move happen
The Solution – Proposed but I’m seeking feedback and expertise):
-- Implement VEEAM Backup and Replication and run the console from the destination host
-- Perform a full replication of the two Virtual guests (Exchange Server and SharePoint Services Server) – let this take as long as it takes – 5 days, 2 weeks, whatever – just get the first replica
-- Once the first replica finishes, target a weekend to complete a ‘follow-up snapshot’ of changes
-- Once the follow-up snapshot completes, re-point DNS for all clients to go to the new systems, at the destination location
Questions/Concerns
-- Am I nuts? Over two weeks, Exchange Databases would be modified quite a bit, assuming I’m depending on VSS (I am right?), it should just be changes to the data and not complete re-copies of the databases, right?
Any and all comments, feedback, guidance and warnings appreciated.
Ok, here is my scenario.
Source Network/Details:
-- 2 Windows Server 2008/64bit Virtual Servers at a Datacenter in the US
-------- 1 Guest is Exchange Server 2007 with 3 large databases around 20GB each (total size of data is around 110GB)
-------- 1 Guests is Windows SharePoint Services 3.0 (with WID/SQL database) – data is about 30GB
-- VMWare host is ESXi (no VCB)
-- Internet throughput is decent
Destination Network/Details:
-- VMWare host is ESXi (but could be ESX and VCB if necessary)
-- Internet up is 3MB, down is about 6MB
The Reason for Asking:
I no longer trust the hosting facility/platform. We’ve had reason to want to move for some time due to various issues, one of which is that we can’t ‘physically’ *touch* these servers and we are at the mercy of the datacenter response time when problems occur. I need these servers running locally
The Challenges:
-- I only have a 24 hour window in which these servers can be *down* to make this move happen
The Solution – Proposed but I’m seeking feedback and expertise):
-- Implement VEEAM Backup and Replication and run the console from the destination host
-- Perform a full replication of the two Virtual guests (Exchange Server and SharePoint Services Server) – let this take as long as it takes – 5 days, 2 weeks, whatever – just get the first replica
-- Once the first replica finishes, target a weekend to complete a ‘follow-up snapshot’ of changes
-- Once the follow-up snapshot completes, re-point DNS for all clients to go to the new systems, at the destination location
Questions/Concerns
-- Am I nuts? Over two weeks, Exchange Databases would be modified quite a bit, assuming I’m depending on VSS (I am right?), it should just be changes to the data and not complete re-copies of the databases, right?
Any and all comments, feedback, guidance and warnings appreciated.
-
- Chief Product Officer
- Posts: 31806
- Liked: 7299 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Need feedback on this migration approach please
Hello, your plan seems fine to me. You would definitely need "fat" ESX in your target environment though, because the current version can only work with "fat" ESX as replication target, and not ESXi. VCB is not needed on target side.
Based on provided network speed (6MB down = 480Mb), first replica should complete in under 8 hours, and subsequent passes will probably take about 1 hour @ full link speed (during off-hours).
I think best would be to do more than 2 synchronizations though, here's how I would do this (from the top of my head):
1. Perform initial synchrhonization.
2. Perform incremental syncrhonization as soon as initial finishes (to apply changes happened during long initial sync). That would only sync changed blocks.
3. Test replica to make sure it starts up fine. Make sure replica is isolated from production in term of networking, use Veeam Backup to "Failover" to it (don't start it manually!). Exchange may throw errors because unable to contact DNS/DC, but as long as VM starts up, all is well. After that, be sure to perform "Undo failover" from Veeam Backup, this would discard test failover snapshot and your replication job will be able to continue to write updates to existing unchanged VMDK.
3. Run daily incremental synchronizations until day X and monitor how much time do they take (so that it does not become a surprise for you later).
4. On day X, stop SMTP service on your source Exchange to prevent incoming email from coming in, and prevent user access to Exchange (don't remember how it is best to do this).
5. Run final incremental replication.
6. Shut down source Exchange.
7. Perform "Failover" to replica in Veeam Backup, wait until it starts up.
8. Make all required DNS changes, enable user access, test Exchange connectivity and do all those Exchange admin things to make sure all is well.
9. If all OK, enable SMTP service for incoming email to start coming in.
10. When you are confident that all is well, remove snapshot created be Veeam Backup during Failover operation by simply deleting it with VIC (removing snapshot would merge all snapshot data into VMDK). You can also delete all *.VRB files found next to VMDK at this time.
Hope this helps.
Based on provided network speed (6MB down = 480Mb), first replica should complete in under 8 hours, and subsequent passes will probably take about 1 hour @ full link speed (during off-hours).
I think best would be to do more than 2 synchronizations though, here's how I would do this (from the top of my head):
1. Perform initial synchrhonization.
2. Perform incremental syncrhonization as soon as initial finishes (to apply changes happened during long initial sync). That would only sync changed blocks.
3. Test replica to make sure it starts up fine. Make sure replica is isolated from production in term of networking, use Veeam Backup to "Failover" to it (don't start it manually!). Exchange may throw errors because unable to contact DNS/DC, but as long as VM starts up, all is well. After that, be sure to perform "Undo failover" from Veeam Backup, this would discard test failover snapshot and your replication job will be able to continue to write updates to existing unchanged VMDK.
3. Run daily incremental synchronizations until day X and monitor how much time do they take (so that it does not become a surprise for you later).
4. On day X, stop SMTP service on your source Exchange to prevent incoming email from coming in, and prevent user access to Exchange (don't remember how it is best to do this).
5. Run final incremental replication.
6. Shut down source Exchange.
7. Perform "Failover" to replica in Veeam Backup, wait until it starts up.
8. Make all required DNS changes, enable user access, test Exchange connectivity and do all those Exchange admin things to make sure all is well.
9. If all OK, enable SMTP service for incoming email to start coming in.
10. When you are confident that all is well, remove snapshot created be Veeam Backup during Failover operation by simply deleting it with VIC (removing snapshot would merge all snapshot data into VMDK). You can also delete all *.VRB files found next to VMDK at this time.
Hope this helps.
-
- Influencer
- Posts: 13
- Liked: never
- Joined: May 20, 2009 5:59 pm
Re: Need feedback on this migration approach please
Thanks Gostev,
So to be clear I understand you.
1. It is fine that the source (old VM, at US datacenter) is running ESXi (does anything get 'installed' there?)
2. The destination (new kit) must be running full ESX (fine - do I need VCB or just full ESX?)
As far as within the Windows guests, are there any other requirements? Does anything get installed within them and/or do I need VSS enabled/configured or is this all happening at the ESX level?
Thanks again.
So to be clear I understand you.
1. It is fine that the source (old VM, at US datacenter) is running ESXi (does anything get 'installed' there?)
2. The destination (new kit) must be running full ESX (fine - do I need VCB or just full ESX?)
As far as within the Windows guests, are there any other requirements? Does anything get installed within them and/or do I need VSS enabled/configured or is this all happening at the ESX level?
Thanks again.
-
- Chief Product Officer
- Posts: 31806
- Liked: 7299 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Need feedback on this migration approach please
1. Yes - ESXi is fine as a source, nothing needs to be installed there. But you want to install Veeam Backup at source site, because of data flow considerations. This is explained in more details here, see my post with different font colors (since you are using ESXi as source, this is exactly your case).
2. Just full ESX, VCB is not needed (VCB can only be used to retrieve VM data, it does not have functionality to write/restore VM data anyway).
Windows guest doesn't need anything installed, small Veeam VSS agent will be put there as a part of snapshot process automatically, and will be removed snapshot is created. You just need to make sure to enable VSS option in Replica Job wizard, and provide valid admin credentials for the guest.
2. Just full ESX, VCB is not needed (VCB can only be used to retrieve VM data, it does not have functionality to write/restore VM data anyway).
Windows guest doesn't need anything installed, small Veeam VSS agent will be put there as a part of snapshot process automatically, and will be removed snapshot is created. You just need to make sure to enable VSS option in Replica Job wizard, and provide valid admin credentials for the guest.
-
- Influencer
- Posts: 13
- Liked: never
- Joined: May 20, 2009 5:59 pm
Re: Need feedback on this migration approach please
awesome, thanks for the detailed response.
-
- Influencer
- Posts: 13
- Liked: never
- Joined: May 20, 2009 5:59 pm
Re: Need feedback on this migration approach please
Ok, I've attempted to kick this thing off but I'm stuck at the moment...
My interpretation of the problem is that NAT seems to be the issue here. When I try to launch the backup, it goes for about 5 minutes and then ends with this... (note, I've replaced the Public IP's with x.x.x.x for the source, y.y.y.y for the target and z.z.z.z for the VM guest running VEEAM to protect the innocent
1 of 15 files processed
Total VM size: 260.00 GB
Processed size: 0.00 KB
Avg. performance rate: 0 KB/s
Backup mode: agentless
Start time: 6/7/2009 12:40:26 PM
End time: 6/7/2009 12:46:26 PM
Backing file "nfc://conn:z.z.z.z,nfchost:ha-host,stg:48a28056-3cb805ba-9117-001ec9541b1a@XXus1vmg02/XXus1vmg02.vmx"
Agents failed to start, server "localhost", client "y.y.y.y"
Cannot connect to server [192.168.250.2:2501].
Note, that I was able to add the source (ESXi) and target (ESX) Servers into the VEEAM console with no problems, including specifying Service Console/SSH settings - I was able to see both datastores, etc.
The IP 192.168.250.2 is the LAN IP for the VEEAM Server (which of course the source or target can't see, by that IP). I've tried the "Run Server on this Side" option, checked and unchecked as Troubleshooting seems to imply it may help in a NAT situation, though there was no effect here.
What am I missing? Do I need to have a VPN in place between the two?
Should I be specifying FQDN's instead of the IP perhaps, so I can override what the destination is?
Help!
My interpretation of the problem is that NAT seems to be the issue here. When I try to launch the backup, it goes for about 5 minutes and then ends with this... (note, I've replaced the Public IP's with x.x.x.x for the source, y.y.y.y for the target and z.z.z.z for the VM guest running VEEAM to protect the innocent
1 of 15 files processed
Total VM size: 260.00 GB
Processed size: 0.00 KB
Avg. performance rate: 0 KB/s
Backup mode: agentless
Start time: 6/7/2009 12:40:26 PM
End time: 6/7/2009 12:46:26 PM
Backing file "nfc://conn:z.z.z.z,nfchost:ha-host,stg:48a28056-3cb805ba-9117-001ec9541b1a@XXus1vmg02/XXus1vmg02.vmx"
Agents failed to start, server "localhost", client "y.y.y.y"
Cannot connect to server [192.168.250.2:2501].
Note, that I was able to add the source (ESXi) and target (ESX) Servers into the VEEAM console with no problems, including specifying Service Console/SSH settings - I was able to see both datastores, etc.
The IP 192.168.250.2 is the LAN IP for the VEEAM Server (which of course the source or target can't see, by that IP). I've tried the "Run Server on this Side" option, checked and unchecked as Troubleshooting seems to imply it may help in a NAT situation, though there was no effect here.
What am I missing? Do I need to have a VPN in place between the two?
Should I be specifying FQDN's instead of the IP perhaps, so I can override what the destination is?
Help!
-
- Veeam Software
- Posts: 268
- Liked: 63 times
- Joined: Jan 01, 2006 1:01 am
- Full Name: Stanislav Simakov
- Contact:
Re: Need feedback on this migration approach please
In this configuration your target host should be able to connect to your backup console on TCP ports used for data transfer (starting from 2500). "Run server on this side" does not have an effect for backups.
-
- Influencer
- Posts: 13
- Liked: never
- Joined: May 20, 2009 5:59 pm
Re: Need feedback on this migration approach please
Thanks but is it actually trying to connect to 192.168.250.2 (nowhere have I given it that IP, the backup console has seemingly passed this to the target host). I need to be using the public IP.ssimakov wrote:In this configuration your target host should be able to connect to your backup console on TCP ports used for data transfer (starting from 2500). "Run server on this side" does not have an effect for backups.
-
- Influencer
- Posts: 13
- Liked: never
- Joined: May 20, 2009 5:59 pm
Re: Need feedback on this migration approach please
My datacenter host will hopefully be opening the 2500+ ports for me shortly (they weren't open), but I'm still wondering if I'm dealing with a NAT issue here, considering it is indicating the private 192.168.250.2 IP. Nothing to worry about?itgroove wrote: Thanks but is it actually trying to connect to 192.168.250.2 (nowhere have I given it that IP, the backup console has seemingly passed this to the target host). I need to be using the public IP.
-
- Influencer
- Posts: 13
- Liked: never
- Joined: May 20, 2009 5:59 pm
Re: Need feedback on this migration approach please
Ok, the ports are opened but no improvement. If I'm reading this correctly, my VEEAM Console server should be listening on 2501, etc. However, regardless of the firewall, I can't get an open connection on that port anyways, on the console? I have no errors to go on other than this from the failed backup attempt...itgroove wrote: My datacenter host will hopefully be opening the 2500+ ports for me shortly (they weren't open), but I'm still wondering if I'm dealing with a NAT issue here, considering it is indicating the private 192.168.250.2 IP. Nothing to worry about?
Backing file "nfc://conn
:74.54.57.2
,nfchost:ha
-host,stg:4
8a28056-3cb
805ba-9117-
001ec9541b1
a@rgus1vmg0
2/rgus1vmg0
2.vmx" Agen
ts failed to start, server "localhost"
, client "hidden_IP" Cannot
connect to server [192.168.25
0.2:2501].
-
- Influencer
- Posts: 13
- Liked: never
- Joined: May 20, 2009 5:59 pm
Re: Need feedback on this migration approach please
Update: Looks like the NAT may not be right, I'll report back shortly. But if anyone has any comments/suggestions, anything is appreciated.
-
- Influencer
- Posts: 21
- Liked: never
- Joined: May 14, 2009 10:55 am
- Full Name: Alex Seesing
- Contact:
Re: Need feedback on this migration approach please
You might consider to use a ordinairy VPN connection between the two sites to overcome the NAT or ports problem.
Cheers
Cheers
----------------------
Excerpt every option. Despite every effort.
Excerpt every option. Despite every effort.
-
- Influencer
- Posts: 13
- Liked: never
- Joined: May 20, 2009 5:59 pm
Re: Need feedback on this migration approach please
And I was doing so well... We got one replica over (got the NAT problem solved) but now the second virtual machine is dying pretty much as soon as it starts with the following...
CreateSnapshot failed, vmRef 496, timeout 1800000, snName "VEEAM BACKUP TEMPORARY SNAPSHOT" snDescription "Please do not delete this snapshot. It is being used by Veeam Backup.", memory False, quiesce True
Creating a quiesced snapshot failed because the (user-supplied) custom pre-freeze script in the virtual machine exited with a non-zero return code.
Any thoughts? The Two VM's were built at the same time, have the same OS, not sure what pre-freeze script it is referring to?
CreateSnapshot failed, vmRef 496, timeout 1800000, snName "VEEAM BACKUP TEMPORARY SNAPSHOT" snDescription "Please do not delete this snapshot. It is being used by Veeam Backup.", memory False, quiesce True
Creating a quiesced snapshot failed because the (user-supplied) custom pre-freeze script in the virtual machine exited with a non-zero return code.
Any thoughts? The Two VM's were built at the same time, have the same OS, not sure what pre-freeze script it is referring to?
-
- Influencer
- Posts: 13
- Liked: never
- Joined: May 20, 2009 5:59 pm
Re: Need feedback on this migration approach please
Nevermind, I did some extra cruising in the forum and the common thread seemed to be to try VSS (can't, can't open all those ports in this environment) or a reboot of the guest. Sure enough, after fighting the guest a little (its an Exchange 2007 box, that is a DC/GC and its partner is no longer nearby), I rebooted it and was able to launch it - seems to be going now.
-
- Influencer
- Posts: 13
- Liked: never
- Joined: May 20, 2009 5:59 pm
Re: Need feedback on this migration approach please
Ok, new problem. The target network had to have the firewall rebooted. I went into VEEAM and tried to stop the job (as it stopped showing any change in progress, even after 2 hours).
But, it is not stopping, it is not doing anything besides warning me about clicking stop but continue to show the non-progress.
Can I safely stop the VEEAM service in order to launch the replication to start up again? I'd hate to lose the 6 days of replication process I started... Does/will it just safely pick up?
But, it is not stopping, it is not doing anything besides warning me about clicking stop but continue to show the non-progress.
Can I safely stop the VEEAM service in order to launch the replication to start up again? I'd hate to lose the 6 days of replication process I started... Does/will it just safely pick up?
-
- Expert
- Posts: 155
- Liked: 40 times
- Joined: Jan 01, 2006 1:01 am
- Contact:
Re: Need feedback on this migration approach please
itgroove;
Can you please open a case with our support to help you troubleshoot your current situation? (support(at)veeam.com) If you restart the service, the job will error out, but you should be able to restart it when ready. We can also provide instructions via support that can cancel this job also via a script.
Thank you
Can you please open a case with our support to help you troubleshoot your current situation? (support(at)veeam.com) If you restart the service, the job will error out, but you should be able to restart it when ready. We can also provide instructions via support that can cancel this job also via a script.
Thank you
-
- Influencer
- Posts: 13
- Liked: never
- Joined: May 20, 2009 5:59 pm
Re: Need feedback on this migration approach please
Hi there, I have an existing ticket - 508399, can we just use that with this history?Ben Milligan wrote:itgroove;
Can you please open a case with our support to help you troubleshoot your current situation? (support(at)veeam.com) If you restart the service, the job will error out, but you should be able to restart it when ready. We can also provide instructions via support that can cancel this job also via a script.
Thank you
I'd be happy to do it by script, if that is safer...
-
- Chief Product Officer
- Posts: 31806
- Liked: 7299 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Need feedback on this migration approach please
If the job did not finish initial synchronization, then no - it will not pick up - you will have to do full sync from the beginning again. As for job stopping, I would wait for more - it may be removing snapshot from source VM, this may take looong time because it was running for 6 days and snapshot can be quite large.
If you don't see snashot on source VM with VIC, then I would just kill Veeam processes on you Veeam server. Since your source is ESXi, the agent runs on Veeam Backup server. The other agent runs on target ESX, just make sure it no longer runs (or reboot ESX host). Then, delete all incomplete replica files, and start the job over again.
If you don't see snashot on source VM with VIC, then I would just kill Veeam processes on you Veeam server. Since your source is ESXi, the agent runs on Veeam Backup server. The other agent runs on target ESX, just make sure it no longer runs (or reboot ESX host). Then, delete all incomplete replica files, and start the job over again.
-
- Chief Product Officer
- Posts: 31806
- Liked: 7299 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Need feedback on this migration approach please
Did not see Ben's reply.
-
- Expert
- Posts: 155
- Liked: 40 times
- Joined: Jan 01, 2006 1:01 am
- Contact:
Re: Need feedback on this migration approach please
itgroove;
Great! I'll take a look at your case and advise you there. Expect a mail shortly.
Thanks!
Great! I'll take a look at your case and advise you there. Expect a mail shortly.
Thanks!
-
- Influencer
- Posts: 13
- Liked: never
- Joined: May 20, 2009 5:59 pm
Re: Need feedback on this migration approach please
Oh crap. I saw the other note suggesting to stop the job, so I tried it. but it is still shows as running (but no change whatsoever in progress). I guess this means the last 5 days of progress is lost? *SIGH*. It still shows replication as 'running' - I don't know what to do next?Ben Milligan wrote:itgroove;
Great! I'll take a look at your case and advise you there. Expect a mail shortly.
Thanks!
I'm confused about the one line there from Gostev. Am I whacking the only thing or just one part of the replica? Any chance someone can look at this with me via a GotoAssist session (I'm happy to provide). I'd love to know where to go next but don't want to kill the service is there is a chance of saving this replica.
Or I can be reached at 250-220-4575
-
- Chief Product Officer
- Posts: 31806
- Liked: 7299 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Need feedback on this migration approach please
No chance to save replica if full sync was not completed
-
- Veeam Software
- Posts: 268
- Liked: 63 times
- Joined: Jan 01, 2006 1:01 am
- Full Name: Stanislav Simakov
- Contact:
Re: Need feedback on this migration approach please
Please run Task Manager and verify if there are any Veeam processes running. If there are no, then actually replication already stopped but the UI does not show the status correctly. To obtain the script to reset jobs status please open a case with support.
Who is online
Users browsing this forum: 00ricbjo, mbrzezinski and 204 guests