Comprehensive data protection for all workloads
Post Reply
itgroove
Influencer
Posts: 13
Liked: never
Joined: May 20, 2009 5:59 pm

Need feedback on this migration approach please

Post by itgroove » May 25, 2009 5:10 pm

Hey folks,
Ok, here is my scenario. :idea:


Source Network/Details:
-- 2 Windows Server 2008/64bit Virtual Servers at a Datacenter in the US
-------- 1 Guest is Exchange Server 2007 with 3 large databases around 20GB each (total size of data is around 110GB)
-------- 1 Guests is Windows SharePoint Services 3.0 (with WID/SQL database) – data is about 30GB
-- VMWare host is ESXi (no VCB)
-- Internet throughput is decent

Destination Network/Details:
-- VMWare host is ESXi (but could be ESX and VCB if necessary)
-- Internet up is 3MB, down is about 6MB

The Reason for Asking:
I no longer trust the hosting facility/platform. We’ve had reason to want to move for some time due to various issues, one of which is that we can’t ‘physically’ *touch* these servers and we are at the mercy of the datacenter response time when problems occur. I need these servers running locally

The Challenges:
-- I only have a 24 hour window in which these servers can be *down* to make this move happen

The Solution – Proposed but I’m seeking feedback and expertise):
-- Implement VEEAM Backup and Replication and run the console from the destination host
-- Perform a full replication of the two Virtual guests (Exchange Server and SharePoint Services Server) – let this take as long as it takes – 5 days, 2 weeks, whatever – just get the first replica
-- Once the first replica finishes, target a weekend to complete a ‘follow-up snapshot’ of changes
-- Once the follow-up snapshot completes, re-point DNS for all clients to go to the new systems, at the destination location

Questions/Concerns
-- Am I nuts? Over two weeks, Exchange Databases would be modified quite a bit, assuming I’m depending on VSS (I am right?), it should just be changes to the data and not complete re-copies of the databases, right?

Any and all comments, feedback, guidance and warnings appreciated.

Gostev
SVP, Product Management
Posts: 25319
Liked: 3767 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Need feedback on this migration approach please

Post by Gostev » May 25, 2009 6:09 pm

Hello, your plan seems fine to me. You would definitely need "fat" ESX in your target environment though, because the current version can only work with "fat" ESX as replication target, and not ESXi. VCB is not needed on target side.

Based on provided network speed (6MB down = 480Mb), first replica should complete in under 8 hours, and subsequent passes will probably take about 1 hour @ full link speed (during off-hours).

I think best would be to do more than 2 synchronizations though, here's how I would do this (from the top of my head):
1. Perform initial synchrhonization.
2. Perform incremental syncrhonization as soon as initial finishes (to apply changes happened during long initial sync). That would only sync changed blocks.
3. Test replica to make sure it starts up fine. Make sure replica is isolated from production in term of networking, use Veeam Backup to "Failover" to it (don't start it manually!). Exchange may throw errors because unable to contact DNS/DC, but as long as VM starts up, all is well. After that, be sure to perform "Undo failover" from Veeam Backup, this would discard test failover snapshot and your replication job will be able to continue to write updates to existing unchanged VMDK.
3. Run daily incremental synchronizations until day X and monitor how much time do they take (so that it does not become a surprise for you later).
4. On day X, stop SMTP service on your source Exchange to prevent incoming email from coming in, and prevent user access to Exchange (don't remember how it is best to do this).
5. Run final incremental replication.
6. Shut down source Exchange.
7. Perform "Failover" to replica in Veeam Backup, wait until it starts up.
8. Make all required DNS changes, enable user access, test Exchange connectivity and do all those Exchange admin things to make sure all is well.
9. If all OK, enable SMTP service for incoming email to start coming in.
10. When you are confident that all is well, remove snapshot created be Veeam Backup during Failover operation by simply deleting it with VIC (removing snapshot would merge all snapshot data into VMDK). You can also delete all *.VRB files found next to VMDK at this time.

Hope this helps. :D

itgroove
Influencer
Posts: 13
Liked: never
Joined: May 20, 2009 5:59 pm

Re: Need feedback on this migration approach please

Post by itgroove » May 25, 2009 7:00 pm

Thanks Gostev,

So to be clear I understand you.

1. It is fine that the source (old VM, at US datacenter) is running ESXi (does anything get 'installed' there?)
2. The destination (new kit) must be running full ESX (fine - do I need VCB or just full ESX?)

As far as within the Windows guests, are there any other requirements? Does anything get installed within them and/or do I need VSS enabled/configured or is this all happening at the ESX level?

Thanks again.

Gostev
SVP, Product Management
Posts: 25319
Liked: 3767 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Need feedback on this migration approach please

Post by Gostev » May 25, 2009 8:11 pm

1. Yes - ESXi is fine as a source, nothing needs to be installed there. But you want to install Veeam Backup at source site, because of data flow considerations. This is explained in more details here, see my post with different font colors (since you are using ESXi as source, this is exactly your case).

2. Just full ESX, VCB is not needed (VCB can only be used to retrieve VM data, it does not have functionality to write/restore VM data anyway).

Windows guest doesn't need anything installed, small Veeam VSS agent will be put there as a part of snapshot process automatically, and will be removed snapshot is created. You just need to make sure to enable VSS option in Replica Job wizard, and provide valid admin credentials for the guest.

itgroove
Influencer
Posts: 13
Liked: never
Joined: May 20, 2009 5:59 pm

Re: Need feedback on this migration approach please

Post by itgroove » May 25, 2009 9:41 pm

awesome, thanks for the detailed response.

itgroove
Influencer
Posts: 13
Liked: never
Joined: May 20, 2009 5:59 pm

Re: Need feedback on this migration approach please

Post by itgroove » Jun 07, 2009 8:47 pm

Ok, I've attempted to kick this thing off but I'm stuck at the moment...

My interpretation of the problem is that NAT seems to be the issue here. When I try to launch the backup, it goes for about 5 minutes and then ends with this... (note, I've replaced the Public IP's with x.x.x.x for the source, y.y.y.y for the target and z.z.z.z for the VM guest running VEEAM to protect the innocent :)

1 of 15 files processed

Total VM size: 260.00 GB
Processed size: 0.00 KB
Avg. performance rate: 0 KB/s
Backup mode: agentless

Start time: 6/7/2009 12:40:26 PM
End time: 6/7/2009 12:46:26 PM

Backing file "nfc://conn:z.z.z.z,nfchost:ha-host,stg:48a28056-3cb805ba-9117-001ec9541b1a@XXus1vmg02/XXus1vmg02.vmx"
Agents failed to start, server "localhost", client "y.y.y.y"
Cannot connect to server [192.168.250.2:2501].

Note, that I was able to add the source (ESXi) and target (ESX) Servers into the VEEAM console with no problems, including specifying Service Console/SSH settings - I was able to see both datastores, etc.

The IP 192.168.250.2 is the LAN IP for the VEEAM Server (which of course the source or target can't see, by that IP). I've tried the "Run Server on this Side" option, checked and unchecked as Troubleshooting seems to imply it may help in a NAT situation, though there was no effect here.

What am I missing? Do I need to have a VPN in place between the two?
Should I be specifying FQDN's instead of the IP perhaps, so I can override what the destination is?


Help!

ssimakov
Veeam Software
Posts: 352
Liked: 52 times
Joined: Jan 01, 2006 1:01 am
Full Name: Stanislav Simakov
Contact:

Re: Need feedback on this migration approach please

Post by ssimakov » Jun 07, 2009 9:02 pm

In this configuration your target host should be able to connect to your backup console on TCP ports used for data transfer (starting from 2500). "Run server on this side" does not have an effect for backups.

itgroove
Influencer
Posts: 13
Liked: never
Joined: May 20, 2009 5:59 pm

Re: Need feedback on this migration approach please

Post by itgroove » Jun 07, 2009 9:15 pm

ssimakov wrote:In this configuration your target host should be able to connect to your backup console on TCP ports used for data transfer (starting from 2500). "Run server on this side" does not have an effect for backups.
Thanks but is it actually trying to connect to 192.168.250.2 (nowhere have I given it that IP, the backup console has seemingly passed this to the target host). I need to be using the public IP.

itgroove
Influencer
Posts: 13
Liked: never
Joined: May 20, 2009 5:59 pm

Re: Need feedback on this migration approach please

Post by itgroove » Jun 07, 2009 9:16 pm

itgroove wrote: Thanks but is it actually trying to connect to 192.168.250.2 (nowhere have I given it that IP, the backup console has seemingly passed this to the target host). I need to be using the public IP.
My datacenter host will hopefully be opening the 2500+ ports for me shortly (they weren't open), but I'm still wondering if I'm dealing with a NAT issue here, considering it is indicating the private 192.168.250.2 IP. Nothing to worry about?

itgroove
Influencer
Posts: 13
Liked: never
Joined: May 20, 2009 5:59 pm

Re: Need feedback on this migration approach please

Post by itgroove » Jun 08, 2009 1:49 am

itgroove wrote: My datacenter host will hopefully be opening the 2500+ ports for me shortly (they weren't open), but I'm still wondering if I'm dealing with a NAT issue here, considering it is indicating the private 192.168.250.2 IP. Nothing to worry about?
Ok, the ports are opened but no improvement. If I'm reading this correctly, my VEEAM Console server should be listening on 2501, etc. However, regardless of the firewall, I can't get an open connection on that port anyways, on the console? I have no errors to go on other than this from the failed backup attempt...

Backing file "nfc://conn
:74.54.57.2
,nfchost:ha
-host,stg:4
8a28056-3cb
805ba-9117-
001ec9541b1
a@rgus1vmg0
2/rgus1vmg0
2.vmx" Agen
ts failed to start, server "localhost"
, client "hidden_IP" Cannot
connect to server [192.168.25
0.2:2501].

itgroove
Influencer
Posts: 13
Liked: never
Joined: May 20, 2009 5:59 pm

Re: Need feedback on this migration approach please

Post by itgroove » Jun 08, 2009 4:40 am

Update: Looks like the NAT may not be right, I'll report back shortly. But if anyone has any comments/suggestions, anything is appreciated.

aseesing
Influencer
Posts: 21
Liked: never
Joined: May 14, 2009 10:55 am
Full Name: Alex Seesing
Contact:

Re: Need feedback on this migration approach please

Post by aseesing » Jun 08, 2009 7:45 am

You might consider to use a ordinairy VPN connection between the two sites to overcome the NAT or ports problem.

Cheers
----------------------
Excerpt every option. Despite every effort.

itgroove
Influencer
Posts: 13
Liked: never
Joined: May 20, 2009 5:59 pm

Re: Need feedback on this migration approach please

Post by itgroove » Jun 16, 2009 2:10 am

And I was doing so well... We got one replica over (got the NAT problem solved) but now the second virtual machine is dying pretty much as soon as it starts with the following...

CreateSnapshot failed, vmRef 496, timeout 1800000, snName "VEEAM BACKUP TEMPORARY SNAPSHOT" snDescription "Please do not delete this snapshot. It is being used by Veeam Backup.", memory False, quiesce True

Creating a quiesced snapshot failed because the (user-supplied) custom pre-freeze script in the virtual machine exited with a non-zero return code.

Any thoughts? The Two VM's were built at the same time, have the same OS, not sure what pre-freeze script it is referring to?

itgroove
Influencer
Posts: 13
Liked: never
Joined: May 20, 2009 5:59 pm

Re: Need feedback on this migration approach please

Post by itgroove » Jun 16, 2009 6:07 am

Nevermind, I did some extra cruising in the forum and the common thread seemed to be to try VSS (can't, can't open all those ports in this environment) or a reboot of the guest. Sure enough, after fighting the guest a little (its an Exchange 2007 box, that is a DC/GC and its partner is no longer nearby), I rebooted it and was able to launch it - seems to be going now.

itgroove
Influencer
Posts: 13
Liked: never
Joined: May 20, 2009 5:59 pm

Re: Need feedback on this migration approach please

Post by itgroove » Jun 19, 2009 7:56 pm

Ok, new problem. The target network had to have the firewall rebooted. I went into VEEAM and tried to stop the job (as it stopped showing any change in progress, even after 2 hours).

But, it is not stopping, it is not doing anything besides warning me about clicking stop but continue to show the non-progress.

Can I safely stop the VEEAM service in order to launch the replication to start up again? I'd hate to lose the 6 days of replication process I started... Does/will it just safely pick up?

Ben Milligan
Expert
Posts: 173
Liked: 40 times
Joined: Jan 01, 2006 1:01 am
Contact:

Re: Need feedback on this migration approach please

Post by Ben Milligan » Jun 19, 2009 8:06 pm

itgroove;

Can you please open a case with our support to help you troubleshoot your current situation? (support(at)veeam.com) If you restart the service, the job will error out, but you should be able to restart it when ready. We can also provide instructions via support that can cancel this job also via a script.

Thank you

itgroove
Influencer
Posts: 13
Liked: never
Joined: May 20, 2009 5:59 pm

Re: Need feedback on this migration approach please

Post by itgroove » Jun 19, 2009 8:11 pm

Ben Milligan wrote:itgroove;

Can you please open a case with our support to help you troubleshoot your current situation? (support(at)veeam.com) If you restart the service, the job will error out, but you should be able to restart it when ready. We can also provide instructions via support that can cancel this job also via a script.

Thank you
Hi there, I have an existing ticket - 508399, can we just use that with this history?

I'd be happy to do it by script, if that is safer...

Gostev
SVP, Product Management
Posts: 25319
Liked: 3767 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Need feedback on this migration approach please

Post by Gostev » Jun 19, 2009 8:15 pm

If the job did not finish initial synchronization, then no - it will not pick up - you will have to do full sync from the beginning again. As for job stopping, I would wait for more - it may be removing snapshot from source VM, this may take looong time because it was running for 6 days and snapshot can be quite large.

If you don't see snashot on source VM with VIC, then I would just kill Veeam processes on you Veeam server. Since your source is ESXi, the agent runs on Veeam Backup server. The other agent runs on target ESX, just make sure it no longer runs (or reboot ESX host). Then, delete all incomplete replica files, and start the job over again.

Gostev
SVP, Product Management
Posts: 25319
Liked: 3767 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Need feedback on this migration approach please

Post by Gostev » Jun 19, 2009 8:16 pm

Did not see Ben's reply.

Ben Milligan
Expert
Posts: 173
Liked: 40 times
Joined: Jan 01, 2006 1:01 am
Contact:

Re: Need feedback on this migration approach please

Post by Ben Milligan » Jun 19, 2009 8:17 pm

itgroove;

Great! I'll take a look at your case and advise you there. Expect a mail shortly.

Thanks!

itgroove
Influencer
Posts: 13
Liked: never
Joined: May 20, 2009 5:59 pm

Re: Need feedback on this migration approach please

Post by itgroove » Jun 19, 2009 8:28 pm

Ben Milligan wrote:itgroove;

Great! I'll take a look at your case and advise you there. Expect a mail shortly.

Thanks!
Oh crap. I saw the other note suggesting to stop the job, so I tried it. but it is still shows as running (but no change whatsoever in progress). I guess this means the last 5 days of progress is lost? *SIGH*. It still shows replication as 'running' - I don't know what to do next?

I'm confused about the one line there from Gostev. Am I whacking the only thing or just one part of the replica? Any chance someone can look at this with me via a GotoAssist session (I'm happy to provide). I'd love to know where to go next but don't want to kill the service is there is a chance of saving this replica.

Or I can be reached at 250-220-4575

Gostev
SVP, Product Management
Posts: 25319
Liked: 3767 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Need feedback on this migration approach please

Post by Gostev » Jun 19, 2009 8:32 pm

No chance to save replica if full sync was not completed :(

ssimakov
Veeam Software
Posts: 352
Liked: 52 times
Joined: Jan 01, 2006 1:01 am
Full Name: Stanislav Simakov
Contact:

Re: Need feedback on this migration approach please

Post by ssimakov » Jun 22, 2009 3:52 pm

Please run Task Manager and verify if there are any Veeam processes running. If there are no, then actually replication already stopped but the UI does not show the status correctly. To obtain the script to reset jobs status please open a case with support.

Post Reply

Who is online

Users browsing this forum: bavo.vdk, Bing [Bot], conrad.stephens, Google [Bot], ZhenYa and 95 guests