Re-ip at DR Site

[homerjnick]

So I was testing replication to our DR site today using a seed and it all went fine.

I included in the options to re-ip the VM since it is a Windows VM and naturally my DR site is on a different subnet.

We were sold Veeam as a DR package with one factor being that we could power up replicated VM's and "Veeam would handle network translation".

So I have server on 172.28.20.xxx and replicate it to my DR site which is on 172.28.36.xxx. I failed over the test server and what I was expecting was to ping 172.28.20.xxx and get a response but there would be network translation meaning that, in theory, the 172.28.36.xxx replicated VM would be responding to the ping.

But no the 172.28.20.xxx VM is dead the replica VM was fine on my 172.28.36.xxx subnet but that is not what we want.

That means replicating Exchange, domain controllers and other IP specific and reliant servers to a DR site and getting them up and running is very messy. I would need to do lots of DNS changes, firewall changes etc...

So is this right? How do others handle this?

[dellock6]

Hi Nick,
re-ip means exaclty what you are seeing in your test: every windows VM gets its IP address changed according to the new subnet mask, so it's still reachable even when being powered in the DR site; and this is how most DR solutions work, if there is some routing to be crossed between production and DR.

What you would like to have is a stretched LAN, where servers in different places have all the same IP subnets, regarding which side of the router they are; but is something you can realize with network activities, not by Veeam. If there is routing in the middle, you need to re-ip the servers.
What happens if you deploy a new VM in the DR site with a 172.28.20.XXX ip? Can you reach it? No because is an IP "inside" your subnet, and your computer calls it without using the network gateway...

Many vendors are submitting some sort of Layer2 encapsulation over Layer3 (Cisco OTV, VMware VXLAN and others), but is not something you can "fix" inside Veeam, which works at a higher OSI level.

You can adjust your network by modyfing DNS entries during a failover so you can reach the servers with their new IPs, but this is out of the scope of Veeam and other DR solutions.

[homerjnick]

Thanks Luca...I suspected that was the answer...I can modify firewall rules and DNS changes etc but I'm sure this will cause issues with domain controllers and other servers...I would be interested to know what other Veeam customers do...

[tsightler]

If you're environment is configured for Dynamic DNS and makes proper use of aliases then failing over with different IP addresses is easy. When the replica's power on with their new IP's they simply register their new addresses in DNS. This is especially easy if using Microsoft DNS with AD integration, simply make sure that you have a DC running at the DR site that is also a secondary DNS server.

That being said, if you simply want to keep the same IP's at the remote site then you will need to make some type of network changes to accommodate this. Options are streched layer 2 networks as Luca suggested (or using layer 2 VPN solutions like Tinc), or routing changes during failover. The biggest problem with making routing changes is that it doesn't easily give you the ability to failover individual servers, which is actually the far more likely failure scenario.

[dellock6]

Totally agree, DC services (all with dns onboard) can be placed in both Prod and DR site so you do not even need to failover.
When failed-over servers power up in DR they will only find the DCs locally to the DR itself so they will register there and update DNS records.

Tom, what about Tinc? I never heard about it, do you have some link do learn something more about it? Thanks

[tsightler]

Tinc is an open source Layer 2 VPN that is designed to build fully meshed layer-2 networks while requiring only a single TCP or UDP port connection. It is similar in concept to OpenVPN in that it basically uses SSL to build a layer-2 VPN (which you can then of course run layer-3 over based on if you using a bridge or routed connection). The different is that OpenVPN is peer-to-peer or hub-spoke, and requires complex routing configuration if you have many nodes and many possible disjointed networks.

With mesh VPNs you can build our your VPN in any topology that makes since and, as long as there is some connections between members of the mesh, then all nodes will always be able to see all other nodes. On top of this you can then choose to build a complex, stretch layer-2 network, or use layer-3 networks and any type of routing protocol such as BGP or OSPF to create an exceptionally resilient VPN network.

There are actually several similar projects, but I fell in love with TINC because it uses a simple to configure private key authentication, a single port, and has a very simple configuration.

http://www.tinc-vpn.org/

Other options with similar capabilities are CloudVPN

http://dev.e-x-a.org/projects/cloudvpn/wiki

I used to have a blog post on my old blog that discussed how I used Linux and Tinc to build a "backup WAN" for my old company. We could lose our entire MPLS provider, and even our physical routers, and network connectivity would simply fail over to the Tinc VPN. I also did some testing with stretching our layer-2 to our remote DR site 700 miles away. I build my own linux appliance that would run tinc, and ran two on each side of the link for a cluster and some scripts to perform some "fancy" proxy ARPs to get traffic to route via the gateway that I wanted based on which physical site the VM was actually located. It worked amazingly well for something I just threw together to play with.

[dellock6]

Really interesting.
I was going to ask you about the ARP tables management, having the same subnet at both ends, but you already gave my an asnwer
Thanks, I will try it the next time I will have a request for a stretched lan.

[rb51]

Hi guys,

Absolutely noob here....

We will be using veeam6 to replicate our VMs to our Provider DR site. Firstly we are placing a DC at the DR site + have already configured proxy VMs at both ends. Basically we will have 2 different subnets. So:

VM1: 192.168.1.10 prod site failsover to VM1_replica: 192.168.100.10 at DR site

What happens to other entries on DNS that point to 192.168.1.10???? Do we need to manually populate DNS with entries to 192.168.100.10 or veeam will take care of it?

Comments/suggestions are much appreciated...

rb51

[dellock6]

If they are all windows VMs, Veeam re-ip will take care of the setting changes. But if it's about dns, you can for sure configure both dns servers in every server on both sites, DNS traffic is really light so it does not have an impact on the wan link; also, in Active Directory you can work with "sites" and thus separate the two DCs in two different sites; in this way at least windows machins joined to the domain will use their local DNS.

[J1mbo]

IMO a layer-2 link is really much more preferable. There are various ways of achieving that (one of which has been described above, another of which is to effectively run two VLANs on your WAN and use one as a layer-2 bridge). Some care needs to be exercised in routing at the remote site in considering the failover scenarios though (primary site surviving, or not).

Besides the simplicity, not all applications can tolerate IP renumber easily (Oracle, for example).

[rb51]

If they are all windows VMs, Veeam re-ip will take care of the setting changes. But if it's about dns, you can for sure configure both dns servers in every server on both sites, DNS traffic is really light so it does not have an impact on the wan link; also, in Active Directory you can work with "sites" and thus separate the two DCs in two different sites; in this way at least windows machins joined to the domain will use their local DNS.

Ciao Luca,

Yes the DC will be set in AD as "offsite".

Still a bit unclear about your dns answer. My question is that we have in our main site DNS
VM1: 192.168.1.10 plus A records pointing to the same IP, e.g.,
Record1: 192.168.1.10
Record2: 192.168.1.10
Record3: 192.168.1.10 etc

When we failover to the DR site Veeam will Re-IP VM1 to VM1_replica 192.168.100.10 and register with the local DC that is fine. However what about the other A records??? Do we need to manually update them to point to 192.168.100.10???? I believe we do....

Comments are appreciated.....

[tkrajewski]

Maybe using CNAME instead of A record would be better to point different FQDNs to the same host? Then you would have

VM1.domain.com A 192.168.1.10
Record1.domain.com CNAME VM1.domain.com
Record2.domain.com CNAME VM1.domain.com
etc

That way, in DR site, VM after startup will record it's VM1.domain.com to new IP and all other records will point to VM1.domain.com that is, in effect, that new IP. A little bit of redesign (if it's possible) but you have to do it once. Does it make sense?

[dellock6]

Absolutely, I was going to suggest the same solution, thanks Tomasz!

Btw: large use of cname is a good practice even without re-ip or DR, usually a single host would have to be limited to only 1 A record, and have all other records pointing to it configured as cname. This is also described in dns RFC (something almost nobody reads... )

[Alan_ORiordan]

I wanted to share my first offsite replication results with you and clarify one point about DNS registration:

I replicated a base build Windows Server 2008 R2 VM with a 40GB VM file (16.4GB) used With a 10Mbit link and Enterprise license (no WAN accelerator) this took 4hrs 14 mins.

The replication job was configured from the DR site's Veeam console with direct links to the VM Hosts so as to get over the Vcenter issue.

I then tried the replica with continuous schedule and was happy to see the process continued with very small/fast updates of approx 2:30 mins.

To fail-over this replica I shut down the LIVE VM simulating the failure

The fail-over, network mapping and Re-IP completed quickly and successfully with DNS updating on the DR site.

My main question is about DNS on the LIVE site, the servers source A record was removed but it was not re-created with the DR IP address even after waiting for a couple of AD replication cycles.

Is this by design somehow? I am curious to know if that is expected behaviour?

After undoing failover all returned to normal, the replica was powered down by Veeam and DNS was sorted out at the LIVE (source) site and DR site. I had to power back on the source VM presumably because that was the state it was in when it was failed over.

Overall happy with the results just would like to complete my understanding regarding the DNS behaviour?

Now on with seeding the REAL VM's to the DR site and getting ready for the real DR test at the end of the month!

[foggy]

Alan, you should configure DNS on failover manually, Veeam B&R just sets the corresponding IPs and does not touch DNS.

[Alan_ORiordan]

Alexander (foggy)

Thanks, but then why was the DNS A record removed from Live site DNS and added to DR site DNS without my input?

Also why did the failed over DNS A record not replicate back to Live if it was in the DR DC's AD integrated DNS forward lookup file?

Alan

[Vitaliy S.]

Hi Alan,

Veeam does not "touch" source VMs, and cannot change DNS or IP settings.

After undoing failover all returned to normal, the replica was powered down by Veeam and DNS was sorted out at the LIVE (source) site and DR site. I had to power back on the source VM presumably because that was the state it was in when it was failed over.
....
Also why did the failed over DNS A record not replicate back to Live if it was in the DR DC's AD integrated DNS forward lookup file?

Did you use failback or undo-failover option in the Veeam backup console?

Thanks!

[Alan_ORiordan]

I used undo failover on this occasion. I will test failback to production next. I need get a feel for how long it will take to failback production machines following our first DR test using Veeam over a 10MBit link.

I can understand that the server would register its Re-IP'd settings on it's boot up at DR but not why that change would then not be replicated back to other sites DNS servers just removed? Perhaps that is an issue for my own separate research, it's just when you said i should "configure DNS on failover manually" that had me wondering what you meant, I understood that as manually re-creating new A records for failed over machines which could get messy if not careful.

From reading other posts in this thread I also wonder what happens with failed over Domain Controllers and Exchange servers which will have other A records (Autodiscover, Mailgate) and SRV records etc. I can see that you would change the DC's AD Site should the failover be permanent otherwise AD replication may not work properly during a temporary failover. Are there any articles that discuss DNS specifically during failover for DC's etc.

Thank you for your commitment to answering forum posts, it's a big help.

[Vitaliy S.]

I used undo failover on this occasion. I will test failback to production next. I need get a feel for how long it will take to failback production machines following our first DR test using Veeam over a 10MBit link.

Yes, you should be using failback if you want changes to be replicated back from the DR site to the production environment.

Take a look at our on-line help center for more details > http://helpcenter.veeam.com/backup/80/v ... lback.html

I can understand that the server would register its Re-IP'd settings on it's boot up at DR but not why that change would then not be replicated back to other sites DNS servers just removed? Perhaps that is an issue for my own separate research, it's just when you said i should "configure DNS on failover manually" that had me wondering what you meant, I understood that as manually re-creating new A records for failed over machines which could get messy if not careful.

Yes, I'm also wondering why you have these records removed. You can use two addresses for all your production VMs, and in this case doing the failover will not change the DNS server records and your VMs will be accessible via alternative address.

[BHB3805]

I also have experienced where DNS records are removed from the production environment after powering on a replica virtual machine in an isolated network. Has me puzzled. Anyone else seeing this issue? Any resolution?

[rstuart]

Can you clarify what you mean by "you can use two addresses for all your production VMs"?

Do you mean adding a second static DNS record for the each VM? For instance, server1.domain.local 192.168.10.40(Prod site) and server1.domain.local 192.168.20.40(DR site)?

[DGrinev]

Hi Robert and welcome to the community!

Do you mean adding a second static DNS record for the each VM?

Yes, that's correct or you can manually configure DNS after the failover. Thanks!