SureBackup Design Question

[vertices]

So we have been using VBR for a while now. Getting around to setting up SureBackup.

We have 2 sites, with a tunnel between, vSphere infrastructure in both, a beefy physical server for Veeam in each. The southern site is a full VBR server, the northern is a backup proxy and repository.
Now we used to have 2 full VBR servers in order to do backups at the northern site, but pull replicas to the southern. Then Veeam came out with proxies and repositories and it was great. We rebuilt around this architecture and it’s been serving us great. The VBR in the southern site controlled all backup and replication jobs, running backups up north, and pulling replicas to the south, as well as running some backups in the south.

Now we get to SureBackup. We have this wonderful Virtual Lab. One of its interface IPs is the same as the production gateway. This obviously causes issues with routing outside its subnet. We added routes and everything manually to the routers on both sides for the masquerade subnet, but I really didn’t see how this would work at all as this would require the virtual lab to route to the production gateway, which is the exact same IP assigned to one of its interfaces.

So we call up support and sure enough, they say that your Virtual Lab must be in the same subnet as the VBR server if you want to run any tests beyond vmware tools heartbeat. This is absolutely bizarre to me. I mean Veeam puts all this effort into proxies and repositories so that we can have a distributed architecture and control it nicely from a single interface. Then they just completely bypass this in their design of SureBackup.

All Veeam has to do is build the testing agents into the proxy agent. Then the proxy agents, just as they do for backups, do the work, and send the results to the VBR server. This would solve this entire problem. Support told us that if we wanted to do what we are doing we have to go back to having 2 VBR servers which to me is ludicrous.

Does anyone have an answer as to why the ping and script checks run directly from the VBR server and not from a local proxy? I just can’t fathom why this design decision was made as it effectively negates the benefits of having proxies and repositories available.

[habibalby]

Hello,
Check this out might help you ;
http://www.veeam.com/blog/category/surebackup
http://www.percula.info/?p=348

[vertices]

Thanks but all that does is reconfirm my feelings that this was a huge oversight. The job of testing (and static routes) should have been placed on an agent. Either a separate proxy or build it into the backup proxy agent.

We are left with an inability to use SureBackup fully because we followed Veeams design of using backup proxies.

If anyone has successfully accomplished running a Virtual Lab, that the VBR server can access for ping and script tests, where the Virtual Lab resides on a different network from the VBR server I'd love to hear how you accomplished this.

[foggy]

One of its interface IPs is the same as the production gateway. This obviously causes issues with routing outside its subnet.

Might be a silly question, but do you assign the production gateway IP to an internal proxy appliance interface (the one that belongs to the isolated virtual lab)? Could you please describe your network topology in more detail, providing addresses used for all the involved computers.

Also, please review this post, might be useful.

[vertices]

Might be a silly question, but do you assign the production gateway IP to an internal proxy appliance interface (the one that belongs to the isolated virtual lab)?

Yes I do. And we have 2 sites with SureBackup jobs in each. The site with the VBR server works fine, the other does not.

I’ve also seen the post. Support tells me directly that it won’t work as long as the VBR is in a different subnet from the VBR server. So not sure how much this is going to help but here goes:

Let’s use this:

VPN between 2 sites:

Site 1:
Production Gateway LAN IP: 10.1.1.1
Veeam 1: Physical full Veeam Server including proxy and repository 10.1.1.80
Virtual Lab at 10.1.1.82 with an interface configured as 10.1.1.1, Masquerade network 10.255.1.x



Site 2:
Production Gateway LAN IP: 10.1.6.1
Veeam 2: Physical Server acting as Veeam Proxy and Repository 10.1.6.80
vCenter at 10.1.6.81
Virtual Lab at 10.1.6.82 with an interface configured as 10.1.6.1, Masquerade network 10.255.2.x


Veeam 1 controls backup jobs in both sites, using the proxy as well as the repository that is local to the job. Veeam 1 also uses the proxy at Veeam 2 to pull replicas from Site 2 to Site 1. There are proper routes on the firewalls at each location. Each firewall has a route to 10.255.2.x pointing to 10.1.6.82.

Everything in Site 1 in regards to the SureBackup jobs running there works perfectly. This is expected as Veeam support says that the Virtual Lab must be in the same subnet as the full VBR server. SureBackup jobs in Site 2 fail on ping or script tests. Support tells us this is due to the VBR server in Site 1 not being able to reach the masquerade network in Site 2. Well we have routes so that isn’t the issue.

However look at the Virtual Lab in Site 2. It has an interface configured as 10.1.6.1 which is the same as the production gateway in Site 2. This is so VMs will still have access to the assigned gateway. So of course it can’t route traffic to Site 1 as that would require the Virtual Lab to route traffic to an IP that is the same as one of its interfaces.

This design makes sense and I can see why Support says it won’t work. What I am saying is that if the ping and script tests ran from a proxy, such as Veeam 2 in the above example, then the testing source would be in the same subnet as the Virtual Lab, thus negating this issue. The proxy on Veeam 2 would perform the testing to the Virtual Lab in Site 2, then send the results back to Veeam 1 in Site 1. Essentially the exact same design as how backup proxies work, but for testing instead.

Unless I am completely missing something here, it would seem that Support is correct, and the only real way to resolve this issue is for Veeam to introduce additional functionality to the proxy, or an additional agent altogether.

[vertices]

So in addition to all the other problems, Our VBR server in Site 1 can't ping the Virtual Lab appliance in Site 2 until we login to the Virtual Lab appliance in Site 2 and ping the VBR server in site 1. Then both can ping each other. So even if we had routing working perfectly and none of the problem's I've outlined above existed, we still have this very basic issue of jobs failing until someone manually logs into the virtual appliance and pings the Veeam server. Then they the jobs can progress. After some time, or after a reboot of the Virtual Appliance, it fails again until someone logs into it and pings the VBR server.

This is identical to the problem at the end of the thread that foggy listed. Note that that person didn't get an answer either.

http://forums.veeam.com/viewtopic.php?f ... ork#p56333

[rkovhaev]

Hi,
Please correct me if I got it wrong, on site1 you have VBR server with SB jobs configured to run on site1 and site2
So, in this case, on firewall on site1 you need to have 2 static routes:
10.255.1.x has to point to 10.1.1.82
10.255.2.x has to point to 10.1.6.82
and on site2 firewall must be able to route back to 10.1.1.0/24

If this is all done, then I suggest running tcpdump on VBR server and tcpdump on proxy appliance and analyzing the traffic flow.
Also please let me know your support case number.

So in addition to all the other problems, Our VBR server in Site 1 can't ping the Virtual Lab appliance in Site 2 until we login to the Virtual Lab appliance in Site 2 and ping the VBR server in site 1. Then both can ping each other. So even if we had routing working perfectly and none of the problem's I've outlined above existed, we still have this very basic issue of jobs failing until someone manually logs into the virtual appliance and pings the Veeam server. Then they the jobs can progress. After some time, or after a reboot of the Virtual Appliance, it fails again until someone logs into it and pings the VBR server.

This is identical to the problem at the end of the thread that foggy listed. Note that that person didn't get an answer either.

http://forums.veeam.com/viewtopic.php?f ... ork#p56333

This one is recently discovered issue with appliance not sending ARP/RARP requests upon initial boot. I am working on getting a new appliance.

[dood]

This one is recently discovered issue with appliance not sending ARP/RARP requests upon initial boot. I am working on getting a new appliance.

Does anyone find a solution for this issue ?
It seems i get the same one on the VBR 6.5.
(Case #00178159)

[foggy]

Seems that our support has custom appliance build for this issue. Please continue working with them to get it and test in your environment.

[dood]

Yes, support gave me a customized appliance (drv-va.iso).
When the iso file replaced, don't forget to edit the existing virtual lab and apply settings to reload this iso file

[dellock6]

Hi,
we hit the same exact issue at a customer today, once we login into the VLAB appliance and we run several pings, finally VLAB and Veeam server start pinging each other, and the surebackup jobs completes the "starting virtual lab routing engine" step.

We already opened a ticket to eventually obtain the modified iso, but there is any chance the new iso will be integrated in a future patch of 6.5, or will it be release inside v7?

Thanks,
Luca.

[foggy]

Good chances that it will be included in v7.

[glupano]

Man!!!!! I have been playing around with that for hours. Could not work out what was happening. I could ping the LAB VM from everything on my production network EXCEPT the VBR server. Logged into the lab linux console and pinged successfully from there then all of a sudden it starts responding from the VBR server. Weird

Thanks for that

[aslastenkin]

Yes, support gave me a customized appliance (drv-va.iso).
When the iso file replaced, don't forget to edit the existing virtual lab and apply settings to reload this iso file

The same issue! Unfortunately our support ended one month ago and we cannot open the case. Could you please provide custom ISO?

[foggy]

Anton, I apologize for being obvious, but you're asking for support while your support has already expired. The proper way in your situation will be to renew your maintenance and request the custom ISO via support service. Thanks.

[aslastenkin]

It is a known bug, and It would be great if you provide public access to the solution, not only for me, am I right? )

[foggy]

All product updates are available only to customers having an active support contract.

[jim3cantos]

Thanks but all that does is reconfirm my feelings that this was a huge oversight. The job of testing (and static routes) should have been placed on an agent. Either a separate proxy or build it into the backup proxy agent.

We are left with an inability to use SureBackup fully because we followed Veeams design of using backup proxies.

If anyone has successfully accomplished running a Virtual Lab, that the VBR server can access for ping and script tests, where the Virtual Lab resides on a different network from the VBR server I'd love to hear how you accomplished this.

Could this be considered as a F.R.?

thanks

[foggy]

Yes, however could you please first share what particular issue do you have, whether you've already tried suggestions mentioned in this thread above, and whether you've contacted support with it?

[jim3cantos]

Yes, however could you please first share what particular issue do you have, whether you've already tried suggestions mentioned in this thread above, and whether you've contacted support with it?

Yes, I did in another thread: http://forums.veeam.com/veeam-backup-re ... ml#p159885

[foggy]

That one looks like another issue or am I missing something?

[jim3cantos]

That one looks like another issue or am I missing something?

well, as surebackup does not seem to work between sites with different subnets and for us it is not possible to add routes to the routers, then I tried to

install a copy of Veeam at the remote site and use this instance to run the SureBackup job

as indicated here: http://forums.veeam.com/vmware-vsphere- ... tml#p78154,
but we are using replication from backup in site B and here we arrive at the issue in the other thread. I am assuming that I have to put the replication job in veeam server at DR site in order to be able to use surebackup. Am I right?

[foggy]

Thanks for explanation.

I am assuming that I have to put the replication job in veeam server at DR site in order to be able to use surebackup. Am I right?

That would complicate the replication (periodic rescan, dummy backup job), I'd leave site A backup server responsible for all the jobs and use site B backup server exclusively for SureBackup.

[jim3cantos]

Ok. In fact this is the first thing I tried but just adding the replicas from infrastructure in application group and then running a surebackup job does not seem to work:

Code: Select all

01/09/2015 11:26:49 Error    No object in backup found for VM 'SRVDA1_replica' (ID 'ac2da991-7834-432b-a880-b00bca039ed7')
01/09/2015 11:26:49 Error    No object in backup found for VM 'SRVDA1_replica' (ID 'ac2da991-7834-432b-a880-b00bca039ed7')
01/09/2015 11:26:49          Job finished

Do I have to import configuration database from backup server at site A or something else?

[foggy]

Here's the procedure that should work.

[jim3cantos]

Here's the procedure that should work.

mmmm...this is only for testing backups?

I would like to be able to test the replicas with re-IP applied at DR site.

[foggy]

Well, I'm afraid this was never tested for replicas. Then we're back to running replication jobs on the remote console, that would imply creating a dummy job mapped to the backup files created by the first backup console and repository rescan as a pre-replication job script.