Hosting the backup server offsite

[Treebeard]

We're running backups in 3 different environments linked by IPSec VPNs. Each site has multiple pairs of independent backup servers and proxies (e.g. svr-bkup01 to svr-bkup-proxy01, svr-bkup02 to svr-bkup-proxy02 etc), as well as offsite repositories which are shared. The offsite repo servers also have Veeam B&R installed as they run jobs themselves.

This setup is a bit of a pain to manage, as each pairing has its own jobs and upgrades require as many as 8 individual installations to cover the various backup servers across all 3 sites.

I'm considering rebuilding this configuration so that the repositories and proxies remain, but the actual backup server runs in Azure. All the various jobs are controlled by the backup server in Azure, and the "heavy lifting" is done by the local proxies and repos.

I believe this setup would be more efficient for resource usage (B&R server in Azure is aware of all jobs running) and for maintaining the jobs. Right now if we want to simulate a DR test, we have to manually import the backups onto whichever offsite server has them. I'm thinking if there's a core backup server running in the cloud with connections to all 3 environments, DR and DR tests can just be done via that backup server without faffing with importing backup chains.

Finally, I think management can be provided just via the Veeam console running on the support staff local machines rather than giving all tiers of staff administrator access to the backup server itself.

Would anyone have any thoughts on this?

[HannesK]

Hello,
and welcome to the forums.

What you describe to plan is what most customers in distributed environments do. Go for it

If you need to do application restore (mainly SQL), then it might be a good idea to add a VBR console to each location. Because otherwise the mount of the SQL backup will always be done on your central VBR server. That ends up with some unwanted cross-site traffic. But except for that, a central VBR server is the "default way" to go for most customers.

Best regards,
Hannes

[oleg.feoktistov]

Hi Tom,

Agree with Hannes. Also, some thoughts to share since you are planning to run VBR in Azure for a distributed environment:
- Make sure backup proxies are scaled to your infrastructure and jobs sizes lest to have jobs ever failover to VBR
in case of on-premise proxies unavailability.
- Configure Proxy Affinity settings for backup repositories to prevent local repositories from communicating to remote proxies.

Thanks,
Oleg

[Treebeard]

Hi both,

Thanks for your comments, and also for the recommendation about the VBR console on the local sites - I hadn't considered that.

Sounds like a very interesting project to start designing

Cheers!

[Treebeard]

Sorry to bump - this project has finally started! I'm just setting up a VM in Azure now, but is it possible to use an Azure SQL Database as the Veeam DB, or does it need to be a "proper" instance?

So far I can connect the Veeam B&R installer to the Azure SQL instance but the installer complains that the SQL user doesn't have the required permissions for the database I created. I guess this is because it's a "managed" database, therefore the user may not have some needed permissions otherwise found with a traditional "sa" account. Any ideas?

[oleg.feoktistov]

Hi Tom,

Correct, Azure SQL is not yet a supported type for this purpose. Though, we have this feature request noted.

Thanks,
Oleg

[Treebeard]

Hi all,

Apologies for another bump but I thought I'd provide an update on this, as the project is now complete!

As stated previously we had multiple B&R instances across three datacentres, and in fact these were also virtualised (including the repos, one vmdk per customer on large vmfs datastores). Each B&R had a dedicated proxy to offload the backup processing. This was all running on top of the very VMware platforms they were backing up.

After a lengthy discussion with one of the UK Veeam experts, we took a list of proposed improvements and best practices, and were able to adopt all of them.

The B&R server running in Azure has been really good for us. We have the Veeam console installed on our jump boxes and techs are discouraged from logging into the Azure B&R server directly. To protect it, we use Azure Backup for VM-level backups as well as an encrypted config export. We use file copy jobs to copy the config off to the two main DCs. Recovery is simply a matter of installing Veeam B&R anywhere, and recovering the config from one of the sites.

All former virtual components (apart from the B&R server in Azure) are now replaced with physical hardware.

Reusing some older kit, we now have a physical repo server at each site with large LUNs mapped from the backup SANs. We are using one huge ReFS volume (primary, fast clone) and an NTFS volume if we want to use Windows dedupe. The repo server OS disks are backed up to other sites using Veeam Agent for Windows. As the backup LUNs are mapped through iSCSI, we would simply just restore the OS into a VM if there was a major hardware failure, and the LUN maps would still work albeit reduced task capacity. The repo servers also have a decent amount of local DAS storage which we use to backup our infrastructure components, making sure the backup SAN storage is retained for customers. Everything follows the 3-2-1 rule.

We also have two physical proxies per site. As they were former VMware hosts used for VDI, they were very high on CPU cores and RAM and so made very good proxies.

Apart from the B&R server, none of the components are domain joined. We still rely on the platforms for DNS, however we include offsite DNS servers in the list so that we don't simply rely on the local domain controllers.

Another design change is to limit our use of Windows dedupe. Previously this was used a lot, and the amount of I/O on the backup SANs was never ending. The relentless I/O caused latency issues on the VMware hosts. Backup windows were also difficult to manage as we had to ensure backups and dedupe never ran at the same time.

The data savings with Fast Clone on ReFS has given us about a 50% reduction across the board. We did get a better dedupe ratio with Windows dedupe, however the overall repo disk size still had to be large so that transforms could happen. Capacity planning was a challenge!

I'm very pleased with how this has turned out, and the feedback from staff has been overwhelmingly positive.

We have in fact run into the SQL database restore issue recently, and will probably look at installing an SQL Server Express instance on some of the jump boxes for smaller DB restores.

Thanks to Veeam for making such a flexible solution to allow this kind of setup.