Hardened Linux - one repo multiple sites

[dhayes16]

Hello all. I hope everyone is doing well.

Quick question. We are deploying a site to site backup and replication solution between 2 sites over a high end vpn. Basically 2 sister bdrs at each site replicating each other. We know there are cloud options for immutability but we would like to deploy a Linux hardened repo for archive. Now I know the limit of the Linux hardened repo is that it can only be connected to one b&r server. So would there be any downside to having one b&r server (where the Linux repo is) handle the entire backup infrastructure processing with a proxy at each location and using the Linux repo as a destination for the servers at both locations?

Thanks for any input. I am not sure I am explaining it right.

[micoolpaul]

Hi, I would rather not have a primary backup at a different location to the restore destination. I’d suggest a local backup for the remote site to the hardened repo and then a backup copy job.

If you’re just adding this in as an archive tier, I’d ask the question, is this my off-site copy as well? Since you mentioned cloud backup, that is a key benefit. So if it’s your off-site copy, it needs to be hosted at a 3rd site or can only be an off-site copy to one of the 2 sites.

Some food for thought…

[dhayes16]

Thanks for taking the time to respond. I would refer you to the following post I made about this design.

microsoft-hyper-v-f25/please-critique-m ... ml#p448873

I should have been more l detailed. Yes we want to add an archive tier to this design. Others have mentioned using an off site vcc provider and possibly using insider protection and such (or possibly aws or Wasabi object) but the customer would like to keep the archive tier on premise at their primary location if possible. And let it be the archive tier for both locations.

Again thanks for the feedback.

[jorgedlcruz]

Hello,
As mentioned by Michael, I think the best recommendation will be:
- Physical Backup Repository per site, as I can not see anything like this in any of your threads, always better and more secure when they are physical. Linux with Hardened Repository, meaning SSH disabled, firewall up, and only allowed iDRAC or iLO, etc.
- Backup Proxies are the HyperV per site so all good, better onhost.
- Backup server on one of the sites, with backup config to one site, and file copy job to copy this key file to the other sites.
- Seems you have a server at each site with hyper-v waiting to become active? Is this outside your current prod infra? I quite do not understand this VBR per site with hyperv. But if you do, all good, as a standby with the config file freshed copy every day.
- Backup jobs to every site HLR with short retention, like 14 days.
- Backup Copy between the sites to the their HLR, just different folders

So this will be one single VBR, seems a VM, with backup config protected and replicated between sites, maybe replica this VM as well. Then this VBR controls all HLR, in case this VBR gets compromised, you either boot up the replica vbr on another site, or import config to a ready to go VM.

PS: There are other options for what you call archive tier, which should not be used as we have a feature call like that that is specifically for glacier, deep archive and azure archive. You could do HLR to every site, always recommended backup locally. And then have a Cloudian, or a minIO, or Scality, Object storage as a backup copy with object lock. Giving you that difference between normal disk presented for backups, and object storage abstraction for the copies. This if budget allows it.

That should be enough.

If in doubt, please contact your local Systems Engineer and have a quick call with him/her.

Cheers

[dhayes16]

Thank you for your reply. I have revised the pic of the project below.

https://imgur.com/gallery/yyXugnG

Each site will have dedicated Windows 2019 servers with 20TB of storage that will be the local on-prem repos running ReFS. Also, these servers will be running Hyper-V for DR purposes. Each one of these servers will contain replicas for the other site for DR purposes. We were thinking of introducing the HLR (only one) at the Seattle location with 40TB of storage to be the backup copy job target for BOTH servers at each location. Sorry I used archive tier incorrectly.

The overall goal here is to protect from ransomware or other malicious attack. We could also look at local on-prem Object storage. I had not considered that.

Thanks for your very valuable input.
Dave

[jorgedlcruz]

Hello,
That makes sense. Seattle is the weakest point as it has backups, and copies on same site. Can you not add another HLR on the other site so you do cross backup copies.

Windows with ReFS sounds good for backups, so the shyntetic flies.

I said object, but seeing you have physical windows, you can even consider small tape at one site to hold backups of that site, and backup copies of other site perhaps.

Glad to be of help.

[soncscy]

As mentioned by Michael, I think the best recommendation will be:
- Physical Backup Repository per site, as I can not see anything like this in any of your threads, always better and more secure when they are physical. Linux with Hardened Repository, meaning SSH disabled, firewall up, and only allowed iDRAC or iLO, etc.

To be honest, I wouldn't keep iDRAC/iLO unless you have some absolute regulatory need for it. Both get deep into their respective systems and have had multiple high-rating CVEs as late as November 2021 (iLO is particularly guilty....)

These remote administration systems are just as convenient for attackers as they are for sysadmins, and that they can sit on public IPs is just a disaster waiting to happen. They're backdoors no matter how you look at them, and in my opinion, neither Dell nor HPE have shown they are taking such issues seriously as the tooling only gets more powerful, not less.

I'm afraid I don't have a good answer on how to forward out reporting information, but I imagine it should be some system on the box itself that warns as various parameters approach their thresholds, and communication can only go out, not in, and it should be set to be overly sensitive so that you're alerted before stuff becomes an issue.

[micoolpaul]

To get information out you can get SNMP and/or Syslog configured to achieve this and if it makes sense to keep iDRAC/iLO enabled, I’d be placing that behind an MFA enabled VPN in an isolated subnet on a dedicated switch as well preferably.