-
- Influencer
- Posts: 17
- Liked: 2 times
- Joined: Aug 31, 2022 2:58 pm
- Contact:
Immutable Backup Rotation Idea (opinions wanted)
I have the following idea for immutable backups, please tell me where there are holes or why it's a bad idea.
-Single host with raid controller in RAID 10 or 6 with large HDD's
-Using something like Proxmox.
-Management NIC always physically unplugged unless work on the host is needed.
-4 hardened immutable Linux repo vm's setup on the Host.
-Each vm with a single passthrough Physical NIC for network access
-Backup Data virtual disks for each VM stored on same proxmox datastore with data deduplication enabled and encryption enabled on the proxmox datastore.
-Separate immutable jobs for each separate VM, happening on different weeks (week 1, week 2, week 3, week 4 and then repeat)
-Cronjob on the proxmox host to disable the physical NIC of all VM's except for the VM that has the job for that week. Enable whichever VM should be enabled as well.
-Disable Encryption (maybe) on the backup copy job so that deduplication on the proxmox host is efficient.
The idea is you functionally get offline immutable backups since only 1 of the VM's is online at a time. With, from my perspective the only "exploit path" would be a bad actor getting in to the online hardened repository and escaping the VM in to the host (all extremely unlikely) and wiping out the hosts data. You also get the benefit of having fast clone within the single VM's and also deduplication on the host side to hopefully reduce data usage to something close to a single VM.
-Single host with raid controller in RAID 10 or 6 with large HDD's
-Using something like Proxmox.
-Management NIC always physically unplugged unless work on the host is needed.
-4 hardened immutable Linux repo vm's setup on the Host.
-Each vm with a single passthrough Physical NIC for network access
-Backup Data virtual disks for each VM stored on same proxmox datastore with data deduplication enabled and encryption enabled on the proxmox datastore.
-Separate immutable jobs for each separate VM, happening on different weeks (week 1, week 2, week 3, week 4 and then repeat)
-Cronjob on the proxmox host to disable the physical NIC of all VM's except for the VM that has the job for that week. Enable whichever VM should be enabled as well.
-Disable Encryption (maybe) on the backup copy job so that deduplication on the proxmox host is efficient.
The idea is you functionally get offline immutable backups since only 1 of the VM's is online at a time. With, from my perspective the only "exploit path" would be a bad actor getting in to the online hardened repository and escaping the VM in to the host (all extremely unlikely) and wiping out the hosts data. You also get the benefit of having fast clone within the single VM's and also deduplication on the host side to hopefully reduce data usage to something close to a single VM.
-
- Veeam Legend
- Posts: 418
- Liked: 244 times
- Joined: Apr 11, 2023 1:18 pm
- Full Name: Tyler Jurgens
- Contact:
Re: Immutable Backup Rotation Idea (opinions wanted)
What is the reasoning for the added complexity? Deploying a VHR using the Veeam standards has that system essentially only accessible by console. All I see with your setup is a bunch of added complexity without added redundancy.
Tyler Jurgens
Veeam Legend x3 | vExpert ** | VMCE | VCP 2020 | Tanzu Vanguard | VUG Canada Leader | VMUG Calgary Leader
Blog: https://explosive.cloud
Twitter: @Tyler_Jurgens BlueSky: @explosive.cloud
Veeam Legend x3 | vExpert ** | VMCE | VCP 2020 | Tanzu Vanguard | VUG Canada Leader | VMUG Calgary Leader
Blog: https://explosive.cloud
Twitter: @Tyler_Jurgens BlueSky: @explosive.cloud
-
- Influencer
- Posts: 17
- Liked: 2 times
- Joined: Aug 31, 2022 2:58 pm
- Contact:
Re: Immutable Backup Rotation Idea (opinions wanted)
I guess the reasoning is a hardened repository remains online at all times and is subject to any bad actor that finds or is aware of an exploit in the veeam service listening on the network (the communication path from veeam backup server to veeam hardened repository). With a hardened repository effectively offline, there is no worry of this exploit.
-
- Veeam Legend
- Posts: 418
- Liked: 244 times
- Joined: Apr 11, 2023 1:18 pm
- Full Name: Tyler Jurgens
- Contact:
Re: Immutable Backup Rotation Idea (opinions wanted)
In your proposed setup I'd be more worried about stability of your infrastructure rather than a bad actor exploiting the Veeam services. For example, you have a problem with Proxmox (or whatever underlying hypervisor you chose) and you now lose access to your critical backups. Or you have an urgent restore request from a repo that is offline. How do you plan to address that? Getting back into that hypervisor to restore network connectivity then perform the restore - assuming that OS didn't crash or anything in the meantime, because without network connectivity, you aren't monitoring it either. I see a lot of introduced risk to reduce a different potential risk.
Backups should be simple and straightforward. You do not want complexity to interfere with recovering from your backups. You should still have another copy of those backups offsite, so if someone were to exploit Veeam's services, then get into your VHR and further exploit the OS to gain root access, you would still have those offsite backups.
Backups should be simple and straightforward. You do not want complexity to interfere with recovering from your backups. You should still have another copy of those backups offsite, so if someone were to exploit Veeam's services, then get into your VHR and further exploit the OS to gain root access, you would still have those offsite backups.
Tyler Jurgens
Veeam Legend x3 | vExpert ** | VMCE | VCP 2020 | Tanzu Vanguard | VUG Canada Leader | VMUG Calgary Leader
Blog: https://explosive.cloud
Twitter: @Tyler_Jurgens BlueSky: @explosive.cloud
Veeam Legend x3 | vExpert ** | VMCE | VCP 2020 | Tanzu Vanguard | VUG Canada Leader | VMUG Calgary Leader
Blog: https://explosive.cloud
Twitter: @Tyler_Jurgens BlueSky: @explosive.cloud
-
- Veeam Legend
- Posts: 212
- Liked: 60 times
- Joined: Mar 22, 2017 11:10 am
- Full Name: Mark Boothman
- Location: Darlington, United Kingdom
- Contact:
Re: Immutable Backup Rotation Idea (opinions wanted)
As Tyler suggests the infra stability is a huge risk. If you have an issue with the host that is all your repo's offline which could be catastrophic
-
- Influencer
- Posts: 17
- Liked: 2 times
- Joined: Aug 31, 2022 2:58 pm
- Contact:
Re: Immutable Backup Rotation Idea (opinions wanted)
well I guess I should add that this would be in addition to our other backup infra. We have 3 locations with server infra and each location have a veeam repository which copy all their backups to all sites (3 sites have a copy of every site including itself) so restores are very simple and fast. This specific idea is to have a copy of all those backups that are effectively offline without actually needing to be offline. Currently I just have a single bare-metal hardened repot (a 4th repo) at our main location. Just trying to expand the "safeness" of the backups without going to the inevitable tape or removeable HDD scenario. I'll also add we are an "air gapped" (not entirely) network, so off-siting to a cloud immutable repo is a no go.
Who is online
Users browsing this forum: Ahrefs [Bot], Semrush [Bot] and 67 guests