Backup of Hardened Repository Server

[chrisr]

I've implemented a Linux Hardened repository setup using some spare hardware and storage connected via iscsi. It's seemingly working well and I have been able to backup, backup copy and restore vms from test jobs, and have been blocked from deleting data from the VBR console during the immutability period.

However, my mind now turns to the fact that the physical server hardware that is the linux server is now a single point of failure and isn't backed up anywhere as yet. I looked at adding a linux computer managed by the backup server, which of course asks for credentials to connect and there doesn't seem to be a single use option.

It wouldn't make sense to use the creds used for the owner of the repo folders (as they would then be stored and negates the single use setting), and the root account is disabled from memory (one of my more linux savvy colleagues did the centos install & config) so how should I go about backing up the repo server (to alternate nas storage of course, i'm guessing i can't backup the repo server to one of the hardened repos)?

As I see it, one option is to add another local linux account for the agent backup, or perhaps use another in guest/non-veeam solution

[PTide]

Hi,

Unfortunately, Linux Agent backup is not supported for Linux servers that hold hardened repository role.

Our hardened repo does not use root, while some of VAL components do.

That is, if you install the agent and will try to use the same machine as hardened repo, the repo will be not-so-hardened (if it will be functional at all!).

Since Linux repository does not hold any special configuration, you should use a backup copy job instead (to have an additional copy of your backups).

As for the box itself - it can be redeployed in a matter of minutes if you have spare hardware.



Thanks!

[chrisr]

Hi, yes I have backup copies configured for the normal jobs (albeit on the same storage for now, but will eventually be on storage in the DR site), so my question is more about backup of the linux box itself - I didn't realise that the agent backup wasn't supported for those that have the repo role and, as a linux noob, I was looking for an easy way to be able to redeploy rather that resetting up from scratch. I suppose that method is not unheard of, as that is the recommended esxi host recovery method from memory

[mamosorre84]

Hi all,

I've the same doubt about linux server as single point of failure.

What do you suggest to avoid this "potential risk"?

If I have two linux hardened repo, can I use them as a single repo configuring some clustering features linux side?

Thanks

Marco S.

[PTide]

Hi,

If you really want to back up the repo, you can install VAL in a standalone mode (this will not impose the problems that I mentioned above).

Thanks!

[mamosorre84]

I don't want to backup the repo, I want to have an "high available" repo

[chrisr]

Hi all,

I've the same doubt about linux server as single point of failure.

What do you suggest to avoid this "potential risk"?

If I have two linux hardened repo, can I use them as a single repo configuring some clustering features linux side?

Thanks

Marco S.

I was thinking about a 2nd linux server and then only mounting half the repos on one server and the other half via the 2nd linux server, potentially a little extreme but it would ensure access to at least 50% of the backups in the event of loosing the single linux host we currently have

[PTide]

I don't want to backup the repo, I want to have an "high available" repo

I think something like active-passive configuration with a shared storage should work just fine. However we've never tested such scenario.
In general, as long as you ensure that the repo node can be resolved and has access to the same storage and has all veeam service up and running, VBR is pretty much agnostic about what's on the backend.

[chrisr]

I don't want to backup the repo, I want to have an "high available" repo

Well yes, I don't want to backup the repos themselves, just the OS/config of the underlying hardware that makes those repos available

[chrisr]

Hi,

If you really want to back up the repo, you can install VAL in a standalone mode (this will not impose the problems that I mentioned above).

Thanks!

That's where I came in - looking at installing the linux agent, but confused about account creds to use as there is no root and it doesn't make sense to undo the hardening by using the repo folder access creds

[PTide]

Just use sudo to install VAL locally in a standalone mode and to operate it.

Thanks!

[palijn]

I think you are confused regarding the whole "no root" thing. If you set up a standard linux os, root exists and there is nothing you can do about it since the system is pretty much designed that way. Root is user id 0, whatever the name. I used to rename the user caroot for fun. Any process running with process owner uid 0 is effectively run by root. Which is exactly what sudo achieves : run the command as the user whose id is 0 (unless you specify another user as argument).
What you probably have is that the root /account/ is disabled, which prevents you from logging in as this user, and not much more.

[Gustav]

One method for this (with three options) is described in my articles:

Part 6: Backup of the Linux server itself
Part 7. Bare Metal Recovery of the Linux server

[javichumellamo]

I think something like active-passive configuration with a shared storage should work just fine. However we've never tested such scenario.
In general, as long as you ensure that the repo node can be resolved and has access to the same storage and has all veeam service up and running, VBR is pretty much agnostic about what's on the backend.

I run a clustered SAMBA service on DRBD some years ago and it worked like a charm. AFAIK you can replicate yout repo with DRBD+Keepalived without any shared storage, said that it seems to be an even more available option than a shared stg because you will not have any single point of failure. I'm about to start playing with a hardened repository and I want to test it on DRBD as soon as I can.

[lolbebis]

We run the hardened repository on a dedicated ESXi with just the repository on it and the backup storage on RDM disks.
That makes taking image snaphots of the OS easy without snapshoting all of the PB of backup data.
In case of a hardware failure we can just restore the vm to another hardware and remap storage.
I guess doing that will add some attack surfaces, but since we use it for a single vm we can lock it down pretty hard.

[veremin]

But does not this add additional security breaches (immutable backups can be destroyed as soon as insider gets access to virtual infrastructure) and complexity during restore (a virtual environment has to be created first before you can get access to backup data)?

[chrisr]

One method for this (with three options) is described in my articles:

Part 6: Backup of the Linux server itself
Part 7. Bare Metal Recovery of the Linux server

Thanks Gustav,

That looks to be an excellent series of articles, with just the info required so I'll find some time to digest these, thanks

[Entropy]

Just chiming in that I used Gustav's approach to backing up the linux repo OS volume and it seems sound to me (and as a newb Linux admin the instructions were very thorough).

[Link State]

If you use a LUN RDM and the storage supports Volume Lock enable it, Infinidat use Snaprotator technology