Hardened Repository with external iSCSI storage

[Steveeo]

Moderator split from veeam-backup-replication-f2/hardened-re ... 00624.html

Morning,

Just going to Hijack this
So I have been trying (grrrr so much time waisted!) to setup a Physical Linux iscsi attached immutable datastore for backup copies (2x one at each site for backup copies over Wan) simples right? Well if it is what am I missing. I have a call with Veeam still open but they actually linked this page and it seems its the only solution but it doesn't actually say how! Oh and I'm a Windows guy not a Linux so that doesn't help!

I've used Ai to whack my exact config/install, I have rebuilt Linux OS twice and removed/re-added several times all with the same issue, XFS enabled but the -i is never applied to the backup chain and no option in Veeam (like on the Linux Veeam hardened iso image) to set immutability times eg 7/14 days etc.

What am I doing wrong OR can I not do it.
Objective
Deploy a new BYO Linux hardened repository on Ubuntu backed by iSCSI block storage, and verify immutability enforcement at the filesystem level in Veeam Backup & Replication v13.
________________________________________
Environment
• Veeam Backup & Replication: v13 (current production build)
• Repository OS: Ubuntu Server (fresh installation)
• Repository type: BYO Linux (not JeOS / Infrastructure Appliance)
• Storage: Dell PowerVault ME5012 presented as iSCSI block device
• Filesystem: XFS with reflink enabled
• Use case: Backup Copy Job repository
• Authentication: Non root Linux user with passwordless sudo
________________________________________
Linux Repository Host Preparation (New Build)
OS installation
• Ubuntu Server installed cleanly
• No prior Veeam components or repository configuration
• SSH enabled
• Bash shell in use
________________________________________
Storage preparation
• iSCSI LUN presented from storage array
• OS sees disk as:
• /dev/sde
________________________________________
Filesystem creation
The disk was formatted as XFS with reflink enabled:
Shell
mkfs.xfs -f -m reflink=1 /dev/sde

Show more lines
Verification:
Shell
xfs_info /dev/sde
# reflink=1 confirmed
Show more lines
________________________________________
Mount point and permissions
Shell
mkdir -p /repo/backups
mount /dev/sde /repo/backups
chown root:root /repo/backups
chmod 700 /repo/backups
chmod -t /repo/backups
``

Verification:
Shell
ls -ld /repo/backups
# drwx------ root root
``

No subdirectories were created manually.
________________________________________
Repository Addition in Veeam
Add Backup Repository
Path used in Veeam:
Backup Infrastructure
→ Add Backup Repository
→ Direct Attached Storage
→ Linux
The option “Linux (Hardened Repository)” is not available for BYO Linux + iSCSI.
________________________________________
Authentication
• Non root Linux user
• “Elevate account privileges automatically”
• No root credentials used
________________________________________
Repository path
/repo/backups
Veeam automatically created:
/repo/backups
All other repository settings were left at defaults.
Repository was added successfully.
________________________________________
Backup Copy Job Testing
Data written to repository
• Backup Copy Job executed
• Backup Copy Job executed again to ensure restore point closure
Restore points were visible on the repository.
________________________________________
Filesystem-level immutability verification
On the Linux repository host:
Shell
lsattr -R /repo/backups
``

Observed result:
Plain Text
---------------------- *.vbk
---------------------- *.vib
---------------------- *.vbm

No files ever displayed the immutable (i) attribute, even after multiple closed restore points existed.
________________________________________
Expected vs Actual Behavior
Expected
• Once restore points are closed on a hardened repository, backup files should be protected using the Linux immutable attribute (chattr +i).
Actual
• Backup files remain fully mutable.
• No immutable attribute is applied at any point.
• Repository behavior matches a standard Linux repository.
________________________________________
Assistance Requested
Please confirm:
1. Whether BYO Linux repositories in VBR v13 are expected to enforce immutability at all, or
2. Whether immutability enforcement in v13 is exclusive to JeOS / Infrastructure Appliance deployments only, even though iSCSI is supported for BYO Linux repositories.
________________________________________
Summary
This is a new Ubuntu build, using:
• supported OS
• supported filesystem (XFS + reflink)
• supported storage (iSCSI block device)
• supported authentication model
Despite this, immutability is never enforced at the filesystem level.

[HannesK]

Hello,
and welcome to the forums.

99% sure you have configured a regular Linux repository and not a Hardened Repository because these two command break Hardened Repository because it cannot write to folders that are owned by root.

chown root:root /repo/backups
chmod 700 /repo/backups

I suggest following one of the guides that exist on the internet instead of hallucinating AI commands... "chmod -t" is something I cannot remember I ever used...

Best regards
Hannes

[Steveeo]

Hi,

I've found various guides. They all seems to be missing the same part - isci connection, in my case the jeos build I cannot connect the iscsi connection as the dell me5012 requires a connection from the device, you cannot do this as there is no root command to try a connection.
I'm either Dumb or it can't be done.

[Mildur]

External storage is not supported yet for our appliances.
This will be a new feature in our summer release (v13.1).

Best,
Fabian

[d.artzen]

When we build our hardened repos some years ago we used this article series to implement it on Ubuntu: https://github.com/GustavBrock/Veeam.Li ... ory%201.md
This was still in V11, but most of the instruction is still valid in the V13, the only direct change came in V12 that the option to add a hardened repo was separated from the normal linux repo (Step 4 in that article series). So you have to make sure you choose the correct server type.
We just adapted it to use a iSCSI target (direct attached NAS) instead of internal disks (Step 3 in the series), how to do that was easily found via Google, Ubuntu is quite good documented. Just search for iSCSI and Ubuntu. Then you follow the guide in formatting the volume with XFS and setting the correct permissions. Has worked fine ever since. Also we did not implement the steps to backup the linux server itself, since it is not recommended to install the veeam agent to a hardened repo.

[HannesK]

Hello,
I split this into a separate topic to avoid confusion in the other thread. I still believe that the described documentation above cannot work because it would require root permissions to write into /repo/backups which Hardened Repository does not have. Also the iSCSI part itself seems to be missing in the description. As I read it again, I also see that you probably really added a regular Linux Repository and not a Hardened Repository because the key thing is "Single-use credentials for hardened repository..." .

Does it say "Hardened" as "Type" in "Backup Infrastructure" -> "Backup Repositories"?

To summarize: for me, the entire description looks wrong and as you say that immutability is not working, I tend to delete the entire thread to avoid other people from trying to copy from that.

Best regards
Hannes

[swarnock@us.tuv.com]

Here is a rough setup for how I do Ubuntu 24 hardened repositories.

Configure iSCSI initiator (client)
sudo iscsiadm -m discoverydb -t st -p <ip of target(server)> -o new --discover

sudo iscsiadm -m node -l -W

sudo iscsiadm -m node --op=update -n node.conn[0].startup -v automatic
sudo iscsiadm -m node --op=update -n node.startup -v automatic

Verify iSCSI connection via "dmesg".

Proceed with creating a file system on the newly attached disk using the regular method. Please note that mkfs will TRIM the entire iSCSI LUN, this can take a significant amount of time.

Create new directory for the repository "mkdir /repository0"

Configure immutability on /repository0 directory to prevent Veeam from borking the repository if the drive mount is unable to mount. (i.e. chattr +i /repository0)

Create mount point via systemd mount point. /etc/systemd/system/repository0.mount

Code: Select all

[Unit]
Description=Veeam Repository
After=open-iscsi.service
[Mount]
What=/dev/disk/by-uuid/7dd3f8cc-f2a9-4c2f-90de-d1c5f91fa30e
Where=/repository0
Type=xfs
Options=defaults
[Install]
WantedBy=multi-user.target

Configure Ubuntu Hardened Repository
Create veeamhardened user on Ubuntu host/guest "sudo adduser veeamhardened"

add veeamhardened user to sudoers "sudo adduser veeamhardened sudo"

Grant veeamhardened user permissions to /repository0 directory.

Configure repository in Veeam console

Remove veeamhardened user from sudoers "sudo deluser veeamhardened sudo"



I do two different things from what I have seen others do. First I deny permissions to the mount directory so that if the volume fails to mount Veeam is unable to write to this directory. This prevents a backup job from starting unless the repository is properly mounted. Second I use Systemd .mount files for the repository mount instead of /etc/fstab. I do this as .mount files do not prevent the OS from starting. In addition this allows me to setup the mount to happen after the iscsi daemon is fully loaded.
Overall this is how I have setup a group of remote branch offices to be as reliable as I can make them.

Sean