[RELEASE] Managed Hardened Repository ISO by Veeam

[gmajestix]

Installation also works on Dell R760 < PERC H965i Front for DATA / BOSS-N1 for OS, Broadcom Adv. Dual 25Gb.

[HannesK]: thanks, I added the server to the list

[mattskalecki]

We're excited to move our managed Hardened Repository project to the next step and upgrade its status from Community Preview to experimentally supported. This means hardened repositories provisioned from this ISO build are now officially supported for use in production environments, and you can open support cases normally in case of any issues (experimental support SLA disclaimer applies only to issues with the ISO Installer and the Configurator Tool specifically).

Am I interpreting this correctly?
[*] Support cases involving issues with backup/restore/etc jobs targeting a repository built from the "Managed Hardened Repository ISO" are given full support as if the target was any other supported repository.
[*] Support cases involving issues with installing or configuring the repository with the "Managed Hardened Repository ISO" are given the lower "experimental support SLA".

If that is correct, I'm very excited to give this a try with our next repo server. Manually hardening these servers is not trivial.

[PTide]

Hi,

Yes, that's correct.

The Veeam bits running inside the Hardened Repo are the same regardless of the nature of the repository, hence all issues related to the actual usage of the repo are treated with the same level of dilligence.

At the same time, issues related to installation/configuration process of the ISO-based Hardened Repo will have lower priority due to the novelty of the deployment method.

Thanks!

[MOBO]

when the repair option arrives , will it be able to upgrader/replace VHR that is installed following the blog posts of Hannesk from 2023 or will i need to start over ?

[HannesK]

Hello,
"repair" as it was developed today (internal builds) only works for systems installed with a Hardened Repository ISO. But we are evaluating alternative options that require manual user interaction to allow migrations.

Best regards
Hannes

[Jneau]

Hello everyone,

I have a question about the VHR image published by Veeam.

In the event that a client saturates its VHR repository, what are the possibilities of being able to remove the flags and delete files to free up space? Do we have to go through veeam support?

I'll admit I've tested it, and the solution is functional and hardening .

Have a nice day,
Julian,

[Gostev]

Yes, you can always remove flags yourself under root, after which backup files can be deleted... no need to go through support.

[Outlaw]

the answer was / is "no".

I just try to understand why customers ask such things, because we try to build things customers ask for

Hello

Perhaps the answer "NO" was too veiled. Somewhere I heard about a practice that you can't say no, it's bad for karma

P.S. I'll wait until there are more such requests from clients

[Jneau]

Yes, you can always remove flags yourself under root, after which backup files can be deleted... no need to go through support.

Hello Gostev,

Unless I'm mistaken, we don't have root access with this VHR Veeam ISO deployment. If I take a note from the community:

“Advantages of VHR VEEAM ISO

The biggest advantage of ISO is that there's no need for further customization or scripting (the system is already hardened by the custom installer).
There is no root user.
Using Rocky Linux as a base, you benefit from 10 years of support.
After the official release, you'll also benefit from Veeam support.”

Julian,

[Gostev]

As long as you are able to boot a VHR server from external media, then you can always get yourself root access by modifying relevant OS security files directly from within another OS you booted. The only thing that can potentially stop you are locked BIOS settings to make the machine boot from external media, however assuming your an admin of said server you'd know its BIOS password.

P.S. Note that physical access to actual server's enclosure gives you a number of ways to bypass even said BIOS lock. You could reflash the BIOS to reset its password, or remove hard drives and connect them to another server to edit those OS files, etc.

[Jneau]

Thank you Gostev,
We agree, access with an external media will allow to modify the accesses. You've answered my question.

Have a nice day,

Julian,

[chris_lalala]

Im just missing some Userguide/Documentation to this release.

E.g. Users automatically created, default Logins, is SSH allowed, permissions on default created users, terminal login, etc.

[HannesK]

Hello,
@chris_lalala: documentation is available in the user guide

@chris.childerhose: I deleted your answer to avoid confusion and split the other question as it describes a technical issue.

Best regards
Hannes
PS: sorry for the delay, I did not get email notifications for unknown reasons

[j.suenram@it-ngo.com]

Hi,
I know this is primary for Bare-Metal-Servers but we have some Servers in Datacenter which present VMs for customers as Remote-Backup-Storage. Will there be an option to extend existing volumes when the VM Disk is resized in the Hypervisor?

Regards

[HannesK]

Hello,
the "resizing" and "add storage" request also exists for physical servers more or less the same. There are ideas on that topic, but no timeline.

For physical servers, one would usually add new disks / RAID-sets and then add that to LVM and extend the file system. Some RAID controllers might even allow extending RAID-sets. Extending the disk size also works with LVM (pvresize). Then the file system also needs to be extended. I won't describe here how to do that, because if done wrong, data loss can happen.

Best regards
Hannes

[chris_lalala]

Hello,
@chris_lalala: documentation is available in the user guide

@chris.childerhose: I deleted your answer to avoid confusion and split the other question as it describes a technical issue.

Best regards
Hannes
PS: sorry for the delay, I did not get email notifications for unknown reasons

Sorry, but this is not a Userguide for the Linux OS. This one is missing E.g. Users automatically created, default Logins, is SSH allowed, permissions on default created users, terminal login, etc.

[HannesK]

Hello,
The passwords are documented here

Does that solve everything?

We don't document the Linux OS details because the customer is not supposed to touch it.

If you wish to configure Linux OS yourself, you can build hardened repository on any supported Linux distribution on your own.

Best regards
Hannes

[padair]

I've had 0.1.17 up and running on a new Dell R760xs for a few weeks now. Install and setup to backup data went without issue.
Server has PERC H755 controller with 6 x 12TB SATA drives. (setup Raid6)
Server also has the BOSS-N1 with 2 x 480GB SSD drives. (setup Raid1)
All is well so far....

EDIT - This now includes upgrading V B&R server to 12.3 and any updates it need to apply to the VHR as well. All working good.

[HannesK]: thanks, added to the list

[mattskalecki]

I successfully installed the VHR ISO on a "white box" server with a "Supermicro X11SPI-TF" motherboard and an "LSI MegaRAID SAS 9361-8i" raid controller.

I ran encountered two errors (both user created):

• I forgot to enable UEFI Secure Boot before running the installer. The installer stopped immediately and directly said that UEFI Secure Boot must be enabled. Thumbs up.
• I initially created the bootable usb stick using Rufus in the default "ISO Image" mode. This resulted in an installer that booted and appeared to work just fine. I was able to setup the network and NTP. But, when I clicked to start the install, it progressed through the first few steps before failing with an "unknown error". Fortunately, I found a forum post here that clarified that "DD Mode" is required when using Rufus. Given how popular Rufus is, and the fact that the non-working mode is the default, I think it would be nice if the documentation had a warning note. It would also be great if the installer could detect that it was improperly created and give a more informative error.

Besides the mysterious "unknown error" issue, I was really happy with the experience. I second the earlier request for a "Test Network" button. I noticed that the NTP settings kind of acted as a network sanity check, because they only allowed me to enable NTP once the NTP server was reachable, but it would be much better for trouble-shooting to see an option like the VMWare ESXi network configurator, which shows ping results to all the intermediate destinations.

Adding the Repo to the VEEAM backup infrastructure was a breeze.

[HannesK]: thanks, added to the list

[HannesK]

Hello,
thanks for the feedback, I added the new servers to the list

For Rufus: the conversation is still going on where and how to document it. Technically, it's "outside Veeam", but we understand that the majority of customers uses it and we have seen several cases, so we want to address that.

Network troubleshooting: thanks, I added +1 -

Best regards
Hannes

[cmln]

Hello all,
I have a question.
I have to install an Immutable with a Linux server, previously I used Ubuntu and that wend fine.
Now I want to use Rocky hardened repository but there is a catch.
The thing is I’m going to use a PowerdEdge R660 with a Powervault 5024 direct attached storage.
Is that supported?
And if it is supported, how do I begin?
Regards
CMLN

[chris.childerhose]

You would just need to try the latest ISO and see if it works to install and set up the DAS properly for the repo. Then report back any problems or errors to support.

[dloseke]

The thing is I’m going to use a PowerdEdge R660 with a Powervault 5024 direct attached storage.
Is that supported?
And if it is supported, how do I begin?

I'm assuming that you're using an HBA355e SAS HBA to connect to the array? In that case, you'd just create a volume on the array, create a server using the SAS HBA ID and map the volume to the server like one would normally do. Once that is done, boot up the ISO and if all is well you should see the local storage/volume as well as the volume you've mapped to the server. I'm assuming that you're using local storage for the OS, but if not, I would assume you could create two volumes, one at around 100GB for the OS, and then a second larger volume for the repo and the installer with some luck would find both of them.

[dloseke]

Not that this would be any way recommended, but I can confirm that the ISO can be deployed on a Synology NAS.

Synology DC923+ running DMS version 7.2.2-72806 Update 2. The underlying hardware is an AMD Ryzen R1600 2.4GHz processor with 4GB of RAM and storage is provided by (4) Synology HAT3300-4T HDD's and (2) Synology SNV3410-400G NVMe M.2 SSD's in a Synology Hybrid RAID (SHR) using the M.2 drives as a SSD Cache.

The VM in this scenario is running on Synology's Virtual Machine Manager with 4 CPU's and 2GB of RAM, Video card is configured as VGA (leaving as vmvga will result in a failure to boot), and 2 virtual disks were created, one is 100GB and the second is 8TB using the VirtIO interface. The network on the NAS two bonded interfaces using the Balance-SLB link aggregation mode.

[HannesK]

Hello,
agree, not a recommended setup, but KVM as hypervisor works, yes. Thanks for reporting!

I would reduce CPUs and add RAM because 2GB RAM is normally not enough for 4-8 Repository tasks that you would probably configure for 4 vCPUs.

Best regards
Hannes

[dloseke]

Thank Hannes. Unfortunately this particular NAS has 4GB of RAM total so I can't use any more than 2GB for the VM, but if I had the choice, I'd certainly be using 4GB if I could. I do have another NAS to play around with that has more RAM, so I'll be giving it a shot on that machine as well and I'm sure that'll yield better results. As such, I did limit the concurrent tasks as this particular machine as well. Thanks again!

[ITP-Stan]

I've installed this on an old HP 8300 TOWER pc with 1 240GB SATA SSD and 2 4TB SATA drives and setup went through OK.
I tried it first with only 1 4TB drive but that didn't work, the need for 2 identical drives for mirror is a hard requirement it seems.

[cmln]

I'm assuming that you're using an HBA355e SAS HBA to connect to the array? In that case, you'd just create a volume on the array, create a server using the SAS HBA ID and map the volume to the server like one would normally do. Once that is done, boot up the ISO and if all is well you should see the local storage/volume as well as the volume you've mapped to the server. I'm assuming that you're using local storage for the OS, but if not, I would assume you could create two volumes, one at around 100GB for the OS, and then a second larger volume for the repo and the installer with some luck would find both of them.

Hello dloseke,
I was thinking the same way.
I wil try it and let you know the result
Thanks regards
CMLN

[HannesK]

@ITP-Stan : Thanks for reporting back on the old PC. To answer your question: The ISO requires two disks. Right now the setup accepts two disks of identical size as smallest disk. The next version system requirement for disk sizes will be:
1) minimum two disks (same as today)
2) all disks must be at least 100GB or larger (same as today)
3) there must be _one_ smallest disk. This new requirement exists to ensure that "repair mode" overwrites only the operating system disk (smallest disk) and not a data disk

A setup with 200GB + 1TB + 2TB + 3TB is valid. 200GB will be used for OS and 6TB for data.

@cmln: it will fail if it's detected as multipath device (which I assume the setup will be)

[ITP-Stan]

Where can I find the documentation for this?
Like the passwords etc.
In helpcenter there is no specific section for this, is it just under B&R?