Standalone backup agents for Linux, Mac, AIX & Solaris workloads on-premises or in the public cloud
VAXman65
Influencer
Posts: 10
Liked: 4 times
Joined: Feb 18, 2015 1:40 pm
Contact:

Rocky Linux now GA. Veeam support ??? (also AlmaLinux)

Post by VAXman65 » 3 people like this post

I built up a Rocky Linux 8.4 system today using the GA code base that is now available. That worked just fine and I was able to install Veeam and get it started. However, once i defined the backup job, it fell over on execution. (Disappointing but not entirely surprising).

Is there a timeframe for Rocky Linux support in Veam?

I also filed Veeam Support - Case # 04874548

FWIW, here is the first sign of trouble in the Job log. I imagine the Veeam code needs a liberal sprinkling with “ || rocky” through the codebase.

[21.06.2021 19:15:24.340] <140649266202368> lpbdeve| ERR |Failed to create DeviceInfoEx for device [/dev/sr0].
[21.06.2021 19:15:24.340] <140649266202368> lpbdeve| >> |No medium found
[21.06.2021 19:15:24.340] <140649266202368> lpbdeve| >> |Failed to probe device [/dev/sr0].
[21.06.2021 19:15:24.340] <140649266202368> lpbdeve| >> |--tr:CRealBlkidAccessor: Failed to probe device [/dev/sr0].
[21.06.2021 19:15:24.340] <140649266202368> lpbdeve| >> |--tr:in virtual lpbdevenumlib::blkid::IBlkidProbe::Ptr lpbdevenumlib::accessor::CRealBlkidAccessor::ProbeDevice(const astr&) const at /mnt/Sources/LpbDevEnumLib/lpbdevenumlib/accessor/impl/RealBlkidAccessor.cpp:38

Regnor
Veeam Legend
Posts: 660
Liked: 190 times
Joined: Jan 31, 2011 11:17 am
Full Name: Max
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by Regnor »

Here's a quote from @gostev regarding Rocky Linux:
We will certainly consider adding support in future depending on its actual adoption levels and market share.
So if a certain percentage of Veeam customers is requesting support for Rocky Linux then it will be evaluated.

aj_potc
Expert
Posts: 103
Liked: 19 times
Joined: Mar 17, 2018 12:43 pm
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by aj_potc »

Sadly, Veeam doesn't support RHEL or CentOS 8.4 yet, so this may be the primary reason that it doesn't work on Rocky 8.4.

Gostev
SVP, Product Management
Posts: 29943
Liked: 5870 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by Gostev » 1 person likes this post

I would say, the primary reason is that we're trying to limit the number of distributions our QC needs to test the product against only to the most widely adopted ones. If Rocky will see good adoption, then we will add support for it - likely along with removing support for some other distribution to free up the QC resources for Rocky.

For RHEL 8.4 support specifically, we worked directly with Red Hat and they added some special code in their custom kernel to ensure Veeam will be able to support RHEL 8.4. I don't know if derivative distributions will automatically have the same mod, but without one we won't be able to support anything based on 5.8 or later kernel versions because of the kernel feature loss that previously enabled I/O interception essential for our snapshot and CBT drivers (see the sticky topic). Which means only file-level backup without snapshots would be possible.

VAXman65
Influencer
Posts: 10
Liked: 4 times
Joined: Feb 18, 2015 1:40 pm
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by VAXman65 »

I was able to confirm that the problem I was having was not with Rocky Linux, but rather with the newer kernel. In this case, the fault was bug-for-bug compatible with CentOS. The same fix worked in both places.

In both CentOS and Rocky Linux, 8.4 went out with kernel 4.18.0-305.3.1. This was the first kernel version of RedHat derivatives that broke volume snapshots with the incorporation of the 5.8 kernel enhancements. For both CentOS and Rocky Linux, I was able to leave the systems overall at the 8.4 version but downgrade the kernel back to kernel-4.18.0-240.22.1. This mixture was my preferred approach for Rocky Linux since it only went GA with 8.4. The alternative is to downgrade across the board to 8.3, which also worked.

So my experience has shown that CentOS 8.4 falls into this same problem and the kernel version that comes with it (at least today) does not have the necessary fixes to work with Veeam. Downgrade of at least the kernel restores functionality. Rocky Linux has proven to be bug for bug compatible and exhibits exactly the same problem and exactly the same solution.

Overall, this has been a positive experience for me, appearing to prove out how well Veeam with work with Rocky on the basis of this edge case where it actually broke!

Longer term, this does point to a CentOS issue and that it may be necessary for Veeam to work with the upstream maintainers to ensure it not only gets into the RedHat distro, but also into the codebase that the RedHat specific version is built from. If this is done, then it would flow automatically into the downstream releases. It may be the case that the fix code is being rolled into only the RedHat commercial version

PTide
Product Manager
Posts: 6218
Liked: 680 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by PTide »

Hi,

Depending on what your current kernel version is, I would suggest to take a look at this post.
As for pushing it upstream - yes, we have been working with the kernel maintainers.

Thanks!

Gostev
SVP, Product Management
Posts: 29943
Liked: 5870 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by Gostev »

Unfortunately, kernel maintainers don't want anything in the kernel that is not consumed by the kernel itself, which makes it extremely challenging to add functionality like I/O interception capabilities to be used by external applications like Veeam or Oracle.

VAXman65
Influencer
Posts: 10
Liked: 4 times
Joined: Feb 18, 2015 1:40 pm
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by VAXman65 »

With CentOS 8 dying at the end of the year, there will be little need/impetus to address this issue for that distro. CentOS Steam is another matter, but it being upstream of RedHat would seem to offer no solution if the point of entry for the code fix is with RedHat.

Rocky Linux incorporating these changes remains downstream from RedHat on an ongoing basis. In theory, code made part of the mainline build feeding into RedHat commercial release should appear in Rocky. However, if RedHat chose to incorporate the fixes only in their commercial code, it would be missed by Rocky Linux, since it would not be part of the code base passed down. This would be an awfully weird way of doing things - isn't the RedHat-only code their (limited) protected IP? I would not think that the fixes Veeam has been pushing and providing would qualify.

The real question would be, what kernel version is in RedHat 8.4 and does it fail today? If it is working, then are the changes in the codebase passed downstream or are they trapped in the RedHat proprietary code?

Gostev
SVP, Product Management
Posts: 29943
Liked: 5870 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by Gostev »

RedHat 8.4 is distributed with good old 4.18 kernel, but also with some stuff from 5.8 kernel backported into it, including the breaking change unfortunately. It's a good question though where those backported changes sit.

VAXman65
Influencer
Posts: 10
Liked: 4 times
Joined: Feb 18, 2015 1:40 pm
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by VAXman65 »

Is the plan here to hope for modification to the 5.8 kernel and hope that RedHat chooses to backport the fix to the breaking change later on?

Or is the approach to provide Redhat the code directly to fix the breaking change so that it would be included with the backported elements they include in 4.18?

Gostev
SVP, Product Management
Posts: 29943
Liked: 5870 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by Gostev »

It's the latter and it's already done. We identified the issue long ago based based on the pre-release 8.4 builds testing, and worked with Red Hat to implement a workaround for their custom kernel. We still need to modify our code slightly to work with that though, so the currently shipping Veeam Agent version won't work with 8.4 yet.

FromOZ
Lurker
Posts: 2
Liked: never
Joined: May 06, 2014 12:03 pm
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by FromOZ »

Gostev wrote: Jun 23, 2021 3:30 pm I would say, the primary reason is that we're trying to limit the number of distributions our QC needs to test the product against only to the most widely adopted ones. If Rocky will see good adoption, then we will add support for it - likely along with removing support for some other distribution to free up the QC resources for Rocky.
The key point here is that with respect to Rocky Linux we are not talking about yet another/new Linux distribution to have to code for, and assign QA resources to. We are talking about RHEL, because "Rocky Linux is a community enterprise operating system designed to be 100% bug-for-bug compatible with America's top enterprise Linux distribution" (see https://rockylinux.org home page with FAQ on this topic).

Rocky Linux is, as CentOS was before, a downstream build of RHEL. The project started because RedHat shifted CentOS from being downstream to upstream. Of course they already had an upstream, Fedora.

The net result/takeaway is:
  1. CentOS is, essentially, dead.
  2. Rocky Linux will be the platform CentOS likely most users will be changing to. It was started by Gregory Kurtzer, founder of the CentOS project, he obviously has good credentials.

Gostev
SVP, Product Management
Posts: 29943
Liked: 5870 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by Gostev »

If most of the former CentOS users migrate to Rocky Linux, then of course we will have no choice but to test and officially support our products on it. As CentOS had a decent market share before all the recent changes.

aj_potc
Expert
Posts: 103
Liked: 19 times
Joined: Mar 17, 2018 12:43 pm
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by aj_potc »

There already are a couple of replacements for CentOS. Even if Rocky does not end up being the most popular, I think the point that @FromOZ is making is that Rocky isn't a new distribution that you have to support, but rather is intended to be identical to what CentOS was: a downstream build of RHEL.

In other words, if things work as intended, you won't have to make a decision about testing and supporting Rocky. If you support RHEL, then support for Rocky is a given.

PTide
Product Manager
Posts: 6218
Liked: 680 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by PTide » 1 person likes this post

Hi,

Well, despite of whatever distro maintainers claim about compatibily, we still test all distros separately, even CentOS and RHEL. We don't announce anything to be supported just because the manufacturer says that it's 100% compatible, because sometimes we catch odd bugs on seemingly identical distros.

Thanks!

arundas
Lurker
Posts: 1
Liked: never
Joined: Jul 08, 2021 2:47 am
Full Name: Arun
Contact:

[MERGED] AlmaLinux 8.4 Support

Post by arundas »

CentOS 8 is going out of support soon. AlmaLinux and RockyLinux has already stepped in and released their versions already. But it seems like the latest Linux kernel 4.18.0-305 have issues with most of backup software like Acronis, R1Soft and Veeam.
https://imgur.com/a/8he1GQZ

Acronis (https://kb.acronis.com/content/68870) and R1Soft are providing their customers with workaround for the issue. Will Veeam be providing support for these OS/Kernel ?

HannesK
Veeam Software
Posts: 11530
Liked: 2204 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by HannesK »

Hello,
and welcome to the forums.

Please see the answers around the same topics above. If you are interested in more details, the sticky topic about kernel 5.8 and later documents the progress on these kernels veeam-agents-for-linux-mac-aix-solaris- ... 70158.html

Best regards,
Hannes

VAXman65
Influencer
Posts: 10
Liked: 4 times
Joined: Feb 18, 2015 1:40 pm
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by VAXman65 »

Today, I received the kmod-veeamsnap.x86_64 5.0.1.4493-1.el8 on both My CentOS 8.4 and Rocky Linux 8.4 systems. I had been using Veeam, though with my OS kernel trapped at 4.18.0-240.22.1.el8. So I re-enabled kernel updates, got 4.18.0-305.19.1.el8_4 and rebooted.

Running the Veeam backup worked perfectly! CentOS and Rocky Linux both worked exactly the same and both ran the backup to conclusion without complaint. Here is the job summary information my automation retuned for the Rocky Linux run:

Code: Select all

Backup session
   ID: {fba57ddf-7338-4329-aa5d-8f2012f5439c}
   Job name: kristall-all
   Job ID: {93a61224-a702-4624-90e1-00b2da035bab}
   State: Success
   Start time: 2021-10-08 12:21:42
   End time: 2021-10-08 12:22:10

2021-10-08 16:21:42 UTC [info] Job kristall-all started at 2021-10-08 16:21:42 UTC
2021-10-08 16:21:43 UTC [info] Preparing to backup
2021-10-08 16:21:45 UTC [info] Creating volume snapshot
2021-10-08 16:21:46 UTC [info] Starting incremental backup to Repository_1
2021-10-08 16:21:47 UTC [info] Backed up V0uv033 5 GB at 899.3 MB/s
2021-10-08 16:21:52 UTC [info] Backed up sdb 1 MB at 5.3 MB/s
2021-10-08 16:21:53 UTC [info] Backed up V3uv033 480 MB at 179.8 MB/s
2021-10-08 16:21:55 UTC [info] Backing up BIOS bootloader on /dev/sda
2021-10-08 16:21:56 UTC [info] Backed up sda 1 GB at 566.6 MB/s
2021-10-08 16:21:58 UTC [info] Backing up summary.xml
2021-10-08 16:22:00 UTC [info] Releasing snapshot
2021-10-08 16:22:05 UTC [info] Applying retention policy
2021-10-08 16:22:05 UTC [info] Full backup file merge completed successfully
2021-10-08 16:22:08 UTC [info] Executing post-job script
2021-10-08 16:22:10 UTC [info] Post-job script has been executed successfully
2021-10-08 16:22:10 UTC [info] Processing finished at 2021-10-08 16:22:10 UTC

PTide
Product Manager
Posts: 6218
Liked: 680 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by PTide »

Hi,

What do you use Rocky for? Desktop/Laptop/Server?

Thanks!

VAXman65
Influencer
Posts: 10
Liked: 4 times
Joined: Feb 18, 2015 1:40 pm
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by VAXman65 » 1 person likes this post

This was my prototype for migration from CentOS to Rocky. These are server instances running as KVM server VMs. In this case, I am moving my Samba AD domain controllers to the platform.

I have a number of server VMs running CentOS. Now that I have proven out Rocky and have a functioning backup solution, I’ll be migrating a number of CentOS 7 systems to Rocky 8.4

doublem
Enthusiast
Posts: 26
Liked: 7 times
Joined: Jul 01, 2016 4:07 pm
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by doublem »

Gostev wrote: Jun 28, 2021 3:43 pm If most of the former CentOS users migrate to Rocky Linux, then of course we will have no choice but to test and officially support our products on it. As CentOS had a decent market share before all the recent changes.
@Gostev

Thinking the other way around. If Veeam supports Rocky Linux, users may be adopting Rocky Linux at a larger scale.

Robert G.
Service Provider
Posts: 3
Liked: 3 times
Joined: Jun 20, 2019 5:59 pm
Location: Germany
Contact:

[MERGED] [Feature Request]: Support for AlmaLinux

Post by Robert G. » 3 people like this post

We would like to request that AlmaLinux be added to the list of supported operating systems. For the following components:
  • Veeam Backup & Replication
    • Backup Proxy Server
    • Backup Repository Server
    • Gateway Server
    • Veeam Agent Computer (Linux)
  • Veeam Agent
    • Veeam Agent for Linux

There are already a few threads where this topic has been discussed:
Like MySQL/MariaDB or ownCloud/Nextcloud, I don't think we will see a clear winner of this "successor battle" in the next few years.
  • RockyLinux seems to be the winner of the community heart
  • AlmaLinux scores with update speed, features, years of experience and professionalism in general

HannesK
Veeam Software
Posts: 11530
Liked: 2204 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by HannesK » 1 person likes this post

Hello,
I merged your request to the existing discussion. We count your request +1 👍

Best regards,
Hannes

saahi
Novice
Posts: 8
Liked: 3 times
Joined: Jan 17, 2017 1:55 pm
Contact:

Re: Rocky Linux now GA. Veeam support ???

Post by saahi »

Our linux guys would love to have support for Almalinux.

aj_potc
Expert
Posts: 103
Liked: 19 times
Joined: Mar 17, 2018 12:43 pm
Contact:

Re: Rocky Linux now GA. Veeam support ??? (also AlmaLinux)

Post by aj_potc » 2 people like this post

Unfortunately, CentOS users have got a real chicken-and-egg problem here.

Those of us running CentOS have to migrate to another distro before it goes out of support at the end of this year (just a few short weeks away).

But Veeam wants to wait until they know which OS has the most users before they announce support.

This means that we could switch our production servers to an OS that Veeam decides not to support in the future. I'd really not like to be in that situation.

It would be great if Veeam could give us some indication here, but I know that may not be a realistic expectation.

PTide
Product Manager
Posts: 6218
Liked: 680 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Rocky Linux now GA. Veeam support ??? (also AlmaLinux)

Post by PTide »

@aj_potc,

Which distro are you inclining towards? What about RHEL (allows for up to 16 machines in production for free).

Thanks!

aj_potc
Expert
Posts: 103
Liked: 19 times
Joined: Mar 17, 2018 12:43 pm
Contact:

Re: Rocky Linux now GA. Veeam support ??? (also AlmaLinux)

Post by aj_potc »

Probably Rocky Linux, as I've successfully converted a few systems already. But I wouldn't be against Alma; they sound like a professionally run operation that I could trust.

I really don't want to deal with the RHEL subscription model in order to keep systems updated, and I'm pretty close to the 16-system limit already. So this doesn't sound like a good path for us.

Thank you for your interest!

villapla
Lurker
Posts: 2
Liked: 2 times
Joined: Jan 21, 2021 9:42 am
Full Name: Juanjo Villaplana
Contact:

Re: Rocky Linux now GA. Veeam support ??? (also AlmaLinux)

Post by villapla » 1 person likes this post

Hi,

We are currently migrating our CentOS 8 servers to AlmaLinux, so we would like to see AlmaLinux as a supported OS for our Veeam infrastructure (Proxy Servers and Repository Servers ---including XFS reflink support---).

We have delayed the migration of our Proxy and Repository servers, but it would be nice to know the Veeam support road-map for RHEL 8 clones as CentOS 8 is about to reach its EOL.

Thanks in advance!

soehl
Enthusiast
Posts: 55
Liked: 8 times
Joined: May 09, 2011 12:43 pm
Full Name: Sebastian
Location: Germany
Contact:

Re: Rocky Linux now GA. Veeam support ??? (also AlmaLinux)

Post by soehl »

but it would be nice to know the Veeam support road-map for RHEL 8 clones as CentOS 8 is about to reach its EOL
+1

danswartz
Expert
Posts: 257
Liked: 30 times
Joined: Apr 26, 2013 4:53 pm
Full Name: Dan Swartzendruber
Contact:

Re: Rocky Linux now GA. Veeam support ??? (also AlmaLinux)

Post by danswartz »

+1 for me too. I have a small almalinux install that serves up a heartbeat datastore for Vsphere 6.7 in my home lab. I'd love to back it up, but... NOPE!

Post Reply

Who is online

Users browsing this forum: aalvarez and 4 guests