Standalone backup agents for Linux, Mac, AIX & Solaris workloads on-premises or in the public cloud
Post Reply
aj_potc
Expert
Posts: 141
Liked: 35 times
Joined: Mar 17, 2018 12:43 pm
Contact:

Veeam Agent for Linux 6 volume backups fail for RHEL 8.7

Post by aj_potc » 1 person likes this post

After updating Veeam Linux agents to the latest version, I'm unable to make volume backups under RHEL 8.7.

Every attempt throws the error:

Code: Select all

Failed to load module [veeamsnap]
To test further, I've spun up a fresh VM, installed RHEL 8.7 from ISO, and then started VAL 6 in workstation mode. When I start a volume backup, I get the same error. This version of RHEL and its kernel should be supported, and worked fine with VAL 5.

I just wanted to see if the dev team is aware of the issue, and if a fix or workaround is known.

I opened a support ticket (case #05879296) with my findings and with logs attached.

Thanks!
HannesK
Product Manager
Posts: 14314
Liked: 2888 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: Veeam Agent for Linux 6 volume backups fail for RHEL 8.7

Post by HannesK »

Hello,
let's see what support finds out. One of the servers had secure boot enabled.

Which of the servers had exactly the same kernel worked before with VAL5? Because 4.18.0-425.10.1 is not listed for VAL 5 in https://www.veeam.com/kb2804 (update of that article for V6 is ongoing)

Best regards,
Hannes
aj_potc
Expert
Posts: 141
Liked: 35 times
Joined: Mar 17, 2018 12:43 pm
Contact:

Re: Veeam Agent for Linux 6 volume backups fail for RHEL 8.7

Post by aj_potc »

Hi Hannes,

One system was updated to kernel 4.18.0-425.10.1 recently, and I believe it was running on that version successfully with VAL 5. I'll see what support says about it.

But I have another test system (in this case Rocky Linux) which is running an older kernel, 4.18.0-372.9.1. That was also working fine with VAL 5, but now fails with VAL 6. The error log for that system is very similar to the one for RHEL, which has the following lines:

Code: Select all

vsnap    |   Checking the compatibility of the kernel module with others
vsnap    |   Loading kernel module [veeamsnap] with parameters [zerosnapdata=1 debuglogging=0 snapstore_block_size_pow=14 change_tracking_block_size_pow=18 logdir=/var/log/veeam fixflags=0 logmaxsize=15728640] and loader [/usr/sbin/veeamsnap-loader].
But with one very odd addition that makes no sense to me:

Code: Select all

vsnap    | WARN|Your Linux kernel version is higher than the version this product build has been tested against, which may cause issues with some functionality.
Any idea why that line would show up for a kernel that is compatible?

After that line, I see the following:

Code: Select all

vsnap    |   Module [veeamsnap] has been loaded.
vsnap    | ERR |The module [veeamsnap] was not loaded.
I realize Rocky Linux is not yet supported, but I find it odd that an older kernel that worked fine with VAL 5 wouldn't work properly under VAL 6. Any ideas about that?

Thanks very much for your help.
VAXman65
Influencer
Posts: 13
Liked: 7 times
Joined: Feb 18, 2015 1:40 pm
Contact:

Re: Veeam Agent for Linux 6 volume backups fail for RHEL 8.7

Post by VAXman65 »

I opened Case #05880975 — Linux system backup job fails with Failed to create volume snapshot

Key part of the job log appears to be:

Code: Select all

[16.02.2023 05:30:46.939] <140423288297216> lpbcore  |   Taking snapshots.
[16.02.2023 05:30:46.939] <140423288297216> vmb      |     [SessionLog][processing] Creating volume snapshot.
[16.02.2023 05:30:53.242] <140423288297216> prtcl    | WARN|Method invocation was not finalized. Method id [1]. Class: [lpbcorelib::interaction::ISnapshotOperation]
[16.02.2023 05:30:53.242] <140423288297216> prtcl    | ERR |Failed to load module [veeamsnap].
[16.02.2023 05:30:53.242] <140423288297216> prtcl    | >>  |--tr:Failed to create Veeam snapshots.
[16.02.2023 05:30:53.242] <140423288297216> prtcl    | >>  |--tr:in void lpbcorelib::service::CSnapshotService::CreateVeeamSnapshots(const SVeeamSnapDeviceRequests&, const CSnapshotResourcePtr&, const Ptr&) at /mnt/Sources/LPBCoreLib/lpbcorelib/service/SnapshotService.cpp:146
[16.02.2023 05:30:53.242] <140423288297216> prtcl    | >>  |--tr:Failed to finish snapshot creation process.
[16.02.2023 05:30:53.242] <140423288297216> prtcl    | >>  |--tr:in virtual lpbcorelib::interaction::ISnapshotResourceLockPtr lpbcorelib::service::CSnapshotOperation::EndCreateSnapshot(lpbcorelib::service::CSnapshotTask::Ptr&) at /mnt/Sources/LPBCoreLib/lpbcorelib/service/SnapshotOperation.cpp:33
[16.02.2023 05:30:53.242] <140423288297216> prtcl    | >>  |--tr:Failed to execute method [1] for class [lpbcorelib::interaction::ISnapshotOperation].
[16.02.2023 05:30:53.242] <140423288297216> prtcl    | >>  |--tr:in void remoting::CStubBase<TStub, TObject>::ExecuteMethod(remoting::method_id_t, remoting::IStub::SDispatchContext&) const [with TStub = lpbcorelib::interaction::proxystub::CSnapshotOperationStub; TObject = lpbcorelib::interaction::ISnapshotOperation; remoting::method_id_t = unsigned int] at /mnt/Sources/LPBCoreLib/lpbcorelib/interaction/proxystub/./../../../ProtocolLib/remoting/StubBase.h:77
[16.02.2023 05:30:53.242] <140423288297216> prtcl    | >>  |--tr:Failed to invoke method [1] in class [lpbcorelib::interaction::ISnapshotOperation].
[16.02.2023 05:30:53.242] <140423288297216> prtcl    | >>  |--tr:in void remoting::CProxyBase::CMethodInvocationStream<TObject>::FinalizeExc() [with TObject = lpbcorelib::interaction::ISnapshotOperation] at /mnt/Sources/LPBCoreLib/lpbcorelib/interaction/proxystub/../../../../ProtocolLib/remoting/ProxyBase.h:92
[16.02.2023 05:30:53.242] <140423288297216> prtcl    | >>  |An exception was thrown from thread [140423288297216].
[16.02.2023 05:30:53.242] <140423288297216> vmb      |     [SessionLog][error] Failed to create volume snapshot.
[16.02.2023 05:30:53.243] <140423288297216> vmb      |     [SessionLog][error] Failed to take volume snapshot.
[16.02.2023 05:30:53.244] <140423288297216> lpbcore  |   Taking snapshots. Failed.
[16.02.2023 05:30:53.244] <140423288297216> vmb      |   [SessionLog][info] Executing post-job script.
[16.02.2023 05:30:53.245] <140423288297216> lpbcommon|   Executing custom script: [/usr/local/sbin/veeam_backup_status-email.sh]. Arguments: []
[16.02.2023 05:30:53.245] <140423279904512>          | Thread started.  Role: 'script output redirector to buffer', thread id: 140423279904512, parent id: 140423288297216.
[16.02.2023 05:30:53.245] <140423288297216> lpbcommon|     Wait child process for finish.
[16.02.2023 05:30:53.245] <140423271511808>          | Thread started.  Role: 'script output redirector to log', thread id: 140423271511808, parent id: 140423288297216.
[16.02.2023 05:30:53.245] <140423263119104>          | Thread started.  Role: 'Pid waiter. timeout: 600', thread id: 140423263119104, parent id: 140423288297216.
[16.02.2023 05:30:53.249] <140423263119104>          | Thread finished. Role: 'Pid waiter. timeout: 600'.
[16.02.2023 05:30:54.246] <140423271511808>          | Thread finished. Role: 'script output redirector to log'.
[16.02.2023 05:30:54.246] <140423279904512>          | Thread finished. Role: 'script output redirector to buffer'.
[16.02.2023 05:30:54.247] <140423288297216> lpbcore  | BackupJobPerformer: Creating backup. Failed.
[16.02.2023 05:30:54.247] <140423288297216> vmb      | [SessionLog][error] Failed to perform backup.
[16.02.2023 05:30:54.247] <140423288297216> vmb      | [SessionLog][error] Failed to load module [veeamsnap].
[16.02.2023 05:30:54.248] <140423288297216> vmb      | [SessionLog][error] Processing finished with errors at 2023-02-16 10:30:54 UTC.
[16.02.2023 05:30:54.248] <140423288297216> lpbcore  | ERR |Failed to load module [veeamsnap].
[16.02.2023 05:30:54.248] <140423288297216> lpbcore  | >>  |--tr:Failed to create Veeam snapshots.
[16.02.2023 05:30:54.248] <140423288297216> lpbcore  | >>  |--tr:in void lpbcorelib::service::CSnapshotService::CreateVeeamSnapshots(const SVeeamSnapDeviceRequests&, const CSnapshotResourcePtr&, const Ptr&) at /mnt/Sources/LPBCoreLib/lpbcorelib/service/SnapshotService.cpp:146
[16.02.2023 05:30:54.248] <140423288297216> lpbcore  | >>  |--tr:Failed to finish snapshot creation process.
[16.02.2023 05:30:54.248] <140423288297216> lpbcore  | >>  |--tr:in virtual lpbcorelib::interaction::ISnapshotResourceLockPtr lpbcorelib::service::CSnapshotOperation::EndCreateSnapshot(lpbcorelib::service::CSnapshotTask::Ptr&) at /mnt/Sources/LPBCoreLib/lpbcorelib/service/SnapshotOperation.cpp:33
[16.02.2023 05:30:54.248] <140423288297216> lpbcore  | >>  |--tr:Failed to execute method [1] for class [lpbcorelib::interaction::ISnapshotOperation].
[16.02.2023 05:30:54.248] <140423288297216> lpbcore  | >>  |--tr:in void remoting::CStubBase<TStub, TObject>::ExecuteMethod(remoting::method_id_t, remoting::IStub::SDispatchContext&) const [with TStub = lpbcorelib::interaction::proxystub::CSnapshotOperationStub; TObject = lpbcorelib::interaction::ISnapshotOperation; remoting::method_id_t = unsigned int] at /mnt/Sources/LPBCoreLib/lpbcorelib/interaction/proxystub/./../../../ProtocolLib/remoting/StubBase.h:77
[16.02.2023 05:30:54.248] <140423288297216> lpbcore  | >>  |--tr:Failed to take volume snapshot.
[16.02.2023 05:30:54.248] <140423288297216> lpbcore  | >>  |--tr:in static lpbcorelib::interaction::ISnapshotResourceLockPtr lpbcorelib::backup::CSnapshotHelper::TakeSnapshot(vmbplatform::model::CBackupJobPtr, const std::__cxx11::list<boost::shared_ptr<lpbcorelib::backup::IBackupObject> >&, const object_id_t&, vmbplatform::model::CSessionLoggerPtr, lpbcorelib::backup::CCbtDataContainer&, bool) at /mnt/Sources/LPBCreLib/lpbcorelib/backup/snapshot/SnapshotHelper.cpp:135
[16.02.2023 05:30:54.248] <140423288297216> lpbcore  | >>  |Backup job has failed.
[16.02.2023 05:30:54.248] <140423288297216> lpbcore  | >>  |--tr:in virtual void lpbcorelib::backup::CBackupJobPerformer::execute() at /mnt/Sources/LPBCoreLib/lpbcorelib/backup/BackupJobPerformer.cpp:210
[16.02.2023 05:30:54.248] <140423288297216> lpbcore  | >>  |An exception was thrown from thread [140423288297216].
[16.02.2023 05:30:54.250] <140423662724032>          |   Closing socket device.
[16.02.2023 05:30:54.250] <140423662724032> lpbcore  | LpbManSession: Processing commands. ok.
[16.02.2023 05:30:54.250] <140423662724032>          | Closing socket device.
[16.02.2023 05:30:54.273] <140423288297216> lpbcore  | JOB STATUS: FAILED.
VAXman65
Influencer
Posts: 13
Liked: 7 times
Joined: Feb 18, 2015 1:40 pm
Contact:

Re: Veeam Agent for Linux 6 volume backups fail for RHEL 8.7

Post by VAXman65 »

My fail is actually on Rocky 8.7, but it appears to be all the same.

[root@hendrick ~]# uname -a
Linux hendrick.lan.kitsnet.us 4.18.0-425.10.1.el8_7.x86_64 #1 SMP Thu Jan 12 16:32:13 UTC 2023 x86_64 x86_64 x86_64 GNU/Linux
[root@hendrick ~]# cat /etc/rocky-release
Rocky Linux release 8.7 (Green Obsidian)
[root@hendrick ~]# veeamconfig --version
v6.0.0.1060
PTide
Product Manager
Posts: 6428
Liked: 729 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Veeam Agent for Linux 6 volume backups fail for RHEL 8.7

Post by PTide »

Hello everyone,

Kindly check whether you have installed the updated UEFI certificate for SecureBoot:

http://repository.veeam.com/backup/linu ... noarch.rpm

Please let us know if it helped.

Thank you!
Gostev
Chief Product Officer
Posts: 31521
Liked: 6699 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Agent for Linux 6 volume backups fail for RHEL 8.7

Post by Gostev »

aj_potc
Expert
Posts: 141
Liked: 35 times
Joined: Mar 17, 2018 12:43 pm
Contact:

Re: Veeam Agent for Linux 6 volume backups fail for RHEL 8.7

Post by aj_potc » 1 person likes this post

In my case, the solution to this problem was different for each of three systems.

On one system, the presence of Secure Boot prevented Veeam from working. I was able to fix that by disabling Secure Boot.

On another system, the veeamsnap module was not properly installed. A reinstallation with yum resolved that.

And in the third case, a Rocky Linux system, I discovered that the veeamsnap module checks for the OS ID value in /etc/os-release, and will not load if it finds "rocky" instead of "rhel".
PTide
Product Manager
Posts: 6428
Liked: 729 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Veeam Agent for Linux 6 volume backups fail for RHEL 8.7

Post by PTide » 1 person likes this post

nd in the third case, a Rocky Linux system, I discovered that the veeamsnap module checks for the OS ID value in /etc/os-release, and will not load if it finds "rocky" instead of "rhel".
Unfortunately Rocky (and other CentOS substitutes) is not supported yet. If you are confident that it will work fine, and really want to try, then use dkms module please.

Thank you!
Post Reply

Who is online

Users browsing this forum: No registered users and 6 guests