No character device registered for veeamsnap kernel module.

[ejenner]

I have a Linux CentOS server which recently stopped backing up. Has been failing for about a week now. I've tried upgrading Veeam to the latest version and reinstalling the agent but that's made no difference.

The error message is "No character device registered for veeamsnap kernel module."

At a guess I'd say OS has performed some sort of automatic update and made itself incompatible.

Has anyone seen this before and any ideas how to resolve?

[wishr]

Hi EJ,

What are your OS and kernel versions?

Thanks

[ejenner]

Sorry for the late reply, our Linux guy says:

Version/Kernel 3.10.0 (bug fix 1062)
64bit

[wishr]

Thanks.

Could you please make sure that your Agent has 3.0.2.1190 build since only this one supports kernel 3.10.0-1062 or higher. If you are already on 3.0.2.1190, I'd recommend continuing the investigation with our support team.

[ejenner]

I recently updated our environment to 4b. Presumably the agent would be the latest version in that case? I've removed and reinstalled the agent since upgrading the Veeam version.

[marcinbojko]

Latest version of agent available for 4b was 3.0.2.1185
Is there a way how to force Veeam B&R to use newer agent, without manually replacing execs?

[wishr]

This is correct, and, unfortunately, this can only be done by manually replacing the executables on the backup server. EJ, I'd suggest doing this in cooperation with our engineers to make sure the things will go smoothly.

[dguinn]

I just worked through a similar problem with support, and the fix was to install dkms first, uninstall the veeam agent, then install veeamsnap, then install veeam. I was running on Oracle Linux 7.6 with the UEK kernel...but it is likely the same for CentOS.

[wishr]

Hi Dan,

You are talking about standalone Agent, correct?

[dguinn]

either one actually. AFAIK they are essentially the same...but I did install it at first standalone, then tried via the console. Neither worked until I did the steps listed above.

[wishr]

Yes, they are essentially the same, but there is a way to upload VAL 3.0.2.1190 distributives to a B&R 9.5 U4b server, so it will be possible to deploy VAL to target machines using Agent Management functionality. Our support engineers should be able to assist with this.

The procedure you described above usually applies to standalone VAL installations.

[dguinn]

That's what the support engineer I worked with walked me through.

[wishr]

What was your support case ID?

[dguinn]

03813728. The phone conversation was to install DKMS, then if that was unsuccessful by itself, to reinstall using the install guide steps.

[ejenner]

The frustrating thing with this is that it had been working perfectly. Just suddenly started to fail and hasn't backed up since.

Here is the KB describing the fix... although it still says at the bottom that support will guide you through it: https://www.veeam.com/kb2804

I'm waiting for them to get back to me.

[wishr]

The kernel you have now (3.10.0-1062) after you CentOS applied updates automatically was not yet released at the time when the team was building and testing 3.0.2.1185. It's not something uncommon, especially in Linux world, when a new platform update breaks our functionality - we've seen this many times and this is one of the reasons why we regularly update our components - to make sure any new release is supported by Veeam and you can continue backing up your crucial workloads.

We always recommend raising a support case at the same day/hour when you started experiencing an issue to make sure you get a solution ASAP, especially in cases when there is a new not yet known issue (does not apply to this particular case, however), so we'll be able to start developing and testing a hotfix to ensure quick delivery.

Yes, the KB you mentioned describes exactly the case with 3.10.0-1062 release. Standalone agents can be updated manually, while Managed Agents can be updated centrally. The latter requires some manual interventions, though, to update the distibs on the B&R server and we strongly recommend doing this together with our support engineers. In case something goes unexpected way, they will be able to revert changes and quickly consult/assist you since there are specific procedures and SLAs in place. Just in general, the technical support team is the first place to contact when you faced an issue.

I checked one more time the case @dguinn had. And his issue had a bit different nature: his kernel version was supported by the current Agent version, but just DKMS and kernel headers were missing.

As far as I see, in the final, the issue was resolved using the method I described above and I'm happy to see our solution worked for you.

Thanks

[ejenner]

Here is the solution from support. Have implemented this and our Linux backups are working again.

This procedure can only be used with B&R 9.5 U4b and VAL 3.0.2.1190.

"To deploy the Agent 3.0.2.1190 to managed RHEL 7.7 systems, first please download the following packages from repository.veeam.com :

http://repository.veeam.com/backup/linu ... x86_64.rpm (precompiled veeamsnap module)

http://repository.veeam.com/backup/linu ... noarch.rpm (dkms veeamsnap module)

http://repository.veeam.com/backup/linu ... x86_64.rpm (main VAL package)

After that, please determine what is your Distribution server - https://helpcenter.veeam.com/docs/backu ... l?ver=95u4 . Usually, by default Veeam Backup Server is used in this role. On the Distribution server, please navigate to C:\Program Files\Veeam\Veeam Distribution Service\Fixes\val\x64\rpm and put all three downloaded RPM packages there.

Once the packages are on the Distribution server, please run a rescan of protection group - the updated packages should be recognized by Veeam and appropriate hosts (RHEL-based, x64 architecture) should be either upgraded or marked for upgrade depending on the protection group settings."

[marcinbojko]

Thanks, it worked - both procedure and 3.0.2.1190 on Centos 7.7

[KernelKurtz]

Bumping this thread as it describes my problem as of today. Backups have been fine for quite a while, but server picked up updates last night and backup failed this AM. Running VAL in standalone mode on CentOS 8. Fails with the above error "No character device registered for veeamsnap kernel module" New kernel version is 4.18.0-240.1.1.el8_3.x86_64 #1 SMP Thu Nov 19 17:20:08 I assume I have to wait for an updated veeamsnap module to make it to the repos? Upgrading currently says nothing to do.

[wishr]

Hi Blair,

The recently released Cumulative Patch 20201202 for VBR 10a should fix the issue.

Thanks

[KernelKurtz]

Good morning, thanks for the reply Fedor. I'm not running Backup and Replication server, just Linux Agent in free standalone mode. The existing module is kmod-veeamsnap-4.0.1.2365-1.el8.x86_64. According to your link, the latest version should be 4.0.1.2372 - "RHEL 8.3 and SLES 15 SP2 distributions support by Veeam Agent for Linux 4.0.1.2372 (included in the update)". It's not available via a regular package update right now, so hoping it will be soon. I'll keep checking. Best regards.

[wishr]

Good morning, Blair, and thanks for clarifying.

We are planning to upload the new package to the repositories but I cannot share any ETA as of now.

[KernelKurtz]

Very good Fedor, much appreciated. That is what I figured.

Have a good one and stay safe.

[wishr]

Stay safe as well!

[PTide]

@KernelKurtz,

Until we update the repository, you might want to extract VAL packages from VBR ISO. You need to unpack VALRedist.msi (located in <ISO>:\Packages\VALRedist.msi):

Code: Select all

msiexec /a valredist.msi

Thanks!

[KernelKurtz]

Thanks PTide. That is a neat trick, learned something new today! I don't work with Windows a lot.

The ISO I downloaded is from August 04 however and the .rpms I extracted are still v. 2365. Is there a newer ISO somewhere with the 2372 rpms?

Cheers!

[matthiasvdc]

Same here, downloaded installation ISO today, installed it on a new server. Added Centos 8 nodes to a backup job, got the same errors.
The Linux agent version included is still 4.0.1.2365-1.e18.x86_64

[wishr]

Hi Everyone,

As per this week Gostev's forum digest, the updated ISO containing Cumulative Patch 20201202 changes is not on the website yet. Stay tuned - we are planning to upload the new version in the upcoming days.

Thanks

[Gostev]

Or, just install the patch no need to wait for an ISO.

[matthiasvdc]

Just to inform that the patch worked. Warning: I had to manually push the veeam upgrade to the linux clients.