-
- Product Manager
- Posts: 6551
- Liked: 765 times
- Joined: May 19, 2015 1:46 pm
- Contact:
Veeam Agent for Linux and RHEL 7.5 crash
Issue
Using Veeam Agent for Linux 2.0 or earlier on the latest release of Red Hat Enterprise Linux (RHEL) 7.5 and its derivatives (CentOS 7.5, Oracle RHCK 7.5) causes the operating system to crash during backup snapshot creation.
Cause
veeamsnap kernel module used in Veeam Agent for Linux 2.0 and earlier is not compatible with newly released RHEL 7.5 kernel 3.10.0-862
More info
We are researching the issue in collaboration with RedHat. Status updates will be posted here and in KB2659
13.06.2018 UPDATE: The fix is available via support team.
03.07.2018 UPDATE: New 2.0.1 version that addresses the issue is available via repositories.
Using Veeam Agent for Linux 2.0 or earlier on the latest release of Red Hat Enterprise Linux (RHEL) 7.5 and its derivatives (CentOS 7.5, Oracle RHCK 7.5) causes the operating system to crash during backup snapshot creation.
Cause
veeamsnap kernel module used in Veeam Agent for Linux 2.0 and earlier is not compatible with newly released RHEL 7.5 kernel 3.10.0-862
More info
We are researching the issue in collaboration with RedHat. Status updates will be posted here and in KB2659
13.06.2018 UPDATE: The fix is available via support team.
03.07.2018 UPDATE: New 2.0.1 version that addresses the issue is available via repositories.
-
- Lurker
- Posts: 1
- Liked: never
- Joined: Apr 14, 2018 3:31 pm
- Full Name: Chris Buchleiter
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
RHEL 7.4 has been updated with the 3.10.0-862 kernel and is exhibiting the same issue.
-
- Product Manager
- Posts: 6551
- Liked: 765 times
- Joined: May 19, 2015 1:46 pm
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
Hi Chris,
Thank you for reporting that!
Thank you for reporting that!
-
- VP, Product Management
- Posts: 6034
- Liked: 2859 times
- Joined: Jun 05, 2009 12:57 pm
- Full Name: Tom Sightler
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
Does this happen even if the module is recompiled from source? I know our module uses some ABI calls that are not marked as stable and thus could break between kernel trains in RHEL7.
-
- Novice
- Posts: 5
- Liked: 1 time
- Joined: Apr 20, 2018 5:06 pm
- Full Name: Steve Linthicum
- Contact:
RHEL 7.5
I've been testing veeam linux agent on a RHEL 7.4 physical oracle server in preparation of the Oracle plugin becoming available later this year. Last week we did our normal monthly patch cycle on this dev host and it patched to RHEL 7.5. Now when I try to run a backup through veeam the server reboots. I tried uninstalling and reinstalling and setting the backup up from scratch but get the same behavior. Run a backup, reboot and backup fails.
Anyone else seeing this issue?
Anyone else seeing this issue?
-
- Product Manager
- Posts: 6551
- Liked: 765 times
- Joined: May 19, 2015 1:46 pm
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
Hi,
This is a known issue, we are working on the fix. Please note that even if you recompile the module from source it won't work on 3.10.0-862
Thanks
This is a known issue, we are working on the fix. Please note that even if you recompile the module from source it won't work on 3.10.0-862
Thanks
-
- Lurker
- Posts: 1
- Liked: never
- Joined: Apr 05, 2018 2:18 pm
- Full Name: Gary Lynch
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
I too am experiencing this problem. Have been evaluating the Agent on a server that is due to go into production soon and yesterday I performed some updates and now the server crashes. Do we know how long it will take to fix? This could dictate when our server can go into production or we use alternative backup methods.
Thanks
Gary
Thanks
Gary
-
- Product Manager
- Posts: 6551
- Liked: 765 times
- Joined: May 19, 2015 1:46 pm
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
Support for 7.5 will be introduced in the next VAL update which will take at least a month to be released.
Thanks
Thanks
-
- Novice
- Posts: 3
- Liked: never
- Joined: Apr 30, 2018 7:39 am
- Full Name: Thomas Gächter
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
hi all,
any news about update for veeamagent (Linux)???
btw. not just Redhat 7.5 are affectet ..... 7.4 with last kernel-update (specter fix!)too.
regards
any news about update for veeamagent (Linux)???
btw. not just Redhat 7.5 are affectet ..... 7.4 with last kernel-update (specter fix!)too.
regards
-
- Influencer
- Posts: 11
- Liked: 1 time
- Joined: Aug 01, 2016 4:11 pm
- Full Name: Tomas Hajek
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
I have 2 systems that are running RHEL 7.5 with Kernel 3.10.0-862.el7.x86_64 and VAL 2.0.0.400, one (Dell PowerEdge R720xd) has been successfully backing up for about 3 weeks and another (Dell PowerEdge R630) for 7 days. I have two other systems, however, (Dell PowerEdge R330's) that exhibit the same crashing behavior which lead me to this post today. Is this expected that some systems work and others don't? Should I submit a support request to include information on working vs. non-working systems? Is it know what the root cause of the issue is? Should I downgrade kernel versions to be on the safe side for my systems that are working (if so what version is safe)?
thanks
thanks
-
- Novice
- Posts: 5
- Liked: 1 time
- Joined: Apr 20, 2018 5:06 pm
- Full Name: Steve Linthicum
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
Support will tell you like they did me, RHEL 7.5 is not supported.
-
- Product Manager
- Posts: 6551
- Liked: 765 times
- Joined: May 19, 2015 1:46 pm
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
Hi,
If the module has been built via dkms on 7.4 system with kernel version < 3.10.0-862, then dkms will fail to rebuild the module after kernel update, BUT weak-updates will kick in and the module might keep working. This is not supported as we don't know potential pitfalls of such approach.
Thanks
We've observed such behaviour during tests. For now it can be described as follows:Is this expected that some systems work and others don't?
If the module has been built via dkms on 7.4 system with kernel version < 3.10.0-862, then dkms will fail to rebuild the module after kernel update, BUT weak-updates will kick in and the module might keep working. This is not supported as we don't know potential pitfalls of such approach.
Thanks
-
- Novice
- Posts: 6
- Liked: never
- Joined: Jun 12, 2017 1:34 pm
- Full Name: Ken Everett
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
I have also seen this issue on a test server that was recently updated to 7.5. For now we will have to delay updating the production servers until this is resolved which becomes a serious issue in another month. We will have to ask for an exception since the servers can't be patched as it will break backups.
-
- Product Manager
- Posts: 6551
- Liked: 765 times
- Joined: May 19, 2015 1:46 pm
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
Hi,
The issue will be addressed in the next release.
Thanks
The issue will be addressed in the next release.
Thanks
-
- Novice
- Posts: 3
- Liked: never
- Joined: Apr 30, 2018 7:39 am
- Full Name: Thomas Gächter
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
Hi,
and when is the delivery date of this new release?!
We get really in trouble with the auditor ..... because of "old" kernels .... or not existing backups!
regards
tom
and when is the delivery date of this new release?!
We get really in trouble with the auditor ..... because of "old" kernels .... or not existing backups!
regards
tom
-
- Product Manager
- Posts: 6551
- Liked: 765 times
- Joined: May 19, 2015 1:46 pm
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
Unfortunately I cannot provide you with any estimates except the one that I've mentioned before. The update will be released as soon as our QA team ensures that everything is in place and work stable.
Thank you
Thank you
-
- Lurker
- Posts: 1
- Liked: never
- Joined: May 17, 2018 2:03 pm
- Full Name: Gustavo Rivero
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
Its been a month since the error was identified. You stated a month for the fix. A month of no backups is significant in a a business. So there is no date on the fix?
-
- Product Manager
- Posts: 6551
- Liked: 765 times
- Joined: May 19, 2015 1:46 pm
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
Hi Gustavo,
In fact neither me nor anyone else has ever stated that the fix will be released within a month. What was stated 3 weeks ago is that "Support for 7.5 will be introduced in the next VAL update which will take at least a month to be released."
Thank you for your understanding
In fact neither me nor anyone else has ever stated that the fix will be released within a month. What was stated 3 weeks ago is that "Support for 7.5 will be introduced in the next VAL update which will take at least a month to be released."
Thank you for your understanding
-
- Chief Product Officer
- Posts: 31780
- Liked: 7280 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
Just to remind here as well, our commitment is to release support for new platform/OS/kernel versions within 90 days of general availability, with the actual historical average time being 2 months, depending primarily on complexity of required fixes. To avoid ending up in situations such as above (having no backup for weeks due to incompatibility), we recommend not to upgrade to new platform/OS/kernel versions before they are officially supported by the corresponding Veeam product. Thank you!
-
- Novice
- Posts: 5
- Liked: 1 time
- Joined: Apr 20, 2018 5:06 pm
- Full Name: Steve Linthicum
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
So what does that say about it also not working in the latest updated 7.4 which is fully supported and there is no 90 day wait?!?!
-
- Product Manager
- Posts: 5792
- Liked: 1214 times
- Joined: Jul 15, 2013 11:09 am
- Full Name: Niels Engelen
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
Steve, RHEL 7.4 has received the kernel update which causes this issue in the beginning of April. If you don't install this update, RHEL 7.4 systems will continue to work fine. Thanks!
Personal blog: https://foonet.be
GitHub: https://github.com/nielsengelen
GitHub: https://github.com/nielsengelen
-
- Lurker
- Posts: 1
- Liked: never
- Joined: May 21, 2018 8:14 pm
- Full Name: B H
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
Is there maybe a possible workaround? I just migrated to a new nfs server, and experiencing the backup veeam bug now.
Not sure why the client is tied to a kernel or a specific version of redhat/centos.
Not sure why the client is tied to a kernel or a specific version of redhat/centos.
-
- Product Manager
- Posts: 6551
- Liked: 765 times
- Joined: May 19, 2015 1:46 pm
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
VAL uses its own kernel module to operate, which depends on kernel's internal structures, one of which got modified.Not sure why the client is tied to a kernel or a specific version of redhat/centos.
Thanks
-
- Novice
- Posts: 3
- Liked: never
- Joined: Apr 30, 2018 7:39 am
- Full Name: Thomas Gächter
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
Hi,
this i can read in the docu:
Prebuilt binaries require kernel >= 2.6.32-131.0.15 for CentOS / RHEL 6 and kernel >= 3.10.0-123 for CentOS / RHEL 7 to function.
For other kernels please install the veeamsnap module using the source rpm package available at the repository.
i downloaded the veeamsnap from this repository ...... with the same error on dkms build!
It's now 6 weeks since this error is well known ...... please let your customers know, when they can get a solution. In this situation i have to think about a backup change to a other solution.
Just because when "Murphy" visit us and we need a restore to a date after 10. April 2018 (!!!) ..... we just are not able to do it ..... it starts really to be a businesses critical issue.
regards
this i can read in the docu:
Prebuilt binaries require kernel >= 2.6.32-131.0.15 for CentOS / RHEL 6 and kernel >= 3.10.0-123 for CentOS / RHEL 7 to function.
For other kernels please install the veeamsnap module using the source rpm package available at the repository.
i downloaded the veeamsnap from this repository ...... with the same error on dkms build!
It's now 6 weeks since this error is well known ...... please let your customers know, when they can get a solution. In this situation i have to think about a backup change to a other solution.
Just because when "Murphy" visit us and we need a restore to a date after 10. April 2018 (!!!) ..... we just are not able to do it ..... it starts really to be a businesses critical issue.
regards
-
- Chief Product Officer
- Posts: 31780
- Liked: 7280 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
Current target is mid-June, we will keep you updated if these timelines change - we're trying to accelerate the release by all means possible, but we cannot sacrifice stability testing in particular.
-
- Veteran
- Posts: 266
- Liked: 30 times
- Joined: Apr 26, 2013 4:53 pm
- Full Name: Dan Swartzendruber
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
Thanks, appreciate the heads-up. I was painfully bitten when I tried installing and running VAL on 3.10.0-862.3.2.el7.x86_64. Everything (apparently) installed okay, yet when I tried to do a backup, the system panicked and rebooted. Ugh...
-
- Chief Product Officer
- Posts: 31780
- Liked: 7280 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
Keep in mind there's a workaround for those who don't really NEED to use the problematic kernel update for whatever reason, and that is to simply go back to the previously used kernel until the new VAL version is released.
You can list all installed kernels using the following command:
Then, you should modify "/boot/grub2/grubenv" by changing the saved_entry to match the kernel you want to load by default.
You can list all installed kernels using the following command:
Code: Select all
awk -F\' /^menuentry/{print\$2} /etc/grub2.cfg
-
- Enthusiast
- Posts: 30
- Liked: 3 times
- Joined: Dec 20, 2016 6:39 am
- Full Name: Manuel Orsatti
- Location: Italy
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
if you don't like modifying "/boot/grub2/grubenv" by hand, you can instead use
where <number> is the position on the list obtained with awk (starting the count from 0)
Code: Select all
grub2-set-default <number>
-
- Chief Product Officer
- Posts: 31780
- Liked: 7280 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
Thanks Manuel, this certainly seems much less error-prone than what I suggested
-
- Novice
- Posts: 9
- Liked: never
- Joined: Feb 21, 2016 9:07 pm
- Full Name: Paul
- Contact:
Re: Veeam Agent for Linux and RHEL 7.5 crash
Hi everyone,
We find ourselves in the same boat. Luckily we can wait about 3 more weeks, maybe until after July 4. After that it's back to TSM if not fixed. From reading this, last news was maybe a month from April 27 so hopefully the update/fix is close. If any new news is available please let us know. Thanks!
We find ourselves in the same boat. Luckily we can wait about 3 more weeks, maybe until after July 4. After that it's back to TSM if not fixed. From reading this, last news was maybe a month from April 27 so hopefully the update/fix is close. If any new news is available please let us know. Thanks!
Who is online
Users browsing this forum: No registered users and 5 guests