Standalone backup agents for Linux and Unix workloads on-premises or in the public cloud
Post Reply
marcwa19197
Influencer
Posts: 19
Liked: never
Joined: Dec 05, 2019 1:30 pm
Contact:

[LINUX] Kernel 5.8 and later

Post by marcwa19197 »

Hello,

i know this is an "big topic" in this forum, i read much threads about it.
I just want to ask for a status for the Kernel 5.8 compatibility. As you might know, Fedora 33 was released a few days ago. Fedora 33 is using Kernel 5.8 as default.

Is there any timeline? If not, i will wait for official compatibility with Fedora 33.
At the moment im using Fedora 32 with an older Kernel, because Kernel 5.8 is not working at the moment.

I read something about an newsletter, how can i subscribe to it?

Thank you very much.

PTide
Product Manager
Posts: 5731
Liked: 573 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Fedora 33 - Kernel 5.8

Post by PTide »

Hi,
Is there any timeline? If not, i will wait for official compatibility with Fedora 33.
Unfortunately there is no timeline for now.
I read something about an newsletter, how can i subscribe to it?
I guess you mean "Word from Gostev"? Just click on a "wrench" icon under the topic name and select "Subscribe"

Thanks!

marcwa19197
Influencer
Posts: 19
Liked: never
Joined: Dec 05, 2019 1:30 pm
Contact:

Re: Fedora 33 - Kernel 5.8

Post by marcwa19197 »

Thank you for the information!
Im getting
"You are not authorised to read this forum."
if i follow your link..

PTide
Product Manager
Posts: 5731
Liked: 573 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Fedora 33 - Kernel 5.8

Post by PTide »

My bad. Please try "User Control Panel > "Board Preferences" tab > "Weekly Digest Subscription" section > "Subscribe to weekly digest" radio button.

Thanks!

marcwa19197
Influencer
Posts: 19
Liked: never
Joined: Dec 05, 2019 1:30 pm
Contact:

Re: Fedora 33 - Kernel 5.8

Post by marcwa19197 »

Thanks, but it was already checked.
No Problem - i will just wait for an announcement :-)

So just for high level planning, do you think Kernel 5.8 will be compatible this year? If not, i will check if i can go to Fedora 33 but using Kernel 5.7.

ascaris
Novice
Posts: 4
Liked: 1 time
Joined: May 08, 2019 6:16 am
Contact:

Re: Fedora 33 - Kernel 5.8

Post by ascaris »

He had said something about one of the major players in Linux being favorable to the idea of including whatever change was needed in their version of the kernel some time ago, if I recall. Has it been announced which one it was and how that was going?

Gostev
SVP, Product Management
Posts: 27411
Liked: 4541 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Fedora 33 - Kernel 5.8

Post by Gostev »

marcwa19197 wrote: Oct 30, 2020 8:07 amThanks, but it was already checked.
No Problem - i will just wait for an announcement :-)
It's a weekly digest. If you're not getting it, you should check your spam filters.

Gostev
SVP, Product Management
Posts: 27411
Liked: 4541 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Fedora 33 - Kernel 5.8

Post by Gostev » 1 person likes this post

marcwa19197 wrote: Oct 30, 2020 8:07 amKernel 5.8 will be compatible this year?
In general, kernel 5.8 or later versions will never be supported for an image-level backup. The next supported kernel version is currently unknown, because it is beyond our control. Veeam Agent for Linux depends on the critical functionality that was removed starting from kernel 5.8, which allowed us to perform the I/O filtering for snapshot and changed block tracking purposes. There are currently a few options on the table on how to address this, including upstreaming the actual drivers in question.

Here are all 4 ways of how this may play out:
1. One major distributions told us they are simply not going to include the breaking change into their custom kernel at all.
2. Another major distribution considers to keep including the kernel patch we provided temporarily, until either of the following two possibilities happen.
3. Linux kernel maintainers may decide to upstream our out of tree drivers into the Linux kernel. By now, most work on our side has be done to enable this, and the drivers are in review by the community.
4. Linux kernel maintainers may decide to restore the lost functionality in some shape or form. Reviewing our submission from (3) has sparkled the corresponding conversations between them.

This was the status as of about one week ago, I did not check for updates since then.

marcwa19197
Influencer
Posts: 19
Liked: never
Joined: Dec 05, 2019 1:30 pm
Contact:

Re: Fedora 33 - Kernel 5.8

Post by marcwa19197 »

Hello Gostev,

thank you very much for that information. At the moment i also receive the Newsletter! Thanks.

So i guess, there is still no update or timeline? Or maybe a Roadmap?

Gostev
SVP, Product Management
Posts: 27411
Liked: 4541 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Fedora 33 - Kernel 5.8

Post by Gostev » 2 people like this post

Here's my understanding - beg a pardon if I don't get certain terms right, as I'm far from the trenches.

Our developers have delivered the patch to restore the missing Linux kernel functionality, but were then told by maintainers that it cannot be accepted into the kernel without any "internal consumer" of its functionality. But, the community quickly determined device-mapper (dm) as the best candidate to become such a consumer.

So in the past month, our developers kept working with kernel maintainers, now on implementing the required changes into dm. However, the joint team is completely stuck at this time because the mission-critical Linux kernel maintainer for dm is currently busy with urgent work at his employer, and cannot spend any time on this until next year.

On a negative news side, the alpha version of RHEL 8.4 appears to have the entire block layer from kernel version 5.8 ported into their custom kernel based on version 4.18. This includes the "breaking change" (the kernel function loss that renders image-level backups impossible). This comes totally unexpected considering how conservative RedHat has been in terms of advancing their kernel version, and unfortunately also at the worst possible moment.

By the way, what is everyone's take on manually patching the kernel with the patch we implemented until it makes it into the tree sooner or later? I do realize it's a lot of work if you have many machines, but might be OK for Linux enthusiasts at least?

marcwa19197
Influencer
Posts: 19
Liked: never
Joined: Dec 05, 2019 1:30 pm
Contact:

Re: [LINUX] Kernel 5.8 and later

Post by marcwa19197 »

Hello Gostev,

thank you very much for this information!

Is there maybe a small instruction how we can patch our kernel manually? Im not an expert on this, but a few steps with commands would be really helpful!
(As it was on Github for Kernel 5.7 https://github.com/mike2307/veeamsnap)

Thank you and have a happy new year!

Finix
Lurker
Posts: 1
Liked: never
Joined: Dec 28, 2020 8:52 am
Full Name: FinixYan
Contact:

Re: [LINUX] Kernel 5.8 and later

Post by Finix »

If device registers submit_bio function in its block_device_operations, it is not hard to support in the current veeam snapshot framework, right? If so, it could at least make much of DM/MD (except multipath) device work.

PTide
Product Manager
Posts: 5731
Liked: 573 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: [LINUX] Kernel 5.8 and later

Post by PTide » 1 person likes this post

Is there maybe a small instruction how we can patch our kernel manually? Im not an expert on this, but a few steps with commands would be really helpful!
(As it was on Github for Kernel 5.7 https://github.com/mike2307/veeamsnap)
There is no patch right now as we are currently working on a prototype. Once we are ready to provide such patch I will post links and instructions in this thread.

Thanks!

KernelKurtz
Novice
Posts: 9
Liked: 3 times
Joined: Dec 08, 2020 5:20 pm
Full Name: Blair Steenerson
Contact:

Re: [LINUX] Kernel 5.8 and later

Post by KernelKurtz » 1 person likes this post

Since it looks like the Linux snapshot functionality is going to be broken shortly without this (what exactly are they thinking? - I know someone who uses Acronis in the enterprise which I believe uses the same functionality - seems unlike the Linux kernel maintainers to purposely break something important and widely used like this), I'd be willing to give the patch a try when it is available.

Will watch this space for more info. Many thanks!

ibschreiber
Influencer
Posts: 18
Liked: 3 times
Joined: Nov 21, 2017 11:51 am
Full Name: Michael Schreiber
Contact:

[MERGED] Ubuntu 20.04 backup stoped working - solution (kind of)

Post by ibschreiber » 1 person likes this post

Hi folks,
backup on my Ubuntu 2004 stoped working after Kernel was upgraded to the 5.8 branch. Bad, really bad. As Veeam can't do anything about it at this point, the only option was to revert back to 5.6. But I didn't want to loose the ability to use newer Kernels when necessarry. So my solution was to install the 18.04 kernels while still allowing Kernels from 20.04 to be installed. how?

1: Install the older Kernels:

Code: Select all

apt install linux-generic-hwe-18.04
(or any other version, the hwe works fine on my laptop)

2: Install veeam (install the .dep to configure the source in apt, then

Code: Select all

sudo apt install veeam
This fails miserably when dkms tries to built veeamsnap for the 5.8 kernel, so....

3: Go to /usr/src/veeamsnap-x.x.x.xxxx and edit the file dkms.conf
add the line

Code: Select all

BUILD_EXCLUSIVE_KERNEL="^5.4.*"
to it so the package will only be built for the 5.4 kernels.

another friendly

Code: Select all

sudo dpkg-reconfigure veeamsnap
and all should be fine.

4: To set the default kernel to the 5.4, I used grub configure and enabled the option to select the last booted kernel as default.
you can also edit /etc/default/grub and set/add

Code: Select all

GRUB_SAVEDEFAULT="true"
GRUB_DEFAULT="saved"
to it.

That's it. Now you can boot into the 5.4 to have your backup working, boot into 5.8 or any other kernels when needed and the apt update process is only broken when veeam releases a new version of veeamsnap. Just repeat Step 3 and you're good to go.

@VEEAM: As veeamsnap is broken on higher kernels anyway, could you please include BUILD_EXCLUSIVE_KERNEL="^5.4.*" (maybe a bit more generalized) in your next release of the linux version. This would really help!
Michael Schreiber
CEO Creatronics GmbH - Greding - Germany
Veeam user and VCSP

marcwa19197
Influencer
Posts: 19
Liked: never
Joined: Dec 05, 2019 1:30 pm
Contact:

Re: [LINUX] Kernel 5.8 and later

Post by marcwa19197 »

PTide wrote: Dec 29, 2020 1:36 pm There is no patch right now as we are currently working on a prototype. Once we are ready to provide such patch I will post links and instructions in this thread.

Thanks!
Thank you very much!
Will also wait and watch this thread for a update.

HannesK
Veeam Software
Posts: 6794
Liked: 1051 times
Joined: Sep 01, 2014 11:46 am
Location: Austria
Contact:

Re: [LINUX] Kernel 5.8 and later

Post by HannesK »

@ibschreiber: I merged your post with the existing discussion.
Now you can boot into the 5.4 to have your backup working, boot into 5.8 or any other kernels when needed
I'm not sure how customers should handle that in practice. Who would reboot the system for backup?

ibschreiber
Influencer
Posts: 18
Liked: 3 times
Joined: Nov 21, 2017 11:51 am
Full Name: Michael Schreiber
Contact:

Re: [LINUX] Kernel 5.8 and later

Post by ibschreiber »

Well I'm working using Kernel 5.4. Jus for trying things out I switch to kernel 5.8. For servers this could be a way to get the backup working again. I think it depends on the requirements in each individual case.
Michael Schreiber
CEO Creatronics GmbH - Greding - Germany
Veeam user and VCSP

PTide
Product Manager
Posts: 5731
Liked: 573 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: [LINUX] Kernel 5.8 and later

Post by PTide »

For veeamsnap to work on kernels 5.8 and later it is required to add blk_interposer functionality.

This feature is being developed in blk-interposer.v2 branch git://git.kernel.org/pub/scm/linux/kernel/git/hare/scsi-devel. One needs only kernel changes (skip the device-mapper).

Therefore, only these two commits are required:

commit 8dddc3dd22db58c690ee2916266b136a1b80875b
Date: Thu Nov 19 17:49:23 2020 +0100
blk_interposer - Block Layer Interposer


commit 71e21b5e3941f4c0cacad47cc119141ca7fd2a66
Date: Fri Dec 11 19:01:19 2020 +0300
block: blk_interposer - change attach/detach logic


If are not into having too much fun with git, you can use the prepared patches from this branch:

0001-blk_interposer-Block-Layer-Interposer.patch
0003-block-blk_interposer-change-attach-detach-logic.patch


or use this cumulative patch:

blk_interposer_cumulative.patch


In order to install a stable vanilla kernel (5.10 for example) you have to download it first:

Code: Select all

 wget https://cdn.kernel.org/pub/linux/kernel/v5.x/linux-5.10.3.tar.xz 
Unpack:

Code: Select all

 tar -xf ./linux-5.10.3.tar.xz 
Apply the patch:

Code: Select all

 cd ./linux-5.10.3
 patch -p1 <../0001-blk_interposer-Block-Layer-Interposer.patch
 patch -p1 <../0003-block-blk_interposer-change-attach-detach-logic.patch
Or apply the cumulative patch:

Code: Select all

 patch -p1 <../blk_interposer_cumulative.patch
Next, configure and build the kernel (as usual), and install it. blk_interposer does not require any additional configuration.

We recommend to build a package (deb or rpm, depending on your packet manager) and install the kernel as a package. For example, "make -j4 deb".

Post Reply

Who is online

Users browsing this forum: No registered users and 4 guests