[LINUX] Kernel 5.16 and later, veeamsnap support

[PhoenixHGIC]

Unfortunately the update has already taken place and rolling back the Kernel is by no means straightforward.
Also due to our security/ compliance commitments as a company, we have to keep up to date with latest releases.

Which distro are you using?

We are using Ubuntu 20.10 (GNU/Linux 5.8.0-43-generic x86_64)

[Ejdesgaard]

The latest news can be seen here:
https://patchwork.kernel.org/project/li ... ies=427209

Associated correspondence on lkml.org, for those who are interested

https://lkml.org/lkml/2021/2/3/797

Good to see progress

[PTide]

@PhoenixHGIC,
VAL works with kernels 5.8 if you use the latest version form this experimеntal branch:
https://github.com/veeam/veeamsnap/tree/experimental
We plan to include it into the next update.

If you would like to request an official fix now, please contact our support team directly and make an inquiry.

Please note the same statements don't apply to 5.9 or later, so be sure not to upgrade to those until further notice.
But in 5.8, there are still some "left-overs" of the removed kernel functionality that enabled us to support it.

Thanks!

[marcwa19197]

Any news on this?
Im following https://patchwork.kernel.org/project/li ... rder=-date but i dont know how they are updating the information....

[Gostev]

Guys, I will share any significant updates here the moment I receive the news. Until then, consider the latest status update from me to be the current state.

[toxic9]

Hi! I have opened an incident on this subject and I never received an answer.
My question is simple, will ever be a new veeam for linux version with this problem fixed?
or will we keep on relying on workarounds from now on?
thanks

[marcwa19197]

As i understand, as soon as the needed kernel-patch is implemented by the linux kernel-team, veeam can work like before on Kernel < 5.8

[PTide]

Hi,

@toxic9,

My question is simple, will ever be a new veeam for linux version with this problem fixed?
or will we keep on relying on workarounds from now on?

The fix needs to be implemented both in VAL and in the Linux kernel. As soon as the kernel patch series that Veeam has developed to restore the required kernel functionality is accepted into the kernel, the VAL is expected to start working out of the box on those kernels that implement the patch.

Thanks!

[toxic9]

so if I understood correctly, the kernel devs should "accept" these changes?
I thought it was only depending on veeam devs.

thanks for your explanation.

[PTide]

Roughly speaking - yes. In order for the whole fix to work the kernel changes proposed by us have to be accepted by the kernel devs. Veeam team, in turn, has to write the kernel patch in such way that it satisifes the kernel maintainers' "sense of beautiful" : ) Well, jokes aside - the (linux) community has pretty strict standards of how things should be done, and also certain technical requirements have to be met.

Thanks!

[marcwa19197]

Im just reverted back to kernel 5.7.

On Fedora 33 just do the following (you need koji installed):

1.) koji download-build --arch=x86_64 kernel-5.7.0-1.fc33
2.) sudo dnf install kernel-5.7.0-1.fc33.x86_64.rpm kernel-core-5.7.0-1.fc33.x86_64.rpm kernel-devel-5.7.0-1.fc33.x86_64.rpm kernel-modules-5.7.0-1.fc33.x86_64.rpm
3.) reboot

This will download Kernel 5.7.0 and install it. After that you need an reboot and Veeam will work again.
I will wait till everything is ready for the newer kernels.

Thank you very much!

[Ejdesgaard]

Is the expectation that the patches will be approved/pulled for the 5.12 merge window or should we expect it to be pushed for 5.13 or later ?

[Gostev]

Since we have absolutely no control over the approval part, we don't have ANY expectations at the moment...

[PhoenixHGIC]

@PhoenixHGIC,
VAL works with kernels 5.8 if you use the latest version form this experimеntal branch:
https://github.com/veeam/veeamsnap/tree/experimental
We plan to include it into the next update.

If you would like to request an official fix now, please contact our support team directly and make an inquiry.

Please note the same statements don't apply to 5.9 or later, so be sure not to upgrade to those until further notice.
But in 5.8, there are still some "left-overs" of the removed kernel functionality that enabled us to support it.

Thanks!

Sorry to be a pain, I logged a case with support as soon as I got your message. They tell me that they are still liaising with you. Is there anything you can do to progress this?

[PTide]

The internal process has come to a final conclusion just today, the support engineer will update the case shortly.

Thanks!

[marcwa19197]

Since we have absolutely no control over the approval part, we don't have ANY expectations at the moment...

No Problem Gostev.
Thank you Veeam for having also product management looking into it and also giving updates in the forum.

[marcwa19197]

Hello,

i guess there is still no update or progress? :-/

Happy easter!

[HannesK]

Hello,
there is progress, but nothing finalized yet.

Happy easter!
Hannes

[Gostev]

The good news is, more and more maintainers are getting involved to review the code and provide feedback, so our developers have been super busy lately implementing all the suggestions. Feels like we're finally getting somewhere, comparing to the crickets situation a few months ago.

[SRMETAS]

Kernel 5.8 is out for at least 6 Months and veeam v11 is brand new and still does not work with it? At least veeamsnap is not working on 5.8. and i have to backup on filelevel which is not really nice!

[Mildur]

This is not Veeams Fault. Veeam depends here on the kernel maintainers.
See the newest development in this case from Antons Newsletter last weekend:

Linux kernel 5.8+ support > overall, the status remains more or less the same: our developers are iterating the new kernel module quickly based on feedback from maintainers. But there are some significant news nevertheless: after talking to Oracle developers last week, we found we're not the only major vendor impacted by the make_request_fn function removal from Linux kernel. Apparently, they too are using this function for I/O interception in their ASMFD (ASM Filter Driver) kernel module. So we showed them the current code of the replacement functionality we developed, and they really liked it. While I'm just happy to see another major vendor having the same need, as this significantly increases the chances of upstreaming the code we developed soon!

[marcwa19197]

Fedora 34 was released a few days ago.
I already upgraded to it. Is there still no news to share? :-/

[HannesK]

Hello,
we can hardly influence the speed of implementation into the Linux kernel. So there won't be real news (except "it's in progress") until it's implemented in the Linux kernel.

Best regards,
Hannes

[PTide]

Hello everyone,

Please make sure to check the latest update:

https://github.com/veeam/veeamsnap/issues/10

We seem to have found a workaround that allows the usage of veeamsnap without patching the kernel (at least with its current versions).

NOTE: the update is experimental and it hasn't been tested as thoroughly as we test our production versions (yet).

Thanks!

[marcwa19197]

Hi,

thanks for the good news!
But is this also an solution for Kernel 5.12? Or is this only for <5.12 so we still have to wait for the kernel changes?

Thanks!

[eidal]

Hi,
At the time of writing, kernel 5.12 is not yet supported as stated by Codelmp ten days ago.
https://github.com/veeam/veeamsnap/issues/10
Fingers crossed...

[PTide]

The patch has been updated so it can be used with kernel 5.12 too.

Thanks!

[marcwa19197]

Tested with Fedora 34 a few minutes ago, worked without problems after sudo make and sudo make install.
Backup using Entire-Machine setting in VAL is running now.

Thanks.

[Ejdesgaard]

The patch has been updated so it can be used with kernel 5.12 too.

Thanks!

* Is this a stopgap solution until the upstream work matures ?
* Did you manage to get Oracle on-board for the upstream development ?

Regards

[Gostev]

1. Yes, exactly: it's based on an ugly hack we found, definitely not a good long-term solution.

2. Yes, they have been very much involved. Currently waiting for them to test our latest drop actually. We're trying a much simpler kernel change based on one recent idea, which should work both for them and us - and has much better chances of being accepted into the kernel sooner.