Linux Kernel 5.0

[mike2307]

Hi there,

I updated my Fedora 29 installation to kernel 5.0.3. The veeamsnap module doesn't compile with this version.
According to my research, the Linux kernel developers changed something related to the block layer and that is causing the issue.
Has anyone tried to figure out how the code needs to be patched to make it working again?

Thanks in advance.

Kind regards,
Michael

[PTide]

Hi,

According to our tests, the module may fail to compile on kernels that have I/O schedulers other than "deadline" and "cfq". We know for sure that it does not work with "bfq" scheduler at all, while it might work with mq-deadline. Other schedulers (kyber, noop, anticipatory etc) hasn't been tested at all.

Thanks!

[rokko74]

I confirm the "blk_init_queue" related compilation issue with Fedora 29 kernel 5.0.3-200.fc29.x86_64 update:

Code: Select all

$ cat /var/lib/dkms/veeamsnap/3.0.1.1046/build/make.log
DKMS make.log for veeamsnap-3.0.1.1046 for kernel 5.0.3-200.fc29.x86_64 (x86_64)
gio 28 mar 2019, 06.26.22, CET
make: ingresso nella directory "/usr/src/kernels/5.0.3-200.fc29.x86_64"
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/log.o
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/mem_alloc.o
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/sparse_bitmap.o
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/container.o
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/container_spinlocking.o
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/queue_spinlocking.o
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/bitmap_sync.o
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/page_array.o
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/rangelist.o
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/rangelist_ex.o
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/rangevector.o
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/blk_util.o
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/blk_direct.o
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/blk_redirect.o
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/blk_deferred.o
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/defer_io.o
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/cbt_map.o
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/tracker.o
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/tracker_queue.o
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/snapimage.o
  CC [M]  /var/lib/dkms/veeamsnap/3.0.1.1046/build/snapdata_collect.o
/var/lib/dkms/veeamsnap/3.0.1.1046/build/snapimage.c: In function ‘snapimage_create’:
/var/lib/dkms/veeamsnap/3.0.1.1046/build/snapimage.c:615:28: error: implicit declaration of function ‘blk_init_queue’; did you mean ‘blk_put_queue’? [-Werror=implicit-function-declaration]
             image->queue = blk_init_queue(NULL, NULL);
                            ^~~~~~~~~~~~~~
                            blk_put_queue
/var/lib/dkms/veeamsnap/3.0.1.1046/build/snapimage.c:615:26: warning: assignment to ‘struct request_queue *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
             image->queue = blk_init_queue(NULL, NULL);
                          ^
/var/lib/dkms/veeamsnap/3.0.1.1046/build/snapimage.c:617:26: warning: assignment to ‘struct request_queue *’ from ‘int’ makes pointer from integer without a cast [-Wint-conversion]
             image->queue = blk_init_queue(NULL, &image->queue_lock);
                          ^
In file included from ./include/linux/seqlock.h:36,
                 from ./include/linux/time.h:6,
                 from ./include/linux/stat.h:19,
                 from ./include/linux/module.h:10,
                 from /var/lib/dkms/veeamsnap/3.0.1.1046/build/stdafx.h:5,
                 from /var/lib/dkms/veeamsnap/3.0.1.1046/build/snapimage.c:1:
/var/lib/dkms/veeamsnap/3.0.1.1046/build/snapimage.c: In function ‘_snapimage_stop’:
/var/lib/dkms/veeamsnap/3.0.1.1046/build/snapimage.c:727:37: error: incompatible type for argument 1 of ‘spinlock_check’
                 spin_lock_irqsave( q->queue_lock, flags );
                                    ~^~~~~~~~~~~~
./include/linux/spinlock.h:241:34: note: in definition of macro ‘raw_spin_lock_irqsave’
   flags = _raw_spin_lock_irqsave(lock); \
                                  ^~~~
/var/lib/dkms/veeamsnap/3.0.1.1046/build/snapimage.c:727:17: note: in expansion of macro ‘spin_lock_irqsave’
                 spin_lock_irqsave( q->queue_lock, flags );
                 ^~~~~~~~~~~~~~~~~
In file included from ./include/linux/seqlock.h:36,
                 from ./include/linux/time.h:6,
                 from ./include/linux/stat.h:19,
                 from ./include/linux/module.h:10,
                 from /var/lib/dkms/veeamsnap/3.0.1.1046/build/stdafx.h:5,
                 from /var/lib/dkms/veeamsnap/3.0.1.1046/build/snapimage.c:1:
./include/linux/spinlock.h:316:67: note: expected ‘spinlock_t *’ {aka ‘struct spinlock *’} but argument is of type ‘spinlock_t’ {aka ‘struct spinlock’}
 static __always_inline raw_spinlock_t *spinlock_check(spinlock_t *lock)
                                                       ~~~~~~~~~~~~^~~~
/var/lib/dkms/veeamsnap/3.0.1.1046/build/snapimage.c:728:17: error: implicit declaration of function ‘blk_stop_queue’; did you mean ‘blk_sync_queue’? [-Werror=implicit-function-declaration]
                 blk_stop_queue( q );
                 ^~~~~~~~~~~~~~
                 blk_sync_queue
/var/lib/dkms/veeamsnap/3.0.1.1046/build/snapimage.c:729:42: error: incompatible type for argument 1 of ‘spin_unlock_irqrestore’
                 spin_unlock_irqrestore( q->queue_lock, flags );
                                         ~^~~~~~~~~~~~
In file included from ./include/linux/seqlock.h:36,
                 from ./include/linux/time.h:6,
                 from ./include/linux/stat.h:19,
                 from ./include/linux/module.h:10,
                 from /var/lib/dkms/veeamsnap/3.0.1.1046/build/stdafx.h:5,
                 from /var/lib/dkms/veeamsnap/3.0.1.1046/build/snapimage.c:1:
./include/linux/spinlock.h:382:64: note: expected ‘spinlock_t *’ {aka ‘struct spinlock *’} but argument is of type ‘spinlock_t’ {aka ‘struct spinlock’}
 static __always_inline void spin_unlock_irqrestore(spinlock_t *lock, unsigned long flags)
                                                    ~~~~~~~~~~~~^~~~
cc1: some warnings being treated as errors
make[1]: *** [scripts/Makefile.build:277: /var/lib/dkms/veeamsnap/3.0.1.1046/build/snapimage.o] Error 1
make[1]: *** Attesa per i processi non terminati....
make: *** [Makefile:1557: _module_/var/lib/dkms/veeamsnap/3.0.1.1046/build] Error 2
make: uscita dalla directory "/usr/src/kernels/5.0.3-200.fc29.x86_64"

[mike2307]

Any news regarding this topic?
When can we expect a fix for that?

Kind regards,
Michael

[PTide]

Hi,

There exists an unofficial fix, please check this AUR

Thanks!

[rokko74]

Hi,
thank You very much P.Tide, it seems to work here with these steps:

Code: Select all

wget https://www.github.com/veeam/veeamsnap/archive/experimental.zip
unzip experimental.zip
cd veeamsnap-experimental/source/
make all && make load

[gothaggis]

on a physical box I was testing, I was getting the compile errors with kernel 5.0.

I decided to test on a vm, using kernel 5.0.7 (specifically I'm using kernel-ml from elrepo.org) and the latest Veeam agent (March 26th) - and I'm not getting compile errors and the agent works. I'm not sure if this is due to something with vm, or maybe 5.0.7 fixes it? Can anyone else confirm?

[PTide]

Hi,

Would you kindly provide more info on the errors you are getting, and the exact kernel versions?

Thanks!

[gothaggis]

The kernel version was 5.0.1 - I don't recall the exact errors, but they were all veeamsnap related, so I imagine they are the same that were posted above. When I'm able to reboot the server, I'll test out 5.0.7 and see if I have the same results as I did in the VM.

[Ryand833]

I just tested kernel 5.0.7 on my Fedora 29 system, same error as kernel 5.0.1, 5.0.4, 5.0.5, etc (regarding blk_init_queue).

I used the experimental veeamsnap build mentioned above and it works with 5.0.7 (exact kernel version: 5.0.7-200.fc29.x86_64)

[PTide]

Ryand,

So, to clarify - the current official build of veeamsnap does not work 5.0.7, 5.0.1, 5.0.4, 5.0.5 and you get the blk_init_queue error, but the experimental build works just fine on 5.0.7, is that correct?

Thanks!

[Ryand833]

Yes, that's right - I tested the official build on all the kernels you listed, each having the same build errors on the veeamsnap module.

I then tested the experimental build on 5.0.7 - the module built successfully, the module loaded successfully, and I was able to perform a full system backup using it.

[sdet00]

I am about to try out Ubuntu 19.04, which ships with 5.0 - has anyone tried this on a Debian based install?

[mfo]

Same error on Ubuntu 19.04 with veeamsnap 3.0.1.1046:

Code: Select all

/var/lib/dkms/veeamsnap/3.0.1.1046/build/snapimage.c:728:17: error: implicit declaration of function ‘blk_stop_queue’; did you mean ‘blk_sync_queue’? [-Werror=implicit-function-declaration]
                 blk_stop_queue( q );
                 ^~~~~~~~~~~~~~
                 blk_sync_queue

[PTide]

Hi,

Please mind the user guide system requirements section:

Ubuntu 10.04 – 18.10

Support for 19.04 is already on our roadmap for future updates.

Thanks!

[gothaggis]

Upgraded my CentOS install to kernel 5.0.9 - and using the experimental veemsnap posted above - Veeam Agent is now able to back up my system.

[swissbuechi]

Upgraded to Ubuntu 19.04. with Kernel 5.0.0.13 and I`m not able to install veeamsnap. Even the experimental veemsnap posted above.

[edward.hang]

I also tried the expermental release without success. I'm on Ubuntu 19.04 which is on 5.0.0.13

[mike2307]

I can confirm that the experimental branch is working fine with kernel versions 5.0.7 to 5.0.9 on Fedora 29 as well as Fedora 30.

[PTide]

Edward, Raphael,

Kindly collect build logs and share them with me.

The logs are likely to be found there: /var/lib/dkms/veeamsnap/<version>/build/make.log

Thanks!

[edward.hang]

Hi P.Tide,

the dkms directory is missing in /var/lib/

I have posted the output instead

Code: Select all

sudo make all && make load
Making all...
ccflags-y= -O2 -Wno-multichar -DDISTRIB_NAME_UBUNTU -DDISTRIB_VERSION_1=19 -DDISTRIB_VERSION_2=04
make -C /lib/modules/5.0.0-13-generic/build M= modules
make[1]: Entering directory '/usr/src/linux-headers-5.0.0-13-generic'
  YACC    scripts/kconfig/zconf.tab.h
  HOSTCC  scripts/kconfig/zconf.lex.o
  YACC    scripts/kconfig/zconf.tab.c
  HOSTCC  scripts/kconfig/zconf.tab.o
  HOSTLD  scripts/kconfig/conf
scripts/kconfig/conf  --syncconfig Kconfig
make[2]: *** No rule to make target 'arch/x86/tools/relocs_32.c', needed by 'arch/x86/tools/relocs_32.o'.  Stop.
make[1]: *** [arch/x86/Makefile:223: archscripts] Error 2
make[1]: Leaving directory '/usr/src/linux-headers-5.0.0-13-generic'
make: *** [Makefile:54: all] Error 2

[PTide]

Edward,

Thanks for the logs! Of these, it is obvious that the assembly of the module veeamsnap has not even started - there was a problem with building "scripts/kconfig/conf". What architecture do you build the module for?

Thanks

[sdet00]

Veeam Snap broken on Ubuntu 19.04 right now, 5.0.0-15 kernel.

[PTide]

Hi,

Due to some major changes in I/O schedulers, support for 19.04 is currently in the development, and will be released as soon as it passes all tests.

Thanks!

[obwankenooby]

Hi there P.Tide,

Any news concerning the Linux Kernel 5.x based Veeam Agent for Linux ?
Running 19.04, would appreciate to backup my endpoint =)

i'd test beta releases if needed.

Thanks,
Regards,
ok

[eMKi]

Hi I am also looking forward to a Kernel 5.x based Veeam Agent for Linux. Forgot to check the compatibility of Veeam with 19.04 before upgrading from 18.10...
I would also beta test if needed.

Kind regards!

[PTide]

The new update that will introduce support for kernel 5.0 is undergoing the final testing and will be released soon.

Thanks!

[sdet00]

Any idea if the update for the 5.0 Kernel is going to work with the 5.1 Kernel as well?

[technovelist]

I'm a complete newbie to veeam and can't figure out what to do after compiling the experimental version.

Can someone post all the steps needed to to take a backup using the experimental version for kernel 5.0? I was able to get to the "loading veeamsnap kernel module from current folder" message, but what do I do after that? I thought I might be able to use "apt-get install veeam" to finish the installation but that didn't work.

I'd appreciate any help.

[PTide]

Any idea if the update for the 5.0 Kernel is going to work with the 5.1 Kernel as well?

Reportedly, it already works fine with Fedora on 5.1.11 kernel.

Can someone post all the steps needed to to take a backup using the experimental version for kernel 5.0?

If "make all" command finished successfully, then you should run "make load" command while you are in the directory where veeamsnap sources are located. To check whether the module has been loaded run lsmod | grep veeamsnap.

Thanks!