Steps to build veemsnap final for internal distribution?

[mikeely]

I'm going to see pushback trying to install the Veeam agent given the build effort required to install veeamsnap due to the fact that it's slagging the host calling all the RPM build tools for a few minutes per each install. Given that most of our hosts are kickstarted and architecturally the same, what steps can I take to build an internal binary-only RPM?

[PTide]

Hi,

You can copy veeamsnap.ko module from the machine where it has been already built to all other machines. Make sure that kernel-headers and kernel versions on the machine where the module has been built do match the versions of those on the machines where you are going to copy the module to!

The module is located at /usr/lib/modules/<you-current-kernel-version>/extra/. Also you'll have to install veeam package so that it ignores veeamsnap dependency and configure the system to autoload veeamsnap module at boot. Please note that the described approach will require you to rebuild the module every time you upgrade the kernel.

Feel free to ask for assistance, should you need one.

Thank you.

[tsightler]

I've been wanting to build some steps to create binary module versions (that also satisfy the dependency) because I know this is going to come up fairly often, but I just haven't managed to find time. I'll see if I can look into it over the next couple of days if someone else doesn't beat me to it.

What specific distro are you targeting?

[mikeely]

I've been wanting to build some steps to create binary module versions (that also satisfy the dependency) because I know this is going to come up fairly often, but I just haven't managed to find time. I'll see if I can look into it over the next couple of days if someone else doesn't beat me to it.

What specific distro are you targeting?

That would be great. We're primarily using Centos7.

[tsightler]

I took some time today to work on this.

First step is to setup for building an RPM, I used the instructions from https://wiki.centos.org/HowTos/SetupRpmBuildEnvironment as my guideline and that worked perfectly.

The veeamsnap module source code can be downloaded from the Veeam github repo, but to make things easier I just created a SRPM with the appropriate spec and kmodtool file, which you can download from here. I will look to move this to Github soon.

Assuming you have the proper rpmbuild environent and kernel-devel packages installed, you can build a kmod RPM with the following command:

Code: Select all

rpmbuild --rebuild veeamsnap-kmod-1.0.0.944-1.el7.src.rpm

This should give you a kmod-veeamsnap-1.0.0.944-1.el7.x86_64.rpm file in your rpmbuild/RPMS/x86_64 directory.

By default the spec file is configured to look for a 3.10.0-514* kernel-devel package (it will accept any sub-version of this kernel train, so, for example, kernel-devel-3.10.0-514.6.1 is just fine). This is the current kernel train for RHEL/CentOS 7.3 (Update 3). If you need to build against a different kernel just make sure you have the proper kernel-devel package install and define the target kernel version on the command line:

Code: Select all

rpmbuild --rebuild --define 'kversion 3.10.0-123.el7' veeamsnap-kmod-1.0.0.944-1.el7.src.rpm

One of this nice things about RHEL/CentOS is that they try to maintain a consistent kABI (Kernel Application Binary Interface) so that you don't need to recompile modules for every single kernel version. This SRPM does produce a kABI compliant module as best as possible, however, the veeamsnap module does use some symbols that are not whitelisted, thus it's possible that the module will need to be rebuilt for future RHEL/CentOS 7 kernel version if these non-whitelisted symbols change, however, at this time I have tested a binary module built against kernel 3.10.0-514.6.1 (the most recent at the time of this posting) with the original 3.10.0-123 kernel from RHEL/CentOS 7 GA, and the same module binary loaded and worked without any issues so, at least for now, it's possible to build a single kmod-veeamsnap RPM that works for all RHEL/CentOS 7 systems as long as they are using an official kernel between 3.10.0-123 and 3.10.0-514.6.1.

If you don't want to deal with setting up an rpmbuild environment or compiling your own module, you can download my pre-compiled RPM and the Veeam Agent for Linux userspace components and then install with the following command:

Code: Select all

yum install kmod-veeamsnap-1.0.0.944-1.el7.x86_64.rpm veeam-1.0.0.944-1.el7.x86_64.rpm

The kmod-veeamsnap binary RPM satisfies the dependency for veeamsnap>1.0.0.944 normally provided by the veeamsnap DKMS based RPM distributed by Veeam, so no need to force install anything!

I've currently tested on 4 different systems (CentOS 7.0, 7.2, and 7.3, and a RHEL 7 Update 2 system), and the binary seemed to work properly on each with no issues. Feedback regarding any problems is welcome!

[tsightler]

Here are links to SRPM and pre-built veeamsnap kmod RPM for RHEL/CentOS 6:

SRPM: veeamsnap-kmod-1.0.0.944-1.el6.src.rpm

Pre-built kmod based RPM (2.6.23-642 kernel): kmod-veeamsnap-1.0.0.944-1.el6.x86_64.rpm
Due to the fact that the veeamsnap module uses a significant number of non-whitelisted kABI functions, and those functions have changed during the life of RHEL/CentOS 6, this module is likely to only work on the latest Update 8 kernels (2.6.32-642 train). If you are running older update version, you should rebuild the SRPM above.

Veeam userspace components (from official Veeam repo): veeam-1.0.0.944-1.el6.x86_64.rpm

[mikeely]

Thanks, this should work out just fine. We've got a set of build servers/repos for all our Centos releases. I'll give them a spin tomorrow and let you know how it went.

[tsightler]

OK, please let me know how it goes and feel free to PM me if you have any issues. This is my first cut and I have some improvements that I want to add. For example, I want to include a script in the source RPM that will automatically download the latest veeamsnap source directly from github but, for simplicity, these SRPMs just have a tarball that I manually created. If you don't want to trust my tarball you can clone the source from https://github.com/veeam/veeamsnap and simply tar up the source directory.

I've had at least one customer that has tested these already with no issues, so I feel pretty confident with them.

[mikeely]

Went pretty well. Only thing was the .spec file was for el6 and the 2.6 kernel:
+ KSRC=/usr/src/kernels/2.6.32-642.el6.x86_64

As you say, that was easy enough to override, and after that it built just fine.

Edit: the last step here is ensuring we stay up to date so that when Veeam releases a new version of the Linux agent we're able to automatically push that to our build environments. What do you recommend here?

[tsightler]

By any chance did you happen to download the EL6 SRPM instead of the EL7 one? I have both linked above, but I'll admit the EL7 link is not very obvious:

EL7 veeamsnap SRPM: http://www.sightunseen.org/files/val/el ... l7.src.rpm
EL6 veeamsnap SRPM: http://www.sightunseen.org/files/val/el ... l6.src.rpm

I'll double check them, perhaps I accidentally botched the creation of the EL7 SRPM, but I don't see how as I'm actually using two different build VMs.

Otherwise, thanks for your support. I'm sure that newer versions of Veeam will come with new builds of veeamsnap as well, and since the dependency requirement are in the RPM it the new user space tools should fail to build if it's not compatible, but that's indeed why I was hoping to make it easy to simply pull down the latest source from github to use with the SRPM. Hopefully I'll have more time to work on that soon.

[mikeely]

I pulled down the EL6. Which, happily, builds fine in the EL7 environment if I munge the spec.

[tsightler]

Yeah, there's not much difference between them except for things like removing i386 arch, since that isn't available in EL7, and a little different kmod stuff. Glad it worked.