Steps to build veemsnap final for internal distribution?

Backup agent for Linux servers and workstations on-premises or in the public cloud

Steps to build veemsnap final for internal distribution?

Veeam Logoby mikeely » Tue Jan 24, 2017 9:57 pm

I'm going to see pushback trying to install the Veeam agent given the build effort required to install veeamsnap due to the fact that it's slagging the host calling all the RPM build tools for a few minutes per each install. Given that most of our hosts are kickstarted and architecturally the same, what steps can I take to build an internal binary-only RPM?
mikeely
Enthusiast
 
Posts: 43
Liked: 3 times
Joined: Mon Nov 07, 2016 7:39 pm
Full Name: Mike Ely

Re: Steps to build veemsnap final for internal distribution?

Veeam Logoby PTide » Wed Jan 25, 2017 3:03 pm

Hi,

You can copy veeamsnap.ko module from the machine where it has been already built to all other machines. Make sure that kernel-headers and kernel versions on the machine where the module has been built do match the versions of those on the machines where you are going to copy the module to!

The module is located at /usr/lib/modules/<you-current-kernel-version>/extra/. Also you'll have to install veeam package so that it ignores veeamsnap dependency and configure the system to autoload veeamsnap module at boot. Please note that the described approach will require you to rebuild the module every time you upgrade the kernel.

Feel free to ask for assistance, should you need one.

Thank you.
PTide
Veeam Software
 
Posts: 2884
Liked: 236 times
Joined: Tue May 19, 2015 1:46 pm

Re: Steps to build veemsnap final for internal distribution?

Veeam Logoby tsightler » Wed Jan 25, 2017 4:15 pm 1 person likes this post

I've been wanting to build some steps to create binary module versions (that also satisfy the dependency) because I know this is going to come up fairly often, but I just haven't managed to find time. I'll see if I can look into it over the next couple of days if someone else doesn't beat me to it.

What specific distro are you targeting?
tsightler
Veeam Software
 
Posts: 4712
Liked: 1716 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Steps to build veemsnap final for internal distribution?

Veeam Logoby mikeely » Wed Jan 25, 2017 5:49 pm

tsightler wrote:I've been wanting to build some steps to create binary module versions (that also satisfy the dependency) because I know this is going to come up fairly often, but I just haven't managed to find time. I'll see if I can look into it over the next couple of days if someone else doesn't beat me to it.

What specific distro are you targeting?

That would be great. We're primarily using Centos7.
mikeely
Enthusiast
 
Posts: 43
Liked: 3 times
Joined: Mon Nov 07, 2016 7:39 pm
Full Name: Mike Ely

Re: Steps to build veemsnap final for internal distribution?

Veeam Logoby tsightler » Thu Jan 26, 2017 9:03 pm 4 people like this post

I took some time today to work on this. I plan to do a full, step-by-step blog post on http://www.veeamhub.io next week, but I thought I'd post something here for now.

First step is to setup for building an RPM, I used the instructions from https://wiki.centos.org/HowTos/SetupRpmBuildEnvironment as my guideline and that worked perfectly.

The veeamsnap module source code can be downloaded from the Veeam github repo, but to make things easier I just created a SRPM with the appropriate spec and kmodtool file, which you can download from here. I will look to move this to Github soon.

Assuming you have the proper rpmbuild environent and kernel-devel packages installed, you can build a kmod RPM with the following command:
Code: Select all
rpmbuild --rebuild veeamsnap-kmod-1.0.0.944-1.el7.src.rpm

This should give you a kmod-veeamsnap-1.0.0.944-1.el7.x86_64.rpm file in your rpmbuild/RPMS/x86_64 directory.

By default the spec file is configured to look for a 3.10.0-514* kernel-devel package (it will accept any sub-version of this kernel train, so, for example, kernel-devel-3.10.0-514.6.1 is just fine). This is the current kernel train for RHEL/CentOS 7.3 (Update 3). If you need to build against a different kernel just make sure you have the proper kernel-devel package install and define the target kernel version on the command line:

Code: Select all
rpmbuild --rebuild --define 'kversion 3.10.0-123.el7' veeamsnap-kmod-1.0.0.944-1.el7.src.rpm

One of this nice things about RHEL/CentOS is that they try to maintain a consistent kABI (Kernel Application Binary Interface) so that you don't need to recompile modules for every single kernel version. This SRPM does produce a kABI compliant module as best as possible, however, the veeamsnap module does use some symbols that are not whitelisted, thus it's possible that the module will need to be rebuilt for future RHEL/CentOS 7 kernel version if these non-whitelisted symbols change, however, at this time I have tested a binary module built against kernel 3.10.0-514.6.1 (the most recent at the time of this posting) with the original 3.10.0-123 kernel from RHEL/CentOS 7 GA, and the same module binary loaded and worked without any issues so, at least for now, it's possible to build a single kmod-veeamsnap RPM that works for all RHEL/CentOS 7 systems as long as they are using an official kernel between 3.10.0-123 and 3.10.0-514.6.1.

If you don't want to deal with setting up an rpmbuild environment or compiling your own module, you can download my pre-compiled RPM and the Veeam Agent for Linux userspace components and then install with the following command:

Code: Select all
yum install kmod-veeamsnap-1.0.0.944-1.el7.x86_64.rpm veeam-1.0.0.944-1.el7.x86_64.rpm


The kmod-veeamsnap binary RPM satisfies the dependency for veeamsnap>1.0.0.944 normally provided by the veeamsnap DKMS based RPM distributed by Veeam, so no need to force install anything!

I've currently tested on 4 different systems (CentOS 7.0, 7.2, and 7.3, and a RHEL 7 Update 2 system), and the binary seemed to work properly on each with no issues. Feedback regarding any problems is welcome!
tsightler
Veeam Software
 
Posts: 4712
Liked: 1716 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Steps to build veemsnap final for internal distribution?

Veeam Logoby tsightler » Fri Jan 27, 2017 2:45 am 2 people like this post

Here are links to SRPM and pre-built veeamsnap kmod RPM for RHEL/CentOS 6:

SRPM: veeamsnap-kmod-1.0.0.944-1.el6.src.rpm

Pre-built kmod based RPM (2.6.23-642 kernel): kmod-veeamsnap-1.0.0.944-1.el6.x86_64.rpm
Due to the fact that the veeamsnap module uses a significant number of non-whitelisted kABI functions, and those functions have changed during the life of RHEL/CentOS 6, this module is likely to only work on the latest Update 8 kernels (2.6.32-642 train). If you are running older update version, you should rebuild the SRPM above.

Veeam userspace components (from official Veeam repo): veeam-1.0.0.944-1.el6.x86_64.rpm
tsightler
Veeam Software
 
Posts: 4712
Liked: 1716 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Steps to build veemsnap final for internal distribution?

Veeam Logoby mikeely » Tue Jan 31, 2017 1:24 am

Thanks, this should work out just fine. We've got a set of build servers/repos for all our Centos releases. I'll give them a spin tomorrow and let you know how it went.
mikeely
Enthusiast
 
Posts: 43
Liked: 3 times
Joined: Mon Nov 07, 2016 7:39 pm
Full Name: Mike Ely

Re: Steps to build veemsnap final for internal distribution?

Veeam Logoby tsightler » Tue Jan 31, 2017 1:38 am

OK, please let me know how it goes and feel free to PM me if you have any issues. This is my first cut and I have some improvements that I want to add. For example, I want to include a script in the source RPM that will automatically download the latest veeamsnap source directly from github but, for simplicity, these SRPMs just have a tarball that I manually created. If you don't want to trust my tarball you can clone the source from https://github.com/veeam/veeamsnap and simply tar up the source directory.

I've had at least one customer that has tested these already with no issues, so I feel pretty confident with them.
tsightler
Veeam Software
 
Posts: 4712
Liked: 1716 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Steps to build veemsnap final for internal distribution?

Veeam Logoby mikeely » Tue Jan 31, 2017 6:09 pm

Went pretty well. Only thing was the .spec file was for el6 and the 2.6 kernel:
+ KSRC=/usr/src/kernels/2.6.32-642.el6.x86_64

As you say, that was easy enough to override, and after that it built just fine.

Edit: the last step here is ensuring we stay up to date so that when Veeam releases a new version of the Linux agent we're able to automatically push that to our build environments. What do you recommend here?
mikeely
Enthusiast
 
Posts: 43
Liked: 3 times
Joined: Mon Nov 07, 2016 7:39 pm
Full Name: Mike Ely

Re: Steps to build veemsnap final for internal distribution?

Veeam Logoby tsightler » Tue Jan 31, 2017 6:43 pm 2 people like this post

By any chance did you happen to download the EL6 SRPM instead of the EL7 one? I have both linked above, but I'll admit the EL7 link is not very obvious:

EL7 veeamsnap SRPM: http://www.sightunseen.org/files/val/el ... l7.src.rpm
EL6 veeamsnap SRPM: http://www.sightunseen.org/files/val/el ... l6.src.rpm

I'll double check them, perhaps I accidentally botched the creation of the EL7 SRPM, but I don't see how as I'm actually using two different build VMs.

Otherwise, thanks for your support. I'm sure that newer versions of Veeam will come with new builds of veeamsnap as well, and since the dependency requirement are in the RPM it the new user space tools should fail to build if it's not compatible, but that's indeed why I was hoping to make it easy to simply pull down the latest source from github to use with the SRPM. Hopefully I'll have more time to work on that soon.
tsightler
Veeam Software
 
Posts: 4712
Liked: 1716 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Steps to build veemsnap final for internal distribution?

Veeam Logoby mikeely » Tue Jan 31, 2017 6:46 pm

I pulled down the EL6. Which, happily, builds fine in the EL7 environment if I munge the spec.
mikeely
Enthusiast
 
Posts: 43
Liked: 3 times
Joined: Mon Nov 07, 2016 7:39 pm
Full Name: Mike Ely

Re: Steps to build veemsnap final for internal distribution?

Veeam Logoby tsightler » Tue Jan 31, 2017 7:29 pm

Yeah, there's not much difference between them except for things like removing i386 arch, since that isn't available in EL7, and a little different kmod stuff. Glad it worked.
tsightler
Veeam Software
 
Posts: 4712
Liked: 1716 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler


Return to Veeam Agent for Linux



Who is online

Users browsing this forum: No registered users and 2 guests