Standalone backup agents for Linux, Mac, AIX & Solaris workloads on-premises or in the public cloud
Post Reply
Alvi
Influencer
Posts: 12
Liked: 10 times
Joined: Jul 08, 2015 10:19 am
Contact:

LUKS support

Post by Alvi » 1 person likes this post

I've been looking around for a backup-solution of our field-workers Linux Laptops.
I am glad that Veeam now offers this BETA.

Just downloaded and tried out my first time this morning, so this is my first emotional oppinion :)
Sorry, it's long, I know...

First the short:
Installation on a Mint Linux 18 64bit (which is Ubuntu 16.04 based): no problems.

Then the backup:

I wanted to create "Entire system", but it failed on choosing "local" as a destination:
It's okay to say "/mnt/backup" is part of the entire system, so it cannot be chosen as a destination.
Veeam told me to exclude that directory... but there is no option to exclude it anywhere.
Otherwise choosing /dev/sdd1 should be an option, but even this failed because veeam needed a not a device I think.
The solution should be: Let me choose a directory, but please exclude that automatically from backups when "entire system - local".

So I just created a volume based backup job and chose any volume and partitions except the partition containing the backups.
I have a dedicated disk for local backups in our laptops, so this no problem for me, but I think the next versions should be able to backup even within the same partition and have to bring in "Exclusion" options for the Target-Directories.

The first run took 86 minutes (1TB read from SSD and 385 GB wrote to SATA-disk).
Most annoying was: During that run the system really often becam unresponsive for seconds.
The load average grew up to 7,26 on my Core i7, so the CPU's really were heavy on duty.
Veeam told me "source" had been the bottleneck. Nice to know my SSD is a bottleneck :)
But it felt my CPU was the bottleneck - it's the first time I really thought my Core i7 4770 is a bottleneck :)

Tried another run 15 minutes later. That run was "quick" in 6 minutes, read 340MB and wrote 138MB.
Wondering what 138 MB of data changed in that short time.

Reboted the machine and tried another run 2 hours later.. That run needed 38 minutes, read again 1TB and wrote 202MB.

In my oppinion an incremental backup run should not run about 40 minutes (just half the time of a full backup) even after a reboot...

Apropos incrmental backup:
Feature request:
As known from our Veeam enterprise: Will there be an reverse incrmental option someday?
I'm in love with that concept of "reverse incremental" the newest Backup-File alwas is complete and I may just discard the old diffs as my storage goes out of space without needing to do a new Full-Backup. That should become a standard in my humble oppinion :)

Restore:
Just tried to restore some data (online without the iso) and mounted the Restore point:
The mounted mount points worked so far... but... not every backed up device could be mounted.
May be the thing is:
I have got a luks-encrypted device and backed it up (/dev/sdc1).
So I think there is mission luks-support (at least for restore? Hope the backup worked, he used a lot of time to fully backup the device first time).
Veeam should try to detect luks devices, ask for a password and try to mount it for restore...?
Or is there any manual way to mount that restore-point using luks?

I think it will be the same on Restore with CD:
Does the CD contain everything to support luks encrypted volumes within the backup?

Next thing is:
The backups target-directory itself is stored on a luks-encrypted volume.
Within my running system it's no problem to mount that backup-target-directory and make it accessible to Veeam, but is the iso capable of mounting that encrypted target-filesystem?

To avoid using encrypted filesystems as a target-directory there should be a password option for backup jobs.
Especially for field-workers and their laptops often it is not possible to backup to clouds, so the backup should get local but secure for the case of loosing the hardware.

TLDR:
My first impression is:
Great beta as expected for Veeam product, but like a beta should be: Working but in need of round some corners!
Thank you for that good job until here, guys!
nielsengelen
Product Manager
Posts: 5619
Liked: 1177 times
Joined: Jul 15, 2013 11:09 am
Full Name: Niels Engelen
Contact:

Re: LUKS support

Post by nielsengelen »

About exclusions, this is currently known and should be improved within an upcoming release ;-).

About what changed, well that can be a lot of things, logs, temp files,... Machines do change quick (quicker then you think) and 138MB isn't that much imo. I guess the incremental was around 60-70MB in file size?

In regards to reverse incremental, this probably won't come soon as it requires much more I/O on the backup target and even within VBR we advise forward incremental (with synthetic full).

LUKS support is something we might have to look into as encryption is getting more and more used, so thanks for the FR.
Personal blog: https://foonet.be
GitHub: https://github.com/nielsengelen
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: LUKS support

Post by PTide »

Hi,
It's okay to say "/mnt/backup" is part of the entire system, so it cannot be chosen as a destination.
Veeam told me to exclude that directory... but there is no option to exclude it anywhere.
You should exclude the device that contains the target directory. If you choose the "Entire machine" VAL attempts to backup all local drives, so you've decided to use "Volume level" mode, which was a valid approach.
Otherwise choosing /dev/sdd1 should be an option, but even this failed because veeam needed a not a device I think.
I'm not sure that I follow you - did you try to choose /dev/sdd1 as a destination?
The solution should be: Let me choose a directory, but please exclude that automatically from backups when "entire system - local"
Imagine if you had CIFS share mounted to a local directory (say /backup) , so if you chose "Entire" - "Local" - "your CIFS" everything would be OK. Now imagine that, for some reason, the CIFS share have been accidentally unmounted. With the approach you've offered VAL would automatically exclude the device where the /backup directory resides at (root device for example) thus leaving you without a backup.
The first run took 86 minutes (1TB read from SSD and 385 GB wrote to SATA-disk).
...which gives us an average speed of ~100 MB/s - that's a normal speed of a SATA drive, looks like the bottleneck stats are wrong, thanks for heads up!
Reboted the machine and tried another run 2 hours later.. That run needed 38 minutes, read again 1TB and wrote 202MB. In my oppinion an incremental backup run should not run about 40 minutes (just half the time of a full backup) even after a reboot...
That hapens due to reboot - the driver is loaded out off the kernel thus CBT is get reset. Will look into that in future releases.
Will there be an reverse incrmental option someday?
Depending on the amount of requests, however not in the first version for sure.

Regarding LUKS support - I agree with Niels, we will look into it later.

Thanks for your feedback!
Alvi
Influencer
Posts: 12
Liked: 10 times
Joined: Jul 08, 2015 10:19 am
Contact:

Re: LUKS support

Post by Alvi »

Did not think about the approach of missing network mounts... now it's making sense :)
The CBT is reset.. no as you say it... sounds somewhat logical...

In summary: Yes, the LUKS approach in my point of view would be one of the prior things, the rest is nice to have and performance improvements for future versions.
Thanks for your feedback!
That's what the BETA is for :)
fprat
Novice
Posts: 6
Liked: never
Joined: Feb 24, 2017 6:27 am
Full Name: Frédéric PRAT
Contact:

Re: LUKS support

Post by fprat »

Hello,

We have the Veeam Backup & Replication solution and we need to perform backup/restore plan of Oracle Linux 6.8 system with LVM over LUKS volume (Entire system backup).
In order to perform this task we use the last Veeam Linux agent.
It seems that the LUKS isn't currently available in the last Veeam Linux agent.

Is there any plan to integrate it in a future release of the Veeam Linux agent.

Thank you.

Best regards

Frédéric PRAT
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: LUKS support

Post by PTide »

Hi,

Backup of LUKS-encrypted volumes should work just fine, the only limitation is that the volumes form such backup will be restored unencrypted. Did you get any error messages during the backup session?

Thank you
fprat
Novice
Posts: 6
Liked: never
Joined: Feb 24, 2017 6:27 am
Full Name: Frédéric PRAT
Contact:

Re: LUKS support

Post by fprat »

Hello,

Thank you for your fast feedback.

Currently, We want to backup/restore an entire linux server with the following disks architecture:

Code: Select all

NAME                                                 MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
fd0                                                    2:0    1     4K  0 disk
sda                                                    8:0    0   100G  0 disk
├─sda1                                                 8:1    0   500M  0 part  /boot
└─sda2                                                 8:2    0  99,5G  0 part
  ├─root_vg-lv_root (dm-0)                           251:0    0  29,3G  0 lvm   /
  ├─root_vg-lv_swap (dm-1)                           251:1    0  15,6G  0 lvm   [SWAP]
  └─root_vg-lv_home (dm-7)                           251:7    0  29,3G  0 lvm   /home
sdb                                                    8:16   0    50G  0 disk
└─sdb1                                                 8:17   0    50G  0 part
  ├─oracle_vg-lv_ora1120 (dm-3)                      251:3    0  29,3G  0 lvm   /oracle/oradb/11.2.0
  └─oracle_vg-lv_diag (dm-4)                         251:4    0     2G  0 lvm   /oracle/diag
sdc                                                    8:32   0   600G  0 disk
└─sdc1                                                 8:33   0   600G  0 part
  └─luks-56a83e13-e115-4eb0-bbd7-21caba21736a (dm-2) 251:2    0   600G  0 crypt
    ├─data_vg-lv_appli_POGSINDV (dm-5)               251:5    0   293G  0 lvm   /appli/oracle/POGSINDV
    └─data_vg-lv_sauvebase (dm-6)                    251:6    0 195,3G  0 lvm   /sauvebase
The backup process haven't returned any errror and I haven't see any error in the log file

During the restoration process (though the veeam bootable linux image), the disk sdc1 is restored empty and we have lose the encrypted disk with the Volume group and the logical volume available under it.
After restoration, the restored disks architecture looks like:

Code: Select all

NAME                                                 MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
fd0                                                    2:0    1     4K  0 disk
sda                                                    8:0    0   100G  0 disk
├─sda1                                                 8:1    0   500M  0 part  /boot
└─sda2                                                 8:2    0  99,5G  0 part
  ├─root_vg-lv_root (dm-0)                           251:0    0  29,3G  0 lvm   /
  ├─root_vg-lv_swap (dm-1)                           251:1    0  15,6G  0 lvm   [SWAP]
  └─root_vg-lv_home (dm-7)                           251:7    0  29,3G  0 lvm   /home
sdb                                                    8:16   0    50G  0 disk
└─sdb1                                                 8:17   0    50G  0 part
  ├─oracle_vg-lv_ora1120 (dm-3)                      251:3    0  29,3G  0 lvm   /oracle/oradb/11.2.0
  └─oracle_vg-lv_diag (dm-4)                         251:4    0     2G  0 lvm   /oracle/diag
sdc                                                    8:32   0   600G  0 disk
└─sdc1                                                 8:33   0   600G  0 part
With the veeam Linux Agent and the Veeam Backup and Replication application, how we must proceed for backuping and restoring LVM over LUKS object?

Thank you for your response.

Best regards

Frédéric PRAT
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: LUKS support

Post by PTide »

Hi Frederic,

Could you please try to perform File-level restore from the backup in order to check if the files that reside on the encrypted disks are available?

Thanks
fprat
Novice
Posts: 6
Liked: never
Joined: Feb 24, 2017 6:27 am
Full Name: Frédéric PRAT
Contact:

Re: LUKS support

Post by fprat »

Hello,

So, I have tested the File-Level restore but before using the File-level restore some préerequisite must applied.
As, during the Entire System restore process, it doesn't restore any object under the encrypted partition, the volume group,the logical volumes and the associated data aren't be restored.
Before, running the File-level restore process, we must create through the Linux Rescue disk:
- The missing volume group
- The missing logical volumes and format they

After this, we can perform the File-Level restore process in the last created logical volume.

So, in summarry (correct me if It is not right), the restore process of an Entire Linux system with LVM over luks volumes will looks like as below:
1. Boot on Veeam recovery media disk
2. Perform the Restore volumes process
3. Reboot
4. Boot on the Linux rescue disk
5. Create missing volume group, logical volumes and format they
6. Reboot
7. Boot on Veeam recovery media disk
8. Perform the Restore files process in order to restore the data in the logical volumes previously created

Disadvantages
Lost of the LUKS encrypted volumes (Data backuped from an encrypted volume are restored in an unencrypted volume)
A simple restore process is transformed into a difficult restore process.

Did you have any others process to restore the backup of a system with lvm over luks volumes.

Thank you for your feedback

Best regards

Frédéric PRAT
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: LUKS support

Post by PTide »

I think that some sort of misunderstanding occured and I'd like to clerify the situation.

Can you see the files that are located on the encrypted device if you press 'R' on the main screen and choose the backup? Also, do you see the logical volumes in the right panel (like on this picture) that are located on an encrypted device?

Thank you.
fprat
Novice
Posts: 6
Liked: never
Joined: Feb 24, 2017 6:27 am
Full Name: Frédéric PRAT
Contact:

Re: LUKS support

Post by fprat »

Hello,

Here is a screen capture of the Veeam Recovery Media view:
Image

It seems that the volume group are correctly backuped but the encrypted volume doesn't appear (sdc1). So it seems that during the restore process, the Veeam recovery media doesn't know where restore the volume group (cf screen capture).
The luks volume is under sdc1:

Code: Select all

NAME                                                 MAJ:MIN RM   SIZE RO TYPE  MOUNTPOINT
fd0                                                    2:0    1     4K  0 disk
sda                                                    8:0    0   100G  0 disk
├─sda1                                                 8:1    0   500M  0 part  /boot
└─sda2                                                 8:2    0  99,5G  0 part
  ├─root_vg-lv_root (dm-0)                           251:0    0  29,3G  0 lvm   /
  ├─root_vg-lv_swap (dm-1)                           251:1    0  15,6G  0 lvm   [SWAP]
  └─root_vg-lv_home (dm-7)                           251:7    0  29,3G  0 lvm   /home
sdb                                                    8:16   0    50G  0 disk
└─sdb1                                                 8:17   0    50G  0 part
  ├─oracle_vg-lv_ora1120 (dm-3)                      251:3    0  29,3G  0 lvm   /oracle/oradb/11.2.0
  └─oracle_vg-lv_diag (dm-4)                         251:4    0     2G  0 lvm   /oracle/diag
sdc                                                    8:32   0   600G  0 disk
└─sdc1                                                 8:33   0   600G  0 part
  └─luks-56a83e13-e115-4eb0-bbd7-21caba21736a (dm-2) 251:2    0   600G  0 crypt
    ├─data_vg-lv_appli_POGSINDV (dm-5)               251:5    0   293G  0 lvm   /appli/oracle/POGSINDV
    └─data_vg-lv_sauvebase (dm-6)                    251:6    0 195,3G  0 lvm   /sauvebase
It seems that the trouble is more during the backup process.
Here is the Agent Source Log and Job.log of the backup.

Best regards

Frédéric PRAT
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: LUKS support

Post by PTide »

Got it, thanks! And the last question - what happens if you specifically choose sdc1 and/or the underlying volumes using the "Volume level backup" mode?

Thanks
fprat
Novice
Posts: 6
Liked: never
Joined: Feb 24, 2017 6:27 am
Full Name: Frédéric PRAT
Contact:

Re: LUKS support

Post by fprat »

I'm not able to select the sdc1 volume in the "Volume level backup" mode (it doesn't appear in the interface).
In order to test, I have selected the sdc volume and the dependant volume group and logical volume and perform the backup.
The backup job is configured as baloow:

Code: Select all

# veeamconfig job info --name TestVolumeBackup
Backup job
   ID: {b68f3b6e-7703-4ae5-9a10-accc4a33e725}
   Name: TestVolumeBackup
   Repository ID: {88788f9e-d8f5-4eb4-bc4f-9b3f5403bcec}
   Repository name: [SrvSauvTlsOoc01]Default Backup Repository
   Creation time: 2017-03-07 14:27:24
   Options:
      Compression: Lz4
      Max Points: 14
   Objects for backup:
   Include Disk: data_vg
   Include Disk: sdc
In the job.log, I see the following message:

Code: Select all

[07.03.2017 15:24:51] <139837375076096> lpbcore|     Partition [/dev/sdc1] has usage type [crypto] and will be skipped.
Here is the Agent.source.log and Job.log of the last backup.

When I process to a "Restore volumes" through the Veeam Recovery Media, it doesn't restore the volume group when I try to restore the sdc volume:
Image

And when I try to restore only teh volume group, the application doesn't propose any option:
Image

As the backup process doesn't want to backup the sdc1 volumes (the encrypted volume), the restore process doesn't know where restore the volume group.

Best regards

Frédéric PRAT
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: LUKS support

Post by PTide »

Ok, I see. For now the only way to restore the LVs is to manually create sdc1 partition via Recovery Media, create VG, and manually restore LVs into that VG.

Regarding LUKS preservation - we will look what can be done with that problem so you will be able to keep encryption in place after restore.

Thanks
fprat
Novice
Posts: 6
Liked: never
Joined: Feb 24, 2017 6:27 am
Full Name: Frédéric PRAT
Contact:

Re: LUKS support

Post by fprat »

Ok, I will perform the restoration process like this.

Thank you for your availability et professionalism.

Best regards

Frédéric PRAT
Post Reply

Who is online

Users browsing this forum: No registered users and 9 guests