Does Veeam Agent for Linux support recover a LVM?

[jorgedlcruz]

Hi guys,
I have a Ubuntu Server, by default Ubuntu always creates a LVM, but trying to recover, VAL doesn't clone the whole partition table, so the LVM doesn't work:


You can see after recover, the partition table is not the same:


Would you mind to help me?

[PTide]

Hi,

You selected sda on the left side, pressed enter and selected "Restore whole disk from ..." and then picked sda from the right pane, is that correct?

[jorgedlcruz]

Yes that's correct, I have tried many times and doesn't work, would you mind to check on your side?

Best regards

[PTide]

Currently automatic restores of LV is not fully supported. For now you can turn sda5 into a PV and create a VG (or add sda5 to an existing VG). After that you can start creating LVs in the VG manually and perform restores to them aftewards.

Thanks

[LaHo]

Hello,

I tested this too.
When I do a single volume backup then I understand that there is a limitation in LVM restore.
My expectation to have a bare machine backup for my linux servers by executing "entire machine (recommended)" backup in veeam is not satisfied.
If I try to restore the entire machine backup with the recovery media iso - there is the point "restore whole disk to..." I expect that will restore whole disk (doesn’t matter LVM or without LVM) and have finally a running bootable system.


But finally it restores sda1 (/boot), while sda2 the LVM Part / (ext4 root) is missing (looks not like a whole disk...)


I am able to restore the / (ext4) to /sda2 as a separate step but booting from that will not be possible.


That makes the restore real complicated for systems having LVM and veeam for linux much inadequate. Unfortunately Ubuntu and Redhat has LVM as a preferred layout.
How will the GA of veeam handle LVM / whole disk restores?
I am the only one who expect from whole disk a whole disk???

Regards,
LaHo

[ovidiu]

Just commenting to get a notification when the topic is answered as I cannot seem to find a button to subscribe without commenting.

[PTide]

Updating this thread with a couple of questions:

I am the only one who expect from whole disk a whole disk???

Let's talk about the degree of automatization. Do you expect the software to automatically restore LV structure in case the VG is spread across many PV residing on different drives, or you just want the automatization to work with basic scenarios (like the on on the picture above)? If the former then do you expect it to work on a set of drives that differ from the original set by capacity or number of disks or both?

For now the simplest way to restore the machine to a bootable state is to perform the following steps:

1. Pick sda, choose "restore whole disk from ...", pick sda from the backup.
2. Pick sda2 and choose "create LVM physical volume". Enter VG name. Make sure that it has the same name as in the backup you're restoring from.
3. Pick free space inside the VG, choose to "restore volume from ...", pick root LV from the backup. Perform the same for the remaining free space and swap LV. Start restore.

Thanks you

[LaHo]

Hello,

I tried your suggested procedure. The difference from your procedure and my first attempt is that you select from the "Current System" while I selected from the "In Backup" location.
However, by selecting "Current-System" > sda > restore whole disk from - generates the lvm and restores lv_root. While selecting "In Backup" > sda > restore whole disk to does not restore lv_root.

After the restore has been completed, I tried to boot the system without success. Booting again from the Veeam.iso and trying to mount the lv_root and boot was successful. Looks to me that the bootloader is missing.
VM Hardware was the same only the vmhardisk has been changed for the restore test.



pvdisplay /dev/sda2


vgdisplay /dev/vg_centos62


lvscan


vgchange -a y /dev/vg_centos62

mount /dev/vg_centos62/lv_root /test and /dev/sda1 (boot) was posible and working. Booting does not start looks like grub bootloader is missing.

Regards,
Laura

[PTide]

Hi,

However, by selecting "Current-System" > sda > restore whole disk from - generates the lvm and restores lv_root

That can happen if there is already a layout on the disk. If you choose a fresh new disk to restore to then you won't get such a result and will have to create LVs manually.

After the restore has been completed, I tried to boot the system without success. Booting again from the Veeam.iso and trying to mount the lv_root and boot was successful. Looks to me that the bootloader is missing.

That's correct, it can be seen from the restore session log that the bootloader wasn't restored to /sda. Could you please clarify once again how excatly did you perform the restore? Was it "backup to disk" or "disk from backup"? In both cases these should have appeared a "loader (sda)" entry in the "Restore" column, did you see that entry?

Thanks!

[LaHo]

Hi,

That can happen if there is already a layout on the disk. If you choose a fresh new disk to restore to then you won't get such a result and will have to create LVs manually.

The disk was empty (new vmdk). My problem is that there are differences between "whole disk" restores:
left side:
selecting "In Backup" > sda > restore whole disk to :: does not restore lv_root.
right side:
selecting "Current-System" > sda > restore whole disk from :: generates the lvm and restores lv_root

Could you please clarify once again how exactly did you perform the restore? Was it "backup to disk" or "disk from backup"?

Do you mean restore from/to? I did both restores as described above.

Still wondering why bootloader is missing by whole disk...

Thank you
LaHo

[PTide]

The disk was empty (new vmdk). My problem is that there are differences between "whole disk" restores:
left side:
selecting "In Backup" > sda > restore whole disk to :: does not restore lv_root.
right side:
selecting "Current-System" > sda > restore whole disk from :: generates the lvm and restores lv_root

something is not right here...In other words, you've tried the following two cases:

1. You took a newly created vmdk-1 and selected "In Backup" > sda > restore whole disk to" and that did not restored lv_root. (totally expected for Beta)

2. You took another newly created vmdk-2 and selected "Current-System" > sda > restore whole disk from" and that did restore lv_root. (unexpected for Beta)

Is that correct?

Thanks

[LaHo]

Hi,

I tried both again, but in each case the LVM is not created by just selecting restore whole disk form/to.
So you are right. I have to follow your instructions to create LVM

1. Pick sda, choose "restore whole disk from ...", pick sda from the backup.
2. Pick sda2 and choose "create LVM physical volume". Enter VG name. Make sure that it has the same name as in the backup you're restoring from.
3. Pick free space inside the VG, choose to "restore volume from ...", pick root LV from the backup. Perform the same for the remaining free space and swap LV. Start restore.

Now I was able to restore the lv_root and lv_swap with success (lv_root is mountable).
But If I try to restore the bootloader I got the error "Unable to convert BootloaderType::type [-1080767520] to string.


Is that a known issue with the beta?

about the degree of automatization.

I would expect for a simple one harddisk LVM configuration (on most server and client installations for the system) that a "whole disk restore" should be a straight forward process" which will restore the data, partiontable and bootloader and be able to boot again, without the need of knowledge of LVM structure and creation. However to have the option to customize LVM or rebuild the new partition on your own is a nice feature.
That should work with same disk size or larger. Nice addon would be to shrink...

Regards
LaHo

[PTide]

But If I try to restore the bootloader I got the error "Unable to convert BootloaderType::type [-1080767520] to string.

"Restore whole disk from" is supposed to restore a bootloader as well...regarding the BootLoaderType error - our QA team saw that error before, however we did not manage to reproduce it. Does it happen every time? Could you please describe settings of the VM that you are trying to restore?

I would expect for a simple one harddisk LVM configuration (on most server and client installations for the system) that a "whole disk restore" should be a straight forward process" which will restore the data, partiontable and bootloader and be able to boot again, without the need of knowledge of LVM structure and creation. That should work with same disk size or larger.

Noted as a feature request.

Nice addon would be to shrink...

Do you mean LV schrink? That option (and many others) is available in a command line interface via lvresize command.

But If I try to restore the bootloader I got the error "Unable to convert BootloaderType::type [-1080767520] to string. Is that a known issue with the beta?

Yes, that's a known bug.

Thanks

[LaHo]

Hi,

that´s good to know.

Thank you very much for your help and time.
I will wait for the next version.

Regards
LaHo

[ASG]

Sorry for digging out such an old post - but as I've been trying to do a restore from a backup to a single disk single lvm setting. Appart from missing cryptsetup on the veeam recovery media (which is a bit of a problem for me since our linux hosts are encrypted) I'm unable to restore the LVM.

Manually adding the VG works, however restoring the partitions inside the LVM doesn't work.


Adding them reduces the free space of the VG but they are not listed in the left pane and are not restored when doing the restore. Deleting the VG is not possible, cleaning the partition table doesn't help in this case as the VG stays.


Edit:
I've added the VG manually and tried to restore the LVs. As you can see the free space was reduced. However the LVs are not listed.


But as you can see here - there is no intention to restore the LVs.


Edit2:
Case #03444489

[PTide]

Hi,

It is a known issue with the most recent version of Recovery Media (3.0.0.865). Please try using ISO v2.0.1

Thanks!

[ASG]

Thanks for your quick reply, I didn't read this in the Top Issue tracker post - maybe it's worth adding it there?

Apart from that - is it possible to get a LUKS cryptsetup on the disk? I've tried adding veeam agent to my own custom livecd that I use for installing (unpacking the image) and encrypting - however I'm only getting the option to restore files (that is mounting the image in a folder). I would write a script that's rsyncing then - however it's pretty static regarding the folder structure... Having the option to create the encrypted volumes via the veeam cd and then let veeam restore seems like the better way.

Edit:
Or is there a way to make the veeam agent mount the partitions/lvs into single folders? That would make the scripting A LOT easier versus the whole filesystem in a single mountpoint.

[PTide]

I didn't read this in the Top Issue tracker post - maybe it's worth adding it there?

Good call, added.

Having the option to create the encrypted volumes via the veeam cd and then let veeam restore seems like the better way.

Agree. Would just CLI interface suffice?

Or is there a way to make the veeam agent mount the partitions/lvs into single folders? That would make the scripting A LOT easier versus the whole filesystem in a single mountpoint.

You mean that you want to see different mountpoints from the original system not in a single /mnt/backup directory, but something like /mnt/backup/vg-root, /mnt/backup/vg-home etc?

Thanks!

[ASG]

Hi PTide,

yes CLI would suffice - I'm doing it on the CLI now anyway as I'll have to setup things a bit special.

And yes, different mountpoints (or maybe an option - some people maybe need the single mountpoint for just restoring a file regardless on which volume it resides) would make things a lot easier in some cases (scripting to rsync the mountpoint to just created volumes for example).

Thanks!
ASG

[ASG]

Just a quick add (since I can't edit my other post)

- Using the old ISO indeed works restoring the LVM - however the system is not bootable since the LVM is different (it was encrypted when backed up) and Ubuntu refuses to boot. Apart from that the system was unencrypted (and encrypting the disks is a must-do) and this wouldn't be a way to do a restore anyway.

- I've added the Agent to my livecd, created the partitions and encrypted lvm manually and then mounted the restore point via the agent gui. Afterwards I've made a script that rsyncs the different partitions onto the target volumes via excludes (since the whole directory-tree is mounted under /mnt/backup) and this works fine. Afterwards chrooting and a bit of changing crypttab and grub-install and the system is booting just fine as it should.

While having the different mountpoints available would be easier (no need to work with excludes etc... when rsyncing the directories) it works with the existing mountpoint and I'm happy. Scripted everything apart from mounting the veeam Backup and gave it a few testruns - everything worked.

[PTide]

Ok, I just want to wrap my head around your setup:

On the original system you had your root LV sitting on an encrypted luks device, right? What about /boot - is it encrypted too?

Thanks!

[ASG]

The original system looks like this:

Code: Select all

sda (RAID)
- sda1 EFI /boot/efi
- sda2 boot /boot/
- sda3 (encrypted luks)
 - LVM
  - ROOT /
  - DATA /data
  - SWAP swap

boot and efi are not encrypted, but the other volumes are encrypted. Some of our servers have a second data volume (since the data on these volumes doesn't need to be backed up) but most of our Linux systems use this layout.

[houla]

Does moving to 3.0.0.957 resolve the LVM issue as 2.0.1.665 does ?

Thanks!

[PTide]

Hi,

If you're referring to VAL 3.0.0.957 then it is a private fix that is dedicated to problems #2 and #8 from the Top Issues list.
If you were referring to ISO v3.0.0.957, then you should consult the support engineer that gave it to you.

Thanks!

[houla]

Referring to the ISO v3.0.0.957 resolving the LVM issue as ISO 2.0.1.665 seems to.

And the ISO v3.0.0.957 was downloaded from a Veeam files directory - no support engineer was involved.

Thanks

[nielsengelen]

If you want to use the ISO for version 3 please contact support for the hotfix. As v3 has brought changes, it may happen that you can’t restore backups when using the ISO for version 2.

[emilec]

I have also been having issues restoring a CentOS 7 system with LVM with VAL version 3.0. Using recovery 3.0.1.1046 with customizations from the OS. The restore wizard seems to be doing what it needs to do and in fact booting from a CentOS recovery ISO shows all the correct partitions. So I dug a bit more and for some reason the bootable partition is not being set (I am still using BIOS mode). The way to fix this was to boot a CentOS ISO and use parted to run "disk_set pmbr_boot on"

Notice the disk flag is blank

Code: Select all

# parted -l
Model: DELL PERC H730P Adp (scsi)
Disk /dev/sda: 2399GB
Sector size (logical/physical): 512B/512B
Partition Table: gpt
Disk Flags:

Number  Start   End     Size    File system  Name  Flags
 1      1049kB  2097kB  1049kB                     bios_grub
 2      2097kB  539MB   537MB   xfs
 3      539MB   34.1GB  33.6GB                     lvm

To fix

Code: Select all

parted /dev/sda
disk_set pmbr_boot on

Details will now show

Code: Select all

Disk Flags: pmbr_boot

So one feature request: please can you include parted in the recovery image? In theory I should be able to complete the recovery, drop to the CLI and fix it before rebooting.

[emilec]

I also got some feedback from Veeam support who suggested I set the boot flag with fdisk. fdisk on CentOS specifically warns against using fdisk (2.23.2) on gpt disks and there is no "a" option to set the bootable flag. However the fdisk (2.27.1) on the veeam recovery media has an "A" option under the expert settings to set legacy BIOS bootable flag. I used that to set it on /dev/sda1 after the restore and the system boots (listed as bios_grub in parted).

Information on the internet seems to be somewhat conflicted on manipulating gpt with fdisk.

[PTide]

Hi,

Good call. We'll consider adding parted into the image.

Thanks!