V6 Agent fails if there's unsupported volumes (DRBD, in our case)

[qurvax]

Hello,
We just noticed this on our experimental boxes with Oracle Linux running 5.4.17-2136.315.5.el8uek.x86_64, veeamsnap Version: 6.0.0.1060.
We have tried to configure B&R 11 server with newest agent for volume based backups, and If there's active DRBD volumes it fails. If we remove those, everything works. Is there any workaround for this? To be clear, we don't want those unsupported volumes backed up, just some of host's "native" volumes. Older agent version (5.0.2?) didn't have this problem, it just skipped those. I do belive it was posible to configure some devices to be exluded, but wasn't able to find any info how to do that for V6.

Code: Select all

[08.03.2023 22:41:20.082] <140540548982528> lsm      |     Add block devices to graph.
[08.03.2023 22:41:20.082] <140540548982528> lsm      |       Device [8:0], path [/dev/sda]. Type: [scsi]
[08.03.2023 22:41:20.082] <140540548982528> lsm      |       Device [8:1], path [/dev/sda1]. Type: [scsi]
[08.03.2023 22:41:20.082] <140540548982528> lsm      |       Device [8:2], path [/dev/sda2]. Type: [scsi]
[08.03.2023 22:41:20.082] <140540548982528> lsm      |       Device [8:3], path [/dev/sda3]. Type: [scsi]
[08.03.2023 22:41:20.082] <140540548982528> lsm      |       Device [8:4], path [/dev/sda4]. Type: [scsi]
[08.03.2023 22:41:20.082] <140540548982528> lsm      |       Device [147:1], path [/dev/drbd1]. Type: [unknown]
[08.03.2023 22:41:20.082] <140540548982528> lsm      |       Device [147:2], path [/dev/drbd2]. Type: [unknown]
[08.03.2023 22:41:20.082] <140540548982528> lsm      |       Device [147:3], path [/dev/drbd3]. Type: [unknown]
[08.03.2023 22:41:20.082] <140540548982528> lsm      |       Device [147:4], path [/dev/drbd4]. Type: [unknown]
[08.03.2023 22:41:20.082] <140540548982528> lsm      |       Device [252:0], path [/dev/dm-0]. Type: [dm]
[08.03.2023 22:41:20.082] <140540548982528> lsm      |       Device [252:1], path [/dev/dm-1]. Type: [dm]
[08.03.2023 22:41:20.082] <140540548982528> lsm      |       Device [252:2], path [/dev/dm-2]. Type: [dm]
[08.03.2023 22:41:20.082] <140540548982528> lsm      |       Device [252:3], path [/dev/dm-3]. Type: [dm]
[08.03.2023 22:41:20.082] <140540548982528> lsm      |       Device [252:4], path [/dev/dm-4]. Type: [dm]
[08.03.2023 22:41:20.082] <140540548982528> lsm      |       Device [252:5], path [/dev/dm-5]. Type: [dm]
[08.03.2023 22:41:20.082] <140540548982528> lsm      |       Device [252:6], path [/dev/dm-6]. Type: [dm]
[08.03.2023 22:41:20.082] <140540548982528> lsm      |   Enum all block devices. ok.
[08.03.2023 22:41:20.082] <140540548982528> lsm      |   Process dm devices.
[08.03.2023 22:41:20.082] <140540548982528> lsm      |     Probing partition table on [/dev/dm-6].
[08.03.2023 22:41:20.083] <140540548982528> lsm      |       Partition table type (on: /dev/dm-6): [dos].
[08.03.2023 22:41:20.083] <140540548982528> dsk      |       Async read mode is [true]
[08.03.2023 22:41:20.083] <140540548982528> dsk      |       I/O statistics for '/dev/dm-6': 512 bytes read unaligned (1 requests)
[08.03.2023 22:41:20.083] <140540548982528> lsm      |       Objects relationship: link child [PartitionTable] 'dos 20484852' <-> parent [BlockDevice] '/dev/dm-6' single flag: '1'.
[08.03.2023 22:41:20.083] <140540548982528> lpbdevenu|       Try get partition from blkid for device [/dev/drbd4].
[08.03.2023 22:41:20.088] <140540548982528> lpbdevenu|       Try get partition from blkid for device [/dev/drbd4]. Failed.
[08.03.2023 22:41:20.088] <140540548982528> lpbdevenu|       Try get partition from sysfs for device [/dev/drbd4].
[08.03.2023 22:41:20.088] <140540548982528> lpbdevenu|       Try get partition from sysfs for device [/dev/drbd4]. Failed.
[08.03.2023 22:41:20.088] <140540548982528> lpbdevenu|       Try get partition by DevNo.
[08.03.2023 22:41:20.088] <140540548982528> lpbdevenu|       Try get partition by DevNo. Failed.
[08.03.2023 22:41:20.088] <140540548982528> lsm      |     Probing partition table on [/dev/dm-6]. Failed.
[08.03.2023 22:41:20.088] <140540548982528> lsm      |   Process dm devices. Failed.
[08.03.2023 22:41:20.088] <140540548982528> lsm      | Enum system. Failed.
[08.03.2023 22:41:20.088] <140540548982528> prtcl    | ERR |Failed to get partition information, possible device type is not supported.
[08.03.2023 22:41:20.088] <140540548982528> prtcl    | >>  |--tr:CBlkidPartitionGetter: Failed to get blkid partition [/dev/drbd4] of device [/dev/dm-6].

[Mildur]

Hello quarvax

Could you please open a support case for this issue?
Doesn‘t sound right to me. If you use volume level backup and don’t select this DRBD volumes, then it shouldn‘t be backed up by the job.

Best,
Fabian

[PTide]

Please also try to use this workaround to exclude DRBD devices.

Thanks!

[qurvax]

@Mildur: yeah, it's kinda strange. Also, it worked with previous version without any tricks. I'll contact support if theres nothing 'obvious' to do anymore.

@PTide: Nope, nothin i add to IgnoreDevices seems to change anything in the logs. Tried dm-x, drbd volumes, underlying block device (phisical disk)...

[PTide]

If you've tried excluding all drbd-related devices ([including /dev/drbd4 and /dev/dm-6) and restarted the service and that did not help, then please open a support case and let me know the number.

Thanks!

[qurvax]

I did try to exclude a bunch of stuff. Didn't see any difference. As if it fails way before any of that matters.
I have contacted support on this, Case #05962098

[PTide]

Ok, I requested an escalation for your case. Let's see what we will find there.

Thanks!

[qurvax]

Thanx PTide, much appreciated. Last news is this got all the way to dev's, after gathering some strace dumps.

[qurvax]

Good news: DEV's were able to determine root cause. Bad news: It's deeeeeep, so will fix "someday".
Whole story:
"Enumerator assumes that if device has partition table, then all dependent devices considered as partitions, but in this case it is virtual DRBD device."©Veeam Support
That means anything that has custom stuff on block device (metadata etc...) that's aint looking like normal partition, will, _probably_, screw V6 agent up.

[PTide]

Thank you for sharing. Will try to speed up the fix as this particular functionality (exclusions) is probably the only way to make the agent work with exotic configs (e.g. DRBD).

[qurvax]

That's more like "agent should NOT assume anything", to be usable. I'm not saying exclusions don't work, because I wasn't able to get this far - agent just doesn't accept the fact, that there may be something exotic, and cannot function with configs as these. This is not some programming error, it's just a limitation (a nasty one tho). So more like "needs improvement". I wonder how i'm the only one out here noticed...

[PTide]

Right. We will figure out how to improve it so that nothing else breaks.

Thanks!