CentOS 6 & error with target Veeam Backup server

[dcit]

Hello,

I am trying Veeam Agent for Linux 4.0.1.2365.1.el6 on CentOS 6.10.

I have not been able to backup to samba as CentOS 6 supports mounting only SMB1, which is insecure and we have it disabled on our servers.

I have been able to backup successfully to unfs3 server @ cygwin (we are considering to backup to Windows machine serving for backup purpose in our dedicated housing in datacenter, for this reason I am trying NFS server on Windows), but only when managing Veeam Agent by CLI, e.g. because GUI is trying NFS version 4 (and is without option to specify version of NFS) and when I have tried to mount it manually before and in GUI select local destination and that mount point I have got Permission denied (chown on that NFS share from cygwin does not work - this is probably reason for that).

Then I have tried to backup to our Veeam Backup server (not our preferred option in this case, as we prefer to have backups in that datacenter, and we have Veeam Backup server only in our internal network), as I was wondering if it will be working. But it fails with errors such as:

Code: Select all

      20:06:22         [error] Backing up sda                                                               00:01:03
      20:07:29         [error] Failed to perform backup
      20:07:29         [error] boost::thread_resource_error: Resource temporarily unavailable
      20:07:29         [error] Failed to upload disk
      20:07:29         [error] Agent failed to process method {DataTransfer.SyncDisk}
      20:07:29         [error] Exception from server: Reconnectable protocol device was closed
      20:07:29         [error] Unable to retrieve next block transmission command. Number of already processed blocks: [0]
      20:07:29         [error] Failed to download disk '0004ffb2'

I have tried to update server Veeam Backup & Replication 10 Cumulative Patch 1 -> Veeam Backup & Replication 10 Cumulative Patch 2. But still errors.

Then I have noticed that there is newer version of Veeam Backup server.

Is it necessary to update to Veeam Backup & Replication 10a to have VAL 4.0.1 working?

Best Regards,
Tomas JIRKA

[dcit]

I have upgraded to Veeam Backup & Replication 10a, but it still ends with error, now specifically:

Code: Select all

      22:51:27         [error] Backing up sda                                  00:01:18
      22:52:49         [error] Failed to perform backup
      22:52:49         [error] boost::thread_resource_error: Resource temporarily unavailable
      22:52:49         [error] Failed to upload disk
      22:52:49         [error] Agent failed to process method {DataTransfer.SyncDisk}
      22:52:49         [error] Processing finished with errors at 2020-08-19 22:52:49 GMT

[Natalia Lupacheva]

Hi,

It doesn't seem like a known issue.
Please provide a support case id, it's mandatory when you are posting an error or a technical issue.

Thanks!

[PTide]

Regarding the SMB version and UI - please check this workaround that our fellow user has suggested.

As for the error (Exception from server: Reconnectable protocol device was closed) - it looks pretty much like a connectivity issue to me. Please work with our support team as Natalia has suggested.

Thank you

[dcit]

I was not sure if I can report it to Veeam support, when we have not bought Veeam Agent for Linux, only Veeam Backup&Replication server for three ESXi servers.

But as you requested, I have created support case, case id: #04347382

Btw tt occurred to me now that for virtual machine on which I am trying it we have actually license for VAL, because that VM is running on ESXi for which we have license in VBR, and that should give license not only for image level backup of whole VM, but for Veeam Agent inside too.

[dcit]

Regarding the SMB version and UI - please check this workaround that our fellow user has suggested.

As for the error (Exception from server: Reconnectable protocol device was closed) - it looks pretty much like a connectivity issue to me. Please work with our support team as Natalia has suggested.

Thank you

I have user that workaround today for backup of vCenter Appliance, so it was useful to know that tip.
But for that CentOS 6 that trick is not working for me. Probably because RHEL 6 / CentOS 6 does not support SMB2 in smbclient.

https://support.hpe.com/hpesc/public/do ... 27394en_us
"The kernel version included in RHEL 6 contains an older cifs.ko kernel
module that only supports SMB1 protocol. SMB2 (and later) protocols
require a later cifs.ko and thus a later kernel. RHEL 7 has initial support
for SMB2 protocol in the cifs.ko kernel module and so can act as a SMB2
client. "

[dcit]

Btw today I have tried to add USB flash disk to our ESXi server and then to add it to test VM.
I have mounted that USB drive to /mnt/usb and I have tried to backup on it, resulted with familiar errors:

Code: Select all

15:29:40         [error] Backing up sda 1.4 GB at 97.1 MB/s (33%)                                                                    00:00:52
15:30:36         [error] Failed to perform backup
15:30:36         [error] std::bad_alloc
15:30:36         [error] Failed to download disk '0004ffb2'
15:30:36         [error] write: Connection reset by peer
15:30:36         [error] Failed to upload disk
15:30:36         [error] Agent failed to process method {DataTransfer.SyncDisk}
15:30:36         [error] Processing finished with errors at 2020-08-21 15:30:36 GMT

[dcit]

Then my colleague plugged USB drive to physical server in housing in datacenter with CentOS 6 too (to be able to backup that server is reason why I was testing it in VM) .
I was hoping that may be there are some VMware USB tunneling limitations and on physical HW it will work fine. But alas, it did not made me happy. In that server we have two SSDs in software RAID1 and some incompatibily with it is probably causing that it backups only bootloader or something like that:

Code: Select all

     Summary                              Data

     Duration:           00:00:12         Processed:          28.6 GB (100%)
     Processing rate:    183.1 KB/s       Read:               1.1 MB
     Bottleneck:         Source           Transferred:        14.3 KB (76x)

      Time             Action                                                                Duration

      15:51:07         Job BackupJob1 started at 2020-08-21 15:51:07 GMT
      15:51:08         Preparing to backup
      15:51:08         Creating volume snapshot                                              00:00:00
      15:51:09         Starting full backup to Repository_1
      15:51:09         Backing up BIOS bootloader on /dev/md0                                00:00:00
      15:51:15         Backed up md0 1.1 MB at 4 MB/s                                        00:00:00
      15:51:15         Backing up summary.xml                                                00:00:00
      15:51:17         Releasing snapshot                                                    00:00:02
      15:51:19         Processing finished at 2020-08-21 15:51:19 GMT

File level backup of some directories (I have tried /etc/ and /usr, after backup of all directories (except /mnt) ended with error [error] Unable find filesystem type for device with number: [0:5]) works fine.

[dcit]

Btw I have tried to change partition types

Code: Select all

  Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *        2048     3121151     1559552   83  Linux
Partition 1 does not end on cylinder boundary.
/dev/sda2         3121152    70230015    33554432   82  Linux swap / Solaris
/dev/sda3        70230016   488396799   209083392   83  Linux

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *        2048     3121151     1559552   83  Linux
/dev/sdb2         3121152    70230015    33554432   82  Linux swap / Solaris
/dev/sdb3        70230016   488396799   209083392   83  Linux

->

   Device Boot      Start         End      Blocks   Id  System
/dev/sda1   *           1         195     1559552   fd  Linux raid autodetect
/dev/sda2             195        4372    33554432   fd  Linux raid autodetect
/dev/sda3            4372       30402   209083392   fd  Linux raid autodetect

   Device Boot      Start         End      Blocks   Id  System
/dev/sdb1   *        2048     3121151     1559552   fd  Linux raid autodetect
/dev/sdb2         3121152    70230015    33554432   fd  Linux raid autodetect
/dev/sdb3        70230016   488396799   209083392   fd  Linux raid autodetect

and then reboot

It rebooted correctly, but it does not helped for veeam agent backup, still the same problem.

[dcit]

I have found that regarding veeam backups of that VM, problem was in not enough memory.

When I have been trying it again I have noticed commit size very close to limit, and big memory consumtion in htop.

Almost just before backup failed:

Code: Select all

cat /proc/meminfo | grep -i commit
CommitLimit:     1199020 kB
Committed_AS:    1146572 kB

So I have tried to add memory to that VM (512MB -> 1024 MB).

Now backup to USB flash disk works, backup to VBR repository works too.

Btw even before that memory rise backup to NFS server (unfs3 version 0.9.22-1 & rpcbind version 0.2.1-1 @ cygwin @ Windows 10) worked. May be it is less memory demanding.

After adding memory I have tried to backup to NFS share by WinNFSd, but that still does not work. But I can accept that veeam needs some NFS feature(s) that this program does not provide.

More problem is that backup of that physical machine does not work, veeam backups of that VM were just for test.

It would be nice if checking "not enough memory" can be added to VAL, as warning/error messages I have received did not mention it, and that mislead me.

[PTide]

But for that CentOS 6 that trick is not working for me. Probably because RHEL 6 / CentOS 6 does not support SMB2 in smbclient.

I didn't know that, thank you for that info!

In that server we have two SSDs in software RAID1 and some incompatibily with it is probably causing that it backups only bootloader or something like that:

That's weird. What happens if you select "Entire machine" mode or select the mountpoint where md0 is mounted at?

More problem is that backup of that physical machine does not work, veeam backups of that VM were just for test.

So, the physical machine has enough RAM, but the Agent still does not back up the whole disk (only the bootloader).

Is that correct?

Thanks!