Unable to allocate memory

[horizn]

Hi,
I am trying to run backup job on one of Debian 6 servers, but unfortunately it end up with "Unable to allocate memory" for some reason. Server has 16GB of memory installed, and about 14GBs are free, as well as there is a lot of disk space:

Code: Select all

[02.02.2017 16:04:29] <3013114736> vsnap  |               Writing snapshot file blocks (path: /veeamsnapshotdata.0, size: 21474836480) ok.
[02.02.2017 16:04:29] <3013114736> vsnap  |             Allocating new snapshot file (path: /veeamsnapshotdata.0, size: 21474836480) ok.
[02.02.2017 16:04:31] <3013114736> vsnap  |           Snapshot file successfully created with 'fiemap'
[02.02.2017 16:04:31] <3013114736> vsnap  |         Try to make snapshot data info for all devices on snapshot device 9:0.
[02.02.2017 16:04:31] <3013114736> vsnap  |       Checking common snapshot data file exist. Data host device [/dev/md0], file [/veeamsnapshotdata.0], size 21474836480. Failed.
[02.02.2017 16:04:31] <3013114736> lpbcore|     Creating snapshot storage. Storage type: stretch file Failed.
[02.02.2017 16:04:31] <3013114736> lpbcore|   Creating machine snapshot using VeeamSnap kernel module. Failed.
[02.02.2017 16:04:31] <3013114736> vsnap  |   Closing VeeamSnap control.
[02.02.2017 16:04:31] <3013114736> vsnap  |   Closing VeeamSnap control. ok.
[02.02.2017 16:04:31] <3013114736> lpbcore| ERR |Cannot allocate memory
[02.02.2017 16:04:31] <3013114736> lpbcore| >>  |Failed to execute IOCTL_SNAPSHOT_COMMON_DATAINFO.
[02.02.2017 16:04:31] <3013114736> lpbcore| >>  |--tr:SnapshotDataMakeCommonFile failed.
[02.02.2017 16:04:31] <3013114736> lpbcore| >>  |--tr:CreateSnapshotStorageCommon failed.
[02.02.2017 16:04:31] <3013114736> lpbcore| >>  |--tr:Failed to create machine snapshot.
[02.02.2017 16:04:31] <3013114736> lpbcore| >>  |--tr:Unable to create snapshot.
[02.02.2017 16:04:31] <3013114736> lpbcore| >>  |--tr:Failed to execute method [0] for class [N10lpbcorelib11interaction9proxystub21CResourcesServiceStubE].
[02.02.2017 16:04:31] <3013114736> lpbcore| >>  |An exception was thrown from thread [3013114736].
[02.02.2017 16:04:34] <3013114736>        |   Closing socket device.
[02.02.2017 16:04:34] <3013114736> lpbcore| Starting proxystub protocol dispatch loop. ok.
[02.02.2017 16:04:34] <3013114736>        | Closing socket device.
[02.02.2017 16:04:34] <3013114736>        | Thread finished. Role: 'peer 127.0.0.1:33075'.
[02.02.2017 16:04:34] <3029900144> lpbcore| Manager process [13554] has been shutdown.
[02.02.2017 16:04:34] <3029900144>        | Thread finished. Role: 'Manager process [13554] shutdown handler.'.
[02.02.2017 16:04:34] <3049687920> lpbcore| Cleaning up resources for job process [13554].
[02.02.2017 16:04:34] <3021507440> lpbcore| ERR |Backup job has failed.
[02.02.2017 16:04:34] <3021507440> lpbcore| >>  |Cannot allocate memory
[02.02.2017 16:04:34] <3021507440> lpbcore| >>  |Failed to execute IOCTL_SNAPSHOT_COMMON_DATAINFO.
[02.02.2017 16:04:34] <3021507440> lpbcore| >>  |--tr:SnapshotDataMakeCommonFile failed.
[02.02.2017 16:04:34] <3021507440> lpbcore| >>  |--tr:CreateSnapshotStorageCommon failed.
[02.02.2017 16:04:34] <3021507440> lpbcore| >>  |--tr:Failed to create machine snapshot.
[02.02.2017 16:04:34] <3021507440> lpbcore| >>  |--tr:Unable to create snapshot.
[02.02.2017 16:04:34] <3021507440> lpbcore| >>  |--tr:Failed to execute method [0] for class [N10lpbcorelib11interaction9proxystub21CResourcesServiceStubE].
[02.02.2017 16:04:34] <3021507440> lpbcore| >>  |--tr:Failed to create volume snapshot.
[02.02.2017 16:04:34] <3021507440> lpbcore| >>  |Backup job has failed.
[02.02.2017 16:04:34] <3021507440> lpbcore| >>  |An exception was thrown from thread [3013114736].
[02.02.2017 16:04:34] <3021507440> lpbcore| Retry is disabled. Attempts counter '3', isRetriable flag 'false'.
[02.02.2017 16:04:34] <3021507440>        | Thread finished. Role: 'backup job retry handler'.
[02.02.2017 16:04:34] <3038292848>        | Closing socket device.
[02.02.2017 16:19:20] <3066473328> lpbcore| LpbCfgSession: Tcp loop. Failed.
[02.02.2017 16:19:20] <3066473328> lpbcore| ERR |LpbCfgSession failed.
[02.02.2017 16:19:20] <3066473328> lpbcore| >>  |read: End of file
[02.02.2017 16:19:20] <3066473328> lpbcore| >>  |--tr:Cannot read data from the socket. Requested data size: [4].
[02.02.2017 16:19:20] <3066473328> lpbcore| >>  |An exception was thrown from thread [3066473328].
[02.02.2017 16:19:20] <3066473328>        | Closing socket device.
[02.02.2017 16:19:20] <3066473328>        | Thread finished. Role: '(async) LPBConfig session (127.0.0.1:33070)'.
[02.02.2017 16:19:20] <3058080624>        |   Closing socket device.
[02.02.2017 16:19:20] <3058080624> lpbcore| Starting proxystub protocol dispatch loop. ok.
[02.02.2017 16:19:20] <3058080624>        | Closing socket device.
[02.02.2017 16:19:20] <3058080624>        | Thread finished. Role: 'peer 127.0.0.1:33069'.

What could be a reason?

[PTide]

Hi,

Please provide some info about filesystems that are in use on the server - post lsblk -af output.

Thank you

[horizn]

Code: Select all

/dev/sdb2: UUID="ca7d8b36-c30a-e7b2-f4a8-c511fab987c4" LABEL="build7:0" TYPE="linux_raid_member" 
/dev/sda2: UUID="ca7d8b36-c30a-e7b2-f4a8-c511fab987c4" LABEL="build7:0" TYPE="linux_raid_member" 
/dev/sda1: UUID="cf23bdec-e70e-48eb-9a4c-dd1cd9c44dc2" TYPE="ext3" 
/dev/sdb5: UUID="200a0976-50a8-4274-984c-10f4a0d2dcfd" TYPE="swap" 
/dev/md0: UUID="c34ebc80-48e2-42a6-aa75-c17394f77e27" TYPE="ext3" 
/dev/loop0: UUID="36353de8-4d78-4d09-b40c-553fd03ec76f" TYPE="ext3" 

[PTide]

It seems that the system failed to create a snapshot for some reason. Please contact our support team directly and post your case ID.

Thank you

[horizn]

It made backup successfully at the second attempt, and then incremental without any issues.

[PTide]

Great! Anyway, please contact our support team so they can get logs from you as we'd like to investigate if there are any hidden bugs that could cause such a failure.

Thank you