Failed to flush file buffers

[DavidS]

I recently encouter a strange error on Veeam fullbackups.

Code: Select all

07.01.2015 18:40:39 :: Error: Incorrect function.
Failed to flush file buffers. File: [E:\Backup\BCK_Name_ALL\BCK_Name_ALL2015-01-07T113550.vbk].

A fullbackup uses about 3-4TB in our environment, fullbackups worked before. I think this error occures since the upgrade to Veeam v8.

As soon as the Backupfile (vbk) reaches a size about 2 TB+, this error occurs and the session stops. If I do a Retry the performance drops to around 5MB/s and at some point the retry also fails with nothing written to the vbk.

What components I can exclude so far are the storage, the network and the server performance.

I suspect it has something to do with the repository, its an Windowsserver Repository, an iSCSI-Storage with a 17TB LUN behind that.

Now the blocksize we used in NTFS is 64k, maybe this is the issue? Would 8k be the better blocksize?

And what about REFS, would a Volume formatted with REFS maybe prevent this failure (although the REFS min. blocksize is alos 64k)

Thanks
David

[dellock6]

Hi David,
this kind of errors are better to be investigated by support, so please open a ticket.

Regarding block size, default NTFS block size is ok, and if any, to gain better performances it should be increased rather than reduced. Veeam smaller blocks are 128KB (wan deduplication at 256kb plus compression adding around 2x reduction) so 64k or 128k is fine. Also, take care of the underlying storage, also those volume should use the same block size, to avoid write amplification. Alignment of the different layers is also another concern, but since you are talking about ReFS I suppose you are using Windows 2012, so alignment usually is not a problem since in modern windows edition (2008 and above) is done automatically.

[DavidS]

Hello Everyone

A little followup to my problem.
As Luca suggested I opened a support ticket.

The support couldn't really help us solving this problem so we invistigated some more hours by ourself.

We adjusted some iSCSI settings for optimization, finally this optimizations solved our problem:

TcpAckFrequenzy

Code: Select all

Edit all Adapters, used for the iSCSI connection with the Value "TcpAckFrequenzy = 1". If you use multipath, edit this settings for every adapter.

	Create this DWORD key with a value of 1 :
	HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces\<Interface GUID>\TcpAckFrequency

Nagle

Code: Select all

Disabled Nagle for iSCSI

Create this DWORD key with a value of 1:
HKLM\SYSTEM\CurrentControlSet\Control\Class\{4D36E97B-E325-11CE-BFC1-08002BE10318}\<Instance Number>\Parameters\iSCSIDisableNagle

Last step -> Reboot Veeam Server

This solved our problem. We have 2 iSCSI connections, one over a 10GB Ethernet and the other with a simple 1GB Ethernet.
Though, we only encoutered this problems with the 10GB Ethernet. It seems Veeam, iSCSI and 10GB aren't the best friends

I hope this solutions may help someone with similar problems.

Regards
David

[spgsit5upport]

No idea what it is about iSCSIDisableNagle, as such entry no longer is used for anything, it is only TcpAckFrequency to deactivate Nagle Algorithm on iSCSI interfaces

https://gallery.technet.microsoft.com/s ... m-66ca7608

[spgsit5upport]

And TcpNoDelay as per this - https://kb.globalscape.com/Knowledgebas ... 10438.aspx