Availability for the Always-On Enterprise
Post Reply
DavidS
Veeam ProPartner
Posts: 11
Liked: 6 times
Joined: Aug 18, 2014 12:00 pm
Full Name: David
Location: Switzerland
Contact:

Failed to flush file buffers

Post by DavidS » Jan 08, 2015 10:11 am

I recently encouter a strange error on Veeam fullbackups.

Code: Select all

07.01.2015 18:40:39 :: Error: Incorrect function.
Failed to flush file buffers. File: [E:\Backup\BCK_Name_ALL\BCK_Name_ALL2015-01-07T113550.vbk].
A fullbackup uses about 3-4TB in our environment, fullbackups worked before. I think this error occures since the upgrade to Veeam v8.

As soon as the Backupfile (vbk) reaches a size about 2 TB+, this error occurs and the session stops. If I do a Retry the performance drops to around 5MB/s and at some point the retry also fails with nothing written to the vbk.

What components I can exclude so far are the storage, the network and the server performance.

I suspect it has something to do with the repository, its an Windowsserver Repository, an iSCSI-Storage with a 17TB LUN behind that.

Now the blocksize we used in NTFS is 64k, maybe this is the issue? Would 8k be the better blocksize?

And what about REFS, would a Volume formatted with REFS maybe prevent this failure (although the REFS min. blocksize is alos 64k)

Thanks
David

dellock6
Veeam Software
Posts: 5437
Liked: 1480 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Failed to flush file buffers

Post by dellock6 » Jan 08, 2015 11:12 am

Hi David,
this kind of errors are better to be investigated by support, so please open a ticket.

Regarding block size, default NTFS block size is ok, and if any, to gain better performances it should be increased rather than reduced. Veeam smaller blocks are 128KB (wan deduplication at 256kb plus compression adding around 2x reduction) so 64k or 128k is fine. Also, take care of the underlying storage, also those volume should use the same block size, to avoid write amplification. Alignment of the different layers is also another concern, but since you are talking about ReFS I suppose you are using Windows 2012, so alignment usually is not a problem since in modern windows edition (2008 and above) is done automatically.
Luca Dell'Oca
EMEA Cloud Architect @ Veeam Software

@dellock6
http://www.virtualtothecore.com/en/
vExpert 2011-2012-2013-2014-2015-2016-2017-2018
Veeam VMCE #1

DavidS
Veeam ProPartner
Posts: 11
Liked: 6 times
Joined: Aug 18, 2014 12:00 pm
Full Name: David
Location: Switzerland
Contact:

[solved] Re: Failed to flush file buffers

Post by DavidS » Feb 10, 2015 1:12 pm 2 people like this post

Hello Everyone

A little followup to my problem.
As Luca suggested I opened a support ticket.

The support couldn't really help us solving this problem so we invistigated some more hours by ourself.

We adjusted some iSCSI settings for optimization, finally this optimizations solved our problem:

TcpAckFrequenzy

Code: Select all

Edit all Adapters, used for the iSCSI connection with the Value "TcpAckFrequenzy = 1". If you use multipath, edit this settings for every adapter.

	Create this DWORD key with a value of 1 :
	HKEY_LOCAL_MACHINE\SYSTEM\CurrentControlSet\Services\Tcpip\Parameters\Interfaces\<Interface GUID>\TcpAckFrequency
Nagle

Code: Select all

Disabled Nagle for iSCSI

Create this DWORD key with a value of 1:
HKLM\SYSTEM\CurrentControlSet\Control\Class\{4D36E97B-E325-11CE-BFC1-08002BE10318}\<Instance Number>\Parameters\iSCSIDisableNagle
Last step -> Reboot Veeam Server

This solved our problem. We have 2 iSCSI connections, one over a 10GB Ethernet and the other with a simple 1GB Ethernet.
Though, we only encoutered this problems with the 10GB Ethernet. It seems Veeam, iSCSI and 10GB aren't the best friends

I hope this solutions may help someone with similar problems.

Regards
David

spgsit5upport
Expert
Posts: 188
Liked: 14 times
Joined: May 28, 2010 10:25 am
Full Name: Seb
Contact:

Re: Failed to flush file buffers

Post by spgsit5upport » Nov 16, 2016 4:30 pm

No idea what it is about iSCSIDisableNagle, as such entry no longer is used for anything, it is only TcpAckFrequency to deactivate Nagle Algorithm on iSCSI interfaces

https://gallery.technet.microsoft.com/s ... m-66ca7608

spgsit5upport
Expert
Posts: 188
Liked: 14 times
Joined: May 28, 2010 10:25 am
Full Name: Seb
Contact:

Re: Failed to flush file buffers

Post by spgsit5upport » Nov 18, 2016 9:30 am


Post Reply

Who is online

Users browsing this forum: No registered users and 20 guests