Hello, I recently started running into a weird problem with veeam 4.0 running on windows 2008 64 bit, I have a support case open but wanted the community take on the matter.
all the veeam backups are going to a 11 gig NTFS volume connected via iscsi, it has been working fine for a while but all of a sudden this started to happen:
The requested operation could not be completed due to a file system limitation
"Failed to write data to the file" in the veeam logs, no corresponding even in the windows event log but it's word for word a known "Microsoft" error:
Hi Boris, this have not been reported on forums before for sure, but I will check with our support if anything like this came through regular support before. Windows 2008 x64 is generally quite common platform for Veeam Backup across all of our customers.
Interesting that the error is in the Veeam job, but not in the Windows Event log. Is this error sporadic or regular, consistently with specific jobs, or seemingly random?
Not sure if it's related, but last night one of our Veeam jobs kicked up the error in the first KB for pre-Vista systems (ours is w2k3): "insufficient system resources exist to complete the requested service" -- a first for us. However, the job auto-retried as scheduled and completed successfully. Also nothing in the Event logs. 3 TB local array, 200 GB VBK with 1-5 GB VRB's (last night's was 2.7 GB). I wasn't planning to look much into it if it remains a one-off thing, but I'll see if it happens again.
EDIT: The KB suggests the cause is a heavily-fragmented file. If it's a specific job or two where you're seeing the issue, try creating a fresh full backup file. That's what I'll try if I see it happen again here. (And if it is a particular job or two, I'm curious how large your VBK/VRB files are and how long they've been running synthetically without a full? My backup VBK in question's been synthetic for about 2 months with daily VRB's.)
I think the issue is the storage I am using, I am doing vsphere storage API (direct iscsi backups) on the same gig interface as the backup destination and I think that the iscsi stack is getting overwhelmed and causing timeouts. I am going to do some tests via DAS storage to see if I can replicate the problem
Makes sense. I found we had more of the same errors over the weekend when simultaneously performing a large local copy job (40 GB of tiny few-K files) to the same storage as our Veeam backups, so I'm pretty sure our errors are also storage I/O related (our DAS is a cheap eSATA enclosure).
Just an update-- In our case, it appears the one backup file referenced above was causing the problems. Creating a fresh one resolved the errors on the related job and any other jobs that bumped up against it during the retry window. FWIW