What is the best file system for a Linux target?

[mschlott]

I see that it takes a long time to remove large files from an EXT3 file system on my Linux Target. I have several backups where my vbk file exceeds 500GB and incrementals are around 50GB. I decided to try EXT4 as a file system but it has it's drawbacks too. I am currently running a mix of EXT3 and EXT4 file systems with modified mount options to increase performance.

What do you use as a Linux target file system and why? If this is a FAQ, I missed it.

mike

[rawtaz]

Would you mind elaborating on which downsides of EXT4 you have in mind?

[tsightler]

You can use whatever Linux filesystem you like, everyone has their own preference. My personal preference is to use EXT4 because it is widely used as the default filesystem on just about every mainstream Linux OS and thus has a wide install base which means that it's proven stable and reliable. It's performance is quite good, and it can be tweaked for either data safety or maximum performance, with the defaults being a good, conservative mix. It's biggest drawback is that it is limited to 16TB (well, technically EXT4 supports larger than 16TB, but the userspace tools are still a work in progress, you can create a >16TB EXT5 filesystem with development versions of e2fstools, but some tools are not yet updated last I checked, admittedly a few months ago).

My next choice would probably be XFS. It's also been very proven, and while certainly not deployed in the Linux space nearly as widely as the EXT3/4 filesystems, it's used by a large number of customers and is probably the most widely seen >16TB non-clustered filesystem seen in Linux deployments, at least in my experience.

As far as tuning EXT4 for performance, I'd suggest using at least "noatime" to disable access time updates, as there's not risk in that. Some would suggest using "data=writeback" but there's some additional increase in risk since this mode does not guarantee that data is written to disk prior, unlike the default mode that groups data writes and occur just prior to journal commits.

Other options that might be useful are "barrier=0". This is only recommended if your drives have full battery backed write cache to save writes in the event of a crash. The "nobh" option is also regularly recommended but I've honestly never been able to measure much of a difference with this option for the Veeam use case.

[mschlott]

Would you mind elaborating on which downsides of EXT4 you have in mind?

The main drawback that I have seen of EXT4 is the read speed, that I think is primarily due to access time modification every time a file is accesed. Here are the mount options that I am currently using with EXT4 to get the best performance.

Code: Select all

noatime,data=writeback,barrier=0,nobh,errors=remount-ro 1 2

[tsightler]

You can improve streaming read for full restores by increasing read-ahead typically with a command like:

Code: Select all

blockdev --setra 16384 /dev/sda

The "/dev/sda" parameter should be replaced by the block device backing your filesystem.

[mschlott]

The reason I originally brought this up, is that io wait time seems to be my biggest bottlneck according to top.

top - 08:08:39 up 20:31, 1 user, load average: 2.50, 2.46, 2.46
Tasks: 268 total, 1 running, 267 sleeping, 0 stopped, 0 zombie
Cpu(s): 2.7%us, 1.1%sy, 0.0%ni, 56.6%id, 39.6%wa, 0.0%hi, 0.0%si, 0.0%st
Mem: 16334104k total, 16179040k used, 155064k free, 20064k buffers

I see %wa fluctuate from 25% to 50% while running 2 synthetic fulls agains a single raid5 lun. Right now this is a EXT4 mounted with noatime, on 1/2 of an apple XServe RAID with ATA drives. This may just be the best performance I can get on these drives with synthetic fulls running. I may just have to move one of them to another Lun as one of them is now running over 20 hours.

Any hints or criticism is welcome.

[tsightler]

Certainly you should expect that a synthetic full would always be limited by disk I/O, not matter what the target since, pretty much a synthetic full is nothing but I/O. As far as whether that's the best you can expect, it's hard to know. How long does a "real" full backup for that job? How big is the full backup? How big are the incrementals? Are you doing a synthetic full with transform? There's simply not enough information to even guess what you should expect, but it sounds like your using an old ATA XServe with 7 ATA drives in RAID5. Those old boxes were pretty good for streaming I/O in their day, but were never great at random I/O since they had limited cache and were constrained by the fact that they only had 7 ATA drives, placing a pretty severe limit on their IOP capacity.