How are you handling NTFS fragmentation?

[MattMannix]

I'm curious how those of you are configuring your repos? Over the years we have used several backup solutions which leveraged backup-to-disk methods writing to NTFS. We have seen some rather heavy fragmentation on those volumes which hurt performance.

I'm wondering what fragmentation looks like on large repo volumes containing veeam data? Has a weekly scheduled defrag job been enough to keep your system in check?

Our linux engineer suggested bypassing NTFS in favor of a NFS volume but I'd prefer to use local storage and not add an additional layer of networking to the mix. I'm currently specing out a system to run Veeam for a large number of stand alone ESXi nodes, so this information would be quite helpful.

If you run your repo on an NTFS share and havn't run defrag in a while I'd love to hear some of your stats and fragmentation levels.

Thanks again.

[Cokovic]

Maybe i can provide you with some more info on that.

Actually i'm running 8 physical servers to backup our whole ESXi environment. Every Veeam backup server has a local attached storage with available disk space from 5TB up to 10TB. Most of these servers are running for over two years now. Scheduled defragmentation is turned off and was never actively performed. Over the last two years i had a lot of job changes and moved Veeam backup files from one server to another.

Here are the stats shown by Windows disk defragmenter after analyzing the repo disks:

Server 1 2% fragmented
Server 2 38% fragmented
Server 3 2% fragmented
Server 4 2% fragmented
Server 5 2% fragmented
Server 6 18% fragmented
Server 7 0% fragmented
Server 8 1% fragmented

Sidenote for Server 2 and Server 6:
I assume that most of the defragmentation is because of transactional SQL backups running on the same disk where the Veeam repo is.

Anyway you should take into account a VBK fragmentation too as Gostev explained here.

[MattMannix]

This is great information. The fragmentation is much lower than I was expecting to see over such a long period of time.

@Cokovic do you mind me asking, on these large volumes do you maintain many backup jobs or just one? How many restore points are you keeping per job. ?
Thanks again for this info, its really helpful.

[bgbGsy]

Hi There. I do schedule a weekly defrag on my server with my Veeam images. Not necessarily because of Veeam performance but because of writing to LTO5 tape afterwards. Because I use reverse incremental backups, if I don't defrag it really hits tape performance which wants to sustain 140 megs/sec+ to stop the tape shoe-shining. By doing a weekly defrag I am able to hit the maximums and also increase the life of the tapes. To put this in perspective my tape write performance without defrag was about 6gb/min and this increases to 8gb/min with defrag.

Obviosuly sequential reads from the disk are much better for the tape drive.

If I wasn't using tape, I probably wouldn't defrag as much.

Hope that info helps.

[Cokovic]

Hey Mat,

in total i'm running 165 jobs which makes an average of about 20 jobs per server backing up 1800+ VMs at the moment. We are using reversed incrementals with 15 restore points.

[m.novelli]

My 2 cents

For Veeam backup generally I use Dell NX3200 servers with local storage (up to 20 TB with 2TB disks, up to 27 TB with 3TB disks)
I configure a RAID5 plus one hot spare formatted with one big NTFS partition. At RAID level I configure 512k or 1 MB blocks, at NTFS level 64k blocks. I perform every week a full active backup. So fa so good with many customers. I don't bother about fragmentation of this Veeam Backup volume. The biggest VBK files of my customers are about 1.5 - 2 TB each.
On virtual machines I perform once or twice a year a defrag + sdelete

Marco

[m.novelli]

One addition: I still use Windows 2008 R2 as backup server/repository. I plan to move to Windows 2012 once the first service pack will be available and I plan to use the dedupe feature of Windows 2012. I suppose this will heavily fragment the partition. Still don't bother about that
I hope that in a couple of years Fusion-io cards will be mainstream and we will be able to forget at all defrag...

Marco