Recommended best allocation unit size for repository volume?

[bbricker]

Quick question- My VB&R server is a Win2k8R2-x64 virtual machine and I'm creating a new repository volume from a 9TB RDM disk to my SAN. What is the recommended allocation unit size when formatting the drive? 64k?

thanks!
Ben

[Gostev]

Tom just recommended 64K in the adjacent thread, and I totally trust him on these sort of questions

[bbricker]

But I think that was for stripe size defined in the SAN, not the allocation unit size set when formatting a partition in Windows.

[Gostev]

I believe that for best performance, NTFS allocation unit size should be the same as SAN stripe size (or be a multiple of it).

[dellock6]

Disk alignment is one of the biggest problem when it comes to SAN performances.
Windows 2008 R2 aligns its partitions around 1024Kb blocks. As a rule of thumb, if the fraction between the upper filesystem and the underlying storage is an integer, you are ok. So, since 1024k / 64k = 16, you are good to go.
If you are going for a vmdk over a vmfs filesystem, also VMware uses 64k blocks.
Also, check best practices from your storage vendor about NTFS luns.

[bbricker]

thanks guys!

[nmace]

I'm setting up a new repository for Veeam. The Veeam VM itself will be a VM in sphere 5.5 with locally attached storage. The VMFS block size in version 5.5 is (I believe) 1 MB. Should I make the raid volume stripe size the same? I'm assuming the NTFS volume should also be 1 MB? Or is there a reason to make the stripe or NTFS size less than the VMFS block size?

[foggy]

For RAID stripe, 256KB is typically recommended, NTFS volumes should be formatted at least with 64K clusters.

You may also find this discussion useful.

[bertdhont]

We have bought a new DAS (HP D3600) with 12 x 4TB SAS HDD.
This DAS is connected through a HP SmartArray P421.
We will use that storage as backup repository for Veeam.

As we do incremental backups and no reversed incrementals, we will create a logical drive in RAID50.
Are there any recommendations about other parameters of that volume, such as:

• Strip Size / Full Stripe Size
  Sectors/Track

And in Windows, is it a good idea to format the disk with an allocation unit size of 64K
(because it is to store only large files)

Regards,

Bert

[dellock6]

To be honest I don't know what's a full stripe in HP terms...
For stripe size, you should align it to the expected deduplication level you are willing to use:
128k for "wan" (256k original, 128k with compression on)
256k for "lan" (512k original, 256k compressed)
512k for "local" (1024k original, 512k compressed).

Also, you should use the same size for both the storage and the file system, or you can incur in write amplification problems. So, use the same value also for NTFS. On Windows 2008 and later ntfs is automatically aligned to the underlying disk, otherwise you should also check and fix alignment problem, this is usually an uderestimated problem in storage configurations but it can lead to serious performance penalties.

Then, also evaluate the raid configuration, balancing space and performances. Are you sure to go for raid50 with only 12 disks??? There are no "SAS" 4TB disks, those are simple SATA disk using a SAS interface. At 7200rpm, raid50 and raid10 will give you really different results...

[cffit]

I'm currently setting up a new VEEAM server and it will have local disk for the repository. The disks are 1.2TB 10K SAS disks. What is the optimal RAID setup for best performance on a backup repository that does incremental backups? I plan to have about 12 disks in the array.

Also, for stripe size at the array level and then OS level block size, what should I go with? I assume the larger the better? Or is 256Kb the best route?

[Delo123]

As backup files are usually gigantic go with the largest...
12 disks is a bit large for raid5, raid 10 would be best if that leaves enough space for you. If not go with raid 6 or 2 raid 5's or a raid 50 if supported...

[dellock6]

Regardless the number of disks, I'd sty away these days from Raid5 as even a single disk is too big to be safe with just one parity. The rebuild time of a 2-4-6 TB disk is insanely long, and for that entire period you have no parity left, if anything happens there's data loss.

[Delo123]

@Luca: That is why we all have Veeam backup copies!

[CPCS]

Hi,

i am just stumbling over the same kind of question, also with HP Array Configuration Utility

To be honest I don't know what's a full stripe in HP terms...

The stripe size is the amopunt of date that is stored on each physical drive in the array.The full stripe size is the ammount of data that the controller can read or write simultaneously on all drives in the array...

For stripe size, you should align it to the expected deduplication level you are willing to use:
128k for "wan" (256k original, 128k with compression on)
256k for "lan" (512k original, 256k compressed)
512k for "local" (1024k original, 512k compressed).

Also, you should use the same size for both the storage and the file system, or you can incur in write amplification problems. So, use the same value also for NTFS. On Windows 2008 and later ntfs is automatically aligned to the underlying disk, otherwise you should also check and fix alignment problem, this is usually an uderestimated problem in storage configurations but it can lead to serious performance penalties.

What do you mean with "local"? The Backup Server will have local storage the repository is laying on but the vm to be backed up are running in another server room, connected via 1GBit lan with the backup server.


The Raid will be Raid5 and under that two logical drives:

• 120GB for Windows 2008R2
• 5,5TB for Veeam Repository

So how should it be configured for each logical drive, specially the Repository?

[CPCS]

and how about the allocation unit size inside Windows for the Repository Partition? Is veeam also creating small files on the repository or only the backup files which will be at least a few MB-TB so i should choose the max ntfs allocation unit size of 64kb??

[albertwt]

But I think that was for stripe size defined in the SAN, not the allocation unit size set when formatting a partition in Windows.

So what about the stripe size in your LUN RAID group or disk group ?
have you set it to 256 KB or make it the same 64 KB same as the largest NTFS Allocataion Unit Size ?

[albertwt]

So how should it be configured for each logical drive, specially the Repository?

@CPCS, I'm on the similar situation as you. So did you end up with some recommendation that you can share here ?

[CPCS]

Suport only reverse to community or giving non concrete answers with no references >-<

After a lot of searching a used this sites to try to get an idea:

https://www.42u.ca/2016/03/21/veeam-rep ... es-part-2/
http://www.virtualtothecore.com/en/veea ... ripe-size/
http://www.mirazon.com/veeam-repository-best-practices/
https://www.koolaid.info/veeam-backup-r ... ion-notes/

This was also intersting (and seems to be the base for this pdf = Veeam Repository Best Practices presentations pdf from http://www.swovug.ca/resources/)

Backup Repository Best Practices: 2015 Edition
https://www.veeam.com/veeamon/recorded-sessions

In the end i used 512 stripe size in the raid for the repository with local target as storage optimization with a lot of hope that this wont break my neck in the future...

[albertwt]

CPCS,

yes, that's what I've been told as well.

Anyway, I've managed to create multiple 15 TB NTFS LUNs on my HP DAS using:

Strip Size: 512 KiB
Full Stripe Size 2560 KiB

Thanks for the sharing.

[mephisto]

Guys,

I'm in doubt about a couple of things, Veeam suggests the backups I/O is normally between 256/512KB, that means the stripe size of my raid array, be RAID-10 or RAID-5 for example should be larger than 256KB, right? I would assume the bigger the better for sequential writes as well?

Windows then formats a NTFS partition by default at 4k cluster size, and allows it to be up to 64KB (can't do higher, perhaps REFS will do?), so I also believe I should have this set up to 64KB as definitely we would not have small files with Veeam backups.

What do you guys think?

[foggy]

Please review considerations above.

[Trin]

I've recently set up a 10TB iSCSI LUN on our Synology RS3617xs+ but I didn't pay attention to the file allocation unit size when doing so. A quick check suggests it's at 4k...

I want to start again but I'm not sure what the recommended size should be. Any advice would be very welcome.

Server - Windows Server 2016 VM
Repository - Synology RS3617xs+, RAID-6

Many thanks

[PTide]

Hi,

First, may I ask you what file system do you use? Is it NTFS or ReFS?

Thanks

[Yakov Sovluk]

Hi!

If I use Local target (Veeam side) and NTFS (storage side), what stripe size should I use?

https://bp.veeam.expert/repository_serv ... y_planning - here I see: "Tip: As can be seen from the field, optimal value for the stripe size is often between 256 KB and 512 KB"

https://www.virtualtothecore.com/en/vee ... ripe-size/ - In main part of article Luca writes that 64К stripe size is slow, but 256K is rather faster, 512K more faster.

But in comments he wrote: "More than Veeam block size, it’s useless to create a stripe size bigger than the filesystem you are going to use on top of it. For example,with NTFS the maximum block size is 64k, so any windows repository will use at most this value."
"On NTFS I always use 64k stripe"
"Go for 64k block size at the storage layer and at the NTFS layer."

So, If I use NTFS, I shoulde make 64K stripe size, because NTFS max cluster size = 64K.

Does something changes in case ReFS? (As I know max ReFS cluster size = 64K too)

[Gostev]

Recommended RAID stripe size is 128KB or 256KB regardless of the file system. This number is based on typical Veeam block size. File system cluster is largely irrelevant since all writes go through system cache that groups them, resulting in cluster sizes having no material impact on I/O performance.

[Yakov Sovluk]

Thanks!

But for local target, where compressed veeam block size = 512K, maybe 512K stripe size will be better?

And:
https://bp.veeam.expert/repository_serv ... y_planning - here I see: "Tip: As can be seen from the field, optimal value for the stripe size is often between 256 KB and 512 KB"

I think 128K is too small for local target and for lan target too.

Or you recomend to use 256К for all cases (lan or local target)?

[Gostev]

It can often be less than 512KB, so our architects generally recommend stripe size of 128KB (or 256KB at most).

[dellock6]

Yakov,
compression is not fixed. We see "on average" a 2x reduction, but in reality each block starts at 1024KB and becomes one time 512KB, another one 500KB, another 444KB, another one again 600KB. So, building a volume with 512KB would probably improve performance, but it would also let you waste a ton of disk space since only one block would be stored inside.

Also, as you also suggested, a user may decide at some point to switch from local target (512KB compressed blocks) to smaller blocks, and thus a large disk cluster size would give even more waste of space. If you think that WAN target uses 256KB blocks, they could become on average 128KB, but again some of them may be even smaller.

That's why we recommend a good balance between performance and space usage, which is 64KB or 128KB.

[ChrisJ83Knights]

Hi all

Should I go for the largest allocation size available when creating the backup server RAIDs?

128K is recommended below but should I be going for 256K?

https://www.veeam.com/blog/vmware-backu ... tices.html

Whilst on the subject should I be going for 64K when using ReFS?

Many thanks