Comprehensive data protection for all workloads
bbricker
Enthusiast
Posts: 54
Liked: 27 times
Joined: Feb 10, 2012 8:43 pm
Contact:

Recommended best allocation unit size for repository volume?

Post by bbricker » 1 person likes this post

Quick question- My VB&R server is a Win2k8R2-x64 virtual machine and I'm creating a new repository volume from a 9TB RDM disk to my SAN. What is the recommended allocation unit size when formatting the drive? 64k?

thanks!
Ben
Gostev
Chief Product Officer
Posts: 31457
Liked: 6647 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Recommended best allocation unit size for repository vol

Post by Gostev » 2 people like this post

Tom just recommended 64K in the adjacent thread, and I totally trust him on these sort of questions ;)
bbricker
Enthusiast
Posts: 54
Liked: 27 times
Joined: Feb 10, 2012 8:43 pm
Contact:

Re: Recommended best allocation unit size for repository vol

Post by bbricker »

But I think that was for stripe size defined in the SAN, not the allocation unit size set when formatting a partition in Windows.
Gostev
Chief Product Officer
Posts: 31457
Liked: 6647 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Recommended best allocation unit size for repository vol

Post by Gostev » 1 person likes this post

I believe that for best performance, NTFS allocation unit size should be the same as SAN stripe size (or be a multiple of it).
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Recommended best allocation unit size for repository vol

Post by dellock6 » 2 people like this post

Disk alignment is one of the biggest problem when it comes to SAN performances.
Windows 2008 R2 aligns its partitions around 1024Kb blocks. As a rule of thumb, if the fraction between the upper filesystem and the underlying storage is an integer, you are ok. So, since 1024k / 64k = 16, you are good to go.
If you are going for a vmdk over a vmfs filesystem, also VMware uses 64k blocks.
Also, check best practices from your storage vendor about NTFS luns.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
bbricker
Enthusiast
Posts: 54
Liked: 27 times
Joined: Feb 10, 2012 8:43 pm
Contact:

Re: Recommended best allocation unit size for repository vol

Post by bbricker »

thanks guys!
nmace
Enthusiast
Posts: 99
Liked: 12 times
Joined: Jul 23, 2012 3:48 pm
Contact:

[MERGED] Repository stripe size

Post by nmace » 1 person likes this post

I'm setting up a new repository for Veeam. The Veeam VM itself will be a VM in sphere 5.5 with locally attached storage. The VMFS block size in version 5.5 is (I believe) 1 MB. Should I make the raid volume stripe size the same? I'm assuming the NTFS volume should also be 1 MB? Or is there a reason to make the stripe or NTFS size less than the VMFS block size?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Recommended best allocation unit size for repository vol

Post by foggy » 1 person likes this post

For RAID stripe, 256KB is typically recommended, NTFS volumes should be formatted at least with 64K clusters.

You may also find this discussion useful.
bertdhont
Service Provider
Posts: 45
Liked: 5 times
Joined: Nov 08, 2013 2:53 pm
Full Name: Bert D'hont
Contact:

[MERGED] Veeam Repository on DAS

Post by bertdhont » 1 person likes this post

We have bought a new DAS (HP D3600) with 12 x 4TB SAS HDD.
This DAS is connected through a HP SmartArray P421.
We will use that storage as backup repository for Veeam.

As we do incremental backups and no reversed incrementals, we will create a logical drive in RAID50.
Are there any recommendations about other parameters of that volume, such as:
  • Strip Size / Full Stripe Size
    Sectors/Track
Image

And in Windows, is it a good idea to format the disk with an allocation unit size of 64K
(because it is to store only large files)

Regards,

Bert
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Recommended best allocation unit size for repository vol

Post by dellock6 » 1 person likes this post

To be honest I don't know what's a full stripe in HP terms...
For stripe size, you should align it to the expected deduplication level you are willing to use:
128k for "wan" (256k original, 128k with compression on)
256k for "lan" (512k original, 256k compressed)
512k for "local" (1024k original, 512k compressed).

Also, you should use the same size for both the storage and the file system, or you can incur in write amplification problems. So, use the same value also for NTFS. On Windows 2008 and later ntfs is automatically aligned to the underlying disk, otherwise you should also check and fix alignment problem, this is usually an uderestimated problem in storage configurations but it can lead to serious performance penalties.

Then, also evaluate the raid configuration, balancing space and performances. Are you sure to go for raid50 with only 12 disks??? There are no "SAS" 4TB disks, those are simple SATA disk using a SAS interface. At 7200rpm, raid50 and raid10 will give you really different results...
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
cffit
Veteran
Posts: 338
Liked: 35 times
Joined: Jan 20, 2012 2:36 pm
Full Name: Christensen Farms
Contact:

Re: Recommended best allocation unit size for repository vol

Post by cffit »

I'm currently setting up a new VEEAM server and it will have local disk for the repository. The disks are 1.2TB 10K SAS disks. What is the optimal RAID setup for best performance on a backup repository that does incremental backups? I plan to have about 12 disks in the array.

Also, for stripe size at the array level and then OS level block size, what should I go with? I assume the larger the better? Or is 256Kb the best route?
Delo123
Veteran
Posts: 361
Liked: 109 times
Joined: Dec 28, 2012 5:20 pm
Full Name: Guido Meijers
Contact:

Re: Recommended best allocation unit size for repository vol

Post by Delo123 »

As backup files are usually gigantic go with the largest...
12 disks is a bit large for raid5, raid 10 would be best if that leaves enough space for you. If not go with raid 6 or 2 raid 5's or a raid 50 if supported...
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Recommended best allocation unit size for repository vol

Post by dellock6 » 1 person likes this post

Regardless the number of disks, I'd sty away these days from Raid5 as even a single disk is too big to be safe with just one parity. The rebuild time of a 2-4-6 TB disk is insanely long, and for that entire period you have no parity left, if anything happens there's data loss.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
Delo123
Veteran
Posts: 361
Liked: 109 times
Joined: Dec 28, 2012 5:20 pm
Full Name: Guido Meijers
Contact:

Re: Recommended best allocation unit size for repository vol

Post by Delo123 »

@Luca: That is why we all have Veeam backup copies! ;)
CPCS
Influencer
Posts: 17
Liked: 4 times
Joined: Mar 01, 2016 1:49 pm
Contact:

Re: Recommended best allocation unit size for repository vol

Post by CPCS » 1 person likes this post

Hi,

i am just stumbling over the same kind of question, also with HP Array Configuration Utility

Image
dellock6 wrote:To be honest I don't know what's a full stripe in HP terms...
HP Help wrote:The stripe size is the amopunt of date that is stored on each physical drive in the array.The full stripe size is the ammount of data that the controller can read or write simultaneously on all drives in the array...
dellock6 wrote: For stripe size, you should align it to the expected deduplication level you are willing to use:
128k for "wan" (256k original, 128k with compression on)
256k for "lan" (512k original, 256k compressed)
512k for "local" (1024k original, 512k compressed).

Also, you should use the same size for both the storage and the file system, or you can incur in write amplification problems. So, use the same value also for NTFS. On Windows 2008 and later ntfs is automatically aligned to the underlying disk, otherwise you should also check and fix alignment problem, this is usually an uderestimated problem in storage configurations but it can lead to serious performance penalties.
What do you mean with "local"? The Backup Server will have local storage the repository is laying on but the vm to be backed up are running in another server room, connected via 1GBit lan with the backup server.


The Raid will be Raid5 and under that two logical drives:
  • 120GB for Windows 2008R2
  • 5,5TB for Veeam Repository
So how should it be configured for each logical drive, specially the Repository?
CPCS
Influencer
Posts: 17
Liked: 4 times
Joined: Mar 01, 2016 1:49 pm
Contact:

Re: Recommended best allocation unit size for repository vol

Post by CPCS » 1 person likes this post

and how about the allocation unit size inside Windows for the Repository Partition? Is veeam also creating small files on the repository or only the backup files which will be at least a few MB-TB so i should choose the max ntfs allocation unit size of 64kb??
albertwt
Veeam Legend
Posts: 879
Liked: 46 times
Joined: Nov 05, 2009 12:24 pm
Location: Sydney, NSW
Contact:

Re: Recommended best allocation unit size for repository vol

Post by albertwt »

bbricker wrote:But I think that was for stripe size defined in the SAN, not the allocation unit size set when formatting a partition in Windows.
So what about the stripe size in your LUN RAID group or disk group ?
have you set it to 256 KB or make it the same 64 KB same as the largest NTFS Allocataion Unit Size ?
--
/* Veeam software enthusiast user & supporter ! */
albertwt
Veeam Legend
Posts: 879
Liked: 46 times
Joined: Nov 05, 2009 12:24 pm
Location: Sydney, NSW
Contact:

Re: Recommended best allocation unit size for repository vol

Post by albertwt »

CPCS wrote:
So how should it be configured for each logical drive, specially the Repository?
@CPCS, I'm on the similar situation as you. So did you end up with some recommendation that you can share here ?
--
/* Veeam software enthusiast user & supporter ! */
CPCS
Influencer
Posts: 17
Liked: 4 times
Joined: Mar 01, 2016 1:49 pm
Contact:

Re: Recommended best allocation unit size for repository vol

Post by CPCS » 1 person likes this post

Suport only reverse to community or giving non concrete answers with no references >-<

After a lot of searching a used this sites to try to get an idea:

https://www.42u.ca/2016/03/21/veeam-rep ... es-part-2/
http://www.virtualtothecore.com/en/veea ... ripe-size/
http://www.mirazon.com/veeam-repository-best-practices/
https://www.koolaid.info/veeam-backup-r ... ion-notes/

This was also intersting (and seems to be the base for this pdf = Veeam Repository Best Practices presentations pdf from http://www.swovug.ca/resources/)

Backup Repository Best Practices: 2015 Edition
https://www.veeam.com/veeamon/recorded-sessions

In the end i used 512 stripe size in the raid for the repository with local target as storage optimization with a lot of hope that this wont break my neck in the future...
albertwt
Veeam Legend
Posts: 879
Liked: 46 times
Joined: Nov 05, 2009 12:24 pm
Location: Sydney, NSW
Contact:

Re: Recommended best allocation unit size for repository vol

Post by albertwt »

CPCS,

yes, that's what I've been told as well.

Anyway, I've managed to create multiple 15 TB NTFS LUNs on my HP DAS using:
Strip Size: 512 KiB
Full Stripe Size 2560 KiB
Thanks for the sharing.
--
/* Veeam software enthusiast user & supporter ! */
mephisto
Expert
Posts: 121
Liked: 7 times
Joined: Nov 07, 2012 6:49 pm
Full Name: Mephisto poa
Contact:

[MERGED] Block size / stripe

Post by mephisto » 1 person likes this post

Guys,

I'm in doubt about a couple of things, Veeam suggests the backups I/O is normally between 256/512KB, that means the stripe size of my raid array, be RAID-10 or RAID-5 for example should be larger than 256KB, right? I would assume the bigger the better for sequential writes as well?

Windows then formats a NTFS partition by default at 4k cluster size, and allows it to be up to 64KB (can't do higher, perhaps REFS will do?), so I also believe I should have this set up to 64KB as definitely we would not have small files with Veeam backups.

What do you guys think?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Recommended best allocation unit size for repository vol

Post by foggy »

Please review considerations above.
Trin
Enthusiast
Posts: 37
Liked: 7 times
Joined: Apr 28, 2017 10:38 am
Contact:

[MERGED] Recommended file allocation unit size query

Post by Trin »

I've recently set up a 10TB iSCSI LUN on our Synology RS3617xs+ but I didn't pay attention to the file allocation unit size when doing so. A quick check suggests it's at 4k...

I want to start again but I'm not sure what the recommended size should be. Any advice would be very welcome.

Server - Windows Server 2016 VM
Repository - Synology RS3617xs+, RAID-6

Many thanks
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Recommended best allocation unit size for repository vol

Post by PTide »

Hi,

First, may I ask you what file system do you use? Is it NTFS or ReFS?

Thanks
Yakov Sovluk
Novice
Posts: 4
Liked: never
Joined: Dec 18, 2018 2:11 pm
Contact:

Re: Recommended best allocation unit size for repository volume?

Post by Yakov Sovluk »

Hi!

If I use Local target (Veeam side) and NTFS (storage side), what stripe size should I use?

https://bp.veeam.expert/repository_serv ... y_planning - here I see: "Tip: As can be seen from the field, optimal value for the stripe size is often between 256 KB and 512 KB"

https://www.virtualtothecore.com/en/vee ... ripe-size/ - In main part of article Luca writes that 64К stripe size is slow, but 256K is rather faster, 512K more faster.

But in comments he wrote: "More than Veeam block size, it’s useless to create a stripe size bigger than the filesystem you are going to use on top of it. For example,with NTFS the maximum block size is 64k, so any windows repository will use at most this value."
"On NTFS I always use 64k stripe"
"Go for 64k block size at the storage layer and at the NTFS layer."

So, If I use NTFS, I shoulde make 64K stripe size, because NTFS max cluster size = 64K.

Does something changes in case ReFS? (As I know max ReFS cluster size = 64K too)
Gostev
Chief Product Officer
Posts: 31457
Liked: 6647 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Recommended best allocation unit size for repository volume?

Post by Gostev »

Recommended RAID stripe size is 128KB or 256KB regardless of the file system. This number is based on typical Veeam block size. File system cluster is largely irrelevant since all writes go through system cache that groups them, resulting in cluster sizes having no material impact on I/O performance.
Yakov Sovluk
Novice
Posts: 4
Liked: never
Joined: Dec 18, 2018 2:11 pm
Contact:

Re: Recommended best allocation unit size for repository volume?

Post by Yakov Sovluk »

Thanks!

But for local target, where compressed veeam block size = 512K, maybe 512K stripe size will be better?

And:
https://bp.veeam.expert/repository_serv ... y_planning - here I see: "Tip: As can be seen from the field, optimal value for the stripe size is often between 256 KB and 512 KB"

I think 128K is too small for local target and for lan target too.

Or you recomend to use 256К for all cases (lan or local target)?
Gostev
Chief Product Officer
Posts: 31457
Liked: 6647 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Recommended best allocation unit size for repository volume?

Post by Gostev »

It can often be less than 512KB, so our architects generally recommend stripe size of 128KB (or 256KB at most).
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Recommended best allocation unit size for repository volume?

Post by dellock6 »

Yakov,
compression is not fixed. We see "on average" a 2x reduction, but in reality each block starts at 1024KB and becomes one time 512KB, another one 500KB, another 444KB, another one again 600KB. So, building a volume with 512KB would probably improve performance, but it would also let you waste a ton of disk space since only one block would be stored inside.

Also, as you also suggested, a user may decide at some point to switch from local target (512KB compressed blocks) to smaller blocks, and thus a large disk cluster size would give even more waste of space. If you think that WAN target uses 256KB blocks, they could become on average 128KB, but again some of them may be even smaller.

That's why we recommend a good balance between performance and space usage, which is 64KB or 128KB.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
ChrisJ83Knights
Enthusiast
Posts: 27
Liked: never
Joined: Jan 10, 2017 3:06 pm
Full Name: Chris Johnson
Contact:

[MERGED] RAID Volume strip size

Post by ChrisJ83Knights »

Hi all

Should I go for the largest allocation size available when creating the backup server RAIDs?

128K is recommended below but should I be going for 256K?

https://www.veeam.com/blog/vmware-backu ... tices.html

Whilst on the subject should I be going for 64K when using ReFS?

Many thanks
Post Reply

Who is online

Users browsing this forum: BartP, dbeerts, knut-weber, Noushad and 195 guests