ReFS 2019 system requirements

[ejenner]

Hi guys,

Anyone know of a system requirements guide for ReFS on Server 2019?

i.e. how much RAM and processor per Terabyte of disk for instance?

[Gostev]

Hello! Just follow Veeam backup repository system requirements, these will cover ReFS needs. Thanks!

[ejenner]

No change for 2019? I suppose the system requirements won't have changed much between the two versions. This is most likely what we're going for.

x2 new repositories

HPE ProLiant DL380 Gen10 Intel Xeon-S 4114 10-Core (2.20GHz 13.75MB) 32GB (2 x 16GB) + second processor + 10 additional 16GB RAM modules (192gb).

These will be paired with DS3600 enclosures and 10TB drives. So about 100TB to start with and expand up to 3 enclosures per server as we go.

[Gostev]

No changes. Your planned configuration looks great! I also like it that you don't skimp on RAM, because it's cheap these days anyway - and v10 will put it to good use, because we're expanding the use of RAM caching. It really helps to accelerate things like Instant VM Recovery performance.

[jasonede]

I've seen hints elsewhere in this thread (veeam-backup-replication-f2/windows-201 ... 57726.html) and others that REFS requires more memory than NTFS and have seen some people talk about 1GB RAM per TB of storage + 8GB for the OS. Is that reasonable? (Assumption that backup server and repository are on same device)

[Gostev]

No, 1GB RAM per 1TB is pre-September 2018 ReFS update best practice. These days you just follow Veeam system requirements and sum RAM requirements for all Veeam components running on your backup server. They are pretty generous and will cover ReFS needs as well.

[d3nz]

Gostev,
Are you sure that requirement 1 GB RAM for 1 TB of ReFS isn't actual?
At the best practices site this requirement is present: https://www.veeambp.com/repository_serv ... tory_types

[Gostev]

Yes, I am positive about that.

[tsightler]

Best practice guide needs and update for this, it's in progress.

[d3nz]

Are you joke guys? )) We are ordering now hundreds gigabytes of RAM for all our repository servers with ReFS and your support approved this requirement at one of my cases with URL to veeambp.com.

[tsightler]

Best practices are not the same as requirements, they are recommendations on how to do something so that it works "best" in a given circumstance. These practices are based on what is observed by customers, not some cooked up lab testing, but they can, and will, change over time, especially when vendors make changes that address issues.

This is especially true in a case like this where the recommendation comes from cases where ReFS was consuming significant amounts of kernel memory and it was clearly observed that customers with more RAM had much greater success than those with minimal RAM. Microsoft released fixes and it seems that currently ReFS, does not need additional RAM vs our normal recommendations compared to NTFS, at least for most cases, but it takes time for this to be proven in the field and we've only recently become comfortable that this best practice recommendation can be dropped (although there's still some evidence it might not hurt for Windows 2019)

That being said, in general I would always council customers to have more RAM in repos vs less RAM, time and again customers with more RAM have greater reliability and performance and this is regardless of using ReFS or not, so you shouldn't feel too bad about having more RAM.

[Gostev]

In any case, this won't be money lost. You can put this RAM into a very good use with v10 by bumping RAM cache size defaults for our Instant Recovery 2.0 engine... this will benefit the instant restore performance greatly.

[d3nz]

Ok.
How much RAM you recommend for 2 sockets server with total 20 cores and connected by SAS datastore with 7200rpm disks with total capacity 490 TB (raid 6 groups) with role Proxy+Repo (SAN) on WinSvr2019Std to serve ReFS with 64K block?
Before it was 8 GB + 20*4 GB + 490 GB = 578 GB.
Now is it 8 GB + 20*4 GB = 88 GB?

[d3nz]

Btw here https://helpcenter.veeam.com/docs/backu ... 95u4#proxy
and here: https://www.veeambp.com/appendix_a_sizing
so different requirements\recomendations for proxy and repository servers.
What requirements is prefer?

[tsightler]

I would consider helpcenter/user guide as minimum amounts for solution to work in typical deployment scenarios, while best practice guide is how Veeam architect would design and deploy system for maximum performance and reliability in a large environment where there can be many different stress factors.

Normal best practice recommendation for proxy is 2GB per core/vCPU (same as task for proxy) so you have to combine requirement for both when running them on the same box. For 20 core box that is proxy/repo it would be:

8GB RAM Base OS +
20 cores * 2GB for Proxy role = 40GB total
20 cores * 4GB for Repo role = 80GB total

So 128GB RAM would be the best practice recommendation for this setup, while minimum requirement would be significantly less. Personally, I'd still recommend more than this, say in the 192-256GB, but I would not recommend less than the 128GB best practice config in any case.

[d3nz]

So 128GB RAM would be the best practice recommendation for this setup, while minimum requirement would be significantly less. Personally, I'd still recommend more than this, say in the 192-256GB, but I would not recommend less than the 128GB best practice config in any case.

What is the purpose of additional RAM exceeding 128GB?

[tsightler]

Mostly this recommendation comes because I keep seeing reports like this:

veeam-backup-replication-f2/windows-201 ... ml#p359163

And this:

veeam-backup-replication-f2/windows-201 ... ml#p359043

Admittedly I don't know the full details of each of these customers environments, but I know enough large, petabyte scale environments that aren't having such issues, or at least not to critical levels, and all of those have more RAM than the best practice minimum thus it's difficult to argue with the fact that customers with more RAM have more reliable operations (admittedly, the vast majority of those customers are on 2016, while the thread is about 2019).

Having more RAM seems to help significantly during high I/O periods, such as merges/synthetic fulls and large delets and the OS cache usage can easily grow to dozens of GBs leading to user space processes (like the Veeam processes) being swapped and slowing the overall system to a crawl. The recent patches attempt to address some of this, but I still see higher memory use in all cases with ReFS, just not as severe. More memory does not completely eliminate the possibility of these issues occurring, but significantly decreases the odds.

In the future I'm thinking that the best practice guide probably needs an update with some type of sliding scale for OS memory vs the fixed 8GB currently recommended. Probably something not related to disk space but rather number of core/parallel tasks as that seems to be a bigger factor than overall disk space. And extra 2-4GB per core just for the OS is my personal recommendation right now and what I counsel the customers I work with to use as a minimum. Most of the customers I work with with large ReFS repos in the 300-500TB range have at least 256GB and many have more.