Repo build sanity check

[bhagen]

It's time to think about buying a couple of new repos, so I've been researching current Veeam recommendations.

I'd like to get some feedback and a sanity check on these potential builds. (What could we improve, is anything just plain "wrong" with any of the components or VBR policies? Could we / should we go a completely different route with the same budget? Etc.)

We're running VBR11a VUL. For our production site, we're going to switch to 7 day immutable backups (once a day), with daily backup copy jobs going to our DR site (32day retention, GFS 4x weekly, 1x monthly, 5x yearly, latest fulls go to tape once a quarter). Daily replication from DR repo backup copy jobs to our DR vmware cluster ( which is identical to our prod cluster). Running CDP on several VMs directly from prod cluster to DR cluster, 30 minute RPO.

Our business is very static; not a ton of data or VM growth over 3 years is anticipated. 7 days of backups currently is around 25TB; all our backup copy jobs from the last 3 years currently take up around 200TB. We also only build our own boxes (both vmware and all our repos) on Supermicro hardware, and have a strict "no cloud" policy.

Potential Builds
Keep cost reasonable; don't buy the best, but don't buy the minimum.
Learn linux, unfortunately, and create hardened repos with immutability
Or run Minio and setup S3 for immutability
40TB repo in production site: either this https://www.supermicro.com/en/products/ ... P-WTRT.cfm or this https://www.supermicro.com/en/products/ ... -240p-tnrt
300TB repo in DR site: https://www.supermicro.com/en/products/ ... 1cr24h.cfm
CPU - Dual Intel Silver 4216 16C 2.1Ghz
RAM – 256Gb
OS Drives – 2x 250GB SSD RAID 10
NICs – 2port 10GB
Scalable / Expandable

Specific questions that we haven't settled yet (feedback please!):
For the prod repo, would an all flash with the above specs be significantly faster / better than running 12-4TB spinny disks?
If all flash is better, SAS SSDs or NVMe?
RAID 6, or 60?
ReFS for windows/minio?
XFS or ZFS for linux?

Thanks for your feeback!

[HannesK]

Hello
1) yes (at least I never hard of any all-flash system that is slower than spinning disks when talking about the same amount of disk space)
2) I would take the protocol / system where price / value is best
3) I prefer one RAID60 over multiple small RAID6. That makes it easier to manage
4) For today, I would go with REFS. With V12 and direct backup to object storage, object storage (minio or whatever makes sense) could become an alternative, yes
5) XFS, because of block cloning support

Best regards,
Hannes

[bhagen]

Thanks @HannesK; great feedback!

What do you think of the hardware specs compared to Veeam recommendations? I *think* these are pretty "middle of the road" specs; correct?

Also, I can't tell from the docs if CPU or RAM is more significant, or if dual CPUs is significantly better than single. Actually, I can't find anything specifically saying that dual CPUs can even be utilized by a veeam job?

[chris.childerhose]

When it comes to CPU or RAM it depends on the Veeam component that you are looking at. This site has the Design specs for the different components of the Veeam system - https://bp.veeam.com/vbr/

You want to check the Design section as Proxy specs are different than Repository, etc.

[bhagen]

Wow...@chris.childerhose, that's an amazing doc that I've never seen before! Glad I'm still in the "quote" stage of my repo purchase; gonna put that on hold until I make it thru this doc.

[chris.childerhose]

No problem. That site is put together by the SAs at Veeam so it is updated regularly and a great best practice to follow. I use it all the time.

[bhagen]

Fantastic resource; it'll be interesting to see how my "gut feeling" on hardware compares to the findings from this doc!

[chris.childerhose]

Yes, it definitely is. Here to help so feel free to post back after going through it for clarity, etc.

[bhagen]

As I was going thru the Design Specs site, I ran across the VSE tool (which I had actually gone thru with a vendor a couple weeks ago). Since I had run rvtools recently, I imported it into VSE.

This is what I came up with for "Site B" (where our backup copies and GFS live):

Role Qty Cores RAM (GB) Repo Cap (TB) Capacity Utilisation
Physical Repository 2 20 128 400 74.39%

Is "Qty" here saying this repo needs 2 CPUs, or is it saying I need two repositories of this size?

Thanks!

[chris.childerhose]

I believe the quantity is two servers.

[bhagen]

Well then...that ain't right.

My current "site b" repo has a cap of 236.4TB (bottom 2 lines in the shot below), and I'm using 149.3TB of it; ~63% used. 3%/year change rate = ~4.2TB...so in 3 years I'll be using a whopping 12.6TB more than I am today. I don't think I'll need two 400TB cap repos!

Plus, I've been wondering for a while why Veeam reports these crazy numbers when I click on Backup Infrastructure - Backup Repositories and look in the right pane:



Clearly I am *not* using 575.2TB on my 127.3TB repository.

[chris.childerhose]

Maybe it meant two repo servers with a total of 400TB?

Also that reporting number are you using ReFS or XFS by chance? The numbers are skewed when you do and I am hoping they fix that soon.

[bhagen]

Yes; all 4 of those are ReFS volumes. I was hoping that was a bug!

Two repo servers with a total of 400TB would make more sense...not sure why I didn't think of that. Interesting it would suggest a 400TB repo for "site a" when I'm changing the number of retention points from 32 to 7...but at least they're only suggesting one repo instead of 2! Also, zero virtual proxy servers in site b?? I currently have 4 proxies in each site, and they're constantly in use.



Regardless, this has been a great exercise! And it looks like I may have (grossly?) overestimated the compute/ram requirements for these repos. My question now is, will these repos even be able to USE two CPUs? Or am I wasting my money buying dual-cpu systems? I have searched for that answer and not found it so far.

[chris.childerhose]

It never hurts to have lots of resources for sure but I am sure overkill for repos buy if it was proxies then definitely.

[dbewernick]

When it comes to the repository quantity and the 400TB repositories, you need to know how the VSE works and were to tweak things. There are default server package sizing values defined for the start, so some tables show you more than you might need.
I recommend working through this with a partner that has some VSE experience.

[bhagen]

I did work with a partner that knew his way around VSE, and after looking at our rvtools he suggested that a 300TB "site b" is what we need. But I didn't think to ask about CPUs, and he didn't mention that.

So again, I know I have the sizing right, just wondering if dual CPUs can even be used by Veeam on a physical repository.

Anybody know for sure?

[chris.childerhose]

It should take advantage of dual CPUs based on tasks as noted here - https://bp.veeam.com/vbr/2_Design_Struc ... ositories/

[Ctek]

Hello
1) yes (at least I never hard of any all-flash system that is slower than spinning disks when talking about the same amount of disk space)
2) I would take the protocol / system where price / value is best
3) I prefer one RAID60 over multiple small RAID6. That makes it easier to manage
4) For today, I would go with REFS. With V12 and direct backup to object storage, object storage (minio or whatever makes sense) could become an alternative, yes
5) XFS, because of block cloning support

Best regards,
Hannes

A1 reply right there, +1 on all facts stated, I would just add, point 4, v12 with object storage, since it not launched yet, and object storage being what it is, I would not wager all my eggs for that basket right now, wait and see a bit how it behaves. Some providers have great performance, some others are more -ish. Make sure you think of 3-2-1 first. Cheers