Considering REFS for a small deployment, advice needed

[mephisto]

Hi guys,

I'm reading a lot about REFS as I'm trying to resolve an issue with merging old incrementals into a full backup (forever incremental) is taking ages. I'm using ZFS over iSCSI at the moment and running the Veeam server as a VM, but I'm planning to move the repository into the local storage of one of the hosts using a HW raid controller. Is HW raid controller a no go here? I've been reading that REFS and Storage Spaces would work best on HBA so Windows itself manages the parity, but the sever I'm testing this does not have a supported HBA controller, only HW raid is available.

Am I right to think that if I start using full synthetic backups instead of forever incremental in REFS will give me the same savings with storage as forever incremental but without the performance issues with merging old incrementals?

Would running the repository as a VM on top of ESXi and VMFS give much of a performance loss comparing to repository running bare metal with a HW raid? I'm planning to use this repository as well not for just this Veeam repository for VMs that I have running, but also I'm planning to use Veeam CC running in another instance but also connected to the same "repository" server just at a different folder. Would that be a bad idea? I would like for now to keep the Veeam backup server and CC separated but we have a small deployment, trying to keep complexity down.

[mephisto]

also after reading a bit more, does it make any difference using forever incremental or synthetic backup with blockcloning?

I believe forever incremental once the rerention is triggered the new full backup would simply be referenced to the previous incrementals, and once those incrementals are pushed out of the retention just the required data would be kept and referenced to the latest full backup file, right? Just trying to get my head around how it works using REFS and the current backup method.

Would reverse incremental also work same way?

[Gostev]

Yes.

[mephisto]

for which question? not sure which one you addressed

[Gostev]

Sorry - it was for the last one in your post, but kind of answers both anyway:D

[billcouper]

You won't use Storage Spaces if there is a hardware RAID card, just create a RAID array with volume formatted as ReFS. (Works for me).

ReFS can use more memory than NTFS or other filesystem, and similar to ZFS the larger your storage capacity the more RAM you should have.
I don't know what you consider 'small' deployment, but factor approximately 256-512MB of memory for every 1TB of storage plus 2GB for the base OS. (Very rough figures but you get the idea).

You can use a single repository server across multiple Veeam instances, but it isn't recommended. (Works for me).

As far as I am aware, if you migrate your existing backup chain to ReFS you will not get any block-cloning benefit. You will need to start a fresh backup chain on ReFS.

You can run the repository server as a VM and create ReFS inside a VMDK/VHDX, but it isn't recommended. (Works for me).

[mephisto]

Thanks mat,e very comprehensive answer! Ok, for the time being I'm going to use REFS over VMFS as I run the backup server as a VM and I'm setting up a different Veeam server for cloud connect as well.

I'll leave storage spaces out of this for the time being.

So best way forward would be bare metal windows with HBA and storage spaces? Would it make considerable difference or we are taking about a few percentage points here?

[nmdange]

You'll get better performance with a physical Windows server with a hardware RAID controller running RAID 6/60 vs. using Storage Spaces with Dual Parity. To get decent performance with storage spaces with parity, you have to use Storage Spaces Direct with at least 4 physical servers and use what's called Mirror-Accelerated Parity. And even then it won't be that great until Windows Server 2019.

The main reason to have a separate bare metal server over a VM with a virtual disk is to keep your backup infrastructure separate from your production infrastructure.

[billcouper]

Using a hardware RAID controller will provide better parity performance compared to Storage Spaces. It will also reduce memory requirement on the repository server, as Storage Spaces does use memory too (the RAID card doesn't).

The main reason you would consider Storage Spaces is to enable the self-healing capability of ReFS. It's a next-gen filesystem that can repair data corruption automatically, but that will only function if it's built on top of Storage Spaces (or S2D: storage spaces direct).

To get good performance on a standalone Storage Spaces server, you would choose Mirror and get 50% storage efficiency. Moving to a 4-node S2D will enable "mirror accelerated parity" to regain some of the performance, while maintaining most of the storage efficiency of parity. It must be noted that S2D requires Datacenter licensing of Windows Server and is therefore prohibitively expensive.

Which brings us to cost, which will probably be a primary factor. Standalone bare-metal Storage Spaces servers running Mirror spaces, or a 4-node S2D cluster might be the "best" or "recommended" solutions, but this isn't a primary storage solution and the cost per TB can quickly increase beyond the point you are comfortable spending for backup storage. Cost consideration is why I'm running my repo servers inside VM's and utilizing existing storage appliances via NFS-VMFS-VMDK layers of abstraction (definitely not recommended but working so far).

To use Storage Spaces in your case would 'ideally' require install a HBA into your storage server and do a mirrored Storage Space, so you have cost for hardware and cost for lost capacity. You will have to consider the price difference compared to using a RAID6 array. And it should be worth noting that ReFS driver has been fairly reliable for ~6 months now, so the risk of data corruption is low (it used to be a very large risk)

[mephisto]

I think I'll stick to a barebone server with Windows 2016 and HW raid controller then, it is very cheap and works, then format with REFS and use as a repository for my Veeam server that runs as a VM from another host. I'm thinking here, do you think it would be possible to have 2x 10GB nics working together on 2016 server as a backup repository for Veeam? Not sure if I can add a repository with multiple connections to a veeam server?

[ITP-Stan]

Just remember to format in REFS with 64K, it's not the default blocksize.
You would have to put the 2 nics in a teaming configuration.
Jumbo frames might also be a good idea, performance wise.