andyg wrote:Nice Post, we have similar hardware space but use ZFS on Linux though, so I'm curious to see what you think of Win 2016 and ReFS as a replacement to ZFS. (do you have a blog or any guides you follow to get your ZFS setup?)
The reason we didn't go for ZFS on Linux is that its not properly supported by Napp-it. As we aren't Solaris experts we chose to manage our ZFS layer largely by webui through the napp-it interface.
Our primary storage is coming off an IBM DS Fibre channel storage unit - we are switching about 70TB of primary storage to a Nimble all Flash array within the next month - at this stage the hyper converged setups were deemed to be too risky for us - Windows 2016 only went RTM recently (despite us having some hyper converged units running storage spaces direct for some time on technical preview)..
For a single node backup/management host, IMHO win2016 and ReFS doesn't cut the mustard due to lack of striping across pool sets and restrictions about the use of SSD acceleration and lack of global hot spares. So unless you have a 4 node hyper converged infrastructure running just for backups, the main benefit of using Re-FS on top of ZFS is that we get maximum performance and capacity on the spindles with the benefits of application aware de-dupe in Server 2016 that is coming in Veeam 9.5. End result is that our backup and management layer costs can be kept down to the best bang for the buck.
However for a multi node hyper converged solution for primary storage I'd run 2016 with storage spaces direct in preference over other solutions right now if we were a hyperv shop (but we are a vmware shop currently and are likely to stay that way for the foreseeable future particularly with the shift to core licensing model of Data Centre Edition).
The beauty of using an architecture like we have is that we are currently exposing the old CIFS share to the Re-FS VM, and copying the old backup files from there and seeding the new Re-FS file system, so that we can carry on the backups where they left off and switch over without an issue to our new architecture. When 9.5 comes along we'll just enable the check box that says Re-FS file system and we'll start getting the speed up performances promised, but in the mean time we'll be hopefully getting the de-dupe benefits as the ReFS system is being seeded.
By continuing to use ZFS under the hood we are also protecting against things like bit rot in a more proven way as only time will tell as to the reliability of Re-FS.
I look after a large dev shop over here, so we have a fair amount of flexibility - our set up really just comes from a 100% virtualised approach (including vcentre and all Veeam components) and lots of experimentation and reading! so we make it all up as we go along