I can give you some (maybe useful, maybe not) insights as we're using Synology NAS and B&R 9.5
We use two NAS (2x RS3617RPxs) for Veeam Backups. They're stuffed with 6TB disks (WD Red Pro) and in a raid6 config, giving ~55TB of useable space. They're connected via iSCSI with a DA cable to the server using jumbo frames. We found that connecting them via a switch results in performance loss, regardless of jumbo-frames. The NAS and servers use 10gbps NICs. The unit price per NAS (including disks and NIC) is about $6000.
iSCSI does not give you the best performance (NFS / CIFS has better performance), BUT iSCSI allows you to use Server 2016 dedup functionality, which dramatically increases the retention periods, especially if you do regular active full backups. And the penalty is not that large, especially for large block sizes which is what you usually have with backups.
We get about 350MB/s write and 650MB/s read performance this way (on non-deduped data). Deduped data reads at about 30MB/s average, but the rate is very inconsistent (between 10MB/s and 150MB/s).
On a closer look:
- About 13TB of VMs (~40VMs total, largest is 2.3TB)
- physical server at the datacenter, connected to one NAS as well as the SAN storage
- physical server at our main office, connected to the other NAS as well as a tape library (MSL 4048 with one LTO-7 Ultrium drive).
Backup jobs run as follows:
- Backup all VMs from the NAS to the SAN (during lunch and at evenings of business days, with weekly active full, starting on friday, 7pm).
- As soon as a backup job finishes, copy the backups to the NAS at the main office. (10gbps line)
- As soon as the weekly active full backup copy job finishes, write the copy of the backups to tape
- Dedup any data that is older than 10 days
Our learnings after playing a lot with this setup:
- We can only achieve this our backup schedule by using post-job scripts
- Use single file per VM for your repositories, as this makes the life of the dedup job a lot easier.
- iSCSI is not the bottleneck. It's usually the disks themselves, because the backup takes place at large block sizes.
- We use GFS Tape Jobs. As GFS Tape Jobs do not support parallel processing, there is no point in having two Tape drives in your library (and those LTO-7 drives are quite expensive, we paid ~$5000 for ours).
- Do not dedup your newest active full, as it would slow down your desaster recovery significantly (30MB/s read instead of 600MB/s read). We're deduping only data that is older than 10 days.
- Dedup needs sh*tloads of RAM. In our case, we have about: 80TB of deduped user data (8TB dedup chunk stores), and 40GB of avaibale RAM (available for the dedup job, the server has 48GB of RAM). This is the limit - increasing the deduped data results in aborted dedup jobs. We will upgrade our servers to 192GB of RAM in the near future in the hope that this will increase the potential deduped data significantly.
The result of this setup is:
- desaster recovery speeds of about 600MB/s (newest restore point), which gives a full recovery time of <8 hours (including some admin overhead) in case of a SAN failure.
- 100 restore points on disk at two sites (which will hopefully go to 200 - 300 restore points after the RAM upgrade).
- Weekly, Monthly, Quarterly and Yearly backups on Tape (2 Tapes per full backup set).