10gbit on NAS : CPU recommendations

[remosito]

Hi everybody,

as I type this a tape backup job is running from our old NAS (secondary storage) to the tape on another Machine. It's an old Home NAS with 1gbit link and 4x2TB NAS disks in RAID5.
I only get 30-40MBps and proxy is given as bottleneck. And indeed CPU on NAS is at 99%. (dest proxy gets me close to 100MBps with 99% utilisation)

Now we just upgraded backbone to 10gbit and plan to hook up ESXi Hosts with 10gbit. And want to get a new NAS with 10gbit as well. Probably 8-12x4TB NAS Disks in RAID6.

Current low througput on the old NAS makes me worry about CPU in the NAS. Does anybody have experience of minimum CPU requirements so not the NAS CPU is the bottleneck ( source proxy ).

Say we go Synology, would any of the CPUs be the bottleneck:

- DS2015xs (Annapurna Labs Alpine AL-514 32-bit Quad Core 1.7 GHz )
- DS3615xs (Intel Core i3-4130 64-bit Dual Core 3.4 GHz )
- RS3614xs+ (Intel Xeon E3-1230 v2 64-bit Quad Core 3.3 GHz )

thanks a lot for any info and tipps

[Gostev]

Hi. Any reason why do you want to go with NAS specifically, as opposed just general purpose server with internal disks (which is generally faster, cheaper and more reliable storage for Veeam)?

[remosito]

3 Main reasons:

- Space. We only have space in a cupboard on one floor where we have free 10Gbit ports on a switch. NAS are rather optimized volume wise for 8-12HDs. Rackmount might be close though no rackmount rails
and the location is rather noise sensitive (which is why I'd like to avoid the Rackmount NAS too. The Box ones have bigger fans. And in case of Synology are 28 vs 42db).

- Time. My time. I have way to much overtime already. I have ordered and setup and maintained over the year the current NAS in way less time than I spent on any of our servers.

- Was stupid enough to turn on some of the "fun" User features (photo, music, Fileversioning ....). Some users are rather attached now.


Obviously one and two are the main reasons.

Anyway. Doesn't really change much about the question. What level of (per core) CPU power is required for the source proxy to not be the bottleneck for 8-12 6TB NAS disks in RAID6 over 10gbit link.
Even general purpose Servers do have different CPUs built in. Is a 2Core4Thread standard Processor enough? Would a Celeron do? 4 Cores required? Xeon?

[PTide]

Hi,

How is your NAS connected to VBR?

Thank you.

[Gostev]

What level of (per core) CPU power is required for the source proxy to not be the bottleneck for 8-12 6TB NAS disks in RAID6 over 10gbit link.
Even general purpose Servers do have different CPUs built in. Is a 2Core4Thread standard Processor enough? Would a Celeron do? 4 Cores required? Xeon?

Wait, are you asking about source proxy server CPU recommendations, or NAS CPU recommendations?

[remosito]

How is your NAS connected to VBR?

VBR runs in a VM on a ESXi Server on the same 10gbit backbone Network as the NAS.

The NAS is added to VBR as a Backup Repository of Type "Linux Server".

[remosito]

Wait, are you asking about source proxy server CPU recommendations, or NAS CPU recommendations?

They are the same in this case. When doing a Tape Job of Files on the NAS to a tape on the VM VBR runs and the Tape is attached. The NAS runs a veeam agent source proxy process. On my old NAS that one reaches 99% utilisation while only delivering 40ish MB/s.

[Gostev]

Got it. Honestly, I think any modern Intel processor that is not Atom should do, as tape archival jobs are not resource intensive for data movers (as no data processing happens in this case). You would want something beefier for VMware/Hyper-V backup proxy server CPU, because they do additional data processing on raw data. But once backup has been created, it's only basic I/O operations to move its existing blocks over to tape.

[remosito]

Thanks for the feedback. Good to hear you don't think I'd need to spring for a 4 Core Xeon and rackmount NAS.

Our old one has a Intel® Atom™ D510, Single Core 1,67 GHz and only gets 40ish MB/s.

Which is why I am a bit hesitant about the Annapurna 1.7Ghz CPU in the DS2015xs. That NAS would fit the general bill the best. But low frequency makes me worry about single core performance a bit (which I expect is the determining one for veeam source proxy performance).

Btw the tape archival one is the only one we use for the moment. But might add different ones in the long run. And I'd assume restoring files/VMs from the NAS would involve heavier workload as well.

[jtek]

I was recently looking at the DS2015xs and when doing some initial searches trying to determine performance a few sites showed some pretty impressive numbers. I think Annapurna Labs Alpine AL-514 32-bit Quad Core 1.7 GHz will be plenty of power. This is one of the articles I am referring to: http://www.anandtech.com/show/8985/syno ... -10g-nas/2

I have recently become a big fan of Synology and started to put them in a lot of my customers locations as their primary backup for veeam and recently started to look at them for some SMB locations for running small 10-20 vdi setups with the 10gig models of course.

[remosito]

Did some more reading of benchmarks over the weekend. And saw the anandtech one as well. While performance is pretty good. At a few places anandtech and others have found single core performance to be the limiting factor. Because of that I am now more leaning towards the DS3615xs with the Intel CPU.

Sure. Costs a bit over a grand more. But on the total cost of moving backbone and Servers onto 10gbit. It's really not that much more....

If ever you do get the DS2015xs would you mind coming back and post if and how the Annapurna gares with the veeam agents?

[Gostev]

Wait, but Annapurna Alpine AL-514 seems to be an ARM processor, so it cannot run Veeam data movers at all?

[jschenck]

Not sure what you budget looks like but I've bought a few Synology's and their performance has been quite good. They have an option to install an SSD and use it as a cache mechanism in front of the spinning disks. The software features of the Synology are amazing.

https://www.synology.com/en-us/products/DS3615xs

[jschenck]

Now that I actually read the discussion thread here I see that you've been looking at the various Synology products. I bought a 2414 and a 3614 - both will easily saturate the 1g network link without breaking a sweat on the CPU. the 2414 does not have the option of putting a PCI card in so the 3614 is the only way to get 10g enet in my situation. I only use these devices as Veeam backup repo targets so 1g enet was/is good enough for me. Our high performance NAS is a set of Netapp FAS 8020's clusters, but that is a different budget level.

Looks like Synology has added the 10g option to more of their product line for this year. I would suggest buying the best that your budget allows since storage is the backbone of the virtual infrastructure. In this budget space Synology is the brand I would look at based on personal experience.

BTW, I saw a significant performance increase by using jumbo frames on the ethernet. Dedicated VLAN setup for the NAS to ESX NFS connection, make sure MTU=9000 setup on the switch and vswitch and storage. The larger enet frames reduces CPU load, fewer packets to process for the same amount of data.

[hoFFy]

I don't get it. I never used a proxy on the NAS. All processing power came from the VBR itself, or when its an very old server for just holding the disk space I used VMs on the source hosts as proxy servers.
What is the reason to use the NAS' CPUs for this?

Hi. Any reason why do you want to go with NAS specifically, as opposed just general purpose server with internal disks (which is generally faster, cheaper and more reliable storage for Veeam)?

Faster - yes
Cheaper - ? Do you have any recommendations which are cheaper with.. lets say 4-6 drives with 3-4TB disks in RAID 6
Which OS do you ave in mind? FreeNAS based on FreeBSD? It should be as simple as possible (not like Lucas' tests with Ceph ) We calculate the prices for our customers based on the time we need for everything... If a NAS is set up in 20 minutes and a small server needs 4 hours, then the decision will always be the NAS.

To clarify: We talk about a NAS as a backup target for small networks. As soon as there are several TBs of data with a high change rate we use dedicated VBR servers

[Gostev]

Sebastian. We're kind of hi-jacking this thread here... but personally, I would just create "all in one" backup appliance for those small environments. Physical server with Veeam installed, and internal storage as a backup repository. Makes things simple for you - just bring that into the new environment, hook into the network and you're done! Otherwise, you'd still need to figure out where to install Veeam itself, etc.

[hoFFy]

Of course this gets a bit off-topic. Shall we open a new thread?
I'm interested in your statement that a new server only for VBR is cheaper than the combination of a VM for VBR and a NAS as a target.
When I sum up the costs for a NAS (lets say QNAP), the disks and a 10GB-NIC in the NAS I'm at about 2000 - 2200€. A new server + disks + NIC is a bit more expensive, isn't it?

[Gostev]

Well, you will need to do some research based on your specific requirements (or open a new topic to get some feedback from other users).

But, for example at the lowest end, for example HP MicroServer will be a better choice than any low-end NAS out there from every perspective. That is, if you are looking for a hardware with big brand name, as of course there are cheaper brands too (like SuperMicro or local micro-manufacturers).

But I think I see where our disconnect on pricing is. When looking at the price, remember that when buying a server, you are buying the whole package that will be able to run the entire solution. But when buying NAS, you are buying storage alone - and will still need a server to run the actual solution (but even VM cannot be considered "free" by any means, as it will steal the exact same amount of resources, just from another physical server). So, you have to account the price of the latter as well, if you want fair apple to apple comparison.

Not to mention that I hate the idea of running backup server in a VM in the first place, because it will be lost along with other production VMs preventing you from performing restores, but that's another long topic. Just something else to consider besides much better performance and reliability of a server-based solution.

[remosito]

Wait, but Annapurna Alpine AL-514 seems to be an ARM processor, so it cannot run Veeam data movers at all?

Thank you so much for that post. Didn't even know the veeamagent doesn't run on ARM based NAS. Our current one has a Intel one core so we didn't run into it.

[remosito]

Now that I actually read the discussion thread here I see that you've been looking at the various Synology products. I bought a 2414 and a 3614 - both will easily saturate the 1g network link without breaking a sweat on the CPU. the 2414 does not have the option of putting a PCI card in so the 3614 is the only way to get 10g enet in my situation. I only use these devices as Veeam backup repo targets so 1g enet was/is good enough for me. Our high performance NAS is a set of Netapp FAS 8020's clusters, but that is a different budget level.

Looks like Synology has added the 10g option to more of their product line for this year. I would suggest buying the best that your budget allows since storage is the backbone of the virtual infrastructure. In this budget space Synology is the brand I would look at based on personal experience.

BTW, I saw a significant performance increase by using jumbo frames on the ethernet. Dedicated VLAN setup for the NAS to ESX NFS connection, make sure MTU=9000 setup on the switch and vswitch and storage. The larger enet frames reduces CPU load, fewer packets to process for the same amount of data.

Thank you for the feedback. Especially in regards to jumbo frames. Might do that too if performance isn't good enough.

When you say significant performance increase. How much are we talking? On what NAS?

[remosito]

I don't get it. I never used a proxy on the NAS. All processing power came from the VBR itself, or when its an very old server for just holding the disk space I used VMs on the source hosts as proxy servers.
What is the reason to use the NAS' CPUs for this?

It's just what Veeam does when you add the NAS as a repository as a "Linux Server". Then the NAS is the VBR (assuming that's short for Veeam Backup Repository). And a veeamagent get's executed on it doing some of the lifting.

Faster - yes
Cheaper - ? Do you have any recommendations which are cheaper with.. lets say 4-6 drives with 3-4TB disks in RAID 6
Which OS do you ave in mind? FreeNAS based on FreeBSD? It should be as simple as possible (not like Lucas' tests with Ceph ) We calculate the prices for our customers based on the time we need for everything... If a NAS is set up in 20 minutes and a small server needs 4 hours, then the decision will always be the NAS.

To clarify: We talk about a NAS as a backup target for small networks. As soon as there are several TBs of data with a high change rate we use dedicated VBR servers

Indeed. If you have to factor in worktime then a NAS becomes much cheaper very fast. If you don't because the employee is paid anyway and not exactly worktime starved, then a roll your own might be cheaper.

[remosito]

Well, you will need to do some research based on your specific requirements (or open a new topic to get some feedback from other users).

No worries about hijacking the thread.

It was already very helpful and we decided on the Intel based "bigger and better" NAS (DS3615xs)

I can see how ones preferences for organizing Virtualization makes a NAS a better or worse choice compared to a standalone roll yourself storage server.

In our case. All we need is storage. Our existing servers cover everything resources wise we need. Except storage. And we don't have enough SAS slots left in any of them to add the required amount of storage on the cheap (we could get hyperexpensive 2.5" high-capacity SSDs I guess. But just one or two would cost us more than the NAS).

And in our case the Veeam Backup Server is in a VM (we had one VM running Windows already for some of the software we need. Made much more sense to add it there than add another standalone server that runs all day long). The Veeam Backup Server VM is replicated on the failover standby server. So I don't worry to much about availability.