Veeam NAS - Qnap 879u-RP

[jbrullemans]

Hi All,

I planning to get my data stored on a NAS. The setup would be as following:

- Veeam Running as VM on my ESX servers
- Veeam Backup as Virtual appliance with fail over to LAN.
- Repository is then the NAS QNAP 879U-RP via CIFS (QNAP 8x2TB RAID 0+1) which is located in an other building the building are connection via switches via fiber 1GB.

What performance would i get in my current setup just to a dell server with 4 local disk in it. And has any got experience with the QNAP 879 or any other QNAP products.
Do i mis anything on this setup. And would any recommend other NAS over the QNAP's?

John

[jpeake]

I have a couple of much smaller QNAP NAS's (TS-419P II) and performance is terrible. 30MBs is all I can get with write speed. Read speed is much better (around 90MBs). CPU on the NAS pegs 100% as soon as backup starts. Just not enough horsepower. I use them for remote DR, WAN is slow anyways.

The 879 has much better CPU, should be much faster I would think.

A Veeam server with local disks will smoke your NAS. Assuming your VM storage is up to the task. What are you using for VM storage?

[dbrett]

jpeake - the 419 uses the Marvell chipset doesn't it? If it's anything like the old 209 I had it will be really bad. I use a variety of the Atom based x59 models and don't have any issues, one has a .vbk file of > 4TB without any problems. I would think the x79 series would be plenty quick enough and I wouldn't be so sure a server with local disks would be *that* much quicker (not without spending $$$ anyway).

We run the NAS as a Linux server rather than a CIFS share - I don't think it's officially supported as the shell is not BASH, but it works well. (We did have a couple of teething troubles but got those sorted out).

[jpeake]

jpeake - the 419 uses the Marvell chipset doesn't it? If it's anything like the old 209 I had it will be really bad. I use a variety of the Atom based x59 models and don't have any issues, one has a .vbk file of > 4TB without any problems. I would think the x79 series would be plenty quick enough and I wouldn't be so sure a server with local disks would be *that* much quicker (not without spending $$$ anyway).

Yes it uses the Marvell CPU, and it's painful. Copying from the NAS is pretty fast though. It really struggles with writes (I have RAID 5 , 4 X 2TB disks).

So for anyone looking at a small NAS, the Marvell ones are cheap for a reason.

[jbrullemans]

About what speed are we talking about. Currently with backing up to disk with reverse incremental de actual day is written with 8-12MB/s . But for the overall process not in de detail is is now about 50-60 depending on how much changes there were down after the last backup.

I hope to get a better performance then currently our dell server does (Xeon 2.6ghz 4gb 8x1 sata raid 5.).
The host can take the performance cx3-10 15k fiberdisks.

[foggy]

John, please take a look at this topic for recommendations on different NAS devices.

[jbrullemans]

Hi Foggy,

I have read the forum only one question remains. On my DELL server with 8x1TB RAID 5 setup perc H700. when i look into the details of the job i.e. exchange server is the max data read is 12MB/s this is not the overall performance of the job but on detail level. Is this the expexted through put of SATA disks? I have done a test with creating another repository with 2 sas 10k raid 1 and i have in detail 110 MB/s. Is this logical? the DELL r501 is a qaud-core with 4gb. The other system is a system of 2006 Xeon dual-core with 2GB RAM. if this the expected performance of a SATA drive i can better buy a server with SAS and use that as a repository.

But for sure some people over have also a NAS running and can exchange me their performance results ( the detailed performance).

John

[dellock6]

Uhm, it could be some configuration problems in the SATA raid. SATA disks are slow, but not that much, nonetheless they could become slow if they are attacched to a slow or badly configured raid card: can you check if write cache is enable on the raid volume? Otherwise you are writing directly on every disk of the raid, plus computing data parity inline, and this could result in bad performances like those you are experiencing.

Luca.

[Yuki]

We are doing exactly this (minus the second building) for several of our clients.

Multiple servers to receive Veeam replicas and stay dark. Actual backups going to QNAPs (anywhere between 4 and 16 bay units, depending on the client).

We are getting about 70-90MBps on single 1Gbit link. We are about to go to 10Gbit on QNAPs, switches and servers to allow for even better rates. Just make sure your Veeam host has the processing power and connectivity to move and crunch the data. We've seen CPUs become the bottleneck on older hardware.

A few more notes:
Server: Dells PE710/PE720
Drives: SAS 10,000RPM and 15,000RPM
RAID: RAID 5 on PERC with either 512MB or 1GB cache
Veeam is assigned 4 CPUS and has 6GB RAM.

QNAPs: all intel based 4-16 bay units.
QNAP drives: 7200RPM Hitachi, Seagate, WD
QNAP RAID: RAID5



Even though our setup is not optimized for speed in terms of RAID levels or networking, we are still getting perfectly adequate performance.

[jbrullemans]

Uhm, it could be some configuration problems in the SATA raid. SATA disks are slow, but not that much, nonetheless they could become slow if they are attacched to a slow or badly configured raid card: can you check if write cache is enable on the raid volume? Otherwise you are writing directly on every disk of the raid, plus computing data parity inline, and this could result in bad performances like those you are experiencing.

Luca.

Hi Luca,

The write back cache in on. it is only two raid sets. 1 RADI1 for OS and application 2x1TB. And raid set 2 5x1TB RAID5 for data storage. Could it be that 5 disks in a raid 5 is to heavy for the raid controller?

[dellock6]

Well, I do not know the intimate technology of Qnap, but a counter-check you could do, if you can, is to recreate the volum with raid10, which has no write penalty due to parity. If it performs well, raid5 could be the problem. Parity-based raid are troublesome on low end storage devices.

[Yuki]

There is NO real hardware raid controller in QNAP. It is all done at the processor level through a RAID driver. As already said above - we have deployed many different QNAPs - from 4 to 16 bay and running RAID5 on them is not a problem.

[Gostev]

Does not matter. What Luca said regarding write penalty stands only has to deal with RAID level, and is true whether RAID is hardware or software based.

[jpeake]

On my pair of QNAP's (TS-419P II), in RAID 5 I am only getting 30MBs (write speeds). I set the other one to RAID 0, and write speeds almost double.

Huge hit when doing RAID parity on a low-end device.

[jpeake]

Here is what I am seeing on my cheap NAS array's. This is using iometer, 128K request size, 100% sequential, 4GB test file.

Arrays have 4 X 2TB SATA drives

RAID/Jumbo/Read (MB/s)/Write(MB/s)
0/Off/100+/50
5/Off /100+/32
0/9000/77/62
5/9000/78/36

[dellock6]

Raid 0 or you meant 10? Anyway, expected results, raid without parity are faster than those with parity (5 or 6 usually), and jumbo frames can help on large sequential transfers like backups.
My only doubt is about the block size, 128K is not a common value for filesystems, maybe 32k is more realistic. On NTFS for example the max value is 64k, but it depends on the size of the partition (http://support.microsoft.com/kb/140365 for more info).

Luca.

[Yuki]

Does not matter. What Luca said regarding write penalty stands only has to deal with RAID level, and is true whether RAID is hardware or software based.

Goestev - yes and no.

What he says is true, but from personal experience with QNAPs i can tell you their RAID is not the bottleneck for people posting here. We regularly push at 80-90MBps sustained speed on a single gigabit link. In some instances we have bonded Nics on QNAPs and are getting more than a single gig link would give. So again - what he says is true, but it is not the issue.

Here is a screenshot of me writing several files to a low end QNAP with RAID.

So as you can see, it can clearly take more than people here seem to allude. This is on a home system, with a built-in realtek gigabit adapter, cheap dumb gigabit switch (no jumbo frames), writing from Windows to an SMB share on QNAP. At clients where systems are optimized (ProCurve switches, jumbo frames, Intel Gigabit adapters (sometimes with LACP/trunking) we see higher rates.

[Yuki]

And here is a QNAP at one of the clients receiving a 6GB backup file via windows copy (Veeam has already finished backing up).

4 Disk RAID5 array as you can see in a second screenshot.
single 1Gbit link on better NIC/Switch/Server than my home setup above.






So, now tell me 4 disk QNAP can't write faster than 30MBps.

As i said before - there is a bottleneck, but it is not the QNAP hardware most likely. I bet your Veeam host may not be keeping up with processing data.

[jpeake]

Yuki - I wish I was getting that kind of write speed on RAID 5. I can get in that ballpark on RAID 0, but wouldn't want to run in that mode for real data storage. Which model is that, what firmware (I have TS-419PII running 3.7.3)? I have the same Seagate drives in mine, and it's slow.

I changed my Iometer spec to 32k data chunks like Luca suggested and reran test this morning. I am showing 40 MB/s, 1300 IOPS, 40ms latency. The host is a HP Proliant DL 360 G5, 8 cores at 2.5Ghz and 12GB RAM. Intel PT Quad Port NIC for iSCSI. Switch is Cisco 2960G, jumbo frames on and flow control on. Maybe i am missing something here. I have two of those NAS's, and would love to use one for secondary onsite backup if I could get faster writes.

[Yuki]

Last two screenshots are from QNAP 459+ Pro running on 3.7.3.20120801.

I don't remember the model of the unit in the first screenshot, but it is also an older one - not the new USB3, SATA III units available now.

[jpeake]

The CPU in that one crushes the Marvel CPU in mine. Maybe that's the big difference. My CPU is pegged at 100% when writing

[Yuki]

Yes, if you are on Marvel - then you are unlikely to see our results. All of the units we are using are Intel based. From Atom - to intel i3 and from 2-16 bay QNAPs. The CPU load on ours stay very low - 5-10% at most.

Not only do we run backups to QNAPs, i was able to run VMs from them via iSCSI target as well, but it is slower than on a decent server. Still something that could work in case of emergency. We've also done replication on these units and on the unit in the second set of screenshots - we are using Water/Fire/Shock proof eSATA drives to automatically off-load Veeam backups for off-siting. Those drives are picked up by storage company on regular schedule to be taken to a safe vault.

[Gostev]

Marvel = Awful performance (first hand experience)

[jpeake]

thanks guys. mystery solved for me.

[MaartenK]

We also have several QNAPs. With Atom and Marvell. With marvell we also see some performance issues. With the Intels Atom CPU or higher the speed is considerable better. LACP also cranks up the (network) speed. For backup with QNAP I would recommend a 8-serie or higher. And replication off course

[chimera]

Marvel = Awful performance (first hand experience)

Same case here, the QNAP TS419U II has absolute #$!@# throuhput, getting around 30MB/s and CPU maxing out at 100% Std vs Jumbo frames make no difference, except that std frames the CPU on the QNAP seems to fluctuate between around 95 and 100%, whereas jumbo frames sit on 100% constantly. It is a temporary solution for this client until they get a decent backup appliance which for now is ok, but given that QNAP released a model with a couple 1Gbps (128MB/s) NIC's yet the CPU can only cope with 1/4 of that theoretical performance (in RAID5) and not a great deal more in RAID0, with the limitations of the Marvel CPU it seems completely pointless releasing it in the first place... or is that to suck you into buying an Intel based model

Has anyone tested QNAP with CIFS vs iSCSI and is there much difference? I'd suspect not being that the CPU is the cause?

[Gostev]

You are right, CPU will be the bottleneck in any case. Could be even more so with iSCSI. I recommend shooting for dual-core Atom based NAS to never deal with CPU issues again.

[briggs]

Hi, I'm hoping the people who originally commented on this thread are still watching...
I have QNAP 459 Pro2's (so the Intel CPU's) and with the 1Gbps NIC, even when located on the same managed switch as the VM cluster, with RAID10 (4x4TB SATA 6Gbps brand new disks!), I can't get anything like these speeds. I can't even get better than 30MB/s infact. What is wrong with my setup, the Veeam jobs on full backups almost always report the bottleneck to be the target...
Thanks, Alan

[MaartenK]

Hi, I'm hoping the people who originally commented on this thread are still watching...
I have QNAP 459 Pro2's (so the Intel CPU's) and with the 1Gbps NIC, even when located on the same managed switch as the VM cluster, with RAID10 (4x4TB SATA 6Gbps brand new disks!), I can't get anything like these speeds. I can't even get better than 30MB/s infact. What is wrong with my setup, the Veeam jobs on full backups almost always report the bottleneck to be the target...
Thanks, Alan

What kind of firmware you are using? have you tried to copy a big file (ex. ISO) with SMB?

I`m got a 459Pro aswell (at home) with 4x 1TB WD Reds and a simple Linksys Gbit switch. I can push arround 90-100MB/sec from my pc (RAID 0).

What kind of speeds do you get when you using iSCSI in combination with Veeam? is it te same?

[dbrett]

I'm using a 459 Pro in our office and multiple customers are using 859 Pros with no issue, can quite happily saturate a 1Gb link.

I am using the Linux agent directly on the Qnap in house and a variety of SMB/Linux elsewhere. We are currently on the latest v3 firmware, 3.8.4 I think. I did try the 4.0 firmware but it didn't pick up eSATA very reliably and I am using that for Copy Jobs.

Have you checked CPU resources on the Qnap while your jobs are running?