Comprehensive data protection for all workloads
Post Reply
Gostev
Chief Product Officer
Posts: 31751
Liked: 7256 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Recommendations for backup storage, backup target

Post by Gostev » 6 people like this post

[UPDATE] This recommendation is still valid in 2021. In fact, V11 is so highly optimized for deployments on general-purpose servers that backup performance can exceed 11GB/s with a single box!

This topic is the continuation of the previous discussion, where some recommendations were no longer correct because they were made a few years ago, and the product architecture has changed significantly since then.

At this time, the best low end primary backup storage for Veeam is Windows or Linux server with internal or direct attached (JBOD) storage. Below, I'm quoting a couple of weekly forum digest articles from earlier this year where I have discussed such backup targets in more details.

#1

What is the best primary storage for Veeam backups? This is certainly in the top 10 questions I get from our partners who want to build the complete solution for their clients in the most optimal manner. Higher-end storage is usually easy - if you have big money to spend, you will get great results with most of the available options, be it serious deduplicating storage device or high-capacity SAN. But, what about lower end storage systems? Most people seem to think "since I need the cheapest storage for my backups, I will just buy the cheapest NAS", which is actually a bad choice! Those cheap NAS devices are neither reliable (top cause of corrupted backup files), nor fast (too few spindles and dual 1Gb Ethernet connectivity at best), nor they are actually the cheapest option (comparing to another one you get with Veeam).

What if I told you that you can get a very reliable and fast backup storage without actually having to buy one? Thanks to B&R's architecture, you can do this. Take any decommissioned physical Windows or Linux server, stuff it with a bunch of hard drives - and you get the most economical, fast and reliable target for Veeam B&R you can possibly get at this price point. Sure, you can probably build even cheaper, or faster, or more reliable target - but the above route will always provide for the best possible combination of all three. And even if you run out of space for more hard drives within the enclosure, you can still grow capacity in a very cost effective manner by direct attaching JBOD, which are quite cheap.

So, what is it exactly that makes a physical server with internal (or direct attached) storage the best choice for storing backups? It's cheap, because in most cases you don't even have to buy one - any virtualization project leaves plenty of those behind (and if not, then you are doing something wrong). It's reliable, because we are talking plain vanilla operating system without any hacks or optimizations, and there is no file shares involved. And it is very fast, because all I/O intensive operations on backup files (such as transformation) are performed locally by data mover running right on the box, instead of over relatively slow Ethernet network. Finally, you can easily grow both capacity (by adding more JBODs) and scalability (by adding additional servers) of this storage platform along with your virtual infrastructure with minimal additional investments. The latter is actually why I tend not to limit this solution to SMB only, as I've seen quite a few of mid-size customers using this storage concept with great success.

#2

Here is really nice write up and real-world proof that my proposed storage architecture scales in both capacity and scalability to support even largest environments of 100 TB and more in size!
We used another product for our backups for years. I was brought in last year because the company had many issues with this application. I was able to clean up almost all of the issues but we ended up switching to Veeam anyway. You see as part of their mitigation to get backups done while they had issues the previous admin used Veeam to do backups on some critical systems. Veeam worked so well that we decided to virtualize 99% of the environment in order to take full advantage of Veeam B&R.

I as began my deployment I utilized the NetApp FAS2240 that our old product used to backup too. I quickly found that doing reverse incremental backups to LUNS on this NetApp was not going to work. After much investigation with NetApp support we found that the way the NetApp writes blocks is not very compatible with the way Veeam writes blocks. It causes a huge amount of resource utilization on the NetApp an so backups and tape outs are extremely slow.

I started searching for another solution and luckily we had a lot of old HP DL380 servers and SAS attached disk arrays that had been retired. I decided to build a new backup server using this equipment for testing. I build a 60Tb backup server using only a DL380 & SAS Attached arrays full of 2TB SATA disks. The performance increase was dramatic. So dramatic that I can now perform all of my backups every night with fulls on the weekends and tape them out every night with no issues. This was something that we were never able to do with our old product alone.

Since the storage I used was all out of warranty I convinced management that we should purchase new backup units to replace these. We are purchaing a new DL380 with attached storage that will have 80Tb of usable space. We are also buying another unit to send to DR so I can run backup copies to it. The cost for this storage significantly less than a NetApp. The NetApp units can easily top 6 figures for only 40Tb where as the HP Storage is less than $50k for 80Tb.
dcolpitts
Veeam ProPartner
Posts: 119
Liked: 24 times
Joined: Apr 01, 2011 10:36 am
Full Name: Dean Colpitts
Location: Atlantic coast of Canada
Contact:

Re: Recommendations for backup storage, backup target

Post by dcolpitts »

Not to burst anyone's bubble, but the DL380 solution mentioned above is the same hardware solution as HP's StoreOnce 4500 & 4700 solutions as both the 4500 and 4700 are built on DL380p Gen8s with attached JBODs, except building it yourself does't get you HP's dedup & replication - although the plain old DL380 is likely better suited for those on a budget...

dcc
m.novelli
Veeam ProPartner
Posts: 556
Liked: 103 times
Joined: Dec 29, 2009 12:48 pm
Full Name: Marco Novelli
Location: Asti - Italy
Contact:

Re: Recommendations for backup storage, backup target

Post by m.novelli » 2 people like this post

Would like to share my experience. I really love servers with internal storage as Veeam Backup repository.

Recently I've installed a Dell R720xd server, with a PERC H710 controller and eleven 4 TB 7.200rpm SATA disk configured in RAID5 (plus one hot spare for more security). The available usable space with this config is about 36 TB.

The server is connected via iSCSI to three VMware hosts and a couple of Dell PowerVault MD storage. The iSCSI chain is 10 Gbit end to end.

So, with that R720xd server (about 10-11k euro of cost) I can run Full Active backups at 230 MB/sec sustained speed on SATA disks!

Marco
guitarfish
Enthusiast
Posts: 98
Liked: 12 times
Joined: Mar 06, 2013 4:12 pm
Contact:

[MERGED] Using Windows Server as backup repository/target

Post by guitarfish »

I presently use Veeam 7.0 to backup two ESXi 5.1 hosts to a QNAP NAS at a couple remote sites. I am using a Windows 7 Pro x64 system as the proxy server. I am now looking to deploy two ESXi 5.5 hosts at our HQ, and I'd like something better than the NAS for a target.

I've been looking at some higher end NAS devices, which are basically Windows Server 2012 appliances. If this is the case, why not just use a Windows Server 2012 R2 system (which I happen to have as a spare) as a backup repository? The system has dual gigabit Ethernet, and I can put a pair of SAS drives in it (RAID-1) and away we go. My total backup size of all VMs will be well under 1TB.

Is using a Windows Server a practical approach for a small setup like mine?
Gostev
Chief Product Officer
Posts: 31751
Liked: 7256 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Recommendations for backup storage, backup target

Post by Gostev »

It is not only practical, but also the recommended approach. And not just for small setups such as yours... read some feedback above ;)
guitarfish
Enthusiast
Posts: 98
Liked: 12 times
Joined: Mar 06, 2013 4:12 pm
Contact:

Re: Recommendations for backup storage, backup target

Post by guitarfish »

Ya know, I remember reading the post above from April, but I couldn't find it. Thank you for directing me to it!

So if I understand this, the Windows Server where the DAS is (the repository) can serve as the proxy server also, which would eliminate the need for me to go from ESXi -> Win 7 proxy -> NAS in my current setups?
Gostev
Chief Product Officer
Posts: 31751
Liked: 7256 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Recommendations for backup storage, backup target

Post by Gostev »

That is correct.
ditguy2012
Novice
Posts: 8
Liked: never
Joined: Dec 03, 2014 7:26 pm
Full Name: Blake
Contact:

[MERGED] Cheap and fast disk target recommendation

Post by ditguy2012 »

I know this question has been asked in the past but I'm hoping to find some updated experiences. What are people using to store their disk to disk backups and do you like it?

We're a dell shop so we naturally got quotes about the Powervault series systems. But I'm wondering if there are faster new technologies out there or is it the same old game: maximize your spindle count and disk speed to get backup and restores going faster. you can also use things like synthetic fulls but when it comes to IOs I wonder if there are less expensive options. flash and ssd only seem to help if there's a deduplication database which there isn't in veeam and we need many terabytes of disk space so flash/ssd seems to expensive.

Thoughts anyone?
m.novelli
Veeam ProPartner
Posts: 556
Liked: 103 times
Joined: Dec 29, 2009 12:48 pm
Full Name: Marco Novelli
Location: Asti - Italy
Contact:

Re: [MERGED] Cheap and fast disk target recommendation

Post by m.novelli » 2 people like this post

Dell PowerEdge R730xd with twelve 4TB NLSAS hard disk and 10 Gbit networking is a killer Veeam Backup server

With SATA disks the SureBackup jobs are a little bit slow and sometimes they fail starting Exchange or SQL Servers VM (I'm talking about the Dell PowerEdge R720xd with previous generation of SAS controller)

Cheers
getzjd
Enthusiast
Posts: 25
Liked: 3 times
Joined: Mar 12, 2012 3:10 am
Full Name: Jay
Location: Cincinnati, OH
Contact:

Re: Recommendations for backup storage, backup target

Post by getzjd »

I just installed a Synology DS1815+ with eight 6TB Western Digital red drives RAID 10 connected via MPIO iSCSI and Windows 2012R2 virtualized Veeam server using direct SAN access mode. My target bottleneck is between 0% and 11% so the Synology is doing a great job. My source is an old CX4 series SAN. We will see how this adjusts in the next few weeks after I cutover to a VNX and off the old CX4 SAN.

Processing rate for 4 jobs totaling 640GB 136 MB/s

Source 98% > Proxy 69% > Network 12% > Target 8%
ditguy2012
Novice
Posts: 8
Liked: never
Joined: Dec 03, 2014 7:26 pm
Full Name: Blake
Contact:

Re: [MERGED] Cheap and fast disk target recommendation

Post by ditguy2012 »

m.novelli wrote:Dell PowerEdge R730xd with twelve 4TB NLSAS hard disk and 10 Gbit networking is a killer Veeam Backup server

With SATA disks the SureBackup jobs are a little bit slow and sometimes they fail starting Exchange or SQL Servers VM (I'm talking about the Dell PowerEdge R720xd with previous generation of SAS controller)

Cheers
Hi m.novelli. Thanks for the reply. What RAID did you do with 12 4TB drives? We considered doing the Dell R730xd and putting 24x 1.2TB 10K drives in it with a 2GB RAID Controller, 2x 10-core CPUs and 64GB of memory.

The other big concern we're seeing is needing such a high amount of disk space for doing synthetic fulls. Apparently we can do incremental forever but the question is if that means doing a full restore requires a bit from every day the server was ever backed up? Is a better idea to do a real full maybe once a month so that you're not doing restores that many days back? We were told for 18TB of data today, growing to 36 over the next 3 years that we needed over 100TB of space to do the following:

fulls weekly, keep for 5 iterations
daily's, keep for 30 days
monthly fulls keep for a year (the day we do monthly's we don't do weekly fulls).
jwest
Lurker
Posts: 1
Liked: never
Joined: Dec 16, 2014 11:41 pm
Full Name: Jason
Contact:

Re: Recommendations for backup storage, backup target

Post by jwest »

I curious if you are going with dedicated storage servers for the Veeam backup repository, are you also using them as the Veeam proxy as well. Any benefit using VM proxy as the dedicated server should have plenty of CPU and RAM resources if using current CPU & RAM technologies. What happens if you run out space? Just add a second dedicated storage server and then split your jobs accordingly?
Dave23
Novice
Posts: 5
Liked: never
Joined: Sep 20, 2013 3:38 pm
Full Name: Dave
Contact:

[MERGED] Storage Devices for backups

Post by Dave23 »

I'm just curious to know what others are using for disk to disk backup solutions for the Veeam environment? Looking for low cost storage device.
k00laid
Veeam Vanguard
Posts: 227
Liked: 55 times
Joined: Jan 13, 2011 5:42 pm
Full Name: Jim Jones
Location: Hurricane, WV
Contact:

Re: Recommendations for backup storage, backup target

Post by k00laid » 2 people like this post

Like m.novelli above I am using a Dell PE 720xd chocked full of drives with Server 2012R2 installed. After RAID 5 my usable for a repository is about 24TB. Further I 1) Formatted the drive with the /L modifier (important!) and then 2) enabled Windows Disk Deduplication on the drive with a dedupe cycle of 8 days to allow a synthetic full operation to happen more efficiently. With that I'm currently holding 65 TB of backups in about 3.46TB of actual disk space. I know that this method has fallen out of favor with Veeam, but I swear by it and as long as you do the correct formatting on the server you shouldn't run into corruption issues.
Jim Jones, Sr. Product Infrastructure Architect @iland / @1111systems, Veeam Vanguard
Gostev
Chief Product Officer
Posts: 31751
Liked: 7256 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Recommendations for backup storage, backup target

Post by Gostev »

It has indeed fallen out of favor for large backup sizes (above few TB). Depending on your full backup size, you can be totally fine though. One other gotcha is the fact that the technology itself is very young comparing to plain vanilla NTFS, and could contain data corruption bugs (we did observe a few corruptions on such volumes in support, but of course it could have been caused by other issues too).
k00laid
Veeam Vanguard
Posts: 227
Liked: 55 times
Joined: Jan 13, 2011 5:42 pm
Full Name: Jim Jones
Location: Hurricane, WV
Contact:

Re: Recommendations for backup storage, backup target

Post by k00laid »

I ran into corruption as well but that had to do with the formatting, once done correctly it has worked perfectly (knocking on wood now) for over a year.
Jim Jones, Sr. Product Infrastructure Architect @iland / @1111systems, Veeam Vanguard
lsumarkb
Novice
Posts: 5
Liked: never
Joined: Mar 18, 2011 3:36 pm
Full Name: Mark T Breaux
Contact:

Re: Recommendations for backup storage, backup target

Post by lsumarkb »

I did a quick google search and I don't see an "/L" modifier. Can you elaborate on the formatting, @k00laid?

@Gostev - If I understand you correctly, your recommended setup would be a physical windows 2012 R2 box that is loaded with drives and all Veeam roles are installed on it? We have our iSCSI storage network split off onto its own switch for vSphere 5.5. Would I hooked this server into that network or does veeam require it to be on the management/vm network?

Thanks for the help!
foggy
Veeam Software
Posts: 21133
Liked: 2140 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Recommendations for backup storage, backup target

Post by foggy »

lsumarkb wrote:I did a quick google search and I don't see an "/L" modifier. Can you elaborate on the formatting, @k00laid?
By default, NTFS volumes are formatted with small size file records. This switch enables formatting with larger ones. A good blog post on that, btw.
lsumarkb
Novice
Posts: 5
Liked: never
Joined: Mar 18, 2011 3:36 pm
Full Name: Mark T Breaux
Contact:

Re: Recommendations for backup storage, backup target

Post by lsumarkb »

@foggy - Thank you sir! If I use ReFS with storage spaces then I don't have to worry about that I assume?
foggy
Veeam Software
Posts: 21133
Liked: 2140 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Recommendations for backup storage, backup target

Post by foggy »

Then you don't have to worry about both points mentioned by Jim, since deduplication is also not supported on ReFS. ;)
lsumarkb
Novice
Posts: 5
Liked: never
Joined: Mar 18, 2011 3:36 pm
Full Name: Mark T Breaux
Contact:

Re: Recommendations for backup storage, backup target

Post by lsumarkb »

Really?! Missed that in the many articles I have been reading on storage spaces, deduplication, and ReFS. MS dropped the ball on that one. Thanks Foggy.
vaicsimmons
Influencer
Posts: 21
Liked: 1 time
Joined: Mar 27, 2015 2:07 pm
Full Name: Chris Simmons
Contact:

[MERGED] Hard Drive & RAID Recommendations

Post by vaicsimmons »

I am setting up Veeam B&R on a new server with local hard drives attached. The server will be running Windows Server 2012 R2 and uses a Supermicro chasis w/ 12 bays and an LSI 9260 connected to the backplane. I planned on using either 4 or 6 TB drives. We are a SMB that has about 1.5 TB of data in total.

I would like to keep 14 daily incrementals, 4 weekly fulls, 12 monthly fulls and 7 yearly fulls. Our file server is pretty stagnant so I don't expect much growth there. I have an Exchange server that houses roughly 50 mailboxes with a database size no larger than 80 GB along with a SQL server that holds less than 10 databases consuming no more than 15 GB. Our business hours are M-F 8:00-5:00. Therefore, my backup window is pretty large on the weekdays and then all weekend long as well.

This server will be kept in the same rack as my ESXi hosts. I will probably end up buying a Synology or Netgear NAS and placing that in another building of ours that s about 1/4 mile down the road and connected via fiber and doing a backup copy job. This will allow me to get my backups offsite as well.

With all of that said, I have a few questions:

1. Should I use SAS or SATA drives and why?

2. Should I spend more for Enterprise class drives or not and why?

3. Any particular brand preferred....HGST, WD, Seagate?

4. Due to my large backup window, is it preferred to roll with RAID 6 to allow for a larger array vs RAID 10?

I am also considering using Windows 2012 R2 deduplication on the volume storing my weekly, monthly and yearly backups. Any comments / feedback using Windows dedup is appreciated as well.
Shestakov
Veteran
Posts: 7328
Liked: 781 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Recommendations for backup storage, backup target

Post by Shestakov »

Hello Chris,

Most of answers can be found in the thread above. Please review and ask additional questions if you have any.
vaicsimmons wrote:Due to my large backup window, is it preferred to roll with RAID 6 to allow for a larger array vs RAID 10?
RAID 10 is a recommended approach due to better performance.
vaicsimmons wrote:I am also considering using Windows 2012 R2 deduplication on the volume storing my weekly, monthly and yearly backups. Any comments / feedback using Windows dedup is appreciated as well.
Here is a great related topic. Please take a look. Thanks!
m.novelli
Veeam ProPartner
Posts: 556
Liked: 103 times
Joined: Dec 29, 2009 12:48 pm
Full Name: Marco Novelli
Location: Asti - Italy
Contact:

Re: Recommendations for backup storage, backup target

Post by m.novelli »

NLSAS disks (similar to WD Red Pro)
RAID 5 + 1 hot spare
vaicsimmons
Influencer
Posts: 21
Liked: 1 time
Joined: Mar 27, 2015 2:07 pm
Full Name: Chris Simmons
Contact:

Re: Recommendations for backup storage, backup target

Post by vaicsimmons »

m.novelli wrote:NLSAS disks (similar to WD Red Pro)
RAID 5 + 1 hot spare
Just curious....why RAID 5??? Just about every opinion that I have read said to stay away from RAID 5 as the chances of a hard drive failure upon a rebuild of the array is pretty high. Therefore, you would lose everything that was on the array.
dellock6
VeeaMVP
Posts: 6163
Liked: 1971 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Recommendations for backup storage, backup target

Post by dellock6 » 1 person likes this post

In general, now that disks are becoming bigger and bigger, any raid using disk at least of 3TB should really use double parity or raid10 with multiple disks (which in a way is still a single disk protection, if you loose two disks of the same R1 sub-group, you've lost data). On large data sets, this is the reason why you see at least the usage of RAID6, but with huge write penalty, or for even bigger data sets storage is moving to different technologies (replication over scale-out or erasure coding).
RAID simply can't keep up anymore with such large sets.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
Gostev
Chief Product Officer
Posts: 31751
Liked: 7256 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Recommendations for backup storage, backup target

Post by Gostev »

Yep. RAID5 was good in early days of small hard drives because of its lower write I/O penalty (lower than RAID6). But these days, most people consider it too risky.
m.novelli
Veeam ProPartner
Posts: 556
Liked: 103 times
Joined: Dec 29, 2009 12:48 pm
Full Name: Marco Novelli
Location: Asti - Italy
Contact:

Re: Recommendations for backup storage, backup target

Post by m.novelli »

vaicsimmons wrote: Just curious....why RAID 5??? Just about every opinion that I have read said to stay away from RAID 5 as the chances of a hard drive failure upon a rebuild of the array is pretty high. Therefore, you would lose everything that was on the array.
RAID6 write penalty is too high IMHO for a storage dedicated to backups

In 15+ years of IT I've never lost a RAID5... All my servers and storage always have firmware updated (both raid card and disk firmware). I use only Dell PowerEdge servers. In the last years I've prepared many Veeam Backup servers with 2 - 3 - 4 and now 6 TB NLSAS disks, so far so good. I always use an hot spare. On a recent Dell PowerEdge T630 where failed a 6TB disk, the automatic hot spare coverage took about 8 hours to complete the rebuild (it was a RAID5 made of nine 6 TB disks)

I can't see why a disk that get intensive writes during backups should have an higher failure rate due to intensives reads in case of array rebuild...

Marco
GarethUK
Influencer
Posts: 22
Liked: 2 times
Joined: Mar 21, 2014 11:41 am
Full Name: Gareth
Contact:

Re: Recommendations for backup storage, backup target

Post by GarethUK » 1 person likes this post

HI,

Another big thumbs up for Dell R720xd and Windows 2012R2. I'm not using deduplication.

We're using Dell R720xd fully populated with an additional tray with both raid sets in a raid 60.

To then add some spice to the mix we create a single 48TB storage space across the whole lot to create a 48TB repo.

We obviously replicate to another date center and I've seen throughput across the network as high as 6-7Gbps.

We previously used a 120TB ZFS appliance but that was truly awful with Veeam.

Regards,

Gareth
gkennedy
Influencer
Posts: 10
Liked: 10 times
Joined: Aug 01, 2013 3:48 am
Full Name: Gavin Kennedy
Contact:

Re: Recommendations for backup storage, backup target

Post by gkennedy »

@GarethUK - do you mind if I PM you regarding your setup. We have the same hardware and looking at doing a similar thing. Would you mind?

Cheers,
Gav
Post Reply

Who is online

Users browsing this forum: bct44, Google [Bot], Gostev, lando_uk, tm67 and 105 guests