Comprehensive data protection for all workloads
Post Reply
davidb1234
Expert
Posts: 162
Liked: 15 times
Joined: Nov 15, 2011 8:47 pm
Full Name: David Borden
Contact:

RAID6 or RAID10 for backup storage on local disk array?

Post by davidb1234 » 1 person likes this post

Veeam backup server is a 12 core monster with 12GB of RAM connected to an MSA60 with 12 x 2TB disks.

We have 2 ESXi 5 hosts connected 8GB FC SAN multipathed. The Veeam server is directly connected to the SAN by 8GB FC.

We have total of 4TB of VMs to back up and are planning on the Reverse Incremental model where it leaves us with a full intact VM backup every night.

If I go RAID10 I'll have about 11TB of Usable space for backups.

If I go RAID6 I'll have about 19TB of Usable space for backups.

Which way did you go and why? Any recommendations? Any regrets? Is RAID10 overkill?

We will also be replicating to a hotsite later this year but I don't think that plays a part in this decision.
averylarry
Veteran
Posts: 264
Liked: 30 times
Joined: Mar 22, 2011 7:43 pm
Full Name: Ted
Contact:

Re: RAID6 or RAID10 for backup storage on local disk array?

Post by averylarry »

I'd go RAID6. You can lose any 2 disks and still have all your data. RAID10 -- if you lose the correct 2 disks, everything is toast.

It seems highly unlikely that your backup storage will be the bottleneck compared with production storage/compression/deduplication etc. I doubt that the likely performance decrease with RAID6 would be noticable. Besides -- the number 1 rule of thumb for storage performance is number of spindles. Since you'll have the same number of spindles either way . . .

(And for those purists out there, yes -- I'm massively simplifying.)
TimeKnight
Enthusiast
Posts: 29
Liked: never
Joined: Jan 19, 2012 3:35 pm
Full Name: Brian Weinberg
Contact:

Re: RAID6 or RAID10 for backup storage on local disk array?

Post by TimeKnight »

I have a similar Veeam Server setup (Dell R510 with a Single CPU Xeon X5650 (6 cores + HT) 2.67GHz and 12.0 GB of RAM. For storage we have 12 2.0TB SATA HDD in a RAID 6 configuration giving us about 18TB of space.)

I seem to be having issues with the Transformation process (I only do it once a week) and it takes 35+ hours.

I am starting to think that the low IOps of 7k disks will require RAID 10 for the amount of random IO that is required for the Transformation Process.
averylarry
Veteran
Posts: 264
Liked: 30 times
Joined: Mar 22, 2011 7:43 pm
Full Name: Ted
Contact:

Re: RAID6 or RAID10 for backup storage on local disk array?

Post by averylarry »

Despite what Veeam says, I think the tranformation process got messed up in v.6. I gave up on it entirely and switched to reverse incremental backups. I don't think (particularly when non-scientifically comparing to v.5) that the slow transformation in v.6 can be entirely explained by system hardware/storage etc. (I'm thinking it's a memory/pagefile problem.)
Gostev
Chief Product Officer
Posts: 31456
Liked: 6647 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: RAID6 or RAID10 for backup storage on local disk array?

Post by Gostev » 1 person likes this post

Most customers recommend RAID10 for performance considerations (especially when using transform or reversed increments).

@Ted, transformation uses exact same code as the reversed incremental backup... same function that applies incremental data into VBK, and saves replaced data to VRB. So, I am curious why reversed incremental works better for you.
averylarry
Veteran
Posts: 264
Liked: 30 times
Joined: Mar 22, 2011 7:43 pm
Full Name: Ted
Contact:

Re: RAID6 or RAID10 for backup storage on local disk array?

Post by averylarry »

I started doing transforms on the weekends. That worked fine in v.5. But, as is typical, my overall backup system/strategy is never changing. Eventually I was so happy with v.5 backups that I started doing a mid-day backup. Incremental backups kept it quick and fast. Snapshots didn't cause any problems. Weekly transforms completed in less than a day.

Fast forward many months from then. I now have v.6 instead of v.5. I now have vSphere 5 instead of vSphere 4. I now use vmfs5 instead of vmfs4. My backup datastore is a completely different/separate SAN with a single LUN and native vmfs5. I have close to double the amount of data to backup.

So really -- too many changes to necessarily blame any single thing. With v.6, tranforms started taking days and would extend into the work week. So I added another backup job in the very early morning and did transforms daily instead of weekly (since tranforms run on the 1st job that runs in any given day). That helped, but still would cause problems because that job has to wait for replication jobs to finish etc. Additionally, snapshots are now causing minor issues during the day. So I basically ended up dumping the mid-day backup and switched to reverse incremental which made the early morning backup job superfluous. Now my backup is a more typical daily job with reverse incremental for everything, and everything also replicates to my WAN.


Also -- (this is simplified) transformation reads from the .vib file and writes to .vrb and .vbk. Reversed incremental reads from the source and writes to .vrb and .vbk. That allows for 2 separate LUNS. Reversed incremental is not 1/3 read, 2/3 write random I/O to a single LUN where transformation is (again, simplified). My backup LUN is a lower tier of storage compared to my production LUNs.
davidb1234
Expert
Posts: 162
Liked: 15 times
Joined: Nov 15, 2011 8:47 pm
Full Name: David Borden
Contact:

Re: RAID6 or RAID10 for backup storage on local disk array?

Post by davidb1234 »

Obviously if we back up every night with reverse incremental we need the transform process to complete in a reasonable amount of time since it will be done every single day.

From these responses I am actually worried now. I figured a few hours maybe to transform after the incremental runs but it seems that part of it takes a LONG time. Is that really the case???

We have 4TB of total VMs to back up however the daily change rate is about 500GB. Can anyone give me a rough ballpart on what they are seeing for transform times when using reverse incremental leaving us with a full intact VM each day?

What good is reverse incremental if the VM dies in the middle of the day and the transform isn't even done running yet from the previous nights backup?

I'd love to do RAID6 but now I am kind of freaked out by these responses.
tsightler
VP, Product Management
Posts: 6009
Liked: 2842 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: RAID6 or RAID10 for backup storage on local disk array?

Post by tsightler » 1 person likes this post

I've worked with a lot of users that are happily using RAID6 for their backup targets. Be sure to choose a good stripe size that makes sense, i.e. generally larger stripe sizes (64K or greater) work best for storing Veeam backups since the blocks are typically large.

However, having such a high change rate is going to work against you. A 500GB change rate for 4TB of data is quite high compared to the typical environment that I see and means that a transform will need to move at least 2TB of data for each nights backup, all on the target storage. The ability to do this depends completely on your targets storage ability to meet the I/O requirements of moving this much data at random I/O. I'd suggest just testing it, it will only take one night of transform before you know whether it will work or not.
davidb1234
Expert
Posts: 162
Liked: 15 times
Joined: Nov 15, 2011 8:47 pm
Full Name: David Borden
Contact:

Re: RAID6 or RAID10 for backup storage on local disk array?

Post by davidb1234 »

I determined that change rate from what backup exec is currently grabbing.

258GB Exchange Databases - Full Backup each night
200GB of SQL databases and changed files on the file server(mostly SQL database .BAK files)

I am guessing that this data change rate wouldn't apply the same way to veeam. I plan to use VEEAM to do application aware processing of SQL full backups and exchange. Our backups are extremely ineffecient right now.

Does Veeam truely need to grab then entirely of the Exchange OR SQL databases or will it do block level backups of just the incremental changes each night?
Gostev
Chief Product Officer
Posts: 31456
Liked: 6647 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: RAID6 or RAID10 for backup storage on local disk array?

Post by Gostev »

Only changed blocks, of course.
averylarry
Veteran
Posts: 264
Liked: 30 times
Joined: Mar 22, 2011 7:43 pm
Full Name: Ted
Contact:

Re: RAID6 or RAID10 for backup storage on local disk array?

Post by averylarry »

reversed incremental is NOT an incremental followed by a transform. It's just a single process that modifies the .vib file while also creating the .vrb file. In my opinion/observation -- reverse incremental doesn't take as long as an incremental followed by a transform.

My Exchange server is about 800Gb of databases with 200 users hitting it pretty hard. My reversed incremental file sizes are about 11Gb (the weekend is larger when the weekly maintenance stuff runs on the Exchange databases).
Gostev
Chief Product Officer
Posts: 31456
Liked: 6647 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: RAID6 or RAID10 for backup storage on local disk array?

Post by Gostev »

I never said this is the same process, I said it is the same code. Backup files interaction is isolated to a few functions which every backup mode uses. So, I would not expect different transform modes to provide drastically different performance...
averylarry
Veteran
Posts: 264
Liked: 30 times
Joined: Mar 22, 2011 7:43 pm
Full Name: Ted
Contact:

Re: RAID6 or RAID10 for backup storage on local disk array?

Post by averylarry »

I was referring to David's previous post --

"What good is reverse incremental if the VM dies in the middle of the day and the transform isn't even done running yet from the previous nights backup?"

which implies a misunderstanding of reverse incremental as an incremental followed by a transform.
Gostev
Chief Product Officer
Posts: 31456
Liked: 6647 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: RAID6 or RAID10 for backup storage on local disk array?

Post by Gostev »

Oh, makes sense now. You are right.
albertwt
Veeam Legend
Posts: 879
Liked: 46 times
Joined: Nov 05, 2009 12:24 pm
Location: Sydney, NSW
Contact:

Re: RAID6 or RAID10 for backup storage on local disk array?

Post by albertwt »

Hi All,

So in this case with the latest edition of Veeam 9.0, does the below suggestion is correct:

If most or all of the backup job is still using Reverse Incremental, then use RAID 10
If converting to or most of the backup job is using Forward Incremental, then use RAID 6

As for the Strip / Full stripe size, choose the largest possible value that is supported by the Storage controller eg.
Strip Size: 256 KB
Full Stripe Size: 1024 KB

Can anyone confirm if that's the correct case ?
--
/* Veeam software enthusiast user & supporter ! */
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: RAID6 or RAID10 for backup storage on local disk array?

Post by dellock6 » 1 person likes this post

Hi Albert,
these rules of thumb are correct, but remember that each storage device has its own features and characteristics, like the cache size of the controller, the way blocks are spread around the disks, and so on. Worth a check with the specific vendor sometimes.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
albertwt
Veeam Legend
Posts: 879
Liked: 46 times
Joined: Nov 05, 2009 12:24 pm
Location: Sydney, NSW
Contact:

Re: RAID6 or RAID10 for backup storage on local disk array?

Post by albertwt »

Cool,

Many thanks for the clarification Lucca.
Your website is always useful and has been a great help to us all.
--
/* Veeam software enthusiast user & supporter ! */
albertwt
Veeam Legend
Posts: 879
Liked: 46 times
Joined: Nov 05, 2009 12:24 pm
Location: Sydney, NSW
Contact:

Re: RAID6 or RAID10 for backup storage on local disk array?

Post by albertwt »

Hi Luca,

Does splitting the Backup Repo into multiple smaller LUNs can improve the backup performance ?
rather than one massive big LUNs for multiple backup job ?
--
/* Veeam software enthusiast user & supporter ! */
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: RAID6 or RAID10 for backup storage on local disk array?

Post by dellock6 »

Depends on the underlying array, if all the LUNs are coming from the same raid group for example, the overall performance are always the same, just divided by the number of created volumes.
Be careful on massive luns, rebuild times of degraded raid can be painful.

Luca (with one C ;))
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
albertwt
Veeam Legend
Posts: 879
Liked: 46 times
Joined: Nov 05, 2009 12:24 pm
Location: Sydney, NSW
Contact:

[MERGED] RAID 50 vs. RAID 6 for Backup Repository ?

Post by albertwt »

Hi All,

I'm currently implementing Disk to Disk backup system within the Data Center infrastructure.

The backend of the backup server is attached using HP D2600 (12x 3 TB 7200 rpm SATA) and D2700 (25x 500 GB 7200rpm SAS) disks.

So which LUN setting is better given the below settings:

Image

Thanks.
--
/* Veeam software enthusiast user & supporter ! */
nielsengelen
Product Manager
Posts: 5618
Liked: 1177 times
Joined: Jul 15, 2013 11:09 am
Full Name: Niels Engelen
Contact:

Re: RAID6 or RAID10 for backup storage on local disk array?

Post by nielsengelen » 1 person likes this post

Looking at the info RAID50 should be better performance wise.
Personal blog: https://foonet.be
GitHub: https://github.com/nielsengelen
albertwt
Veeam Legend
Posts: 879
Liked: 46 times
Joined: Nov 05, 2009 12:24 pm
Location: Sydney, NSW
Contact:

Re: RAID6 or RAID10 for backup storage on local disk array?

Post by albertwt »

Great,

Thanks for the assistance Niels.

The reason that LUN was using RAID 6 or RAID50 was due to the number of disks is more than 12 in each RAID group.

Hence I will destroy that LUN on RAID6 and change it into RAID 50 with the largest strip size possible.
--
/* Veeam software enthusiast user & supporter ! */
Post Reply

Who is online

Users browsing this forum: Bing [Bot], Google [Bot], IvanK and 183 guests