Comprehensive data protection for all workloads
Post Reply
tom.starren
Enthusiast
Posts: 26
Liked: 1 time
Joined: Dec 03, 2014 12:31 pm
Full Name: Tom Starren
Contact:

How many repository volumes (Windows 2012 R2 Dedup)

Post by tom.starren »

Hi All,

Currently we have got 2 physical HP servers with each a storage unit attached with 12x4TB (RAID5 volume of 40TB) in each of our datacenter. The storage units have the new backups, archive backups of the datacenter and archive backups of the remote site all on the same big volume (each one has his own repository on the same volume). On one of the sites we also have some EndPoint protection repository, again same volume.

Because of a grow and the need for a longer archive time on of the volumes is filling up to 75% and we have decided to buy an additional 2 storage units (with again 12x 4TB disks) which are connected to the two servers (each server has now got 2x12x4TB disks connected to it as DAS). I know the dedup process is single threaded but can do multiple volumes at once, I want to split the storage up in several volumes/repositories.

We know have B&R v8 with latest patch installed and are using VMware vShpere 6. I've created backup jobs for our VMs based upon OS/role and large servers (SQL, Exchange, fileserver) have there own jobs (keeping the backup files as small as possible (some of the servers are around 1,5TB). Each jobs has then a local and remote copy job attached for archiving (remote job runs through WAN accelerator).

Does someone have a guideline/tip how I can do this a smart way? I've someone needs more information, please let me know.

Kind Regards,
Tom
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: How many repository volumes (Windows 2012 R2 Dedup)

Post by Vitaliy S. »

Hi Tom,

Just to make sure we are on the same page - would you like to know how to setup your repositories in a better way or you're asking for something else? If you want to keep your backup files as small as possible you can try to leverage per-VM backup chains feature (available in v9). In addition to scale-out backup repository that would allow you manage your backup repositories much easier.

Let me know if I understood your question correctly.
tom.starren
Enthusiast
Posts: 26
Liked: 1 time
Joined: Dec 03, 2014 12:31 pm
Full Name: Tom Starren
Contact:

Re: How many repository volumes (Windows 2012 R2 Dedup)

Post by tom.starren »

Hi Vitaliy,

I'm searching for a good way to setup the repositories. This could include changing the RAID level.

We are not planning to upgrade to a new version of VEEAM, but if there is a feature in there which could really help, we can think about it. We have some large VMs (those are already in a separate job), the small servers are combined.
skrause
Veteran
Posts: 487
Liked: 105 times
Joined: Dec 08, 2014 2:58 pm
Full Name: Steve Krause
Contact:

Re: How many repository volumes (Windows 2012 R2 Dedup)

Post by skrause »

You really should not use RAID5 for disks over 1TB. The chance of a second disk failure during rebuild are huge with larger disks. If you have to use a parity RAID with disks that large you should go RAID6. Veeam recommends using RAID 10 for the best performance but you will lose quite a lot of capacity to do that.

Make sure you use the /L flag for formatting the volumes if you are planning on using Windows Deduplication.
Steve Krause
Veeam Certified Architect
tom.starren
Enthusiast
Posts: 26
Liked: 1 time
Joined: Dec 03, 2014 12:31 pm
Full Name: Tom Starren
Contact:

Re: How many repository volumes (Windows 2012 R2 Dedup)

Post by tom.starren »

Currently the volume is formatted with the /L option, I've learned the hard way that it was needed. RAID10 isn't an option, we just expanded the capacity and then I would half it again, RAID6 is manageable.

Do you have any tips on how many and how large you would made the volumes?
skrause
Veteran
Posts: 487
Liked: 105 times
Joined: Dec 08, 2014 2:58 pm
Full Name: Steve Krause
Contact:

Re: How many repository volumes (Windows 2012 R2 Dedup)

Post by skrause »

It is really a personal preference thing. I run 5-7TB per repository volume in my setup (but my disk is on a lower-end SAN and not direct attached) and it works ok but you will get better overall space utilization (and Windows DeDupe ratios) the larger your volumes are if you are really trying to best leverage Windows deduplication. The biggest thing is to figure out what your RAID controller/disks can handle in terms of concurrent tasks I/O. Since they are all hitting the same RAID array/controller on the backend, I would probably go with one volume per RAID set on each server as that will spread you out across the most spindles and give you the best performance possible for the RAID level you use.

Scale-out repositories and per-VM chains in v9 will make using smaller volumes for repositories more manageable since you can point jobs at the scale-out repository and have multiple smaller volumes as part of it. If you are running Windows Deduplication it really won't matter too much if you use per-VM chains or large jobs with multiple servers as the deduplication will probably be fairly similar.
Steve Krause
Veeam Certified Architect
tom.starren
Enthusiast
Posts: 26
Liked: 1 time
Joined: Dec 03, 2014 12:31 pm
Full Name: Tom Starren
Contact:

Re: How many repository volumes (Windows 2012 R2 Dedup)

Post by tom.starren »

We have a HP P441 controller with two D3600 enclosures attached to it with a total of 24x 4TB disks (and this two times in 2 different locations). Current deduplication rate is 82% on the 40TB volume. Splitting the volumes would decrease this number, but the dedup process will have more resources to dedup the files. The per-VM chains will help keeping backup files small and this helps in the dedup process also.
nmdange
Veteran
Posts: 527
Liked: 142 times
Joined: Aug 20, 2015 9:30 pm
Contact:

Re: How many repository volumes (Windows 2012 R2 Dedup)

Post by nmdange »

My preference is the largest volume you can get while staying under 64TB (chkdsk limit). With that many drives, I'd probably say use RAID 5+0 or RAID 6, and ideally at least 1 hotspare as well.
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: How many repository volumes (Windows 2012 R2 Dedup)

Post by Gostev »

64TB is also a VSS volume size limit, and VSS is leveraged by Windows dedupe.
Delo123
Veteran
Posts: 361
Liked: 109 times
Joined: Dec 28, 2012 5:20 pm
Full Name: Guido Meijers
Contact:

Re: How many repository volumes (Windows 2012 R2 Dedup)

Post by Delo123 »

Hi Tom,

i have posted a bit of our setup some time ago:

http://forums.veeam.com/veeam-backup-re ... 99-15.html

Maybe it can give you some ideas... Doing a single raid 5 on 12 or even 24 disks sounds dangerous. At least try 2 or 4 raid 5's and stripe them with storage spaces.... Anyway, let me know if you want to chat a bit :) ps. We are currently updating our 36bay supermicros with additional hgst 60bay 6tb loaded jbods for extra space :)

Ps. Our 108tb storage pool currenty holds just over 500tb of data :) costs for the supermicro was under 10k so quite nice if you ask me :)
Post Reply

Who is online

Users browsing this forum: Bing [Bot], Semrush [Bot] and 256 guests