How many repository volumes (Windows 2012 R2 Dedup)

[tom.starren]

Hi All,

Currently we have got 2 physical HP servers with each a storage unit attached with 12x4TB (RAID5 volume of 40TB) in each of our datacenter. The storage units have the new backups, archive backups of the datacenter and archive backups of the remote site all on the same big volume (each one has his own repository on the same volume). On one of the sites we also have some EndPoint protection repository, again same volume.

Because of a grow and the need for a longer archive time on of the volumes is filling up to 75% and we have decided to buy an additional 2 storage units (with again 12x 4TB disks) which are connected to the two servers (each server has now got 2x12x4TB disks connected to it as DAS). I know the dedup process is single threaded but can do multiple volumes at once, I want to split the storage up in several volumes/repositories.

We know have B&R v8 with latest patch installed and are using VMware vShpere 6. I've created backup jobs for our VMs based upon OS/role and large servers (SQL, Exchange, fileserver) have there own jobs (keeping the backup files as small as possible (some of the servers are around 1,5TB). Each jobs has then a local and remote copy job attached for archiving (remote job runs through WAN accelerator).

Does someone have a guideline/tip how I can do this a smart way? I've someone needs more information, please let me know.

Kind Regards,
Tom

[Vitaliy S.]

Hi Tom,

Just to make sure we are on the same page - would you like to know how to setup your repositories in a better way or you're asking for something else? If you want to keep your backup files as small as possible you can try to leverage per-VM backup chains feature (available in v9). In addition to scale-out backup repository that would allow you manage your backup repositories much easier.

Let me know if I understood your question correctly.

[tom.starren]

Hi Vitaliy,

I'm searching for a good way to setup the repositories. This could include changing the RAID level.

We are not planning to upgrade to a new version of VEEAM, but if there is a feature in there which could really help, we can think about it. We have some large VMs (those are already in a separate job), the small servers are combined.

[skrause]

You really should not use RAID5 for disks over 1TB. The chance of a second disk failure during rebuild are huge with larger disks. If you have to use a parity RAID with disks that large you should go RAID6. Veeam recommends using RAID 10 for the best performance but you will lose quite a lot of capacity to do that.

Make sure you use the /L flag for formatting the volumes if you are planning on using Windows Deduplication.

[tom.starren]

Currently the volume is formatted with the /L option, I've learned the hard way that it was needed. RAID10 isn't an option, we just expanded the capacity and then I would half it again, RAID6 is manageable.

Do you have any tips on how many and how large you would made the volumes?

[skrause]

It is really a personal preference thing. I run 5-7TB per repository volume in my setup (but my disk is on a lower-end SAN and not direct attached) and it works ok but you will get better overall space utilization (and Windows DeDupe ratios) the larger your volumes are if you are really trying to best leverage Windows deduplication. The biggest thing is to figure out what your RAID controller/disks can handle in terms of concurrent tasks I/O. Since they are all hitting the same RAID array/controller on the backend, I would probably go with one volume per RAID set on each server as that will spread you out across the most spindles and give you the best performance possible for the RAID level you use.

Scale-out repositories and per-VM chains in v9 will make using smaller volumes for repositories more manageable since you can point jobs at the scale-out repository and have multiple smaller volumes as part of it. If you are running Windows Deduplication it really won't matter too much if you use per-VM chains or large jobs with multiple servers as the deduplication will probably be fairly similar.

[tom.starren]

We have a HP P441 controller with two D3600 enclosures attached to it with a total of 24x 4TB disks (and this two times in 2 different locations). Current deduplication rate is 82% on the 40TB volume. Splitting the volumes would decrease this number, but the dedup process will have more resources to dedup the files. The per-VM chains will help keeping backup files small and this helps in the dedup process also.

[nmdange]

My preference is the largest volume you can get while staying under 64TB (chkdsk limit). With that many drives, I'd probably say use RAID 5+0 or RAID 6, and ideally at least 1 hotspare as well.

[Gostev]

64TB is also a VSS volume size limit, and VSS is leveraged by Windows dedupe.

[Delo123]

Hi Tom,

i have posted a bit of our setup some time ago:

http://forums.veeam.com/veeam-backup-re ... 99-15.html

Maybe it can give you some ideas... Doing a single raid 5 on 12 or even 24 disks sounds dangerous. At least try 2 or 4 raid 5's and stripe them with storage spaces.... Anyway, let me know if you want to chat a bit ps. We are currently updating our 36bay supermicros with additional hgst 60bay 6tb loaded jbods for extra space

Ps. Our 108tb storage pool currenty holds just over 500tb of data costs for the supermicro was under 10k so quite nice if you ask me