Deduplicating storage device

[andreas2012]

Hi,

We have an environment with 12 ESXi Dell Poweredge hosts running vSAN.
There is around 250 VMs and around 200 physical servers that is backed up with Veeam.

The environment is configured with a primary backup target that is a Windows server with just a bunch of disk, running Windows Server 2016 with deduplication. This backup target is for fast restore possibility and primarily contains only latest backup
Second we have a Fujitsu CS800 deduplication NAS appliance device as a backup target for Backup-Copy jobs, here we save several versions. This box is also configured to replicate its data over to another location with the software that is running inside the CS800.
Third we have a tape station for offline storage.

This configuration is working ok, but now we need to replace the CS800 box with a new box, and we are looking for a high end deduplication box. Fujitsu is not an option here, so is there anyone out there that could recommend some boxes from the big vendors. It should be for example either Lenovo, Dell, Quantum

Or if there is any other recommendations please comment.

Thanks for reply

/Andreas

[HannesK]

Hello,
my personal opinion is that deduplication boxes never pay of against REFS / XFS. Even if you would require the double amount of disk space (which is very unlikely because the deduplication factor come from unnecessary full backups), it would still be faster and cheaper. I'm talking about faster restore. For backup (copy) most deduplication boxes are okay.

For the box itself: I would go for a box with Veeam integration. That eliminates Lenovo as far as I know. Dell (DataDomain) is working good for backup as far as I hear from customer. For Quantum there is something special with V10 and XFS integration. If I remember correctly you need a RAM upgrade (I recommend to talk to Quantum about details).

Best regards,
Hannes

[andreas2012]

Hi,

Thanks for reply.
So what you are saying is it might be better for example to have 2 physical servers with a lot of local disks and these should then be running Windows Server 2019 with ReFS ?

Could you also please explain a bit more when it comes to "Even if you would require the double amount of disk space (which is very unlikely because the deduplication factor come from unnecessary full backups)"

Thanks for reply.

/Andreas

[HannesK]

Hello,
I don't know how much storage you need. 2 RU servers can have 24 LFF disks, 4 RU servers about 60 LFF disks. That means hundreds of TB are possible in one server.

What I try to say is that I do not care about disk space usage. I care about costs.

Best regards,
Hannes

[andreas2012]

Hi,

The data the NAS box is giving me is the following

Data Size Before Reduction
151.07 TB

Data Size After Reduction
32.68 TB

Total Reduction Ratio
4.62 : 1

Deduplication Ratio
3.13

Compression Ratio
1.48


So I guess I need to figure out how many disks and how large disks i need to support this +grow..... agree ?

Also how is it to compare ReFS deduplication with a NAS Appliance Deduplication box ?

Thanks for helping, this is not my strongest field

/Regards
Andreas

[HannesK]

well, the comparison is not easy. Because you deactivated Veeam compression / dedupe before storing to the CS800. So the "Data Size Before Reduction" is probably 2x higher than reality.

I would go with the restore point simulator default settings for compression (50% or 2x), tick the REFS checkbox and set the daily change rate to 3-5%. Then you add the growth (I used 10% and 5 years). You still need to fill out the amount of source data. I guessed about 60TB

https://rps.dewin.me/?m=3&s=61440&r=14& ... 4,12,0,3&e

[andreas2012]

Hi,

Ok If I then understand you correct I should by RAW disk capacity around 60TB, right ?
I guess there are many things that comes into play here, but its a start I guess...
Also do you have any comments regarding type of disks, number of disks, size and raid type ?

/Andreas

[HannesK]

Hello Andreas,

Ok If I then understand you correct I should by RAW disk capacity around 60TB, right ?

I guessed that your source data is 60TB. you need to adjust all input values to your real values.

general rules apply... at least double parity RAID...the faster the disks, the faster the backup / restore. https://www.veeambp.com/repository_serv ... das-or-san

With block cloning, I don't believe that many customers still go for 10k disks.

Best regards,
Hannes

[andreas2012]

Hi,

Ok I see, I will look into it.
When it comes to the disks I think we will go for HD SAS 12G 14TB 7.2K 512e HOT PL 3.5.
These are very big disks, so I am a title worry about rebuild time, but guess that depends on the workload on the server.
And we will also spec this server with 1xIntel Xeon Silver 4210R 10C 2.40 GHz and 6x16GB (1x16GB) 1Rx4 DDR4-2933 R ECC
And 10G fibre, so hopefully this will be good as it gets With windows server 2019 deduplication and refs.

/Andreas

[HannesK]

yep, rebuild times would also worry me

deduplication and refs.

please make it an "or" not an "and" . My recommendation is REFS. No windows deduplication.

Yes, you can only combine it with a reg key, but it has only experimental support. feedback is not good so far about speed.

[andreas2012]

Hi,

Well If I don`t use deduplication then I have to have a lot of disk since we have

Data Size Before Reduction
151.07 TB

Data Size After Reduction
32.68 TB

Or would I enable the following inside Veeam and format the repository with Refs
Data reducation>Enable inline data deduplication
Compression level = none
Storage Optimization = LAN target

/Andreas

[NightBird]

A real example of ReFS fast block cloning efficiency on backup copy.

Take a look, backup copy GFS 12 monthly.

Space usage on the repository


Real space size



Forget deduplicating appliances.

For example take a DELL 740xd2 or a HPE Apollo 4200 with a bunch of disks
imagine the capacity of 24 LFF of 14TB in raid60 + 1 LFF for hotspare good choice :p it must be around 270TB usable space repository

[andreas2012]

Hi,

Thanks for screenshots. I guess I just have to try and test to see if I either should use deduplication or if I should use fast block cloning.
I have never looked into fast block cloning before, but if I go for this is there some best practice to these jobs, I have several different jobs like example below.

one job that do a full backup every day of 10 machines
one job that do a forever incremental every day of 10 machines
one job that do incremental backups and one day a week we do a synthetic full backup

/Andreas

[HannesK]

Hello,

if I should use fast block cloning.

yes, block cloning is the way to go. sorry, I forgot to mention it. Normally I always link the blog article

I understood that the initial question was about backup copy jobs.

one job that do a full backup every day of 10 machines

that's not relevant for the backup copy job. the backup copy job is incremental forever per default.

Actually I do not understand your job design, but in general: REFS block cloning only saves disk space for synthetic fulls (which are default for backup copy jobs)

Best regards,
Hannes

[andreas2012]

Hi,

Ok I understand, yes originally it was for Backup Copy Job, but I might buy another server to use as a primary backup repository also for regular backup jobs.
As you say "REFS block cloning only saves disk space for synthetic fulls" so if I use any other backup job methods I guess I should use deduplicaton instead ?

/Andreas

[HannesK]

yes, if you use "active full", you can go with deduplication.

for incremental forever, I would always go with block cloning because it's sooooo much faster

[andreas2012]

Ok, thanks for every answer