Discussions specific to the VMware vSphere hypervisor
Post Reply
LeoKurz
Veeam ProPartner
Posts: 27
Liked: 6 times
Joined: Mar 16, 2011 8:36 am
Full Name: Leonhard Kurz
Contact:

Migrate Repository to dedupe appliance

Post by LeoKurz » Sep 10, 2015 7:03 am

Hello,

I have to migrate backups from direct attached repositories to a dedup appliance (HP StroeOnce). I'd like to de-dedupe and uncompress the backups so they can be reduced by the appliance. Any idea how to do this? Copy jobs only copy the latest backup and any future backups... Powershell?

__Leo

Shestakov
Veeam Software
Posts: 6974
Liked: 714 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Migrate Repository to dedupe appliance

Post by Shestakov » Sep 10, 2015 8:19 am

Hello Leo,
Backup copy job copies will only synthetically create one restore point from the all chain files. However file copy job is able to copy all files at once.
However, you can manually copy all the files to the new repository, rescan it and re-map jobs to the files in the new repository.
Does it work for you, how many jobs do you have?

LeoKurz
Veeam ProPartner
Posts: 27
Liked: 6 times
Joined: Mar 16, 2011 8:36 am
Full Name: Leonhard Kurz
Contact:

Re: Migrate Repository to dedupe appliance

Post by LeoKurz » Sep 10, 2015 8:32 am

Thanx for your fast reply. Copy the files over will not do the job as the files are deduped and compressed an therefore the applinace will do a very poor job reducing the files any further. When you copy the VMs de-deduped and uncompressed, the appliance will significantly reduce the amount of needed space as it dedupes accross all VMs in all jobs. I know the restore speed etc. is very poor but the appliance will be used as secondary backup space.

__Leo

Shestakov
Veeam Software
Posts: 6974
Liked: 714 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Migrate Repository to dedupe appliance

Post by Shestakov » Sep 10, 2015 8:53 am

You are correct, using storage deduplication and compression with disabled inline ones is a recommended approach, but since you have all your files deduped/compressed, it looks more logical to copy them as they are and apply the storage deduplication to newly created backups and let existing ones be deleted by retention.
The way to undedupe/uncompress backup files is to restore VMs and backup them again with no inline deduplication. But I believe even if you automate the process, it will take much longer with not significant benefit.
And could you reply if we are talking about huge number of files/jobs? Thanks!

LeoKurz
Veeam ProPartner
Posts: 27
Liked: 6 times
Joined: Mar 16, 2011 8:36 am
Full Name: Leonhard Kurz
Contact:

Re: Migrate Repository to dedupe appliance

Post by LeoKurz » Sep 10, 2015 3:25 pm

It's about 15 Jobs with about 150 VMs and roughly 30TB space. The problem is, that the appliance was sized assuming we could use overall deduplication. And some of the backups have to be kept for a pretty long time, so the space consummation hurts...

BTW, when you configure a repository that the content should be de-deuplicated and you restore a backup from tape to this repository, will the backups be de-deduplicated during restore when the backups on tape are default deduplicated and compressed ?

__Leo

Shestakov
Veeam Software
Posts: 6974
Liked: 714 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Migrate Repository to dedupe appliance

Post by Shestakov » Sep 11, 2015 11:01 am

LeoKurz wrote:BTW, when you configure a repository that the content should be de-deuplicated and you restore a backup from tape to this repository, will the backups be de-deduplicated during restore when the backups on tape are default deduplicated and compressed ?
Yes, they will.
But we are talking here about 3 stages(types) of deduplication: inline backup deduplication(processed by VBR), disk-repository hardware deduplication and tape drive deduplication. You don`t need all 3 of them at once, like in the scenario you described above.
You can either enable inline backup deduplication or hardware + tape deduplication(recommended). Thanks!

agregersen
Novice
Posts: 5
Liked: never
Joined: Sep 14, 2015 8:23 am
Full Name: Anders Gregersen
Contact:

Re: Migrate Repository to dedupe appliance

Post by agregersen » Sep 14, 2015 12:45 pm

I did almost the reverse (moving from a dedup box to a plain storage box). We used it for copy jobs. We had inline dedup and no or dedup friendly compression and experienced almost a 50% reduction of the data we stored.
Our box was limited to 1gbit Ethernet and post process deduping - it just couldn't keep up. Now we run 10gbit on a iscsi volume and use Veeams own dedup and compression. We have about 126 vms and about 13TB of data.

Shestakov
Veeam Software
Posts: 6974
Liked: 714 times
Joined: May 21, 2014 11:03 am
Full Name: Nikita Shestakov
Location: Prague
Contact:

Re: Migrate Repository to dedupe appliance

Post by Shestakov » Sep 14, 2015 5:34 pm

Thanks for participation, Anders.

Basically compression/deduplication is a trade-off between fast processing(CPU resource) and backup size (Network, disk resources).

Out of curiousity, what`s the bottleneck for your jobs? Thanks!

Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 32 guests