Discussions related to using object storage as a backup target.
Post Reply
PedroF
Lurker
Posts: 1
Liked: never
Joined: May 28, 2020 9:57 am
Full Name: Pedro Ferreira
Contact:

S3-Compatible Storage Sizing

Post by PedroF »

Hi,

I am designing a solution where we would like to replace the use of a tape library with an Obejct storage with locking implementation.

We wanted to use Dell EMC ECS as on premise S3 target in order to take advantage of the immutability feature that S3 offers but they still don't support it, so we are looking at Cloudian because it does what we want.

My question is the difficulty in correctly dimensioning the space I will need for the solution since Veeam does source side dedupe.

I usually use the link below to do my calculations but as Veeam dedup the data it sends, can i look at the destination as a "forever increment"?
http://rps.dewin.me/

What is the best approach to correctly size the S3-compatible repository?

Thanks in advance.

Pedro
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: S3-Compatible Storage Sizing

Post by veremin » 1 person likes this post

Just enable ReFS checkbox - object storage integration works in similar fashion. Thanks!
dariusz.tyka
Enthusiast
Posts: 54
Liked: 4 times
Joined: Jan 21, 2019 1:38 pm
Full Name: Dariusz Tyka
Contact:

Re: S3-Compatible Storage Sizing

Post by dariusz.tyka »

And what about if we enable GFS for backup job? I suppose we need to switch to backup copy job and specify required weekly, monthly & yearly restore points? And still have RFS/XFS check box enabled?

My second question is if we should go for VCC option or object storage as an tape replacement - from required disk space consumption? It will be secondary target for our backups - so we will have copy/move options enabled on object storage or use BCJ to offload backups to VCC provider.
I got quote from one VCC provider and can have the same price for both object storage and VCC repository. If I'm correct for object storage with GFS backup job option we still can use RFS/XFS to calculate required disk space. For VCC repository it is not the same as we use BCJ and for those all weekly/monthly/yearly restore points are full ones.

Dariusz
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: S3-Compatible Storage Sizing

Post by veremin »

Let's summon the allmighty creator himself - @tdewin. While we're guessing, he might know the exact answers.
dariusz.tyka
Enthusiast
Posts: 54
Liked: 4 times
Joined: Jan 21, 2019 1:38 pm
Full Name: Dariusz Tyka
Contact:

Re: S3-Compatible Storage Sizing

Post by dariusz.tyka »

Hi Vladimir,

can you only confirm that data stored on object storage (capacity tier) is stored in forever forward incremental format- even if we select to keep certain backups longer for archival purposes?

Dariusz
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: S3-Compatible Storage Sizing

Post by Gostev »

Confirming.
dariusz.tyka
Enthusiast
Posts: 54
Liked: 4 times
Joined: Jan 21, 2019 1:38 pm
Full Name: Dariusz Tyka
Contact:

Re: S3-Compatible Storage Sizing

Post by dariusz.tyka »

Thanks Gostev. Then if we have the same price for object storage and Veeam cloud connect repository better choice for us will be object storage. As less data will be stored there so we should pay less. But it will require some additional testing.
tdewin
Veeam Software
Posts: 1775
Liked: 646 times
Joined: Mar 02, 2012 1:40 pm
Full Name: Timothy Dewin
Contact:

Re: S3-Compatible Storage Sizing

Post by tdewin » 1 person likes this post

The way ReFS and object storage works is indeed very similar. Both link to the same block (with the same size) in the previous "files". So yes, just enabling ReFS/XFS should give you an idea of how much object storage is used.
dariusz.tyka
Enthusiast
Posts: 54
Liked: 4 times
Joined: Jan 21, 2019 1:38 pm
Full Name: Dariusz Tyka
Contact:

Re: S3-Compatible Storage Sizing

Post by dariusz.tyka »

One additional question to my previous one - REFS can also be present on VCC repositories and then I assume both object storage and VCC repository will require same available space to hold our backups. Is there any option to check if VCC repository is REFS one?
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: S3-Compatible Storage Sizing

Post by Gostev »

No, tenant knows nothing about service provider's infrastructure. So, you will need to check with your service provider on this.

Even more importantly, you will want to ask how do they charge. Some may charge for total size of backup files w/out considering any potential space savings, other for actual used space only. I've seen both approaches, as even before ReFS/XFS we had service providers using deduplication storage appliances.
lrhazi
Enthusiast
Posts: 94
Liked: 3 times
Joined: Jan 03, 2017 8:15 pm
Full Name: Mohamed Lrhazi
Contact:

[MERGED] How to calculate S3 size estimate

Post by lrhazi »

Hello,
We would like to use S3 to replace our tape backups... and am wondering how to estimate the S3 storage size.
I am thinking about enabling GFS on the backup jobs. and I started with a test backup job, with just one VM in it. GFS has only recently been turned on, on the job.

Here is what the backups look like currently.
https://imgur.com/EccyPCg

The VM size is about 80GB. Retention policy is set to 7 restore points. GFS is set to 4 weekly, 12 monthly and 7 yearly.
The Capacity Tier is set to copy backups immediately and Move files older than 14 days.

I do not recall when I enabled GFS on this job, maybe two weeks ago, or less.

If I check the bucket in S3, it says total size is just 74GB, even though I seem to have two fullbacks already there. I assume this is because of deduplication?

What would you say a good estimate is for the amount of S3 storage this job will consume after 1 year? 2 years...? assuming the rate of change of the VM does not change.

Thank you so much.
Mohamed.
HannesK
Product Manager
Posts: 14287
Liked: 2877 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: S3-Compatible Storage Sizing

Post by HannesK »

Hello,
I merged your question to the existing one and also added an answer to the FAQ post338749.html#p338749

Best regards,
Hannes
Post Reply

Who is online

Users browsing this forum: Wallachuck and 7 guests