Discussions related to using object storage as a backup target.
Post Reply
ctg49
Enthusiast
Posts: 65
Liked: 45 times
Joined: Feb 14, 2018 1:47 pm
Full Name: Chris Garlington
Contact:

SOFS - Evacuate backups from Object Storage Repository

Post by ctg49 » 2 people like this post

Curious about a use case we have. We're testing two different providers for potentially leveraging an S3 object storage repository in a SOFS setup (which works wonderfully so far, btw), and I'd like to utilize the exact same backup dataset I tested with one, with the other. I tried evacuating the S3 repository only to discover there's no button there, grayed out or otherwise. I was able to set it to maintenance mode, but that didn't change anything.

Is there something special that needs to be done to enable evacuation of an object storage repository within a SOFS configuration?

Thanks!
Gostev
Chief Product Officer
Posts: 31812
Liked: 7302 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: SOFS - Evacuate backups from External Repository

Post by Gostev »

Hi Chris, currently there's no Evacuate functionality for object storage extents as for regular extents. We saved some time postponing the implementation of this functionality, because we did not believe there is a big use case. This is because usually, due to download charges it is cheaper to just keep the backups there until they are no longer needed, and then delete them.

However, if you go to backup properties, you will see there's an option to download backup files back to Performance Tier (right-click on the required files).

Also, if you're evaluating some major and well known providers, please do share your conclusion with us.

Thanks!
ctg49
Enthusiast
Posts: 65
Liked: 45 times
Joined: Feb 14, 2018 1:47 pm
Full Name: Chris Garlington
Contact:

Re: SOFS - Evacuate backups from External Repository

Post by ctg49 » 14 people like this post

Perfect; found the option to copy back to performance tier, exactly what I was looking for.

This is definitely a very limited use case scenario, under normal circumstances I'd never transfer data back due to incurred costs. This is really just something we can make use of for the time being for testing purposes due to free trials :) I'd also like to very specifically thank the VEEAM team for making the cloud object storage tiering system basically idiot-proof. VEEAM is consistently one of the few products I use daily that actually works the way I'd expect it to, and I appreciate that.

We've actually tested Wasabi already, and gotten a very positive experience from them. My leadership got alerted to them by others on campus, specifically regarding their partnership with VEEAM. Very attractive pricing. Our performance was as follows:

Upload speeds - Test job processed 140GB at 151MB/s, and uploaded 78.7GB in approximately 17 minutes (around 78MB/s).
Restoral speeds, file level recovery - Downloaded mix of small files… ~500 files - 178MB total, approximately 3-5MB/s.
Downloaded single large file, 1.3GB total, approximately 50MB/s.

Restoral speeds, full VM - Restored 40GB VM in 3m30s, at approximately 160MB/s transfer speeds.
Local restoral of same VM took 4m30s, at speeds varying between 110-150MB/s.

Note that restoral from Wasabi was actually faster than local storage... local storage is a deduped NTFS volume on a RAID5 array.

Hoping to be testing S3 later today once I get an account via our on-campus cloud provider system.
Gostev
Chief Product Officer
Posts: 31812
Liked: 7302 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: SOFS - Evacuate backups from External Repository

Post by Gostev »

Chris, thank you so much for sharing real-world numbers! Yes, I too heard good things about Wasabi pricing... and I love the fact that they don't charge for API calls (this part of the object storage bill comes as a nasty surprise for many).
ctg49
Enthusiast
Posts: 65
Liked: 45 times
Joined: Feb 14, 2018 1:47 pm
Full Name: Chris Garlington
Contact:

Re: SOFS - Evacuate backups from Object Storage Repository

Post by ctg49 » 9 people like this post

Finally got a chance to test our S3 bucket (US East - Ohio), results for that are as follows:

Upload speeds - Test job processed 140GB at 176MB/s, and uploaded 78.7 in approximately 15m35s (around 84MB/s).

Restoral speeds, file level recovery - Downloaded mix of small files… ~500 files, - 178MB total, approximately 3-5MB/s (done in ~30s).
Downloaded single large file, 1.3GB total, approximately 47MB/s.

Restoral speeds, full VM - Restored 40GB VM in 4m25s, at approximately 115MB/s transfer speeds.
Refrained from local restoral as results would be same as above.


End result, roughly comparable speeds, if not a little faster on Wasabi.
AlexHeylin
Veteran
Posts: 563
Liked: 173 times
Joined: Nov 15, 2019 4:09 pm
Full Name: Alex Heylin
Contact:

Re: SOFS - Evacuate backups from Object Storage Repository

Post by AlexHeylin »

Hi @Gostev

How does an SP do "go to backup properties, you will see there's an option to download backup files back to Performance Tier (right-click on the required files)." for tenant backups (stored on SP SOBR with object extent)? We can't see the backup properties etc (I've raised this as causing challenges before).

Or, is there an alternative way to effectively evacuate an object extent that's part of SP SOBR, and is full of tenant backups some of which might live for many months?

We're looking to implement a new Wasabi immutable extent, and "move" all our S3 data from the old non-immutable Wasabi extent. I'm quite happy to seal the old extent and leave it to live for 90 days (minimum payment period), but after that all the data needs to be gone from it. Surely just sealing it will leave all the "full backup" blocks sat in the old extent, and won't create them in the new one because they exist in the old one?

Thanks
Gostev
Chief Product Officer
Posts: 31812
Liked: 7302 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: SOFS - Evacuate backups from Object Storage Repository

Post by Gostev »

@veremin can you please explain what is the approach for VCC service providers?
veremin
Product Manager
Posts: 20412
Liked: 2300 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: SOFS - Evacuate backups from Object Storage Repository

Post by veremin »

How does an SP do "go to backup properties, you will see there's an option to download backup files back to Performance Tier (right-click on the required files)." for tenant backups (stored on SP SOBR with object extent)?
You can right-click on a tenant account and select Download. This way tenant data will be downloaded from Capacity Tier to Performance Tier. Thanks!
AlexHeylin
Veteran
Posts: 563
Liked: 173 times
Joined: Nov 15, 2019 4:09 pm
Full Name: Alex Heylin
Contact:

Re: SOFS - Evacuate backups from Object Storage Repository

Post by AlexHeylin »

Wonderful - thanks! Any chance of a docs update to say this?
Gostev
Chief Product Officer
Posts: 31812
Liked: 7302 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: SOFS - Evacuate backups from Object Storage Repository

Post by Gostev »

But he linked the existing doc? :D
AlexHeylin
Veteran
Posts: 563
Liked: 173 times
Joined: Nov 15, 2019 4:09 pm
Full Name: Alex Heylin
Contact:

Re: SOFS - Evacuate backups from Object Storage Repository

Post by AlexHeylin » 1 person likes this post

Oh yeah, sorry - I haz the dumb :roll:. Unfortunately support don't seem to know about that doc, and it didn't come up in searches on how to do this. I'll RTFM in future :wink:
Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 9 guests