Discussions related to using object storage as a backup target.
Post Reply
FredrikJ
Novice
Posts: 3
Liked: never
Joined: Apr 16, 2019 8:57 am
Full Name: Fredrik
Contact:

One-off backups to S3

Post by FredrikJ »

Hi,
We have a routine to keep decommissioned VMs in cold storage for a specific amount of time depending on the customers wishes. To instantly free up space from our main backup storage we would like to do a one-off backup of these VMs and offload to a S3 provider.

I have tried both VeeamZIP and backup export to a new scaleout that has a local extent for staging the backups on as well as a S3 object storage. However we are finding that the SOBR tiering job is not offloading any backups to the S3 storage despite being set to move all backups older than 0 days.

Is this simply not a supported way of offloading backups?
Is it something that will be supported in a future version?
Are there any supported workarounds to get the functionality we seek?

Regards,
Fredrik
wishr
Veteran
Posts: 3077
Liked: 453 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: One-off backups to S3

Post by wishr »

Hi Frederik,

The offloading process should work. Please find a few questions below.

Have you had a chance to check that article and confirm you are trying to offload inactive chains?
What do you mean saying "However we are finding that the SOBR tiering job is not offloading any backups to the S3 storage despite being set to move all backups older than 0 days."?
Have you tried to run the offload job manually?

Thanks,
Fedor
FredrikJ
Novice
Posts: 3
Liked: never
Joined: Apr 16, 2019 8:57 am
Full Name: Fredrik
Contact:

Re: One-off backups to S3

Post by FredrikJ »

Hi,

That article only mentions scheduled backup jobs and backup copy jobs. I dont see any mention of VeeamZIP or backup export.
When running the offload job, either manually or on schedule, it returns "2019-04-16 13:08:25 :: 0 backup files will be moved to the capacity tier"

Regards,
Fredrik
wishr
Veteran
Posts: 3077
Liked: 453 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: One-off backups to S3

Post by wishr »

Both VeeamZip and exported backups cannot be offloaded to S3, at least currently. Is there any reason to use those instead of scheduled backups?

Thanks in advance
FredrikJ
Novice
Posts: 3
Liked: never
Joined: Apr 16, 2019 8:57 am
Full Name: Fredrik
Contact:

Re: One-off backups to S3

Post by FredrikJ »

Since all the VMs are in cold storage/shut down there is not really any reason to have backup jobs running for them. Reason we want to offload to S3 is to save local storage space.

Regards,
Fredrik
Gostev
Chief Product Officer
Posts: 31459
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: One-off backups to S3

Post by Gostev » 1 person likes this post

Fredrik, your feature request certainly makes sense. However, one-off backups to S3 would require Veeam to support backing up directly to S3. I am sure this functionality will come with time, once we become reasonably confident about leveraging object storage overall. Thanks!
backupquestions
Expert
Posts: 186
Liked: 21 times
Joined: Mar 13, 2019 2:30 pm
Full Name: Alabaster McJenkins
Contact:

[MERGED] Veeamzip and object storage?

Post by backupquestions »

Is there a way to utilize the age out to Object storage with a veeamzip vbk?

For example I receive legal hold request and take a veeamzip one off full backup of a vm... I need to now hold this for centuries (lol)... I can put it on a local nas sure, but would be nice to have it go out to Wasabi via placing the vbk on the local SOBR... It's a sealed chain as it's just a one off vbk, but the thing is, there would be no retention policy attached to it so there is no way for Veeam to ever delete it... Though I could probably just do it from bucket...

Having said all of that, I think veeam doesn't transmit vbk itself to wasabi and instead transmits small blocks of data which might not live on the object storage as actual vbk/vib?

Anyway.... Possible or no? If not, is the general recommendation just to throw veeamzip one off stuff onto usb drive or nas? I don't want to waste my main veeam repo space with stuff that I would likely never even need to restore from.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: One-off backups to S3

Post by veremin »

Possible or no?
Not possible
If not, is the general recommendation just to throw veeamzip one off stuff onto usb drive or nas?
Correct, removable media is the best way to store such backups.

Thanks!
CantThinkOfAName
Service Provider
Posts: 3
Liked: never
Joined: Jul 18, 2019 12:09 pm
Contact:

[MERGED] Moving One-Off Backups to S3

Post by CantThinkOfAName »

Hi,

yes, there already was a post about this topic but I think our use case is different enough to warrant a new one.

We need to archive some no longer used machines to S3. Doesn't need to directly go to S3, it's fine if we could do it the usual way (backup to Scale-Out Repo and later the Offloading Job moves it to S3).

As I understand the concept of Backup Chain Legitimacy, a backup can never become inactive if no newer full backup is created, yes?
Then, is there any way we could take one initial full backup and somehow mark it as inactive so the Offloading Job can move it? Because there will never be another backup following it, these machines are destined to be deleted afterwards.

Regards
Gostev
Chief Product Officer
Posts: 31459
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Moving One-Off Backups to S3

Post by Gostev »

Hello!

This is not a supported use case currently... we're looking to deliver this by enabling offloading of VeeamZIP backups to object storage in future releases.

Thanks!
DE&C
Service Provider
Posts: 29
Liked: 25 times
Joined: Aug 07, 2017 11:51 am
Full Name: William
Location: Zurich, Switzerland
Contact:

Re: One-off backups to S3

Post by DE&C »

Hi Anton

A lot of my customers would like to have VeeamZip on a SOBR that offloads the data (from VeeamZip) to the capacity tier (s3-like storage). Maybe with 3 options
- a) offloaded after the normal period configured on the SOBR (example of my customer x: 14 days)
- B) the option on the SOBR to configure for VeeamZip a dedicated period (3 days ->rest stays for 14 days as configured)
- C) push a button that offloads earlier all VeeamZip data to the capacity tier

Looking forward to the version when VeeamZip is offloaded to S3 :)

Best regards
Doron
AuGL
Enthusiast
Posts: 51
Liked: 3 times
Joined: May 07, 2019 12:22 am
Full Name: Glenn
Contact:

Re: One-off backups to S3

Post by AuGL »

I also have a requirement to use VeeamZIP to archive servers that are being decommissioned, and store them offsite in AWS S3.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: One-off backups to S3

Post by veremin »

Thanks, your feature request has been noted.
dmoy_anchoragecap
Lurker
Posts: 1
Liked: never
Joined: Jan 05, 2021 6:18 pm
Full Name: David Moy
Contact:

Re: One-off backups to S3

Post by dmoy_anchoragecap »

I would also like to have this feature added
dariusz.tyka
Enthusiast
Posts: 54
Liked: 4 times
Joined: Jan 21, 2019 1:38 pm
Full Name: Dariusz Tyka
Contact:

Re: One-off backups to S3

Post by dariusz.tyka »

I would also like to see such feature within Veeam.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: One-off backups to S3

Post by veremin » 1 person likes this post

The upcoming release of Veeam Backup and Replication (v11) includes this feature. Thanks!
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: One-off backups to S3

Post by veremin » 1 person likes this post

v11 has been released and now full backups created with VeeamZIP and Export Backup features are processed by the Capacity Tier policies.
dariusz.tyka
Enthusiast
Posts: 54
Liked: 4 times
Joined: Jan 21, 2019 1:38 pm
Full Name: Dariusz Tyka
Contact:

Re: One-off backups to S3

Post by dariusz.tyka »

Hi Veremin,

it also includes the movement to archive tier from capacity tier?
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: One-off backups to S3

Post by veremin »

dariusz.tyka
Enthusiast
Posts: 54
Liked: 4 times
Joined: Jan 21, 2019 1:38 pm
Full Name: Dariusz Tyka
Contact:

Re: One-off backups to S3

Post by dariusz.tyka »

Thanks. Do you know if there is any document regarding the minimal permission for Glacier storage? Like is it for S3 bucket. Or maybe those are same?
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: One-off backups to S3

Post by veremin »

We have Amazon S3 Glacier permissions documented here, it should be exactly what you're looking for. And don't forget about setting the ones mentioned in Amazon S3 Object Storage section. Thanks!
dariusz.tyka
Enthusiast
Posts: 54
Liked: 4 times
Joined: Jan 21, 2019 1:38 pm
Full Name: Dariusz Tyka
Contact:

Re: One-off backups to S3

Post by dariusz.tyka » 1 person likes this post

Is it possible to somehow narrow down those permissions required for Glacier like it was for S3 (single bucket name)?
As now those permissions are quite powerful and can affect any S3 or EC2 resource within VPC.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: One-off backups to S3

Post by veremin »

I haven't checked it personally, but you can try to limit required S3 permissions by setting bucket name as a resource parameter in JSON file (the entity the permissions are applied to) and see whether it helps - some inspiration can be taken from this KB article.

Don't think the same approach will work for EC2 permissions, though, because there is no static EC2 entity that the permissions can be assigned to - proxy appliance gets created when offload starts and gets removed as soon as it stops.

Thanks!
Post Reply

Who is online

Users browsing this forum: No registered users and 7 guests