Discussions related to using object storage as a backup target.
rganer
Novice
Posts: 8
Liked: never
Joined: Sep 16, 2014 9:06 pm
Full Name: Rodney Garner
Contact:

Testing Veeam B&R v8 and I want to archive data to Amazon s3

Post by rganer »

I'm new to Veeam backup and I am trying to understand how to move my backups from my local repository to Amazon s3 using Veeam 8 that came out today. I saw that there was a cloud edition for V7 of veeam but that doesnt seem to work with the V8 trial install/license.

I need to know all the components necessary to backup hypev vms to a iscsi storage repository then copy new backups to Amazon from the san. Amazon doesnt appear as an option unders service providers in the V8 trial I installed today.

Any help would be appreciated so I can make a purchasing decision on veeam this month.
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by Gostev »

For direct backup to Amazon, consider using Amazon Storage Gateway.

However, note that direct backup to public clouds has severe drawbacks comparing to Cloud Connect. The difference is pretty dramatic - simply copying existing backup filess to the cloud (and then back, before you can restore) vs. native Veeam repository in the cloud that supports all B&R functionality (including forever-incremental backups, WAN optimization, WAN acceleration, and most importantly - restore directly from cloud repository).
awellendorf
Novice
Posts: 5
Liked: never
Joined: Oct 31, 2012 11:38 pm
Full Name: Aaron Wellendorf
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by awellendorf »

I'm also looking to do this but to Amazon Glacier. I had used v7 Cloud Edition and was able to send backup to AWS Glacier without the AWS Storage Gateway. Now it seems that with v8 I have to use the AWS Storage Gateway to do this. AWS Storage Gateway is an extra $125/month/Gateway. Anyone else doing this on v8 yet and if so what are your opinions?
bsoosay
Enthusiast
Posts: 34
Liked: never
Joined: Oct 02, 2014 7:47 am
Full Name: Benjamin Soosay
Contact:

[MERGED] : Cloud Settings - AWS

Post by bsoosay »

I'm using Veeam Backup and Replication version 7.

With this version, can I push my backups to the cloud, specifically looking at AWS.

If this is possible, how do I do this and where do I find the configuration settings for this please.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by veremin »

You might want to consider using Amazon Storage Gateway to get seamless tape-like backup experience to Amazon S3 and Amazon Glacier. However, backing up to a Veeam Cloud Connect provider is a better choice with v8, as explained above. Thanks.
rganer
Novice
Posts: 8
Liked: never
Joined: Sep 16, 2014 9:06 pm
Full Name: Rodney Garner
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by rganer »

The list of cloud connect providers seem to be very small. Will this list expand anytime soon? Will amazon support veeam cloud connect?

Also I found some documentation that veeam is suggesting using Cloudberry enterprise edition to any one install veeam v8 and need access to amazon s3 backup buckets.

Im evaluating the software and like it but my supervisor is dead set on using Amazon s3 to archive our backups offsite. I need to make sure how arechive to Amazon will work with veeam v8 if we decide to use this backup software. Right now its a little confusing as to what is supported. V7 had cloud edtition which is similar to cloudberry but v8 has cloud connect but it doesnt seem to support the Amazon s3 service.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by veremin »

Will this list expand anytime soon?
Yes, this will be expanded with time. However, during the first month after v8 release, the listing is exclusive to the 25 launch partners.
Also I found some documentation that veeam is suggesting using Cloudberry enterprise edition to any one install veeam v8 and need access to amazon s3 backup buckets.
May I ask you to provide a link for that document?
I'm evaluating the software and like it but my supervisor is dead set on using Amazon s3 to archive our backups offsite.
Not sure whether it helps, but you can try to highlight benefits of using Cloud Connect over Cloud Edition, like ability to restore granular data without copying the whole image, etc.

Thanks.
rganer
Novice
Posts: 8
Liked: never
Joined: Sep 16, 2014 9:06 pm
Full Name: Rodney Garner
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by rganer »

v.Eremin wrote:May I ask you to provide a link for that document?
http://www.veeam.com/backup-cloud-editi ... g-faq.html
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by veremin »

Indeed, you can stick to the approach outlined in that document, and get the said license at no charge. However, I still strongly suggest to consider using Cloud Connect in conjunction with a local service provider suiting your needs, since the benefits you'll get make all the difference. Thanks!
rganer
Novice
Posts: 8
Liked: never
Joined: Sep 16, 2014 9:06 pm
Full Name: Rodney Garner
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by rganer »

Thanks for you help. I'm still testing how to get backups to Amazon. Hopefully sometime soon cloud connect will support Amazon s3 storage. The current list of service providers are a no go for my Vice President. He likes well known service providers as we are looking to move off of Seagate Evault backup to sometime more cost effective.

I am looking int Amazon AWS Storage Gateway as an option to use with Veeam v8. Aws storage gateway is looking like the better alternative for my relatively small environment.
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by Gostev »

Does recovery time matter for your VP? Make sure he realizes that with Amazon, granular recoveries directly from the cloud will not be possible (you will need hours/days and lots of $$$ to download all required backups locally first). So, I will question "cost effectiveness", unless of course he plans cloud backups that he will never have to do restores from.

More generally speaking though, Amazon is not the best choice for offsite backup, and ironically this is specifically because they are "well known" and have lots of users.

For example, a couple of years ago Amazon has lost data of some 100'000 users. Two weeks later, they have reported that they have restored data for 98% (or so) of impacted end users. Now, think more about this. This means 2000 (sic!) users lost their data completely... the fact that the provider was "well known" did not really help. Nor it will help you to avoid ending up in those thousands of users.

Besides, 98% of "lucky" ones had zero control on their data restore process. About 50'000 or so of those users had to wait over a week (sic!) for their data to become available, and could not do anything about it... you cannot really pick up the phone and call Amazon, and expect them to start recovering YOUR data first? They will instead recover everyone in some unpredictable order, because they have no insight on importance of content availability for the particular user (nor do they care). They may spend 10 days recovering family picture archives (without knowing this can wait), and you will wait for your very important backup (without them knowing you are waiting). Way too many users for any sort of personal approach to be plausible, hard to blame them for this!

With local service provider, it's a different experience - you can even be there on-site to make sure you are in that 1% of users whose data will be first to be recovered ;)

More importantly, the only purpose of backup is recovery, so think about those disaster recovery scenarios why are you looking to backup to cloud in the first place... for some reason, people never consider this. How many days it will take to download all your backups in case of entire site loss? Do some math on what will be the cost of such data recovery (especially with Glacier, but even with S3)? On the other hand, with local provider, you can hop in the car with 12-bay NAS, and be back in a few hours with all of your backups in hands. You don't get this flexibility with those huge public clouds?

So there are lots and lots of considerations behind choosing the service providers for off-site backups, and just going with "well known" ones for perceived cost effectiveness has lots of implications around the very reason of doing off-site backups in a first place: recovering from entire site disasters.
Bobin_NZ
Influencer
Posts: 12
Liked: never
Joined: Jun 25, 2014 5:39 am
Full Name: Latham Keen
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by Bobin_NZ »

I have been backing up our historical data using fast-glazier and a backup up copy job every 3 months. I tried using the VTL and the storage gateway in AWS, but the AWS VTL isn't support in Veeam. The engineer I discuses the problem with said it was something they where working on. Apparently (when I was trying this) the VTL drivers weren't supported in Veeam. I think (at the time) on Symantec was able to interface with it.

If anyone has had success with this in Veeam 8 or 7 please let me know. This would be a great way to stick in historical backups in to glazier
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by veremin » 1 person likes this post

The recent release of AWS VTL Gateway brings support to both B&R v7 Patch #4 and B&R v8. Thanks.
Bobin_NZ
Influencer
Posts: 12
Liked: never
Joined: Jun 25, 2014 5:39 am
Full Name: Latham Keen
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by Bobin_NZ »

Thanks Eremin. I will start testing it out. Appreciate the quick reply.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by veremin »

You're welcome. Should other questions regarding Amazon S3 or Cloud Connect arise, feel free to ask. Thanks.
Tomsyr
Enthusiast
Posts: 37
Liked: 1 time
Joined: Jul 01, 2014 3:39 pm
Full Name: Tom Conklin
Location: Central New York
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by Tomsyr »

A few questions about using AWS S3...
I understand that doing a full recovery of a server isn't practical, but the way I understand it, we should be able to do a file or directory level recovery by running Veeam B&R on an instance in our VPC, and make it available there, or download it to a local resource.
If our goal is off-site storage in the event of a disaster, our main risk is the restore of ALL the data, but AWS would give us the Off-Site protection, and access to critical files immediately...
I saw a comment about data loss on AWS - They state 11 9's of availability. Can you provide a link that shows where users lost data on AWS?
Regarding the comment about picking up the phone for a restore, we're using a MSP for our AWS resources and paying for Amazon support. So far they have been very responsive for other needs (GovCloud is a PITA ;)).

Currently, the AWS gateway appliance is not available to my sites since we have our VPC/S3 in GovCloud. If we were to use V8, wouldn't all the benefits of WAN optimization be addressed anyway?

Lastly - are you aware of any CloudConnect partners that are capable of managing ITAR related data - this basically means that only USA citizens can access our data, which includes the hardware and software that runs the systems.
We are currently evaluating how we are going to store our data off-site, and we HATE tapes and USB drives.

Thanks in advance!
Tom
tsightler
VP, Product Management
Posts: 6009
Liked: 2843 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by tsightler »

Tomsyr wrote:I saw a comment about data loss on AWS - They state 11 9's of availability. Can you provide a link that shows where users lost data on AWS?
I believe they actually claim 99.99% availbility for their S3 data storage service (object storage), and 11 9's of "durability". While this may technically be accurate, it can be somewhat misleading as they are referring to the inability to recovery any given object. But if you have a 1TB backup file, that's not going to be stored as 1 object, instead it will be broken down into many smaller objects based on the exact technology you use to push it there. Assuming you use something like Cloudberry this will likely be 10MB "chunks". Which means that a single 1TB backup could easily be 100,000 objects. The risk is that the loss of even one of those chunks could render the entire 1TB backup useless, unlike if your storing a bunch of images, or videos, where the loss of one object is the loss of one picture or Vine.

I'm not saying that 11 9's of durability is bad or anything though, it's probably as high as any other storage medium out there, but just wanted to clarify some of the concepts.

Also to be clear, Veeam doesn't natively use any object storage, although you can potentially use 3rd party option to push Veeam backups to S3 or other object storage. However, if you run a Veeam repository in Amazon you'll need to use EBS storage. It's availability and durability guarantees are different from S3, as are it's pricing. And there have been documented instances of data loss for EBS, by far the largest back in 2011 where, after 3 days of outages and days of recovery, Amazon eventually apologized and said not all data was recoverable. In some cases people reported that they lost data in S3 as well, but that appears to be more of a case that they lost access to their S3 data because they were dependent on systems running in EC2 with EBS backed volumes and once those were offline the data in their S3 buckets wasn't really useful. In some cases Amazon specifically told customers to restore from their own backups. It's pretty easy to Google it up, but here are a few links about that incident.

http://www.bizjournals.com/seattle/blog ... l?page=all
http://www.quora.com/Has-Amazon-S3-ever ... ermanently

Note that I'm not intending to say anything negative about Amazon services. They have proven to be robust and reliable but, like anything built by humans, they can fail, and you have to compare not just the costs, but also the potential risk associated with using them. It's really no different than any other utility.
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by Gostev »

Tomsyr wrote:Lastly - are you aware of any CloudConnect partners that are capable of managing ITAR related data - this basically means that only USA citizens can access our data, which includes the hardware and software that runs the systems.
Hi, Tom. I have asked our VCP team to reach out to our current and upcoming US-based Cloud Connect providers and ask this.
rick.johnson
Veeam Software
Posts: 1
Liked: never
Joined: Nov 04, 2014 4:11 pm
Full Name: Rick Johnson
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by rick.johnson »

Hi Tom,

I've provided a link below to our Veeam Cloud Connect partner look-up, the feedback i got from ODS (OffsiteDataSync), Cirrity and NewCloud Network is that they in fact familiar with ITAR and are compliant.

http://www.veeam.com/find-a-veeam-cloud-provider.html
mchesterton
Service Provider
Posts: 2
Liked: 1 time
Joined: Oct 28, 2014 5:09 pm
Full Name: Matthew Chesterton
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by mchesterton » 1 person likes this post

Hi Tom,

We (OffsiteDataSync) meet the needs and provide service to organizations and entities with ITAR compliance constraints. I'd be happy to discuss this further if / when you are ready.
Tomsyr
Enthusiast
Posts: 37
Liked: 1 time
Joined: Jul 01, 2014 3:39 pm
Full Name: Tom Conklin
Location: Central New York
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by Tomsyr »

Thank you all for your replies!
Tom
AntonZ
Influencer
Posts: 13
Liked: never
Joined: Dec 04, 2014 4:22 am
Full Name: Anton
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by AntonZ »

Hi guys!
The question to Veeam team and to service providers.
Currently I have 80TBs of backups. My full compressed backup is 5TBs. I need to store everything which is older than 2 months in cloud. So let's count. I will round. My best case scenario is to use Amazon Glaciar - 1 cent per 1gb => $1000 per 100TB of data. My worst case scenario is to use Amazon S3 - 3 cents per 1gb => $3000 per 100TB of data. These numbers work for me.
Now Veeam tells me "you know, you don't need Amazon, you can use any other service provider, let's say, OffsiteDataSync, and pay 16 cents per GB, which is 16(!) or 4 times more expensive for me + you will have an obvious limit of 64TB.
So what are my options with cloud?
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by veremin »

Hi, Anton, kindly, check the adjacent thread talking about similar issue. We've outlined there major benefits of using Cloud Connect against utilization of cheap online storage; should clarify situation for you.

Also, have you tried to estimate the price of restoring 5TB from Amazon Glacier, as well as, the time required for it?

Thanks.
AntonZ
Influencer
Posts: 13
Liked: never
Joined: Dec 04, 2014 4:22 am
Full Name: Anton
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by AntonZ »

Hi, this is simple. I have a lot of data, which I'll touch only in the case of audit. One week for recovery is fine for me. Also I'm not ready to pay 16 times more for a quicker restore.
I have to store weekly backups for a year, monthly backups for a year, yearly backups for 7 years. Every backup is full...
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by Gostev »

AntonZ wrote:Also I'm not ready to pay 16 times more for a quicker restore.
Then just pick a different provider, who does not charge you 16 times more? Regular purchasing considerations still apply - look for the best value for the money you can spend.
AntonZ wrote:Now Veeam tells me "you know, you don't need Amazon, you can use any other service provider, let's say, OffsiteDataSync, and pay 16 cents per GB, which is 16(!) or 4 times more expensive for me + you will have an obvious limit of 64TB.
Veeam does not recommend specific service providers, it is completely up to you who to choose. And you should certainly pick the one that works best for you based on your requirements, including price. As I have already demonstrated in the above-linked topic using a specific example - even today, the prices may vary 10 times between two Cloud Connect service providers. Also, consider that v8 has been launched less than a month ago, meaning there is still little to no provider competition out there (very few Cloud Connect service providers are available to choose from). We have hundreds more already signed up and coming in the next few months.
AntonZ wrote:So what are my options with cloud?
Well, we give you all the possible options, really. If you like Amazon prices, and don't care about fast backups and restores - we do support backup to Amazon via VTL Gateway, as already mention on the previous page. Just be sure you fully understand recovery time and especially recovery costs with Amazon cloud storage (unlike backup, getting your data out of Amazon can be extremely expensive).
tinto1970
Expert
Posts: 109
Liked: 32 times
Joined: Sep 26, 2013 8:40 am
Full Name: Alessandro T.
Location: Bologna, Italy
Contact:

[MERGED] : Anyone using AmazonStorageGateway and S3?

Post by tinto1970 »

Hi all,

we are evaluating the chance to put a copy, or whole our backup files on Amazon web services.

Using Veeam Backup copy seems the more obvious, but I'm worried about performances, because the .vbk will be rewritten i.e. daily

Using S3 as primary backup target would allow me to use a "classic" full+differential backup policy, without activating the 'synthetic full' creation.

Is there anyone who could share his experience with similar utilization of AWS S3? Any suggestion or comment will be appreciated.
Thanks in advance.
Alessandro aka Tinto | VMCE 2024 | Veeam Legend | VCP-DCV 2023 | VVSPHT2023 | vExpert 2024
blog.tinivelli.com
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by veremin »

Hi, Alessandro,

Some considerations are provided above, so, please, take a look.

Speaking about copying data offsite, haven't you thought about usage of backup copy job pointed to Cloud Provider of your choice?

Thanks.
abelliniSIBA
Enthusiast
Posts: 32
Liked: 10 times
Joined: Nov 19, 2014 2:01 pm
Full Name: Alessandro Bellini
Location: Italy, Milan
Contact:

[MERGED] : S3 repository

Post by abelliniSIBA »

Hi all,
I have a local Veeam server (win2k8r2) + Wan Accelerator Local (win 2k8r2) + Wan Accelerator Target on AWS (EC2 instance win2012) and we need to store data on S3 (as repo, mounted as local disk)
on the forum, i found that topic. http://forums.veeam.com/post100360.html ... or#p100360
where Luca says ".. In order to save on price of EBS disk for example, you can create a VM with a small disk and attach an S3 bucket as slave disk to that VM."
About that,
Can i ask you if we can use this repository as Veeam Repository?
If so, have you some guide/info to be sure Veeam can contact this repository? (now i'm using a third-parties software to mount S3 bucket as local drive)
With this architecture i have a message from Veeam, for my test task, "can't contact target Repository"
Thanks all for your help
Regards

Alessandro
Veolia
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by veremin »

Your post has been merged into existing topic discussing similar matter. Kindly, take a look at the answers provided, and ask for any assistance, should it be needed.
abelliniSIBA
Enthusiast
Posts: 32
Liked: 10 times
Joined: Nov 19, 2014 2:01 pm
Full Name: Alessandro Bellini
Location: Italy, Milan
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by abelliniSIBA »

v.Eremin wrote:Your post has been merged into existing topic discussing similar matter. Kindly, take a look at the answers provided, and ask for any assistance, should it be needed.
Thanks for your help.
Anyway, can you confirm me that the only way to export backup/restore to/from cloud, having deduplication and compression feature, is to use cloud providers?
Thanks in advance for your help
Veolia
Post Reply

Who is online

Users browsing this forum: No registered users and 17 guests