Discussions related to using object storage as a backup target.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by veremin »

Actually, you can ship compressed and deduplicated VB&R backups to amazon S3 storage. However, in order to achieve that Amazon Storage Gateway will be required, as well as, tape jobs pointed to it. Thanks.
AtechAddam
Lurker
Posts: 1
Liked: never
Joined: Jun 14, 2015 2:59 am
Full Name: Addam Reynolds
Contact:

[MERGED] : Backup Copy Job questions/queries/worries

Post by AtechAddam »

Hi All,

Sorry if this is a bit of a n00b question.

I have a customer that wishes to have their daily backup copied off to Amazon S3.

We have the backup set up going to network storage and then I have created a backup copy job to copy this over to S3.

This appears to be working, however it is a continuous job. I have set the schedule so that it doesn't interfere with any of their other backups, however once the job pauses, it does not restart.

I have had to cancel the job and start it again so that it runs at the next interval.

Other than this, it runs continuously and even though I've set a schedule, it doesn't appear to be running to this schedule.

I'd prefer it to copy this backup over to S3, finish the job and then the next morning at 1am for it to start again.

I'm wondering if this is by design, or if I am doing something wrong?

Is a backup copy job the best way to do this, or would a normal backup be better?

Thanks,

Addam
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by veremin »

Hi, Addam,

Your post has been merged into existing discussion, so, see the answers provided above for more information about backup to Amazon S3.

Thanks.
tym
Enthusiast
Posts: 83
Liked: 9 times
Joined: Mar 26, 2015 9:30 pm
Full Name: Tim Diekhans
Contact:

[MERGED] : VEEAM to Amazon Glacier

Post by tym »

Hi

I just wanted to start moving from tape to cloud backup and what did I find? VEEAM cancelled support for Amazon Glacier in that they discontinued the cloud edition. Very sad and a huge drawback. Do I have to use the amazon gateway now? I dont want to use another backup software like CloudBerry Backup for this.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by veremin »

Yep, you need to leverage Amazon Storage Gateway in order to get tape-like experience with Amazon Glacier or Amazon S3 storage.

Alternatively, you can use Service Provider lookup, find an SP meeting your requirements and just point your backup and/or backup copy jobs to the cloud repositories provided by them. This will provide much better performance for backups and more importantly for restores (enabling restores directly from the cloud), plus greater manageability due to being fully integrated.

Thanks.
Gostev
Chief Product Officer
Posts: 31459
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: [MERGED] : VEEAM to Amazon Glacier

Post by Gostev »

tym wrote:I just wanted to start moving from tape to cloud backup and what did I find? VEEAM cancelled support for Amazon Glacier in that they discontinued the cloud edition. Very sad and a huge drawback.
So, your first sentence is the most important. Thing is, most users are unwilling to move from tape to cloud until they can backup and restore their data to/from the cloud faster than using tape. As you correctly noted, we did try Amazon-based solution, but most of our users found backup and restore performance to be unacceptable (at least for image-based backups), which is exactly why this functionality was discontinued and replaced with Cloud Connect (which provides at least by an order of magnitude faster backup and restore performance, making it a real alternative to tape).
Meyercord
Enthusiast
Posts: 36
Liked: 6 times
Joined: Jul 14, 2014 4:31 pm
Full Name: AJ Meyercord
Contact:

[MERGED] : Cloud backup using Amazon Gateway

Post by Meyercord »

Is anyone writing backups to cloud storage without the Cloud Connect product? I need a reliable way to get my data offsite into inexpensive long-term cloud storage like S3 or Glacier, but I want to do it directly without relying on a third-party partner.

I've tested backing up to virtual tape using Amazon's VTL solution, but because Veeam requires writing the entire backup set to disk before performing a restore, recovering even a single file or VM would require downloading 4-6TB of data, making it prohibitively expensive and creating a significant delay.

One potential solution I can think of is to host a repository on Amazon Storage Gateway Cached Volume (http://docs.aws.amazon.com/storagegatew ... cepts.html), but I have a number of concerns and I was hoping someone here had either looked into this as well or even implemented it in some form and could comment on their experience:

1) How would you deal with maintaining restore points over the long-term (5-7 years)? Would you continually write them to a single massive volume, or create smaller volumes and swap them out as they fill up? Does it make sense to take Amazon EBS snapshots at intervals and then delete the older Veeam recovery points after they had been snapshot?

2) Are snapshots even practical? The cost to store snapshot data is >3 times the S3 storage cost ($0.095/GB vs $0.030/GB)

3) Is there any way to move the data from S3 to Glacier to reduce cost ($0.010/GB vs $0.030/GB)? I don't see anything in the documentation indicating that it's possible to move Gateway data to Glacier.

4) In the situations above, if I roll off older restore points by swapping out volumes, or snapshotting, or moving them to Glacier, how does Veeam deal with the disappearance of these recovery points? How can I find them in my inventory years from now when I need to restore a file or VM from a specific date or based on other criteria? Do I have to keep everything online all the time just so Veeam doesn't lose track of it? Is there any way to tell Veeam that the data has been archived, and then locate that archived data years later?

5) For performing restores would it be beneficial to build a Veeam proxy server in Amazon EC2 that would have faster access to the data in S3? If I did this, could would it be possible to restore a VMWare guest directly to EC2? Is this a practical way to restore files or VMs, either to an EC2 instance or back to my local LAN?

Am I overthinking this? Is is better to just use Cloud Connect or the Amazon Virtual Tape Library? If you've used either of these, Id like to hear about your experience
mongie
Expert
Posts: 152
Liked: 24 times
Joined: May 16, 2011 4:00 am
Full Name: Alex Macaronis
Location: Brisbane, Australia
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by mongie » 1 person likes this post

I believe in V9 there is a mount server which would allow your point 5 to be valid.

To speed restores in V8, I think you could do this with a full veeam server, and manual imports to the database.
Tailwind
Lurker
Posts: 1
Liked: never
Joined: Oct 25, 2015 7:57 pm
Full Name: Tailwind Support

[MERGED} Veeam to QNAP to Amazon S3

Post by Tailwind »

Hello everyone,

I'm looking for some advice:

I am using B&R to backup 20 HyperV VMs to a QNAP backup device. Currently have no offsite solution though, so I am trying to get something in place ASAP!
Now, QNAP has the ability to upload to Amazon S3 cloud storage, I am trying to look for the best Veeam backup method to use with the intention of uploading the backups to Amazon S3 cloud storage, whilst also keeping the cost as low as possible. Suggestions anyone?
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by foggy »

Please review this thread for suggestions and feel free to ask for any further clarification, if required.
SisTecno
Lurker
Posts: 2
Liked: never
Joined: Nov 09, 2015 12:18 pm
Full Name: Sistemas Tecnocasa
Contact:

[MERGED] Can I configure AWS as Service Provider

Post by SisTecno »

Hi,

We are configuring the Veeam Backup & Replication v8 and we want to use the AWS as a Service Provider to make the Cloud Backups, is it posible.
Can you explain me how to configure.

Thanks
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Can I configure AWS as Service Provider

Post by veremin »

What kind of storage are you planning to use in combination with that VM? EBS, S3, Glacier, etc.?
SisTecno
Lurker
Posts: 2
Liked: never
Joined: Nov 09, 2015 12:18 pm
Full Name: Sistemas Tecnocasa
Contact:

Re: Can I configure AWS as Service Provider

Post by SisTecno »

I want to use S3 for the newers backup and want to move to Glacier the olders. I want to do so making a Budget and in the options make a lifetime to move to glacier older than 3 months
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by veremin »

In this case, you would need Amazon Storage Gateway. More information regarding the applicable scenario, as well as, drawbacks it contains can be found on the first page of this discussion; kindly, take a look. Thanks.
113
Lurker
Posts: 1
Liked: never
Joined: Jan 07, 2016 7:27 pm
Full Name: jd
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by 113 »

Hello,

Has anyone tried restoring Data from AWS using veeam vtl?

Thanks
readie
Expert
Posts: 158
Liked: 30 times
Joined: Dec 05, 2010 9:29 am
Full Name: Bob Eadie
Contact:

[MERGED] : Amazon Glacier in 2016.

Post by readie »

Starting a new topic, for reasons explained below.
We have been looking at Glacier for 'archiving', as the costs of storage are very low, and we don't expect to have to restore.
Came across 'Veeam Backup and Replication Cloud Edition' and it looked just the job. Simple GUI which allows connection to many different cloud storage providers, including Amazon S3 and Glacier. Started to investigate. Is it a separate product from B&R V8 which we are running? Additional purchase?
Then realised that the videos and descriptions I had been seeing were all dated around 2013/2014, and it seems that Veeam have now 'bundled' it into V8 which we have.
However, the bundling seems to have complicated the situation, and now we either need a 'Cloud Service Provider' (additional costs) and/or Amazon's AWS Gateway (additional monthly cost $125+).
Some have suggested using Synology (which we have) which has the Glacier software built in to the latest version, but it will be a pity to use Synology rather than Veeam to do this bit of our backup strategy.
I should value any comments from Veeam themselves, or from anyone else who has managed to use Glacier as part of their backup strategy.
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by veremin »

We don't allow duplicate discussions, so your post has been merged into the existing topic. You can find plenty of real world feedback and considerations here, kindly please review and feel free to ask additional questions if required. Thanks.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by veremin »

113 wrote:Has anyone tried restoring Data from AWS using veeam vtl?
Can you elaborate on your question, as Veeam doesn't provide any virtual tape library? Or you're talking about Amazon AWS Gateway?
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: [MERGED] : Amazon Glacier in 2016.

Post by dellock6 »

readie wrote:However, the bundling seems to have complicated the situation, and now we either need a 'Cloud Service Provider' (additional costs) and/or Amazon's AWS Gateway (additional monthly cost $125+).
Some have suggested using Synology (which we have) which has the Glacier software built in to the latest version, but it will be a pity to use Synology rather than Veeam to do this bit of our backup strategy.
I should value any comments from Veeam themselves, or from anyone else who has managed to use Glacier as part of their backup strategy.
Cloud Edition is not available anymore as the technology was not able to scale efficiently above few VMs. The "bundle" in v8 is not the same technology, as Cloud Connect is a completely new and more efficient solution.
If you do not want to go for Cloud Connect, there are around many solutions that can work as a cloud gateway towards Glacier, Synology is not at all the only option. Whatever can be presented to Veeam as a local disk or a SMB share can be the target for a backup or better a backup copy, then it's up to the appliance (hardware or software) to cache locally and send blocks towards Glacier. Just be aware that the limited cache of these solutions makes them slow to upload Veeam files, which are usually way bigger than their cache. The new Per-VM chains coming in v9 will definitely help.
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
readie
Expert
Posts: 158
Liked: 30 times
Joined: Dec 05, 2010 9:29 am
Full Name: Bob Eadie
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by readie » 1 person likes this post

Thanks for merging v.Eremin, and for clarification of the situation Luca. After much research we are now thinking seriously of Crashplan, as all our repositories at the moment are on a single Windows 2012 R2 Server as shared iSCSI drive mounts (to various NAS stores) so we think the Windows server with Crashplan will be able to copy the ones we want up to the cloud.
Only thing I can't get my head round is how the permanent archiving system of Crashplan will manage with our incremental or reverse incremental backups . . . should we only backup the vbk and vbm files, and not the vbi? Would Backup Copies be less of a problem?
Anyone tried this, and/or any suggestions would be welcome.
(As an aside, I'm thinking of handling my home 3 PCs using VEB to backup two to a spare HD on the third, and then a single home Crashplan subscription to get it all up to the cloud.)
Sorry - realised that this is becoming a Crashplan thread rather than a Veeam thread, but leveraging what Veeam gives us to start with.
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by veremin »

Based on quick google research, I don't think that the said solution will manage efficiently backup modes that imply transform operations (forward forever incremental, reversed incremental). In other words, it seems that each time .vbk file will be copied anew.

Though, this statement needs to be proven.

Thanks.
readie
Expert
Posts: 158
Liked: 30 times
Joined: Dec 05, 2010 9:29 am
Full Name: Bob Eadie
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by readie »

Yes I suspect you're right, and we might start just for archiving our infrequent backup copy archives (annual, quarterly, or fortnightly). Of course Crashplan is not going to 'understand' Veeam's backup chain etc . . . . It's cheap, but more thought required!
Thanks for your input.
Bob
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by veremin »

You're welcome. I'd be interested to hear the results of your findings, so, keep us updated on how it goes. Thanks.
dellock6
Veeam Software
Posts: 6137
Liked: 1928 times
Joined: Jul 26, 2009 3:39 pm
Full Name: Luca Dell'Oca
Location: Varese, Italy
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by dellock6 »

Many of this solutions are file-based replica, and not block-based. I did some researches some years ago, so it may have been changed, but those software are designed for the use case of a computer file system, where files are small, and thus they can be copied again if modified.
With Veeam, if we just inject few blocks into the vbk, the file is completely new from the point of view of the replication software, thus it has to be copied entirely again. So, Forever forward and reversed modes are to be excluded. You may try active full, but then each cycle you have to copy the whole full backup file.

I've seen some replication software able to work at the block-level, so they might be able to just replicate the changed block after a transform, but they are pricey, so every penny saved by going to these cheap storage solutions are lost.

Sorry for sounding like a marketing boy, but this is again the advantage you get by using Veeam Cloud Connect over plain cloud storage...

Luca
Luca Dell'Oca
Principal EMEA Cloud Architect @ Veeam Software

@dellock6
https://www.virtualtothecore.com/
vExpert 2011 -> 2022
Veeam VMCE #1
readie
Expert
Posts: 158
Liked: 30 times
Joined: Dec 05, 2010 9:29 am
Full Name: Bob Eadie
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by readie »

Yea I know you're right, but we might start just with some occasional archives - e.g. at the moment we keep an annual archive on external USB HDs, and dumping them up to Crashplan (will take ages) might be worth a try. We have got a fairly good Internet (500Mbs - though it's not that upload speed) compared with some.
Bob
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.
SuperStrike
Influencer
Posts: 10
Liked: never
Joined: Jan 24, 2016 12:34 pm
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by SuperStrike »

dellock6 wrote:Many of this solutions are file-based replica, and not block-based. I did some researches some years ago, so it may have been changed, but those software are designed for the use case of a computer file system, where files are small, and thus they can be copied again if modified.
With Veeam, if we just inject few blocks into the vbk, the file is completely new from the point of view of the replication software, thus it has to be copied entirely again. So, Forever forward and reversed modes are to be excluded. You may try active full, but then each cycle you have to copy the whole full backup file.

I've seen some replication software able to work at the block-level, so they might be able to just replicate the changed block after a transform, but they are pricey, so every penny saved by going to these cheap storage solutions are lost.

Sorry for sounding like a marketing boy, but this is again the advantage you get by using Veeam Cloud Connect over plain cloud storage...
Luca
CrashPlan uses block-level de-duplication backup.

CrashPlan detects the duplicate blocks of data and does not send them again. If the file changes, only the changed blocks are transferred.

I think CrashPlan is great for vbk backup.
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by veremin »

So, you have tested CrasPlan with the .vbks produced by forever incremental modes - forward forever incremental and reversed incremental ones? When the retention comes, and full backup gets updated, CrashPlan will send just the new blocks, updating the .vbk file already copied to it? Thanks.
readie
Expert
Posts: 158
Liked: 30 times
Joined: Dec 05, 2010 9:29 am
Full Name: Bob Eadie
Contact:

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Post by readie » 1 person likes this post

We looked at Crashplan and thought I had checked that it does NOT do block level backups, so rather gave it up. (We also found that it was DESPERATELY slow as it only managed 2 -3 Mbps, even though we have 100+Mbps available upload . . . they let everyone share their available bandwidth!) However, I have just found this on their site . . .
"After initial backup of the file is complete, only new or changed information is sent when the file is backed up.
When CrashPlan scans a file, it knows that the file changed and the progress bar runs through the file as if the information is new. But as it goes, it discovers the information hasn't actually changed and only transmits the new information to the backup destination.
For the technically savvy: CrashPlan does incremental deltas by block within the file."

However, having ditched Crashplan I have been testing Google Cloud Nearline (using Cloudberry) which is definitely block level. I have had to do a cunning tweak, to convince Cloudberry that it is backing up the same file, even though Veeam has changed the name/date of the VBK file. I have a little batch file which creates a generic named (Windows) 'hard pointer' to the VBK file, and ask Cloudberry to backup this hard link. It does take quite some time to analyse the file before sending, as it has to compare every block with what it had before (I presume), but does then just backup the incremental blocks. . . . I haven't yet looked at restoring, as my initial tests are still progressing. A 2TB VBK file took 5 days to backup (first full backup) and its first incremental is tomorrow. Looking forward to 'per VM backup chains' as a means of spreading these initial full backups.
Happy to keep you all posted as we progress.
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.
marchawk
Lurker
Posts: 1
Liked: never
Joined: Feb 08, 2016 3:53 am
Full Name: Marc Hawk
Contact:

[MERGED] Veeam Synthetic Full - Process via AWS

Post by marchawk »

Hi

I am looking into what options I have to start to replace my tape backups with a sync to AWS S3 and subsequent offiste Synthetic Full processing, my CIO wa s not impressed with the last tape invoice I just dropped on his desk.

I an ideal scenario I'd love to have :

- Veeam server onsite running a two week synthetic full schedule - Synthetic fulls on first and 3rd Friday of each month. Incremental each day there after.
- Incremental backups both stored on local oniste disk and transferred to AWS S3. Transfer of incremental veeam jobs to S3 would be daily as Veeam jobs complete.
- THIS IS THE PART I NEED GUIDANCE WITH: Veeam Synthetic full job then processed in AWS S3 (would need some form of EC2 instance with Veeam product I assume?). Want to only transfer incremental jobs to S3 then process synthetic fulls with some typeof compute form AWS.
- Synthetic Full also processed on local disk which would prominently be used for restores.

Can anyone please advise if this is possible or is their is a better way to approach this.

Thanks in advance
barcode2328
Lurker
Posts: 1
Liked: never
Joined: Feb 22, 2016 3:56 pm
Full Name: Brent Johnson
Contact:

[MERGED] : Veeam V9 with Amazon

Post by barcode2328 »

Hello,

We are looking for a new backup solution and one of them on the table is Veeam. There is only one question I have though, and I can't seem to find a direct answer for it. I am hoping that someone can help me out.

Will Veeam v9 backup directly to Amazon (AWS)?

That is our biggest thing we are looking for since we utilize AWS quite a bit right now. We do have storage gateways setup currently with Amazon which we use to send backups up there right now with our existing software. I am hoping to utilize those same gateways if we end up going with Veeam to backup up to AWS as well.

Researching this out on the internet I found that v8 can probably do this, and that on Amazon's side is capable; but then I hear stuff about using CloudBerry and that it might not be supported anymore? I don't want some roundabout way to get our backups up to AWS, I just want it pretty seamless getting them up there. And of course if we were to become a new user with Veeam we would be jumping on v9 I would imagine

Thanks for any answers or pointers. :D
Post Reply

Who is online

Users browsing this forum: Semrush [Bot] and 32 guests