Testing Veeam B&R v8 and I want to archive data to Amazon s3

Availability for the Always-On Enterprise

[MERGED] : Amazon Glacier in 2016.

Veeam Logoby readie » Fri Jan 08, 2016 8:49 am

Starting a new topic, for reasons explained below.
We have been looking at Glacier for 'archiving', as the costs of storage are very low, and we don't expect to have to restore.
Came across 'Veeam Backup and Replication Cloud Edition' and it looked just the job. Simple GUI which allows connection to many different cloud storage providers, including Amazon S3 and Glacier. Started to investigate. Is it a separate product from B&R V8 which we are running? Additional purchase?
Then realised that the videos and descriptions I had been seeing were all dated around 2013/2014, and it seems that Veeam have now 'bundled' it into V8 which we have.
However, the bundling seems to have complicated the situation, and now we either need a 'Cloud Service Provider' (additional costs) and/or Amazon's AWS Gateway (additional monthly cost $125+).
Some have suggested using Synology (which we have) which has the Glacier software built in to the latest version, but it will be a pity to use Synology rather than Veeam to do this bit of our backup strategy.
I should value any comments from Veeam themselves, or from anyone else who has managed to use Glacier as part of their backup strategy.
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.
readie
Expert
 
Posts: 156
Liked: 30 times
Joined: Sun Dec 05, 2010 9:29 am
Full Name: Bob Eadie

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Veeam Logoby v.Eremin » Fri Jan 08, 2016 8:59 am

We don't allow duplicate discussions, so your post has been merged into the existing topic. You can find plenty of real world feedback and considerations here, kindly please review and feel free to ask additional questions if required. Thanks.
v.Eremin
Veeam Software
 
Posts: 13266
Liked: 968 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Veeam Logoby v.Eremin » Fri Jan 08, 2016 9:10 am

113 wrote:Has anyone tried restoring Data from AWS using veeam vtl?

Can you elaborate on your question, as Veeam doesn't provide any virtual tape library? Or you're talking about Amazon AWS Gateway?
v.Eremin
Veeam Software
 
Posts: 13266
Liked: 968 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: [MERGED] : Amazon Glacier in 2016.

Veeam Logoby dellock6 » Fri Jan 08, 2016 10:09 am

readie wrote:However, the bundling seems to have complicated the situation, and now we either need a 'Cloud Service Provider' (additional costs) and/or Amazon's AWS Gateway (additional monthly cost $125+).
Some have suggested using Synology (which we have) which has the Glacier software built in to the latest version, but it will be a pity to use Synology rather than Veeam to do this bit of our backup strategy.
I should value any comments from Veeam themselves, or from anyone else who has managed to use Glacier as part of their backup strategy.


Cloud Edition is not available anymore as the technology was not able to scale efficiently above few VMs. The "bundle" in v8 is not the same technology, as Cloud Connect is a completely new and more efficient solution.
If you do not want to go for Cloud Connect, there are around many solutions that can work as a cloud gateway towards Glacier, Synology is not at all the only option. Whatever can be presented to Veeam as a local disk or a SMB share can be the target for a backup or better a backup copy, then it's up to the appliance (hardware or software) to cache locally and send blocks towards Glacier. Just be aware that the limited cache of these solutions makes them slow to upload Veeam files, which are usually way bigger than their cache. The new Per-VM chains coming in v9 will definitely help.
Luca Dell'Oca
EMEA Cloud Architect @ Veeam Software

@dellock6
http://www.virtualtothecore.com
vExpert 2011-2012-2013-2014-2015-2016
Veeam VMCE #1
dellock6
Veeam Software
 
Posts: 5047
Liked: 1330 times
Joined: Sun Jul 26, 2009 3:39 pm
Location: Varese, Italy
Full Name: Luca Dell'Oca

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Veeam Logoby readie » Mon Jan 11, 2016 9:10 pm 1 person likes this post

Thanks for merging v.Eremin, and for clarification of the situation Luca. After much research we are now thinking seriously of Crashplan, as all our repositories at the moment are on a single Windows 2012 R2 Server as shared iSCSI drive mounts (to various NAS stores) so we think the Windows server with Crashplan will be able to copy the ones we want up to the cloud.
Only thing I can't get my head round is how the permanent archiving system of Crashplan will manage with our incremental or reverse incremental backups . . . should we only backup the vbk and vbm files, and not the vbi? Would Backup Copies be less of a problem?
Anyone tried this, and/or any suggestions would be welcome.
(As an aside, I'm thinking of handling my home 3 PCs using VEB to backup two to a spare HD on the third, and then a single home Crashplan subscription to get it all up to the cloud.)
Sorry - realised that this is becoming a Crashplan thread rather than a Veeam thread, but leveraging what Veeam gives us to start with.
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.
readie
Expert
 
Posts: 156
Liked: 30 times
Joined: Sun Dec 05, 2010 9:29 am
Full Name: Bob Eadie

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Veeam Logoby v.Eremin » Tue Jan 12, 2016 12:15 pm

Based on quick google research, I don't think that the said solution will manage efficiently backup modes that imply transform operations (forward forever incremental, reversed incremental). In other words, it seems that each time .vbk file will be copied anew.

Though, this statement needs to be proven.

Thanks.
v.Eremin
Veeam Software
 
Posts: 13266
Liked: 968 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Veeam Logoby readie » Tue Jan 12, 2016 3:43 pm

Yes I suspect you're right, and we might start just for archiving our infrequent backup copy archives (annual, quarterly, or fortnightly). Of course Crashplan is not going to 'understand' Veeam's backup chain etc . . . . It's cheap, but more thought required!
Thanks for your input.
Bob
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.
readie
Expert
 
Posts: 156
Liked: 30 times
Joined: Sun Dec 05, 2010 9:29 am
Full Name: Bob Eadie

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Veeam Logoby v.Eremin » Tue Jan 12, 2016 3:57 pm

You're welcome. I'd be interested to hear the results of your findings, so, keep us updated on how it goes. Thanks.
v.Eremin
Veeam Software
 
Posts: 13266
Liked: 968 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Veeam Logoby dellock6 » Tue Jan 12, 2016 8:50 pm

Many of this solutions are file-based replica, and not block-based. I did some researches some years ago, so it may have been changed, but those software are designed for the use case of a computer file system, where files are small, and thus they can be copied again if modified.
With Veeam, if we just inject few blocks into the vbk, the file is completely new from the point of view of the replication software, thus it has to be copied entirely again. So, Forever forward and reversed modes are to be excluded. You may try active full, but then each cycle you have to copy the whole full backup file.

I've seen some replication software able to work at the block-level, so they might be able to just replicate the changed block after a transform, but they are pricey, so every penny saved by going to these cheap storage solutions are lost.

Sorry for sounding like a marketing boy, but this is again the advantage you get by using Veeam Cloud Connect over plain cloud storage...

Luca
Luca Dell'Oca
EMEA Cloud Architect @ Veeam Software

@dellock6
http://www.virtualtothecore.com
vExpert 2011-2012-2013-2014-2015-2016
Veeam VMCE #1
dellock6
Veeam Software
 
Posts: 5047
Liked: 1330 times
Joined: Sun Jul 26, 2009 3:39 pm
Location: Varese, Italy
Full Name: Luca Dell'Oca

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Veeam Logoby readie » Thu Jan 14, 2016 5:03 pm

Yea I know you're right, but we might start just with some occasional archives - e.g. at the moment we keep an annual archive on external USB HDs, and dumping them up to Crashplan (will take ages) might be worth a try. We have got a fairly good Internet (500Mbs - though it's not that upload speed) compared with some.
Bob
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.
readie
Expert
 
Posts: 156
Liked: 30 times
Joined: Sun Dec 05, 2010 9:29 am
Full Name: Bob Eadie

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Veeam Logoby SuperStrike » Thu Jan 28, 2016 12:45 pm

dellock6 wrote:Many of this solutions are file-based replica, and not block-based. I did some researches some years ago, so it may have been changed, but those software are designed for the use case of a computer file system, where files are small, and thus they can be copied again if modified.
With Veeam, if we just inject few blocks into the vbk, the file is completely new from the point of view of the replication software, thus it has to be copied entirely again. So, Forever forward and reversed modes are to be excluded. You may try active full, but then each cycle you have to copy the whole full backup file.

I've seen some replication software able to work at the block-level, so they might be able to just replicate the changed block after a transform, but they are pricey, so every penny saved by going to these cheap storage solutions are lost.

Sorry for sounding like a marketing boy, but this is again the advantage you get by using Veeam Cloud Connect over plain cloud storage...
Luca


CrashPlan uses block-level de-duplication backup.

CrashPlan detects the duplicate blocks of data and does not send them again. If the file changes, only the changed blocks are transferred.

I think CrashPlan is great for vbk backup.
SuperStrike
Novice
 
Posts: 8
Liked: never
Joined: Sun Jan 24, 2016 12:34 pm

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Veeam Logoby v.Eremin » Thu Jan 28, 2016 2:33 pm

So, you have tested CrasPlan with the .vbks produced by forever incremental modes - forward forever incremental and reversed incremental ones? When the retention comes, and full backup gets updated, CrashPlan will send just the new blocks, updating the .vbk file already copied to it? Thanks.
v.Eremin
Veeam Software
 
Posts: 13266
Liked: 968 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: Testing Veeam B&R v8 and I want to archive data to Amazo

Veeam Logoby readie » Fri Jan 29, 2016 2:56 pm 1 person likes this post

We looked at Crashplan and thought I had checked that it does NOT do block level backups, so rather gave it up. (We also found that it was DESPERATELY slow as it only managed 2 -3 Mbps, even though we have 100+Mbps available upload . . . they let everyone share their available bandwidth!) However, I have just found this on their site . . .
"After initial backup of the file is complete, only new or changed information is sent when the file is backed up.
When CrashPlan scans a file, it knows that the file changed and the progress bar runs through the file as if the information is new. But as it goes, it discovers the information hasn't actually changed and only transmits the new information to the backup destination.
For the technically savvy: CrashPlan does incremental deltas by block within the file."

However, having ditched Crashplan I have been testing Google Cloud Nearline (using Cloudberry) which is definitely block level. I have had to do a cunning tweak, to convince Cloudberry that it is backing up the same file, even though Veeam has changed the name/date of the VBK file. I have a little batch file which creates a generic named (Windows) 'hard pointer' to the VBK file, and ask Cloudberry to backup this hard link. It does take quite some time to analyse the file before sending, as it has to compare every block with what it had before (I presume), but does then just backup the incremental blocks. . . . I haven't yet looked at restoring, as my initial tests are still progressing. A 2TB VBK file took 5 days to backup (first full backup) and its first incremental is tomorrow. Looking forward to 'per VM backup chains' as a means of spreading these initial full backups.
Happy to keep you all posted as we progress.
Bob Eadie
Computer Manager at Bedford School, UK (since 1999).
Veeam user since 2009.
readie
Expert
 
Posts: 156
Liked: 30 times
Joined: Sun Dec 05, 2010 9:29 am
Full Name: Bob Eadie

[MERGED] Veeam Synthetic Full - Process via AWS

Veeam Logoby marchawk » Mon Feb 08, 2016 4:23 am

Hi

I am looking into what options I have to start to replace my tape backups with a sync to AWS S3 and subsequent offiste Synthetic Full processing, my CIO wa s not impressed with the last tape invoice I just dropped on his desk.

I an ideal scenario I'd love to have :

- Veeam server onsite running a two week synthetic full schedule - Synthetic fulls on first and 3rd Friday of each month. Incremental each day there after.
- Incremental backups both stored on local oniste disk and transferred to AWS S3. Transfer of incremental veeam jobs to S3 would be daily as Veeam jobs complete.
- THIS IS THE PART I NEED GUIDANCE WITH: Veeam Synthetic full job then processed in AWS S3 (would need some form of EC2 instance with Veeam product I assume?). Want to only transfer incremental jobs to S3 then process synthetic fulls with some typeof compute form AWS.
- Synthetic Full also processed on local disk which would prominently be used for restores.

Can anyone please advise if this is possible or is their is a better way to approach this.

Thanks in advance
marchawk
Lurker
 
Posts: 1
Liked: never
Joined: Mon Feb 08, 2016 3:53 am
Full Name: Marc Hawk

[MERGED] : Veeam V9 with Amazon

Veeam Logoby barcode2328 » Mon Feb 22, 2016 4:16 pm

Hello,

We are looking for a new backup solution and one of them on the table is Veeam. There is only one question I have though, and I can't seem to find a direct answer for it. I am hoping that someone can help me out.

Will Veeam v9 backup directly to Amazon (AWS)?

That is our biggest thing we are looking for since we utilize AWS quite a bit right now. We do have storage gateways setup currently with Amazon which we use to send backups up there right now with our existing software. I am hoping to utilize those same gateways if we end up going with Veeam to backup up to AWS as well.

Researching this out on the internet I found that v8 can probably do this, and that on Amazon's side is capable; but then I hear stuff about using CloudBerry and that it might not be supported anymore? I don't want some roundabout way to get our backups up to AWS, I just want it pretty seamless getting them up there. And of course if we were to become a new user with Veeam we would be jumping on v9 I would imagine

Thanks for any answers or pointers. :D
barcode2328
Lurker
 
Posts: 1
Liked: never
Joined: Mon Feb 22, 2016 3:56 pm
Full Name: Brent Johnson

PreviousNext

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Bing [Bot] and 17 guests