Offsite Incremental to Synthetic Full Utility

Availability for the Always-On Enterprise

Offsite Incremental to Synthetic Full Utility

Veeam Logoby jfowler » Thu Nov 18, 2010 3:24 pm 1 person likes this post

If I wanted to just copy over the first .vbk and then all the .vib Incremental files to a cheap NAS at a remote site every night, is there a way I could takes those .vib files and build a synthetic full outside of the backup job? Hopefully something I could easily script? Looking for a way to avoid copying the synthetic full that is created every week locally to the remote site.

Also, Is there a way to specify a location for each of the backup types? Like so:

D:\Backups\Job1.vbk
E:\Incremental\Job12010-11-17T220058.vib
F:\Archives\Job12010-11-11T220058.vrb
jfowler
Novice
 
Posts: 7
Liked: 1 time
Joined: Thu Dec 17, 2009 5:00 pm
Full Name: Jeremy Fowler

Re: Offsite Incremental to Synthetic Full Utility

Veeam Logoby Gostev » Thu Nov 18, 2010 3:42 pm

Hi Jeremy, no it is not possible to build new full outside backup job or specify individual locations for VBK, VIB and VRB files. Please clarify why do you need to create synthetic full offsite?
Gostev
Veeam Software
 
Posts: 21390
Liked: 2349 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Offsite Incremental to Synthetic Full Utility

Veeam Logoby jfowler » Thu Nov 18, 2010 4:15 pm

Gostev wrote:Hi Jeremy, no it is not possible to build new full outside backup job or specify individual locations for VBK, VIB and VRB files.


How do I submit a feature request? :)

Gostev wrote:Please clarify why do you need to create synthetic full offsite?


So I don't have hundreds of incrementals to restore from when disaster occurs or during DR testing. Here is what I want to do:

1. Do Full backup and copy the .vbk file to cheap 4TB NAS (Iomega StorCenter Pro ix4-100) Talking about 2TB of data here...
2. Ship NAS to remote site
3. Incremental backup job locally every weekday night
4. Rsync Incremental files to offsite NAS
5. Synthetic Fulls built on Fridays locally
6. Do the same remotely from a server at the remote site (WITHOUT having to rsync the new .vbk file to save on bandwidth 2+TB of data)
7. Convert incremental to reversal and store as archives on large capacity, lower cost drives (locally only, offsite incrementals are deleted)
8. Sleep soundly knowing my DR plan works
jfowler
Novice
 
Posts: 7
Liked: 1 time
Joined: Thu Dec 17, 2009 5:00 pm
Full Name: Jeremy Fowler

Re: Offsite Incremental to Synthetic Full Utility

Veeam Logoby Gostev » Thu Nov 18, 2010 4:52 pm

Sounds good. Will see what we can do in the next releases to facilitate this.
Gostev
Veeam Software
 
Posts: 21390
Liked: 2349 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Offsite Incremental to Synthetic Full Utility

Veeam Logoby tsightler » Thu Nov 18, 2010 5:00 pm

This is a great feature request, and one I have previously indicated would be an excellent idea. We currently get similar functionality by basically, running two Veeam jobs, one to local storage, and a second to remote Linux target. That way, synthetic fulls are performed locally and remotely, without significant WAN impact. It would be really nice to, instead of running two jobs, actually rsync the local incremental to the remote site and have the remote site be able to create the synthetic fulls by running some standalone utility.

Of course, a nice, WAN efficient, asynchronous replication built into Veeam would pretty much solve this issue on it's own. :mrgreen:
tsightler
Veeam Software
 
Posts: 4768
Liked: 1737 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Offsite Incremental to Synthetic Full Utility

Veeam Logoby Mindflux » Thu Nov 18, 2010 11:21 pm

tsightler wrote:This is a great feature request, and one I have previously indicated would be an excellent idea. We currently get similar functionality by basically, running two Veeam jobs, one to local storage, and a second to remote Linux target. That way, synthetic fulls are performed locally and remotely, without significant WAN impact. It would be really nice to, instead of running two jobs, actually rsync the local incremental to the remote site and have the remote site be able to create the synthetic fulls by running some standalone utility.

Of course, a nice, WAN efficient, asynchronous replication built into Veeam would pretty much solve this issue on it's own. :mrgreen:

There was a post by someone who said all their remote backups were taking forever even though both the source/destination were remote. Support confirmed that the data had to travel TWICE over wan (once each way) to back up with veeam without a local install

How is yours different or am I misunderstanding you?
Mindflux
Enthusiast
 
Posts: 26
Liked: never
Joined: Wed Nov 10, 2010 7:52 pm
Full Name: RyanW

Re: Offsite Incremental to Synthetic Full Utility

Veeam Logoby Gostev » Thu Nov 18, 2010 11:48 pm

It was me who confirmed, not support :) that topic was about backing up VM located in site B, by backup server located in site A, to a CIFS share located in site B. Tom is talking about backing up VM located in site A, by backup server located in site A, to a Linux (agent enabled) target located in site B.
Gostev
Veeam Software
 
Posts: 21390
Liked: 2349 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Offsite Incremental to Synthetic Full Utility

Veeam Logoby tsightler » Fri Nov 19, 2010 4:35 am

Mindflux wrote:There was a post by someone who said all their remote backups were taking forever even though both the source/destination were remote. Support confirmed that the data had to travel TWICE over wan (once each way) to back up with veeam without a local install

How is yours different or am I misunderstanding you?


We currently install a Veeam server at each site so that each of our facilities has a "local" Veeam server. That "local" install pushes the backup to a remote linux target at a DR location, which is the most WAN efficient manner currently available. If we needed to restore one of these "remote" backups at the DR location we simply import the backup into the Veeam install local to the DR site and perform the restore.

Basically, as long as either the source, or the target is local to the Veeam server then the data only traverses the WAN once. In the case you are talking about, both the ESX server, and the backup target (a CIFS share) were at the remote site, while the Veeam server was at the "main" site, thus the traffic had to traverse the WAN twice.
tsightler
Veeam Software
 
Posts: 4768
Liked: 1737 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: Offsite Incremental to Synthetic Full Utility

Veeam Logoby gglynn » Wed Jan 19, 2011 7:02 pm

Gostev wrote:Sounds good. Will see what we can do in the next releases to facilitate this.


I'm bumping this topic to see if any further consideration has been given to the development of this utility. This would be extremely useful to me and several of my customers.
gglynn
Novice
 
Posts: 5
Liked: never
Joined: Thu Aug 12, 2010 4:54 pm
Full Name: George Glynn

Re: Offsite Incremental to Synthetic Full Utility

Veeam Logoby virtualwatts » Wed Jan 19, 2011 8:16 pm

Second!! We also could use that utility. That is our Number 2 Enhancement Request - right behind a better UI.
And there I was thinking everyone else has amazing fast WAN speeds and can move 30GB-40GB nightly with ease

Right now we are attempting to run incremental with synth full on local.
We sneakernet the VBK then push the VIB via WAN to DR.
Our next step will be to push the Veeam server and restore it to the DR, where we will attempt a synth full remote from a restored Veeam VM.

We spent quite a bit of time on the phone with Veeam tech going over the details and the response was "well, that should work" and then "technically that is not a supported solution". Oddly, it was the exact use case we presented to the sales guys.

Why? Because Veeam replication is syncronous and our WAN capacity cannot handle the large volumes and long run time.
Our WAN cannot handle the VBK push each night. Backing up from DR would be impractical as well for same reason.
Our ultimate desire is to push VIB files to a DR and have the Veeam in the DR import and synth full.

We are awaiting additional storage in our EMC array on DR before testing this out.

Just FYI - BackupExec 2010 has introduced this functionality in their VMware backup solution, so for no other reason perhaps Veeam should do it to stay ahead of the competition. Another reason of course is that they can sell production licenses for both source and target.

Rick
virtualwatts
Enthusiast
 
Posts: 50
Liked: never
Joined: Thu Nov 18, 2010 2:41 pm
Full Name: Rick Watts

Re: Offsite Incremental to Synthetic Full Utility

Veeam Logoby Gostev » Thu Jan 20, 2011 3:18 pm

virtualwatts wrote:Our WAN cannot handle the VBK push each night. Backing up from DR would be impractical as well for same reason.
Our ultimate desire is to push VIB files to a DR and have the Veeam in the DR import and synth full.

Hi Rick, so why you cannot use what tsightler had shared above for now?
Gostev
Veeam Software
 
Posts: 21390
Liked: 2349 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: Offsite Incremental to Synthetic Full Utility

Veeam Logoby gglynn » Wed Jan 26, 2011 10:24 pm

Gostev wrote:Hi Rick, so why you cannot use what tsightler had shared above for now?


I can't speak for Rick, but the reason I can't use tsightler's suggestion is that I need the backup to exist both locally (because it's the only backup software we use, and we might want to use U-AIR to recover something from a backup quickly) and at the DR site, and I don't have time to run the backup twice every time.
gglynn
Novice
 
Posts: 5
Liked: never
Joined: Thu Aug 12, 2010 4:54 pm
Full Name: George Glynn

Re: Offsite Incremental to Synthetic Full Utility

Veeam Logoby Vitaliy S. » Thu Jan 27, 2011 10:45 am

George, have you considered using rsync in this case? We recommend using rsync to sync backup files offsite. You can search this forum for offsite and rsync for more information on this.
Vitaliy S.
Veeam Software
 
Posts: 19558
Liked: 1102 times
Joined: Mon Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov

Re: Offsite Incremental to Synthetic Full Utility

Veeam Logoby gglynn » Thu Jan 27, 2011 8:39 pm

Vitaliy S. wrote:George, have you considered using rsync in this case? We recommend using rsync to sync backup files offsite. You can search this forum for offsite and rsync for more information on this.


We're using rsync now, but it's not ideal. The problem is that rsync is most effective when you're doing reverse incremental backups, because that's the only way you end up with a file with changed blocks so you don't have to replicate the entire full backup over the wire again, which is key, because we don't have a super-fast link to our offsite. This is also why we'd want to keep a local copy of the backup. However, in our opinion, reverse incremental isn't the safest backup mode, because we'd prefer to not touch our VBK if we don't have to.

Synthetic fulls are great, because if the process to create one fails in the middle, you still have the original full VBK and incremental VIB backup files intact and untouched. If we had a standalone utility capable of creating a synthetic full from a full and a series of incrementals, then we could do one full backup when we bring a new VM online, sneakernet it over to the offsite, and then just replicate the smaller nightly (or more often) incrementals to the offsite over the wire, creating synthetic fulls at the main site and the offsite once a week or so, independently, without the need to ship a massive amount of data across the wire. This is ideal because the data we need to do this is already there at the offsite, it's just not in its most useful form without the requested utility.

Personally, I have a Veeam B&R server installed at both the main and offsite locations, so I could even get away with a new job type inside B&R: a standalone "Create Synthetic Full" job. However, I still think it would be easier to just have a standalone utility for most people.
gglynn
Novice
 
Posts: 5
Liked: never
Joined: Thu Aug 12, 2010 4:54 pm
Full Name: George Glynn

Re: Offsite Incremental to Synthetic Full Utility

Veeam Logoby foxj77 » Fri Jan 28, 2011 12:16 pm

We are in the same setup as jfowler.

I'd love the feature too!
foxj77
Novice
 
Posts: 4
Liked: never
Joined: Thu Jan 06, 2011 2:33 pm

Next

Return to Veeam Backup & Replication



Who is online

Users browsing this forum: Dilupa, Google [Bot], opg70, Yahoo [Bot] and 20 guests