v6 - How to have a local and off-site backup copy?

[sly_cv]

Hi-
I'd like to configure my backups so that I have a local copy in my data center and a copy off-site in our DR site.

I think the ideal situation would be to either:
1) have a backup job that puts data in two repositories.
2) Have a job that copies data from one repository to a second.

Is either of these possible? Or is there another way of accomplishing this goal?

This is what my setup loos like now:

Code: Select all

          Data Center                 ||      DR Site  
[ESXi Host] ---- [Windows Server]  ---||----[Windows Server]
                   Proxy                         Repository
                   Repository

P.S. Do I need a proxy at my DR site?

[Gostev]

Hi Sam,

v6 does not bring any changes comparing to v5 to the above scenario. Regarding your options:
1. There is no such capability in the product.
2. Since our File Copy jobs always do "full backup" instead of deltas, this would be very inefficient.

Just like with v5, you have two options:
1. Two jobs, one backing up to the local repository, and another one to DR site repository.
2. One job backing up to the local repository, and using rsync to sync backups to DR site.

Internally at Veeam, we are using scenario 1. One customer who already had a chance to try both 1 and 2 with v6 is reporting that option 1 works better for him.

No, you do not need a proxy at DR site, just repository (unless your target in DR site is a CIFS share).

Thanks!

[chad5k1]

We have exactly the same setup.
If you run option 1, do you need another veeam license for the DR site? Or would you just run both jobs from the Data center?

[Vitaliy S.]

Richard, you need to have license for source hosts only. Please review our sticky F.A.Q. for more information. Thanks!

[chad5k1]

Perfect thanks Vitaliy. In which case, do you think it is better to run both backups from one site, or run one from each site?

[Vitaliy S.]

Not sure I fully understand your question, could you please elaborate?

[chad5k1]

So as I understand it with option 1, there are a couple of options
a) Run two backup jobs, both from one site (which one?), but each with a different destination (one destination local data center, one destination DR site).
b) Install two veeam servers (one at each site) and run one backup job from each site (Datacenter and DR), with local destinations
Does that make it a bit clearer?

[foggy]

If your question regards to Veeam B&R server location (production or DR site), then it no longer matters with new v6 architecture. The actual backup is performed by backup proxies. Option 1 in Anton's post above implies having a backup proxy on the production site and setting up two backup jobs within your single Veeam B&R console to backup VMs on your production host. The first one will perform backups locally to store them on the same production site and the second one will put backups offsite (to another (DR) site).

[chad5k1]

Yes, just having a read and I'd started to come to the same conclusion. So a proxy on the DR site and the production site that each do local backups.
Thanks all.

[brandonfro]

If I implement Option 1, does that mean if I suffer total loss at the source site, then I won't have access to the multiple restore points created by Veeam? Even with Veeam installed off-site? Is the solution to getting around this problem having two jobs, one at each site?

[k00laid]

With option 1 and having a B&R server at the DR site as well is there now an automated way to get those jobs imported into Veeam at the remote site?

[Vitaliy S.]

If I implement Option 1, does that mean if I suffer total loss at the source site, then I won't have access to the multiple restore points created by Veeam? Even with Veeam installed off-site? Is the solution to getting around this problem having two jobs, one at each site?

No, this doesn't mean that. If you loose your production site completely, please import all backup files to a new backup server and perform a restore. Besides, we still have standalone utility called extract.exe which allows you to restore VMs even without presence of the Veeam backup server.

With option 1 and having a B&R server at the DR site as well is there now an automated way to get those jobs imported into Veeam at the remote site?

Backup jobs cannot be shared between two different backup servers. You will have to recreate your backup jobs and map them to existing backup files to continue running incremental runs.

[Gostev]

You will have to recreate your backup jobs and map them to existing backup files to continue running incremental runs.

Actually, you cannot do this between backup servers at this time. You can only map backup if it was produced by another job on the same server.

[k00laid]

Actually, you cannot do this between backup servers at this time. You can only map backup if it was produced by another job on the same server.

Can you expand on that? Does that mean I need to continue to run the offsite job from the offsite Veeam server? I thought this was a feature that was supposed to be there in v6. Any time frame on being able to share backups between servers?

[Gostev]

No, it does not mean that. Not sure how it is connected.

[holografique]

v6 does not bring any changes comparing to v5 to the above scenario. Regarding your options:
1. There is no such capability in the product.
2. Since our File Copy jobs always do "full backup" instead of deltas, this would be very inefficient.

Just like with v5, you have two options:
1. Two jobs, one backing up to the local repository, and another one to DR site repository.
2. One job backing up to the local repository, and using rsync to sync backups to DR site.

Internally at Veeam, we are using scenario 1.

Hi Gostev,

While this may work for some customers, this still poses a huge limitation in the product for being able to handle true offsite archive copies of backups for environments with hundreds, or potentially thousands of virtual machines (especially in the service provider space).

The biggest issues is that having to have a second set of jobs just for the purpose of sending backup copies offsite imposes DOUBLE the read disk IO load on the primary Tier 1 storage array where the production vm's live (whether VADP or direct host backup is used). This method does not scale well and causes storage performance issues as the number of vm's and the total amount of data grows.

When comparing this to other products, the most effective and scalable method is having the ability to have a media server (or proxy) that is responsible for replicating copies of the backup jobs directly off the backup storage target (typically a Tier2/Tier3 storage array) over to another proxy at the offsite location. This allows the environment to have jobs running just for the purpose of getting backup archives offsite (without the need for tape), AND most importantly it keeps all of the loads associated with these archive jobs isolated to the backup storage array and away from the primary Tier1 storage array storage array. Again, this is very important when dealing with large scale environments.

Any possibility of seeing this capability in the future? I was under the impression this capability would be available in v6.0

thanks

-omar

[chad5k1]

I would certainly like to see this functionality. It would be a huge deal.

[Gostev]

Hi Omar,

We will have every possible feature in the future, and I agree this one would be "nice to have". However, being the end user of the product as well, I don't really agree with your assessment of double jobs being the absolute evil. We are having great success doing exactly this in our production environment with no noticeable impact on production storage, and our VMware administrator cannot be more happy. Because CBT allows to pick up changed blocks only - "DOUBLE" disk I/O for each changed blocks, although does look scary when put in all capital letters, is not really noticeable in reality.

I do agree that this may still be a concern for really large environments, however keep in mind that to date, our product has been primarily focused at small and medium businesses - with the new v6 architecture being the first real step into enterprise environments. And it would be premature to add enterpise-focused features before we even had enterprise-class architecture. One step at a time...

Thanks!

[holografique]

thanks for the reply. and yes, my post was specifically stated at the enterprise and service provider level. again, where there are thousands of vm's that have to be backed up, where thousands of vm's are already generating large amounts of disk IO load, any amount of additional loads can have a negative impact during production hours, that is a reality, and one that many large customers have to deal with and monitor very closely. especially in the service provider space where the provider does not control when customer workloads are generated. those environments are even more sensitive to having any product or design methodology create additional IO loads against the primary storage tiers. By having the ability to pull copies of backups directly from the backup storage target, this allows the most amount of flexibility in allowing those jobs to continue running at any point in time with NO risk to impacting production workloads.

I can tell you from first-hand experience as a partner, you are missing out on large opportunities because of the inability to have this capability. Several we have already worked with and tried to sell them on the architecture, but was unable to because of this performance load risk. You are also missing out on the opportunity to compete with other appliance based de-dupe products that have their built in replication capabilities to get the backup copies offsite. I can tell you for sure, as soon as you have this capability, you are going to make a huge impact by offering a much lower cost method for getting backups offsite for archive purposes, without the need for overly expensive de-dup based appliances (you know which one's im talking about )

I look forward to the enterprise-class architecture...

thanks!

[JoeIanni]

I would like to agree with holografique assessment, I would love to see a 2 stage approach to offsite backups as well as offsite replication. This will also fit nicely with VM's that are on the larger side with large amounts of changes. This would eliminate the fuss of working out a time window to preform offsite replications and or offsite backup just to minimise snapshots size and I/O load.

[fgw]

you can always create a "Fle Copy" job to pick up the backup files created by your backup jobs and copy them to a remote server. you can e

the only nice feature i can think of which is missing here is the ability to select a certain cob as post job activity. right now you can only select a command to run after a job finishes without errors. would be nice to simply select one (or more?) of the other jobs defined on the server to be started when a job is finished.

this would also allow for cascading jobs ...

another option would be to add a feature to the "Schedule" tab which lets you select a list of jobs which all needs to be finished before a job starts. this way the copy job could be configured to wait until all backup jobs are finished and finally start to transfer to backup files to the remote location.

yes i know this probably can be done with the help of powershell scripts. but it would be much simpler if one can select this from within the gui.

may be veeam is considering implementing such a feature ...

[JoeIanni]

From my experience, my clients will utilise both replication and backups to meet their respective Backup/DR policies. so I would have to disagree you fgw, I would like to see offsite replication work with a single snaphot (Staged approach, as option) and not two individual one's (Doing a local backup as well as a having a replica for the same VM), for the following reasons:

1. One snaphot required per VM.
2. Window time for the snapshot is greatly reduced.
3. Less impact to production environment due to decreased number of snapshots.
4. Transfer of images can be controlled and not dependant on work production times or other such factors.
5. De-duplication my be possible with replicas in a staged approach.
6. I believe more jobs could be replicated across a same size link due to statement in item 4 and 5.
7. No need to have a Veeam Server ready to import backup images at the remote site, if offsite backups are employed.
8. RTO's are much better with replica's than offsite backups.
9. Less management required for offsite replica's then offsite backups in the event of a DR.

[fgw]

joe, i guess we are not talking about the same scenario.

what i was talking about is as follows:

• run a backup job, which involves snapshots - but thats the way it is
• run a file copy job to copy the backupfiles to the remote site

actually just one snapshot operation involved here!

my approach is a bit different.

in my environment there are about 115 vm's with about 2.8TB storage configured in total.
a full backup takes about 3 to 4 hours to complete - not that bad! this is required only at first run anyway.
subsequent daily (tue - sat) incrementals, in fact all following backups, will take about 1 hour.

as i'm too lazy to write the script to check all my backupjobs to detect when they are done and trigger my offsite copy, i simply run another backup on sundays to a repository at the remote site. well, in fact its written to a windows share on a server at the remote site as windows repositories are broken in the current release.
once again, this will take some hours on the first run but runs much quicker on every following sunday.
as there is no production going on on weekends its no problem in my environment.

finally i will end up having

• daily backups locally (tue - sat)
• a weekly backupfile containing all vm's (or may be several files if vm's are split into more than one job) at the remote site which can also be copied onto tape

[bhwong]

1. I don't quite understand the need to have 2 backup jobs for a local and remote backup. Does this mean that for every VM to be backed up at both local and remote sites, there will be 2 read and 2 snapshots taken?

2. Why is a backup for remote preferred over replication?

3. If I have 2 sites that need to backup to one another, do I need to install a Veeam server at each site? Or is one Veeam server sufficient to manage backup jobs at both sites? In this environment, do I also need a proxy and repository server at each site to perform backup and replication?

[foggy]

1. I don't quite understand the need to have 2 backup jobs for a local and remote backup. Does this mean that for every VM to be backed up at both local and remote sites, there will be 2 read and 2 snapshots taken?

It depends on the way you select to go when deciding to have both local and remote backups. As Anton has described in his first post in this thread, you can either have two jobs pointing to different repositories (local and remote) or one local backup job and rsync syncing backups to remote site.

2. Why is a backup for remote preferred over replication?

Nobody is talking about preference here. It's all about your backup strategy, but we like to say that backup is not a backup until you have at least 3 copies of it. It's quite a common practice to have replicas for short term retention and good RTO (immediate failover) and backups at both local and remote sites for a much longer (and less regular) retention (space-efficient data archive).

3. If I have 2 sites that need to backup to one another, do I need to install a Veeam server at each site? Or is one Veeam server sufficient to manage backup jobs at both sites? In this environment, do I also need a proxy and repository server at each site to perform backup and replication?

Single Veeam B&R server is sufficient, but you do need to have proxy servers and repositories at each location for efficient backup and replication.

[bhwong]

It depends on the way you select to go when deciding to have both local and remote backups. As Anton has described in his first post in this thread, you can either have two jobs pointing to different repositories (local and remote) or one local backup job and rsync syncing backups to remote site.

Many users feedback that rsync is not effective as it has to read the whole VM at both ends to know the difference before it can sync it over whereby Veeam Replication can skip this, thus Veeam replication will be much faster than using rsync. True?

What is the technical challenge here that prevent Veeam from able to do both backup and replication with just 1 snapshot and 1 read on the same VM?

Nobody is talking about preference here. It's all about your backup strategy, but we like to say that backup is not a backup until you have at least 3 copies of it. It's quite a common practice to have replicas for short term retention and good RTO (immediate failover) and backups at both local and remote sites for a much longer (and less regular) retention (space-efficient data archive).

What is the reason for having at least 3 copies of backup to consider a complete backup?

For replicas at remote site, how does Veeam deal with remote subnet when restoring so that local users can still access the replicas as if these VMs appear to be original VMs running at local site?

3. If I have 2 sites that need to backup to one another, do I need to install a Veeam server at each site? Or is one Veeam server sufficient to manage backup jobs at both sites? In this environment, do I also need a proxy and repository server at each site to perform backup and replication?

Single Veeam B&R server is sufficient, but you do need to have proxy servers and repositories at each location for efficient backup and replication.[/quote]

I do not have a huge environment, just 2 ESXi hosts at HQ and 3 ESXi at DC (Data Center). If I need to backup VMs in both HQ and DC to one another, what is the bare minimum of servers I need to deploy? Are 3 sufficient as listed below or do I also need to deploy Repository Servers at both ends as well?

1 Veeam B&R at HQ
1 Proxy at HQ
1 Proxy at DC

[foggy]

Many users feedback that rsync is not effective as it has to read the whole VM at both ends to know the difference before it can sync it over whereby Veeam Replication can skip this, thus Veeam replication will be much faster than using rsync. True?

That's true, so you need to take VM size into account. Btw, here's a good reading on this.

What is the technical challenge here that prevent Veeam from able to do both backup and replication with just 1 snapshot and 1 read on the same VM?

No technical reasons actually, it's just a matter of feature priority.

What is the reason for having at least 3 copies of backup to consider a complete backup?

Just to be on the safe side! You cannot rely on a single backup, 2 copies is absolutely a must (and one of them should be offsite).

For replicas at remote site, how does Veeam deal with remote subnet when restoring so that local users can still access the replicas as if these VMs appear to be original VMs running at local site?

You can set up the re-IP rules in the replica job if you have different subnets in local and remote sites but you should configure routing between two subnets manually.

[scott_mac]

We're a small business but have a fair amount of data - 5 VM's but a total of around 3Tb of data.

I backup to a local CIFS share overnight (A Netgear ReadyNAS) and then use the built in 'Replicate' (ok, it's an add-on you have to purchase) to copy the backup to an offsite location. I believe it is pretty much a front end to rsync but it appears to work very well (i've tested restores but not yet had to rely on one!)

It does struggle a bit when reading through the changes if you use Reverse Incremental, but for forward incremental it only slows down a bit when you have the regular full backup, if you have sufficient bandwidth across your WAN though, this won't be an issue. Our WAN is only 3MBps but I am able to backup all of the servers every night and replicate them before the start of our working day with ease, due to the speed limitations I do bring the offsite unit in to work when a full backup has completed as one of the servers creates a 400+Gb file... I am currently testing the forever incremental system to see how that affects the speed.

I appreciate that this may not be suitable in Enterprise or for all, but as a relatively cost effective solution for an SMB it gives us a good level of DR for a relatively modest outlay.

[seapro220]

[merged]

Hello all.
I've started using the B&R File Copy program and have found it useful, for some processes. I do my normal B&R backups, then schedule the File Copy to copy that data to another source, just to have a 'copy' of that data that's not physically close by. What i've found is that i have to manually change the File Copy program as my 'full' B&R file names' change each week. Is it possible to include any type of variable, like (source location) *.vbk to destination x.vbk? This would make this process simplified and automated - which is what I was trying to do in the first place. It seems like such a simple ....
I'm using B&R 6.1 version.

thanks -

[foggy]

Mark, since File Copy job always performs a full backup, this is considered to be inefficient approach to having offsite copy of backups. Please review this topic for some other ways of achieving the same that allow to sync only changes.