Backup with Rsync

[n2it-dk]

Hi, ive Got a job to do. Ill try and define below and I need some clarifications and help to achieve this the best possible way.

The scenario goes like this.

There are huge amounts of data Thaysen need to be backed op Daily and this data should be located in backupdestination 1. The data in backupdestination 1 should allso be backed up to backupdestination 2 in a remote site.

Doing a regular backup to backupdestination 1 and then do a filecopy of backupdestination 1 to backupdestination 2 isnt the way to go, since I cant enable cbt on the filecopy and I need this for the dataload.

Im thinking of two ways of doing this the first is to make a replica to backupdestination 1 and then do a backup with cbt og ba lupdestination 1 to backupdestination 2. (Im not sure this Will work)

Second solution would be to male regular backup to backupdestination 1 with cbt enabled and then have a rsync making backup of the vbk files in backupdestination 1 to backupdestination 2 and enable the switch in rsync to only backup the changes in the vbk file, so that the backupwindpw is kept down in time. (I dont know how to enable and configure this)

The target backup destination is iSCSI mounted As LUN's in the vmware servers and allso the servers are a mix between ms, Linux and novell.

Data to be backed up is huge, counting several/Many TB's

I kind of think this should be enabled running rsync in a Linux server and veeam is running in a Windows server. Just need some help to get this working.

Please let me know your thoughts and ideas.

Regards

[Gostev]

I would go with second option (rsync) as I know this is what our customers are successfully using today, and it is much better traffic and speed wise, as replication would have to deal with uncompressed VMDK data, while in this scenario we are talking about highly compressed backup files. Just search for "rsync" here on this forum for more information on how they have this set up.

Here's one helpful thread with some tips and tricks on using rsync for instance: off-site backup ideas

[n2it-dk]

Hi Gostev, yes thanks for your reply. Ive seen this post you regere to, and what tsigthler is writing about using rsync from linux is actually what I need, so Tom could you specify how you have serup this?

Ps. Im writing this on my iPhone, so sorry for any misspells or hard to understand questions. 

[larry]

I just ran in to this, I was using Rsync, switch to have a remote veeam server backup over my 100 mb wan. Doing the second backup is taking half the time on the first pass then hours less on each other pass. Just be relly sure not to do two backups to the same VM at once. I have 6 100 gig VMs I am doing each night now. With Rsync, rsyn needs to compare checksums of every block every night - Veeam just check CB and flys. I been testing only for a week now with full VM and Veeam has been way past better. I also find replication very fast too, much faster than Rsync after first pass. I backup and replicate both ways every nigh with full ones on Sat and Sun Here is a cut and paste from my log you will see now it takes just a few minutes compared to first. Rsync took 6 hours to do just the 165 gig one.
8 of 8 files processed

Total VM size: 35.00 GB
Processed size: 35.00 GB
Processing rate: 238 MB/s
Backup mode: SAN/NBD with changed block tracking
Start time: 5/14/2010 9:50:18 AM
End time: 5/14/2010 9:52:48 AM
Duration: 0:02:30
8 of 8 files processed

Total VM size: 35.00 GB
Processed size: 35.00 GB
Processing rate: 9 MB/s
Backup mode: SAN/NBD with changed block tracking
Start time: 5/14/2010 1:00:48 AM
End time: 5/14/2010 2:06:23 AM
Duration: 1:05:35
10 of 10 files processed

Total VM size: 165.00 GB
Processed size: 165.00 GB
Processing rate: 11 MB/s
Backup mode: SAN/NBD with changed block tracking
Start time: 5/14/2010 2:08:30 AM
End time: 5/14/2010 6:23:29 AM
Duration: 4:14:58

10 of 10 files processed

Total VM size: 165.00 GB
Processed size: 165.00 GB
Processing rate: 11 MB/s
Backup mode: SAN/NBD with changed block tracking
Start time: 5/14/2010 2:08:30 AM
End time: 5/14/2010 6:23:29 AM
Duration: 4:14:58

over 100 mb wan

[n2it-dk]

Larry, could you clearify how you do this? Are you running Veeam at both sites? Or Veeam and Rsync? Allso how are your infrastructure? What kind of servers and allso what kind of storage?

Thanks

[larry]

I have 2 veeam servers. One at each end. They are windows 2008 sp2 X64 4 gig ram, 2 256 gig raid 1 and 6 1 gig sata disks on an intel raid. everything is fully patched and all updates.I have backupexec 2010 ( single server version ) loaded on each server also. I backup the veeam backups which are just local files to tape. I have ESX vspear servers with lefthand iscsi sans at each site. The veeam servers have two 1 gig nics, one on the iscsi network and one on production.

My old setup used the same hardware but backupexec with vcb and rsync off site. Backup windows was 12 hours, now 2 hours. At the end of the night now I have 3 copies plus tape before I 2 copies.

Veeam is beating my SAN in off site backup.

my day is over so any more info you need I will get to you on Monday

[n2it-dk]

So, Larry.. From what I can read from above, you backup same server(s) with two different Veeam installations. If Im right, I didnt know that this were actually verified as working, kind of think there might be issues with the ctk files that Veeam uses to keep track records.

Please enlighten me.

[n2it-dk]

Think I found a way around this, with powershell and some workarounds. Still if anyone is having some good samples of how to copy vbk files from iSCSI Luns to another iSCSI Lun, that would still be great.