Sync Master Backups to Wan in Block Level

[nehalem]

Good morning all,

I am doing a backup with reverse incremental policy i am having files .VBK and .VRB in the backup repository folder of my primary Veeam Server (NTFS Volume)
In each cycle of the backup job, i assume that the master VBK is changed, and also some new VRBs are created. So far so good.

So my scenario is to replicate/copy these files to a remote Veeam server behind a WAN, to be available for import and restoration
I am reading on this forum that using Rsync is the best method to sync these files because of its availability to make incremental changes to the files without the need
of re-copy all the file.

My question is which options do i have to use to make it in block level mode sync, and also Rsync can make block level sync while source an target is NTFS (Windows)

Is there any tested / working command to do this between 2 Windows folders?

The command i am using these options. But every time a file changes (eg. VBK of reversed incremental backup) i assume that it copies everything from scratch

rsync -rav --progress --inplace /cygdrive/d/FileCenter/ //san/dskd/backup/filecenter/ -verbose --delete --stats

In order to see if it is really working in this demo command the target is local machine (1Gbps), not the really WAN machine.

Thanks

[tsightler]

Normally, when running locally, rsync reverts to basically being the same as a copy, because if it didn't it would actually probably be worse than a copy. Rsync has two basic features, one, to scan the directories being copied and only copy files that have changed, that still works even on local copies (so there can still be advantages to using rsync locally), but the second feature is to minimize the network bandwidth used to copy the files. To do this rsync basically reads the file on both ends, hashes the results, and then sends the bytes that are different. Actually, when using rsync without the "--inplace" option, the target file has to be both read and written (it writes the "new" file to a hidden file and, only once the file is complete does it delete the original and replace it with the new copy).

In other words, rsync is basically aimed at reducing network bandwidth but it does this by using disk bandwidth, and to do this well it really needs an rsync process on both ends (either rsync via ssh, or an rsyncd process running on one end).

[chad5k1]

I've been having real trouble with rsync recently. Does anyone have any other recommendations that run on Windows?
I've been using deltacopy - which is basically a port of rsync to run in windows.
I have a 100mb wan connection and about 300gb to move every night.
Thoughts appreciated.

[gwhite]

[merged]

Question for the Veeam masses...but first some background info.

Our backups currently total around 2.5TB, but will likely be larger. We run reverse incremental backups and have applied the registry key to stop the v5 .vbk filename timestamping.

We have two FreeNAS servers set up for our backup. The primary NAS has a SAMBA share that Veeam Backup writes data to. We then set up the rsync server on the primary NAS and the rsync client on the secondary NAS.

Day one we let the rsync client pull all the data from the primary to the secondary NAS...took a while, but the sync finished successfully. The next day after the backup ran, the rsync job ran...and ran...and ran. It looked like what it was doing was scanning each side for differences...like it's supposed to. Problem is 12 hours later it hadn't done anything as far as copying any files/data.

I've seen posts on here that people are using rsync with good results. Could you please share with me what your rsync set up and configs look like on the server and client side (or however you have it set up)?

We would like to take advantage of the delta-copy feature in rsync, but if there is a better solution out there for moving files this large incrementally, I would appreciate any info you can share.

Thank you!

[Vitaliy S.]

Hello Greg,

Here's one helpful thread with some rsync script examples, might help. Thanks.

[A3_h]

I use the following command rsync.exe -rlt -P --delete --progress --inplace --stats "/cygdrive/D/Backups" "ms098::VeeamBackup/"
And yes i takes a lot of time to "check" the vm's on changes. Especialy the exchange wich is 300Gb takes a couple of hours.. But everday the differentail backup is 50Gb..
So i am also interested i speedup tip because we have to sync about 100Gb a day over 20Mbit line. (theoretically capable for about 209GB a day) because my backup window will be shorter because of the checkin change blocks feature of rsync.

Show examples of rsync configs will be appreciated...