Block size for replication

[rfn]

Hi,

We have just had two EqualLogic PS4000's installed with the intent to have one as the primary and the other as a secondary, solely for replication. The secondary is installed in a another building and there is a 2 GB fiber connection between the buildings.

It works great... there's nothing wrong with the product but our snapshots and replication is taking way more space than anticipated. We're using a Kerio Connect mailserver that stores all mails as separate files which makes is very easy to backup and restore, but it also keeps small index files in every mail folder. These files are constantly updated and that triggers a copy when ever a snapshot/replication occurs. We have about 3 GB of changed data each day, but snapshots are 2 GB each hour(!) and I'm pretty sure that it's because of these small files because EqualLogic has a block size of 64 MB and if we have 500 of these small files changing all the time and they're all on different blocks, then it's a lot data that has to me moved!

My question is if this will work better with Veeam? Does it operate with a smaller block size so that the changed amount of data corresponds better with what is changed in the filesystem?

Regards,
René Frej Nielsen

[Bunce]

I think I've seen 1MB block sizes posted before, however according to the FAQ it can be manually set in V5:
http://www.veeam.com/forums/viewtopic.p ... ize#p18664

[Vitaliy S.]

That's correct, having lots of small index files changed will produce large incrementals. Currently 1 MB block size is used to retrieve changed blocks, while with v5 you'll be able to lower the block size for changed blocks.

[Gostev]

Yep, it should be about 64 times better with Veeam v4

[rfn]

I have now checked the messagestore and there's even more small files than just the index files. They are all between 1-10 KB so doesn't really take up any space in the filesystem, but it's bad for block based replication. That's something that I wasn't aware of when we decided on the SAN solution

Oh well, I think I'll have a look at Veeam or think of some other solution, like maybe a traditional backup solution that stores its files on the secondary SAN. Do you know if the block size can be changed in version 5 for each backup job, so that for the mailserver it can be really small and for other more traditional jobs it can be the standard 1 MB?

[Vitaliy S.]

Yes, you will be able configure this parameter for each backup job separately.

[rfn]

Very interesting... I'll check out the trial once version 5 is out!

[tsightler]

I have now checked the messagestore and there's even more small files than just the index files. They are all between 1-10 KB so doesn't really take up any space in the filesystem, but it's bad for block based replication. That's something that I wasn't aware of when we decided on the SAN solution

Oh well, I think I'll have a look at Veeam or think of some other solution, like maybe a traditional backup solution that stores its files on the secondary SAN. Do you know if the block size can be changed in version 5 for each backup job, so that for the mailserver it can be really small and for other more traditional jobs it can be the standard 1 MB?

Might be better to go with a file based replication. It's the perfect job for rsync.

[rfn]

You're probably right... Does rsync exist for Windows? The problem is that Kerio recommends that nothing touches its live file system, so to be safe it would be better with some kind of snapshotting.

[tsightler]

No problem, just wrap rsync with a script mounts a shadow copy of the drive and rsync that file. Yes, rsync exist for Windows and it's still a great tool. There's "cwrsync" which is just a packaging of Cygwin and Rsync in a nice Windows installer, it works great:

http://www.itefix.no/i2/node/10650

Or, if you want a GUI with a scheduler you might use DeltaCopy:

http://www.aboutmyip.com/AboutMyXApp/DeltaCopy.jsp

It can run as a Windows service, uses the rsync algorithm, but I don't know if it can run an external script to create the shadow copy.

[rfn]

Great... Thank you!

I'll take a look at those tools and great idea of using a shadow copy.