Comprehensive data protection for all workloads
Post Reply
rfn
Expert
Posts: 141
Liked: 5 times
Joined: Jan 27, 2010 9:43 am
Full Name: René Frej Nielsen
Contact:

Block size for replication

Post by rfn »

Hi,

We have just had two EqualLogic PS4000's installed with the intent to have one as the primary and the other as a secondary, solely for replication. The secondary is installed in a another building and there is a 2 GB fiber connection between the buildings.

It works great... there's nothing wrong with the product but our snapshots and replication is taking way more space than anticipated. We're using a Kerio Connect mailserver that stores all mails as separate files which makes is very easy to backup and restore, but it also keeps small index files in every mail folder. These files are constantly updated and that triggers a copy when ever a snapshot/replication occurs. We have about 3 GB of changed data each day, but snapshots are 2 GB each hour(!) and I'm pretty sure that it's because of these small files because EqualLogic has a block size of 64 MB and if we have 500 of these small files changing all the time and they're all on different blocks, then it's a lot data that has to me moved!

My question is if this will work better with Veeam? Does it operate with a smaller block size so that the changed amount of data corresponds better with what is changed in the filesystem?

Regards,
René Frej Nielsen
Bunce
Veteran
Posts: 259
Liked: 8 times
Joined: Sep 18, 2009 9:56 am
Full Name: Andrew
Location: Adelaide, Australia
Contact:

Re: Block size for replication

Post by Bunce »

I think I've seen 1MB block sizes posted before, however according to the FAQ it can be manually set in V5:
http://www.veeam.com/forums/viewtopic.p ... ize#p18664
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Block size for replication

Post by Vitaliy S. »

That's correct, having lots of small index files changed will produce large incrementals. Currently 1 MB block size is used to retrieve changed blocks, while with v5 you'll be able to lower the block size for changed blocks.
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Block size for replication

Post by Gostev »

Yep, it should be about 64 times better with Veeam v4 ;)
rfn
Expert
Posts: 141
Liked: 5 times
Joined: Jan 27, 2010 9:43 am
Full Name: René Frej Nielsen
Contact:

Re: Block size for replication

Post by rfn »

I have now checked the messagestore and there's even more small files than just the index files. They are all between 1-10 KB so doesn't really take up any space in the filesystem, but it's bad for block based replication. That's something that I wasn't aware of when we decided on the SAN solution :shock:

Oh well, I think I'll have a look at Veeam or think of some other solution, like maybe a traditional backup solution that stores its files on the secondary SAN. Do you know if the block size can be changed in version 5 for each backup job, so that for the mailserver it can be really small and for other more traditional jobs it can be the standard 1 MB?
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Block size for replication

Post by Vitaliy S. »

Yes, you will be able configure this parameter for each backup job separately.
rfn
Expert
Posts: 141
Liked: 5 times
Joined: Jan 27, 2010 9:43 am
Full Name: René Frej Nielsen
Contact:

Re: Block size for replication

Post by rfn »

Very interesting... I'll check out the trial once version 5 is out!
tsightler
VP, Product Management
Posts: 6011
Liked: 2843 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Block size for replication

Post by tsightler »

rfn wrote:I have now checked the messagestore and there's even more small files than just the index files. They are all between 1-10 KB so doesn't really take up any space in the filesystem, but it's bad for block based replication. That's something that I wasn't aware of when we decided on the SAN solution :shock:

Oh well, I think I'll have a look at Veeam or think of some other solution, like maybe a traditional backup solution that stores its files on the secondary SAN. Do you know if the block size can be changed in version 5 for each backup job, so that for the mailserver it can be really small and for other more traditional jobs it can be the standard 1 MB?
Might be better to go with a file based replication. It's the perfect job for rsync.
rfn
Expert
Posts: 141
Liked: 5 times
Joined: Jan 27, 2010 9:43 am
Full Name: René Frej Nielsen
Contact:

Re: Block size for replication

Post by rfn »

You're probably right... Does rsync exist for Windows? The problem is that Kerio recommends that nothing touches its live file system, so to be safe it would be better with some kind of snapshotting.
tsightler
VP, Product Management
Posts: 6011
Liked: 2843 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Block size for replication

Post by tsightler »

No problem, just wrap rsync with a script mounts a shadow copy of the drive and rsync that file. Yes, rsync exist for Windows and it's still a great tool. There's "cwrsync" which is just a packaging of Cygwin and Rsync in a nice Windows installer, it works great:

http://www.itefix.no/i2/node/10650

Or, if you want a GUI with a scheduler you might use DeltaCopy:

http://www.aboutmyip.com/AboutMyXApp/DeltaCopy.jsp

It can run as a Windows service, uses the rsync algorithm, but I don't know if it can run an external script to create the shadow copy.
rfn
Expert
Posts: 141
Liked: 5 times
Joined: Jan 27, 2010 9:43 am
Full Name: René Frej Nielsen
Contact:

Re: Block size for replication

Post by rfn »

Great... Thank you!

I'll take a look at those tools and great idea of using a shadow copy.
Post Reply

Who is online

Users browsing this forum: No registered users and 254 guests