Discussions specific to the VMware vSphere hypervisor
Post Reply
lars@norstat.no
Expert
Posts: 109
Liked: 14 times
Joined: Nov 01, 2011 1:44 pm
Full Name: Lars Skjønberg
Contact:

Storage Optimization for local replication

Post by lars@norstat.no » Nov 01, 2013 1:45 am

Is there some speed gain in changing Storage optimization from local target to Local Target 16TB+ when using Direct SAN -> NBD ?

I know that i change the datablocks from 1024 to 8192 and it seems to me that this gives some gain in speed, but i have no conclusive evidence for it ....

foggy
Veeam Software
Posts: 18264
Liked: 1561 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Storage Optimization for local replication

Post by foggy » Nov 01, 2013 10:13 am

This could result in less processing overhead due to smaller number of blocks to process, however I'm not sure this will be a noticeable improvement in case of local replication. Storage optimization settings are more relevant for WAN replication, where using a smaller block size will result in significantly less traffic across the link.

Gostev
SVP, Product Management
Posts: 24793
Liked: 3536 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Storage Optimization for local replication

Post by Gostev » Nov 01, 2013 3:35 pm

The performance may increase slightly in case of local replication. However, the drawback of using this setting is a few times bigger increments (restore points).

lars@norstat.no
Expert
Posts: 109
Liked: 14 times
Joined: Nov 01, 2011 1:44 pm
Full Name: Lars Skjønberg
Contact:

Re: Storage Optimization for local replication

Post by lars@norstat.no » Nov 01, 2013 4:50 pm

A few times bigger ? So a snapshot / restore point that would be 5 GB today would be 10, 15, 40 GB ? Or is this something that is hard to really calculate ?

tsightler
VP, Product Management
Posts: 5420
Liked: 2243 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Storage Optimization for local replication

Post by tsightler » Nov 01, 2013 6:47 pm

It's hard to calculate, but the point is that the block size defines the smallest amount of data that will be moved even a single byte is changed within that block. If your changes are all clumped in a small area of the disk it might not make much difference, if they're spread over a larger area (think Exchange/SQL), then it's likely to increase significantly, say 4-8x (it's an 8x larger block size, so 8x is probably the top end).

lars@norstat.no
Expert
Posts: 109
Liked: 14 times
Joined: Nov 01, 2011 1:44 pm
Full Name: Lars Skjønberg
Contact:

Re: Storage Optimization for local replication

Post by lars@norstat.no » Nov 01, 2013 11:17 pm

Hmm, so then the best thing would be to use different block size for different servers like small for SQL/Exchange servers and large for file/app servers ...

Will experiment a little with this then ...

lars@norstat.no
Expert
Posts: 109
Liked: 14 times
Joined: Nov 01, 2011 1:44 pm
Full Name: Lars Skjønberg
Contact:

Re: Storage Optimization for local replication

Post by lars@norstat.no » Nov 05, 2013 6:09 pm

What i have found really surprised me. Using the 16TB+ setting reduced the time for replication some and on my test machine completed in around 4 minutes, down from 5 minutes.

But when i choose WAN optimization the changed blocks where so few i guess that the replication time was reduced to only 3 minutes. I really can't see the significant processing overhead anywhere. Guess that would be on the Proxy, but i can't really see what would cause the overhead ? The Proxy is reading directly from the SAN via Fiberchannel ... Could that have anything to do with it ?

Maybe the processing overhead only occurs on iSCSI ?

Gostev
SVP, Product Management
Posts: 24793
Liked: 3536 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Storage Optimization for local replication

Post by Gostev » Nov 05, 2013 7:27 pm

You cannot draw any conclusions based on a single test with so little changes (just a few minutes worth). This is similar to making any conclusions on something by polling just 1 random guy :) any small fluctuation in the environment will affect the results of your testing.

Try daily replicating some real VM with two jobs having different block sizes for a week, using the same processing modes. And look not only at times, but the amount of data read by incremental runs as well (those should be noticeably bigger with larger block size).

Also, keep in mind that the results will differ significantly depending on application running in the test VM. In terms of performance, applications making a few big, consolidated changes to the disk will benefit from large block sizes (for example, archive file servers where the files are mostly added, and rarely/never modified). Application updating small chunks of data in files all over the disk (for example, Exchange and SQL) may benefit from smaller block size in some cases... unless they are really busy, and create huge amount of daily changes (like 10% or more of the entire disk) - in which case again large block size may play better for performace (but not the incremental data size).

There is only one thing you can be sure of - smaller block size will always give you smaller incremental backup sizes. Processing time is a different question, but normally smaller blocks means slower processing for local replication (more block to process for the engine), and faster processing for replication over WAN (when WAN bandwidth is the primary bottleneck).

Post Reply

Who is online

Users browsing this forum: tcranger and 24 guests