quick performance test btw Hot add and Network

[bdufour]

quick test on a lighter VM of mine, as this is a common question, which route to go on the proxy side - hot add/nbd. we use production VMs as proxies, to save on cost, and this has proven to work well for us. so we backup and replicate these VMs and with NBD it makes that much easier. also, the consolidation issues have been non existent while on NBD - which is obvious.

10ge virtual backbone network.
aggregated (LACP) to Data Domain dedup with DDboost - aggregated over 1ge - NICs arent 10ge. although, our next appliance we will go the 10ge route, as our 10ge network is relatively new.


Using backup proxy XX for disk Hard disk 1 [hotadd]
Hard disk 1 (80.0 GB) 6.7 GB read at 147 MB/s [CBT]
Using backup proxy XX for disk Hard disk 2 [hotadd]
Hard disk 2 (200.0 GB) 1.1 GB read at 150 MB/s [CBT]

Using backup proxy XX for disk Hard disk 1 [nbd]
Getting list of guest file system local users
Hard disk 1 (80.0 GB) 6.2 GB read at 135 MB/s [CBT]
Using backup proxy XX for disk Hard disk 2 [nbd]
Hard disk 2 (200.0 GB) 1004.0 MB read at 148 MB/s [CBT]

[DGrinev]

Hi Blake,

Can you share the bottleneck stats for both tests. Thanks!

[bdufour]

sure!
target on both, which makes sense according to my judgment.

Using backup proxy XX for disk Hard disk 1 [hotadd] 00:28
Hard disk 1 (80.0 GB) 6.7 GB read at 147 MB/s [CBT]00:48
Using backup proxy XX for disk Hard disk 2 [hotadd] 00:25
Hard disk 2 (200.0 GB) 1.1 GB read at 150 MB/s [CBT]00:11
Removing VM snapshot 00:07
Network traffic will be encrypted
Saving GuestIndexData.zip 00:02
Saving GuestMembers.xml 00:00
Publishing guest file system index to catalog
Finalizing 00:10
Truncating transaction logs 00:02
Truncating SQL server transaction logs 00:00
Swap file blocks skipped: 176.0 MB
Deleted file blocks skipped: 328.0 MB
Busy: Source 8% > Proxy 23% > Network 15% > Target 86%
Primary bottleneck: Target


Using backup proxy XX for disk Hard disk 1 [nbd] 00:00
Getting list of guest file system local users 00:00
Hard disk 1 (80.0 GB) 6.2 GB read at 135 MB/s [CBT]00:51
Using backup proxy XX for disk Hard disk 2 [nbd] 00:00
Hard disk 2 (200.0 GB) 1004.0 MB read at 148 MB/s [CBT]00:13
Removing VM snapshot 00:06
Network traffic will be encrypted
Saving GuestIndexData.zip 00:01
Saving GuestMembers.xml 00:00
Publishing guest file system index to catalog
Finalizing 00:09
Truncating transaction logs 00:01
Truncating SQL server transaction logs 00:00
Swap file blocks skipped: 188.0 MB
Deleted file blocks skipped: 180.0 MB
Busy: Source 56% > Proxy 24% > Network 22% > Target 82%
Primary bottleneck: Target

[foggy]

On fast networks NBD can be faster due to no overhead on attaching/detaching disks.

[bdufour]

yes foggy, ive read that nbd is optimal with 10ge. although, i ran nbd on a full 1ge network, and it was very similar in performance to hot add over 1ge if my memory serves my correctly. i dont have those results unfortunately.

[foggy]

This is also possible - too many variables, for example hotadd operation could take longer in that environment making hotadd transport overall slower and comparable to NBD in terms of job duration.

[bdufour]

indeed variables will never be constant among different networks. i thinks its worth testing in individual environments to understand the best course of action for an admin. the choice, for me, was based on the production proxies needing to be backed up and replicated, so nbd was the clear choice - then i noticed very similar performance to hot add. thanks for the input.