vSphere 4.1 and USB Passthrough

[kac_pres]

Has anyone tried to connect an external USB hard drive (1TB or larger) to a host and use USB passthrough to connect it to a Veeam VM. From there do backups to the locally attached USB drive?

I think this would be a rock-solid feature. In case of a fire, run in there, grab the drive, and run out.

[Gostev]

I was looking forward to vSphere 4.1 because of this feature alone (I knew about it since last November, it was really hard not to share with anyone because I was so excited).

Vitaly has been configuring and testing this all day on the RTM vSphere 4.1 code, he will post an update soon along with any required information.

[Vitaliy S.]

Hi Kendrick,

I've just finished testing this feature, and I can confirm that this kind of setup works perfectly with ESX(i) 4.1. Unfortunately, I wasn't able to test the performance as the only device I had was a pretty slow USB stick.

Here is the VMware's KB article describing how to add USB device to a VM on ESX(i) 4.1:
http://kb.vmware.com/kb/1022290

Thanks!

[bwestover]

This is a very exciting feature indeed!

I have been waiting for USB Passthrough support on ESX for years!

One use case we might have is easy offsite storage. We would like to hook up a USB hard drive to the host that the Veeam VM is on, and backup our critical VMs to it. Then swap it out for another, and take the original offsite.

We do this now, but the hard disk is on another physical machine so the backups have to be transferred over the network.

[Frosty]

I would love to know whether this gives decent USB performance (i.e. USB 2.0, not USB 1.1). Are you using VMDirectPath to attach the USB devices in the VM? I'm thinking about trying to do the same sort of thing, but with External SATA via VMDirectPath ... just waiting on my new ESX hosts and storage to arrive so I can try it out.

[kac_pres]

Frosty, I can tell you right now that eSATA won't work. I agree that eSATA would be a kick @$$ feature for something like this because speeds would be awesome, but eSATA is not supported for VMDirectPath. At the current moment, only NICS and FiberChannel HBAs are supported for VMDirectPath.

[Exrace]

Don't forget about RDX drives. I have a setup using RDX drive as a mounted device in esx and it works fine.
I tried getting a RDX drive to work in passthrough in ESXi 4.0 but it didn't work...wonder if 4.1 has support for that now?

[Frosty]

Frosty, I can tell you right now that eSATA won't work. I agree that eSATA would be a kick @$$ feature for something like this because speeds would be awesome, but eSATA is not supported for VMDirectPath. At the current moment, only NICS and FiberChannel HBAs are supported for VMDirectPath.

Damn. Although I note also that there is SAS support, not just FC; but that probably doesn't help me much! I suppose there is the faint possibility that we could come up with something which works even though its not supported ...

[bctirado]

I have this setup and configured, performance feels very much like USB 1 not 2. New bases took me 3 days to complete vs the normal 6ish hours, incrementals take 12 hours vs the normal 1 hour.

My setup-
Veeam installed on a 2008r2 VM on a 4.1 ESX host
Configured to not backup up itself but instead a seperate ESX host running ESX 4u2

I should also note that a direct file transfer hovers around 6.6mb on throughput for large files.

Anyone else having transfer rate pains with the new passthrough?

Brian

[Vitaliy S.]

Hi Brian,

I believe you might get much better results by using fast USB drives instead of USB sticks. If you have any, please try. It would be interesing to see what will be the results. Thanks!

[bctirado]

Hi Vitaliy

I'm actually using a usb hard drive, similiar to what we use for a particular type of backup. On a physical box the same device performs at USB 2 speeds and drops the backup times to what is listed above.

Brian

[TrevorBell]

Just having a play with this also backing up to a USB drive i have ordered a usb 2.0 for tomorrow so will post some speeds.

Trev

[TrevorBell]

I have done a quick test with 2 production vm`s both were 7 mb/s next pass was at only 8 mb/s so not very fast but ill try tomorrow on brand new 1tb drive

trev

[TrevorBell]

Update :

I have now attached a USB 2.0 device to the ESX 4.1 server and here are the results-:

first VM 17 mb/s ( Network Mode ) second VM 19mb/s ( SAN Mode )

second pass

first VM 132 mb/s ( Network Mode ) second VM 163 mb/s ( SAN Mode )

The two VM`s are the same as yesterdays test i can only assume poor 7-8 mb/s yesterday was due to USB 1 device even CBT backups made little difference.But now with 2.0 USB device im getting very fast speeds, the Vm`s have little data change so the figures might be misleading but a good test for people thinking of going 4.1 and attaching usb devices directly to the ESX host for backups

hope that helps

Trev

** Please wait till Veeam 4.1.2 is released before you try this as there is still issues running 4.1.1 with ESX 4.1 **

[Felix]

As others already wrote, vSphere USB Passthrough performance seems limited to about 6-7 MBytes.

I've done my own testing with HD Tune Pro 3.50 and a 500GB WD Elements USB 2.0 drive:
Physical Host, Windows 7 x64: 33/23 MByte/s R/W
VMware Fusion 3.1.3, Mac OS X 10.6.4 x86_64: 23/17 MByte/s R/W
vSphere ESXi 4.1.0, Windows XP x64 SP2 VM: 7/6 MByte/s R/W

Hardware used for testing was a 2009 Nehalem Mac Pro 8x2.26GHz Xeon E5520 w/HT and 16GB DDR3-ECC. Additionally I reverified the vSphere performance numbers on a whitebox system, Core 2 Duo E6400, 2.4GHz, 4GB DDR2 and got the same meager 6-7 MByte/s.

I also compared USB passthrough performance between WIndows 7 x64 and Windows XP Pro SP3 x86 on VMware Fusion, but saw little difference in performance. I've not yet tested with Win7 or W2K8 on vSphere, but I expect similar results.
If vMotion compatibility is enabled, datarates will be slightly lower and you'll see much higher cpu lood than in non-vMotion mode. A single VM seems to peg one core at 100% during benchmark in vMotion mode on the Core 2 Duo system, while non-vMotion was somewhere around 8% CPU load on that box.

@TrevorBell: I can't quite understand your claims of USB 1.0 vs. USB 2.0 device: An USB 1.1 device maxes out at 12 MBit/s, so you'd see maybe 1.5 MByte/s instead of 7-8 MByte/s. The numbers you posted above under "Update:" can probably be attributed to compression in the first run and to little change + changed block tracking in second pass. So I don't think you're getting more than 6-7 MByte/s throughput fromt he USB device. You could check with HD Tune Pro's Benchmark or Disk Monitor during backup, which shows troughput and IOPS.

Felix

[Bunce]

If Trev was going by the Veaam job stats, they don't reflect actual transfer rates.

Those stats are basically: Entire VM Size / Time taken.

Given the entire size of a VM is hardly ever backed up, they're not really useful.

What I'm hoping for in V5 to get an actual, real-life transfer rate is something like:

(
- TOTAL Size of VM Files -> LESS
- Zero's blocks not backed up -> LESS
- Blocks not backed up due to compression -> LESS
- <Any other nifty technology which reduces the amount backed up> -> PLUS
- Any overhead traffic required (eg for replication)
)
DIVIDED BY
(
- Total JobTime LESS
- Time taken to prepare job LESS
- Tme taken to take and remove snapshots
)

The actual numers of each of these items would be nice to see as well as the metrics of what makes up the VBK/VBR files (so we can report what we're transferring / saving to management etc)

[Gostev]

This will not be in v5 for sure.

[TrevorBell]

Felix,

i didnt claim anything just posted a couple of tests i did..

[ideo-fdufour]

I'm evaluating Veeam Backup & Replication, so that's my first step with the product, and wondering if my backup job setup is maximize...

Setup:
vSphere Essentials and Veeam Essentials.
Veeam installed in a VM on the ESX server.

Backup job:
Virtual Appliance
VM to backup: Exchange 130 Gig (2 vHD)
Destination: USB drive on Veeam VM (Z:\VeeamBackup\Exchange)
Backup Mode: Reversed incremental
Enable application-aware image processing (enabled)

I got poor processing rate (3MB/s) took more than 11 hour to backup the 130 Gig VM

Is there something wrong with my backup job setup ?

Any input will be much appreciated!!

Regards,
Fred

[Vitaliy S.]

Hello Fred,

I'd say it is somehow expected with USB Passthrough enabled, please review this topic for more information. If I were you I would change your destination target to any network share/linux box/DAS to get higher performance rates.

Thanks!

[FMFB]

Does ESXi 4.0 supports following device:
500GB WD Elements USB 2.0 drive:
thanx

[Vitaliy S.]

You can check the hardware matrix on VMware site for supported devices, but I'm pretty sure it should work.