Comprehensive data protection for all workloads
TrevorBell
Expert
Posts: 353
Liked: 7 times
Joined: Feb 13, 2009 10:13 am
Full Name: Trevor Bell
Location: Worcester UK
Contact:

vStorage API in Virtual Appliance mode

Post by TrevorBell » Sep 28, 2009 5:40 pm

[UPDATE 1/11/2010] As of ESX4 Update1, vStorage API Virtual Appliance processing mode no longer requires that you have VMware Advanced or later license. In Update 1, VMware had fixed what appeared to be the SCSI Hot Add feature licensing glitch, so now you can enjoy all benefits for Virtual Appliance mode with any VMware vSphere license level, including VMware Essentials and Standard.

What sort of setup is required for this just a VM runnign VB4 with iscsi san lun for the images ?

Gostev
SVP, Product Management
Posts: 24612
Liked: 3463 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: vStorage API in Virtual Applicance mode

Post by Gostev » Sep 28, 2009 6:15 pm

Trevor, the beauty is - no setup is needed. You just install Veeam Backup in a VM, and select this backup mode - that's it. And Veeam will be able to do direct-from-SAN backups through ESX I/O stack (instead of network stack). Speed should be quite similar to SAN mode (per VMware), and you will be able to backup all VMs located on the same shared storage as Veeam Backup VM (but not from other storages which are not connected to ESX on which Veeam Backup VM runs).

But please don't bother with this right now, because we have confirmed that this mode is indeed not working properly in BETA1 - we will try to fix this by BETA2.

TrevorBell
Expert
Posts: 353
Liked: 7 times
Joined: Feb 13, 2009 10:13 am
Full Name: Trevor Bell
Location: Worcester UK
Contact:

Re: vStorage API in Virtual Applicance mode

Post by TrevorBell » Sep 28, 2009 6:22 pm

oki keep me posted when you need a test doing... ill gt a vm ready with alot of space to do a good test when needed..

Gostev
SVP, Product Management
Posts: 24612
Liked: 3463 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: vStorage API in Virtual Applicance mode

Post by Gostev » Oct 14, 2009 10:24 pm

I will really appreciate if someone can do some testing in Virtual Appliance mode with fast SAN storage, and compare performance with direct SAN access mode. Due to our SAN currently having performance issues, it is really hard to compare performance between different modes, as they all produce equally low speed :(

Thanks in advance! :D

tsightler
VP, Product Management
Posts: 5399
Liked: 2229 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: vStorage API in Virtual Applicance mode

Post by tsightler » Oct 15, 2009 12:56 am

I plan to test this, but due to time constraints with real work getting in the way I probably won't be able to get to this until Friday afternoon.

dkeathii
Enthusiast
Posts: 28
Liked: never
Joined: May 07, 2009 3:46 pm
Full Name: daniel
Contact:

Re: vStorage API in Virtual Applicance mode

Post by dkeathii » Oct 15, 2009 5:14 am

I'll do some local testing, but my san isn't really a speed demon.

dkeathii
Enthusiast
Posts: 28
Liked: never
Joined: May 07, 2009 3:46 pm
Full Name: daniel
Contact:

Re: vStorage API in Virtual Applicance mode

Post by dkeathii » Oct 15, 2009 6:14 am

I fired it off first from the vc server(physical box outside the VMs) using san mode, it switched to san/nbd and from the network usage looked like it was pulling one through the san interface and the other through the lan interface. I was seeing about 40MB/s. Both source and destination were on the same SAN, different LUNs.

I installed veeam rep. on a guest VM and tried the same job. Same SAN as the first test, same situation (Guests on same SAN, different LUNS)
I got some funkyness inside task manager, and the job failed. See screenshots below...or not, I don't see how to attach anything. I'll email it to you.

This won't cause me any issues as I intend to continue using the VC server, but since you asked... :)

TrevorBell
Expert
Posts: 353
Liked: 7 times
Joined: Feb 13, 2009 10:13 am
Full Name: Trevor Bell
Location: Worcester UK
Contact:

Re: vStorage API in Virtual Applicance mode

Post by TrevorBell » Oct 15, 2009 8:23 am

Ill have a play with this today not much else to do :oops:

Gostev
SVP, Product Management
Posts: 24612
Liked: 3463 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: vStorage API in Virtual Applicance mode

Post by Gostev » Oct 15, 2009 10:00 am

Daniel, the screenshots look really weird indeed... could you please send me the logs, we would like to investigate.

TrevorBell
Expert
Posts: 353
Liked: 7 times
Joined: Feb 13, 2009 10:13 am
Full Name: Trevor Bell
Location: Worcester UK
Contact:

Re: vStorage API in Virtual Applicance mode

Post by TrevorBell » Oct 15, 2009 10:51 am

First impressions :

I made a New vm win 2008 32bit, 40 gig c drive, picked one VM and backed it up to the 40g drive was getting 41Mb/s not too bad....
Now i have made a 100gig LUN off the SAN and doing some more testing i have to say the speeds are pretty good. First set of figures are normal san backup mode followed by the Virtual appliance mode.


San Mode Backups
CSDOM01
6 of 6 files processed
Total VM size: 24.00 GB
Processed size: 24.00 GB
Processing rate: 74 MB/s
Backup mode: SAN/NBD with changed block tracking
Start time: 10/14/2009 7:50:50 PM
End time: 10/14/2009 7:56:20 PM
Duration: 00:05:29

CSPRT01
6 of 6 files processed
Total VM size: 24.00 GB
Processed size: 24.00 GB
Processing rate: 62 MB/s
Backup mode: SAN/NBD with changed block tracking
Start time: 10/14/2009 7:44:14 PM
End time: 10/14/2009 7:50:50 PM
Duration: 00:06:35

CSPDL01
6 of 6 files processed
Total VM size: 24.00 GB
Processed size: 24.00 GB
Processing rate: 36 MB/s
Backup mode: SAN/NBD with changed block tracking
Start time: 10/14/2009 7:32:51 PM
End time: 10/14/2009 7:44:14 PM
Duration: 00:11:22

Now for New Virtual Mode
CSPRT01
6 of 6 files processed
Total VM size: 24.00 GB
Processed size: 24.00 GB
Processing rate: 79 MB/s
Backup mode: HOTADD with changed block tracking
Start time: 15/10/2009 11:14:41
End time: 15/10/2009 11:19:52
Duration: 00:05:10

CSPDL01
6 of 6 files processed
Total VM size: 24.00 GB
Processed size: 24.00 GB
Processing rate: 75 MB/s
Backup mode: HOTADD with changed block tracking
Start time: 15/10/2009 11:09:14
End time: 15/10/2009 11:14:41
Duration: 00:05:27

CSDOM01
6 of 6 files processed
Total VM size: 24.00 GB
Processed size: 24.00 GB
Processing rate: 95 MB/s
Backup mode: HOTADD with changed block tracking
Start time: 15/10/2009 11:19:52
End time: 15/10/2009 11:24:10
Duration: 00:04:17

TrevorBell
Expert
Posts: 353
Liked: 7 times
Joined: Feb 13, 2009 10:13 am
Full Name: Trevor Bell
Location: Worcester UK
Contact:

Re: vStorage API in Virtual Applicance mode

Post by TrevorBell » Oct 15, 2009 10:59 am

All figures are LIVE PRODUCTION network not a test envoriment :shock:

Here is our Terminal Server VM only about 2-3 people using it but here are the figures -:

6 of 6 files processed
Total VM size: 24.00 GB
Processed size: 24.00 GB
Processing rate: 23 MB/s
Backup mode: SAN/NBD with changed block tracking
Start time: 10/14/2009 7:56:21 PM
End time: 10/14/2009 8:14:01 PM
Duration: 00:17:40

6 of 6 files processed
Total VM size: 24.00 GB
Processed size: 24.00 GB
Processing rate: 31 MB/s
Backup mode: HOTADD with changed block tracking
Start time: 15/10/2009 11:24:11
End time: 15/10/2009 11:37:24
Duration: 00:13:13

Gostev
SVP, Product Management
Posts: 24612
Liked: 3463 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: vStorage API in Virtual Applicance mode

Post by Gostev » Oct 15, 2009 11:06 am

Trevor, thank you for testing this. Most important thing for me is that speed is comparable with SAN mode, and even a bit faster. If there was an issue I suspected, speed would be 5-10 times smaller than with SAN mode (just like with "original" BETA2 performance drop). This looks good, thanks again for checking this!

TrevorBell
Expert
Posts: 353
Liked: 7 times
Joined: Feb 13, 2009 10:13 am
Full Name: Trevor Bell
Location: Worcester UK
Contact:

Re: vStorage API in Virtual Applicance mode

Post by TrevorBell » Oct 15, 2009 11:17 am

Anton,

The VM i made was 2 gig memory and 4 CPU when the operation is running it is intense with 78% guest memory used too....so proves you need to set aside some hefty Ram and Cpu

Im thinking now of moving to this mode for all my backups and then i can remove the Poweredge 860 which is the physical backup server.....another server out the server room!! very good feature this :)

Gostev
SVP, Product Management
Posts: 24612
Liked: 3463 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: vStorage API in Virtual Applicance mode

Post by Gostev » Oct 15, 2009 11:45 am

TrevorBell wrote:Im thinking now of moving to this mode for all my backups and then i can remove the Poweredge 860 which is the physical backup server.....another server out the server room!! very good feature this :)
Ditto! You should go 100% virtual, like our production.

TrevorBell
Expert
Posts: 353
Liked: 7 times
Joined: Feb 13, 2009 10:13 am
Full Name: Trevor Bell
Location: Worcester UK
Contact:

Re: vStorage API in Virtual Applicance mode

Post by TrevorBell » Oct 15, 2009 12:48 pm

I can also report very good restore times using this method and also very good replication too, better than SAN mode in my enviroment anyways

Thanks

Trev

Gostev
SVP, Product Management
Posts: 24612
Liked: 3463 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: vStorage API in Virtual Applicance mode

Post by Gostev » Oct 15, 2009 3:21 pm

So I was thinking more on why Virtual Appliance mode works faster for you, and I think I have one idea. This could due to the VM having faster memory I/O than your physical server, because VM is running on modern server hardware (very fast memory bus). On the other hand, physical host you use for Veeam Backup is probably based on some older hardware?

Thing is, unless CPU is 100% loaded, memory I/O performace is what mostly affects backup performance with "ideal" storage access (faster memory improves on-the-fly dedupe and compression speed).

Gostev
SVP, Product Management
Posts: 24612
Liked: 3463 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: vStorage API in Virtual Applicance mode

Post by Gostev » Oct 16, 2009 12:27 pm

[UPDATE 1/11/2010] As of ESX4 Update1, vStorage API Virtual Appliance processing mode no longer requires that you have VMware Advanced or later license. In Update 1, VMware had fixed what appeared to be the SCSI Hot Add feature licensing glitch, so now you can enjoy all benefits for Virtual Appliance mode with any VMware vSphere license level, including VMware Essentials and Standard.

We are still researching Daniel's issue reported early in this thread, but it looks like Virtual Appliance mode requires that you have VMware license with the following feature (see below in red):
Attachments
licenses.png
licenses.png (151.47 KiB) Viewed 5675 times

floss
Enthusiast
Posts: 31
Liked: never
Joined: Feb 15, 2009 8:31 pm
Contact:

Re: vStorage API in Virtual Applicance mode

Post by floss » Oct 21, 2009 3:08 pm

Hi I am running Backup via The Vstorage API in VA mode and its works superb i am seeing backup speeds of on 125-150mb per sec , one question i can see when the backup runs four extra disks are created then deleted on my Virtual backup sever can someone expalin in detail how the mode work in terms of the scanning and data transfer througn the ESX i/0 stack

Gostev
SVP, Product Management
Posts: 24612
Liked: 3463 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: vStorage API in Virtual Applicance mode

Post by Gostev » Oct 21, 2009 3:37 pm

Hello. In this mode, virtual disks of the backed up VM are attached directly to the Veeam Backup server by leveraging SCSI Hot Add functionality (this is exactly what you are observing), so their content becomes available for Veeam Backup server "directly" (just like Veeam Backup VM's own disks). So all data transfer is performed directly from storage via ESX storage I/O stack, instead of network I/O stack as in case of the network backup. All the rest is still the same as with other backup modes (including changes scanning).

TrevorBell
Expert
Posts: 353
Liked: 7 times
Joined: Feb 13, 2009 10:13 am
Full Name: Trevor Bell
Location: Worcester UK
Contact:

Re: VB4 RC HotADD mode Error

Post by TrevorBell » Oct 26, 2009 4:43 pm

Guys HOT ADD is the real way forward for VB if you get chance to try it out and have enterprise Vmware license it really is the fastest option for backups in my enviroment getting 10% faster than traditional API backups. :) Hopefully you all will see the same results in your enviroments.

Gostev
SVP, Product Management
Posts: 24612
Liked: 3463 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VB4 RC HotADD mode Error

Post by Gostev » Oct 26, 2009 7:56 pm

Actually it should work with "Advanced" license and up. Simplest check is this: if you have VMotion, you should be able to use it.

Biggest reason for such a great performance is due to the fact that VMware had completely re-written ESX iSCSI initiator in ESX4, and sounds like they did great job with this.

I believe that for powerful modern physical backup server and HBA with latest drivers, performance should be a few percent faster than for Virtual Appliance mode (ESX still has some overhead comparing to physical). But in all other cases, Virtual Appliance mode will be winner. ESX servers typically run on high-end server hardware which beats regular physical servers on memory speed, while this is defining factor for Veeam Backup performance if CPU and source/target storage are not bottlenecks.

I think the biggest benefit this mode brings is not even performance, but the fact that you can remove one more physical server from environment. However, you have to have ESX host with some good spare CPU capacity - Veeam Backup can chew all CPU you throw at him when it pumps the raw data from storage at 100MB/s. So while you are removing one physical server, you are taking big chunk of power from another one (although this may not be a problem for off-hours backups).

Trevor, could you please send me the logs from this job, I would like the devs to review and find out what have caused the error above (just in case).

tsightler
VP, Product Management
Posts: 5399
Liked: 2229 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: VB4 RC HotADD mode Error

Post by tsightler » Oct 26, 2009 9:14 pm

Gostev wrote:Actually it should work with "Advanced" license and up. Simplest check is this: if you have VMotion, you should be able to use it.
Don't think this is quite true. We have vMotion, but it was purchased as an addon to our "Standard" edition back in the early vMotion days (ESX 2.x). Since we've always been under maintenance, we still have vMotion (along with Storage vMotion) with our Standard edition ESX 4 servers, but Standard edition does not include Hot Add support. Disappointing because we were hoping to use that mode but we'll get over it.

Gostev
SVP, Product Management
Posts: 24612
Liked: 3463 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VB4 RC HotADD mode Error

Post by Gostev » Oct 26, 2009 9:34 pm

Yep, my rule of thumb does not account for custom feature sets :) I just thought it would be easiest question for us to ask the customer to determine the vSphere license level.

On the other note, I wonder if it is possible to upgrade only 1 ESX host to "Advanced"?

tsightler
VP, Product Management
Posts: 5399
Liked: 2229 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: VB4 RC HotADD mode Error

Post by tsightler » Oct 26, 2009 10:19 pm

Gostev wrote:Yep, my rule of thumb does not account for custom feature sets :) I just thought it would be easiest question for us to ask the customer to determine the vSphere license level.
Perhaps, but in my case the answer would be misleading. It seems you could be 100% sure by simply asking them if they list "Hot-Pluggable Vitrual HW" in the licensed features on the Configuration tab. Perhaps your way covers enough of the users that's it's not a big deal, but I'm a stickler for the details, it's a personality flaw. :mrgreen:
Gostev wrote:On the other note, I wonder if it is possible to upgrade only 1 ESX host to "Advanced"?
I don't see why not. We're seriously considering upgrading our core datacenter hosts to Enterprise Plus because there are currently some big discounts and we think it may pay off once the core count in server processors goes above 6, but I doubt we'd upgrade the servers at our remote sites.

Gostev
SVP, Product Management
Posts: 24612
Liked: 3463 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VB4 RC HotADD mode Error

Post by Gostev » Oct 26, 2009 10:27 pm

On the other hand, remote sites probably do not even need this mode to start with? Because most typically they are very small (2-3 ESX with few VMs to backup) and all ESXes are with local storage (so only "Network" mode can be used)? At least this is what we are seeing in huge accounts like retail chains.

tsightler
VP, Product Management
Posts: 5399
Liked: 2229 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: VB4 RC HotADD mode Error

Post by tsightler » Oct 26, 2009 10:49 pm

Maybe, maybe not. Our remote sites generally only have 2 servers, and 5-7 VM's, typically 500-1TB of data or more, but they do have small, iSCSI SAN's because with think vMotion is worth it, giving us much flexibility during hardware/OS maintenance. I suppose with Veeam 4/block change tracking we could probably use two ESX servers with local storage, and Veeam replication running every few minutes and perhaps actually be more robust without the SAN.

Most of our remote sites have no physical servers at all, or if they do have a physical server it's a Linux server, so hot add mode would be great for these because it's ideal for running Veeam within a VM and still does direct SAN backups. The network mode would be almost as good, but is much slower with ESX 4, although perhaps not a big deal with block change tracking. We plan to do a lot of testing with our remote sites with Veeam once v4 finally hit "release" status. We've never really run Veeam at the remote sites, but we're thinking it might be able to cover pretty much all of our backups now.

Gostev
SVP, Product Management
Posts: 24612
Liked: 3463 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: VB4 RC HotADD mode Error

Post by Gostev » Oct 27, 2009 8:42 am

Agree - with 2 separate local storages all eggs are no longer in one basket.

Gostev
SVP, Product Management
Posts: 24612
Liked: 3463 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: vStorage API in Virtual Applicance mode

Post by Gostev » Oct 28, 2009 8:59 pm

This thread was moved here from the closed beta forum.

schaffeb
Enthusiast
Posts: 50
Liked: never
Joined: Apr 14, 2009 10:44 pm
Full Name: Ben S
Contact:

Re: vStorage API in Virtual Applicance mode

Post by schaffeb » Oct 29, 2009 3:32 pm

I've been testing replication in Virtual Applicance mode, which works great. However, even after the replication job is done, I still see the disk of the vm I replicated in the Veeam VM's explorer. Shouldn't it be disconnecting after the replica is complete?

Gostev
SVP, Product Management
Posts: 24612
Liked: 3463 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: vStorage API in Virtual Applicance mode

Post by Gostev » Oct 29, 2009 3:48 pm

Ben, probably the explorer just does not autorefresh itself properly... disks are actually disconnected right after backup is done (otherwise, snapshot removal would be impossible).

Post Reply

Who is online

Users browsing this forum: Baidu [Spider] and 27 guests