Using VSAN in production?

[zoltank]

This isn't a B&R question per se, but I've found the members here to be well versed in real world usage of VMware.

We're looking at a refresh of our virtual environment late this year or early next, and I'm taking the opportunity to reexamine our architecture. Currently we have a 9TB SAN with dual everything connecting to three hosts with a total of 55 VMs. However, even with this I don't like that the SAN is single point of failure, especially since we've had a dual controller reset on the SAN before (absolutely not fun). However, I understand a physical SAN, I understand its limitations, and it's a proven technology.

VSAN makes a lot of sense to me on the theoretical level, and I can see it being the future of virtualized storage. However, it's only been a year since VMware released their VSAN product.

So, is VMware VSAN ready for a production environment with business critical VMs on it?

[Vitaliy S.]

I believe VMware Virtual SAN 6.0 looks like a good offer from VMware.

As to the previous version of vSan you can review (if you haven't seen them yet) a couple of comments over these links:
http://www.virtualizationteam.com/stora ... y-yet.html
http://www.reddit.com/r/vmware/comments ... nightmare/
http://technodrone.blogspot.com/2014/03 ... truth.html

[dellock6]

And as additional note, since you are evaluating a "Server SAN" solution, remember there are also other technologies available, like Nutanix or HP StoreVirtual VSA (that is also supported with our storage snapshots integration by the way).
All of them have good feedback from user and proven usability, and we have integration and best practices papers with all of them, when not having dedicated software integration.

[zoltank]

I'm really only looking at VMware VSAN since eliminating a vendor in our virtual environment is a benefit to us; one fewer neck to choke.

[emachabert]

Beware of vendor lock-in
By the way looks like a good technical option.

[dellock6]

Whenever you pick a vendor, there it comes a vendor lock-in (even with Veeam). The problem is not vendor lock-in, but if you pick up a vendor that after the sell cycle is going to choke you, that one is a bad lock-in.

I was comparing other Server SAN solutions in technical terms, not for reducing the number of suppliers. If so, you should also consider EVO:RAIL, they all come with a one single point of contact for all the components.

[zoltank]

I'm also evaluating EVO:RAIL, though I'm more wary of purchasing a hardware solution from a software vendor. I'm pretty comfortable with a Dell server on the VMware HCL list. With our SAN we've ended up in the annoying situation of having VMware saying talk to IBM (our SAN vendor), and having IBM saying talk to VMware.

Sorry, when I said "I'm really only looking at VMware VSAN since eliminating a vendor in our virtual environment is a benefit to us; one fewer neck to choke.", what I meant wasn't that we're only looking at VSAN to eliminate a vendor, what I meant was we're only looking at VMware's implementation of VSAN in order not to add an additional vendor.

We're at VSAN, first and foremost, because in theory it should be more reliable than a hardware SAN, and we love reliable.

[dellock6]

Got your point.
Well, it's reliable as any scale-out storage can be, because it doesn't have a single point of failure. But this is on paper, you're better to test it out in your environment if possible; not saying it's not good, but specs on the paper are one thing, in real life usually things change and the details are important.
Just one note for EVO:RAIL. VMware is not selling it, they just give licenses to OEM to build and sell their own appliances, so you will end up having an HP, Dell, EMC, and others solutions. But one requirement of EVO:RAIL is that the hardware vendor has to guarantee a single point of contact for support (sounds like I'm promoting it by now... sorry, just trying to give correct informations).

Luca.

[zoltank]

I just learned that about EVO:RAIL this morning, which has really reduced its appeal to me.

Unfortunately, I can't really effectively test it out. We're looking at a complete refresh of our VMware environment (standing up a completely separate and new environment and then moving out VMs over to it), so we need to decide beforehand if we'll be packing our hosts with storage or purchasing a SAN.

[tsightler]

I just learned that about EVO:RAIL this morning, which has really reduced its appeal to me.

I'm curious as to why this reduced it's appeal. Earlier you mentioned that you were wary about buying hardware from a software vendor, so I would have thought this would make it better. Instead of buying your own hardware and software separately, and then mashing it all together and having them point at each other when something doesn't work, you can buy a certified combination of hardware/software and have a single point of support. This would seem to be the best possible option. Of course, I'm not sure about the entire EVO:RAIL concept as I think it removes some flexibility going forward although perhaps I just like building things myself to much and thus this commoditization is killing me. But I am curious of your thoughts around this point.

Thanks for the great conversation!

[emachabert]

It is still the same mess, software/hardware tennis match , but this time you just count the points. I can't wait to manage a performance issue on EVO:Rack
Let's learn about this new technology, soon designs will be sharper and limits will be well known.

I'm thinking like Tom, I like building things I know will run smoothly at light speed
Before converging systems, we need converged people. Vertical silos are dead, we need a new type of admins, with great understanding of software, storage and network.

Regarding new features of Vsphere, VVols are a biggest point to me because it definitively change the way we've been managing VM storage for the last 10years. I see many benefits on that point.

[zoltank]

I'm curious as to why this reduced it's appeal. Earlier you mentioned that you were wary about buying hardware from a software vendor, so I would have thought this would make it better. Instead of buying your own hardware and software separately, and then mashing it all together and having them point at each other when something doesn't work, you can buy a certified combination of hardware/software and have a single point of support. This would seem to be the best possible option. Of course, I'm not sure about the entire EVO:RAIL concept as I think it removes some flexibility going forward although perhaps I just like building things myself to much and thus this commoditization is killing me. But I am curious of your thoughts around this point.

If I go with EVO:RAIL I'm paying a premium and giving up my ability to customize my host/storage specifically to my needs and price point. I'd be willing to do that if it meant the same vendor supported both the hardware and software, but that's clearly not the case. I'm essentially just buying a pre-configured server which still has me going to the hardware vendor for support and possibly locks me in to waiting on the hardware vendor to certify/release vSphere versions for their specific EVO:RAIL. It seems like I'd be able to have the same level of support by just making sure all the hardware in the server I buy is on the HCL.

[emachabert]

Yep, a VCE like offer, but at server scale.
I think it will be good for smb, branch office etc. I played with those servers at VMworld last year, in less than 20min you have 4 esx, 1 vcenter and 1 logInsight up and running. You enter the technical information in the wizaard and then it builds everything automatically.

[umenz]

back to the initilal question.

I think Server Hardware with NVMe Cache is faster than "entry" SANs in the same pricing league.
Why not use vSAN in Production. Any catastophal experences ?

[dellock6]

I'd say that after the first release and its problems (as any other v1 product) VSAN has proved itself to be a really stable solution. I don't see limits as of today not to use it in production.

Luca

[Gostev]

I think Server Hardware with NVMe Cache is faster than "entry" SANs in the same pricing league.

I love VSAN as a technology.

However, I always felt it comes out too expensive due to high software licensing costs. The total price just did not seem right to me when it did not provide most of the features you can get from those entry-level SANs - most of which these days are just scaled-down version of their enterprise-class version, and thus are extremely feature-rich.

May be things have changed around VSAN licensing last time I looked though (when it first came out). But here is the thing - they changed in the SAN market too, where due to fierce competition you can now get entry-level all-flash SAN for under USD 20,000 (I am thinking HP 3PAR 8000 series, for example). It is some serious hardware with functionality VSAN cannot even dream about.

Can you share your pricing estimates for VSAN with NVMe Cache? What is the licensing like for 4 nodes, which is the absolute minimum for VSAN?

[umenz]

yes, vSAN for each node USD 2.500.. and should configured balanced. Oversized for my next project I think.

But my Intension was to Cache local storage in the same way as All-Flash Appliances do that.
- HP SmartCache is limited with 4GB RAM of the Controller, Needs RAID5 for the Flash to enable write cache...

so I am searching for a "smaller" solution and found Starwind as an option.
My opinon is that a level between Hardware and VMware can accelerate more than a controller can do ?

thanks
Ulrich

[emachabert]

I don't get the raid 5 for flash opt using smartarray.
If you use hp smartcache, you simply build a raid1 with two ssd then you choose on which volume you wan't to enable smartcache (read/write). 4gb on the smart array is not mandatory and depends on the configuration and expected workload parterns.

You could also look at HP storevirtual VSA with SSD and SaS and enable auto tiering (adaptive optimization).

[atuzov]

Hello umenz,

StarWind Virtual SAN will definitely do the job for you.
StarWind has RAM and SSD server-side caching feature which will accelerate the performance between hardware and VMware.

As a nice bonus, Veeam is fitting perfectly to StarWind as proved by many Customers. Veeam was also selected by StarWind as a backup solution of choice for StarWind HyperConverged Appliance (HCA).

You can find more info here:
https://www.starwindsoftware.com/starwind-virtual-san

[atuzov]

Umenz.
Also, i reckon the StarWind forums would be a good source of information as well. You will be able to get more answers to any StarWind related questions.

[dellock6]

Just a note regarding the VSAN and HP cache: the entire concept of vsan is to leverage single disks for capacity and SSD for caching, so why would you configure raid or caching at the controller level? All disks need to be added as single devices to the VSAN cluster, and the server will use all of them creating the desired level of protection.

I'd honestly look for a simple and good hba in the vmware hcl if I'd design a vsan cluster, with no raid at all.

[umenz]

Hi Luca, starting the question was to use Hardware or Software to get more.
but a good hint when I use Software we need not Hardware RAID.

[zoltank]

After reading in the Veeam newsletter this past weekend about the precipitous decline in VMware support, I'm quite happy about our decision to stick with a traditional SAN for this refresh.