Veeam Customer Support Sucks!!!!

[ceebee]

I have never seen a product company this arrogant on customer support than Veeam! The response is we are closed over the weekend and since our engineers are based out of Russia - the first call back we can get is on 4th!
Wow so much so for a company that is rated for best product of the year. Neither Am i impressed with the product nor with the customer support.
(I was calling regarding the initial back-up times for exchange 2003 (Veeam 4.0 with ESX 3.5) taking in access of 48+ hours for a 230 GB store with 850 GB total disk space and was not sure what was going on.)

[Gostev]

Hello Chet, welcome to our forums. I am not with our support, but are you on Premium 24/7 support contract? Standard support is indeed only provided during business hours (and today is Saturday), please refer to the following document for more information:
http://www.veeam.com/files/veeam_softwa ... policy.pdf

Also, please note that if you are not registered with Veeam as a support contact for your company, then your request will be automatically classified as coming from evaluator in our call center, and thus will also be served during business hours and with lower priority.

Your issue looks to be environmental (storage or networking), so indeed it would be best to have our technical staff to take a look at full logs as well as live enviroment over the webex if possible. In all cases, I recommend that you send all logs from Help | Support Information to support@veeam.com to speed up the resolution.

Thank you!

[ceebee]

Yes we have 24/7 support. But that is a misnomer as the support is truly not 24/7. I have seen others report the same for exchange initial back-up but not as slow as 4 MBps.
What can i do to get some help during the weekend? Unfortunately if we wait until Monday we will miss out on our maintenance window.
Appreciate the help.

[ssimakov]

Chet,

I think we have a miscommunication here. According to our records you are not associated as a support contact with your company account so obviously your support call was not identified as Premium customer support call. This has been fixed a minute ago and you will be contacted shortly.

[wocomike]

How do I go about getting my account flagged properly? I submitted some log files on Thursday night and have not had a response. Ticket ID 518377

[Gostev]

Usually, the current support contact should get in touch with our support and request to get your name added. But you can just contact our support with this request, and they will get this sorted for you.

There are larger companies where one location is already using Veeam Backup, and another location is only in the process of evaluation, which is why you need to be in the list of support contacts to get Standard or Premium support... otherwise the system will classify your request as coming from evaluator.

[withanh]

Chet,

From my experience it sounds like your Exchange is being backed up in network copy mode (or whatever the actual term is) and the speed you are reporting is about right for a 230GiB store and 850GiB drive. 4MiB/s is about what I was getting with my Exchange backups. My store isn't nearly as large as yours, only 70GiB store and 200GiB drive.

A couple of weeks ago, I was having problems with one of the controllers on my SAN and my Exchange backup speed was very similar to yours:

Code: Select all

6 of 6 files processed 

Total VM size: 200.02 GB
Processed size: 200.02 GB
Processing rate: 6 MB/s
Backup mode: SAN/NBD with changed block tracking
Start time: 12/22/2009 5:11:22 PM
End time: 12/23/2009 3:26:41 AM
Duration: 10:15:18

Backing up file "[VMFS-iSCSI02_Delta] vanc-exch01/vanc-exch01-flat.vmdk"
Unable to establish direct connection to the shared storage (SAN).
Please ensure that:
- HBA is properly installed in the Veeam Backup server computer, or software iSCSI initiator is configured correctly.
- SAN volume can be seen by operating system in the Windows Disk Management snap-in on the Veeam Backup server.
- Read access is allowed for the Veeam Backup server computer on the corresponding LUN (refer to your SAN documentation).

Direct SAN connection is not available, failing over to network mode...

But with SAN mode working my speeds are:

Code: Select all

6 of 6 files processed 

Total VM size: 200.02 GB
Processed size: 200.02 GB
Processing rate: 19 MB/s
Backup mode: SAN/NBD with changed block tracking
Start time: 12/28/2009 10:09:54 PM
End time: 12/29/2009 1:05:00 AM
Duration: 2:55:05

Then I started playing with Virtual Appliance mode and HOTADD and my speeds are now:

Code: Select all

6 of 6 files processed 

Total VM size: 200.02 GB
Processed size: 200.02 GB
Processing rate: 51 MB/s
Backup mode: HOTADD with changed block tracking
Start time: 12/31/2009 10:00:53 PM
End time: 12/31/2009 11:07:46 PM
Duration: 1:06:52

I would take a look at how your Veeam Backup server is accessing your ESX(i) infrastructure. If your Veeam server is physical, make sure you have your iSCSI SAN connected to your Veeam server, or a Fibre Channel HBA on the box. Make sure your backup jobs are set to use SAN/NAS mode.

If your Veeam server is virtual (like mine - and required for Virtual Appliance/HOTADD mode) then I'm not sure how your ESX(i) DataStores should be connected to the Veeam server (or even if they need to). You should be using Virtual Appliance/HOTADD mode for the backup jobs.

My DataStores are all iSCSI so I have them connected to my virtual Veeam server anyway since it's so easy to connect iSCSI. I hope this helps at least explain why you're seeing the speeds you're getting.

Also, have faith in the Veeam support, they've been great to work with. I have been using Veeam for only a few months and called in to support a couple of times gettings things set up properly. I got frustrated with their support, but it was mostly my impatience I also did have a tech that I was having trouble working with, I asked for an escalation to a supervisor and once the supervisor got involved (and I kept working with the same tech) things improved greatly. If anyone asks me for a reference on Veeam, the product gets A+, the support gets B+/A- and that's because of the afore mentioned single incident.

h

[lethallynx]

I am very happy with the product and very happy with the support.

I called a few days before new years day for some support and got a call back within the hour and the problem was resolved.

Sounds like you just got unlucky, I am sure your next support call will go much better.

[ceebee]

The back-up is happening over the SAN and not Network. The settings are the same for exchange as for other servers. The Primary SAN drives are 146 GB FC connected via a Brocade 4GB switch. The back-up SAN is running 1TB SATA drives
No one from Veeam has been able to pin point the root cause yet. It has been 5 days now and all i have is some amount of email support flowing.


Moreover the incremental back-up deleted the Full back-up!!!! How super is that? I mean c'mon which back-up product exists today that will go out and delete the full back-up cos it thinks its corrupt. I have not seen why it would be corrupt. If indeed it is then it should error out. Even pidely products do the basic checking.
The reasoning i get from Veeam support is that the Full back-up was corrupt. So basically i am not to trust the full back-up now? Thatz beyond crazy now.
You take a full back-up before any major upgrades or changes and look at the logs to ensure that all is well. Veeam tell me that the integrity check is not performed until the incremental back-up is started. Fantastic!
I am getting one story after the other - Which one to believe is up in the air. Any takers??


For example look at the support of VMWare - The moment you call within 5 minutes they are on a webex session with you - NO QUESTIONS ASKED! That is the way support should be. Period!

[Gostev]

You take a full back-up before any major upgrades or changes and look at the logs to ensure that all is well.

Correct, but there are issues with the storage, then the backup logs will show that backup went OK, but the data in it will be corrupted - would you agree? Although this happens once in a blue moon, so I don't think this is your issue. I will take a look at the logs when I am back from vacation, I believe the reason here is that your full backup simply was not created successfully.

Veeam tell me that the integrity check is not performed until the incremental back-up is started. Fantastic!

Other products do not do integrity checks on created backups at all... they just drop those fulls on storage, and never touch afterwards - assuming they are correct.

For example look at the support of VMWare - The moment you call within 5 minutes they are on a webex session with you - NO QUESTIONS ASKED! That is the way support should be. Period!

I agree.

[ceebee]

What do we need to do to get support like VMWare? It would help tremendously.

[Gostev]

Probably, charge for it as much as VMware does

Seriously though, I am not with our support organization, and I am not a customer, so I cannot really judge the situation. I know they have been doing big reorganization lately, so hopefully it will improve the quality. The new structure should scale much better for the exponential customer base growth we are experiencing as a company. Also, we had many complaints on our call center, and it is being completely revamped.

This is why we really appreciate all feedback (including negative) - without it, how can we know that something is not working very well?

Also, specifically to your case (and the issue with performance) - it is always hard for our support to deal with storage issues, so it is hard to blame them. They are not storage experts, they know our product very well, but not those 3rd party storage devices. Based on all the information from thousands of customers, we know for sure that our product does not have performance problems, but it is really hard for us to troubleshoot issues with 3rd party storage performance. We always try to help and assist though, so it may indeed take much more time to work on such issues, as opposed when issues are with our product (some bug or improper behaviour) - the latter issues are usually resolved in no time.

[ceebee]

I am a firm believer in "You get what you pay for". You just validated that statement. Thanks you!
Let's how long it takes them to answer another call that i logged.

[withanh]

I opened a support case this morning and was contacted within about 15 minutes.

[drbarker]

Hi Chet,

The forums are a good place for gathering info about different experiences from different hardware...

Out of idle curiosity:

- What type of iSCSI & FC SAN are you using
- What is the veeam backup server, and what fc card do you have in the backup server
- How many spindles are you using in both the FC & iSCSI storage?
- When are you running the Veeam backup (during the exchange db maintenance period overnight?)

[ceebee]

Ben M. got on our machine and confirmed that Veeam was about 3-4 times slow when compared to a file transfer via the windows command.
Development is looking into the issue but I am waiting for their response to what could be the root cause.
They keep telling me it is environmental but they wont give me any specifics.
No the exchange back-up is not run during any maintenance window. We have since then switched over to NT back-up as Veeam does not work!

[Gostev]

They keep telling me it is environmental but they wont give me any specifics.

Hello Chet, I've just reviewed your case, and I can definitely see that support told you specifics there. You clearly have some serious enviromental issues with networking and/or your vCenter. Most VMware API operations complete with connection timeouts, connections are very unstable and often drop, preventing backup activities from working properly.

I have also noticed that you have been already told about this environmental issue before (in one of your previous support cases), but it is still there.

Please appreciate the fact that our product relies on many components of your virtual infrastructure, and errors/failures are to be expected if some pieces of your environment are not working properly. No product can work in a failing environment. And product failures do not automatically mean that the product is "buggy" as you posted in another thread. If our product had real issues, then many thousands of our customers would also be affected by those. Which, as you see, is not the case - most other customers have great experience with the product, both performance and reliability wise.

[ceebee]

The snapshots were created by Veeam and not manually and not by any VMware specific process. Who ever creates the snapshot is responsible for the deletion also. Veeam might leverage the VMWare API but that is besides the point. It is between Veeam and VMWare to resolve. I as a customer should not be caught in the middle.

[ceebee]

In my opinion it is very simple. If Veeam is the product that i am going to rely on and it does not work in certain instances with the VCenter then it is Veeams responsibility to help why.
How can VMWare help in this case? It is very easy to point that it is a VC issue without stating why it is being caused. In my opinion it my Infrastructure as messed up as you guys are stating then nothing would have worked.
All i am looking for is a pointer as to why i am getting this error and i get the message that it is VC issue.
I don't think this is going to get any resolution. I would expect Veeam to work with VMWare technical support to isolate the find the root cause.

[Gostev]

Chet, you are correct that whoever creates snapshot should delete it. This is exactly the operation our product is unable to perform because of unstable vCenter connectivity and constant connection drops we have been seeing in your environment. How possibly our product can remove the created snapshot if all connection attempts to vCenter keep are failing (or drop soon after being established) during certain periods of time? Our product is simply unable to issue the snapshot removal command in such conditions

But the issue may not be connected to vCenter either, this could be regular hardware issue (failing router, cabling, NIC etc). So this may no be just between Veeam and VMware to resolve.

To me, this all points to network connectivity issues, but it would definitely help to involve VMware so that they could ensure vCenter does NOT have issues - for example, high load - which could cause connection drops (I've seen this before, but only a few times and in very large environments - hundred plus of hosts). At the same time, I am seeing issues due to faulty network equipment reported every other week (mostly performance issues due to 1Gbit > 100Mbit failover on some port, and unreliable connectivity is second most commonly observed issue).

Please note that we are NOT saying that "your environment is completely messed up", but there are definitely some issues with vCenter server connectivity. This issue may not be affecting "core" functionality (ability of ESX hosts to run VMs), which is why everything else may work well for you. But any application connecting to vCenter with VMware API would suffer from this issue.

I think we all should work together, it is unlikely that we will be able to resolve environmental issue just between Veeam and VMware - without involving you or your technical staff.

Thank you.

[Ztech-Windowbook]

The standard stock answer for all Veeam support calls is "it's a VMware problem, call them."
Been working on a ticket for a week now. Promised a call-back, never received one. Promised log file analysis, this never happened.
Veeam's VoIP system is horrendous, from any phone cell or land line - drop-outs, jitter, unintelligible voices.
Add to that the majority of heavily-accented support techs and this is a recipe for disaster.

We have initiated a support agreement with a local company to avoid Veeam support entirely.
I would not recommend this product as it is too slow and buggy to be effective.

[Ben Milligan]

Hello there! Ben Milligan from Veeam Support. I'm sorry to hear you've had this experience in the past with our team but I assure you we're here to help.

I head up our team of engineers based here in the US and I can also assure you that they are native English speakers. We do have engineers based in EMEA however this is the half of our team that handles European time zones and languages typically. We do sometimes refer to VMware, but only in the case where we need their help to resolve something. We are happy to work jointly with them in cases where it is not fully known where the problem resides (Veeam and VMware), as well. Be that as it may, I appreciate your feedback.

I have researched all the cases your company has opened with us, and unfortunately did not find one that is currently open. The most recent case was opened and closed on 2/8 of this year. Is this the case you are referring to? Please advise the case number for the one you mentioned and I'll be happy to review it in depth to ensure its handled properly.

Thanks!

[Gostev]

Anyone else noticed that the last response in this thread is over 5 years old? Nothing speaks better for the quality of our support

[emachabert]

Sometimes, I feel like living in a parallel universe

[TommyB]

Add to that the majority of heavily-accented support techs and this is a recipe for disaster.

Wow, I can't believe you actually wrote something disrespectful like this... apart from everything else that you wrote, *this* actually makes me think that you are a closed-minded, unfriendly person that always tries to find something to grouse about. This deserves a forum ban.

My experiences with Veeam support have always been very positive and helped me to resolve various issues - and I never heard "it is VmWare's fault".

[b.lagace]

I know what his problem is...he forgot to read the manual !

Be serious! I have over a dozen tickets with Veeam no matter how complex, and yes sometimes I needed to convince them to look twice, in the end...everything gets solved. Offcourse you need to put some effort in it yourself!

And never forgot : support team is not the development team. So if a bug finds his way into the released problems, these guys are just the messengers trying to understand and figure it out so they can steer the developpers.

Have a happy monday everyone!

epic to choose just as post of the week

[adruet]

I have opened about ten calls in the last three years, not only do I have a very high opinion of the product and its developpers for always being able to implement things we thought we could only dream about, but the support is great. They really listen to you, forward request to dev for improvements, help you with great courtasy.
I am in EMEA (France), and even if I check that I can be called back in English, they often call me back with someone who speaks French. They are not native speakers, but I can tell you that their French is impressive ! Not only are they technically skilled, but they are also very nice to work with, and it's always a pleasure.

I am not making compliments over products very often, Veeam B&R is probably the product that has impressed me the most in last couple years since the begining of virtualisation, and their support quality is about as impressive as the products.

[readie]

Over the past three or four years I have found the support given over these Forums excellent, speedy, and knowledgeable. In fact I normally pose the problem here first before raising a formal support call (which often is not required in the end). Sometimes the help comes from 3rd party forum users, but very often from Veeam themselves.

[rccl_ecain]

Wow, I can't believe you actually wrote something disrespectful like this... apart from everything else that you wrote, *this* actually makes me think that you are a closed-minded, unfriendly person that always tries to find something to grouse about. This deserves a forum ban.

My experiences with Veeam support have always been very positive and helped me to resolve various issues - and I never heard "it is VmWare's fault".

Great point. Honestly, you would think this person may be a troll, but there are just some people who should never be working in this industry to begin with.

[andyd@lacma.org]

The standard stock answer for all Veeam support calls is "it's a VMware problem, call them."
Been working on a ticket for a week now. Promised a call-back, never received one. Promised log file analysis, this never happened.
Veeam's VoIP system is horrendous, from any phone cell or land line - drop-outs, jitter, unintelligible voices.
Add to that the majority of heavily-accented support techs and this is a recipe for disaster.

We have initiated a support agreement with a local company to avoid Veeam support entirely.
I would not recommend this product as it is too slow and buggy to be effective.

You've been "working on a ticket for a week now?" That's nothing. I have a ticket that was escalated to Level 3 months ago which was opened on 10-19-2014. One of the primary reasons I dumped Symantec was due to their poor support. Unfortunately, Veeam is even worse. They just handle things differently. You can't count on receiving tape support in a timely fashion because they don't believe any tape issue is ever a #1 Severity. Interesting way of handling these issues.