VCC supplier performance

[dtwiley]

Are there any guidelines for minimum performance of VCC solutions?

The reason I ask is because we are a Veeam partner and engaged with a VCC provider to allow our client base to backup to cloud storage and the performance is VERY unpredictable.

Basic copy jobs dropped to as low as 600k/s and routine merges on machines as small as 60GB took 2 hours even though that nights deltas were just 1.6GB! We had a merge job for a copy job which had just "30GB processed, 25GB transferred" take 167 hours!

17/08/2019 12:15:19 :: Hard disk 2 (3.0 TB) 27.8 GB read at 1 MB/s (05:58:07)
17/08/2019 18:16:09 :: Starting full backup file merge [GFS] (149:02:22)
17/08/2019 18:39:41 :: Synthetic full restore point 08/04/2019 00:00:00 created successfully (148:38:45)

In this example, the transfer too 6 hours (the client has a 300mbit leased line and it was idle other than this transfer), then when the processing was handed off to the VCC provider it took 149 hours to great a single synthetic backup!

They've acknowledged the problem as "too much load on the cluster this client is on", but its been going on for 6 weeks now and they've built a new cluster, moved the client and merges still take several hours for what is quite a small backup.

I appreciate that resource is contended as there are several clients backing up to the same resource (more than likely overnight at the same time), but the provider should be catering for this as you'd expect this kind of behaviour.

The problem is that Veeam is a great product and we've been installing it for years locally for clients and they love it, its never let us down. But with the service now being impacted by 3rd party VCC partners who don't have adequate infrastructure, monitoring and capacity planning, the solution looks very bad to our clients and is soiling the good name of Veeam.

If there aren't already, I urge Veeam to put some minimum SLAs around VCC provider performance. I've already engaged with our Veeam account manager and have been told the VCC provider we're using are one of the largest in the UK so its not like we've chosen a small provider.

It feels like they've massively under estimated the storage requirement for VCC and it cant handle the load.

[Gostev]

Good feedback, but I don't necessarily agree that Veeam is in position to impose minimum SLA requirements. Another way to look at it: this would be similar to requiring Airbus not to sell their planes to airlines who have their on-time arrival (or flight cancellation) metrics worse than certain threshold?

Also, consider the following: what is bad SLA for you, could be good enough SLA for another customer - for the price the provider is asking, and for their use case. Since I like analogies and you're from UK, think Ryanair vs. BA.

So in my opinion, SLAs must be an integral part of the contract between you and the service provider - and not something Veeam requires and controls. In fact, many service providers practice this already, just because most customers will simply not subscribe to their service otherwise. For example, one particular U.S. provider I met gives 10% discount off of monthly bill for every 1 hour when agreed-upon SLAs were not met, up to 90% discount (something along these lines). They are not the cheapest provider though, as always - you get what you pay for.