Maintain control of your Microsoft 365 data
Post Reply
AlexHeylin
Veteran
Posts: 563
Liked: 173 times
Joined: Nov 15, 2019 4:09 pm
Full Name: Alex Heylin
Contact:

[Rant] QA lacking in VB / VBM v6, all sub versions

Post by AlexHeylin » 1 person likes this post

Every release of v6 has caused us problems. When we've upgraded because support told us to upgrade to fix the problem the last upgrade caused it's created at least one new problem, sometimes more than one.

Right now 6.1.0.222 is still being pushed to unsuspecting users & partners, but has at least new problems which we're aware of. At least one of these is a new problem in 6.1.0.222 which was not there in the previous v6 release.

One of which fills the logs with errors and causes backup results checks to fail. The excessive errors make it hard to identify if the backup was "successful" or not - because it always ends with a state of Error.

The other means it's not possible to exclude any SPO sites because VBO is unable to retrieve the list. The latest fix (425545_6d882908c1) added a new one - if you have a SPO site / M365 team in the exclusion list then under certain circumstances it will make the job throw an error and HARD fail when it tries to expand the exclusions at the beginning of the job.

Is it too much to ask that two things happen:
1. QA and test in real environments before general release, and especially before pushing it to partners and users.
2. Break up fixes and new features into separate releases. It seems that the frantic rate you're pushing out new features has likely broken things that did work, and re-broken things which got broken in the last version and were supposed to be fixed in this version.

Veeam are capable of doing MUCH better than this, so I'm unable to work out why VBO appears to be being slung out the door without proper testing and expecting users and partners to spend our time finding the issues, opening cases for them, and updating to resolve them - only to be hit by a new bug that just got created in the new version.

Being told in veeam-backup-for-microsoft-365-f47/vbo- ... 80107.html that the issue was confirmed, then that it did not exist did not reduce our annoyance with VBO v6 and the way it seems to have been repeatedly booted out the door while broken in new ways.

Image

I wish I'd never upgraded any of our instances to v6. If you haven't, I advise you don't.
At this rate we're going to look for another product because we can't spend this amount of time dealing with software issues on what is a fairly low profit service. That makes me really sad, but this is unsustainable.
AlexHeylin
Veteran
Posts: 563
Liked: 173 times
Joined: Nov 15, 2019 4:09 pm
Full Name: Alex Heylin
Contact:

Re: [Rant] QA lacking in VB / VBM v6, all sub versions

Post by AlexHeylin »

Case #05588000 - Since upgrade to 6.1.0.222 backups are full of failures
Case #05508116 - More freaking warnings from VBO365 v6 screwing up our backup result checks
Case #05471196 - Adding backup applications fails with app missing permission error
Case #05444449 - Failed to process team: Sales. No team found with Group Id a3e24107-7f46-4f13-85f8-fc302ae91d15
Case #05444422 - Backup result email says Warning - GUI says Failed
Case #05442544 - Silence warnings on sharepoint cannot change web part export mode
Case #05442225 - Authentication issue after upgrade to v6 GA
Case #05433725 - Three problems with upgrade from v6 GA to 6.0.0.367

For a product that's been out for 166 days, I don't think that's an acceptable amount of tickets. Especially as many of those are for issues which were introduced after the GA of 6.0.0.367, and some appear to be for issues created while fixing issues introduced since 6.0.0.367.

While Veeam's rapid development cycle is generally a great thing, something appears to be going wrong here.
Mike Resseler
Product Manager
Posts: 8191
Liked: 1322 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: [Rant] QA lacking in VB / VBM v6, all sub versions

Post by Mike Resseler » 4 people like this post

Hey Alex,

I could take a few of those cases and explain how they have happened and are out of our control, but I am not going to do this. You are right that we are facing issues. I am working with the teams on trying to solve them. Both on the short term (get the issues fixed) and on the long term (have less issues after a release). This is taking almost all of my time now and will be something I need to work on for the next couple of months.

1. QA and test in real environments before general release, and especially before pushing it to partners and users.
We do that. We actually have a lot of environments. And not only demo/ test environments. The issue you describe above is because we request a new parameter (which we didn't do before) and it turned out that older environments can return an unexpected value (to be technically correct, a date should be returned)

2. Break up fixes and new features into separate releases. It seems that the frantic rate you're pushing out new features has likely broken things that did work, and re-broken things which got broken in the last version and were supposed to be fixed in this version.
We do this as good as possible. We have major versions (6 for example) and minor versions (6a) and in between we release CP's which contain fixes

Veeam are capable of doing MUCH better than this
I honestly agree with this, and as said, I am working on this.
AlexHeylin
Veteran
Posts: 563
Liked: 173 times
Joined: Nov 15, 2019 4:09 pm
Full Name: Alex Heylin
Contact:

Re: [Rant] QA lacking in VB / VBM v6, all sub versions

Post by AlexHeylin » 2 people like this post

Thanks Mike - I'm glad that we agree on the last point.

I'm sure some of this is outside your control / can't test in EVERY environment, but I'm glad you understand the impact this has had and the frustration (and time chewed up) that's arisen from it.

One thing you may not realise is that sometimes we techs are under management pressure because "Veeam uses too much time / is too expensive to run... We can get a much better deal from vendor x who says you just press a button and it all works [ALL techs know that's never true!] - we should use that instead!!" etc. There comes a point where we can't defend Veeam if the situation gets too bad - it's bad politics for us, and might even risk our career progression / job.
Mike Resseler
Product Manager
Posts: 8191
Liked: 1322 times
Joined: Feb 08, 2013 3:08 pm
Full Name: Mike Resseler
Location: Belgium
Contact:

Re: [Rant] QA lacking in VB / VBM v6, all sub versions

Post by Mike Resseler » 4 people like this post

Understood loud and clearly. And yes, I do realize the management pressure. I have been in the same situation in previous jobs (both as consultant as internal IT)
A.Rogers
Enthusiast
Posts: 32
Liked: 12 times
Joined: Jan 08, 2021 5:05 pm
Full Name: Alan Rogers
Contact:

Re: [Rant] QA lacking in VB / VBM v6, all sub versions

Post by A.Rogers » 1 person likes this post

I think Mike's response to the thread has been fair and honest so I won't bash anyone too much. But we are also having the same problems with these updates. 6A has decimated our backups for over a week and has cost me hours of troubleshooting. Vanilla v6 would have done the same if I hadn't caught a random blog post warning of problems.
infused
Service Provider
Posts: 178
Liked: 13 times
Joined: Apr 20, 2013 9:25 am
Full Name: Hayden Kirk
Contact:

Re: [Rant] QA lacking in VB / VBM v6, all sub versions

Post by infused »

Few issues here as well, not as much as the rest of you it seems. But it's making us quite nervous about the product, especially since we just signed another point plan a month ago.
A.Rogers
Enthusiast
Posts: 32
Liked: 12 times
Joined: Jan 08, 2021 5:05 pm
Full Name: Alan Rogers
Contact:

Re: [Rant] QA lacking in VB / VBM v6, all sub versions

Post by A.Rogers » 3 people like this post

Performance wise, the latest update 6.1.0.254 looks to be pretty decent. Would definitely recommend upgrading if you haven't already.
Polina
Veeam Software
Posts: 3195
Liked: 774 times
Joined: Oct 21, 2011 11:22 am
Full Name: Polina Vasileva
Contact:

Re: [Rant] QA lacking in VB / VBM v6, all sub versions

Post by Polina »

Hi Alan,

Thanks a lot for your feedback! I forwarded it to our RnD team )
karsten123
Service Provider
Posts: 482
Liked: 120 times
Joined: Apr 03, 2019 6:53 am
Full Name: Karsten Meja
Contact:

Re: [Rant] QA lacking in VB / VBM v6, all sub versions

Post by karsten123 » 1 person likes this post

Same experience here as Alan.
Performance of 6a CP1 is good.
AlexHeylin
Veteran
Posts: 563
Liked: 173 times
Joined: Nov 15, 2019 4:09 pm
Full Name: Alex Heylin
Contact:

Re: [Rant] QA lacking in VB / VBM v6, all sub versions

Post by AlexHeylin » 3 people like this post

Having tracked down our latest issues to a broken OS install (for which VBO was the canary), I'd like to anti-rant that while it still has some "unintended changes" from v5 (which will be reversed in v7) 6.1.0.254 does seem to resolve most of the issues created in the earlier v6 releases. Hopefully we won't find any more issues when we upgrade the rest of our fleet to 6.1.0.254.
Polina
Veeam Software
Posts: 3195
Liked: 774 times
Joined: Oct 21, 2011 11:22 am
Full Name: Polina Vasileva
Contact:

Re: [Rant] QA lacking in VB / VBM v6, all sub versions

Post by Polina » 1 person likes this post

Hi Alex,

I'm glad to know that, and really appreciate sharing it here.
Post Reply

Who is online

Users browsing this forum: No registered users and 8 guests