[Rant] QA lacking in VB / VBM v6, all sub versions

[AlexHeylin]

Every release of v6 has caused us problems. When we've upgraded because support told us to upgrade to fix the problem the last upgrade caused it's created at least one new problem, sometimes more than one.

Right now 6.1.0.222 is still being pushed to unsuspecting users & partners, but has at least new problems which we're aware of. At least one of these is a new problem in 6.1.0.222 which was not there in the previous v6 release.

One of which fills the logs with errors and causes backup results checks to fail. The excessive errors make it hard to identify if the backup was "successful" or not - because it always ends with a state of Error.

The other means it's not possible to exclude any SPO sites because VBO is unable to retrieve the list. The latest fix (425545_6d882908c1) added a new one - if you have a SPO site / M365 team in the exclusion list then under certain circumstances it will make the job throw an error and HARD fail when it tries to expand the exclusions at the beginning of the job.

Is it too much to ask that two things happen:
1. QA and test in real environments before general release, and especially before pushing it to partners and users.
2. Break up fixes and new features into separate releases. It seems that the frantic rate you're pushing out new features has likely broken things that did work, and re-broken things which got broken in the last version and were supposed to be fixed in this version.

Veeam are capable of doing MUCH better than this, so I'm unable to work out why VBO appears to be being slung out the door without proper testing and expecting users and partners to spend our time finding the issues, opening cases for them, and updating to resolve them - only to be hit by a new bug that just got created in the new version.

Being told in veeam-backup-for-microsoft-365-f47/vbo- ... 80107.html that the issue was confirmed, then that it did not exist did not reduce our annoyance with VBO v6 and the way it seems to have been repeatedly booted out the door while broken in new ways.



I wish I'd never upgraded any of our instances to v6. If you haven't, I advise you don't.
At this rate we're going to look for another product because we can't spend this amount of time dealing with software issues on what is a fairly low profit service. That makes me really sad, but this is unsustainable.

[AlexHeylin]

Case #05588000 - Since upgrade to 6.1.0.222 backups are full of failures
Case #05508116 - More freaking warnings from VBO365 v6 screwing up our backup result checks
Case #05471196 - Adding backup applications fails with app missing permission error
Case #05444449 - Failed to process team: Sales. No team found with Group Id a3e24107-7f46-4f13-85f8-fc302ae91d15
Case #05444422 - Backup result email says Warning - GUI says Failed
Case #05442544 - Silence warnings on sharepoint cannot change web part export mode
Case #05442225 - Authentication issue after upgrade to v6 GA
Case #05433725 - Three problems with upgrade from v6 GA to 6.0.0.367

For a product that's been out for 166 days, I don't think that's an acceptable amount of tickets. Especially as many of those are for issues which were introduced after the GA of 6.0.0.367, and some appear to be for issues created while fixing issues introduced since 6.0.0.367.

While Veeam's rapid development cycle is generally a great thing, something appears to be going wrong here.

[Mike Resseler]

Hey Alex,

I could take a few of those cases and explain how they have happened and are out of our control, but I am not going to do this. You are right that we are facing issues. I am working with the teams on trying to solve them. Both on the short term (get the issues fixed) and on the long term (have less issues after a release). This is taking almost all of my time now and will be something I need to work on for the next couple of months.

1. QA and test in real environments before general release, and especially before pushing it to partners and users.
We do that. We actually have a lot of environments. And not only demo/ test environments. The issue you describe above is because we request a new parameter (which we didn't do before) and it turned out that older environments can return an unexpected value (to be technically correct, a date should be returned)

2. Break up fixes and new features into separate releases. It seems that the frantic rate you're pushing out new features has likely broken things that did work, and re-broken things which got broken in the last version and were supposed to be fixed in this version.
We do this as good as possible. We have major versions (6 for example) and minor versions (6a) and in between we release CP's which contain fixes

Veeam are capable of doing MUCH better than this
I honestly agree with this, and as said, I am working on this.

[AlexHeylin]

Thanks Mike - I'm glad that we agree on the last point.

I'm sure some of this is outside your control / can't test in EVERY environment, but I'm glad you understand the impact this has had and the frustration (and time chewed up) that's arisen from it.

One thing you may not realise is that sometimes we techs are under management pressure because "Veeam uses too much time / is too expensive to run... We can get a much better deal from vendor x who says you just press a button and it all works [ALL techs know that's never true!] - we should use that instead!!" etc. There comes a point where we can't defend Veeam if the situation gets too bad - it's bad politics for us, and might even risk our career progression / job.

[Mike Resseler]

Understood loud and clearly. And yes, I do realize the management pressure. I have been in the same situation in previous jobs (both as consultant as internal IT)

[A.Rogers]

I think Mike's response to the thread has been fair and honest so I won't bash anyone too much. But we are also having the same problems with these updates. 6A has decimated our backups for over a week and has cost me hours of troubleshooting. Vanilla v6 would have done the same if I hadn't caught a random blog post warning of problems.

[infused]

Few issues here as well, not as much as the rest of you it seems. But it's making us quite nervous about the product, especially since we just signed another point plan a month ago.

[A.Rogers]

Performance wise, the latest update 6.1.0.254 looks to be pretty decent. Would definitely recommend upgrading if you haven't already.

[Polina]

Hi Alan,

Thanks a lot for your feedback! I forwarded it to our RnD team )

[karsten123]

Same experience here as Alan.
Performance of 6a CP1 is good.

[AlexHeylin]

Having tracked down our latest issues to a broken OS install (for which VBO was the canary), I'd like to anti-rant that while it still has some "unintended changes" from v5 (which will be reversed in v7) 6.1.0.254 does seem to resolve most of the issues created in the earlier v6 releases. Hopefully we won't find any more issues when we upgrade the rest of our fleet to 6.1.0.254.

[Polina]

Hi Alex,

I'm glad to know that, and really appreciate sharing it here.