Build numbers

[m.novelli]

Hi guys , don't like the new build numbering with patch date at the end, for example 11.0.0.837 P20210525

It's confusing and at the bottom of the backup report email the build is only 11.0.0.837 , so I don't know the real patch level of the customer / Veeam installation

Why did you complicated a real simple number like the build?

Marco

[Gostev]

Hi, Marco. Changing the build number for patches would prevent us from quickly shipping them like we have been doing in the past year. Advancing the build number opens a whole can of worms around upgrade-related complexities you did not even realize existed with Windows software versions management and Windows Installer requirements. We do advance the build number in minor releases like 10a or 11a, however those come as a new ISO and not just a small EXE. Unlike patches these require much longer release cycle, allowing us to address these additional complexities and test everything well.

Having said that, actually we've been adding patch levels all over the place where the build number is shown (like at the bottom of the console), and this is the one place we missed.

@Egor Yakovlev let's fix "the bottom of the backup report email" in 11a to show the patch level too.

Thanks!

[Egor Yakovlev]

Noted and logged to be fixed with 11a.
Cheers!

[m.novelli]

Egor I liked you answer

[AlexHeylin]

Please use only properly semantic versioning for version numbers (including patches etc).

For example any VBR install lower than "11.0.1.1261 P20211211" has a security issue which needs to be upgraded to "11.0.1.1261 P20211211" to resolve. However, it's not possible to compare "11.0.1.1261" to "11.0.1.1261 P20211211" using 3rd party implementations of parsing for semantic versioning because "11.0.1.1261 P20211211" stops it being a semantic version number so it becomes unparsable, or at best not reliably parsable. Version numbers are supposed to be machine parsable, for good reasons including comparing versions programmatically.

This makes managing the software estate (for example at an MSP where Veeam's management tools may not be universally deployed, or even deployable) much more difficult than it needs to be.

You've already stretched the version number to major.minor.maintenance.build and given that some of those numbers don't / barely increase within a major version number it seems you could shift the patch number into that space and make this work. For example the latest build of VBR v10 was 10.0.x.y which was followed by 11.0.x.y so the minor isn't used at all in this case.

Thanks

[Gostev]

Also remember that hotfixes are made for the specific build and we make them by re-building the specific modules in the specific code branch => the created module preserves the original build number of that branch. While cumulative patches discussed here are nothing but multiple such hotfixes shipped together with a convenient installer on top, so you don't have to deploy them manually.

[soncscy]

Sorry, but what makes it not parsable or unreliably parsable?

I have my own opinions on the patch naming conventions, despite getting the reason why it's done that way, but if it's predictable, it's parsable.

Code: Select all

bluenix:~ boredpanda$ ver="11.0.1.1261 P20211211"
bluenix:~ boredpanda$ IFS=' '
bluenix:~ boredpanda$ read -a somearr <<< "$ver"
bluenix:~ boredpanda$ if [ ${#somearr[@]} -gt 1 ]; then echo "Veeam version ${somearr[0]}, patch "${somearr[1]}; else echo "Veeam version ${somearr[0]}, no cumulative patches"; fi
Veeam version 11.0.1.1261, patch P20211211
bluenix:~ boredpanda$ ver="11.0.1.1261"
bluenix:~ boredpanda$ read -a somearr <<< "$ver"
bluenix:~ boredpanda$ if [ ${#somearr[@]} -gt 1 ]; then echo "Veeam version ${somearr[0]}, patch "${somearr[1]}; else echo "Veeam version ${somearr[0]}, no cumulative patches"; fi
Veeam version 11.0.1.1261, no cumulative patches
bluenix:~ boredpanda$ 

This is simple bash of course, but making conditions for it are pretty simple, and updating the patch table is as simple as just adding a new patch value. It's maybe a few minutes of work to integrate into existing code. The P[date] nomenclature is mildly annoying, but easily handled by a quick function to convert to a datetime value and compared accordingly.

[m.novelli]

I really dislike the new version numbering
Silly and hard human readable

Marco

[Gostev]

Could you clarify what specifically do you find silly about YYYYMMDD?

[AlexHeylin]

@soncscy - the point is not that code can't be written to parse it. The point is that it does not parse in several 3rd party implementations of semantic versioning comparison. It's not always possible / practical to use custom code to do the comparison - and it should not be required, if the convention was being stuck to.

Given the minor component is rarely if ever incremented by Veeam, I suggest that much better use could be made of the available numbering space. One simple example - just shift the patch (3rd segment) left into the minor (2nd segment). Another would be that each of the Pyyyymmdd releases is released as a minor release - i.e the first Pyyyymmdd release for 11.0.1234 would result in 11.1(.1234) - or 11.1234.1 if you prefer it that way. Any of these are more standard than what's being done now. Otherwise why not just number releases as "11.0.0.0.0.0.0.1234.5678.sadkhj Updated on Tuesday 01/03/03, at about 5 o'clock"

Don't get me wrong - I understand why it's being done the way it is, but I don't think it's helpful to customers or partners (even if it is to Veeam Dev). To customers each Pyyyymmdd release is in effect a minor release - so incrementing the minor segment would be much clearer than adding a space and a date to the patch segment.

I do agree that if you're going to add a date as you are then Pyyyymmdd does make sense - and THANKS for picking a format that will at least alpha sort in the correct order, and not Pmmddyy or something equally daft. However it can break parsers which stick to the standard and often expect strictly positive numeric segments. Some of us rely on those parsers. Ideally we'd get those parsers updated / improved, however the developers will quite justifiably say "we're sticking to the standard, and we can't deal with every deviation that other people put in their version strings".

[Gostev]

incrementing the minor segment would be much clearer than adding a space and a date to the patch segment

You may have missed my previous post then. Incrementing the minor segment of a build number means issuing a new product build. So at this point we're no longer talking about hotfixes in principle. Releasing a new build introduces significant overhead and it takes a few times longer to ship. For example, 11a was released as a new build.

Note that we did ship every update as a new build prior to V10 but this was not working well at all in terms of the speed of delivery. Our inability to swiftly release essential reliability and security patches was really impacting our customers and thus also our support organization (due to the sheer number of customers we have and keep adding). Which is why the change was made which allowed us to dramatically increase the speed of patch releases.

[m.novelli]

Could you clarify what specifically do you find silly about YYYYMMDD?

- As an European Human I think in DD MM YYYY
- Full date as version number is stupid, imagine to buy SQL Server 20220302. Windows 10 numbering is also very stupid.
- with every patch you forget to put the numbers in all places: now the email for Backup Configuration Job doesn't show the correct patch level
- reading Veeam Backup & Replication 11.0.1.1261 P20220302 without dot or slash beetwend YYYYMMDD is hardly human readable

That's silly, no excuses

[Gostev]

In my opinion it's much more silly to suggest a naming approach that does not even support a proper sorting, just because this is how the minority of world's population think (including me though)!

Note that this "full date" is not a "version number", but a patch level. Using your own example, I never saw anyone say that "SQL 2022 KB4583458" is a stupid naming for a patch level. Windows uses the same exact patch naming too, and no one ever objected. Now, P20220302 has the same exact length but it also carries actually useful information, as opposed to just a meaningless KB number. I had received some good feedback about this idea back when we first introduced this naming convention in V10, and you're the first to suddenly call it "stupid" 2 years later... so just think about it

Adding slashes is not a problem of course, if the community is fine about updating any parsers they may have already built in the past 2 years.

Finally, the UI bug you mentioned is irrelevant to this discussion, as it would have to be fixed no matter of the patch naming convention we used.

[Regnor]

- As an European Human I think in DD MM YYYY

We use DD MM YYYY and other parts of the world MM DD YYYY, which is always confusing.
I personally got used to YYYYMMDD some time ago, as we use that schema for all documentations, scripts, etc; and no one complained about it so far.

On the topic. I think dates as version/release numbers are great as I can't keep build numbers in my head; I will even forget the most current one.
But seeing P2022XXXX or P2020XXXX will be a good indicator on how far behind an installation is.

[Egor Yakovlev]

you forget to put the numbers in all places

Confirmed: found 2 email reports that use different logic for version retrieval, rather than one we have implemented in other reports.
Logged a bug and it will be fixed in v12 code.

/Thanks!

[AlexHeylin]

@Gostev
Yes - I probably did miss your reply about avoiding a full rebuild into a new version, so sorry about that.

I'll say again - THANKS for using Pyyyymmdd which alpha sorts correctly. Please stick to that, because if you're going to stick to this versioning convention it's far less evil than the other date string options. In tech, all dates should be yyyy mm dd (largest to smallest units) order to avoid confusion and ambiguity (with the bonus of alpha sorting into order). We have software which randomly uses both mm/dd/yyyy and dd/mm/yyyy in the same user interface and it's a nightmare for any dates where both dd & mm are < 13 because we can't be sure which format the date is in, so we can't be sure what the date actually is.

OK, I get why you're doing this. I don't really get why you can't just move everything to the left and use that virtually unused minor segment, but if you say this would cause you problems then I can't argue with that.

I never saw anyone say that "SQL 2022 KB4583458" is a stupid naming for a patch level

Ok, I'll say it - that is a stupid naming convention. Not least because MS patch numbers aren't strictly ascending (or unique!). The next patch level after that might be "SQL 2022 KB237974" Pyyyymmdd is better than that!

Thanks

[doktornotor]

- As an European Human I think in DD MM YYYY

Yeah, and other parts of the world don't. YYYYMMDD is ISO 8601 standard, which is not ambiguous. Please don't touch that.

[Steve-nIP]

Yes, ISO 8601 is the best date format. Everything else is nonsense. We don't read time in a little endian way- seconds, minutes, hours, or mixed endian like the US, we'd end up with minutes, seconds, hours!

[davide.asts]

Hi, Marco. Changing the build number for patches would prevent us from quickly shipping them like we have been doing in the past year. Advancing the build number opens a whole can of worms around upgrade-related complexities you did not even realize existed with Windows software versions management and Windows Installer requirements. We do advance the build number in minor releases like 10a or 11a, however those come as a new ISO and not just a small EXE. Unlike patches these require much longer release cycle, allowing us to address these additional complexities and test everything well.

Ok, to make it simple: you have complicated the life of your customers to make your life simpler.

Wow...

[m.novelli]

"Ok, to make it simple: you have complicated the life of your customers to make your life simpler.

Wow...
"

THIS

Marco

[veremin]

The main reason for this decision was the ability to speed up the delivery of cumulative patches that address reliability and security issues. And we cannot see how this obvious improvement makes our customers' lives complicated:

Note that we did ship every update as a new build prior to V10 but this was not working well at all in terms of the speed of delivery. Our inability to swiftly release essential reliability and security patches was really impacting our customers and thus also our support organization (due to the sheer number of customers we have and keep adding). Which is why the change was made which allowed us to dramatically increase the speed of patch releases.

[AlexHeylin]

we cannot see how this obvious improvement makes our customers' lives complicated

Respectfully - that's because you're not managing a fleet of VBR installs across a range of machines, OSes, and deployment and management scenarios in the way some of your customers are. The complications for customers are what's being discussed in this thread.

I hope we've all agreed that if it's a choice between faster development & better tested software versus more usable version numbering then we agree with you and we'll sacrifice the simpler version numbering to get the faster development & better tested software. I think some of us were (are?) not sure that it is an either / or choice.

[Gostev]

I get your point but then what we should really be discussing here is features to simplify managing and updating a fleet of VBR installs, as opposed to belaboring the minutiae of our patch naming convention?

But you're correct, that's exactly the choice. A hotfix for the specific build will always be a few times faster to deliver than a new product build. Last month someone in support has put together the actual historical numbers and found out that we're talking 3x less time to ship on average between the legacy and new approaches.

[Origin 2000]

Confirmed: found 2 email reports that use different logic for version retrieval, rather than one we have implemented in other reports.
Logged a bug and it will be fixed in v12 code.

The Enterprise Manager also display the "incomplete" Version number under Configuration->Backupserver.

I vote against the current version scheme but to be fair i would say that its not so easy to make everyone happy.

Regards,
Joerg

[AlexHeylin]

Here's a great example of the problem.

As you can see, the equals function run against a non-existent version number using the VBR P convention gets stripped down to just the numeric components and then compared, so it returns all machine running 11.0.1.1261 which covers all versions from 2021-10-07 to 2022-03-12 making it impossible to differentiate between them.

Unfortunately anything less than "11.0.1.1261 P20220302" is vulnerable to four CVEs as per the release notes. This makes easy differentiation quite important!

[m.novelli]

Nice catch AlexHeylin , one more issue with this "dumb" numbering scheme

Marco

[Gostev]

the equals function run against a non-existent version number using the VBR P convention gets stripped down

Wow. Alex, what tool are you using? Seems like a bug, as the definition of Equals is "exactly the same" (as opposed to Like, for example).

When you are asked to compare something data against a base value, why would you modify the base value into something else first? This is just insane, no?

Let's forget Veeam for a moment and take the very latest ESXi 6.7 EP 23 patch for example. Its official release name is ESXi670-202201001. Why would any software management tool strip off 202201001 before comparing this to the value on some ESXi host? And why would anyone use a tool with such a flawed logic?

[AlexHeylin]

I checked and internally the version comparison is done via a MySQL function. Ha! - I can rewrite that.. I thought. Yeah, I can and did then I found this in querying the 'software' table in the DB.


The version number is stored in standard semantic format in the DB, so the Pyyyymmdd isn't even available in the data.

I don't want to get into a debate about this specific tool. It does more than you can imagine, and replaces several full time members of staff (and does a better job), so while some of it is clunky and dated it's the current release of an off the shelf tool commonly used for systems management. Many other similar systems are available, all of which do broadly the same thing with varying level of "hackability" by the system admins. They're ubiquitous at MSPs. I'm presenting this data as an example of why this non-standard numbering scheme causes problems for some customers / partners who are using tools which they're not going to (/can't) stop using just because Veeam picked a non-standard numbering scheme. We're just going to have to work a lot harder than we'd otherwise have to to maintain Veeam's software, which isn't endearing. Obviously we're going to request they look at this use case - but as I said above they're very likely going to reply "The software correctly handles data which complies with the standard", which is true.

To touch on the ESXi example - that's clearly not semantic numbering to start with so to some extent all bets are off, however as such I think the comparison code might treat it as major version 'ESXi670-202201001' with no other data, so would directly compare ESXi versions on the basis of each being a separate major version. An ESXi version number would never appear in the 'software' table though, and would appear in the tables dedicated to VMWare. However, based on the info on the page you linked the ACTUAL version number might well be '6.7.17167734' or '6.7.?.17167734' - regardless of what they call it. In the same way, Windows 11 is version 10.0.y.z - yes, it's actually Windows 10 but it's branded as Windows 11 and has the new GUI if y <= 22000. They didn't even make it 10.1.y.x which would have made slightly more sense, though obviously 11.0.x.y would make the most sense! The point I'm making is there's often a difference between the branding / "known as" name and the actual version string.

[Egor Yakovlev]

Hi all.

I do agree that today's patch level is quite tricky to retrieve in the first place. Will see if we can tune it onwards, however it's format will remain for above-mentioned reasons. On a side note, patch level is logged quite often in our service logs and can be pulled\reformatted by a simple 1-liner.

For example, like this(I choose to create a regkey with "suitable" build number format):

Code: Select all

$version = (Get-ChildItem C:\ProgramData\Veeam\Backup\Svc.VeeamUI.log | Select-String -Pattern "version=" | Select-Object -Last 1).ToString().Split('=') | Select-Object -last 2
New-ItemProperty -Path "HKLM:\SOFTWARE\Veeam\Veeam Backup and Replication" -Name "BuildVersionWithoutP" -Value ($version[0].Split('&patch')[0] + "." + $version[1].Split('P')[1]) -Force

I'm sure LT\CW can parse registry

/Thanks!

[AlexHeylin]

Hi Egor,
Thanks for that. Indeed it can, though unfortunately not into the software table so it won't pull it as a data field as such. This will have to be written as a monitor to alert that an insecure version is installed. Not ideal, but thanks for letting us know how to pull the version number. Surely the FULL Veeam version number is the version stored in

[HKEY_LOCAL_MACHINE\SOFTWARE\Microsoft\Windows\CurrentVersion\Uninstall\{171D06E1-5B07-46EA-AC1B-A8739582BCA5}]
"DisplayVersion"="11.0.1.1261"



OHHHHHH!!!! Well that would explain why the software table in our system says "11.0.1.1261" for all versions of v11a.



@Gostev - are you expecting our management system to pick up something ( Pyyyymmdd) that Veeam isn't specifying as the software version in the only place that matters in Windows?

Seems like a bug

Yep, I'll agree with that - in Veeam's patching, because as far as Windows (and thus everything else) is concerned you're patching without changing the version number to the version number you refer to it as. You might be installing '11.0.1.1261 P20220302', '11.0.1.1261 P20211211', '11.0.1.1261 P20211123', or '11.0.1.1261' but you're always telling Windows (and the rest of the world) you've installed '11.0.1.1261'


Friends don't let friends do patch / version management like this.

I never saw anyone say that "SQL 2022 KB4583458" is a stupid naming for a patch level.

You've already seen me say exactly that above, but I want to add that having 3+ versions of software that are all externally (outside Veeam) numbered with the same version is definitely worse. To use your example, that's like MS numbering all their SQL versions as "SQL 2022" from release through to EOS (a ten year life cycle)