[V12] System Requirements for our 2022 release

[scott.anderson]

IT is an interesting industry. It sometimes forgets that its actually a service industry.

The vast majority of IT professionals are supporting the operation of business, not running the business.

So its easy to say, stop using this old OS or application. It's not always that easy for a business or organisation to stop using them however.

As a MSP we have wide range of businesses that we support including for example manufacturing organisations with engraving machines that are the size of a small bus, cost upwards of $500,000 and come with a dedicated PC provided by the manufacturer (running XP!!!). So when asking the business to upgrade their XP PC means spending half a million $ you can imagine the conversation.

Another example we have is an irrigation CoOp in the Australian outback covering an area about the size of half of Europe. They have hardened PC's running/monitoring remote pump stations (when the temp can get up to 50+ Centigrade for 8 months of the year it needs some serious hardware) most of which are still running XP or Win 7. Each PC pre-Covid was costing anything up to $15k and they have over 100 of them. And when it can take a couple of days to physically reach some of the locations (serious outback cross country driving), the decision to replace any PC's just because of their OS has to be taken seriously.

On the flip side, as an IT professional I completely agree that anyone using unsupported OS/Applications should be talked to sternly

As a vendor Veeam is in a no win situation. Keeping the old stuff is expensive and can hinder/impede new features to support new OS and applications. But the good news that seems to be missed by many is that Veeam is quite good at supporting its own older versions. If the new version no longer supports the OS/Application version you need, then just continue using the older version of Veeam. v9.5 looks and feels old now, but it still works and is still supported.

[micoolpaul]

Hi Scott,

I hate to be the person that says ‘well, actually…’ but in this case it’s an important point to clarify. The 9.x releases are absolutely not under any support.

This has been the case since the 1st January 2022.

I did a write up of this over on the community hub: https://community.veeam.com/blogs-and-p ... -2022-1849

9.x’s successor v10 goes end of support in Feb 2023, so people need to consider moving away from that too…

[mkaec]

...As a MSP we have wide range of businesses that we support including for example manufacturing organisations...

Thanks for the samples of legitimate cases where it makes sense to stay with EOL software. I think most of us just get to see people being lazy. I've had multiple vendors tell my users to downgrade Java to a version with numerous known security exploits in order to keep accessing their web applications. Recently, the help desk has been busy turning TLS 1.0 back on after Microsoft turned it off via the latest CU because a vendor didn't get the memo that their site should be supporting TLS 1.2 by now. No million dollar hardware involved here. They just need a programmer to keep their stuff current.

[joachimeberhard]

I support an industry with about 50 Office PCs, 20 Servers and over 100 Industry machines.

Office PCs + Servers are all Windows 2019 Server or Enterprise LTSC covered by Software Assurance.

When it comes to Servers componets, I would say yes, you can require the most current.

However, when it comes to client support, we will still need the longest support I can get.

I still support machines running Windows NT 4.0 and Windows XP. And even more industrial machines running Windows 7.

Because you see, one machine costs 10 times as much as the entire IT including Servers, Workstations and Licenses.

So from my standpoint, I dont worry about old Servers or Exchange versions not being supported.

But what I see in the Industry still A LOT is NT4, XP, 7 (only those 3 OS Versions were popular in industry) and all the old SQL Server Versions starting from even 2005.

And yes, being able to backup a very important Windows 7 Industry machine daily with Veeam was one of the key selling points along with "reverse incremental" backups for full backups to Tape.

I hope this helps you, as I thought it is really important to tell you guys who your customers are.

Thanks for reading

[Regnor]

Just thinking; if a big part of the production machines is running on outdated hardware with End of Life software, does the backup software need to be up-to-date and covered by support? Or can you just run older releases on your own?
Of course official support would best, especially if you're having problems with restoring such old software. But does it really matter that much?

[vmtech123]

If you are running XP that was EOL in 2014. 8 years ago. I'd prefer Veeam not get large and bloated supporting stuff that has no business being on anyone's network. It's also going to increase the price due to testing and development and support wait times etc.

If you are running XP, you might as well run ESXI 5.5 and Veeam V7. Your hardware is off maintenance most likely in that case so why not run yourself a SAN that isn't on maintenance too.

I am joking, but you get the point. Planning has to be done when software is installed and plan those OS upgrades a few years early to not get into this mess. Waiting until the last month before EOL always causes issues. MS releases a new OS ever 2-3 years. That means there are 3 potential upgrades by the time this happens and 10 years. Long enough that it should have been planned out internally.,

Put the blame on the company that never provided an upgrade path from XP, or the business that bought software from such a small company without doing research or planning for upgrades.

I sure hope those XP machines are stand alone and not network connected also. Yikes.

[vmtech123]

I'll add, the percentage of people on XP would be less than 1%, by a long shot. The cost of Veeam supporting it will be large. Everything from Support, R and D, testing, releases etc. Do you expect the people maintaining a good product life cycle to cover these costs? Perhaps they could price it out add an added cost for legacy support, but then I'd still suggest just using an old version of Veeam. How long would you expect them to keep XP supported? at some point you are going to want to be able to backup 2030, but are you still going to want to include XP?

Remember when MS dropped all their driver support for legacy stuff? It was the best thing to happen to windows in a long time.

What do you do when the OS goes on those and you need MS support, or your software goes bad? do the vendors still support the software? if so, why won't they provide an update?

[joachimeberhard]

Granted, NT4 and XP are extreme cases, but I actually care about Win7 support and SQL.

ESXi and Server Software should indeed be latest.

But I still know some big German industrial machines manufacturers still shipping SQL 2005/8 Express and stuff. Today.

But when talking about 1M€ production machines in quantity, they will still be there for a long time...

And this is what I am hinting at...

And good software that supports this is rare these days.

[Regnor]

But I still know some big German industrial machines manufacturers still shipping SQL 2005/8 Express and stuff. Today.

Those companies are the problem. They're not really interested in keeping their products up-to-date and/or want to sell new hardware and services instead.
And as they're often holding a monopoly/oligopoly, customers can't really do anything against it or put pressure on them.

Finally the (backup) admin needs to step in and take responsibility for those systems, while no one else cares...
Not a great position to be in.
If the system breaks and you can't restore them, it's your fault.
If the bad guys take over the infrastructure via one of those unpatched EoL systems, it's your fault.

[joachimeberhard]

My dream solution would be sort of bootable Veeam ISO that can be used to make OS independant manual backups to your Veeam repos and restore from there.

Sort of like the old Ghost 2001 was, but with access to Veeam repos.

A man can dream...

[Regnor]

This would probably be sufficent for such systems as most of them are rather static and don't change at all.
Take a look at my feature request from 2014
veeam-agent-for-windows-f33/feature-req ... ilit=agent

[joachimeberhard]

Yes, something like this would be awesome.
Programming on industrial machines changes, but usually slowly.

And as I said, from the cost factor alone, but also the increasing mindset that throwing away old computers for "environmental intolerant software requirements" is a big thing.
A thing where big tech is on the wrong path. And people see through it...

But this is also a chance for new companies that focus on supporting environmental and cost friendly long-term solutions.

And I know from many IT peers in manufacturing industry, we WILL support old software, because the demand is now rising, and it will be there for decades to come.
Because of the cost factor alone in manufacturing industry.

[xudaiqing]

Manufacturing industry had quite different need compare to office PC and servers, change is extremely hard after install. Many industry systems never install any patch after the system is installed on site even if it is not using an legacy system.
One major fact is cost, an industry accident could cost billions of dollars and legal liabilities, which means any upgrade only feasible when entire factory is shutdown for upgrade and need to go through rampup process again. This could mean weeks or months of production lost.

[ITP-Stan]

A client of mine, also in manufacturing, has some (laser) +1M$ machines running Windows 95, Windows XP and Windows 7.
Ofcourse we take necesary security steps for these OS'es, but we still like to back-up at least the WIN7 ones with Veeam Agent.

I don't know how much R&D is necessary to keep an old version of the Veeam Agent working on Windows 7 and allowing the Agent to back-up to a Veeam B&R repo on v12+.
I wouldn't mind using an older unsupported Veeam agent on those machines, but it would be bothersome if it can't connect to a up-to-date Veeam B&R to store the backup.
I does not have to be a managed agent, just being able to store the back-up in the Veeam B&R repo.

Maybe Veeam B&R can be made to accept connections from older clients in some kind of legacy mode?

[vmtech123]

What happens when there is an XP or windows 95 or windows 7 issue where you need Microsoft support? or hardware issues? I'm sure that stuff isn't easy or even possible to get anymore.

[Gostev]

A client of mine, also in manufacturing, has some (laser) +1M$ machines running Windows 95, Windows XP and Windows 7.
Ofcourse we take necesary security steps for these OS'es, but we still like to back-up at least the WIN7 ones with Veeam Agent.

Please note that we never supported Windows 95 or Windows XP. And Windows 7 remains supported with V12 thus giving you many more happy years of Windows 7 backups, as V12 will be supported at least through 2025.

[Dima P.]

Stan,

A client of mine, also in manufacturing, has some (laser) +1M$ machines running Windows 95, Windows XP and Windows 7.

Anton is right - you can now, with v11a, and with upcoming v12 version use Windows Agent to protect Windows 7 SP1 without any issues. Been wondering if 95 or XP can in your case be protected on a file level via nas backup job or such machine does not have much unstructured data to protect and full image backup is required for possible Bare Metal Recovery? Thank you!

[joachimeberhard]

A LTS release is one of the best and most appreciated new Veeam approaches.

From what I lurked on the forums, this is what you plan for V12 yes? Making it LTS?

If thats the case, its one of the smartest and most customer friendly moves.

Appreciated. Most useful.

[Gostev]

Eventually we decided against LTS because it is not doable with our current R&D resources. What has changed our mind is some security issues we had to go back and fix in still-supported V10, which required moths of work to rewrite big chunks of the code and then fully re-test all functionality, as those changes touched pretty much everything. This made us realize that if we had (for example) the latest 9.5 update declared as LTS, there the same changes would require a few times longer, because it misses years of other architectural security improvements... so with enough bad luck, having LTS versions means we may end up spending most of our time just re-writing and patching those really old versions, instead of moving the product forward. This may be doable down the road once we double/triple our R&D, but certainly not at this stage.

I would question however this whole notion that customers who seem perfectly fine about running unsupported/unpatched core platform versions in their production environment are somehow unable or restricted from running unsupported Veeam versions which are still compatible with those outdated platforms. I would definitely expect some consistency as it comes to such policies.

Also, it seems unfair to not demand gorilla vendors like Microsoft or VMware, which both has relatively infinite resources, to continue providing support for their outdated software - and yet demand that Veeam continues to support same.

[mkaec]

Microsoft offers private paid support options after a product exits public support. Does Veeam offer something similar? That could address the resource issue while providing what customers with million dollar Windows 7 systems need.

[infused]

No one should be running outdated software like VMWare 6.0. If you're an MSP, you should be on top of this and planned for it years ago.

[Regnor]

@mkaec: Do you actually see companies paying Microsoft for the extended support? From my experience so far, no company with legacy/EoL Windows versions did.
I've only heard about the government paying for it; and they don't have dependencies or industrial machines, they're just too slow upgrading.

[xudaiqing]

What happens when there is an XP or windows 95 or windows 7 issue where you need Microsoft support? or hardware issues? I'm sure that stuff isn't easy or even possible to get anymore.

Normally the hardware manifacture will come and fix or replace the system. However they will need the backup file to restore configs for us. Usually in form of some file on the original file system.

[ITP-Stan]

I would question however this whole notion that customers who seem perfectly fine about running unsupported/unpatched core platform versions in their production environment are somehow unable or restricted from running unsupported Veeam versions which are still compatible with those outdated platforms. I would definitely expect some consistency as it comes to such policies.

As I said in my previous post, I'm fine running an unsupported Agent version.
In my experience however, older agent versions can't connect to an up-to-date Veeam B&R repo to store the back-ups.
And it's bothersome to have seperate Veeam B&R instance and seperate backup storage.
And subscription license wise is it even possible to keep an older Veeam B&R instance active next to a latest version?

To compare to Microsoft like you suggest.
Old Windows 7 OS can connect perfectly fine to newer Microsoft Server OS.

And we don't need Microsoft support (ever).
Either we fix it ourselves or the manufacturer will come re-install the system.

[Gostev]

I see this as a possibility that eventually, older agent versions will be able to connect to much newer backup servers. However, this likely won't happen until our agents become more or less "feature complete" and their protocol stops changing significantly between major releases to accommodate all the new features and improvements. However, at this time the overhead of supporting multiple protocol versions is very significant.

Importantly, a number of those protocol changes are done in order to improve security, and I can foresee some cases when we simply won't be able to keep the backup server still answering on the "old" protocol endpoint.

[mkaec]

@mkaec: Do you actually see companies paying Microsoft for the extended support? From my experience so far, no company with legacy/EoL Windows versions did.
I've only heard about the government paying for it; and they don't have dependencies or industrial machines, they're just too slow upgrading.

I think most companies do the upgrade versus pay Microsoft for post-EOL support. But I don't have experience with those industrial companies that have million-dollar XP systems. The calculus would be different for them.

[xudaiqing]

Industrial computers normally had fixed software stack and won't change what's installed on it after deployment, with closed dedicated network, usually not much could go wrong for them. Also in most case the standard procedure is to replace or reimage the system and restore application config and data if it's not an easy fix.

[micoolpaul]

Eventually we decided against LTS because it is not doable with our current R&D resources. What has changed our mind is some security issues we had to go back and fix in still-supported V10, which required moths of work to rewrite big chunks of the code and then fully re-test all functionality, as those changes touched pretty much everything. This made us realize that if we had (for example) the latest 9.5 update declared as LTS, there the same changes would require a few times longer, because it misses years of other architectural security improvements... so with enough bad luck, having LTS versions means we may end up spending most of our time just re-writing and patching those really old versions, instead of moving the product forward. This may be doable down the road once we double/triple our R&D, but certainly not at this stage.

I would question however this whole notion that customers who seem perfectly fine about running unsupported/unpatched core platform versions in their production environment are somehow unable or restricted from running unsupported Veeam versions which are still compatible with those outdated platforms. I would definitely expect some consistency as it comes to such policies.

Also, it seems unfair to not demand gorilla vendors like Microsoft or VMware, which both has relatively infinite resources, to continue providing support for their outdated software - and yet demand that Veeam continues to support same.

Hi Gostev/Veeam Team,

I understand the rationale behind not delivering an LTS version of Veeam, at the beginning of the thread you mentioned:

One other change we're planning for later this year is introducing an extended support policy. More on this when final decisions are made, but what's important now is that any environments affected by these changes will potentially be able to stay on V11 for longer than usual.

Was this comment purely when the idea of LTS was circulating at Veeam? Or is there potentially more to this? I'm working with Enterprise customers that don't want to jump straight to GA, but are conflicted due to the immediate "End of Fix" status when the next release goes GA. The lifecycle policy wording specifically stating that updates/patches/hotfixes no longer being created unless by exception, is causing issues with security-conscious organisations. To jump straight into a new version invites risks of course, and so organisations want to test and upgrade with planned regression testing to be confident in the deployment, but this could leave them in violation of frameworks they've agreed to ensuring all applications are under active development/support (appreciate that you do have 'support' in the way of Veeam staff assisting with issues, but not development effort).

This is a topic I'm repeatedly seeing, and I'm sure that Veeam are hearing this too. Will we see any lifecycle policy changes with v12 afterall?

[Gostev]

Yes, this comment was purely when the idea of LTS was circulating at Veeam. There are no lifecycle policy changes with v12 after all. However, security issues have always been one of the exceptions to the support policy anyway. And so are major issues experienced by our Enterprise customers for whom it's unrealistic to jump to the next major release overnight.