CVE-2023-5869 in PostgreSQL

[AlexHeylin]

Should we upgrade PG manually, or are Veeam working on a patch for this?
As PG is not covered by MS Updates, what's Veeam's approach to handling security issues etc in PG now it's the default DB?

[tyler.jurgens]

I expect it's the same as every other database you would have used for Veeam - you need to manage it yourself.

Veeam never patched SQL Server, nor SQL Express during its updates (at least, not that I'm aware of). It would deploy the version it needed to get the install going, then it was up to the administrator to keep it patched. I'm guessing many didn't bother.

The better question would be: Is there a compatibility matrix for Veeam + Postgres so you can easily see which versions you can target for patching?

[AlexHeylin]

Hi Tyler,
You're right of course, Veeam didn't (to my knowledge) patch MS SQL. However MS did, so that was taken care of. I'm assuming Veeam discussed internally what they are going to do about patching PG SQL - which isn't supported by MS update so isn't taken care of automatically.

If they're not going to do it, that's OK (though not great) but as you said, we're going to need updated compatibility matrix etc if we're doing this manually.

I'm not keen on having yet another bit of software to manually maintain, especially one that secures the backups. Running a fleet of Veeam (MSP) already takes a lot of maintaining, and adding a DB which we have to maintain completely separately and manually feels like a step in the wrong direction.

[Mildur]

Hello

We currently don't have plans to include PostgreSQL updates in our update process. Updating the configuration database engine is still a task of the customer/partner.

Supported PostgreSQL versions are in our user guide. You can update to PostgreSQL 14.10 or 15.5 which will mitigate the security vulnerability for the PostgreSQL server.

Local or remote installation of the following versions of PostgreSQL:

- PostgreSQL 14.x
- PostgreSQL 15.x (PostgreSQL 15.1 is included in the Veeam Backup & Replication setup, but we strongly recommend to download and install the latest PostgreSQL 15.x version)

For the issue of this CVE, it's important to note that the security vulnerability can only be leveraged if an attacker gets authenticated against the database. For that, he needs to be able to connect to the database and have its credentials.

My recommendation:
- For all in One installations (backup server + PostgreSQL server on the same machine) limit access to the backup server to the remote backup console.
- For advanced deployments where backup server and PostgreSQL server are on different machines, make sure that only the backup server is allowed to access the database server and it's databases.


Best,
Fabian

[AlexHeylin]

Thanks Fabian, especially for the mitigation advice. Unfortunately our security scanning system doesn't know about mitigations, so we're going to need to patch to resolve the alerts. I'll do the first few machines manually, then look at if we can automate the upgrade via RMM script.
Thanks both.

[AlexHeylin]

Automated install looks fine. I couldn't get automated upgrade to work. Weirdly if I believe the UI then PGSQL upgraded while VBR was running and VBR didn't seem to notice, at least not so much it surfaced in the GUI.

It would be good if Veeam were able to spend any time trying to get automated upgrades to work and share their results. At some point I think they're going to need to do it during a VBR upgrade anyway.

[DerOest]

I'm currently testing the 12.1 upgrade and noticed that the "integrated" Postgres database did not get upgraded.

For MS SQL it was easy - that comes basically automatically by means of WSUS/Windows Update.

But now that PostgreSQL is included and the new default - you should take care of this!
To me it feels like a lot of systems will stay unpatched if you don't include it in your updates!

Veeam as a compony displays a huge drive for resiliency and security - just look at the new features in 12.1
So to me it looks strange, why Veeam would not update all integral components!


At least give a fat warning/notice if the Veeam installer detects outdated PostgreSQL versions and link to easy documentation detailing the upgrade procedure.
I'm a backup administrator, not a database administrator!

[Mildur]

Hi DerOeast

At least give a fat warning/notice if the Veeam installer detects outdated PostgreSQL versions

Thanks for the request. We're already planning to research on this topic, but we don't have an ETA yet.

and link to easy documentation detailing the upgrade procedure.
I'm a backup administrator, not a database administrator!

I'm not a database administrator either, but I can share information I've found on PostgreSQL's websites.

Upgrading to a minor version (e.g., 15.4 --> 15.5) is straightforward. Make sure you have a backup of the database. Veeam's configuration backup is sufficient if the PostgreSQL server only holds the Veeam database.
Steps:
1.) Stop the database services
2.) Install the new minor version
3.) Apply additional configurations if required (Check the release notes. Not all minor versions require it)
4.) Start the database services
Source: https://www.postgresql.org/support/versioning/ (Chapter Upgrading)

Upgrading to a minor release does not normally require a dump and restore; you can stop the database server, install the updated binaries, and restart the server. For some releases, manual changes may be required to complete the upgrade, so always read the release notes before upgrading.

Upgrade to a Major version (example 14 --> 15) requires more steps and a PostgreSQL command: https://www.postgresql.org/docs/15/pgupgrade.html

I recommend keeping it simple and updating only within your major version. V12 was shipped with PostgreSQL v15.1, so you should perform a minor update to v15.5. Don't forget to create a configuration backup.

Best,
Fabian

[DerOest]

Hi Mildur - much appreciated reply!

Sure, minor upgrades should be no problem. Simply running the windows installer with "next next next" leads to an upgraded instance.

But here is where my headache comes from - this is a minor release upgrade 15.4 -> 15.5:

https://www.postgresql.org/docs/release/15.5/

However, several mistakes have been discovered that could lead to certain types of indexes yielding wrong search results or being unnecessarily inefficient. It is advisable to REINDEX potentially-affected indexes after installing this update. See the fourth through seventh changelog entries below.

"wrong results" sounds scary, and i have no means to understand indepth if we/Veeam-DB would be affected.
Neither would i know how to run such an reindex (and i don't intend to learn that, as again, i'm not a database admin...)

And i think Anton Gostev wrote somewhere else that Veeam now runs on 1 Million Servers - at that scale, not every customer should have to reinvent the wheel - it should better be done by one central player

[Gostev]

There's no disagreement that this will become important as PostgreSQL adoption increases, and we do plan to manage PostgreSQL upgrades eventually. It just has not been a priority because for now, with almost the entire above-mentioned million of VBR servers still uses Microsoft SQL Server configuration databases. But of course, this ratio will start changing quickly with the default install deploying PostgreSQL starting from V12. So I'm hoping we will have required dev resources available to make V13 setup wizard upgrade PostgreSQL as well - at least the requirement for this is already created by the responsible PM.

[DerOest]

Sounds absolutely reasonable, thanks again for this insight.
You guys make Veeam so special, I have never encountered another (non-opensource^^) company so engaged with the community. You make it so easy to get in touch - I know you really take a lot of time out of your days to keep this going and i absolutely appreciate that!

[AlexHeylin]

Nudge based on veeam-backup-replication-f2/postgresql- ... 92352.html