8.0.3.1044 - PostgreSQL database operation failed with transient error.

[hans.ovli]

Hi, after upgradingto V8 (8.0.3.1044) we have jobs that fails with: "Failed to create a restore point: PostgreSQL database operation failed multiple times with transient error."
Are there any known fix or approach to this? We also have unindexed repos after the upgrade. I am asking here since the support is not answering(since monday 16th) and this is getting urgent now. It seems mail is backed up ok, but most of the sharepoint,OneDrive and Teams job fails.
For your reference the support ticket is 07466079.
Thx in advance!

[Mildur]

Hi Hans

I moved your question to a new topic. Have you already used our Talk to Manager Button button to make our support management aware of the case?

The error itself is currently being investigated by our R&D team, but I cannot confirm if your situation is related to other cases.
Therefore, please continue the investigation with our customer support team. Use the "Talk to Manager" option if you don't receive an answer within a reasonable time.

Best,
Fabian

[daniel.rajczewski]

Hi,
I having problem with Microsoft 365 backups after upgrade VBM365 from v7 to v8 and migration database from localDB to PostgreSQL.
I made upgrade on 14 september and create support case on 16 september, (Case ID 07421954) still my problem wasn`t to be resolved.
Every day PostgreSQL service is stuck, i having error 'PostgreSQL database operation failed multiple times with transient error.'
I edit settings for PostgreSQL from this link https://helpcenter.veeam.com/docs/vbo36 ... tml?ver=80 but this dont work.
Today all services VBM, PostgreSQL, nats-server are running but i cant open VBM console.
I having error
'Endpoint handshake failed. Phase:CustomWorkflow. Could not finish read operation in timely manner'
Sorry but i very frustraited, this product doesnt work after upgrade / migratiom.
My support case having month i only sending new logs.
I having lacks on backup chains - on one day backup jobs was end success, but by next two days are failed.
I escalate this case to manager, but still no one can resolved my problem.
This situatuon is not accepted for me!

[Mildur]

Hi Daniel

I'm sorry to hear about your experiences with V8. I merged your topic together with a similar one.
The situation with "PostgreSQL database operation failed multiple times with transient error" is currently investigated by our R&D team. I'll ask our support management if they can compare your case with cases with the same error message.

BEst,
Fabian

[hans.ovli]

I have escalated through "Talk to Manager" and i have received a reply yesterday where they asked for logs which i sent right a way.
Now i am still waiting for an answer.

[daniel.rajczewski]

It is possible to rollback to version 7?
In my observation when backup jobs are saving to Object Storage, all information aboth change from last backup job on ver 7 was saving to PersistentCache folder, from version 8 all this data is saving to PostgreSQL Database.
I having on console 15 organizations, all organizations are having create 4 backup jobs (Mailbox, OneDrive, SharePoint, Teams) when few backup jobs are running at this same time PostgreSQL database is stucking, and all jobs having error 'PostgreSQL database operation failed multiple times with transient error.'
For the test I having second machine where I having backup to Object Repository for 4 organization, i having configurated 20 backup jobs and on this machive problem aren`t exist.
I having third machine with backup on premise (local repository) - 17 organizations wit 35 backup jobs and problem aren`t exist.
It is possible to back with backups created on VBM ver 8 againt to ver 7?

[Mildur]

Hi Daniel

No, unfortunately not. Upgraded repositories cannot be connected to an older build (V7a or earlier).



Best,
Fabian

[daniel.rajczewski]

Thanks Fabian for info,
Great! so I`m with damaged product withouth backups from few days!
Sorry for my sarsasm...

[m.novelli]

It seems VBO v8 to be little debacle. Never jump quickly on new major version

Marco

[hans.ovli]

I have escalated this matter for the second time yesterday.
Still not solved, one of our biggest customer has no Teams/OneDrive/Shrepoint backup since 15 of October
This can potentially get bad for us unfortunately.

[Mildur]

I will review the status of these cases from this topic.

Best,
Fabian

[maiquelbf]

Hi,

Does this problem have a solution already?

Maiquel

[Polina]

Hi @maiquelbf

This error is generic and can be caused by a variety of reasons; hence, each case has to be researched separately. Please open a support case and post its ID here for follow-up.

One thing you can check in advance, though, is whether your database is running on HDD or SDD. HDD is a common root cause of performance problems.

Thanks!

[daniel.rajczewski]

Dear Veeam,
When problem with performance of PostgreSQL will be resolved?!
Its been 2,5 months when I created Support Case and still broblem exist, no one contact with me, my first Case was close, despite the fact that I expected resolve.
I had to create Veeam Backup M365 machines, and spreade backup jobs between servers.
Now I having 7 servers, before upgrade to V8 and change DB to PosgtreSQL i had 2 servers, and problem still existing. On one day on week PostgreSQL is failing.
Additional 5 machines generate costs and usage of resources.
On every day i need to view all consoles, and all jobs because jobs are still running, but on window i having error.

Code: Select all

PostgreSQL database operation failed multiple times with transient error. . The maximum number of retries (10) was exceeded while executing database operations with 'ExtendedNpgsqlRetryingExecutionStrategy'. See the inner exception for the most recent failure.. Exception while reading from stream.

The problem is existing on backup to Object Storage.
Today it show on next machine where I saving backups on local disk on JET format.

I'm increasingly considering changing my strategy to protect M365 data by different software and leaving the repositories as an archive with VBM Comminity Edition until the end of retention is reached.
The problem is existing, and i dont see any steps to be resolved.

[Polina]

Hi Daniel,

First, let me say that I am sorry for the inconvenience you are facing.
As said above the "transient error" can be caused by various reasons, and requires an investigation in each case. Unfortunately, there's no silver bullet to fix it once and for all possible scenarios.

I checked the information on your first case, it seems that it was indeed closed automatically because our support team hasn't got any response from you for a while.
Could you please provide me with an ID of the case you have currently open?

Thanks!

[daniel.rajczewski]

Hi Polina,
Case ID #07501638

[DaStivi]

Hi Daniel has your issues been fixed? did you see or tuned the postgreSQL? according to this post someone helped this.. not sure if its really related to your issue but mostly all the issues sounds like postgreSQL "indexing" issues:
post527658.html#p527658

i do hessitate to update because all of the bad feedback i see on the forum about vb365 v8

[Polina]

@daniel.rajczewski

Did you have a chance to proceed with the latest steps provided by the support engineer?

[hans.ovli]

Just to note:
If there is a workaround for this i will like to know!
My ticket regarding "PostgreSQL database operation failed with transient error." are still not resolved.
R&N still researching, damn this is frustrating.

[daniel.rajczewski]

Hi @Polina
I makes this changes, for now problem doesnt exist. But I split few backup jobs from one console to three.

[hans.ovli]

I can now confirm that this error now has been fixed. It was performance issue with the disk that the PostgreSQL DB reside.
When we upgrade from standard SSD with 500 IOPS to Premium SSD with 5000 IOPS the issue disappeared.
The 365 platform now runs smoothly. My only objection to this problem is why that was'nt tried as a solution weeks ago.
Anyways this is now all good.