Comprehensive data protection for all workloads
Post Reply
guitarfish
Enthusiast
Posts: 98
Liked: 12 times
Joined: Mar 06, 2013 4:12 pm
Contact:

Backup Copy job "Success" despite fatal error

Post by guitarfish »

Over the last couple weeks we started getting errors in our Backup Copy jobs. I worked with Veeam support as well as a couple other vendors, and found the error was caused by the Intrusion Prevention System on a firewall. The specific error was "Failed to decrypt TLS data: (336130329)". The issue is resolved, but we didn't actually realize it was happening for about a week, because B&R was reporting the jobs as "Success", while in fact they were fatally flawed.

When a BC job completes, an email is sent. Here's an example:
Image

All looks well, but look at the actual log of said job:
Image

The TLS error results in the data not being transferred completely, and the cloud copy is not valid. I found it quite alarming that the job was listed as a Success. This error should be flagged in red and the job should've been reported as Failed. For now, my trust in the daily emails is gone.
Dima P.
Product Manager
Posts: 14417
Liked: 1576 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: Backup Copy job "Success" despite fatal error

Post by Dima P. »

Hello guitarfish,

Thank you for your feedback, we will share it with RnD folks! Any reason you are still using the legacy copy job engine instead of new mirror / periodic backup copy jobs?
guitarfish
Enthusiast
Posts: 98
Liked: 12 times
Joined: Mar 06, 2013 4:12 pm
Contact:

Re: Backup Copy job "Success" despite fatal error

Post by guitarfish »

I recently upgraded to v12, haven't gotten much further yet.
chris.childerhose
Veeam Vanguard
Posts: 573
Liked: 132 times
Joined: Aug 13, 2014 6:03 pm
Full Name: Chris Childerhose
Location: Toronto, ON
Contact:

Re: Backup Copy job "Success" despite fatal error

Post by chris.childerhose »

It will be interesting to see if the same is reported when you upgrade to the new v12 copy jobs and backup chains.
-----------------------
Chris Childerhose
Veeam Vanguard / Veeam Legend / Veeam Ceritified Architect / VMCE
vExpert / VCAP-DCA / VCP8 / MCITP
Personal blog: https://just-virtualization.tech
Twitter: @cchilderhose
Dima P.
Product Manager
Posts: 14417
Liked: 1576 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: Backup Copy job "Success" despite fatal error

Post by Dima P. »

Thank you for the details! guitarfish, can you please share the case ID you were referring to?
JaySt
Service Provider
Posts: 415
Liked: 75 times
Joined: Jun 09, 2015 7:08 pm
Full Name: JaySt
Contact:

Re: Backup Copy job "Success" despite fatal error

Post by JaySt »

seeing this error as well. Any more information about this perhaps? was this patched somewhere in the last couple of patches or was it perhaps fixable by upgrading the chain and the job to the new mirror mode?
Veeam Certified Engineer
Dima P.
Product Manager
Posts: 14417
Liked: 1576 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: Backup Copy job "Success" despite fatal error

Post by Dima P. »

seeing this error as well. Any more information about this perhaps?
Can you please share your case with us? We need more logs to pinpoint the root cause.
it perhaps fixable by upgrading the chain and the job to the new mirror mode
Correct, new backup copy has different under the hood logic, so we were not able to reproduce this issue.
JaySt
Service Provider
Posts: 415
Liked: 75 times
Joined: Jun 09, 2015 7:08 pm
Full Name: JaySt
Contact:

Re: Backup Copy job "Success" despite fatal error

Post by JaySt » 1 person likes this post

no case yet. I'll probably put effort in upgrading the job and chain first. seems like the better plan.
Veeam Certified Engineer
chris.childerhose
Veeam Vanguard
Posts: 573
Liked: 132 times
Joined: Aug 13, 2014 6:03 pm
Full Name: Chris Childerhose
Location: Toronto, ON
Contact:

Re: Backup Copy job "Success" despite fatal error

Post by chris.childerhose »

Hopefully, the job and chain upgrade solves it for you. I will be interested once that is done.
-----------------------
Chris Childerhose
Veeam Vanguard / Veeam Legend / Veeam Ceritified Architect / VMCE
vExpert / VCAP-DCA / VCP8 / MCITP
Personal blog: https://just-virtualization.tech
Twitter: @cchilderhose
JaySt
Service Provider
Posts: 415
Liked: 75 times
Joined: Jun 09, 2015 7:08 pm
Full Name: JaySt
Contact:

Re: Backup Copy job "Success" despite fatal error

Post by JaySt » 1 person likes this post

wrong assumption about the chain format. It was allready upgraded. will probabl need to pick it up with support later.
Veeam Certified Engineer
jcheung22
Lurker
Posts: 2
Liked: never
Joined: Nov 10, 2016 3:20 pm
Full Name: jcheung
Contact:

Re: Backup Copy job "Success" despite fatal error

Post by jcheung22 »

Hi. I'm seeing this error too on backup copy jobs. Seems to be intermittent, i.e. some VHDX files for some VMs get this error, then the error disappears the following night. We have B&R v12.1.1.56 and Meraki MX firewalls.

@guitarfish what was your fix? Thanks.
MarkBoothmaa
Veeam Legend
Posts: 181
Liked: 49 times
Joined: Mar 22, 2017 11:10 am
Full Name: Mark Boothman
Location: Darlington, United Kingdom
Contact:

Re: Backup Copy job "Success" despite fatal error

Post by MarkBoothmaa »

Your not doing SSL decryption on the firewalls are you?
JaySt
Service Provider
Posts: 415
Liked: 75 times
Joined: Jun 09, 2015 7:08 pm
Full Name: JaySt
Contact:

Re: Backup Copy job "Success" despite fatal error

Post by JaySt »

jcheung22 wrote: Mar 04, 2024 3:40 pm We have B&R v12.1.1.56 and Meraki MX firewalls.
ah! Meraki in my case as well. They have been suspect for a while and they still are. Something going on there.
Veeam Certified Engineer
jcheung22
Lurker
Posts: 2
Liked: never
Joined: Nov 10, 2016 3:20 pm
Full Name: jcheung
Contact:

Re: Backup Copy job "Success" despite fatal error

Post by jcheung22 »

Thanks, not sure about the SSL decryption - will check. Currently on MX 18.107.8 firmware,
Post Reply

Who is online

Users browsing this forum: Google [Bot] and 87 guests