Comprehensive data protection for all workloads
Post Reply
Railtracer
Novice
Posts: 4
Liked: never
Joined: May 02, 2022 2:41 pm
Full Name: Samuel Patten
Contact:

After rotating drives, Job does the error/restarting message.

Post by Railtracer »

I have been trying to track down information on this and feeling pretty stupid. After upgrading to V12 I am seeing Errors for the first run of two backup copy jobs in immediate mode after the drives get rotated out, Veeam then restarts the job and completes. I recall this being a message level discrepancy that was previously changed to a warning message. I am not sure if I am seeing the currently expected behavior or not, my research has not clarified it for me.

Just to make sure I am describing this intelligibly: After every drive rotation, immediate mode copy jobs "fail" with "Error: Rotated drive was changed, restarting backup copy job" They then proceed to complete as expected. Am I crazy when I am thinking this was previously changed to a non error condition or should I actually be engaging support? The fail email notifications always look bad, but the job is still getting the job done.
tomtom94
Novice
Posts: 9
Liked: never
Joined: Dec 02, 2022 4:53 pm
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by tomtom94 »

Its an old bug that got introduced with V11, seems to be back with V12.

We had the same error upgrading from V10 to V11 and now in V12.
Currently using a mail filter to get rid of the mails so the monitoring system doesnt go red every day.

Here is the old thread with V11: veeam-backup-replication-f2/v11-and-rot ... 72491.html
Railtracer
Novice
Posts: 4
Liked: never
Joined: May 02, 2022 2:41 pm
Full Name: Samuel Patten
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by Railtracer »

Yeah it does behave exactly like that. I suppose that is better than me doing something wrong. It also corresponded to when one of these external drives died and got replaced. Hopefully we can get some kind of official word on if this behavior has returned.
lupowolfi
Service Provider
Posts: 11
Liked: 4 times
Joined: May 06, 2016 9:56 am
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by lupowolfi »

We receive the same message for some of our customers after upgrading to V12.
#02954883
praxxzz
Novice
Posts: 9
Liked: never
Joined: Mar 08, 2017 7:33 am
Full Name: Markus Praxenthaler
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by praxxzz »

We have the same message as well.
lupowolfi
Service Provider
Posts: 11
Liked: 4 times
Joined: May 06, 2016 9:56 am
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by lupowolfi »

lupowolfi wrote: May 24, 2023 12:05 pm We receive the same message for some of our customers after upgrading to V12.
#02954883
Apparently I shared the wrong case number and can no longer edit the post...
Case #06080970 would be the correct one
tomtom94
Novice
Posts: 9
Liked: never
Joined: Dec 02, 2022 4:53 pm
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by tomtom94 »

Finally found some time to open a support ticket: #02595537
lupowolfi
Service Provider
Posts: 11
Liked: 4 times
Joined: May 06, 2016 9:56 am
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by lupowolfi »

Support answered as followes:

"...this is a cosmetic issue and can be safely ignored as long as it happens only right after the disk is changed and the next session is successful. We are aware of such behavior and will fix it in one of the upcoming patches. ..."

Still, I hope the problem will be fixed quickly, because even though it is only a cosmetic issue, we lose time answering questions from concerned customers. :roll:
lupowolfi
Service Provider
Posts: 11
Liked: 4 times
Joined: May 06, 2016 9:56 am
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by lupowolfi » 1 person likes this post

Nearly half a year and two patches later, the issue still seems to persist. :(
How is the status?
Gostev
Chief Product Officer
Posts: 31561
Liked: 6725 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by Gostev »

@Egor Yakovlev please comment.
Egor Yakovlev
Veeam Software
Posts: 2537
Liked: 683 times
Joined: Jun 14, 2013 9:30 am
Full Name: Egor Yakovlev
Location: Prague, Czech Republic
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by Egor Yakovlev »

This bug was fixed back in December 2022.
We are running internal investigation to find the roots of the problem.

Will come back as soon as I get details.
Egor Yakovlev
Veeam Software
Posts: 2537
Liked: 683 times
Joined: Jun 14, 2013 9:30 am
Full Name: Egor Yakovlev
Location: Prague, Czech Republic
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by Egor Yakovlev » 3 people like this post

Here are some news: initially reported problem was with Backup Copy Immediate mode and image-level backups as a source(virtual machines), and this one was fixed long time ago.
Unfortunately, it appears that the same issue revealed with agent-based backups later on - we have a confirmation in QA labs and fix is on the way.
We also see some improvements can be done around RD sessions behavior as a whole, and planning to adjust it.

/Thanks for bringing this up!
Dima P.
Product Manager
Posts: 14417
Liked: 1576 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by Dima P. » 1 person likes this post

Hello folks,

We found the root cause of the problem and plan to fix it in the upcoming version. Thank you for sharing this issue!
xasz
Influencer
Posts: 13
Liked: never
Joined: May 21, 2017 8:03 pm
Full Name: Michael Schneider

Re: After rotating drives, Job does the error/restarting message.

Post by xasz »

We have the same problem. I have watched this thread for an reply.
@Dima P. Are we talking about a upcoming Major Version or an Patch?
We have lot's of customers using RDX drives and it really hard to monitor :)
Dima P.
Product Manager
Posts: 14417
Liked: 1576 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by Dima P. »

Michael,

Unfortunately the fix is quite big for a private fix or a patch, so we will have in the upcoming v12 update. Thank you!
FrazierCO
Novice
Posts: 5
Liked: never
Joined: Jun 30, 2021 2:28 pm
Full Name: Howard Frazier
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by FrazierCO »

@Dima P. What is the timeline for the release of the v12 update which will include the fix for this issue?
Dima P.
Product Manager
Posts: 14417
Liked: 1576 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by Dima P. »

Hello Howard,

Unfortunately I cant give you dates but we are working hard to deliver the mentioned update as soon as possible. Stay tuned for product updates and announcements!
lupowolfi
Service Provider
Posts: 11
Liked: 4 times
Joined: May 06, 2016 9:56 am
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by lupowolfi »

I only recently discovered this issue seems to cause another bigger problem.

Apparently if the rotated drive backed repository is configured to delete all backups upon drive change, this triggers the deletion of the media, even if the drive was not changed.
We have a few customers who swap their RDX drives only on a weekly basis.
Normally you would be left with a week of backups on the media (1 full + 6 increments).
But because of this problem, the media content gets deleted daily and a full backup is created.
So only one backup point remains on the media.
Gostev
Chief Product Officer
Posts: 31561
Liked: 6725 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by Gostev »

This seems to be a completely different issue than what is being discussed in this thread? This thread is about invalid notification, while what you're describing is basically data loss situation.

In any case, this is not an expected behavior: the backups should only ever be deleted when the media is rotated. I suggest you open a support case to investigate what you're seeing. This is not currently a known issue afaik.
mxxrxxs
Lurker
Posts: 2
Liked: never
Joined: Oct 10, 2023 8:22 am
Full Name: Maurice
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by mxxrxxs »

Any news on a potential (aimed) date for fixing?
FrazierCO
Novice
Posts: 5
Liked: never
Joined: Jun 30, 2021 2:28 pm
Full Name: Howard Frazier
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by FrazierCO »

+1

This really is a problem... Is it possible to upgrade the priority?
Dima P.
Product Manager
Posts: 14417
Liked: 1576 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by Dima P. »

Hi folks,

The fix will be provided with the upcoming 23H2 update. Thank you!
mxxrxxs
Lurker
Posts: 2
Liked: never
Joined: Oct 10, 2023 8:22 am
Full Name: Maurice
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by mxxrxxs »

any date when that 23H2 patch will be available?
Gostev
Chief Product Officer
Posts: 31561
Liked: 6725 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by Gostev » 1 person likes this post

We're planning to ship it before the end of this year.
lupowolfi
Service Provider
Posts: 11
Liked: 4 times
Joined: May 06, 2016 9:56 am
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by lupowolfi »

Gostev wrote: Sep 19, 2023 10:57 am This seems to be a completely different issue than what is being discussed in this thread? This thread is about invalid notification, while what you're describing is basically data loss situation.

In any case, this is not an expected behavior: the backups should only ever be deleted when the media is rotated. I suggest you open a support case to investigate what you're seeing. This is not currently a known issue afaik.
You are right, the issue seems to be a different one. (Case # 06356719)
Apparently if the option "Delete all backups" is set on a repository with rotated media, a backup copy job might delete content from media, even though the physical media has not been changed.
In my eyes this is a regression from the earlier version introduced in v12 with the new backup chain format and "Parent Backup" and "Child Backups" concept.
TeltisMM
Lurker
Posts: 1
Liked: never
Joined: Jan 27, 2024 6:17 am
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by TeltisMM »

Is it possible to put the information about the specific error and job details for event 490 (and maybe all others where its currently missing) back into windows event log?

I remember a while ago we had something like this: Job details: Failed to pre-process the job Error: Rotated drive was changed, restarting backup copy job
Dima P.
Product Manager
Posts: 14417
Liked: 1576 times
Joined: Feb 04, 2013 2:07 pm
Full Name: Dmitry Popov
Location: Prague
Contact:

Re: After rotating drives, Job does the error/restarting message.

Post by Dima P. » 1 person likes this post

TeltisMM,

This issue was addressed in the latest Veeam B&R 12.1 version, the false-positive error is no longer added to the job log. Thank you!
Post Reply

Who is online

Users browsing this forum: Semrush [Bot] and 106 guests