-
- Enthusiast
- Posts: 52
- Liked: never
- Joined: Oct 28, 2015 9:36 pm
- Full Name: Joe Brancaleone
- Contact:
Veeam service on repo not removing immutable attribute on older files
I have had a case open Case #07148096 on this issue. So far I have had no useful suggestions, nor does this seem to be a known issue. Around mid to late January I updated our Veeam B&R environment from 12 to 12.1, and also had our Linux sysadmin reboot the Linux hardened repos to apply kernel updates.
It seems that since then, Veeam jobs will end in a failure status when attempting to apply the 42 restore point retention policy, because the oldest vbk and vib files in the backup chain are still immutable. This means that for some reason the 14 most recent backups immutable policy is not being enforced when it comes to *removing* the immutability attribute.
I have tried starting new backup chains w/ new fulls on different repositories, but the issue persists across all repositories. I have had to work with the sysadmin to get root access to manually remove immutability from older files using chattr -i, so that Veeam can apply some retention policies and our repo capacities are under control.
Does this problem sound familiar to anyone? I don't even know where the breakage has happened between the VBR server issuing the info on immutable policies, and the local lock files/metadata that is supposed to be tracking immutability expirations.
According to this white paper on pg 17-19:
https://contenthub.gmevents.ae/wp-conte ... ackups.pdf
the Veeam service which sets and unsets extended attributes veeamimmureposvsc runs as root. Presumably that service is operational in terms of having root access, since it can clearly set the immutable attribute on newly created files. But something is preventing it from removing the immutable attribute on older files. Or it is not properly reading the lock and file metadata to get updated info on immutability expiration?
It seems that since then, Veeam jobs will end in a failure status when attempting to apply the 42 restore point retention policy, because the oldest vbk and vib files in the backup chain are still immutable. This means that for some reason the 14 most recent backups immutable policy is not being enforced when it comes to *removing* the immutability attribute.
I have tried starting new backup chains w/ new fulls on different repositories, but the issue persists across all repositories. I have had to work with the sysadmin to get root access to manually remove immutability from older files using chattr -i, so that Veeam can apply some retention policies and our repo capacities are under control.
Does this problem sound familiar to anyone? I don't even know where the breakage has happened between the VBR server issuing the info on immutable policies, and the local lock files/metadata that is supposed to be tracking immutability expirations.
According to this white paper on pg 17-19:
https://contenthub.gmevents.ae/wp-conte ... ackups.pdf
the Veeam service which sets and unsets extended attributes veeamimmureposvsc runs as root. Presumably that service is operational in terms of having root access, since it can clearly set the immutable attribute on newly created files. But something is preventing it from removing the immutable attribute on older files. Or it is not properly reading the lock and file metadata to get updated info on immutability expiration?
-
- Veeam Software
- Posts: 2313
- Liked: 553 times
- Joined: Jun 28, 2016 12:12 pm
- Contact:
Re: Veeam service on repo not removing immutable attribute on older files
Hi @joebranca,
Thank you for your post here and sorry to hear that the issue is still on-going.
The case was recently escalated to our Advanced Support Team, and I can see that you recently uploaded fresh logs for their review. The behavior is not expected, so let's allow the Advanced Support Team a bit of time to review the newest logs and we will get to the bottom of this issue.
From my time in Support, I've seen something similar to this when the .lock files on the Linux Hardened Repository have somehow gotten unexpected owner/group, but let's allow the Advanced Support Team time to check this a bit more deeply. I also suspect as you do that the Immutable Service (or the lock files it works with) are having permission difficulties, and I'll comment the case as such.
Thank you for your post here and sorry to hear that the issue is still on-going.
The case was recently escalated to our Advanced Support Team, and I can see that you recently uploaded fresh logs for their review. The behavior is not expected, so let's allow the Advanced Support Team a bit of time to review the newest logs and we will get to the bottom of this issue.
From my time in Support, I've seen something similar to this when the .lock files on the Linux Hardened Repository have somehow gotten unexpected owner/group, but let's allow the Advanced Support Team time to check this a bit more deeply. I also suspect as you do that the Immutable Service (or the lock files it works with) are having permission difficulties, and I'll comment the case as such.
David Domask | Product Management: Principal Analyst
-
- Enthusiast
- Posts: 52
- Liked: never
- Joined: Oct 28, 2015 9:36 pm
- Full Name: Joe Brancaleone
- Contact:
Re: Veeam service on repo not removing immutable attribute on older files
Thanks for the update. This has been quite the head scratcher!
-
- Service Provider
- Posts: 25
- Liked: 6 times
- Joined: Jan 03, 2020 10:08 am
- Full Name: Oliver Palz
- Contact:
Re: Veeam service on repo not removing immutable attribute on older files
Any news on this one?
I just opened case #07299050 with the same issue...
I just opened case #07299050 with the same issue...
--
You wanna Talk? Check my Calendar @Bookings
MS Bookings: https://bit.ly/3028OME
Xing: https://www.xing.com/profile/Oliver_Palz
LinkedIn: https://www.linkedin.com/in/oliverpalz/
You wanna Talk? Check my Calendar @Bookings
MS Bookings: https://bit.ly/3028OME
Xing: https://www.xing.com/profile/Oliver_Palz
LinkedIn: https://www.linkedin.com/in/oliverpalz/
-
- Product Manager
- Posts: 10062
- Liked: 2675 times
- Joined: May 13, 2017 4:51 pm
- Full Name: Fabian K.
- Location: Switzerland
- Contact:
Re: Veeam service on repo not removing immutable attribute on older files
Hi Oliver
I can't see a solution yet in the other case.
Please keep working with your support engineer.
Best,
Fabian
I can't see a solution yet in the other case.
Please keep working with your support engineer.
Best,
Fabian
Product Management Analyst @ Veeam Software
-
- Lurker
- Posts: 1
- Liked: never
- Joined: Oct 01, 2018 9:46 am
- Full Name: Pieter
- Contact:
Re: Veeam service on repo not removing immutable attribute on older files
Hi,
We have the same issue after an update from 12 to 12.1
The flags can be reset manually as root but that is only a temp workaround.
Case # 07295148
Kr
We have the same issue after an update from 12 to 12.1
The flags can be reset manually as root but that is only a temp workaround.
Case # 07295148
Kr
-
- Product Manager
- Posts: 14951
- Liked: 3148 times
- Joined: Sep 01, 2014 11:46 am
- Full Name: Hannes Kasparick
- Location: Austria
- Contact:
Re: Veeam service on repo not removing immutable attribute on older files
Hello,
just by chance... did anyone of you follow a Red Had blog post for installation and has 2775 permissions on the repository paths? SGID is not supported anymore for security reasons and the Red Hat blog post was updated to "chmod 700" some weeks ago.
Best regards,
Hannes
just by chance... did anyone of you follow a Red Had blog post for installation and has 2775 permissions on the repository paths? SGID is not supported anymore for security reasons and the Red Hat blog post was updated to "chmod 700" some weeks ago.
Best regards,
Hannes