Discussions specific to the Microsoft Hyper-V hypervisor
Post Reply
pesos
Expert
Posts: 139
Liked: 9 times
Joined: Nov 12, 2014 9:40 am
Full Name: John Johnson
Contact:

can't get hardened repository working

Post by pesos »

We've set up an ubuntu server per the instructions here:
https://nolabnoparty.com/en/veeam-v11-h ... lity-pt-1/
https://www.starwindsoftware.com/blog/v ... ory-part-1

Have created and recreated the repository partition a few times now and it never works. I am unable to create a subfolder within the repo from veeam (agent invoke failed with non retryable error
permission denied
failed to create directory '/veeamrepo/test/'
agent failed to process method [FileSystem.DirectoryCreate}

If I try to just continue creating the repository using the /veeamrepo root without a subfolder I get "file system behind the provided path is not XFS: /veeamrepo"

Mildur
Veeam Legend
Posts: 1464
Liked: 608 times
Joined: May 13, 2017 4:51 pm
Full Name: Fabian Kessler
Location: St. Gallen, Switzerland
Contact:

Re: can't get hardened repository working

Post by Mildur »

Is the User („single-use credentials“) you have used to configure the backup repo the owner of the path on the Linux System?
Are you using Veeam Agent V5.0?
VMCE 9 | Veeam Legends 2021
Working with Veeam since 2017 for a VCSP in Switzerland

pesos
Expert
Posts: 139
Liked: 9 times
Joined: Nov 12, 2014 9:40 am
Full Name: John Johnson
Contact:

Re: can't get hardened repository working

Post by pesos »

I am using build 11.0.0.837 p20210525, so whatever that pushes to the linux box is what's installed.

The local account (same one used for single use creds) was set up as the owner of the path, per the blogs linked above
sudo chown -R locveeam:locveeam /mnt/veeamrepo
sudo chmod 700 veeamxfsrepo01-02

thanks!

wishr
Veeam Software
Posts: 2687
Liked: 362 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: can't get hardened repository working

Post by wishr »

Hi John,

When configuring any Veeam components and software we recommend relying only on the official documentation posted on our website.

Since you are experiencing a technical issue, please open a case with our technical support team directly, as troubleshooting is not possible through the forums (as mentioned when you click "New Topic").

Thanks

pesos
Expert
Posts: 139
Liked: 9 times
Joined: Nov 12, 2014 9:40 am
Full Name: John Johnson
Contact:

Re: can't get hardened repository working

Post by pesos »

No problem, will open a case. Just wanted to make sure I wasn't doing something really linux-noob-stupid.

wishr
Veeam Software
Posts: 2687
Liked: 362 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: can't get hardened repository working

Post by wishr »

The only thing that looks strange to me is that the path is different between the two lines you posted above.

pesos
Expert
Posts: 139
Liked: 9 times
Joined: Nov 12, 2014 9:40 am
Full Name: John Johnson
Contact:

Re: can't get hardened repository working

Post by pesos »

yeah that 2nd line was a copy paste from the blog. it had the right path when i ran it on my actual machine

wishr
Veeam Software
Posts: 2687
Liked: 362 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: can't get hardened repository working

Post by wishr »

Got it :) What does the below line show in the output?

Code: Select all

ll /mnt/

pesos
Expert
Posts: 139
Liked: 9 times
Joined: Nov 12, 2014 9:40 am
Full Name: John Johnson
Contact:

Re: can't get hardened repository working

Post by pesos »

Image

wishr
Veeam Software
Posts: 2687
Liked: 362 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: can't get hardened repository working

Post by wishr »

At the first sight, it looks fine so I would suggest continuing the investigation with our tech. support engineers. Probably there is an issue with the mount or XFS configuration.

pesos
Expert
Posts: 139
Liked: 9 times
Joined: Nov 12, 2014 9:40 am
Full Name: John Johnson
Contact:

[MERGED] hardened repository not working

Post by pesos »

case 04851782

Unable to get hardened repository working. Have rebuilt from scratch and still no luck. Once we actually try to start running backups, veeam attempts to create a subfolder for the backup files and fails:

7/9/2021 1:56:48 PM :: Error: Permission denied
Failed to create directory '/mnt/hardrepo1/nextcloudHR/'
Agent failed to process method {FileSystem.DirectoryCreate}.

Have gone through the exact configs and steps taken with support and they've verified it all looks correct.

Natalia Lupacheva
Veeam Software
Posts: 867
Liked: 199 times
Joined: Apr 27, 2020 12:46 pm
Full Name: Natalia Lupacheva
Contact:

Re: can't get hardened repository working

Post by Natalia Lupacheva »

Hi John,

I've moved your post to the existing thread to keep similar issues together.
I see you have quite a long discussion with Support and already tried several ways to figure out how to fix this failure. So please keep working with Support team to beat this problem.

Thanks!

pesos
Expert
Posts: 139
Liked: 9 times
Joined: Nov 12, 2014 9:40 am
Full Name: John Johnson
Contact:

Re: can't get hardened repository working

Post by pesos »

Thanks Natalia. Finally a breakthrough with support today after a 3rd rebuild of the linux server.

Turns out it's looking like it's a bug in the Veeam software that reverts the /mnt/repofolder ownership/perms back to root once it's added in the veeam console... which in turn prevents the repo user veeam leverages from being able to create subfolders for actual job storage. We caught it in the act this time, reverted ownership to the repo user account and then we were able to run jobs. Support is going to repro in their lab and look at sorting out where the ownership is getting reversed.

soncscy
Veeam Legend
Posts: 483
Liked: 239 times
Joined: Aug 04, 2019 2:57 pm
Full Name: Harvey Carel
Contact:

Re: can't get hardened repository working

Post by soncscy »

Hi John,

I've set up quite a few of these, and never had this issue, so I'm not quite sure that it's accurate to say that the software does it.

What distribution are you using? An exact link to the download you used would be great, since even the same distribution release might be a different branch.

pesos
Expert
Posts: 139
Liked: 9 times
Joined: Nov 12, 2014 9:40 am
Full Name: John Johnson
Contact:

Re: can't get hardened repository working

Post by pesos » 1 person likes this post

It’s the current/latest version of Ubuntu server. Just passing along what we experienced and support is leaning towards after multiple repros. Will definitely follow up here once they can repro in their own lab and provide more detail.

pesos
Expert
Posts: 139
Liked: 9 times
Joined: Nov 12, 2014 9:40 am
Full Name: John Johnson
Contact:

Re: can't get hardened repository working

Post by pesos » 1 person likes this post

Update:

“The issue is being looked at by our tier 3 support team as well as our research and development teams, so the way we were configuring the repository initially is not the cause of the permissions denied failures. There is likely an issue with the default umask setting for users on specific distros of Linux. When Veeam is deploying the components to the repository the permission end up being reverted to root 755. The good news is that at this time the issue is being actively worked on for a more sustainable resolution, the unfortunate part is that at this time to only solution to what we noticed is manual correction as we did on our call.”

Gostev
SVP, Product Management
Posts: 28902
Liked: 5271 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: can't get hardened repository working

Post by Gostev »

By current/latest version of Ubuntu, do you mean Ubuntu 21.04?

pesos
Expert
Posts: 139
Liked: 9 times
Joined: Nov 12, 2014 9:40 am
Full Name: John Johnson
Contact:

Re: can't get hardened repository working

Post by pesos »

Hi Anton,

It's 20.04.2 LTS

Gostev
SVP, Product Management
Posts: 28902
Liked: 5271 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: can't get hardened repository working

Post by Gostev »

Thanks, I wonder if this is a change in .2 because normally V11 should have received plenty of testing with .1...

pesos
Expert
Posts: 139
Liked: 9 times
Joined: Nov 12, 2014 9:40 am
Full Name: John Johnson
Contact:

Re: can't get hardened repository working

Post by pesos » 1 person likes this post

I asked if there was a recommended distro and heard back that:

"Many other debian distros have shown to not encounter this issue, specifically older version of Ubuntu such as version of 19. The main cause of this issue is when the user's umask is 077. If you notice that the user account when created in the Linux server has the umask of 077 then you would need to follow the same steps."

tomaskalabis
Novice
Posts: 4
Liked: never
Joined: Oct 19, 2016 6:36 pm
Full Name: Tomas Kalabis
Contact:

Re: can't get hardened repository working

Post by tomaskalabis »

same problem here.
ubuntu 20.10
vbr 11.0.0.837
i have 3 backup jobs to linux repo all failing at: 23.08.2021 10:33:34 :: Failed to pre-process the job Error: Permission denied
Failed to create directory '/veeam/HOSTING - Backup ALL to XFSsecure-repo/'
Agent failed to process method {FileSystem.DirectoryCreate}.

wishr
Veeam Software
Posts: 2687
Liked: 362 times
Joined: Aug 07, 2018 3:11 pm
Full Name: Fedor Maslov
Contact:

Re: can't get hardened repository working

Post by wishr »

Hi Tomas,

Have you already opened a ticket? If so, could you please share your case ID, as requested when you click "New topic"?

Thanks

pesos
Expert
Posts: 139
Liked: 9 times
Joined: Nov 12, 2014 9:40 am
Full Name: John Johnson
Contact:

Re: can't get hardened repository working

Post by pesos » 1 person likes this post

Support should be able to get you the specific steps to fix the permission-breaking that veeam is doing. I didn’t note the exact steps but here is the basic idea:

“ I have good and unfortunate news regarding what we discovered on our webex for the Linux repository. The issue is being looked at by our tier 3 support team as well as our research and development teams, so the way we were configuring the repository initially is not the cause of the permissions denied failures. There is likely an issue with the default umask setting for users on specific distros of Linux. When Veeam is deploying the components to the repository the permission end up being reverted to root 755. The good news is that at this time the issue is being actively worked on for a more sustainable resolution, the unfortunate part is that at this time to only solution to what we noticed is manual correction as we did on our call. ”

Post Reply

Who is online

Users browsing this forum: No registered users and 9 guests