Discussions specific to the VMware vSphere hypervisor
davecla
Enthusiast
Posts: 25
Liked: 4 times
Joined: Feb 03, 2016 9:40 pm
Full Name: Dave Clarke
Contact:

Backup Copy Jobs not working when repository is Linux

Post by davecla » Aug 01, 2018 5:25 am

I've got a local veeam repository on linux have been running backup copy job to a NTFS based repository at secondary site for years - all was working well.

Wanted to move repository at remote site to linux. Created new repository and copied over local backups. Remapped Backup Job. Backup of servers at remote site to repository at remote site are working fine.

Want to move backup copy jobs to new remote linux repository. Copied over files and remapped copy job. Copy job now failed with error "No connection could be made because the target machine actively refused it xx.xx.xx.xx:2500.

I've got no idea how the backup job can work, but they backup copy does not. Anyone got any pointers?

thx
dave

nitramd
Expert
Posts: 136
Liked: 21 times
Joined: Feb 16, 2017 8:05 pm
Contact:

Re: Backup Copy Jobs not working when repository is Linux

Post by nitramd » Aug 01, 2018 1:22 pm

Firewall issue? Veeam B&R needs access to ports 2500-5000 on the repository.

Have you checked your Linux log files for clues?

zx81
Service Provider
Posts: 11
Liked: 1 time
Joined: Nov 24, 2016 6:57 am
Location: Perth, Australia
Contact:

[MERGED] Linux Backup repository refusing connections

Post by zx81 » Aug 08, 2018 8:18 am

Case ID# 03124279

I'm having intermittent problems with a Veeam linux repository. This repository is shared between a Veeam Backup and Replication server and a separate Veeam Cloud Connect server. I understand that you can share a repository in this way, but must take into account the number of concurrent tasks configured for this repository on each Veeam Backup server. And that the two servers obviously can't know how many tasks the other is running. I understand that the recommendation is for 1 task per core.

The repository in question is a physical machine with 2 CPUs, 12 cores and 96GB of RAM. I'm seeing intermittent backup copy job failures with the error No connection could be made because the target machine actively refused it 10.x.x.x:2514.

Is there a way to debug this issue on the linux repository server itself? When the problem is happening I can't find any issues on the linux server. It has plenty of free RAM, CPU and disk IO. It's clear that at some point the backup repository is refusing backup copy job data for some reason, but I've searched the syslog and auth logs for issues related to ssh or anything else and can't find any problems. I also notice that each backup copy job even when idle maintains a running process in the form /tmp/VeeamAgentxxxx, so I'm not sure how to work out if I'm exceeding the recommended number of tasks at a given time.

In short, does Veeam log anything on the linux repository that could help me determine why it seems to be refusing connections some of the time? How can I identify which processes to count to ensure I'm not exceeding 1 task per core?

Thanks,

PTide
Veeam Software
Posts: 4658
Liked: 386 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Linux Backup repository refusing connections

Post by PTide » Aug 08, 2018 4:59 pm

Hi,

Have you tried to run a backup job while the repo is refusing backup copy jobs? What is your source repository?

Thanks

PTide
Veeam Software
Posts: 4658
Liked: 386 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Backup Copy Jobs not working when repository is Linux

Post by PTide » Aug 08, 2018 5:00 pm

Hi Dave, sorry for the delay with response.

Have you contacted our support team already? If not then please do so and let us know your case ID.

Thanks

nitramd
Expert
Posts: 136
Liked: 21 times
Joined: Feb 16, 2017 8:05 pm
Contact:

Re: Backup Copy Jobs not working when repository is Linux

Post by nitramd » Aug 08, 2018 8:54 pm

@zx81, have you double checked your firewall settings? Ports 2500-5000 should be open on the repository.

zx81
Service Provider
Posts: 11
Liked: 1 time
Joined: Nov 24, 2016 6:57 am
Location: Perth, Australia
Contact:

Re: Backup Copy Jobs not working when repository is Linux

Post by zx81 » Aug 09, 2018 2:02 am

@nitramd There aren't any firewall rules between the Veeam Server and the repository. The problem is intermittent, the backups work most of the time. I could see how a firewall error that didn't open the range completely could cause this issue, but unfortunately that's not the case here.

zx81
Service Provider
Posts: 11
Liked: 1 time
Joined: Nov 24, 2016 6:57 am
Location: Perth, Australia
Contact:

Re: Linux Backup repository refusing connections

Post by zx81 » Aug 09, 2018 2:06 am

PTide wrote: Have you tried to run a backup job while the repo is refusing backup copy jobs? What is your source repository?
Hi,

Unfortunately I've never managed to catch the problem while it's live. Whenever I login to check and then force the job to Sync Now, it works. The source repository is another Linux based repository at a different site.

nitramd
Expert
Posts: 136
Liked: 21 times
Joined: Feb 16, 2017 8:05 pm
Contact:

Re: Backup Copy Jobs not working when repository is Linux

Post by nitramd » Aug 09, 2018 4:51 pm

@zx81,

Have a look at this post regarding a socket being forcibly closed - it may or may not apply: https://forums.veeam.com/veeam-backup-r ... 46532.html

PTide
Veeam Software
Posts: 4658
Liked: 386 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Backup Copy Jobs not working when repository is Linux

Post by PTide » Aug 10, 2018 11:49 am

zx81,

I've briefly reviewed your case and noticed that there were actually two support engineers who handled your case, and one of them (from VCSP support team) asked you to send them the logs so they could find out why exactly the target datamover process wasn't listening. Have you done that? The thing is that there are many reasons why the error could have occurred, including the target datamover process agent crash. On the other hand, if it was really due to lack of CPU, the datamover shouldn't have crashed. Please let the support team investigate the root cause.

Thank you.

zx81
Service Provider
Posts: 11
Liked: 1 time
Joined: Nov 24, 2016 6:57 am
Location: Perth, Australia
Contact:

Re: Backup Copy Jobs not working when repository is Linux

Post by zx81 » Aug 13, 2018 1:18 am

There are no outstanding requests for logs for the case (ID# 03124279) I referenced? I've already provided logs from the backup server, including that of the target repository that is refusing connections. So I'm not sure where you are getting that I haven't proactively been working with support on this case?

Maybe you're talking about yet another case for Veeam Cloud Connect? I'm also battling several Veeam problems that aren't the subject of a case (including ReFS hard hangs) and you're right, I haven't found time to get those logs yet. I have to prioritize the work on several Veeam problems.

PTide
Veeam Software
Posts: 4658
Liked: 386 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Backup Copy Jobs not working when repository is Linux

Post by PTide » Aug 13, 2018 5:54 am

I'm referencing case 03124279. Right, there seemingly were no outstanding requests, however the engineers have switched at some point, so I wanted to clarify whether or not the substitute engineer got your logs (I've contacted him already, so he will review the case again).

Thanks

Andreas Neufert
Veeam Software
Posts: 3142
Liked: 545 times
Joined: May 04, 2011 8:36 am
Full Name: @AndyandtheVMs Veeam PM
Location: Germany
Contact:

Re: Backup Copy Jobs not working when repository is Linux

Post by Andreas Neufert » Aug 13, 2018 6:56 am

Default SSHD Service on Linux is not optimized for the Veeam usage (multiple connections). Please change:
etc/ssh/sshd_conf settings:
ClientAliveInterval 30
TCPKeepAlive yes
ClientAliveCountMax 99999
MaxSessions 200
MaxStartups 100:30:200

davecla
Enthusiast
Posts: 25
Liked: 4 times
Joined: Feb 03, 2016 9:40 pm
Full Name: Dave Clarke
Contact:

Re: Backup Copy Jobs not working when repository is Linux

Post by davecla » Aug 26, 2018 9:28 pm 2 people like this post

Changes to /ssh/sshd_conf per @Andreas fixed our "target machine actively refused it xx.xx.xx.xx:2500" error.

All jobs completing without error now. Thanks all.

Andreas Neufert
Veeam Software
Posts: 3142
Liked: 545 times
Joined: May 04, 2011 8:36 am
Full Name: @AndyandtheVMs Veeam PM
Location: Germany
Contact:

Re: Backup Copy Jobs not working when repository is Linux

Post by Andreas Neufert » Aug 27, 2018 5:14 pm

Hi Dave, thanks. Which Linux OS Version do you use?

Post Reply

Who is online

Users browsing this forum: aaron.grenier, Google [Bot] and 31 guests