Comprehensive data protection for all workloads
Post Reply
FFulleringer
Novice
Posts: 8
Liked: 2 times
Joined: Mar 21, 2018 2:32 pm
Full Name: Cosme Fulanito
Contact:

SQL transaction logs jobs prevent other backup jobs from running

Post by FFulleringer »

Hello everyone! I'm hoping to get a little help. I'm having trouble when scheduling SQL Server logs backup (I set them to run every 15 minutes), if these jobs are active every other backup job won't start, they keep forever waiting for backup infrastructure resources availability and eventually fail with the error: Backup proxy not ready.
My backup server is Veeam Backup and Replication 11 running on Windows Server 2019, it is also the backup proxy since I only have 1 server dedicated to this. My repository is an HPE StoreOnce. Both server and storage were recently installed.
Before this I had a Windows Server 2012 and an IBM v7000 lun for repository, never had an issue like this (the number of virtual machines and databases haven't changed). The jobs were configured the same way as now.

The concurrent tasks for the backup proxy is set to 6, since the server has 6CPU and the concurrent tasks for the StoreOnce repository is unlimited.

Since the transaction logs jobs run constantly the resources never become available for the backup jobs. the SVC.Veeambackup log file won't display any error but it shows how the tasks remain sort of idle:

>[RTS] Tasks count statistics: [Completed: 0], [Pending: 13], [Preparing: 0], [Ready: 0], [ResponseGiven: 0], [WaitingForComplete: 27]


Like I said, it's the first time I have this problem but also the first time I use a Storeonce repository, maybe something is misconfigured.

Will appreciate any clue or idea provided.

Thanks a lot!
HannesK
Product Manager
Posts: 14844
Liked: 3086 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: SQL transaction logs jobs prevent other backup jobs from running

Post by HannesK »

Hello,
and the concurrent tasks for the StoreOnce repository is unlimited.
that's a bad idea. As you have an all-in-one system with only 6 CPU cores, it should be maximum 6. Probably you are already "overbooked" and below minimum system requirements if you sum up all your roles.

I currently cannot see, how you run out of proxy tasks if everything is configured exactly the same as before. Normally, I would just say "increase the proxy tasks", but the environment is already overbooked.

I see that you ran Server 2012 before: did you run an older VBR version on that server, or was it also V11?

Best regards,
Hannes
PetrM
Veeam Software
Posts: 3626
Liked: 608 times
Joined: Aug 28, 2013 8:23 am
Full Name: Petr Makarov
Location: Prague, Czech Republic
Contact:

Re: SQL transaction logs jobs prevent other backup jobs from running

Post by PetrM »

Hello,

Just one thing:
FFulleringer wrote:'m having trouble when scheduling SQL Server logs backup (I set them to run every 15 minutes), if these jobs are active every other backup job won't start, they keep forever waiting for backup infrastructure resources availability and eventually fail with the error: Backup proxy not ready.
It cannot be related to T-log backup jobs as they do not consume task slots: log backup is a critical operation for a database that must be performed according to the strictly defined schedule to prevent the complete filling up of the production disk, so we never wait for infrastructure resources availability in this case. I guess there are some "orphaned" tasks pending somewhere in our service cache and most probably the issue will disappear for some time after the Veeam Backup Service restart. Anyway, I recommend opening a support case for a proper technical analysis.

Thanks!
FFulleringer
Novice
Posts: 8
Liked: 2 times
Joined: Mar 21, 2018 2:32 pm
Full Name: Cosme Fulanito
Contact:

Re: SQL transaction logs jobs prevent other backup jobs from running

Post by FFulleringer »

HannesK wrote: Oct 20, 2022 5:40 am Hello,

that's a bad idea. As you have an all-in-one system with only 6 CPU cores, it should be maximum 6. Probably you are already "overbooked" and below minimum system requirements if you sum up all your roles.

I currently cannot see, how you run out of proxy tasks if everything is configured exactly the same as before. Normally, I would just say "increase the proxy tasks", but the environment is already overbooked.

I see that you ran Server 2012 before: did you run an older VBR version on that server, or was it also V11?

Best regards,
Hannes
Thanks for your reply. The 2012 was running Veeam B&R 9.5. Not 11. I just don't know how many CPUS the Storeonce has but the HPE engineer who installed the storage told me to keep it unlimited (tasks for the repository). Tasks for the Proxy are in 1xCPU.
FFulleringer
Novice
Posts: 8
Liked: 2 times
Joined: Mar 21, 2018 2:32 pm
Full Name: Cosme Fulanito
Contact:

Re: SQL transaction logs jobs prevent other backup jobs from running

Post by FFulleringer »

PetrM wrote: Oct 20, 2022 2:55 pm Hello,

Just one thing:


It cannot be related to T-log backup jobs as they do not consume task slots: log backup is a critical operation for a database that must be performed according to the strictly defined schedule to prevent the complete filling up of the production disk, so we never wait for infrastructure resources availability in this case. I guess there are some "orphaned" tasks pending somewhere in our service cache and most probably the issue will disappear for some time after the Veeam Backup Service restart. Anyway, I recommend opening a support case for a proper technical analysis.

Thanks!
Today I checked the backups from the weekend, one of them was still "running". It backed up 3 VM's and stopped, waiting for resources availability. Backup Proxy Not Ready for 20 hours. Edited the job, disabled guest proccesing and the job started running again, finished in about 5 minutes.

That's why I think the problem is Transaction logs / Application Awareness related.

Thank you.
FFulleringer
Novice
Posts: 8
Liked: 2 times
Joined: Mar 21, 2018 2:32 pm
Full Name: Cosme Fulanito
Contact:

Re: SQL transaction logs jobs prevent other backup jobs from running

Post by FFulleringer »

Opened support case. Case # 05688419.
PetrM
Veeam Software
Posts: 3626
Liked: 608 times
Joined: Aug 28, 2013 8:23 am
Full Name: Petr Makarov
Location: Prague, Czech Republic
Contact:

Re: SQL transaction logs jobs prevent other backup jobs from running

Post by PetrM »

HI Cosme,

It's an interesting observation but anyway I would treat it as a technical problem because I don't see any objective or design-related reasons for the jobs waiting for T-logs backup completion. I believe our engineers should have a look at debug logs and probably Veeam Backup Service trace or dump to sort out the issue.

Thanks!
Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 78 guests