Standalone backup agents for Linux, Mac, AIX & Solaris workloads on-premises or in the public cloud
Post Reply
Flole
Influencer
Posts: 11
Liked: 1 time
Joined: Feb 06, 2021 2:43 am
Contact:

VAL6 and B&R - Bottleneck/Source info apparently wrong

Post by Flole »

When I run a backup to a Veeam B&R repo I am seeing the "Percent busy state" stats in B&R, and also in the logs of the agent. For some reason in my case the Agent is always at 0%, even though I am sure it's the bottleneck due to the source being I/O limited, everything else has a single-digit percentage.

Also in the Job.log it looks like this:

Code: Select all

... bottleneck: 0/10/0/8
Is this a known issue? Is anybody else seeing this? For obvious reasons it comes to the wrong conclusion then when showing the bottleneck. Sometimes the Agent indeed shows "Agent" as bottleneck, but mostly it's showing "Target".
HannesK
Product Manager
Posts: 14314
Liked: 2888 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: VAL6 and B&R - Bottleneck/Source info apparently wrong

Post by HannesK »

Hello,
the "always 0%" sounds like a technical issue. Please provide a support case ID for this issue, as requested when you click New Topic.
Without case number, the topic will eventually be
deleted by moderators
.

In general: please check the sticky FAQ on the forums about bottleneck analysis.

Thank you,
Hannes
Flole
Influencer
Posts: 11
Liked: 1 time
Joined: Feb 06, 2021 2:43 am
Contact:

Re: VAL6 and B&R - Bottleneck/Source info apparently wrong

Post by Flole »

When I checked the "known issues" at post310953.html#p310953 it says:

"If you use Veeam Agent for Linux with a paid license installed, you can submit a support case to the Veeam Support Team at www.veeam.com/support.html. If you use Veeam Agent for Linux FREE, please submit issues by creating a separate topic for each issue. Please do not post log snippets, instead include the link to full logs package."

As I don't have a paid license I was under the impression that I can not submit a support case and I should submit an issue by creating a separate topic for each issue, which I did. Is that no longer correct?

I think I read the FAQ you are referring to. I remember reading something about "having a bottleneck is normal" and so on, and that's what I expect aswell, but the low numbers in my case seem odd and I am 100% sure the target is not the bottleneck as I ran 2 backups after each other and only a few Megabytes needed transferring and writing to the target while the Agent was reading multiple Terrabytes from the disk.
HannesK
Product Manager
Posts: 14314
Liked: 2888 times
Joined: Sep 01, 2014 11:46 am
Full Name: Hannes Kasparick
Location: Austria
Contact:

Re: VAL6 and B&R - Bottleneck/Source info apparently wrong

Post by HannesK »

Hello,
one can also open cases with the free version.

Which version are you using? Backup & Replication V12 and Veeam Agent for Linux V6? It looks similar in my lab, but I would not bet that source is really no problem in my environment

Best regards,
Hannes
Flole
Influencer
Posts: 11
Liked: 1 time
Joined: Feb 06, 2021 2:43 am
Contact:

Re: VAL6 and B&R - Bottleneck/Source info apparently wrong

Post by Flole »

Yes exactly that's what I'm using. A simple test by unloading the kernel module, restarting the service and then running a backup right after a backup finished should cause it to re-read the entire disk as CBT isn't persistent. In that case only a minimal amount of data should need to be transferred/written to the target IMO and the source should usually be the bottleneck.
PTide
Product Manager
Posts: 6428
Liked: 729 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: VAL6 and B&R - Bottleneck/Source info apparently wrong

Post by PTide »

If you opened a case please post the number.
Without the number I won't be able to fish it out of the queue and elevate its priority in order for our R&D to get it reviewed.

Thanks!
Flole
Influencer
Posts: 11
Liked: 1 time
Joined: Feb 06, 2021 2:43 am
Contact:

Re: VAL6 and B&R - Bottleneck/Source info apparently wrong

Post by Flole »

Since things got a lot worse now and backup performance significantly dropped for no reason down to 10 MB/s (it was well beyond 100 Mb/s before) and the target is mostly idle I went ahead and opened a support ticket to try to figure out what on earth is going on here now.

Looking at the logs I am seeing

Code: Select all

 0%, workload src: 0/7/2 (cpu 7, processing 0), ntf: 99/11/0
which seems to indicate the network on the target is the bottleneck? Absolutely ridiculous and completely wrong, I ran iperf (with a single connection) while the backup was still running, and it got almost 1 Gbit/s throughput (well, no surprise, Veeam is sending a burst and then doing nothing for quite a while until it sends another short burst). The throughput chart in B&R is empty/unusable aswell, that makes it even more unclear what might be going on.

The case number is 05961721.
Flole
Influencer
Posts: 11
Liked: 1 time
Joined: Feb 06, 2021 2:43 am
Contact:

Re: VAL6 and B&R - Bottleneck/Source info apparently wrong

Post by Flole »

Also it happened for the second time now that I opened a case, selected the EU storage bucket, selected that I want SFTP for uploading and got US SFTP credentials. That used to work in the past and seems broken now aswell.
PTide
Product Manager
Posts: 6428
Liked: 729 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: VAL6 and B&R - Bottleneck/Source info apparently wrong

Post by PTide »

Thanks for the case number, we will look into it soon.

As for the EU/US bucket - your logs are on s3.eu-west-1.amazonaws.com

Maybe the credentials were only labeled as US? Please elaborate.

Thanks!
Flole
Influencer
Posts: 11
Liked: 1 time
Joined: Feb 06, 2021 2:43 am
Contact:

Re: VAL6 and B&R - Bottleneck/Source info apparently wrong

Post by Flole »

I uploaded them using the Webinterface, the credentials show a server with "us" in it's name.
Post Reply

Who is online

Users browsing this forum: Baidu [Spider] and 8 guests