VAL6 and B&R - Bottleneck/Source info apparently wrong

[Flole]

When I run a backup to a Veeam B&R repo I am seeing the "Percent busy state" stats in B&R, and also in the logs of the agent. For some reason in my case the Agent is always at 0%, even though I am sure it's the bottleneck due to the source being I/O limited, everything else has a single-digit percentage.

Also in the Job.log it looks like this:

Code: Select all

... bottleneck: 0/10/0/8

Is this a known issue? Is anybody else seeing this? For obvious reasons it comes to the wrong conclusion then when showing the bottleneck. Sometimes the Agent indeed shows "Agent" as bottleneck, but mostly it's showing "Target".

[HannesK]

Hello,
the "always 0%" sounds like a technical issue. Please provide a support case ID for this issue, as requested when you click New Topic.
Without case number, the topic will eventually be
deleted by moderators.

In general: please check the sticky FAQ on the forums about bottleneck analysis.

Thank you,
Hannes

[Flole]

When I checked the "known issues" at post310953.html#p310953 it says:

"If you use Veeam Agent for Linux with a paid license installed, you can submit a support case to the Veeam Support Team at www.veeam.com/support.html. If you use Veeam Agent for Linux FREE, please submit issues by creating a separate topic for each issue. Please do not post log snippets, instead include the link to full logs package."

As I don't have a paid license I was under the impression that I can not submit a support case and I should submit an issue by creating a separate topic for each issue, which I did. Is that no longer correct?

I think I read the FAQ you are referring to. I remember reading something about "having a bottleneck is normal" and so on, and that's what I expect aswell, but the low numbers in my case seem odd and I am 100% sure the target is not the bottleneck as I ran 2 backups after each other and only a few Megabytes needed transferring and writing to the target while the Agent was reading multiple Terrabytes from the disk.

[HannesK]

Hello,
one can also open cases with the free version.

Which version are you using? Backup & Replication V12 and Veeam Agent for Linux V6? It looks similar in my lab, but I would not bet that source is really no problem in my environment

Best regards,
Hannes

[Flole]

Yes exactly that's what I'm using. A simple test by unloading the kernel module, restarting the service and then running a backup right after a backup finished should cause it to re-read the entire disk as CBT isn't persistent. In that case only a minimal amount of data should need to be transferred/written to the target IMO and the source should usually be the bottleneck.

[PTide]

If you opened a case please post the number.
Without the number I won't be able to fish it out of the queue and elevate its priority in order for our R&D to get it reviewed.

Thanks!

[Flole]

Since things got a lot worse now and backup performance significantly dropped for no reason down to 10 MB/s (it was well beyond 100 Mb/s before) and the target is mostly idle I went ahead and opened a support ticket to try to figure out what on earth is going on here now.

Looking at the logs I am seeing

Code: Select all

 0%, workload src: 0/7/2 (cpu 7, processing 0), ntf: 99/11/0

which seems to indicate the network on the target is the bottleneck? Absolutely ridiculous and completely wrong, I ran iperf (with a single connection) while the backup was still running, and it got almost 1 Gbit/s throughput (well, no surprise, Veeam is sending a burst and then doing nothing for quite a while until it sends another short burst). The throughput chart in B&R is empty/unusable aswell, that makes it even more unclear what might be going on.

The case number is 05961721.

[Flole]

Also it happened for the second time now that I opened a case, selected the EU storage bucket, selected that I want SFTP for uploading and got US SFTP credentials. That used to work in the past and seems broken now aswell.

[PTide]

Thanks for the case number, we will look into it soon.

As for the EU/US bucket - your logs are on s3.eu-west-1.amazonaws.com

Maybe the credentials were only labeled as US? Please elaborate.

Thanks!

[Flole]

I uploaded them using the Webinterface, the credentials show a server with "us" in it's name.