XFS performance question

[Troubleshooter]

Good afternoon Team!
I am interested to learn with you how to troubleshoot this issue about an XFS repository with vmware.
Backups are really really slow.

We have an xfs repo with an iscsi connection to a qnap through a trunk 10+10 gbps in load balancing, HDD only.
The linux vm is located on the same vmware infrastructure ( i know is not the best practice, but at that moment there was no other choice )
I see bottleneck sometimes on the source and sometimes on the target... always 99%.

Can you please tell me what information do you need and which test i can do to understand better?
This isnt an hardened repository, and the chain has been moved from an old refs repository with the Move function integrated in veeam.

I really appreciate your answer.
Thanks guys!

[david.domask]

Hi Tommaso,

Just to confirm, you're seeing performance issues with just all writes or specifically when fast clone is used, for example during Synthetic Full and merge operations?

I would start with the fio utility, as we have documentation on establishing base-line tests. This will only test normal writes though, and if the issue is with fast clone performance, you would need to just test cp with --reflink on a decently sized file, as this should help establish a baseline for expected performance.

Remember that bottlenecks don't necessarily indicate a problem -- something will always be "slowest" in the environment, and the actual read/write performance is more important to focus on.

[karsten123]

I would suggest to use real multi path io for your iscsi connection. Trunking iscsi nics is never a good idea. Or use a single connection.
Do you use jumbo frames? Do you use the same nics for iscsi and backup traffic?
What is the QNAP model, how many drives do you have, what type of drives are they and what RAID type do you use? Is it a thin LUN? Is it possible that your parity check is still running?
And what are the real performance numbers for your backup jobs?