For full or single disk restore we recommend to use a hot-add proxy instead of NDB. You will get most likely better restore performance. My suggestion: Deploy a Linux or windows proxy on your ESXI host for the restore.
You can also open a case and let our customer support team investigate the bottleneck. Something we cannot do here in the forum. And I don't think another customers restore experience will help you to solve it in your specific environment.
I would start with testing a restore with a hot-add proxy and if that doesn't speed up things, contact our customer support team.
I can only agree with Fabian, the best way is indeed Hot-Add. When we were still using NBD the restore speed dropped drastically starting with vsphere 7U2, if I remember correctly. Installing some Linux VMs as Hot-Add Proxies on the hosts solved that nicely and the speed is better as it ever was before.