Why do you think that you need to directly attach your tape drives to fiber ports on your server? There's no problem with an FC switch inbetween (except maybe total bandwidth - but in practice that was no problem here). I've had 6 LTO4 tapes connected to my server using one FC 4Gb Link via a FC Switch - worked like a charm.
Currently I don't use the switch any more because I didn't have any redundancy for it, but thats just me. Main point is: Windows has to see the tape drives. How that happens (FC direct, FC via Switch, SAS, Ethernet, whatever) is completely transparent to Veeam.