If you manage to virtualize a server accessing tape you can go for it (I'm pretty sure you won't be able to do that with Hyper-V, but you can do that with VMWare if you present a physical HBA to a virtual machine exclusively). But you can connect any physical server to a tape drive and install Veeam agent on it (Veeam agent will "talk" to the tape). So that means "yes" and "no" depending on what you call Veeam server. You don't have to install Veeam Backup and Replication on a server for it to "talk" to tape, but you have to install Veeam agent instead.
If I understand your idea of design properly: short answer is Yes. Long answer is above.