Having a single backup server will, indeed, give you more control of job schedule, so that you could avoid job overlaps.
Can you please tell me how many jobs you have in total? Based on your infrastructure description, I would say that a single backup server should be more than enough. Also what kind of VM stun do you refer to? The one that might happen when VM snapshot gets committed?