First: Welcome to the forums!
If I understand it correctly (I can be wrong, still at my first coffee
) you are taking backups with the agents. Those backups are stored in the VBR repository. Correct?
If so, then replication won't work. What you should consider is do a backup copy job to that secondary repository off-site.
But, unfortunately, you won't be able to exclude files from that backup (sorry about that)