Offsite backup conundrum

[Vertigo]

Hi everyone

Ok so I'm currently using Veeam Agent for Windows at home to backup my PC to my own server. This works really well and I'm very impressed with the performance.

The problem is now offsite backup. What I currently have to do is regularly plug in an external drive, copy the Veeam files from the server and then keep the drive at work. This is cumbersome and I tend to forget to do it regularly enough.

What I want is to do the the same via the Internet. Put a spare drive into a machine at work, then VPN in from home and copy the Veeam files that way.

The problem, of course, is bandwidth. Whilst I can obviously "seed" the drive at work manually to start with, I need some method of updating the files at block level so only changes are transmitted.

This is proving virtually impossible. I've tried a few "file sync" utilities which claim to be able to do block-level updates to files but they just don't work, possibly because the "main" Veeam file changes name as each expired incremental is merged into it daily.

Ironically, the way Veeam merges each incremental into the main file as it expires is pretty much exactly what I need. If only I could have Veeam do the "same again" and update a second set of files in a different location once a week, that'd work superbly as I'd only be sending changes across the wire.

Is what I need possible with Veeam at all? I don't even mind paying for a more functional licence if it does what I need (although obviously I'm not a business so not going to be spending hundreds/thousands).

I was almost thinking I could put another instance of Veeam Agent on the server itself and then backup the backup files to the remote location. Clunky but would that work?!

Thanks for any advice.
Toby.

[wishr]

Hi Toby,

I suggest you simply spin up Veeam Backup & Replication Community Edition at home that is free of charge and utilize Backup Copy Jobs because they are designed specifically for copying data to alternate locations. You may additionally use Copy Jobs Mapping to speed up the initial copy.

Thanks

[Egor Yakovlev]

Another scenario is to have Veeam Agent for Windows "Server edition" - it supports multiple jobs creation, which will allow you to have 1 job for local backup, and 1 job for offsite backup on same agent.
Cheers!

[Vertigo]

Thanks both.

I did look at Backup Copy Jobs which seemed to do what I wanted but I don't think it uses block-level differencing does it? In this case it would end up just copying all the backup files in their entirety which puts me back where I started.
I was also under the impression that the Backup & Replication was purely for VMs rather than just basic file/partition backup?

With regards to the server edition of the Agent, yes that would let me just have two separate jobs on the PC, one to my server daily and one to remote storage weekly but I can't find any pricing on it and I'd assume that, as a "server" version, it's not going to be cheap?

Cheers.

[wishr]

Hi Vertigo,

Please check this section and adjacent ones for info regarding block tracking and other technologies we provide to minimize the amount of data to be transferred during Copy Job runs.

B&R also offers Managed Mode for Agents that might be interesting for you, and moreover, with Community Edition you can deploy up to three Server Edition Agents for free! Our intention at Veeam is to provide data protection solutions for all types of workloads: physical, virtual, and cloud.

Thanks

[Vertigo]

So are you saying I should deploy B&R Community to my server (Windows Server 2016 box) and I can then deploy Server Edition Agents to my client PC (Windows 10 Pro), all for free and it should do what I need?

Sounds too good to be true to be honest

[wishr]

Exactly as you described - too good to be true, but it is . With CE you get ten instances that can be used for backing up any workload of any type and each Server Edition Agent consumes three instances. Workstation Edition Agent consumes just one so you can use it in conjunction with Backup Copy Jobs and utilize the remaining instances for something else.

[Vertigo]

Ok sounds great, I'll give it a go when I get a chance.

I appreciate my situation is rather atypical in that I'm a home user but run a server with a full server OS. Many of the solutions I've looked at restrict the deployment of their free or "home user" editions to non-Server OSes, obviously as a way of ensuring business users don't abuse these licence types.

Thanks

[Vertigo]

Ok so I've been playing with this a bit and it all looks pretty awesome.

I've installed B&R on my server, deployed the Windows Agent to a test PC with a fresh Windows install to start with as it's a smaller, more manageable dataset.
I've got the main backup job working to a repository on the server no problems.

I've then established a VPN to my office and used the Add Server function to deploy a gateway server to my PC and added a repository to it.
I've then configured a backup copy job to backup the original backup job files to the remote machine.

This all seems to be working but I do have a few quick questions...

1. When the time comes to do this for my main PC, the data size will be much larger and so I'd be looking to seed the backup copy. I understand I can do this by copying the main backup files to the target PC (by a faster means such as an external drive) but where exactly do I put them in the repository folder on the target box? Do they need to go into a folder with a certain name or anything?

2. Whilst my main backup runs daily, I want to schedule the backup copy job to run once a week, in the early hours of Saturday morning when it won't interfere with regular business bandwidth at the office. I presume I can set the interval to every 7 days at 2am but how do I tell it which day of the week to run? Do I have to start it on a Saturday so it then runs every Saturday thereafter? I'm a bit confused by the copy interval thing at the moment.

3. As I mentioned, the remote machine is accessed via a VPN which I need to establish only when required. Can I just use the before/after script settings within the copy job to do this? Would the job ever try to access the destination repository at other times? Obviously, when the VPN is down, the repository won't be accessible.

Thanks for the awesome software and help so far!

Toby.

[wishr]

Hi Toby,

Please find the answers below.

1. Currently, Backup Copy Mapping does not support Managed Agent Backups. Please refer to the "Limitations" section here for details.
2. You may use Backup Copy Windows while keeping the interval set at 7 days.
3. There is no backup data transmission outside of allowed Backup Copy Window, however, retention, as well as other operations such as backup merge, may be applied/performed even outside of the allowed backup copy window, thus a stable connection with the backup copy target is highly recommended. This will be a bit simplified in the next version of B&R that is expected to be released very soon.

Thanks!

[Vertigo]

Thanks again for the help. I've been playing around and it actually seems to be working pretty well.

Regards the backup copy over the VPN, this seems to be successfully establishing the VPN, completing the data transfer and then shutting down the connection again. It remains to be seen whether other operations, such as those you mention, will cause problems but I'm going to leave my test scenario running for a week or so (with retention at only a few days) to see how it goes. Unfortunately I can't realistically maintain the VPN indefinitely.

It's unfortunate I can't make use of the mapping functionality. Is there any other way I can seed the remote copy, such as doing it initially to a local device and then moving the files to the remote repository? If not then I'll just have to do the initial backup over the VPN which will take a couple of days.

[wishr]

Hi Toby,

You may overcome the current limitation of Backup Copy Mapping by using the Standalone Agent. So Backup & Replication will be responsible for only copying the backup created by VAW operating independently of the backup server. This way you'll be able to seed the copy as you explained.

Thanks

[Vertigo]

Thanks for that, although I may just allow it to seed over the VPN as I can just leave it running for a couple of days over Christmas.

Out of interest, does the job interval start from the beginning or the end of the previous interval?
For example, if I were to create a new backup copy job on Sunday at 10am but set the interval to every 7 days at 2am, the next interval would be the following Sunday at 2am.
If the initial backup took a couple of days, however, as it will for me, does the 7 day interval start when the initial backup finishes, which could be Tuesday sometime, so it would then run future incrementals on Tuesdays?

Basically I want to run weekly at 2am on Sundays as it's out of office hours and I'm not interfering with business bandwidth. If I know that the initial backup is going to take a couple of days and the interval runs from the end I could start the initial backup on a Friday so it ends on Sunday and then runs every Sunday thereafter, if that makes sense. This is hard to test empirically without actually doing a multiple-day full backup.

[wishr]

You are most welcome!

It all depends on the configuration of the copy job. Please check this and adjacent guide sections.

Thanks.

[Vertigo]

Ah ok I think I understand.

If I want to run each week at Sunday at 2am, it looks like I need to create/start the job on Sunday AFTER 2am (not hard). If I were to start it before 2am then it would fail as it wouldn't be able to complete the initial backup before the next interval kicked in at 2am.

It also seems to say that a new interval would be forced at 2am each Sunday, regardless of how long the previous interval/backup took to complete, which is ideal.

It also seems that each interval will "wait" for something to do, so if an interval occurs but there are no new restore points available to backup, it would wait until there were, so as soon as a new restore point appears, it would be copied immediately rather than waiting for the next Sunday 2am interval. That's not ideal for my scenario as it would potentially leave the VPN up while it waits for a new restore point to appear but, in reality, shouldn't be an issue because, on a weekly schedule, it's highly unlikely there wouldn't be any new restore points. A setting to skip the interval completely if there are no restore points would be very hand though

[asdffdsa6131]

if you want to copy veeam backup files over a vpn or to the cloud, the way you are doing it is not practical.
as you mention, veeam will merge incrementals in the main file and this you would need to upload that large main file.
there is no need for block-level file copy programs.

1. use daily forever foward incremental backups and full backups once a week or month.
https://helpcenter.veeam.com/docs/backu ... l?ver=95u4
"If the synthetic full backup and/or active full backups are not enabled, Veeam Backup & Replication will produce a forever forward incremental backup chain."
2. copy those files over the vpn, using any file copy program such as fastcopy, which does checksum.
3. use rclone to copy the files to the cloud, which also does checksum of files

[Vertigo]

Sorry but this is exactly the opposite of my experience.

I tried the method you suggested to start with but the issue is that the name and content of the full backup is changed as each expired incremental is merged into it, so the entire file would need to be re-copied over the VPN. No block-level copy program I tried could cope with it.

Using backup copy jobs in Veeam is working perfectly. The expired incrementals are merged into the full backup over the VPN in a couple of minutes, so clearly it doesn't need to read/access the entire full backup to do the merge.

[asdffdsa6131]

the setup i described, i have been using for over a year now, each and every day.
i set the "keep backups" to 730 days and as a result, i never ran into your issue of expiring incrementals and the resultant merge.

[Vertigo]

That's hardly a practical solution, for me at least. Firstly the storage requirements of keeping two years worth of daily backups will skyrocket. Secondly if I want to do a full BMR restore, it'll have to merge 700+ incrementals together in order to construct a current, full backup.

[asdffdsa6131]

your original post was about using veeam agent and off-site backup and vpn.
my solution is FREE, VERY simple and works well over a vpn and copying to cloud and does not require veeam backup and replication.

for vpn, i use the FREE veeampn.
i have fios symmetrical gigabit connections.
each site has a cheap NAS.
the NAS itself has software to copy the .vib and .vbk files to cloud, i use wasabi.

i use this procedure on over two dozens computers/servers, about 12 per site.

i do a daily incremental and a monthly full, no synthetics, no defrag, no compact.
for each computer, i schedule each monthly full on a different day, so never more then one .vbk to copy per day.

a BMR would at most would need the last full and up to 30 days of incremental.
and once a month, i spend five minutes, pruning/deleteing out the old .vib and .vbk as needed.