Backup copy nightmare!

[mephisto]

Hey guys,

I'm trialling a solution for a client where we are virtualizing all infra-structure and testing offsite backup jobs. I'm trying to use backup copy but for some very odd reason it runs only for some backups and not for others.

The backup job created with several VMs about 20 days ago with reverse incremental to local disk runs fine, but the backup copy is always waiting for new restore points, even if I run it manually the backup copy doesn't do anything. I've already changed the "copy every" from 5 minutes to 100 days and nothing happens, never.

If I create a new backup job with a couple of VMs, it works just fine. What the hell? Both copy jobs are going to offsite repository, exactly the same repository.

I really don't like this "copy every" option, and as I've seen quite a lot people don't like or get it too.

It would be so much easier to make something like "after backup X finish, run this backup copy" so there is no margin for these problems.

I've researched the forum and found lots of posts plus several articles online with people having the same problem so I believe it is more confusing than helpful to have this option there. I'm happy with Veeam, we have other clients working perfectly fine on this but for this new specific client, the backup job for me is just a flawed way to provide offsite backups, I really don't like or feel confident about it.

[foggy]

I think it's more about proper understanding of the "Copy every" setting. In most cases the confusion comes from where people think of it as a time when the backup copy job should start copying data (which is wrong). Instead, consider it as an interval to look for the new restore points within (or how often to look for them). Another good explanation is given here.

The backup job created with several VMs about 20 days ago with reverse incremental to local disk runs fine, but the backup copy is always waiting for new restore points, even if I run it manually the backup copy doesn't do anything. I've already changed the "copy every" from 5 minutes to 100 days and nothing happens, never.

Are you sure there are new restore points created by the regular backup job within the specified copy interval, that were not processed by the copy job yet? 5 minutes is way too less, you do not run your backup that frequent, right? And simply changing it to 100 (or whatever) days is not sufficient to start copying something, since there're still no new restore points available.

If I create a new backup job with a couple of VMs, it works just fine. What the hell? Both copy jobs are going to offsite repository, exactly the same repository.

The newly created job always has something to copy over since it hasn't copied anything yet.

It would be so much easier to make something like "after backup X finish, run this backup copy" so there is no margin for these problems.

And this is exactly how it works, given it is scheduled properly.

[mephisto]

I think it's more about proper understanding of the "Copy every" setting. In most cases the confusion comes from where people think of it as a time when the backup copy job should start copying data (which is wrong). Instead, consider it as an interval to look for the new restore points within (or how often to look for them). Another good explanation is given here.

I understand sort of how it works, it keeps looking for new restore points inside the timeframe I give it. For example I left the copy every 1 day, ran a manual backup and the copy job it still stuck:

Are you sure there are new restore points created by the regular backup job within the specified copy interval, that were not processed by the copy job yet? 5 minutes is way too less, you do not run your backup that frequent, right? And simply changing it to 100 (or whatever) days is not sufficient to start copying something, since there're still no new restore points available.

As you could see on Windows explorer, new restore points using reverse incremental are created, actually several of them because I was trying to make the backup copy to trigger. That is why I've tried 5 minutes, 10, 1 day, 2 days, 100 days. It really doesn't matter what is there.

The newly created job always has something to copy over since it hasn't copied anything yet.

I don't get this. The new backup job was created to disk, then afterwards it was completed that I created the copy backup. Not sure what do you mean. The other copy job I'm referring too never worked.

And this is exactly how it works, given it is scheduled properly.

Yeah, but there is no real way to "force" it.

evne if you select sync now, it doesn't work. I would love to see a button called "I don't care, just copy the data"

[foggy]

As you could see on Windows explorer, new restore points using reverse incremental are created, actually several of them because I was trying to make the backup copy to trigger. That is why I've tried 5 minutes, 10, 1 day, 2 days, 100 days. It really doesn't matter what is there.

Not sure about this particular job since you're saying you've changed its settings several times already. Basically, if nothing has been copied by the backup copy job during the current sync interval and there's a new restore point created within it (and also none of the limitations apply), the job should process this restore point (and as you say, it works this way if you create a new backup copy job). If you feel this doesn't work like this, feel free to contact support to investigate the reasons.

[Gostev]

It would be so much easier to make something like "after backup X finish, run this backup copy" so there is no margin for these problems.

Just remember that a single Backup Copy job can include VMs from multiple primary backup jobs. If you have Backup Copy waiting for one of them, then by the time it finishes, other may still be running. Now, if you have Backup Copy wait for all of them, then that's a lot of time wasted when the copy might already have been taking place.

[mephisto]

I've literally run a backup, took about 2 hours to local disk and then tried to make the copy work around this. No luck. All VMs were processed successfully in a single go, new restore point created. I'm sorry but I still really dislike that there is no real option how to enforce a backup copy to happen. You need to wait on the software to decide whatever or not it want to copy the files across, very frustrating.

[veremin]

You can always "force" a backup copy job, using "sync now" option. If there is a given restore point in the specified interval that has not been already copied to target location, "sync now" should make backup copy job transfer it.

If this doesn't happen in your case, feel free to open a ticket with our support team and let them to confirm your configuration.

Thanks.

[87Marc]

Gostev,

I noticed some interesting behavior which could be the source of frustration for some people. When I Schedule the backup copy job to the same backup repository that the backup resides on the backup copy job does nothing, presumably because there is already a copy of the backup on that repository for the period in question. When I schedule the backup copy job to a different repository it performs as expected (or at least how it is perceived to be expected).

Does Veeam have logic that tells it not to copy backups when the repositories are the same? Does it just change the flag on the backup set to not allow it to be deleted?

[Vitaliy S.]

Hi Marc,

Yes, that's correct. Backup copy job has a built-in logic that checks target location of the repository and does not copy files to the same repository where source files reside. Can you please tell me why do you want to move files to the same device?

Thanks!

[87Marc]

Vitaliy,

I was only running some test backup copy jobs. But I do have a question.

If I only have one local repository and remote repository and I run a backup Job for all my VMs to the local repository and I setup a backup copy job to the remote repository with a GFS scenario, how do I also make sure the local repository retains those same GFS backups? I don't see any obvious and easy way to set this up.

[Gostev]

With our reference architecture, you should land backups on fast primary storage, and only keep a handful of restore points there (allowing for fast backups and restores). For GFS archives, you normally want to dedicate a slower storage with lower cost per TB.

In your case, assuming your dedicated local repository has enough space to host GFS, it is what we call "secondary" repository. Meaning, you are missing small but fast primary backup repository, which is where you want to land your backups to. Depending on the size of your environment, a server with a few fat SSDs or 10K/15K SAS drives will make a perfect primary backup repository.

If your backup server is physical, you can simply make it your primary repository by adding some internal storage.

Thanks!

[tsightler]

If I only have one local repository and remote repository and I run a backup Job for all my VMs to the local repository and I setup a backup copy job to the remote repository with a GFS scenario, how do I also make sure the local repository retains those same GFS backups? I don't see any obvious and easy way to set this up.

Assuming your local repository has enough performance and space, you just need to create a second repository on the local disk. A repository is just a folder on a local system so, say for example you wanted to keep 7 days in the primary, and then a GFS retention of 7 days and 8 weekly backups in the same local set of disks you can simply create two repos:

E:\Daily_Repo
E:\GFS_Repo

So while these two repos would share the same disks, they would still be two different repos from a Veeam perspective and thus you can run backup jobs to the Daily_Repo and a Backup Copy job to the GFS_Repo. There are certainly some performance considerations here since the copy is happening from and to the same set of disks, but based on the size of the backups and performance of the disk that might be OK. You'll just need to test.

[mephisto]

With our reference architecture, you should land backups on fast primary storage, and only keep a handful of restore points there (allowing for fast backups and restores). For GFS archives, you normally want to dedicate a slower storage with lower cost per TB.

How would it work having a fast primary repository with for example 4 snapshots kept and then having the need to a longer retention on a secondary repository with slower storage?

So far I know the backup job just does this, get s backup that already exists and copy somewhere else. Does it work differently than this and it actually "build snapshots" on the backup copy jobs? I'm very confused how this would work.

[foggy]

Backup copy job does not copy files but rather synthetically creates restore points on target repository using data available in the source repository. It is always incremental and copies the latest VM state during each synchronization interval.

[mephisto]

Backup copy job does not copy files but rather synthetically creates restore points on target repository using data available in the source repository. It is always incremental and copies the latest VM state during each synchronization interval.

Thanks for that. Can it do reverse incrementals too?

[foggy]

No, it is always incremental, the first restore point in the chain is full (VBK), all subsequent restore points are incremental (VIB) files.

[mephisto]

No, it is always incremental, the first restore point in the chain is full (VBK), all subsequent restore points are incremental (VIB) files.

Hmm, shame it can't do reverse, any specific reason for that?

Also, can it merge the incrementals into the full backup after retention time has been reached?

[foggy]

Also, can it merge the incrementals into the full backup after retention time has been reached?

This is exactly what it is doing. Please review the description referred above more carefully.