-
- Expert
- Posts: 121
- Liked: 7 times
- Joined: Nov 07, 2012 6:49 pm
- Full Name: Mephisto poa
- Contact:
Backup copy nightmare!
Hey guys,
I'm trialling a solution for a client where we are virtualizing all infra-structure and testing offsite backup jobs. I'm trying to use backup copy but for some very odd reason it runs only for some backups and not for others.
The backup job created with several VMs about 20 days ago with reverse incremental to local disk runs fine, but the backup copy is always waiting for new restore points, even if I run it manually the backup copy doesn't do anything. I've already changed the "copy every" from 5 minutes to 100 days and nothing happens, never.
If I create a new backup job with a couple of VMs, it works just fine. What the hell? Both copy jobs are going to offsite repository, exactly the same repository.
I really don't like this "copy every" option, and as I've seen quite a lot people don't like or get it too.
It would be so much easier to make something like "after backup X finish, run this backup copy" so there is no margin for these problems.
I've researched the forum and found lots of posts plus several articles online with people having the same problem so I believe it is more confusing than helpful to have this option there. I'm happy with Veeam, we have other clients working perfectly fine on this but for this new specific client, the backup job for me is just a flawed way to provide offsite backups, I really don't like or feel confident about it.
I'm trialling a solution for a client where we are virtualizing all infra-structure and testing offsite backup jobs. I'm trying to use backup copy but for some very odd reason it runs only for some backups and not for others.
The backup job created with several VMs about 20 days ago with reverse incremental to local disk runs fine, but the backup copy is always waiting for new restore points, even if I run it manually the backup copy doesn't do anything. I've already changed the "copy every" from 5 minutes to 100 days and nothing happens, never.
If I create a new backup job with a couple of VMs, it works just fine. What the hell? Both copy jobs are going to offsite repository, exactly the same repository.
I really don't like this "copy every" option, and as I've seen quite a lot people don't like or get it too.
It would be so much easier to make something like "after backup X finish, run this backup copy" so there is no margin for these problems.
I've researched the forum and found lots of posts plus several articles online with people having the same problem so I believe it is more confusing than helpful to have this option there. I'm happy with Veeam, we have other clients working perfectly fine on this but for this new specific client, the backup job for me is just a flawed way to provide offsite backups, I really don't like or feel confident about it.
-
- Veeam Software
- Posts: 21139
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Backup copy nightmare!
I think it's more about proper understanding of the "Copy every" setting. In most cases the confusion comes from where people think of it as a time when the backup copy job should start copying data (which is wrong). Instead, consider it as an interval to look for the new restore points within (or how often to look for them). Another good explanation is given here.
Are you sure there are new restore points created by the regular backup job within the specified copy interval, that were not processed by the copy job yet? 5 minutes is way too less, you do not run your backup that frequent, right? And simply changing it to 100 (or whatever) days is not sufficient to start copying something, since there're still no new restore points available.mephisto wrote:The backup job created with several VMs about 20 days ago with reverse incremental to local disk runs fine, but the backup copy is always waiting for new restore points, even if I run it manually the backup copy doesn't do anything. I've already changed the "copy every" from 5 minutes to 100 days and nothing happens, never.
The newly created job always has something to copy over since it hasn't copied anything yet.mephisto wrote:If I create a new backup job with a couple of VMs, it works just fine. What the hell? Both copy jobs are going to offsite repository, exactly the same repository.
And this is exactly how it works, given it is scheduled properly.mephisto wrote:It would be so much easier to make something like "after backup X finish, run this backup copy" so there is no margin for these problems.
-
- Expert
- Posts: 121
- Liked: 7 times
- Joined: Nov 07, 2012 6:49 pm
- Full Name: Mephisto poa
- Contact:
Re: Backup copy nightmare!
I understand sort of how it works, it keeps looking for new restore points inside the timeframe I give it. For example I left the copy every 1 day, ran a manual backup and the copy job it still stuck:foggy wrote:I think it's more about proper understanding of the "Copy every" setting. In most cases the confusion comes from where people think of it as a time when the backup copy job should start copying data (which is wrong). Instead, consider it as an interval to look for the new restore points within (or how often to look for them). Another good explanation is given here.
As you could see on Windows explorer, new restore points using reverse incremental are created, actually several of them because I was trying to make the backup copy to trigger. That is why I've tried 5 minutes, 10, 1 day, 2 days, 100 days. It really doesn't matter what is there.foggy wrote: Are you sure there are new restore points created by the regular backup job within the specified copy interval, that were not processed by the copy job yet? 5 minutes is way too less, you do not run your backup that frequent, right? And simply changing it to 100 (or whatever) days is not sufficient to start copying something, since there're still no new restore points available.
I don't get this. The new backup job was created to disk, then afterwards it was completed that I created the copy backup. Not sure what do you mean. The other copy job I'm referring too never worked.foggy wrote: The newly created job always has something to copy over since it hasn't copied anything yet.
Yeah, but there is no real way to "force" it.foggy wrote: And this is exactly how it works, given it is scheduled properly.
evne if you select sync now, it doesn't work. I would love to see a button called "I don't care, just copy the data"
-
- Veeam Software
- Posts: 21139
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Backup copy nightmare!
Not sure about this particular job since you're saying you've changed its settings several times already. Basically, if nothing has been copied by the backup copy job during the current sync interval and there's a new restore point created within it (and also none of the limitations apply), the job should process this restore point (and as you say, it works this way if you create a new backup copy job). If you feel this doesn't work like this, feel free to contact support to investigate the reasons.mephisto wrote:As you could see on Windows explorer, new restore points using reverse incremental are created, actually several of them because I was trying to make the backup copy to trigger. That is why I've tried 5 minutes, 10, 1 day, 2 days, 100 days. It really doesn't matter what is there.
-
- Chief Product Officer
- Posts: 31809
- Liked: 7302 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Backup copy nightmare!
Just remember that a single Backup Copy job can include VMs from multiple primary backup jobs. If you have Backup Copy waiting for one of them, then by the time it finishes, other may still be running. Now, if you have Backup Copy wait for all of them, then that's a lot of time wasted when the copy might already have been taking place.mephisto wrote:It would be so much easier to make something like "after backup X finish, run this backup copy" so there is no margin for these problems.
-
- Expert
- Posts: 121
- Liked: 7 times
- Joined: Nov 07, 2012 6:49 pm
- Full Name: Mephisto poa
- Contact:
Re: Backup copy nightmare!
I've literally run a backup, took about 2 hours to local disk and then tried to make the copy work around this. No luck. All VMs were processed successfully in a single go, new restore point created. I'm sorry but I still really dislike that there is no real option how to enforce a backup copy to happen. You need to wait on the software to decide whatever or not it want to copy the files across, very frustrating.
-
- Product Manager
- Posts: 20411
- Liked: 2300 times
- Joined: Oct 26, 2012 3:28 pm
- Full Name: Vladimir Eremin
- Contact:
Re: Backup copy nightmare!
You can always "force" a backup copy job, using "sync now" option. If there is a given restore point in the specified interval that has not been already copied to target location, "sync now" should make backup copy job transfer it.
If this doesn't happen in your case, feel free to open a ticket with our support team and let them to confirm your configuration.
Thanks.
If this doesn't happen in your case, feel free to open a ticket with our support team and let them to confirm your configuration.
Thanks.
-
- Lurker
- Posts: 2
- Liked: never
- Joined: Sep 30, 2014 2:32 pm
- Contact:
Re: Backup copy nightmare!
Gostev,
I noticed some interesting behavior which could be the source of frustration for some people. When I Schedule the backup copy job to the same backup repository that the backup resides on the backup copy job does nothing, presumably because there is already a copy of the backup on that repository for the period in question. When I schedule the backup copy job to a different repository it performs as expected (or at least how it is perceived to be expected).
Does Veeam have logic that tells it not to copy backups when the repositories are the same? Does it just change the flag on the backup set to not allow it to be deleted?
I noticed some interesting behavior which could be the source of frustration for some people. When I Schedule the backup copy job to the same backup repository that the backup resides on the backup copy job does nothing, presumably because there is already a copy of the backup on that repository for the period in question. When I schedule the backup copy job to a different repository it performs as expected (or at least how it is perceived to be expected).
Does Veeam have logic that tells it not to copy backups when the repositories are the same? Does it just change the flag on the backup set to not allow it to be deleted?
-
- VP, Product Management
- Posts: 27377
- Liked: 2800 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: Backup copy nightmare!
Hi Marc,
Yes, that's correct. Backup copy job has a built-in logic that checks target location of the repository and does not copy files to the same repository where source files reside. Can you please tell me why do you want to move files to the same device?
Thanks!
Yes, that's correct. Backup copy job has a built-in logic that checks target location of the repository and does not copy files to the same repository where source files reside. Can you please tell me why do you want to move files to the same device?
Thanks!
-
- Lurker
- Posts: 2
- Liked: never
- Joined: Sep 30, 2014 2:32 pm
- Contact:
Re: Backup copy nightmare!
Vitaliy,
I was only running some test backup copy jobs. But I do have a question.
If I only have one local repository and remote repository and I run a backup Job for all my VMs to the local repository and I setup a backup copy job to the remote repository with a GFS scenario, how do I also make sure the local repository retains those same GFS backups? I don't see any obvious and easy way to set this up.
I was only running some test backup copy jobs. But I do have a question.
If I only have one local repository and remote repository and I run a backup Job for all my VMs to the local repository and I setup a backup copy job to the remote repository with a GFS scenario, how do I also make sure the local repository retains those same GFS backups? I don't see any obvious and easy way to set this up.
-
- Chief Product Officer
- Posts: 31809
- Liked: 7302 times
- Joined: Jan 01, 2006 1:01 am
- Location: Baar, Switzerland
- Contact:
Re: Backup copy nightmare!
With our reference architecture, you should land backups on fast primary storage, and only keep a handful of restore points there (allowing for fast backups and restores). For GFS archives, you normally want to dedicate a slower storage with lower cost per TB.
In your case, assuming your dedicated local repository has enough space to host GFS, it is what we call "secondary" repository. Meaning, you are missing small but fast primary backup repository, which is where you want to land your backups to. Depending on the size of your environment, a server with a few fat SSDs or 10K/15K SAS drives will make a perfect primary backup repository.
If your backup server is physical, you can simply make it your primary repository by adding some internal storage.
Thanks!
In your case, assuming your dedicated local repository has enough space to host GFS, it is what we call "secondary" repository. Meaning, you are missing small but fast primary backup repository, which is where you want to land your backups to. Depending on the size of your environment, a server with a few fat SSDs or 10K/15K SAS drives will make a perfect primary backup repository.
If your backup server is physical, you can simply make it your primary repository by adding some internal storage.
Thanks!
-
- VP, Product Management
- Posts: 6035
- Liked: 2860 times
- Joined: Jun 05, 2009 12:57 pm
- Full Name: Tom Sightler
- Contact:
Re: Backup copy nightmare!
Assuming your local repository has enough performance and space, you just need to create a second repository on the local disk. A repository is just a folder on a local system so, say for example you wanted to keep 7 days in the primary, and then a GFS retention of 7 days and 8 weekly backups in the same local set of disks you can simply create two repos:87Marc wrote:If I only have one local repository and remote repository and I run a backup Job for all my VMs to the local repository and I setup a backup copy job to the remote repository with a GFS scenario, how do I also make sure the local repository retains those same GFS backups? I don't see any obvious and easy way to set this up.
E:\Daily_Repo
E:\GFS_Repo
So while these two repos would share the same disks, they would still be two different repos from a Veeam perspective and thus you can run backup jobs to the Daily_Repo and a Backup Copy job to the GFS_Repo. There are certainly some performance considerations here since the copy is happening from and to the same set of disks, but based on the size of the backups and performance of the disk that might be OK. You'll just need to test.
-
- Expert
- Posts: 121
- Liked: 7 times
- Joined: Nov 07, 2012 6:49 pm
- Full Name: Mephisto poa
- Contact:
Re: Backup copy nightmare!
Gostev wrote:With our reference architecture, you should land backups on fast primary storage, and only keep a handful of restore points there (allowing for fast backups and restores). For GFS archives, you normally want to dedicate a slower storage with lower cost per TB.
How would it work having a fast primary repository with for example 4 snapshots kept and then having the need to a longer retention on a secondary repository with slower storage?
So far I know the backup job just does this, get s backup that already exists and copy somewhere else. Does it work differently than this and it actually "build snapshots" on the backup copy jobs? I'm very confused how this would work.
-
- Veeam Software
- Posts: 21139
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Backup copy nightmare!
Backup copy job does not copy files but rather synthetically creates restore points on target repository using data available in the source repository. It is always incremental and copies the latest VM state during each synchronization interval.
-
- Expert
- Posts: 121
- Liked: 7 times
- Joined: Nov 07, 2012 6:49 pm
- Full Name: Mephisto poa
- Contact:
Re: Backup copy nightmare!
Thanks for that. Can it do reverse incrementals too?foggy wrote:Backup copy job does not copy files but rather synthetically creates restore points on target repository using data available in the source repository. It is always incremental and copies the latest VM state during each synchronization interval.
-
- Veeam Software
- Posts: 21139
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Backup copy nightmare!
No, it is always incremental, the first restore point in the chain is full (VBK), all subsequent restore points are incremental (VIB) files.
-
- Expert
- Posts: 121
- Liked: 7 times
- Joined: Nov 07, 2012 6:49 pm
- Full Name: Mephisto poa
- Contact:
Re: Backup copy nightmare!
Hmm, shame it can't do reverse, any specific reason for that?foggy wrote:No, it is always incremental, the first restore point in the chain is full (VBK), all subsequent restore points are incremental (VIB) files.
Also, can it merge the incrementals into the full backup after retention time has been reached?
-
- Veeam Software
- Posts: 21139
- Liked: 2141 times
- Joined: Jul 11, 2011 10:22 am
- Full Name: Alexander Fogelson
- Contact:
Re: Backup copy nightmare!
This is exactly what it is doing. Please review the description referred above more carefully.mephisto wrote:Also, can it merge the incrementals into the full backup after retention time has been reached?
Who is online
Users browsing this forum: AdsBot [Google], Amazon [Bot], Bing [Bot], EricinIT and 66 guests