Archive Job option

[lxzndr]

Are there plans for have the option for an Archive job? I like the copy job feature as an alternative to manually copying the data to another location, but it would be nice to be able to archive the data separately without any incremental parts to it.

I would like do my daily backup jobs, copy job for offsite, and have an archive copy job that will give me weekly, monthly, etc copies on a deduplication device. there is no need for there to be a daily copy to that device, only the archive copy. I would also want the ability to be able to run an archive job on a the offsite copy so that we can offsite archive as well as the onsite archive, and have that archive job created from the copy job files, not built from the source backup files to avoid any WAN traffic on those. (particularly for those of us without enterprise license)

As others have said before, it makes sense to have archives go to dedupe device, but doesn't make sense to have a daily changing vbk file going there.

I have been running a script to copy the backup vbk files to archive device, but then I don't have any Veeam history on those copies.

Mike

[foggy]

I would like do my daily backup jobs, copy job for offsite, and have an archive copy job that will give me weekly, monthly, etc copies on a deduplication device. there is no need for there to be a daily copy to that device, only the archive copy.

To achieve that, you could set the "copy every" interval to one week and set minimum number of restore points to keep (2).

I would also want the ability to be able to run an archive job on a the offsite copy so that we can offsite archive as well as the onsite archive, and have that archive job created from the copy job files, not built from the source backup files to avoid any WAN traffic on those. (particularly for those of us without enterprise license)

Currently, you can try to use a "dummy" backup job mapped to the files produced by the backup copy job and use this job as a source for the "archive" backup copy job.

[lxzndr]

Currently, you can use a "dummy" backup job mapped to the files produced by the backup copy job and use this job as a source for the "archive" backup copy job.

I'm not sure I understand how to setup the "dummy" job for this.

[foggy]

Create a new backup job pointing to the remote repository and map it to the files produced by the backup copy job (do not run this job). Then specify this backup job as a source for the backup copy job that will run in the remote site for archiving purposes.

[lxzndr]

Can or will a copy job use the files of another copy job as its source?

I have a copy job that copies a set of servers to offsite repository daily and keeps them for 7 days.
I want to have another copy job that archives one of those servers once a week (keep 4 weeks, and 36 months of archives)

If I create second copy job for that server, will it pull from the offsite repository to build on the offsite archive repository? Or will it only use the local repository files to send across the wan?
the offsite and offsite archive are in the same location (actually two different drive sets on the same server)

Thank you,
Mike

[foggy]

Mike, currently backup copy job cannot be used as a source for another backup copy job. A possible workaround could be to use a "dummy" backup job, map it to the backup copy job files, and then use this job as a source for the backup copy job.

[lxzndr]

When I go to create the dummy backup job, it says that the files are already in use by the copy job, do I want to map it to this job instead?
Will that mess up the copy job? Do I need to include all the servers in the original copy job in the dummy job? or just the ones I want to use in the archive job?
Due to other reasons long ago, we have two veeam servers, one in each of our locations. do I set up the dummy and archive on the Local site (to the originals) or on the Remote site (where the offsite copy is stored, and where the archive repository is) I'm sure it probably doesn't really matter, but is one a better location than the other? Having it local makes it easier to see all the backups for the server in one spot. having it remote may help with processing?

Mike

[foggy]

You're right, it does not work this simple way. Moreover, I've just talked to R&D guys and it turns out that backup copy job cannot process backups created by another backup copy job (hardcoded limitation). We will discuss the ability to address this in the future, though.

Currently you have to run both backup copies over WAN or use some 3d party utility to copy files in the remote site to some other location.

[lxzndr]

It would be great if there was the option to store archival copies of the copy job in a different location than the copy. Or to be able to use the copy job as source for another copy job.
Or have an Archive job option, that doesn't do any incremental backups at all. just builds full backup copies from previous and changed blocks.

Also it can be confusing how the copy job archival processing works. Does it use the backup closest to the selected day for the archive? I notice it only tags them as archives when the file would normally be purged due to the retention limit.

Should be an option to specify the day to run the copy job, but not limit when it operate. How would I define the job if I wanted to copy only Friday (or closest to friday) backup? but I also don't want to limit the amount of time it gets to run (in case it takes more than allotted) and not get messed up if someone needs to edit the job (which starts a new cycle). Maybe allow the option to schedule the copy job the same as a regular backup job? have the continuous as an option, but not the only scheduling.

One goal would be to schedule it to start on Saturday 1am, and use the most recent backup to make the copy (be that Friday, or Tuesday, or even Saturday 12:30am as long as it is newer than the previous backup copy was) then not run again until next Saturday. Allow it run until completion (as long as it completes before the next start time) Keeping 4 weekly copies, and 36 Monthly copies. (or maybe 4W, 4M, 4Q, 3Y)

[foggy]

Also it can be confusing how the copy job archival processing works. Does it use the backup closest to the selected day for the archive? I notice it only tags them as archives when the file would normally be purged due to the retention limit.

Probably, the corresponding user guide section will make it clearer for you.

And thanks for the feedback!

[lxzndr]

That covers a daily job, but not what happens if it only runs once a week. If I say I want an archive copy 1st of every month, but my job only runs every 7 days, then what determines the Monthly archive? That is the part that I haven't seen covered in the guide.

What is the best way to accomplish the goal mentioned above?
Define the job to run every 7 days starting at 1am, with limit of when it can run from now until that start time, and then clear the restrictions after the first backup runs?

I will say that with the copy job, Veeam has improved for GFS archiving, but isn't quite there yet. 1 week of daily (or weekday) stored to fast storage as initial backup, and then weekly/monthly/etc copies to archival (deduplicating) storage using the copy (without incrementals in the archive or reading the previous backups there), plus same 1 week of daily copies offsite, and weekly/monthly/etc offsite archive is what we would love to get from it.

[lando_uk]

I would also like the option to create multiple full copies for archive rather than incrementals. I get paranoid that my full will become corrupt and I'll loose the whole chain.

I'd much rather have lots of fulls that are then deduped by the OS.

[veremin]

That covers a daily job, but not what happens if it only runs once a week. If I say I want an archive copy 1st of every month, but my job only runs every 7 days, then what determines the Monthly archive? That is the part that I haven't seen covered in the guide.

With backup copy job running on weekly basis, you can simply propagate the logic described in the said section of User Guide; just replace days with weeks, so to say, Monday = 1st week, Tuesday = 2nd week. The idea is quite simple. The backup copy job has the retention settings. Once the specified retention policy is exceeded, the full backup starts moving ("transforming") forward during each run: be it daily run or the weekly one (as is in your case). When the full backup reaches the day chosen for GFS archive purpose, the new GFS point is created.

I would also like the option to create multiple full copies for archive rather than incrementals. I get paranoid that my full will become corrupt and I'll loose the whole chain.

The GFS restore points are stored as full backups, and backup copy job itself has built-in integrity check of the restore point with automated remediation of the detected corruption. If this isn't enough, you might switch to remote backup job and select to run full backups each time.

Thanks.

[larry]

Question on best way to do this task.
At our DR site we store a backupcopy from our main site, I push it there. This backup contains SQL data and I do not use the WAN accelerator for the job. Full backups once a week on main job. 150 meg wan link for backups only at night. Some other jobs do use the WAN accelerator but this job performed best without.
We want to move our retention data, weekly monthly and yearly to a separate storage location. While increasing our daily retention in the current location. For our DR site we are moving retention data to USB SATA 4tb disks. We will rotate the USB disks out as they fill.

I do not want to send the data over the WAN twice. I want the daily data and retention data split between servers and repositories at the DR site.
Should I change the main backup copy to a pull backup job and make a backup copy from that? I hate to hit production twice but could.
Make an ugly script to robo full backups? Then swap USD drive when full.

Any way to use a backupcopy as a source and use a local proxy to move the data?
Can the Veeam server at the DR site use the backup copies in the repositories the Veeam server at the main site made?
thanks
Is everyone’s data doubling every year?

[foggy]

Larry, am I understanding right that you just want to store backup files created by regular retention and GFS retention of the backup copy job in different locations? Then I'm afraid your options are either using two backup copy jobs or utilizing some external "ugly script" or tool to copy files to different location.

[larry]

ok thanks
My vote is for a Archive job to do this.
Options to include.
Don't check for existing files. This way removable media is automatic with archives. If I have my archive folder set to drive X folder X just do it. If folder not there, make it.
Store media name with Archive record, with media name during a restore if files are not there Veeam can display mount media name. Allow user to choses where to find files this way I can mount a two year old disk to a different drive letter while leaving current setup alone.
Will need a import Archive.

Maybe a flag for destruction of archive, I select to save for X years and after that time create a warning to mount old media so it can be deleted when job runs. I need to find all old backups and destroy after they are expired currently.

I like current retention options and can do all our necessity retentions.

[foggy]

Thanks for the reasonable feedback; much appreciated.

[DaFresh]

Hello,

Sorry but this post is quiet confusing for me. As Mike, my archiving needs are simple, I need to produce a full backup (or a copy of a full backup...) of critical VMs every 6 months, with a retention policy like 1years. these VMs are backuped every day, so the best option would be the use of the Backup Copy mode, for example every first Sunday of January and July.

Could someone explain how schedule properly that simple use case with in the Backup Copy mode ?

Thanks,

Cédric

[Gostev]

Hi, Cédric

Why not just create a dedicated backup job with the required schedule and retention?

Seems like Backup Copy is unnecessary complication in this case. Also, it is always a good idea to create check point backups of critical VMs by obtaining the data directly from production storage (as this reduces the chance of introducing data corruption).

Thanks!
Anton

[DaFresh]

Hi Anton,

Well, I was having that in my minds as a secondary option. I find more elegant to use the Backup Copy mode as it only require to *copy*, thus is more light for the underlaying hardware, faster, and safer than a full that could overlap the backup window.

I think it would have been maybe more flexible to allow, in addition to "Minute, Hour, Day", something like "Week, Month" ? If it was doable, that would have done the trick !

Cheers

[veremin]

If you want to, you can still stick to backup copy job. As to weekly, monthly backup copy job schedule, you can achieve that by setting 7 day or 30 day copy interval, respectively. Thanks.

[puters]

I saw mention before of using an 'ugly script' or 3rd party tool to move the archive restore points out to deduped storage - what impact would this have on Veeam's retention policies?

At the moment I'm thinking of making a Powershell script to manage retention and move out archive restore points to a Windows Deduplication enabled volume. I have moved one of the yearly restore points and Veeam has replaced it with a ~30MB file of the same name. I'm just not sure if Veeam is going to run into errors if it tries to remove a backup that is no longer there.

Would the best option perhaps be to have the backup copy jobs set to keep just one of each restore point (yearly / monthly / weekly) and the script can copy them out for a longer retention elsewhere?

[foggy]

Retention shouldn't be confused by the fact that some of the GFS points are not physically there anymore. But why move? You can copy them for longer retention to a third location and let backup copy job retention to do its job, maintaining the required number of points in its target location.

[puters]

Thanks, I'll give it a go. The DDPEval tool predicted a 42% space saving on a given backup set, should save us a fair bit of space.