Supported Ways to use USB rotation

[Zew]

Hey all,

I was just wondering a couple things about creating secondary resources for jobs, let me provide a bit of background into our infrastructure setup.

I have Veeam B&R installed on a Server 2008 R2 VM, on one of our hosts within a HA cluster.

We have a SAN that is dedicated to backups and has a volume shared via CIFS to this VM server (No problems getting this setup)
However since this is a VM it doesn't have any actual USB ports (and hardware passthrough is crippling to HA/vMotioning this VM),
so instead since the SAN has a USB port I just add it to that and it auto shares the devices via CIFS with the name of the volume it was formatted as (NTFS FileSystem)

I added it as another repository and selected the (This is a rotating drive checkbox under advanced options of the repo)...

Now here comes the unintuitive part, I would assume that you could edit a Job ( I have one created for each individual VM in our cluster) and select storage, where I would select use secondary target for this job.. which just pops up a thing saying you need to create a Backup Copy Job.... I would have thought I could just specify the secondary repo I created, and tell it when to duplicate the data for that job over to the secondary repo... why do I have to create another job to accomplish this task? It be nice to have all jobs associated to a VM including its duplicate jobs all under one job so to speak (I'm almost embarrassed to say it but Backup Exec got this right, but everything else about it is well, terrible)

Suggestions

... If you are not going to be changing the job layout, then at the very least allow it to create a "Backup Copy Job" should one not exist.

The other non intuitive part about setting up USB offsite duplicate jobs is the scheduling and linking..

So here we are creating a new job to attempt a duplication (Cough, shudders, mmm), create "Backup Copy Job" give it a name, oh can't be the same name as a normal job, cause its another job!!! :@ OK, named VM Copy Job...
Then the only scheduling option is "Copy Every:" and specifies only Minute, Hour, Day

Like Why can't I schedule a date n time for it to run? Then I have to select the VM again in the next step (which if it would just allow it to be done in my initial job I wouldn't have to specify it again, this is redundant for me)

Then I specify the target (which could be done in the initial job once you select the checkbox to store in another repo) and then schedule is simply when its allowed to run...

Pretty much I am clearly not understanding the way B&R does it job replication and isn't as intuitive as it should be..

I may also not I was informed that USB rotation disc are only supported on a windows server (like I have to have a single server hosting a file share?! really?!) looking for confirmation on this one as well...

I know this is a long post, but this seem like something Veeam still isn't getting right, from looking at Veeams history on the subject it was only possible to have USB rotation with scripts and tweaking, so good job integrating it, but I think it still needs some work to be done on it....

[foggy]

It's a question of understanding the backup copy jobs concept, actually. Take them as a completely separate thing, it's not just as you specify the secondary location to place your backup files to by the regular backup job. Backup copy actually does not copy files at all. What it does is synthetically creates restore points on the target repository using data available in the source repository, copying the latest VM state during each synchronization interval. And it does not necessarily stand behind particular backup job - you can point it either to the entire source repository or select individual VMs from infrastructure (not necessarily backed up by a single regular backup job), or one or several backup jobs, so it's flexible to accommodate different scenarios. Like in your case, you can have a single backup copy job for all the backup jobs that you have created for individual VMs (btw, is there any reason for this?).

So if you're creating your first backup job, just simply skip the Secondary Target step of the wizard, you can link the created backup job to a backup copy job later, if the need be. As a result, the backup job will be used as a source for the backup copy job and the restore points created with the backup job will be automatically synced to a secondary location.

[Zew]

Thanks Foggy,

But this is still not as intuitive as I think it could be. Let me explain what I'm expecting over what you had told me (Let's face it, Your explanation was pretty convoluted and I have to read it over several times just to grasp the concept)
I come from Canada and they think I'm slow, ehhh... anyway

Going back to how the terrible BE did it, and I actually liked it that way (sad to say)

VM backup Job (Full) -> (Specify incremental job run times, in the existing job) -> specify duplicate stage and point to rotation "pool" of USB discs and tell it when you want this job to run, and which backup set to use (I usually picked full)
everything is nice centralized you only have one job that has multiple stages and it actually kind of nice, it was many other issues that drove me aware from that product.

Now, your telling me that in order to duplicate a job I HAVE to create another job, and give it another name, and point it to an existing VM, or a existing backup job... It just makes no sense to me...

So let me just re-phrase my question...

How can I setup a duplicate job on the existing main backup job I have and tell it to run at a specific time?
I don't want "automatic sync" so to speak, I want to schedule exactly when my duplicate jobs run and to what repository they run onto.

As for your question on my reasoning for my backup jobs, Most of our VM's are setup to be pretty stand alone- don't rely on services by other VM's to reduce downtime if any particular VM as an issue, then only the services on that VM will be affected (makes diagnosing issue all lot easier, and makes restoring snapshots also easier, this doesn't obviously work for 2008 R2 DC's, but for most other services it works great.) Anyway that why I backup VM's individually, if we had our SQL server virtualized (working on it) then I'd add multiple VM's for certain services e.g. SharePoint I would backup both the FE server and the SQL server hosting it's files. Anyway running on a bit of a tangent here. All I want to setup is a primary backup job, and tell it to make full backups onto USB shared disc which in this case is really a CIFS shared by the SAN. Sounds pretty simple yet Veeam doesn't have a simple way to implement my style of duplication jobs...

Is that the only way is to do it via a Backup Copy Job, cause it would be a waste to setup another regular job and simple copy it to the USB CIFS device as it would basically have to re-read and snapshot and all the stuff it does for a regular job all over again. I don't want to do that, could you provide maybe some ideas as to a more feasible way to achieve what I'm after?

[Gostev]

Yes, Backup Copy job is the only to copy backup in Veeam in a fully integrated manner (with backup copies tracked in the configuration, without conflicts with primary jobs when it starts up, with support for rotated media etc.)

There is also File Copy jobs, which will treat backup files as regular files.

I appreciate your opinion that Backup Copy may seem a little complex for simplest scenarios with a single job making a single copy, but you have to keep in mind that it was built to address the needs of customers with environments of any size, and it does FAR more than a simple copying of file from one place to another. I am sure you will end up appreciating its capabilities as your environment grows.

Thanks!

[Zew]

Thanks Gostev,

I understand it is meant to cover more complex scenarios and that's fantastic to think scalable, always good to see.

And I love Veeam definitely amazingly documented, great KB articles, and layout and design. However, why can't I schedule when a Backup Copy job is suppose to run?

It would also be far more intuitive to have create copy job or create tape job (radio button popup) when selecting secondary repo in the storage section of a regular job, instead of one that just specifies you need to create one.

If Copy backup job was suppose to be designed to accommodate all scenarios, how can it not accommodate my very simple one?

As I mentioned before I want full control of when my jobs run, including primary job, and now "Copy Backup jobs" I still think you should revamp the design to allow for duplication to be done right in an existing job.. would be far more intuitive for simpler scenarios..

So basically you are telling me there is literally no way I can tell it to take an existing job from the source and tell it to duplicate to a USB device when I want it to? I'm forced to setup a duplicate job (separate) and link it to my other job, which then runs immediately after that job completes... this is how I'm understanding it based on the documentation I have read.

OFFTOPIC, but is there anyway I can fix the typo in my title of this thread, my OCD is driving me crazy every time I see it...

[Gostev]

However, why can't I schedule when a Backup Copy job is suppose to run?

You can do that, of course. The Backup Copy interval start time is configurable when you create a Backup Copy job (see first step of the wizard).

[foggy]

So basically you are telling me there is literally no way I can tell it to take an existing job from the source and tell it to duplicate to a USB device when I want it to? I'm forced to setup a duplicate job (separate) and link it to my other job, which then runs immediately after that job completes... this is how I'm understanding it based on the documentation I have read.

Basically, backup copy job will do right this: monitor the backup job linked to it and start copying new data as soon as backup job session is finished and a new restore point is created.

[Zew]

You can do that, of course. The Backup Copy interval start time is configurable when you create a Backup Copy job (see first step of the wizard).

So if I have a main Copy Job set to Fri 8 PM, to have the copy job run on Saturday 8 PM, set the "Copy Every:" to 1 Day "starting at" 8PM?

I'm still having such a hard time wrapping my head around how this works, when it should be so easy... really I just want to say second backup "duplicate" this backup source, to this destination at this time.

[Zew]

Basically, backup copy job will do right this: monitor the backup job linked to it and start copying new data as soon as backup job session is finished and a new restore point is created.

So let me try and understand this:

Create Back-up Copy Job;
1) Name JOb, and schedule it (This schedule time is based on the time the VM main backup job is suppose to run correct?, as in If my Main job runs at Friday 8pm, when I specify 1 Day @ 8 PM, it will run on Saturday 8PM, correct?)
2) Pick VM, what if I pick multiple VM's that have backup Jobs starting at different times, then they will simply run based on the schedule I created, so if VM 1 main job is at Friday 9PM, its copy job will start @ Saturday 9PM, and if VM 2 has its main backup job On Friday 11PM, this back copy job would run on Saturday 11PM?
3) Target, this is something at least is dead simple and understandable, pick the target repo for the backup copy.
4) Direct
5) "Shedule" but really you schedule it in step 1, this is more like When is this backup copy job allowed to run.
6) Summary "Be nice if this included the times it will run based of the VM's selected or something

And now as I asked Gostev, If I don't specify this job, created above, to the main job it will only run based on the schedule I placed in step 1, if I do link it to the main job (By going the storage section and checking off "configure secondary destination for this job" (which is actually incorrectly worded as it should say pick a secondary JOB for this job) then it will run as soon as the main job is run.

This is how I am understanding it so far.

This is the one thing Veeam that is amazing complex and convoluted for what it should be... (I had zero issues understanding BE duplication steps... Super Easy (Pick Source, Pick destination, pick time, done)

So to make my question now is clear as possible. How Do I setup my replication "Backup Copy Job" to run Exactly 24 hours from the main job, instead of exactly right after?

[Gostev]

When you have a single nail in your hand, anything other than a hammer made of wooden stick with metal brick on top is too complex. This statement is totally correct, and is impossible to argue. Problems start to arise when you have thousands of nails to go through in a very short time, and some of them are screws. To be able to achieve that, you need much more complex tools. And yes, these tools are amazingly more complex comparing to a hammer. And no, you can never get a simplicity of hammer from them. And no, you cannot use them in the same way you have used hammer, such as nailing a nail with a back of an electric screwdriver... it's a totally different concept, it's a paradigm shift you have to go through.

Usual rules apply - a software designed for protecting dozens of VMs is no good for protecting dozens of thousands, and vice versa. Likewise, a concept that works well when you have only a few jobs and a basic requirement of weekly entire backup copy locally on a fixed weekly schedule will not work well when you have dozens of jobs (each with unpredictable end times), more typical daily backup copy requirement, and when some of the VMs have to be copied locally and other - offsite over WAN.

Our Backup Copy job is a separate process with automated scheduling, because it is designed to selectively pick VMs which backups need to be copied onsite or offsite. So, it has to support scenarios with dozens of source jobs, each running and completing on their own schedule, in the most optimal manner. Likewise, simply copying a backup file that each job produces will not cut it, because ability to select VMs is essential (usually, it's impossible to backup ALL of your VMs offsite).

Hopefully, this clarifies the logic behind Backup Copy design.

P.S. This was written before I saw a huge update to your previous post, so this response is not meant to ignore your questions.

[Gostev]

So to make my question now is clear as possible. How Do I setup my replication "Backup Copy Job" to run Exactly 24 hours from the main job, instead of exactly right after?

What you are asking for can be easily done with a simple File Copy job we provide, and this can be scheduled and will work exactly as you want it to.

However, can you explain why do you want to wait for 24 hours, instead of making a copy of your backup right after it has been created by the main job? The latter seems to be more logical, otherwise you will not be meeting the 3-2-1 rule during the window between your main job completion and copy job start time, thus increasing the risk of data loss. It will always take 24 hours until you will be in compliance with your policy on the required number of backup copies.

Or may be there is some reason for this delay that I am currently missing?

Additionally, how does your approach addresses the following common scenarios. Let's imagine source VMs had massive changes on Friday (Windows service pack install), or some network switch failed over from 1Gb to 100Mb due to an office cleaner hitting a patch cord, or production or backup storage were to busy with an unexpected external workload, slowing down backup dramatically - or something else along these lines. As the result, your main job will take 30 hours to complete, instead of usual 3 hours. Because of that, when a copy job starts 24 hours later after main job starts, it will simply fail due to nothing to copy, and you will be left with NO backup copy for an entire week, missing your offsite RPO by as many as 7 days.

Are you not even considering something like this to happen?

And trust me, I am just scratching the surface here behind all the scenarios our Backup Copy jobs are designed to handle transparently...

[Zew]

Hey Gostev,

I guess your are right, I just wanted the job to start 24 hours later just cause I like to stack my backup jobs 1 hour in between each backup job, determined after testing how long on average backup job runs. I just wouldn't want the Backup Copy Job to run while the next job starts up. I suppose I could just allow the defaults to apply on backup Copy Jobs and just see how it fans out. I appreciate the time you are taking to help me clarify a proper way to setup backup copy jobs correctly, and apologies if I came off a bit arrogant, that is not my intention.

haha, as for the office cleaning personal, our server room isn't accessible to cleaners, as I take it upon myself to keep a cleanly server room.

For now I will do as you suggest and run the job as you suggest, Create Backup Copy Job, and then associate backup job to primary job.

Again thanks for your time in clarifying this for me.

[Gostev]

Hey, no problems at all. By the way, I think you will want to specify the following behavior mod (I've just copied the description from What's New document). Otherwise, depending on when you create Backup Copy job and copy interval specified, the resulting behavior may not be as you desire.

Forward lookup: Backup Copy jobs can now be configured to wait for the new restore point to be created by a primary backup job, instead of using the latest non-copied restore point already available. To enable the new behavior, create the BackupCopyLookForward (DWORD) = 1 registry value under
HKLM\SOFTWARE\Veeam\Veeam Backup and Replication key.

[Zew]

I was just wondering something I found a bit odd, I was checking the USB device I had configured for my Backup Copy Job... and was a little surprised to see my Software Repository VM (Which hold close to 250 GB of data) is showing up as only 120 MB on the USB device for the folder name of the VM I had backed up... Am I missing something?

[foggy]

Were all the VMs successfully synced up by this job during previous synchronization interval? Were there any warning/error messages?

[Zew]

Hey Foggy,

Thanks for the suggestions, Just wanted to note for everyone who maybe following this thread, that I changed out my USB drive being shared by the SAN, and added a new USB device added to the SAN with the same share name (Seen by Veeam as a CIFS repo) and it worked awesome, I checked the USB device after the weekend jobs, and saw the couple hundred Gigs for the VM in question.

Sooo yippy, looks like you can use rotate USB via a CIFS share, seem to be working for me at least

[foggy]

Yep, you can implement rotated drives scenario with CIFS repository. There're some limitations comparing to Windows-type repository, however overall scenario works just fine. Glad you were finally able to get everything work.

[Zew]

I know it's been awhile since I posted here, and just had one last question.

Since I didn't want to create a whole to thread since this still falls under this. I have my Backup Copy Jobs set, and as others have noticed you can only bring it down to 2 minimum restore points (Full and Incremental).

Thing is (this isn't how I will fully manage it up...) When I got change a BCJ target it always complains about wanting me to move the Backup data first ( I just want Veeam to be quiet and simply copy a full backup to the new target should it not find it) This is after all a BCJ and the source with the data required to copy is clearly available still (So why should I have to manually move the data when BR can do that for me, via its source).

I already tried adding the ForceCreateMissingVBK registery under both the standard Veeam hive and under the B&R subkey AS per (http://www.veeam.com/kb1154)

but every time I specify a new target with no data it still complains that it wants to see the data there... what am I missing?!?

On a Really good note though, even though I am sharing the drive via CIFS and have it set as a rotated drive in the B&R Backup Repo settings, when I changed the drive to one that had no data, B&R BJC copy data to the blank drive just like I want it to! So the above task isn't the end of the world, if it doesn't work out, just means if I ever do need to change repos on a job, I'll either A) have to re-create the Jobs (Via a PS script, which I have read about) or B) Copy the data as Veeam keep prompting to do.

[foggy]

Just to make sure I'm getting you right: when you change the backup copy job target to an empty folder, it works, however not in case when the folder contains some older backups?

[Zew]

When I physically change the disc being shared (TO be safe what I do is stop Veeam B&R service, log into my NAS, stop the Samba service, umount the disc, add new disc, remount the disc, and restart the Samba service, which shares the disc via CIFS with the exact same share path) when its a new disc Empty Data the job will copy data from the Source to the Destination.

However If I instead for testing purposes wanted to create a new repo target in B&R, and then goto a Backup Copy Job and attempt to specify this new target, I get an annoying prompt asking me to migrate the data to the new repo before the Backup Copy Job will accept it as a new target... sooo annoying... but since I am swapping my disc and sharing it via the same CIFS share path, it's not the end of the world.

Right now I was using a 1TB 3.5" 7200 RPM disc formatted as NTFS, and then shared via CIFS/SMB via my NAS. (Even though my NAS uses ZFS for the main pool, I decided on a NTFS formatted disc so in the event need to recover VM's offsite, a technical wouldn't need to build a ZFS type server to get the data, any old windows machine running B&R will suffice at this point. I actually haven't tested this, but I'm assuming it should work. However the Disc kept getting overfilled and Backup Copy Jobs would fail. I didn't think my over VM selection would reach that size limit, so I shared a 2TB disc in hopes it will suffice, will find out after the weekend jobs.

[foggy]

In a rotated drives scenario, there's no need to manually change repository in the job settings each time, just swap the drive.

[Zew]

Totally understandable, just be a nice feature if you did need to change repository (due to a different path name or otherwise) since you cannot change the "path" directory in the given backup repository directly. You have to add the new repository to the available list of repositories (with the new path). However you cannot change the repo in the copy job till you manually move the data from the old repo to the new repo manually (which is a pain and kind of pointless when you tell it to simply populate the data from the original source repository, but it doesn't work in the case) or you simply totally recreate the backup job.. both are a pain when the Backup Copy Job should simply accept the new Repository, realize its blank and simply re-create the data from the source repository.

So in short its really annoying if you need to change a Backup Copy Job's repository, and it expects you to populate the data manually or totally recreate the job from scratch.

[foggy]

You can also use the Remove from Backups command to achieve that, without the need to copy files or re-create the job.

[Zew]

It be a nice feature request to have that functionality built in. Even if at least a pop saying "New repository missing required data, Copy data from original source?" and then click yes and it runs that PS script providing the variables with the name of the job you are attempting to change the repo on.

You guys do amazing work, just small things here and there I see could use some adjustments. This one being small as changing a copy job's repo is seldom done. So such a feature request I can see being down in the list. But still worth mentioning,

Thanks for the link to the script, I'll keep it my code repo should I ever need it. Usually I've just been manually copying the data myself in the event of.