Comprehensive data protection for all workloads
Locked
farid.shahab
Novice
Posts: 3
Liked: never
Joined: Dec 31, 2013 5:33 am
Full Name: Farid Shahab
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by farid.shahab »

v.Eremin wrote:Kindly take a look at the answer in this existing discussion. Thanks.
Hi Eremin,

Can you share this topic with me please? I am having the same problem. the RDX removable disk is not being recognized at the repository selection. thanks. Farid

farid.shahab
Novice
Posts: 3
Liked: never
Joined: Dec 31, 2013 5:33 am
Full Name: Farid Shahab
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by farid.shahab »

Andreas Neufert wrote:Use Windows Disk Manager to create a NTFS Partition and select a drive letter for this. Windows will remember the drive letter.
I think it is not about the drive letter, it is because Veeam doesn't recognise removable device as a repository backup, veeam only accept local disks for its repository. please have a slution for this. Farid

Gostev
SVP, Product Management
Posts: 28221
Liked: 5007 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by Gostev »

farid.shahab wrote:Veeam doesn't recognise removable device as a repository backup, veeam only accept local disks for its repository
That is certainly not true. Plugged-in removable drive is no different from local disk.

bunger
Expert
Posts: 117
Liked: 8 times
Joined: Jun 23, 2010 5:39 pm
Full Name: Bill Unger
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by bunger »

Gostev - I have an open case with support ( currently escalated to Senior Support ) as the regkey and hotfix don't seem to fix the issue for our environment.

While I am awaiting a fix or "next steps" from them, is there a workaround I could use? Would the scripts previously mentioned do the trick?

foggy
Veeam Software
Posts: 20100
Liked: 1878 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by foggy »

Bill, what is the actual scenario that doesn't work?

bunger
Expert
Posts: 117
Liked: 8 times
Joined: Jun 23, 2010 5:39 pm
Full Name: Bill Unger
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by bunger »

foggy wrote:Bill, what is the actual scenario that doesn't work?
I have a backup job that backs up VMs to an internal hard drive. I then has an associated Backup Copy job that should copy the latest backup to a swappable external hard drive. The Backup Copy job is failing with the following error:


1/6/2014 3:43:29 PM :: Error: The system cannot find the path specified.

CreateDirectory(A:\Copy Backup to Rotating) function call failed.
Cannot create folder. Folder path: [A:\Copy Backup to Rotating].
--tr:FC: Failed to create directory. Directory path: [A:\Copy Backup to Rotating].
--tr:Failed to call DoRpc. CmdName: [FcCreateDir] inParam: [<InputArguments><DirName value="A:\Copy Backup to Rotating" /></InputArguments>].

We are running v7 R2, have the registry key defined, and the hot fix installed. Still no go...

veremin
Product Manager
Posts: 18417
Liked: 1819 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by veremin »

Hi, Bill, I'm wondering whether these scripts can solve your issues with using backup copy job in conjunction with rotated drives. Thanks.

bunger
Expert
Posts: 117
Liked: 8 times
Joined: Jun 23, 2010 5:39 pm
Full Name: Bill Unger
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by bunger »

I just heard back from Senior Support and don't feel like I am getting very far. Here is what they have to say:

We have additionally discussed this with R&D team.
As it turned out ForceCreateMissingVBK was initially designed to be used for regular backup jobs that use reversed incremental method.
For backup copy it would also work if there is only single .vbk and no .vib files. If there was a backup chain (.VBK + .VIBs) it won't work as simply ignoring previous chain would possibly cause configuration conflicts in future.


And then...

As far as I understand case summary you need to use backup copy job to copy backups to a repository with rotated drives, right ?
By default after changing the medias copy job will stop working as it will not be able to find files produced during previous sessions.
To get it to work again you need to clean up configuration records.
In order to do so, please open Veeam GUI -> Backups, then right click on one produced by corresponding copy job and click "Remove from backups" (please see attached screenshot).


Neither of these helps me get the backup copy job working to removable media. The point I am stuck on is that I thought the registry key and hotfix addressed the issue, but it didn't for us. And based on the responses above, it doesn't sound there IS a fix?!

veremin
Product Manager
Posts: 18417
Liked: 1819 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by veremin »

If applicable, you can use the scripts mentioned in my previous message. They should automate the cleaning procedure for you ( backup-> delete) for you, making backup copy job work without issues with removable media.

Thanks.

Gostev
SVP, Product Management
Posts: 28221
Liked: 5007 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by Gostev » 2 people like this post

Hi all, as promised in the weekly forum digest couple of weeks ago, here is the beta code for full support of all use cases around rotated drives. We are looking to include this code in the next product patch, but would like to get your feedback first, to make sure we have addressed all your requirements. The goal here was to remove the need for ANY scripting on your side completely. I would hate to find out we've missed something after the patch is released, so I appreciate your help with this.

This code adds rotated drives support for all backup modes of primary backup jobs, as well as for Backup Copy jobs. It also adds optional backup repository auto cleanup functionality. This code must be installed on B&R 7.0 R2.

Registry values controlling the functionality:

ForceCreateMissingVBK (DWORD)
Set to 1 to enable support for rotated drives. With this functionality enabled, if any backup file from the latest full backup chain is missing (such as when the existing hard drive is replaced by another one), jobs will start the new backup chain and create the new full backup (instead of failing out).

ForceDeleteBackupFiles (DWORD)
Enables automatic cleanup of the backup repository from ALL existing files if any backup file from the latest full backup chain is missing (such as when the existing hard drive is replaced by the hard drive brought back from offsite location, and containing some older backup files).
Set to 3 to make the job delete the entire contents of the backup job’s folder only.
Set to 5 to make the job delete the entire contents of the root backup repository folder, thus deleting any files belonging to other jobs along. Note that if backup repository is pointed at the volume's root folder, the entire volume's contents will be erased.

Download here:
ftp://ftp00445433:ctDjaMKR@supportftp.veeam.com/

bunger
Expert
Posts: 117
Liked: 8 times
Joined: Jun 23, 2010 5:39 pm
Full Name: Bill Unger
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by bunger »

I have been working on the powershell script this morning and it doesn't seem to work.

My backup architecture is as follows:

Backup Job called "Nightly Backup" backs up VMs to internal drive and this backup location is d:\NightlyBackup

Backup Copy Job ( Attached to "Nightly Backup" ) should copy recent backup to rotating external drive. This rotating location is A:\

When running and stepping through the Powershell script, it is checking the location d:\NightlyBackup for the existence of a VBK. If it can't find a VBK, then it stops the scheduler, deletes the backup from the GUI, deletes the files from media, and then restarts the scheduler.

The problems that I see with the script is that it references the main backup job and not the backup copy job.
- It looks to d:\NightlyBackup for the existence of a VBK ( which always exists as it is not rotating )
- it runs $backup.Delete(), which is initialized to the backup job and not the backup copy job
- it deletes the backup files from the d:\NightlyBackup location and not A:\

What am I missing here?

bunger
Expert
Posts: 117
Liked: 8 times
Joined: Jun 23, 2010 5:39 pm
Full Name: Bill Unger
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by bunger »

Oh man. Is it Friday yet?

I just realized I was entering the main backup job name in the script instead of the copy job. My bad. <sigh> Ignore my previous post.

tsightler
VP, Product Management
Posts: 5843
Liked: 2676 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by tsightler »

You may also be interested in Anton's post above, from earlier today just before your post, where he's asking for testers of the beta patch that allows all of this to "just work" by setting a few registry keys, no scripting required.

bunger
Expert
Posts: 117
Liked: 8 times
Joined: Jun 23, 2010 5:39 pm
Full Name: Bill Unger
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by bunger »

I just installed the patch for clients - we'll see how it plays...

STORM121
Novice
Posts: 7
Liked: never
Joined: Dec 26, 2011 3:40 am
Full Name: David L Litz
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by STORM121 »

Do we replace the current dll files with the new ones and do we have to create these new registry entries?

teknomage
Service Provider
Posts: 23
Liked: 2 times
Joined: Jul 21, 2010 8:55 pm
Full Name: Mike
Location: Fargo, ND
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by teknomage »

I won't have a chance to try this in my lab until tomorrow, but I have a question. When the .DLL files are replaced, are those DWORDs created automatically or do they need to be created manually? If they need to be created manually, what hive do they need to be created in?

foggy
Veeam Software
Posts: 20100
Liked: 1878 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by foggy »

Provided .DLL files should replace the original ones (recommended to keep the old files somewhere, just to be on the safe side) and the keys should be created manually under Veeam B&R registry hive: HKEY_LOCAL_MACHINE\SOFTWARE\VeeaM\Veeam Backup and Replication.

GregZTL
Novice
Posts: 3
Liked: 2 times
Joined: Jan 13, 2014 2:27 pm
Full Name: Greg Smith
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by GregZTL »

Swapping to the new DLLs appears to break my ability to create new Repos with the message 'Detecting server configuration Error: Method not found: 'Veeam.Backup.Common.EOSPlatform Veeam.Backup.Model.CphysicalHostInfo.get_OsPlatform()' at the 'Apply' step. I'm running v7.0.0.690. Changing back to the original DLLs makes it work again, so I've used this as a work-around while I try the patch out.

foggy
Veeam Software
Posts: 20100
Liked: 1878 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by foggy »

GregZTL wrote:I'm running v7.0.0.690.
Greg, as it is stated in Anton's post introducing the fix, you should be running R2 update (v7.0.0.771) to be able to use it:
Gostev wrote:This code must be installed on B&R 7.0 R2.

GregZTL
Novice
Posts: 3
Liked: 2 times
Joined: Jan 13, 2014 2:27 pm
Full Name: Greg Smith
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by GregZTL »

Ah, thanks foggy, a bit of a fail there on my part! I assumed that since I'd downloaded the most recent installer (.iso) I'd be on the latest version, I've found the R2а update to install and try again.

foggy
Veeam Software
Posts: 20100
Liked: 1878 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by foggy »

It would be much appreciated if you could post the results of your testing here as we are highly interested in any feedback. Thanks!

sheath777
Influencer
Posts: 11
Liked: 2 times
Joined: Nov 27, 2013 10:17 pm
Full Name: Steve Heath
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by sheath777 »

I'm using a 64bit machine. Do the keys need to be added here:

HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication

Or here:

HKEY_LOCAL_MACHINE\SOFTWARE\Wow6432Node\Veeam\Veeam Backup and Replication

I've seen other posts that say to place reg keys in the Wow6432Node on 64bit machines. Thanks!

teknomage
Service Provider
Posts: 23
Liked: 2 times
Joined: Jul 21, 2010 8:55 pm
Full Name: Mike
Location: Fargo, ND
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by teknomage »

I have a question about this registry key:
ForceDeleteBackupFiles (DWORD)
Enables automatic cleanup of the backup repository from ALL existing files if any backup file from the latest full backup chain is missing (such as when the existing hard drive is replaced by the hard drive brought back from offsite location, and containing some older backup files).
Set to 3 to make the job delete the entire contents of the backup job’s folder only.
Set to 5 to make the job delete the entire contents of the root backup repository folder, thus deleting any files belonging to other jobs along. Note that if backup repository is pointed at the volume's root folder, the entire volume's contents will be erased.
I've tested the new patch with the following equipment:
  • Physical server
    USB attached RDX drive (removable hard drives) with three drives
    ESXi host with two shell (tiny) VMs
    I have added the DWORD and set its value to 3.
I have created a job two backup my two VMs (I've tried forward and reverse incremental) and ran the job manually once per RDX drive for a couple of cycles. In my testing, I have run the job three times per cartridge (nine times total) and even through I have my retention policy set to 2, I I now have three .vbk files on my cartridges. At no point did I see the backup job clear out the backup folder I have defined.

Can anyone confirm they are seeing the same results or help me find a flaw in my testing procedures?

Gostev
SVP, Product Management
Posts: 28221
Liked: 5007 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by Gostev »

sheath777 wrote:HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication
This.
teknomage wrote:I have created a job two backup my two VMs (I've tried forward and reverse incremental) and ran the job manually once per RDX drive for a couple of cycles. In my testing, I have run the job three times per cartridge (nine times total) and even through I have my retention policy set to 2, I I now have three .vbk files on my cartridges. At no point did I see the backup job clear out the backup folder I have defined.
Well, the whole idea behind the rotated media is that you have the job create backup files on the external drive, then you take the drives offsite. Of course, you still want the backup files on the drives you take offsite, so I am not sure why you expect them to be deleted from your cartridges at this point.

I don't know how RDX works, but assuming "cartridges" are separate folders/volumes/locations on the device, the deletion will happen if you do something like this:
1. Run the job against Cartridge 1, this will create some backup files there. Take the cartridge offsite once full, or as needed.
2. Run the job against Cartridge 2, this will create some backup files there (including the new full backup), take the cartridge offsite.
3. Run the job against Cartridge 3, this will create some backup files there (including the new full backup), take the cartridge offsite.
4. No more cartridges left in the device, so you bring Cartridge 1 (with the oldest backups) from the offsite location back to your office, and plug it back in.
5. Run the job against Cartridge 1 again. The job will not see any backup files from the previous step 3 on this cartridge, understand that this is some old cartridge that you now want to be reused, and will clean it up from any existing files automatically.

Normally, people just use external USB or eSATA hard drives for this purpose. This is what we ourselves were doing at Veeam in the early days.

Does it make sense?

teknomage
Service Provider
Posts: 23
Liked: 2 times
Joined: Jul 21, 2010 8:55 pm
Full Name: Mike
Location: Fargo, ND
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by teknomage »

That's essentially what I'm trying to accomplish except for filling the drive/cartridge each time. My intentions are to duplicate what we're doing currently with tape drives, which is to do a full backup every night and to take that backup drive offsite daily (regardless if its full or not).

My test is obviously not representative of an actual environment because of the tiny backup jobs. What you described will get me more than I was intending RPO-wise since most of the customers will have total backups smaller than the 1TB media I will typically be using. I will do more testing with more realistic backups jobs.

When does the clearing of the backup folder take place? I assume the job will realize that the folder/media is full and clear it before attempting to do the actual backup.

FYI: This is the type of drive we're currently standardized on using http://www.tandbergdata.com/us/index.cf ... -quikstor/

Gostev
SVP, Product Management
Posts: 28221
Liked: 5007 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by Gostev »

teknomage wrote:I assume the job will realize that the folder/media is full and clear it before attempting to do the actual backup.
The erase is unconditional. As long as the job cannot find the previous restore point in the backup location, it will delete all existing files from the location defined by value of the corresponding registry key.

teknomage
Service Provider
Posts: 23
Liked: 2 times
Joined: Jul 21, 2010 8:55 pm
Full Name: Mike
Location: Fargo, ND
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by teknomage »

Does it only erase if the drive is full?

Will the jobs be able to control the number of restore points through the setting in the job or is that not valid because the backup chain is not consistent?

Gostev
SVP, Product Management
Posts: 28221
Liked: 5007 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by Gostev »

The erase is unconditional, meaning no disk space checks are performed.

You are implicitly controlling the number of restore points you keep by inserting the hard drive with the oldest restore points and letting the job delete them. But there are no hard drives tracking or anything like that in the current code. We will erase and use whatever drive you give us.

teknomage
Service Provider
Posts: 23
Liked: 2 times
Joined: Jul 21, 2010 8:55 pm
Full Name: Mike
Location: Fargo, ND
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by teknomage »

That is not the behavior I saw in my testing. When I cycled through the drives the second and third time it kept adding .vbk files. After three times through i had three .vbk files.

Here are the registry settings I have configured. Are these correct?

[HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication]
"ForceCreateMissingVBK"=dword:00000001
"ForceDeleteBackupFiles"=dword:00000003

Gostev
SVP, Product Management
Posts: 28221
Liked: 5007 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: v7 Backup Copy Job removable media (rotated drives)

Post by Gostev »

Yes, your settings look good. Please open a support case and provide the debug logs (explain that this is beta, so that the logs are forwarded straight to R&D). Thanks!

Locked

Who is online

Users browsing this forum: cschweigert, oleg.feoktistov, olivier.rossi, PKaufmann, restore-helper and 50 guests