feature request - FileToTape Files Exclusions...and DFS

Everything about backing up to tape

feature request - FileToTape Files Exclusions...and DFS

Veeam Logoby richhartdr » Mon Sep 09, 2013 3:22 pm

Hey guys...just a feature request for v7 and tape support: Can't believe this is not in there...but it's not. You can't exclude folders or files from a FileToTape backup job... There are file masks for what to INCLUDE...but not what to EXCLUDE.

I ran into this while trying to exclude a hidden dfsrprivate (Microsoft DFS) folder that was under the root directory I'm trying to backup...Veeam did not like that folder at all and would lock up, not to mention it's unneeded as far as backing up anyway. This was a deal breaker so far, but I'm trying to see if I can backup this folder (with the dfsrprivate folder within) some other way...using permissions or anything. If there are any ideas or people have backed up DFS shares, please enlighten me! :) This said, Veeam will not be working for us to use for tape...sadly...we were excited to possibly get off of using two backup software's (the other being Symantec).

Basically, we DFS our remote office file server network shares back to corporate, and backup and archive all offices from these DFS destinations to tape...was hoping Veeam would allow us to switch...

Thanks for a great product no less though!
richhartdr
Enthusiast
 
Posts: 25
Liked: 4 times
Joined: Tue Dec 07, 2010 9:34 pm
Full Name: Don Richhart

Re: feature request - FileToTape Files Exclusions...and DFS

Veeam Logoby dualdj1 » Tue Sep 10, 2013 2:42 am

Second on an Exclusion option.
dualdj1
Enthusiast
 
Posts: 43
Liked: 4 times
Joined: Tue Feb 05, 2013 6:56 pm
Full Name: Jason K. Brandt

Re: feature request - FileToTape Files Exclusions...and DFS

Veeam Logoby v.Eremin » Tue Sep 10, 2013 8:16 am

Thanks, guys, for the feedback; highly-appreciated. We may consider adding this functionality in one of the future releases.

For now, if you want to exclude a given folder from archiving to tape and , meanwhile, you don’t want to go over all files one by one, manually including them, you can create a simple script that will add a new “file to tape" job and handle the manual file selection routine for you. Thanks.
v.Eremin
Veeam Software
 
Posts: 13266
Liked: 968 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: feature request - FileToTape Files Exclusions...and DFS

Veeam Logoby richhartdr » Tue Sep 10, 2013 4:05 pm

@v.Eremin - Good thought, but please don't hold back the details... :P

I'm browsing the forum PS beginner pages, command reference doc (http://veeampdf.s3.amazonaws.com/guide/ ... QL2ernA%3D), and searching all over the forums and yet fail to see any remove- *folder* cmdlets with ANYTHING related to TAPE jobs...(it's new in v7 is probably the reason) So is it documented somewhere else, or do I try and view all the new commands (and understand them) just from powershell itself? Do you have more specifics on your thoughts for this "simple script" for me being a noob and this being my first powershell (for Veeam)?

Ex. Would you expect something like the following workflow?
1) Use Windows powershell to grab a list of folders, excluding the one I need excluded, and apply to a variable (table?). (no Veeam-specific powershell snap-in required)
2) Use Veeam powershell cmdlets to create (or edit???) a file-to-tape job. (would require Veeam specific TAPE cmdlets, which I can't find)
3) Add the folders from step 1's variable (table?) to the newly created Veeam job. (again, would require Veeam specific TAPE cmdlets)
4) Run the job. (normal Veeam commands and I know how to do this)

...basically, yes, I could create a "script" to backup my VM infrastructure and files on my own, but then why would I buy Veeam or pay maintenance? In fact, the answer of "jsut create a simple script" could probably be used for most forum questions, if the poster was not obligated to offer any ideas how they see this being buiilt in their mind...even if it some top level generics like the workflow above... ;)

...I really wonder how a simple folder/file exclusion option was missed as a development topic when tape was being added...seems a no-brainer...like the file masks, which Veeam DID include.

Thanks for your help though, I know this probably sounds like I don't appreciate your answer...I do....but all to often I see "just create a simple script" posted with nothing to back it. I'm sorry your audience here is not a veteran Veeam PS creator...and as such I require a little more <sigh> hand holding...

Cheers v.Eremin
richhartdr
Enthusiast
 
Posts: 25
Liked: 4 times
Joined: Tue Dec 07, 2010 9:34 pm
Full Name: Don Richhart

Re: feature request - FileToTape Files Exclusions...and DFS

Veeam Logoby richhartdr » Tue Sep 10, 2013 6:26 pm

Can someone tell me the Veeam powershell commands (or better, tell me where to learn v7 and tape commands myself!) for the following:

1) List the file/folders, server, and file masks currently in a tape job (hopefully knowing how these are storred, I'll be able to pick an output format from my "Get-ChildItem -Directory -Recurse -Exclude *dfsrprivate*" command so I can edit or add an entry to the tapejob)

2) Edit OR Add/Remove an existing file/folders, server, and file masks currently in a tape job

...I'll construct the rest to, hopefully, get what I need. Thanks in advance...

PS - Not looking for handouts, but not finding a lot of documentation on the v7 powershell tape commands yet...and the built-in get-help docs from the Veeam plug-in are not very helpful either.
richhartdr
Enthusiast
 
Posts: 25
Liked: 4 times
Joined: Tue Dec 07, 2010 9:34 pm
Full Name: Don Richhart

Re: feature request - FileToTape Files Exclusions...and DFS

Veeam Logoby richhartdr » Tue Sep 10, 2013 8:29 pm

Getting closer...this is making a new one, but I'll figure out the edit afterwards...

Can I merge these?
Add-VBRTapeFilesJob -Name "test" -Server "servername" -Path A:\folder\folder\folder -MediaPool "Full Network Backup" -MediaPoolIncremental "Incremental Network Backup"
...and...
gci -Path \\folder\folder\folder -Exclude *dfsrprivate*,*anyotherfileorfolder*
...something like:
Add-VBRTapeFilesJob -Name "test" -Server "servername" -Path A:\folder\folder\folder | -Exclude *dfsrprivate*,*anyotherfileorfolder* -MediaPool "Full Network Backup" -MediaPoolIncremental "Incremental Network Backup"

The -path parameter take pipes, but I'm not fully sure how they work yet...still looking...
richhartdr
Enthusiast
 
Posts: 25
Liked: 4 times
Joined: Tue Dec 07, 2010 9:34 pm
Full Name: Don Richhart

Re: feature request - FileToTape Files Exclusions...and DFS

Veeam Logoby richhartdr » Tue Sep 10, 2013 9:36 pm

Boom...well, got this far today...I can create a new files2tape job and populate it with subfolders excluding the dfsrprivate like I need:

Code: Select all
write-host "Starting Veeam Files2Tape Job test"
Add-PSSnapin -Name VeeamPSSnapIn -ErrorAction SilentlyContinue
$JobName = "test"
$DirList = Get-ChildItem -Directory A:\folder\letter -Exclude *dfsrprivate*,*plusexcludeadditionalitems*
Add-VBRTapeFilesJob -Name $JobName -Server "servername" -Path $DirList -MediaPool "Full Network Backup" -MediaPoolIncremental "Incremental Network Backup"


So...remaining questions:
1) Does this look "clean" and is it the best way?
2) Will it cover all files/folders? I noticed that running this gci (Get-ChildItem) command, you had to specifically call out -hidden or -system to display those...but what I have should do the trick since now I'm not relying on Veeam to exclude...
3) How can I just EDIT a job versus creating a new one...I'm guessing making a new one (and deleting the old one) will casue havoc on job history/restores? Implications of this?
4) Best practice for kicking this off (after adding del job, or re-making as edit job, etc.)...from Windows Scheduler? Other?

...thanks for everyone's inspiration on this, my first Veeam powershell! Woot! ;P
richhartdr
Enthusiast
 
Posts: 25
Liked: 4 times
Joined: Tue Dec 07, 2010 9:34 pm
Full Name: Don Richhart

Re: feature request - FileToTape Files Exclusions...and DFS

Veeam Logoby v.Eremin » Wed Sep 11, 2013 8:29 am

Does this look "clean" and is it the best way?

Yep, it does. I was about to create something similar, but, then saw you had done it yourself

Will it cover all files/folders?

As far as I can understand, it will cover all the folders (except hidden one) that are located under A:\folder\latter, as well as, files that are stored in these folders.

How can I just EDIT a job versus creating a new one

This is a known issue that nowadays you can’t change source settings of existing file to tape job via Powershell. This problem is likely to be addressed in one of the upcoming releases.

Best practice for kicking this off (after adding del job, or re-making as edit job, etc.)...from Windows Scheduler?

It’s completely up to you how to initiate start of this job. Thanks.
v.Eremin
Veeam Software
 
Posts: 13266
Liked: 968 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Re: feature request - FileToTape Files Exclusions...and DFS

Veeam Logoby richhartdr » Wed Sep 11, 2013 11:45 am

Thank you for your time and verification v.Eremin, it is very much appreciated. :)
richhartdr
Enthusiast
 
Posts: 25
Liked: 4 times
Joined: Tue Dec 07, 2010 9:34 pm
Full Name: Don Richhart

Re: feature request - FileToTape Files Exclusions...and DFS

Veeam Logoby richhartdr » Wed Sep 11, 2013 11:57 am

Actually, one last question(s) since you answered that I can only make a new job each time. How exactly with this behave in Veeam? Will it keep the same hierarchy in the Files section of Veeam, or will each job make a new tree? And can I delete the previous job each time and retain the files and the information to allow me to restore for the tape (and therefore job) it was originally backed up from? Also, will it retain the ability to run incrementals...or will every time be a full?

Sorry for the last blast, but got to be sure it's in working order doing this custom. :)
richhartdr
Enthusiast
 
Posts: 25
Liked: 4 times
Joined: Tue Dec 07, 2010 9:34 pm
Full Name: Don Richhart

Re: feature request - FileToTape Files Exclusions...and DFS

Veeam Logoby richhartdr » Wed Sep 11, 2013 6:26 pm

Additional question since the original does not seem to scale well with having multiple file paths to backup...(each one being a list of several folders of course, excluding the dfsrprivate). Questions is this:

I have the actual command:
Code: Select all
Add-VBRTapeFilesJob -Name $JobName -Server "This Server" -Path $DirListSum2 -MediaPool "Full Network Backup" -MediaPoolIncremental "Incremental Network Backup"


...and when trying to build the $DirLsitSum variable...it just will not take anything I throw at it (object, etc)...minus a single string:
Code: Select all
#Tried this:
$DirList1 = Get-ChildItem -Directory \\folder1 -Exclude *dfsrprivate* #string
$DirList2 = Get-ChildItem -Directory \\folder2 -Exclude *dfsrprivate* #string
$DirList3 = Get-ChildItem -Directory \\folder3 -Exclude *dfsrprivate* #string
$DirListSum = $DirList1,$DirList2,$DirList3 #object

#NEED to convert object back to string before submitting to Veeam's Add-VBRTapeFilesJob cmdlet
$DirListSum2 = $DirListSum | select-object #converted from object to string (I think)

#Using just this variable works fine:
Add-VBRTapeFilesJob -Name $JobName -Server "This Server" -Path $DirList1 -MediaPool "Full Network Backup" -MediaPoolIncremental "Incremental Network Backup"
$DirList1

#This also does NOT work:
Add-VBRTapeFilesJob -Name $JobName -Server "This Server" -Path $DirListSum -MediaPool "Full Network Backup" -MediaPoolIncremental "Incremental Network Backup"

#I've also tried [string] commands, -join commands, etc.  Help on Add-VBRTapeFilesJob's -Path parameter is not enough to figure this out minus trial and error...which I can't seem to solve.


Also...this might be all for nothing since people are getting <1MB/s on their files2tape jobes anyway...and that's certainly not production doable:
viewtopic.php?f=29&t=17802

Thanks!
richhartdr
Enthusiast
 
Posts: 25
Liked: 4 times
Joined: Tue Dec 07, 2010 9:34 pm
Full Name: Don Richhart

Re: feature request - FileToTape Files Exclusions...and DFS

Veeam Logoby richhartdr » Wed Sep 11, 2013 6:49 pm

This forum is awesome! Makes me figure out my own solutions...this works, but I swear I tried this earlier!
Code: Select all
$DirList = Get-ChildItem -Directory \\folder1,\\folder2,\\folder3 -Exclude *dfsrprivate*
Add-VBRTapeFilesJob -Name $JobName -Server "This Server" -Path $DirList -MediaPool "Full Network Backup" -MediaPoolIncremental "Incremental Network Backup"


I'll post the full code after I clean it up now...but like I said the <1MB/s limitation is a deal breaker unfortunately...so now I wait...
richhartdr
Enthusiast
 
Posts: 25
Liked: 4 times
Joined: Tue Dec 07, 2010 9:34 pm
Full Name: Don Richhart

Re: feature request - FileToTape Files Exclusions...and DFS

Veeam Logoby richhartdr » Wed Sep 11, 2013 6:59 pm

Final Code Snippet:
Code: Select all
#Veeam Files_To_Tape using \\folder1, \\folder2, and \\folder3 network paths:

#Enable to pass parameters from the command line.  Not applicable for the way my jobs are different though.
#param ([string]$JobName1,$JobName2 = "$(Read-Host 'Enter the new Veeam files2tape job name. [e.g. Files2Tape Network Backup]')")

#Notification job is starting
write-host "Veeam Files2Tape Job 'Files2Tape Network Backup' Started"

#Add the Veeam PowerShell Snap-in, if it's already loaded, continue silently
Add-PSSnapin -Name VeeamPSSnapIn -ErrorAction SilentlyContinue

#Create Variable(s)
$JobName1 = "test"

#Delete variable contents (for testing)
#Remove-Variable -Name DirList1 -Force

#Get a list of all subdirectories under several paths.  The -Directory parameter also allows for folder exclusions
$DirList1 = Get-ChildItem -Directory \\folder1,\\folder2,\\folder3 -Exclude *dfsrprivate*,*anyotherfolderexclusion*

#Delete any previous job(s) with the same name
Remove-VBRJob -Job $JobName #I'm still researching how to get this to delete more than one, and also to not prompt...seems simple in the help, but I don't have somethign quite right

#Create new job and configure settings
#Add-VBRTapeFilesJob -Name $JobName -Server "This Server" -Path $DirList1 -MediaPool "Full Network Backup" -MediaPoolIncremental "Incremental Network Backup"

#Notification job has completed
write-host "Veeam Files2Tape Job 'Files2Tape Network Backup' Completed"
richhartdr
Enthusiast
 
Posts: 25
Liked: 4 times
Joined: Tue Dec 07, 2010 9:34 pm
Full Name: Don Richhart

Re: feature request - FileToTape Files Exclusions...and DFS

Veeam Logoby richhartdr » Wed Sep 11, 2013 7:52 pm

feature request (unless 'm missing it) - no option to select "Eject media once the job finishes" in the Add-VBRTapeFilesJob cmdlet...also several other things you can't set via powershell that's in the GUI as well. I keep hitting caveat after caveat...yuck.
richhartdr
Enthusiast
 
Posts: 25
Liked: 4 times
Joined: Tue Dec 07, 2010 9:34 pm
Full Name: Don Richhart

Re: feature request - FileToTape Files Exclusions...and DFS

Veeam Logoby v.Eremin » Thu Sep 12, 2013 12:09 pm

Hi, Don.

Will it keep the same hierarchy in the Files section of Veeam, or will each job make a new tree?

There will be only one files tree under “Files” node.

And can I delete the previous job each time and retain the files and the information to allow me to restore for the tape (and therefore job) it was originally backed up from?

Yes, you will be able to restore files. By default, Veeam Backup & Replication will restore the latest version of files available on tape. If you want to restore files of different (previous) version, you should click "Backup Set" option in restoration wizard.

Also, will it retain the ability to run incrementals...or will every time be a full?

The new job will perform a full run.

(unless 'm missing it)

You’re correct here. Probably, we will add missing commandlets in one of the future releases. So, thanks for your feedback, much appreciated.

Thanks.
v.Eremin
Veeam Software
 
Posts: 13266
Liked: 968 times
Joined: Fri Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin

Next

Return to Tape



Who is online

Users browsing this forum: No registered users and 10 guests