Comprehensive data protection for all workloads
Post Reply
kortex
Influencer
Posts: 12
Liked: never
Joined: May 16, 2019 12:49 pm
Full Name: Manuel Berfelde
Contact:

Export list of guest files

Post by kortex »

Hi everyone,

is it possible to generate a list of all guest files in a backup? Maybe export the index?

A user deleted random files from a 13TB fileserver (don't ask *shakes head*) and we would like to compare the backup to the production data.

Thanks!
PetrM
Veeam Software
Posts: 3229
Liked: 520 times
Joined: Aug 28, 2013 8:23 am
Full Name: Petr Makarov
Location: Prague, Czech Republic
Contact:

Re: Export lsit of guest files

Post by PetrM »

Hi Manuel,

For example, you may start file level restore and try to get the list of files by querying C:\VeeamFLR folder on mount server.
However, such operation requires significant time as long as you have many files.

Thanks!
oleg.feoktistov
Veeam Software
Posts: 1912
Liked: 635 times
Joined: Sep 25, 2019 10:32 am
Full Name: Oleg Feoktistov
Contact:

Re: Export lsit of guest files

Post by oleg.feoktistov »

Agree with Petr.
You can get a list of all guest folders/files starting FLR and executing powershell cmd against C:\VeeamFLR folder:

Code: Select all

Get-ChildItem -Path "C:\VeeamFLR\<ServerName>_id\Volume1" -Recurse > "C:\GuestFiles.txt"
Or:

Code: Select all

Get-ChildItem -Path "C:\VeeamFLR\<ServerName>_id\Volume1" -Recurse | Export-Csv -Path "C:\GuestFiles.csv"
You can also highlight folders, which remained untouched for sure, with -Exclude parameter.
Thanks!
Oleg
kortex
Influencer
Posts: 12
Liked: never
Joined: May 16, 2019 12:49 pm
Full Name: Manuel Berfelde
Contact:

Re: Export lsit of guest files

Post by kortex »

Thanks for the responses!

I have already thought about that but we are talking about 13TB of small files.
As PetrM said: this will take ages.

I was hoping I could use the index Veeam generates during the backup...
kortex
Influencer
Posts: 12
Liked: never
Joined: May 16, 2019 12:49 pm
Full Name: Manuel Berfelde
Contact:

Re: Export lsit of guest files

Post by kortex »

btw: I tested the get-childitem way for a smaller machine but veeam decided to close the recovery session "due to reaching inactivity timeout"
soncscy
Veteran
Posts: 643
Liked: 312 times
Joined: Aug 04, 2019 2:57 pm
Full Name: Harvey
Contact:

Re: Export lsit of guest files

Post by soncscy »

Just off the cuff, you might try this with cygwin or change the mount point of the VeeamFLR to another drive and use Windows Subsystem for Linux.

Coreutils is magnitudes faster than Powershell (ps isn't bad...but it has nothing on coreutils) and use that instead. You're still looking at a few minutes but I've gone through billions of files with coreutil programs without even peaking the CPU.

Just locally with about 100k files on a laptop SSD:

time (ls -R | wc -l)

91670

real 0m1.748s
user 0m0.553s
sys 0m1.163s

I can run some tests when I'm in office tomorrow on a much larger subset (8 million some files). but I'm confident core utils will help you here.
PetrM
Veeam Software
Posts: 3229
Liked: 520 times
Joined: Aug 28, 2013 8:23 am
Full Name: Petr Makarov
Location: Prague, Czech Republic
Contact:

Re: Export lsit of guest files

Post by PetrM »

kortex wrote:veeam decided to close the recovery session "due to reaching inactivity timeout"
One more option is to try this registry value: InactiveFLRSessionTimeout (Type: REG_DWORD, Path: HKEY_LOCAL_MACHINE\SOFTWARE\Veeam\Veeam Backup and Replication\)
Default is 1800 (in sec, Decimal) but the value can be increased according to your specific requirements.

I'd suggest to contact our support team so that they help you to follow the procedure described above.

Thanks!
soncscy
Veteran
Posts: 643
Liked: 312 times
Joined: Aug 04, 2019 2:57 pm
Full Name: Harvey
Contact:

Re: Export lsit of guest files

Post by soncscy » 1 person likes this post

Morning all!

To bump this, I strongly believe coreutils will help. Here is a test from a demo-fileshare mounted via FLR with 8.5 million files:

lab-test+Administrator@lab-test /cygdrive/c/Veeamflr
$ time (ls -R | wc -l)
8502029

real 3m8.201s
user 0m11.185s
sys 0m11.608s

lab-test+Administrator@lab-test /cygdrive/c/Veeamflr
$ time (ls -R VeeamFLR > temp/output.csv)

real 3m13.111s
user 0m10.890s
sys 0m9.421s

I dunno about you, but 3 minutes isn't "that" bad to dump 8 million files to CSV, and I'm pretty confident this should be expedient for you :) Granted, you're gonna need to play a little with the formatting from shell to get an output that's usable for you to diff, but there are many examples online if you search.

Running the Get-ChildItem operation in powerhell got up around 1.5 hours before I just killed it. (and dumped a 2+ gb file versus the bash one that clocked in around 90 mb

PS C:\Users\Administrator\Desktop\> Get-ChildItem -Path C:\VeeamFLR -Recurse | Export-Csv "C:\temp\psexport.csv"
PS C:\Users\Administrator\Desktop\> ^C
PS C:\Users\Administrator\Desktop\> (Get-History)[-1].EndExecutionTime - (Get-History)[-1].StartExecutionTime

Days : 0
Hours : 1
Minutes : 47
Seconds : 30
Milliseconds : 334
Ticks : 64503346348
TotalDays : 0.0746566508657407
TotalHours : 1.79175962077778
TotalMinutes : 107.505577246667
TotalSeconds : 6450.3346348
TotalMilliseconds : 6450334.6348

Just dump all hopes of powershell on this I say :) PS is fine for many things, but it cannot compete for data munging with basic shell commands
oleg.feoktistov
Veeam Software
Posts: 1912
Liked: 635 times
Joined: Sep 25, 2019 10:32 am
Full Name: Oleg Feoktistov
Contact:

Re: Export lsit of guest files

Post by oleg.feoktistov »

Hi Harvey,

Have to agree on that. Since Powershell is object based, output of Get-ChildItem is passed to Export-Csv as a list of objects with properties.
Whereas, in Linux bash interpret commands output as plain text, which turns out to be very handy at dumping data swiftly :wink:

Best regards,
Oleg
Post Reply

Who is online

Users browsing this forum: Bing [Bot] and 260 guests