Download file from VM to local host

[jw_ic]

Is there a way to download a file from the vm restore point, to your local host where you are running the REST commands?

Trying to do

https://emserver:9398/api/catalog/vms/v ... on=restore

in the request body I have json:

{
"FileRestoreSpec": {
"ForDirectDownload": "",
"_xmlns": "http://www.veeam.com/ent/v1.0",
"_xmlns:xsd": "http://www.w3.org/2001/XMLSchema",
"_xmlns:xsi": "http://www.w3.org/2001/XMLSchema-instance"
}
}

Then I get back:

<?xml version="1.0" encoding="utf-8"?>
<Task xmlns:xsd="http://www.w3.org/2001/XMLSchema" xmlns:xsi="http://www.w3.org/2001/XMLSchema-instance" Href="https://emserver:9398/api/tasks/task-5" Type="Task" xmlns="http://www.veeam.com/ent/v1.0">
<Links>
<Link Href="https://emserver:9398/api/tasks/task-5" Rel="Delete" />
<Link Href="https://emserver:9398/api/restoreSessio ... mat=Entity" Name="Restore job session" Type="RestoreSession" Rel="Related" />
</Links>
<TaskId>task-5</TaskId>
<State>Finished</State>
<Operation>RestoreFile</Operation>
<Result Success="true">
<Message>Restore operations started successfully.</Message>
</Result>
</Task>


then, if I try:
https://emserver:9398/api/catalog/vms/v ... n=download

I get:

{"FirstChanceExceptionMessage":null,"StackTrace":null,"Message":"{\"FirstChanceExceptionMessage\":null,\"StackTrace\":null,\"Message\":\"Resource does not exist\",\"StatusCode\":404,\"Status\":\"NotFound\"}","StatusCode":404,"Status":"NotFound"}

[chris.arceneaux]

Hi James,

Per our documentation, the API call you referenced doesn't allow direct download. Below is another API call with this ability:

(POST) /vmRestorePoints/{ID}/mounts/{ID}/{filepath}?action=restore

[jw_ic]

Sure, yes trying this.

POST:

Code: Select all

https://emserver:9398/api/vmRestorePoints/6eebf77d-5c23-4eaa-997e-09634555fb21/mounts/1/C:/tech/scripts/wssa.bat?action=restore

And getting

Code: Select all

{
    "FirstChanceExceptionMessage": null,
    "StackTrace": null,
    "Message": "The method or operation is not implemented.",
    "StatusCode": 500,
    "Status": "InternalServerError"
}

Headers including:

[{"key":"X-RestSvcSessionId","value":"MDJlZGE5MjQtZDNlMy00MWM5LWI0ZTUtN2YwYjM3OTdhOWQw","description":"","type":"text","enabled":true}]
[{"key":"Accept","value":"application/json","description":"","type":"text","enabled":true}]
[{"key":"Content-Type","value":"application/json","description":"","type":"text","enabled":true}]

Body: ( Raw / json )

Code: Select all

{
   "FileRestoreSpec": {
      "ForDirectDownload": "",
      "_xmlns": "http://www.veeam.com/ent/v1.0",
      "_xmlns:xsd": "http://www.w3.org/2001/XMLSchema",
      "_xmlns:xsi": "http://www.w3.org/2001/XMLSchema-instance"
   }
}

fyi; Using Postman to test this out.

[jw_ic]

In the REST API log I see this:

Code: Select all

[06.01.2021 10:38:27] <62> Error    Failed to start file restore from mount point. vmRestorePointIdStr: 6eebf77d-5c23-4eaa-997e-09634555fb21, mountIdStr: 1, path: C:/tech/scripts/wssa.bat. RequestUri: [URI template match was not found].
[06.01.2021 10:38:27] <62> Error    The method or operation is not implemented. (System.NotImplementedException)
[06.01.2021 10:38:27] <62> Error       at Veeam.Backup.Enterprise.RestAPIService.CRestApiFlrBrowserScope`2.StartFileRestoreFromMountPoint(String vmPointIdStr, String mountIdStr, String displayPath, FileRestoreSpecType restoreSpec)
[06.01.2021 10:38:27] <62> Error       at Veeam.Backup.Enterprise.RestAPIService.CRestRestoreMountPointControllerStub.<StartFileRestoreFromMountPoint>d__8.MoveNext()
[06.01.2021 10:49:45] <59> Info     [GET] request to [https://veeamem:9398/api/vmRestorePoints/6eebf77d-5c23-4eaa-997e-09634555fb21/mounts/1/C:/tech/scripts?action=listFiles&amp;pageSize=10&amp;page=1] deserialized. Message: [null]; Parameters: [].
[06.01.2021 10:49:45] <59> Info     [02eda924d3e341c9b4e57f0b3797a9d0] Retrieving mount point file items list. vmPointIdStr: 6eebf77d-5c23-4eaa-997e-09634555fb21, mountIdStr: 1, path: C:/tech/scripts, pageSize: , pageNum: 
[06.01.2021 10:49:54] <38> Info     [POST] request to [https://veeamem:9398/api/vmRestorePoints/6eebf77d-5c23-4eaa-997e-09634555fb21/mounts/1/C:/tech/scripts/wssa.bat?action=restore] deserialized. Message: [{
[06.01.2021 10:49:54] <38> Info        "FileRestoreSpec": {
[06.01.2021 10:49:54] <38> Info           "ForDirectDownload": "",
[06.01.2021 10:49:54] <38> Info           "_xmlns": "http://www.veeam.com/ent/v1.0",
[06.01.2021 10:49:54] <38> Info           "_xmlns:xsd": "http://www.w3.org/2001/XMLSchema",
[06.01.2021 10:49:54] <38> Info           "_xmlns:xsi": "http://www.w3.org/2001/XMLSchema-instance"
[06.01.2021 10:49:54] <38> Info        }
[06.01.2021 10:49:54] <38> Info     }]; Parameters: [].
[06.01.2021 10:49:54] <38> Info     [02eda924d3e341c9b4e57f0b3797a9d0] Starting file restore from mount point. vmPointIdStr: 6eebf77d-5c23-4eaa-997e-09634555fb21, mountIdStr: 1, path: C:/tech/scripts/wssa.bat
[06.01.2021 10:49:54] <38> Error    Failed to start file restore from mount point. vmRestorePointIdStr: 6eebf77d-5c23-4eaa-997e-09634555fb21, mountIdStr: 1, path: C:/tech/scripts/wssa.bat. RequestUri: [URI template match was not found].
[06.01.2021 10:49:54] <38> Error    The method or operation is not implemented. (System.NotImplementedException)
[06.01.2021 10:49:54] <38> Error       at Veeam.Backup.Enterprise.RestAPIService.CRestApiFlrBrowserScope`2.StartFileRestoreFromMountPoint(String vmPointIdStr, String mountIdStr, String displayPath, FileRestoreSpecType restoreSpec)
[06.01.2021 10:49:54] <38> Error       at Veeam.Backup.Enterprise.RestAPIService.CRestRestoreMountPointControllerStub.<StartFileRestoreFromMountPoint>d__8.MoveNext()

[oleg.feoktistov]

Hi James,

EM REST API interprets converted self-closing xml tags as a json object, and all the rest is redundant. For your request body just use this:

Code: Select all

{
      "ForDirectDownload": {}
}

Thanks,
Oleg

[jw_ic]

Was able to get father with the suggestion of using

Code: Select all

{
      "ForDirectDownload": {}
}

So thank you Oleg!

I am using Postman and it put the file in the Body for the results? Is there a way to save it as a file vs display it? and can you specify where it downloads to on the local file system?

Thanks

[jw_ic]

fyi; I do have a case open: Case #04576926

[oleg.feoktistov]

Yes, postman displays file contents in response body by default. The easiest way to save files from postman UI is to expand "Send" button and choose "Send and Download" instead. Upon request completion, Postman should prompt you to save the file to your filesystem.
If we are talking about full automation, code would depend on the language and libraries you use, but the algorithm is all the same:

• Create request to the download endpoint.

• Create writable stream on your file system. Use full path (e.g. "/home/admin/Downloads/file.log").

• Send request and pipe response to a writable stream.

• Close writable stream and check the file on your fs.

Hope it helps,
Oleg

[jw_ic]

Ah ok, didn't notice that on postman.

So when automating, I would be using PowerShell. Would you have a quick example for downloading?

[oleg.feoktistov]

Just pipe out the response to Out-FIle cmdlet and pass -FilePath with a precise file name:

Code: Select all

 Invoke-RestMethod -Method Get -Uri $uri -Headers $headers | Out-File -FilePath 'C:\file.log' 

Or use success stream redirection:

Code: Select all

 Invoke-RestMethod -Method Get -Uri $uri -Headers $headers > 'C:\file.log' 

With such approach responses will be redirected to a file only on success.

Thanks!

[jw_ic]

Easy enough, thanks again for your help!

[jw_ic]

@oleg.feoktistov

For the powershell to create the json object I did:

Code: Select all

$jsonBase = @{}
$jsonBase.Add("ForDirectDownload","{}")
$json = $jsonBase | ConvertTo-Json
$json

But was getting:

Code: Select all

Invoke-WebRequest : {"FirstChanceExceptionMessage":null,"StackTrace":null,"Message":"restoreSpec.ForDirectDownload : Error converting value \"{}\" to type 'Veeam.Backup.Interaction.RestAPI.Resources.FileRestoreSpecTypeForDirectDownload'. Path 
'ForDirectDownload', line 2, position 30. ","StatusCode":400,"Status":"BadRequest"}
At line:1 char:11
+ $result = Invoke-WebRequest -Uri $uri -method post -Headers $headers  ...
+           ~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~~
    + CategoryInfo          : InvalidOperation: (System.Net.HttpWebRequest:HttpWebRequest) [Invoke-WebRequest], WebException
    + FullyQualifiedErrorId : WebCmdletWebResponseException,Microsoft.PowerShell.Commands.InvokeWebRequestCommand

Thanks

[jw_ic]

Oh, maybe I have to do the ?action=restore before I can do ?action=download? Or can I just do ?action=download

[jw_ic]

Looks like that was the case, aka need to do the ?action=restore first, then ?action=download second. So with the above Invoke-Rest command I got the file downloaded. The file I tried was a text file, so now need to try a more binary file and make sure that works as well. Wondering now though if there is a way to get how long it will take to download the file, and if the file has finished downloading. But first thing is first is try to get a binary type file.

[jhoughes]

Yes, the restore action must always be processed first.

PowerShell will not return the prompt for you until the download has completed, so you'll know when it's completed. I don't remember of the top of my head if you'll have a progress bar in the PowerShell window or not, as it's been a while since I've done a "stare and compare" recovery.

There won't be any method to determine the time to recover beyond estimating it for your setup, as it would be dependent upon the setup of the session and basic overhead, the size of the file, the transfer speed of the download, along with the file operations on the host running the PowerShell session, and nothing in the API would calculate that automatically.

[oleg.feoktistov]

I don't have PS console at hand to duly test it, and I don't normally use powershell to interact with REST API,
but the script shared here looks good to me in terms of setting up a progress bar for your file downloads.
Try one and see if it works. Thanks!

[jw_ic]

Should have posted this sooner but here was the code to downloading a file via the rest endpoint with powershell script. Just in case it helps someone else out trying to do the same.



vmFileRestore.ps1

Code: Select all

Param(
    [parameter(Mandatory=$true)]
    [String] $empadmin,
    [parameter(Mandatory=$true)]
    [String] $emadminpwd,        
    [parameter(Mandatory=$true)]
    [String] $vmserver,
    [parameter(Mandatory=$true)]
    [String] $restoredate,
    [parameter(Mandatory=$true)]
    [String] $source_dir
)

#------------------------------------------------------------------------------------------------------------------
#region TEST VARIABLES
<# 

    $emadmin = "emadmin"
    $emadminpwd = "empasswd"
    $vmserver = "vmservername"
    $restoredate = "4/28/2021"
    $source_dir = "d:\sql\backups"

    $restoredate
    $restoredate.GetType()
    $restoredate2 = $restoredate | Get-Date -Format "yyyyMMdd"
    $restoredate2.GetType()
    $restoredate2

    $dest_dir = "c:/restore"
    $dest_dir
#>
#endregion

#------------------------------------------------------------------------------------------------------------------
#region SETUP
#------------------------------------------------------------------------------------------------------------------
# Set TLS Accepted 
[Net.ServicePointManager]::SecurityProtocol = [Net.SecurityProtocolType]::Tls -bor [Net.SecurityProtocolType]::Tls11 -bor [Net.SecurityProtocolType]::Tls12

add-type @"
    using System.Net;
    using System.Security.Cryptography.X509Certificates;
    public class TrustAllCertsPolicy : ICertificatePolicy {
        public bool CheckValidationResult(
            ServicePoint srvPoint, X509Certificate certificate,
            WebRequest request, int certificateProblem) {
            return true;
        }
    }
"@

#------------------------------------------------------------------------------------------------------------------
## Trust All Unknown Certs 
[System.Net.ServicePointManager]::CertificatePolicy = New-Object TrustAllCertsPolicy

#------------------------------------------------------------------------------------------------------------------
## Set Veeam Environment 
$uri_base = "https://veeamEntMgr:9398/api"
$uri_login  =  $uri_base + "/sessionMngr/?v=latest"

#------------------------------------------------------------------------------------------------------------------
# ToDo move Creds to PS file
$SSPassword = ConvertTo-SecureString $emadminpwd -AsPlainText -Force
$credentials = New-Object System.Management.Automation.PSCredential ($emadmin, $SSPassword)

#------------------------------------------------------------------------------------------------------------------
# Login to Enterprise Manager
$response = Invoke-WebRequest $uri_login -Method Post -Credential $credentials

#------------------------------------------------------------------------------------------------------------------
# Get the Session ID Returned
$session = $response.Headers["X-RestSvcSessionId"]

#------------------------------------------------------------------------------------------------------------------
# Create Headers with the SessionID
$headers = New-Object "System.Collections.Generic.Dictionary[[String],[String]]"
$headers.Add("X-RestSvcSessionId", $session)
$headers.Add("Accept", "application/json")
$headers.Add("Content-Type", "application/json")
$headers.Add("Accept-Encoding", "gzip, deflate, br")
$headers

#endregion 

#------------------------------------------------------------------------------------------------------------------
#region FIND BAK FILES STEPS
#------------------------------------------------------------------------------------------------------------------
# (GET) /catalog/vms/{vmname}/vmRestorePoints to be able to find which restorepoint to use in next step
$uri = "${uri_base}/catalog/vms/${vmserver}/vmRestorePoints"
$vmrp = invoke-RestMethod -Uri $uri -Method GET -Headers $headers
$vmrp

#------------------------------------------------------------------------------------------------------------------
# Loop to find specific vmRestorePoints id based on $restoredate  ( Measure how long this takes)
measure-command {
    $restore = @{}
    $vmrp.Refs | ForEach-Object { 

            Write-Host "VM Found, Possible Restore Points: "
            Write-Host "Name: " $_.Name 
            Write-Host " UID:  " $_.UID
            Write-Host "Href: " $_.Href
            Write-Host "Type: " $_.Type
            Write-Host "----------------------" 

            if ( $_.Name -match $restoredate ) {
                Write-Host "Restore Point Based on Date: " $restoredate
                $restore.status = "True"
                $restore.Name = $_.Name
                $restore.UID =  $_.UID
                $restore.Href = $_.Href
                $restore.Type = $_.Type

            }  else {
                $restore.status = "False"
            }    
    }

    if ( $restore.status = "True" ) {
        Write-Host "Match Name: " $restore.Name 
        Write-Host "Match UID: "  $restore.UID
        Write-Host "Match Href: " $restore.Href
        Write-Host "Match Type: " $restore.Type
        Write-Host "----------------------" 
    }
}

#------------------------------------------------------------------------------------------------------------------
# Storing uri reference for restore point
$vmrp_uri = $restore.Href
$vmrp_uri

#------------------------------------------------------------------------------------------------------------------
# Store vmRestorePoint ID to $VMRP_ID ; Would return VMRP_ID for future Reference to be able to download later 
$split = $restore.UID -split ":"
$VMRP_ID = $split[3]
$VMRP_ID
                 
#------------------------------------------------------------------------------------------------------------------
# Mount to access Filesystems
$uri = "${uri_base}/catalog/vms/${vmserver}/vmRestorePoints/${VMRP_ID}?action=browse"
$mount_task = invoke-RestMethod -Uri $uri -Method POST -Headers $headers
$mount_task

#------------------------------------------------------------------------------------------------------------------
# Find Guestfs down links 
# $result = $mount_task.Links |  Select-Object rel, href | Where-Object { $_.Rel -match "Down"}
# $down_uri = $result.Href
# $down_uri

#------------------------------------------------------------------------------------------------------------------
# Find file in Source Directory
# $uri = "${uri_base}/catalog/vms/${vmserver}/vmRestorePoints/${VMRP_ID}/guestfs/${source_dir}/?action=listAll"
# $result = invoke-RestMethod -Uri $uri -Method GET -Headers $headers
# $result

#------------------------------------------------------------------------------------------------------------------
# Get specific file based on restore date 

# $result.Directories.DirectoryEntries.Count
# $result.Directories.DirectoryEntries | Select-Object Name
# $result.Files.FileEntries.Count
# $file = $result.Files.FileEntries | Select-Object Name, links, href | Where-Object { $_.Name -match "20210209"}
# $uri = $file.Links.Href
# $uri 

# $source_filename = $file.Name
# $source_filename

# $original_file = $source_dir + "/" + $source_filename
# $original_file

# $VMRP_ID

#endregion

#------------------------------------------------------------------------------------------------------------------                           
#region DOWNLOAD STEPS 
# ------------------------------------------------------------------------------------------------------------------                
# Mount VMDK this would be later step to download the file
$uri="${uri_base}/vmRestorePoints/${VMRP_ID}/mounts"
$mount_task = invoke-RestMethod -Uri $uri -Method POST -Headers $headers
$mount_task_uri = $mount_task.Href 
$mount_task_uri

#------------------------------------------------------------------------------------------------------------------
# Poll Mount Task Staus for Finish 
$count=0
do { 
    $mount_result = invoke-RestMethod -Uri $mount_task_uri -Method GET -Headers $headers 
    write-host "status: " + $mount_result.state + "Count: " $count  
    $count++
    start-sleep 5
 } until ( $mount_result.state -eq "Finished"  )
 write-host "Mount Finished"
 
 $mount_result 

#------------------------------------------------------------------------------------------------------------------
# Get Mount uri
$r1 = $mount_result.Links | Select-Object *
$related = $r1 | Select-Object rel, href | Where-Object { $_.Rel -match "Related"}
$mount_uri = $related.href
$mount_uri

#------------------------------------------------------------------------------------------------------------------
# Find .bak file in mount based on date
$uri = "${mount_uri}/${source_dir}/?action=listAll"
$result = invoke-RestMethod -Uri $uri -Method GET -Headers $headers
$result

#------------------------------------------------------------------------------------------------------------------
# Get specific file based on restore date 
##$result.Directories.DirectoryEntries.Count
##$result.Directories.DirectoryEntries | Select-Object Name
$result.Files.FileEntries.Count
$file = $result.Files.FileEntries | Select-Object Name, links, href | Where-Object { $_.Name -match $restoredate2 }  # filename.bak
$uri = $file.Links.Href
$uri 

$r1 = $file.Links | Select-Object *
$related = $r1 | Select-Object rel, href | Where-Object { $_.Rel -match "Restore"}
$download_uri = $related.href
$download_uri

$restoreTo= $dest_dir +"/" + $file.Name
$restoreTo

#------------------------------------------------------------------------------------------------------------------
# Run the Restore/Download Job:
$restore_task = Invoke-WebRequest -Uri $download_uri -method post -Headers $headers -Body $json
$restore_task | Select-Object -Property *
$object = $restore_task  | ConvertFrom-Json
$restore_task_uri = $object.Href 
$restore_task_uri

#------------------------------------------------------------------------------------------------------------------
## Get restore job task status
$count=0
do { 
    $mount_result = invoke-RestMethod -Uri $restore_task_uri -Method GET -Headers $headers 
    write-host "status: " $mount_result.state + "Count: " $count 
    $count++
    start-sleep 5
 } until ( $mount_result.state -eq "Finished"  )
 write-host "Mount Finished"

$mount_result 

#------------------------------------------------------------------------------------------------------------------
# Download File to restore
$r1 = $mount_result.Links | Select-Object *
$related = $r1 | Select-Object rel, href | Where-Object { $_.Rel -match "Download"}
$download_uri = $related.href
$download_uri

If(!(test-path $dest_dir)) {
      New-Item -ItemType Directory -Force -Path $dest_dir
}

$ProgressPreference = 'Continue'  #on
#$ProgressPreference = 'SilentlyContinue'  #off

measure-command {  
    Invoke-WebRequest -Method Get -Uri $download_uri -Headers $headers -OutFile $restoreTo -ContentType "application/octet-stream"
}

#------------------------------------------------------------------------------------------------------------------
# test download
<#
    $source = 'http://speedtest.tele2.net/100MB.zip'
    $destination = 'c:\restore\100MB.zip'
    Start-BitsTransfer -Source $source -Destination $destination
    Invoke-WebRequest -Method Get -Uri $source -Headers $headers -OutFile $destination -ContentType "application/octet-stream"
#>

exit 0

#endregion