Error: Cannot find aspnetcore-runtime-6.*-linux-x64.tar.gz

[micoolpaul]

Hi,

Having updated to VBfMA v4, I'm seeing that I'm unable to deploy workers at present, the error I'm seeing is:

Code: Select all

[27.07.2022 13:03:47]   41 (1) Error: Worker VBA-868bef3e-483d-4fb0-bf0b-c9d6bd33e42b: Failed to create a new VM of size Standard_F2s_v2 in region westeurope. Deleting the VM...
[27.07.2022 13:03:47]   41 (1) Error: Cannot find aspnetcore-runtime-6.*-linux-x64.tar.gz
[27.07.2022 13:03:47]   41 (1) Type: System.Exception
[27.07.2022 13:03:47]   41 (1) Stack:
[27.07.2022 13:03:47]   41 (1)    at Veeam.Core.Verifiers.Verify(Boolean b, String message, Object arg1)
[27.07.2022 13:03:47]   41 (1)    at Veeam.Azure.Controller.Services.Worker.WorkerAgentDeploymentParameters.GetDotnetRuntimeFilePath()
[27.07.2022 13:03:47]   41 (1)    at Veeam.Azure.Controller.Services.Worker.WorkerAgentDeploymentParameters.UploadFilesToContainerAsync(IStorageBlobManager storageBlobManager, AzureStorageAccount storageAccount, WorkerDeploymentOptions deploymentOptions, String workerAgentContainer, CancellationToken cancellationToken)
[27.07.2022 13:03:47]   41 (1)    at Veeam.Azure.Controller.Services.Worker.WorkerAgentDeploymentParameters.Prepare(String vmName, WorkerDeploymentOptions deploymentOptions, WorkerQueues workerQueues, IAzureEngine engine, WorkerConfigurationCheck workerConfiguration, List`1 allTags, WorkerProfile workerProfile, CancellationToken cancellationToken)
[27.07.2022 13:03:47]   41 (1)    at Veeam.Azure.Controller.Services.Worker.WorkerCreatorService.CreateNewWorkerVmAsync(AzureRegionId regionId, String vmName, WorkerProfile workerProfile, WorkerDeploymentOptions deploymentOptions, WorkerQueues workerQueues, CancellationToken cancellationToken)
[27.07.2022 13:03:47]   41 (1) Error: Worker VBA-868bef3e-483d-4fb0-bf0b-c9d6bd33e42b: failed to create a new VM in region westeurope. Cannot find aspnetcore-runtime-6.*-linux-x64.tar.gz. Deleting the VM...
[27.07.2022 13:03:47]   41 (1) Deleting Azure Service Bus queue 'VBA-868bef3e-483d-4fb0-bf0b-c9d6bd33e42b_VbToWorker' from namespace 'veeamdwgiv5zw867h59idnus'...
[27.07.2022 13:03:47]   41 (1) Deleting Azure Service Bus queue 'VBA-868bef3e-483d-4fb0-bf0b-c9d6bd33e42b_WorkerToVb' from namespace 'veeamdwgiv5zw867h59idnus'...
[27.07.2022 13:03:48]   41 (1) Azure Service Bus queue 'VBA-868bef3e-483d-4fb0-bf0b-c9d6bd33e42b_VbToWorker' has been deleted from the namespace 'veeamdwgiv5zw867h59idnus'
[27.07.2022 13:03:48]   41 (1) Azure Service Bus queue 'VBA-868bef3e-483d-4fb0-bf0b-c9d6bd33e42b_WorkerToVb' has been deleted from the namespace 'veeamdwgiv5zw867h59idnus'
[27.07.2022 13:03:48]   41 (1) Error: Worker VBA-868bef3e-483d-4fb0-bf0b-c9d6bd33e42b: deployment failed, reservation in worker pool was canceled.
[27.07.2022 13:03:48]   41 (1) Error: Cannot find aspnetcore-runtime-6.*-linux-x64.tar.gz
[27.07.2022 13:03:48]   41 (1) Type: System.Exception
[27.07.2022 13:03:48]   41 (1) Stack:
[27.07.2022 13:03:48]   41 (1)    at Veeam.Core.Verifiers.Verify(Boolean b, String message, Object arg1)
[27.07.2022 13:03:48]   41 (1)    at Veeam.Azure.Controller.Services.Worker.WorkerAgentDeploymentParameters.GetDotnetRuntimeFilePath()
[27.07.2022 13:03:48]   41 (1)    at Veeam.Azure.Controller.Services.Worker.WorkerAgentDeploymentParameters.UploadFilesToContainerAsync(IStorageBlobManager storageBlobManager, AzureStorageAccount storageAccount, WorkerDeploymentOptions deploymentOptions, String workerAgentContainer, CancellationToken cancellationToken)
[27.07.2022 13:03:48]   41 (1)    at Veeam.Azure.Controller.Services.Worker.WorkerAgentDeploymentParameters.Prepare(String vmName, WorkerDeploymentOptions deploymentOptions, WorkerQueues workerQueues, IAzureEngine engine, WorkerConfigurationCheck workerConfiguration, List`1 allTags, WorkerProfile workerProfile, CancellationToken cancellationToken)
[27.07.2022 13:03:48]   41 (1)    at Veeam.Azure.Controller.Services.Worker.WorkerCreatorService.CreateNewWorkerVmAsync(AzureRegionId regionId, String vmName, WorkerProfile workerProfile, WorkerDeploymentOptions deploymentOptions, WorkerQueues workerQueues, CancellationToken cancellationToken)
[27.07.2022 13:03:48]   41 (1)    at Veeam.Azure.Controller.Factories.WorkerVmFactory.CreateNewWorkerVmAsync(AzureRegionId regionId, String vmName, WorkerProfile workerProfile, WorkerQueues workerQueues, CancellationToken cancellationToken)
[27.07.2022 13:03:48]   41 (1)    at Veeam.Azure.Controller.Factories.WorkerVmFactory.CreateNewWorkerVmAsync(AzureRegionId regionId, String vmName, WorkerProfile workerProfile, WorkerQueues workerQueues, CancellationToken cancellationToken)
[27.07.2022 13:03:48]   41 (1)    at Veeam.Azure.Workers.WorkerFactory.CreateAsync(AzureRegionId region, WorkerProfile profile, String name, CancellationToken cancellationToken)
[27.07.2022 13:03:48]   41 (1)    at Veeam.Azure.Workers.WorkerDeployer.DeployNewWorker(CreatingWorkerInfo creatingWorkerInfo, CancellationToken cancellationToken)
[27.07.2022 13:03:48]   79 (1) Processing the worker message (component WorkerProvider, message: WorkerNotAvailableMessage(Reason: Worker deployment failed, see logs for details (eventID: 1105914088), Region: westeurope Purpose: Small), profile: Small)
[27.07.2022 13:03:48]   79 (1) Worker provider: no workers are available in region westeurope for Small purpose because Worker deployment failed, see logs for details (eventID: 1105914088)
[27.07.2022 13:03:48]   79 (1) Failed to assign a worker. No workers are available. The operation will be retried (profile: Small)

I can see it's interacting with the Servicebus successfully and all other tasks are successful such as triggering snapshots, finding the storage accounts etc. This was working fine the day before when running as v3, updated last night.

To confirm if this was a missing package from VBfMA instead of the worker VM, I listed the installed packages and could see two aspnetcore packages:
aspnetcore-runtime-3.1/bionic,now 3.1.27-1 amd64 [installed]
aspnetcore-runtime-6.0/bionic,now 6.0.7-1 amd64 [installed,automatic]

This makes me believe I'm seeing an issue with the worker deploying the required packages?

I'd raise a support case but this is an NFR license for testing in the lab. Anyone else seen this?

[nielsengelen]

Hi Michael,

This looks like a new technical issue. Can you open a case and let us know the ID so we can get someone on it?

[micoolpaul]

Thanks, have opened Case #05550950.

Have uploaded two sets of log files, one from last 24 hours which should capture the v4 behaviour, and one from the last 48 hours to provide a comparison to v3 behaviour.

[nielsengelen]

Hi Michael,

Got a follow-up internally and we were aware of the issue and got it solved. An update was pushed via the updater (which should either be available or already installed automatically).

Can you see if you still face the issue?

[micoolpaul]

Afternoon!

I can confirm it's working now, it automatically patched as it had no updates available.

Thanks for the prompt response!