Strange error resolved

VMware specific discussions

Strange error resolved

Veeam Logoby TommyB » Wed Feb 03, 2016 4:05 pm

Hi,

just in case anyone else runs into this, I just want to describe a error situation I had for the last 2 weeks.

Out of nothing, several backups started to fail with the following error:

Processing VMware vCenter Server Appliance Error: Failed to open VDDK disk [[LUN1] VMware vCenter Server Appliance/VMware vCenter Server Appliance_1.vmdk] ( is read-only mode - [true] )
Failed to create folder.
Folder path: [C:\Windows\TEMP\VeeamBackup\VeeamAgent-3836\1b467bef.533am].
Logon attempt with parameters [VC/ESX: [192.168.0.230];Port: 443;Login: [root];VMX Spec: [moref=vm-26];Snapshot mor: [snapshot-6907];Transports: [hotadd:nbd];Read Only: [true]] failed because of the following errors: Failed


After trying to re-boot the VMs and all servers without success to see if it helps I contacted VEEAM support.

While they were checking the log files I sent, I had to do some (completely unrelated) testing regarding the windows task scheduler and decided to use the VEEAM proxy server, since this machine is not needed during business hours.

So I logged in and tried to set up a scheduled task - when I tried to save it, it threw an error and suddenly I realised that the local C: drive was completely filled up (0 bytes free).

After extending the partition by another 20 GB, the following backups worked again without any error.

Probably, error messages should be reworked in that area ;-)
TommyB
Expert
 
Posts: 115
Liked: 15 times
Joined: Wed Aug 28, 2013 9:46 am
Location: Germany.Europe.Terra.Sol.Milkyway.Localgroup.Virgo
Full Name: Thomas Braun

Re: Strange error resolved

Veeam Logoby Shestakov » Wed Feb 03, 2016 6:23 pm

Hi Thomas,

So your backup jobs were failing because some unrelated task filled up the backup proxy and once you freed the space, it started working again?
Is that a virtual or physical machine? Were that the only proxy chosen in the job settings?

By the way, do you use Veeam ONE Monitor alarms?
Thanks!
Shestakov
Veeam Software
 
Posts: 4861
Liked: 396 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

Re: Strange error resolved

Veeam Logoby alanbolte » Wed Feb 03, 2016 8:43 pm

The error message says Veeam was unable to create a folder on C:. Were you able to create the folders on the proxy's C: drive manually despite the error message? Or was it unclear that the problem was on the particular proxy?
alanbolte
Expert
 
Posts: 635
Liked: 170 times
Joined: Mon Jun 18, 2012 8:58 pm
Full Name: Alan Bolte

Re: Strange error resolved

Veeam Logoby TommyB » Fri Feb 05, 2016 10:47 am

Shestakov wrote:So your backup jobs were failing because some unrelated task filled up the backup proxy and once you freed the space, it started working again?
Is that a virtual or physical machine? Were that the only proxy chosen in the job settings?

By the way, do you use Veeam ONE Monitor alarms?


Yes, the proxy is a VM, and I think the full disk situation was caused by a combination of factors:

- The VMs disk was designed to small (20GB) from the beginning
- Lots of Windows Updates
- Lots of temporary files from the Veeam agent running on the proxy

The point is that the error message points to "Folder path: [C:\Windows\TEMP\VeeamBackup\VeeamAgent-3836\1b467bef.533am]. ", but it is not very clear about on *which* machine this is located.

Obviously a warning message including the full path (including computer name) would have been much more efficient to find the cause.

Or Veeam backup could check if the proxy has enough space before running the job and give an appropriate error message.

I have Veeam One installed but, to be honest, I'm not actively using it (but probably I should :oops: )

Thomas
TommyB
Expert
 
Posts: 115
Liked: 15 times
Joined: Wed Aug 28, 2013 9:46 am
Location: Germany.Europe.Terra.Sol.Milkyway.Localgroup.Virgo
Full Name: Thomas Braun

Re: Strange error resolved

Veeam Logoby TommyB » Fri Feb 05, 2016 10:52 am

alanbolte wrote:The error message says Veeam was unable to create a folder on C:. Were you able to create the folders on the proxy's C: drive manually despite the error message? Or was it unclear that the problem was on the particular proxy?


No, I could not create that folder - 0 bytes left... :wink:

The problem is that the error message does not mention the proxy servers name at all - maybe you can take a look at the log files, the Case # is 01677583
TommyB
Expert
 
Posts: 115
Liked: 15 times
Joined: Wed Aug 28, 2013 9:46 am
Location: Germany.Europe.Terra.Sol.Milkyway.Localgroup.Virgo
Full Name: Thomas Braun

Re: Strange error resolved

Veeam Logoby Shestakov » Fri Feb 05, 2016 10:56 am

TommyB wrote:Obviously a warning message including the full path (including computer name) would have been much more efficient to find the cause.

Or Veeam backup could check if the proxy has enough space before running the job and give an appropriate error message.

I have Veeam One installed but, to be honest, I'm not actively using it (but probably I should :oops: )

VBR doesn`t check periodically if VMs has a sufficient space, it just informs you that backup is failed because of the certain reason.

Veeam ONE Monitor is what you need to prevent it. You should receive "Guest disk space" alarm when the free space went below the threshold on the proxy.
Shestakov
Veeam Software
 
Posts: 4861
Liked: 396 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

Re: Strange error resolved

Veeam Logoby TommyB » Fri Feb 05, 2016 2:58 pm

Shestakov wrote:VBR doesn`t check periodically if VMs has a sufficient space,


Hmm ... IMHO it should at least check if a backup proxy has enough space - because it is part of the backup processing.
TommyB
Expert
 
Posts: 115
Liked: 15 times
Joined: Wed Aug 28, 2013 9:46 am
Location: Germany.Europe.Terra.Sol.Milkyway.Localgroup.Virgo
Full Name: Thomas Braun

Re: Strange error resolved

Veeam Logoby Shestakov » Fri Feb 05, 2016 3:20 pm

Well, it checks when the job is started and tells you the reason why it`s failed.
However, I`ve understood your feedback. Thank you for the pointing it out!
Shestakov
Veeam Software
 
Posts: 4861
Liked: 396 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov

Re: Strange error resolved

Veeam Logoby TommyB » Mon Feb 08, 2016 10:05 am

Shestakov wrote:Well, it checks when the job is started and tells you the reason why it`s failed.


Hmm... even your support was not able to immediately pinpoint the actual cause from the error message.

So there is potential for improvement (well... there always is, regardless how hard you may try :D )

Thomas
TommyB
Expert
 
Posts: 115
Liked: 15 times
Joined: Wed Aug 28, 2013 9:46 am
Location: Germany.Europe.Terra.Sol.Milkyway.Localgroup.Virgo
Full Name: Thomas Braun

Re: Strange error resolved

Veeam Logoby Shestakov » Mon Feb 08, 2016 12:25 pm

Thomas,
that`s a tricky thing to troubleshoot the issue via forums. To pinpoint the root of the problem immediately it`s better to take a closer look at logs.
Anyways I agree there is potential to improve. Thanks for the feedback once again!
Shestakov
Veeam Software
 
Posts: 4861
Liked: 396 times
Joined: Wed May 21, 2014 11:03 am
Location: Saint Petersburg
Full Name: Nikita Shestakov


Return to VMware vSphere



Who is online

Users browsing this forum: No registered users and 15 guests