Comprehensive data protection for all workloads
Post Reply
mrt
Enthusiast
Posts: 53
Liked: 2 times
Joined: Feb 10, 2011 7:27 pm
Contact:

backups writing to C:\Windows\SysWOW64

Post by mrt »

Why might my backups be suddenly writing to C:\Windows\SysWOW64 when this directory is not specified in the backup repo node? Other volumes specified in the node are available and have plenty of free space. Now C: is full and things are failing left and right. These jobs are also affected by the nbd failover bug.

The jobs are configured to make active fulls on the first Sunday of every month, which today happens to be.
I'll get in touch with support but am hoping for a quick answer here. Thanks
kellison
Enthusiast
Posts: 25
Liked: never
Joined: Dec 22, 2009 5:56 pm
Full Name: Ken Ellison
Contact:

[MERGED] Veeam 6.0 Server Hard Drive Space

Post by kellison »

I upgraded my Veeam server to 6.0 over the weekend - it is a VM Windows 2008 R2 server. Upgrade went well but last night was the first rn of backups. I had about 10 GB of hard drive space available on my "C" drive when I left for the day. The Veeam is installed on this drive. This morning I found that over half my backups had failed. Looking at the event logs I found out the "C" drive had run out of hard drive space. I then shut down the server and added 20 more GB to the "C" drive, and proceeded to run a backup. It is at 29 percent (running VERY SLOW) on the first VM of 6 and it has already used up 10 GB of hard drive space. What is causing this and how to I fix this? My retention period for my backups is 7 days.
HELP !!!
engagingit
Lurker
Posts: 1
Liked: never
Joined: Dec 06, 2011 1:37 pm
Full Name: Allan Napier
Contact:

Re: Veeam 6.0 Server Hard Drive Space

Post by engagingit »

Happening to me as well, a copy of the backup file is being made at c:\windows\syswow64

I'm assuming there is a flag to make it write directly to the usb drive rather than save locally first but I can't seem to find it.
real_tarantoga
Expert
Posts: 105
Liked: 22 times
Joined: Aug 20, 2009 12:32 pm
Location: Germany
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by real_tarantoga »

My veeam Backup Server is updated - and no more usable. :(
No way to disable excessive writing into %windir%\syswow64? (not resolved)
No way to chose the default repository while upgrading? (resolved)
A really annoying bug with the "synthetic full" check box! (resolved)

... and why a default repository on %systemdrive% ?!? What happens to veeam?

looking at your answer with anticipation, Stefan :wink:
kellison
Enthusiast
Posts: 25
Liked: never
Joined: Dec 22, 2009 5:56 pm
Full Name: Ken Ellison
Contact:

Re: Veeam 6.0 Server Hard Drive Space

Post by kellison »

Update - I had to stop my backup - it used up the entire 20 GB that I had added....
Vitaliy S.
VP, Product Management
Posts: 27374
Liked: 2799 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by Vitaliy S. »

Please contact our technical team as logs investigation is required. Thanks!
Gostev
Chief Product Officer
Posts: 31806
Liked: 7299 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by Gostev »

real_tarantoga wrote:and why a default repository on %systemdrive% ?!? What happens to veeam?
For consistency, and because usually, there are no other drives. The default repository is for evaluators only, of course we expect that for production usage, customers will create proper repositories pointing to the actual backup storage.

Regarding the actual problem discussed in this topic, we have received only a few reports like this, so this seems to affect very specific configuration of backup server only. Everyone who is affected should open a support case to let us figure out what is common.

Thanks!
real_tarantoga
Expert
Posts: 105
Liked: 22 times
Joined: Aug 20, 2009 12:32 pm
Location: Germany
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by real_tarantoga »

short status update

the support recommandation to use another backup proxy was almost right, but ... also 64bit backup proxies didn't work.
i tried on a 32bit server - and it was running.
I will now create a backup proxy server pool on an dedicated esx host cluster (DRS disabled), with 2003 r2 & 2008 r1 enterpise servers, giving them a lot of vCPUs ... and i will update my results here.

for now it seems to be an explicit 64bit problem.

Stefan
Sethbartlett
Veteran
Posts: 282
Liked: 26 times
Joined: Nov 10, 2010 6:51 pm
Full Name: Seth Bartlett
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by Sethbartlett »

You can also try deleting your current repository and creating a new one pointing to the same location and updating the job, this has seemed to work.
Skype: Sethbartlett88 - Make sure to label who you are and why you want to add me ;)
Twitter: @sethbartlett
If my post was helpful, please like it. Sometimes twitter is quicker to hit me up if you need me.
Gostev
Chief Product Officer
Posts: 31806
Liked: 7299 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by Gostev »

Must be related to automatic creation of repositories during the upgrade.
real_tarantoga
Expert
Posts: 105
Liked: 22 times
Joined: Aug 20, 2009 12:32 pm
Location: Germany
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by real_tarantoga »

Hello board,
the workaround with virtual machines used as backup proxies was very successful.
Even more I could run more concurrent jobs as anytime before! No real stress or heavy performance impact neighter on the VMs nor on the two ESX servers. So veeam jobs have processed the backup for 100 VMs over night - some uncounted terra bytes ... That's cool.

Other than mentioned above I could also use x64 machines.
The problem wasn't related to the architecture but to a mishandling of the existent backup jobs and/or backup files on the imported backup repository during the upgrade process.

The backup proxy on the upgraded veeam b&r 5.0.2.255:
  • works for all newly created jobs - it is not possible to map old vbk files to the new jobs, so I have to start with a full backup.
  • doesn't work for any job with exisisting vbk/vbr files, works after removing the backup files through the GUI
  • doesn't work for old jobs with german vowels (umlauts) - even if the manually given file name doesn't include umlauts! - and works when the jobs will be renamed. That always worked in the older program versions ...
  • doesn't work for jobs (vbm+vbk) moved from syswow64 to the backup repository, backups must be deleted
I wonder why the program continues to use the %systemroot%\syswow64 folder to put the meta file here for only some jobs, while other (new) jobs create subfolders on the repository and another group of jobs put the meta file right beside the existing vbk files ...

Finally, the problem seems to be resolved by working around step by step from issue to issue - and losing a lot of backups!!!
For the first time I'm deeply disappointed with the software and also with the support - not with the reaction time, but with the given answers and result (there's no solution or lossless workaround until now). I've got the impression they were "groping about in the dark" (?) as well.

Regards, Stefan
Gostev
Chief Product Officer
Posts: 31806
Liked: 7299 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by Gostev »

Hi Stefan, sorry for the issue you've experienced. I have forwarded your feedback to our support and QC management. I've also verified that the issue is not wide spread (just a few support cases). With any software, there is always a risk with running into issues when making a decision to upgrade to the new major release immediately upon its availability. It is fair to say that no amount of testing by limited QC group can cover all the configuration and upgrade scenarios our 35'000 customers might have. Thanks.
kellison
Enthusiast
Posts: 25
Liked: never
Joined: Dec 22, 2009 5:56 pm
Full Name: Ken Ellison
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by kellison »

I have to agree with Stefan - after going back and forth with support for a whole day - "try this - try this" I finally gave up and created a new datastore, added it as a new drive letter to my Veeam VM server, deleted the backups and repository and basically started from scratch again on my backups. Strange though that my other repository that was brought over from the upgrade is working fine. In fact, I had 2 different backups in the repository that I had deleted and the other backup worked fine. On my DR site that was also upgraded I am not having any trouble with the repositories (so far)... I'm not seeing any huge improvement in speed but guess will just be thankful that I have the backups working again.
flavor4real
Expert
Posts: 205
Liked: 5 times
Joined: Nov 22, 2010 7:57 pm
Full Name: DS
Contact:

[MERGED] vbk get push to a different path

Post by flavor4real »

Hello,
How is it going? All of my Veeam jobs worked fine until now. All of the sudden the one backup job creates their vbk in c:\windows\SysWOW64\

There were no prior issues with veeam and I'm kind of wondering. I already removed the VBK, verified the veeam job setting and tested again. Still does the same

Thanks
Gostev
Chief Product Officer
Posts: 31806
Liked: 7299 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by Gostev »

Hi, what product version are you running? This sounds like known v5 upgrade issue that was resolved loooooooooooong ago... I believe, in 6.1 release.
flavor4real
Expert
Posts: 205
Liked: 5 times
Joined: Nov 22, 2010 7:57 pm
Full Name: DS
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by flavor4real »

We are running 6.0. We have to wait until we get approval for the 6.1 version.

I received a respond from the Veeam Support, pointing me to KB ID: 1475 and I have a question. the KB below says:
=====================================================================
1) The repository for the backup job(s) will need to be deleted however you cannot delete the repository with jobs linked to it so what you can do is create another repository pointing to the same location.

2) Move the Veeam files (VBK, VRB, VIB, VBM) from the SysWow64 folder back into their original locations with the rest of the backup files for your jobs.

3) As long as you have a VBM file for each set of backups you can then simply go into Backup Repositories (under Backup Infrastructure) and re-scan the repositories that you just created. Veeam should then see that the backup files reside in that directory.

4) Go into your Backup jobs and point back to the newly created repository and click the Map Backups link. Your jobs should pick up where they left off and you shouldn't have this issue again.

5) Delete the original repository which should now have no jobs linked to it.
=====================================================================

Point 2 outlines, that the VBM need to be located into their original locations with the rest of the backup files for your job. My VMB's are located in the C:\Windows\SysWOW64 .. Do I have to move them?

Thanks
flavor4real
Expert
Posts: 205
Liked: 5 times
Joined: Nov 22, 2010 7:57 pm
Full Name: DS
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by flavor4real »

I have another question.
If I have one mapped drive and a few backup jobs save the generated backup files (vib's/ vbk's) in the location. Does each backup job need to have their own folder structure within the location?
Example:
Z:\Backupjob1\ ...*.vbk,...*.vib
Z:\Backupjob2\ ...*.vbk,...*.vib
Z:\Backupjob3\ ...*.vbk,...*.vib

or can everything be dumped like
Z:\ ...*.vbk,...*.vib

I'm asking because I'm trying to get my issue resolved and I saw that under V6.0, it seems to be that veeam creates a folder with the Job name under the storage path and have the vbk, vib, and the VMB in it.

If so, do I have to create these folder manually and move the VMB's in it, then rescan the repository?
Thanks
foggy
Veeam Software
Posts: 21138
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by foggy »

flavor4real wrote:Point 2 outlines, that the VBM need to be located into their original locations with the rest of the backup files for your job. My VMB's are located in the C:\Windows\SysWOW64 .. Do I have to move them?
Yes, move them along with all backup files.

flavor4real wrote:If so, do I have to create these folder manually and move the VMB's in it, then rescan the repository?
The subfolder for the job is automatically created in the backup repository during the first job run following the upgrade to v6. If you do not have the folder for this job, please create it manually and place all the job's files in it.
veremin
Product Manager
Posts: 20400
Liked: 2298 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by veremin »

flavor4real wrote: Does each backup job need to have their own folder structure within the location?
Example: Z:\Backupjob1\ ...*.vbk,...*.vib
or can everything be dumped like : Z:\ ...*.vbk,...*.vib
As far as I’m concerned, such structured architecture is necessary. Otherwise, step 4 of the described method will definitely fail with the following error:

Code: Select all

Move all backup files to a new repository first.
Thanks.
flavor4real
Expert
Posts: 205
Liked: 5 times
Joined: Nov 22, 2010 7:57 pm
Full Name: DS
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by flavor4real »

Very Interesting ... thanks guys.
flavor4real
Expert
Posts: 205
Liked: 5 times
Joined: Nov 22, 2010 7:57 pm
Full Name: DS
Contact:

[MERGED] Create New Repository and move Backup files

Post by flavor4real »

Hello,
I've search and found some posts but I have more questions.
I've upgraded from V5 to V6 and the backup jobs started to put their backup files to the ..\windows\sysWoW64 folder. I found many information but it's not really working out.

I have 4 existing repositories, pointing to 4 individual storage paths, and holding the vbk's/... for 3-4 backup jobs. Since previously all vbk's/vib's were simple dumped in each storage location.
I've therefore created the required folder structure Example:
Z:\Backupjob1\ ...*.vbk,...*.vib
Z:\Backupjob2\ ...*.vbk,...*.vib
Z:\Backupjob3\ ...*.vbk,...*.vib

and moved the jobs and the vbm in these locations. I rescanned the repositories and mapped each job to it. Now, Once I run the job it still put's the backup files in the ..\windows\sysWoW64 folder and creates a new vbm file.

so, what else do I need to do?
I've read that I need to create new repositories, I've done that.
Then I need to move the existing backups from the old repository to the new one. How do I do that?
Then I need to rescan
Then I need to map the jobs

Thanks,
veremin
Product Manager
Posts: 20400
Liked: 2298 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: [MERGED] Create New Repository and move Backup files

Post by veremin »

flavor4real wrote: I've read that I need to create new repositories, I've done that.
Then I need to move the existing backups from the old repository to the new one. How do I do that?
Then I need to rescan
Then I need to map the jobs
Thanks,
1.In your OS Just manually copy/paste existing folders (Backup Job1 and etc.) with backup files (.vbk,.vbm,.vib) stored in it to a new repository.
2.Then Rescan new repository.
3.After having done that, right-click the job, go to the “Storage” option and select newly-created repository from the list.
4.Map the job.

Additionally, If you still think that there isn’t enough information, you can easily use this topic as a source of knowledge.

That’s all.
Thanks.
flavor4real
Expert
Posts: 205
Liked: 5 times
Joined: Nov 22, 2010 7:57 pm
Full Name: DS
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by flavor4real »

does that mean that my storage admin has to attach 4 new storage to my Veeam Backup Server? So, then i would have 8 storage and i can move the data from the existing 4 to the new 4 and then remove the old 4.....
foggy
Veeam Software
Posts: 21138
Liked: 2141 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by foggy »

You do not need additional storage actually, just create new repositories pointing to the same location and move the backup files from the SysWow64 folder there, according to the new folder structure.

What you might be missing, is not just picking the new repository while mapping the job, but also pointing it to the existing backup files (using the Map backup link on the Storage step of the wizard).

Btw, here is another topic with the procedure outlined in detail: Moving To New Repository.

In case you're still stuck with getting the jobs writing to the required folders, I would recommend asking support to assist you via webex, this would probably be the quickest way to address that.
flavor4real
Expert
Posts: 205
Liked: 5 times
Joined: Nov 22, 2010 7:57 pm
Full Name: DS
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by flavor4real »

I see the new added Backup Repository in the storage selection mapping under the the job. however, once selected, I do not see any of the sub folders. I rescanned the repositories several times.

WebEx is not allowed in our environment ...
flavor4real
Expert
Posts: 205
Liked: 5 times
Joined: Nov 22, 2010 7:57 pm
Full Name: DS
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by flavor4real »

Version 6.0. We have not approval to use Version 6.1 yet.

Will a fresh install of Veeam 6.0 (instead of a upgrade) fix this issue? I could then just import the existing backup files.
Gostev
Chief Product Officer
Posts: 31806
Liked: 7299 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by Gostev »

Hi, I'd like to remind that this is NOT a support forum, as explained when you click New Topic.

Please open a support case, and our technical support will be able to assist you whether or not webex is allowed. The team behind this forum is not involved in support, and we cannot assist you efficiently with technical issues via forum posts. Besides, most issues require debug logs research. Thank you for understanding!
flavor4real
Expert
Posts: 205
Liked: 5 times
Joined: Nov 22, 2010 7:57 pm
Full Name: DS
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by flavor4real »

Understood... sorry, just brain storming
ENSFCa
Novice
Posts: 7
Liked: 1 time
Joined: Aug 17, 2012 11:08 am
Full Name: Andy Palmer
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by ENSFCa » 1 person likes this post

Hi, just wanted to chime in as I noticed this thread and, for me it is relevant.
We had an issue where this folder was filling our System drive on VM File Level Restore. We'd chose Windows FLR on our large data store VM and the system would hang for ages, eventually creating a huge file in this folder and requiring a services stop and restart (and cleanup of this folder and the C:\VeeamFLR folder too).
It's still not actually solved yet (case #155330, closed but awaiting fix) however the Veeam techies did find the cause - something called reparse points (http://en.wikipedia.org/wiki/NTFS_reparse_point).
We are using a Heirarchical Storage System which, as it should, archives old files and replaces the original file with a stub - these stubs are attempted to be verified by Veeam and each one takes a while to time-out - hence the hang and the massive cache file being produced as the system tries to deal with it.
Work-Around: use the 'Other' FLR option which mounts the VM in vCentre (via NFS) and allows you to browse the disk - it's slower and leaves bits behind in vCentre, but it's reliable.
flavor4real
Expert
Posts: 205
Liked: 5 times
Joined: Nov 22, 2010 7:57 pm
Full Name: DS
Contact:

Re: backups writing to C:\Windows\SysWOW64

Post by flavor4real » 1 person likes this post

I just wanted to give an update:

All the solutions did not worked. The Veeam Engineer WebEx'd in and tried a few things but there was no solution. so, I went and recreated one backup job and tested it. the Backup job created the folder structure based on the job names and put the *.vbm and backup file in. I therefore recreated all backup jobs and disabled the old ones. At first i received a slow backup speed of ~30-60 Mb/sec but after the initial backup, everything started to work again. I will now, slowly decommission the old backup jobs and remove the old backups.

So, the worst scenario is that you have to recreate the backup jobs, which is fine. You will not loos any data if you keep the old job and files intact and just slowly decommission then after your grace period.

The only issue I've experienced is that my backup storage locations disk space get pushed to full. I had to manually remove old backups one by one each day to ensure that I still can restore from the old data but have enough space for the new data to come in. After all, new jobs means a new retention period, so you need a bit of extra space. I managed.

Thanks for the support guys.
Post Reply

Who is online

Users browsing this forum: Google [Bot], Gostev, Majestic-12 [Bot], sally123, Semrush [Bot], Willy M. and 161 guests