Comprehensive data protection for all workloads
spiritie
Service Provider
Posts: 189
Liked: 40 times
Joined: Mar 01, 2016 10:16 am
Full Name: Gert
Location: Denmark
Contact:

Re: .vib files are left behind

Post by spiritie »

FrancWest wrote: Apr 09, 2021 12:59 pm you're welcome! Was this a completely new install of V11, or did the repository with the redundant files already exist before upgrading to V10?

According to the information I got this should be fixed in V11 RTM (and thus also GA).
This was upgraded server from v10. I forgot to add in my previous post that the leftover .vib files have an "created date" after v11 was installed.
FrancWest
Veteran
Posts: 488
Liked: 93 times
Joined: Sep 17, 2017 3:20 am
Full Name: Franc
Contact:

Re: .vib files are left behind

Post by FrancWest »

We are still at V10, so I have to rely on the word from Veeam that it's fixed in V11.
spiritie
Service Provider
Posts: 189
Liked: 40 times
Joined: Mar 01, 2016 10:16 am
Full Name: Gert
Location: Denmark
Contact:

Re: .vib files are left behind

Post by spiritie »

I've opened a case as well asking for the hotfix for our v10 installations: #04748788

I create Veeam forum post as well a few months ago, but Veeam suspected it to be something with our installation, nice to see it being fixed now:
veeam-backup-replication-f2/help-with-f ... 70546.html
pirx
Veeam Legend
Posts: 568
Liked: 72 times
Joined: Dec 20, 2015 6:24 pm
Contact:

Re: .vib files are left behind

Post by pirx »

Can someone please explain the naming of the hotfix files?

I received a ZIP with name Fix_307232_6f30225612.zip, this includes a exe file veeam_backup_10.0.1.4854_PrivateFix_TF305723.exe where the Install Wizard says its Cumulative Patch TF303125.

There is no readme or any connection to my case ID.

@FranceWest: support told me, my case is a different one than yours, thus I wanted to install the patch today. It seems there are different file lock issues, the one I have is unique...
Gostev
Chief Product Officer
Posts: 31455
Liked: 6646 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: .vib files are left behind

Post by Gostev »

Looks like something broke on the builder which is causing the incorrect hotfix naming, thanks for reporting.
FrancWest
Veteran
Posts: 488
Liked: 93 times
Joined: Sep 17, 2017 3:20 am
Full Name: Franc
Contact:

Re: .vib files are left behind

Post by FrancWest »

pirx wrote: Apr 12, 2021 4:53 pm @FranceWest: support told me, my case is a different one than yours, thus I wanted to install the patch today. It seems there are different file lock issues, the one I have is unique...
ok, thanks for the update. I'll wait for support to come back to me then.
pirx
Veeam Legend
Posts: 568
Liked: 72 times
Joined: Dec 20, 2015 6:24 pm
Contact:

Re: .vib files are left behind

Post by pirx »

Gostev wrote: Apr 12, 2021 7:33 pm Looks like something broke on the builder which is causing the incorrect hotfix naming, thanks for reporting.
The thing is, support told me this is how it should look like. I don't want to hijack this topic, but the content of a hotfix/private fix should at least contain a short readme for which problem/casenumber it is. I also know now that this problem (Trying to offload the storage that is locked by the source job. Error code: 0x00000020' ) is not fixed in V11 and we have to wait for 11a.
spiritie
Service Provider
Posts: 189
Liked: 40 times
Joined: Mar 01, 2016 10:16 am
Full Name: Gert
Location: Denmark
Contact:

Re: .vib files are left behind

Post by spiritie »

Hi All

I can attest to a case where VBR v11 didn't clean correctly up

On 5-3-2021 9/68 VM's in a particular backup job failed with:
Error: Exception of type 'Veeam.Backup.AgentProvider.AgentClosedException' was thrown.
The job then did a re-try and successfully backed up the remaining 9 VM's.

It left behind 9 ghost .VIB files I found with the PowerShell script.

I found the Task.<VM>.log files for that date at it is filled with the error message from above and a few different ones.

So there are still some cases where it doesn't clean up properly.
ConradGoodman
Enthusiast
Posts: 98
Liked: 5 times
Joined: Apr 21, 2020 11:45 am
Full Name: Conrad Goodman
Contact:

Re: .vib files are left behind

Post by ConradGoodman »

Thanks for this post.

I have just cleared several TB off of 4 Veeam servers.

I used the PS script from FrancWest to find the orphaned files in our standard backup repositories.

Unfortunately I cannot get the script to work for Backup Copy Jobs in our Scale-Out Repos.

I am struggling to modify the script to filter on Backup Copy Jobs only in the first line:

Code: Select all

$backup = get-vbrbackup | Where-Object { ($_.jobtype -eq "Copy")}
obviously 'Copy' is not gonna fly as a jobtype, but I can't find the right parameter anywhere.

Can anyone help with this please? I will then set these scripts up as monitoring all servers so if a file is orphaned I get an alert.
FrancWest
Veteran
Posts: 488
Liked: 93 times
Joined: Sep 17, 2017 3:20 am
Full Name: Franc
Contact:

Re: .vib files are left behind

Post by FrancWest »

This command returns all copy jobs. I’m not sure if I included all backup copy job types, but these are the ones present in our environment:

Code: Select all

get-vbrbackup | Where-Object { ($_.jobtype -eq "BackupSync") -or ($_.jobtype -eq "SimpleBackupCopyPolicy") -or ($_.jobtype -eq "SimpleBackupCopyWorker")}
You can get all backups and their type with the command:

Code: Select all

Get-vbrbackup | fl name, jobtype
However, since I received the hotfix I don’t have any redundant backup files anymore. So if you are on V10 you can ask for the hotfix (but you have to manually clean out the redundant backup files).
ConradGoodman
Enthusiast
Posts: 98
Liked: 5 times
Joined: Apr 21, 2020 11:45 am
Full Name: Conrad Goodman
Contact:

Re: .vib files are left behind

Post by ConradGoodman »

Thanks, will have a look at that code in a bit.

We are on V11 throughout, and no leftover files anywhere since the upgrade. Just pre upgrade mess.

That said, if it happened once, no harm in having some monitoring of the situation moving forwards... Having all your disks swallowed up and not knowing why is not good.
FrancWest
Veteran
Posts: 488
Liked: 93 times
Joined: Sep 17, 2017 3:20 am
Full Name: Franc
Contact:

Re: .vib files are left behind

Post by FrancWest »

I still don’t get it why Veeam is not warning customers about this issue. As far as I know there isn’t even a KB article about this issue. We are talking about possibly terabytes of redundant files that are left behind on repositories and occupying precious disk space.
ConradGoodman
Enthusiast
Posts: 98
Liked: 5 times
Joined: Apr 21, 2020 11:45 am
Full Name: Conrad Goodman
Contact:

Re: .vib files are left behind

Post by ConradGoodman » 1 person likes this post

Have to agree mate, that is disappointing. Thanks to the title of this thread, I found it on my first Google.

I dare say there are a bunch of Veeam customers who are buying disks, or changing their Scale-Out performance tier to capacity tier offload settings to compensate.

Certainly in my case, across 4 servers we are talking 10's of TBs of lost space.
Post Reply

Who is online

Users browsing this forum: dbeerts, Google [Bot], jackyku and 115 guests