Comprehensive data protection for all workloads
Post Reply
B.F.
Expert
Posts: 160
Liked: 9 times
Joined: Jan 28, 2014 5:41 pm
Contact:

Failed to repair VM '<VMNAME>'

Post by B.F. »

For reason's I'm not sure yet, we have a backup to disk that failed:
Full backup file merge failed Error: Insufficient system resources exist to complete the requested service. Failed to write data to the file [<FILEPATH .vbk>]

Not sure what resources were insufficient since there's still boat loads of space available on the disk.

Now when I try to run or retry the backup:
Error: Failed to repair VM '<VMName>' in backup '5/24/2016 1:05:43 PM'. Unknown algorithm 'TransformForeverIncremental'

Any suggestions on how to repair this?

Thanks
PTide
Product Manager
Posts: 6408
Liked: 724 times
Joined: May 19, 2015 1:46 pm
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by PTide »

Hi,

Please contact our support team so they can investigate the error and post your case ID here.

Thanks
B.F.
Expert
Posts: 160
Liked: 9 times
Joined: Jan 28, 2014 5:41 pm
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by B.F. »

Ticket #01825333 opened.
electricd7
Expert
Posts: 121
Liked: 7 times
Joined: Mar 27, 2012 10:13 pm
Full Name: Chad Killion
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by electricd7 »

We are seeing this exact same behavior. Are you by chance backing up to a scale-out repository? We didn't start seeing these until we started using the new scale-out repositories. I have had a case open (#01813976) since 5/31 and still no resolution. At first they pointed to my repository as we were using 2012 storage spaces to span disks into a large volume to use as the repository. They had me change to just a single large disk (we are using VMware so we just created a large VMDK) and started a new chain at that new repository (a member of a new scale-out repository as the only member.) We just crossed the retention period where a merge needed to happen and have the same error again on the new storage!
B.F.
Expert
Posts: 160
Liked: 9 times
Joined: Jan 28, 2014 5:41 pm
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by B.F. »

Our repository is quite basic and had been used for a while now. The error only happened once and ended up needing to start a new chain of backups as well as added some more system memory to the backup server. So far so good.
ccatlett1984
Enthusiast
Posts: 83
Liked: 9 times
Joined: Oct 31, 2013 5:11 pm
Full Name: Chris Catlett
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by ccatlett1984 »

I just upgraded last week (from v8) and am seeing the same thing on a few jobs, but not all.

I have opened case # 01880151
prabhash.jena
Enthusiast
Posts: 35
Liked: 2 times
Joined: Jan 16, 2015 1:32 pm
Full Name: Prabhash Kumar Jena
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by prabhash.jena »

B.F. wrote:Our repository is quite basic and had been used for a while now. The error only happened once and ended up needing to start a new chain of backups as well as added some more system memory to the backup server. So far so good.
Hi,

Were Veeam able to fix this or you ended up creating new job?

Regards
Prabhash
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by veremin »

A new backup chain was started. Most likely, a new active full backup was created. Besides that, system resources available for a backup server were increased. Thanks.
prabhash.jena
Enthusiast
Posts: 35
Liked: 2 times
Joined: Jan 16, 2015 1:32 pm
Full Name: Prabhash Kumar Jena
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by prabhash.jena »

Hi,

I applied some fixes such as removing backup from configuration, rescan of scale out repository, run an active full, exclude the VM in question from backup job, perform a quick backup of the VM in question reboot of repository server but nothing worked. I created new job and am keeping the data from old job for restoration. I have plenty of resources on the servers so I am sure that's not the issue.

This is not a pleasant experience, 1 VM backup file corruption leading to corruption of whole job. I believe there can be a database fix for this.

Regards
Prabhash
veremin
Product Manager
Posts: 20270
Liked: 2252 times
Joined: Oct 26, 2012 3:28 pm
Full Name: Vladimir Eremin
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by veremin »

I believe there can be a database fix for this.
And have you tried to reach our support team to check the availability of the said fix? Thanks.
dpatel
Influencer
Posts: 13
Liked: 2 times
Joined: Nov 14, 2016 10:35 am
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by dpatel »

Hi,

We are facing same problem on our one of the backup job. We are using Veeam 9.0.0.1715 with scale-out repository.

On same repository other backups are working fine without any error.

Thanks,
Daxesh
foggy
Veeam Software
Posts: 21069
Liked: 2115 times
Joined: Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by foggy »

Please contact technical support for troubleshooting. Thanks.
Steen
Service Provider
Posts: 51
Liked: 3 times
Joined: Mar 13, 2015 1:20 pm
Full Name: Steen
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by Steen »

I have same problem on several job.
Tha backup-chain is destroyed and forced to make new backup-job.
Sadly this seeams to be an widely spread problem and no fix
Regards Steen
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by Gostev »

The fix is to add some RAM to a repository server (or whatever server runs the target data mover for the corresponding backup repository).
Steen
Service Provider
Posts: 51
Liked: 3 times
Joined: Mar 13, 2015 1:20 pm
Full Name: Steen
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by Steen »

Hi.
How much memory?
We have 32GB on B&R and 32 GBV on all ours repositories so i dont think that is the problem.
Regards Steen
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by Gostev »

Depends primarily on the number of jobs you are running concurrently.
Steen
Service Provider
Posts: 51
Liked: 3 times
Joined: Mar 13, 2015 1:20 pm
Full Name: Steen
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by Steen »

1 job with 1 vm. Still fails
Regards Steen
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by Gostev »

Once the issue has been introduced, it will fail regardless. I am talking about the time when the issue was introduced - in other words, how many jobs are running concurrently normally in your environment? Earlier you mentioned you have "multiple jobs".
Steen
Service Provider
Posts: 51
Liked: 3 times
Joined: Mar 13, 2015 1:20 pm
Full Name: Steen
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by Steen »

We have 1 B&R and 4 repositories
Maximum amount of of concurent jobs is 24 (one concurrent snapshot/CSV)
Regards Steen
Gostev
Chief Product Officer
Posts: 31457
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by Gostev »

So assuming totally even spread, that is 6 jobs per repository... but if spread is not even, then even more. Depending on the job size, you may indeed be running out of memory on backup repository, because you need 6-8 GB RAM per concurrent job according to our System Requirements.
Steen
Service Provider
Posts: 51
Liked: 3 times
Joined: Mar 13, 2015 1:20 pm
Full Name: Steen
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by Steen »

Yes. Its extrem high requiremnts.
altough if i look right now the usage of memory 8incl OS) is on repositories is about
1.6 GB
3.6 GB
4.6 GB
4.4 Gb

running 8 jobs

( i thought it was 4 GB / job )
Regards Steen
tsightler
VP, Product Management
Posts: 6009
Liked: 2843 times
Joined: Jun 05, 2009 12:57 pm
Full Name: Tom Sightler
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by tsightler »

What OS version is your repository?
Steen
Service Provider
Posts: 51
Liked: 3 times
Joined: Mar 13, 2015 1:20 pm
Full Name: Steen
Contact:

Re: Failed to repair VM '<VMNAME>'

Post by Steen »

2012 R2
Regards Steen
Post Reply

Who is online

Users browsing this forum: Google [Bot], Ivan239, RValensise, tyler.jurgens, ybarrap2003 and 173 guests