Comprehensive data protection for all workloads
Locked
jai64
Influencer
Posts: 23
Liked: 2 times
Joined: Jun 20, 2009 10:43 am
Full Name: Joe Iovinelli
Contact:

Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by jai64 »

Has anyone had any problem upgrading to v4?

I upgraded last night and only 1 of 5 backup jobs ran. The rest hung for hours @ 0%.

I restarted and rebooting the server. Then tried to run again (and new jobs), but now the are all still stuck at 0%.

It says "Repairing Backup from Previous Rollback". It is also eating space like crazy and leaving .update files.

Called support at 4:30AM. Still no word...or help.

~joe
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Backup 4.0 Upgrade Issues.

Post by Gostev »

Joe, Veeam Backup 4.0 has completely new backup storage format (transactional storage) and the upgrade procedure of v3 storage is very complex. Storage upgrade happens on the first launch of the specific job after the product upgrade (this is why you see the job "hanging" at 0%). Based on our earlier investigation of couple of similar support cases, storage upgrade may be unable to update certain larger v3 backup files to v4 format.

If possible, I recommend that you create the new jobs and delete existing v3 ones for which upgrade failed, this is an immediate workaround. We are already testing fix for this issue, and support will contact you as soon as the fix will be available.

Sorry for this inconvenience, upgrade is always most complex thing to test :(
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Backup 4.0 Upgrade Issues < READ IF UPGRADING FROM v3

Post by Gostev »

Thread stickied.

For future readers: this is one-time upgrade issue does not affect all backups, but only some. So, if your job are running fine after upgrade, you do not need any fixes or anything.

UPDATE - IMPORTANT - PLEASE READ THIS POST CAREFULLY
The download package has been updated on http://www.veeam.com at 10pm GMT Oct 29
Current package does not have this issue.

If you have run into the above mentioned upgrade problem with the original RTM package, follow this procedure:
1. Uninstall the existing version.
2. Download and install updated package (the new package has "_u" in its name), pointing it to the same configuration database.
3. Rename or copy away the .upgrade files next to VBK files for all failed jobs.
4. Run all jobs, first run should now complete successfully.
5. If all is well, delete .upgrade files preserved on step 3.

If you have replication jobs affected by installing the original RTM package, please contact support for assistance.

If you upgraded using the original RTM package, and first run for each job after upgrade completed successfully (which is more common situation), or if you decided to create "fresh" jobs instead - you do not need to download and reinstall the new package. It has all the same components as the original package, except one upgrade-related component.
jai64
Influencer
Posts: 23
Liked: 2 times
Joined: Jun 20, 2009 10:43 am
Full Name: Joe Iovinelli
Contact:

Re: Veeam Backup 4.0 Upgrade Issues.

Post by jai64 »

That puts me in a bit of a pickle.

I have 5TB of backups for my 70 servers and a bunch of complex jobs setup. I wish I would have read about the conversion in the Upgrade instructions???? It was only a few lines and seemed like a minor patch. It said nothing about a backup conversion processes?

I only have enough space to do new backups for a few days. I will have to delete all my old backup files, because it will not properly prune the old stuff out.

And yes my backup files are large? I suspect most peoples are. I have a lot of big servers.

How long until a fix?

~ Not happy
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Backup 4.0 Upgrade Issues < READ IF UPGRADING FROM v3

Post by Gostev »

The fix will be available later today.
toreagan
Novice
Posts: 5
Liked: never
Joined: Apr 13, 2009 9:26 pm
Full Name: to
Contact:

Re: Veeam Backup 4.0 Upgrade Issues < READ IF UPGRADING FROM v3

Post by toreagan »

I have the same issue. Half of my jobs are stuck at 0% and I've tried "stopping" the job(s), but no luck... they are still hung. How can I successfully stop the job(s) without having to reboot the server? where will the patch be available from... new full download?

Please Advise... Thanks,
~Trent
jai64
Influencer
Posts: 23
Liked: 2 times
Joined: Jun 20, 2009 10:43 am
Full Name: Joe Iovinelli
Contact:

Re: Veeam Backup 4.0 Upgrade Issues < READ IF UPGRADING FROM v3

Post by jai64 »

Does that also mean that until this conversion is completed on a job, you can also not do a restore from that job?

~ joe
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Backup 4.0 Upgrade Issues < READ IF UPGRADING FROM v3

Post by Gostev »

Patch is already available and verified with one of the customers over webex.
Support will contact you shortly with the patch (2 files) and instructions on applying the fix.

Joe, you can restore from backup even if storage update procedure failed, no problems there. You just cannot make new backups on v3 jobs, until the fix is applied.
BarryCoombs
Enthusiast
Posts: 30
Liked: never
Joined: May 22, 2009 2:46 pm
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by BarryCoombs »

Is this patch going to be made available to everyone, or rolled up into a new release, just thinking about the planned customer upgrades I have over the next few weeks. Would prefer to not need to log a support call if I experience this.
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by Gostev »

Barry, we will update the package on our web-site tonight. Anyone who will download the product after this will not run into the issue. But overall, this issue does not affect everyone who is upgrading, it is quite rare: so far just 3 support cases for a few hundreds of successful upgrades.

I will update this thread once the new package is available for download.
BarryCoombs
Enthusiast
Posts: 30
Liked: never
Joined: May 22, 2009 2:46 pm
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by BarryCoombs »

ok cheers, I will re-download tomorrow. I came across the issue on our demo Veeam box but just deleted the old backups as they weren't important anyway.
jai64
Influencer
Posts: 23
Liked: 2 times
Joined: Jun 20, 2009 10:43 am
Full Name: Joe Iovinelli
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by jai64 »

The update worked great. Thank you for the quick response.

~joe
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by Gostev »

Thank you for updating the topic :)
donikatz
Expert
Posts: 124
Liked: never
Joined: Jan 01, 2006 1:01 am
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by donikatz »

Wish I'd read this before upgrading, since I had this same problem. I have enough storage for new backups, though, so I just started fresh ones. Thanks for the update, good info. Loving 4.0 speed! :)
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by Gostev »

The download package has been updated on http://www.veeam.com

If you have run into the above mentioned problem with the previous package:
1. Uninstall the existing version.
2. Download and install updated package (the new package has "_u" in its name), pointing it to the same configuration database.
3. Rename or copy away the .upgrade files next to VBK files for all failed jobs.
4. Run all jobs, first run should now complete successfully.
5. If all is well, delete .upgrade files preserved on step 3.

If you have replication jobs affected by installing the original RTM package, please contact support for assistance.

If you upgraded using the original RTM package, and first run for each job after upgrade completed successfully (which is more common situation), or if you decided to create "fresh" jobs instead - you do not need to download and reinstall the new package. It has all the same components as the original package, except one upgrade-related component.

Thanks Doni :D
HenrikS
Influencer
Posts: 22
Liked: never
Joined: Oct 29, 2009 2:09 pm
Full Name: Henrik Schewe
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by HenrikS »

I totally agree, this should have been mentioned in the upgrade notes.
I also ran into this problem, but since We've only been using veeam in a test environment, i just purged all old backups and reinstalled veeam and the DB.
tjestr
Enthusiast
Posts: 44
Liked: never
Joined: Mar 05, 2009 9:33 am
Full Name: Falko Dohse
Location: Hamburg
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by tjestr »

For the upgrade process, do i need the double space (for vbk files) to complete the upgrade succesfully?
HenrikS
Influencer
Posts: 22
Liked: never
Joined: Oct 29, 2009 2:09 pm
Full Name: Henrik Schewe
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by HenrikS »

If you have installed the updated package from veeam.com, it should not need the double space; no!
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by Vitaliy S. »

Falko,

Henrik is right, you don't need to have double space for the proper upgrade of the VBK files. Just use the the new package with "_u" in its name.
Felix
Enthusiast
Posts: 37
Liked: 2 times
Joined: Oct 30, 2009 2:43 am
Full Name: Felix Buenemann
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by Felix »

Argh, had I read this thread earlier, just yesterday thrashed two weeks of v3 Backups, because Veeam Backup 4.0 was trying to upgrade a 800GB VBK which exceeded the free space on the backup raid. This *really* should've been noted in the release notes.
colsztyn
Novice
Posts: 4
Liked: never
Joined: Jul 22, 2009 6:42 pm
Full Name: Christopher A. Olsztyn
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by colsztyn »

After upgrading to the original 4.0.80 and running my first test job, an existing job, it has been "hanging" at the 0 percent complete and "initializing target session". Is it anticipated that this will eventually complete or do I need to restart the server.

Also, can I get the patch to fix this issue.

Sincerely,

Chris
Felix
Enthusiast
Posts: 37
Liked: 2 times
Joined: Oct 30, 2009 2:43 am
Full Name: Felix Buenemann
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by Felix »

Read the thread, redownloading the latest version and following the steps in the thread should (supposedly) fix your problems.
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by Gostev »

Thanks Felix! Correct.
jeff@pcb.ca
Novice
Posts: 4
Liked: never
Joined: Jul 14, 2009 10:01 pm
Full Name: Jeff Bourassa
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by jeff@pcb.ca »

I have updated to the latest Backup 4.0 (just downloaded, looks to be the same build file however re-installed over the existing). I still have 2 servers that are stuck on the 0% problem. And after the update are still stuck on this. Should I leave them for the weekend? Or is there a way to force kill the .update process, clear the job and start with a new job?

I agree that some heads up on this issue would have been helpful.
Vitaliy S.
VP, Product Management
Posts: 27055
Liked: 2710 times
Joined: Mar 30, 2009 9:13 am
Full Name: Vitaliy Safarov
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by Vitaliy S. »

Hello Jeff, have you exactly followed the instructions posted by Anton here around what to do with the .upgrade files?
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by Gostev »

Upgrade shoud complete pretty fast, no need to leave the jobs hanging for the weekend, it won't help. If you follow the procedure I listed above closely, the upgrade should complete successfully. Or yes, alternatively you can always simply delete the jobs for which automatic VBK storage upgrade failed, and create the new job - if this works for you.
jeff@pcb.ca
Novice
Posts: 4
Liked: never
Joined: Jul 14, 2009 10:01 pm
Full Name: Jeff Bourassa
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by jeff@pcb.ca »

I can't stop the jobs in the console, otherwise I would and simply delete them. However I will go through the step as listed over the weekend and see if that clears things up.

Thanks
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by Gostev »

Jeff, if you need any help at all, please do not hesitate to open a support case, and they will assist you. If the job won't stop, I believe you can simply kill veeam manager and agent processes with the task manager. Or, reboot Veeam Backup server. Then, you will be able to delete the job.
jingxi02
Novice
Posts: 9
Liked: never
Joined: Apr 25, 2009 11:03 pm
Full Name: Leslie Lei
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by jingxi02 »

I ran into the same issue after I upgraded it to version 4. The job just hung on "Constructing roll back file" for over day. It won't stop or cancel. I end up have to move the old backup files to a different folder and delete the backups in the Veeam console and start the new Full back on the same job. If I need anything from the old backup, I just import them again.

Anyway, Veeam 4 works much better and solved alot of old issues. Some of my job was running only 15MB/s. The same job is running 120MB/s with Veeam 4. Huge improvement. Thanks alot. :P :P :P :P :mrgreen: :mrgreen: :lol: :lol:
Gostev
Chief Product Officer
Posts: 31460
Liked: 6648 times
Joined: Jan 01, 2006 1:01 am
Location: Baar, Switzerland
Contact:

Re: Veeam Backup 4.0 Upgrade Issue < READ IF UPGRADING FROM v3

Post by Gostev »

jingxi02 wrote:I end up have to move the old backup files to a different folder and delete the backups in the Veeam console and start the new Full back on the same job. If I need anything from the old backup, I just import them again.
Yep, this is exactly what I would do too. Despite major storage changes, Veeam Backup 4.0 can still import backups made by any previous version of Veeam Backup. Glad you are enjoying the new version! :)
Locked

Who is online

Users browsing this forum: Amazon [Bot], Google [Bot] and 183 guests