Availability for the Always-On Enterprise
mitchellm3
Novice
Posts: 5
Liked: 2 times
Joined: Apr 12, 2016 8:08 pm
Contact:

Re: 9.5. Update 1. Feedback

Post by mitchellm3 » Jan 27, 2017 7:15 pm 1 person likes this post

We are a pretty large Veeam shop. We backup over 1500+ VMs per night. We upgraded from 9.0 update2 to 9.5 update 1 two nights ago. The upgrade went well. We noticed that in the morning we have many jobs that are just stuck waiting on backup infrastructure resource availability. We then have to kill the jobs and re-run them. When backups kick off in the afternoon they move real well but after a few hours the jobs start to get stuck.

We opened a call with support case#02050995 and they alluded to the fact that we aren't the only customer seeing this issue. We didn't get many details so far since it is going to development but I believe the problem may have to do with the fact that we use storage array integrated snapshots. We do this across many 3par arrays but they didn't mention that it was 3par specific. We do have another site that does 200+ VMs on a 3par and it works great but we didn't have array based snapshots turned on. We are turning that on this evening to see if it has the same issues.

The good news is that our backup issues finally don't seem to point to our StoreOnce 6600's.

Protey
Novice
Posts: 5
Liked: 1 time
Joined: Apr 29, 2016 6:00 am
Full Name: Anatolii
Contact:

Re: 9.5. Update 1. Feedback

Post by Protey » Jan 30, 2017 5:52 am

On two servers updates with problems, do not run for the job schedule. Changed start time on each job and everything worked.
And another problem, after installing Windows Update KB 3205404 ( Windows Server 2012 R2 Microsoft .NET Framework Updates for 3.5, 4.5.2, 4.6/4.6.1, 4.6.2 (KB3205404) ) services Veeam backup not start. Rollback this update and all run.

enzoparone
Lurker
Posts: 1
Liked: never
Joined: Oct 13, 2016 8:39 pm
Full Name: Enzo Parone
Contact:

Re: 9.5. Update 1. Feedback

Post by enzoparone » Jan 30, 2017 8:08 am

I have installed Update 1 without problem :D

m_zolkin
Veeam Software
Posts: 23
Liked: 11 times
Joined: Aug 26, 2009 1:13 pm
Full Name: Mikhail Zolkin
Contact:

Re: 9.5. Update 1. Feedback

Post by m_zolkin » Jan 30, 2017 9:50 am

mitchellm3 wrote: We opened a call with support case#02050995 and they alluded to the fact that we aren't the only customer seeing this issue.
We have assigned a senior engineer to the ticket #02050995. It doesn't really look like a common issue. Stay tuned.
VP, WW Customer Technical Support

radkins22
Lurker
Posts: 2
Liked: never
Joined: Nov 08, 2016 2:20 pm
Full Name: Ronald Adkins
Contact:

Re: 9.5. Update 1. Feedback

Post by radkins22 » Jan 30, 2017 4:35 pm

I upgraded from 9.5 to 9.5 update 1. The upgrade appeared to go well without any errors, but when I checked my backups the next morning I had several failures. I tracked the issue down to the one physical proxy server I have. (For reference I also have 26 virtual proxy servers.) This physical proxy normally backs up several servers using direct san access, but it also backs up several servers across the network using the network transport mode. After the upgrade it continued backing up the san based servers fine, but when it came to the servers it normally backs up in network mode it was not failing over to network mode. It just sat there until the job timed out and failed. I tried stopping/starting the Veeam services then rebooting, but it still would not fail over to network mode. I had to manually remove the Veeam components, completely remove it from the Veeam environment then re-add it before I could get it to work properly. Everything has been working fine since.

BLWL
Influencer
Posts: 17
Liked: 4 times
Joined: Jan 27, 2015 7:24 am
Full Name: Bjorn L
Contact:

Re: 9.5. Update 1. Feedback

Post by BLWL » Jan 31, 2017 11:54 am 1 person likes this post

Installed Update 1 yesterday.

Install went smooth, but had to reboot server before installing update and manually starting services.

All 20 jobs successful from last night (had

Using storage integration with NetApp.

GJ Veeam.

mitchellm3
Novice
Posts: 5
Liked: 2 times
Joined: Apr 12, 2016 8:08 pm
Contact:

Re: 9.5. Update 1. Feedback

Post by mitchellm3 » Feb 20, 2017 1:56 am

After working with VEEAM support for several weeks we think we are finally getting out of the woods. Support created 3 new DLLs for us so the manager could do more and handle all our array snapshots. Backups are rolling along and we don't see jobs waiting on resources even though there are no jobs running on the particular proxy or the storage that job uses.

Another problem that was uncovered...once things started moving along...is we have some proxy servers with repositories that point to the same Catalyst store. When we first designed our solution we did this because we wanted the best deduplication. However, we have since moved away from that thinking...now giving proxy servers their own catalyst stores because dedupe is really good. Anyway, we have a bunch of jobs that get weird storage errors because VEEAM doesn't like multiple repositories pointing to the same catalyst store. This apparently changed with 9.5 but they said they will be fixing that in the next release.

Post Reply

Who is online

Users browsing this forum: Google [Bot] and 20 guests