VSS timeout with Exchange 2010

VMware specific discussions

Re: VSS timeout with Exchange 2010

Veeam Logoby cliffm » Mon Aug 22, 2011 9:13 pm

VC is VMware vCentre server. VC is some jargon that Veeam support started using but yes, my apologies, it probably did not make sense. It took me a long time to realise that this had something to do with the issue as I was fully focused for weeks on VSS being the problem. But as soon as I turned the VC off my Veeam jobs started working. As soon as I turn it on the Veeam jobs stop working. Veeam has been OK backing up everything else except a Microsoft SBS. For the SBS I need the VC turned off or to run the backups on a different network without a VC. The SBS is the only Microsoft server we have, everything else is Linux. So I can't comment on any other flavours or versions of MS Server. Veeam has however backed up various XP machines using VSS with no issue; for whatever reason it is just the SBS that has caused me grief so I can understand the focus on VSS. I don't know if the VC is the root of the issue but at this stage it is the only reliable reproducible thing I have found that gets me a working backup every time. Something somewhere in the complexity is not communicating in the Veeam/ESXi/VC/VMware tools mix, I don't have the resources to test all the combinations. But; at least for my case I have got something that works; which is all I need for now.
cliffm
Enthusiast
 
Posts: 37
Liked: 3 times
Joined: Fri Jun 03, 2011 12:41 am
Full Name: Cliff Meakin

Re: VSS timeout with Exchange 2010

Veeam Logoby tsightler » Mon Aug 22, 2011 10:31 pm

When you are using VC to perform backups are you sure that the account that you used to add the VC to Veeam has full administrative permissions on the VM?

The reason I'm asking is that I've seen at least a case or two where the user tried to be more restrictive with the permissions that just giving "full access" to the Veeam account to vCenter. This mostly worked, except in cases where the Veeam server was firewalled or otherwise unable to access the Windows VM it is trying to back up. In this scenario Veeam cannot activate the VSS runtime service via the administrative share and must access the host via the VMware Tools interface, however, the account being used by the Veeam server doesn't have permissions to perform this.

When running the job via the ESX host directly it's typically running with an account that has full admin privileges on the ESX host and then it works fine.
tsightler
Veeam Software
 
Posts: 4768
Liked: 1737 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: VSS timeout with Exchange 2010

Veeam Logoby J1mbo » Tue Aug 23, 2011 8:22 am

Surely the problem here is that Veeam can't confirm what's happened with vCentre due to what is effectively a race between the VSS quiesce request and the vCentre need to update the database with what is going on. So it would be expected behaviour (for me anyway).

I'd suggest running vCentre in it's own VM anyway (if you update to vSphere 5 there will be the vCentre appliance option, to save on licensing another Microsoft instance, at the expense of the vRAM nonsence).
J1mbo
Expert
 
Posts: 261
Liked: 29 times
Joined: Tue May 03, 2011 12:51 pm
Full Name: James Pearce

Re: VSS timeout with Exchange 2010

Veeam Logoby jbrullemans » Tue Aug 23, 2011 9:48 am

We also had this problem in the beginning of May this year. Contacted MS contacted EMC contacted Veeam. No one new what the problem was and where the time out came from. On one i day i increased the memory of VC and all the problems dissapeard.
jbrullemans
Enthusiast
 
Posts: 38
Liked: 1 time
Joined: Wed Mar 09, 2011 8:22 am
Full Name: John Brullemans

Re: VSS timeout with Exchange 2010

Veeam Logoby tsightler » Tue Aug 23, 2011 12:59 pm

J1mbo wrote:Surely the problem here is that Veeam can't confirm what's happened with vCentre due to what is effectively a race between the VSS quiesce request and the vCentre need to update the database with what is going on. So it would be expected behaviour (for me anyway).

I'd suggest running vCentre in it's own VM anyway (if you update to vSphere 5 there will be the vCentre appliance option, to save on licensing another Microsoft instance, at the expense of the vRAM nonsence).


That's a good point, and certainly possible, but I didn't understand that from his posts. If vCenter itself is installed on, or uses the SQL database on the SBS server with which he's having a problem then you're right, that's probably the issue, and makes perfect sense.
tsightler
Veeam Software
 
Posts: 4768
Liked: 1737 times
Joined: Fri Jun 05, 2009 12:57 pm
Full Name: Tom Sightler

Re: VSS timeout with Exchange 2010

Veeam Logoby cliffm » Tue Aug 23, 2011 8:49 pm

I had the VC installed on a VM that was on a brand new IBM server, using approx 1/3 of all the server resources. I also had Veeam server installed (64 bit) on the same VM as the VC. The SBS is on the same host but doers not have anything other that VMware Tools installed in it. It was not until I tried Veeam server (32 bit) on another network that I started to see any success with this issue. Eventually I tried turning the VC off completely and the Veeam jobs started working on my original production network. As for admin access, support also asked me about this. Occasionally Veeam jobs would succeed and all writers were stable after a successful backup. So I don't think it is a permissions thing because nothing has changed permissions wise from success to failure or vice versa.

I bought Veeam to backup this pesky MS SBS. It has turned out to be quite a path of discovery! However for me now the VC is turned off because a backup is more important to me than a VC. This I can live with until there is some clarity about what the actual issue is here, and of course a fix!
cliffm
Enthusiast
 
Posts: 37
Liked: 3 times
Joined: Fri Jun 03, 2011 12:41 am
Full Name: Cliff Meakin

Re: VSS timeout with Exchange 2010

Veeam Logoby gerdawg » Wed Aug 24, 2011 1:52 am

I have the same issue after setting up a DAG between my two exchange 2010 SP1 servers.

Does this link seem to be the culprit?
________________________________________________________________________________________________________________________________
Understanding Backup, Restore and Disaster Recovery

http://technet.microsoft.com/en-us/libr ... 76874.aspx

Be aware of these limitations when using VSS for backup and restore of Exchange data:

The VSS plug-in that ships with Exchange 2010 can be used to back up volumes containing active mailbox database copies or standalone (non-replicated) mailbox databases only. It can't be used to back up volumes containing passive mailbox database copies. To back up passive mailbox database copies, you need either Microsoft System Center Data Protection Manager or a third-party Exchange-aware VSS-based application.

Passive mailbox database copies are backed up using a separate VSS writer in the Microsoft Exchange Replication service. The Microsoft Exchange Replication service VSS Writer doesn't support restores. Although you can back up a passive mailbox database copy using Microsoft System Center Data Protection Manager or a third-party Exchange-aware VSS-based application, you can't perform a VSS restore directly to a passive mailbox database copy. However, you can perform a VSS restore to an alternate location, suspend replication to the passive copy, and then copy the database and log files from the alternate location to the location of the passive database copy in the file system.
_________________________________________________________________________________
I am not sure how the mailbox is being called by veaam and exactly what Exchange writers it uses but at least for me I've narrowed down having two active/active databases on one server and not having a problem. It seem to me that it's also plausible as right now I am in the middle of moving many of our users to different mailbox stores based on location. My one mailbox store is over a 10mb link and DAG between the two databases (and the move to that server's active database) takes place over that link. I was wondering if it's also possible that simply seeding the DAG database requires the VSS writer to be in-use and this causes the issue with Veaam/Microsoft's VSS writer when another application tries to gain access to perform the backup for Veeam. What concerns me most is the fact that the passive database from my other mail server resides on the same server that at times backs up and other times does not and I am wondering if the tech net article above is the reason for this.

Usually after a few tries or possibly letting the database sit idle and running the job again later that night fixes the issue. What I do notice is when that happens my traffic between sites are lower - which in turn means the replication of DAG between them is also lower.

For the record Veeam is setup to backup the server from the same site and uses 1 gbps link to do so for the exchange server that is on the same switch.
gerdawg
Influencer
 
Posts: 19
Liked: never
Joined: Wed Jun 22, 2011 3:15 am
Full Name: Gerry D

Re: VSS timeout with Exchange 2010

Veeam Logoby gerdawg » Thu Aug 25, 2011 3:08 am

This may be a silly question but should i be backing up BOTH of my exchange servers in a DAG configuration? It still seems if one backup is going on one server that it is frozen on the other server.

Should I just be copying the active/passive database on the one server and NOT backing up my other server that has an active/passive on it (the inverse of Server 1).

If that is the case, can you easily restore a passive database copy? All in all a passive copy should hold the exact same information that an active copy has regardless of location. Is there a reason to backup my secondary exchange server other than for configuration purposes?

I read the Microsoft best practices guide for backing up DAG servers but I didn't see anything in regards to this that made sense to me, I have read elsewhere by Microsoft that it is acceptable to backup the passive database rather than an active but I wanted to see what everyone else thoughts were on this first. Other types of backup products (BE) give you the option to backup an "active" copy only, which I am guessing is fine but to take it one step further...let's say I have a connectivity issue one night and my primary exchange server swaps the database to passive for both stores and puts an active copy of both DB"s on server 2 which isn't backed up by Veeam. Would a passive copy of BOTH databases by Veeam for Server 1 still be recoverable using AIR?

I just want to make sure I'm doing the right thing, sometimes Microsoft says on thing but you guys have the experience when it comes to backing up a Virtual Exchange server running DAG so any help is appreciated.

Thanks!
gerdawg
Influencer
 
Posts: 19
Liked: never
Joined: Wed Jun 22, 2011 3:15 am
Full Name: Gerry D

Re: VSS timeout with Exchange 2010

Veeam Logoby gerdawg » Thu Aug 25, 2011 11:11 pm

Would anyone be able to take a stab at the question above? Is it necessary to backup an exchange DAG server from both servers or would backing up a passive database from one of the servers be sufficient?
gerdawg
Influencer
 
Posts: 19
Liked: never
Joined: Wed Jun 22, 2011 3:15 am
Full Name: Gerry D

Re: VSS timeout with Exchange 2010

Veeam Logoby foggy » Fri Aug 26, 2011 8:49 am

Gerry, there are a few existing topics on this that might be helpful to you, for example this one (particularly, the last page).
Thanks.
foggy
Veeam Software
 
Posts: 14742
Liked: 1079 times
Joined: Mon Jul 11, 2011 10:22 am
Full Name: Alexander Fogelson

Re: VSS timeout with Exchange 2010

Veeam Logoby gerdawg » Fri Aug 26, 2011 3:32 pm

Foggy,

Thank you for the reply. I've looked through the forums before posting the above question and cannot find an answer from someone @ veeam that will tell me if it's an OK process or not just to backup one member of the DAG cluster or not.

(viewtopic.php?f=2&t=5586&p=31803&hilit=dag+backup#p31803 and viewtopic.php?f=2&t=5586&p=31803&hilit=dag+backup#p31803
)

To be clear, my question is - Can I only backup one member of a DAG cluster regardless if it holds and active/passive - passive/passive - active/active database configuration using Veeam or do I have to backup both of my server in my DAG configuration. Microsoft recommends that you only backup the passive configuration per my reading, however I can't see how you can control that in the event of a failover.

What's Veeam's stance on backing up a DAG database? - Am I supported for a restore of any database on my DAG cluster regardless if it was backed up as Passive/Active if I'm only backing up one server in my DAG configuration. It seems like it would be a waste to have to backup both of my DAG servers if they are supposed to contain the same information.
gerdawg
Influencer
 
Posts: 19
Liked: never
Joined: Wed Jun 22, 2011 3:15 am
Full Name: Gerry D

Re: VSS timeout with Exchange 2010

Veeam Logoby Gostev » Fri Aug 26, 2011 3:48 pm

Veeam does not have official stance on backing up a DAG database, so it is up to customer what to backup. AFAIK, most customers backup both in a single job, and because of dedupe this does not really affects backup size, but you have both VMs backed up.

Kindly please do not further derail this topic with DAG-specific questions, as OP is talking about different issue, not connected with DAG. There are existing topics discussing DAG specific that you can continue posting into. I will clean this topic up a bit later to remove all offtopic. Thanks.
Gostev
Veeam Software
 
Posts: 21390
Liked: 2349 times
Joined: Sun Jan 01, 2006 1:01 am
Location: Baar, Switzerland

Re: VSS timeout with Exchange 2010

Veeam Logoby gerdawg » Fri Aug 26, 2011 7:29 pm

Gostev,

Thank you for the reply. That said, I'll disagree that the topic entirely a derail from the OP's issue and that mostly my fault as I did not specify that I am having a freezing issue as well when Veeam is trying to backup our exchange servers, so I apologize.

I am having the same issue as the OP with the guest freezing. However my issues are FURTHER complicated by using two separate exchange servers, which I would believe happen irrespective of DAG.

I have searched the forums looking for some information on this configuration and my issue of the DB writer freezing and this is something Veeam claims "is supported" however I have yet to find a post that provides insight on how to make that happen reliably. My main concern and the reason I posted in the thread is that I'm looking for information on how to stop the unreliable freezing issues of our VSS writer when trying to backup our Exchange server. Any insight you can provide there further would be appreciated - or if this could be moved into a more specific topic that would provide better understanding of the freezing issue along with DAG that might be more helpful.

Thanks!
gerdawg
Influencer
 
Posts: 19
Liked: never
Joined: Wed Jun 22, 2011 3:15 am
Full Name: Gerry D

Re: VSS timeout with Exchange 2010

Veeam Logoby Bunce » Sat Aug 27, 2011 12:16 pm

The freezing issue has been done to death and workarounds posted.Search my posts for the FCS modification commands.

Its not up to veeam to tell you what to backup.. Thats your job to work out based on your particular needs, environment, and XGE administration knowledge.

The link Anton provided gives examples of what customers (including myself) have chosen to backup in various DAG scenarios, so use them, or don't use them, as a guide.
Bunce
Expert
 
Posts: 254
Liked: 8 times
Joined: Fri Sep 18, 2009 9:56 am
Location: Adelaide, Australia
Full Name: Andrew

Re: VSS timeout with Exchange 2010

Veeam Logoby gerdawg » Sat Aug 27, 2011 2:14 pm

Bunce,

I saw your FCS mod commands and unfortunately they don't help me at all in my situation. What has worked however at this point is not backing up my secondary server or only backing it up after the first Veeam server is finished. For reference I have figured out what my issue is and my work around works for me just fine to backup what I need.

That said, I'm not asking Veeam to tell me what to backup. I am asking them what they believe the best practice is for using the product that they have developed. This shouldn't turn into a pissing match as you can go to any other product vendor and ask this very question and they already have pre-canned information to guide you in a direction of what they recommend for their product. Perhaps asking in these forums is the issue, and i should call support for my issue and guidance. Based on other posts I have read it seems like whenever the question is asked of what's "preferred" to backup with Veeam for certain technologies the answer is always, whatever it's your choice. Yes, clearly, it's my choice and my responsibility of what I want to backup. However you should keep in mind, the Veeam product, regardless of the fact the Microsoft's VSS writers are being use, ONLY backs up disks which complicates what "best practice" can and will be for a customer that can no longer choose to backup EXACTLY what they WANT - such as backing up active vs passive databases residing on a single server. Perhaps I'm off base by asking in this post which was resurrected fresh by another individual and concurred with an issue as I was having, and since there isn't an official company answer, fine, but this shouldn't be a difficult question that a customer is asking nor should it be a problem that I am asking it.

That said, Veeam forums have been very informative and I'm appreciative of the advice I've been given so far from all of the board members, I was just looking to find out what the official stance was based on a problem I was having with my freezing of the VSS, and I'm clear based on the previous post that Veeam does not have an official stance on what they deem should be backed up for Exchange (DAG). I think my specific issue likely stems from the fact that I have 2 separate Veeam Servers (so two different jobs) backing up DAG servers at two different sites and I have yet to find a post that deals with my specific issue.

As Gostev has already suggested - if this post is off topic for this particular thread and should likely be moved to another area with close to the same issue, but is a DAG thread, please feel free to merge and/or move it there.
gerdawg
Influencer
 
Posts: 19
Liked: never
Joined: Wed Jun 22, 2011 3:15 am
Full Name: Gerry D

PreviousNext

Return to VMware vSphere



Who is online

Users browsing this forum: Google Feedfetcher and 13 guests