VSS Freeze operation for SQL Server

[ian0x0r]

This is a slightly weird one, and I cant see anything addressing this issue specifically in the forums already.

I am backing up a SQL 2005 SP3 server running on 2003 SP2 and invoking the Guest level image processing. I get an error stating
Freezing guest operating system
VSSControl: Failed to freeze guest, wait timeout

If I check the veeam job logs, I can see it does freeze the I/O for 4 seconds, If i check the sql server event logs I can see that I/O to the databases is also frozen (which takes 4 seconds) and then succesfully backed up. VSSADMIN list writer and providers does not show any errors.

If I modify the job to ignore application processing errors, the job will complete with a warning, but it looks like it has done the VSS piece within the guest operating system succesfully.

Does anyone have any suggestions as to what maybe going on here?

Thanks

Ian

[Gostev]

Hi, is this the only VM affected with this issue? What applications are you running on this VM? What applications are using this SQL database?

[ian0x0r]

Hi Anton,

The sql server is hosting SQL services for Citrix, an old BES installation which is not in use, a helpdesk system and a seperate SQL instance for vCentre and Update Manager.

Have these caused any issues before? I checked for any installations of backup exec agent as per suggestion in another post, but there is no agent present.

Thanks

Ian

[Gostev]

Hi Ian, I think vCenter SQL instance presence may potentially be causing the issue, as with SQL server frozen, vCenter may not be able to perform some operation requiring database record update. Thanks.

[ian0x0r]

That does kinda make sense thinking about it. A lot of the none vCentre related databases were recently migrated from a different SQL server which did backup without any issue.

Any ideas on a work around for this other than setting ignore application processing errors? It would be interesting to know if other people are having a similair issue backing up a SQL server hosting vCentre database.

Thanks again

Ian

[kkoehn]

I've got the same issue with an ESXi VM running Windows Server 2003 and SQL 2000 (don't laugh). We periodically have very slow backup performance (2MB/sec) and oftentimes get the time out error with VSS. Thought maybe the virtual disk on our SAN (EMC Clariion AX4-5i) that this VM was stored on was too busy but moving it to another virtual disk didn't improve performance... Ironically this happens on our Exchange server, too... both VMs have been having troubles with 95% hung snapshot issues as well...

Bad VM? Not sure...

[MattG]

I am having VSS freeze issues backing up SQL 2008 running on 2008 R2 VM. The SQL server houses the vCenter Database.

Can we get some clarification on this? Can Veeam VSS backup a SQL server housing the vCenter database? I do not see this noted in the release notes.

-MattG

[Vitaliy S.]

Matt, could you please clarify if your vCenter Server is located on the same machine as SQL Server housing vCenter database?

[MattG]

vCenter is on a VM. vCenter stores it DB on a separate SQL 2008 VM. The SQL server VM will not complete the VSS job.

Since it is SQL I want to be able to get VSS to work. Support is telling me that I cannot VSS a SQL DB that contains the vCenter DB.

-MattG

[Vitaliy S.]

Matt,

It seems like there was a misunderstanding between you and our support team.

You can backup your vCenter Server together with SQL server hosting the database with Application-Aware Image Processing enabled. Just add a host which hosts SQL VM to your backup console via IP address and then add this VM to the backup job from this standalone host. Should work.

If you configure the job through vCenter server, the job will be timing out when VM freeze happens (because vCenter is unable to process API calls being "frozen").

Please let me know how it works for you. Thanks!

[MattG]

So I have 4 ESXi hosts in my cluster. All VMs are in the cluster. Veeam is configured to use vCenter to pull and manage the inventory.

The solution to backup vCenter SQL DB would be to remove the vCenter integration with Veeam and make all 4 ESXi hosts directly mapped to the Veeam console. Once this occurs then I would not be able to use VM and Templates folders to include/exclude and my jobs would now be per sevrer. So with 4 servers, I would need 4 jobs. This would be the only way to catch VMs that migrate between hosts. If I explicitly reference a VM, and it vMotions then the job would not back it up (I just tested this).

Unless I am missing something, this is not even a workable workaround?

-MattG

[Vitaliy S.]

Matt,

There is no need to remove anything from the backup console. You can have both vCenter Server and standalone host (added by IP address) within the same backup server. Your existing jobs will not be affected by this change. Standalone host connection will only be used for vCenter Server backup job.

[MattG]

Vitaly,

Thanks for clearing me up. I did not add it by IP which meant I couldn't add it. I have added ESX01 to a DRS Host Group to tie SQL to ESX01 and am now attempting to backup the SQL server. I will let you know how I make out.

Thanks,
-MattG

[MattG]

It worked.

-MattG

[Vitaliy S.]

Thanks for the update, glad that it worked for you.

[Oletho]

I can confirm that this workaround also solved my problem with vCenter server.

Ole Thomsen

[ian0x0r]

Forgetting that I had posted this question, just came back across it googling the error message

I can confirm the add the individual host does work, however, is there a way to use Virtual appliance mode whilst replicating from an individual host, rather than from vCenter? Would I need to add all hosts to the B&R console or is virtual appliance mode not possible in this scenario?

Thanks

Ian

[Vitaliy S.]

Hi Ian,

Yes, it will work, however you will need to deploy backup server on every host to use the Virtual Appliance mode. This is a requirement, because backup server should be able to access datastore where target VM resides in order to hot add its disks. Thanks.

[ian0x0r]

Ah Ok, thansk for clarifying this.

I think I can live with network backup mode for this one job

Ian

[Hammy]

I am having an issue with getting my SQL VM Guest to backup after I re-add Virtual centre to the Backup and replication client.

I done as suggested added the host as an IP Address and created the job and it worked a treat lovely old job. I also created a number of other jobs for the hosts I added as IP and ran them all successfully, however as soon as I add Virtual Centre back in so I can create a job which would cover it for any vmotion stuff the VSS Freeze issue re-apperars. I have gone through the properties of the SQL job and confirmed it has been added via the IP Address host and not V Centre but still won't work. If I then remove VCentre I have had to go back through properties and when I hit calculate again when the SQL Guest is selected I end up having to add it in again. This then works again.

Am I doing something wrong any help much appreciated.

H

[Vitaliy S.]

That's really strange that you have such behavior while adding vCenter Server to the console, let our support team investigate this issue more closer.

[imo]

Hi,

Try creating a new job, add the servers via the standalone ESX host ip address and run the job. you should now have a successfull backup.



\Ibrar Ashraf

[ian0x0r]

There was one thing I did for a while with my SQL server hosting vCenter DB, which was to ignore any processing errors if you are using application aware image processing. You would get a warning on the Veeam job, but looking at event log on SQL box the DB backup and truncate procedure seemed to work as expected. I did then however move to adding the host via IP to back it up without any error messages. I suppose if you have DRS you could set up host affinity for the SQL server.
Hv e you tried adding the vcenter server to the inventory first, before adding hosts by IP?

[Hammy]

Thanks for the suggestions guys I will have a look at trying them today. Been away on a couple of days hols hence slow response. I will update later with results as I need to find a way round this as not having VC in there is causing me issues with setting up Surebackup as it wants VC to create the Virtual Lab side of things.

[Gostev]

SureBackup works just fine without vCenter though. May be you should investigate your issue with support, as it is likely unrelated to missing vCenter. Thanks.

[Michael_6835]

just wanted to drop a line that this solution does indeed work. Thank you.

There seems to be a pretty big disconnect between support (phone) and that given in the forums. When i first received my first error, I contacted support. After reviewing the logs that I sent in, I was told that I need to upgrade my Veeam server to 3 processors and upgrade to the latest version. I can appreciate upgrading to the next version, but the support rep sent me a MS article pointing me to replication and also told me the VSS issue had something to do with the VMware side of things.


It would be nice to receive the same level of support when you call in. I did upgrade, but I definitely don't need 3 processors in my Veeam server and since this post actually solved my issue, i don't believe the upgrade had much to do with it.