-
- Veteran
- Posts: 392
- Liked: 33 times
- Joined: Jul 18, 2011 9:30 am
- Full Name: Hussain Al Sayed
- Location: Bahrain
- Contact:
[ID#5156540]Snapshot left behind by Veeam Backup - vRDM LUNs
Hello,
Veeam Backup was running fine, by mistake I stopped the job which was running on the Exchange 2007 VM. This exchange has got 1 VMDK for OS and 13 vRDM LUNs for Database.
VM-ctk.vmdk files left behind in the RDM Mapping LUN for every vRDM.
VM-ctk.vmdk file left behind in the LUN where the VMDK OS located.
Backup is no more can run due to UnFreeze Error, cannot create a Shadow copy.
ddb.deletable in all the CTK files are set to ddb.deletable = "true"
What if V2V cloning would be resolve the issue?
If V2V cloning done, what a about the vRDM will they be reconvert it to VMDK?
Thanks,
Veeam Backup was running fine, by mistake I stopped the job which was running on the Exchange 2007 VM. This exchange has got 1 VMDK for OS and 13 vRDM LUNs for Database.
VM-ctk.vmdk files left behind in the RDM Mapping LUN for every vRDM.
VM-ctk.vmdk file left behind in the LUN where the VMDK OS located.
Backup is no more can run due to UnFreeze Error, cannot create a Shadow copy.
ddb.deletable in all the CTK files are set to ddb.deletable = "true"
What if V2V cloning would be resolve the issue?
If V2V cloning done, what a about the vRDM will they be reconvert it to VMDK?
Thanks,
-
- VP, Product Management
- Posts: 27365
- Liked: 2794 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: [ID#5156540]Snapshot left behind by Veeam Backup - vRDM
Hello Hussain,
V2V cloning will convert your vRDM disks into VMDKs, see this topic for more info: http://communities.vmware.com/message/1748804
I would recommend using a python script mentioned in this topic to consolidate all your existing snapshots: Virtual Appliance mode leaving Snapshots
Thank you.
V2V cloning will convert your vRDM disks into VMDKs, see this topic for more info: http://communities.vmware.com/message/1748804
I would recommend using a python script mentioned in this topic to consolidate all your existing snapshots: Virtual Appliance mode leaving Snapshots
Thank you.
-
- Veteran
- Posts: 392
- Liked: 33 times
- Joined: Jul 18, 2011 9:30 am
- Full Name: Hussain Al Sayed
- Location: Bahrain
- Contact:
Re: [ID#5156540]Snapshot left behind by Veeam Backup - vRDM
hello,
Thanks but the python script indicates that the vmdk files should have ddb.deletable = false but in my case all the vmdks point to this particular VM is ddb.deletable = false which means are allowed to be deleted.
Beside that, I don't have a snapshot left in the directory of the VM. All I Have is ctk-vmdk which worries me.
[root@SX01 MAIL001]# ls
MAIL001-56e9e21b.vswp MAIL001.nvram MAIL001.vmxf vmware-56.log vmware.log
MAIL001-aux.xml MAIL001.vmdk vmware-0.log vmware-57.log
MAIL001-ctk.vmdk MAIL001.vmsd vmware-54.log vmware-58.log
MAIL001-flat.vmdk MAIL001.vmx vmware-55.log vmware-59.log
[root@SX01 MAIL001]#
RDM Mapping LUN:
[root@ESX01 MAIL001]# ls
MAIL001_10-ctk.vmdk MAIL001_14-rdm.vmdk MAIL001_2.vmdk MAIL001_7-ctk.vmdk
MAIL001_10-rdm.vmdk MAIL001_14.vmdk MAIL001_3-ctk.vmdk MAIL001_7-rdm.vmdk
MAIL001_10.vmdk MAIL001_15-ctk.vmdk MAIL001_3-rdm.vmdk MAIL001_7.vmdk
MAIL001_11-ctk.vmdk MAIL001_15-rdm.vmdk MAIL001_3.vmdk MAIL001_8-ctk.vmdk
MAIL001_11-rdm.vmdk MAIL001_15.vmdk MAIL001_4-ctk.vmdk MAIL001_8-rdm.vmdk
MAIL001_11.vmdk MAIL001_16-ctk.vmdk MAIL001_4-rdm.vmdk MAIL001_8.vmdk
MAIL001_12-ctk.vmdk MAIL001_16-rdm.vmdk MAIL001_4.vmdk MAIL001_9-ctk.vmdk
MAIL001_12-rdm.vmdk MAIL001_16.vmdk MAIL001_5-ctk.vmdk MAIL001_9-rdm.vmdk
MAIL001_12.vmdk MAIL001_1-ctk.vmdk MAIL001_5-rdm.vmdk MAIL001_9.vmdk
MAIL001_13-ctk.vmdk MAIL001_1-rdm.vmdk MAIL001_5.vmdk MAIL001-ctk.vmdk
MAIL001_13-rdm.vmdk MAIL001_1.vmdk MAIL001_6-ctk.vmdk MAIL001-rdm.vmdk
MAIL001_13.vmdk MAIL001_2-ctk.vmdk MAIL001_6-rdm.vmdk MAIL001.vmdk
MAIL001_14-ctk.vmdk MAIL001_2-rdm.vmdk MAIL001_6.vmdk
[root@SX01 MAIL001]#
Thanks but the python script indicates that the vmdk files should have ddb.deletable = false but in my case all the vmdks point to this particular VM is ddb.deletable = false which means are allowed to be deleted.
Beside that, I don't have a snapshot left in the directory of the VM. All I Have is ctk-vmdk which worries me.
[root@SX01 MAIL001]# ls
MAIL001-56e9e21b.vswp MAIL001.nvram MAIL001.vmxf vmware-56.log vmware.log
MAIL001-aux.xml MAIL001.vmdk vmware-0.log vmware-57.log
MAIL001-ctk.vmdk MAIL001.vmsd vmware-54.log vmware-58.log
MAIL001-flat.vmdk MAIL001.vmx vmware-55.log vmware-59.log
[root@SX01 MAIL001]#
RDM Mapping LUN:
[root@ESX01 MAIL001]# ls
MAIL001_10-ctk.vmdk MAIL001_14-rdm.vmdk MAIL001_2.vmdk MAIL001_7-ctk.vmdk
MAIL001_10-rdm.vmdk MAIL001_14.vmdk MAIL001_3-ctk.vmdk MAIL001_7-rdm.vmdk
MAIL001_10.vmdk MAIL001_15-ctk.vmdk MAIL001_3-rdm.vmdk MAIL001_7.vmdk
MAIL001_11-ctk.vmdk MAIL001_15-rdm.vmdk MAIL001_3.vmdk MAIL001_8-ctk.vmdk
MAIL001_11-rdm.vmdk MAIL001_15.vmdk MAIL001_4-ctk.vmdk MAIL001_8-rdm.vmdk
MAIL001_11.vmdk MAIL001_16-ctk.vmdk MAIL001_4-rdm.vmdk MAIL001_8.vmdk
MAIL001_12-ctk.vmdk MAIL001_16-rdm.vmdk MAIL001_4.vmdk MAIL001_9-ctk.vmdk
MAIL001_12-rdm.vmdk MAIL001_16.vmdk MAIL001_5-ctk.vmdk MAIL001_9-rdm.vmdk
MAIL001_12.vmdk MAIL001_1-ctk.vmdk MAIL001_5-rdm.vmdk MAIL001_9.vmdk
MAIL001_13-ctk.vmdk MAIL001_1-rdm.vmdk MAIL001_5.vmdk MAIL001-ctk.vmdk
MAIL001_13-rdm.vmdk MAIL001_1.vmdk MAIL001_6-ctk.vmdk MAIL001-rdm.vmdk
MAIL001_13.vmdk MAIL001_2-ctk.vmdk MAIL001_6-rdm.vmdk MAIL001.vmdk
MAIL001_14-ctk.vmdk MAIL001_2-rdm.vmdk MAIL001_6.vmdk
[root@SX01 MAIL001]#
-
- VP, Product Management
- Posts: 27365
- Liked: 2794 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: [ID#5156540]Snapshot left behind by Veeam Backup - vRDM
Having CTK files on your datastore is absolutely normal as these files are used to keep track of changed blocks in your VM.habibalby wrote:Beside that, I don't have a snapshot left in the directory of the VM. All I Have is ctk-vmdk which worries me.
If CTK files are gone, the VMware API will let us know that change information is not available and we will ask VMware to re-enable change tracking back for this VM.
-
- Veteran
- Posts: 392
- Liked: 33 times
- Joined: Jul 18, 2011 9:30 am
- Full Name: Hussain Al Sayed
- Location: Bahrain
- Contact:
Re: [ID#5156540]Snapshot left behind by Veeam Backup - vRDM
Hello,
I have another support case with VMWare and they are saying the VSS it seems its gone
Writer name: 'Microsoft Exchange Writer'
Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
Writer Instance Id: {fecfee60-3ad5-4550-81a2-c8697a3e4c37}
State: [9] Failed
Last error: Timed out
I have another support case with VMWare and they are saying the VSS it seems its gone
Writer name: 'Microsoft Exchange Writer'
Writer Id: {76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}
Writer Instance Id: {fecfee60-3ad5-4550-81a2-c8697a3e4c37}
State: [9] Failed
Last error: Timed out
-
- VP, Product Management
- Posts: 27365
- Liked: 2794 times
- Joined: Mar 30, 2009 9:13 am
- Full Name: Vitaliy Safarov
- Contact:
Re: [ID#5156540]Snapshot left behind by Veeam Backup - vRDM
Not sure how VMware support team can assist you with Microsoft Exchange VSS writer , but If I were you I would look through Event logs on the Exchange server and try to match event IDs with Microsoft KB articles.
If it doesn't help, then it's better to reach Microsoft support team directly for assistance.
If it doesn't help, then it's better to reach Microsoft support team directly for assistance.
-
- Veteran
- Posts: 392
- Liked: 33 times
- Joined: Jul 18, 2011 9:30 am
- Full Name: Hussain Al Sayed
- Location: Bahrain
- Contact:
Re: [ID#5156540]Snapshot left behind by Veeam Backup - vRDM
Hello,
VMware will not help me with VSS issue, but from the logs they have seen and the way how I have descried the issue for them they pin-point the VSS is the issue, still they have to comeback and will see where this dilemma get me off
I'm trying to match the event IDs with Microsoft and other blogs, I have seen lots of people having this trouble and some of them mentioned Veeam product.
Let me see who it goes, if you have any tips or tricks can assist please let me know.
Highly appreciated.
VMware will not help me with VSS issue, but from the logs they have seen and the way how I have descried the issue for them they pin-point the VSS is the issue, still they have to comeback and will see where this dilemma get me off
I'm trying to match the event IDs with Microsoft and other blogs, I have seen lots of people having this trouble and some of them mentioned Veeam product.
Let me see who it goes, if you have any tips or tricks can assist please let me know.
Highly appreciated.
-
- Veteran
- Posts: 392
- Liked: 33 times
- Joined: Jul 18, 2011 9:30 am
- Full Name: Hussain Al Sayed
- Location: Bahrain
- Contact:
Re: [ID#5156540]Snapshot left behind by Veeam Backup - vRDM
Here's the timestamps of the Veeam Job when it runs and Event IDs from Exchange Server. The Events clearly shows the VSS aborting the Database and timing out.
Code: Select all
Veeam:
Start time: 11/28/2011 12:52:15 PM
End time: 11/28/2011 12:56:01 PM
Duration: 0:03:45
Freezing guest operating system
Unfreeze error: [Backup job failed.
Cannot create a shadow copy of the volumes containing writer's data.
A VSS critical writer has failed. Writer name: [Microsoft Exchange Writer]. Class ID: [{76fe1ac4-15f7-4bcd-987e-8e1acb462fb7}]. Instance ID: [{fecfee60-3ad5-4550-81a2-c8697a3e4c37}]. Writer's state: [VSS_WS_FAILED_AT_FREEZE]. Error code: [0x800423f2].]
Exchange: Administrative Event.
Log Name: Application
Source: ESE
Date: 28/11/2011 12:54:37
Event ID: 2007
Task Category: ShadowCopy
Level: Error
Keywords: Classic
User: N/A
Computer: MAIL001.domain.com
Description:
Information Store (336) Shadow copy instance 12 aborted.
Exchange Application Event:
Log Name: Application
Source: MSExchangeIS
Date: 28/11/2011 12:54:03
Event ID: 9606
Task Category: Exchange VSS Writer
Level: Information
Keywords: Classic
User: N/A
Computer: MAIL001.domain.com
Description:
Exchange VSS Writer (instance 60193528-c79f-4e6a-a36e-74427522ad2d) has prepared for backup successfully.
Log Name: Application
Source: ESE
Date: 28/11/2011 12:54:14
Event ID: 2005
Task Category: ShadowCopy
Level: Information
Keywords: Classic
User: N/A
Computer: MAIL001.domain.com
Description:
Information Store (336) Shadow copy instance 12 starting. This will be a Full shadow copy.
Log Name: Application
Source: MSExchangeIS
Date: 28/11/2011 12:54:14
Event ID: 9811
Task Category: Exchange VSS Writer
Level: Information
Keywords: Classic
User: N/A
Computer: MAIL001.domain.com
Description:
Exchange VSS Writer (instance 12) has successfully prepared the database engine for a full or copy backup of storage group 'Second Storage Group'. The following 1 databases are mounted and will be backed up:
Public Folder Database
Log Name: Application
Source: MSExchangeIS
Date: 28/11/2011 12:54:14
Event ID: 9811
Task Category: Exchange VSS Writer
Level: Information
Keywords: Classic
User: N/A
Computer: MAIL001.domain.com
Description:
Exchange VSS Writer (instance 12) has successfully prepared the database engine for a full or copy backup of storage group 'Staff-SG'. The following 4 databases are mounted and will be backed up:
Staff-DB1-H
Staff-DB2-N
Staff-DB3-L
Staff-DB4-O
Log Name: Application
Source: MSExchangeIS
Date: 28/11/2011 12:54:14
Event ID: 9811
Task Category: Exchange VSS Writer
Level: Information
Keywords: Classic
User: N/A
Computer: MAIL001.domain.com
Description:
Exchange VSS Writer (instance 12) has successfully prepared the database engine for a full or copy backup of storage group 'Students-SG2'. The following 5 databases are mounted and will be backed up:
Students-DB10
Students-DB6
Students-DB7
Students-DB8
Students-DB9
Log Name: Application
Source: MSExchangeIS
Date: 28/11/2011 12:54:14
Event ID: 9811
Task Category: Exchange VSS Writer
Level: Information
Keywords: Classic
User: N/A
Computer: MAIL001.domain.com
Description:
Exchange VSS Writer (instance 12) has successfully prepared the database engine for a full or copy backup of storage group 'Students-SG'. The following 5 databases are mounted and will be backed up:
Students-DB1
Students-DB2
Students-DB3
Students-DB4
Students-DB5
Log Name: Application
Source: MSExchangeIS
Date: 28/11/2011 12:54:14
Event ID: 9608
Task Category: Exchange VSS Writer
Level: Information
Keywords: Classic
User: N/A
Computer: MAIL001.domain.com
Description:
Exchange VSS Writer (instance 60193528-c79f-4e6a-a36e-74427522ad2d:12) has prepared for Snapshot successfully.
Log Name: Application
Source: ESE
Date: 28/11/2011 12:54:14
Event ID: 2001
Task Category: ShadowCopy
Level: Information
Keywords: Classic
User: N/A
Computer: MAIL001.domain.com
Description:
Information Store (336) Shadow copy instance 12 freeze started.
Log Name: Application
Source: ESE
Date: 28/11/2011 12:54:14
Event ID: 2001
Task Category: ShadowCopy
Level: Information
Keywords: Classic
User: N/A
Computer: MAIL001.domain.com
Description:
MSExchangeIS (336) Second Storage Group: Shadow copy instance 12 freeze started.
Log Name: Application
Source: ESE
Date: 28/11/2011 12:54:14
Event ID: 2001
Task Category: ShadowCopy
Level: Information
Keywords: Classic
User: N/A
Computer: MAIL001.domain.com
Description:
MSExchangeIS (336) Staff-SG: Shadow copy instance 12 freeze started.
Log Name: Application
Source: ESE
Date: 28/11/2011 12:54:14
Event ID: 2001
Task Category: ShadowCopy
Level: Information
Keywords: Classic
User: N/A
Computer: MAIL001.domain.com
Description:
MSExchangeIS (336) Students-SG: Shadow copy instance 12 freeze started.
Log Name: Application
Source: ESE
Date: 28/11/2011 12:54:14
Event ID: 2001
Task Category: ShadowCopy
Level: Information
Keywords: Classic
User: N/A
Computer: MAIL001.domain.com
Description:
MSExchangeIS (336) Students-SG2: Shadow copy instance 12 freeze started.
Log Name: Application
Source: MSExchangeIS
Date: 28/11/2011 12:54:15
Event ID: 9610
Task Category: Exchange VSS Writer
Level: Information
Keywords: Classic
User: N/A
Computer: MAIL001.domain.com
Description:
Exchange VSS Writer (instance 60193528-c79f-4e6a-a36e-74427522ad2d:12) has frozen the storage group(s) successfully.
Log Name: Application
Source: ESE
Date: 28/11/2011 12:54:37
Event ID: 2003
Task Category: ShadowCopy
Level: Information
Keywords: Classic
User: N/A
Computer: MAIL001.domain.com
Description:
Information Store (336) Shadow copy instance 12 freeze ended.
Log Name: Application
Source: ESE
Date: 28/11/2011 12:54:37
Event ID: 2007
Task Category: ShadowCopy
Level: Error
Keywords: Classic
User: N/A
Computer: MAIL001.domain.com
Description:
Information Store (336) Shadow copy instance 12 aborted.
Log Name: Application
Source: MSExchangeIS
Date: 28/11/2011 12:54:37
Event ID: 9614
Task Category: Exchange VSS Writer
Level: Information
Keywords: Classic
User: N/A
Computer: MAIL001.domain.com
Description:
Exchange VSS Writer (instance 60193528-c79f-4e6a-a36e-74427522ad2d:12) has aborted the backup successfully.
Log Name: Application
Source: MSExchangeIS
Date: 28/11/2011 12:54:57
Event ID: 9648
Task Category: Exchange VSS Writer
Level: Information
Keywords: Classic
User: N/A
Computer: MAIL001.domain.com
Description:
Exchange VSS Writer (instance 60193528-c79f-4e6a-a36e-74427522ad2d:12) has processed the backup shutdown event successfully.
Log Name: Application
Source: VSS
Date: 28/11/2011 12:57:57
Event ID: 8224
Task Category: None
Level: Information
Keywords: Classic
User: N/A
Computer: MAIL001.domain.com
Description:
The VSS service is shutting down due to idle timeout.
Who is online
Users browsing this forum: No registered users and 15 guests