Guest processing backup generating an error

[neilh1703]

Good day,

I am encounting issues when performing an application-aware backup of a new linux db VM running SuSE 12.5, Oracle 19c.

The host is a Suse 12.5, Oracle 19c db and is currently in use. and struggling working out what is going on. Veeam 10a.

After Getting information from Vsphere it says

Code: Select all

Unable to perform application-aware processing of Oracle database (SID: ???) : Oracle error has occurred. ORA-12547: TNS:lost contact

--tr:OracleSession: failed to init server.
--tr:Failed to create oracle session.
--tr:Failed to create oracle session holder. SID: ???.  

The Agent log says:

Code: Select all

[11.05.2021 14:19:03.530] <140193661417216> oralib| Trying to initialize oracle interface with dblink []
[11.05.2021 14:19:03.530] <140193661417216> oralib|   Initializing oracle session
[11.05.2021 14:19:03.600] <140193661417216> oralib|   Initializing oracle session Failed.
[11.05.2021 14:19:03.600] <140193661417216> oralib| WARN|Failed to connect as SYSDBA: insufficient privileges or channel error. Connecting by default. Login: XXXX
[11.05.2021 14:19:03.600] <140193661417216> oralib| >>  |Oracle error has occurred. ORA-12547: TNS:lost contact
[11.05.2021 14:19:03.600] <140193661417216> oralib| >>  |
[11.05.2021 14:19:03.600] <140193661417216> oralib| >>  |--tr:OracleSession: failed to init server.
[11.05.2021 14:19:03.600] <140193661417216> oralib| >>  |--tr:Failed to create oracle session.

I have confirmed I can login as the Oracle user XXXX with sysdba priviliges.

The host was originally cloned from another db VM.

Oracle is running fine on the new VM and is use.

Our DBAs have confirmed all environmental variables are set correctly and /etc/oratab has the correct information.

I have opened a case with Veeam support and they are struggling to see why the application-aware side is failing.

I have tried to compare the original db backup with the clone and it appears not be finding the clone's SID when performing SSH Oracle checks

Original server task log:

Code: Select all

...
[29.04.2021 13:00:41] <29> Info     [SSH oracle] Explore instances. Oracle username: Oracle_user. DbConfig: <dbconfig><dbrecids><db sid="XXX" recId="0" stamp="" firstPassRecId="29897"></db></dbrecids><excludes /></dbconfig>
...

Cloned severs task log:

Code: Select all

...
[11.05.2021 05:28:09] <20> Info     [SSH oracle] Explore instances. Oracle username: Oracle_username. DbConfig: <dbconfig><dbrecids /><excludes /></dbconfig>
...

Does anyone know how it captures the the SID? Both backups jobs have Guest OS credentials of root, Oracle account with sysdba priviliges 'Oracle_user'. I see prior to the message it installs the oracleagent_x64 successfully to the host

Case: 04784718

[PTide]

Hi,

I found a bug in our internal systems which looks similarly to the issue that you've described.
Would you provide contents of /etc/oratab and inventory.xml from both machines, please? (you can PM those if you want)

P.S. Meahwhile, please keep working with the support team

Thanks!

[neilh1703]

Hi,

Please see below:

Uncloned

oratab:

Code: Select all

# HGQ:/oracle/HGQ/19:N # uncomment this line after upgrade - added by SAP RUNINSTALLER 2020-10-16_11-50-24
# HDC:/oracle/HDC/19:N # uncomment this line after upgrade - added by SAP RUNINSTALLER 2021-02-16_18-50-05
#HDC:/oracle/HDC/19.0.0:N
HDC:/oracle/HDC/19:N

inventory.xml:

Code: Select all

<?xml version = '1.0' encoding = 'UTF-8' standalone = 'yes'?>
<!-- Copyright (c) 1999, 2021, Oracle. All rights reserved. -->
<!-- Do not modify the contents of this file by hand. --><INVENTORY>
   <VERSION_INFO>
      <SAVED_WITH>13.9.4.0.0</SAVED_WITH>
      <MINIMUM_VER>2.1.0.6.0</MINIMUM_VER>
   </VERSION_INFO>
   <HOME_LIST>
      <HOME NAME="OraDb11g_home2" LOC="/oracle/HDC/11204" TYPE="O" IDX="10"/>
      <HOME NAME="OraDB19Home1" LOC="/oracle/HDC/19.0.0" TYPE="O" IDX="11"/>
      <HOME NAME="OraDb11g_home1" LOC="/oracle/ABC/112_64" TYPE="O" IDX="1"/>
      <HOME NAME="agent13c1" LOC="/oemagent/13c/agent_13.4.0.0.0" TYPE="O" IDX="12"/>
      <HOME NAME="OraHome3" LOC="/oemagent/12c/plugins/oracle.sysman.emas.discovery.plugin_12.1.0.8.0" TYPE="O" IDX="6" REMOVED="T"/>
      <HOME NAME="OraHome5" LOC="/oemagent/12c/plugins/oracle.sysman.oh.agent.plugin_12.1.0.5.0" TYPE="O" IDX="8" REMOVED="T"/>
      <HOME NAME="OraHome6" LOC="/oemagent/12c/plugins/oracle.sysman.db.agent.plugin_12.1.0.8.0" TYPE="O" IDX="9" REMOVED="T"/>
      <HOME NAME="OraHome4" LOC="/oemagent/12c/plugins/oracle.sysman.oh.discovery.plugin_12.1.0.5.0" TYPE="O" IDX="7" REMOVED="T"/>
      <HOME NAME="OraHome2" LOC="/oemagent/12c/plugins/oracle.sysman.db.discovery.plugin_12.1.0.8.0" TYPE="O" IDX="5" REMOVED="T"/>
      <HOME NAME="sbin12c1" LOC="/oemagent/12c/sbin" TYPE="O" IDX="3" REMOVED="T"/>
      <HOME NAME="agent12c1" LOC="/oemagent/12c/core/12.1.0.5.0" TYPE="O" IDX="2" REMOVED="T"/>
      <HOME NAME="OraHome1" LOC="/oemagent/12c/plugins/oracle.sysman.xa.discovery.plugin_12.1.0.6.0" TYPE="O" IDX="4" REMOVED="T"/>
   </HOME_LIST>
   <COMPOSITEHOME_LIST/>
</INVENTORY>

Cloned:

oratab:

Code: Select all

# HGQ:/oracle/HGQ/19:N # uncomment this line after upgrade - added by SAP RUNINSTALLER 2020-10-16_11-50-24
# HDC:/oracle/HDC/19:N # uncomment this line after upgrade - added by SAP RUNINSTALLER 2021-02-16_18-50-05
#HDC:/oracle/HDC/19.0.0:N
HDX:/oracle/HDX/19:N
#HDC:/oracle/HDC/19:N

inventory.xml

Code: Select all

<?xml version = '1.0' encoding = 'UTF-8' standalone = 'yes'?>
<!-- Copyright (c) 1999, 2021, Oracle. All rights reserved. -->
<!-- Do not modify the contents of this file by hand. --><INVENTORY>
   <VERSION_INFO>
      <SAVED_WITH>13.9.4.0.0</SAVED_WITH>
      <MINIMUM_VER>2.1.0.6.0</MINIMUM_VER>
   </VERSION_INFO>
   <HOME_LIST>
      <HOME NAME="OraDB19Home1" LOC="/oracle/HDX/19.0.0" TYPE="O" IDX="11"/>
      <HOME NAME="agent13c1" LOC="/oemagent/13c/agent_13.4.0.0.0" TYPE="O" IDX="12"/>
      <HOME NAME="OraHome3" LOC="/oemagent/12c/plugins/oracle.sysman.emas.discovery.plugin_12.1.0.8.0" TYPE="O" IDX="6" REMOVED="T"/>
      <HOME NAME="OraHome5" LOC="/oemagent/12c/plugins/oracle.sysman.oh.agent.plugin_12.1.0.5.0" TYPE="O" IDX="8" REMOVED="T"/>
      <HOME NAME="OraHome6" LOC="/oemagent/12c/plugins/oracle.sysman.db.agent.plugin_12.1.0.8.0" TYPE="O" IDX="9" REMOVED="T"/>
      <HOME NAME="OraHome4" LOC="/oemagent/12c/plugins/oracle.sysman.oh.discovery.plugin_12.1.0.5.0" TYPE="O" IDX="7" REMOVED="T"/>
      <HOME NAME="OraHome2" LOC="/oemagent/12c/plugins/oracle.sysman.db.discovery.plugin_12.1.0.8.0" TYPE="O" IDX="5" REMOVED="T"/>
      <HOME NAME="sbin12c1" LOC="/oemagent/12c/sbin" TYPE="O" IDX="3" REMOVED="T"/>
      <HOME NAME="agent12c1" LOC="/oemagent/12c/core/12.1.0.5.0" TYPE="O" IDX="2" REMOVED="T"/>
      <HOME NAME="OraHome1" LOC="/oemagent/12c/plugins/oracle.sysman.xa.discovery.plugin_12.1.0.6.0" TYPE="O" IDX="4" REMOVED="T"/>
   </HOME_LIST>
   <COMPOSITEHOME_LIST/>
</INVENTORY>

Thanks

[neilh1703]

I have managed to get this solved as follows:

As we are using the root account to perform the Guest Processing, I have added the following variables to the /root/.bashrc

Code: Select all

export ORACLE_SID=HDX
export ORACLE_BASE=/oracle/HDX
export ORACLE_HOME=/oracle/HDX/19
export PATH=$PATH:$ORACLE_HOME/bin

Relinked the Oracle binaries

Checked the Oracle account in Guest Processing is a member of SYS.

Ensured the correct SID is in /etc/oratab.

Confirmed I can enter sqlplus as the Oracle account mentioned in Guest Processing.

Tested the backup job with the scripts and it is working fine. I have also checked the agent log.

Posting the details in case anyone else has similar issue