Guidelines for Monitoring, Diagnosing, and Resolving Problems

Enterprise Administration > Windchill ESI > Administering Windchill ESI in an ORACLE Applications Environment > Guidelines for Monitoring, Diagnosing, and Resolving Problems

This section provides you with:

• General guidelines on how to monitor and diagnose problems and lists some specific techniques you can use to resolve them.

• A listing of specific problems, their causes, and resolutions.

Monitoring Problems

As a Windchill ESI Administrator you are alerted to problems through reporting channels such as user reports which includes trouble tickets logged via a help desk or through e-mails and through automated system alerts.

User reports are a passive, reactive means of detecting problems. To improve Windchill ESI system stability and lower the overall cost-of-ownership, it is recommended that you supplement user reports with automated system alerts which is a proactive approach that can prevent problems before they become critical.

The following describe how to monitor problems using the following:

• TIBCO BusinessWorks monitoring services

• Error handling processes and logging services

• Event rules and problem-detection approaches

Monitoring with BusinessWorks

TIBCO BusinessWorks is used by the Windchill ESI Error Handling process to monitor the Windchill EAI software components and to handle system problems. TIBCO Administrator provides you with a Graphical User Interface and microagents to monitor and maintain hardware and software components within TIBCO administrative domains. These tools can be configured to monitor BusinessWorks process engines, adapters, log file entries, available disk space, operating system parameters, and more. When defining a deployment configuration, TIBCO Administrator provides facilities to create alert and escalation rules triggered by predefined events, such as a component failure, suspended process, or log event.

You can use BusinessWorks to monitor the following:

• BusinessWorks engine problems

• Adapter problems

• JMS problems

BusinessWorks Engine Problems

TIBCO BusinessWorks engines need to be running for every process used in Windchill ESI. Therefore the BusinessWorks monitoring component must be able to do the following:

• Handle events raised when a failure of a BusinessWorks engine causes messages to be lost or for processing to stop

• Send alerts to TIBCO Administrator monitoring when failures occur

• Escalate actions as failure frequency increases

Failure Categories

In a deployment, failures can be separated into the following categories:

• Any Failure: Catches any failure and performs an action

• First Failure: Catches the first failure of an engine and performs an action

• Second Failure: Catches the second failure of an engine and performs an action

• Subsequent Failure: Catches any failure that occurs subsequent to the second failure and performs an action

Suggested Deployment Configuration

The following lists suggested actions that you can configure during deployment, for the various types of failures:

• Any Failure: Raise an alert to the administrator

• First Failure: Restart the engine

• Second Failure: Restart the engine and raise a second failure alert

• Subsequent Failure: Restart the engine and send e-mail to the administrator

A configurable counter and timer exists which determines when to reset the failure count to first. Use the timer setting on this counter to set to a particular time frame in which more than two failures in that time frame raises considerable concern about the system’s overall integrity.

1. Navigate down through the TIBCO Administrator GUI:

◦ Application Management > <Application Name> > Configuration > Process Archive.par

2. Click the Monitoring tab:

3. Click the Add button under Events. The Add Event screen will appear:

4. Select the type of event you would like to monitor. For example, when a JMS queue fails, a process becomes suspended. You may configure an alert to monitor for a process suspension:

A systems administrator has the ability to restart the engine using TIBCO BusinessWorks Administrator. For more information, see topic JMS Problems below.

Adapter Problems

The adapter’s distribution target system needs to be running for many processes used in Windchill ESI. Therefore, the TIBCO BusinessWorks monitoring component must be able to do the following:

• Restart adapter instances when they fail

• Send alerts to TIBCO Administrator when failures occur

• Escalate actions as failure frequency increases

Failure Categories

In a deployment, failures can be separated into the following categories

• Any Failure: Catches any failure and performs an action

• First Failure: Catches the first failure of an adapter and performs an action

• Second Failure: Catches the second failure of an adapter and performs an action

• Subsequent Failure: Catches any failure that occurs subsequent to the second failure and performs an action

Suggested Deployment Configuration

The following lists suggested actions that you can configure during deployment, for the various types of failure:

• Any Failure: Raise an alert to the administrator

• First Failure: Restart the adapter

• Second Failure: Restart the adapter and raise a Second Failure alert

• Subsequent Failure: Restart the adapter and send an e-mail to the administrator

JMS Problems

An EMS Server needs to be running for the master process flow and error handling processes to work properly. The TIBCO Administrator must be able to do the following:

• Handle events raised when an EMS server failure occurs

• Send errors to the error handling process when a failure occurs

Unfortunately, TIBCO BusinessWorks does not create an event when EMS servers fail. To combat this, you can do one of the following:

• No suspend: When a JMS queue does not respond or cannot be connected to at runtime after a repeat-on-error-until-true group with the suspend checkbox option not selected, BusinessWorks creates a process engine log entry indicating a timeout of the JMS activity. You could configure a log event in TIBCO Administrator to monitor the log for this and raise an alert.

• Suspend: When a JMS queue does not respond or cannot be connected to at runtime after a repeat-on-error-until-true group with the suspend checkbox option selected, BusinessWorks suspends the process and you can configure a TIBCO Administrator rule base to raise an alert.

By default, Windchill ESI is configured for the suspend approach, as this allows for the process that caught the JMS queue failure to continue after restarting the EMS server.

Special Guidelines for EMS Server Problems

Windchill PDMLink allows using a single JMS provider at a time. Therefore, the TIBCO EMS server becomes an integral part of the Windchill ESI architecture, and Windchill PDMLink users may use it for functions other than Windchill ESI applications. Therefore, it may not be prudent to configure Windchill ESI to automatically restart the EMS server when there is a problem with Windchill ESI that is related to a JMS problem. Rather, the default and recommended configuration is to have the error-handling process place the current BusinessWorks job in suspend mode when it detects a serious JMS problem.

Suspend mode allows you to manually intervene and does not pose a risk of data loss. Only the affected BusinessWorks job - that is, the product data transaction that is being published - is suspended while other jobs in the same process engine may continue. If other jobs encounter the same JMS problem, they get individually suspended as well. You can individually restart jobs via the TIBCO BusinessWorks Administrator. Jobs get resumed from the point of suspension, not from the last checkpoint.

TIBCO Administrator does not provide built-in administration domain monitoring of TIBCO EMS. You may, however, configure TIBCO BusinessWorks to issue alerts in the event that the EMS server is suspended or requires restarting. End-users can use the basic administration console in TIBCO EMS to configure TIBCO BusinessWorks to issue alerts to in the event that the EMS server is suspended or requires to be restarted. For more information, see topic Configuring E-Mail Alerts below.

Oracle Applications Problems

The TIBCO Adapter for Oracle Applications log files can report problems such as an invalid Oracle Applications Windchill ESI user account. These situations generally occur because of problems such as typographical errors made during installation or passwords expiring after a predetermined period. These can be unexpected and difficult to diagnose. Therefore, you may wish to configure TIBCO BusinessWorks to monitor these log files for the relevant message text and to issue alerts should the event occur.

Using Error Handling Processes and Logging Services

In addition to configuring alerts for problems with the underlying TIBCO products, you can also use the error handling and logging shared services to detect and pinpoint Windchill ESI problems.

Developing Event Rules and Problem-Detection Approaches

Since event rules are closely tied to hardware deployment, it is not feasible for Windchill ESI to provide a predefined, ready-to-use configuration. However, you can develop and adjust event rules as needed, over time, in a non-invasive manner to the underlying TIBCO products and Windchill ESI business logic component. You can prioritize developing rules based on the most common or troublesome issues you have encountered, and thus gradually, move from a reactive to a proactive problem-detection approach.

Diagnosing Problems

After detecting a problem that cannot be corrected automatically or by the user, you need to begin diagnosing the problem. This involves categorizing and localizing the problem to determine its root cause.

Localizing Problems

To localize the source of the problem, you need to ask questions such as:

• Is the problem associated with a business process issue such as a system-of-record violation, a functional issue such as invalid data, or a technical issue such as a server being down?

• Is the problem associated with Windchill PDMLink, TIBCO, or the distribution target ERP system?

• If it is a problem related to TIBCO, is it associated with TIBCO EMS, BusinessWorks, or Adapter for Oracle Applications?

• Is the problem associated with the BusinessWorks deployment and/or run-time agents?

• Is the problem associated with the underlying physical network and computing, rather than Windchill ESI?

• Can the problem scenario be duplicated in a test system with the same configuration as the production environment?

Categorizing Problems: Key Areas of Focus for Troubleshooting

To categorize problems, you need to focus on key problem areas and get familiar with error handling reports such as error logs and error handling codes.

Most system-related, technical issues can be categorized according to the location of the root cause. Refer to the Windchill Enterprise Systems Integration Installation and Configuration - Oracle Applications section and applicable documentation provided by your systems integrator, as necessary, to verify and restore correct configuration settings.

It is also important for you to become familiar with the business process and functional troubleshooting information. Users who are not familiar with this information may escalate such issues to you.

The following categories of problems and their descriptions are not intended to be exhaustive with detailed step-by-step procedures. Rather, they are provided to help you to focus on some of the key or potential root causes of technical issues:

• Problems originating from Windchill ESI

• Problems originating from TIBCO components such as:

◦ EMS

◦ BusinessWorks

◦ Adapter for Oracle Applications

• Problems originating from the distribution target.

• Problems indicated in Windchill ESI logs.

Windchill ESI Problems

The following lists how to deal with problems that may originate from Windchill ESI services:

• Verify that the Windchill server and application are running

• Verify that the Windchill JMS client is connecting to the TIBCO EMS server

• Verify that the Windchill ESI user account for the EMS server is correctly configured

• Verify all other JMS-related configurations

• Scan for error messages in the Windchill administrative logs

• Validate the distribution targets and their attributes for the given release

• Verify correct operation of the Windchill Enterprise Integration Access Remote Procedure Calls (RPCs)

• Verify the correctness of the values set for the various Windchill ESI preferences

TIBCO Problems

The following list how to deal with problems that may originate from TIBCO components:

EMS Problems

• Verify that the EMS server is running

• Verify the EMS server is correctly configured

• Ensure that the required Windchill ESI JMS queues are defined

• Verify that the BusinessWorks Windchill ESI user account for the EMS server is correctly configured

• Verify the secure queue authentication configuration

• Isolate any Java Virtual Machine (JVM) issues, such as unsupported versions or memory overflow errors

BusinessWorks Problems

• Verify that all required TIBCO services are running. Process engines do not start if the following required services are not running. Verify that all required TIBCO services are running:

◦ TIBCO Administrator <version> (Domain name)

◦ TIBCO HAWK Agent (Domain name)

• Verify the existence and contents of required Windchill ESI BusinessWorks application properties and configuration files:

◦ ESIORAEMailMessageLookups.properties

◦ ESIORADefaults.properties

◦ ESIORAErrorHandlingCodes.properties

◦ ESIORALookups.properties

◦ ESIORAMessageLookups.properties

◦ FilesToRead_ORA.properties

• Verify that the BusinessWorks JMS client is connecting to the TIBCO EMS server

• Verify all other EMS-related configuration

• Isolate any Java Virtual Machine (JVM) issues, such as unsupported versions or memory overflow errors

• Verify that the Windchill ESI components are deployed to the correct domain

• Verify the Windchill ESI business logic configuration and deployment settings, including the global variable values

You must restart the process engine for new global variable values to take effect.

• Verify the Java classpath

• Validate the configuration settings in the TRA file

• Validate the BusinessWorks Administrator server configuration by:

◦ Authorizing user accounts

◦ Making sure that multiple repositories with the same Rendezvous configuration (network, service, and daemon) on the same subnet have unique names, regardless of the BusinessWorks administration domain

◦ Ensuring that the server-based repository (.dat) files are not deleted directly. To properly delete a server-based project from an administration domain, follow these steps:

▪ Undeploy the project (this removes all of the .tra and .cmd files)

▪ Stop the administration server service

▪ Delete the contents of <Tibco_Home>/tra/domain/<Domain_Name>/application/<Application_Name>/working folder

▪ Restart the administration server service

The meaning of <Application_Name> in the above referenced path may be understood from the fact that the deployment name of an application is of the form <Domain_Name>-<Application_Name>.

• Isolate any issues with internationalization or locale configuration settings. Check out the following:

◦ com_infoengine_locale attribute in the JMS header of Windchill ESI messages

◦ ESIOMAdapter/Locale global variable

◦ Defaults and cross-referencing lookup file entries used in data mapping

Adapter for Oracle Applications Problems

The following lists how to deal with problems that may originate in an Oracle Applications distribution target. Refer to the Windchill Enterprise Systems Integration Installation and Configuration - Oracle Applications section for more details.

• Verify that all Oracle Applications servers are running.

• Verify that the Oracle Applications system is running a version that is supported

• Verify the existence of a valid ESISYS Oracle Applications user account used by the TIBCO Adapter for Oracle Applications in all ESITarget instances.

• Verify that the ESISYS user account has the required security authorization profile.

• Isolate any issues with Internationalization or locale configuration.

Oracle Applications Problems

• Verify that all Oracle Applications servers are running.

• Verify that the Oracle Applications system is running a version that is supported

• Verify the existence of a valid ESISYS Oracle Applications user account used by the TIBCO Adapter for Oracle Applications in all ESITarget instances.

• Verify that the ESISYS user account has the required security authorization profile.

• Isolate any issues with Internationalization or locale configuration.

Logs and Error Handling Codes

To help diagnose problems, you need to be thoroughly familiar with the key Windchill ESI logs, described in the Windchill Enterprise Systems Integration Installation and Configuration - Oracle Applications section. You also need to be familiar with the EAI error-handling architecture, and the error codes. The following lists the available logs and how to access them:

Logs

How to Access

Windchill Enterprise Systems Transaction Log

Windchill PDMLink user interface

TIBCO BusinessWorks Process engine logs

Windchill ESI application messages appear with a role designation of ESI to distinguish them from the standard TIBCO product messages.

TIBCO BusinessWorks Administrator. You can find the log files at <TIBCO_HOME>/tra/domain/<domain_name>/application/logs/<Application_Name>-Process_Archive.log[.sequence#]

TIBCO EMS logs

File name and path are determined by log file related configuration parameters in the tibemsd.conf file.

TIBCO Adapter for Oracle Applications logs

<TIBCO_HOME>/tra/domain/<domain_name>/application/logs/<Application_Name>-ESIOMAdapter.log.[.sequence#]

Oracle Applications logs

Consult your Oracle Applications administrator.

You should also be familiar with the Oracle Applications distribution targets and the data objects and attributes impacted by Windchill ESI. You may have direct responsibility for viewing and manipulating data in Oracle Applications, or may simply have to coordinate these activities with Oracle Applications administrators. To view the data that was transferred into Oracle Applications by Windchill ESI, the following windows are particularly important:

• Master Item

• Bills of Materials

• Engineering Change Orders

• Routings

Resolving Problems

After you have monitored, detected, and diagnosed problems, you need to resolve them. The following lists some general techniques you can use when troubleshooting as well as a list of specific problems with their solutions.

Techniques for Resolving Problems

Following lists some of the troubleshooting techniques you can use to resolve problems.

Coordinating Troubleshooting Teams and Escalating Problems

As a Windchill ESI administrator, you may have to involve and coordinate a wide variety of specialists to fully resolve production issues. These parties may include end-users, functional experts, Windchill PDMLink administrators, Oracle Applications administrators, database specialists, operating system specialists, network administrators, and others. If, after diagnosing and localizing an issue, you determine that the problem cannot be solved in-house, you may need to escalate it to systems integration partners or to PTC Technical Support, or both, for assistance. Systems integration partners may be particularly critical to help you resolve problems involving special configuration or customizations made to the standard Windchill ESI product.

Manually Overriding Checkpoints

The Windchill ESI application leverages checkpoint activities at key points in the TIBCO BusinessWorks processing flow. A checkpoint saves the process data and state of the current running process instance so that it can be recovered at a later time in the event of a failure. If a process engine fails, all process instances can be recovered and can resume execution at the location of their last checkpoint in the process definition. Only the most recent state is saved by a checkpoint. If you have multiple checkpoints in a process, only the state from the last checkpoint is available for recovering the process.

The location of Windchill ESI checkpoints has been strategically designed to ensure robust transaction integrity while minimizing adverse performance effects. Occasionally, you may need to manually override an activated checkpoint to restart transaction processing from scratch when the process engine is restarted.

Overriding checkpoints should be done with extreme caution, as it can lead to duplicate or incomplete data, incorrect export history records, and data corruption.

However, if circumstances warrant it, you can override checkpoints by deleting the <Tibco_Home>/tra/domain/<Domain_Name>/application/<Application_Name>/working directory and its contents. Depending on the process engine mode, there may be many other /working subdirectories under the /tibco path that would need to be deleted. Do a search for /working to locate and remove all of these subdirectories.

Resolving Solaris Platforms Issues

On Solaris platforms you may have to resolve problems that arise from:

• Environment variable issues

• Designer deployment configuration issues

The following describe techniques to resolve these.

Environment Variable Issues

On various Unix platforms certain TIBCO components fail to launch because of library dependencies. If there are any *.sh or *.csh files present in the bin directory of that particular component, they should be executed and then the ‘ldd’ command can be used to identify library dependencies. All the required library paths should be added to the tibco.env.CUSTOM_EXT_PREPEND variable present in the *.tra file.

Designer Deployment Configuration Issues

Occasionally, the TIBCO Administrator and the TIBCO Runtime Agent (TRA) become corrupted and without warning, shut down on the Solaris platforms. If this occurs while TIBCO Designer is running, deployment configurations and TIBCO Designer may behave strangely, for example:

• Server-based repositories do not appear in the project pull-down list for the current domain when opening projects

• Deployment components are displayed as new, even though the deployment history show they had been deployed

• The deploy and undeploy options are unavailable for deployed, changed, or even new deployment components

If any of these symptoms occur, reboot the Solaris machine, restart the TRA and TIBCO Administrator, and then restart Designer.

Finding and Killing TIBCO Processes on UNIX platforms

Occasionally, as part of resolving problems you may need to kill all running TIBCO processes.

On UNIX platforms, you can use the UNIX command ‘ps’ to list all the processes that are running. However, the standard ‘all’ flag, ‘-a’, does not necessarily work to list low-level TIBCO processes such as TIBCO Rendezvous. To view such processes, you can execute the following command:

# ps -ef | grep <filter>

For example:

# ps -ef | grep tibco

The above results in several lines that look like this:

root 5559 1 0 09:10:02 pts/1 0:18
/opt/tibco/administrator/<version>/bin/tibcoadmin --propfile /opt/

You can force-kill the processes listed by using the ‘kill’ command with the flag -9 and the process ID (the second column from your ‘ps -ef’ command)

# kill -9 5559

Using the -9 flag ensures the process is really dead. Generally speaking, to ensure all TIBCO processes are dead, you should run

ps -ef | grep tibco
ps -ef | grep hawk

and

ps -ef | grep rvd

and verify that no running processes are found.

Configuring E-Mail Alerts

As mentioned earlier, Windchill ESI can be configured to send you e-mail alert messages when the Windchill ESI business logic experiences an error:

• Sending an ESIPostResult message to Windchill PDMLink

• Waiting for an ESIResultResponse message from Windchill PDMLink

• Parsing the ESIResultResponse message

This configuration is performed through the following global variables:

• ESIMail/ToAddress

• ESIMail/FromAddress

• ESIMail/CCAddress

• ESIMail/BCCAddress

The server is configured using the global variable: ESIMail/SMTPHostServer.

Refer to theWindchill Enterprise Systems Integration Installation and Configuration - Oracle Applications section for details on how to configure the global variables.

The Windchill ESI business logic attempts to retry any EMS-related actions the number of times specified by the TIBCO BusinessWorks global variable, ESIJMS/RetryCount. Therefore, you may receive multiple e-mail messages for the same issue. After a final unsuccessful attempt, the Windchill ESI business logic suspends the associated BusinessWorks process, allowing you to examine the errors described in the e-mail alert messages, fix the underlying problems, and resume or kill the BusinessWorks process using TIBCO Administrator. Refer to the TIBCO Administrator User's Guide for details on how to use this interface.

Resolving Specific Problems

The following section lists specific problems, provides possible causes, and suggests solutions to resolve them.

TIBCO Administrator shows status as unknown for BWengine or adapter processes

Occasionally, as part of resolving problems you may need to kill all running TIBCO processes.

# ps -ef | grep <filter>

For example:

# ps -ef | grep tibco

The above results in several lines that look like this:

root 5559 1 0 09:10:02 pts/1 0:18
/opt/tibco/administrator/<version>/bin/tibcoadmin --propfile /opt/

You can force-kill the processes listed by using the ‘kill’ command with the flag -9 and the process ID (the second column from your ‘ps -ef’ command)

# kill -9 5559

Using the -9 flag ensures the process is really dead. Generally speaking, to ensure all TIBCO processes are dead, you should run

ps -ef | grep tibco
ps -ef | grep hawk

and

ps -ef | grep rvd

and verify that no running processes are found.

Configuring E-Mail Alerts

As mentioned earlier, Windchill ESI can be configured to send you e-mail alert messages when the Windchill ESI business logic experiences an error:

• Sending an ESIPostResult message to Windchill PDMLink

• Waiting for an ESIResultResponse message from Windchill PDMLink

• Parsing the ESIResultResponse message

This configuration is performed through the following global variables:

• ESIMail/ToAddress

• ESIMail/FromAddress

• ESIMail/CCAddress

• ESIMail/BCCAddress

The server is configured using the global variable: ESIMail/SMTPHostServer.

Refer to theWindchill Enterprise Systems Integration Installation and Configuration - Oracle Applications section for details on how to configure the global variables.

Resolving Specific Problems

The following section lists specific problems, provides possible causes, and suggests solutions to resolve them.

TIBCO Administrator shows status as unknown for BWengine or adapter processes

There are two possible causes for this issue:

• A particular service’s microhawk agent is taking time to send its status to the Administrator; it should be resolved in few minutes.

• A configuration change has not been picked up by hawk service; restarting the hawk service will resolve this issue

Adapter fails to start with JMS transport

After installing TIBCO Runtime Agent <version> and TIBCO Runtime Agent <version>.1, TIBCO Adapter projects using Enterprise Message Service as transport will not start up from TIBCO Designer and give the following error:

Message "Code = AESDKC-0156,Category = JmsComm, Severity = errorRole, Description = could not open JMS shared library jms." displays

This could be due to incompatible Adapter SDK libraries. To resolve this issue:

• On Windows: Remove the libeay32.dll and ssleay32.dll files from <TIBCO_HOME>/adapters/sdk/version/lib

• On UNIX: Remove the libssl and libcrypto openssl libraries from the <TIBCO_HOME>/adapters/sdk/version/lib directory.

Unresolved library dependencies

Message "Code = AESDKC-0156,Category = JmsComm, Severity = errorRole, Description = could not open JMS shared library jms." displays

This could be due to incompatible Adapter SDK libraries. To resolve this issue:

• On Windows: Remove the libeay32.dll and ssleay32.dll files from <TIBCO_HOME>/adapters/sdk/version/lib

• On UNIX: Remove the libssl and libcrypto openssl libraries from the <TIBCO_HOME>/adapters/sdk/version/lib directory.

“Coyote connector has not been started” error appears in TIBCO BusinessWorks Engine Logs

This could be due to incorrect values for ESIOthers/WSHost and ESIOthers/WSPort variables. Providing the proper values should resolve this issue.

All EMS Server configurations disappear after the EMS server is started manually

This could be because the command to start the EMS server has been changed in the 5.1.4 version. In EMS version 4.x the command to start EMS was "./tibemsd". In EMS v5.1.4 the command is "./tibemsd64 -config ../tibco/cfgmgmt/ems/data/tibemsd.conf". The command uses relative path and it should be executed from "<TIBCO_HOME>\ems\<version>\bin".

To resolve this problem stop the process started by the command:

"./tibemsd"

and start the ems server with correct command:

"./tibemsd64 -config ../tibco/cfgmgmt/ems/data/tibemsd.conf"

Deployment of Applications using the TIBCO Administrator GUI displays a Failure message

This could happen if any previous deployment or un-deployment accidentally acquires a lock and has not released it.

To resolve this issue run the AppManage command from command prompt to diagnose and resolve the problem accordingly:

• Browse to <Tibco_Home>\tra\<version>\bin

• Run the following command:

AppManage -deploy -app <Application name> -user <username>
-pw <password> -domain <Domain_Name>

For Example:

AppManage -deploy -app ORA_ESI_Solution -user admin -pw admin
-domain I4590

For more information refer to the <Tibco_Home>\tra\domain\<Domain_Name>\logs\ApplicationManagement.log file.

TIBCO BusinessWorks returns error messages

TIBCO BusinessWorks returns error messages such as the following:

-2003 Feb 19 13:27:12:294 GMT -5 Engine Error [] PE-Error
process initialization failed for
ProcessDefinitions/Services/WCResult_Service

--Initialization error in
[ProcessDefinitions/Services/WCResult_Service/JMSRepeatUntilTru
e_ESIPostResult_Result/JMSSender_ESIResult_PostResult]

--javax.naming.ServiceUnavailableException: Failed to query
JNDI: Failed to connect to the server at tcp://localhost:7222.
Root exception is javax.jms.JMSException: Failed to connect to
the server at tcp://localhost:7222

This could happen if the EMS server name is invalid.

To resolve this problem confirm that the ESIJMS/JNDIContextURL global variable in the deployment configuration of the process engine matches the value in the QueueConnectionFactory URL, in the factories.conf file located in <Tibco_Home>/ems/<version>/tibco/cfgmgmt/ems/data

The ESIJMS/JNDIContextURL global variable should be set to:

tibjmsnaming://<machinename>:<port>

and the QueueConnectionFactory URL should be set to:

tcp://<machine name>:<port>.

The machine name and port values should match.

For example, if ESIJMS/JNDIContextURL is set to tibjmsnaming://mymachine.mycompany.com:7222, then the QueueConnectionFactory URL should be set to tcp://mymachine.mycompany.com:7222.

Refer to “Global Variables for Process Engines” theWindchill Enterprise Systems Integration Installation and Configuration - Oracle Applications for details on this variable.

BusinessWorks returns error messages

BusinessWorks returns error messages such as the following:

-2003 Feb 19 18:57:29:051 GMT -5 Engine Error [] PE-Error process
initialization failed for
ProcessDefinitions/DataProcessing/JMS_ESIEvent_TransactionRelease_
End_PD

--Initialization error in
[ProcessDefinitions/DataProcessing/JMS_ESIEvent_TransactionRelease
_End_PD/JMSReceiver_Event_ESIEvent]

--Could not establish connection or session with EMS provider.

--javax.naming.AuthenticationException: Not permitted: invalid name
or password. Root exception is javax.jms.JMSSecurityException:
invalid name or password

This could happen if the user name and password for the JMS server is invalid.

To resolve this issue confirm that the ESIJMS/Username and ESIJMS/Password global variables in the deployment configuration of the process engine match the values of the username and password specified on the EMS server for the queues.

The username and password values set for the EMS server is in the users.conf file which is located in the <Tibco_Home>/ems/<version>/tibco/cfgmgmt/ems/data directory. Refer to "Global Variables for Process Engines" in theWindchill Enterprise Systems Integration Installation and Configuration - Oracle Applications section for details on this variable.

The passwords are obfuscated if they were set using the EMS Administration tool. If you do not remember the passwords you set, you will have to reset them.

Upon starting a TIBCO component on UNIX, you receive errors:

Upon starting a TIBCO component on UNIX, you receive the following errors:

/usr/lib/dld.sl: Call to mmap() failed - TEXT (UNIX path)

/usr/lib/dld.sl: Permission denied

For example,

/usr/lib/dld.sl: Call to mmap() failed - TEXT
/opt/tibco/tra/5.12/lib/libmaverick50.sl

/usr/lib/dld.sl: Permission denied

This could happen if permissions for the file have not been set properly.

To resolve this issue use the command chmod 555 on the file listed after TEXT. For example,

# chmod 555 /opt/tibco/tra/<version>/lib/libmaverick50.sl

BusinessWorks, Windchill ESI, or both are failing to connect to EMS

This could happen if the EMS server is not configured properly. When you specify the name of the EMS server as localhost, that server is only recognized on the box that it is running on. No other machine can connect to it. An application that is set to localhost will attempt to find the EMS server running on the same machine. If it is not found, it will error. When you specify a machine name as your server name, other machines can connect to your EMS server.

To resolve this issue check that the QueueConnectionFactory value located in the <Tibco_Home>/ems/<version>/tibco/cfgmgmt/ems/data/factories.conf file and the global variable ESIJMS/JNDIContextURL in the deployment of the Process Engine are set accordingly.

• QueueConnectionFactory in factories.conf file = tcp://<machinename>:7222 Where machine name is the machine where the EMS server is running.

• Set the global variable ESIJMS/JNDIContextURL in BW Engine = tibjmsnaming://<machinename of where EMS server is running>:7222

It does not matter where this EMS server resides. It can reside on the same machine as Windchill PDMLink, same machine as the TIBCO process engine, or a different machine altogether. As long as the above values described are set appropriately (and the machines are on the same network), Windchill ESI and EAI components are able to connect to the correct EMS server.

To determine which machine and username is connected to an EMS server, type the following command in the TIBCO EMS administration tool:

>show connections

This gives you a list of which users are connected and from which machine.

Transaction remains in pending state

This could happen if the JMS server queues are not subscribed to as described in Configuring and Administering JMS Queues.

Common root causes are: the JMS server is not running when the Windchill method server starts, misconfigurations in the JMS properties of the Windchill adapter, or misconfigurations in the JMS server.

When Windchill ESI services are successful in subscribing to the JMS queue meant for receiving result messages, the following information is included in the Windchill PDMLink method server log:

Thread-10: subscription to queue "com.ptc.windchill.esi.Result" successful

If this information does not appear in the log, it means Windchill was unable to subscribe to the said queue successfully. In this case, all Windchill ESI transactions are left in a pending state. The transactions are processed once Windchill is able to subscribe to the queue successfully.

Normally, when Windchill ESI services are unable to subscribe to the queue because the JMS server is unavailable, Info*Engine receives an exception. Windchill ESI services log this exception in the Windchill Adapter transaction log. If TIBCO EMS is the JMS provider, the message contains the following text:

javax.ems.JMSException: Failed to connect to the server at tcp

As a detection measure, you may consider configuring your monitoring software to look for this or similar text. To resolve the issue, make sure the EMS server is up before the method server, and resolve any issues with the Windchill adapter JMS configuration and the JMS server configuration.

To avoid this issue, ensure the following:

• Only one WCESI user is connected to EMS server (can be viewed from EMS Administration Tool> Show connections)

• The number of ESISYS connections with ClientID (BW-ESIMaster_JMSConnection-queue-<Application Name>-Process_Archive) should be equal to number of ERP instances configured (can be viewed from EMS Administration Tool> Show connections)

• All the connections are from either TIBCO or Windchill server in the current testing suite. There are no connections from a previous suite or from an alien machine (can be viewed from EMS Administration Tool> Show connections)

• Windchill and Process Archive connect to same JMS Queue (can be viewed from EMS Administration Tool> Show queues)

• The queue com.ptc.windchill.esi.Result has only one receiver (can be viewed from EMS Administration Tool> Show queues)

• There should be no messages in any of the queues (can be viewed from EMS Administration Tool> Show queues)

• Windchill has subscribed to the DataResponse queue successfully. Connect to JMS server and check if the DataResponse queue is created and the file WCESI user is granted send rights on the DataResponse queue. If show queues displays * in front of DataResponse queue name, it indicates that the queue is temporary and needs to be created. This issue is observed if the EAR is deployed manually. Run the following commands in JMS Administration window to resolve the issue:

◦ create queue <DataResponse>

◦ setprop queue <DataResponse> secure

◦ grant queue <DataResponse> <EAI User> receive

◦ grant queue <DataResponse> <WC ESI User> send

◦ setprop factory QueueConnectionFactory url=tcp://<JMSServer>:7222

◦ commit

• Process Archive is connected to the same DataResponse queue that Windchill has subscribed to. Check the JMS Administration window, to confirm that the DataResponse queue is subscribed to by the Process Archive. In case of manual deployment this step is missed sometimes. If the DataResponse queue is subscribed to, check TIBCO Administrator--> Application Management --> Application Name --> Configuration-->Deployment Name--> Advanced -->ESIJMS/DataResponseQueue

TIBCO BusinessWorks Process engine logs error messages

TIBCO BusinessWorks Process engine log contains the following message text:

Job terminated: Please check following 1) FilesToRead_ORA.properties exists at %TIBCO_HOME%\\esi\\bin 2) %TIBCO_HOME%\esi\bin exists in tibco.env.STD_EXT_CP path in %TIBCO_HOME%\
bw\<version>\bin\bwengine.tra

These symptoms usually indicate that, when starting up, the TIBCO BusinessWorks process engine could not load the Windchill ESI business logic properties files. These properties files contain the cross-reference and default values used in data mapping, texts used in logging, etc. There are several possible causes for this issue:

1. The properties files are not properly installed in the <TIBCO HOME>\esi\bin directory.

2. The FilesToRead_ORA.properties file specifies an invalid value for the Path variable. The Path variable must point to the place where the files are located. This value must contain a slash at the end of the path name. For example: C:/tibco/esi/bin (Windows) or /opt/tibco/esi/bin (UNIX).

3. The properties files' security authorizations do not allow read access to the TIBCO BusinessWorks process engine.

To resolve this issue:

1. If the FilesToRead_ORA.properties file specifies an invalid value for the Path variable, enter in the correct value, save the file, and then restart the TIBCO BusinessWorks process engine.

2. If the properties files have restricted read access, grant this access to the TIBCO BusinessWorks process engine. See theWindchill Enterprise Systems Integration Installation and Configuration - Oracle Applications section for further details on configuring the Windchill ESI business logic properties files and specifying class paths during deployment.

The TIBCO BusinessWorks process engine does not connect to the TIBCO EMS server

This could happen if the BusinessWorks engine was started before the EMS server.

To avoid this problem always start the EMS server before the BusinessWorks engine. For further information on detecting this problem, see topic JMS Problems.

On a Windows server, Adbagent fails to start with the following error: The ordinal 3823 could not be located in dynamic link library LIBEAY32.dll

To resolve this issue run the following commands:

1. MOVE /Y <Tibco_Home>/adapter/sdk/<version>/bin/libeay32.dll <Tibco_Home>/adapter/sdk/5.5/bin/libeay32_bk.dll

2. MOVE /Y <Tibco_Home>/adapter/sdk/<version>/bin/ssleay32.dll <Tibco_Home>/adapter/sdk/5.5/bin/ssleay32_bk.dll

3. COPY /Y <Tibco_Home>/tibrv/<version>/bin/libeay32.dll <Tibco_Home>/adapter/sdk/<version>/bin/libeay32.dll

4. COPY /Y <Tibco_Home>/tibrv/<version>/bin/ssleay32.dll <Tibco_Home>/adapter/sdk/<version>/bin/ssleay32.dll

Windchill ESI fails to publish any business objects to the distribution target system

Windchill ESI fails to publish any business objects to the distribution target system. Errors include overall business object failures, "missing a required field" errors, etc. In addition, the TIBCO BusinessWorks Process engine log does not contain any Windchill ESI message text, as shown in the following example:

2003 Aug 11 18:55:31:722 GMT -4 Engine ESI [] PE-ESI Job-15000
[ProcessDefinitions/Services/Logging_Service/WriteLog_ESIMaster
]: ,98,,3,1,0,40012,,,, 2003 Aug 11 18:55:32:563 GMT -4 Engine
ESI [] PE-ESI Job-15000
[ProcessDefinitions/Services/Logging_Service/WriteLog_ESIMaster
]: ,98,,3,1,0,40013,,,, 2003 Aug 11 18:55:32:864 GMT -4 Engine
ESI [] PE-ESI Job-15000
[ProcessDefinitions/Services/Logging_Service/WriteLog_ESIMaster
]: ,98,,3,1,0,40016,,,,

1. The properties files are not properly installed in the <TIBCO HOME>\esi\bin directory.

3. The properties files' security authorizations do not allow read access to the TIBCO BusinessWorks process engine.

To resolve this problem verify the following:

1. If the FilesToRead_ORA.properties file specifies an invalid value for the Path variable, enter in the correct value, save the file, and then restart the TIBCO BusinessWorks process engine.

TIBCO BusinessWorks indicates an error similar to one of the following

TIBCO BusinessWorks indicates an error similar to one of the following:

• ORA-01461 can bind a LONG value only for insert into a LONG column

• [DataDirect][ODBC Oracle driver]String data, right truncated. Error in parameter

This could happen if you have attempted to map a field which has a longer field length in Windchill PDMLink than is allowed in Oracle Applications. The most likely fields having this issue are CN number and Reference Designator.

Windchill PDMLink should be modified or business processes should be in place to prevent CN Numbers or Reference Designators over 10 characters in length. Refer to the Windchill Enterprise Systems Integration Installation and Configuration - Oracle Applications section for further details on managing field length differences between systems.

TIBCO Administrator fails with an "Apache/Tomcat 404" Web browser error

This could happen if you are using an unsupported web browser version.

To resolve this problem upgrade your web browser to Microsoft Internet Explorer version 6.0 SP1, or later.

The TIBCO Adapter for Oracle Applications log contains error messages

The TIBCO Adapter for Oracle Applications log contains error messages similar to the following:

2004 Mar 11 14:43:20:226 GMT -5
BOMConfiguration.BOMConfiguration Error [Adapter] AEADB-700071
Duplicate agent found on DEV01MACHINE.mydomain.mycompany.com
(132.253.82.60)2004 Mar 11 14:43:20:226 GMT -5
MasterConfiguration.MasterConfiguration Info [Adapter] AEADB-
700095 Terminating agent

This could happen if multiple developers are using the same adapter deployment configuration names on the same subnet. Differentiating adapter deployment configurations by their domain is insufficient.

To avoid this problem ensure that all adapter deployment configuration names are unique across the subnet.

When starting an adapter instance or process engine in TIBCO Administrator, the status remains stuck on "Starting Up"

When starting an adapter instance or process engine in TIBCO Administrator, the status remains stuck on "Starting Up" and never changes to "Started".

This is a known issue with the TIBCO Adapter for Oracle Applications and the TIBCO Runtime Agent (TRA). It is possible that starting the TIBCO services automatically (as is the default on Windows systems) may cause them to sometimes start in the wrong sequence and prevent proper communication between the Administrator Server and TRA.

Check the component's log file to determine if it has, in fact, started successfully, despite the "Starting Up" status display. In most cases, it probably has. If not, try the following procedure:

1. Shut down the TIBCO Administrator application and all running TIBCO components and services.

2. Start up the TIBCO services manually, in the following order:

a. TIBCO Administrator <version> (Domain Name)

b. TIBCO HAWK Agent (Domain Name)

3. Restart the TIBCO Administrator application and try to start up the components again.

Multiple "Activity timed out" errors appear in the TIBCO BusinessWorks process engine log

Multiple "Activity timed out" errors appear in the TIBCO BusinessWorks process engine log; however, Windchill ESI reports a successful result in the Enterprise Systems Transaction Log Graphical User Interface.

This is an expected behavior. Windchill ESI automatically compensates for certain timeout events with subsequent Oracle Applications queries, and processing proceeds normally.

No action is required.

The Distribution Targets table is not displayed in the properties page of custom parts

The Distribution Targets table is not displayed in the properties page of custom parts like 'wt.wadm.FADProduct' after the targets are created in the database.

This could happen if the default version of the file <Windchill>\codebase\netmarkets\jsp\tgt\distributionList.jsp is not designed to display the distribution targets table for custom parts.

To enable the Distribution Targets table for custom parts such as wt.wadm.FADProduct:

1. Open the file: <Windchill>\codebase\netmarkets\jsp\tgt\distributionList.jsp

2. Modify the if statement as follows, by adding the custom part type.

if (oid.indexOf("wt.doc") != -1 ||

oid.indexOf("wt.epm") != -1 ||

oid.indexOf("wt.part") != -1 ||

oid.indexOf("com.ptc.windchill.mpml.processplan.MPMProcessPlan") != -1 ||

oid.indexOf("com.ptc.windchill.mpml.resource.MPMProcessMaterial") != -1 ||

oid.indexOf("com.ptc.windchill.mpml.resource.MPMTooling") != -1 ||

oid.indexOf("com.ptc.windchill.mpml.resource.MPMSkill") != -1 ||

oid.indexOf("wt.wadm.FADProducts") != -1)

3. Save the file and restart servlet engine.