This section provides you with:

As a Windchill ESI Administrator you are alerted to problems through reporting channels such as user reports which includes trouble tickets logged via a help desk or through e-mails and through automated system alerts.

User reports are a passive, reactive means of detecting problems. To improve Windchill ESI system stability and lower the overall cost-of-ownership, it is recommended that you supplement user reports with automated system alerts which is a proactive approach that can prevent problems before they become critical.

The following describe how to monitor problems using the following:

TIBCO BusinessWorks is used by the Windchill ESI Error Handling process to monitor the Windchill EAI software components and to handle system problems. TIBCO Administrator provides you with a Graphical User Interface and microagents to monitor and maintain hardware and software components within TIBCO administrative domains. These tools can be configured to monitor BusinessWorks process engines, adapters, log file entries, available disk space, operating system parameters, and more. When defining a deployment configuration, TIBCO Administrator provides facilities to create alert and escalation rules triggered by predefined events, such as a component failure, suspended process, or log event.

You can use BusinessWorks to monitor the following:

TIBCO BusinessWorks engines need to be running for every process used in Windchill ESI. Therefore the BusinessWorks monitoring component must be able to do the following:

In a deployment, failures can be separated into the following categories:

The following lists suggested actions that you can configure during deployment, for the various types of failures:

A configurable counter and timer exists which determines when to reset the failure count to first. Use the timer setting on this counter to set to a particular time frame in which more than two failures in that time frame raises considerable concern about the system’s overall integrity.

A systems administrator has the ability to restart the engine using TIBCO BusinessWorks Administrator. For more information, see topic JMS Problems below.

The adapter’s distribution target system needs to be running for many processes used in Windchill ESI. Therefore, the TIBCO BusinessWorks monitoring component must be able to do the following:

In a deployment, failures can be separated into the following categories

The following lists suggested actions that you can configure during deployment, for the various types of failure:

A configurable counter and timer exists which determines when to reset the failure count to first. Use the timer setting on this counter to set to a particular time frame in which more than two failures in that time frame raises considerable concern about the system’s overall integrity.

An EMS Server needs to be running for the master process flow and error handling processes to work properly. The TIBCO Administrator must be able to do the following:

Unfortunately, TIBCO BusinessWorks does not create an event when EMS servers fail. To combat this, you can do one of the following:

By default, Windchill ESI is configured for the suspend approach, as this allows for the process that caught the JMS queue failure to continue after restarting the EMS server.

Windchill PDMLink allows using a single JMS provider at a time. Therefore, the TIBCO EMS server becomes an integral part of the Windchill ESI architecture, and Windchill PDMLink users may use it for functions other than Windchill ESI applications. Therefore, it may not be prudent to configure Windchill ESI to automatically restart the EMS server when there is a problem with Windchill ESI that is related to a JMS problem. Rather, the default and recommended configuration is to have the error-handling process place the current BusinessWorks job in suspend mode when it detects a serious JMS problem.

Suspend mode allows you to manually intervene and does not pose a risk of data loss. Only the affected BusinessWorks job - that is, the product data transaction that is being published - is suspended while other jobs in the same process engine may continue. If other jobs encounter the same JMS problem, they get individually suspended as well. You can individually restart jobs via the TIBCO BusinessWorks Administrator. Jobs get resumed from the point of suspension, not from the last checkpoint.

TIBCO Administrator does not provide built-in administration domain monitoring of TIBCO EMS. You may, however, configure TIBCO BusinessWorks to issue alerts in the event that the EMS server is suspended or requires restarting. End-users can use the basic administration console in TIBCO EMS to configure TIBCO BusinessWorks to issue alerts to in the event that the EMS server is suspended or requires to be restarted. For more information, see topic Configuring E-Mail Alerts below.

The TIBCO Adapter for Oracle Applications log files can report problems such as an invalid Oracle ApplicationsWindchill ESI user account. These situations generally occur because of problems such as typographical errors made during installation or passwords expiring after a predetermined period. These can be unexpected and difficult to diagnose. Therefore, you may wish to configure TIBCO BusinessWorks to monitor these log files for the relevant message text and to issue alerts should the event occur.

In addition to configuring alerts for problems with the underlying TIBCO products, you can also use the error handling and logging shared services to detect and pinpoint Windchill ESI problems.

Since event rules are closely tied to hardware deployment, it is not feasible for Windchill ESI to provide a predefined, ready-to-use configuration. However, you can develop and adjust event rules as needed, over time, in a non-invasive manner to the underlying TIBCO products and Windchill ESI business logic component. You can prioritize developing rules based on the most common or troublesome issues you have encountered, and thus gradually, move from a reactive to a proactive problem-detection approach.

After detecting a problem that cannot be corrected automatically or by the user, you need to begin diagnosing the problem. This involves categorizing and localizing the problem to determine its root cause.

To localize the source of the problem, you need to ask questions such as:

To categorize problems, you need to focus on key problem areas and get familiar with error handling reports such as error logs and error handling codes.

Most system-related, technical issues can be categorized according to the location of the root cause. Refer to the Administering Windchill ESI in an ORACLE Applications Environment and applicable documentation provided by your systems integrator, as necessary, to verify and restore correct configuration settings.

It is also important for you to become familiar with the business process and functional troubleshooting information. Users who are not familiar with this information may escalate such issues to you.

The following categories of problems and their descriptions are not intended to be exhaustive with detailed step-by-step procedures. Rather, they are provided to help you to focus on some of the key or potential root causes of technical issues:

The following lists how to deal with problems that may originate from Windchill ESI services:

The following list how to deal with problems that may originate from TIBCO components:

The following lists how to deal with problems that may originate in an Oracle Applications distribution target. Refer to the Administering Windchill ESI in an ORACLE Applications Environment for more details.

The following lists how to deal with problems that may originate in an Oracle Applications distribution target. Refer to the Administering Windchill ESI in an ORACLE Applications Environment for more details.

To help diagnose problems, you need to be thoroughly familiar with the key Windchill ESI logs, described in the Administering Windchill ESI in an ORACLE Applications Environment. You also need to be familiar with the EAI error-handling architecture, and the error codes. The following lists the available logs and how to access them:

After you have monitored, detected, and diagnosed problems, you need to resolve them. The following lists some general techniques you can use when troubleshooting as well as a list of specific problems with their solutions.

Following lists some of the troubleshooting techniques you can use to resolve problems.

As a Windchill ESI administrator, you may have to involve and coordinate a wide variety of specialists to fully resolve production issues. These parties may include end-users, functional experts, Windchill PDMLink administrators, Oracle Applications administrators, database specialists, operating system specialists, network administrators, and others. If, after diagnosing and localizing an issue, you determine that the problem cannot be solved in-house, you may need to escalate it to systems integration partners or to PTC Technical Support, or both, for assistance. Systems integration partners may be particularly critical to help you resolve problems involving special configuration or customizations made to the standard Windchill ESI product.

The Windchill ESI application leverages checkpoint activities at key points in the TIBCO BusinessWorks processing flow. A checkpoint saves the process data and state of the current running process instance so that it can be recovered at a later time in the event of a failure. If a process engine fails, all process instances can be recovered and can resume execution at the location of their last checkpoint in the process definition. Only the most recent state is saved by a checkpoint. If you have multiple checkpoints in a process, only the state from the last checkpoint is available for recovering the process.

The location of Windchill ESI checkpoints has been strategically designed to ensure robust transaction integrity while minimizing adverse performance effects. Occasionally, you may need to manually override an activated checkpoint to restart transaction processing from scratch when the process engine is restarted.

However, if circumstances warrant it, you can override checkpoints by deleting the <Tibco_Home>/tra/domain/<Domain_Name>/application/<Application_Name>/working directory and its contents. Depending on the process engine mode, there may be many other /working subdirectories under the /tibco path that would need to be deleted. Do a search for /working to locate and remove all of these subdirectories.

On Solaris platforms you may have to resolve problems that arise from:

The following describe techniques to resolve these.

On various Unix platforms certain TIBCO components fail to launch because of library dependencies. If there are any *.sh or *.csh files present in the bin directory of that particular component, they should be executed and then the ‘ldd’ command can be used to identify library dependencies. All the required library paths should be added to the tibco.env.CUSTOM_EXT_PREPEND variable present in the *.tra file.

Occasionally, the TIBCO Administrator and the TIBCO Runtime Agent (TRA) become corrupted and without warning, shut down on the Solaris platforms. If this occurs while TIBCO Designer is running, deployment configurations and TIBCO Designer may behave strangely, for example:

If any of these symptoms occur, reboot the Solaris machine, restart the TRA and TIBCO Administrator, and then restart Designer.

Occasionally, as part of resolving problems you may need to kill all running TIBCO processes.

On UNIX platforms, you can use the UNIX command ‘ps’ to list all the processes that are running. However, the standard ‘all’ flag, ‘-a’, does not necessarily work to list low-level TIBCO processes such as TIBCO Rendezvous. To view such processes, you can execute the following command:

For example:

The above results in several lines that look like this:

You can force-kill the processes listed by using the ‘kill’ command with the flag -9 and the process ID (the second column from your ‘ps -ef’ command)

Using the -9 flag ensures the process is really dead. Generally speaking, to ensure all TIBCO processes are dead, you should run

and

and verify that no running processes are found.

As mentioned earlier, Windchill ESI can be configured to send you e-mail alert messages when the Windchill ESI business logic experiences an error:

This configuration is performed through the following global variables:

The server is configured using the global variable: ESIMail/SMTPHostServer.

Refer to theAdministering Windchill ESI in an ORACLE Applications Environment for details on how to configure the global variables.

The Windchill ESI business logic attempts to retry any EMS-related actions the number of times specified by the TIBCO BusinessWorks global variable, ESIJMS/RetryCount. Therefore, you may receive multiple e-mail messages for the same issue. After a final unsuccessful attempt, the Windchill ESI business logic suspends the associated BusinessWorks process, allowing you to examine the errors described in the e-mail alert messages, fix the underlying problems, and resume or kill the BusinessWorks process using TIBCO Administrator. Refer to the TIBCO Administrator User's Guide for details on how to use this interface.

The following section lists specific problems, provides possible causes, and suggests solutions to resolve them.

Occasionally, as part of resolving problems you may need to kill all running TIBCO processes.

On UNIX platforms, you can use the UNIX command ‘ps’ to list all the processes that are running. However, the standard ‘all’ flag, ‘-a’, does not necessarily work to list low-level TIBCO processes such as TIBCO Rendezvous. To view such processes, you can execute the following command:

For example:

The above results in several lines that look like this:

You can force-kill the processes listed by using the ‘kill’ command with the flag -9 and the process ID (the second column from your ‘ps -ef’ command)

Using the -9 flag ensures the process is really dead. Generally speaking, to ensure all TIBCO processes are dead, you should run

and

and verify that no running processes are found.

As mentioned earlier, Windchill ESI can be configured to send you e-mail alert messages when the Windchill ESI business logic experiences an error:

This configuration is performed through the following global variables:

The server is configured using the global variable: ESIMail/SMTPHostServer.

Refer to theAdministering Windchill ESI in an ORACLE Applications Environment for details on how to configure the global variables.

The Windchill ESI business logic attempts to retry any EMS-related actions the number of times specified by the TIBCO BusinessWorks global variable, ESIJMS/RetryCount. Therefore, you may receive multiple e-mail messages for the same issue. After a final unsuccessful attempt, the Windchill ESI business logic suspends the associated BusinessWorks process, allowing you to examine the errors described in the e-mail alert messages, fix the underlying problems, and resume or kill the BusinessWorks process using TIBCO Administrator. Refer to the TIBCO Administrator User's Guide for details on how to use this interface.

The following section lists specific problems, provides possible causes, and suggests solutions to resolve them.

There are two possible causes for this issue:

After installing TIBCO Runtime Agent <version> and TIBCO Runtime Agent <version>.1, TIBCO Adapter projects using Enterprise Message Service as transport will not start up from TIBCO Designer and give the following error:

This could be due to incompatible Adapter SDK libraries. To resolve this issue:

After installing TIBCO Runtime Agent <version> and TIBCO Runtime Agent <version>.1, TIBCO Adapter projects using Enterprise Message Service as transport will not start up from TIBCO Designer and give the following error:

This could be due to incompatible Adapter SDK libraries. To resolve this issue:

This could be due to incorrect values for ESIOthers/WSHost and ESIOthers/WSPort variables. Providing the proper values should resolve this issue.

This could be because the command to start the EMS server has been changed in the 5.1.4 version. In EMS version 4.x the command to start EMS was "./tibemsd". In EMS v5.1.4 the command is "./tibemsd64 -config ../tibco/cfgmgmt/ems/data/tibemsd.conf". The command uses relative path and it should be executed from "<TIBCO_HOME>\ems\<version>\bin".

To resolve this problem stop the process started by the command:

and start the ems server with correct command:

This could happen if any previous deployment or un-deployment accidentally acquires a lock and has not released it.

To resolve this issue run the AppManage command from command prompt to diagnose and resolve the problem accordingly:

For Example:

For more information refer to the <Tibco_Home>\tra\domain\<Domain_Name>\logs\ApplicationManagement.log file.

TIBCO BusinessWorks returns error messages such as the following:

This could happen if the EMS server name is invalid.

To resolve this problem confirm that the ESIJMS/JNDIContextURL global variable in the deployment configuration of the process engine matches the value in the QueueConnectionFactory URL, in the factories.conf file located in <Tibco_Home>/ems/<version>/tibco/cfgmgmt/ems/data

The ESIJMS/JNDIContextURL global variable should be set to:

tibjmsnaming://<machinename>:<port>

and the QueueConnectionFactory URL should be set to:

tcp://<machine name>:<port>.

The machine name and port values should match.

For example, if ESIJMS/JNDIContextURL is set to tibjmsnaming://mymachine.mycompany.com:7222, then the QueueConnectionFactory URL should be set to tcp://mymachine.mycompany.com:7222.

Refer to “Global Variables for Process Engines” theAdministering Windchill ESI in an ORACLE Applications Environment for details on this variable.

BusinessWorks returns error messages such as the following:

This could happen if the user name and password for the JMS server is invalid.

To resolve this issue confirm that the ESIJMS/Username and ESIJMS/Password global variables in the deployment configuration of the process engine match the values of the username and password specified on the EMS server for the queues.

The username and password values set for the EMS server is in the users.conf file which is located in the <Tibco_Home>/ems/<version>/tibco/cfgmgmt/ems/data directory. Refer to "Global Variables for Process Engines" in theAdministering Windchill ESI in an ORACLE Applications Environment for details on this variable.

The passwords are obfuscated if they were set using the EMS Administration tool. If you do not remember the passwords you set, you will have to reset them.

Upon starting a TIBCO component on UNIX, you receive the following errors:

For example,

This could happen if permissions for the file have not been set properly.

To resolve this issue use the command chmod 555 on the file listed after TEXT. For example,

This could happen if the EMS server is not configured properly. When you specify the name of the EMS server as localhost, that server is only recognized on the box that it is running on. No other machine can connect to it. An application that is set to localhost will attempt to find the EMS server running on the same machine. If it is not found, it will error. When you specify a machine name as your server name, other machines can connect to your EMS server.

To resolve this issue check that the QueueConnectionFactory value located in the <Tibco_Home>/ems/<version>/tibco/cfgmgmt/ems/data/factories.conf file and the global variable ESIJMS/JNDIContextURL in the deployment of the Process Engine are set accordingly.

It does not matter where this EMS server resides. It can reside on the same machine as Windchill PDMLink, same machine as the TIBCO process engine, or a different machine altogether. As long as the above values described are set appropriately (and the machines are on the same network), Windchill ESI and EAI components are able to connect to the correct EMS server.

To determine which machine and username is connected to an EMS server, type the following command in the TIBCO EMS administration tool:

>show connections

This gives you a list of which users are connected and from which machine.

This could happen if the JMS server queues are not subscribed to as described in Configuring and Administering JMS Queues.

Common root causes are: the JMS server is not running when the Windchill method server starts, misconfigurations in the JMS properties of the Windchill adapter, or misconfigurations in the JMS server.

When Windchill ESI services are successful in subscribing to the JMS queue meant for receiving result messages, the following information is included in the Windchill PDMLink method server log:

If this information does not appear in the log, it means Windchill was unable to subscribe to the said queue successfully. In this case, all Windchill ESI transactions are left in a pending state. The transactions are processed once Windchill is able to subscribe to the queue successfully.

Normally, when Windchill ESI services are unable to subscribe to the queue because the JMS server is unavailable, Info*Engine receives an exception. Windchill ESI services log this exception in the Windchill Adapter transaction log. If TIBCO EMS is the JMS provider, the message contains the following text:

As a detection measure, you may consider configuring your monitoring software to look for this or similar text. To resolve the issue, make sure the EMS server is up before the method server, and resolve any issues with the Windchill adapter JMS configuration and the JMS server configuration.

To avoid this issue, ensure the following:

TIBCO BusinessWorks Process engine log contains the following message text:

These symptoms usually indicate that, when starting up, the TIBCO BusinessWorks process engine could not load the Windchill ESI business logic properties files. These properties files contain the cross-reference and default values used in data mapping, texts used in logging, etc. There are several possible causes for this issue:

To resolve this issue:

This could happen if the BusinessWorks engine was started before the EMS server.

To avoid this problem always start the EMS server before the BusinessWorks engine. For further information on detecting this problem, see topic JMS Problems.

To resolve this issue run the following commands:

Windchill ESI fails to publish any business objects to the distribution target system. Errors include overall business object failures, "missing a required field" errors, etc. In addition, the TIBCO BusinessWorks Process engine log does not contain any Windchill ESI message text, as shown in the following example:

These symptoms usually indicate that, when starting up, the TIBCO BusinessWorks process engine could not load the Windchill ESI business logic properties files. These properties files contain the cross-reference and default values used in data mapping, texts used in logging, etc. There are several possible causes for this issue:

To resolve this problem verify the following:

TIBCO BusinessWorks indicates an error similar to one of the following:

This could happen if you have attempted to map a field which has a longer field length in Windchill PDMLink than is allowed in Oracle Applications. The most likely fields having this issue are CN number and Reference Designator.

Windchill PDMLink should be modified or business processes should be in place to prevent CN Numbers or Reference Designators over 10 characters in length. Refer to the Administering Windchill ESI in an ORACLE Applications Environment for further details on managing field length differences between systems.

This could happen if you are using an unsupported web browser version.

To resolve this problem upgrade your web browser to Microsoft Internet Explorer version 6.0 SP1, or later.

The TIBCO Adapter for Oracle Applications log contains error messages similar to the following:

This could happen if multiple developers are using the same adapter deployment configuration names on the same subnet. Differentiating adapter deployment configurations by their domain is insufficient.

To avoid this problem ensure that all adapter deployment configuration names are unique across the subnet.

When starting an adapter instance or process engine in TIBCO Administrator, the status remains stuck on "Starting Up" and never changes to "Started".

This is a known issue with the TIBCO Adapter for Oracle Applications and the TIBCO Runtime Agent (TRA). It is possible that starting the TIBCO services automatically (as is the default on Windows systems) may cause them to sometimes start in the wrong sequence and prevent proper communication between the Administrator Server and TRA.

Check the component's log file to determine if it has, in fact, started successfully, despite the "Starting Up" status display. In most cases, it probably has. If not, try the following procedure:

Multiple "Activity timed out" errors appear in the TIBCO BusinessWorks process engine log; however, Windchill ESI reports a successful result in the Enterprise Systems Transaction Log Graphical User Interface.

This is an expected behavior. Windchill ESI automatically compensates for certain timeout events with subsequent Oracle Applications queries, and processing proceeds normally.

No action is required.

The Distribution Targets table is not displayed in the properties page of custom parts like 'wt.wadm.FADProduct' after the targets are created in the database.

This could happen if the default version of the file <Windchill>\codebase\netmarkets\jsp\tgt\distributionList.jsp is not designed to display the distribution targets table for custom parts.

To enable the Distribution Targets table for custom parts such as wt.wadm.FADProduct:

Verify the following settings while manually deploying the EAR:

Now deploy the application, but do not start the services.

EMS: Run the following commands in the JMS Administration window to configure JMS. Replace <DataResponse> and <EAI User> with the applicable values. For example, com.ptc.windchill.esi.DataResponse.ORAPTCPROD and ESISYS and so on.

Now start the services.

Verify the following:

If no logs are generated for service startup and process id shows "-1", try running the corresponding *.sh or *.cmd file from <Tibco_Home>/tra/domain/<Domain_Name>/application/<Application_Name> folder. This would indicate any dependency failures, which should be addressed before starting the service.

This happens if the Windchill ESI preference Publish Promotion Requests has a value No. Set the preference to Yes in order that a single RTM workflow is created upon releasing a promotion request.