Guidelines for Monitoring, Diagnosing, and Resolving Problems
This section provides you with:
• General guidelines on how to monitor and diagnose problems and lists some specific techniques you can use to resolve them.
• A listing of specific problems, their causes, and resolutions.
Monitoring Problems
As a Windchill ESI Administrator you are alerted to problems through reporting channels such as user reports which includes trouble tickets logged via a help desk or through emails and through automated system alerts.
User reports are a passive, reactive means of detecting problems. To improve Windchill ESI system stability and lower the overall cost-of-ownership, it is recommended that you supplement user reports with automated system alerts which is a proactive approach that can prevent problems before they become critical.
The following describe how to monitor problems using the following:
• TIBCO BusinessWorks monitoring services
• Error handling processes and logging services
• Event rules and problem-detection approaches
Monitoring with BusinessWorks
TIBCO Administrator is used by the Windchill ESI Error Handling process to monitor the Windchill EAI software components and to handle system problems. TIBCO Administrator provides you with a Graphical User Interface and microagents to monitor and maintain hardware and software components within TIBCO administrative domains. These tools can be configured to monitor BusinessWorks process engines, adapters, log file entries, available disk space, operating system parameters, and more. When defining a deployment configuration, TIBCO Administrator provides facilities to create alert and escalation rules triggered by predefined events, such as a component failure, suspended process, or log event.
You can use BusinessWorks to monitor the following:
• BusinessWorks engine problems
• Adapter problems
• JMS problems
BusinessWorks Engine Problems
TIBCO BusinessWorks engines need to be running for every process used in Windchill ESI. Therefore the BusinessWorks monitoring component must be able to do the following:
• Handle events raised when a failure of a BusinessWorks engine causes messages to be lost or for processing to stop
• Send alerts to TIBCO Administrator monitoring when failures occur
• Escalate actions as failure frequency increases
Failure Categories
In a deployment, failures can be separated into the following categories:
• Any Failure: Catches any failure and performs an action
• First Failure: Catches the first failure of an engine and performs an action
• Second Failure: Catches the second failure of an engine and performs an action
• Subsequent Failure: Catches any failure that occurs subsequent to the second failure and performs an action.
Suggested Deployment Configuration
The following lists suggested actions that you can configure during deployment, for the various types of failures:
• Any Failure: Raise an alert to the administrator
• First Failure: Restart the engine
• Second Failure: Restart the engine and raise a second failure alert
• Subsequent Failure: Restart the engine and send email to the administrator
A configurable counter and timer exists which determines when to reset the failure count to first. Use the timer setting on this counter to set to a particular time frame in which more than two failures in that time frame raises considerable concern about the system's overall integrity. The following screen shots show examples of the suggested deployment.
1. Navigate down through the TIBCO Administrator GUI:
◦ Application Management > <Application Name> > Configuration > Process Archive.par:
2. Click the Monitoring tab:
3. Click the Add button under Events. The Add Event screen will appear:
4. Select the type of event you would like to monitor. For example, when a JMS queue fails, a process becomes suspended. You may configure an alert to monitor for a process suspension:
A systems administrator has the ability to restart the engine using TIBCO BusinessWorks Administrator. For more information, see the JMS Problems section.
Adapter Problems
The adapter's distribution target system needs to be running for many processes used in Windchill ESI. Therefore, the TIBCO BusinessWorks monitoring component must be able to do the following:
• Restart adapter instances when they fail
• Send alerts to TIBCO Administrator when failures occur
• Escalate actions as failure frequency increases
Failure Categories
In a deployment, failures can be separated into the following categories
• Any Failure: Catches any failure and performs an action
• First Failure: Catches the first failure of an adapter and performs an action
• Second Failure: Catches the second failure of an adapter and performs an action
• Subsequent Failure: Catches any failure that occurs subsequent to the second failure and performs an action.
JMS Problems
An EMS Server needs to be running for the master process flow and error handling processes to work properly. The TIBCO Administrator must be able to do the following:
• Handle events raised when an EMS Server failure occurs
• Send errors to the error handling process when a failure occurs
Unfortunately, TIBCO BusinessWorks does not create an event when EMS servers fail. To combat this, you can do one of the following:
• No suspend: When a JMS queue does not respond or cannot be connected to at runtime after a repeat-on-error-until-true group with the suspend checkbox option not selected, BusinessWorks creates a process engine log entry indicating a timeout of the JMS activity. You could configure a log event in TIBCO Administrator to monitor the log for this and raise an alert.
• Suspend: When a JMS queue does not respond or cannot be connected to at runtime after a repeat-on-error-until-true group with the suspend checkbox option selected, BusinessWorks suspends the process and you can configure a TIBCO Administrator rule base to raise an alert.
By default, Windchill ESI is configured for the suspend approach, as this allows for the process that caught the JMS queue failure to continue after restarting the EMS server.
Special Guidelines for EMS Server Problems
Windchill PDMLink allows a single JMS provider at a time. Therefore, the TIBCO EMS server becomes an integral part of the Windchill ESI architecture, and Windchill PDMLink users may use it for functions other than Windchill ESI applications. Therefore, it may not be prudent to configure Windchill ESI to automatically restart the EMS server when there is a problem with Windchill ESI that is related to a JMS problem. Rather, the default and recommended configuration is to have the error-handling process place the current BusinessWorks job in suspend mode when it detects a serious JMS problem.
Suspend mode allows you to manually intervene and does not pose a risk of data loss. Only the affected BusinessWorks job - that is, the product data transaction that is being published - is suspended while other jobs in the same process engine may continue. If other jobs encounter the same JMS problem, they get individually suspended as well. You can individually restart jobs via the TIBCO BusinessWorks Administrator. Jobs get resumed from the point of suspension, not from the last checkpoint.
TIBCO Administrator does not provide built-in administration domain monitoring of TIBCO EMS. You may, however, configure TIBCO BusinessWorks to issue alerts in the event that the EMS server is suspended or requires restarting. End-users can use the basic administration console in TIBCO EMS to configure TIBCO BusinessWorks to issue alerts to in the event that the EMS server is suspended or requires to be restarted. For more information, see Configuring E-Mail Alerts section.
SAP Problems
The TIBCO Adapter for SAP log files can report problems such as an invalid or locked SAP ESI user account. These situations generally occur because of problems such as typographical errors made during installation or passwords expiring after a predetermined period; these can be unexpected and difficult to diagnose. Therefore, you may wish to configure TIBCO BusinessWorks to monitor these log files for the relevant message text and to issue alerts should the event occur.
Using Error Handling Processes and Logging Services
In addition to configuring alerts for problems with the underlying TIBCO products, you can also use the error handling and logging shared services to detect and pinpoint Windchill ESI problems as described in the section Logs and Error Handling Codes.
Developing Event Rules and Problem-Detection Approaches
Since event rules are closely tied to hardware deployment, it is not feasible for Windchill ESI to provide a predefined, ready-to-use configuration. However, you can develop and adjust event rules as needed, over time, in a non-invasive manner to the underlying TIBCO products and Windchill ESI business logic component. You can prioritize developing rules based on the most common or troublesome issues you have encountered, and thus gradually, move from a reactive to a proactive problem-detection approach.
Diagnosing Problems
After detecting a problem that cannot be corrected automatically or by the user, you need to begin diagnosing the problem. This involves categorizing and localizing the problem to determine its root cause.
Localizing Problems
To localize the source of the problem, you need to ask questions such as:
• Is the problem associated with a business process issue such as a system-of-record violation, a functional issue such as invalid data, or a technical issue such as a server being down?
• Is the problem associated with Windchill PDMLink, TIBCO, or the distribution target ERP system?
• If it is a problem related to TIBCO, is it associated with TIBCO EMS, BusinessWorks, or Adapter for SAP?
• Is the problem associated with the BusinessWorks deployment and/or run-time agents?
• Is the problem associated with the underlying physical network and computing, rather than Windchill ESI?
• Can the problem scenario be duplicated in a test system with the same configuration as the production environment?
Categorizing Problems: Key Areas of Focus for Troubleshooting
To categorize problems, you need to focus on key problem areas and get familiar with error handling reports such as error logs and error handling codes.
Most system-related, technical issues can be categorized according to the location of the root cause. Refer to theWindchill Enterprise Systems Integration Installation and Configuration Guide - SAP and applicable documentation provided by your systems integrator, as necessary, to verify and restore correct configuration settings.
It is also important for you to become familiar with the business process and functional troubleshooting information. Users who are not familiar with this information may escalate such issues to you.
The following categories of problems and their descriptions are not intended to be exhaustive with detailed step-by-step procedures. Rather, they are provided to help you to focus on some of the key or potential root causes of technical issues:
• Problems originating from Windchill ESI
• Problems originating from TIBCO components such as:
◦ EMS
◦ BusinessWorks
◦ Adapter for SAP
• Problems originating from the distribution target.
• Problems indicated in Windchill ESI logs.
Windchill ESI Problems
The following lists how to deal with problems that may originate from Windchill ESI services:
• Verify that the Windchill server and application are running
• Verify that the Windchill JMS client is connecting to the TIBCO EMS server
• Verify that the Windchill ESI user account for the EMS server is correctly configured
• Verify all other JMS-related configurations
• Scan for error messages in the Windchill administrative logs
• Validate the distribution targets and their attributes for the given release
• Verify correct operation of the Windchill Enterprise Integration Access Remote Procedure Calls (RPCs)
• Verify the correctness of the values set for the various Windchill ESI preferences
TIBCO Problems
The following lists how to deal with problems that may originate from TIBCO components:
EMS Problems
• Verify that the EMS server is running
• Verify that the EMS server is correctly configured
• Ensure that the required Windchill ESI JMS queues are defined
• Ensure that there is one and only one listener per queue at a given time
• Verify that the BusinessWorks ESI user account for the EMS server is correctly configured
• Verify the secure queue authentication configuration
• Isolate any Java Virtual Machine (JVM) issues, such as unsupported versions or memory overflow errors
BusinessWorks Problems
• Verify that all required TIBCO services are running. Process engines do not start if the following required services are not running:
◦ TIBCO Administrator <version> (Domain name)
◦ TIBCO HAWK Agent (Domain name)
• Verify the existence and contents of required Windchill ESI BusinessWorks application properties and configuration files:
◦ ESIEMailMessageLookups.properties
◦ ESIDefaults.properties
◦ ESIErrorHandlingCodes.properties
◦ ESILookups.properties
◦ ESIMessageLookups.properties
◦ FilesToRead_SAP.properties
• Verify that the BusinessWorks JMS client is connecting to the TIBCO EMS server
• Verify all other EMS-related configuration
• Isolate any Java Virtual Machine (JVM) issues, such as unsupported versions or memory overflow errors
• Verify that the Windchill ESI components are deployed to the correct domain
• Verify the ESI business logic configuration and deployment settings, including the global variable values
|
|
You must restart the process engine for new global variable values to take effect.
|
• Validate the configuration settings in the TRA file
• Validate the BusinessWorks Administrator server configuration by:
◦ Authorizing user accounts
◦ Making sure that multiple repositories with the same Rendezvous configuration (network, service, and daemon) on the same subnet have unique names, regardless of the BusinessWorks administration domain
◦ Ensuring that the server-based repository (.dat) files are not deleted directly. To properly delete a server-based project from an administration domain, follow these steps:
▪ Undeploy the project (this removes all of the .tra and .cmd files)
▪ Stop the administration server service
▪ Delete the contents of <Tibco_Home>/tra/domain/<Domain_Name>/application/<Application_Name>/working folder
▪ Restart the administration server service
|
|
The meaning of <Application_Name> in the above referenced path may be understood from the fact that the deployment name of an application is of the form <Domain_Name>-<Application_Name>.
|
• Isolate any issues with internationalization or locale configuration settings. Check out the following:
◦ com_infoengine_locale attribute in the JMS header of Windchill ESI messages
◦ ESISAPAdapter/Locale global variable
◦ Default and cross-referencing lookup file entries used in data mapping
Adapter for SAP Problems
The following lists how to deal with problems that may originate in the Adapter for SAP.
• Verify that the adapter instance is deployed and running
• Validate adapter deployment configuration settings
• Validate the SAP system connection parameters defined by the global variables Group ESISAPAdapter.
|
|
Adapter timeout errors may be caused by a mismatch between parameters in the adapter configuration and the attributes on the ESITarget object created in Windchill. These values are case-sensitive. Some values, such as ESISAPAdapter/SystemID and ESISAPAdapter/Locale, are not used by the adapter to log on to the SAP system. The adapter does not immediately produce a direct error, but the BusinessWorks process engine will not be able to communicate with the adapter, so a timeout error is likely to occur. If the other values, such as ESISAPAdapter/ApplicationServer, ESISAPAdapter/SystemNumber, ESISAPAdapter/Client, ESISAPAdapter/Username, or ESISAPAdapter/Password are incorrect, the adapter is not able to log on to the SAP system and an error message appears in the TIBCO Adapter for SAP logs.
In addition, if the ESIFlags/isMultiplePE variable is set to true the adapter will ignore the Transaction/Destination value from the ESIResponse and will call the default adapter in the application. If there are dedicated Process Archives per ERP instance, this variable should be set to true.
The structure of ESIJMS/DataResponseQueue should be com.ptc.windchill.esi.DataResponse.<ESISAPAdapter/SystemID>.< ESISAPAdapter/Client>
|
• Validate that the Windchill ESI distribution target (ESITarget attributes) sent from Windchill PDMLink are consistent with the SAP connection parameters defined in the adapter deployment configuration.
• Validate the BusinessWorks domain user name and password (on the Configuration tab of the adapter deployment configuration)
• Validate that the SAP user name and password (on the Custom tab of the adapter deployment configuration) are correct and that the account is not locked in SAP
• Isolate any problems with Internationalization or locale configuration settings
◦ Within each adapter deployment configuration, the ESISAPAdapter/Locale global variable value (on the Custom tab) must be consistent with the Locale Encoding parameter value (on the Advanced tab).
|
|
The ESISAPAdapter/Locale global variable value is used in Rendezvous message subject names to identify the correct target adapter.
|
◦ All locale parameter values must be valid (For example, chosen from the list of acceptable possible values).
• Confirm that the number of adapter connections is compatible with ESI data loads and the distribution target SAP system configuration.
Other Configuration Problems
Verify following settings while manually deploying the EAR:
• DataResponse Queue name: Should have the <SystemID>.<Client> (like PT3.800) appended to it.
• JMS URL in ESIJMS and ESISAPAdapter group: Should have the hostname and not localhost.
• Java Max Heap Size and Java Thread Stack Size settings: These should be more than 512 MB and 512 KB respectively.
• Set > > > > > > to a higher value (for example, 6) and set adr3.locale to UTF8.
Now deploy the application, but do not start the services.
EMS: Run following commands in a JMS Administration window to configure JMS. Replace <DataResponse>, <EAI User> and <WCESI User> with the applicable values; for example com.ptc.windchill.esi.DataResponse.PT2.800, ESISYS and WCESI.
• create queue <DataResponse>
• setprop queue <DataResponse> secure
• grant queue <DataResponse> <EAI User> receive
• grant queue <DataResponse> <WC ESI User> send
• setprop factory QueueConnectionFactory url=tcp://<JMSServer>:7222
• commit
Now start the services.
Service Startup Errors
Verify the following:
• <Tibco_Home>/tra/domain/<Domain_Name>/application/logs
If no logs are generated for service startup and process id shows "-1", try running the corresponding *.sh or *.cmd file from <Tibco_Home>/tra/domain/<Domain_Name>/application/<Application_Name> folder. This would indicate any dependency failures, which should be addressed before starting the service.
SAP Problems
The following lists how to deal with problems that may originate in an SAP distribution target. Refer to the Windchill Enterprise Systems Integration Installation and Configuration Guide - SAP for more details.
• Verify that all SAP application and database servers are running
• Verify that the SAP system is running a version that is supported.
• Verify that the SAP system meets the minimum Support Package levels assumed by Windchill ESI
• Verify that the SAP system has applied the specific OSS Notes assumed by Windchill ESI
• Ensure that all APIs assumed by Windchill ESI are available and characterized to Windchill ESI specifications as described in Windchill Enterprise Systems Integration Installation and Configuration Guide - SAP.
• Ensure that all APIs assumed by Windchill ESI are remotely enabled
• Validate the functional configuration assumed by Windchill ESI
• Verify the existence of the ESISYS SAP user account used by the TIBCO Adapter for SAP in all ESITarget systems and clients.
• Verify that the ESISYS user account has the required security authorization profile
• Verify that the ESISYS user account is not locked and does not have an expired password
• Verify that the correct date format setting is defined for the ESISYS user account
• Verify that a sufficient number of SAP gateway connections are available to the TIBCO Adapter for SAP
• Confirm that table record locks are not blocking updates by Windchill ESI.
• Isolate any issues with Internationalization or locale configuration settings.
Logs and Error Handling Codes
To help diagnose problems, you need to be thoroughly familiar with the key Windchill ESI logs, described in the Windchill Enterprise Systems Integration Installation and Configuration Guide - SAP. You also need to be familiar with the EAI error-handling architecture, and the error codes. The following lists the available logs and how to access them:
|
Logs
|
How to Access
|
|
Windchill Enterprise Systems Transaction Log
|
Windchill PDMLink user interface
|
|
TIBCO BusinessWorks Process engine logs
|
|
Windchill ESI application messages appear with a role designation of ESI to distinguish them from the standard TIBCO product messages.
|
|
TIBCO Administrator tracing or <Tibco_Home>/tra/domain/<Domain_Name>/application/logs/<Application Name>.Process_Archive.log[.sequence #]
|
|
TIBCO EMS logs
|
File name and path are determined by log file configuration parameter in the tibemsd.conf file. If the logfile_max_size configuration parameter is not set to zero (0), there may be multiple log files appended with sequence numbers.
|
|
TIBCO Adapter for SAP logs
|
TIBCO Administrator tracing or <Tibco_Home>/tra/domain/<Domain_Name>/application/logs/<Application_Name>.[Adapter Configuration].log[.sequence #]
|
|
TIBCO Adapter for SAP gateway trace.
|
<Tibco_Home>/adapter/adr3/<version>/bin/dev_rfc.trc
|
|
When a TIBCO Adapter for SAP error occurs, TIBCO creates the gateway trace (dev_rfc.trc) file in the <Tibco_Home>/adapter/adr3/<version>/bin directory. The error information in this file is often more detailed and helpful than the information that is provided in the standard adapter logs in the <Tibco_Home>/adapter/adr3/<version>/logs directory.
|
|
|
SAP gateway and application server logs
|
Consult your SAP Basis administrator. For Windchill ESI API call sequences that include CALO_INIT_API (that is, BOMs, CNs, and Part/Material Revisions), you may obtain additional logging information in the SAP application log (transaction SLG1). Refer to SAP online application help for further details on using this transaction.
You should also be familiar with the SAP distribution targets and the data objects and attributes impacted by Windchill ESI. You may have direct responsibility for viewing and manipulating data in SAP, or may simply have to coordinate these activities with SAP Basis administrators. To view the data that was transferred into SAP by Windchill ESI, the following transactions are particularly important:
MM03 (View Material)
CS03 (View Material BOM)
CV03N (View Document)
CC03 (View Change Master)
CA03 (View Process Plan)
|
Resolving Problems
After you have monitored, detected, and diagnosed problems, you need to resolve them. The following lists some general techniques you can use while troubleshooting, as well as a list of specific problems and solutions.
Techniques for Resolving Problems
Following lists some of the troubleshooting techniques you can use to resolve problems.
Coordinating Troubleshooting Teams and Escalating Problems
As a Windchill ESI administrator, you may have to involve and coordinate a wide variety of specialists to fully resolve production issues. These parties may include end-users, functional experts, Windchill PDMLink administrators, SAP Basis administrators, database specialists, operating system specialists, network administrators, and others. If, after diagnosing and localizing an issue, you determine that the problem cannot be solved in-house, you may need to escalate it to systems integration partners or to PTC Technical Support, or both, for assistance. Systems integration partners may be particularly critical to help you resolve problems involving special configuration or customizations made to the standard Windchill ESI product.
Manually Overriding Checkpoints
The Windchill ESI application leverages checkpoint activities at key points in the TIBCO BusinessWorks processing flow. A checkpoint saves the process data and state of the current running process instance so that it can be recovered at a later time in the event of a failure. If a process engine fails, all process instances can be recovered and can resume execution at the location of their last checkpoint in the process definition. Only the most recent state is saved by a checkpoint. If you have multiple checkpoints in a process, only the state from the last checkpoint is available for recovering the process.
The location of Windchill ESI checkpoints has been strategically designed to ensure robust transaction integrity while minimizing adverse performance effects. Occasionally, you may need to manually override an activated checkpoint to restart transaction processing from scratch when the process engine is restarted.
|
|
Overriding checkpoints should be done with extreme caution, as it can lead to duplicate or incomplete data, incorrect export history records, and data corruption.
|
However, if circumstances warrant it, you can override checkpoints by deleting the <Tibco_Home>/tra/domain/<Domain_Name>/application/<Application_Name>/working directory and its contents. Depending on the process engine mode, there may be many other /working subdirectories under the /tibco path that would need to be deleted. Do a search for /working to locate and remove all of these subdirectories.
Resolving Solaris Platforms Issues
On Solaris platforms you may have to resolve problems that arise from:
• Environment variable issues
• Designer deployment configuration issues
The following describe techniques to resolve these.
Environment Variable Issues
On various Unix platforms certain TIBCO components fail to launch because of unresolved library dependencies.
If there is any *.sh or *.csh files present in the bin directory of that particular component, they should be executed and then the 'ldd' command can be used to identify unresolved library dependencies.
All required library paths should be added to the tibco.env.CUSTOM_EXT_PREPEND variable found in the *.tra file.
Designer Deployment Configuration Issues
Occasionally, the TIBCO Administrator and the TIBCO Runtime Agent (TRA) become corrupted and without warning, shut down on the Solaris platforms. If this occurs while TIBCO Designer is running, deployment configurations and TIBCO Designer may behave strangely, for example:
• Server-based repositories do not appear in the project pull-down list for the current domain when opening projects
• Deployment components are displayed as new, even though the deployment history show they had been deployed
• The deploy and undeploy options are unavailable for deployed, changed, or even new deployment components
If any of these symptoms occur, reboot the Solaris machine, restart the TRA and TIBCO Administrator, and then restart Designer.
Finding and Killing TIBCO Processes on UNIX platforms
Occasionally, as part of resolving problems you may need to kill all running TIBCO processes.
On UNIX platforms, you can use the UNIX command 'ps' to list all the processes that are running. However, the standard 'all' flag, '-a', does not necessarily work to list low-level TIBCO processes such as TIBCO Rendezvous. To view such processes, you can execute the following command:
# ps -ef | grep <filter>
For example:
# ps -ef | grep tibco
The above results in several lines that look like this:
root 5559 1 0 09:10:02 pts/1 0:18
<Tibco_Home>/administrator/<version>/bin/tibcoadmin --propfile /opt/
You can force-kill the processes listed by using the 'kill' command with the flag '-9' and the process ID (the second column from your 'ps -ef' command)
# kill -9 5559
Using the '-9' flag ensures the process is really dead. Generally speaking, to ensure all TIBCO processes are dead, you should run
ps -ef | grep tibco
ps -ef | grep hawk
ps -ef | grep rvd
and verify that no running processes are found.
Configuring Email Alerts
As mentioned earlier, Windchill ESI can be configured to send you email alert messages when the Windchill ESI business logic experiences an error:
• Sending an ESIPostResult message to Windchill PDMLink
• Waiting for an ESIResultResponse message from Windchill PDMLink
• Parsing the ESIResultResponse message
This configuration is performed through the following global variables:
• ESIMail/ToAddress
• ESIMail/FromAddress
• ESIMail/CCAddress
• ESIMail/BCCAddress
The server is configured using the global variable: ESIMail/SMTPHostServer.
Refer to theWindchill Enterprise Systems Integration Installation and Configuration Guide - SAP for details on how to configure the global variables.
The Windchill ESI business logic attempts to retry any EMS-related actions the number of times specified by the TIBCO BusinessWorks global variable, ESIJMS/RetryCount. Therefore, you may receive multiple email messages for the same issue. After a final unsuccessful attempt, the Windchill ESI business logic suspends the associated BusinessWorks process, allowing you to examine the errors described in the email alert messages, fix the underlying problems, and resume or kill the BusinessWorks process using TIBCO Administrator. Refer to the TIBCO Administrator User's Guide for details on how to use this interface.
Resolving Specific Problems
The following section lists specific problems, provides possible causes, and suggests solutions to resolve them.
TIBCO Administrator shows status as unknown for BWengine or adapter processes
There are two possible causes for this issue:
• A particular service's microhawk agent is taking time to send its status to the Administrator; it should be resolved in a few minutes.
• A configuration change has not been picked up by the hawk service; restarting the hawk service will resolve this issue
Adapter fails to start with JMS transport
After installing TIBCO Runtime Agent <version> and TIBCO Runtime Agent <version>.1, TIBCO Adapter projects using Enterprise Message Service as transport will not start up from TIBCO Designer and give the following error:
Message "Code = AESDKC-0156,Category = JmsComm, Severity = errorRole, Description = could not open JMS shared library jms." displays
This could be due to incompatible Adapter SDK libraries. To resolve this issue:
• On Windows: Remove the libeay32.dll and ssleay32.dll files from the <TIBCO_HOME>/adapters/sdk/version/lib folder.
• On UNIX: Remove the libssl and libcrypto openssl libraries from the <TIBCO_HOME>/adapters/sdk/version/lib directory.
Adapter fails to start showing a status of "Starting up" in the Administrator
If the process ID "-1" is allocated to the Adapter process, then this indicates some error launching the Adapter. This error is generally a library dependency. The following are the known errors:
• Case 1: Error while loading shared libraries: librfccm.so: wrong ELF class: ELFCLASS64
• Case 2: Error while loading shared libraries: librfccm.so: wrong ELF class: ELFCLASS32
• Case 3: Similar issues have been observed for the following libraries
◦ libresolv.so.2 sunw_2.2.2
◦ libstdc++-libc6.2-2.so.3
◦ libstdc++.so.5
◦ Errors are sometimes observed due to missing dll files in the SysWOW64 folder of the Windows machine.
• Case 4: Incorrect Java library settings
Unresolved library dependencies
The following solutions may fix this issue:
• You might have used 64 bit SAPJCo libraries. SAP Adapter is a 32 bit application on Windows X64 platforms. Using 32 bit libraries will resolve the issue.
• You might have used 32 bit SAPJCo libraries. SAP Adapter is a 64 bit application on certain platforms like Linux x64, Solaris X64 and Solaris SPARC. Using 64 bit libraries will resolve the issue.
• Install compatibility packages to resolve the dependencies.
• If Java environmental variables are already set, ensure that the versions are compatible. TIBCO Applications also install JRE 1.5 and 1.6. Users can always remove any java settings they have and let the TIBCO application set the respective java variables.
• On Solaris machines if Java variables are set, ensure that the class path contains 64 bit Java libraries, as the SAP adapter is a 64 bit application.
TIBCO Adapter for SAP instance stops working and status shows "Error"
This issue occurs due to adapter stack overflow error. TIBCO support has accepted it as a known issue and suggested increasing a parameter adr3.stacksize (for example, to 512 KB).
To set adr3.stacksize navigate to > > > > > .
'Coyote connector has not been started' error appears in TIBCO BusinessWorks Engine Logs
This could be due to incorrect values for ESIOthers/WSHost and ESIOthers/WSPort variables. Providing the proper values should resolve this issue.
"Input data invalid" message in ETL
This error indicates a schema validation failure in the 'Invoke an adapter request response service' activity. A detailed description and stack trace are logged into the Process Archive logs. The logs will indicate the exact reason of the schema mismatch.
For example:
validation error: data "xs:string('Hinge, Right Hand, Male, Removable, 0.187 Dia Pin, SS')"length must be at most xs:int('40') CHARACTERs ({com.tibco.xml.validation}SIMPLE_E_LENGTH_TOO_LONG) at /aeRequestInputType[1]/{http://www.tibco.com/xmlns/ae2xsd/2002/05/ae/700/basic/functionModules}__caret_request_caret_BAPI__MATERIAL__SAVEREPLICA_caret_BAPI__MATERIAL__SAVEREPLICA[1]/MATERIALDESCRIPTION[1]/item[2]/MATL__DESC[1]com.tibco.xml.validation.exception.k: data "xs:string('Hinge, Right Hand, Male, Removable, 0.187 Dia Pin, SS')" length must be at most xs:int('40') CHARACTERs
This could be due to a schema validation failure.
To resolve this issue rename the highlighted object and publish the transaction.
All EMS Server configurations disappear after the EMS server is started manually
This could be because the command to start the EMS server has been changed in the 5.1.4 version. In EMS version 4.x the command to start EMS was "./tibemsd". In EMS v5.1.4 the command is "./tibemsd64 -config ../tibco/cfgmgmt/ems/data/tibemsd.conf". The command uses a relative path and it should be executed from "<TIBCO_HOME>\ems\<version>\bin".
To resolve this problem stop the process started by the command:
"./tibemsd"
and start the EMS server with the correct command:
"./tibemsd64 -config ../tibco/cfgmgmt/ems/data/tibemsd.conf"
Deployment of Applications using the TIBCO Administrator GUI displays a Failure message
This could happen if any previous deployment or undeployment might accidentally have acquired a lock, and has not released it.
Run the AppManage command from command prompt to diagnose and resolve the problem accordingly:
1. Browse to:
◦ <Tibco_Home>\tra\<version>\bin
2. Run the following command:
AppManage deploy <Application name> -user <username> -pw <password> -domain <Domain_Name>
For example:
AppManage -'deploy -app SAP_ESI_Solution -user admin -pw admin -domain I4590
For more information refer to the <Tibco_Home>\tra\domain\<Domain_Name>\logs\ApplicationManagement.log file.
Cannot create a .dat file using the Repoconvert command
This happens when the value of tibco.class.path.extended property is missing the following entry in the RepoConvert.tra file:
• %TPCL_HOME%/tomcat/server/lib
Add the missing entry to the tibco.class.path extended property.
1. Open the RepoConvert.tra file, located in the following directory:
◦ <TIBCO_HOME>\tra\<version>\bin
2. Look for tibco.class.path.extended and append the path <TIBCO_HOME>/tpcl/<version>/tomcat/server/lib
3. Save the RepoConvert.tra file.
The Distribution Targets table is not displayed in the properties page of custom parts like 'wt.wadm.FADProduct' after the targets are created in the database
The default version of the file <Windchill>\codebase\netmarkets\jsp\tgt\distributionList.jsp is not designed to display the distribution targets table for custom parts.
To enable the Distribution Targets table for custom parts such as wt.wadm.FADProduct.
1. Open the file: <Windchill>\codebase\netmarkets\jsp\tgt\distributionList.jsp
2. Modify the if statement as follows, by adding the custom part type.
For example, if the object type is wt.wadm.FADProducts modify the if statement as follows:
if (oid.indexOf("wt.doc") != -1 ||
oid.indexOf("wt.epm") != -1 ||
oid.indexOf("wt.part") != -1 ||
oid.indexOf("com.ptc.windchill.mpml.processplan.MPMProcessPlan") != -1 ||
oid.indexOf("com.ptc.windchill.mpml.resource.MPMProcessMaterial") != -1 ||
oid.indexOf("com.ptc.windchill.mpml.resource.MPMTooling") != -1 ||
oid.indexOf("com.ptc.windchill.mpml.resource.MPMSkill") != -1 ||
oid.indexOf("wt.wadm.FADProducts") != -1)
3. Save the file and restart servlet engine.
SAPAdapter.PartConfiguration Error [Application] Error Message
The following message appears in the adapter log file (located in the adapter/adr3/<version>/logs directory) when Windchill ESI logs on to the SAP via the TIBCO Adapter for SAP:
SAPAdapter.SAP-ESISAPAdapterConfiguration Error [Application] AER3-000183 RFC error; Group : 104, Key : Name or password is incorrect. Please re-enter, Message : Name or password incorrect. Please re-enter
In addition, the following text appears in the dev_rfc.trc file (located in the /adapter/adr3/<version>/bin directory):
T:1756 ======> User not authorized. Session terminated
T:1756 <* RfcReceive [1] : returns 3:RFC_SYS_EXCEPTION
>TS> Fri Jan 24 11:41:57 2003
T:1780 ======> Name or password is incorrect. Please re-enter
T:1780 <* RfcReceive [2] : returns 3:RFC_SYS_EXCEPTION
T:1780 RfcGetAttributes: Invalid handle [1]T:2288 RfcGetAttributes:
Invalid handle [1]T:1756 RfcGetAttributes: Invalid handle [1]T:1764
RfcGetAttributes: Invalid handle [1]>TS> Fri Jan 24 11:42:46 2003
T:2648 <* RfcCall [1] : returns 18:RFC_INVALID_HANDLE
This happens when an invalid SAP user name and password is being used.
To resolve this issue provide the correct user name and password in the adapter configuration. For more information, see the Windchill Enterprise Systems Integration Installation and Configuration Guide - SAP.
Adapter Log File Error Messages
The log file for the Adapter for SAP displays the following message just as the adapter is starting:
SAPAdapter.SAP-ESISAPAdapterConfiguration Error [Application] AER3-000183
RFC error; Group : 104, Key : User is locked. Please notify the person responsible, Message : User is locked. Please notify the person responsible.
The adapter appears as if the start-up was successful. The following success message appears in the adapter log:
SAPAdapter.SAP-ESISAPAdapterConfiguration Info [Adapter] AER3-000082
Successful initialization of Adapter
However, when Windchill ESI issues a call to the adapter, the following error message appears in the adapter log file:
SAPAdapter.SAP-ESISAPAdapterConfiguration Error [Application] AER3-000072
Client connection SAP-ESISAPAdapterConfiguration InboundConnection0Client0 is
invalid
In addition, the following text appears in the dev_rfc.trc file:
T:1256 ======> User is locked. Please notify the person
responsible
T:1256 <* RfcReceive [1] : returns 3:RFC_SYS_EXCEPTION
T:1256 <* RfcReceive [4] : returns 3:RFC_SYS_EXCEPTION
T:1256 RfcGetAttributes: Invalid handle [3]
This happens when the SAP user name and password have become locked.
To resolve this issue an SAP administrator would have to unlock the account and you would need to verify the user name and password that is specified in the adapter configuration. For more information, see the Windchill Enterprise Systems Integration Installation and Configuration Guide - SAP.
TIBCO BusinessWorks returns error messages
TIBCO BusinessWorks returns error messages such as the following:
-2003 Feb 19 13:27:12:294 GMT -5 Engine Error [] PE-Error
process initialization failed for
ProcessDefinitions/Services/WCResult_Service
--Initialization error in
[ProcessDefinitions/Services/WCResult_Service/JMSRepeatUntilTru
e_ESIPostResult_Result/JMSSender_ESIResult_PostResult]
--javax.naming.ServiceUnavailableException: Failed to query
JNDI: Failed to connect to the server at tcp://localhost:7222.
Root exception is javax.jms.JMSException: Failed to connect to
the server at tcp://localhost:7222
This happens when the EMS server name is invalid or the EMS server is not running.
To resolve this issue, start the EMS server if it is not running.
Or confirm that ESIJMS/JNDIContextURL global variable in the deployment configuration of the process engine matches the value in the QueueConnectionFactory URL, in the factories.conf file located in <Tibco_Home>/ems/<version>/tibco/cfgmgmt/ems/data.
The ESIJMS/JNDIContextURL global variable should be set to:
tibjmsnaming://<machine_name>:<port>
and the QueueConnectionFactory URL should be set to:
tcp://<machine name>:<port>.
The machine name and port values should match.
For example, if ESIJMS/JNDIContextURL is set to tibjsnaming://mymachine.mycompany.com:7222, then the QueueConnectionFactory URL should be set to tcp://mymachine.mycompany.com:7222.
Refer to "Process Engine Global Variable Groups" theWindchill Enterprise Systems Integration Installation and Configuration Guide - SAP for details on this variable.
Additional BusinessWorks error messages
BusinessWorks returns error messages such as the following:
-2003 Feb 19 18:57:29:051 GMT -5 Engine Error [] PE-Error process
initialization failed for
ProcessDefinitions/DataProcessing/JMS_ESIEvent_TransactionRelease_
End_PD
--Initialization error in
[ProcessDefinitions/DataProcessing/JMS_ESIEvent_TransactionRelease
_End_PD/JMSReceiver_Event_ESIEvent]
--Could not establish connection or session with EMS provider.
--javax.naming.AuthenticationException: Not permitted: invalid name
or password. Root exception is javax.jms.JMSSecurityException:
invalid name or password
This happens when the user name or password for the JMS server is invalid.
Confirm that the ESIJMS/Username and ESIJMS/Password global variables in the deployment configuration of the process engine match the values of the username and password specified on the EMS server for the queues.
The username and password values set for the EMS server are in the users.conf file which is located in the <Tibco_Home>/ems/<version>/tibco/cfgmgmt/ems/data directory. Refer to "Process Engine Global Variable Groups" in the Windchill Enterprise Systems Integration Installation and Configuration Guide - SAP for details on this variable.
The passwords are obfuscated if they were set using the EMS Administration tool. If you do not remember the passwords you set, you will have to reset them.
Starting a TIBCO component on UNIX causes errors
Upon starting a TIBCO component on UNIX, you receive the following errors:
/usr/lib/dld.sl: Call to mmap() failed - TEXT (UNIX path)
/usr/lib/dld.sl: Permission denied
For example,
/usr/lib/dld.sl: Call to mmap() failed - TEXT
<Tibco_Home>/tra/<version>/lib/libmaverick50.sl
/usr/lib/dld.sl: Permission denied
This happens when permissions for the file have not been set properly.
To resolve this issue use the command chmod 555 on the file listed after TEXT. For example,
# chmod 555<Tibco_Home>/tra/<version>/lib/libmaverick50.sl
BusinessWorks, Windchill ESI, or both are failing to connect to EMS
The EMS server is not configured properly. When you specify the name of the EMS server as localhost, that server is only recognized on the box that it is running on. No other machine can connect to it. An application that is set to localhost will attempt to find the EMS server running on the same machine. If it is not found, it will error. When you specify a machine name as your server name, other machines can connect to your EMS server.
Check that the QueueConnectionFactory value located in the <Tibco_Home>/ems/<version>/tibco/cfgmgmt/ems/data/factories.conf file and the global variable ESIJMS/JNDIContextURL in the deployment of the Process Engine are set accordingly.
• QueueConnectionFactory in the factories.conf file should be set to tcp://<machinename>:7222, where machine name is the name of the machine where the EMS server is running.
• The global variable ESIJMS/JNDIContextURL in the BW Engine should be set to tibjsnaming://<machine name> where <machine name> is the name of the machine where the EMS server is running.
It does not matter where this EMS server resides. It can reside on the same machine as Windchill PDMLink, same machine as the TIBCO process engine, or a different machine altogether. As long as the above values described are set appropriately (and the machines are on the same network), Windchill ESI and EAI components are able to connect to the correct EMS server.
To determine which machine and username are connected to an EMS server, type the following command in the TIBCO EMS administration tool:
>show connections
This gives you a list of which users are connected and from which machine.
Transaction remains in pending state
This could happen if the JMS server queues are not subscribed to as described in
Configuring and Administering JMS Queues.
Common root causes are: the JMS server is not running when the Windchill method server starts, misconfigurations in the JMS properties of the Windchill adapter, or misconfigurations in the JMS server.
When Windchill ESI services are successful in subscribing to the JMS queue meant for receiving result messages, the following information is included in the Windchill PDMLink method server log:
Thread-10: subscription to queue "com.ptc.windchill.esi.Result" successful
If this information does not appear in the log, it means Windchill was unable to subscribe to the said queue successfully. In this case, all Windchill ESI transactions are left in a pending state. The transactions are processed once Windchill is able to subscribe to the queue successfully.
Normally, when Windchill ESI services are unable to subscribe to the queue because the JMS server is unavailable, Info*Engine receives an exception. Windchill ESI services log this exception in the Windchill Adapter transaction log. If TIBCO EMS is the JMS provider, the message contains the following text:
javax.ems.JMSException: Failed to connect to the server at tcp
As a detection measure, you may consider configuring your monitoring software to look for this or similar text. To resolve the issue, make sure the EMS server is up before the method server, and resolve any issues with the Windchill adapter JMS configuration and the JMS server configuration.
To avoid this issue, ensure the following:
• Only one WCESI user is connected to EMS server (can be viewed from EMS Administration Tool> Show connections)
• The number of ESISYS connections with ClientID (BW-ESIMaster_JMSConnection-queue-<Application Name>-Process_Archive) should be equal to the number of ERP instances configured (can be viewed from EMS Administration Tool> Show connections)
• All the connections are from either TIBCO or Windchill server in the current testing suite. There are no connections from a previous suite or from an alien machine (can be viewed from EMS Administration Tool> Show connections)
• Windchill and Process Archive connect to the same JMS Queue (can be viewed from EMS Administration Tool> Show queues)
• The queue com.ptc.windchill.esi.Result has only one receiver (can be viewed from EMS Administration Tool> Show queues)
• There should be no messages in any of the queues (can be viewed from EMS Administration Tool> Show queues)
• Windchill has subscribed to the DataResponse queue successfully. Connect to the JMS server and check if the DataResponse queue is created and the WCESI user is granted send rights on the DataResponse queue. If 'show queues' displays a * in front of the DataResponse queue name, it indicates that the queue is temporary and needs to be created. This issue is observed if the EAR is deployed manually. Run the following commands in the JMS Administration window to resolve the issue:
◦ create queue <DataResponse>
◦ setprop queue <DataResponse> secure
◦ grant queue <DataResponse> <EAI User> receive
◦ grant queue <DataResponse> <WC ESI User> send
◦ setprop factory QueueConnectionFactory url=tcp://<JMSServer>:7222
◦ commit
• The Process Archive is connected to the same DataResponse queue that Windchill has subscribed to. Check the JMS Administration window, to confirm that the DataResponse queue is subscribed to by the Process Archive. In case of manual deployment this step is missed sometimes. If the DataResponse queue is subscribed to, check TIBCO Administrator--> Application Management --> Application Name --> Configuration-->Deployment Name--> Advanced -->ESIJMS/DataResponseQueue
The TIBCO BusinessWorks process engine does not connect to the TIBCO EMS server
This could happen if the BusinessWorks engine was started before the EMS server.
Always start the EMS server before the BusinessWorks engine.
TIBCO BusinessWorks Process engine logs error messages
TIBCO BusinessWorks Process engine log contains the following message text:
Job terminated:
Please check following
1) FilesToRead_SAP.properties exists at %TIBCO_HOME%\\esi\\bin
2) %TIBCO_HOME%\esi\bin exists in tibco.env.STD_EXT_CP path in %TIBCO_HOME%\bw\<version>\bin\bwengine.tra
These symptoms usually indicate that, when starting up, the TIBCO BusinessWorks process engine could not load the Windchill ESI business logic properties files. These properties files contain the cross-reference and default values used in data mapping, texts used in logging, etc. There are several possible causes for this issue:
1. The properties files are not properly installed in the <TIBCO HOME>\esi\bin directory.
2. The FilesToRead_SAP.properties file specifies an invalid value for the Path variable. The Path variable must point to the place where the files are located. This value must contain a slash at the end of the path name. For example: C:/tibco/esi/ (Windows) or /opt/tibco/esi/ (UNIX).
3. The properties files' security authorizations do not allow read access to the TIBCO BusinessWorks process engine.
To resolve this issue:
1. If the FilesToRead_SAP.properties file specifies an invalid value for the Path variable, enter in the correct value, save the file, and then restart the TIBCO BusinessWorks process engine.
2. If the properties files have restricted read access, grant this access to the TIBCO BusinessWorks process engine. See the Windchill Enterprise Systems Integration Installation and Configuration Guide - SAP for further details on configuring the Windchill ESI business logic properties files and specifying class paths during deployment.
TIBCO Administrator fails with an "Apache/Tomcat 404" Web browser error
You may be using an unsupported web browser version.
Upgrade your web browser to Microsoft Internet Explorer version 6.0 SP1, or later.
When starting an adapter instance or process engine in TIBCO Administrator, the status remains stuck on "Starting Up," and never changes to "Started."
This is a known issue with the TIBCO Adapter for SAP and the TIBCO Runtime Agent (TRA). It is possible that starting the TIBCO services automatically (as is the default on Windows systems) may cause them to sometimes start in the wrong sequence and prevent proper communication between the Administrator Server and TRA.
To resolve this issue check the component's log file to determine if it has, in fact, started successfully, despite the "Starting Up" status display. In most cases, it probably has. If not, try the following procedure:
1. Shut down the TIBCO Administrator application and all running TIBCO components and services.
2. Start up the TIBCO services manually, in the following order:
a. TIBCO Administrator <version> (Domain Name)
b. TIBCO HAWK Agent (Domain Name)
3. Restart the TIBCO Administrator application and try to start up the components again.
Multiple "Activity timed out" errors appear in the TIBCO BusinessWorks process engine log
Multiple "Activity timed out" errors appear in the TIBCO BusinessWorks process engine log; however, Windchill ESI reports a successful result in the Enterprise Systems Transaction Log Graphical User Interface.
This is an expected behavior. Windchill ESI automatically compensates for certain timeout events with subsequent SAP queries, and processing proceeds normally.
No action is required.
Multiple Release To Manufacturing workflows are created upon releasing a promotion request from Windchill PDMLink
This happens if the Windchill ESI preference Publish Promotion Requests has a value No. Set the preference to Yes in order that a single RTM workflow is created upon releasing a promotion request.
|
|
With the Windchill ESIPublish Promotion Requests set to No, releasing a promotion request results in as many RTM workflows as there are promotables in the promotion request.
|
A document associated to a part (or to a manufacturing object) does not get published to certain distribution targets associated to it
This can occur if the document is associated to additional distribution targets as compared to the part (or the manufacturing object). In such a case, the document is published only to those distribution targets that are associated to the part (or to the manufacturing object). This is only an expected behavior. In order that the document is published to the additional targets, release the document either as a standalone object or in association with a Change Notice or a promotion request.
Only the top-level document in a CAD document structure is published upon releasing a part associated to the structure
This is an expected behavior. A CAD document structure is published only if the structure was released as a standalone object, or in association with a Change Notice or a promotion request.
Only the top-level document in a CAD document structure is published upon releasing the structure
This can occur if any of the following is true:
• The distribution target attribute Number of Levels to Publish when Publishing a CAD Document Structure is set to 0 (zero), or to an empty string. It should be set to a number specifying the required number of levels instead.
• The document masters that represent the first level children in the structure are not resolvable to iterations using the distribution target attribute Saved Filter to be used when Publishing a CAD Document Structure.
Ensure that these distribution target attributes are set appropriately.
The distribution target attribute Saved Filter to be used when Publishing a Change Notice does not get used when publishing a CAD document structure in association with a Change Notice
This is an expected behavior. ESI services use the attribute Saved Filter to be used when Publishing a CAD Document Structure instead. Regardless of whether a CAD document structure is released as a standalone object, or in association with a Change Notice or a promotion request, ESI services use the distribution target attributes that appear under the CAD Document Settingssection of the Create/Edit Distribution Target UI.
An attempt to publish a control characteristic results in a failure
This problem can occur if any of the following is true:
• The control characteristic is consumed by a process plan’s operation but is no longer associated to its owner (part or process plan). Also, the Windchill ESI preference Ignore Orphaned Control Characteristics has a value No.
• The associated model item's EPM document is no longer associated to the control characteristic’s owner.
In order for the control characteristic to be published successfully, ensure that the control characteristic and its model item's EPM document are both associated to the control characteristic's owner.
