ThingWorx Analytics 8.3 Release Notes
The following sections describe the new functionality, enhancements, bug fixes, and known issues in ThingWorx Analytics 8.3.
New Functionality
|
ThingWorx Analytics Installation
|
|---|
|
This release brings several enhancements to the installation of ThingWorx Analytics functionality:
• Bundled Installer – ThingWorx Analytics Server is now available in a bundled installer that also includes Descriptive Analytics (a new microservice – see the next table). This bundled installation, available for both Linux and Windows operating systems, provides a convenient way to install any or all of the included components. You can choose the components at the time of installation.
• Subset Installation – When installing theThingWorx Analytics Server, you can now select which microservices to include. At a minimum, the Results and Data microservices must be installed, but all of the others are optional. At the time of installation you can choose to install all of the microservices or a subset of them.
• User Account Selection – Previously, ThingWorx Analytics installers, both Linux and Windows, created a default twxanalytics user account for the server to use. In this release, an advanced option is available to override the default account and specify a user account. The specified account must already exist in the system. The installation process will first verify the user account, then set ownership and permissions accordingly, and configure systemd or Windows services to run as the specified user.
• Common Workers – ThingWorx Analytics Server architecture has changed so that most of the microservices are lightweight and require few resources. The microservices are no longer constrained by a one-to-one coupling with an embedded worker. Each microservice accepts job requests and queries from the Analytics edge agent and submits any asynchronous jobs to one or more separate workers. The workers handle the actual work of the submitted job requests. At installation, three workers, which can each accept any type of job, are set up. The number of workers can be modified after installation.
• Upgrade from Earlier Version – The new installer provides an upgrade path from some earlier versions of ThingWorx Analytics Server. When the 8.3 installer runs, it detects any previous installation of the server, from release 8.1.x to the current release, and prompts the user to upgrade. The upgrade process migrates all the data related to existing jobs and results. When the upgrade is complete, the installer will continue its deployment process. You can run the installer on an up-to-date installation for the purpose of repairing or modifying the installation.
Note that an upgrade to the ThingWorx Analytics Server will likely require an upgrade to your ThingWorx server as well. The ThingWorx server must be upgraded before you run the ThingWorx Analytics installer.
|
|
Descriptive Analytics Services
|
|---|
|
An IoT environment provides access to a large amount of data from a variety of sources. Using analytics and statistical calculations, that raw data can be transformed to create meaningful insights. Descriptive Analytics functionality provides an out-of-the-box library of services that can perform common statistical calculations, frequency transformations, and statistical process control calculations. Packaging these on-demand services in a library reduces the amount of time and coding required to perform calculations and data transformations. Application developers can use this generated intelligence from their raw data as input to other algorithms or machine learning models. They can also use it to build solution-specific visualizations (mashups or widgets) for their end users or create alerts based on logged property values.
These services are made available in two new microservers: StatisticalCalculation and StatisticalMonitoring. Both microservers are installed as part of the Descriptive Analytics microservice, during the ThingWorx Analytics bundled installation (see above). Their services can be accessed via your Thing model in ThingWorx Composer. The types of services available in each microserver are listed below:
StatisticalCalculation Microserver:
• Mean / Standard Deviation
• Median / Minimum / Maximum / Mode
• Five Number
• Data Distribution
• Confidence Interval
• Fast Fourier Transform
• Sampling Frequency
StatisticalMonitoring Microserver:
• Continuous points above or below a threshold
• Continuous points that are inside or outside of a range
• Increasing or decreasing trends
• Alternating directions
• Rate of change
• Rate of change that is inside or outside of a range
|
|
Analytics Builder and Analytics Manager Accessible from within ThingWorx Composer
|
|---|
|
In this release, a new Analytics menu has been added to ThingWorx Composer so that Analytics Builder and Analytics Manager can both be launched natively from within the ThingWorx Composer interface. This enhancement eliminates the need to configure the User Management Subsystem in ThingWorx Composer and the need to use a console mashup to access Analytics Builder or Analytics Manager. The consoles are still available but are no longer mandatory. Once the ThingWorx Analytics Extension has been imported and configured in ThingWorx, the Analytics menu is accessible by clicking the new Analytics menu icon:  . |
|
Predictive Scoring Using the Analytics Server Connector
|
|---|
|
In this release, you can perform predictive scoring on models by using the TW.AnalysisServices.AnalyticsServer.AnalyticsServerConnector connector in Analytics Manager. The prediction microservice in the Analytics Server is used to score the models. With this connector, you can perform predictive scoring on both, non-time series as well as time series models.
You cannot run a simulation predictive scoring job.
The analysis replay functionality of Analytics Manager is not supported by the Analytics Server connector.
For more information, see How Predictive Scoring Using the Analytics Server Connector Works.
The ThingWorx Analytics Help Center includes tutorials to perform predictive scoring on non-time series and time series models. You can find the tutorials at Tutorial: Use the Analytics Server Connector for Predictive Scoring on Non-time Series Models and Tutorial: Use the Analytics Server Connector for Predictive Scoring on Time Series Models.
|
Enhancements
|
Enhancement Description
|
Reference #
|
||
|---|---|---|---|
|
ThingWorx Analytics Server: Redundancy Filtering
In this release, the option to enable redundancy filtering has been added to both the Training and the Signals microservices. When enabled, redundancy filtering operates by calculating mutual information between the goal variable and each feature. It then iteratively ranks each feature, in combination with previously-selected features, by the amount of information gain it provides. During training, this ranking is used to improve feature selection for the predictive model. During a signals job, the information gain ranking is used to indicate the predictive strength of each feature.
|
TA-360
TA-834
|
||
|
Analytics Builder: Redundancy Filtering UI Updates
The new option to enable redundancy filtering for Training and Signals (see above), is also exposed to the Analytics Builder user interface. The option is available during the following actions:
• Creating a new predictive model – The option is available in the Model Settings section of the Advanced Model Configuration tab.
• Identifying new signals – The option is available when a signals job is run from a dataset, without being based on an existing model. When a signals job is run on an existing model, the redundancy filter option retains its value from when the model was trained.
|
TA-866
TA-836
|
||
|
ThingWorx Analytics Server: Time Series Data
In this release, the following enhancements have been made to working with time series data in both the Training and the Signals microservices:
• lookahead parameter – In the past, the lookahead field has always defaulted to 1 (or 0 if goal history was not in use). This default meant that outcomes could be predicted for only one time step ahead. This parameter is now user-configurable. By entering a number greater than 1, outcomes can be predicted further ahead.
• useGoalHistory parameter – When useGoalHistory* is enabled, recent goal values are used to predict future goal values. This behavior is the default for scoring time series predictive models. However, in some scenarios, it’s not possible to know the value of the goal variable during predictive scoring, either because the value of the goal feature is not observable during scoring or because measuring it is physically or financially difficult. By disabling useGoalHistory, you can train a time series model that can be used to predict the value of a time series goal without providing recent values for the goal field.
|
TA-805
TA-806
|
||
|
Analytics Builder: Time Series Data UI Updates
For time series data, the new Lookahead and Use Goal History parameters (see above), are also exposed to the Analytics Builder user interface. These option are available during the following actions:
• Creating a new predictive model – Both options are available in the Time Series section of the Advanced Model Configuration tab.
• Identifying new signals – The options are available when a signals job is run from a dataset, without being based on an existing model. When signals run on an existing model, both options retain their values from when the model was trained.
|
TA-808
TA-809
|
||
|
ThingWorx Analytics Server and Analytics Builder: SVM Learner Added to Predictive Modeling Techniques
For generating predictive models, a new Support Vector Machines (SVM) learner has been added to the set of available learner techniques. Applicable only for Boolean goals, it can be used to predict the classification of data into one of two categories. It tries to classify data in a way that maps the largest separation between the categories. An SVM learner can be used alone or as part of a Majority Vote ensemble.
|
TA-1122
TA-1123
|
||
|
ThingWorx Analytics Server and Analytics Builder: Majority Vote Added to Predictive Modeling Ensemble Techniques
In this release, the Majority Vote ensemble technique has been added to the set of available ensembles. Applicable only for Boolean goals, the Majority Vote ensemble tallies the scores for each learner in the ensemble and selects the one with the largest tally.
|
TA-1124
TA-1125
|
Bug Fixes
|
Bug Fix Description
|
Reference #
|
|---|---|
|
Anomaly Detection: Error messages contain misspellings and misleading wording
In this release, the anomaly alert messaging has been updated to fix a misspelling and to clarify that it is the property value which might be anomalous when an anomaly alert is generated.
|
TA-253
|
|
Analytics Builder: Retraining a time series model does not retain the default lookback window size
Previously, when a time series model was built using the default lookback window size of 0, the lookback size was not retained when the time series model was retrained. This issue manifested itself in two scenarios:
• If the time series model had one feature, and a lookback size of 0, the retraining job would fail.
• If the time series model had more than one feature, and a lookback size of 0, the retraining job would succeed, but only as a standard training model, not as a time series model
This issue has been resolved by ensuring that the lookback size is always retained when retraining a time series model.
|
TA-472
|
|
ThingWorx Analytics: DistributionQuery job error message
In previous releases, when a DistributionQuery filter returns no rows, a NullPointException error was generated and the job failed. In this release, the job will complete and report that no rows were found that matched the filter.
|
TA-475
|
|
ThingWorx Analytics: importantFieldCount is not returned in GetJobInfo
When GetJobInfo is run on a predictive scoring job, the importantFieldCount should be included in the additionalParmeters section of the results. Previously, the importantFieldCount was not included. This issue has been resolved so that importantFieldCount is included in the GetJobInfo results.
|
TA-476
|
|
ThingWorx Analytics: Model validation fails when the dataset metadata includes Integer/Continuous fields
When the metadata for a dataset contains fields defined as dataType Integer with opType Continuous, the model validation process fails to complete and generates an error. For the purposes of outputting to a PMML format, the validation process cannot accept an Integer dataType for a Continuous goal. To resolve this issue and successfully output a PMML model, all Integer/Continuous fields are converted internally to dataType Double with opType Continuous. Now the validation process can complete successfully and a model can be generated.
|
TA-496
|
|
Analytics Builder: Deleting a filter does not delete all jobs associated with it
Messaging in Analytics Builder has been updated to clarify that models, signals, profiles, and scoring jobs that use a filter are not deleted when the filter is deleted. The delete action removes the filter from the datasetFilters property of the AnalyticsServerFiltersInstance Thing. However, because a SQL version of the filter is stored with each job, the job results can still be viewed after the filter is deleted from Thing storage.
|
TA-502
|
|
Analytics Builder: No validation message is displayed for a duplicate dataset name
When creating a dataset, if a name is entered that is not unique, the create dataset job cannot be submitted. However, no message is displayed to explain that the dataset name is a duplicate. This issue has been resolved so that an error is generated if a duplicate name is submitted.
|
TA-525
|
|
ThingWorx Analytics Server: Many warnings in the log file of type BlockedThreadChecker
The log file for the Edge microservice becomes difficult to use because of the large volume of warning messages it includes. To resolve this issue, the BlockedThreadChecker warnings are now written to a separate file.
|
TA-543
|
|
ThingWorx Analytics Server: maxAllowedFields at the learner level is overridden by the ensemble level setting
During Training, the value for the maxAllowedFields can be set at the ensemble level or at the individual learner level. However, the individual learner level setting for this parameter is being overridden by the ensemble level setting. This issue has been resolved so that the learner level setting is not overridden.
|
TA-840
|
|
ThingWorx Analytics Server: No exception is generated when a time series model is trained with multiple temporal or entity fields
If a time series model is trained from a dataset that includes more than one temporal field or more than one entity field, the job should fail and generate an error message. Instead, the system selects the first instance of each field and leaves the other fields as informational, with no indication to the user. This issue has been resolved so that an exception is generated and the training job fails.
|
TA-1131
|
|
Analytics Builder: Signals jobs are created as standard, instead of timeseries, when initiated from models built with auto-windowing
When a timeseries model is created with a Lookback of 0, which indicates auto-windowing, a signals job run on that model will only run as a standard dataset, not as a timeseries dataset. To resolve this issue, when a signals job on an existing model detects a Lookback of 0, it will use the detected auto-windowing size from the trained model instead.
|
TA-1148
|
|
Analytics Builder: Sampling Strategy parameter is not maintained when a model is retrained
When an existing model is retrained, the Sampling Strategy parameter returns to its default value instead of maintaining the value of the original model. This issue has been resolved so the original value is maintained.
|
TA-1224
|
|
ThingWorx Analytics Installer: Filter out unnecessary characters from the Appkey
On occasion, when an Appkey is copied from Composer and pasted into a field during the Analytics installation, some extra characters, spaces, or returns are copied and pasted inadvertently. These extra characters can cause downstream problems. To resolve this issue, all extraneous characters will be filtered out when the Appkey is entered during Analytics installation.
|
TA-1240
|
|
ThingWorx Analytics Installer: Remove sudo from the Linux installer
The use of sudo in many Analytics server installation scripts has been problematic for some customers. Since users are already required to run the installation as the root user, the sudo command is not necessary. To resolve this issue, sudo has been removed from all system commands in the server installer or installer utility scripts.
|
TA-1241
|
|
ThingWorx Analytics Server: Predictive scoring does not work for timeseries models that include hyphenated field names
When a timeseries model is created using a dataset with hyphenated field names, predictive scoring on that model generates an error. This issue has been resolved by updated the way these fields are parsed for timeseries transformation.
|
TA-1245
|
|
ThingWorx Analytics Server: Negative MI calculations during Signals jobs
In some cases, during a signals job, calculating normalized Mutual Information (MI) can result in an invalid negative value due to numerical precision. An illegal argument exception is generated and the job fails. To resolve this issue, whenever the approximation of MI returns a negative value, the value is set to zero and the job can complete successfully.
|
TA-1295
|
|
ThingWorx Analytics Server: Allow predictive scoring with important fields and prescriptive scoring to handle Boolean or Ordinal opType goals
When retrieving important fields from a predictive scoring job or running a prescriptive scoring job on a model generated by the Training microserver, an error occurs if the goal is a Boolean or Ordinal opType. The calculation expects the goal value from the scoring job to be a number and fails when it finds a True/False string or an ordered set of strings. To resolve this issue, an update has been made to allow important fields, for predictive scoring, and prescriptive scores to be calculated successfully for a PMML model with a Boolean or Ordinal goal. This resolution applies to models that were generated by the Training microserver. For third-party PMML models, this solution may not be applicable.
|
TA-1371
TA-1604
|
|
ThingWorx Analytics Server: FieldImportance calculations not handling out-of-range values as expected
During predictive scoring with important fields, when a record included a field value that was outside of the expected minimum or maximum for that field, unexpected results were sometimes returned:
• When Full Range was selected, the original predictive score associated with that record was ignored when calculating the importance for that field. However, the importance scores for other fields in the same record did take into account the original score.
• When Distance from Max or Distance from Min was selected, negative values were sometimes returned if the original predictive score for a record was either greater than the maximum or less than the minimum expected values.
These issues have been resolved so that FieldImportance calculations can handle out-of-range values more effectively.
|
TA-1483
TA-1579
|
|
Analytics Builder: No indication of loading after selecting a Signal to view
When a specific signal is selected on the Signal results page, processing the display of the value distribution can take a few seconds. Previously, there was no indication that loading was happening. This issue has been resolved.
|
TA-1485
|
|
Analytics Builder: Sidebar menu disappears when viewing Model Results In this release, Analytics Builder can be launched natively from within ThingWorx Composer by clicking the new Analytics icon: . It’s no longer necessary to launch Analytics Builder via an external console mashup. When Analytics Builder is launched natively, there are no sidebar menu issues.If you continue to use the console mashup to launch Analytics Builder, be sure to use the Refresh button (  ) on each individual Analytics Builder tab instead of the browser refresh (F5). |
TA-1514
|
|
Analytics Builder: Misspelling on the New Predictive Model dialog box
Spelling of the word “technique” has been corrected in the Ensemble Techniques section of the Advanced Model Configuration tab in the New Predictive Model dialog box.
|
TA-1519
|
|
Analytics Builder: PVA shows double values for actual results when input values are integers
By design, predicted vs. actual (PVA) results are displayed as double values, rather than integers. Because the data for PVA results is binned for charting, the data points on the chart do not represent individual records. Each PVA point represents a range, whether it contains one or multiple records. The chart point location reflects the predicted and actual values at the center of that range.
As a clarification, tooltips that display actual and predicted results on a Bubble Plot, in model results, have been updated. They now indicate that the records in the bubble are near the displayed values.
|
TA-1520
|
|
Analytics Manager: Error generated when publishing a model with integer data type fields
The issue of converting integer data type fields to number data types has been resolved for models uploaded or published using the new Analytics Server Connector. The new connector handles the necessary data type conversions properly.
There are no current plans to resolve the issue for models uploaded using the ThingPredictor Connector.
|
TA-1538
|
|
Analytics Builder: Filter strings are not passed to Signals and Profiles jobs when run from an existing model
When Signals and Profiles jobs are run from an existing model that includes a filter, the filter query string is not passed to those jobs. This issue can lead to misleading results from both Signals and Profiles jobs. A logic error has been fixed to resolve this issue.
|
TA-1569
|
|
ThingWorx Analytics Server: API documentation for AnalyticsDatasetRef shows uppercase options for formats
The API documentation of the AnalyticsDatasetRef Data Shape incorrectly lists options for the format field in all uppercase letters. The format options must be in all lowercase letters. This error has been corrected in the API documentation.
|
TA-1613
TW-34562
|
|
Analytics Builder: Refreshing multi-page lists returns to the first page
When refreshing a list page with multiple pages (such as the Models list page), the refresh always displayed the first page rather than the previous current page. This issue has been resolved so that the current page is redisplayed after a refresh.
|
TA-1620
|
|
ThingWorx Analytics Server: Binned Metrics in Validation are returned in the wrong order
The ContinuousModelEvaluator calculates Pearson Correlation and RMSE in different bins of the validation dataset. However, in previous releases, when the results were returned, their order was swapped and the values were mapped to the wrong keys. As a result, Pearson Correlation values could exceed 1 and RMSE results could be returned as negative values. Because of this incorrect mapping, the normalized RMSE values were actually a normalization of the Pearson Correlation, which is not a relevant result. This issue has been resolved by fixing the mapping so that all values are mapped to their correct keys.
|
TA-1628
|
|
Analytics Manager: Error Message field did not display the full error message on the UI
This issue has been fixed. A tooltip is displayed on the UI with the error message.
|
AM-1275
|
Known Issues
|
Known Issue Description
|
Reference #
|
|---|---|
|
ThingWorx Analytics Server: ZooKeeper doesn’t start after a successful Analytics Server installation
When an Analytics Server Linux installation has been completed successfully on an Ubuntu 16.04 server, the ZooKeeper component will sometimes not start when the other microservices do. To verify whether ZooKeeper is running, use the twas.sh script to check the status of the installed services. For example: /opt/ThingWorxAnalytics/ThingWorxAnalyticsServer/bin/twas.sh status. If ZooKeeper is running, it will be listed as active (running). If ZooKeeper is not running, it will be listed as failed. To resolve the issue, use the twas.sh script to restart the Analytics Server. For example: /opt/ThingWorxAnalytics/ThingWorxAnalyticsServer/bin/twas.sh restart. Recheck using the status script and ZooKeeper should be running.
This issue will be fully resolved in the next release.
|
TA-1797
|
|
ThingWorx Analytics Server: Two-at-a-time Signals request does not return MI for all field combinations
When running a request for Signals, and specifying maxAtATime = 2, Mutual Information (MI), in relation to the goal, is not returned for all field combinations as expected. Instead, individual (one-at-a-time) MI scores are returned for all fields and then are filtered down to the top 25% most relevant fields. Two-at-a-time signals are calculated only for those fields. There is currently no way to modify this filtering behavior.
|
TA-729
|
|
Analytics Manager: An existing simulation event fails if it is triggered after server restart
Analysis agents must be restarted after a ThingWorx server restart. Without restart of the agent, any new simulation jobs will fail to execute.
|
—
|
|
Analytics Manager: A time window event does not work for non-time series models uploaded using the Analytics Server Connector when all properties of the thing, including the key field are mapped
|
AM-1289
|
|
ThingWorx Analytics Server: The reported version number is an internal versioning number
The Version Info service currently returns a version number that indicates both the external version and the internal build version, but not a path release version. Possible changes in version information are under consideration for a future enhancement.
|
TA-1217
|
|
Analytics Builder: Filtering on list pages does not work as expected
When filtering is applied to a list page with multiple pages (such as the Models list page), the filtered results are not aggregated for display on a single page. Instead, the filtered results remain on their original pages and multiple pages must be clicked through to find all the filtered results.
To improve handling of list page displays, a configurable parameter has been added to set the number of table items that can be displayed per page. A more permanent resolution for the filtering with pagination issue will be provided in a future release.
|
TA-1596
|
|
Analytics Builder: Screens do not load properly in IE 11
The View Datasets and View Models screens do not load properly in the IE 11 browser for the support Windows operating systems. This issue will be resolved in a future release. To avoid the issue, use another browser, such as Chrome, Firefox, or Safari.
|
TW-31953
|
End of Life Information
|
ThingWatcher
|
|---|
|
ThingWatcher has been discontinued as a standalone SDK in 8.3. However, ThingWatcher functionality is available in the Anomaly Detection features native to the ThingWorx platform. For more information, see Anomaly Detection.
|
|
ThingPredictor
|
|---|
|
ThingPredictor software media is no longer available for download as of 8.3. However, support will continue for 8.x users until the next major release. To replace ThingPredictor capabilities, an improved set of predictive services is being introduced in 8.3. For more information, see How Predictive Scoring Using the Analytics Server Connector Works.
|
|
DataConnect / Data Analytics Definition
|
|---|
|
DataConnect and the Data Analytics Definition are not available as of 8.3. Replacement functionality will be introduced in a future release.
|