Support for Multiple Consumers of IoT Hub Data
As of v.4.2.0 of the ThingWorx Azure IIoT OPC UA integration, support is provided for multiple consumers of Azure IoT Hub data. Previously the ThingWorx Azure IoT Hub Connector automatically unpublished OPC UA nodes when those nodes were unbound or deleted from ThingWorx Platform. "Unpublish" here means that the nodes are deleted from the Azure IoT Hub. The Connector could also unpublish nodes on the Azure IoT Hub that were never bound in the ThingWorx Platform. For customers who want to integrate with ThingWorx and other consumers of telemetry, such as Microsoft Azure Data Lake, this behavior prevented that integration.
The following additions to the Connector's extension provide the following capabilities:
To disable and enable the automatic publishing and unpublishing of nodes, a configuration option, called Enable Auto Node Publishing, is provided on the AzureIoTHubTemplate Thing. When set to the default value, true, the ThingWorx Platform continues the behavior of automatically publishing and unpublishing nodes when they are added to or deleted from ThingWorx. It also deletes nodes that were never bound in ThingWorx.
For customers integrating with another application such as Azure Data Lake, this option can be set to false so that nodes deleted by ThingWorx are not automatically unpublished from the Azure IoT Hub. After setting the option to false, you need to start up or restart all Connectors communicating with that Azure IoT Hub so that the new value is applied in the Connectors. The Connectors will no longer unpublish OR publish nodes as a result of creating, updating, or deleting Remote Things bound to UA endpoints. It will no longer delete nodes that were never bound to ThingWorx Platform. In this configuration, property updates do not cause publishing or unpublishing to occur automatically.
Two dedicated services, called PublishUANodes and UnpublishUANodes, are available on the AzureIotHubTemplate Thing to publish and unpublish nodes manually when automatic publishing/unpublishing is disabled.
The following sections provide more details about this functionality. Click the title of a section to display its content. Click the title again to hide the content:
How It Works
When automatic node publishing/unpublishing is enabled, you can expect the following behavior:
1. Scenario: You can use Postman to publish a new node to the Azure IoT Hub without using ThingWorx. The node and its data are available to applications such as Azure Data Lake.
2. Scenario: ThingWorx and another application both subscribe to the same node on the Azure IoT Hub. If you delete a Thing on ThingWorx that is bound to the node, the Connector unpublishes the node. Data from the node no longer flows to the other application.
If the node is bound to a property on another Thing, the Connector does not unpublish the node.
By setting Enable Auto Node Publishing to false, you prevent the automatic publishing and unpublishing of nodes, with the following results:
1. Scenario: A new node is published to the Azure IoT Hub using Postman but not ThingWorx. Result: A restart of the Connector is required when you change the value of Enable Auto Node Publishing. After restarting, the Connector takes no action and the new node remains in the Azure IoT Hub. It is available for subscription by another application, such as Azure Data Lake.
2. Scenario: A new node is added in ThingWorx. Binding the node to a Thing results in the Connector publishing the node with the sample and publish rates set in ThingWorx Composer.
When disabling automatic publishing/unpublishing, you need to monitor node subscriptions and manage the removal of nodes ("unsubscribing") either through the services, PublishUANodes and UnpublishUANodes, or a tool such as Postman.
Using the Azure Portal to manage node removal is not recommended.
Publishing Intervals
Currently, when you set a new Publishing Interval in ThingWorx Composer for a particular property, all properties coming from the IoT Edge device are also set to the new Publishing Interval. What if you want to have the interval change for only that property?
It is not possible to have two properties bound to the same tag publish at different rates. If you have prop1 and prop2 with both bound to the RampDouble tag, they both publish at the same rate. If the publishing interval for prop1 is four seconds and for prop2 is two seconds, both properties publish at the shorter rate between the two properties, which in this example is two seconds.
For example, a ThingWorx Thing that represents a machine/line has a number of properties coming from a PLC through the OPC server (TKS). Depending on what the value represents and how it will be used, there are different scan rates/sampling intervals required to best fit their use. Here are example properties with example desirable update rates:
Desired Update Rate
Current Work Order
30 seconds
Next Work Order
120 seconds
Last Downtime Event
60 seconds
Seconds Running
60 seconds
Good Part Count
10 seconds
Rejected Part Count
10 seconds
Active Error Code
1 second
Current Status
1 second
Part of the importance of publishing interval configuration is the network traffic and infrastructure load, but also translates into load on a ThingWorx Platform and on the data persistence provider storing this data. Configuration enables your system to achieve the correct level of responsiveness while maintaining performance.
To run multiple nodes with different publishing intervals, you must upgrade to the latest approved version of the Microsoft Azure Industrial IoT (IIoT) stack. Earlier versions do not provide this capability. For instructions on upgrading, refer to Upgrading the Azure IIoT Stack.
If, after upgrading, you notice that publishing intervals are being reset for all properties rather than just the one you changed in ThingWorx Composer, make sure that your namespace is running the latest approved version and that your edge modules are no longer communicating with microservices on an older namespace. The publishing intervals for properties all being changed when you change it for one is a good indicator that there is a mismatch of microservices versions.
Microsoft does not support two namespaces sharing the same unique set of underlying Azure resources. In addition, Microsoft does not support the simultaneous operation of old and new deployments using the same Azure Resources.. Check whether two namespaces are sharing the same Azure Resources. This might occur after an upgrade.
If you are a system administrator with access to the Azure Portal and your IoT Edge devices, you can go to IoT Edge and click IoT Edge Deployments to see how your Kubernetes pods are working and also whether layered deployment is successful.
FAQ: Automatic Node Publishing and Unpublishing
How do I unpublish a node that is not bound to a ThingWorx Thing?
PTC Cloud customers and on premise customers can take one of the following actions:
If you have another ThingWorx Platform instance that manages subscriptions, the related Connector instance can unpublish a node that is not bound to a Thing in the platform.
Use the dedicated Unpublish service when Enable Auto Node Publishing is set to false.
In addition to the actions listed above, PTC on premise customers can use third-party methods such as Postman.
What happens when I bind a Thing to a Node?
When Enable Auto Node Publishing is set to the default behavior (true), the ThingWorx Platform synchronizes the published nodes for that endpoint with the mappings found in the AzureOpcUaPropertyMapDataTable. If multiple Things are tied to that endpoint, the platform accounts for all those subscriptions, not just those for the Thing that is having its bindings updated. This synchronization operation could lead to any combination of new publishing, unpublishing, or updating of nodes already published, according to what is found in the table. If other endpoints are bound to other Things, they are not synchronized at the same time as the binding updates are being made to a Thing that is bound to the synchronized endpoint.
When Enable Auto Node Publishing is set to false, the publishing and unpublishing of nodes never happens as a result of the synchronization process
What happens when I change the sample and publish rates from the Azure Portal or Postman, and it is different than the sample and publish rate set in ThingWorx Composer?
The rates used reflect the new settings.
How do I change Enable Auto Node Publishing to ‘true' or ‘false’?
Go to your AzureIotHubTemplate Thing and click Configuration. Scroll to the Enable Auto Node Publishing check box. By default it is selected, indicating that ThingWorx automatic publishing and unpublishing of nodes is in use. Clear the check box to turn it off. Save the Thing,and then restart all related Connectors to apply the change.
What happens when I change Enable Auto Node Publishing to ‘true’ after I already manage my subscriptions outside of ThingWorx?
When bindings are updated for endpoints, published nodes synchronize for that endpoint via ThingWorx subscriptions. With no action other than switching the flag to true, no publishing or unpublishing takes place.
What is the financial impact (cost) of having more published nodes?
More messages flowing through the Azure IoT Hub will cost more. Refer to Microsoft Azure's pricing policies for Azure IoT Hub.
What is the performance impact?
What happens to performance when a single node has multiple subscriptions? What performance metric is valuable here?
Subscriptions from outside of ThingWorx Platform should not impair the platform's processing of telemetry. If multiple subscriptions exist in the platform, processing should be no slower than if those subscriptions were to distinct nodes.
Applications do not really subscribe to individual nodes. They subscribe to all telemetry coming through the Azure IoT Hub. ThingWorx Platform keeps a table to map properties to the nodes you care about, but the platform fields all of the telemetry that comes through the Hub, regardless of whether it is mapped. It is not really accurate to talk about other consumers subscribing to individual nodes. They also subscribe to all telemetry sent to the Hub and may or may not ignore some of it by their own internal logic.
As a consequence of more permanent orphan publications, does telemetry data get to ThingWorx bounded nodes more slowly?
The Azure IoT Hub Connector takes some amount of time to determine that incoming telemetry is not mapped. For this reason, the processing may be expected to be marginally slower than if those nodes were not published. However, the processing should not be slower than if those nodes were subscribed to by a ThingWorx Platform.
“Orphan publications” in this context means that published nodes that are not subscribed by a ThingWorx Platform.