Using the eMessage Connector in a ThingWorx High Availability (HA) Clustering Environment
When using ThingWorx HA Clustering, your incoming traffic ("ingress") should be routed through a load balancer that sits in front of at least two eMessage Connectors. The Connectors will distribute the ingress messages. In addition, an HA Cluster installation requires a ThingWorx Connection Server that sits beside the eMessage Connector, as shown in the following diagram. The Connection Server should never be between the eMessage Connector and the ThingWorx Platform.
The rest of this topic explains the following aspects of running the ThingWorx eMessage Connector in a ThingWorx High Availability (HA) Clustering environment:
The eMessage Connector uses a round robin policy for distributing ingress messages across the available ThingWorx Platform instances. This policy provides load distribution and fault tolerance.
Here are the rules for this round robin policy:
1. Only connected and writable ThingWorx instances are considered for routing an incoming message. In this context, writable means that the buffer for writing messages has not exceeded the high watermark configured in the property, cx-server.transport.websockets.connections.highWaterMark.
2. Each message is routed to the next connected and writable ThingWorx instance, based on a circular list of ThingWorx instances.
3. If one instance in the list is not connected or not writable, the eMessage Connector routes the message to the next instance in the list.
4. If all ThingWorx instances are either not connected or not writable, request messages result in an exception being returned. For the eMessage Connector, the result is a 500 Internal Error response being sent to the edge device.
When an eMessage Connector is disconnected from all ThingWorx instances, all Edge WebSockets are closed. However, the Connector does not prevent a new Edge WebSocket from connecting.
Multi-Part Request Messages
The following rules govern how multi-part messages are routed by an eMessage Connector in an HA cluster:
1. The first chunk of a multi-part message is routed like any other message. It is sent to the next connected and writable ThingWorx instance according to the round-robin policy.
2. Subsequent chunks of the multi-part message routed to the same ThingWorx instance that the first chunk was routed to.
3. If the ThingWorx instance to which the chunks are routed becomes disconnected or not writable, the Connection Server fails the request and sends an error response to the edge device.
The service discovery feature enables the Connection Server to react to dynamic configuration of ThingWorx Platform instances in a ThingWorx High Availability Clustering environment. Administrators do not have to configure the platform instances manually and restart the Connection Server when the platform instances change.
More specifically, the service discovery feature enables the Connection Server to respond to platforms being added to and removed from the cluster. Detection of ThingWorx Platform instances going up and down keeps the Connections Server instances connected correctly in the cluster. The Remote Things connected through HAProxy can still receive property updates from the Edge when one platform instance in the cluster goes down.
Configuration of
service discovery for an eMessage Connector is required when operating in a ThingWorx High Availability Clustering environment.