Using the Connection Server in a ThingWorx High Availability (HA) Clustering Environment
This topic explains the following aspects of running the ThingWorx Connection Server in a ThingWorx High Availability (HA) Clustering environment:
When using ThingWorx HA Clustering, your incoming traffic ("ingress") should be routed through a load balancer that sits in front of at least two Connection Servers. The Connection Servers will distribute the ingress messages, as explained in the next section. For complete information about HA clustering, refer to the
ThingWorx High Availability section of the ThingWorx Platform 9 Help Center.
How the Connection Server Handles Ingress Messages
The Connection Server uses a round robin policy for distributing ingress messages across the available ThingWorx Platform instances. This policy provides load distribution and fault tolerance.
Here are the rules for this round robin policy:
1. Only connected and writable ThingWorx instances are considered for routing an incoming message. In this context, writable means that the buffer for writing messages has not exceeded the high watermark configured in the property, cx-server.transport.websockets.connections.highWaterMark.
2. Each message is routed to the next connected and writable ThingWorx instance, based on a circular list of ThingWorx instances.
3. If one instance in the list is not connected or not writable, the Connection Server routes the message to the next instance in the list.
4. If all ThingWorx instances are either not connected or not writable, request messages result in an exception being returned. For the Connection Server, the result is a 500 Internal Error response being sent to the edge device.
When a Connection Server is disconnected from all ThingWorx instances, all Edge WebSockets are closed. However, the Connection Server does not prevent a new Edge WebSocket from connecting.
Multi-Part Request Messages
The following rules govern how multi-part messages are routed by a Connection Server in an HA cluster:
1. The first chunk of a multi-part message is routed like any other message. It is sent to the next connected and writable ThingWorx instance according to the round-robin policy.
2. Subsequent chunks of the multi-part message routed to the same ThingWorx instance that the first chunk was routed to.
3. If the ThingWorx instance to which the chunks are routed becomes disconnected or not writable, the Connection Server fails the request and sends an error response to the edge device.
Service Discovery
The service discovery feature enables the Connection Server to react to dynamic configuration of ThingWorx Platform instances in a ThingWorx High Availability Clustering environment. Administrators do not have to configure the platform instances manually and restart the Connection Server when the platform instances change.
More specifically, the service discovery feature enables the Connection Server to respond to platforms being added to and removed from the cluster. Detection of ThingWorx Platform instances going up and down keeps the Connection Server instances connected correctly in the cluster. The Remote Things connected through HAProxy can still receive property updates from the Edge when one platform instance in the cluster goes down.
Configuration of
service discovery for a Connection Server is required when operating in a ThingWorx High Availability Clustering environment.