Analytics Server System Architecture for 8.5

Beginning in the 8.5 release, ThingWorx Analytics Server architecture has been simplified to include fewer components than past releases. This reduction is achieved mainly by consolidating the Analytics processes into two functional areas represented by the following microservices:

• Synchronous – Handles stateless jobs such as, synchronous predictive scoring, synchronous prescriptive scoring, and synchronous data query services.

• Asynchronous – Handles requests through a job management framework that processes the request and produces a set of results. Asynchronous jobs include training models, batch scoring, and retrieving batch job results. All related job states and configuration information is stored in a single database.

Some additional flexibility is provided in this release, depending on which microservices are installed. If only the Synchronous microservice is selected during installation, ZooKeeper and the Analytics Workers will not be installed. The synchronous jobs are stateless and do not require job management or a database.

If both microservices are installed, or only the Asynchronous microservice, then ZooKeeper and one or more common Analytics Workers will be included in the installation automatically.

Although the component structure of the Analytics Server has changed in this release, its representation in the ThingWorx Thing model has not. If you plan to deploy the Analytics Server with ThingWorx, a full set of Analytics Things will be created on installation, just as in earlier releases. For backwards compatibility, nothing about the APIs available via ThingWorx have changed. Analytics services are still grouped into the same logical areas by Thing. The same payloads and pathways are required to run the services.

A deployment of ThingWorx Analytics includes a series of logical components. The components are shown in the following architecture diagram and described in the Components section below.

1. Client – A system that can be used to submit job requests and queries to the ThingWorx Analytics microservices. You can interact with the microservices in one of the following ways:

◦ Via ThingWorx – When ThingWorx Analytics is deployed with ThingWorx, an adapter acts as an integration point between the two servers. The adapter automatically registers with ThingWorx and instantiates the Things that represent the connected ThingWorx Analytics services. The adapter uses the connected services to handle job requests submitted through the ThingWorx server.

The adapter does not scale horizontally, so there can be only one per system. The number of resources (cores) assigned to the adapter also controls the number of simultaneous requests the system can handle (about twice the number of cores the adapter is provided).

◦ Via FactoryTalk Analytics DataFlowML – When ThingWorx Analytics is deployed with DataFlowML, job requests and queries can be communicated from within a DataFlowML pipeline directly to the native Analytics APIs. For more information about accessing ThingWorx Analytics functionality from DataFlowML, see the FactoryTalk Analytics DataFlowML documentation.

2. Microservices – Two ThingWorx Analytics microservices are available. Each microservice includes a REST API component. During installation, one or both of these microservices can be installed. Note that if you select only the Synchronous microservice during installation, ZooKeeper and the Analytics Workers will not be installed. The synchronous jobs are stateless and do not require job management or a database.

When the Asynchronous microservice is installed, whether by itself or in combination with the Synchronous microservice, then ZooKeeper and three common Analytics Workers are automatically included in the installation. When the Asynchronous microservice receives job requests and other queries from the adapter, the REST APIs submit them to the common workers. The workers create a queued system for executing submitted jobs as specified by the requests.

3. Database – An embedded H2 database that handles the data from the asynchronous jobs. Before the 8.5 architecture restructuring, each microservice included its own database. With the consolidation of asynchronous processes into a single microservice, the database requirements are simplified. One database can handle the data from all of the asynchronous jobs. A jobType column has been added so that queries can locate the relevant data.

4. Common Workers – Common workers can process any type of asynchronous job requests. Workers pick up the jobs from the ZooKeeper system and do the work. Currently, three workers are provided during the installation process. However, the number of workers can be scaled up after the installation is complete. Each worker can handle one job at a time, so if there is only one worker in the system, only one job can be processed at a time for the system. Keep in mind, multiple workers will allow more job requests to be processed simultaneously, but will not increase the speed with which any given job is handled.

Since the workers do the heaviest processing in the system, they require allocation of the most system resources. Recommended: 4 GB of memory and at least one full core per worker.

5. ZooKeeper – A service that manages the asynchronous job queues and provides configuration information. Other components in the system register with ZooKeeper and obtain the configuration information they need. ZooKeeper also provides coordination between the microservice and the common workers.