Tips for Choosing an API for Properties
The C SDK provides multiple functions that you can use to transfer data to the ThingWorx Platform. These functions are:
• twApi_WriteProperty()
• twApi_SetSubscriberProperty()
• twApi_SetSubscribedPropertyVTQ()
• twApi_PushSubscribedProperties()
These API calls offer both benefits and disadvantages that should be carefully considered.
twApi_WriteProperty()
int twApi_WriteProperty(enum entityTypeEnum entityType,
char * entityName, char * propertyName,
twPrimitive * value, int32_t timeout, char forceConnect)
This function directly targets a specific property on the ThingWorx Platform and creates the equivalent of an HTTP PUT request encoded as an AlwaysOn message.
The HTTP/1.1 RFC defines PUT as follows:
The PUT method requests that the enclosed entity be stored under the supplied Request-URI. If the Request-URI refers to an already existing resource, the enclosed entity SHOULD be considered as a modified version of the one residing on the origin server. If the Request-URI does not point to an existing resource … the origin server can create the resource with that URI.
The new value is transmitted to the server immediately if a connection is available.
In the ThingWorx world each property has a set of co-ordinates that take the place of the URI in an HTTP PUT request. They are:
• entityType — almost always THING
• entityName — The name of the Thing
• propertyName
The PUT request provides a value that must be a single ThingWorx primitive type such as a STRING, NUMBER, INFOTABLE, or BOOLEAN. No allowance for a VTQ (Value, Time, Quality) value is supported. Only the V (value) part can be provided. When it receives this request, the platform uses the time that the data arrived on the platform to complete the VTQ that is logged when data arrives. Quality is assumed to be GOOD.
A PUT request AlwaysOn message is stored to disk in the Offline Message Store without any timestamp information if the following conditions are all true:
• The device calling twApi_WriteProperty() is offline when this function is called.
• A new connection cannot be established.
• An offline message store is configured,
Later, when the connection is established, this data will adopt the receipt time on the server.
This API call should not be used to deliver data that is time sensitive in nature such as data that will be stored in a value stream. Its primary use should be to immediately modify a property value on the platform, perhaps to change a property of a thing controlling conjuration of the platform or another device.
Performance Impact
From a performance standpoint, this function will require a full message roundtrip which on average can take as a long as 400ms to complete. This effectively caps upstream data rates for any property change to 2 samples/sec.
The only optimization which is available is to set property name to *. When this is done, any InfoTable provided as a value will be assumed to be a set of PropertyName->Primitive value pairs. This allows a single PUT request to target multiple properties on a single thing but not to provide multiple values for a single property of a thing with distinct time stamps.
twApii_SetSubscribedPropertyVTQ()
int twApi_SetSubscribedPropertyVTQ(char * entityName,
char * propertyName, twPrimitive * value, DATETIME timestamp,
char * quality, char fold, char pushUpdate)
This API call is representative of twApi_SetSubscribedProperty() which is identical to twApi_SetSubscribedPropertyVTQ() where timestamp = the time at which the function was called. Quality will be set to GOOD. The difference in the functions is that the VTQ version supports the providing of an arbitrary time and quality value.
Neither of these functions send data directly to the platform. They instead store the new property value in a list of Value-time-quality data structures in memory. The number of values which can be stored in memory is a function of the maximum offline message store size. These two different features currently share the same configuration setting even though they are not connected in any way.
The actually emptying of what could be called the “In Memory Property Store” (as opposed to the Offline Message Store) is accomplished by calling:
int twApi_PushSubscribedProperties(char * entityName, char forceConnect)
This function is responsible for emptying of the In Memory Property Store. Normally an SDK application periodically calls this function to drain this store, creating one or more AlwaysOn messages containing multiple VTQ change entries. Note that the entityName parameter can be used to selectively drain the In Memory Property Store for just a single entity. If entityName is NULL, the entire store will be drained.
The method used to send these stored values to the server is very different than twApi_WriteProperty(). Instead of forming a PUT request, this function uses the twApi_InvokeService() API call to form a POST request.
HTTP/1.1 RFC for the definition of POST is…
The POST method is used to request that the origin server accept the entity enclosed in the request as a new subordinate of the resource identified by the Request-URI … The posted entity is subordinate to that URI in the same way that a file is subordinate to a directory containing it, a news article is subordinate to a newsgroup to which it is posted, or a record is subordinate to a database.
The bottom line here is that a POST request creates a new server resource. In the ThingWorx world, post request are used to call services or, potentially to create new model objects. In this context, the POST request will be used to perform a service call, targeted at a particular Thing/Entity.
The service called to post data to a Thing on the platform is called UpdateSubscribedPropertyValues() which is provided by the RemoteThing Thing template. It takes an argument called Values which is an InfoTable who’s columns are Value, Time and Quality. As many VTQ values as you like can be passed as rows to this InfoTable up until the InfoTable would exceed MaxMessageSize for an AlwaysOn Message on the platform side. Note that this service is provide on a specific thing so the VTQ data passed in this InfoTable must be for only one Thing per a service call.
Since each row in the Values InfoTable is a VTQ value, this type of service call forms the Ideal message for the Offline Message Store because delivery delays will not effect the time stamps built into the VTQ values. Any times that reach the platform are the time the data was captured, not the time it was received by the platform.
Performance Impact
twApi_SetSubscribedPropertyVTQ() always offers better performance than twApi_WriteProperty() because of the options available to optimize how data is sent to the platform. If we use the average AlwaysOn message roundtrip time of 400ms, assume the recommended max message size of 1,000,000 bytes and assume a VTQ is of a fixed length of 13 bytes (A number, a time stamp and the string GOOD) then ignoring the minimal overhead for the header and InfoTable structure you are looking at 190K VTQ samples a second being able to be pushed up to the server.
You are never going to get that kind of throughput in this situation because, unlike file transfer, each VTQ requires much more processing on the platform side. Past experience has shown that the platform will cap out around 10K VTQ samples a second. This pairs well with the maximum acquisition capabilities of most edge hardware.
Recommendations
The current ThingWorx EMS uses only twApi_WriteProperty(). It has been reported by customers that all data sent through the EMS is stored with its VTQ’s time stamps set to its arrival time on the platform. This is consistent with the implementation of twApi_WriteProperty(). This of course will cause the data to become time shifted if the resulting messages are generated while the device is offline. This can cause serious problems when this data is later analyzed.
There are two remedies for this problem on the EMS.
Simple API Change
Replace the use of twApi_WriteProperty() with a 1:1 call to twApi_SetSubscribedProperty() immediately followed by twApi_PushSubscribedProperties(). This will replace the PUT message with a POST message, preserving the time the data was captured on the server.
There will be no throughput performance increase. The EMS will still be capped at around 2-3 VTQ values per second.
Complex API Change
By attempting to make better use of the caching capabilities of twApi_SetSubscribedProperty(), The EMS could improve its data throughput by multiple orders of magnitude. This will require more complex decision making done by the EMS and new settings as well.
Since twApi_SetSubscribedProperty() does not send data, only cache it locally, knowing when to call twApi_PushSubscribedProperties() becomes an issue it has not had to deal with in the past. When should twApi_PushSubscribedProperties() be called? Well there are a few factors to consider. The first is, is the In Memory Property Store almost full? Are we online or can we get online? How long has the data been sitting on the Store? If the data collection rate is low, a customer will not appreciate data not being sent, even if the timestamp is preserved. If MaxMessageSize is set at 1,000,000 bytes, this is not a performance factor as the C SDK will automatically break down cached VTQs into multiple messages.
How do you solve this problem? Most customers poll twApi_PushSubscribedProperties(), twice a second should be more than enough to handle a 10k/s VTQ transfer rate.
Other Recommendations
There are two additional common sense changes I would suggest we make to the C SDK in regards to the twApi_SetSubscribedProperty() API.
The first is, break out the Max Offline Message Store size setting from the In Memory Property Store setting. Using one setting for two different caches seems wasteful to me as you might want the Offline Message Store to be much bigger than the In Memory Property Store but they currently must always be the same size.
Second, change the way the In Memory Property Store behaves when it is full. Right now, it drops new values. It should auto empty itself into the Offline Message Store if its available and as a last resort, discard the oldest values or fold on the newest values to attempt to salvage as much data as possible.