File Transfer Subsystem
The file transfer subsystem provides the required methods to manage file transfers between Remote Things, file repositories, and federated servers.
Unless specified, file transfers between an edge client and the ThingWorx Platform require the client to be connected from the beginning of the transfer through completion. ThingWorx Platform file transfers can be initiated while the edge client is disconnected by instructing the ThingWorx Platform to perform the copy using the queueable functionality. Queueable file transfers are placed in an offline queue and execute once the agent is connected.
 |
If you are using PostgreSQL as your persistence provider, the offline queue is persisted to support failover for ThingWorx high availability.
|
After the transfer is initiated, the edge client cannot disconnect or the transfer will fail. File transfers that are queued will expire if they do not begin before the Time To Live (sec) of a Queueable File Transfer configuration setting value.
Generally, file transfers are controlled by the ThingWorx Platform. However, you can have edge-controlled file transfers, which are activated by adding the EdgeControlled Thing Shape to the Remote Thing that is participating in the file transfer. Edge-controlled file transfers must have either the source or the destination set as a FileRepository Thing on the ThingWorx Platform. In an edge-controlled file transfer, the transfer is controlled by the edge client. The edge invokes the Touch service to update the timestamp of the file transfer as it progresses. If the edge does not do this, the ThingWorx Platform times out the file transfer. The edge indicates whether the transfer was successful or completed with an error.
You can embed context by providing the metadata parameter when initiating the file transfer. This field is a JSON object, which can contain any arbitrary JSON. Certain base functionality will result in the ThingWorx Platform using this field to supply additional instructions to the edge client. For these transfers, the information in the metadata field should not be modified by any process other than the ThingWorx Platform.
File Transfer between Federated Servers
To transfer files between federated servers, do the following:
1. Configure a federation between ThingWorx ServerA and ThingWorx ServerB. For more information, see Configuring a Federation.
2. Add ThingA to ServerA using the FileRepository Thing Template.
a. Select the Published checkbox for ThingA so it is accessible by other ThingWorx servers.
3. Add RemoteThingA to ServerB using the RemoteThingWithFileTransfer Thing Template.
a. In the Identifier field, enter ThingA@ServerA.
4. Copy your file (for example, test.txt) to the SystemRepository root folder of ServerB.
5. Open the FileTransferSubsystem on ServerB and execute the copy service with the following parameter values:
◦ sourceRepo: SystemRepository
◦ sourcePath: /
◦ sourceFile: test.txt
◦ targetRepo: RemoteThingA
◦ targetPath: /
◦ targetFile: test.txt
◦ Do not change the other parameters from their default values.
6. Go to the /ThingworxStorage/repository/ThingA folder on ServerA and verify that the test.txt file was successfully copied there.
Configuration
|
File Transfer Settings
|
Data Type
|
Default
|
Notes
|
|---|---|---|---|
|
Min Threads Allocated to File Transfer Pool
|
NUMBER
|
10
|
|
|
Max Threads Allocated to File Transfer Pool
|
NUMBER
|
10
|
Asynchronous file transfers could be lost if ThingWorx goes down. For example, this setting has the default value of 10, and 50 long-running file transfers are submitted. If ThingWorx goes down, 40 would be lost.
|
|
Max Queue Entries Before Adding New Working Thread
|
NUMBER
|
100
|
The maximum number of active transfers allowed at a time.
|
|
Idle Thread Timeout (sec)
|
NUMBER
|
600000
|
|
|
FileTransfer Idle Timeout (sec)
|
NUMBER
|
30
|
|
|
Max FileTransfer block size (bytes)
|
NUMBER
|
128000
|
This variable imposes a maximum block size during file transfer at a system level. EMS configuration still takes precedence, however, if the EMS is configured with a larger buffer_size than what is configured in this variable, then this variable will cap this block size. If larger block sizes (128KB) are configured through EMS, this value must be increased. The maximum compile level is 1MB.
|
|
Max FileTransfer size (bytes)
|
NUMBER
|
100000000
|
If the source file is larger than this value, the transfer fails and an error message is displayed.
|
|
Max File Transfers Allowed in Offline Queue
|
NUMBER
|
50000
|
|
|
Max File Transfers Allowed Per Thing in Offline Queue
|
NUMBER
|
10
|
|
|
Time to Live (sec) of a Queueable File Transfer
|
NUMBER
|
86400
|
A file transfer will be removed from the offline queue after this specified amount of time.
|
|
Total Max Edge-Controlled File Transfers Allowed
|
NUMBER
|
500
|
The total concurrent edge-controlled transfers across the platform. The range is 1–1000 transfers. This is separate from the maximum allowed always-on transfers (which are controlled by the Max Queue Entries Before Adding New Working Thread setting.)
|
|
Total Max Edge-Controlled File Transfers Allowed Per Thing
|
NUMBER
|
2
|
The maximum number of concurrent transfers allowed to or from an edge-controlled Thing. The range is 1–100 transfers. For example, a value of 2 means that one edge-controlled Thing can only have two active transfers (upload or download) at a given time. Subsequent requests to dequeue file transfer egress are rejected until there is capacity.
|
|
Idle Timeout for Edge-Controlled File Transfers (sec)
|
NUMBER
|
600
|
The maximum time an active job can remain active without being acted on (for example, by data transfer or job state update). The range is 1–3600 seconds. This is similar to the idle timeout for always-on transfer jobs, but it is usually longer to account for the ping cycle of polling devices.
|