ThingWorx Model Definition and Composer > System > Subsystems > Audit Subsystem > Audit Archive, Export, and Clean-up Operations > Exporting Audit Data > Sample Application for Exporting Audit Data
  
Sample Application for Exporting Audit Data
This sample application is a ThingWorx SchedulerThing that uses the SSH File Transfer Protocol (SFTP) Extension from the Tools section of the ThingWorx Marketplace to push the exported audit records to a remote SFTP server. You can configure the SchedulerThing to execute at the time and frequency best suited for your environment. The SchedulerThing maintains a property called lastExportDate, which is updated each time an export runs successfully.
The ThingWorx Audit Subsystem manages the export of audit entries as shown in the following figure and described below:
Flow of audit data during a custom export, from Online to Offline (Archived) to Customer File Repository
Online Audit Data
Audit data that is generated by the platform is persisted in the ThingWorx database. This data can be queried and filtered by audit category and date range. The amount of data in the Online storage is limited to the retention time configured for the DaysOnline parameter of the Audit Subsystem.
Offline Audit Data
Audit data is archived periodically, based on the frequency specified for the AuditArchiveScheduler Thing. The default frequency is every 24 hours.
When exporting audit records using this sample application, the ExportAuditData service includes a call to archive the current online audit data. This call ensures that the exported audit data will not have any gaps for audit entries generated but not yet archived by the AuditArchiveScheduler thing.
Installation and Configuration of the Extension
To install and configure the SFTP extension, follow these steps:
1. Download the "SSH File Transfer Protocol" Extension and the user guide for the extension from the ThingWorx Marketplace
2. Follow the instructions in the ThingWorx SFTP Extension User Guide to install the extension.
3. From ThingWorx Composer, create a new SFTP thing and select the SFTPRepositoryTemplate that was imported with the extension. On the Configuration page for the SFTP thing, configure the desired remote server location and credentials. See the user guide for the extension if you require details.
4. From Composer, create a new thing that uses the Scheduler thing template. Continue to the next section to set up this scheduler.
Setup of the SFTP Audit Exporter Scheduler
Configure the new scheduler as follows:
1. On the General Information page, configure the following properties:
Name — SFTPAuditExporter
Thing Template — Scheduler
2. On the Properties page, add the following properties:
Property
BaseType
Thing Template
HasDefaultValue
Persistent
lastExportDate
DATETIME
N/A
Selected, default value is 1/1/1970 00:00:00
Selected
targetRepository
THINGNAME
FileRepository
N/A
Selected
sftpThing
THINGNAME
SftpRepositoryTemplate
N/A
Selected
sftpDirectory
STRING
N/A
Selected, default value is *.*
Selected
3. On the Subscriptions page, add the following subscription information and code:
Source — Leave this field blank.
Event — Specify ScheduledEvent.
Enabled — Select this check box.
Script — Add the code from the code block below.
Here is the script to copy into the Script field:

// Sample script for a scheduleThing that will periodically call the ExportAuditData service on the
// AuditSubsystem to capture audit records prior to those records aging off the system.
// These then are transferred to an external SFTP server using the SFTP Thingworx Extension.

logger.info("Audit export starting");

// Property validation checks
if (me.sftpThing == undefined) {
throw "sftpThing property not set. Cannot continue.";
}
if (me.targetRepository == undefined) {
throw "targetRepository property not set. Cannot continue.";
}
if (me.sftpDirectory == undefined || me.sftpDirectory == null || me.sftpDirectory == "") {
// set a default value if not setup previously.
me.sftpDirectory = ".";
}

var exportEndTime = new Date();
var uploadDirFormatted = dateFormat(exportEndTime, "yyyy-MM"); // store in directories by month

var exportParams = {
startDate: me.lastExportDate, // Start after the last successsful export
endDate: exportEndTime, // Use the end time recorded above
targetRepositoryName: me.targetRepository, // destination repository from the scheduler's properties

// Set the targetPath to custom folder for exporting. The files will be deleted from this folder on
// completion of the export.
targetPath: "scheduledExport",

// Add the current date to the targetFileName
// The targetFileName must be unique. The ExportAuditData service will not overwrite an existing file.
targetFileName: "AuditArchive-" + dateFormat(exportEndTime, "yyyy-MM-dd-HHmm"),

// The locale for the exported audit messages. If undefined, the system default is used
locale: undefined
};

try {

Subsystems["AuditSubsystem"].ExportAuditData(exportParams);

// The export to the target repository now needs to be transferred to an external location.
// This example uses the SFTP Extension to transfer to a remote SFTP server.
// The location and credentials for the SFTP server are configured on the SFTP Thing.

// Check to make sure the destination directory exists as the SFTP server will not auto create it.
var checkDirParams = {
path: me.sftpDirectory
};
var findDirObj = {
name: uploadDirFormatted
};
if (Things[me.sftpThing].ListDirectories(checkDirParams).Find(findDirObj) == undefined) {
var createDirParams = {
path: me.sftpDirectory + "/" + uploadDirFormatted
};
Things[me.sftpThing].CreateFolder(createDirParams);
}

var uploadParams = {
FileRepository: me.targetRepository,
RepoFilePath: exportParams.targetPath + "/" + exportParams.targetFileName + ".zip",
RemoteFilePath: me.sftpDirectory + "/" + uploadDirFormatted + "/" + exportParams.targetFileName + ".zip"
};
Things[me.sftpThing].UploadFile(uploadParams);

// Upon successfully storing the export file, update the start time
// breadcrumb to the time the export was started.
me.lastExportDate = exportEndTime;

} catch (err) {
logger.error("Failed to export the auditing records: " + err);
} finally {
// Delete the temporary export file from the local repository
var deleteFileParams = {
path: exportParams.targetPath + "/" + exportParams.targetFileName + ".zip"
};
Things[me.targetRepository].DeleteFile(deleteFileParams);
}
logger.info("Audit export completed");
Configuration
The SFTPAuditExporter has the following configuration properties that must be set prior to using it:
targetRepository — Specifies the ThingWorx FileRepository thing to use as a temporary location for the exported audit entries. When it exports audit entries, this application stages them locally prior to sending out to the configured SFTP server. Once the transfer has completed, the local file will be removed.
sftpThing — Specifies the SFTP thing that you created in Step 3 of the Installation section above.
sftpDirectory — Specifies the parent directory on the remote SFTP server in which to store the audit entry files. Assuming that the remote server has a default directory for each user, the default value, *.*, specifies that the default directory should be used. This value can be changed, if desired.
In addition, on the Configuration page for this scheduler, the schedule time can be chosen to adjust when to perform the export. By default, the schedule executes each day at midnight.
Operational Notes
When the audit entries are saved on the remote SFTP server, the saved files will be named AuditArchive-<year>-<month>-<day>-<hour><minute>.zip, where the time indicates when the archive file was created. Each zip file contains a single, JSON-formatted file with the localized audit records. The archive files are organized into subdirectories by month, using the naming convention,<year>-<month>. For example, 2017-08.
To change the locale used for the audit entries, edit the scheduler script and set the locale field in the exportParams definition.