Archiving and Purging Online Audit Data (Direct Persistence)
The Direct Persistence implementation of the Audit Subsystem (default) provides a less expensive storage option for audit data than the Data Table implementation. To archive online audit data, use the ArchiveAuditHistoryDirectPersistence. This service copies online audit data to offline archive zip files within the AuditArchiveFileRepository. For the Direct Persistence implementation, the audit data is stored in the AuditArchiveDirectPersistence directory of the repository.
The input parameter for the ArchiveAuditHistoryDirectPersistence service is the olderThanDate, which has the ThingWorx base type, DATETIME, in a format based on your locale set in ThingWorx Composer. For example:
Default locale (U.S. English) — The format is MM-DD-YYYY.
de locale (German) — The format is DD-MM-YYYY.
When setting a value for this parameter, you are specifying the date before which to archive audit entries in the online storage. For example, if you want to archive all online audit entries prior to March 27, 2020, specify 03-27-2020 for the Default locale or 27-03-2020 for the de locale as the olderThanDate. Here are examples of the Input page for each locale, Default and de (German):
Default Locale Input
de Locale Date Input
The format in the example above is DD-MM-YYYY. This olderThanDate is converted to the format, YYYY-MM-DD for any locale when the archive file is created. The size of each archive file is controlled by the audit subsystem parameter, Maximum entries for online audit data. The default value is 50000.
The archive file name includes a chunk number. For example, The chunk size is also a configuration parameter of the audit subsystem, Archive Audit Chunk Size. This parameter determines the maximum number of rows from the database to be compressed within a single chunk. For example, if the chunk size parameter is set to its minimum value, 10000 rows and there are 35000 rows in the database for a single day, the archive service creates four chunks, with the numbering starting at 0:
If the archive service is called more than once on the same day and chunks have already been created, each new call to the service creates a separate chunk, even if the previous chunk was not full. in the example above, the last chunk would have room for more rows, but if the service is called again on the same day, the name of the first archive file created is
For the Direct Persistence implementation, the Number of days of online audit data to maintain parameter of the audit subsystem controls how long audit data remains online. This parameter specifies the number of days that online data should remain in the online storage. The default number of days is 60. If you keep this default value, any data that is older than 60 days is automatically archived to offline storage after 60 days.
Online data can also be manually archived by invoking the AuditArchiveSchedulerDirectPersistence service. To archive online audit data manually:
1. From the Browse panel in ThingWorx Composer, go to System > Subsystems, and select the AuditSubsystem.
2. In the AuditSubsystem page, select Services, and then locate the AuditArchiveSchedulerDirectPersistence service.
3. Click to run the ArchiveAuditHistoryDirectPersistence service.
The Input/Output page appears, with input content similar to the examples above.
4. Set an olderThanDate, click Execute. in the lower right corner of the page. Results are displayed in the output side of the page.
If desired, you can save the olderThanDate as an input set that can be reused.
The ArchiveAuditHistoryDirectPersistence service copies all audit data from online storage to the offline and stores it in the directory AuditArchiveDirectPersistence of the AuditArchiveFileRepository. The archive files are compressed (zip) and their names are derived from the date on which they were created. For example, an archive file created on March 26, 2020, is named
Since the archive operation copies the online audit data but does not delete it, the online data remains in the database until it is purged from the online storage, either manually or based on a scheduler. To purge online audit data manually, navigate to the Audit Subsystem Services page and run the PurgeAuditData service
The purge operation deletes only data that has been archived. If it finds some audit messages to delete that were not previously archived, it first archives them, and only after that deletes them.
Never disable the purge scheduler. Doing so prevents the system from periodically moving the audit data offline and then deleting it. Allowing the data to accumulate without periodic purge operations may eventually bring the platform down.
Was this helpful?