Clean-up of the Offline Audit Archives
When auditing is enabled, you need to monitor the disk space to determine if you need to clean up the AuditArchiveFileRepository of the audit subsystem. Depending on the chosen implementation, the AuditArchiveFileRepository contains either of two subdirectories, one for each implementation:
• Direct Persistence — AuditArchiveDirectPersistence
• Data Table — AuditArchive
If you choose to use the Data Table implementation and later switch to the Direct Persistence implementation, the audit data from the Data Table implementation is preserved. The directory structure on the file repository will look like this:
 | The reverse is also true - if you switch from Direct Persistence to Data Table, the Direct Persistence data is preserved. |
The following metrics can help you make the decision about the necessity to clean up the repository:
• AuditArchiveDiskSpaceUsed (shown in MB) — how much disk space is used by the archive repository.
• AuditArchiveFreeDiskSpace (shown in MB) — how much disk space remains free.
There are two scheduler Things for the CleanUpOfflineAudit service:
• AuditArchiveCleanupScheduler — By default, schedules a job on the last Friday of every month at 6 PM. You can change the schedule by navigating to the AuditArchiveCleanupScheduler in ThingWorx Composer and setting a different CRON expression for it in its Configuration page.
The job calls the CleanUpOfflineAudit service. Depending on the currently active implementation, CleanUpOfflineAudit deletes subdirectories of archived audit data only from one of the above directories. When the Data Table implementation is active, only subdirectories from AuditArchive are deleted. Similarly, when the Direct Persistence is active, only subdirectories from AuditArchiveDirectPersistence are deleted.
The subdirectories that are deleted are those older than the value of the DaysToArchive property of this scheduler. You can set the value of this property from ThingWorx Composer. Its default value is 180 days.
In addition to using the scheduler, you can call the CleanUpOfflineAudit service manually. When called manually, the service takes the date as input and removes data from the subdirectory for the currently active implementation, AuditArchive for the Data Table implementation or AuditArchiveDirectPersistence for the Direct Persistence implementation.
• AuditArchiveCleanupNotificationScheduler — Notifies users that the clean-up service will be run. By default, this scheduler Thing is triggered on the last Friday of every month at 6 AM and fires an event to notify any subscribers to the event that the clean-up service will be run at 6 PM (the default time that the job scheduled by the AuditArchiveCleanupScheduler will run). As with the scheduler for the cleanup service, you can change the day and/or the frequency for running this notification service. You can also subscribe to the event and add your own logic in a script that you want to run based on the event. Refer to the section below, Subscribing
to the Event, for details.
The clean-up service, CleanUpOfflineAudit, takes as input olderThanDate. This argument is of type DateTime, in a format that is based on the locale The service deletes audit entries from offline storage that are older than the specified date.
 | You can also run the CleanUpOfflineAudit service manually. |
For output, the clean-up service provides information about the directories that were removed successfully and, if applicable, directories that the service failed to remove. Note that a failure to remove a file or directory does not stop the clean-up process. The service skips the failing directory and moves on to the next one.
The example that follows show how the clean-up service works.
Example 1. Using the CleanUpOfflineAudit Service
For example, suppose the service is invoked with the date 2020-03-07 and the repository subdirectory for the Direct Persistence implementation, AuditArchiveDirectPersistence, contains the following subdirectories:
After the clean-up service runs, the repository contains only the following subdirectories:
The subdirectory named 2020-03-07 is for the date specified as the olderThanDate, and the subdirectory named 2020-03-08 is later than the olderThanDate. These two remain, while the clean-up service deletes the other three subdirectories because they are older than the olderThanDate.
Only administrators have permission to invoke the clean-up service. For a non-admin user to be able to invoke it, an administrator needs to give the user the proper permissions, using a service override. For information on service overrides, refer to the topic, Service Overrides in the security section of this Help Center.
If the clean-up service fails for some reason, an error message is returned with the reason for the failure.
Best Practice for Setting the Offline Retention Time Value
If the metrics reporting feature is enabled, relevant metrics about the audit subsystem and its audit archive file repository are available to you through ThingWorx Composer. To access the metrics, select > . You can see the entire list of metrics in the topic, Metrics for Auditing Activities. As stated above, the following metrics are good indicators of the need to clean up archived audit data in the repository:
• AuditArchiveDiskSpaceUsed (shown in MB) — how much disk space is used by the archived data in the AuditArchiveFileRepository.
• AuditArchiveFreeDiskSpace (shown in MB) — how much disk space remains free.
If the disk space used is high and free space low, it is recommended that you clean up the repository as soon as possible by invoking the clean-up service manually and specifying a date that the service will use as the starting point for removing all files and directories that are older than the date you specify. Also, consider setting DaysToArchive to a lower value so that you maintain the amount of free disk space that your environment requires.
Subscribing to the Event
The AuditArchiveCleanupNotificationScheduler triggers an event, called AuditCleanupNotification, and notifies subscribers about the upcoming file clean-up operation. You can subscribe to the event and add your own logic in a script that you want to run based on the event.
The job associated with the AuditArchiveCleanupScheduler has a property called DaysToArchive. By default the value of this property is 180 days (six months). You can change this value from ThingWorx Composer. The scheduled job runs monthly to clean up all the data that is older than the specified DaysToArchive. You can also modify the frequency of the automatic service invocation in Composer.