ThingWorx Model Definition and Composer > System > Subsystems > Audit Subsystem > Audit Archive, Export, and Clean-up Operations > Clean-up of the Audit Archives
  
Clean-up of the Audit Archives
When auditing is enabled, you need to monitor the disk space to determine if you need to clean up the File Repository of the Audit Subsystem where archived audit files are stored (AuditArchiveFileRepository). The following metrics can help you make the decision about the necessity to clean up the repository:
AuditArchiveDiskSpaceUsed (shown in MB) — how much disk space is used by the archive repository.
AuditArchiveFreeDiskSpace (shown in MB) — how much disk space remains free.
There are two scheduler things for the ExportAuditData service:
AuditArchiveCleanupScheduler — By default, schedules a job on the last Friday of every month at 6 PM. The job calls the CleanUpOfflineAudit service and deletes all the offline audit data directories from the AuditArchiveFileRepository that is older than the DaysToArchive property of this scheduler thing. This property can be changed from ThingWorx Composer. Its default value is 180 days. The CleanUpOfflineAudit service can be called manually as well. When called manually, the service takes the date as input and removes from the repository all the audit data directories that are older than that date.
AuditArchiveCleanupNotificationScheduler - Notifies users that the clean-up service will be run. By default, this scheduler thing is triggered on the last Friday of every month at 6 AM and fires an event to notify any subscribers to the event that the clean-up service will be run at 6 PM (the default time that the job scheduled by the AuditArchiveCleanupScheduler will run). As with the schedule for the cleanup service, you can change the day and/or the frequency that this notification service runs. You can also subscribe to the event and add your own logic in a script that you want to run based on the event. See the section below, Subscribing to the Event, for details.
The clean-up service, CleanUpOfflineAudit, takes as input olderThanDate (of type DateTime, in the formatYYYY-MM-DD). The service deletes audit entries from offline storage that are older than the specified date. Note that you can also run this service manually.
For output, the clean-up service provides information about the directories that were removed successfully and, if applicable, directories that the service failed to remove. Note that a failure to remove a file or directory does not stop the clean-up process. The service skips the failing directory and moves on to the next one.
For example, suppose the service is invoked with the date 2017-06-07 and the root directory of the Audit Archive File Repository contains subdirectories with the following names:
After the clean-up service runs, the repository looks like this:
Only administrators have permission to invoke the clean-up service. For a non-admin user to be able to invoke it, an administrator needs to give the user the proper permissions, using a Service Override. For information on Service Overrides, see the topic, Service Overrides in the Security section of this Help Center.
When the clean-up service finishes successfully, a message is written to the server log that shows the amount of space that was cleaned up. If the clean-up service fails for some reason, an error message is returned with the reason for the failure.
Best Practice for Setting the Offline Retention Time Value
If the metrics reporting feature is enabled, relevant metrics about the Audit Subsystem and its Audit Archive File Repository are available to you through ThingWorx Composer (select Monitoring > Subsystem). You can see the entire list of metrics in the topic, Metrics for Auditing Activities. As stated above, the following metrics should be good indicators of the need to clean up archived audit data in the repository:
AuditArchiveDiskSpaceUsed (shown in MB) — how much disk space is used by the archived data in the AuditArchiveFileRepository.
AuditArchiveFreeDiskSpace (shown in MB) — how much disk space remains free.
If the disk space used is high and free space low, it is recommended that you clean up the repository as soon as possible by invoking the clean-up service manually and specifying a date that the service will use as the starting point for removing all files and directories that are older than the date you specify. Also, consider setting DaysToArchive to a lower value so that you maintain the amount of free disk space that your environment requires.
Subscribing to the Event
The AuditArchiveCleanupNotificationScheduler triggers an event, called AuditCleanupNotification, and notifies subscribers about the upcoming file clean-up operation. You can subscribe to the event and add your own logic in a script that you want to run based on the event.
The job associated with the AuditArchiveCleanupScheduler has a property called DaysToArchive. By default the value of this property is 180 days (six months). You can change this value from ThingWorx Composer. The scheduled job runs monthly to clean up all the data that is older than the specified DaysToArchive. You can also modify the frequency of the automatic service invocation in Composer.