Archive Cleanup

The Archive Cleanup page enables you to manage disk space usage by cleaning up archived messages and defragmenting the Error Queue:

Archive Cleanup

Copies of messages that leave Rhapsody (through an output communication point or by deletion from a queue) and all notification events raised in Rhapsody are kept for archival purposes. Users can view or resend these messages. However, it is not advisable that messages be kept indefinitely, as they do take up disk space. The archive cleanup is the mechanism by which messages are cleaned up.

An archive cleanup task can be run manually or scheduled to run every x hours (where x is a whole number greater than one) at y minutes past the hour (0 - 59), daily at a specified time, weekly (at a specified day and time), or monthly (at a specified day and time). Alternatively, you can use a cron expression to schedule an archive cleanup task.

The settings for archive cleanup are set in the Archive Cleanup panel of the Archive Cleanup page:

Manual Cleanup

You can also run an archive cleanup task manually.

Click the Cleanup Now link to start an archive cleanup task manually. A manual archive cleanup task cleans up any messages and logs older than the time specified in the Keep archive data for, Keep logs for and Keep notification events for fields.

Automatic Cleanup

For an automatic cleanup the following settings must be specified:

• Keep archive data for - the number of hours or days messages will be retained in the archive. After this time, the messages will no longer be available for viewing or resending.

• Keep logs for - the number of days worth of binary logs to keep with Rhapsody. This is the number of days the log messages will be visible via the Management Console.

  This setting only relates to the binary logs used for visibility in the Management Console, and not the textual logs kept in the <rhapsody>\logs directory.

• Cleanup notification event history - the number of days to keep notification events (they are kept indefinitely when the checkbox is unchecked). By default, notification events are archived and deleted after 90 days.
• Cleanup backups - the number of days to keep backups (they are kept indefinitely when the checkbox is unchecked). By default, backups are kept indefinitely. 
• Cleanup - when the archive cleanup task should run. You can schedule archive cleanup to run according to your requirements, by selecting and completing the cleanup fields. Alternatively, select the Advanced button and enter the appropriate details in the Cron Expression field to schedule archive cleanup using a cron expression.

Select the Apply button to start the automatic archive cleanup process.

Scheduling Archive Cleanup

The archive cleanup task can have a minor impact on engine performance. Each component queue is locked for a small period of time (generally sub-second) while the queue is analyzed for live messages.

Archive Cleanup can be scheduled to run every x hours (where x is a whole number greater than one) at y minutes past the hour (0 - 59), daily at a specified time, weekly (at a specified day and time), or monthly (ar a specified day and time). By default, the archive cleanup process runs every hour (on the hour).

However, you can schedule the archive cleanup task to run according to your requirements, by completing the frequency fields. Alternatively, select the Advanced button and enter the appropriate details in the Cron Expression field to schedule an archive cleanup task using a cron expression. You must use the syntax described in Syntax Rules.

Syntax Rules

Archive cleanup uses the cron expressions as understood by the Quartz Scheduler.

The format for the expression is <second> <minute> <hour> <day-of-month> <month> <day-of-week>, where each field is defined as:

• <second> - the second at which the action should occur, 0 through 59. Do not set as *, otherwise the action is run every second.
• <minute> - the minute at which the action should occur, 0 through 59. Do not set as *, otherwise the action is run every minute.
• <hour> - the hour at which the action should occur, 0 through 23. Use * for every hour.
• <day-of-month> - the day of the month on which the action should occur, 1 through 31. Use * for every day. Use ? if the day is unknown.
• <month> - the month in which the action should occur, 1 through 12. Use * for every month.
• <day-of-week> - the day of the week on which the action should occur. Enter three-letter acronyms for the days of the week. For example, Mon, Tues, Wed, Thu, and so on. Alternatively, you can use 1 through 7, where 1 is Sunday, 2 is Monday and 7 is Saturday. Use * for every day. Use ? if the day is unknown.

Special Characters

The following special characters are also allowed:

Example Expressions

The following table lists some examples of basic expressions:

Cleanup Status

The archive cleanup task may detect files that are eligible for clean up but cannot be deleted at that moment in time. Typically, this is because the file has been read at most five minutes prior to Archive Cleanup attempting to delete it. An archive cleanup task can be run again at a later time when it will attempt to delete these files again.

If the number of files that fail to be deleted continues to grow, then you should look in the archive cleanup logs to see which files could not be removed. By default, the logs are stored in log/archiveCleanupLog/cleanup_log.txt, where all successfully and unsuccessfully deleted files are logged during an archive cleanup. The log is a Log4J file which can be configured in the log4j.properties file with the logger name ArchiveCleanup.

In the worst case scenario, files that fail to be deleted during Archive Cleanup will be marked for deletion when the JVM exits. If you encounter issues where the archive cleanup task continually fails to delete a file, contact Rhapsody Support.

Archive cleanup also calculates the number of messages on the Error Queue that are eligible for defragmentation.

Detecting Old Error Messages

Archive Cleanup automatically detects when defragmentation is required on the Error Queue and notifies the user with a count of messages that are eligible for defragmentation and the following notifications:

• A warning banner on the Archive Cleanup page. This banner is automatically dismissed only when there are no more messages eligible for defragmentation on the Error Queue.
• An alert in the System Monitor. Refer to Custom Thresholds to configure this alert. The alert is also presented as a warning icon in the navigation menu.

Refer to Error Queue Defragmentation for details on how Rhapsody defragments the Error Queue.

Live Message Scanning During Archive Cleanup

Live message scanning enables archive cleanup to determine which messages are live in the communication point input queue. You can set the input queue size threshold for live messages using the rhapsody.properties file in order to determine how to perform live message scanning during archive cleanup:

Error Queue Defragmentation

Rhapsody stores message metadata in files, with one file usually containing metadata for multiple messages. A file can only be removed once all of the messages it references are eligible for archive cleanup. Consequently, messages left on the Error Queue prevent not only themselves from being cleaned up, but also other messages that were originally received at the same time. Reducing the file size can alleviate the issue, but greatly increases the overhead required to perform a cleanup.

Rhapsody can defragment messages on the Error Queue by re-writing them into new files in the message store, so that the old files can then be removed by the archive cleanup task. The Error Queue defragmentation task can have a minor impact on engine performance.

When a message has been defragmented, it retains the message data of the original message but not the full event path. A defragmented message only retains the first and last message events.

In Message View, you can view the truncated path of a defragmented message and its message body and properties, and display the details of the original message by clicking its message ID. If the original message has been cleaned up, clicking the message ID displays a message indicating the original message has been cleaned up.

Messages determined to be eligible for defragmentation are normally error messages older than Keep archive data for period plus a configurable duration (two days by default). 

While Error Queue defragmentation functionality can assist you to manage the Error Queue, it is not a substitute for Error Queue management.

Running the Defragmentation Task

To run an Error Queue defragmentation task:

1. Configure the settings for the defragmentation task. Refer to Configuring the Defragmentation Task for details.
2. Run the task either:
   • Manually by clicking the Defrag Now link.
   • Automatically after archive cleanup or by scheduling it (both are disabled by default). Refer to Scheduling the Defragmentation Task for details on scheduling the defragmentation task.
3. Review the status of the defragmentation task. Refer to Defragmentation Task Status for details.

Configuring the Defragmentation Task

You can manage Error Queue defragmentation tasks through the Error Queue defragmentation panel on Archive Cleanup page:

You can set the following configuration options through the Management Console:

You can set the following configuration options through the  rhapsody.properties  file:

# Priority for the thread that runs the message defrag
#CleanupService.messageDefrag.threadPriority=1
# Minimum disk space available to allow defrag
#CleanupService.messageDefrag.minimumSpaceMegabytes=2048

Scheduling the Defragmentation Task

You can schedule to run the defragmentation task according to the following schedule:

Refer to Scheduling Archive Cleanup for details on scheduling.

Defragmentation Task Status

The status of any current  defragmentation task and results of the last defragmentation task are displayed on the archive cleanup page in the Management Console: