Chapter 16. Using Audit Logs

Table of Contents
16.1. Audit Log Types
16.2. Audit Log File Formats
16.3. Audit Configuration
16.4. Generating Reports

OpenIDM auditing can publish and log all relevant system activity to the targets you specify. Auditing can include data from reconciliation as a basis for reporting, access details, and activity logs that capture operations on internal (managed) objects and external (system) objects. Auditing provides the data for all the relevant reports, including orphan account reports.

The auditing interface allows you to push auditing data to files and to the OpenIDM repository.

16.1. Audit Log Types

This section describes the types of audit log OpenIDM provides.

Access Log

OpenIDM writes messages concerning access to the REST API in this log.

Default file: openidm/audit/access.csv

Activity Log

OpenIDM logs operations on internal (managed) and external (system) objects to this log type.

Entries in the activity log contain identifiers both for the reconciliation or synchronization action that triggered the activity, and also for the original caller and the relationships between related actions.

Default file: openidm/audit/activity.csv

Reconciliation Log

OpenIDM logs the results of a reconciliation run, including situations and the resulting actions taken to this log type. The activity log contains details about the actions, where log entries display parent activity identifiers, recon/reconID.

Default file: openidm/audit/recon.csv

Where an action happens in the context of a higher level business function, the log entry points to a parent activity for the context. The relationships are hierarchical. For example, a synchronization operation could result from scheduled reconciliation for an object type. OpenIDM also logs the top level root activity with each entry, making it possible to query related activities.

16.2. Audit Log File Formats

This section describes the audit log file formats to help you map these to the reports you generate.

Access Log Fields

Access messages are split into the following fields.

"_id"

UUID for the message object, such as "0419d364-1b3d-4e4f-b769-555c3ca098b0"

"action"

Action requested, such as "authenticate"

"ip"

IP address of the client. For access from the local host, this can appear for example as "0:0:0:0:0:0:0:1%0".

"principal"

Principal requesting the operation, such as "openidm-admin"

"roles"

Roles associated with the principal, such as "[openidm-admin, openidm-authorized]"

"status"

Result of the operation, such as "SUCCESS"

"timestamp"

Time when OpenIDM logged the message, in UTC format, for example "2012-11-18T08:48:00.160Z"

Activity Log Fields

Activity messages are split into the following fields.

"_id"

UUID for the message object, such as "0419d364-1b3d-4e4f-b769-555c3ca098b0"

"action"

Action performed, such as "create". See the section on Event Types for a list.

"activityId"

UUID for the activity corresponding to the UUID of the resource context

"after"

JSON representation of the object resulting from the activity

"before"

JSON representation of the object prior to the activity

"message"

Human readable text about the activity

"objectId"

Object identifier such as "managed/user/DDOE1"

"parentActionId"

UUID of the action leading to the activity

"requester"

Principal requesting the operation

"rev"

Object revision number

"rootActionId"

UUID of the root cause for the activity. This matches a corresponding "rootActionId" in a reconciliation message.

"status"

Result of the operation, such as "SUCCESS"

"timestamp"

Time when OpenIDM logged the message, in UTC format, for example "2012-11-18T08:48:00.160Z"

Reconciliation Log Fields

Reconciliation messages are split into the following fields.

"_id"

UUID for the message object, such as "0419d364-1b3d-4e4f-b769-555c3ca098b0"

"action"

Synchronization action, such as "CREATE". See the section on Actions for a list.

"ambiguousTargetObjectIds"

When the situation is AMBIGUOUS or UNQUALIFIED and OpenIDM cannot distinguish between more than one target object, OpenIDM logs the identifiers of the objects in this field in comma-separated format. This makes it possible to figure out what was ambiguous afterwards.

"entryType"

Kind of reconciliation log entry, such as "start", or "summary".

"message"

Human readable text about the reconciliation action

"reconciling"

What OpenIDM is reconciling, "source" for the first phase, "target" for the second phase

"reconId"

UUID for the reconciliation operation, which is the same for all entries pertaining to the reconciliation run.

"rootActionId"

UUID of the root cause for the activity. This matches a corresponding "rootActionId" in an activity message.

"situation"

The situation encountered. See the section describing synchronization situations for a list.

"sourceObjectId"

UUID for the source object.

"status"

Result of the operation, such as "SUCCESS"

"targetObjectId"

UUID for the target object

"timestamp"

Time when OpenIDM logged the message, in UTC format, for example "2012-11-18T08:48:00.160Z"

16.3. Audit Configuration

OpenIDM exposes the audit logging configuration under http://localhost:8080/openidm/config/audit for the REST API, and in the file conf/audit.json where you installed OpenIDM. The default conf/audit.json file contains the following object.

{
    "eventTypes" : {
        "activity" : {
            "filter" : {
                "actions" : [
                    "create",
                    "update",
                    "delete",
                    "patch",
                    "action"
                ]
            },
            "watchedFields" : [ ],
            "passwordFields" : [ "password" ]
        },
        "recon" : { }
    },
    "logTo" : [
        {
            "logType" : "csv",
            "location" : "audit",
            "recordDelimiter" : ";"
        },
        {
            "logType" : "repository",
            "useForQueries" : true
        }
    ],
    "exceptionFormatter" : {
        "type" : "text/javascript",
        "file" : "bin/defaults/script/audit/stacktraceFormatter.js"
    }
}

16.3.1. Event Types

The eventTypes configuration specifies what events OpenIDM writes to audit logs. OpenIDM supports two eventTypes: activity for the activity log, and recon for the reconciliation log. The filter for actions under activity logging shows the actions on managed or system objects for which OpenIDM writes to the activity log.

The filter actions list enables you to configure the conditions that result in actions being written to the activity log.

read

When an object is read by using its identifier.

create

When an object is created.

update

When an object is updated.

delete

When an object is deleted.

patch

When an object is partially modified.

query

When a query is performed on an object.

action

When an action is performed on an object.

You can optionally add a filter triggers list that specifies the actions that are logged for a particular trigger. For example, the following addition to the audit.json file specifies that only create and update actions are logged for an activity that was triggered by a recon.

   ...
            "filter" : {
                "actions" : [
                    "create",
                    "update",
                    "delete",
                    "patch",
                    "action"
                ],
                "triggers" : {
                    "recon" : [
                        "create",
                        "update"
                    ]
                }
            },
            "watchedFields" : [ ],   
   ...
   

If a trigger is provided, but no actions are specified, nothing is logged for that trigger. If a trigger is omitted, all actions are logged for that trigger. In the current OpenIDM release, only the recon trigger is implemented. For a list of reconciliation actions that can be logged, see Synchronization Actions.

The watchedFields parameter enables you to specify a list of fields that should be "watched" for changes. When the value of one of the fields in this list changes, the change is logged in the audit log, under the column "changedFields". Fields are listed in comma-separated format, for example:

"watchedFields" : [ "email", "address" ]

The passwordFields parameter enables you to specify a list of fields that are considered passwords. This parameter functions much like the watchedFields parameter in that changes to these field values are logged in the audit log, under the column "changedFields". In addition, when a password field is changed, the boolean "passwordChanged" flag is set to true in the audit log. Fields are listed in comma-separated format, for example:

"passwordFields" : [ "password", "username" ]

16.3.2. Log To List

The logTo list enables you to specify the format of the log, where it is written, and various parameters for each log type.

logType

The format of the audit log. The log type can be one of the following:

  • csv - write to a comma-separated variable format file.

    The "location" property indicates the name of the directory in which the file should be written, relative to the directory in which you installed OpenIDM.

    Audit file names are fixed, access.csv, activity.csv, and recon.csv.

    The "recordDelimiter" property enables you to specify the separator between each record.

  • repository - write to the OpenIDM database repository.

    OpenIDM stores entries under the /openidm/repo/audit/ context. Such entries appear as audit/access/_id, audit/activity/_id, and audit/recon/_id, where the _id is the UUID of the entry, such as 0419d364-1b3d-4e4f-b769-555c3ca098b0.

    In the OrientDB repository, OpenIDM stores log records in the audit_access, audit_activity, and audit_recon tables.

    In a JDBC repository, OpenIDM stores records in the auditaccess, auditactivity, and auditrecon tables.

    The "useForQueries" boolean property indicates whether the repository logger should be used to service reads and query requests. The value is true by default. If "useForQueries" is set to false, the CSV file is used to service read and query requests.

16.3.3. Exception Formatter

The exceptionFormatter property specifies the name and type of file that handles the formatting and display of exceptions thrown by the audit logger. Currently, "text/javascript" is the only supported type.

The "file" property provides the path to the script file that performs the formatting. The default exception formatter is "bin/defaults/script/audit/stacktraceFormatter.js".

16.4. Generating Reports

When generating reports from audit logs, you can correlate information from activity and reconciliation logs by matching the "rootActionId" on entries in both logs.

The following MySQL query shows a join of the audit activity and audit reconciliation tables using root action ID values.

mysql> select distinct auditrecon.activity,auditrecon.sourceobjectid,
 auditrecon.targetobjectid,auditactivity.activitydate,auditrecon.status
 from auditactivity inner join auditrecon
 auditactivity.rootactionid=auditrecon.rootactionid
 where auditrecon.activity is not null group by auditrecon.sourceobjectid;
+----------+--------------------------+----------------------+---------------------+---------+
| activity | sourceobjectid           | targetobjectid       | activitydate        | status  |
+----------+--------------------------+----------------------+---------------------+---------+
| CREATE   | system/xmlfile/account/1 | managed/user/juser   | 2012-01-17T07:59:12 | SUCCESS |
| CREATE   | system/xmlfile/account/2 | managed/user/ajensen | 2012-01-17T07:59:12 | SUCCESS |
| CREATE   | system/xmlfile/account/3 | managed/user/bjensen | 2012-01-17T07:59:12 | SUCCESS |
+----------+--------------------------+----------------------+---------------------+---------+
3 rows in set (0.00 sec)