Chapter 10. Configuring Synchronization

The following example triggers a reconciliation operation based on the systemLdapAccounts_managedUser mapping. The mapping is defined in the file conf/sync.json.

$ curl
 --header "X-OpenIDM-Username: openidm-admin"
 --header "X-OpenIDM-Password: openidm-admin"
 --request POST
 "http://localhost:8080/openidm/recon?_action=recon&mapping=systemLdapAccounts_managedUser"
    

By default, an assigned reconciliation run ID is returned immediately when the reconciliation operation is initiated. Clients can make subsequent calls to the reconciliation service, using this reconciliation run ID to query its state and to call operations on it.

For example, the reconciliation run initiated previously would return something similar to the following:

{"_id":"0890ad62-4738-4a3f-8b8e-f3c83bbf212e"}

To have the entire reconciliation run complete before the reconciliation run ID is returned, set the waitForCompletion property to true when the reconciliation is initiated. For example:

$ curl
 --header "X-OpenIDM-Username: openidm-admin"
 --header "X-OpenIDM-Password: openidm-admin"
 --request POST
 "http://localhost:8080/openidm/recon?_action=recon&
 mapping=systemLdapAccounts_managedUser&waitForCompletion=true"
    

You can cancel a reconciliation run by sending a REST call with the cancel action, specifying the reconciliation run ID. For example, the following call cancels the reconciliation run initiated in the previous section:

$curl
 --header "X-OpenIDM-Username: openidm-admin"
 --header "X-OpenIDM-Password: openidm-admin"
 --request POST
 "http://localhost:8080/openidm/recon/0890ad62-4738-4a3f-8b8e-f3c83bbf212e?_action=cancel"
    

The output for a reconciliation cancelation request is similar to the following:

    {"_id":"0890ad62-4738-4a3f-8b8e-f3c83bbf212e",
     "action":"cancel",
     "status":"SUCCESS"}
    

If you specified that the call should wait for completion before the ID is returned, you can obtain the reconciliation run ID from the list of active reconciliations, as described in the following section.

You can display a list of reconciliation processes that have completed, and those that are in progress, by running a RESTful GET on "http://localhost:8080/openidm/recon". The following example displays all reconciliation runs.

$ curl
 --header "X-OpenIDM-Username: openidm-admin"
 --header "X-OpenIDM-Password: openidm-admin"
 --request GET
 "http://localhost:8080/openidm/recon"
    

The output of such a request is similar to the following, with one item for each reconciliation run.

{"reconciliations":[
 {"_id":"d3040cc9-ec2e-41b8-86c4-72393087a626",
 "mapping":"systemLdapAccounts_managedUser",
 "state":"SUCCESS",
 "stage":"COMPLETED_SUCCESS",
 "stageDescription":"reconciliation completed.",
 "progress":{
   "source":{
      "existing":{
         "processed":1001,
         "total":"1001"
      }
   },
   "target":{
      "existing":{
         "processed":1001,
         "total":"1001"
      },
      "created":0
   },
   "links":{
      "existing":{
         "processed":1001,
         "total":"1001"
      },
      "created":0
   }
 },
 "started":"2012-11-18T08:48:00.031Z",
 "ended":"2012-11-18T08:48:00.160Z"},
    

Reconciliation operations are logged in the file /path/to/openidm/audit/recon.csv and in the repository. You can read and query the reconciliation audit logs over the REST interface as outlined in the following examples.

By default all audit/recon query responses are formatted based on the entryType of the entry. Fields that are not required for the specific entry type are stripped away from the response. For example, a summary entry would not need to include a null targetObjectId field, as this field would not make sense in the context of a summary. You can specify that this auto-formatting be disabled and return the full entry for all entry types. To disable entry formatting, include formatted=false as a query parameter in the request.

To return all reconciliation operations logged in the audit log, run a RESTful GET on the audit/recon endpoint. For example:

$ curl
 --header "X-OpenIDM-Username: openidm-admin"
 --header "X-OpenIDM-Password: openidm-admin"
 --request GET
 "http://localhost:8080/openidm/audit/recon"

The following code sample shows an extract of the audit log after the first reconciliation operation in Sample 1.

{
  "entries": [
    {
      "rootActionId": "d0578abf-f38e-4ede-a7dc-5ee9eaa8ce53",
      "messageDetail": null,
      "message": "Reconciliation initiated by openidm-admin",
      "timestamp": "2013-05-08T07:58:33.296Z",
      "reconId": "5cf09dfa-e85c-4d52-ab6c-8ba7c2e3d34f",
      "entryType": "start",
      "_id": "11381e20-3679-469d-a71c-c557c2bd091e",
      "status": "SUCCESS",
      "exception": "",
      "mapping": "systemXmlfileAccounts_managedUser"
    },
    {
      "messageDetail": null,
      "rootActionId": "d0578abf-f38e-4ede-a7dc-5ee9eaa8ce53",
      "situation": "ABSENT",
      "actionId": "86995423-8a43-4fc7-9c3c-9e450e0234cb",
      "targetObjectId": "managed/user/scarter",
      "action": "CREATE",
      "entryType": "",
      "_id": "9f59bb8a-31c6-41af-8f27-02094391ba0c",
      "reconId": "5cf09dfa-e85c-4d52-ab6c-8ba7c2e3d34f",
      "status": "SUCCESS",
      "exception": "",
      "reconciling": "source",
      "ambiguousTargetObjectIds": "",
      "timestamp": "2013-05-08T07:58:33.791Z",
      "message": null,
      "sourceObjectId": "system/xmlfile/account/scarter"
    },
    {
      "messageDetail": null,
      "rootActionId": "d0578abf-f38e-4ede-a7dc-5ee9eaa8ce53",
      "situation": "ABSENT",
      "actionId": "dea9b5c5-7a75-4cab-b8e4-176bea0a94a6",
      "targetObjectId": "managed/user/bjensen",
      "action": "CREATE",
      "entryType": "",
      "_id": "4fd285ef-a409-4875-abd0-5d70965fe172",
      "reconId": "5cf09dfa-e85c-4d52-ab6c-8ba7c2e3d34f",
      "status": "SUCCESS",
      "exception": "",
      "reconciling": "source",
      "ambiguousTargetObjectIds": "",
      "timestamp": "2013-05-08T07:58:33.793Z",
      "message": null,
      "sourceObjectId": "system/xmlfile/account/bjensen"
      },
      {
        "rootActionId": "d0578abf-f38e-4ede-a7dc-5ee9eaa8ce53",
        "messageDetail": {
          "ended": "2013-05-08T07:58:33.813Z",
          "started": "2013-05-08T07:58:33.294Z",
          "situationSummary": {
            "SOURCE_MISSING": 0,
            "FOUND": 0,
            "SOURCE_IGNORED": 0,
            "UNQUALIFIED": 0,
            "UNASSIGNED": 0,
            "TARGET_IGNORED": 0,
            "CONFIRMED": 0,
            "AMBIGUOUS": 0,
            "ABSENT": 2,
            "MISSING": 0
          },
          "progress": {
            "links": {
              "created": 2,
              "existing": {
                "total": "0",
                "processed": 0
              }
            },
            "target": {
              "created": 2,
              "existing": {
                "total": "0",
                "processed": 0
              }
            },
            "source": {
              "existing": {
                "total": "2",
                "processed": 2
              }
            }
          },
          "stageDescription": "reconciling target entries",
          "stage": "ACTIVE_RECONCILING_TARGET",
          "state": "ACTIVE",
          "mapping": "systemXmlfileAccounts_managedUser"
        },
        "message": "SOURCE_IGNORED: 0 MISSING: 0 FOUND: 0 AMBIGUOUS: 0 UNQUALIFIED: 0
           CONFIRMED: 0 SOURCE_MISSING: 0 ABSENT: 2 TARGET_IGNORED: 0 UNASSIGNED: 0 ",
        "timestamp": "2013-05-08T07:58:33.813Z",
        "reconId": "5cf09dfa-e85c-4d52-ab6c-8ba7c2e3d34f",
        "entryType": "summary",
        "_id": "a8a81f9f-fa8f-49f4-a0d6-c88b5fc4be2a",
        "status": "SUCCESS",
        "exception": "",
        "mapping": "systemXmlfileAccounts_managedUser"
      }
    ]
}

Most of the fields in this audit log are self-explanatory. Each distinct reconciliation operation is identified by its reconId. Each entry in the log is identified by a unique _id. The first log entry indicates the status for the complete reconciliation operation. Successive entries indicate the status for each record affected by the reconciliation.

To obtain information on a specific audit log entry, include its entry _id in the endpoint. For example:

$ curl
 --header "X-OpenIDM-Username: openidm-admin"
 --header "X-OpenIDM-Password: openidm-admin"
 --request GET
 "http://localhost:8080/openidm/audit/recon/9f59bb8a-31c6-41af-8f27-02094391ba0c"

The following sample output shows the results of a read operation on a specific reconciliation audit entry.

{
 "messageDetail": null,
 "rootActionId": "d0578abf-f38e-4ede-a7dc-5ee9eaa8ce53",
 "situation": "ABSENT",
 "actionId": "86995423-8a43-4fc7-9c3c-9e450e0234cb",
 "targetObjectId": "managed/user/scarter",
 "action": "CREATE",
 "entryType": "",
 "_id": "9f59bb8a-31c6-41af-8f27-02094391ba0c",
 "reconId": "5cf09dfa-e85c-4d52-ab6c-8ba7c2e3d34f",
 "status": "SUCCESS",
 "exception": "",
 "reconciling": "source",
 "ambiguousTargetObjectIds": "",
 "timestamp": "2013-05-08T07:58:33.791Z",
 "message": null,
 "sourceObjectId": "system/xmlfile/account/scarter"
}

To query the audit log for a particular reconciliation operation, use the audit-by-recon-id keyword, specifying the reconciliation ID, as follows:

$ curl
 --header "X-OpenIDM-Username: openidm-admin"
 --header "X-OpenIDM-Password: openidm-admin"
 --request GET
 "http://localhost:8080/openidm/audit/recon/?_queryId=audit-by-recon-id&reconId=<reconID>"

Output similar to the following is returned, for the specified reconciliation operation:

{
  "start": {
    "rootActionId": "3c098d6f-a5e5-483c-a8ce-82911a10b0a9",
    "messageDetail": null,
    "message": "Reconciliation initiated by openidm-admin",
    "timestamp": "2013-05-14T08:20:41.343Z",
    "reconId": "1ef8e7b6-33dc-4f92-810a-b51913508a68",
    "entryType": "start",
    "_id": "ce58ef39-425d-4711-94b9-a9c77df23001",
    "status": "SUCCESS",
    "exception": "",
    "mapping": "systemXmlfileAccounts_managedUser"
  },
  "result": [
    {
      "messageDetail": null,
      "rootActionId": "3c098d6f-a5e5-483c-a8ce-82911a10b0a9",
      "situation": "ABSENT",
      "actionId": "1a391fe8-201b-4f59-ad05-92ee804488a8",
      "targetObjectId": "managed/user/scarter",
      "action": "CREATE",
      "entryType": "",
      "_id": "6dc0a18a-826d-487d-a29f-5cd8d2f55465",
      "reconId": "1ef8e7b6-33dc-4f92-810a-b51913508a68",
      "status": "SUCCESS",
      "exception": "",
      "reconciling": "source",
      "ambiguousTargetObjectIds": "",
      "timestamp": "2013-05-14T08:20:41.763Z",
      "message": null,
      "sourceObjectId": "system/xmlfile/account/scarter"
    },
    {
      "messageDetail": null,
      "rootActionId": "3c098d6f-a5e5-483c-a8ce-82911a10b0a9",
      "situation": "ABSENT",
      "actionId": "0aaba292-1dd3-4e98-a0e2-04bec9ae5209",
      "targetObjectId": "managed/user/bjensen",
      "action": "CREATE",
      "entryType": "",
      "_id": "1cda457e-54e2-451b-8a40-ef93dec7e60c",
      "reconId": "1ef8e7b6-33dc-4f92-810a-b51913508a68",
      "status": "SUCCESS",
      "exception": "",
      "reconciling": "source",
      "ambiguousTargetObjectIds": "",
      "timestamp": "2013-05-14T08:20:41.760Z",
      "message": null,
      "sourceObjectId": "system/xmlfile/account/bjensen"
    }
  ],
  "summary": {
    "rootActionId": "3c098d6f-a5e5-483c-a8ce-82911a10b0a9",
    "messageDetail": {
      "ended": "2013-05-14T08:20:41.783Z",
      "started": "2013-05-14T08:20:41.342Z",
      "situationSummary": {
        "SOURCE_MISSING": 0,
        "FOUND": 0,
        "SOURCE_IGNORED": 0,
        "UNQUALIFIED": 0,
        "UNASSIGNED": 0,
        "TARGET_IGNORED": 0,
        "CONFIRMED": 0,
        "AMBIGUOUS": 0,
        "ABSENT": 2,
        "MISSING": 0
      },
      "progress": {
        "links": {
          "created": 2,
          "existing": {
            "total": "0",
            "processed": 0
          }
        },
        "target": {
          "created": 2,
          "existing": {
            "total": "0",
            "processed": 0
          }
        },
        "source": {
          "existing": {
            "total": "2",
            "processed": 2
          }
        }
      },
      "stageDescription": "reconciling target entries",
      "stage": "ACTIVE_RECONCILING_TARGET",
      "state": "ACTIVE",
      "mapping": "systemXmlfileAccounts_managedUser"
    },
    "message": "SOURCE_IGNORED: 0 MISSING: 0 FOUND: 0 AMBIGUOUS: 0 UNQUALIFIED: 0
       CONFIRMED: 0 SOURCE_MISSING: 0 ABSENT: 2 TARGET_IGNORED: 0 UNASSIGNED: 0 ",
    "timestamp": "2013-05-14T08:20:41.783Z",
    "reconId": "1ef8e7b6-33dc-4f92-810a-b51913508a68",
    "entryType": "summary",
    "_id": "94c5893c-4ea8-4464-aff2-5533d6369722",
    "status": "SUCCESS",
    "exception": "",
    "mapping": "systemXmlfileAccounts_managedUser"
  }
}

Note that the response structure of a read request on audit/recon differs from the response structure of a query by recon-id. The read response contains a flat list of entries. The query response separates the summary and start event type entries. Other entries are listed under the result entry.

To query the audit log for a specific reconciliation situation, use the audit-by-recon-id-situation keyword, specifying the reconciliation ID and the situation that you want to query. For example, the following query returns all ABSENT records found during the specified reconciliation operation:

$ curl
 --header "X-OpenIDM-Username: openidm-admin"
 --header "X-OpenIDM-Password: openidm-admin"
 --request GET
 "http://localhost:8080/openidm/audit/recon?_queryId=audit-by-recon-id-situation&situation=
 ABSENT&reconId=fd2a59df-1fcd-444d-97ca-2c8a7ec6dc6c"

Output similar to the following is returned, with one entry for each record that matches the situation queried:

{
  "result": [
    {
      "messageDetail": null,
      "rootActionId": "3c098d6f-a5e5-483c-a8ce-82911a10b0a9",
      "situation": "ABSENT",
      "actionId": "1a391fe8-201b-4f59-ad05-92ee804488a8",
      "targetObjectId": "managed/user/scarter",
      "action": "CREATE",
      "entryType": "",
      "_id": "6dc0a18a-826d-487d-a29f-5cd8d2f55465",
      "reconId": "1ef8e7b6-33dc-4f92-810a-b51913508a68",
      "status": "SUCCESS",
      "exception": "",
      "reconciling": "source",
      "ambiguousTargetObjectIds": "",
      "timestamp": "2013-05-14T08:20:41.763Z",
      "message": null,
      "sourceObjectId": "system/xmlfile/account/scarter"
    },
    {
      "messageDetail": null,
      "rootActionId": "3c098d6f-a5e5-483c-a8ce-82911a10b0a9",
      "situation": "ABSENT",
      "actionId": "0aaba292-1dd3-4e98-a0e2-04bec9ae5209",
      "targetObjectId": "managed/user/bjensen",
      "action": "CREATE",
      "entryType": "",
      "_id": "1cda457e-54e2-451b-8a40-ef93dec7e60c",
      "reconId": "1ef8e7b6-33dc-4f92-810a-b51913508a68",
      "status": "SUCCESS",
      "exception": "",
      "reconciling": "source",
      "ambiguousTargetObjectIds": "",
      "timestamp": "2013-05-14T08:20:41.760Z",
      "message": null,
      "sourceObjectId": "system/xmlfile/account/bjensen"
    }
  ]
}

The activity logs track all operations on internal (managed) and external (system) objects. Entries in the activity log contain identifiers for the reconciliation or synchronization action that triggered the activity, and for the original caller and the relationships between related actions.

You can access the activity logs over REST with the following call:

$ curl
 --header "X-OpenIDM-Username: openidm-admin"
 --header "X-OpenIDM-Password: openidm-admin"
 --request GET
 "http://localhost:8080/openidm/audit/activity"

The following extract of the activity log shows the last entry in the log, which was a password change for user bjensen.

{
  "entries": [
    {
      "rootActionId": "3c098d6f-a5e5-483c-a8ce-82911a10b0a9",
      "action": "create",
      "objectId": "managed/user/bjensen",
      "_rev": "0",
      "_id": "22ef6d20-bd84-4267-9db8-745825a46ad1",
      "timestamp": "2013-05-14T08:20:41.712Z",
      "message": null,
      "activityId": "aa964020-b343-4d23-8e00-745ed4b79f50",
      "parentActionId": "0aaba292-1dd3-4e98-a0e2-04bec9ae5209",
      "requester": "openidm-admin",
      "rev": "0",
      "passwordChanged": true,
      "status": "SUCCESS",
      "before": null,
      "changedFields": [
        "/password"
      ],
      "after": {
        "securityAnswer": {
          "$crypto": {
            "type": "x-simple-encryption",
            "value": {
              "key": "openidm-sym-default",
              "iv": "c6gZKz4sSL2YNTBwwFXIzw==",
              "cipher": "AES/CBC/PKCS5Padding",
              "data": "7FATEBtuLlLfMXGokfLrtJ2rXPEsE1YTIXSoGQCum4w="
            }
          }
        },
       ...

To return activity information for a specific action, include the _id of the action in the endpoint, for example:

$ curl
 --header "X-OpenIDM-Username: openidm-admin"
 --header "X-OpenIDM-Password: openidm-admin"
 --request GET
 "http://localhost:8080/openidm/audit/activity/22ef6d20-bd84-4267-9db8-745825a46ad1"

Results similar to the following are returned:

{
  "rootActionId": "3c098d6f-a5e5-483c-a8ce-82911a10b0a9",
  "action": "create",
  "objectId": "managed/user/bjensen",
  "timestamp": "2013-05-14T08:20:41.712Z",
  "message": null,
  "activityId": "aa964020-b343-4d23-8e00-745ed4b79f50",
  "after": {
    "securityAnswer": {
      "$crypto": {
        "type": "x-simple-encryption",
        "value": {
          "key": "openidm-sym-default",
          "iv": "c6gZKz4sSL2YNTBwwFXIzw==",
          "cipher": "AES/CBC/PKCS5Padding",
          "data": "7FATEBtuLlLfMXGokfLrtJ2rXPEsE1YTIXSoGQCum4w="
        }
      }
    },
    "stateProvince": "",
    "userName": "bjensen@example.com",
    "roles": "openidm-authorized",
    "description": "Created By XML1",
    "accountStatus": "active",
    "password": {
      "$crypto": {
        "type": "x-simple-encryption",
        "value": {
          "key": "openidm-sym-default",
          "iv": "bqhRyLW1lI+KZROcpgyukg==",
          "cipher": "AES/CBC/PKCS5Padding",
          "data": "qO8A76GqNqftVVwOlasyPw=="
        }
      }
    },
    "securityQuestion": "1",
    "givenName": "Barbara",
    "address2": "",
    "lastPasswordAttempt": "Tue May 14 2013 10:20:41 GMT+0200 (SAST)",
    "address1": "",
    "familyName": "Jensen",
    "passwordAttempts": "0",
    "country": "",
    "city": "",
    "_rev": "0",
    "lastPasswordSet": "",
    "postalCode": "",
    "phoneNumber": "1234567",
    "_id": "bjensen",
    "email": "bjensen@example.com"
  },
  "changedFields": [
    "/password"
  ],
  "_id": "22ef6d20-bd84-4267-9db8-745825a46ad1",
  "_rev": "0",
  "parentActionId": "0aaba292-1dd3-4e98-a0e2-04bec9ae5209",
  "requester": "openidm-admin",
  "rev": "0",
  "passwordChanged": true,
  "status": "SUCCESS",
  "before": null
}

Each action in the activity log has a rootActionId and a parentActionId. The rootActionId is the ID that was assigned to the incoming or initiating request. The parentActionId is the ID that is associated with the overall action. So, for example, if an HTTP request invokes a script that changes a user's password, the HTTP request is assigned the rootActionId and the action taken by the script is assigned the parentActionId. You can query the activity log for the details of a specific action by including the parentActionId in the query. For example:

$ curl
 --header "X-OpenIDM-Username: openidm-admin"
 --header "X-OpenIDM-Password: openidm-admin"
 --request GET
 "http://localhost:8080/openidm/audit/activity?_queryId=audit-by-activity-parent-action&
           parentActionId=90ce7473-98c7-4747-bfbb-32bfa3ce5d92"

The following sample output shows the result of a query that requests details of the password change for bjensen.

{
  "result": [
    {
      "rootActionId": "3c098d6f-a5e5-483c-a8ce-82911a10b0a9",
      "action": "create",
      "objectId": "managed/user/bjensen",
      "_rev": "0",
      "_id": "22ef6d20-bd84-4267-9db8-745825a46ad1",
      "timestamp": "2013-05-14T08:20:41.712Z",
      "message": null,
      "activityId": "aa964020-b343-4d23-8e00-745ed4b79f50",
      "parentActionId": "0aaba292-1dd3-4e98-a0e2-04bec9ae5209",
      "requester": "openidm-admin",
      "rev": "0",
      "passwordChanged": true,
      "status": "SUCCESS",
      "before": null,
      "changedFields": [
        "/password"
      ],
      "after": {
        "securityAnswer": {
          "$crypto": {
            "type": "x-simple-encryption",
            "value": {
              "key": "openidm-sym-default",
              "iv": "c6gZKz4sSL2YNTBwwFXIzw==",
              "cipher": "AES/CBC/PKCS5Padding",
              "data": "7FATEBtuLlLfMXGokfLrtJ2rXPEsE1YTIXSoGQCum4w="
            }
          }
        },
        "stateProvince": "",
        "userName": "bjensen@example.com",
        "roles": "openidm-authorized",
        "description": "Created By XML1",
        "accountStatus": "active",
        "password": {
          "$crypto": {
            "type": "x-simple-encryption",
            "value": {
              "key": "openidm-sym-default",
              "iv": "bqhRyLW1lI+KZROcpgyukg==",
              "cipher": "AES/CBC/PKCS5Padding",
              "data": "qO8A76GqNqftVVwOlasyPw=="
            }
          }
        },
        "securityQuestion": "1",
        "givenName": "Barbara",
        "address2": "",
        "lastPasswordAttempt": "Tue May 14 2013 10:20:41 GMT+0200 (SAST)",
        "address1": "",
        "familyName": "Jensen",
        "passwordAttempts": "0",
        "country": "",
        "city": "",
        "_rev": "0",
        "lastPasswordSet": "",
        "postalCode": "",
        "phoneNumber": "1234567",
        "_id": "bjensen",
        "email": "bjensen@example.com"
      }
    }
  ]
}

For more information about the entries in these logs, see the chapter that covers Using Audit Logs.

Connector configuration files map external resource objects to OpenIDM objects, and are described in detail in the chapter on Connecting to External Resources. Connector configuration files are named openidm/conf/provisioner.resource-name.json, where resource-name reflects the connector technology and external resource, such as openicf-xml.

An excerpt from an example connector configuration follows. The example shows the name for the connector and two attributes of an account object type. In the attribute mapping definitions, the attribute name is mapped from the nativeName, the attribute name used on the external resource, to the attribute name used in OpenIDM. Thus the example shows that the sn attribute in LDAP is mapped to lastName in OpenIDM. The homePhone attribute can have multiple values.

{
    "name": "MyLDAP",
    "objectTypes": {
        "account": {
            "lastName": {
                "type": "string",
                "required": true,
                "nativeName": "sn",
                "nativeType": "string"
            },
            "homePhone": {
                "type": "array",
                "items": {
                    "type": "string",
                    "nativeType": "string"
                },
                "nativeName": "homePhone",
                "nativeType": "string"
            }
        }
    }
}

In order for OpenIDM to access external resource objects and attributes, the object and its attributes must match the connector configuration. Note that the connector file only maps external resource objects to OpenIDM objects. To construct attributes and to manipulate their values, you use the synchronization mappings file.

The synchronization mappings file (openidm/conf/sync.json) represents the core configuration for OpenIDM synchronization.

The sync.json file describes a set of mappings. Each mapping specifies how attributes from source objects correspond to attributes on target objects. The source and target indicate the direction for the data flow, so you must define a separate mapping for each data flow. For example, if you want data flows from an LDAP server to the repository and also from the repository to the LDAP server, you must define two separate mappings.

You identify external resource sources and targets as system/name/object-type, where name is the name used in the connector configuration file, and object-type is the object defined in the connector configuration file list of object types. For objects in OpenIDM's internal repository, you use managed/object-type, where object-type is defined in openidm/conf/managed.json. The name for the mapping by convention is set to a string of the form source_target, as shown in the following example.

{
    "mappings": [
        {
            "name": "systemLdapAccounts_managedUser",
            "source": "system/MyLDAP/account",
            "target": "managed/user",
            "properties": [
                {
                    "target": "familyName",
                    "source": "lastName"
                },
                {
                    "target": "homePhone",
                    "source": "homePhone"
                },
                {
                    "target": "phoneExtension",
                    "default": "0047"
                },
                {
                    "target": "mail",
                    "comment": "Set mail if non-empty.",
                    "source": "email",
                    "condition": {
                        "type": "text/javascript",
                        "source": "(object.email != null)"
                    }
                },
                {
                    "target": "displayName",
                    "source": "",
                    "transform": {
                        "type": "text/javascript",
                        "source": "source.lastName +', ' + source.firstName;"
                    }
                }
            ]
        }
    ]
}

In this example, the source is the external resource, MyLDAP, and the target is OpenIDM's repository, specifically the managed user objects. The properties reflect OpenIDM attribute names. For example, the mapping has the attribute lastName defined in the MyLDAP connector configuration file mapped to familyName in the OpenIDM managed user object. Notice that the attribute names come from the connector configuration, rather than the external resource itself.

You can create attributes on the target as part of the mapping. In the preceding example, a phoneExtension attribute with a default value of 0047 is created on the target. The "default" property can also be used to specify the value to assign to the target property if the "source" property and the "transform" script yield a null value. If no value is specified, the default value is null.

You can also set up conditions under which OpenIDM maps attributes as shown for the email attribute in the example. By default, OpenIDM synchronizes all attributes. In the example, the mail attribute is set only if the script for the condition returns true.

OpenIDM also enables you to transform attributes. In the example, the value of the displayName attribute is set using a combination of the lastName and firstName attribute values from the source. For transformations, the source property is optional. However, the source object is only available when you specify the source property. Therefore, in order to use source.lastName and source.firstName to calculate the displayName, the example specifies "source" : "".

To add a flow from the repository to MyLDAP, you would define a mapping with source managed/user and target system/MyLDAP/account, named for example managedUser_systemLdapAccounts.

The following image shows the paths to objects in the OpenIDM namespace.

[D]

OpenIDM stores managed objects in the repository, and exposes them under /openidm/managed. System objects on external resources are exposed under /openidm/system.

During synchronization, your scripts always have access to a source object and a target object. Examples already shown in this section use source.attributeName to retrieve attributes from the source objects. Your scripts can also write to target attributes using target.attributeName syntax.

{
    "onUpdate": {
        "type": "text/javascript",
        "source": "if (source.email != null) {target.mail = source.email;}"
    }
}

See the Scripting Reference appendix for more on scripting.

By default, all mappings participate in automatic synchronization operations. You can prevent a specific mapping from participating in automatic synchronization by setting the "enableSync" property of that mapping to false. In the following example, automatic synchronization is disabled. This means that changes to objects in the internal repository are not automatically propagated to the LDAP directory. To propagate changes to the LDAP directory, reconciliation must be launched manually.

{
    "mappings" : [
        {
            "name" : "managedUser_systemLdapAccounts",
            "source" : "managed/user",
            "target" : "system/ldap/account",
            "enableSync" : false,
             ....
}
   

OpenIDM supports reversible encryption of attribute values for managed objects. Attribute values to encrypt include passwords, authentication questions, credit card numbers, and social security numbers. If passwords are already encrypted on the external resource, they are generally excluded from the synchronization process. For more information, see Managing Passwords.

You configure encryption in the managed object configuration (in the openidm/conf/managed.json file). The following example shows a managed object configuration that encrypts and decrypts securityAnswer, ssn, and password attributes using the default symmetric key, and additional scripts for extra passwords.

{
    "objects": [
        {
            "name": "user",
            "properties": [
                {
                    "name": "securityAnswer",
                    "encryption": {
                        "key": "openidm-sym-default"
                    }
                },
                {
                    "name": "ssn",
                    "encryption": {
                        "key": "openidm-sym-default"
                    }
                },
                {
                    "name": "password",
                    "encryption": {
                        "key": "openidm-sym-default"
                    }
                }
            ],
            "onStore": {
                "type": "text/javascript",
                "file": "script/encryptExtraPassword.js"
            },
            "onRetrieve": {
                "type": "text/javascript",
                "file": "script/decryptExtraPassword.js"
            }
        }
    ]
}

Do not use the default symmetric key, openidm-sym-default, in production. See the chapter on Securing and Hardening OpenIDM for instructions on adding your own symmetric key.

You can protect specific sensitive data stored in the repository by marking the corresponding properties as "private". Private data, whether it is encrypted or not, is not accessible over the REST interface. Properties that are marked as private are removed from an object when that object is retrieved over REST.

To mark a property as private, set its "scope" to "private" in the conf/managed.json file.

The following extract of the managed.json file shows how HTTP access is prevented on the password and securityAnswer properties.

       "properties" : [
           {
               "name" : "securityAnswer",
               "encryption" : {
                   "key" : "openidm-sym-default"
               },
               "scope" : "private"
           },
           {
               "name" : "password",
               "encryption" : {
                   "key" : "openidm-sym-default"
               },
               "scope" : "private"
   

A potential caveat with using private properties is that such properties are removed if an object is updated by using an HTTP PUT request. A PUT request replaces the entire object in the repository. Because properties that are marked as private are ignored in HTTP requests, these properties are effectively removed from the object when the update is done. To work around this limitation, do not use PUT requests if you have configured private properties. Instead, use a PATCH request to update only those properties that need to be changed.

For example, to update the familyName of user joe, replace only the familyName and not the entire user object, as follows:

$ curl
 --header "X-OpenIDM-Username: openidm-admin"
 --header "X-OpenIDM-Password: openidm-admin"
 --header "Content-Type: application/json"
 --request POST
 --data '[{"replace":"familyName","value": "Brown"}]'
 "http://localhost:8080/openidm/managed/user?_action=patch&_queryId=for-userName&uid=joe"
   

OpenIDM enables you to construct and manipulate attributes using scripts that are triggered when an object is created (onCreate), updated (onUpdate), or deleted (onDelete), or when a link is created (onLink), or removed (onUnlink).

The following example derives a DN for an LDAP entry when the entry is created in the internal repository.

{
    "onCreate": {
        "type": "text/javascript",
        "source":
            "target.dn = 'uid=' + source.uid + ',ou=people,dc=example,dc=com'"
    }
}

When two mappings exist to synchronize the same objects bidirectionally, you can use the links property in one mapping to have OpenIDM use the same internally managed link for both mappings. Otherwise, if no links property is specified, OpenIDM maintains a link for each mapping.

The following excerpt shows two mappings, one from MyLDAP accounts to managed users, and another from managed users to MyLDAP accounts. In the second mapping, the link property tells OpenIDM to reuse the links created in the first mapping, rather than create new links.

{
    "mappings": [
        {
            "name": "systemMyLDAPAccounts_managedUser",
            "source": "system/MyLDAP/account",
            "target": "managed/user"
        },
        {
            "name": "managedUser_systemMyLDAPAccounts",
            "source": "managed/user",
            "target": "system/MyLDAP/account",
            "links": "systemMyLDAPAccounts_managedUser"
        }
    ]
}

Based on the values assigned to objects during target reconciliation, OpenIDM assigns situations, listed here with their default actions.

The following sections reiterate in detail how OpenIDM assigns situations during each of the two synchronization phases.

OpenIDM starts reconciliation and LiveSync by reading a list of objects from the resource. For reconciliation, the list includes all objects available through the connector. For LiveSync, the list contains only changed objects. The connector can filter objects out of the list, too. You can filter objects out of the list by using the validSource property.

OpenIDM then iterates over the list, checking each entry against the validSource filter, classifying objects according to their situations as described in Section 10.6.1, “Synchronization Situations”. OpenIDM uses the list of links for the current mapping to classify objects. Finally, OpenIDM executes the action configured for the situation.

The following table shows how OpenIDM assigns the appropriate situation during source reconciliation, depending on whether a valid source exists (Source Qualifies), whether a link with the appropriate type exists in the repository (Link Exists), and how many target objects are found, either based on links or correlation results.

During source reconciliation, OpenIDM cannot detect situations where no source object exists, such as the UNASSIGNED situation. When no source object exists, OpenIDM detects the situation during the second reconciliation phase, target reconciliation. During target reconciliation, OpenIDM iterates over all target objects that do not have a representation on the source, checking each object against the validTarget filter, determining the appropriate situation, and executing the action configured for the situation.

The following table shows how OpenIDM assigns the appropriate situation during target reconciliation, depending on whether a valid target exists (Target Qualifies), whether a link with an appropriate type exists in the repository (Link Exists), whether a source object exists (Source Exists), and whether the source object qualifies (Source Qualifies). Not all situations assigned during source reconciliation are assigned during target reconciliation.

Certain situations occur only during automatic synchronization (when OpenIDM pushes changes made in the repository out to external systems) and LiveSync (when OpenIDM polls external system change logs for changes and updates the repository).

The following table shows the situations that pertain only to automatic sync and LiveSync, when records are deleted from the source or target resource.

In addition to the static synchronization actions described in the previous section, you can provide a script that is run in specific synchronization situations. The following extract of a sample sync.json file specifies that when a synchronization operation assesses an entry as ABSENT, the workflow named managedUserApproval is invoked. The parameters for the workflow are passed in as properties of the action parameter.

{
    "situation" : "ABSENT",
    "action" : {
        "workflowName" : "managedUserApproval",
        "type" : "text/javascript",
        "file" : "bin/defaults/script/workflow/workflow.js"
    }
}
      

The variables available to these scripts are described in Variables Available in Scripts in the Scripting Appendix.

To optimize a reconciliation operation, the reconciliation process does not attempt to correlate source objects to target objects if the set of target objects is empty when the correlation is started. This considerably speeds up the process the first time the reconciliation is run. You can change this behavior for a specific mapping by adding the correlateEmptyTargetSet property to the mapping definition and setting it to true. For example:

{
    "mappings": [
        {
            "name"                     : "systemMyLDAPAccounts_managedUser",
            "source"                   : "system/MyLDAP/account",
            "target"                   : "managed/user",
            "correlateEmptyTargetSet"  : true
        },
    ]
}

Be aware that this setting will have a performance impact on the reconciliation process.

All links are queried at the start of a correlation and the results of that query are used. You can disable the prefetching of links, so that the correlation process looks up each link in the database as it processes each source or target object. You can disable the prefetching of links by adding the prefetchLinks property to the mapping, and setting it to false, for example:

{
    "mappings": [
        {
            "name": "systemMyLDAPAccounts_managedUser",
            "source": "system/MyLDAP/account",
            "target": "managed/user"
            "prefetchLinks" : false
        }
    ]
}

Be aware that this setting will have a performance impact on the reconciliation process.

By default, reconciliation is executed in a multi-threaded manner, that is, numerous threads are dedicated to the same reconciliation run. Multithreading generally improves reconciliation run performance. The default number of threads for a single reconciliation run is ten (plus the main reconciliation thread). Under normal circumstances, you should not need to change this number, however the default might not be appropriate in the following situations:

To change the number of threads, set the taskThreads property in the conf/sync.json file, for example:

    "mappings" : [
        {
            "name" : "systemXmlfileAccounts_managedUser",
            "source" : "system/xmlfile/account",
            "target" : "managed/user",
            "taskThreads" : 20
            ...
         }
    ]
}

A value of 0 specifies that reconciliation is run on the main reconciliation thread, that is, in a serial manner.

Queries on managed objects in the repository must be defined in the configuration file for the repository, which is either openidm/conf/repo.orientdb.json, or openidm/conf/repo.jdbc.json.

The following example shows a correlation query defined in openidm/conf/repo.orientdb.json.

"for-userName" : "SELECT * FROM ${unquoted:_resource} WHERE userName = ${uid}"

By default, a ${value} token replacement is assumed to be a quoted string. If the value is not a quoted string, use the unquoted: prefix, as shown above.

The following correlation query example shows the JavaScript to call the query defined for OrientDB. The _queryId property value matches the name of the query specified in openidm/conf/repo.orientdb.json, for-userName. The source.name value replaces ${uid} in the query. OpenIDM replaces ${unquoted:_resource} in the query with the name of the table that holds managed objects.

{
  "correlationQuery": {
    "type": "text/javascript",
    "source":
      "var query = {'_queryId' : 'for-userName', 'uid' : source.name}; query;"
  }
}

The query can return zero or more objects, so the situation OpenIDM assigns to the source object depends on the number of target objects returned.

With a JDBC-based repository, the query defined in openidm/conf/repo.jdbc.json is more complex due to how the tables are indexed. The correlation query you define in openidm/conf/sync.json is the same, however.

Correlation queries on system objects access the connector. The connector then executes the query on the external resource.

Your correlation query JavaScript must return a map holding a generic query with the following elements.

varmap={"query": {"Equals": {"field": "uid", "values": [ source.userName ]}}};
map;

Each scheduled reconciliation and LiveSync task requires a schedule configuration file in opendidm/conf. By convention, files are named openidm/conf/schedule-schedule-name .json, where schedule-name is a logical name for the scheduled synchronization operation, such as reconcile_systemXmlAccounts_managedUser.

Schedule configuration files have the following format:

{
 "enabled"       : true,
 "persisted"     : false,
 "type"          : "cron",
 "startTime"     : "(optional) time",
 "endTime"       : "(optional) time",
 "schedule"      : "cron expression",
 "misfirePolicy" : "optional, string",
 "timeZone"      : "(optional) time zone",
 "invokeService" : "service identifier",
 "invokeContext" : "service specific context info"
}

For an explanation of each of these properties, see Scheduling Tasks and Events.

To schedule a reconciliation or LiveSync task, set the invokeService property to either "sync" (for reconciliation) or "provisioner" for LiveSync.

The value of the invokeContext property depends on the type of scheduled event. For reconciliation, the properties are set as follows:

{
    "invokeService": "sync",
    "invokeContext": {
        "action": "reconcile",
        "mapping": "systemLdapAccount_managedUser"
    }
}

The "mapping" is either referenced by its name in the openidm/conf/sync.json file, or defined inline by using the "mapping" property, as shown in the example in Alternative Mappings.

For LiveSync, the properties are set as follows:

{
    "invokeService": "provisioner",
    "invokeContext": {
        "action": "liveSync",
        "source": "system/OpenDJ/__ACCOUNT__"
    }
}

The "source" property follows OpenIDM's convention for a pointer to an external resource object and takes the form system/resource-name/ object-type.

Mappings for synchronization are usually stored in openidm/conf/sync.json for reconciliation, LiveSync, and for pushing changes made to managed objects to external resources. You can, however, provide alternative mappings for scheduled reconciliation by adding the mapping to the schedule configuration instead of referencing a mapping in sync.json.

{
    "enabled": true,
    "type": "cron",
    "schedule": "0 08 16 * * ?",
    "invokeService": "sync",
    "invokeContext": {
        "action": "reconcile",
        "mapping": {
            "name": "CSV_XML",
            "source": "system/Ldap/account",
            "target": "managed/user",
            "properties": [
                {
                    "source": "firstname",
                    "target": "firstname"
                },
                ...
            ],
            "policies": [...]
        }
    }
}