= Configuration Tool
:Author: Martin Look
:Email: mlook@axway.com
:source-highlighter: prettify
ifdef::env-github[]
:outfilesuffix: .adoc
:!toc-title:
:caution-caption: :fire:
:important-caption: :exclamation:
:note-caption: :paperclip:
:tip-caption: :bulb:
:warning-caption: :warning:
endif::[]

For the configuration of the `.fed` and `.env` files the Maven plugin uses Python scripts.
The scripts can also be used as a standalone tool.

== Build FED

The FED builder takes the `.pol` and `.env` archive and JSON files to configure the environmentalized fields and certificates of the project.
The script is invoked with the `jython` interpreter provided with the package and deployment tools of the API Gateway.

....
$ buildfed.sh --help
Usage: buildfed.py OPTIONS

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -v, --verbose         Enable verbose messages [optional]
  -e FILEPATH, --env=FILEPATH
                        Path of environment archive (.env)
  -p FILEPATH, --pol=FILEPATH
                        Path of policy archive (.pol)
  -c FILEPATH, --config=FILEPATH
                        Path of JSON configuration file
  --prop=FILEPATH       Path of JSON property file [multiple]
  --cert=FILEPATH       Path of JSON certificate configuration file [optional]
  --cert-expiration=DAYS
                        Check if certificates expire within the next days
                        [optional]
  --cert-config-update  Enable writing of info section for 'update'
                        certificates within the configuration file [optional]
  --output-fed=FILEPATH
                        Path of output deployment archive file (.fed)
                        [optional]
  --output-env=FILEPATH
                        Path of output environment archive file (.env)
                        [optional]
  -D NAME:VALUE, --define=NAME:VALUE
                        Define a command line property [multiple]
  -F NAME:FILE, --fromFile=NAME:FILE
                        Define a command line property from file content [multiple]
  --passphrase-in=PASSPHRASE
                        Passphrase of input archive files [optional]
  --passphrase-out=PASSPHRASE
                        Passphrase for output archive files [optional]
  -s, --simulate        Enable simulation mode [optional]
  -b DIRECTORY, --base-dir=DIRECTORY
                        Base directory for certificate files [multiple]
  --secrets-file=FILEPATH
                        Path of JSON file containing confidential properties
                        [optional]
  --secrets-key=FILEPATH
                        Path to key file to decrypt confidential properties [optional]
....

If environmentalized fields or certificates are not configured, the build fails.
Missing fields or certificates are automatically added to the configuration file.

[cols="2,5a", options="header"]
|===
|Option
|Description

|-e, --env
|The `.env` file as generated by the `projpack` tool.
The option is mandatory.

|-p, --pol
|The `.pol` file as generated by the `projpack` tool.
The option is mandatory.

|-c, --config
|The JSON configuration file for environmentalized fields.
The option is mandatory.

|--prop
|Path to a JSON file containing the property configuration.
Multiple files are supported by specifying the parameter multiple times.

If a property occurs in multiple files the value from the last file is used.

Specify a single dash '-' as filename to read properties from `stdin`.
The input must be a flat key/value JSON document (e.g., `{"p1": "v1", "p2": "v2"}`).

|--cert
|An optional JSON file containing the certificate configuration.

|--cert-expiration
|Optional number of days for certificate expiration check.

If at least one certificate expires with the next given days an error will be raised.

|--cert-config-update
|Enable writing of info section for 'update' certificates within the configuration file.

Since version v0.7.0 the info section of "update" certificates are not written any more.
Use this to enable the previous behavior.

|--output-fed
|The path of the configured `.fed` file.
If missing, no `.fed` file is generated.

|--output-env
|The path of the configured `.env` file.
If missing, no `.env` file is generated. 

|--define
|Define a command line property `name:value`.
Command line properties can be used instead of custom defined properties.
Multiple properties are supported by specifying the parameter multiple times.

|--fromFile
|Define a command line property from file content `name:file`.
The file has to contain an UTF-8 encoded text.
The content of the file will be placed into the defined property.
Multiple files are supported by specifying the parameter multiple times.

|--passphrase-in
|Passphrase for source archives.
This passphrase is used for reading the `.pol` and `.env` files.

For all source archives the same passphrase is used.
Having different passphrases for `.pol` and `.env` file is not supported.

|--passphrase-out
|Passphrase for output archives.
This passphrase is used on writing the configured archives.
If not set the output archives have the same passphrase as the source archives.

|-s, --simulation
|Enable simulation mode.

In simulation mode, no output files (`.fed` or `.env`) will be written.

The simulation mode will ignore non existing certificate files and will not apply the values of environmentalized fields.

[NOTE]
====
The simulation mode can be used to prepare packages for CI/CD pipelines.

In this case configuration parameters and certificates may be stored in a configuration database.
So during the preparation phase the certificates may not be available and the configuration files may contain placeholders.
To avoid errors due to incompatible types (e.g. placeholder string used for an integer field) the values will not be applied to the entity store.
====

|-b, --base-dir
|Base directory for certificate files.

Multiple directories can be specified by adding the parameter multiple times.

If specified, the given directories are scanned for certificate files.
If multiple certificate files exists the first one will be used.

|--secrets-file
|Path of JSON file containing confidential properties.
The file has to be created by the `encrypt` tool.

|--secrets-key
|Path to key file to decrypt confidential properties.

The key file has to be the same as on creating the secrets file.

This parameter is required if a secrets file is specified.
|===


== Encrypt Secrets

The `encrypt` tools is used to generate an initial secrets file and to encrypt the values of the properties.
The script is invoked with the `jython` interpreter provided with the package and deployment tools of the API Gateway.

The tool requires a path to the secrets file and a key file to encrypt the values.
If the secrets file doesn't exist a new file will be created.
For existing files the given key is checked against the key used on file creation.

....
$ encrypt.sh -h
Usage: encrypt OPTIONS

Encrypt secrets.

Options:
  --version             show program's version number and exit
  -h, --help            show this help message and exit
  -v, --verbose         Enable verbose messages [optional]
  --secrets-file=FILEPATH
                        Path of JSON file containing confidential properties
  --secrets-key=FILEPATH
                        Path to key file to encrypt confidential properties
  --secrets-key-new=FILEPATH
                        Path to new key file to change key [optional]
....

[cols="2,5a", options="header"]
|===
|Option
|Description

|--secrets-file
|Path of JSON file containing confidential properties.

|--secrets-key
|Path to key file to encrypt confidential properties.

.Key File
****
The key file contains any arbitrary sequence of bytes.
It is treated as a binary file.

If you edit this file with a text editor be aware of the encoding and the end of line sequence.
In this case use ASCII characters in a single line (no line feed at the end) to prevent any incompatibility with other line end formats or editor encodings.
****

|--secrets-key-new
|Path to new key file.

Use this option to change the key.
All values will be re-encrypted with the new key.
|===

To add new properties tag the values with the `encrypt:` prefix.
Values having this prefix will be encrypted on running the tool.

[source,json]
----
{
  "secrets": {
    "__": "3QjoMSfhSelmvMlvcgCdyHf+oTyVnHlyneA3stpN0iQKJ1BUIrY9OA==", <1>
    "my.password": "encrypt:changeme", <2>
    "cert.password": "eL5+ogfSxQue8+NA0/l859g/2nTFwxBUp/7l7z/sMOE=" <3>
  }
}
----
<1> Marker to check the key. Don't delete or change it.
<2> The prefix `encrypt:` indicates that the value `changeme` has to be encrypted.
<3> Values without the prefix are already encrypted and will not be changed.

NOTE: The `encrypt` tool use the same cipher as the entity store.

*Examples*

Encrypt values with the given key:

.Command Line
....
$ encrypt.sh --secrets-file=gateway.crypt.json --secrets-key=secrets.key
....

Change the key of a secrets file:

.Command Line
....
$ encrypt.sh --secrets-file=gateway.crypt.json --secrets-key=secrets.key --secrets-key-new=new_secrets.key
....

== Configuration Files

For the configuration of the environment specific deployment archive, various configuration files are used.
All configuration files are JSON documents.
The schema of the JSON is not validated, so any arbitrary property can be added.
Properties not known by the plugin will be ignored.

If a configuration file is updated by the plugin, all properties, including custom properties, are written.

TIP: When a configuration file is written by the plugin, all JSON properties are sorted.
This makes diff & merge easy.

Following kind of configuration files are supported:

.Configuration Files
[cols="2,<4a,^1,^1", options="header"]
|===
|File|Description|Mandatory|Updated footnote:[Files will be updated by plugin or config tool automatically]

|Environmentalized fields
|Configuration file for environmentalized fields.
|yes
|yes

|Certificates
|Configuration file for certificates stored in the project.
|no
|yes

|Properties
|Configuration file for properties references by the environmentalized fields and certificate configuration.
|no
|no

|Secrets
|Configuration file confidential properties.
|no
|no
|===

=== Environmentalized Fields

For configuring environmentalized fields a JSON file having an `entities` property is used.
For each configured environmentalized entity a property exists.
The name of the property is the _short hand key_ of the entity.

The value of the field can be specified directly by a `value` property.
It can also be specified indirectly by a property or environment variables (controlled via the `source` property).

Properties are name/value pairs.
The value of the property is defined ... _(in order of precedence)_

  . ... as predefined system property.
  . ... as a command line property via the command line of the configuration tool.
  . ... within a separate property configuration file.
  . ... within the environmentalized fields file (under `properties`).

The plugin and the configuration tool provides a set of predefined system properties.

.Predefined System Properties
|===
|System Property|Description|Provided by

|_system.artifact.group
|Group of the project artifact.
|Plugin

|_system.artifact.name
|Name of the project artifact.
|Plugin

|_system.artifact.ver
|Version of the project artifact.
|Plugin

|_system.artifact.id
|Complete ID of the project artifact.
|Plugin

|_system.artifact.info
|JSON describing the artifact and its dependencies (content of the `gateway.info.json` file).
If the `gateway.info.json` file is not available the property contains an empty JSON document `{}`.
|Plugin

|_system.build.datetime
|Build date and time in ISO format (YYYY-MM-DD'T'HH:MM:SS.S)
|Config Tool
|===


.src/main/axwgw/gateway.config.json
[source,json]
----
{
    "entities": { <1>
        "/[CircuitContainer]name=Hello World/[FilterCircuit]name=Hello World Message/[SetAttributeFilter]name=Set name": { <2>
            "description": "Name for the 'Hello World' message.", 
            "fields": {
                "attributeValue#0": { <3>
                    "source": "property", <4>
                    "type": "string", <5> 
                    "used": true, <6>
                    "value": "foobar" <7>
                }
            }
        },
        "/[CircuitContainer]name=Hello World/[FilterCircuit]name=Hello World Message/[SetAttributeFilter]name=Build time": {
            "description": "Build time in ISO format.", 
            "fields": {
                "attributeValue#0": {
                    "source": "property",
                    "type": "string",
                    "used": true,
                    "value": "_system.build.datetime" <8>
                }
            }
        },
        "/[CircuitContainer]name=Hello World/[FilterCircuit]name=Hello World Message/[SetAttributeFilter]name=Network zone": {
            "description": "Network zone the API Gateway is located.", 
            "fields": {
                "attributeValue#0": {
                    "source": "value"
                    "type": "string",
                    "used": true,
                    "value": "internal" <9>
                }
            }
        },
        "/[CircuitContainer]name=Hello World/[FilterCircuit]name=Hello World Message/[SetAttributeFilter]name=User": {
            "description": "Name of the user building the .fed", 
            "fields": {
                "attributeValue#0": {
                    "source": "env" <10>
                    "type": "string",
                    "used": true,
                    "value": "USERNAME" <11>
                }
            }
        },
        "/[CircuitContainer]name=Hello World/[FilterCircuit]name=Hello World Message/[SetAttributeFilter]name=Secret": {
            "description": "Some secret information", 
            "fields": {
                "attributeValue#0": {
                    "source": "secrets" <12>
                    "type": "string",
                    "used": true,
                    "value": "secret" <13>
                }
            }
        }
    }
    "properties": { <14>
        "foobar": "myvalue"
    }
}
----
<1> Environmentalized entities are configured under an `entities` attribute.
<2> Short hand key of the environmentalized entity.
<3> Name and index of the environmentalized field.
<4> Declares a property (named `foobar`, see "value") as the source of the field value.
<5> Type of the field (just for documentation, don't change it).
<6> Indicates if the configured field is used.
If _false_ the field is no longer environmentalized or the entity is renamed or removed.
The property is automatically maintained by the plugin. 
<7> Name of the property containing the value.
<8> Use value of the predefined system property `_system.build.datetime`.
<9> Literal value for the field.
<10> Specifies an environment variable as the source for the field value.
<11> Field value is retrieved from the `USERNAME` environment variable.
<12> Specifies an property from the secrets file as the source for the field value.
<13> Field value is retrieved from the `secret` property in the secrets file.
<14> An optional local definition of properties.
If the same property is defined in a separate property file (see below), the separate property has precedence.


NOTE: If `value` is _null_ the field is treated as undefined and the build will fail.

=== Certificates
The configuration for certificates is stored in a separate JSON file.
It specifies the alias of the certificates within the project and the source of the replacement certificate.

When the file is specified it will be automatically generated/updated according to the certificates stored in your project.

.src/main/axwgw/gateway.certs.json
[source,json]
----
{
    "certificates": { <1>
        "extern-crt": { <2>
            "origin": { <3>
                "info": {
                    "not_after": "2020-05-21T07:04:00+02:00", <4>
                    "subject": "CN=extern, O=Axway, L=Berlin, ST=Berlin, C=DE" <5>
                }
            },
            "update": { <6>
                "file": "cert/extern.crt", <7>
                "type": "crt" <8>
            }
        }, 
        "server-p12": {
            "origin": {
                "info": {
                    "not_after": "2020-05-21T07:02:00+02:00", 
                    "subject": "CN=server, O=Axway, L=Berlin, ST=Berlin, C=DE"
                }
            },
            "update": {
                "file": "cert/server.p12", 
                "password": "server", <9>
                "source": "password", <10>
                "type": "p12"
            }
        }, 
        "test": {
            "origin": {
                "info": {
                    "not_after": "2021-09-30T16:01:15+02:00", 
                    "subject": "CN=DST Root CA X3, O=Digital Signature Trust Co."
                }
            },
            "update": null <11>
        },
        "test2": { <12>
            "update": {
                "file": "cert/server.p12", 
                "password": "server.password", <13>
                "source": "property", <14>
                "type": "p12"
            }
        },
        "test3": {
            "update": {
                "file": "cert/server.p12", 
                "password": "TEST3_PASSWORD", <15>
                "source": "env", <16>
                "type": "p12"
            }
        },
        "test4": {
            "origin": {
                "info": {
                    "not_after": "2021-09-30T16:01:15+02:00", 
                    "subject": "CN=DST Root CA X3, O=Digital Signature Trust Co."
                }
            },
            "update": {
                "type": "empty" <17>
            }
        }
    }
}
----
<1> Certificates are configured under a `certificates` attribute.
<2> Unique alias for storing the certificate in the certificate store.
<3> Information of the origin certificate.
A missing `origin` attribute indicates the origin certificate store doesn't has a certificate with this alias.
<4> Expiration date of the origin certificate.
<5> Subject of the origin certificate.
<6> Defines the certificate to update the certificate with the same alias within the certificate store.
A missing `update` attribute indicates a new/unconfigured certificate.
<7> Path to the new certificate file.
<8> Type of the certificate.
`crt` for certificates and `p12` for certificates with key.
<9> Literal password for the `.p12` file.
<10> Declares a property (`server.password`, see "password") as the source for the password.
<11> _null_ value indicates that the certificate will not be updated.
<12> Certificate without a `origin` attribute.
This certificate will be added to the certificate store.
<13> Password for the `.p12` file is retrieved from the property configuration file.
<14> Declares the property "password" as the source of the password for the `.p12` file.
<15> The password is retrieved from the `TEST3_PASSWORD` environment variable.
<16> Specifies an environment variable as the source of the password.
<17> Type `empty` indicates that a certificate will be updated with an _empty_ certificate and therefore will be removed.

=== Properties

Properties used in the configuration files for environmentalized fields and certificates can be resolved by a separate configuration file.
The file is a JSON document containing a `properties` property.
For each property a name/value pair exists.

.gateway.props.json
[source,json]
----
{
    "properties": { <1>
        "name1": "value1", <2>
        "name2": "value2"
    }
}
----
<1> Properties are configured under a `properties` attribute.
<2> For each configured property a name/value pair has to exist.
The property is identified by its _name_.

[NOTE]
====
Properties may contain sensitive information (e.g. credentials).
The shift of properties to a separate file enables them to be exclude from the source code management system.

In productive environments secretes may be stored in a secured configuration database.
For the build process the property file may be temporarily generated from the configuration database.
====

=== Secrets

A secrets file is used to store confidential configurations (e.g. passwords).
The values of the properties are encrypted and can be access with a key only.
All values are encrypted with the same key.

.gateway.crypt.json
[source,json]
----
{
  "secrets": { <1>
    "__": "3QjoMSfhSelmvMlvcgCdyHf+oTyVnHlyneA3stpN0iQKJ1BUIrY9OA==", <2>
    "my.password": "encrypt:changeme", <3>
    "cert.password": "eL5+ogfSxQue8+NA0/l859g/2nTFwxBUp/7l7z/sMOE=" <4>
  }
}
----
<1> The `secrets` property is required.
<2> Marker to check the key. Don't delete or change it.
<3> The prefix `encrypt:` indicates that the value `changeme` has to be encrypted by the `encrypt` tool.
<4> Values without the prefix are already encrypted.