openapi: 3.0.0 info: version: v1 title: Forwarders API description: >- Use the Forwarders API to forward data (metrics, events) to external services. servers: - description: NGINX Controller API url: 'https://{{CONTROLLER_FQDN}}/api/v1' paths: /analytics/forwarders: post: tags: - Forwarders summary: Create a Forwarder description: Creates a new Forwarder resource. operationId: createForwarder requestBody: description: >- Contains the desired settings for the Forwarder resource. content: application/json: schema: $ref: '#/components/schemas/Forwarder' responses: '202': description: >- The request to create a Forwarder resource succeeded. The resource will be created when the configuration is complete. content: application/json: schema: $ref: '#/components/schemas/Forwarder' '400': $ref: '#/components/responses/BadRequest' '401': $ref: '#/components/responses/Unauthorized' '403': $ref: '#/components/responses/Forbidden' '409': $ref: '#/components/responses/Conflict' '500': $ref: '#/components/responses/Internal' get: tags: - Forwarders summary: Get a list of Forwarders. description: Returns a list of all Forwarder resources. operationId: listForwarders responses: '200': description: >- Successfully retrieved a list of all Forwarder resources. content: application/json: schema: $ref: '#/components/schemas/ForwarderListResponse' '401': $ref: '#/components/responses/Unauthorized' '403': $ref: '#/components/responses/Forbidden' '500': $ref: '#/components/responses/Internal' '/analytics/forwarders/{forwarderName}': get: tags: - Forwarders summary: Get a Forwarder operationId: getForwarder description: Gets information about an existing Forwarder resource. parameters: - $ref: '#/components/parameters/ForwarderName' responses: '200': description: >- Successfully retrieved the requested Forwarder resource. content: application/json: schema: $ref: '#/components/schemas/Forwarder' '401': $ref: '#/components/responses/Unauthorized' '403': $ref: '#/components/responses/Forbidden' '404': $ref: '#/components/responses/NotFound' '500': $ref: '#/components/responses/Internal' put: tags: - Forwarders summary: Create or update a Forwarder description: Creates new or updates an existing Forwarder resource. operationId: updateForwarder requestBody: description: >- Contains the desired settings for the Forwarder resource. content: application/json: schema: $ref: '#/components/schemas/Forwarder' parameters: - $ref: '#/components/parameters/ForwarderName' responses: '202': description: >- Successfully initialized the update process of a Forwarder resource. content: application/json: schema: $ref: '#/components/schemas/Forwarder' '400': $ref: '#/components/responses/BadRequest' '401': $ref: '#/components/responses/Unauthorized' '403': $ref: '#/components/responses/Forbidden' '500': $ref: '#/components/responses/Internal' delete: tags: - Forwarders summary: Delete a Forwarder operationId: deleteForwarder description: Deletes a Forwarder resource. parameters: - $ref: '#/components/parameters/ForwarderName' responses: '202': description: >- The request to delete a Forwarder resource succeeded. The resource is marked for deletion. content: application/json: schema: $ref: '#/components/schemas/Forwarder' '401': $ref: '#/components/responses/Unauthorized' '403': $ref: '#/components/responses/Forbidden' '404': $ref: '#/components/responses/NotFound' '500': $ref: '#/components/responses/Internal' components: responses: BadRequest: description: > Bad input parameter, or possibly a bad URI. Check the input for typos and try again. content: application/json: schema: $ref: '#/components/schemas/ErrorModel' Unauthorized: description: | User authentication is invalid or missing. content: application/json: schema: $ref: '#/components/schemas/ErrorModel' Forbidden: description: | The request failed due to insufficient privileges. content: application/json: schema: $ref: '#/components/schemas/ErrorModel' Internal: description: > The request cannot be processed because of an internal server error. content: application/json: schema: $ref: '#/components/schemas/ErrorModel' NotFound: description: | The requested resource was not found or is unavailable. content: application/json: schema: $ref: '#/components/schemas/ErrorModel' Conflict: description: > The request failed due to a conflict with an existing resource. content: application/json: schema: $ref: '#/components/schemas/ErrorModel' parameters: ForwarderName: in: path name: forwarderName schema: type: string required: true description: | The name of the Forwarder resource. schemas: Forwarder: type: object required: - metadata - desiredState properties: metadata: $ref: '#/components/schemas/ResourceMeta' desiredState: $ref: '#/components/schemas/DesiredState' currentStatus: $ref: '#/components/schemas/CurrentStatus' example: metadata: name: splunk displayName: Splunk - Metrics description: Metrics forwarder going to Splunk HEC kind: forwarder uid: d290f1ee-6c54-4b01-90e6-d701748f0851 links: rel: /api/v1/analytics/forwarders/splunk createTime: '2019-07-29T09:12:33.001Z' updateTime: '2019-07-29T10:12:33.001Z' desiredState: collectorType: SPLUNK integrationRef: ref: /platform/integrations/new_splunk_hec streams: - inputDataType: METRICS outputFormat: SPLUNK selector: names=http.request.bytes_rcvd&filter=app='myapp' - inputDataType: METRICS outputFormat: SPLUNK selector: names=http.request.count&filter=app='myapp' currentStatus: state: selfConfigState: isConfigured: false isConfiguring: true isError: false isDeleting: false total: 1 configured: 0 configuring: 1 error: 0 deleting: 0 collectorType: SPLUNK integrationRef: ref: /platform/integrations/splunk_hec streams: - inputDataType: METRICS outputFormat: SPLUNK selector: names=http.request.bytes_rcvd&filter=app='myapp' - inputDataType: METRICS outputFormat: SPLUNK selector: names=http.request.count&filter=app='myapp' CurrentStatus: type: object required: - integrationRef - streams - state - collectorType properties: state: $ref: '#/components/schemas/ConfigState' integrationRef: $ref: '#/components/schemas/ResourceRef' collectorType: type: string enum: - DATADOG - SPLUNK - SYSLOG - OTLP_HTTP - OTLP_GRPC description: > The type of Collector receiving forwarded data. The parameter sets the communication protocol and how the Forwarder monitors the liveness of the Collector. streams: type: array items: $ref: '#/components/schemas/ForwarderStream' DesiredState: type: object required: - integrationRef - streams - collectorType properties: integrationRef: $ref: '#/components/schemas/ResourceRef' collectorType: type: string enum: - DATADOG - SPLUNK - SYSLOG description: > The type of Collector receiving forwarded data. The parameter sets the communication protocol and how the Forwarder monitors the liveness of the Collector. streams: type: array items: $ref: '#/components/schemas/ForwarderStream' ForwarderStream: type: object required: - outputFormat - inputDataType - selector description: > There is no global limitation on number of streams. When creating a Stream, be sure that the metrics you select are not already included in an existing Stream. When you add the same metrics to multiple Streams -- which is known as "overlapping Streams" -- you may see significant performance degradation and data duplication. The duplicated data may not be interpreted correctly by the Collector. properties: outputFormat: type: string enum: - SPLUNK - DATADOG - SYSLOG - OTLP description: > The format in which you want to send data to the Collector. example: SPLUNK inputDataType: type: string enum: - METRICS - EVENTS description: The type of data that the stream is processing. example: METRICS selector: type: string description: > The selector object limits the amount of data sent from NGINX Controller to the desired Collector. The format of this parameter depends on the `inputDataType` value (either `METRICS` or `EVENTS`). An empty selector indicates that all collected data is going to be forwarded to the Collector. Ensure sending all your data out of NGINX Controller does not cause additional charges. Selector elements and rules for `inputDataType`=`METRICS`: Allowed parameters: `excluded_names`, `names`, `filter`. The `filter` parameter and its behavior is not specific to `METRICS` `inputDataType`. `excluded_names` parameter consists of zero or more metric names that should not be forwarded `names` parameter consists of zero or more metric names that should be forwarded The specific rules for these parameters are as follows: - if no value is provided or the only value is a wildcard (`*`), then all metric names will be forwarded. - if an explicit list of metric names is provided, all of these metrics will be forwarded. - any metric name can contain wildcard (`*`) character(s), in which case it will be resolved to all valid metrics that match the pattern. - `excluded_names` can be composed of metrics names and a wildcard (`*`) which should not be forwarded - any metric resolved from a `names` wildcard can be removed by an explicit or wildcard metric from `excluded_names` - any metric provided explicitly (full name without a wildcard) in the `names` parameter cannot be excluded by a wildcard or an explicit metric from the `excluded_names` parameter. That is, `names=system.cpu.idle&excluded_names=system.*` results in an error - `excluded_names` parameter is always applied after the resolution of the metrics from `names` parameter Make sure that all metrics provided in the `names` and `excluded_names` parameters resolve to an existing catalog metric or metrics. Sample use: - `names=http.request.count, http.request.bytes_sent` indicates that only samples of those two metrics will be forwarded - `filter=app='myapp' & (env='prod' or env='dev')` indicates that only samples related to desired app/env will be forwarded, also objects that does not contain filtered dimensions won't be sent - `names=http.request.count, http.request.bytes_sent&filter=app='myapp' AND (env='prod' or env='dev')` combines both above - `excluded_names=nginx.http.request.*` all metrics except matching `http.request.*` will be forwarded - `excluded_names=nginx.http.request.time.*&names=nginx.http.request.*` `excluded_names` can be combined with the `names` parameter to exclude metrics that were selected with a wildcard in the `names` parameters Filtering rules for `inputDataType`=`EVENTS`: Allowed parameters: `filter`. The `filter` parameter and its behavior is not specific to `EVENTS` `inputDataType`. Filter parameter: Filter parameter is shared between all supported data types and has the same behavior when used with any of them. It consists of one or more predicates in the form `` where: - `` is the name of the dimension, - `` is one of the supported operators (`=`, `!=`, `<`, `<=`, `>=` `>`, `in` or `not`), - `` is value of the dimension(s) that you want to filter on, - ` and ` is case sensitive. For example: 'count > 100'. Predicates can be combined into logical expressions using `OR`, `AND`, and `(` `)`. Wrapping predicates in single quotes is recommended to ensure that the full query string is processed correctly. For matching values, wildcard (`*`) use is supported. Make sure that all dimensions provided in the `filter` parameters resolve to an existing catalog dimension. Any expression with dimension not present in filtered metric/event will evaluate to `false`. This enables user to create filter which will forward metric/event with dimension not valid for particular metric/event but as whole filter expression will evaluate to true and only parts will evaluate to false. Example: - `filter=file_path='/etc/hosts' OR instance='instance_name'` will forward `http.request.body_bytes_sent` as `file_path` dimension is invalid but whole expression will evaluate to true - `filter=file_path='/etc/hosts' AND instance='instance_name'` will not forward `http.request.body_bytes_sent` as `file_path` dimension is invalid and whole expression will always evaluate to false, but will forward `sytem.disk.used` which has both dimensions example: >- names=http.request.count,http.request.*,*latency*&filter=((app!='app1' OR app='app2') AND environment in ('aa', 'bb')) AND (tags!='*tag1*' AND interface='ens*') ForwarderListResponse: type: object properties: items: type: array items: $ref: '#/components/schemas/Forwarder' description: List of Forwarder resources. example: items: - metadata: name: splunk displayName: Splunk - Metrics description: Metrics forwarder going to Splunk HEC kind: forwarder uid: d290f1ee-6c54-4b01-90e6-d701748f0851 links: rel: /api/v1/analytics/forwarders/splunk createTime: '2019-07-29T09:12:33.001Z' updateTime: '2019-07-29T10:12:33.001Z' desiredState: collectorType: SPLUNK integrationRef: ref: /platform/integrations/splunk_hec streams: - inputDataType: METRICS outputFormat: SPLUNK selector: names=http.request.bytes_rcvd - inputDataType: METRICS outputFormat: SPLUNK selector: names=http.request.count currentStatus: state: selfConfigState: isConfigured: true isConfiguring: false isError: false isDeleting: false total: 1 configured: 1 configuring: 0 error: 0 deleting: 0 conditions: - type: error message: integration reference not found collectorType: SPLUNK integrationRef: ref: /platform/integrations/splunk_hec streams: - inputDataType: METRICS outputFormat: SPLUNK selector: >- names=http.request.bytes_rcvd&filter=app='myapp' - inputDataType: METRICS outputFormat: SPLUNK selector: names=http.request.count&filter=app='myapp' - metadata: name: splunk displayName: Splunk - Metrics description: Metrics forwarder going to Splunk HEC kind: forwarder uid: d290f1ee-6c54-4b01-90e6-d701748f0851 links: rel: /api/v1/analytics/forwarders/splunk createTime: '2019-07-29T09:12:33.001Z' updateTime: '2019-07-29T10:12:33.001Z' desiredState: collectorType: SPLUNK integrationRef: ref: /platform/integrations/splunk_hec streams: - inputDataType: METRICS outputFormat: SPLUNK selector: >- names=http.request.bytes_rcvd&filter=app='myapp' - inputDataType: METRICS outputFormat: SPLUNK selector: names=http.request.count&filter=app='myapp' currentStatus: state: selfConfigState: isConfigured: true isConfiguring: false isError: false isDeleting: false total: 0 configured: 1 configuring: 0 error: 0 deleting: 0 conditions: - type: error message: integration reference not found collectorType: SPLUNK integrationRef: ref: /platform/integrations/splunk_hec streams: - inputDataType: METRICS outputFormat: SPLUNK selector: >- names=http.request.bytes_rcvd&filter=app='myapp' - inputDataType: METRICS outputFormat: SPLUNK selector: names=http.request.count&filter=app='myapp' SelfLinks: type: object description: > The SelfLinks object contains a link from the resource to itself. This object is used only in responses. properties: rel: type: string example: /api/v1/services/environments/prod description: > `rel` contains the complete path fragment of a URI and can be used to construct a query to the object. ResourceMeta: type: object required: - name properties: name: type: string pattern: >- ^[^A-Z\s\x00-\x1f\x60\x7f\;\*\"\[\]\{\}\\\/%\?:=&\~\^|#<>]+$ not: type: string enum: - . - .. minLength: 1 maxLength: 1024 example: resource-name description: > Resource name is a unique identifier for a resource within the context of a namespace. Resource names must conform to [RFC 1738 Section 2.2](https://www.ietf.org/rfc/rfc1738.txt) and have a valid syntax for email addresses. The following rules are enforced: - do not utilize URL encoding; - do not include spaces; - do not use uppercase characters, for example, 'A-Z'; extended character sets are supported; - do not use the following characters: `"`, `*`, `:`, `;`, `/`, `\`, `%`, `?`, `hash`, `=`, `&`, `|`, `~`, `^`, `{`, `}`, `[`, `]`, `<`, `>`; - cannot start or end with an `@` sign; - cannot be only `.` or `..` For example: For a collection resource located at `https://controller.example.com/api/v1/services/apps/shopping_@1` the resource name is "shopping_@1". displayName: type: string example: My Display Name description: > `displayName` is a user friendly resource name. It can be used to define a longer, and less constrained, name for a resource. Display names: - are optional (defaults to an empty string if no value is provided), - do not have to be unique, - cannot be assigned by the server. description: type: string example: >- This is a sample description string. It provides information about the resource. description: > `description` is a free-form text property. You can use it to provide information that helps to identify the resource. Descriptions: - are optional (defaults to an empty string if no value is provided), - do not have to be unique, - cannot be assigned by the server. kind: type: string example: - description: > Kind is a string representation of an API resource's data type. It is assigned by the server and cannot be changed. When creating a `kind`, the server uses hyphens to connect word segments; singleton and collection item resources are not pluralized. uid: type: string format: uuid example: d290f1ee-6c54-4b01-90e6-d701748f0851 description: > Unique Identifier (UID) UID is a unique identifier in time and space for a resource. When you create a resource, the server assigns a UID to the resource. Refer to [IETF RFC 4122](https://tools.ietf.org/html/rfc4122) for more information. tags: type: array items: type: string example: - production_public - dev - new_app - us-west-1 - emea description: > You can assign `tags` to a resource as a way to help map, scope, and organize resources. The system uses tag selectors to specify selection criteria that match resources that have particular tags. ref: type: string example: /services/environments/prod description: > The `ref` field contains a reference to another NGINX Controller resource. links: $ref: '#/components/schemas/SelfLinks' createTime: type: string format: date-time example: '2019-07-29T09:12:33.001Z' description: > A timestamp that represents the server time when the resource was created. Create time is not guaranteed to be set in "happens-before" order across separate operations. In JSON format, `create_time` type is encoded as a string in the [RFC 3339](https://www.ietf.org/rfc/rfc3339.txt). For example: 2018-04-01T01:30:15.01Z Create Time is assigned by the server and cannot be changed. updateTime: type: string format: date-time example: '2019-07-29T10:12:33.001Z' description: > A timestamp that represents the server time when the resource was last modified. Resources that have never been updated do not have an `update_time` stamp. The default value for resources that have never been updated is the local language-specific equivalent of "null". In JSON format, `update_time` type is encoded as a string as described in [RFC 3339](https://www.ietf.org/rfc/rfc3339.txt). NamedLinks: allOf: - $ref: '#/components/schemas/SelfLinks' - type: object description: > Contains information about the object being referred to. These are generally details -- like the object name and display name -- that are useful to a consumer of the API that performs further processing. This object is only present in responses. properties: name: type: string example: production description: | The name of the linked resource. displayName: type: string example: Production Environment description: A user friendly resource name. ResourceRef: type: object required: - ref properties: ref: type: string example: /services/environments/prod description: | A reference to another NGINX Controller resource. links: $ref: '#/components/schemas/NamedLinks' ConfigStateTally: type: object properties: isConfigured: type: boolean description: The configuration operation is complete. isConfiguring: type: boolean description: >- The configuration of the resource, or of its child(ren), is in process. isError: type: boolean description: >- An error occurred while configuring the resource or its child(ren). isDeleting: type: boolean description: >- A delete operation is in progress for the resource or its child(ren). total: type: integer description: >- The total number of resources to which the configuration operation applies. configured: type: integer description: >- The number of resources that have a complete and valid configuration. configuring: type: integer description: >- The number of resources that are in the process of being configured. error: type: integer description: >- The number of resources that have encountered an error during the configuration process. deleting: type: integer description: >- The number of resources that are in the process of being deleted. ConfigCondition: type: object properties: type: type: string description: The condition type. message: type: string description: >- A human-readable message that provides additional information about the configuration operation. ConfigState: type: object description: > A representation of the resource's current configuration state that comprises the status of the resource itself (`selfConfigState`) and any child resources (`childrenConfigState`). The conditions array provides additional information during configuration changes. properties: selfConfigState: $ref: '#/components/schemas/ConfigStateTally' childrenConfigState: $ref: '#/components/schemas/ConfigStateTally' conditions: type: array items: $ref: '#/components/schemas/ConfigCondition' ErrorDetail: type: object required: - description properties: description: type: string example: >- Error doing : . This can lead to . Try to resolve the issue. description: > A detailed error message returned by the server. These messages contain the following information, where applicable: - What happened. - Why it happened. - What the consequences are (if any). - Recommended action to take to resolve the issue. ErrorModel: type: object required: - message - code properties: message: type: string example: Error doing . description: > A human-readable message, in English, that describes the error. code: type: integer example: 1234567 description: > A numeric error code that can be used to identify errors for support purposes. details: type: array items: $ref: '#/components/schemas/ErrorDetail'