//tag::ref-doc[]
= Spring Cloud Data Flow Tasklauncher Sink

[NOTE]
Use of this app currently requires manual registration.
This app is a REST client of Spring Cloud Data Flow.
Frequently users have experienced errors caused by using a version incompatible with the deployed SCDF server.
To address this issue, as of SCDF 2.9.x, this app is included in the Spring Cloud Dataflow repository, and build and release, to ensure API compatibility and interoperability with the current SCDF server version.
As a consequence, the `dataflow-tasklauncher` sink is no longer part of the pre-packaged apps release, and no longer included in the bulk import.

To manually register the `dataflow-tasklauncher` for either the `rabbit` or `kafka` binder, use one of the following URLs:

 docker:springcloud/spring-cloud-dataflow-tasklauncher-sink-<binder>:<SCDF-version>

 maven://org.springframework.cloud:spring-cloud-dataflow-tasklauncher-sink-<binder>:<SCDF-version>

Example: `docker:springcloud/spring-cloud-dataflow-tasklauncher-sink-kafka:2.9.0`

== About
This module consumes LaunchRequest messages from a `PollableMessageSource` and uses the Data Flow REST client to launch a registered task on a configured https://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/#configuration-local-tasks[task platform].
The client must be configured to connect to a remote Data Flow Server, including any required authentication (see Configuration Options below).

This application launches a registered task definition using the Data Flow Server https://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/#api-guide-resources-task-executions-launching[REST API].

== Input

a Task Launch Request including:

* the task name (required and created as a task with the target Data Flow Server)
* deployment properties (key value pairs, optional).
* program arguments for the task (a list, optional).

The message payload can be a JSON document:

[source,json]
----
{
  "name":"foo",
  "deploymentProps": {"key1":"val1","key2":"val2"},
  "args":["--debug", "--foo", "bar"]
}
----

Minimally, it must include the task name.

[source,json]
----
{"name":"foo"}
----

=== Options

The **$$dataflow-tasklauncher$$** $$sink$$ supports the following configuration properties:

//tag::configuration-properties[]
Properties grouped by prefix:


=== spring.cloud.dataflow.client.authentication

$$access-token$$:: $$OAuth2 Access Token.$$ *($$String$$, default: `$$<none>$$`)*
$$client-id$$:: $$OAuth2 Client Id.$$ *($$String$$, default: `$$<none>$$`)*
$$client-secret$$:: $$OAuth2 Client Secret.$$ *($$String$$, default: `$$<none>$$`)*
$$scope$$:: $$OAuth2 Scopes.$$ *($$Set<String>$$, default: `$$<none>$$`)*
$$token-uri$$:: $$OAuth2 Token Uri.$$ *($$String$$, default: `$$<none>$$`)*

=== spring.cloud.dataflow.client.authentication.basic

$$password$$:: $$The login password.$$ *($$String$$, default: `$$<none>$$`)*
$$username$$:: $$The login username.$$ *($$String$$, default: `$$<none>$$`)*

=== spring.cloud.dataflow.client.authentication.oauth2

$$client-registration-id$$:: $$<documentation missing>$$ *($$String$$, default: `$$<none>$$`)*
$$password$$:: $$<documentation missing>$$ *($$String$$, default: `$$<none>$$`)*
$$username$$:: $$<documentation missing>$$ *($$String$$, default: `$$<none>$$`)*

=== spring.cloud.dataflow.client

$$enable-dsl$$:: $$Enable Data Flow DSL access.$$ *($$Boolean$$, default: `$$false$$`)*
$$server-uri$$:: $$The Data Flow server URI.$$ *($$String$$, default: `$$http://localhost:9393$$`)*
$$skip-ssl-validation$$:: $$Skip Ssl validation.$$ *($$Boolean$$, default: `$$false$$`)*
//end::configuration-properties[]

== Using the TaskLauncher
The dataflow-tasklauncher sink consumes `LaunchRequest` messages, as described above, and launches a task using the target Data Flow server (given by `--spring.cloud.dataflow.client.server-uri`).
The task launcher periodically polls its input source for launch requests but will pause polling when the platform has reached it's concurrent task execution limit, given by `spring.cloud.dataflow.task.platform.<platform-type>.accounts[<account-name>].maximum-concurrent-tasks`.
This prevents the SCDF deployer's deployment platform from exhausting its resources under heavy task load.
The poller is scheduled using a `DynamicPeriodicTrigger`. By default the initial polling rate is 1 second, but may be configured to any duration. When polling is paused, or if there are no launch requests present, the trigger period will increase, applying exponential backoff, up to a configured maximum (30 seconds by default).

NOTE: This version of the Data Flow task launcher is certified for the corresponding Spring Cloud Dataflow Server version.

The SCDF server may be configured to launch tasks on multiple platforms.
Each task launcher instance is configured for a single platform, given by the `platformName` property (`default` if not specified).
This limitation is enforced because if the server has multiple task platforms configured, it may be the case that some of its task platforms are at the limit and some are not.
In this situation, we can only consume the next launch request if we know for which task platform it is targeted.
For this reason, if the SCDF server is configured for multiple task platforms (or a single non-default platform), we assume that all launch requests are targeted for that platform.
The task launcher will set the required deployment property `spring.cloud.dataflow.task.platformName` if the request does not provide it.

NOTE: If the request includes the deployment property `spring.cloud.dataflow.task.platformName`, and the value is not the same as the tasklauncher's `platformName`, the task launcher will throw an exception.

To launch tasks on multiple platforms, you must configure a task launcher instance per platform and use a link:../router-sink/[router sink], or https://docs.spring.io/spring-cloud-stream/docs/current/reference/htmlsingle/#partitioning[partitioning strategy], to route requests to the correct instance.

NOTE: When the poller is paused it puts pressure
on the message broker, so some tuning will be necessary in extreme cases to balance resource utilization.

=== Client Authentication

If the Data Flow server requires authentication, the client must pass credentials with authorization to launch a task.
The Data Flow client supports both basic and OAuth2 authentication.

For basic authentication set the username and password:

```
--spring.cloud.dataflow.client.authentication.basic.username=<username> --spring.cloud.dataflow.client.authentication.basic.password=<password>
```

For OAuth2 authentication, set the `client-id`, `client-secret`, and `token-uri` at a minimum. These values correspond to values set in the SCDF server's OAuth2 configuration.
For more details, see https://docs.spring.io/spring-cloud-dataflow/docs/current/reference/htmlsingle/#configuration-local-security[the Security section in the Data Flow reference].

```
--spring.cloud.dataflow.client.authentication.client-id=<client-id> --spring.cloud.dataflow.client.authentication.client-secret=<client-secret> spring.cloud.dataflow.client.authentication.token-uri: <token-uri>
```

//end::ref-doc[]
