Note that this property accepts only String keys, while Decaton consumer supports consuming keys of arbitrary type. This means that records with non-String keys may just pass through this filter.
Reloadable: yes" }, "decaton.processing.rate.per.partition" : { "type" : "integer", "default" : -1, "description" : "Maximum rate of processing tasks per-partition in second.
If the value N is:
See also {@link RateLimiter RateLimiter}.
Reloadable: yes" }, "decaton.partition.concurrency" : { "type" : "integer", "default" : 1, "description" : "Concurrency used to process tasks coming from single partition.
Reloading this property will be performed for each assigned partition as soon as the current pending tasks of the assigned partition have done.
Reloadable: yes" }, "decaton.max.pending.records" : { "type" : "integer", "default" : 10000, "description" : "Number of records to pause source partition if pending count exceeds this number.
Reloading this property will be performed for each assigned partition as soon as the current pending tasks of the assigned partition have done.
Reloadable: yes" }, "decaton.commit.interval.ms" : { "type" : "integer", "default" : 1000, "description" : "Interval in milliseconds to put in between offset commits.
Too frequent offset commit would cause high load on brokers while it doesn't essentially prevent duplicate processing.
Reloadable: yes" }, "decaton.group.rebalance.timeout.ms" : { "type" : "integer", "default" : 1000, "description" : "Timeout for consumer group rebalance.
Decaton waits up to this time for tasks currently pending or in-progress to finish before allowing a rebalance to proceed.
Any tasks that do not complete within this timeout will not have their offsets committed before the rebalance, meaning they may be processed multiple times (as they will be processed again after the rebalance). If {@link com.linecorp.decaton.processor.runtime.ProcessorProperties#CONFIG_PARTITION_CONCURRENCY #CONFIG_PARTITION_CONCURRENCY} is greater than 1, this situation might also cause other records from the same partition to be processed multiple times (see {@link OutOfOrderCommitControl OutOfOrderCommitControl}).
Generally, this should be set such that {@link com.linecorp.decaton.processor.runtime.ProcessorProperties#CONFIG_MAX_PENDING_RECORDS #CONFIG_MAX_PENDING_RECORDS} can be comfortably processed within this timeout.
Reloadable: yes" }, "decaton.processing.shutdown.timeout.ms" : { "type" : "integer", "default" : 0, "description" : "Timeout for processor close. Decaton waits up to this time for tasks currently pending or in-progress to finish. Any tasks that do not complete within this timeout will mean async task processing code may still be running even after {@link ProcessorSubscription#close ProcessorSubscription#close()} returns, which might lead to errors from e.g. shutting down dependencies of this {@link ProcessorSubscription ProcessorSubscription} that are still in use from async tasks.
Generally, this should be set such that {@link com.linecorp.decaton.processor.runtime.ProcessorProperties#CONFIG_MAX_PENDING_RECORDS #CONFIG_MAX_PENDING_RECORDS} can be comfortably processed within this timeout.
Reloadable: yes" }, "decaton.logging.mdc.enabled" : { "type" : "boolean", "default" : true, "description" : "Control whether to enable or disable decaton specific information store in SLF4J's {@link MDC MDC}. This option is enabled by default, but it is known to cause some object allocations which could become a problem in massive scale traffic. This option intend to provide an option for users to disable MDC properties where not necessary to reduce GC pressure.
Reloadable: yes" }, "decaton.client.metrics.micrometer.bound" : { "type" : "boolean", "default" : false, "description" : "Controls whether to enable or disable binding Micrometer's KafkaClientMetrics to decaton consumers. This is disabled for backwards compatibility, but recommended if you rely on Micrometer since JMX metrics are deprecated. The downside is a possible increase in metrics count.
Reloadable: no" }, "decaton.deferred.complete.timeout.ms" : { "type" : "integer", "default" : -1, "description" : "Control time to \"timeout\" a deferred completion.
Decaton allows {@link DecatonProcessor DecatonProcessor}s to defer completion of a task by calling {@link ProcessingContext#deferCompletion ProcessingContext#deferCompletion()}, which is useful for processors which integrates with asynchronous processing frameworks that sends the processing context to somewhere else and get back later.
However, since leaking {@link Completion Completion} returned by {@link ProcessingContext#deferCompletion ProcessingContext#deferCompletion()} means to create a never-completed task, that causes consumption to suck completely after {@link com.linecorp.decaton.processor.runtime.ProcessorProperties#CONFIG_MAX_PENDING_RECORDS #CONFIG_MAX_PENDING_RECORDS} records stacked up, which is not desirable for some use cases.
By setting this timeout, Decaton will try to \"timeout\" a deferred completion after the specified period. By setting the timeout to sufficiently large value, which you can be sure that none of normal processing to take, some potentially leaked completion will be forcefully completed and decaton can continue to consume the following tasks.
Be very careful when using this feature since forcefully completing a timed out completion might lead to some data loss if the corresponding processing hasn't yet complete.
This timeout can be disabled by setting -1, and it is the default.
Reloadable: yes" }, "decaton.processor.threads.termination.timeout.ms" : { "type" : "integer", "default" : 9223372036854775807, "description" : "Timeout for processor threads termination.
When a partition is revoked for rebalance or a subscription is about to be shutdown, all processors will be destroyed. At this time, Decaton waits synchronously for the running tasks to finish until this timeout.
Even if timeout occurs, Decaton will continue other clean-up tasks. Therefore, you can set this timeout only if unexpected behavior is acceptable in the middle of the last {@link DecatonProcessor#process(ProcessingContext, Object) DecatonProcessor#process(ProcessingContext, Object)} which timed out.
Reloadable: yes" }, "decaton.per.key.quota.processing.rate" : { "type" : "integer", "default" : -1, "description" : "Control per-key processing rate per second quota. Only effective when per-key quota is enabled by {@link SubscriptionBuilder#enablePerKeyQuota SubscriptionBuilder#enablePerKeyQuota}.
Reloadable: yes" }, "decaton.retry.task.in.legacy.format" : { "type" : "boolean", "default" : false, "description" : "Controls whether to produce retry tasks in deprecated {@link DecatonTaskRequest DecatonTaskRequest} format.
CAUTION!!! YOU MAY NEED TO SET THIS TO TRUE WHEN YOU UPGRADE FROM 8.0.1 OR EARLIER
Please read Decaton 9.0.0 Release Note carefully.
Reloadable: yes" }, "decaton.legacy.parse.fallback.enabled" : { "type" : "boolean", "default" : false, "description" : "Controls whether to parse records as {@link DecatonTaskRequest DecatonTaskRequest} format when task metadata header is missing when {@link Deserializer Deserializer} is used, instead of parsing task directly with the deserializer and fill reasonably-default task metadata.
CAUTION!!! YOU MAY NEED TO SET THIS TO TRUE WHEN YOU UPGRADE FROM 8.0.1 OR EARLIER
Please read Decaton 9.0.0 Release Note carefully.
Reloadable: yes" } } }