openapi: 3.2.0 info: title: OpenAPI spec for ClickHouse Cloud Click Pipes API version: '1.0' contact: name: ClickHouse Support url: https://clickhouse.com/docs/en/cloud/manage/openapi?referrer=openapi-1107336 email: support@clickhouse.com servers: - url: https://api.clickhouse.cloud security: - basicAuth: [] tags: - name: ClickPipes paths: /v1/organizations/{organizationId}/services/{serviceId}/clickpipes: get: summary: List ClickPipes description: Returns a list of ClickPipes. operationId: clickPipeGetList parameters: - in: path name: organizationId description: ID of the organization that owns the service. required: true schema: type: string format: uuid - in: path name: serviceId description: ID of the service that owns the ClickPipe. required: true schema: type: string format: uuid responses: '200': description: Successful response content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 200 requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid result: type: array items: $ref: '#/components/schemas/ClickPipe' '400': description: The request cannot be processed due to a client error. Please verify your request parameters and try again. content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 400 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '500': description: An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance. content: application/json: schema: type: object properties: status: type: integer description: HTTP status code. example: 500 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid tags: - ClickPipes post: summary: Create ClickPipe description: Create a new ClickPipe. operationId: clickPipeCreate parameters: - in: path name: organizationId description: ID of the organization that owns the service. required: true schema: type: string format: uuid - in: path name: serviceId description: ID of the service to create the ClickPipe for. required: true schema: type: string format: uuid requestBody: content: application/json: schema: $ref: '#/components/schemas/ClickPipePostRequest' responses: '200': description: Successful response content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 200 requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid result: $ref: '#/components/schemas/ClickPipe' '400': description: The request cannot be processed due to a client error. Please verify your request parameters and try again. content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 400 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '500': description: An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance. content: application/json: schema: type: object properties: status: type: integer description: HTTP status code. example: 500 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid tags: - ClickPipes /v1/organizations/{organizationId}/services/{serviceId}/clickpipes/context: get: summary: Get ClickPipes service context description: '**This endpoint is in beta.** API contract is stable, and no breaking changes are expected in the future. Returns service-level ClickPipes capabilities and Private Preview workload identity context, including the GCP service account to grant access to customer source resources.' operationId: clickPipesServiceContextGet parameters: - in: path name: organizationId description: ID of the organization that owns the service. required: true schema: type: string format: uuid - in: path name: serviceId description: ID of the service to get ClickPipes context for. required: true schema: type: string format: uuid responses: '200': description: Successful response content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 200 requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid result: $ref: '#/components/schemas/ClickPipesServiceContext' '400': description: The request cannot be processed due to a client error. Please verify your request parameters and try again. content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 400 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '500': description: An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance. content: application/json: schema: type: object properties: status: type: integer description: HTTP status code. example: 500 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid tags: - ClickPipes x-badges: - name: Beta position: after /v1/organizations/{organizationId}/services/{serviceId}/clickpipes/{clickPipeId}: get: summary: Get ClickPipe description: Returns the specified ClickPipe. operationId: clickPipeGet parameters: - in: path name: organizationId description: ID of the organization that owns the service. required: true schema: type: string format: uuid - in: path name: serviceId description: ID of the service that owns the ClickPipe. required: true schema: type: string format: uuid - in: path name: clickPipeId description: ID of the requested ClickPipe. required: true schema: type: string format: uuid responses: '200': description: Successful response content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 200 requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid result: $ref: '#/components/schemas/ClickPipe' '400': description: The request cannot be processed due to a client error. Please verify your request parameters and try again. content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 400 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '500': description: An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance. content: application/json: schema: type: object properties: status: type: integer description: HTTP status code. example: 500 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid tags: - ClickPipes patch: summary: Update ClickPipe description: Update the specified ClickPipe. Source fields not present in the per-source update schemas are immutable after creation. For Kafka sources, values submitted for immutable fields (type, format, brokers, topics, consumerGroup, offset, schemaRegistry, exactlyOnce) are not applied, except schema registry credentials, which are rejected. operationId: clickPipeUpdate parameters: - in: path name: organizationId description: ID of the organization that owns the service. required: true schema: type: string format: uuid - in: path name: serviceId description: ID of the service to create the ClickPipe for. required: true schema: type: string format: uuid - in: path name: clickPipeId description: ID of the requested ClickPipe. required: true schema: type: string format: uuid requestBody: content: application/json: schema: $ref: '#/components/schemas/ClickPipePatchRequest' responses: '200': description: Successful response content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 200 requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid result: $ref: '#/components/schemas/ClickPipe' '400': description: The request cannot be processed due to a client error. Please verify your request parameters and try again. content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 400 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '500': description: An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance. content: application/json: schema: type: object properties: status: type: integer description: HTTP status code. example: 500 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid tags: - ClickPipes delete: summary: Delete ClickPipe description: Delete the specified ClickPipe. operationId: clickPipeDelete parameters: - in: path name: organizationId description: ID of the organization that owns the service. required: true schema: type: string format: uuid - in: path name: serviceId description: ID of the service that owns the ClickPipe. required: true schema: type: string format: uuid - in: path name: clickPipeId description: ID of the ClickPipe to delete. required: true schema: type: string format: uuid responses: '200': description: Successful response content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 200 requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '400': description: The request cannot be processed due to a client error. Please verify your request parameters and try again. content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 400 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '500': description: An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance. content: application/json: schema: type: object properties: status: type: integer description: HTTP status code. example: 500 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid tags: - ClickPipes /v1/organizations/{organizationId}/services/{serviceId}/clickpipes/{clickPipeId}/settings: get: summary: Get ClickPipe settings description: Returns the advanced settings for the specified ClickPipe. operationId: clickPipeSettingsGet parameters: - in: path name: organizationId description: ID of the organization that owns the service. required: true schema: type: string format: uuid - in: path name: serviceId description: ID of the service that owns the ClickPipe. required: true schema: type: string format: uuid - in: path name: clickPipeId description: ID of the ClickPipe to get settings for. required: true schema: type: string format: uuid responses: '200': description: Successful response content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 200 requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid result: $ref: '#/components/schemas/ClickPipeSettings' '400': description: The request cannot be processed due to a client error. Please verify your request parameters and try again. content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 400 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '500': description: An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance. content: application/json: schema: type: object properties: status: type: integer description: HTTP status code. example: 500 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid tags: - ClickPipes put: summary: Update ClickPipe settings description: Update the advanced settings for the specified ClickPipe. Send key-value pairs where values can be strings, numbers, or booleans. operationId: clickPipeSettingsUpdate parameters: - in: path name: organizationId description: ID of the organization that owns the service. required: true schema: type: string format: uuid - in: path name: serviceId description: ID of the service that owns the ClickPipe. required: true schema: type: string format: uuid - in: path name: clickPipeId description: ID of the ClickPipe to update settings for. required: true schema: type: string format: uuid requestBody: content: application/json: schema: $ref: '#/components/schemas/ClickPipeSettingsPutRequest' responses: '200': description: Successful response content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 200 requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid result: $ref: '#/components/schemas/ClickPipeSettings' '400': description: The request cannot be processed due to a client error. Please verify your request parameters and try again. content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 400 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '500': description: An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance. content: application/json: schema: type: object properties: status: type: integer description: HTTP status code. example: 500 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid tags: - ClickPipes /v1/organizations/{organizationId}/services/{serviceId}/clickpipes/schemaDiscovery: post: summary: Discover ClickPipe source schema description: '**This endpoint is in beta.** API contract is stable, and no breaking changes are expected in the future. Infers the schema (field names and ClickHouse data types) of a ClickPipe source without creating a pipe. Supported for Kafka, Kinesis, Pub/Sub, and object storage sources. Object storage inference runs on the destination service, which must be running.' operationId: clickPipeSchemaDiscovery parameters: - in: path name: organizationId description: ID of the organization that owns the service. required: true schema: type: string format: uuid - in: path name: serviceId description: ID of the service to run schema discovery against. required: true schema: type: string format: uuid requestBody: content: application/json: schema: $ref: '#/components/schemas/ClickPipeSchemaDiscoveryRequest' responses: '200': description: Successful response content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 200 requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid result: $ref: '#/components/schemas/ClickPipeSchemaDiscoveryResponse' '400': description: The request cannot be processed due to a client error. Please verify your request parameters and try again. content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 400 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '500': description: An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance. content: application/json: schema: type: object properties: status: type: integer description: HTTP status code. example: 500 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid tags: - ClickPipes x-badges: - name: Beta position: after /v1/organizations/{organizationId}/services/{serviceId}/clickpipes/{clickPipeId}/scaling: patch: summary: Update ClickPipe scaling description: 'Change scaling settings for the specified ClickPipe. This endpoint supports Kafka, Kinesis, and object storage pipes (S3, GCS, Azure Blob). **Note:** For database ClickPipes (PostgreSQL, MySQL, MongoDB, BigQuery), use the Update CDC ClickPipes scaling endpoint instead.' operationId: clickPipeScalingUpdate parameters: - in: path name: organizationId description: ID of the organization that owns the service. required: true schema: type: string format: uuid - in: path name: serviceId description: ID of the service that owns the ClickPipe. required: true schema: type: string format: uuid - in: path name: clickPipeId description: ID of the ClickPipe to update scaling settings. required: true schema: type: string format: uuid requestBody: content: application/json: schema: $ref: '#/components/schemas/ClickPipeScalingPatchRequest' responses: '200': description: Successful response content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 200 requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid result: $ref: '#/components/schemas/ClickPipe' '400': description: The request cannot be processed due to a client error. Please verify your request parameters and try again. content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 400 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '500': description: An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance. content: application/json: schema: type: object properties: status: type: integer description: HTTP status code. example: 500 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid tags: - ClickPipes /v1/organizations/{organizationId}/services/{serviceId}/clickpipes/{clickPipeId}/state: patch: summary: Update ClickPipe state description: Start, stop or resync ClickPipe. Stopping a ClickPipe will stop the ingestion process from any state. Starting is allowed for ClickPipes in the "Stopped" state or with a "Failed" state. Resyncing is only for Postgres and MySQL pipes and can be done from any state. operationId: clickPipeStateUpdate parameters: - in: path name: organizationId description: ID of the organization that owns the service. required: true schema: type: string format: uuid - in: path name: serviceId description: ID of the service that owns the ClickPipe. required: true schema: type: string format: uuid - in: path name: clickPipeId description: ID of the ClickPipe to update state. required: true schema: type: string format: uuid requestBody: content: application/json: schema: $ref: '#/components/schemas/ClickPipeStatePatchRequest' responses: '200': description: Successful response content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 200 requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid result: $ref: '#/components/schemas/ClickPipe' '400': description: The request cannot be processed due to a client error. Please verify your request parameters and try again. content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 400 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '500': description: An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance. content: application/json: schema: type: object properties: status: type: integer description: HTTP status code. example: 500 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid tags: - ClickPipes /v1/organizations/{organizationId}/services/{serviceId}/clickpipesCdcScaling: get: summary: Get CDC ClickPipes scaling description: 'Get scaling settings for database ClickPipes (PostgreSQL, MySQL, MongoDB, BigQuery). The infrastructure is shared between all database ClickPipes in the service, both for initial load and CDC. For billing purposes, 2 CPU cores and 8 GB of RAM correspond to one compute unit. **Note:** For Kafka, Kinesis, and object storage pipes (S3, GCS, Azure Blob), see Get ClickPipe. **This endpoint becomes available once at least one database ClickPipe was provisioned.**' operationId: clickPipeCdcScalingGet parameters: - in: path name: organizationId description: ID of the organization that owns the service. required: true schema: type: string format: uuid - in: path name: serviceId description: ID of the service that owns the ClickPipe. required: true schema: type: string format: uuid responses: '200': description: Successful response content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 200 requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid result: $ref: '#/components/schemas/ClickPipesCdcScaling' '400': description: The request cannot be processed due to a client error. Please verify your request parameters and try again. content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 400 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '500': description: An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance. content: application/json: schema: type: object properties: status: type: integer description: HTTP status code. example: 500 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid tags: - ClickPipes patch: summary: Update CDC ClickPipes scaling description: 'Update scaling settings for database ClickPipes (PostgreSQL, MySQL, MongoDB, BigQuery). The infrastructure is shared between all database ClickPipes in the service, both for initial load and CDC. Scaling settings may take a few minutes to fully propagate. For billing purposes, 2 CPU cores and 8 GB of RAM correspond to one compute unit. If your organization tier changes, database ClickPipes will be rescaled appropriately. **Note:** For Kafka, Kinesis, and object storage pipes (S3, GCS, Azure Blob), see Get ClickPipe. **This endpoint becomes available once at least one database ClickPipe was provisioned.**' operationId: clickPipeCdcScalingUpdate parameters: - in: path name: organizationId description: ID of the organization that owns the service. required: true schema: type: string format: uuid - in: path name: serviceId description: ID of the service that owns the ClickPipe. required: true schema: type: string format: uuid requestBody: content: application/json: schema: $ref: '#/components/schemas/ClickPipesCdcScalingPatchRequest' responses: '200': description: Successful response content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 200 requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid result: $ref: '#/components/schemas/ClickPipesCdcScaling' '400': description: The request cannot be processed due to a client error. Please verify your request parameters and try again. content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 400 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '500': description: An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance. content: application/json: schema: type: object properties: status: type: integer description: HTTP status code. example: 500 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid tags: - ClickPipes /v1/organizations/{organizationId}/services/{serviceId}/clickpipesReversePrivateEndpoints: get: summary: List reverse private endpoints description: Returns a list of reverse private endpoints for the specified service. operationId: clickPipeReversePrivateEndpointGetList parameters: - in: path name: organizationId description: ID of the organization that owns the service. required: true schema: type: string format: uuid - in: path name: serviceId description: ID of the service that owns the Reverse Private Endpoint. required: true schema: type: string format: uuid responses: '200': description: Successful response content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 200 requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid result: type: array items: $ref: '#/components/schemas/ReversePrivateEndpoint' '400': description: The request cannot be processed due to a client error. Please verify your request parameters and try again. content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 400 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '500': description: An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance. content: application/json: schema: type: object properties: status: type: integer description: HTTP status code. example: 500 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid tags: - ClickPipes post: summary: Create reverse private endpoint description: Create a new reverse private endpoint. operationId: clickPipeReversePrivateEndpointCreate parameters: - in: path name: organizationId description: ID of the organization that owns the service. required: true schema: type: string format: uuid - in: path name: serviceId description: ID of the service that owns the Reverse Private Endpoint. required: true schema: type: string format: uuid requestBody: content: application/json: schema: $ref: '#/components/schemas/CreateReversePrivateEndpoint' responses: '200': description: Successful response content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 200 requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid result: $ref: '#/components/schemas/ReversePrivateEndpoint' '400': description: The request cannot be processed due to a client error. Please verify your request parameters and try again. content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 400 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '500': description: An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance. content: application/json: schema: type: object properties: status: type: integer description: HTTP status code. example: 500 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid tags: - ClickPipes /v1/organizations/{organizationId}/services/{serviceId}/clickpipesReversePrivateEndpoints/{reversePrivateEndpointId}: get: summary: Get reverse private endpoint description: Returns the reverse private endpoint with the specified ID. operationId: clickPipeReversePrivateEndpointGet parameters: - in: path name: organizationId description: ID of the organization that owns the service. required: true schema: type: string format: uuid - in: path name: serviceId description: ID of the service that owns the Reverse Private Endpoint. required: true schema: type: string format: uuid - in: path name: reversePrivateEndpointId description: ID of the reverse private endpoint to get. required: true schema: type: string format: uuid responses: '200': description: Successful response content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 200 requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid result: $ref: '#/components/schemas/ReversePrivateEndpoint' '400': description: The request cannot be processed due to a client error. Please verify your request parameters and try again. content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 400 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '500': description: An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance. content: application/json: schema: type: object properties: status: type: integer description: HTTP status code. example: 500 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid tags: - ClickPipes delete: summary: Delete reverse private endpoint description: Delete the reverse private endpoint with the specified ID. operationId: clickPipeReversePrivateEndpointDelete parameters: - in: path name: organizationId description: ID of the organization that owns the service. required: true schema: type: string format: uuid - in: path name: serviceId description: ID of the service that owns the Reverse Private Endpoint. required: true schema: type: string format: uuid - in: path name: reversePrivateEndpointId description: ID of the reverse private endpoint to delete. required: true schema: type: string format: uuid responses: '200': description: Successful response content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 200 requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '400': description: The request cannot be processed due to a client error. Please verify your request parameters and try again. content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 400 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '500': description: An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance. content: application/json: schema: type: object properties: status: type: integer description: HTTP status code. example: 500 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid tags: - ClickPipes patch: summary: Update reverse private endpoint description: Update mutable fields for an existing reverse private endpoint. customPrivateDnsMappings is a full replacement list. Use an empty array to clear mappings. operationId: clickPipeReversePrivateEndpointUpdate parameters: - in: path name: organizationId description: ID of the organization that owns the service. required: true schema: type: string format: uuid - in: path name: serviceId description: ID of the service that owns the Reverse Private Endpoint. required: true schema: type: string format: uuid - in: path name: reversePrivateEndpointId description: ID of the reverse private endpoint to update. required: true schema: type: string format: uuid requestBody: content: application/json: schema: $ref: '#/components/schemas/UpdateReversePrivateEndpoint' responses: '200': description: Successful response content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 200 requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid result: $ref: '#/components/schemas/ReversePrivateEndpoint' '400': description: The request cannot be processed due to a client error. Please verify your request parameters and try again. content: application/json: schema: type: object properties: status: type: number description: HTTP status code. example: 400 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid '500': description: An internal server error has occurred. If this issue persists, please contact ClickHouse Cloud support for assistance. content: application/json: schema: type: object properties: status: type: integer description: HTTP status code. example: 500 error: type: string description: Detailed error description. requestId: type: string description: Unique id assigned to every request. UUIDv4 format: uuid tags: - ClickPipes components: schemas: UpdateReversePrivateEndpoint: properties: customPrivateDnsMappings: type: array description: 'Optional private DNS names for Reverse Private Endpoint. Can be used as data source destination address. Must be unique across the ClickHouse service. Generally available for Google Private Service Connect (PSC). For AWS PrivateLink (VPC endpoint service and VPC resource), available in Private Preview; contact ClickHouse support to enable it for your service. Not supported for MSK multi-VPC. Supports exact names and leading wildcard names such as *.example.com' items: $ref: '#/components/schemas/CustomPrivateDnsMapping' example: - privateDnsName: my-service.example.com - privateDnsName: '*.example.com' MskIamUser: properties: accessKeyId: description: IAM access key ID. type: string secretKey: description: IAM secret key. type: string ClickPipePatchPostgresPipeSettings: properties: syncIntervalSeconds: description: Interval in seconds to sync data from Postgres during CDC replication. type: - integer - 'null' minimum: 1 example: 60 pullBatchSize: description: Number of rows to pull in each batch during CDC replication. type: - integer - 'null' minimum: 1 example: 1000 ClickPipeKafkaSource: properties: type: description: Type of the Kafka source. type: string enum: - kafka - redpanda - msk - gcmk - confluent - warpstream - azureeventhub - dokafka format: description: Format of the Kafka source. type: string enum: - JSONEachRow - Avro - AvroConfluent - Protobuf brokers: description: Brokers of the Kafka source. type: string topics: description: Topics of the Kafka source. type: string consumerGroup: description: Consumer group of the Kafka source. If not provided "clickpipes-<>" will be used. type: - string - 'null' example: my-clickpipe-consumer-group authentication: description: 'Authentication method of the Kafka source. SERVICE_ACCOUNT_WORKLOAD_IDENTITY is in Private Preview. ClickPipes uses the GCP service account returned in gcpWorkloadIdentity.principal by the operation with operationId clickPipesServiceContextGet; grant it access to the source resources. Supported authentication methods: kafka: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512, MUTUAL_TLS, msk: SCRAM-SHA-512, IAM_ROLE, IAM_USER, MUTUAL_TLS, gcmk: PLAIN, MUTUAL_TLS, SERVICE_ACCOUNT_WORKLOAD_IDENTITY, confluent: PLAIN, MUTUAL_TLS, warpstream: PLAIN, azureeventhub: PLAIN, redpanda: SCRAM-SHA-256, SCRAM-SHA-512, MUTUAL_TLS, dokafka: SCRAM-SHA-256, MUTUAL_TLS' type: string enum: - PLAIN - SCRAM-SHA-256 - SCRAM-SHA-512 - IAM_ROLE - IAM_USER - MUTUAL_TLS - SERVICE_ACCOUNT_WORKLOAD_IDENTITY iamRole: description: 'IAM role for the Kafka source. Use with IAM role authentication. Read more in ClickPipes documentation: https://clickhouse.com/docs/en/integrations/clickpipes/kafka#iam' type: - string - 'null' example: arn:aws:iam::123456789012:role/MyRole offset: oneOf: - $ref: '#/components/schemas/ClickPipeKafkaOffset' - type: 'null' schemaRegistry: oneOf: - $ref: '#/components/schemas/ClickPipeKafkaSchemaRegistry' - type: 'null' caCertificate: description: PEM encoded CA certificates to validate the broker's certificate. type: - string - 'null' reversePrivateEndpointIds: type: array description: Reverse private endpoint UUIDs used for a secure private connection to the Kafka source. items: type: string exactlyOnce: description: Enable exactly-once delivery. Guarantees every Kafka record is inserted exactly once across restarts and rebalances. Can only be set at pipe creation. type: - boolean - 'null' ClickPipePostgresPipeSettings: properties: syncIntervalSeconds: description: Interval in seconds to sync data from Postgres during CDC replication. type: integer minimum: 1 example: 60 pullBatchSize: description: Number of rows to pull in each batch during CDC replication. type: integer minimum: 1 example: 1000 publicationName: description: PostgreSQL publication name to use for CDC replication. If not provided, ClickPipes will create one automatically. type: string example: clickpipes_publication replicationMode: description: 'Replication mode: "cdc" (change data capture with initial snapshot), "snapshot" (one-time snapshot only), or "cdc_only" (CDC without initial snapshot).' type: string enum: - cdc - snapshot - cdc_only example: cdc replicationSlotName: description: PostgreSQL replication slot name. Only valid for "cdc_only" mode. For "cdc" mode, ClickPipes creates the slot automatically. type: string example: clickpipes_slot allowNullableColumns: description: 'Preserve nullability from Postgres in the destination ClickHouse table. When true, columns without NOT NULL constraints are created as Nullable(...). When false, all columns are non-nullable and NULL values are replaced with the default value for the type. Note: Nullable types have performance overhead in ClickHouse.' type: boolean example: false initialLoadParallelism: description: Number of parallel workers to use per table in the initial snapshot phase. type: integer minimum: 1 example: 1 snapshotNumRowsPerPartition: description: Number of rows per partition during the snapshot phase. type: integer minimum: 1000 example: 100000 snapshotNumberOfParallelTables: description: Number of tables to snapshot in parallel during the initial load phase. type: integer minimum: 1 example: 1 enableFailoverSlots: description: Enable failover support for the replication slot on PG17 and newer. Only applicable when ClickPipes creates the replication slot (i.e., replicationSlotName is NOT provided). type: boolean example: false deleteOnMerge: description: Enable hard delete behavior in ReplacingMergeTree for PostgreSQL DELETE operations. type: boolean example: false ClickPipePatchRequest: properties: name: description: Name of the ClickPipe. type: - string - 'null' source: oneOf: - $ref: '#/components/schemas/ClickPipePatchSource' - type: 'null' destination: oneOf: - $ref: '#/components/schemas/ClickPipePatchDestination' - type: 'null' fieldMappings: type: array description: Field mappings of the ClickPipe. This will not update the table schema, only the ClickPipe configuration. items: $ref: '#/components/schemas/ClickPipeFieldMapping' settings: oneOf: - $ref: '#/components/schemas/ClickPipeSettings' - type: 'null' ClickPipePostPubSubServiceAccountSource: properties: format: description: Format of messages in the Pub/Sub topic. GCP Pub/Sub ClickPipes are in limited preview — contact support to enable this feature for your organization. type: string enum: - JSONEachRow - Avro - Protobuf example: JSONEachRow projectId: description: GCP project ID that owns the Pub/Sub topic. type: string example: my-gcp-project topic: description: Pub/Sub topic name (not the fully-qualified path). type: string example: my-topic authentication: description: Authenticate with a GCP service account JSON key. type: string enum: - SERVICE_ACCOUNT example: SERVICE_ACCOUNT seekType: description: Starting position strategy for consuming the subscription. The seekTimestamp companion is required only when seekType is "timestamp"; setting it for a mismatched seek type is rejected. type: string enum: - latest - earliest - timestamp example: earliest seekTimestamp: description: RFC 3339 / ISO 8601 timestamp to seek to. Required when seekType is "timestamp"; must be omitted otherwise. type: - string - 'null' format: date-time example: '2026-04-10T12:00:00Z' filter: description: Optional Pub/Sub subscription filter expression (CEL). Maximum 256 characters. type: - string - 'null' maxLength: 256 enableOrdering: description: Whether to enable ordered delivery of messages (requires messages to be published with ordering keys). type: - boolean - 'null' ackDeadline: description: Acknowledgement deadline for messages, in seconds. Must be between 10 and 600. type: - integer - 'null' minimum: 10 maximum: 600 serviceAccountKey: $ref: '#/components/schemas/ServiceAccount' required: - format - projectId - topic - authentication - seekType - serviceAccountKey additionalProperties: false ClickPipePostObjectStorageSource: properties: type: description: Type of the ObjectStorage source. type: string enum: - s3 - gcs - dospaces - azureblobstorage - cloudflarer2 - ovhobjectstorage format: description: Format of the files. type: string enum: - JSONEachRow - JSONAsObject - CSV - CSVWithNames - TabSeparated - TabSeparatedWithNames - Parquet - Avro url: description: 'Provide a path to the file(s) you want to ingest. You can specify multiple files using bash-like wildcards. For more information, see the documentation on using wildcards in path: https://clickhouse.com/docs/en/integrations/clickpipes/object-storage#limitations' type: string example: https://datasets-documentation.s3.eu-west-3.amazonaws.com/http/**.ndjson.gz delimiter: description: Delimiter used in the files. type: - string - 'null' example: ',' compression: description: Compression algorithm used for the files. type: - string - 'null' enum: - none - gzip - gz - brotli - br - xz - LZMA - zstd - auto example: auto isContinuous: description: If set to true, the pipe will continuously read new files from the source. If set to false, the pipe will read the files only once. New files have to be uploaded lexically order. type: - boolean - 'null' queueUrl: description: Queue URL for event-based continuous ingestion. For S3, provide an SQS queue URL. For GCS, provide a Pub/Sub subscription (e.g. projects/{project}/subscriptions/{name}). When provided, files are ingested based on event notifications rather than lexicographical order. Only applicable when isContinuous is true and authentication is not public. type: - string - 'null' example: https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue skipInitialLoad: description: If set to true, skips the initial load and only ingests files delivered by queue notifications. Only applicable when queueUrl is provided. type: - boolean - 'null' startAfter: description: Skip all files up to and including this object key during the initial load. Cannot be provided when skipInitialLoad is true. type: - string - 'null' example: events/2026-06-01/ authentication: description: Authentication method. IAM_USER is for S3, GCS, and DigitalOcean Spaces. IAM_ROLE is for S3 only. SERVICE_ACCOUNT is for GCS only. For GCS, SERVICE_ACCOUNT_WORKLOAD_IDENTITY is in Private Preview. ClickPipes uses the GCP service account returned in gcpWorkloadIdentity.principal by the operation with operationId clickPipesServiceContextGet; grant it access to the source resources. CONNECTION_STRING is for Azure Blob Storage. PUBLIC uses no authentication. type: - string - 'null' enum: - IAM_ROLE - IAM_USER - CONNECTION_STRING - SERVICE_ACCOUNT - SERVICE_ACCOUNT_WORKLOAD_IDENTITY iamRole: description: 'IAM role to be used with IAM role authentication. Read more in ClickPipes documentation: https://clickhouse.com/docs/en/integrations/clickpipes/object-storage#authentication' type: - string - 'null' example: arn:aws:iam::123456789012:role/MyRole connectionString: description: Connection string for Azure Blob Storage authentication. Required when authentication is CONNECTION_STRING. type: - string - 'null' example: DefaultEndpointsProtocol=https;AccountName=myaccount;AccountKey=mykey;EndpointSuffix=core.windows.net path: description: 'Path to the file(s) within the Azure container. Used for Azure Blob Storage sources. You can specify multiple files using bash-like wildcards. For more information, see the documentation on using wildcards in path: https://clickhouse.com/docs/en/integrations/clickpipes/object-storage#limitations' type: - string - 'null' example: data/logs/*.json azureContainerName: description: Container name for Azure Blob Storage. Required when type is azureblobstorage. type: - string - 'null' example: mycontainer accessKey: oneOf: - $ref: '#/components/schemas/MskIamUser' - type: 'null' serviceAccountKey: description: Base64-encoded GCP service account JSON key. Required when authentication is SERVICE_ACCOUNT. type: - string - 'null' ClickPipePatchDestination: properties: columns: type: array description: Columns of the destination table. This will not update the table schema, only the ClickPipe configuration. items: $ref: '#/components/schemas/ClickPipeDestinationColumn' ClickPipeMutateMySQLSource: properties: type: description: Type of the MySQL source. Defaults to "mysql" if not specified. type: - string - 'null' enum: - mysql - rdsmysql - auroramysql - mariadb - rdsmariadb credentials: $ref: '#/components/schemas/PLAIN' host: description: MySQL server hostname or IP address. To use a reverse private endpoint, pass the endpoint hostname here. type: string format: hostname example: my-mysql-server.example.com port: description: MySQL server port. type: integer minimum: 1 maximum: 65535 example: 3306 settings: $ref: '#/components/schemas/ClickPipeMySQLPipeSettings' authentication: description: Authentication method for MySQL connection. type: string enum: - basic - IAM_ROLE example: basic iamRole: description: IAM role ARN for IAM authentication (required for IAM_ROLE authentication). type: string example: arn:aws:iam::123456789012:role/MyApplicationRole tlsHost: description: TLS/SSL host for secure connections. type: string example: my-mysql-server.example.com caCertificate: description: PEM encoded CA certificate to validate the MySQL server certificate. type: string example: '-----BEGIN CERTIFICATE----- ...' disableTls: description: Disable TLS for the MySQL connection. Use with caution in production environments. Defaults to false when omitted. type: boolean example: false default: false skipCertVerification: description: Skip TLS certificate verification for the MySQL connection. Use with caution in production environments. type: boolean example: false serverId: description: Optional MySQL server_id the pipe declares itself as in the MySQL replication topology. Must be unique across replicas connected to the source. If omitted, one is assigned automatically. type: integer minimum: 1 maximum: 4294967295 example: 4242 tableMappings: type: array description: List of table mappings defining which MySQL tables to replicate and how they map to ClickHouse tables. items: $ref: '#/components/schemas/ClickPipeMySQLPipeTableMapping' required: - host - port - settings - tableMappings ClickPipeDestinationColumn: properties: name: description: Name of the column. type: string type: description: Type of the column. type: string ClickPipePatchPostgresPipeRemoveTableMapping: properties: sourceSchemaName: description: PostgreSQL source schema name. type: - string - 'null' example: public sourceTable: description: PostgreSQL source table name. type: - string - 'null' example: users targetTable: description: ClickHouse target table name, optionally prefixed with schema name (e.g., "my_schema_my_table"). The table will be created automatically if it does not exist. For snapshot mode, the target table must be empty. type: - string - 'null' example: public_users tableEngine: description: 'ClickHouse table engine: "ReplacingMergeTree" (handles updates/deletes), "MergeTree" (append-only), or "Null" (forward data to materialized views without storing it).' type: - string - 'null' enum: - MergeTree - ReplacingMergeTree - 'Null' example: ReplacingMergeTree partitionKey: description: 'Custom partitioning column used for parallel snapshotting. Only beneficial for PostgreSQL 13 (no benefit for PG14+, which supports indexed ctid scans). Must be an indexed column of type: `smallint`, `integer`, `bigint`, `timestamp without time zone`, or `timestamp with time zone`. Unrelated to ClickHouse partitioning.' type: - string - 'null' example: id partitionByExpr: description: ClickHouse PARTITION BY expression applied to the destination table when ClickPipes creates it. type: - string - 'null' example: toYYYYMM(created_at) ClickPipeScaling: properties: replicas: description: Desired number of replicas. Only for scalable pipes. type: integer minimum: 1 maximum: 40 concurrency: description: Desired number of concurrency. Only for S3 pipes. If set to 0, concurrency is auto-scaled based on the cluster memory. type: integer deprecated: true replicaCpuMillicores: description: CPU in millicores for each replica. Only for streaming pipes. type: integer minimum: 125 maximum: 2000 replicaMemoryGb: description: Memory in GB for each replica. Only for streaming pipes. type: number minimum: 0.5 maximum: 8 ClickPipeSettings: properties: streaming_max_insert_wait_ms: description: Streaming max insert wait time. Configures the max wait period before inserting data into the ClickHouse. type: - integer - 'null' minimum: 500 maximum: 60000 example: 5000 object_storage_concurrency: description: Object storage concurrency. Number of concurrent file processing threads type: - integer - 'null' minimum: 1 maximum: 35 example: 1 object_storage_polling_interval_ms: description: Object storage polling interval. Configures the refresh interval for querying continuous ingest for new object storage data type: - integer - 'null' minimum: 100 maximum: 3600000 example: 30000 object_storage_max_insert_bytes: description: Max insert bytes. Number of bytes to process in a single insert batch type: - integer - 'null' minimum: 10485760 maximum: 53687091200 example: 10737418240 object_storage_max_file_count: description: Max file count. Maximum number of files to process in a single insert batch type: - integer - 'null' minimum: 1 maximum: 10000 example: 100 clickhouse_max_threads: description: Max threads. Maximum number of concurrent threads for file processing type: - integer - 'null' minimum: 0 maximum: 64 example: 8 clickhouse_max_insert_threads: description: Max insert threads. Maximum number of concurrent insert threads type: - integer - 'null' minimum: 0 maximum: 16 example: 1 clickhouse_min_insert_block_size_bytes: description: Min insert block size bytes. Minimum size of data block for insert (in bytes) type: - integer - 'null' minimum: 0 maximum: 10737418240 example: 1073741824 clickhouse_max_download_threads: description: Max download threads. Maximum number of concurrent download threads type: - integer - 'null' minimum: 0 maximum: 32 example: 4 clickhouse_parallel_distributed_insert_select: description: Parallel distributed insert select. Parallel distributed insert select setting type: - integer - 'null' minimum: 0 maximum: 2 example: 2 kafka_read_committed: description: Kafka Read Committed. Whether Kafka consumers read only committed messages type: boolean example: false object_storage_use_cluster_function: description: use cluster function. Whether to use ClickHouse cluster function for distributed processing type: - boolean - 'null' example: true clickhouse_parallel_view_processing: description: parallel view processing. Whether to enable pushing to attached views concurrently instead of sequentially type: - boolean - 'null' example: false MutualTLS: properties: certificate: description: PEM encoded client certificate for mTLS authentication. type: string privateKey: description: PEM encoded client private key for mTLS authentication. type: string format: password ClickPipeFieldMapping: properties: sourceField: description: Source field name. type: string destinationField: description: Destination field name. type: string ClickPipePostPubSubSource: oneOf: - $ref: '#/components/schemas/ClickPipePostPubSubServiceAccountSource' - $ref: '#/components/schemas/ClickPipePostPubSubWorkloadIdentitySource' ClickPipeBigQuerySource: oneOf: - $ref: '#/components/schemas/ClickPipeBigQueryServiceAccountSource' - $ref: '#/components/schemas/ClickPipeBigQueryWorkloadIdentitySource' ClickPipeSchemaDiscoveryMeta: type: object description: Source-specific schema discovery metadata. additionalProperties: type: string ClickPipePostPubSubWorkloadIdentitySource: properties: format: description: Format of messages in the Pub/Sub topic. GCP Pub/Sub ClickPipes are in limited preview — contact support to enable this feature for your organization. type: string enum: - JSONEachRow - Avro - Protobuf example: JSONEachRow projectId: description: GCP project ID that owns the Pub/Sub topic. type: string example: my-gcp-project topic: description: Pub/Sub topic name (not the fully-qualified path). type: string example: my-topic authentication: description: SERVICE_ACCOUNT_WORKLOAD_IDENTITY is in Private Preview. ClickPipes uses the GCP service account returned in gcpWorkloadIdentity.principal by the operation with operationId clickPipesServiceContextGet; grant it access to the source resources. type: string enum: - SERVICE_ACCOUNT_WORKLOAD_IDENTITY example: SERVICE_ACCOUNT_WORKLOAD_IDENTITY seekType: description: Starting position strategy for consuming the subscription. The seekTimestamp companion is required only when seekType is "timestamp"; setting it for a mismatched seek type is rejected. type: string enum: - latest - earliest - timestamp example: earliest seekTimestamp: description: RFC 3339 / ISO 8601 timestamp to seek to. Required when seekType is "timestamp"; must be omitted otherwise. type: - string - 'null' format: date-time example: '2026-04-10T12:00:00Z' filter: description: Optional Pub/Sub subscription filter expression (CEL). Maximum 256 characters. type: - string - 'null' maxLength: 256 enableOrdering: description: Whether to enable ordered delivery of messages (requires messages to be published with ordering keys). type: - boolean - 'null' ackDeadline: description: Acknowledgement deadline for messages, in seconds. Must be between 10 and 600. type: - integer - 'null' minimum: 10 maximum: 600 required: - format - projectId - topic - authentication - seekType additionalProperties: false ClickPipePatchPubSubSource: properties: authentication: description: Authentication method to use with GCP Pub/Sub. SERVICE_ACCOUNT_WORKLOAD_IDENTITY is in Private Preview. ClickPipes uses the GCP service account returned in gcpWorkloadIdentity.principal by the operation with operationId clickPipesServiceContextGet; grant it access to the source resources. type: - string - 'null' enum: - SERVICE_ACCOUNT - SERVICE_ACCOUNT_WORKLOAD_IDENTITY example: SERVICE_ACCOUNT ackDeadline: description: Acknowledgement deadline for messages, in seconds. Must be between 10 and 600. type: - integer - 'null' minimum: 10 maximum: 600 serviceAccountKey: oneOf: - $ref: '#/components/schemas/ServiceAccount' - type: 'null' required: - authentication ClickPipesCdcScalingPatchRequest: properties: replicaCpuMillicores: description: CPU in millicores for DB ClickPipes. type: integer minimum: 1000 maximum: 32000 multipleOf: 1000 example: 2000 replicaMemoryGb: description: Memory in GiB for DB ClickPipes. Must be 4× the CPU core count. type: number minimum: 4 maximum: 128 multipleOf: 4 example: 8 ClickPipeKafkaSchemaRegistry: properties: url: description: Schema URL. HTTPS required. type: string example: https://psrc-aa00.us-east-2.aws.confluent.cloud/schemas/ids/100004 authentication: description: Authentication type of the schema registry. type: string enum: - PLAIN caCertificate: description: PEM encoded CA certificates to validate the schema registry's certificate. type: - string - 'null' ClickPipePostSource: properties: kafka: oneOf: - $ref: '#/components/schemas/ClickPipePostKafkaSource' - type: 'null' objectStorage: oneOf: - $ref: '#/components/schemas/ClickPipePostObjectStorageSource' - type: 'null' kinesis: oneOf: - $ref: '#/components/schemas/ClickPipePostKinesisSource' - type: 'null' pubsub: oneOf: - $ref: '#/components/schemas/ClickPipePostPubSubSource' - type: 'null' postgres: $ref: '#/components/schemas/ClickPipeMutatePostgresSource' mysql: oneOf: - $ref: '#/components/schemas/ClickPipeMutateMySQLSource' - type: 'null' bigquery: oneOf: - $ref: '#/components/schemas/ClickPipeMutateBigQuerySource' - type: 'null' mongodb: oneOf: - $ref: '#/components/schemas/ClickPipeMutateMongoDBSource' - type: 'null' validateSamples: description: Validate data samples received from data source. It will validate the connection and data availability and correctness. If not enabled, only connection will be validated. This has no effect on Postgres or MySQL pipes, they always only validate the connection and table definitions. This is experimental and can be removed in the future. type: boolean CreateReversePrivateEndpoint: properties: description: description: Reverse private endpoint description. Maximum length is 255 characters. type: string example: My reverse private endpoint type: description: Reverse private endpoint type. type: string enum: - VPC_ENDPOINT_SERVICE - VPC_RESOURCE - MSK_MULTI_VPC - GCP_PSC_SERVICE_ATTACHMENT example: VPC_ENDPOINT_SERVICE vpcEndpointServiceName: description: VPC endpoint service name. type: - string - 'null' example: com.amazonaws.vpce.us-east-1.vpce-svc-12345678901234567 vpcResourceConfigurationId: description: VPC resource configuration ID. Required for VPC_RESOURCE type. type: - string - 'null' example: rcfg-12345678901234567 vpcResourceShareArn: description: VPC resource share ARN. Required for VPC_RESOURCE type. type: - string - 'null' example: arn:aws:ram:us-east-1:123456789012:resource-share/share-12345678901234567 mskClusterArn: description: MSK cluster ARN. Required for MSK_MULTI_VPC type. type: - string - 'null' example: arn:aws:kafka:us-east-1:123456789012:cluster/my-cluster mskAuthentication: description: MSK cluster authentication type. Required for MSK_MULTI_VPC type. type: - string - 'null' enum: - SASL_IAM - SASL_SCRAM example: SASL_IAM gcpServiceAttachment: description: 'Private Preview. GCP PSC service attachment URI. Required for GCP_PSC_SERVICE_ATTACHMENT type. Format: projects/{project}/regions/{region}/serviceAttachments/{name}.' type: - string - 'null' example: projects/my-project/regions/us-central1/serviceAttachments/my-service customPrivateDnsMappings: type: array description: 'Optional private DNS names for Reverse Private Endpoint. Can be used as data source destination address. Must be unique across the ClickHouse service. Generally available for Google Private Service Connect (PSC). For AWS PrivateLink (VPC endpoint service and VPC resource), available in Private Preview; contact ClickHouse support to enable it for your service. Not supported for MSK multi-VPC. Supports exact names and leading wildcard names such as *.example.com' items: $ref: '#/components/schemas/CustomPrivateDnsMapping' example: - privateDnsName: my-service.example.com - privateDnsName: '*.example.com' ClickPipePatchMongoDBPipeRemoveTableMapping: properties: sourceDatabaseName: description: MongoDB source database name. type: - string - 'null' example: mydb sourceCollection: description: MongoDB source collection name. type: - string - 'null' example: users targetTable: description: ClickHouse target table name. The table will be created automatically if it does not exist. type: - string - 'null' example: mydb_users tableEngine: description: 'ClickHouse table engine: "ReplacingMergeTree" (handles updates/deletes), "MergeTree" (append-only), or "Null" (forward data to materialized views without storing it).' type: - string - 'null' enum: - MergeTree - ReplacingMergeTree - 'Null' example: ReplacingMergeTree required: - sourceDatabaseName - sourceCollection - targetTable ClickPipeSchemaDiscoveryResponse: properties: fields: type: array description: Inferred schema fields with their ClickHouse data types. items: $ref: '#/components/schemas/ClickPipeSchemaDiscoveryField' meta: oneOf: - $ref: '#/components/schemas/ClickPipeSchemaDiscoveryMeta' - type: 'null' ClickPipePatchMySQLPipeRemoveTableMapping: properties: sourceSchemaName: description: MySQL source database name. type: - string - 'null' example: my_database sourceTable: description: MySQL source table name. type: - string - 'null' example: users targetTable: description: ClickHouse target table name, optionally prefixed with schema name (e.g., "my_database_my_table"). The table will be created automatically if it does not exist. For snapshot mode, the target table must be empty. type: - string - 'null' example: my_database_users tableEngine: description: 'ClickHouse table engine: "ReplacingMergeTree" (handles updates/deletes), "MergeTree" (append-only), or "Null" (forward data to materialized views without storing it).' type: - string - 'null' enum: - MergeTree - ReplacingMergeTree - 'Null' example: ReplacingMergeTree partitionKey: description: Custom partitioning column used for parallel snapshotting. Must be an indexed column of an integer, date, datetime or timestamp type. Unrelated to ClickHouse partitioning. type: - string - 'null' example: id partitionByExpr: description: ClickHouse PARTITION BY expression applied to the destination table when ClickPipes creates it. type: - string - 'null' example: toYYYYMM(created_at) required: - sourceSchemaName - sourceTable - targetTable ClickPipeMutateKafkaSchemaRegistry: properties: url: description: Schema URL. HTTPS required. type: string example: https://psrc-aa00.us-east-2.aws.confluent.cloud/schemas/ids/100004 authentication: description: Authentication type of the schema registry. type: string enum: - PLAIN caCertificate: description: PEM encoded CA certificates to validate the schema registry's certificate. type: - string - 'null' credentials: $ref: '#/components/schemas/ClickPipeKafkaSchemaRegistryCredentials' ReversePrivateEndpoint: properties: description: description: Reverse private endpoint description. Maximum length is 255 characters. type: string example: My reverse private endpoint type: description: Reverse private endpoint type. type: string enum: - VPC_ENDPOINT_SERVICE - VPC_RESOURCE - MSK_MULTI_VPC - GCP_PSC_SERVICE_ATTACHMENT example: VPC_ENDPOINT_SERVICE vpcEndpointServiceName: description: VPC endpoint service name. type: - string - 'null' example: com.amazonaws.vpce.us-east-1.vpce-svc-12345678901234567 vpcResourceConfigurationId: description: VPC resource configuration ID. Required for VPC_RESOURCE type. type: - string - 'null' example: rcfg-12345678901234567 vpcResourceShareArn: description: VPC resource share ARN. Required for VPC_RESOURCE type. type: - string - 'null' example: arn:aws:ram:us-east-1:123456789012:resource-share/share-12345678901234567 mskClusterArn: description: MSK cluster ARN. Required for MSK_MULTI_VPC type. type: - string - 'null' example: arn:aws:kafka:us-east-1:123456789012:cluster/my-cluster mskAuthentication: description: MSK cluster authentication type. Required for MSK_MULTI_VPC type. type: - string - 'null' enum: - SASL_IAM - SASL_SCRAM example: SASL_IAM gcpServiceAttachment: description: 'Private Preview. GCP PSC service attachment URI. Required for GCP_PSC_SERVICE_ATTACHMENT type. Format: projects/{project}/regions/{region}/serviceAttachments/{name}.' type: - string - 'null' example: projects/my-project/regions/us-central1/serviceAttachments/my-service customPrivateDnsMappings: type: array description: 'Optional private DNS names for Reverse Private Endpoint. Can be used as data source destination address. Must be unique across the ClickHouse service. Generally available for Google Private Service Connect (PSC). For AWS PrivateLink (VPC endpoint service and VPC resource), available in Private Preview; contact ClickHouse support to enable it for your service. Not supported for MSK multi-VPC. Supports exact names and leading wildcard names such as *.example.com' items: $ref: '#/components/schemas/CustomPrivateDnsMapping' example: - privateDnsName: my-service.example.com - privateDnsName: '*.example.com' id: description: Reverse private endpoint ID. type: string format: uuid example: 12345678-1234-1234-1234-123456789012 serviceId: description: ClickHouse service ID reverse private endpoint is associated with. type: string format: uuid example: 12345678-1234-1234-1234-123456789012 endpointId: description: Reverse private endpoint endpoint ID. type: string example: vpce-12345678901234567 dnsNames: type: array description: Reverse private endpoint internal DNS names. items: type: string example: - vpce-12345678901234567-abcdefg.execute-api.us-east-1.vpce.amazonaws.com privateDnsNames: type: array description: Reverse private endpoint private DNS names. items: type: string example: - vpce-12345678901234567-abcdefg.execute-api.us-east-1.vpce.amazonaws.com status: description: Reverse private endpoint status. type: string enum: - Unknown - Provisioning - Deleting - Ready - Failed - PendingAcceptance - Rejected - Expired example: Ready ClickPipeScalingPatchRequest: properties: replicas: description: Number of replicas to scale to. Use to scale Kafka pipes. type: - integer - 'null' minimum: 1 maximum: 40 concurrency: description: Number of concurrency to scale to. Use to scale S3 pipes. type: - integer - 'null' minimum: 0 maximum: 34 deprecated: true replicaCpuMillicores: description: CPU in millicores for each replica. Use to scale streaming pipes. type: - integer - 'null' minimum: 125 maximum: 2000 replicaMemoryGb: description: Memory in GB for each replica. Use to scale streaming pipes. type: - number - 'null' minimum: 0.5 maximum: 8 ClickPipeMutateDestination: properties: database: description: Destination database. type: string table: description: Destination table. Required field for all pipe types except database pipes (Postgres, MySQL, BigQuery). type: string managedTable: description: Is the table managed by ClickPipes? Required field for all pipe types except database pipes (Postgres, MySQL, BigQuery). type: boolean tableDefinition: $ref: '#/components/schemas/ClickPipeDestinationTableDefinition' columns: type: array description: Columns of the destination table. Required field for all pipe types except database pipes (Postgres, MySQL, BigQuery). items: $ref: '#/components/schemas/ClickPipeDestinationColumn' roles: type: array description: Optional. Roles to grant to the ClickHouse user that ClickPipe creates. If omitted, the user is granted the default role (`default_role`). Add your custom roles here if required. The role names `clickpipes` and `clickpipes_system` are reserved for internal use and cannot be assigned. items: type: string ClickPipeSchemaDiscoverySource: properties: kafka: oneOf: - $ref: '#/components/schemas/ClickPipePostKafkaSource' - type: 'null' kinesis: oneOf: - $ref: '#/components/schemas/ClickPipePostKinesisSource' - type: 'null' pubsub: oneOf: - $ref: '#/components/schemas/ClickPipePostPubSubSource' - type: 'null' objectStorage: oneOf: - $ref: '#/components/schemas/ClickPipePostObjectStorageSource' - type: 'null' ClickPipeDestination: properties: database: description: Destination database. type: string table: description: Destination table. Required field for all pipe types except database pipes (Postgres, MySQL, BigQuery). type: string managedTable: description: Is the table managed by ClickPipes? Required field for all pipe types except database pipes (Postgres, MySQL, BigQuery). type: boolean tableDefinition: $ref: '#/components/schemas/ClickPipeDestinationTableDefinition' columns: type: array description: Columns of the destination table. Required field for all pipe types except database pipes (Postgres, MySQL, BigQuery). items: $ref: '#/components/schemas/ClickPipeDestinationColumn' ClickPipeSettingsPutRequest: properties: streaming_max_insert_wait_ms: description: Streaming max insert wait time. Configures the max wait period before inserting data into the ClickHouse. type: - integer - 'null' minimum: 500 maximum: 60000 example: 5000 object_storage_concurrency: description: Object storage concurrency. Number of concurrent file processing threads type: - integer - 'null' minimum: 1 maximum: 35 example: 1 object_storage_polling_interval_ms: description: Object storage polling interval. Configures the refresh interval for querying continuous ingest for new object storage data type: - integer - 'null' minimum: 100 maximum: 3600000 example: 30000 object_storage_max_insert_bytes: description: Max insert bytes. Number of bytes to process in a single insert batch type: - integer - 'null' minimum: 10485760 maximum: 53687091200 example: 10737418240 object_storage_max_file_count: description: Max file count. Maximum number of files to process in a single insert batch type: - integer - 'null' minimum: 1 maximum: 10000 example: 100 clickhouse_max_threads: description: Max threads. Maximum number of concurrent threads for file processing type: - integer - 'null' minimum: 0 maximum: 64 example: 8 clickhouse_max_insert_threads: description: Max insert threads. Maximum number of concurrent insert threads type: - integer - 'null' minimum: 0 maximum: 16 example: 1 clickhouse_min_insert_block_size_bytes: description: Min insert block size bytes. Minimum size of data block for insert (in bytes) type: - integer - 'null' minimum: 0 maximum: 10737418240 example: 1073741824 clickhouse_max_download_threads: description: Max download threads. Maximum number of concurrent download threads type: - integer - 'null' minimum: 0 maximum: 32 example: 4 clickhouse_parallel_distributed_insert_select: description: Parallel distributed insert select. Parallel distributed insert select setting type: - integer - 'null' minimum: 0 maximum: 2 example: 2 kafka_read_committed: description: Kafka Read Committed. Whether Kafka consumers read only committed messages type: boolean example: false object_storage_use_cluster_function: description: use cluster function. Whether to use ClickHouse cluster function for distributed processing type: - boolean - 'null' example: true clickhouse_parallel_view_processing: description: parallel view processing. Whether to enable pushing to attached views concurrently instead of sequentially type: - boolean - 'null' example: false ClickPipeObjectStorageSource: properties: type: description: Type of the ObjectStorage source. type: string enum: - s3 - gcs - dospaces - azureblobstorage - cloudflarer2 - ovhobjectstorage format: description: Format of the files. type: string enum: - JSONEachRow - JSONAsObject - CSV - CSVWithNames - TabSeparated - TabSeparatedWithNames - Parquet - Avro url: description: 'Provide a path to the file(s) you want to ingest. You can specify multiple files using bash-like wildcards. For more information, see the documentation on using wildcards in path: https://clickhouse.com/docs/en/integrations/clickpipes/object-storage#limitations' type: string example: https://datasets-documentation.s3.eu-west-3.amazonaws.com/http/**.ndjson.gz delimiter: description: Delimiter used in the files. type: - string - 'null' example: ',' compression: description: Compression algorithm used for the files. type: - string - 'null' enum: - none - gzip - gz - brotli - br - xz - LZMA - zstd - auto example: auto isContinuous: description: If set to true, the pipe will continuously read new files from the source. If set to false, the pipe will read the files only once. New files have to be uploaded lexically order. type: - boolean - 'null' queueUrl: description: Queue URL for event-based continuous ingestion. For S3, provide an SQS queue URL. For GCS, provide a Pub/Sub subscription (e.g. projects/{project}/subscriptions/{name}). When provided, files are ingested based on event notifications rather than lexicographical order. Only applicable when isContinuous is true and authentication is not public. type: - string - 'null' example: https://sqs.us-east-1.amazonaws.com/123456789012/MyQueue skipInitialLoad: description: If set to true, skips the initial load and only ingests files delivered by queue notifications. Only applicable when queueUrl is provided. type: - boolean - 'null' startAfter: description: Skip all files up to and including this object key during the initial load. Cannot be provided when skipInitialLoad is true. type: - string - 'null' example: events/2026-06-01/ authentication: description: Authentication method. IAM_USER is for S3, GCS, and DigitalOcean Spaces. IAM_ROLE is for S3 only. SERVICE_ACCOUNT is for GCS only. For GCS, SERVICE_ACCOUNT_WORKLOAD_IDENTITY is in Private Preview. ClickPipes uses the GCP service account returned in gcpWorkloadIdentity.principal by the operation with operationId clickPipesServiceContextGet; grant it access to the source resources. CONNECTION_STRING is for Azure Blob Storage. PUBLIC uses no authentication. type: - string - 'null' enum: - IAM_ROLE - IAM_USER - CONNECTION_STRING - SERVICE_ACCOUNT - SERVICE_ACCOUNT_WORKLOAD_IDENTITY iamRole: description: 'IAM role to be used with IAM role authentication. Read more in ClickPipes documentation: https://clickhouse.com/docs/en/integrations/clickpipes/object-storage#authentication' type: - string - 'null' example: arn:aws:iam::123456789012:role/MyRole connectionString: description: Connection string for Azure Blob Storage authentication. Required when authentication is CONNECTION_STRING. type: - string - 'null' example: DefaultEndpointsProtocol=https;AccountName=myaccount;AccountKey=mykey;EndpointSuffix=core.windows.net path: description: 'Path to the file(s) within the Azure container. Used for Azure Blob Storage sources. You can specify multiple files using bash-like wildcards. For more information, see the documentation on using wildcards in path: https://clickhouse.com/docs/en/integrations/clickpipes/object-storage#limitations' type: - string - 'null' example: data/logs/*.json azureContainerName: description: Container name for Azure Blob Storage. Required when type is azureblobstorage. type: - string - 'null' example: mycontainer ClickPipeBigQueryPipeSettings: properties: replicationMode: description: Replication mode. BigQuery only supports snapshot mode. type: string enum: - snapshot allowNullableColumns: description: Allow nullable columns in the destination table. type: boolean initialLoadParallelism: description: Number of parallel workers during initial load. type: number snapshotNumRowsPerPartition: description: Number of rows to snapshot per partition. type: number snapshotNumberOfParallelTables: description: Number of parallel tables to snapshot. type: number required: - replicationMode ClickPipeDestinationTableEngine: properties: type: description: Engine type of the destination table. type: string enum: - MergeTree - ReplacingMergeTree - SummingMergeTree - 'Null' versionColumnId: description: Column name to use as version for ReplacingMergeTree engine. type: - string - 'null' columnIds: type: array description: Column names to sum for SummingMergeTree engine. items: type: string ClickPipeMySQLSource: properties: type: description: Type of the MySQL source. Defaults to "mysql" if not specified. type: - string - 'null' enum: - mysql - rdsmysql - auroramysql - mariadb - rdsmariadb host: description: MySQL server hostname or IP address. To use a reverse private endpoint, pass the endpoint hostname here. type: string format: hostname example: my-mysql-server.example.com port: description: MySQL server port. type: integer minimum: 1 maximum: 65535 example: 3306 authentication: description: Authentication method for MySQL connection. type: string enum: - basic - IAM_ROLE example: basic iamRole: description: IAM role ARN for IAM authentication (required for IAM_ROLE authentication). type: string example: arn:aws:iam::123456789012:role/MyApplicationRole tlsHost: description: TLS/SSL host for secure connections. type: string example: my-mysql-server.example.com caCertificate: description: PEM encoded CA certificate to validate the MySQL server certificate. type: string example: '-----BEGIN CERTIFICATE----- ...' disableTls: description: Disable TLS for the MySQL connection. Use with caution in production environments. type: boolean example: false skipCertVerification: description: Skip TLS certificate verification for the MySQL connection. Use with caution in production environments. type: boolean example: false serverId: description: Optional MySQL server_id the pipe declares itself as in the MySQL replication topology. Must be unique across replicas connected to the source. If omitted, one is assigned automatically. type: integer minimum: 1 maximum: 4294967295 example: 4242 settings: $ref: '#/components/schemas/ClickPipeMySQLPipeSettings' tableMappings: type: array description: List of table mappings defining which MySQL tables to replicate and how they map to ClickHouse tables. items: $ref: '#/components/schemas/ClickPipeMySQLPipeTableMapping' required: - host - port - settings - tableMappings ServiceAccount: properties: serviceAccountFile: description: Google Cloud service account JSON key file content, base64 encoded. type: string required: - serviceAccountFile AzureEventHub: properties: connectionString: description: Connection string for Azure EventHub source. type: string ClickPipePostRequest: properties: name: description: Name of the ClickPipe. type: string source: $ref: '#/components/schemas/ClickPipePostSource' destination: $ref: '#/components/schemas/ClickPipeMutateDestination' fieldMappings: type: array description: Field mappings of the ClickPipe. Note that all destination columns must be included in the mappings. items: $ref: '#/components/schemas/ClickPipeFieldMapping' scaling: $ref: '#/components/schemas/ClickPipeScaling' settings: $ref: '#/components/schemas/ClickPipeSettings' ClickPipePostBigQueryServiceAccountSource: properties: snapshotStagingPath: description: GCS bucket path for staging snapshot data (e.g., gs://my-bucket/staging/). Data will be automatically cleaned up after initial load. type: string settings: $ref: '#/components/schemas/ClickPipeBigQueryPipeSettings' tableMappings: type: array description: Table mappings for BigQuery pipe. items: $ref: '#/components/schemas/ClickPipeBigQueryPipeTableMapping' authentication: description: Authenticate with a Google Cloud service account JSON key. Defaults to SERVICE_ACCOUNT when omitted. type: string enum: - SERVICE_ACCOUNT projectId: description: GCP project ID that owns the BigQuery resources. type: string example: my-gcp-project credentials: $ref: '#/components/schemas/ServiceAccount' required: - snapshotStagingPath - settings - tableMappings - credentials additionalProperties: false ClickPipeStatePatchRequest: properties: command: description: 'Command to change the state: ''start'', ''stop'', ''resync''.' type: string enum: - start - stop - resync ClickPipePostKinesisSource: properties: format: description: Format of the Kinesis stream. type: string enum: - JSONEachRow - Avro - AvroConfluent streamName: description: Name of the Kinesis stream. type: string example: my-stream region: description: AWS region of the Kinesis stream. type: string example: us-east-1 useEnhancedFanOut: description: Use enhanced fan-out for the Kinesis stream. type: - boolean - 'null' iteratorType: description: Type of iterator to use when reading from the Kinesis stream. If AT_TIMESTAMP is used, the timestamp field must be provided. type: string enum: - TRIM_HORIZON - LATEST - AT_TIMESTAMP timestamp: description: UNIX timestamp to start reading from the Kinesis stream. Required if iteratorType is AT_TIMESTAMP. type: - integer - 'null' example: 1615766400 authentication: description: Authentication method to use with the Kinesis stream. type: string enum: - IAM_ROLE - IAM_USER iamRole: description: IAM role to use for authentication. Required if IAM_ROLE is used. type: - string - 'null' example: arn:aws:iam::123456789012:role/MyRole accessKey: oneOf: - $ref: '#/components/schemas/MskIamUser' - type: 'null' ClickPipeMongoDBSource: properties: uri: description: MongoDB connection URI. Supports both standard URIs (mongodb://...) and SRV URIs (mongodb+srv://...). Embedded credentials are redacted from API responses, so the returned value can differ from what was submitted. type: string example: mongodb+srv://cluster0.example.mongodb.net/mydb readPreference: description: MongoDB read preference for replica set reads. type: string enum: - primary - primaryPreferred - secondary - secondaryPreferred - nearest example: secondaryPreferred tlsHost: description: TLS/SSL host for secure connections. type: string example: cluster0.example.mongodb.net disableTls: description: Disable TLS for the MongoDB connection. Defaults to false (TLS enabled). type: boolean example: false skipCertVerification: description: Skip TLS certificate verification for the MongoDB connection. Use with caution in production environments. type: boolean example: false caCertificate: description: PEM encoded CA certificate to validate the MongoDB server certificate. type: string example: '-----BEGIN CERTIFICATE----- ...' settings: $ref: '#/components/schemas/ClickPipeMongoDBPipeSettings' tableMappings: type: array description: List of collection mappings defining which MongoDB collections to replicate and how they map to ClickHouse tables. items: $ref: '#/components/schemas/ClickPipeMongoDBPipeTableMapping' required: - uri - readPreference ClickPipeKafkaOffset: properties: strategy: description: Offset strategy. type: string enum: - from_beginning - from_latest - from_timestamp timestamp: description: A minute precision UTC timestamp to start from. Required for "from_timestamp" strategy. type: - string - 'null' example: 2021-01-01T00:00 ClickPipeMySQLPipeTableMapping: properties: sourceSchemaName: description: MySQL source database name. type: string example: my_database sourceTable: description: MySQL source table name. type: string example: users targetTable: description: ClickHouse target table name, optionally prefixed with schema name (e.g., "my_database_my_table"). The table will be created automatically if it does not exist. For snapshot mode, the target table must be empty. type: string example: my_database_users excludedColumns: type: array description: List of column names to exclude from replication. Column names must be unique within this list. items: type: string example: - internal_id - temp_data uniqueItems: true useCustomSortingKey: description: Whether to use a custom sorting key. If true, sortingKeys must be provided. If false or omitted, the default sorting key is the MySQL primary key. type: boolean example: false sortingKeys: type: array description: Ordered list of column names to use as the sorting (ORDER BY) key in ClickHouse. Only used when useCustomSortingKey is true. Column names must be unique within this list. items: type: string example: - created_at_date - event_id uniqueItems: true tableEngine: description: 'ClickHouse table engine: "ReplacingMergeTree" (handles updates/deletes), "MergeTree" (append-only), or "Null" (forward data to materialized views without storing it).' type: string enum: - MergeTree - ReplacingMergeTree - 'Null' example: ReplacingMergeTree partitionKey: description: Custom partitioning column used for parallel snapshotting. Must be an indexed column of an integer, date, datetime or timestamp type. Unrelated to ClickHouse partitioning. type: string example: id partitionByExpr: description: ClickHouse PARTITION BY expression applied to the destination table when ClickPipes creates it. type: string example: toYYYYMM(created_at) required: - sourceSchemaName - sourceTable - targetTable ClickPipePatchMySQLSource: properties: credentials: $ref: '#/components/schemas/PLAIN' host: description: MySQL server hostname or IP address. To use a reverse private endpoint, pass the endpoint hostname here. type: - string - 'null' format: hostname example: my-mysql-server.example.com port: description: MySQL server port. type: - integer - 'null' minimum: 1 maximum: 65535 example: 3306 authentication: description: Authentication method for MySQL connection. type: - string - 'null' enum: - basic - IAM_ROLE example: basic iamRole: description: IAM role ARN for IAM authentication (required for IAM_ROLE authentication). type: - string - 'null' example: arn:aws:iam::123456789012:role/MyApplicationRole tlsHost: description: TLS/SSL host for secure connections. type: - string - 'null' example: my-mysql-server.example.com caCertificate: description: PEM encoded CA certificate to validate the MySQL server certificate. type: - string - 'null' example: '-----BEGIN CERTIFICATE----- ...' disableTls: description: Disable TLS for the MySQL connection. Use with caution in production environments. type: - boolean - 'null' example: false skipCertVerification: description: Skip TLS certificate verification for the MySQL connection. Use with caution in production environments. type: - boolean - 'null' example: false serverId: description: Optional MySQL server_id the pipe declares itself as in the MySQL replication topology. Must be unique across replicas connected to the source. If omitted, one is assigned automatically. type: - integer - 'null' minimum: 1 maximum: 4294967295 example: 4242 settings: $ref: '#/components/schemas/ClickPipePatchMySQLPipeSettings' tableMappingsToAdd: type: array description: Table mappings to add to the pipe. Can be an empty array if no tables are being added. items: $ref: '#/components/schemas/ClickPipeMySQLPipeTableMapping' minItems: 0 tableMappingsToRemove: type: array description: Table mappings to remove from the pipe. Only sourceSchemaName, sourceTable, and targetTable are required for removal. items: $ref: '#/components/schemas/ClickPipePatchMySQLPipeRemoveTableMapping' minItems: 0 required: - host - port PLAIN: properties: username: description: Database username. type: string example: postgres_user password: description: Database password. type: string format: password example: your_secure_password ClickPipeBigQueryServiceAccountSource: properties: snapshotStagingPath: description: GCS bucket path for staging snapshot data (e.g., gs://my-bucket/staging/). Data will be automatically cleaned up after initial load. type: string settings: $ref: '#/components/schemas/ClickPipeBigQueryPipeSettings' tableMappings: type: array description: Table mappings for BigQuery pipe. items: $ref: '#/components/schemas/ClickPipeBigQueryPipeTableMapping' authentication: description: Authenticated with a Google Cloud service account JSON key. type: string enum: - SERVICE_ACCOUNT projectId: description: GCP project ID that owns the BigQuery resources. type: string example: my-gcp-project required: - snapshotStagingPath - settings - tableMappings - authentication ClickPipeDestinationTableDefinition: properties: engine: $ref: '#/components/schemas/ClickPipeDestinationTableEngine' sortingKey: type: array description: Sorting key of the destination table. List of columns. items: type: string partitionBy: description: Partition key SQL expression. type: string primaryKey: description: Primary key of SQL expression. type: string ClickPipePatchMySQLPipeSettings: properties: syncIntervalSeconds: description: Interval in seconds to sync data from MySQL during CDC replication. type: - integer - 'null' minimum: 1 example: 60 pullBatchSize: description: Number of rows to pull in each batch during CDC replication. type: - integer - 'null' minimum: 1 example: 1000 useCompression: description: Enable compression for the MySQL connection. type: - boolean - 'null' example: false ClickPipePatchPostgresSource: properties: credentials: $ref: '#/components/schemas/PLAIN' host: description: PostgreSQL server hostname or IP address. To use a reverse private endpoint, pass the endpoint hostname here. type: - string - 'null' format: hostname example: my-postgres-server.example.com port: description: PostgreSQL server port. type: - integer - 'null' minimum: 1 maximum: 65535 example: 5432 database: description: PostgreSQL database name to replicate from. type: - string - 'null' example: production_db tlsHost: description: TLS/SSL host for secure connections. type: - string - 'null' example: my-postgres-server.example.com caCertificate: description: PEM encoded CA certificate to validate the Postgres server certificate. type: - string - 'null' example: '-----BEGIN CERTIFICATE----- ...' disableTls: description: Disable TLS for the Postgres connection. Use with caution in production environments. type: - boolean - 'null' example: false skipCertVerification: description: Skip TLS certificate verification for the Postgres connection. Use with caution in production environments. type: - boolean - 'null' example: false settings: $ref: '#/components/schemas/ClickPipePatchPostgresPipeSettings' tableMappingsToAdd: type: array description: Table mappings to add to the pipe. Can be an empty array if no tables are being added. items: $ref: '#/components/schemas/ClickPipePostgresPipeTableMapping' minItems: 0 tableMappingsToRemove: type: array description: Table mappings to remove from the pipe. Only sourceSchemaName, sourceTable, and targetTable are required for removal. items: $ref: '#/components/schemas/ClickPipePatchPostgresPipeRemoveTableMapping' minItems: 0 ClickPipePatchKinesisSource: properties: authentication: description: Authentication method to use with the Kinesis stream. type: - string - 'null' enum: - IAM_ROLE - IAM_USER iamRole: description: IAM role to use for authentication. Required if IAM_ROLE is used. type: - string - 'null' example: arn:aws:iam::123456789012:role/MyRole accessKey: oneOf: - $ref: '#/components/schemas/MskIamUser' - type: 'null' ClickPipePostgresSource: properties: type: description: Type of the Postgres source. Defaults to "postgres" if not specified. type: - string - 'null' enum: - postgres - supabase - neon - alloydb - planetscale - rdspostgres - aurorapostgres - cloudsqlpostgres - azurepostgres - crunchybridge - tigerdata host: description: PostgreSQL server hostname or IP address. To use a reverse private endpoint, pass the endpoint hostname here. type: string format: hostname example: my-postgres-server.example.com port: description: PostgreSQL server port. type: integer minimum: 1 maximum: 65535 example: 5432 database: description: PostgreSQL database name to replicate from. type: string example: production_db authentication: description: Authentication method for Postgres connection. type: string enum: - basic - IAM_ROLE example: IAM_ROLE iamRole: description: IAM role ARN for IAM authentication (required for IAM_ROLE authentication). type: string example: arn:aws:iam::123456789012:role/MyApplicationRole tlsHost: description: TLS/SSL host for secure connections. type: string example: my-postgres-server.example.com caCertificate: description: PEM encoded CA certificate to validate the Postgres server certificate. type: string example: '-----BEGIN CERTIFICATE----- ...' disableTls: description: Disable TLS for the Postgres connection. Use with caution in production environments. type: boolean example: false skipCertVerification: description: Skip TLS certificate verification for the Postgres connection. Use with caution in production environments. type: boolean example: false settings: $ref: '#/components/schemas/ClickPipePostgresPipeSettings' tableMappings: type: array description: List of table mappings defining which PostgreSQL tables to replicate and how they map to ClickHouse tables. items: $ref: '#/components/schemas/ClickPipePostgresPipeTableMapping' ClickPipesCdcScaling: properties: replicaCpuMillicores: description: CPU in millicores for DB ClickPipes. type: integer minimum: 1000 maximum: 32000 multipleOf: 1000 example: 2000 replicaMemoryGb: description: Memory in GiB for DB ClickPipes. Must be 4× the CPU core count. type: number minimum: 4 maximum: 128 multipleOf: 4 example: 8 ClickPipeMutateMongoDBSource: properties: credentials: $ref: '#/components/schemas/PLAIN' uri: description: MongoDB connection URI. Supports both standard URIs (mongodb://...) and SRV URIs (mongodb+srv://...). Embedded credentials are redacted from API responses, so the returned value can differ from what was submitted. type: string example: mongodb+srv://cluster0.example.mongodb.net/mydb readPreference: description: MongoDB read preference for replica set reads. type: string enum: - primary - primaryPreferred - secondary - secondaryPreferred - nearest example: secondaryPreferred tlsHost: description: TLS/SSL host for secure connections. type: string example: cluster0.example.mongodb.net disableTls: description: Disable TLS for the MongoDB connection. Defaults to false (TLS enabled). type: boolean example: false default: false skipCertVerification: description: Skip TLS certificate verification for the MongoDB connection. Use with caution in production environments. type: boolean example: false caCertificate: description: PEM encoded CA certificate to validate the MongoDB server certificate. type: string example: '-----BEGIN CERTIFICATE----- ...' settings: $ref: '#/components/schemas/ClickPipeMongoDBPipeSettings' tableMappings: type: array description: List of collection mappings defining which MongoDB collections to replicate and how they map to ClickHouse tables. items: $ref: '#/components/schemas/ClickPipeMongoDBPipeTableMapping' required: - uri - readPreference - settings - tableMappings ClickPipeKinesisSource: properties: format: description: Format of the Kinesis stream. type: string enum: - JSONEachRow - Avro - AvroConfluent streamName: description: Name of the Kinesis stream. type: string example: my-stream region: description: AWS region of the Kinesis stream. type: string example: us-east-1 useEnhancedFanOut: description: Use enhanced fan-out for the Kinesis stream. type: - boolean - 'null' iteratorType: description: Type of iterator to use when reading from the Kinesis stream. If AT_TIMESTAMP is used, the timestamp field must be provided. type: string enum: - TRIM_HORIZON - LATEST - AT_TIMESTAMP timestamp: description: UNIX timestamp to start reading from the Kinesis stream. Required if iteratorType is AT_TIMESTAMP. type: - integer - 'null' example: 1615766400 authentication: description: Authentication method to use with the Kinesis stream. type: string enum: - IAM_ROLE - IAM_USER iamRole: description: IAM role to use for authentication. Required if IAM_ROLE is used. type: - string - 'null' example: arn:aws:iam::123456789012:role/MyRole ClickPipePatchSource: properties: kafka: oneOf: - $ref: '#/components/schemas/ClickPipePatchKafkaSource' - type: 'null' objectStorage: oneOf: - $ref: '#/components/schemas/ClickPipePatchObjectStorageSource' - type: 'null' kinesis: oneOf: - $ref: '#/components/schemas/ClickPipePatchKinesisSource' - type: 'null' pubsub: oneOf: - $ref: '#/components/schemas/ClickPipePatchPubSubSource' - type: 'null' postgres: $ref: '#/components/schemas/ClickPipePatchPostgresSource' mysql: oneOf: - $ref: '#/components/schemas/ClickPipePatchMySQLSource' - type: 'null' mongodb: oneOf: - $ref: '#/components/schemas/ClickPipePatchMongoDBSource' - type: 'null' validateSamples: description: Validate data samples received from data source. It will validate the connection and data availability and correctness. If not enabled, only connection will be validated. This has no effect on Postgres or MySQL pipes, they always only validate the connection and table definitions. This is experimental and can be removed in the future. type: boolean ClickPipePatchMongoDBSource: properties: credentials: $ref: '#/components/schemas/PLAIN' uri: description: MongoDB connection URI (mongodb:// or mongodb+srv://). Credentials are redacted in responses; a masked value is never saved as a credential, so a real credential equal to [REDACTED] cannot be set. To change the connection, send the full URI with real credentials; omit this field or resend the redacted value to leave it unchanged. type: - string - 'null' example: mongodb+srv://cluster0.example.mongodb.net/mydb readPreference: description: MongoDB read preference for replica set reads. type: - string - 'null' enum: - primary - primaryPreferred - secondary - secondaryPreferred - nearest example: secondaryPreferred tlsHost: description: TLS/SSL host for secure connections. type: - string - 'null' example: cluster0.example.mongodb.net disableTls: description: Disable TLS for the MongoDB connection. Defaults to false (TLS enabled). type: - boolean - 'null' example: false skipCertVerification: description: Skip TLS certificate verification for the MongoDB connection. Use with caution in production environments. type: - boolean - 'null' example: false caCertificate: description: PEM encoded CA certificate to validate the MongoDB server certificate. type: - string - 'null' example: '-----BEGIN CERTIFICATE----- ...' settings: $ref: '#/components/schemas/ClickPipePatchMongoDBPipeSettings' tableMappingsToAdd: type: array description: Collection mappings to add to the pipe. Can be an empty array if no collections are being added. items: $ref: '#/components/schemas/ClickPipeMongoDBPipeTableMapping' minItems: 0 tableMappingsToRemove: type: array description: Collection mappings to remove from the pipe. Only sourceDatabaseName, sourceCollection, and targetTable are required for removal. items: $ref: '#/components/schemas/ClickPipePatchMongoDBPipeRemoveTableMapping' minItems: 0 required: - uri - readPreference ClickPipePubSubSource: properties: format: description: Format of messages in the Pub/Sub topic. GCP Pub/Sub ClickPipes are in limited preview — contact support to enable this feature for your organization. type: string enum: - JSONEachRow - Avro - Protobuf example: JSONEachRow projectId: description: GCP project ID that owns the Pub/Sub topic. type: string example: my-gcp-project topic: description: Pub/Sub topic name (not the fully-qualified path). type: string example: my-topic authentication: description: Authentication method to use with GCP Pub/Sub. SERVICE_ACCOUNT_WORKLOAD_IDENTITY is in Private Preview. ClickPipes uses the GCP service account returned in gcpWorkloadIdentity.principal by the operation with operationId clickPipesServiceContextGet; grant it access to the source resources. type: string enum: - SERVICE_ACCOUNT - SERVICE_ACCOUNT_WORKLOAD_IDENTITY example: SERVICE_ACCOUNT seekType: description: Starting position strategy for consuming the subscription. The seekTimestamp companion is required only when seekType is "timestamp"; setting it for a mismatched seek type is rejected. type: string enum: - latest - earliest - timestamp example: earliest seekTimestamp: description: RFC 3339 / ISO 8601 timestamp to seek to. Required when seekType is "timestamp"; must be omitted otherwise. type: - string - 'null' format: date-time example: '2026-04-10T12:00:00Z' filter: description: Optional Pub/Sub subscription filter expression (CEL). Maximum 256 characters. type: - string - 'null' maxLength: 256 enableOrdering: description: Whether to enable ordered delivery of messages (requires messages to be published with ordering keys). type: - boolean - 'null' ackDeadline: description: Acknowledgement deadline for messages, in seconds. Must be between 10 and 600. type: - integer - 'null' minimum: 10 maximum: 600 required: - format - projectId - topic - authentication - seekType ClickPipeBigQueryWorkloadIdentitySource: properties: snapshotStagingPath: description: GCS bucket path for staging snapshot data (e.g., gs://my-bucket/staging/). Data will be automatically cleaned up after initial load. type: string settings: $ref: '#/components/schemas/ClickPipeBigQueryPipeSettings' tableMappings: type: array description: Table mappings for BigQuery pipe. items: $ref: '#/components/schemas/ClickPipeBigQueryPipeTableMapping' authentication: description: Authenticated with the ClickPipes service tenant identity. SERVICE_ACCOUNT_WORKLOAD_IDENTITY is in Private Preview. ClickPipes uses the GCP service account returned in gcpWorkloadIdentity.principal by the operation with operationId clickPipesServiceContextGet; grant it access to the source resources. type: string enum: - SERVICE_ACCOUNT_WORKLOAD_IDENTITY projectId: description: GCP project ID that owns the BigQuery resources. Older pipes created outside OpenAPI may omit this field. type: string example: my-gcp-project required: - snapshotStagingPath - settings - tableMappings - authentication CustomPrivateDnsMapping: properties: privateDnsName: description: 'Optional private DNS names for Reverse Private Endpoint. Can be used as data source destination address. Must be unique across the ClickHouse service. Generally available for Google Private Service Connect (PSC). For AWS PrivateLink (VPC endpoint service and VPC resource), available in Private Preview; contact ClickHouse support to enable it for your service. Not supported for MSK multi-VPC. Supports exact names and leading wildcard names such as *.example.com' type: string example: '*.my-service.example.com' ClickPipePostgresPipeTableMapping: properties: sourceSchemaName: description: PostgreSQL source schema name. type: string example: public sourceTable: description: PostgreSQL source table name. type: string example: users targetTable: description: ClickHouse target table name, optionally prefixed with schema name (e.g., "my_schema_my_table"). The table will be created automatically if it does not exist. For snapshot mode, the target table must be empty. type: string example: public_users excludedColumns: type: array description: List of column names to exclude from replication. Column names must be unique within this list. items: type: string example: - internal_id - temp_data uniqueItems: true useCustomSortingKey: description: Whether to use a custom sorting key. If true, sortingKeys must be provided. If false or omitted, the default sorting key is the PostgreSQL primary key. type: boolean example: false sortingKeys: type: array description: Ordered list of column names to use as the sorting (ORDER BY) key in ClickHouse. Only used when useCustomSortingKey is true. Column names must be unique within this list. items: type: string example: - created_at_date - event_id uniqueItems: true tableEngine: description: 'ClickHouse table engine: "ReplacingMergeTree" (handles updates/deletes), "MergeTree" (append-only), or "Null" (forward data to materialized views without storing it).' type: string enum: - MergeTree - ReplacingMergeTree - 'Null' example: ReplacingMergeTree partitionKey: description: 'Custom partitioning column used for parallel snapshotting. Only beneficial for PostgreSQL 13 (no benefit for PG14+, which supports indexed ctid scans). Must be an indexed column of type: `smallint`, `integer`, `bigint`, `timestamp without time zone`, or `timestamp with time zone`. Unrelated to ClickHouse partitioning.' type: string example: id partitionByExpr: description: ClickHouse PARTITION BY expression applied to the destination table when ClickPipes creates it. type: string example: toYYYYMM(created_at) ClickPipeBigQueryPipeTableMapping: properties: sourceDatasetName: description: Source BigQuery dataset name. type: string sourceTable: description: Source table name. type: string targetTable: description: Target ClickHouse table name. type: string excludedColumns: type: array description: Columns to exclude from the target table. items: type: string useCustomSortingKey: description: Whether to use a custom sorting key for the target table. type: boolean sortingKeys: type: array description: Ordered list of columns to use as sorting key for the target table. items: type: string tableEngine: description: Table engine to use for the target table. type: string enum: - MergeTree - ReplacingMergeTree - 'Null' required: - sourceDatasetName - sourceTable - targetTable ClickPipeSource: properties: kafka: oneOf: - $ref: '#/components/schemas/ClickPipeKafkaSource' - type: 'null' objectStorage: oneOf: - $ref: '#/components/schemas/ClickPipeObjectStorageSource' - type: 'null' kinesis: oneOf: - $ref: '#/components/schemas/ClickPipeKinesisSource' - type: 'null' pubsub: oneOf: - $ref: '#/components/schemas/ClickPipePubSubSource' - type: 'null' postgres: oneOf: - $ref: '#/components/schemas/ClickPipePostgresSource' - type: 'null' mysql: oneOf: - $ref: '#/components/schemas/ClickPipeMySQLSource' - type: 'null' bigquery: oneOf: - $ref: '#/components/schemas/ClickPipeBigQuerySource' - type: 'null' mongodb: oneOf: - $ref: '#/components/schemas/ClickPipeMongoDBSource' - type: 'null' ClickPipeMySQLPipeSettings: properties: syncIntervalSeconds: description: Interval in seconds to sync data from MySQL during CDC replication. type: integer minimum: 1 example: 60 pullBatchSize: description: Number of rows to pull in each batch during CDC replication. type: integer minimum: 1 example: 1000 replicationMode: description: 'Replication mode: "cdc" (change data capture with initial snapshot), "snapshot" (one-time snapshot only), or "cdc_only" (CDC without initial snapshot).' type: string enum: - cdc - snapshot - cdc_only example: cdc replicationMechanism: description: 'MySQL replication mechanism: "GTID" (Global Transaction Identifier) or "FILE_POS" (binary log file and position). Defaults to "GTID" if not specified. MariaDB supports "GTID" only. For "FILE_POS" on MySQL, contact support.' type: string enum: - GTID - FILE_POS example: GTID useCompression: description: Enable compression for the MySQL connection. type: boolean example: false allowNullableColumns: description: 'Preserve nullability from MySQL in the destination ClickHouse table. When true, columns without NOT NULL constraints are created as Nullable(...). When false, all columns are non-nullable and NULL values are replaced with the default value for the type. Note: Nullable types have performance overhead in ClickHouse.' type: boolean example: false initialLoadParallelism: description: Number of parallel workers to use per table in the initial snapshot phase. type: integer minimum: 1 example: 1 snapshotNumRowsPerPartition: description: Number of rows per partition during the snapshot phase. type: integer minimum: 1000 example: 100000 snapshotNumberOfParallelTables: description: Number of tables to snapshot in parallel during the initial load phase. type: integer minimum: 1 example: 1 deleteOnMerge: description: Enable hard delete behavior in ReplacingMergeTree for MySQL DELETE operations. type: boolean example: false required: - replicationMode ClickPipesGcpWorkloadIdentityContext: properties: supported: description: Whether the ClickPipes deployment supports GCP workload identity, which is in Private Preview. The principal field identifies the GCP service account used for source access. type: boolean ready: description: Whether the service tenant identity is ready for workload identity authentication. type: - boolean - 'null' principal: description: GCP service account used by ClickPipes for workload identity authentication. Grant this service account access to customer source resources. type: - string - 'null' example: ch-deadbeef@clickpipes-development.iam.gserviceaccount.com ClickPipeKafkaSchemaRegistryCredentials: properties: username: description: Username for the schema registry. type: string password: description: Password for the schema registry. type: string ClickPipePatchKafkaSource: properties: authentication: description: 'Authentication method of the Kafka source. SERVICE_ACCOUNT_WORKLOAD_IDENTITY is in Private Preview. ClickPipes uses the GCP service account returned in gcpWorkloadIdentity.principal by the operation with operationId clickPipesServiceContextGet; grant it access to the source resources. Supported authentication methods: kafka: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512, MUTUAL_TLS, msk: SCRAM-SHA-512, IAM_ROLE, IAM_USER, MUTUAL_TLS, gcmk: PLAIN, MUTUAL_TLS, SERVICE_ACCOUNT_WORKLOAD_IDENTITY, confluent: PLAIN, MUTUAL_TLS, warpstream: PLAIN, azureeventhub: PLAIN, redpanda: SCRAM-SHA-256, SCRAM-SHA-512, MUTUAL_TLS, dokafka: SCRAM-SHA-256, MUTUAL_TLS' type: - string - 'null' enum: - PLAIN - SCRAM-SHA-256 - SCRAM-SHA-512 - IAM_ROLE - IAM_USER - MUTUAL_TLS - SERVICE_ACCOUNT_WORKLOAD_IDENTITY iamRole: description: 'IAM role for the Kafka source. Use with IAM role authentication. Read more in ClickPipes documentation: https://clickhouse.com/docs/en/integrations/clickpipes/kafka#iam' type: - string - 'null' example: arn:aws:iam::123456789012:role/MyRole caCertificate: description: PEM encoded CA certificates to validate the broker's certificate. type: - string - 'null' reversePrivateEndpointIds: type: array description: Reverse private endpoint UUIDs used for a secure private connection to the Kafka source. items: type: string credentials: description: Credentials for Kafka source. Choose one that is supported by the authentication method. oneOf: - $ref: '#/components/schemas/PLAIN' - $ref: '#/components/schemas/MskIamUser' - $ref: '#/components/schemas/AzureEventHub' - $ref: '#/components/schemas/MutualTLS' ClickPipePostBigQueryWorkloadIdentitySource: properties: snapshotStagingPath: description: GCS bucket path for staging snapshot data (e.g., gs://my-bucket/staging/). Data will be automatically cleaned up after initial load. type: string settings: $ref: '#/components/schemas/ClickPipeBigQueryPipeSettings' tableMappings: type: array description: Table mappings for BigQuery pipe. items: $ref: '#/components/schemas/ClickPipeBigQueryPipeTableMapping' authentication: description: Authenticate with the ClickPipes service tenant identity. Customer credentials must not be provided. SERVICE_ACCOUNT_WORKLOAD_IDENTITY is in Private Preview. ClickPipes uses the GCP service account returned in gcpWorkloadIdentity.principal by the operation with operationId clickPipesServiceContextGet; grant it access to the source resources. type: string enum: - SERVICE_ACCOUNT_WORKLOAD_IDENTITY example: SERVICE_ACCOUNT_WORKLOAD_IDENTITY projectId: description: GCP project ID that owns the BigQuery resources. type: string example: my-gcp-project required: - snapshotStagingPath - settings - tableMappings - authentication - projectId additionalProperties: false ClickPipeMongoDBPipeTableMapping: properties: sourceDatabaseName: description: MongoDB source database name. type: string example: mydb sourceCollection: description: MongoDB source collection name. type: string example: users targetTable: description: ClickHouse target table name. The table will be created automatically if it does not exist. type: string example: mydb_users tableEngine: description: 'ClickHouse table engine: "ReplacingMergeTree" (handles updates/deletes), "MergeTree" (append-only), or "Null" (forward data to materialized views without storing it).' type: string enum: - MergeTree - ReplacingMergeTree - 'Null' example: ReplacingMergeTree required: - sourceDatabaseName - sourceCollection - targetTable ClickPipeSchemaDiscoveryField: properties: name: description: Name of the inferred field. type: string example: user_id type: description: Inferred ClickHouse data type of the field. type: string example: Int64 optional: description: Whether the field is optional (nullable) in the source. type: - boolean - 'null' ClickPipePostKafkaSource: properties: type: description: Type of the Kafka source. type: string enum: - kafka - redpanda - msk - gcmk - confluent - warpstream - azureeventhub - dokafka format: description: Format of the Kafka source. type: string enum: - JSONEachRow - Avro - AvroConfluent - Protobuf brokers: description: Brokers of the Kafka source. type: string topics: description: Topics of the Kafka source. type: string consumerGroup: description: Consumer group of the Kafka source. If not provided "clickpipes-<>" will be used. type: - string - 'null' example: my-clickpipe-consumer-group authentication: description: 'Authentication method of the Kafka source. SERVICE_ACCOUNT_WORKLOAD_IDENTITY is in Private Preview. ClickPipes uses the GCP service account returned in gcpWorkloadIdentity.principal by the operation with operationId clickPipesServiceContextGet; grant it access to the source resources. Supported authentication methods: kafka: PLAIN, SCRAM-SHA-256, SCRAM-SHA-512, MUTUAL_TLS, msk: SCRAM-SHA-512, IAM_ROLE, IAM_USER, MUTUAL_TLS, gcmk: PLAIN, MUTUAL_TLS, SERVICE_ACCOUNT_WORKLOAD_IDENTITY, confluent: PLAIN, MUTUAL_TLS, warpstream: PLAIN, azureeventhub: PLAIN, redpanda: SCRAM-SHA-256, SCRAM-SHA-512, MUTUAL_TLS, dokafka: SCRAM-SHA-256, MUTUAL_TLS' type: string enum: - PLAIN - SCRAM-SHA-256 - SCRAM-SHA-512 - IAM_ROLE - IAM_USER - MUTUAL_TLS - SERVICE_ACCOUNT_WORKLOAD_IDENTITY iamRole: description: 'IAM role for the Kafka source. Use with IAM role authentication. Read more in ClickPipes documentation: https://clickhouse.com/docs/en/integrations/clickpipes/kafka#iam' type: - string - 'null' example: arn:aws:iam::123456789012:role/MyRole offset: oneOf: - $ref: '#/components/schemas/ClickPipeKafkaOffset' - type: 'null' schemaRegistry: oneOf: - $ref: '#/components/schemas/ClickPipeMutateKafkaSchemaRegistry' - type: 'null' caCertificate: description: PEM encoded CA certificates to validate the broker's certificate. type: - string - 'null' reversePrivateEndpointIds: type: array description: Reverse private endpoint UUIDs used for a secure private connection to the Kafka source. items: type: string exactlyOnce: description: Enable exactly-once delivery. Guarantees every Kafka record is inserted exactly once across restarts and rebalances. Can only be set at pipe creation. type: - boolean - 'null' credentials: description: Credentials for Kafka source. Choose one that is supported by the authentication method. oneOf: - $ref: '#/components/schemas/PLAIN' - $ref: '#/components/schemas/MskIamUser' - $ref: '#/components/schemas/AzureEventHub' - $ref: '#/components/schemas/MutualTLS' protobufSchema: description: Base64-encoded .proto source or serialized FileDescriptorSet. Supported only with Protobuf format and cannot be combined with schemaRegistry. type: string example: c3ludGF4ID0gInByb3RvMyI7IG1lc3NhZ2UgRXZlbnQge30= maxLength: 1048576 minLength: 1 writeOnly: true ClickPipeMongoDBPipeSettings: properties: syncIntervalSeconds: description: Interval in seconds to sync data from MongoDB during CDC replication. type: integer minimum: 1 example: 60 pullBatchSize: description: Number of rows to pull in each batch during CDC replication. type: integer minimum: 1 example: 100000 replicationMode: description: 'Replication mode: "cdc" (change data capture with initial snapshot), "snapshot" (one-time snapshot only), or "cdc_only" (CDC without initial snapshot).' type: string enum: - cdc - snapshot - cdc_only example: cdc snapshotNumRowsPerPartition: description: Number of rows per partition during the snapshot phase. type: integer minimum: 1000 example: 100000 snapshotNumberOfParallelTables: description: Number of collections to snapshot in parallel during the initial load phase. type: integer minimum: 1 example: 1 deleteOnMerge: description: Enable hard delete behavior in ReplacingMergeTree for MongoDB DELETE operations. type: boolean example: false useJsonNativeFormat: description: Store JSON values in native ClickHouse JSON format. When disabled, JSON data is stored as String. type: boolean example: true required: - replicationMode ClickPipeMutatePostgresSource: properties: type: description: Type of the Postgres source. Defaults to "postgres" if not specified. type: - string - 'null' enum: - postgres - supabase - neon - alloydb - planetscale - rdspostgres - aurorapostgres - cloudsqlpostgres - azurepostgres - crunchybridge - tigerdata credentials: $ref: '#/components/schemas/PLAIN' host: description: PostgreSQL server hostname or IP address. To use a reverse private endpoint, pass the endpoint hostname here. type: string format: hostname example: my-postgres-server.example.com port: description: PostgreSQL server port. type: integer minimum: 1 maximum: 65535 example: 5432 database: description: PostgreSQL database name to replicate from. type: string example: production_db settings: $ref: '#/components/schemas/ClickPipePostgresPipeSettings' authentication: description: Authentication method for Postgres connection. type: string enum: - basic - IAM_ROLE example: IAM_ROLE iamRole: description: IAM role ARN for IAM authentication (required for IAM_ROLE authentication). type: string example: arn:aws:iam::123456789012:role/MyApplicationRole tlsHost: description: TLS/SSL host for secure connections. type: string example: my-postgres-server.example.com caCertificate: description: PEM encoded CA certificate to validate the Postgres server certificate. type: string example: '-----BEGIN CERTIFICATE----- ...' disableTls: description: Disable TLS for the Postgres connection. Use with caution in production environments. Defaults to false when omitted. type: boolean example: false default: false skipCertVerification: description: Skip TLS certificate verification for the Postgres connection. Use with caution in production environments. type: boolean example: false tableMappings: type: array description: List of table mappings defining which PostgreSQL tables to replicate and how they map to ClickHouse tables. items: $ref: '#/components/schemas/ClickPipePostgresPipeTableMapping' ClickPipeSchemaDiscoveryRequest: properties: source: $ref: '#/components/schemas/ClickPipeSchemaDiscoverySource' ClickPipe: properties: id: description: Unique ClickPipe ID. type: string format: uuid serviceId: description: ID of the service this ClickPipe belongs to. type: string format: uuid name: description: Name of the ClickPipe. type: string example: my_postgres_pipe maxLength: 255 minLength: 1 state: description: 'Current lifecycle state of the ClickPipe. For database pipes: "Provisioning" (initial setup), "Setup" (configuring replication), "Snapshot" (initial data load), "Running" (actively replicating), "Pausing" (transitioning to paused state), "Paused" (temporarily paused), "Modifying" (applying configuration updates), "Resync" (swapping resync tables with original tables), "Failed" (error occurred), "Unknown". For streaming/object storage pipes (Kafka, Kinesis, S3): "Unknown" (initial state), "Provisioning" (setting up resources), "Running" (actively ingesting data), "Stopping" (transitioning to stopped state), "Stopped" (manually stopped, can be restarted), "Completed" (batch ingestion finished for object storage), "Failed" (error occurred, pipe stopped), "InternalError" (internal system error).' type: string enum: - Unknown - Provisioning - Running - Degraded - Stopping - Stopped - Failed - Completed - InternalError - Setup - Snapshot - Paused - Pausing - Modifying - Resync example: Running scaling: $ref: '#/components/schemas/ClickPipeScaling' source: $ref: '#/components/schemas/ClickPipeSource' destination: $ref: '#/components/schemas/ClickPipeDestination' fieldMappings: type: array description: Field mappings of the ClickPipe. Note that all destination columns must be included in the mappings. items: $ref: '#/components/schemas/ClickPipeFieldMapping' settings: $ref: '#/components/schemas/ClickPipeSettings' createdAt: description: Creation timestamp of the ClickPipe in ISO 8601 format. type: string format: date-time updatedAt: description: Last update timestamp of the ClickPipe in ISO 8601 format. type: string format: date-time ClickPipeMutateBigQuerySource: oneOf: - $ref: '#/components/schemas/ClickPipePostBigQueryServiceAccountSource' - $ref: '#/components/schemas/ClickPipePostBigQueryWorkloadIdentitySource' ClickPipePatchMongoDBPipeSettings: properties: syncIntervalSeconds: description: Interval in seconds to sync data from MongoDB during CDC replication. type: - integer - 'null' minimum: 1 example: 60 pullBatchSize: description: Number of rows to pull in each batch during CDC replication. type: - integer - 'null' minimum: 1 example: 100000 ClickPipePatchObjectStorageSource: properties: skipInitialLoad: description: If set to true, skips the initial load and only ingests files delivered by queue notifications. Only applicable when queueUrl is provided. type: - boolean - 'null' startAfter: description: Skip all files up to and including this object key during the initial load. Cannot be provided when skipInitialLoad is true. type: - string - 'null' example: events/2026-06-01/ authentication: description: Authentication method. IAM_USER is for S3, GCS, and DigitalOcean Spaces. IAM_ROLE is for S3 only. SERVICE_ACCOUNT is for GCS only. For GCS, SERVICE_ACCOUNT_WORKLOAD_IDENTITY is in Private Preview. ClickPipes uses the GCP service account returned in gcpWorkloadIdentity.principal by the operation with operationId clickPipesServiceContextGet; grant it access to the source resources. CONNECTION_STRING is for Azure Blob Storage. PUBLIC uses no authentication. type: - string - 'null' enum: - IAM_ROLE - IAM_USER - CONNECTION_STRING - SERVICE_ACCOUNT - SERVICE_ACCOUNT_WORKLOAD_IDENTITY iamRole: description: 'IAM role to be used with IAM role authentication. Read more in ClickPipes documentation: https://clickhouse.com/docs/en/integrations/clickpipes/object-storage#authentication' type: - string - 'null' example: arn:aws:iam::123456789012:role/MyRole connectionString: description: Connection string for Azure Blob Storage authentication. Required when authentication is CONNECTION_STRING. type: - string - 'null' example: DefaultEndpointsProtocol=https;AccountName=myaccount;AccountKey=mykey;EndpointSuffix=core.windows.net path: description: 'Path to the file(s) within the Azure container. Used for Azure Blob Storage sources. You can specify multiple files using bash-like wildcards. For more information, see the documentation on using wildcards in path: https://clickhouse.com/docs/en/integrations/clickpipes/object-storage#limitations' type: - string - 'null' example: data/logs/*.json azureContainerName: description: Container name for Azure Blob Storage. Required when type is azureblobstorage. type: - string - 'null' example: mycontainer accessKey: oneOf: - $ref: '#/components/schemas/MskIamUser' - type: 'null' serviceAccountKey: description: Base64-encoded GCP service account JSON key. Required when authentication is SERVICE_ACCOUNT. type: - string - 'null' ClickPipesServiceContext: properties: gcpWorkloadIdentity: $ref: '#/components/schemas/ClickPipesGcpWorkloadIdentityContext' securitySchemes: basicAuth: type: http scheme: basic description: 'Use key ID and key secret obtained in ClickHouse Cloud console: https://clickhouse.com/docs/cloud/manage/openapi' x-tagGroups: - name: Organization tags: - Organization - Billing - User management - Role Management - UDF - name: Service tags: - Service - Backup - name: API keys tags: - API keys - name: Prometheus tags: - Prometheus - name: ClickPipes tags: - ClickPipes - name: ClickStack tags: - ClickStack - name: Postgres tags: - Postgres