openapi: 3.0.0 info: version: 2017-07-25 x-release: v4 title: AWS Glue DataBrew Datasets Jobs API description: Glue DataBrew is a visual, cloud-scale data-preparation service. DataBrew simplifies data preparation tasks, targeting data issues that are hard to spot and time-consuming to fix. DataBrew empowers users of all technical levels to visualize the data and perform one-click data transformations, with no coding required. x-logo: url: https://api.apis.guru/v2/cache/logo/https_twitter.com_awscloud_profile_image.png backgroundColor: '#FFFFFF' termsOfService: https://aws.amazon.com/service-terms/ contact: name: Mike Ralphson email: mike.ralphson@gmail.com url: https://github.com/mermade/aws2openapi x-twitter: PermittedSoc license: name: Apache 2.0 License url: http://www.apache.org/licenses/ x-providerName: amazonaws.com x-serviceName: databrew x-aws-signingName: databrew x-origin: - contentType: application/json url: https://raw.githubusercontent.com/aws/aws-sdk-js/master/apis/databrew-2017-07-25.normal.json converter: url: https://github.com/mermade/aws2openapi version: 1.0.0 x-apisguru-driver: external x-apiClientRegistration: url: https://portal.aws.amazon.com/gp/aws/developer/registration/index.html?nc2=h_ct x-apisguru-categories: - cloud x-preferred: true servers: - url: http://databrew.{region}.amazonaws.com variables: region: description: The AWS region enum: - us-east-1 - us-east-2 - us-west-1 - us-west-2 - us-gov-west-1 - us-gov-east-1 - ca-central-1 - eu-north-1 - eu-west-1 - eu-west-2 - eu-west-3 - eu-central-1 - eu-south-1 - af-south-1 - ap-northeast-1 - ap-northeast-2 - ap-northeast-3 - ap-southeast-1 - ap-southeast-2 - ap-east-1 - ap-south-1 - sa-east-1 - me-south-1 default: us-east-1 description: The AWS Glue DataBrew multi-region endpoint - url: https://databrew.{region}.amazonaws.com variables: region: description: The AWS region enum: - us-east-1 - us-east-2 - us-west-1 - us-west-2 - us-gov-west-1 - us-gov-east-1 - ca-central-1 - eu-north-1 - eu-west-1 - eu-west-2 - eu-west-3 - eu-central-1 - eu-south-1 - af-south-1 - ap-northeast-1 - ap-northeast-2 - ap-northeast-3 - ap-southeast-1 - ap-southeast-2 - ap-east-1 - ap-south-1 - sa-east-1 - me-south-1 default: us-east-1 description: The AWS Glue DataBrew multi-region endpoint - url: http://databrew.{region}.amazonaws.com.cn variables: region: description: The AWS region enum: - cn-north-1 - cn-northwest-1 default: cn-north-1 description: The AWS Glue DataBrew endpoint for China (Beijing) and China (Ningxia) - url: https://databrew.{region}.amazonaws.com.cn variables: region: description: The AWS region enum: - cn-north-1 - cn-northwest-1 default: cn-north-1 description: The AWS Glue DataBrew endpoint for China (Beijing) and China (Ningxia) security: - hmac: [] tags: - name: Jobs paths: /jobs/{name}: parameters: - $ref: '#/components/parameters/X-Amz-Content-Sha256' - $ref: '#/components/parameters/X-Amz-Date' - $ref: '#/components/parameters/X-Amz-Algorithm' - $ref: '#/components/parameters/X-Amz-Credential' - $ref: '#/components/parameters/X-Amz-Security-Token' - $ref: '#/components/parameters/X-Amz-Signature' - $ref: '#/components/parameters/X-Amz-SignedHeaders' delete: operationId: DeleteJob description: Deletes the specified DataBrew job. responses: '200': description: Success content: application/json: schema: $ref: '#/components/schemas/DeleteJobResponse' examples: DeleteJob200Example: summary: Default DeleteJob 200 response x-microcks-default: true value: status: SUCCEEDED '480': description: ConflictException content: application/json: schema: $ref: '#/components/schemas/ConflictException' examples: DeleteJob480Example: summary: Default DeleteJob 480 response x-microcks-default: true value: status: SUCCEEDED '481': description: ResourceNotFoundException content: application/json: schema: $ref: '#/components/schemas/ResourceNotFoundException' examples: DeleteJob481Example: summary: Default DeleteJob 481 response x-microcks-default: true value: status: SUCCEEDED '482': description: ValidationException content: application/json: schema: $ref: '#/components/schemas/ValidationException' examples: DeleteJob482Example: summary: Default DeleteJob 482 response x-microcks-default: true value: status: SUCCEEDED parameters: - name: name in: path required: true description: The name of the job to be deleted. schema: type: string minLength: 1 maxLength: 240 summary: Amazon Glue DataBrew Delete Job x-microcks-operation: delay: 0 dispatcher: FALLBACK tags: - Jobs get: operationId: DescribeJob description: Returns the definition of a specific DataBrew job. responses: '200': description: Success content: application/json: schema: $ref: '#/components/schemas/DescribeJobResponse' examples: DescribeJob200Example: summary: Default DescribeJob 200 response x-microcks-default: true value: status: SUCCEEDED '480': description: ResourceNotFoundException content: application/json: schema: $ref: '#/components/schemas/ResourceNotFoundException' examples: DescribeJob480Example: summary: Default DescribeJob 480 response x-microcks-default: true value: status: SUCCEEDED '481': description: ValidationException content: application/json: schema: $ref: '#/components/schemas/ValidationException' examples: DescribeJob481Example: summary: Default DescribeJob 481 response x-microcks-default: true value: status: SUCCEEDED parameters: - name: name in: path required: true description: The name of the job to be described. schema: type: string minLength: 1 maxLength: 240 summary: Amazon Glue DataBrew Describe Job x-microcks-operation: delay: 0 dispatcher: FALLBACK tags: - Jobs /jobs/{name}/jobRun/{runId}: parameters: - $ref: '#/components/parameters/X-Amz-Content-Sha256' - $ref: '#/components/parameters/X-Amz-Date' - $ref: '#/components/parameters/X-Amz-Algorithm' - $ref: '#/components/parameters/X-Amz-Credential' - $ref: '#/components/parameters/X-Amz-Security-Token' - $ref: '#/components/parameters/X-Amz-Signature' - $ref: '#/components/parameters/X-Amz-SignedHeaders' get: operationId: DescribeJobRun description: Represents one run of a DataBrew job. responses: '200': description: Success content: application/json: schema: $ref: '#/components/schemas/DescribeJobRunResponse' examples: DescribeJobRun200Example: summary: Default DescribeJobRun 200 response x-microcks-default: true value: status: SUCCEEDED '480': description: ResourceNotFoundException content: application/json: schema: $ref: '#/components/schemas/ResourceNotFoundException' examples: DescribeJobRun480Example: summary: Default DescribeJobRun 480 response x-microcks-default: true value: status: SUCCEEDED '481': description: ValidationException content: application/json: schema: $ref: '#/components/schemas/ValidationException' examples: DescribeJobRun481Example: summary: Default DescribeJobRun 481 response x-microcks-default: true value: status: SUCCEEDED parameters: - name: name in: path required: true description: The name of the job being processed during this run. schema: type: string minLength: 1 maxLength: 240 - name: runId in: path required: true description: The unique identifier of the job run. schema: type: string minLength: 1 maxLength: 255 summary: Amazon Glue DataBrew Describe Job Run x-microcks-operation: delay: 0 dispatcher: FALLBACK tags: - Jobs /jobs/{name}/jobRuns: parameters: - $ref: '#/components/parameters/X-Amz-Content-Sha256' - $ref: '#/components/parameters/X-Amz-Date' - $ref: '#/components/parameters/X-Amz-Algorithm' - $ref: '#/components/parameters/X-Amz-Credential' - $ref: '#/components/parameters/X-Amz-Security-Token' - $ref: '#/components/parameters/X-Amz-Signature' - $ref: '#/components/parameters/X-Amz-SignedHeaders' get: operationId: ListJobRuns description: Lists all of the previous runs of a particular DataBrew job. responses: '200': description: Success content: application/json: schema: $ref: '#/components/schemas/ListJobRunsResponse' examples: ListJobRuns200Example: summary: Default ListJobRuns 200 response x-microcks-default: true value: status: SUCCEEDED '480': description: ResourceNotFoundException content: application/json: schema: $ref: '#/components/schemas/ResourceNotFoundException' examples: ListJobRuns480Example: summary: Default ListJobRuns 480 response x-microcks-default: true value: status: SUCCEEDED '481': description: ValidationException content: application/json: schema: $ref: '#/components/schemas/ValidationException' examples: ListJobRuns481Example: summary: Default ListJobRuns 481 response x-microcks-default: true value: status: SUCCEEDED parameters: - name: name in: path required: true description: The name of the job. schema: type: string minLength: 1 maxLength: 240 - name: maxResults in: query required: false description: 'The maximum number of results to return in this request. ' schema: type: integer minimum: 1 maximum: 100 - name: nextToken in: query required: false description: The token returned by a previous call to retrieve the next set of results. schema: type: string minLength: 1 maxLength: 2000 - name: MaxResults in: query schema: type: string description: Pagination limit required: false - name: NextToken in: query schema: type: string description: Pagination token required: false summary: Amazon Glue DataBrew List Job Runs x-microcks-operation: delay: 0 dispatcher: FALLBACK tags: - Jobs /jobs: parameters: - $ref: '#/components/parameters/X-Amz-Content-Sha256' - $ref: '#/components/parameters/X-Amz-Date' - $ref: '#/components/parameters/X-Amz-Algorithm' - $ref: '#/components/parameters/X-Amz-Credential' - $ref: '#/components/parameters/X-Amz-Security-Token' - $ref: '#/components/parameters/X-Amz-Signature' - $ref: '#/components/parameters/X-Amz-SignedHeaders' get: operationId: ListJobs description: Lists all of the DataBrew jobs that are defined. responses: '200': description: Success content: application/json: schema: $ref: '#/components/schemas/ListJobsResponse' examples: ListJobs200Example: summary: Default ListJobs 200 response x-microcks-default: true value: status: SUCCEEDED '480': description: ValidationException content: application/json: schema: $ref: '#/components/schemas/ValidationException' examples: ListJobs480Example: summary: Default ListJobs 480 response x-microcks-default: true value: status: SUCCEEDED parameters: - name: datasetName in: query required: false description: The name of a dataset. Using this parameter indicates to return only those jobs that act on the specified dataset. schema: type: string minLength: 1 maxLength: 255 - name: maxResults in: query required: false description: 'The maximum number of results to return in this request. ' schema: type: integer minimum: 1 maximum: 100 - name: nextToken in: query required: false description: 'A token generated by DataBrew that specifies where to continue pagination if a previous request was truncated. To get the next set of pages, pass in the NextToken value from the response object of the previous page call. ' schema: type: string minLength: 1 maxLength: 2000 - name: projectName in: query required: false description: The name of a project. Using this parameter indicates to return only those jobs that are associated with the specified project. schema: type: string minLength: 1 maxLength: 255 - name: MaxResults in: query schema: type: string description: Pagination limit required: false - name: NextToken in: query schema: type: string description: Pagination token required: false summary: Amazon Glue DataBrew List Jobs x-microcks-operation: delay: 0 dispatcher: FALLBACK tags: - Jobs /jobs/{name}/startJobRun: parameters: - $ref: '#/components/parameters/X-Amz-Content-Sha256' - $ref: '#/components/parameters/X-Amz-Date' - $ref: '#/components/parameters/X-Amz-Algorithm' - $ref: '#/components/parameters/X-Amz-Credential' - $ref: '#/components/parameters/X-Amz-Security-Token' - $ref: '#/components/parameters/X-Amz-Signature' - $ref: '#/components/parameters/X-Amz-SignedHeaders' post: operationId: StartJobRun description: Runs a DataBrew job. responses: '200': description: Success content: application/json: schema: $ref: '#/components/schemas/StartJobRunResponse' examples: StartJobRun200Example: summary: Default StartJobRun 200 response x-microcks-default: true value: status: SUCCEEDED '480': description: ConflictException content: application/json: schema: $ref: '#/components/schemas/ConflictException' examples: StartJobRun480Example: summary: Default StartJobRun 480 response x-microcks-default: true value: status: SUCCEEDED '481': description: ResourceNotFoundException content: application/json: schema: $ref: '#/components/schemas/ResourceNotFoundException' examples: StartJobRun481Example: summary: Default StartJobRun 481 response x-microcks-default: true value: status: SUCCEEDED '482': description: ServiceQuotaExceededException content: application/json: schema: $ref: '#/components/schemas/ServiceQuotaExceededException' examples: StartJobRun482Example: summary: Default StartJobRun 482 response x-microcks-default: true value: status: SUCCEEDED '483': description: ValidationException content: application/json: schema: $ref: '#/components/schemas/ValidationException' examples: StartJobRun483Example: summary: Default StartJobRun 483 response x-microcks-default: true value: status: SUCCEEDED parameters: - name: name in: path required: true description: The name of the job to be run. schema: type: string minLength: 1 maxLength: 240 summary: Amazon Glue DataBrew Start Job Run x-microcks-operation: delay: 0 dispatcher: FALLBACK tags: - Jobs /jobs/{name}/jobRun/{runId}/stopJobRun: parameters: - $ref: '#/components/parameters/X-Amz-Content-Sha256' - $ref: '#/components/parameters/X-Amz-Date' - $ref: '#/components/parameters/X-Amz-Algorithm' - $ref: '#/components/parameters/X-Amz-Credential' - $ref: '#/components/parameters/X-Amz-Security-Token' - $ref: '#/components/parameters/X-Amz-Signature' - $ref: '#/components/parameters/X-Amz-SignedHeaders' post: operationId: StopJobRun description: Stops a particular run of a job. responses: '200': description: Success content: application/json: schema: $ref: '#/components/schemas/StopJobRunResponse' examples: StopJobRun200Example: summary: Default StopJobRun 200 response x-microcks-default: true value: status: SUCCEEDED '480': description: ResourceNotFoundException content: application/json: schema: $ref: '#/components/schemas/ResourceNotFoundException' examples: StopJobRun480Example: summary: Default StopJobRun 480 response x-microcks-default: true value: status: SUCCEEDED '481': description: ValidationException content: application/json: schema: $ref: '#/components/schemas/ValidationException' examples: StopJobRun481Example: summary: Default StopJobRun 481 response x-microcks-default: true value: status: SUCCEEDED parameters: - name: name in: path required: true description: The name of the job to be stopped. schema: type: string minLength: 1 maxLength: 240 - name: runId in: path required: true description: The ID of the job run to be stopped. schema: type: string minLength: 1 maxLength: 255 summary: Amazon Glue DataBrew Stop Job Run x-microcks-operation: delay: 0 dispatcher: FALLBACK tags: - Jobs components: schemas: EncryptionMode: type: string enum: - SSE-KMS - SSE-S3 OutputFormatOptions: type: object properties: Csv: allOf: - $ref: '#/components/schemas/CsvOutputOptions' - description: Represents a set of options that define the structure of comma-separated value (CSV) job output. description: Represents a set of options that define the structure of comma-separated (CSV) job output. StatisticsConfiguration: type: object properties: IncludedStatistics: allOf: - $ref: '#/components/schemas/StatisticList' - description: List of included evaluations. When the list is undefined, all supported evaluations will be included. Overrides: allOf: - $ref: '#/components/schemas/StatisticOverrideList' - description: List of overrides for evaluations. description: 'Configuration of evaluations for a profile job. This configuration can be used to select evaluations and override the parameters of selected evaluations. ' StatisticOverrideList: type: array items: $ref: '#/components/schemas/StatisticOverride' minItems: 1 SampleMode: type: string enum: - FULL_DATASET - CUSTOM_ROWS DatabaseTableOutputOptions: type: object required: - TableName properties: TempDirectory: allOf: - $ref: '#/components/schemas/S3Location' - description: Represents an Amazon S3 location (bucket name and object key) where DataBrew can store intermediate results. TableName: allOf: - $ref: '#/components/schemas/DatabaseTableName' - description: A prefix for the name of a table DataBrew will create in the database. description: Represents options that specify how and where DataBrew writes the database output generated by recipe jobs. AccountId: type: string maxLength: 255 DataCatalogOutput: type: object required: - DatabaseName - TableName properties: CatalogId: allOf: - $ref: '#/components/schemas/CatalogId' - description: The unique identifier of the Amazon Web Services account that holds the Data Catalog that stores the data. DatabaseName: allOf: - $ref: '#/components/schemas/DatabaseName' - description: The name of a database in the Data Catalog. TableName: allOf: - $ref: '#/components/schemas/TableName' - description: The name of a table in the Data Catalog. S3Options: allOf: - $ref: '#/components/schemas/S3TableOutputOptions' - description: Represents options that specify how and where DataBrew writes the Amazon S3 output generated by recipe jobs. DatabaseOptions: allOf: - $ref: '#/components/schemas/DatabaseTableOutputOptions' - description: Represents options that specify how and where DataBrew writes the database output generated by recipe jobs. Overwrite: allOf: - $ref: '#/components/schemas/OverwriteOutput' - description: A value that, if true, means that any data in the location specified for output is overwritten with new output. Not supported with DatabaseOptions. description: Represents options that specify how and where in the Glue Data Catalog DataBrew writes the output generated by recipe jobs. TagValue: type: string maxLength: 256 CsvOutputOptions: type: object properties: Delimiter: allOf: - $ref: '#/components/schemas/Delimiter' - description: A single character that specifies the delimiter used to create CSV job output. description: Represents a set of options that define how DataBrew will write a comma-separated value (CSV) file. DescribeJobRunResponse: type: object required: - JobName properties: Attempt: allOf: - $ref: '#/components/schemas/Attempt' - description: The number of times that DataBrew has attempted to run the job. CompletedOn: allOf: - $ref: '#/components/schemas/Date' - description: The date and time when the job completed processing. DatasetName: allOf: - $ref: '#/components/schemas/DatasetName' - description: The name of the dataset for the job to process. ErrorMessage: allOf: - $ref: '#/components/schemas/JobRunErrorMessage' - description: A message indicating an error (if any) that was encountered when the job ran. ExecutionTime: allOf: - $ref: '#/components/schemas/ExecutionTime' - description: The amount of time, in seconds, during which the job run consumed resources. JobName: allOf: - $ref: '#/components/schemas/JobName' - description: The name of the job being processed during this run. ProfileConfiguration: allOf: - $ref: '#/components/schemas/ProfileConfiguration' - description: Configuration for profile jobs. Used to select columns, do evaluations, and override default parameters of evaluations. When configuration is null, the profile job will run with default settings. ValidationConfigurations: allOf: - $ref: '#/components/schemas/ValidationConfigurationList' - description: List of validation configurations that are applied to the profile job. RunId: allOf: - $ref: '#/components/schemas/JobRunId' - description: The unique identifier of the job run. State: allOf: - $ref: '#/components/schemas/JobRunState' - description: The current state of the job run entity itself. LogSubscription: allOf: - $ref: '#/components/schemas/LogSubscription' - description: The current status of Amazon CloudWatch logging for the job run. LogGroupName: allOf: - $ref: '#/components/schemas/LogGroupName' - description: The name of an Amazon CloudWatch log group, where the job writes diagnostic messages when it runs. Outputs: allOf: - $ref: '#/components/schemas/OutputList' - description: One or more output artifacts from a job run. DataCatalogOutputs: allOf: - $ref: '#/components/schemas/DataCatalogOutputList' - description: One or more artifacts that represent the Glue Data Catalog output from running the job. DatabaseOutputs: allOf: - $ref: '#/components/schemas/DatabaseOutputList' - description: Represents a list of JDBC database output objects which defines the output destination for a DataBrew recipe job to write into. RecipeReference: $ref: '#/components/schemas/RecipeReference' StartedBy: allOf: - $ref: '#/components/schemas/StartedBy' - description: The Amazon Resource Name (ARN) of the user who started the job run. StartedOn: allOf: - $ref: '#/components/schemas/Date' - description: The date and time when the job run began. JobSample: allOf: - $ref: '#/components/schemas/JobSample' - description: Sample configuration for profile jobs only. Determines the number of rows on which the profile job will be executed. If a JobSample value is not provided, the default value will be used. The default value is CUSTOM_ROWS for the mode parameter and 20000 for the size parameter. DescribeJobResponse: type: object required: - Name properties: CreateDate: allOf: - $ref: '#/components/schemas/Date' - description: The date and time that the job was created. CreatedBy: allOf: - $ref: '#/components/schemas/CreatedBy' - description: The identifier (user name) of the user associated with the creation of the job. DatasetName: allOf: - $ref: '#/components/schemas/DatasetName' - description: The dataset that the job acts upon. EncryptionKeyArn: allOf: - $ref: '#/components/schemas/EncryptionKeyArn' - description: The Amazon Resource Name (ARN) of an encryption key that is used to protect the job. EncryptionMode: allOf: - $ref: '#/components/schemas/EncryptionMode' - description:
The encryption mode for the job, which can be one of the following:
SSE-KMS - Server-side encryption with keys managed by KMS.
SSE-S3 - Server-side encryption with keys managed by Amazon S3.
The job type, which must be one of the following:
PROFILE - The job analyzes the dataset to determine its size, data types, data distribution, and more.
RECIPE - The job applies one or more transformations to a dataset.
TIMEOUT.
JobSample:
allOf:
- $ref: '#/components/schemas/JobSample'
- description: Sample configuration for profile jobs only. Determines the number of rows on which the profile job will be executed.
ValidationConfigurationList:
type: array
items:
$ref: '#/components/schemas/ValidationConfiguration'
minItems: 1
ValidationMode:
type: string
enum:
- CHECK_ALL
ParameterMap:
type: object
additionalProperties:
$ref: '#/components/schemas/ParameterValue'
JobRun:
type: object
properties:
Attempt:
allOf:
- $ref: '#/components/schemas/Attempt'
- description: The number of times that DataBrew has attempted to run the job.
CompletedOn:
allOf:
- $ref: '#/components/schemas/Date'
- description: The date and time when the job completed processing.
DatasetName:
allOf:
- $ref: '#/components/schemas/DatasetName'
- description: The name of the dataset for the job to process.
ErrorMessage:
allOf:
- $ref: '#/components/schemas/JobRunErrorMessage'
- description: A message indicating an error (if any) that was encountered when the job ran.
ExecutionTime:
allOf:
- $ref: '#/components/schemas/ExecutionTime'
- description: The amount of time, in seconds, during which a job run consumed resources.
JobName:
allOf:
- $ref: '#/components/schemas/JobName'
- description: The name of the job being processed during this run.
RunId:
allOf:
- $ref: '#/components/schemas/JobRunId'
- description: The unique identifier of the job run.
State:
allOf:
- $ref: '#/components/schemas/JobRunState'
- description: The current state of the job run entity itself.
LogSubscription:
allOf:
- $ref: '#/components/schemas/LogSubscription'
- description: The current status of Amazon CloudWatch logging for the job run.
LogGroupName:
allOf:
- $ref: '#/components/schemas/LogGroupName'
- description: The name of an Amazon CloudWatch log group, where the job writes diagnostic messages when it runs.
Outputs:
allOf:
- $ref: '#/components/schemas/OutputList'
- description: One or more output artifacts from a job run.
DataCatalogOutputs:
allOf:
- $ref: '#/components/schemas/DataCatalogOutputList'
- description: One or more artifacts that represent the Glue Data Catalog output from running the job.
DatabaseOutputs:
allOf:
- $ref: '#/components/schemas/DatabaseOutputList'
- description: Represents a list of JDBC database output objects which defines the output destination for a DataBrew recipe job to write into.
RecipeReference:
allOf:
- $ref: '#/components/schemas/RecipeReference'
- description: The set of steps processed by the job.
StartedBy:
allOf:
- $ref: '#/components/schemas/StartedBy'
- description: 'The Amazon Resource Name (ARN) of the user who initiated the job run. '
StartedOn:
allOf:
- $ref: '#/components/schemas/Date'
- description: 'The date and time when the job run began. '
JobSample:
allOf:
- $ref: '#/components/schemas/JobSample'
- description: A sample configuration for profile jobs only, which determines the number of rows on which the profile job is run. If a JobSample value isn't provided, the default is used. The default value is CUSTOM_ROWS for the mode parameter and 20,000 for the size parameter.
ValidationConfigurations:
allOf:
- $ref: '#/components/schemas/ValidationConfigurationList'
- description: List of validation configurations that are applied to the profile job run.
description: Represents one run of a DataBrew job.
DatasetName:
type: string
minLength: 1
maxLength: 255
Job:
type: object
required:
- Name
properties:
AccountId:
allOf:
- $ref: '#/components/schemas/AccountId'
- description: The ID of the Amazon Web Services account that owns the job.
CreatedBy:
allOf:
- $ref: '#/components/schemas/CreatedBy'
- description: The Amazon Resource Name (ARN) of the user who created the job.
CreateDate:
allOf:
- $ref: '#/components/schemas/Date'
- description: The date and time that the job was created.
DatasetName:
allOf:
- $ref: '#/components/schemas/DatasetName'
- description: A dataset that the job is to process.
EncryptionKeyArn:
allOf:
- $ref: '#/components/schemas/EncryptionKeyArn'
- description: 'The Amazon Resource Name (ARN) of an encryption key that is used to protect the job output. For more information, see Encrypting data written by DataBrew jobs '
EncryptionMode:
allOf:
- $ref: '#/components/schemas/EncryptionMode'
- description: The encryption mode for the job, which can be one of the following:
SSE-KMS - Server-side encryption with keys managed by KMS.
SSE-S3 - Server-side encryption with keys managed by Amazon S3.
The job type of the job, which must be one of the following:
PROFILE - A job to analyze a dataset, to determine its size, data types, data distribution, and more.
RECIPE - A job to apply one or more transformations to a dataset.
TIMEOUT.
Tags:
allOf:
- $ref: '#/components/schemas/TagMap'
- description: Metadata tags that have been applied to the job.
JobSample:
allOf:
- $ref: '#/components/schemas/JobSample'
- description: A sample configuration for profile jobs only, which determines the number of rows on which the profile job is run. If a JobSample value isn't provided, the default value is used. The default value is CUSTOM_ROWS for the mode parameter and 20,000 for the size parameter.
ValidationConfigurations:
allOf:
- $ref: '#/components/schemas/ValidationConfigurationList'
- description: List of validation configurations that are applied to the profile job.
description: Represents all of the attributes of a DataBrew job.
DatabaseOutputMode:
type: string
enum:
- NEW_TABLE
Date:
type: string
format: date-time
TableName:
type: string
minLength: 1
maxLength: 255
DatabaseOutput:
type: object
required:
- GlueConnectionName
- DatabaseOptions
properties:
GlueConnectionName:
allOf:
- $ref: '#/components/schemas/GlueConnectionName'
- description: The Glue connection that stores the connection information for the target database.
DatabaseOptions:
allOf:
- $ref: '#/components/schemas/DatabaseTableOutputOptions'
- description: Represents options that specify how and where DataBrew writes the database output generated by recipe jobs.
DatabaseOutputMode:
allOf:
- $ref: '#/components/schemas/DatabaseOutputMode'
- description: 'The output mode to write into the database. Currently supported option: NEW_TABLE.'
description: Represents a JDBC database output object which defines the output destination for a DataBrew recipe job to write into.
ColumnName:
type: string
minLength: 1
maxLength: 255
ResourceNotFoundException: {}
JobSize:
type: integer
CreatedBy:
type: string
OverwriteOutput:
type: boolean
StatisticList:
type: array
items:
$ref: '#/components/schemas/Statistic'
minItems: 1
Output:
type: object
required:
- Location
properties:
CompressionFormat:
allOf:
- $ref: '#/components/schemas/CompressionFormat'
- description: The compression algorithm used to compress the output text of the job.
Format:
allOf:
- $ref: '#/components/schemas/OutputFormat'
- description: The data format of the output of the job.
PartitionColumns:
allOf:
- $ref: '#/components/schemas/ColumnNameList'
- description: The names of one or more partition columns for the output of the job.
Location:
allOf:
- $ref: '#/components/schemas/S3Location'
- description: The location in Amazon S3 where the job writes its output.
Overwrite:
allOf:
- $ref: '#/components/schemas/OverwriteOutput'
- description: A value that, if true, means that any data in the location specified for output is overwritten with new output.
FormatOptions:
allOf:
- $ref: '#/components/schemas/OutputFormatOptions'
- description: Represents options that define how DataBrew formats job output files.
MaxOutputFiles:
allOf:
- $ref: '#/components/schemas/MaxOutputFiles'
- description: Maximum number of files to be generated by the job and written to the output folder. For output partitioned by column(s), the MaxOutputFiles value is the maximum number of files per partition.
description: Represents options that specify how and where in Amazon S3 DataBrew writes the output generated by recipe jobs or profile jobs.
DatabaseName:
type: string
minLength: 1
maxLength: 255
TagMap:
type: object
minProperties: 1
maxProperties: 200
additionalProperties:
$ref: '#/components/schemas/TagValue'
RecipeName:
type: string
minLength: 1
maxLength: 255
EntityTypeList:
type: array
items:
$ref: '#/components/schemas/EntityType'
minItems: 1
Statistic:
type: string
pattern: ^[A-Z\_]+$
minLength: 1
maxLength: 128
DatabaseTableName:
type: string
minLength: 1
maxLength: 255
BucketOwner:
type: string
pattern: ^[0-9]{12}$
minLength: 12
maxLength: 12
ConflictException: {}
CompressionFormat:
type: string
enum:
- GZIP
- LZ4
- SNAPPY
- BZIP2
- DEFLATE
- LZO
- BROTLI
- ZSTD
- ZLIB
JobRunErrorMessage:
type: string
Bucket:
type: string
minLength: 3
maxLength: 63
RecipeReference:
type: object
required:
- Name
properties:
Name:
allOf:
- $ref: '#/components/schemas/RecipeName'
- description: The name of the recipe.
RecipeVersion:
allOf:
- $ref: '#/components/schemas/RecipeVersion'
- description: 'The identifier for the version for the recipe. '
description: Represents the name and version of a DataBrew recipe.
ProfileConfiguration:
type: object
properties:
DatasetStatisticsConfiguration:
allOf:
- $ref: '#/components/schemas/StatisticsConfiguration'
- description: 'Configuration for inter-column evaluations. Configuration can be used to select evaluations and override parameters of evaluations. When configuration is undefined, the profile job will run all supported inter-column evaluations. '
ProfileColumns:
allOf:
- $ref: '#/components/schemas/ColumnSelectorList'
- description: 'List of column selectors. ProfileColumns can be used to select columns from the dataset. When ProfileColumns is undefined, the profile job will profile all supported columns. '
ColumnStatisticsConfigurations:
allOf:
- $ref: '#/components/schemas/ColumnStatisticsConfigurationList'
- description: 'List of configurations for column evaluations. ColumnStatisticsConfigurations are used to select evaluations and override parameters of evaluations for particular columns. When ColumnStatisticsConfigurations is undefined, the profile job will profile all supported columns and run all supported evaluations. '
EntityDetectorConfiguration:
allOf:
- $ref: '#/components/schemas/EntityDetectorConfiguration'
- description: Configuration of entity detection for a profile job. When undefined, entity detection is disabled.
description: 'Configuration for profile jobs. Configuration can be used to select columns, do evaluations, and override default parameters of evaluations. When configuration is undefined, the profile job will apply default settings to all supported columns. '
Key:
type: string
minLength: 1
maxLength: 1280
JobName:
type: string
minLength: 1
maxLength: 240
LastModifiedBy:
type: string
StopJobRunResponse:
type: object
required:
- RunId
properties:
RunId:
allOf:
- $ref: '#/components/schemas/JobRunId'
- description: The ID of the job run that you stopped.
ValidationConfiguration:
type: object
required:
- RulesetArn
properties:
RulesetArn:
allOf:
- $ref: '#/components/schemas/Arn'
- description: The Amazon Resource Name (ARN) for the ruleset to be validated in the profile job. The TargetArn of the selected ruleset should be the same as the Amazon Resource Name (ARN) of the dataset that is associated with the profile job.
ValidationMode:
allOf:
- $ref: '#/components/schemas/ValidationMode'
- description: Mode of data quality validation. Default mode is “CHECK_ALL” which verifies all rules defined in the selected ruleset.
description: Configuration for data quality validation. Used to select the Rulesets and Validation Mode to be used in the profile job. When ValidationConfiguration is null, the profile job will run without data quality validation.
ColumnStatisticsConfigurationList:
type: array
items:
$ref: '#/components/schemas/ColumnStatisticsConfiguration'
minItems: 1
GlueConnectionName:
type: string
minLength: 1
maxLength: 255
StartedBy:
type: string
S3TableOutputOptions:
type: object
required:
- Location
properties:
Location:
allOf:
- $ref: '#/components/schemas/S3Location'
- description: Represents an Amazon S3 location (bucket name and object key) where DataBrew can write output from a job.
description: Represents options that specify how and where DataBrew writes the Amazon S3 output generated by recipe jobs.
MaxOutputFiles:
type: integer
minimum: 1
maximum: 999
ListJobsResponse:
type: object
required:
- Jobs
properties:
Jobs:
allOf:
- $ref: '#/components/schemas/JobList'
- description: A list of jobs that are defined.
NextToken:
allOf:
- $ref: '#/components/schemas/NextToken'
- description: A token that you can use in a subsequent call to retrieve the next set of results.
ColumnSelectorList:
type: array
items:
$ref: '#/components/schemas/ColumnSelector'
minItems: 1
OutputList:
type: array
items:
$ref: '#/components/schemas/Output'
minItems: 1
ExecutionTime:
type: integer
Delimiter:
type: string
minLength: 1
maxLength: 1
EncryptionKeyArn:
type: string
minLength: 20
maxLength: 2048
StatisticOverride:
type: object
required:
- Statistic
- Parameters
properties:
Statistic:
allOf:
- $ref: '#/components/schemas/Statistic'
- description: The name of an evaluation
Parameters:
allOf:
- $ref: '#/components/schemas/ParameterMap'
- description: A map that includes overrides of an evaluation’s parameters.
description: 'Override of a particular evaluation for a profile job. '
ColumnSelector:
type: object
properties:
Regex:
allOf:
- $ref: '#/components/schemas/ColumnName'
- description: A regular expression for selecting a column from a dataset.
Name:
allOf:
- $ref: '#/components/schemas/ColumnName'
- description: The name of a column from a dataset.
description: Selector of a column from a dataset for profile job configuration. One selector includes either a column name or a regular expression.
ServiceQuotaExceededException: {}
ColumnNameList:
type: array
items:
$ref: '#/components/schemas/ColumnName'
maxItems: 200
LogGroupName:
type: string
minLength: 1
maxLength: 512
AllowedStatisticList:
type: array
items:
$ref: '#/components/schemas/AllowedStatistics'
minItems: 1
JobRunState:
type: string
enum:
- STARTING
- RUNNING
- STOPPING
- STOPPED
- SUCCEEDED
- FAILED
- TIMEOUT
JobRunList:
type: array
items:
$ref: '#/components/schemas/JobRun'
DeleteJobResponse:
type: object
required:
- Name
properties:
Name:
allOf:
- $ref: '#/components/schemas/JobName'
- description: The name of the job that you deleted.
Arn:
type: string
minLength: 20
maxLength: 2048
NextToken:
type: string
minLength: 1
maxLength: 2000
ParameterValue:
type: string
minLength: 1
maxLength: 32768
ValidationException: {}
StartJobRunResponse:
type: object
required:
- RunId
properties:
RunId:
allOf:
- $ref: '#/components/schemas/JobRunId'
- description: A system-generated identifier for this particular job run.
OutputFormat:
type: string
enum:
- CSV
- JSON
- PARQUET
- GLUEPARQUET
- AVRO
- ORC
- XML
- TABLEAUHYPER
ListJobRunsResponse:
type: object
required:
- JobRuns
properties:
JobRuns:
allOf:
- $ref: '#/components/schemas/JobRunList'
- description: A list of job runs that have occurred for the specified job.
NextToken:
allOf:
- $ref: '#/components/schemas/NextToken'
- description: A token that you can use in a subsequent call to retrieve the next set of results.
ColumnStatisticsConfiguration:
type: object
required:
- Statistics
properties:
Selectors:
allOf:
- $ref: '#/components/schemas/ColumnSelectorList'
- description: 'List of column selectors. Selectors can be used to select columns from the dataset. When selectors are undefined, configuration will be applied to all supported columns. '
Statistics:
allOf:
- $ref: '#/components/schemas/StatisticsConfiguration'
- description: 'Configuration for evaluations. Statistics can be used to select evaluations and override parameters of evaluations. '
description: 'Configuration for column evaluations for a profile job. ColumnStatisticsConfiguration can be used to select evaluations and override parameters of evaluations for particular columns. '
JobType:
type: string
enum:
- PROFILE
- RECIPE
ProjectName:
type: string
minLength: 1
maxLength: 255
Timeout:
type: integer
minimum: 0
MaxRetries:
type: integer
minimum: 0
EntityType:
type: string
pattern: ^[A-Z_][A-Z\\d_]*$
minLength: 1
maxLength: 128
RecipeVersion:
type: string
minLength: 1
maxLength: 16
EntityDetectorConfiguration:
type: object
required:
- EntityTypes
properties:
EntityTypes:
allOf:
- $ref: '#/components/schemas/EntityTypeList'
- description: Entity types to detect. Can be any of the following:
USA_SSN
USA_ITIN
USA_PASSPORT_NUMBER
PHONE_NUMBER
USA_DRIVING_LICENSE
BANK_ACCOUNT
CREDIT_CARD
IP_ADDRESS
MAC_ADDRESS
USA_DEA_NUMBER
USA_HCPCS_CODE
USA_NATIONAL_PROVIDER_IDENTIFIER
USA_NATIONAL_DRUG_CODE
USA_HEALTH_INSURANCE_CLAIM_NUMBER
USA_MEDICARE_BENEFICIARY_IDENTIFIER
USA_CPT_CODE
PERSON_NAME
DATE
The Entity type group USA_ALL is also supported, and includes all of the above entity types except PERSON_NAME and DATE.
AllowedStatistics: allOf: - $ref: '#/components/schemas/AllowedStatisticList' - description: Configuration of statistics that are allowed to be run on columns that contain detected entities. When undefined, no statistics will be computed on columns that contain detected entities. description: Configuration of entity detection for a profile job. When undefined, entity detection is disabled. AllowedStatistics: type: object required: - Statistics properties: Statistics: allOf: - $ref: '#/components/schemas/StatisticList' - description: One or more column statistics to allow for columns that contain detected entities. description: Configuration of statistics that are allowed to be run on columns that contain detected entities. When undefined, no statistics will be computed on columns that contain detected entities. DataCatalogOutputList: type: array items: $ref: '#/components/schemas/DataCatalogOutput' minItems: 1 S3Location: type: object required: - Bucket properties: Bucket: allOf: - $ref: '#/components/schemas/Bucket' - description: The Amazon S3 bucket name. Key: allOf: - $ref: '#/components/schemas/Key' - description: The unique name of the object in the bucket. BucketOwner: allOf: - $ref: '#/components/schemas/BucketOwner' - description: The Amazon Web Services account ID of the bucket owner. description: Represents an Amazon S3 location (bucket name, bucket owner, and object key) where DataBrew can read input data, or write output from a job. DatabaseOutputList: type: array items: $ref: '#/components/schemas/DatabaseOutput' minItems: 1 LogSubscription: type: string enum: - ENABLE - DISABLE JobRunId: type: string minLength: 1 maxLength: 255 JobSample: type: object properties: Mode: allOf: - $ref: '#/components/schemas/SampleMode' - description:A value that determines whether the profile job is run on the entire dataset or a specified number of rows. This value must be one of the following:
FULL_DATASET - The profile job is run on the entire dataset.
CUSTOM_ROWS - The profile job is run on the number of rows specified in the Size parameter.
The Size parameter is only required when the mode is CUSTOM_ROWS. The profile job is run on the specified number of rows. The maximum value for size is Long.MAX_VALUE.
Long.MAX_VALUE = 9223372036854775807
description: A sample configuration for profile jobs only, which determines the number of rows on which the profile job is run. If aJobSample value isn't provided, the default is used. The default value is CUSTOM_ROWS for the mode parameter and 20,000 for the size parameter.
JobList:
type: array
items:
$ref: '#/components/schemas/Job'
MaxCapacity:
type: integer
Attempt:
type: integer
CatalogId:
type: string
minLength: 1
maxLength: 255
parameters:
X-Amz-Security-Token:
name: X-Amz-Security-Token
in: header
schema:
type: string
required: false
X-Amz-Signature:
name: X-Amz-Signature
in: header
schema:
type: string
required: false
X-Amz-Credential:
name: X-Amz-Credential
in: header
schema:
type: string
required: false
X-Amz-Content-Sha256:
name: X-Amz-Content-Sha256
in: header
schema:
type: string
required: false
X-Amz-Date:
name: X-Amz-Date
in: header
schema:
type: string
required: false
X-Amz-SignedHeaders:
name: X-Amz-SignedHeaders
in: header
schema:
type: string
required: false
X-Amz-Algorithm:
name: X-Amz-Algorithm
in: header
schema:
type: string
required: false
securitySchemes:
hmac:
type: apiKey
name: Authorization
in: header
description: Amazon Signature authorization v4
x-amazon-apigateway-authtype: awsSigv4
externalDocs:
description: Amazon Web Services documentation
url: https://docs.aws.amazon.com/databrew/
x-hasEquivalentPaths: true