openapi: 3.0.0 info: version: 2017-07-25 x-release: v4 title: AWS Glue DataBrew Datasets ProfileJobs API description: Glue DataBrew is a visual, cloud-scale data-preparation service. DataBrew simplifies data preparation tasks, targeting data issues that are hard to spot and time-consuming to fix. DataBrew empowers users of all technical levels to visualize the data and perform one-click data transformations, with no coding required. x-logo: url: https://api.apis.guru/v2/cache/logo/https_twitter.com_awscloud_profile_image.png backgroundColor: '#FFFFFF' termsOfService: https://aws.amazon.com/service-terms/ contact: name: Mike Ralphson email: mike.ralphson@gmail.com url: https://github.com/mermade/aws2openapi x-twitter: PermittedSoc license: name: Apache 2.0 License url: http://www.apache.org/licenses/ x-providerName: amazonaws.com x-serviceName: databrew x-aws-signingName: databrew x-origin: - contentType: application/json url: https://raw.githubusercontent.com/aws/aws-sdk-js/master/apis/databrew-2017-07-25.normal.json converter: url: https://github.com/mermade/aws2openapi version: 1.0.0 x-apisguru-driver: external x-apiClientRegistration: url: https://portal.aws.amazon.com/gp/aws/developer/registration/index.html?nc2=h_ct x-apisguru-categories: - cloud x-preferred: true servers: - url: http://databrew.{region}.amazonaws.com variables: region: description: The AWS region enum: - us-east-1 - us-east-2 - us-west-1 - us-west-2 - us-gov-west-1 - us-gov-east-1 - ca-central-1 - eu-north-1 - eu-west-1 - eu-west-2 - eu-west-3 - eu-central-1 - eu-south-1 - af-south-1 - ap-northeast-1 - ap-northeast-2 - ap-northeast-3 - ap-southeast-1 - ap-southeast-2 - ap-east-1 - ap-south-1 - sa-east-1 - me-south-1 default: us-east-1 description: The AWS Glue DataBrew multi-region endpoint - url: https://databrew.{region}.amazonaws.com variables: region: description: The AWS region enum: - us-east-1 - us-east-2 - us-west-1 - us-west-2 - us-gov-west-1 - us-gov-east-1 - ca-central-1 - eu-north-1 - eu-west-1 - eu-west-2 - eu-west-3 - eu-central-1 - eu-south-1 - af-south-1 - ap-northeast-1 - ap-northeast-2 - ap-northeast-3 - ap-southeast-1 - ap-southeast-2 - ap-east-1 - ap-south-1 - sa-east-1 - me-south-1 default: us-east-1 description: The AWS Glue DataBrew multi-region endpoint - url: http://databrew.{region}.amazonaws.com.cn variables: region: description: The AWS region enum: - cn-north-1 - cn-northwest-1 default: cn-north-1 description: The AWS Glue DataBrew endpoint for China (Beijing) and China (Ningxia) - url: https://databrew.{region}.amazonaws.com.cn variables: region: description: The AWS region enum: - cn-north-1 - cn-northwest-1 default: cn-north-1 description: The AWS Glue DataBrew endpoint for China (Beijing) and China (Ningxia) security: - hmac: [] tags: - name: ProfileJobs paths: /profileJobs: parameters: - $ref: '#/components/parameters/X-Amz-Content-Sha256' - $ref: '#/components/parameters/X-Amz-Date' - $ref: '#/components/parameters/X-Amz-Algorithm' - $ref: '#/components/parameters/X-Amz-Credential' - $ref: '#/components/parameters/X-Amz-Security-Token' - $ref: '#/components/parameters/X-Amz-Signature' - $ref: '#/components/parameters/X-Amz-SignedHeaders' post: operationId: CreateProfileJob description: Creates a new job to analyze a dataset and create its data profile. responses: '200': description: Success content: application/json: schema: $ref: '#/components/schemas/CreateProfileJobResponse' examples: CreateProfileJob200Example: summary: Default CreateProfileJob 200 response x-microcks-default: true value: status: SUCCEEDED '480': description: AccessDeniedException content: application/json: schema: $ref: '#/components/schemas/AccessDeniedException' examples: CreateProfileJob480Example: summary: Default CreateProfileJob 480 response x-microcks-default: true value: status: SUCCEEDED '481': description: ConflictException content: application/json: schema: $ref: '#/components/schemas/ConflictException' examples: CreateProfileJob481Example: summary: Default CreateProfileJob 481 response x-microcks-default: true value: status: SUCCEEDED '482': description: ResourceNotFoundException content: application/json: schema: $ref: '#/components/schemas/ResourceNotFoundException' examples: CreateProfileJob482Example: summary: Default CreateProfileJob 482 response x-microcks-default: true value: status: SUCCEEDED '483': description: ServiceQuotaExceededException content: application/json: schema: $ref: '#/components/schemas/ServiceQuotaExceededException' examples: CreateProfileJob483Example: summary: Default CreateProfileJob 483 response x-microcks-default: true value: status: SUCCEEDED '484': description: ValidationException content: application/json: schema: $ref: '#/components/schemas/ValidationException' examples: CreateProfileJob484Example: summary: Default CreateProfileJob 484 response x-microcks-default: true value: status: SUCCEEDED parameters: [] requestBody: required: true content: application/json: schema: type: object required: - DatasetName - Name - OutputLocation - RoleArn properties: DatasetName: description: The name of the dataset that this job is to act upon. type: string minLength: 1 maxLength: 255 EncryptionKeyArn: description: The Amazon Resource Name (ARN) of an encryption key that is used to protect the job. type: string minLength: 20 maxLength: 2048 EncryptionMode: description:

The encryption mode for the job, which can be one of the following:

type: string enum: - SSE-KMS - SSE-S3 Name: description: The name of the job to be created. Valid characters are alphanumeric (A-Z, a-z, 0-9), hyphen (-), period (.), and space. type: string minLength: 1 maxLength: 240 LogSubscription: description: Enables or disables Amazon CloudWatch logging for the job. If logging is enabled, CloudWatch writes one log stream for each job run. type: string enum: - ENABLE - DISABLE MaxCapacity: description: The maximum number of nodes that DataBrew can use when the job processes data. type: integer MaxRetries: description: The maximum number of times to retry the job after a job run fails. type: integer minimum: 0 OutputLocation: description: Represents an Amazon S3 location (bucket name, bucket owner, and object key) where DataBrew can read input data, or write output from a job. type: object properties: Bucket: allOf: - $ref: '#/components/schemas/Bucket' - description: The Amazon S3 bucket name. Key: allOf: - $ref: '#/components/schemas/Key' - description: The unique name of the object in the bucket. BucketOwner: allOf: - $ref: '#/components/schemas/BucketOwner' - description: The Amazon Web Services account ID of the bucket owner. Configuration: description: 'Configuration for profile jobs. Configuration can be used to select columns, do evaluations, and override default parameters of evaluations. When configuration is undefined, the profile job will apply default settings to all supported columns. ' type: object properties: DatasetStatisticsConfiguration: allOf: - $ref: '#/components/schemas/StatisticsConfiguration' - description: 'Configuration for inter-column evaluations. Configuration can be used to select evaluations and override parameters of evaluations. When configuration is undefined, the profile job will run all supported inter-column evaluations. ' ProfileColumns: allOf: - $ref: '#/components/schemas/ColumnSelectorList' - description: 'List of column selectors. ProfileColumns can be used to select columns from the dataset. When ProfileColumns is undefined, the profile job will profile all supported columns. ' ColumnStatisticsConfigurations: allOf: - $ref: '#/components/schemas/ColumnStatisticsConfigurationList' - description: 'List of configurations for column evaluations. ColumnStatisticsConfigurations are used to select evaluations and override parameters of evaluations for particular columns. When ColumnStatisticsConfigurations is undefined, the profile job will profile all supported columns and run all supported evaluations. ' EntityDetectorConfiguration: allOf: - $ref: '#/components/schemas/EntityDetectorConfiguration' - description: Configuration of entity detection for a profile job. When undefined, entity detection is disabled. ValidationConfigurations: description: List of validation configurations that are applied to the profile job. type: array items: $ref: '#/components/schemas/ValidationConfiguration' minItems: 1 RoleArn: description: The Amazon Resource Name (ARN) of the Identity and Access Management (IAM) role to be assumed when DataBrew runs the job. type: string minLength: 20 maxLength: 2048 Tags: description: Metadata tags to apply to this job. type: object minProperties: 1 maxProperties: 200 additionalProperties: $ref: '#/components/schemas/TagValue' Timeout: description: The job's timeout in minutes. A job that attempts to run longer than this timeout period ends with a status of TIMEOUT. type: integer minimum: 0 JobSample: description: A sample configuration for profile jobs only, which determines the number of rows on which the profile job is run. If a JobSample value isn't provided, the default is used. The default value is CUSTOM_ROWS for the mode parameter and 20,000 for the size parameter. type: object properties: Mode: allOf: - $ref: '#/components/schemas/SampleMode' - description:

A value that determines whether the profile job is run on the entire dataset or a specified number of rows. This value must be one of the following:

Size: allOf: - $ref: '#/components/schemas/JobSize' - description:

The Size parameter is only required when the mode is CUSTOM_ROWS. The profile job is run on the specified number of rows. The maximum value for size is Long.MAX_VALUE.

Long.MAX_VALUE = 9223372036854775807

summary: Amazon Glue DataBrew Create Profile Job x-microcks-operation: delay: 0 dispatcher: FALLBACK tags: - ProfileJobs /profileJobs/{name}: parameters: - $ref: '#/components/parameters/X-Amz-Content-Sha256' - $ref: '#/components/parameters/X-Amz-Date' - $ref: '#/components/parameters/X-Amz-Algorithm' - $ref: '#/components/parameters/X-Amz-Credential' - $ref: '#/components/parameters/X-Amz-Security-Token' - $ref: '#/components/parameters/X-Amz-Signature' - $ref: '#/components/parameters/X-Amz-SignedHeaders' put: operationId: UpdateProfileJob description: Modifies the definition of an existing profile job. responses: '200': description: Success content: application/json: schema: $ref: '#/components/schemas/UpdateProfileJobResponse' examples: UpdateProfileJob200Example: summary: Default UpdateProfileJob 200 response x-microcks-default: true value: status: SUCCEEDED '480': description: AccessDeniedException content: application/json: schema: $ref: '#/components/schemas/AccessDeniedException' examples: UpdateProfileJob480Example: summary: Default UpdateProfileJob 480 response x-microcks-default: true value: status: SUCCEEDED '481': description: ResourceNotFoundException content: application/json: schema: $ref: '#/components/schemas/ResourceNotFoundException' examples: UpdateProfileJob481Example: summary: Default UpdateProfileJob 481 response x-microcks-default: true value: status: SUCCEEDED '482': description: ValidationException content: application/json: schema: $ref: '#/components/schemas/ValidationException' examples: UpdateProfileJob482Example: summary: Default UpdateProfileJob 482 response x-microcks-default: true value: status: SUCCEEDED parameters: - name: name in: path required: true description: The name of the job to be updated. schema: type: string minLength: 1 maxLength: 240 requestBody: required: true content: application/json: schema: type: object required: - OutputLocation - RoleArn properties: Configuration: description: 'Configuration for profile jobs. Configuration can be used to select columns, do evaluations, and override default parameters of evaluations. When configuration is undefined, the profile job will apply default settings to all supported columns. ' type: object properties: DatasetStatisticsConfiguration: allOf: - $ref: '#/components/schemas/StatisticsConfiguration' - description: 'Configuration for inter-column evaluations. Configuration can be used to select evaluations and override parameters of evaluations. When configuration is undefined, the profile job will run all supported inter-column evaluations. ' ProfileColumns: allOf: - $ref: '#/components/schemas/ColumnSelectorList' - description: 'List of column selectors. ProfileColumns can be used to select columns from the dataset. When ProfileColumns is undefined, the profile job will profile all supported columns. ' ColumnStatisticsConfigurations: allOf: - $ref: '#/components/schemas/ColumnStatisticsConfigurationList' - description: 'List of configurations for column evaluations. ColumnStatisticsConfigurations are used to select evaluations and override parameters of evaluations for particular columns. When ColumnStatisticsConfigurations is undefined, the profile job will profile all supported columns and run all supported evaluations. ' EntityDetectorConfiguration: allOf: - $ref: '#/components/schemas/EntityDetectorConfiguration' - description: Configuration of entity detection for a profile job. When undefined, entity detection is disabled. EncryptionKeyArn: description: The Amazon Resource Name (ARN) of an encryption key that is used to protect the job. type: string minLength: 20 maxLength: 2048 EncryptionMode: description:

The encryption mode for the job, which can be one of the following:

type: string enum: - SSE-KMS - SSE-S3 LogSubscription: description: Enables or disables Amazon CloudWatch logging for the job. If logging is enabled, CloudWatch writes one log stream for each job run. type: string enum: - ENABLE - DISABLE MaxCapacity: description: The maximum number of compute nodes that DataBrew can use when the job processes data. type: integer MaxRetries: description: The maximum number of times to retry the job after a job run fails. type: integer minimum: 0 OutputLocation: description: Represents an Amazon S3 location (bucket name, bucket owner, and object key) where DataBrew can read input data, or write output from a job. type: object properties: Bucket: allOf: - $ref: '#/components/schemas/Bucket' - description: The Amazon S3 bucket name. Key: allOf: - $ref: '#/components/schemas/Key' - description: The unique name of the object in the bucket. BucketOwner: allOf: - $ref: '#/components/schemas/BucketOwner' - description: The Amazon Web Services account ID of the bucket owner. ValidationConfigurations: description: List of validation configurations that are applied to the profile job. type: array items: $ref: '#/components/schemas/ValidationConfiguration' minItems: 1 RoleArn: description: The Amazon Resource Name (ARN) of the Identity and Access Management (IAM) role to be assumed when DataBrew runs the job. type: string minLength: 20 maxLength: 2048 Timeout: description: The job's timeout in minutes. A job that attempts to run longer than this timeout period ends with a status of TIMEOUT. type: integer minimum: 0 JobSample: description: A sample configuration for profile jobs only, which determines the number of rows on which the profile job is run. If a JobSample value isn't provided, the default is used. The default value is CUSTOM_ROWS for the mode parameter and 20,000 for the size parameter. type: object properties: Mode: allOf: - $ref: '#/components/schemas/SampleMode' - description:

A value that determines whether the profile job is run on the entire dataset or a specified number of rows. This value must be one of the following:

Size: allOf: - $ref: '#/components/schemas/JobSize' - description:

The Size parameter is only required when the mode is CUSTOM_ROWS. The profile job is run on the specified number of rows. The maximum value for size is Long.MAX_VALUE.

Long.MAX_VALUE = 9223372036854775807

summary: Amazon Glue DataBrew Update Profile Job x-microcks-operation: delay: 0 dispatcher: FALLBACK tags: - ProfileJobs components: schemas: ColumnSelectorList: type: array items: $ref: '#/components/schemas/ColumnSelector' minItems: 1 UpdateProfileJobResponse: type: object required: - Name properties: Name: allOf: - $ref: '#/components/schemas/JobName' - description: The name of the job that was updated. EntityTypeList: type: array items: $ref: '#/components/schemas/EntityType' minItems: 1 ColumnStatisticsConfiguration: type: object required: - Statistics properties: Selectors: allOf: - $ref: '#/components/schemas/ColumnSelectorList' - description: 'List of column selectors. Selectors can be used to select columns from the dataset. When selectors are undefined, configuration will be applied to all supported columns. ' Statistics: allOf: - $ref: '#/components/schemas/StatisticsConfiguration' - description: 'Configuration for evaluations. Statistics can be used to select evaluations and override parameters of evaluations. ' description: 'Configuration for column evaluations for a profile job. ColumnStatisticsConfiguration can be used to select evaluations and override parameters of evaluations for particular columns. ' StatisticsConfiguration: type: object properties: IncludedStatistics: allOf: - $ref: '#/components/schemas/StatisticList' - description: List of included evaluations. When the list is undefined, all supported evaluations will be included. Overrides: allOf: - $ref: '#/components/schemas/StatisticOverrideList' - description: List of overrides for evaluations. description: 'Configuration of evaluations for a profile job. This configuration can be used to select evaluations and override the parameters of selected evaluations. ' StatisticOverrideList: type: array items: $ref: '#/components/schemas/StatisticOverride' minItems: 1 SampleMode: type: string enum: - FULL_DATASET - CUSTOM_ROWS Statistic: type: string pattern: ^[A-Z\_]+$ minLength: 1 maxLength: 128 TagValue: type: string maxLength: 256 EntityType: type: string pattern: ^[A-Z_][A-Z\\d_]*$ minLength: 1 maxLength: 128 BucketOwner: type: string pattern: ^[0-9]{12}$ minLength: 12 maxLength: 12 ConflictException: {} StatisticOverride: type: object required: - Statistic - Parameters properties: Statistic: allOf: - $ref: '#/components/schemas/Statistic' - description: The name of an evaluation Parameters: allOf: - $ref: '#/components/schemas/ParameterMap' - description: A map that includes overrides of an evaluation’s parameters. description: 'Override of a particular evaluation for a profile job. ' EntityDetectorConfiguration: type: object required: - EntityTypes properties: EntityTypes: allOf: - $ref: '#/components/schemas/EntityTypeList' - description:

Entity types to detect. Can be any of the following:

The Entity type group USA_ALL is also supported, and includes all of the above entity types except PERSON_NAME and DATE.

AllowedStatistics: allOf: - $ref: '#/components/schemas/AllowedStatisticList' - description: Configuration of statistics that are allowed to be run on columns that contain detected entities. When undefined, no statistics will be computed on columns that contain detected entities. description: Configuration of entity detection for a profile job. When undefined, entity detection is disabled. CreateProfileJobResponse: type: object required: - Name properties: Name: allOf: - $ref: '#/components/schemas/JobName' - description: The name of the job that was created. ColumnSelector: type: object properties: Regex: allOf: - $ref: '#/components/schemas/ColumnName' - description: A regular expression for selecting a column from a dataset. Name: allOf: - $ref: '#/components/schemas/ColumnName' - description: The name of a column from a dataset. description: Selector of a column from a dataset for profile job configuration. One selector includes either a column name or a regular expression. AllowedStatistics: type: object required: - Statistics properties: Statistics: allOf: - $ref: '#/components/schemas/StatisticList' - description: One or more column statistics to allow for columns that contain detected entities. description: Configuration of statistics that are allowed to be run on columns that contain detected entities. When undefined, no statistics will be computed on columns that contain detected entities. ValidationMode: type: string enum: - CHECK_ALL Bucket: type: string minLength: 3 maxLength: 63 ServiceQuotaExceededException: {} ParameterMap: type: object additionalProperties: $ref: '#/components/schemas/ParameterValue' Key: type: string minLength: 1 maxLength: 1280 JobName: type: string minLength: 1 maxLength: 240 AllowedStatisticList: type: array items: $ref: '#/components/schemas/AllowedStatistics' minItems: 1 ColumnName: type: string minLength: 1 maxLength: 255 ResourceNotFoundException: {} AccessDeniedException: {} JobSize: type: integer Arn: type: string minLength: 20 maxLength: 2048 StatisticList: type: array items: $ref: '#/components/schemas/Statistic' minItems: 1 ParameterValue: type: string minLength: 1 maxLength: 32768 ValidationConfiguration: type: object required: - RulesetArn properties: RulesetArn: allOf: - $ref: '#/components/schemas/Arn' - description: The Amazon Resource Name (ARN) for the ruleset to be validated in the profile job. The TargetArn of the selected ruleset should be the same as the Amazon Resource Name (ARN) of the dataset that is associated with the profile job. ValidationMode: allOf: - $ref: '#/components/schemas/ValidationMode' - description: Mode of data quality validation. Default mode is “CHECK_ALL” which verifies all rules defined in the selected ruleset. description: Configuration for data quality validation. Used to select the Rulesets and Validation Mode to be used in the profile job. When ValidationConfiguration is null, the profile job will run without data quality validation. ColumnStatisticsConfigurationList: type: array items: $ref: '#/components/schemas/ColumnStatisticsConfiguration' minItems: 1 ValidationException: {} parameters: X-Amz-Algorithm: name: X-Amz-Algorithm in: header schema: type: string required: false X-Amz-Credential: name: X-Amz-Credential in: header schema: type: string required: false X-Amz-Content-Sha256: name: X-Amz-Content-Sha256 in: header schema: type: string required: false X-Amz-Security-Token: name: X-Amz-Security-Token in: header schema: type: string required: false X-Amz-SignedHeaders: name: X-Amz-SignedHeaders in: header schema: type: string required: false X-Amz-Date: name: X-Amz-Date in: header schema: type: string required: false X-Amz-Signature: name: X-Amz-Signature in: header schema: type: string required: false securitySchemes: hmac: type: apiKey name: Authorization in: header description: Amazon Signature authorization v4 x-amazon-apigateway-authtype: awsSigv4 externalDocs: description: Amazon Web Services documentation url: https://docs.aws.amazon.com/databrew/ x-hasEquivalentPaths: true