openapi: 3.0.0 info: version: '2017-03-31' x-release: v4 title: 'AWS Glue #X Amz Target=AWSGlue.BatchCreatePartition #X Amz Target=AWSGlue.BatchCreatePartition #X Amz Target=AWSGlue.GetCrawler API' description: Glue

Defines the public endpoint for the Glue service.

x-logo: url: https://api.apis.guru/v2/cache/logo/https_twitter.com_awscloud_profile_image.png backgroundColor: '#FFFFFF' termsOfService: https://aws.amazon.com/service-terms/ contact: name: Mike Ralphson email: mike.ralphson@gmail.com url: https://github.com/mermade/aws2openapi x-twitter: PermittedSoc license: name: Apache 2.0 License url: http://www.apache.org/licenses/ x-providerName: amazonaws.com x-serviceName: glue x-origin: - contentType: application/json url: https://raw.githubusercontent.com/aws/aws-sdk-js/master/apis/glue-2017-03-31.normal.json converter: url: https://github.com/mermade/aws2openapi version: 1.0.0 x-apisguru-driver: external x-apiClientRegistration: url: https://portal.aws.amazon.com/gp/aws/developer/registration/index.html?nc2=h_ct x-apisguru-categories: - cloud x-preferred: true servers: - url: http://glue.{region}.amazonaws.com variables: region: description: The AWS region enum: - us-east-1 - us-east-2 - us-west-1 - us-west-2 - us-gov-west-1 - us-gov-east-1 - ca-central-1 - eu-north-1 - eu-west-1 - eu-west-2 - eu-west-3 - eu-central-1 - eu-south-1 - af-south-1 - ap-northeast-1 - ap-northeast-2 - ap-northeast-3 - ap-southeast-1 - ap-southeast-2 - ap-east-1 - ap-south-1 - sa-east-1 - me-south-1 default: us-east-1 description: The AWS Glue multi-region endpoint - url: https://glue.{region}.amazonaws.com variables: region: description: The AWS region enum: - us-east-1 - us-east-2 - us-west-1 - us-west-2 - us-gov-west-1 - us-gov-east-1 - ca-central-1 - eu-north-1 - eu-west-1 - eu-west-2 - eu-west-3 - eu-central-1 - eu-south-1 - af-south-1 - ap-northeast-1 - ap-northeast-2 - ap-northeast-3 - ap-southeast-1 - ap-southeast-2 - ap-east-1 - ap-south-1 - sa-east-1 - me-south-1 default: us-east-1 description: The AWS Glue multi-region endpoint - url: http://glue.{region}.amazonaws.com.cn variables: region: description: The AWS region enum: - cn-north-1 - cn-northwest-1 default: cn-north-1 description: The AWS Glue endpoint for China (Beijing) and China (Ningxia) - url: https://glue.{region}.amazonaws.com.cn variables: region: description: The AWS region enum: - cn-north-1 - cn-northwest-1 default: cn-north-1 description: The AWS Glue endpoint for China (Beijing) and China (Ningxia) security: - hmac: [] tags: - name: '#X Amz Target=AWSGlue.GetCrawler' paths: /#X-Amz-Target=AWSGlue.GetCrawler: parameters: - $ref: '#/components/parameters/X-Amz-Content-Sha256' - $ref: '#/components/parameters/X-Amz-Date' - $ref: '#/components/parameters/X-Amz-Algorithm' - $ref: '#/components/parameters/X-Amz-Credential' - $ref: '#/components/parameters/X-Amz-Security-Token' - $ref: '#/components/parameters/X-Amz-Signature' - $ref: '#/components/parameters/X-Amz-SignedHeaders' post: operationId: GetCrawler description: Retrieves metadata for a specified crawler. responses: '200': description: Success content: application/json: schema: $ref: '#/components/schemas/GetCrawlerResponse' examples: GetCrawler200Example: summary: Default GetCrawler 200 response x-microcks-default: true value: JobRunId: jr_abc123 State: SUCCEEDED '480': description: EntityNotFoundException content: application/json: schema: $ref: '#/components/schemas/EntityNotFoundException' examples: GetCrawler480Example: summary: Default GetCrawler 480 response x-microcks-default: true value: JobRunId: jr_abc123 State: SUCCEEDED '481': description: OperationTimeoutException content: application/json: schema: $ref: '#/components/schemas/OperationTimeoutException' examples: GetCrawler481Example: summary: Default GetCrawler 481 response x-microcks-default: true value: JobRunId: jr_abc123 State: SUCCEEDED requestBody: required: true content: application/json: schema: $ref: '#/components/schemas/GetCrawlerRequest' parameters: - name: X-Amz-Target in: header required: true schema: type: string enum: - AWSGlue.GetCrawler summary: Amazon Glue Get Crawler x-microcks-operation: delay: 0 dispatcher: FALLBACK tags: - '#X Amz Target=AWSGlue.GetCrawler' components: schemas: ScheduleState: type: string enum: - SCHEDULED - NOT_SCHEDULED - TRANSITIONING DescriptionString: type: string minLength: 0 maxLength: 2048 x-pattern: '[\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\r\n\t]*' MongoDBTarget: type: object properties: ConnectionName: allOf: - $ref: '#/components/schemas/ConnectionName' - description: The name of the connection to use to connect to the Amazon DocumentDB or MongoDB target. Path: allOf: - $ref: '#/components/schemas/Path' - description: The path of the Amazon DocumentDB or MongoDB target (database/collection). ScanAll: allOf: - $ref: '#/components/schemas/NullableBoolean' - description:

Indicates whether to scan all the records, or to sample rows from the table. Scanning all the records can take a long time when the table is not a high throughput table.

A value of true means to scan all records, while a value of false means to sample the records. If no value is specified, the value defaults to true.

description: Specifies an Amazon DocumentDB or MongoDB data store to crawl. CrawlerConfiguration: type: string MongoDBTargetList: type: array items: $ref: '#/components/schemas/MongoDBTarget' NullableInteger: type: integer EntityNotFoundException: {} AccountId: type: string minLength: 0 maxLength: 12 NullableBoolean: type: boolean MessagePrefix: type: string minLength: 1 maxLength: 255 x-pattern: '[\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\t]*' LastCrawlInfo: type: object properties: Status: allOf: - $ref: '#/components/schemas/LastCrawlStatus' - description: Status of the last crawl. ErrorMessage: allOf: - $ref: '#/components/schemas/DescriptionString' - description: If an error occurred, the error information about the last crawl. LogGroup: allOf: - $ref: '#/components/schemas/LogGroup' - description: The log group for the last crawl. LogStream: allOf: - $ref: '#/components/schemas/LogStream' - description: The log stream for the last crawl. MessagePrefix: allOf: - $ref: '#/components/schemas/MessagePrefix' - description: The prefix for a message about this crawl. StartTime: allOf: - $ref: '#/components/schemas/Timestamp' - description: The time at which the crawl started. description: Status and error information about the most recent crawl. Schedule: type: object properties: ScheduleExpression: allOf: - $ref: '#/components/schemas/CronExpression' - description: 'A cron expression used to specify the schedule (see Time-Based Schedules for Jobs and Crawlers. For example, to run something every day at 12:15 UTC, you would specify: cron(15 12 * * ? *).' State: allOf: - $ref: '#/components/schemas/ScheduleState' - description: The state of the schedule. description: A scheduling object using a cron statement to schedule an event. ClassifierNameList: type: array items: $ref: '#/components/schemas/NameString' Role: type: string CrawlerSecurityConfiguration: type: string minLength: 0 maxLength: 128 DatabaseName: type: string JdbcMetadataEntry: type: string enum: - COMMENTS - RAWTYPES Path: type: string CatalogTargetList: type: array items: $ref: '#/components/schemas/CatalogTarget' RecrawlBehavior: type: string enum: - CRAWL_EVERYTHING - CRAWL_NEW_FOLDERS_ONLY - CRAWL_EVENT_MODE TablePrefix: type: string minLength: 0 maxLength: 128 CrawlerState: type: string enum: - READY - RUNNING - STOPPING OperationTimeoutException: {} CatalogTarget: type: object required: - DatabaseName - Tables properties: DatabaseName: allOf: - $ref: '#/components/schemas/NameString' - description: The name of the database to be synchronized. Tables: allOf: - $ref: '#/components/schemas/CatalogTablesList' - description: A list of the tables to be synchronized. ConnectionName: allOf: - $ref: '#/components/schemas/ConnectionName' - description: The name of the connection for an Amazon S3-backed Data Catalog table to be a target of the crawl when using a Catalog connection type paired with a NETWORK Connection type. EventQueueArn: allOf: - $ref: '#/components/schemas/EventQueueArn' - description: A valid Amazon SQS ARN. For example, arn:aws:sqs:region:account:sqs. DlqEventQueueArn: allOf: - $ref: '#/components/schemas/EventQueueArn' - description: A valid Amazon dead-letter SQS ARN. For example, arn:aws:sqs:region:account:deadLetterQueue. description: Specifies an Glue Data Catalog target. CatalogTablesList: type: array items: $ref: '#/components/schemas/NameString' minItems: 1 DynamoDBTarget: type: object properties: Path: allOf: - $ref: '#/components/schemas/Path' - description: The name of the DynamoDB table to crawl. scanAll: allOf: - $ref: '#/components/schemas/NullableBoolean' - description:

Indicates whether to scan all the records, or to sample rows from the table. Scanning all the records can take a long time when the table is not a high throughput table.

A value of true means to scan all records, while a value of false means to sample the records. If no value is specified, the value defaults to true.

scanRate: allOf: - $ref: '#/components/schemas/NullableDouble' - description:

The percentage of the configured read capacity units to use by the Glue crawler. Read capacity units is a term defined by DynamoDB, and is a numeric value that acts as rate limiter for the number of reads that can be performed on that table per second.

The valid values are null or a value between 0.1 to 1.5. A null value is used when user does not provide a value, and defaults to 0.5 of the configured Read Capacity Unit (for provisioned tables), or 0.25 of the max configured Read Capacity Unit (for tables using on-demand mode).

description: Specifies an Amazon DynamoDB table to crawl. VersionId: type: integer LastCrawlStatus: type: string enum: - SUCCEEDED - CANCELLED - FAILED CrawlerLineageSettings: type: string enum: - ENABLE - DISABLE UpdateBehavior: type: string enum: - LOG - UPDATE_IN_DATABASE ConnectionName: type: string DeltaTarget: type: object properties: DeltaTables: allOf: - $ref: '#/components/schemas/PathList' - description: A list of the Amazon S3 paths to the Delta tables. ConnectionName: allOf: - $ref: '#/components/schemas/ConnectionName' - description: The name of the connection to use to connect to the Delta table target. WriteManifest: allOf: - $ref: '#/components/schemas/NullableBoolean' - description: Specifies whether to write the manifest files to the Delta table path. CreateNativeDeltaTable: allOf: - $ref: '#/components/schemas/NullableBoolean' - description: Specifies whether the crawler will create native tables, to allow integration with query engines that support querying of the Delta transaction log directly. description: Specifies a Delta data store to crawl one or more Delta tables. EventQueueArn: type: string MillisecondsCount: type: integer LogStream: type: string pattern: '[^:*]*' minLength: 1 maxLength: 512 RecrawlPolicy: type: object properties: RecrawlBehavior: allOf: - $ref: '#/components/schemas/RecrawlBehavior' - description:

Specifies whether to crawl the entire dataset again or to crawl only folders that were added since the last crawler run.

A value of CRAWL_EVERYTHING specifies crawling the entire dataset again.

A value of CRAWL_NEW_FOLDERS_ONLY specifies crawling only folders that were added since the last crawler run.

A value of CRAWL_EVENT_MODE specifies crawling only the changes identified by Amazon S3 events.

description: When crawling an Amazon S3 data source after the first crawl is complete, specifies whether to crawl the entire dataset again or to crawl only folders that were added since the last crawler run. For more information, see Incremental Crawls in Glue in the developer guide. NullableDouble: type: number format: double JdbcTargetList: type: array items: $ref: '#/components/schemas/JdbcTarget' EnableAdditionalMetadata: type: array items: $ref: '#/components/schemas/JdbcMetadataEntry' SchemaChangePolicy: type: object properties: UpdateBehavior: allOf: - $ref: '#/components/schemas/UpdateBehavior' - description: The update behavior when the crawler finds a changed schema. DeleteBehavior: allOf: - $ref: '#/components/schemas/DeleteBehavior' - description: The deletion behavior when the crawler finds a deleted object. description: A policy that specifies update and deletion behaviors for the crawler. DeleteBehavior: type: string enum: - LOG - DELETE_FROM_DATABASE - DEPRECATE_IN_DATABASE Crawler: type: object properties: Name: allOf: - $ref: '#/components/schemas/NameString' - description: The name of the crawler. Role: allOf: - $ref: '#/components/schemas/Role' - description: The Amazon Resource Name (ARN) of an IAM role that's used to access customer resources, such as Amazon Simple Storage Service (Amazon S3) data. Targets: allOf: - $ref: '#/components/schemas/CrawlerTargets' - description: A collection of targets to crawl. DatabaseName: allOf: - $ref: '#/components/schemas/DatabaseName' - description: The name of the database in which the crawler's output is stored. Description: allOf: - $ref: '#/components/schemas/DescriptionString' - description: A description of the crawler. Classifiers: allOf: - $ref: '#/components/schemas/ClassifierNameList' - description: A list of UTF-8 strings that specify the custom classifiers that are associated with the crawler. RecrawlPolicy: allOf: - $ref: '#/components/schemas/RecrawlPolicy' - description: A policy that specifies whether to crawl the entire dataset again, or to crawl only folders that were added since the last crawler run. SchemaChangePolicy: allOf: - $ref: '#/components/schemas/SchemaChangePolicy' - description: The policy that specifies update and delete behaviors for the crawler. LineageConfiguration: allOf: - $ref: '#/components/schemas/LineageConfiguration' - description: A configuration that specifies whether data lineage is enabled for the crawler. State: allOf: - $ref: '#/components/schemas/CrawlerState' - description: Indicates whether the crawler is running, or whether a run is pending. TablePrefix: allOf: - $ref: '#/components/schemas/TablePrefix' - description: The prefix added to the names of tables that are created. Schedule: allOf: - $ref: '#/components/schemas/Schedule' - description: For scheduled crawlers, the schedule when the crawler runs. CrawlElapsedTime: allOf: - $ref: '#/components/schemas/MillisecondsCount' - description: If the crawler is running, contains the total time elapsed since the last crawl began. CreationTime: allOf: - $ref: '#/components/schemas/Timestamp' - description: The time that the crawler was created. LastUpdated: allOf: - $ref: '#/components/schemas/Timestamp' - description: The time that the crawler was last updated. LastCrawl: allOf: - $ref: '#/components/schemas/LastCrawlInfo' - description: The status of the last crawl, and potentially error information if an error occurred. Version: allOf: - $ref: '#/components/schemas/VersionId' - description: The version of the crawler. Configuration: allOf: - $ref: '#/components/schemas/CrawlerConfiguration' - description: Crawler configuration information. This versioned JSON string allows users to specify aspects of a crawler's behavior. For more information, see Setting crawler configuration options. CrawlerSecurityConfiguration: allOf: - $ref: '#/components/schemas/CrawlerSecurityConfiguration' - description: The name of the SecurityConfiguration structure to be used by this crawler. LakeFormationConfiguration: allOf: - $ref: '#/components/schemas/LakeFormationConfiguration' - description: Specifies whether the crawler should use Lake Formation credentials for the crawler instead of the IAM role credentials. description: Specifies a crawler program that examines a data source and uses classifiers to try to determine its schema. If successful, the crawler records metadata concerning the data source in the Glue Data Catalog. S3Target: type: object properties: Path: allOf: - $ref: '#/components/schemas/Path' - description: The path to the Amazon S3 target. Exclusions: allOf: - $ref: '#/components/schemas/PathList' - description: A list of glob patterns used to exclude from the crawl. For more information, see Catalog Tables with a Crawler. ConnectionName: allOf: - $ref: '#/components/schemas/ConnectionName' - description: The name of a connection which allows a job or crawler to access data in Amazon S3 within an Amazon Virtual Private Cloud environment (Amazon VPC). SampleSize: allOf: - $ref: '#/components/schemas/NullableInteger' - description: Sets the number of files in each leaf folder to be crawled when crawling sample files in a dataset. If not set, all the files are crawled. A valid value is an integer between 1 and 249. EventQueueArn: allOf: - $ref: '#/components/schemas/EventQueueArn' - description: A valid Amazon SQS ARN. For example, arn:aws:sqs:region:account:sqs. DlqEventQueueArn: allOf: - $ref: '#/components/schemas/EventQueueArn' - description: A valid Amazon dead-letter SQS ARN. For example, arn:aws:sqs:region:account:deadLetterQueue. description: Specifies a data store in Amazon Simple Storage Service (Amazon S3). NameString: type: string minLength: 1 maxLength: 255 x-pattern: '[\u0020-\uD7FF\uE000-\uFFFD\uD800\uDC00-\uDBFF\uDFFF\t]*' GetCrawlerResponse: type: object properties: Crawler: allOf: - $ref: '#/components/schemas/Crawler' - description: The metadata for the specified crawler. Timestamp: type: string format: date-time LakeFormationConfiguration: type: object properties: UseLakeFormationCredentials: allOf: - $ref: '#/components/schemas/NullableBoolean' - description: Specifies whether to use Lake Formation credentials for the crawler instead of the IAM role credentials. AccountId: allOf: - $ref: '#/components/schemas/AccountId' - description: Required for cross account crawls. For same account crawls as the target data, this can be left as null. description: Specifies Lake Formation configuration settings for the crawler. DeltaTargetList: type: array items: $ref: '#/components/schemas/DeltaTarget' CrawlerTargets: type: object properties: S3Targets: allOf: - $ref: '#/components/schemas/S3TargetList' - description: Specifies Amazon Simple Storage Service (Amazon S3) targets. JdbcTargets: allOf: - $ref: '#/components/schemas/JdbcTargetList' - description: Specifies JDBC targets. MongoDBTargets: allOf: - $ref: '#/components/schemas/MongoDBTargetList' - description: Specifies Amazon DocumentDB or MongoDB targets. DynamoDBTargets: allOf: - $ref: '#/components/schemas/DynamoDBTargetList' - description: Specifies Amazon DynamoDB targets. CatalogTargets: allOf: - $ref: '#/components/schemas/CatalogTargetList' - description: Specifies Glue Data Catalog targets. DeltaTargets: allOf: - $ref: '#/components/schemas/DeltaTargetList' - description: Specifies Delta data store targets. description: Specifies data stores to crawl. S3TargetList: type: array items: $ref: '#/components/schemas/S3Target' PathList: type: array items: $ref: '#/components/schemas/Path' LogGroup: type: string pattern: '[\.\-_/#A-Za-z0-9]+' minLength: 1 maxLength: 512 DynamoDBTargetList: type: array items: $ref: '#/components/schemas/DynamoDBTarget' LineageConfiguration: type: object properties: CrawlerLineageSettings: allOf: - $ref: '#/components/schemas/CrawlerLineageSettings' - description: '

Specifies whether data lineage is enabled for the crawler. Valid values are:

' description: Specifies data lineage configuration settings for the crawler. GetCrawlerRequest: type: object required: - Name title: GetCrawlerRequest properties: Name: allOf: - $ref: '#/components/schemas/NameString' - description: The name of the crawler to retrieve metadata for. JdbcTarget: type: object properties: ConnectionName: allOf: - $ref: '#/components/schemas/ConnectionName' - description: The name of the connection to use to connect to the JDBC target. Path: allOf: - $ref: '#/components/schemas/Path' - description: The path of the JDBC target. Exclusions: allOf: - $ref: '#/components/schemas/PathList' - description: A list of glob patterns used to exclude from the crawl. For more information, see Catalog Tables with a Crawler. EnableAdditionalMetadata: allOf: - $ref: '#/components/schemas/EnableAdditionalMetadata' - description:

Specify a value of RAWTYPES or COMMENTS to enable additional metadata in table responses. RAWTYPES provides the native-level datatype. COMMENTS provides comments associated with a column or table in the database.

If you do not need additional metadata, keep the field empty.

description: Specifies a JDBC data store to crawl. CronExpression: type: string parameters: X-Amz-SignedHeaders: name: X-Amz-SignedHeaders in: header schema: type: string required: false X-Amz-Signature: name: X-Amz-Signature in: header schema: type: string required: false X-Amz-Security-Token: name: X-Amz-Security-Token in: header schema: type: string required: false X-Amz-Content-Sha256: name: X-Amz-Content-Sha256 in: header schema: type: string required: false X-Amz-Date: name: X-Amz-Date in: header schema: type: string required: false X-Amz-Credential: name: X-Amz-Credential in: header schema: type: string required: false X-Amz-Algorithm: name: X-Amz-Algorithm in: header schema: type: string required: false securitySchemes: hmac: type: apiKey name: Authorization in: header description: Amazon Signature authorization v4 x-amazon-apigateway-authtype: awsSigv4 externalDocs: description: Amazon Web Services documentation url: https://docs.aws.amazon.com/glue/ x-hasEquivalentPaths: true