openapi: 3.0.3 info: title: Web Data Extraction API version: 1.0.0 description: A single API for web scraping contact: name: Zyte (Formerly Scrapinghub) url: https://www.zyte.com servers: - url: https://api.zyte.com/v1 description: Zyte Extraction API Production server security: - BasicAuth: [] paths: /extract: post: operationId: extract summary: Process a single URL, return the result description: | Process a single URL, return the result. This endpoint blocks until the result is ready. It is intended for short-running operations. At least one of the following request fields must be set to true: - browserHtml - httpResponseBody - httpResponseHeaders - screenshot - An automatic extraction request field: - article - articleList - articleNavigation - forumThread - jobPosting - jobPostingNavigation - pageContent - product - productList - productNavigation - serp All automatic extraction data types support performing extraction using either a browser request or an HTTP request. Choose which using extractFrom; for serp use serpOptions.extractFrom instead. When no option is specified, currently automatic extraction defaults to using a browser request, except for serp, where an HTTP request is used by default instead. In the future, however, the default value may depend on the target website. When automatic extraction uses a browser request, it can be combined with any fields compatible with browserHtml, e.g. screenshot. When automatic extraction uses an HTTP request, it can be combined with any fields compatible with httpResponseBody. serp cannot be combined with any other fields besides serpOptions and url. You cannot combine multiple automatic extraction request fields (e.g. product and productList) on the same request. You cannot combine httpResponseBody with a request field that is exclusive of browser requests (e.g. httpResponseBody and browserHtml). httpResponseHeaders can be requested alone or with any other valid combination of request fields except for serp. The request body size limit is 5MiB. requestBody: required: true description: An extraction request body content: application/json: schema: $ref: '#/components/schemas/ExtractRequest' examples: DownloadHttp: summary: Retrieve raw HTTP content from a page value: url: https://example.com httpResponseBody: true DownloadCustomHttpRequestHeaders: summary: Retrieve raw HTTP content from a page, using custom HTTP headers value: url: https://example.com httpResponseBody: true customHttpRequestHeaders: - name: X-APOLLO-OPERATION-NAME value: nearByNodes DownloadHttpPostWithHttpRequestBody: summary: Retrieve raw HTTP content from a page using a POST request value: url: https://example.com httpResponseBody: true httpRequestMethod: POST httpRequestBody: WyJCV0kiLCAiRkxMIl0= customHttpRequestHeaders: - name: Content-Type value: application/json DownloadHttpPostWithHttpRequestText: summary: Retrieve raw HTTP content from a page using a POST request value: url: https://example.com httpResponseBody: true httpRequestMethod: POST httpRequestText: '{"name":"John Doe","email":"johndoe@example.com","age":32,"address":{"street":"123 Main St","city":"Anytown","state":"CA","zip":"12345"},"phone_numbers":["+1 555 555 1212","+1 555 555 1313"]}' customHttpRequestHeaders: - name: Content-Type value: application/json DownloadHttpHeaders: summary: Retrieve HTTP headers from a page using a POST request value: url: https://example.com httpResponseHeaders: true DownloadHtml: summary: Open a page in a browser, return HTML value: url: https://example.com browserHtml: true DownloadHtmlWithRequestHeaders: summary: Open a page in a browser and return HTML, setting a proper Referer header value: url: https://example.com browserHtml: true requestHeaders: referer: https://search.example DownloadScreenshot: summary: | Open a page in a browser and return a JPEG screenshot of the content visible on the browser window value: url: https://example.com screenshot: true DownloadScreenshotFullPagePng: summary: | Open a page in a browser and return a full-page PNG screenshot value: url: https://example.com screenshot: true screenshotOptions: fullPage: true format: png DownloadHtmlEchoData: summary: echoData and jobId example description: | Open a page in a browser, return HTML. Pass the echoData and jobId fields through - they'll be returned unchanged in the output. value: url: https://example.com/foo browserHtml: true echoData: seedUrl: https://example.com foo: bar jobId: 123/234/12 DownloadHtmlActions: summary: actions example description: | 1. Open the target page in a browser 2. Type "Zyte" in the search box 3. Click the Search button 4. Wait for the results page to load value: url: https://example.com/search browserHtml: true actions: - action: type selector: value: '#searchbox' type: css text: Zyte - action: click selector: value: '#searchbtn' type: css ExtractProduct: summary: Extract Product information description: | Extract Product information from a page: price, name, etc. value: url: https://example.com/foo product: true ExtractProductWithHtml: summary: Extract Product information, as well as browser HTML description: | Extract Product information, as well as browser HTML. Make a request from Spanish geolocation. value: url: https://example.com/foo product: true browserHtml: true geolocation: ES ExtractProductRaw: summary: Extract Product information using an HTTP request description: | Extract Product information using an HTTP request. value: url: https://example.com/foo product: true extractFrom: httpResponseBody ExtractProductRawWithBody: summary: Extract Product information, as well as httpResponseBody description: | Extract Product information using an HTTP request, as well as httpResponseBody and httpResponseHeaders. Make a request from Spain. value: url: https://example.com/foo product: true extractFrom: httpResponseBody httpResponseBody: true httpResponseHeaders: true geolocation: ES ExtractArticleCustomAttributes: summary: Extract Custom Attributes along with Article information description: | Extract Custom Attributes along with Article information. value: url: https://example.com/foo article: true customAttributes: summary: type: string description: A two sentence article summary article_sentiment: type: string enum: - positive - negative - neutral responses: '200': description: Successful response. Contains the output requested. content: application/json: schema: $ref: '#/components/schemas/Response200' '400': description: | Malformed request. See the error details to identify the exact problem in the request. content: application/problem+json: schema: $ref: '#/components/schemas/Problem' examples: Unrecognized field: value: type: /request/invalid title: Bad Request status: 400 detail: >- Unrecognized field "foo" Invalid JSON value: value: type: /request/invalid-json title: Invalid JSON status: 400 detail: >- The submitted request body is not a valid JSON. Location: line 2, column 26. Details: Unrecognized token 'False': was expecting (JSON String, Number, Array, Object or token 'null', 'true' or 'false') '401': headers: WWW-Authenticate: schema: type: string description: | Authentication problem. See the error details. content: application/problem+json: schema: $ref: '#/components/schemas/Problem' examples: Invalid authentication data: value: type: /auth/not-valid title: Authentication Info Invalid status: 401 detail: >- No valid authentication info found in the request. Check the documentation for the correct authentication schema. Invalid API key: value: type: /auth/key-not-found title: Authentication Key Not Found status: 401 detail: >- The authentication key is not valid or can't be matched. '403': description: | Your account is suspended or not allowed to make the request. content: application/problem+json: schema: $ref: '#/components/schemas/Problem' examples: Account suspended: value: type: /auth/account-suspended title: Account Suspended status: 403 detail: >- Account is suspended, check billing details. '421': description: | The request failed and should not be retried content: application/problem+json: schema: $ref: '#/components/schemas/Problem' examples: Incompatible parameters: value: type: /website/domain-unreachable title: Domain Unreachable status: 421 detail: >- The domain is invalid or unreachable. Please check the domain name and try again. Verify the domain name and ensure it is registered and valid before restarting the crawl. '422': description: | The request couldn't be processed. Check the details. content: application/problem+json: schema: $ref: '#/components/schemas/Problem' examples: Incompatible parameters: value: type: /request/unprocessable title: Unprocessable Request status: 422 detail: >- Incompatible parameters were found in the request. Check details '429': headers: Retry-After: schema: type: integer format: int32 minimum: 0 description: | Too many requests, see the details. content: application/problem+json: schema: $ref: '#/components/schemas/Problem' examples: Domain limit: value: type: /limits/over-domain-limit title: Over Domain Requests Limit status: 429 detail: >- Too many requests. Retry in N seconds from 'Retry-After' header. User limit: value: type: /limits/over-user-limit title: Over User Requests Limit status: 429 detail: >- Too many requests to a specific domain. Retry in N seconds from 'Retry-After' header. Organisation limit: value: type: /limits/over-org-domain-limit title: Over Organisation Requests limit for the requested domain status: 429 detail: >- Too many requests to a specific domain. Retry in N seconds from 'Retry-After' header. '451': description: | Extraction for the domain is forbidden. content: application/problem+json: schema: $ref: '#/components/schemas/ForbiddenDomainProblem' examples: Domain forbidden: value: type: /download/domain-forbidden title: Domain Forbidden status: 451 detail: >- Extraction for the domain is forbidden. blockedDomain: blocked-domain.example '500': description: | Request timeout or internal server error. content: application/problem+json: schema: $ref: '#/components/schemas/Problem' examples: Internal error: value: type: /server/internal title: Internal Server Error status: 500 detail: >- The server encountered an internal error. Please contact support or wait for us to resolve the issue. Timeout: value: type: /server/timed-out title: Request Timed Out status: 500 detail: >- The request took too long and timed out. Try it again. Contact support if it fails consistently. '503': description: | System overload. See the details. headers: Retry-After: schema: type: integer format: int32 minimum: 0 content: application/problem+json: schema: $ref: '#/components/schemas/Problem' examples: Global request limit: value: type: /limits/over-global-limit title: Global Requests Limit Reached status: 503 detail: >- Too many requests to the service. Retry in N seconds from 'Retry-After' header. '520': description: | A downloading error, possibly requiring user action. headers: Retry-After: schema: type: integer format: int32 minimum: 0 content: application/problem+json: schema: $ref: '#/components/schemas/Problem' examples: Website ban: value: type: /download/temporary-error title: Website Ban status: 520 detail: >- Zyte API could not get a ban-free response in a reasonable time. See https://docs.zyte.com/zyte-api/usage/errors.html#ban-responses '521': description: | Permanent downloading error. content: application/problem+json: schema: $ref: '#/components/schemas/Problem' examples: Internal download error: value: type: /download/internal-error title: Internal Downloading Error status: 521 detail: >- Server encountered a problem while downloading. Check request and contact support. default: description: | Error. Check the code and problem object for additional information. Note: The client should be ready for the absence of a problem object. In this case, the HTTP status code should be used. content: application/problem+json: schema: $ref: '#/components/schemas/Problem' components: securitySchemes: BasicAuth: type: http scheme: basic schemas: HTTPHeader: type: object description: A header name and value. required: - name - value properties: name: type: string description: The name of the header minLength: 1 value: type: string description: The value of the header example: name: Content-Type value: text/html; charset=utf-8 ExtractRequest: type: object required: - url properties: url: description: | An absolute URL to extract data from. The host name must be a domain name, it cannot be an IP address. example: https://example.com/item-page type: string maxLength: 8192 requestHeaders: $ref: '#/components/schemas/RequestHeaders' tags: type: object nullable: true description: | Assign arbitrary key-value pairs to the request that you can use for filtering in the [Stats API](/zyte-api/usage/stats.md). Keys must be strings. Values must be strings or `null`. For example: `{"tags": {"foo": "bar", "baz": null}}`. additionalProperties: type: string ipType: description: | [Type of IP address](/zyte-api/usage/features.md) from which the request should be sent. If not specified, Zyte API will use an IP type that, for the target website, does not cause bans or unexpected response data. If you believe Zyte API is using the wrong default IP type for a website, please [reach out to our expert anti-ban team](https://support.zyte.com/support/tickets/new). [See an example](/zyte-api/usage/features.md). type: string enum: - datacenter - residential httpRequestMethod: description: | Request [HTTP method](https://developer.mozilla.org/en-US/docs/Web/HTTP/Methods). Can only be used in combination with httpResponseBody. [See an example](/zyte-api/usage/http.md). See also: httpRequestText, httpRequestBody, customHttpRequestHeaders, httpResponseHeaders. type: string enum: - GET - POST - PUT - DELETE - OPTIONS - TRACE - PATCH - HEAD httpRequestBody: description: | [Base64](https://en.wikipedia.org/wiki/Base64)-encoded data to send as request body. Can only be used in combination with httpResponseBody. It usually needs to be used in combination with httpRequestMethod. If you only need to send UTF-8-encoded text, use httpRequestText instead to skip Base64-encoding. Note that you cannot combine both fields on the same request. [See an example](/zyte-api/usage/http.md). See also: customHttpRequestHeaders. type: string format: byte maxLength: 400000 httpRequestText: description: | UTF-8 text to send as request body. Can only be used in combination with httpResponseBody. It usually needs to be used in combination with httpRequestMethod. If you need to send a binary or non-UTF-8 request body, use httpRequestBody instead. Note that you cannot combine both fields on the same request. [See an example](/zyte-api/usage/http.md). See also: customHttpRequestHeaders. type: string minLength: 1 maxLength: 400000 example: '{"name":"John Doe","email":"johndoe@example.com","age":32,"address":{"street":"123 Main St","city":"Anytown","state":"CA","zip":"12345"},"phone_numbers":["+1 555 555 1212","+1 555 555 1313"]}' customHttpRequestHeaders: description: | HTTP request headers. Can only be used in combination with httpResponseBody. To set headers with other outputs, see requestHeaders. Setting HTTP request headers has some caveats: - Zyte API sends some headers automatically for [ban avoidance](/zyte-api/usage/errors.md), and may silently override or drop some of your custom headers for that purpose. However, your custom headers may override those automatic headers, and in doing so they can break the ban avoidance capabilities of Zyte API, as some websites may ban based on the presence, values, or order of certain headers. - You cannot set the `Cookie` header. Use requestCookies instead. - If you set multiple headers with the same name, only the last header value will be sent. To overcome this limitation, [join the header values with a comma into a single header value](https://stackoverflow.com/a/4371395). For example, replace `"customHttpRequestHeaders": [{"name": "foo", "value": "bar"}, {"name": "foo", "value": "baz"}]` with `"customHttpRequestHeaders": [{"name": "foo", "value": "bar,baz"}]`. [See an example](/zyte-api/usage/http.md). See also: httpRequestMethod, httpRequestText, httpRequestBody, httpResponseHeaders. type: array maxItems: 200 items: $ref: '#/components/schemas/CustomHttpRequestHeader' httpResponseBody: description: | Set to `true` to get the HTTP response body in the httpResponseBody response field. This field is not compatible with [browser automation](/zyte-api/usage/browser.md). [See an example](/zyte-api/usage/http.md). See also: httpRequestMethod, httpRequestText, httpRequestBody, customHttpRequestHeaders. type: boolean default: false httpResponseHeaders: description: | Set to `true` to get the HTTP response headers in the httpResponseHeaders response field. [See an example](/zyte-api/usage/features.md). See also: customHttpRequestHeaders, requestHeaders. type: boolean default: false browserHtml: description: | Set to `true` to get the [browser HTML](/zyte-api/usage/browser.md) in the browserHtml response field. This field is not compatible with [HTTP requests](/zyte-api/usage/http.md). If you use actions, the browser HTML is generated *after* action execution has finished or timed out. By default, [iframes](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe) are empty. See includeIframes. To access content from the [shadow DOM](https://developer.mozilla.org/en-US/docs/Web/Web_Components/Using_shadow_DOM), check out the corresponding example in the [actions documentation](/zyte-api/usage/browser.md). [See an example](/zyte-api/usage/browser.md). See also: screenshot, requestHeaders. type: boolean default: false screenshot: description: | Set to `true` to get a page screenshot in the screenshot response field. This field is not compatible with [HTTP requests](/zyte-api/usage/http.md). To adjust the screenshot contents you can use screenshotOptions and viewport. If you use actions, the screenshot is generated *after* action execution has finished or timed out. [See an example](/zyte-api/usage/browser.md). See also: browserHtml, requestHeaders. type: boolean default: false screenshotOptions: $ref: '#/components/schemas/ScreenshotOptions' extractFrom: $ref: '#/components/schemas/ExtractFrom' article: description: | Set to `true` to get article data in the article response field. The target page should only contain a single article, such as a blog post or a news article. For pages with multiple articles consider using articleList instead. To combine this field with [HTTP requests](/zyte-api/usage/http.md), set extractFrom to `"httpResponseBody"`. If you use actions, data extraction happens *after* action execution has finished or timed out. See also: [List of all automatic extraction request fields](/zyte-api/usage/extract.md), articleNavigation, browserHtml, screenshot, requestHeaders. default: false type: boolean articleOptions: description: | Additional options for article extraction. $ref: '#/components/schemas/ExtractionOptions' articleList: description: | Set to `true` to get article list data in the articleList response field. The target page should contain multiple articles, usually as links or short snippets. Examples of such pages are main or category pages of news sites, main pages of blogs showing multiple posts, and other pages with multiple articles. Article list data is especially useful to get basic information about articles on a website, like a headline and a link to the article details, using a smaller number of requests, when article attributes are extracted directly from a article list page, without making individual article requests. To implement article crawling from article list pages, use articleNavigation, which also enables navigation through pagination links. To combine this field with [HTTP requests](/zyte-api/usage/http.md), set extractFrom to `"httpResponseBody"`. If you use actions, data extraction happens *after* action execution has finished or timed out. See also: [List of all automatic extraction request fields](/zyte-api/usage/extract.md), browserHtml, screenshot, requestHeaders. type: boolean default: false articleListOptions: description: | Additional options for articleList extraction. $ref: '#/components/schemas/ExtractionOptions' articleNavigation: description: | Set to `true` to get article navigation data in the articleNavigation response field. The target page should contain multiple articles and/or subcategories that can be followed. Article navigation data is especially useful for implementing article crawling, i.e. following links to article pages, as well as to subcategories and pagination that can in turn link to more article pages. Article navigation data can also be used to get basic information of articles and subcategories on a website, obtaining the URLs and link names of the articles and subcategories, without making individual requests for those articles. To combine this field with [HTTP requests](/zyte-api/usage/http.md), set extractFrom to `"httpResponseBody"`. If you use actions, data extraction happens *after* action execution has finished or timed out. See also: [List of all automatic extraction request fields](/zyte-api/usage/extract.md), article, articleList, browserHtml, screenshot, requestHeaders. type: boolean default: false articleNavigationOptions: description: | Additional options for articleNavigation extraction. $ref: '#/components/schemas/ExtractionOptions' forumThread: description: | Set to `true` to get forum threads data in the forumThread response field. The target page should contain an individual forum thread page on a forum website. To combine this field with [HTTP requests](/zyte-api/usage/http.md), set extractFrom to `"httpResponseBody"`. If you use actions, data extraction happens *after* action execution has finished or timed out. See also: [List of all automatic extraction request fields](/zyte-api/usage/extract.md), article, browserHtml, screenshot, requestHeaders. type: boolean default: false forumThreadOptions: description: | Additional options for forumThread extraction. $ref: '#/components/schemas/ExtractionOptions' jobPosting: description: | Set to `true` to get job posting data in the jobPosting response field. The target page should contain individual job posting page on a company website or on a job website. To combine this field with [HTTP requests](/zyte-api/usage/http.md), set extractFrom to `"httpResponseBody"`. If you use actions, data extraction happens *after* action execution has finished or timed out. See also: [List of all automatic extraction request fields](/zyte-api/usage/extract.md), browserHtml, screenshot, requestHeaders. type: boolean default: false jobPostingOptions: description: | Additional options for jobPosting extraction. $ref: '#/components/schemas/ExtractionOptions' jobPostingNavigation: description: | Set to `true` to get job posting navigation data in the jobPostingNavigation response field. The target page should contain multiple job postings and/or subcategories that can be followed. Job posting navigation data is especially useful for implementing job posting crawling, i.e. following links to job posting pages, as well as pagination that can in turn link to more job posting pages. Job posting navigation data can also be used to get basic information of job postings on a website, obtaining the URLs and link names of the job postings, without making individual requests for them. To combine this field with [HTTP requests](/zyte-api/usage/http.md), set extractFrom to `"httpResponseBody"`. If you use actions, data extraction happens *after* action execution has finished or timed out. See also: [List of all automatic extraction request fields](/zyte-api/usage/extract.md), jobPosting, browserHtml, screenshot, requestHeaders. type: boolean default: false jobPostingNavigationOptions: description: | Additional options for jobPostingNavigation extraction. $ref: '#/components/schemas/ExtractionOptions' pageContent: description: | Set to `true` to get page content data in the pageContent response field. The target page can contain any type of data. Page content data is especially useful for understanding the layout and hierarchy of information on a page, enabling advanced processing such as content extraction, user experience analysis, and automated page summarization. Page content data can also be used to capture the main content intended for users, along with auxiliary navigation components such as headers, footers, sidebars, and pagination controls. This makes it possible to distinguish core content from supporting links used for site-wide navigation. To combine this field with [HTTP requests](/zyte-api/usage/http.md), set extractFrom to `"httpResponseBody"`. If you use actions, data extraction happens *after* action execution has finished or timed out. See also: [List of all automatic extraction request fields](/zyte-api/usage/extract.md), browserHtml, screenshot, requestHeaders. type: boolean default: false pageContentOptions: description: | Additional options for pageContent extraction. $ref: '#/components/schemas/ExtractionOptions' product: description: | Set to `true` to get product data in the product response field. The target page should only contain a single product. For pages with multiple products consider using productList instead. To combine this field with [HTTP requests](/zyte-api/usage/http.md), set extractFrom to `"httpResponseBody"`. If you use actions, data extraction happens *after* action execution has finished or timed out. [See an example](/zyte-api/usage/extract.md). See also: [List of all automatic extraction request fields](/zyte-api/usage/extract.md), productNavigation, browserHtml, screenshot, requestHeaders. type: boolean default: false productOptions: description: | Additional options for product extraction. type: object properties: extractFrom: $ref: '#/components/schemas/ExtractionOptions/properties/extractFrom' model: type: string enum: - '2024-02-01' - '2024-09-16' description: | Model version to use for product extraction. If not specified, the "2024-09-16" version is used. Available product models: - "2024-02-01" - "2024-09-16" See [Model pinning](/zyte-api/usage/extract/index.md). productList: description: | Set to `true` to get product list data in the productList response field. The target page should contain a list or a grid of products. Product list data is especially useful to get basic information about products on a website using a smaller number of requests, when product attributes are extracted directly from a product list page, without making individual product requests. To implement product crawling from product list pages, use productNavigation, which also enables navigation through pagination links. To combine this field with [HTTP requests](/zyte-api/usage/http.md), set extractFrom to `"httpResponseBody"`. If you use actions, data extraction happens *after* action execution has finished or timed out. See also: [List of all automatic extraction request fields](/zyte-api/usage/extract.md), browserHtml, screenshot, requestHeaders. type: boolean default: false productListOptions: description: | Additional options for productList extraction. $ref: '#/components/schemas/ExtractionOptions' productNavigation: description: | Set to `true` to get product navigation data in the productNavigation response field. The target page should contain multiple products and/or subcategories that can be followed. Product navigation data is especially useful for implementing product crawling, i.e. following links to product pages, as well as to subcategories and pagination that can in turn link to more product pages. Product navigation data can also be used to get basic information of products and subcategories on a website, obtaining the URLs and link names of the products and subcategories, without making individual requests for those products. To combine this field with [HTTP requests](/zyte-api/usage/http.md), set extractFrom to `"httpResponseBody"`. If you use actions, data extraction happens *after* action execution has finished or timed out. See also: [List of all automatic extraction request fields](/zyte-api/usage/extract.md), product, productList, browserHtml, screenshot, requestHeaders. type: boolean default: false productNavigationOptions: description: | Additional options for productNavigation extraction. $ref: '#/components/schemas/ExtractionOptions' customAttributes: type: object description: | Schema of the custom attributes to extract. This is a subset of the OpenAPI specification, using JSON syntax. Zyte custom attributes extraction uses a Large Language Model (LLM) operated by Zyte to obtain any structured data specified by this schema from any unstructured web page. This allows to perform extraction similar to standard schemas, such as article or product, but much more flexibly. When this field is specified, the customAttributes.values field in the response would contain the extracted data. When custom attributes extraction is requested, a standard extraction field must also be specified (e.g. product). This determines the part of the web page which would be passed to the LLM for custom attributes extraction, e.g. when a web page is a product, we're only going to pass the product information, ignoring other parts of the page, such as menu or footer, which makes extraction cheaper and more accurate. [See detailed documentation](/zyte-api/usage/custom-attributes.md). Additionally, to see a request example, scroll up to the right-hand sidebar **Request samples**, and select “Extract Custom Attributes along with Article information” under **Example**. nullable: true additionalProperties: $ref: '#/components/schemas/CustomAttribute' maxProperties: 20 customAttributesOptions: type: object description: Additional options for custom attributes extraction. properties: method: type: string description: | Method to use for custom attributes extraction: * "generate" (default) generates extracted data with the help of a generative Large Language Model (LLM). It is the most powerful and versatile extraction method, but also the most expensive one, with [variable per-request cost](/zyte-api/pricing.md). * "extract" locates extracted data in the requested web page with the help of a non-generative LLM. It only supports a subset of the schema (only string, integer and number types), and can't perform generative tasks such as summarization or data transformation. It is however much cheaper compared to the generative method and has a [fixed per-request cost](/zyte-api/pricing.md). enum: - generate default: generate maxInputTokens: type: integer minimum: 1 description: | Limit on the number of input tokens for custom attribute extraction with the "generate" method. This includes the schema as well, but not our internal fixed prompt with the LLM instruction. When the number of tokens for schema and page text is above the specified maxInputTokens, we truncate the page text to fit in maxInputTokens. This may result in quality degradation or data not extracted from the page because it was truncated. Tokens are words or word pieces, for example ``{"price": "2.00 $"}`` is 9 tokens: ``{"``, ``price``, ``":``, `` "``, ``2``, ``.``, ``00``, `` $``, ``"}``. maxOutputTokens: type: integer minimum: 1 description: | Limit on the number of output tokens for extracted custom attributes with the "generate" method. This field can be set to limit the extraction cost, but may result in quality degradation. See an example of token counting in the maxInputTokens field above. geolocation: $ref: '#/components/schemas/CountryCode' javascript: description: | Forces JavaScript execution on a [browser request](/zyte-api/usage/browser.md) to be enabled (`true`) or disabled (`false`). By default Zyte API enables or disables JavaScript execution for a request depending on which option makes it easier to avoid bans. Use this request field to override that choice. Passing this request field when requesting automatic extraction ( product, article, etc.) may impact the quality of the returned data, as it might override the optimal value for automatic extraction. This field is not compatible with [HTTP requests](/zyte-api/usage/http.md). [See an example](/zyte-api/usage/browser.md). type: boolean actions: $ref: '#/components/schemas/ActionSequence' jobId: description: | ID of the [Scrapy Cloud](/scrapy-cloud/get-started.md) job from which this request has been sent, to be returned in the jobId response field. This field is meant to help with request tracking. [scrapy-zyte-api](https://scrapy-zyte-api.readthedocs.io/en/latest/index.html) fills this request field automatically. [See an example](/zyte-api/usage/features.md). See also: echoData. type: string maxLength: 100 example: example-job-1 echoData: description: | This field is returned in the echoData response field, verbatim. This field can be useful, for example, to keep track of the original request order when [sending multiple requests in parallel](/zyte-api/usage/optimize.md). The request can be rejected if the data is too big. [See an example](/zyte-api/usage/features.md). See also: jobId. viewport: $ref: '#/components/schemas/Viewport' followRedirect: description: | Whether to follow [HTTP redirection](https://developer.mozilla.org/en-US/docs/Web/HTTP/Redirections) or not. Only supported in [HTTP requests](/zyte-api/usage/http.md), [browser requests always follow redirection](/zyte-api/usage/browser.md). type: boolean verifyCertificate: description: | Whether to validate TLS certificates of the target website. When omitted or false, certificates are not validated (matching historical behavior); set to `true` to fail with an error response instead of returning page content when validation fails. Only supported in [HTTP requests](/zyte-api/usage/http.md), in [browser requests](/zyte-api/usage/browser.md) certificates are always validated and this parameter is silently ignored. type: boolean sessionContext: $ref: '#/components/schemas/SessionContext' sessionContextParameters: $ref: '#/components/schemas/SessionContextParameters' session: $ref: '#/components/schemas/Session' networkCapture: $ref: '#/components/schemas/NetworkCaptureFilterSequence' device: description: | Type of device to emulate during your request. A desktop device is emulated by default. Can only be used in combination with httpResponseBody. type: string enum: - desktop - mobile cookieManagement: description: | Cookie management method It determines how to handle user cookies, defined through requestCookies, and automatic cookies, cookies automatically generated by Zyte API. `auto` (default) uses user cookies if defined, or automatic cookies otherwise. `discard` uses user cookies if defined, or no cookies otherwise. enum: - auto - discard default: auto requestCookies: type: array description: | A list of cookies to be sent with a request. You can use the contents of the responseCookies response field as a value for this request field. [See an example](/zyte-api/usage/features.md). items: $ref: '#/components/schemas/Cookie' maxItems: 100 responseCookies: description: | Set to `true` to get the list of cookies set during a request in the responseCookies response field. [See an example](/zyte-api/usage/features.md). See also: requestCookies. type: boolean default: false serp: type: boolean description: | Set to `true` to get the data of a search engine results page (SERP) in the serp response field. The target URL should be a search engine URL. Currently, you cannot combine this field with any other request fields besides serpOptions and url. See also: [List of all automatic extraction request fields](/zyte-api/usage/extract.md). serpOptions: $ref: '#/components/schemas/SerpOptions' includeIframes: type: boolean description: | Whether to add the content of [iframes](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe) into browserHtml. Note that iframes are visible in screenshots even if this is set to `false`. See also: browserHtml. default: false additionalProperties: false Response200: required: - url properties: url: type: string description: | URL the data was extracted from. Could be different from the input URL in case of [redirection](/zyte-api/usage/http.md). See also: statusCode. example: https://example.com/item-page/ statusCode: type: integer description: | The HTTP status code retrieved from the target page. If [redirection is followed](/zyte-api/usage/http.md), this is the status code of the response *after* redirection. See also: url. example: 200 httpResponseBody: description: | [Base64-encoded](https://en.wikipedia.org/wiki/Base64) HTTP response body. To get this response field, set the httpResponseBody request field to `true`. Unlike browserHtml, this field supports binary response bodies, such as image files or PDF files. This is the reason why this field is Base64-encoded, JSON does not support binary data. [See an example](/zyte-api/usage/http.md). type: string format: byte httpResponseHeaders: description: | HTTP response headers. To get this response field, set the httpResponseHeaders request field to `true`. The `Content-Encoding` header value (e.g. `gzip`, `br`, etc.) should not be used to decompress httpResponseBody, Zyte API already decompresses the body of compressed responses. The `Set-Cookie` header value, when present, contains the header value received from the main HTTP response. These cookies could have changed later on, e.g. during browser rendering. Usually you will want to ignore this header in favor of responseCookies, which provides the *final* cookies. [See an example](/zyte-api/usage/features.md). type: array items: $ref: '#/components/schemas/HTTPHeader' browserHtml: description: | [Browser HTML](/zyte-api/usage/browser.md). To get this response field, set the browserHtml request field to `true`. Browser HTML does not include the contents of [iframes](https://developer.mozilla.org/en-US/docs/Web/HTML/Element/iframe) or the [shadow DOM](https://developer.mozilla.org/en-US/docs/Web/Web_Components/Using_shadow_DOM). [See an example](/zyte-api/usage/browser.md). type: string example: Downloaded data. session: $ref: '#/components/schemas/Session' screenshot: description: | [Base64-encoded](https://en.wikipedia.org/wiki/Base64) page screenshot file data. To get this response field, set the screenshot request field to `true`. screenshotOptions.format determines the file format of the screenshot data. [See an example](/zyte-api/usage/browser.md). type: string format: byte article: allOf: - $ref: '#/components/schemas/Article' - description: | Article data. To get this response field, set the article request field to `true`. articleList: allOf: - $ref: '#/components/schemas/ArticleList' - description: | Article list data. To get this response field, set the articleList request field to `true`. articleNavigation: allOf: - $ref: '#/components/schemas/ArticleNavigation' - description: | Article navigation data. To get this response field, set the articleNavigation request field to `true`. forumThread: allOf: - $ref: '#/components/schemas/ForumThread' - description: | Forum thread data. To get this response field, set the forumThread request field to `true`. jobPosting: allOf: - $ref: '#/components/schemas/JobPosting' - description: | Job posting data. To get this response field, set the jobPosting request field to `true`. jobPostingNavigation: allOf: - $ref: '#/components/schemas/JobPostingNavigation' - description: | Job posting navigation data. To get this response field, set the jobPostingNavigation request field to `true`. pageContent: allOf: - $ref: '#/components/schemas/PageContent' - description: | Page content data. To get this response field, set the pageContent request field to `true`. product: allOf: - $ref: '#/components/schemas/Product' - description: | Product data. To get this response field, set the product request field to `true`. productList: allOf: - $ref: '#/components/schemas/ProductList' - description: | Product list data. To get this response field, set the productList request field to `true`. productNavigation: allOf: - $ref: '#/components/schemas/ProductNavigation' - description: | Product navigation data. To get this response field, set the productNavigation request field to `true`. customAttributes: type: object properties: values: type: object additionalProperties: true description: | Values of extracted custom attributes, extracted according to the requested customAttributes schema. metadata: type: object properties: inputTokens: type: integer description: | Total number of used input tokens, excluding our internal fixed prompt with the LLM instruction, when using the "generate" method. outputTokens: type: integer description: | Total number of used output tokens, when using the "generate" method. textInputTokens: type: integer description: | Total number of input tokens used for the text of the web page, excluding the schema and our internal fixed prompt with the LLM instruction, when using the "generate" method. Already included in the customAttributes.metadata.inputTokens field. textInputTokensBeforeTruncation: type: integer description: | textInputTokens before the text was truncated to fit into the input limits, either set via customAttributesOptions.maxInputTokens or due to the model limitation returned in customAttributes.metadata.maxInputTokens, when using the "generate" method. maxInputTokens: type: integer description: | Maximum number of allowed input tokens for the model, when using the "generate" method. excludedPIIAttributes: type: array items: type: string description: | A list of all attributes dropped from the output due to a risk of PII (Personally Identifiable Information) extraction. error: type: string description: | * The ``extraction/unparsable-response`` error is given when the LLM response could not be parsed or recovered. If this error happens, we suggest simplifying the task or reducing the number of attributes. * The ``extraction/schema-size-exceeded`` error is given when the schema did not fit into the input limits, leaving no space for the input text, and therefore the LLM could not be used. If this error happens, we suggest either making the schema smaller (fewer attributes and/or shorter descriptions), or increasing customAttributesOptions.maxInputTokens. echoData: description: | Arbitrary data set on the echoData request field. [See an example](/zyte-api/usage/features.md). type: object jobId: description: | [Scrapy Cloud](/scrapy-cloud/get-started.md) job ID set on the jobId request field. [See an example](/zyte-api/usage/features.md). type: string maxLength: 100 example: example-job-1 actions: description: | Debug information about the execution of the action sequence set in the actions request field. Action order in the response always matches that of the request. type: array items: $ref: '#/components/schemas/ActionResult' responseCookies: description: | List of cookies set during the request. To get this response field, set the responseCookies request field to `true`. [See an example](/zyte-api/usage/features.md). See also: requestCookies. type: array items: $ref: '#/components/schemas/Cookie' networkCapture: type: array description: | Responses captured by filters specified in the networkCapture request parameter. items: $ref: '#/components/schemas/CapturedResponse' serp: $ref: '#/components/schemas/Serp' Problem: type: object properties: type: type: string format: uri-reference description: | A URI reference that uniquely identifies the problem type, only in the context of the provided API. Opposed to the specification in RFC-7807, it is neither recommended to be dereferenceable and point to human-readable documentation nor globally unique for the problem type. default: about:blank example: /problem/connection-error title: type: string description: > A short summary of the problem type. Written in English and readable for engineers, usually not suited for non-technical stakeholders, and not localized. example: Service Unavailable status: type: integer format: int32 description: > The HTTP status code generated by Zyte API for this occurrence of the problem. minimum: 100 maximum: 600 exclusiveMaximum: true example: 503 detail: type: string description: > A human-readable explanation specific to this occurrence of the problem that is helpful to locate the source of the problem and gives advice on how to proceed. Written in English and readable for engineers, usually not suited for non-technical stakeholders, and not localized. example: Connection to database timed out ForbiddenDomainProblem: allOf: - $ref: '#/components/schemas/Problem' properties: blockedDomain: type: string description: > The domain which extraction cannot be performed. example: forbiddendomain.com SessionContext: description: | User-defined name-value pairs to [request a server-managed session](/zyte-api/usage/features.md) initialized with sessionContextParameters). For every subsequent request with the same session context, Zyte API will either reuse an available session created for the same session context or create a new session using sessionContextParameters). Server-managed sessions expire after 4 hours or 3 ban responses. If you are targeting websites that silently expire their sessions before the 4-hour mark, i.e. they revert the effects of your sessionContextParameters but requests continue working as expected otherwise, consider using [client-managed sessions](/zyte-api/usage/features.md) for higher session control. [See an example](/zyte-api/usage/features.md). See also: requestCookies, responseCookies. type: array items: type: object maxItems: 10 properties: name: type: string description: Name of the context identifier. minLength: 1 maxLength: 30 nullable: false value: type: string description: Value of the context identifier. minLength: 1 maxLength: 100 nullable: false required: - name - value SessionContextParameters: description: | Parameters to create a server-managed session for a given sessionContext). [See an example](/zyte-api/usage/features.md). See also: actions. type: object properties: actions: $ref: '#/components/schemas/SessionContextActionSequence' ActionResult: description: | Returns detailed information about the elapsed time and errors for a particular action. type: object properties: action: description: The type of action submitted type: string example: waitForSelector elapsedTime: description: Elapsed time in seconds type: number status: description: | Status of execution of a particular action * success - When the action finishes execution successfully without any errors * continued - When the action fails, but the execution of the action sequence is continued * returned - When the action fails and stops execution * notExecuted - When a a prior action has failed, thereby not executing the current action type: string enum: - success - continued - returned - notExecuted example: success error: description: Detailed information about the underlying error. type: string example: Request timeout while waiting for selector '#form-input' interactionLogs: description: | Messages logged with `console.log()` from [browser scripts](/zyte-api/ide/index.md). type: array items: $ref: '#/components/schemas/InteractionLogEntry' required: - action - elapsedTime - status InteractionLogEntry: description: Interaction log entry type: object properties: time: description: The ISO 8601 format of the time type: string level: description: The log level type: string enum: - debug - info - warning - error - warn message: description: The log message type: string ActionTimeout: description: Maximum wait time in seconds. type: number minimum: 0.0 default: 5.0 maximum: 15.0 UrlPattern: description: | A string to compare with a URL according to `urlMatchingOptions`. type: string example: - https://example.com/api - /api/store/fulfilment PatternMatchingOptions: description: | How to compare a user-defined string with a target string: - `contains` matches if the user-defined string is a substring of the target string. - `exact` matches if the user-defined string is an exact match of the target string. - `startsWith` matches if the target string starts with the user-defined string. - `endsWith` matches if the target string ends with the user-defined string. Comparisons are case-sensitive. Regular expressions or wildcard characters are not supported. type: string enum: - startsWith - endsWith - contains - exact default: contains ActionSelector: description: | A CSS or XPath selector to search for an element. properties: type: description: The type of selector - CSS or XPath type: string enum: - css - xpath value: type: string minLength: 1 maxLength: 500 state: description: | State can be either of the following values and defaults to visible * 'visible' - The element has a non-empty bounding box and no visibility:hidden. Note that an element without content or with display:none has an empty bounding box, and is not considered visible. * 'hidden' - The element is either detached from the DOM, or has an empty bounding box or visibility:hidden. This is the opposite of the 'visible' option. * 'attached' - The element is present in the DOM; it can be visible or hidden type: string enum: - attached - visible - hidden default: visible required: - type - value onError: description: | Handle errors encountered while executing a particular action. * continue - When a particular action fails, the action sequence continues, executing the next actions * return - When a particular actions fails, the action sequence stops, not executing any more actions When an action sequence finishes prematurely the service will return the entire response body up until the point of execution. type: string enum: - continue - return default: return ActionSequence: description: | Sequence of browser actions to execute. Select an action below to see its API reference. When using actions, you get the actions response field with debug information about action execution. [See an example](/zyte-api/usage/browser.md). type: array items: oneOf: - $ref: '#/components/schemas/click' - $ref: '#/components/schemas/doubleClick' - $ref: '#/components/schemas/evaluate' - $ref: '#/components/schemas/goto' - $ref: '#/components/schemas/hide' - $ref: '#/components/schemas/hover' - $ref: '#/components/schemas/interaction' - $ref: '#/components/schemas/keyPress' - $ref: '#/components/schemas/reload' - $ref: '#/components/schemas/scrollBottom' - $ref: '#/components/schemas/scrollTo' - $ref: '#/components/schemas/searchKeyword' - $ref: '#/components/schemas/select' - $ref: '#/components/schemas/setLocation' - $ref: '#/components/schemas/type' - $ref: '#/components/schemas/waitForNavigation' - $ref: '#/components/schemas/waitForRequest' - $ref: '#/components/schemas/waitForResponse' - $ref: '#/components/schemas/waitForSelector' - $ref: '#/components/schemas/waitForTimeout' Action: description: Action to perform. type: object properties: onError: $ref: '#/components/schemas/onError' example: action: click selector: type: css value: '#main' GoToOptions: description: Used to customise navigation options type: object properties: waitUntil: description: | When to consider navigation succeeded, defaults to load. Events can be either: * load - consider navigation to be finished when the load event is fired. * networkidle0 - consider navigation to be finished when there are no more than 0 network connections for at least 500 ms * domcontentloaded - consider navigation to be finished when the DOMContentLoaded event is fired. type: string enum: - load - networkidle0 - domcontentloaded default: load timeout: description: Maximum navigation time in seconds, defaults to 30 seconds. Pass 0 to disable timeout. type: integer default: 30 minimum: 0 ScreenshotOptions: description: | Options for the screenshot taken when the screenshot request field is `true`. type: object properties: format: description: | File format. JPEG screenshots are taken with a quality of 75%. type: string enum: - png - jpeg default: jpeg fullPage: description: | When `true`, the screenshot features the full page. When `false`, it features only what is visible on the browser window (viewport). Full page screenshots: - Are only available in JPEG format. - Have a minimum resolution of 1920x1080, i.e. for pages smaller than 1920x1080, the screenshot looks the same regardless of the value of `fullPage`. - Any image exceeding 5000 (width) x 10000 (height) pixels will be clipped to those dimensions. type: boolean default: false ExtractionOptions: description: | Options for automatic extraction. type: object properties: extractFrom: allOf: - $ref: '#/components/schemas/ExtractFrom' description: | extractFrom override for this particular extraction type. Using extractFrom instead is recommended. userHtml: type: string description: | HTML content provided by the user for extraction. Required when extractFrom is set to `"userHtml"`. minLength: 1 maxLength: 2621440 ExtractFrom: type: string enum: - httpResponseBody - browserHtml - userHtml description: | [Extraction source](/zyte-api/usage/extract/index.md). `httpResponseBody` extracts from httpResponseBody. It is usually faster and cheaper. `browserHtml` extracts from both browserHtml and screenshot. It typically improves quality over `httpResponseBody`, but is not as robust in case of rendering issues. `userHtml` extracts from user-provided HTML content. When this option is set, the userHtml field must contain the HTML to extract from. No download request is performed and no browser rendering takes place. The url field is still required. If not specified, `browserHtml` is currently used by default for [AI extraction](/zyte-api/usage/extract/index.md), while `httpResponseBody` is used by default for [non-AI extraction](/zyte-api/usage/extract/index.md). In the future, the default value may depend on the target website. This field does not work with serp; for `serp`, use serpOptions.extractFrom instead. RequestHeaders: description: | HTTP request headers. Can only be used in a [browser request](/zyte-api/usage/browser.md). For [HTTP requests](/zyte-api/usage/http.md), see customHttpRequestHeaders. At the moment it only supports the [Referer header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referer). [See an example](/zyte-api/usage/browser.md). properties: referer: description: | [Referer header](https://developer.mozilla.org/en-US/docs/Web/HTTP/Headers/Referer). type: string example: https://search.example/ CustomHttpRequestHeader: properties: name: type: string maxLength: 200 example: X-APOLLO-OPERATION-NAME value: type: string maxLength: 7000 example: nearByNodes Session: description: | Parameters to create or reuse a [client-managed session](/zyte-api/usage/features.md). If `id` does not match one of your running sessions, a new session is created with that session ID. Otherwise, the matching running session is reused. Client-managed sessions may expire due to any of the following: - 15 minutes (900 seconds) have passed since the session was created. - 2 minutes (120 seconds) have passed since the session use. - For 3 times in a row, requests using this session got banned. For 5-10 minutes after a session expires, Zyte API keeps track of the expired session and does not allow re-using it. After that time, attempts to reuse the session will instead create a new session. [See an example](/zyte-api/usage/features.md). example: id: ab837d21-f848-42b2-8e88-47ea9d84bad0 properties: id: description: | User-defined session ID. It must be a [version 4 UUID](https://en.wikipedia.org/wiki/Universally_unique_identifier#Version_4_(random)), i.e. a randomly-generated UUID. type: string type: object Cookie: type: object properties: name: type: string maxLength: 4085 description: Cookie name value: type: string maxLength: 4085 description: Cookie value domain: type: string maxLength: 253 minLength: 1 description: Domain the cookie belongs to path: type: string description: Path the cookie belongs to expires: type: integer format: int64 description: Unix time in seconds. httpOnly: type: boolean secure: type: boolean sameSite: type: string enum: - Strict - Lax - Extended - None required: - name - value - domain PostalAddress: description: Postal address to be set type: object properties: addressCountry: description: The country code in [ISO 3166-1 alpha-2](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2) type: string example: US addressRegion: description: The region in which the address is. This value is specific to the website. type: string example: California streetAddress: description: The street address. type: string postalCode: description: The postal code. type: string NetworkCaptureFilterSequence: type: array maxItems: 10 description: | Filters to capture browser network responses. HTTP responses received during browser rendering (including action execution) will be returned in the networkCapture response field if they match any of the filters defined here. You can capture up to 10 responses, provided the sum of their bodies does not exceed 5 MiB. If they do exceed that limit, only the first captured responses within the limit are returned. [See an example](/zyte-api/usage/browser.md). items: $ref: '#/components/schemas/NetworkCaptureFilter' NetworkCaptureFilter: type: object # Note: In the rendered docs, this description only appears in the # networkCapture.filter response field, so the wording is tailored for # that. description: | Filter defined in the networkCapture request field that matched the captured response. properties: filterType: type: string enum: - url - resourceType httpResponseBody: type: boolean default: false description: | Set to `true` to get the body of the captured response in the [networkCapture[].httpResponseBody](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/response/200/networkCapture.httpResponseBody) response field. required: - filterType discriminator: propertyName: filterType mapping: url: '#/components/schemas/UrlFilter' resourceType: '#/components/schemas/ResourceTypeFilter' UrlFilter: description: An object specifying how to capture responses by matching URL allOf: - $ref: '#/components/schemas/NetworkCaptureFilter' - properties: value: type: string minLength: 3 maxLength: 8192 description: | A string to compare with the URL of network responses according to `matchType`. matchType: $ref: '#/components/schemas/PatternMatchingOptions' - required: - value - matchType ResourceTypeFilter: description: An object specifying how to capture responses by resource type allOf: - $ref: '#/components/schemas/NetworkCaptureFilter' - properties: resourceType: type: string enum: - document - xhr description: | A resource type for a network response to match: - `document` is the source HTML document, which might change during browser rendering or through actions. - `xhr` is a response obtained using [XMLHttpRequest](http://devdoc.net/web/developer.mozilla.org/en-US/docs/XMLHttpRequest.1.html). - required: - resourceType CapturedResponse: type: object properties: interceptionStatus: type: object description: | Exit status of the network capture. If `interceptionStatus.status` is `error`, `httpResponseBody` is not delivered. Possible causes of error include all matching responses exceeding the maximum total body size of 5 MiB. properties: status: type: string enum: - success - error error: type: string description: | Error message. This field is only present if `interceptionStatus.status` is `error`. statusCode: type: integer description: HTTP status code of the captured response. httpResponseBody: type: string format: byte description: | [Base64](https://en.wikipedia.org/wiki/Base64)-encoded body of the captured response. To get this response field, set the [networkCapture[].httpResponseBody](https://docs.zyte.com/zyte-api/usage/reference.html#operation/extract/request/networkCapture.httpResponseBody) request field to `true`. url: type: string format: uri description: Captured response URL. headers: type: object description: Captured response headers. filter: $ref: '#/components/schemas/NetworkCaptureFilter' request: type: object description: Captured request that got the captured response. properties: url: type: string description: URL of the captured request. headers: type: object description: Headers of the captured request. method: type: string description: HTTP method of the captured request. body: type: string description: Body of the captured request, if any. Viewport: type: object description: | [Browser viewport](https://developer.mozilla.org/en-US/docs/Glossary/Viewport). properties: width: type: integer description: Viewport width, in pixels. default: 1920 minimum: 320 maximum: 5120 height: type: integer description: Viewport height, in pixels. default: 1080 minimum: 360 maximum: 4096 CountryCode: description: | [ISO 3166-1 alpha-2](https://en.wikipedia.org/wiki/ISO_3166-1_alpha-2) code of a country from which the request should be sent, i.e. the request [geolocation](/zyte-api/usage/features.md). If not specified, Zyte API will use a geolocation that, for the target website, does not cause bans or unexpected locale changes in the response data, such as the wrong language, currency, date format, time zone, etc. If you believe Zyte API is using the wrong default geolocation for a website, please [reach out to our expert anti-ban team](https://support.zyte.com/support/tickets/new). For some websites, however, you might want to set a custom geolocation. For example, you may be interested in visiting the same URL from different locations. Zyte API provides 2 sets of geolocations. Standard geolocations are `AU`, `BE`, `BR`, `CA`, `CN`, `DE`, `ES`, `FR`, `GB`, `IN`, `IT`, `JP`, `KR`, `MX`, `NL`, `PL`, `RU`, `TR`, `US`, and `ZA`. All other geolocations are [extended geolocations](/zyte-api/usage/features.md). [See an example](/zyte-api/usage/features.md). example: US type: string enum: - AW - AF - AO - AI - AX - AL - AD - AE - AR - AM - AS - AQ - TF - AG - AU - AT - AZ - BI - BE - BJ - BQ - BF - BD - BG - BH - BS - BA - BL - BY - BZ - BM - BO - BR - BB - BN - BT - BV - BW - CF - CA - CC - CH - CL - CN - CI - CM - CD - CG - CK - CO - KM - CV - CR - CU - CW - CX - KY - CY - CZ - DE - DJ - DM - DK - DO - DZ - EC - EG - ER - EH - ES - EE - ET - FI - FJ - FK - FR - FO - FM - GA - GB - GE - GG - GH - GI - GN - GP - GM - GW - GQ - GR - GD - GL - GT - GF - GU - GY - HK - HM - HN - HR - HT - HU - ID - IM - IN - IO - IE - IR - IQ - IS - IL - IT - JM - JE - JO - JP - KZ - KE - KG - KH - KI - KN - KR - KW - LA - LB - LR - LY - LC - LI - LK - LS - LT - LU - LV - MO - MF - MA - MC - MD - MG - MV - MX - MH - MK - ML - MT - MM - ME - MN - MP - MZ - MR - MS - MQ - MU - MW - MY - YT - NA - NC - NE - NF - NG - NI - NU - NL - NO - NP - NR - NZ - OM - PK - PA - PN - PE - PH - PW - PG - PL - PR - KP - PT - PY - PS - PF - QA - RE - RO - RU - RW - SA - SD - SN - SG - GS - SH - SJ - SB - SL - SV - SM - SO - PM - RS - SS - ST - SR - SK - SI - SE - SZ - SX - SC - SY - TC - TD - TG - TH - TJ - TK - TM - TL - TO - TT - TN - TR - TV - TW - TZ - UG - UA - UM - UY - US - UZ - VA - VC - VE - VG - VI - VN - VU - WF - WS - YE - ZA - ZM - ZW OrganicResult: type: object properties: description: type: string description: Result excerpt. example: > Squid is a caching proxy for the Web supporting HTTP, HTTPS, FTP, and more. It reduces bandwidth and improves response times by caching and reusing frequently- ... name: type: string description: Result title. example: squid-cache.org url: $ref: '#/components/schemas/OrganicResultURL' rank: type: integer example: 1 description: | Result position among organic results in the search page. The first result of a search page is always 1, regardless of the value of serp.pageNumber. displayedUrlText: type: string example: 1.4M+ followers description: Extra information present in serp result Metadata: type: object description: Metadata. properties: displayedQuery: type: string description: Search query as seen in the webpage. example: squid proxy searchedQuery: type: string description: Search query as specified in the input URL. example: squid proxy totalOrganicResults: type: integer format: int64 description: | Total number of organic results reported by the search engine. minimum: 0 example: 10000 dateDownloaded: type: string description: | The timestamp at which the data was downloaded. Timezone: UTC. Format: ISO 8601 format: "YYYY-MM-DDThh:mm:ssZ" example: '2024-02-29T13:01:54Z' Serp: type: object properties: organicResults: type: array items: $ref: '#/components/schemas/OrganicResult' url: $ref: '#/components/schemas/SearchURL' pageNumber: type: integer description: Current page number minimum: 1 metadata: $ref: '#/components/schemas/Metadata' OrganicResultURL: type: string pattern: ^https?://[\S]+$ description: Result URL. example: https://www.squid-cache.org/ additionalProperties: false SearchURL: type: string pattern: ^https?://[\S]+$ description: | Search URL. Should match url. example: https://search.example/search?q=squid+proxy additionalProperties: false SerpOptions: type: object description: Options for SERP extraction. properties: extractFrom: type: string enum: - browserHtml - httpResponseBody description: | Input to use for extraction, either httpResponseBody or browserHtml. If not specified, `httpResponseBody` is currently used by default. In the future, the default value may depend on the target website. keyword: type: string description: | The keyword or search phrase to query in the SERP extraction process. Example: `"best scraping framework"` click: allOf: - properties: action: enum: - click description: Click on an element. selector: $ref: '#/components/schemas/ActionSelector' button: description: Mouse button to click type: string enum: - left - right - middle default: left delay: description: Time to wait between mousedown and mouseup, in seconds. type: number minimum: 0 maximum: 3 default: 0 waitForNavigationTimeout: description: | Maximum waiting time in seconds for the navigation event during the click action. If navigation happens within the defined duration, then waiting is halted and the next action is executed after the new is page is loaded. If the page loading does not finish then the next action ends with an error, and following actions may not be executed, depending on the onError property. If no navigation happens within the defined duration then the next action is executed. type: number minimum: 0 maximum: 20 default: 0 required: - selector - action - $ref: '#/components/schemas/Action' doubleClick: allOf: - properties: action: enum: - doubleClick description: Double click on an element. selector: $ref: '#/components/schemas/ActionSelector' required: - selector - action - $ref: '#/components/schemas/Action' evaluate: allOf: - properties: action: enum: - evaluate description: | Run JavaScript code in the page context. This is a very powerful action. Use cases include: - Sending an API request from the page context, and writing the response somewhere in the DOM, so that the [browser HTML](/zyte-api/usage/browser.md) output includes it. source: description: JavaScript code to run. type: string maxLength: 6000 required: - source - action - $ref: '#/components/schemas/Action' goto: allOf: - properties: action: enum: - goto description: | Navigate to a new page. This action waits until page load event is fired with a default timeout of 30 seconds. url: description: URL to navigate page to. The url should include scheme type: string options: $ref: '#/components/schemas/GoToOptions' required: - url - action - $ref: '#/components/schemas/Action' hide: allOf: - properties: action: enum: - hide description: Hide an element. selector: $ref: '#/components/schemas/ActionSelector' required: - action - selector - $ref: '#/components/schemas/Action' hover: allOf: - properties: action: enum: - hover description: | Hover over a visible element. Elements that are either hidden or not present will cause the action to exit with an error. selector: $ref: '#/components/schemas/ActionSelector' required: - selector - action - $ref: '#/components/schemas/Action' interaction: allOf: - properties: action: enum: - interaction description: | Execute a [browser script](//zyte-api/ide/index.md). id: description: Script identifier type: string args: description: Input arguments type: object required: - id - action - $ref: '#/components/schemas/Action' keyPress: allOf: - properties: action: enum: - keyPress description: Press a key on the keyboard. key: type: string maxLength: 14 description: | Key to press. A single character or special key from the [list of supported keys](/zyte-api/ide/api/index.md). Key names are case-sensitive. Only one key can be executed at a time. Key combinations are not supported. required: - key - action - $ref: '#/components/schemas/Action' reload: allOf: - properties: action: enum: - reload description: | Reload the page. This action waits until page load event is fired with a default timeout of 30 seconds. options: $ref: '#/components/schemas/GoToOptions' required: - action - $ref: '#/components/schemas/Action' scrollBottom: allOf: - properties: action: enum: - scrollBottom description: | Continuously scroll down the page while it keeps loading more content. The action halts if any of the following conditions are met: - the timeout or the total browser execution time is reached - the page does not load any new content for the duration of maxScrollDelay - maxPageHeight or maxScrollCount have been reached timeout: description: Maximum wait time, in seconds. type: number minimum: 0.0 default: 15.0 maximum: 30.0 maxScrollDelay: description: | The maximum amount of time to wait for each scroll to complete, in seconds. If the page does not not load any content during this time, the action is deemed to have been completed. type: number default: 5 minimum: 0.5 maximum: 10 maxPageHeight: description: Maximum height (in pixels) until which the browser keeps scrolling down the page type: integer maxScrollCount: description: | The maximum number of scrolls to perform. If the page does not yield any fresh content, then the action will finish execution before maxScrollCount is reached. type: integer scrollStep: description: | The number of pixels for each scroll. It can be used for gradual scrolling. If it's specified, maxScrollDelay will be used as fixed time waiting instead of waiting for new contents. type: integer default: 0 minimum: 0 required: - action - $ref: '#/components/schemas/Action' scrollTo: allOf: - properties: action: enum: - scrollTo description: | Scroll the window to a particular place in the document. To set the target location, use one (and only one) of the following: - `top` and `left`, to set the target coordinates in pixels. - `selector`, to target the center of an HTML element. top: description: Specifies the number of pixels along the Y axis to scroll the window. type: integer left: description: Specifies the number of pixels along the X axis to scroll the window. type: integer default: 0 selector: allOf: - $ref: '#/components/schemas/ActionSelector' - description: If passed scrolls to specified selector instead of scrolling to specified coordinates within page. If selector is not found no scroll is performed. If more than one elements match selector it scrolls to the first one. required: - action - $ref: '#/components/schemas/Action' searchKeyword: allOf: - properties: action: enum: - searchKeyword description: | Perform keyword search on the page. This action uses website-specific knowledge to find and use a search box. It may not work on some websites. If that’s the case, please [reach out to us](https://support.zyte.com/support/tickets/new). If there is no search box on a page, an error is returned. keyword: description: The keyword to be searched for type: string required: - keyword - action - $ref: '#/components/schemas/Action' select: allOf: - properties: action: enum: - select description: | Pick single or multiple values from a `` has the multiple attribute, all values are considered, otherwise only the first one is taken into account. type: array items: type: string required: - selector - values - action - $ref: '#/components/schemas/Action' setLocation: allOf: - properties: action: enum: - setLocation description: | Configure a physical address on the website. This action uses website-specific knowledge to find and fill a location form. It may not work on some websites. If that’s the case, please [reach out to us](https://support.zyte.com/support/tickets/new). address: $ref: '#/components/schemas/PostalAddress' required: - action - $ref: '#/components/schemas/Action' type: allOf: - properties: action: enum: - type description: Type text into an element. selector: $ref: '#/components/schemas/ActionSelector' text: description: | Text to type into a focused element. To press a special key, use the `keyPress` action instead. type: string delay: description: Time to wait between key presses, in seconds. type: number minimum: 0 default: 0 required: - selector - text - action - $ref: '#/components/schemas/Action' waitForNavigation: allOf: - properties: action: enum: - waitForNavigation description: | Wait until the page navigates to a new URL or reloads. If `waitForNavigation` is the first action, the specified options will be applied to the initial navigation. Use it if the default timeout of 30 seconds or the default `waitUntil` value (`load`) is not sufficient for the initial navigation. Mind, however, that using `waitForNavigation` as the first action has an important drawback: any error with the initial navigation will nonetheless result in a successful API response, as any other [browser action failure](/zyte-api/usage/errors.md). timeout: type: number maximum: 45.0 minimum: 31.0 default: 31.0 description: | Maximum wait time, in seconds. waitUntil: default: load description: | When to consider that navigation succeeded: - `load` - [load event](https://developer.mozilla.org/en-US/docs/Web/API/Window/load_event), default. - `domcontentloaded` - [DOMContentLoaded event](https://developer.mozilla.org/en-US/docs/Web/API/Window/DOMContentLoaded_event). - `networkidle0` - no ongoing network connections for at least 0.5 seconds. type: string enum: - load - domcontentloaded - networkidle0 required: - timeout - action nullable: false - $ref: '#/components/schemas/Action' waitForRequest: allOf: - properties: action: enum: - waitForRequest description: Wait until the request to a specific URL has been sent. urlPattern: $ref: '#/components/schemas/UrlPattern' urlMatchingOptions: $ref: '#/components/schemas/PatternMatchingOptions' timeout: $ref: '#/components/schemas/ActionTimeout' example: # To wait for a request to https://example.org/store/api/ref=sspa_dk_left_sx_aax_0 - urlPattern: https://example.org/store/api # To wait for a request to https://cdn123.example.org/api/store?q=1234 - urlPattern: api/store urlMatchingOptions: contains # To wait for a request to https://example.org/afsk123/ref=sspa_dk_left_sx_aax_0 - urlPattern: https://example.org/ urlMatchingOptions: startsWith required: - urlPattern - action - $ref: '#/components/schemas/Action' waitForResponse: allOf: - properties: action: enum: - waitForResponse description: Wait until the response from a specific URL has been received. urlPattern: $ref: '#/components/schemas/UrlPattern' urlMatchingOptions: $ref: '#/components/schemas/PatternMatchingOptions' timeout: $ref: '#/components/schemas/ActionTimeout' example: # To wait for a response from https://cdn123.example.org/store/api?q=1234 - urlPattern: /store/api urlMatchingOptions: contains # To wait for a response from https://example.org/store/ref=sspa_dk_left_sx_aax_0 - urlPattern: https://example.org/store/ urlMatchingOptions: startsWith required: - urlPattern - action - $ref: '#/components/schemas/Action' waitForSelector: allOf: - properties: action: enum: - waitForSelector description: | Wait for the selector to appear. If at the moment of calling the method the selector already exists, the action will return immediately. Also, the action will return immediately after the first matching selector appears. For a usage example, see the [web scraping tutorial](/web-scraping/tutorial/js.md). selector: $ref: '#/components/schemas/ActionSelector' timeout: $ref: '#/components/schemas/ActionTimeout' required: - selector - action - $ref: '#/components/schemas/Action' waitForTimeout: allOf: - properties: action: enum: - waitForTimeout description: | Pause script execution for the given number of seconds before continuing. If the value of timeout is greater than the remaining browser execution time, then this action ends with an error. timeout: $ref: '#/components/schemas/ActionTimeout' required: - action - $ref: '#/components/schemas/Action' SessionContextActionSequence: description: | Actions to run to initialize a server-managed session for a given sessionContext). type: array items: oneOf: - $ref: '#/components/schemas/click' - $ref: '#/components/schemas/doubleClick' - $ref: '#/components/schemas/evaluate' - $ref: '#/components/schemas/goto' - $ref: '#/components/schemas/hide' - $ref: '#/components/schemas/hover' - $ref: '#/components/schemas/interaction' - $ref: '#/components/schemas/keyPress' - $ref: '#/components/schemas/reload' - $ref: '#/components/schemas/scrollBottom' - $ref: '#/components/schemas/scrollTo' - $ref: '#/components/schemas/searchKeyword' - $ref: '#/components/schemas/select' - $ref: '#/components/schemas/setLocation' - $ref: '#/components/schemas/type' - $ref: '#/components/schemas/waitForNavigation' - $ref: '#/components/schemas/waitForRequest' - $ref: '#/components/schemas/waitForResponse' - $ref: '#/components/schemas/waitForSelector' - $ref: '#/components/schemas/waitForTimeout' CustomAttribute: type: object properties: description: type: string maxLength: 300 type: type: string enum: - boolean - string - number - integer - array - object discriminator: propertyName: type mapping: boolean: '#/components/schemas/CustomAttributeBoolean' string: '#/components/schemas/CustomAttributeString' number: '#/components/schemas/CustomAttributeNumber' integer: '#/components/schemas/CustomAttributeInteger' array: '#/components/schemas/CustomAttributeArray' object: '#/components/schemas/CustomAttributeObject' CustomAttributeBoolean: allOf: - $ref: '#/components/schemas/CustomAttribute' type: object required: - type CustomAttributeString: allOf: - $ref: '#/components/schemas/CustomAttribute' - properties: enum: type: array minItems: 2 maxItems: 100 items: type: string maxLength: 50 minLength: 1 format: type: string enum: - html - uri - html-text - xpath required: - type CustomAttributeNumber: allOf: - $ref: '#/components/schemas/CustomAttribute' - properties: enum: type: array minItems: 2 maxItems: 10 items: type: number type: object required: - type CustomAttributeInteger: allOf: - $ref: '#/components/schemas/CustomAttribute' - properties: enum: type: array minItems: 2 maxItems: 10 items: type: integer type: object required: - type CustomAttributeArray: allOf: - $ref: '#/components/schemas/CustomAttribute' properties: items: $ref: '#/components/schemas/CustomAttribute' type: object required: - type - items CustomAttributeObject: allOf: - $ref: '#/components/schemas/CustomAttribute' properties: properties: type: object additionalProperties: $ref: '#/components/schemas/CustomAttribute' type: object required: - type - properties Article: type: object properties: headline: description: Article headline or title. type: string example: Article headline articleBody: description: | Clean text of the article, including sub-headings, with newline separators. type: string example: Article body ... articleBodyHtml: description: | Simplified and standardized HTML of the article body, including sub-headings, image captions and embedded content (videos, tweets, etc.). type: string example:

Article body ...

...
description: description: | A short summary of the article. It can be either human-provided (if available), or auto-generated. type: string example: Article summary datePublished: description: | Publication date. ISO-formatted with 'T' separator, may contain a timezone. If the actual publication date is not found, "dateModified" value is taken. type: string example: '2019-06-19T00:00:00' datePublishedRaw: description: | Same date as "datePublished", but before parsing/normalization, i.e. as it appears on the website. type: string example: June 19, 2019 dateModified: description: | The date when the article was most recently modified. ISO-formatted with 'T' separator, may contain a timezone. type: string example: '2019-06-21T00:00:00' dateModifiedRaw: description: | Same date as "dateModified", but before parsing/normalization, i.e. as it appears on the website. type: string example: June 21, 2019 authors: description: Authors of the article. type: array items: $ref: '#/components/schemas/Author' example: - name: Alice nameRaw: Alice and Bob - name: Bob nameRaw: Alice and Bob inLanguage: description: | Language of the article, as an ISO 639-1 language code. Example: "en". Sometimes article language is not the same as the web page overall language; to get the detected web page languages, see "webPageInfo". type: string example: en breadcrumbs: description: | A list of breadcrumbs (a specific navigation element) with optional `name` and `url`. example: - name: Home url: https://example.com/ - name: Cell Phones url: https://example.com/cell-phones - name: Cell Phones & Accessories type: array items: $ref: '#/components/schemas/Breadcrumb' mainImage: $ref: '#/components/schemas/Image' description: The main image of the item. images: description: All images of the item (may include the main image). type: array items: $ref: '#/components/schemas/Image' videos: description: A list of all videos inside the article body. type: array items: type: object properties: url: description: Absolute URL of the video. type: string example: https://example.com/video.mp4 required: - url audios: description: A list of all audios inside the article body. type: array items: type: object properties: url: description: Absolute URL of the audio. type: string example: https://example.com/audio.mp3 required: - url url: description: URL of a page where this article was extracted. type: string example: https://example.com/article/ canonicalUrl: description: Canonical URL of the article, if available. type: string example: https://example.com/article metadata: $ref: '#/components/schemas/Metadata__metadata' required: - url - metadata Author: description: Author of the article. type: object properties: name: description: Full name of the author, e.g. "Alice". type: string nameRaw: description: Text from which this author name was extracted, e.g. "Alice and Bob". type: string example: name: Alice nameRaw: Alice and Bob required: - name Breadcrumb: description: | Breadcrumb item (a specific navigation element) with optional `name` and `url`. example: name: Home url: https://example.com/ type: object properties: name: description: Text of the breadcrumb, as it appears on the website. type: string url: description: Absolute URL of the breadcrumb. type: string Image: description: Image. type: object properties: url: description: URL of an image. type: string example: http://example.com/item-1/image1.jpeg required: - url Metadata__metadata: description: Extracted item metadata for single-item data types. type: object properties: probability: description: | Probability that extracted item is of requested data type. It is closer to 0 in case this page does not contain requested data type. For example, when single product extraction is requested with "product: true", but a page does not contain a product, probability would be close to 0. If an item of requested type can be extracted from a page, then probability is closer to 1. Recommended probability threshold is 0.5, but we will return extracted data even if probability is very low. type: number minimum: 0.0 maximum: 1.0 example: 0.87 dateDownloaded: $ref: '#/components/schemas/DateDownloaded' required: - probability - dateDownloaded DateDownloaded: description: | The timestamp at which the data was downloaded. Timezone: UTC. Format: ISO 8601 format: "YYYY-MM-DDThh:mm:ssZ" type: string example: '2019-06-19T08:27:43Z' ArticleList: type: object properties: articles: description: List of articles available on this page. type: array items: type: object properties: url: description: | URL of a detailed article page. Pass this URL with "article: true" in the request to extract detailed information about the article. type: string example: https://example.com/articles/1/ headline: description: Article headline or title. type: string example: Article headline articleBody: description: | Text of the article as it appears on the list page, including sub-headings, with newline separators. type: string example: Article body ... datePublished: description: | Publication date. ISO-formatted with 'T' separator, may contain a timezone. type: string example: '2019-06-19T00:00:00' datePublishedRaw: description: | Same date as "datePublished", but before parsing/normalization, i.e. as it appears on the website. type: string example: June 19, 2019 authors: description: Authors of the article. type: array items: $ref: '#/components/schemas/Author' example: - name: Alice nameRaw: Alice and Bob - name: Bob nameRaw: Alice and Bob inLanguage: description: | Language of the article, as an ISO 639-1 language code. Example: "en". Sometimes article language is not the same as the web page overall language; to get the detected web page languages, see "webPageInfo". type: string example: en mainImage: $ref: '#/components/schemas/Image' description: The main image of the item. images: description: All images of the item (may include the main image). type: array items: $ref: '#/components/schemas/Image' metadata: $ref: '#/components/schemas/MetadataListItem' required: - metadata url: description: URL of a page where this article list was extracted. type: string example: https://example.com/articles/ metadata: $ref: '#/components/schemas/MetadataList' required: - url - metadata MetadataListItem: description: Item-level metadata for list data types. properties: probability: description: | Probability that extracted item in a list is a valid item. Items which are unlikely to be valid are not returned, so normally no extra thresholding is needed for list items. This probability is not calibrated. type: number minimum: 0.0 maximum: 1.0 example: 0.34 required: - probability MetadataList: description: Top-level metadata for list data types. properties: dateDownloaded: $ref: '#/components/schemas/DateDownloaded' required: - dateDownloaded ArticleNavigation: type: object properties: nextPage: $ref: '#/components/schemas/PaginationNext' pageNumber: $ref: '#/components/schemas/PageNumber' items: description: List of articles available on this page. type: array items: type: object properties: url: description: | URL of a detailed article page. Pass this URL with "article: true" in the request to extract detailed information about the article. type: string example: https://example.com/articles/1/ name: description: The name of the article or article link text. type: string example: Article name datePublished: description: | Publication date. ISO-formatted with 'T' separator, may contain a timezone. type: string example: '2019-06-19T00:00:00' datePublishedRaw: description: | Same date as "datePublished", but before parsing/normalization, i.e. as it appears on the website. type: string example: June 19, 2019 metadata: $ref: '#/components/schemas/MetadataListItem' required: - url - metadata url: description: URL of a page containing the list of articles. type: string example: https://example.com/articles/ metadata: $ref: '#/components/schemas/MetadataList' required: - url - metadata PaginationNext: description: A link to the next page in the list. type: object properties: url: description: URL of the next page in the list. type: string example: http://example.com/foo?p=3 name: description: Text of the link to the next page, if available. type: string example: '3' required: - url PageNumber: description: Integer describing the current page number. Starts at 1. type: integer example: 2 ForumThread: type: object properties: topic: description: Topic that is discussed on the page. type: object properties: name: description: Name of the topic. type: string example: How do you cook rice? required: - name posts: description: List of posts available on this page, including the first or top post. type: array items: type: object properties: text: description: | Text of the post. type: string example: Cooking rice is a hobby of mine. Here is how I cook it. datePublished: description: | Publication date. ISO-formatted with 'T' separator, may contain a timezone. type: string example: '2019-06-19T00:00:00' datePublishedRaw: description: | Same date as "datePublished", but before parsing/normalization, i.e. as it appears on the website. type: string example: June 19, 2019 reactions: description: Details of reactions to this post. type: object properties: likes: description: | Number of up-votes or likes/stars received by the post. type: integer minimum: 0 example: 3 replies: description: | Number of replies received by the post. type: integer minimum: 0 example: 2 metadata: $ref: '#/components/schemas/MetadataListItem' required: - metadata url: description: URL of a page where this forum post list was extracted. type: string example: https://example.com/forum/thread/1/ metadata: $ref: '#/components/schemas/MetadataList' required: - url - metadata JobPosting: type: object properties: jobTitle: description: The title of the job. type: string example: Regional Manager datePublished: description: | Publication date of the job posting. ISO-formatted with 'T' separator, may contain a timezone. type: string example: '2019-06-19T00:00:00' datePublishedRaw: description: | Same date as 'datePublished', but before parsing/normalization, i.e. as it appears on the website. type: string example: 19 June 2019 validThrough: description: | The date after which the job posting is not valid, e.g. the end of an offer. ISO-formatted with ‘T’ separator, may contain a timezone. type: string example: '2019-08-20T00:00:00' description: description: | A description of the job posting including sub-headings, with newline separators. type: string example: Job Description ... descriptionHtml: description: | Simplified HTML of the description, including sub-headings, image captions and embedded content. type: string example:
HTML for Job Description ... employmentType: description: | Type of employment (e.g. full-time, part-time, contract, temporary, seasonal, internship). type: string example: Full-time hiringOrganization: description: Information about the organization offering the job position. type: object properties: name: description: Name of the organization. type: string example: ACME Corp. required: - name baseSalary: description: | The base salary of the job or of an employee in the proposed role. type: object properties: raw: description: Salary amount as it appears on the website. example: $53,251 a year type: string valueMax: description: | The maximum value of the base salary as a number string. In case of only one value given for the salary instead of a range, valueMax is used to represent it. example: '53251.0' type: string currency: description: | Currency associated with the salary amount. ISO 4217 standard. type: string example: USD currencyRaw: description: Currency associated with the salary amount, without normalization. type: string example: $ jobLocation: description: | A (typically single) geographic location associated with the job position. type: object properties: raw: description: Job location as it appears on the website. type: string example: West New York, NJ 07093 required: - raw url: description: URL of a page where this job posting was extracted. type: string example: https://example.com/job metadata: $ref: '#/components/schemas/Metadata__metadata' required: - url - metadata JobPostingNavigation: type: object properties: nextPage: $ref: '#/components/schemas/PaginationNext' pageNumber: $ref: '#/components/schemas/PageNumber' items: description: List of job postings available on this page. type: array items: type: object properties: url: description: | URL of a detailed job posting page. Pass this URL with "jobPosting: true" in the request to extract detailed information about the job posting. type: string example: https://example.com/jobs/1/ name: description: The name of the job posting or job posting link text. type: string example: Job posting name metadata: $ref: '#/components/schemas/MetadataListItem' required: - metadata - url url: description: URL a of page. type: string example: https://example.com/jobs/ metadata: $ref: '#/components/schemas/MetadataList' required: - url - metadata PageContent: type: object properties: breadcrumbs: description: | A list of breadcrumbs (a specific navigation element). example: - name: Home url: https://example.com/ - name: Category url: https://example.com/category - name: Subcategory type: array items: $ref: '#/components/schemas/Breadcrumb' headline: description: A page headline. type: string example: Example page headline title: description: A page title extracted from the `` tag of the page. type: string example: Example page title itemMain: description: | Text of the primary content of the page. It does not include navigation elements (headers, footers, sidebars or pagination links). type: string example: Example content snippet showing part of the page’s main text… itemMainXPath: description: | XPath for `itemMain`. It is an XPath 1.0 expression that points to the smallest HTML element that contains all of `itemMain`. The expression may only work with an HTML5-compliant parser. type: string example: //*[@id='homepage-container']/*[1] navigationHeader: description: | Navigation items from the header. They are typically for site-wide navigation, not page-specific. type: array items: type: object properties: url: description: URL. type: string example: https://example.com/category/ name: description: Name. type: string example: Category name required: - url navigationFooter: description: | Navigation items from the footer. They are typically for site-wide navigation, not page-specific. type: array items: type: object properties: url: description: URL. type: string example: https://example.com/policy/ name: description: Name. type: string example: Privacy Policy required: - url navigationSidebar: description: | Navigation items from the sidebars. They are typically for site-wide navigation, not page-specific. type: array items: type: object properties: url: description: URL. type: string example: https://example.com/sidebar-link/ name: description: Name. type: string example: Sidebar link required: - url pagination: description: | Pagination items. Items to navigate content pages, either relative to the current page (e.g. current, next, previous) or absolute (e.g. first, last, specific page number). type: array items: type: object properties: url: description: URL. type: string example: https://example.com/?page=2 name: description: Name. type: string example: Next required: - url nextPage: $ref: '#/components/schemas/PaginationNext' url: description: URL of the page. type: string example: https://example.com/example-page/ canonicalUrl: description: Canonical URL of the page, if available. type: string example: https://example.com/canonical-url-page/ metadata: $ref: '#/components/schemas/Metadata__metadata' required: - url - metadata Product: type: object required: - url - metadata properties: name: $ref: '#/components/schemas/Name' price: $ref: '#/components/schemas/Price' currency: $ref: '#/components/schemas/Currency' currencyRaw: $ref: '#/components/schemas/CurrencyRaw' regularPrice: $ref: '#/components/schemas/RegularPrice' availability: $ref: '#/components/schemas/Availability' sku: $ref: '#/components/schemas/Sku' mpn: $ref: '#/components/schemas/Mpn' gtin: description: | Standardized GTIN product identifier which is unique for a product across different sellers. type: array items: $ref: '#/components/schemas/Gtin' brand: description: | Brand or manufacturer of the product. type: object properties: name: description: Name of the brand. type: string example: Product brand required: - name breadcrumbs: description: | A list of breadcrumbs (a specific navigation element) with optional `name` and `url`. example: - name: Home url: https://example.com/ - name: Cell Phones url: https://example.com/cell-phones - name: Cell Phones & Accessories type: array items: $ref: '#/components/schemas/Breadcrumb' mainImage: $ref: '#/components/schemas/Image' description: The main image of the item. images: description: All images of the item (may include the main image). type: array items: $ref: '#/components/schemas/Image' description: description: Description of the product. type: string example: product description descriptionHtml: description: > Simplified HTML of the description, including sub-headings, image captions and embedded content. type: string example: <article>HTML description for Product ... aggregateRating: description: | The overall rating, based on a collection of reviews or ratings. ![](https://docs.zyte.com/_static/images/schemas/rating.png) type: object properties: ratingValue: description: The average rating value. type: number example: 4.0 bestRating: description: The highest value allowed in this rating system. type: number example: 5.0 reviewCount: description: The total number of reviews or ratings for the product. type: integer minimum: 0 example: 24 color: $ref: '#/components/schemas/Color' size: $ref: '#/components/schemas/Size' weight: $ref: '#/components/schemas/Weight' material: description: | The materials from which the product is made. Contains all product materials on the page. type: string example: Metal, Plastic style: $ref: '#/components/schemas/Style' additionalProperties: description: | A list of properties or characteristics. * name field contains the property name, * value field contains the property value. ![](https://docs.zyte.com/_static/images/schemas/product_info.png) type: array items: $ref: '#/components/schemas/AdditionalProperty' features: description: | A list of features of the Product. The features of a Product can be found generally on the product page arranged in a list, which is usually bulleted. type: array items: type: string example: - Multi-System Compatible - HD Ready 1366 x 768 LED Panel - REFRESH RATE 100Hz PQI url: $ref: '#/components/schemas/Url' canonicalUrl: $ref: '#/components/schemas/CanonicalUrl' metadata: $ref: '#/components/schemas/Metadata__metadata' variants: description: | Array of product variants, using the same Product schema. Represents extra information available about the variants of a product. All variants are included into this array, including the variant shown on the page. If some field in this array is empty, it means that either the value is the same as in the top-level product, or that extraction API did not manage to extract it. type: array items: type: object properties: name: $ref: '#/components/schemas/Name' price: $ref: '#/components/schemas/Price' currency: $ref: '#/components/schemas/Currency' currencyRaw: $ref: '#/components/schemas/CurrencyRaw' regularPrice: $ref: '#/components/schemas/RegularPrice' availability: $ref: '#/components/schemas/Availability' sku: $ref: '#/components/schemas/Sku' mpn: $ref: '#/components/schemas/Mpn' gtin: description: | Standardized GTIN product identifier which is unique for a product across different sellers. type: array items: $ref: '#/components/schemas/Gtin' mainImage: $ref: '#/components/schemas/Image' description: The main image of the item. images: description: All images of the item (may include the main image). type: array items: $ref: '#/components/schemas/Image' color: $ref: '#/components/schemas/Color' size: $ref: '#/components/schemas/Size' style: $ref: '#/components/schemas/Style' additionalProperties: description: | A list of properties or characteristics. * name field contains the property name, * value field contains the property value. ![](https://docs.zyte.com/_static/images/schemas/product_info.png) type: array items: $ref: '#/components/schemas/AdditionalProperty' url: $ref: '#/components/schemas/Url' canonicalUrl: $ref: '#/components/schemas/CanonicalUrl' Name: description: The name of the product. type: string example: Product name Price: description: > The price at which the product is being offered. If there is only one price associated with the offer, it is returned in this field. type: string pattern: ^[0-9]+(\.[0-9]+)?$ example: '149' Currency: description: > The ISO 4217 standard of the currency in which the price is in. type: string pattern: ^[A-Z]{3}$ example: USD CurrencyRaw: description: > The currency as given on the website, without extra normalization (for example, both "$" and "USD" are possible currencies). type: string example: $ RegularPrice: description: > The price before any discount or special offer. type: string pattern: ^[0-9]+(\.[0-9]+)?$ example: '199.00' Availability: description: > Availability, as a string. Allowed values: * `"InStock"` - includes limited availability, presale, preorder, and in-store only. * `"OutOfStock"` - includes discontinued and sold out. example: InStock type: string enum: - InStock - OutOfStock Sku: description: | The Stock Keeping Unit (SKU), i.e. a merchant-specific identifier for the product - identifier assigned by the seller. ![](https://docs.zyte.com/_static/images/schemas/sku.png) example: A123DK9823 type: string Mpn: description: The Manufacturer Part Number (MPN) of the product. It is issued by the manufacturer, and is the same across different e-commerce websites. type: string example: code-123 Gtin: type: object description: > Standardized GTIN product identifier which is unique for a product across different sellers. example: type: isbn13 value: 9781933624341 properties: type: description: | `gtin14` corresponds to former names *EAN/UCC-14*, *SCC-14*, *DUN-14*, *UPC Case Code*, *UPC Shipping Container Code*. `gtin13` also includes the *jan* (japanese article number). enum: - gtin8 - gtin13 - gtin14 - isbn10 - isbn13 - ismn - issn - upc type: string value: description: The GTIN value as a string. type: string required: - type - value Color: description: Color of the product. type: string example: Red Size: description: | A standardized size of a product, specified through a simple textual string (for example "XL", "32Wx34L"). A single product dimension (height, width) is not considered as the size. type: string example: XL Weight: type: object properties: value: description: | A weight value expressed as a floating point number. type: number example: 120.0 unit: description: | A normalized unit of weight, like kilogram / ounce / pound and others. type: string example: kilogram rawUnit: description: | A unit of weight without normalization - how it was extracted from the page. Normalized version of the rawUnit is in 'unit' attribute. type: string example: kg Style: description: | Style of the product. It can also be referred as pattern/finish on the product page. Example values: "Polka dots", "Striped", "Nickel finish with Translucent glass", etc. type: string example: Striped AdditionalProperty: description: | Additional propertiy or characteristics. * name field contains the property name, * value field contains the property value. ![](https://docs.zyte.com/_static/images/schemas/product_info.png) example: name: batteries value: 1 Lithium ion batteries required. (included) type: object properties: name: description: Property name. type: string value: description: Property value. type: string required: - name Url: description: URL of a page where this product was extracted. type: string example: https://example.com/product/ CanonicalUrl: description: Canonical URL of the product, if available. type: string example: https://example.com/product/ ProductList: type: object properties: breadcrumbs: description: | A list of breadcrumbs (a specific navigation element) with optional `name` and `url`. example: - name: Home url: https://example.com/ - name: Cell Phones url: https://example.com/cell-phones - name: Cell Phones & Accessories type: array items: $ref: '#/components/schemas/Breadcrumb' products: description: List of products available on this page. type: array items: type: object properties: url: description: | URL of a detailed product page. Pass this URL with "product: true" in the request to extract detailed information about the product. type: string example: https://example.com/products/1/ name: description: The name of the product. type: string example: Product name price: $ref: '#/components/schemas/Price' currencyRaw: $ref: '#/components/schemas/CurrencyRaw' currency: $ref: '#/components/schemas/Currency' regularPrice: $ref: '#/components/schemas/RegularPrice' mainImage: $ref: '#/components/schemas/Image' description: The main image of the item. metadata: $ref: '#/components/schemas/MetadataListItem' required: - metadata url: description: URL of a page where this product list was extracted. type: string example: https://example.com/products/ metadata: $ref: '#/components/schemas/MetadataList' categoryName: description: Name of the category in which the listed products are. type: string example: Sports & Outdoors required: - url - metadata ProductNavigation: type: object properties: categoryName: description: Name of the category in which the listed products are found. type: string example: Sports & Outdoors nextPage: $ref: '#/components/schemas/PaginationNext' pageNumber: $ref: '#/components/schemas/PageNumber' items: description: List of products available on this page. type: array items: $ref: '#/components/schemas/NavigationRequest' subCategories: description: List of subcategory links found on this page. type: array items: $ref: '#/components/schemas/NavigationRequest' url: description: URL of the page. type: string metadata: $ref: '#/components/schemas/MetadataList' required: - url - metadata NavigationRequest: type: object properties: url: description: URL of the item. type: string example: https://example.com/products/category/1 name: description: The name or link text of the item. type: string example: In Her Wake method: description: HTTP method associated with the navigation request for product, subcategory type: string enum: [GET, POST, PUT, DELETE, OPTIONS, TRACE, PATCH] headers: description: List of headers associated with the request. type: array items: type: object properties: name: type: string description: Header name. value: type: string description: Header value. body: description: Base64-encoded body of the request. type: string format: byte metadata: $ref: '#/components/schemas/MetadataListItem' required: - url - metadata