SAP Community - Document Information Extraction

How to upload documents in bulk using a background job - Document information Extraction

2025-05-21T09:50:18.336000+02:00

hi guys,

Could you let me know if there's an option to upload documents to Document Information Extraction via a batch job or another tool/program, in a more automated way than uploading document by document manually by a user? And if yes, how to achieve this?

I have a case where I will receive hundreds of scanned documents every day (purchase orders) in a specific folder, from which a job or program should pick them up one by one and try to upload them to Document Information Extraction (DIE).

best,

Ewa

Table change not possible. There is a db trigger on the table

2025-07-17T11:01:03.997000+02:00

"Table change not possible. There is a DB trigger on the table." I am facing this issue; how to solve this?

Oppt Prioritization score custom Agent in SAP CX AI Toolkit

2025-07-22T17:26:06.300000+02:00

Hi Experts,

I am trying to create a custom AI agent for opportunities called Oppt Prioritization score agent. For this i created an AI Tool by giving the following prompt:

You are an opportunity prioritization agent in the SAP CX AI Toolkit for SAP Sales Cloud V2. Analyze all opportunities with Opprtunity.Status with Open status.

Calculate the prioritization score based on the following weighted criteria:
1. Opportunity.Expected_Value (50% weight)
2. Opportunity.Probability (30% weight)
3. Opportunity.Close_Date within 30 days from the Opportunity.Current_Date (20% weight)
Assumptions
4.Opportunity.Probability is assumed to be a percentage (0-100).
5.The Opportunity.Close_Date condition is binary, based on whether it falls within 30 days from the Opportunity.Current_Date, interpreted as future dates for prioritization.

Priotitization Score = 0.5*{Opportunity.Expected_Value} + 0.3*{Opportunity.Probability} + 0.2*{cases}
1 {ifOpportunity.Close_Date is within 30 days from the Opportunity.Current_Date} else
0
end{cases}

Provide a one-sentence explanation per opportunity justifying its ranking and, sorted by Amount (descending). Use a professional tone and avoid technical jargon.
For each opportunity, provide a brief, actionable follow-up recommendation (e.g., “Schedule a discovery call to discuss budget constraints”). Use a concise, professional tone and leverage intelligent Q&A to ensure recommendations are contextually relevant.
Include a summary of recent interactions for each opportunity, generated using the toolkit’s AI summary capabilities. Use a professional tone and ensure recommendations support team collaboration.
Ensure Output Fits Within 5000 Chatacters
Compact Formatting--> Remove unnecessary spaces & line breaks.
Limit Answer Length--> Max 500 Characters per answer, truncate with "..."

I assigned this AI tool to the custom AI Agent.The details (PFA screenshot) which are used are

Expected value = 15000 USD

Probability = 80

Close Date = 25 Jul 2025

The calculation for close date is incorrect, it should be 1, but it is coming as 0. Any suggestions how can i achieve the condition for the close date?

SAP Business Process Automation with Document Information Extraction - extract data using AI

2025-07-24T17:10:26.475000+02:00

Greetings,

I am building a automation process and want to use the activity "Extract Data Using AI (Premium), reference can be found here IRPA SDK V2.

It is marked as Alpha, so I'm not sure what that means. I am using Document Information Extraction SDK version 1.75.21 (the latest) but this activity is not available within the SDK.

Can you please let me know when it will be available as it is key to the process I am building. As we have many document types I do not want to extract based on a template. We do have DOX premium service to allow for gen AI extraction based on the schema only.

Many thanks in advance for your help, Adrian

"message": "Missing parameter: options" in SAP Document AI Upload API

2025-07-31T12:56:28.550000+02:00

Hi Team,

While working with the document upload api (below) in SAP document AI API from BAS

../document-information-extraction/v1/document/jobs getting the error in network response -

{
"error": {
"code": "E95",
"message": "Required form-data not provided.",
"details": [
{
"code": "0",
"message": "Missing parameter: options"
}
]
}
}

while the same is working fine from Postman

Any help is greatly appreciated. Thank you in advance.

SAP Document AI

Is it possible to structure DOX files in folders and subfolders?

2025-08-07T14:51:43.064000+02:00

When we load documents via the DOX web UI or via API endpoint, the documents are just places in one big unstructured stack and the UI seems to have no option to structure these files and even not very good search functionality.

I have below test setup with currently only 3 documents, but when wh go live I expect we will have maybe hundreds of documents loaded every day and the documents will be for different solutions like purchase order, sales invoices, and some custom made documents for certificates.
It is not nice that these documents are just in a big stack and I can't even filter on template or schema.

Or can I? Are there some tools that I didn't see?

Kind Regards
Thomas Madsen Nielsen

Customizing Extraction Fields in SAP Document Information Extraction (DOX)

2025-08-12T19:47:26.625000+02:00

Hello,

I’m currently evaluating SAP Document Information Extraction (DOX) in SAP Document AI using a trial account. Although I’ve noticed that it extracts every text very accurately, not every text is being mapped to the extraction fields.

I wanted to add/customize header fields myself to map the extracted text to header fields or line items better (while following this link https://help.sap.com/docs/document-ai/sap-document-ai/add-data-fields to add data fields) but schema configuration doesn’t appear in my trial environment. It also states that I cannot change SAP schemas as shown in the attached image below:

So I was wondering if it's possible to edit/customize extraction fields in the premium version of SAP Document AI? If so, does this customization occur before the optical character recognition (OCR) to guide the model, or only after OCR to map extracted text to fields?

Defining custom field mappings before OCR would be extremely valuable for my team's use case, so I’d greatly appreciate if anyone can explain what’s possible before I consider upgrading to a full subscription. Thank you.

How do I make DOX learn?

2025-08-25T14:36:45.387000+02:00

Hi

I trying to load some purchase orders with different layouts. I Need to extract at least one field that is not included in the SAP standard schema (EAN no.), so I made a copy of the SAP_purchaseOrder_schema and added the ean header field to the copy.

The new field can have setup type Manual or Auto (with a default extractor). As I understand the documentation, a custom field must have setup type Manual. Also I understand that only setup type Auto will use the AI functionality and if I chose Manual setup type, I must make a template. Does this mean that fields with Auto are handled by the AI and fields with Manual are depending on the template?

So I have made the new field with setup type Manual and I have made a template.

Then I uploaded a file. Some data was extracted wrongly so I entered Edit mode and corrected the errors and saved the document again. Then I addet the document to the template.

Now I expected DOX to have learned from the corrections I made, but if I load exactly same document again, it still makes the same errors.

So, can anyone explain how DOX can learn and what setup influence the learning and what I can to to teach DOX?

Also I would like to understand, if I have 3 purchase orders with 3 very different layouts, can I handle this by adding the 3 documents to one template or must I create 3 templates or is it even better to create 3 different schemas?

I hope it is allright that I asked 3 questions in this thread as I think they are all related to the subject - how does DOX learn and fix extraction errors?

Kind Regards
Thomas Madsen Nielsen

The limited no of documents in the Trial version of SAP Document AI must increase to make sense

2025-09-02T13:47:49.518000+02:00

Hi

Depending on where you read documentation, the limited no of document that can be loaded in a Trial DOX is either 30 or 50 documents per month - with the addition that it counts one document per every 3 pages in a file.

This very little - almost useless. On top of this the usage statistics in BTP are unreliable, so the counted usage is higher than reality.

I have a BTP trial account with a trial DOX where I have loaded 17 document. 7 in August and 10 in September, but the usage statistics says I have loaded 40 documents in September and now my quota is used. I know 100% sure I loaded the same document 10 times in september and the document was only 2 pages, so it should count 10 documents instead of 40.

Also it is frustrating that the usage statistics says I loaded 47 documents, but even if that was correct (it isn't), the documentation says that the quota is 50 documents per 30 days (rolling period), so according to documentation my quota is not used in any way.

If SAP wants to sell SAP Document AI, they should increase the quota. If we can't really test it, why should we buy it?

Also I find it disturbing that the usage statistics count usage that does not exist. I have seen this problem not only in trail versions, but also in Pay-As-You-Go.

Just my 2 cents.

Thomas Madsen Nielsen

How to Integrate SAP Document AI with SAP Build Process Automation

2025-09-04T14:02:04.049000+02:00

Perfect 👍 Here’s your full rephrased + enhanced message including the part about creating an Action for file upload. You can use this directly in your SAP Community or Blog:

I have created an instance of SAP Document AI and built a few custom schemas to read PDFs. Everything works fine when tested directly in Document AI.

Now, I want to integrate it with SAP Build Process Automation (BPA). For this, I have already created a destination.

My question:
👉 How can I create an Action in BPA to upload a file to Document AI?

Do we need to use a Custom Script to achieve this, or is it recommended to only create Actions in BPA and then call them in a process?

If the recommended approach is to use an Action, can someone guide me on how to create an Action in BPA for file upload?

Specifically:

How should the input parameters (like PDF file and options JSON) be defined in the Action?
How do we configure the multipart/form-data request inside the Action?
Once created, can this Action be directly reused in different BPA processes for uploading files?

Would really appreciate your guidance. Thanks in advance!

For reference, below are the steps I followed in Postman which work successfully:

Step 1: Upload the PDF

Call the Upload API with multipart/form-data.

file → PDF document
options → JSON with schema details

Example:

{
  "schemaId": "69d84f47-b658-46fb-a330-d9aadd602112",
  "clientId": "default",
  "documentType": "invoice"
}

👉 Response returns a Job ID.

Step 2: Check Job Status

Use the Get Job API with the Job ID.
Proceed when status = SUCCEEDED.

Step 3: Fetch Extracted Data

Call the Get Result API with the Job ID to get the extracted data in JSON format (header details, line items, etc.).

✅ Now, I’m looking for help on how to configure this same upload flow as an Action in SAP BPA, so I can call it directly from an automation.

SAP Document AI - item splitted to 2 pages

2025-09-09T13:36:18.246000+02:00

Hi everyone,

In SAP Document AI (Premium version), I run into a limitation when invoices or other documents contain line items that are split across two pages.

Instead of treating them as one item, the service always creates two separate items. I know this is a known limitation, but I’m trying to understand if there are any workarounds or configuration options that can help.

So far I’ve tried:

Enabling Instant Learning
Setting dataFeedbackCollection = true

…but it doesn’t improve the results.

Has anyone found a reliable way to handle split line items?
Are there service settings or template tricks that help?
Or is post-processing/merging the only practical solution at this point?

Would love to hear how others are dealing with this.

Thank you

Blog Post : Integrate SAP Document AI with SAP Build Process Automation on a Trial Account

2025-09-16T19:38:52.286000+02:00

Introduction

In today's digital world, automating document-heavy processes is key to improving efficiency and accuracy. Manually extracting data from invoices, purchase orders, or delivery notes is time-consuming and prone to errors. This is where the powerful combination of SAP Document AI and SAP Build Process Automation comes into play.

SAP Document AI uses machine learning to intelligently extract structured data from unstructured documents. SAP Build Process Automation allows you to design, build, and run automated workflows. By integrating these two services, you can create robust automations that read, interpret, and process documents just like a human would, but faster and without fatigue.

In this blog post, we will walk through the steps to integrate these two services on a trial account, using a standard invoice schema to extract data.

Prerequisites

An active SAP BTP Trial Account.
Access to SAP Build Process Automation in your trial subaccount.
Basic familiarity with the SAP Build Process Automation studio.

Step 1: Subscribe to SAP Document AI

The easiest way to get started with SAP Document AI on SAP BTP is by using the booster tool.

Log in to your SAP BTP Trial Account cockpit.
Navigate to your Subaccount and go to the Booster.
Search for "SAP Document AI" and click on Start.
Run the booster. It will automatically:
1. Subscribe you to the service.
2. Create a service instance and a service key.
Once the booster completes, go to your Subaccount -> Instances and Subscriptions.
Find your newly created service instance for SAP Document AI and navigate to its Service Keys. Open the default service key.
Crucial: From the JSON of the service key, note down the following values. You will need them in the next step:
1. uaa.clientid
2. uaa.clientsecret
3. uaa.url

Note: To access the SAP Document AI service on the browser (e.g., to check schemas or test manually), ensure your user has the appropriate roles assigned by the booster, such as Document_Information_Extraction_Viewer.

Step 2: Generate an Access Token in your Automation

To call any SAP Document AI API, you need a valid OAuth access token. We will generate this using the client credentials from the service key.

In your SAP Build Process Automation project, create three String variables:
1. uaaUrl (Value: The uaa.url from your service key)
2. clientId (Value: The uaa.clientid from your service key)
3. clientSecret (Value: The uaa.clientsecret from your service key)
Create a fourth variable to hold the combined client ID and secret, encoded in Base64. Name it clientInfo.
Use a Script activity to construct the request for the access token. Set the Output parameter of this script to a variable of type Any (e.g., optionsGet).
1. Script Code for Access Token Request:

return {
    method: 'GET',
    url: uaaUrl + '/oauth/token?grant_type=client_credentials',
    responseType: 'json', 
    resolveBodyOnly: true,
    headers:{
        Authorization:'Basic ' + clientInfo
    }
};

After this script, use an Rest Call Web Service activity.
The response contains an access_token field. Extract this token into a new string variable or directly pass it to the variable. The value should be:
1. "Bearer " + step2.obj.access_token

Step 3: Upload a Document for Processing

Now that we have an access token, we can upload an invoice document for processing. We will use the pre-built SAP_invoice_schema.

Ensure you have the path to your invoice file stored in a variable (e.g., fileObj).
Use another Script activity to construct the multipart/form-data request for document upload. The output will be your request configuration.
Use an Rest Call Web Service activity to execute this request.
1. Script Code for Document Upload:

// Define the extraction options
let optionsPayload = {
    documentType: "custom",
    schemaName: "SAP_invoice_schema", // Using the pre-built schema
    // Below commented values are optional, if you want you can use as per requirement.
    //headerFields: ["deliveryNoteNumber", "purchaseOrderNumber", "deliveryDate"],
    //lineItemFields: ["materialNumber", "quantity", "unitOfMeasure"],
    clientId: "default"
};

// Construct the HTTP request options
let requestOptions = {
    method: "POST",
    url: "https://aiservices-trial-dox.cfapps.us10.hana.ondemand.com/document-information-extraction/v1/document/jobs",
    responseType: "json",
    resolveBodyOnly: true,
    metadataType: irpa_core.enums.request.metadataType.formData, // Crucial for file upload
    headers: {
        Authorization: access_Token // Use the variable from previous step
    },
    metadata: [
        {
            name: "file",
            file: fileObj, // Variable holding the file path
        },
        {
            name: "options",
            value: JSON.stringify(optionsPayload)
        }
    ],
    timeout: 30000 // Set a longer timeout for file upload
};

return requestOptions;

From the uploadResponse, extract two crucial values:
1. The job id (e.g., into jobId). You will need this to check the result.
2. The initial status. It will be 'PENDING', meaning the AI is processing the document.

Step 4: Retrieve the Extracted Data

Processing with AI is not instantaneous. We need to poll the service until the status changes from PENDING to DONE.

Create a Data Type: First, create a simple Data Type (e.g., ExtractionResult) with two fields:
1. Status (Type: String)
2. Data (Type: Any)
Create a Variable: Create a variable (e.g., finalResult) of this custom type ExtractionResult.
Set Initial Value: Initialize this variable. Set Status to 'PENDING' and leave Data blank.
Build the Polling Request: Create a Script activity to build a GET request to check the job status. The output will be used in an HTTP Request.
1. Script Code for Retrieving Job Status/Result:

let requestOptions = {
    method: "GET",
    // Use the jobId from the upload step in the URL
    url: "https://aiservices-trial-dox.cfapps.us10.hana.ondemand.com/document-information-extraction/v1/document/jobs/" + jobId,
    responseType: "json",
    resolveBodyOnly: true,
    headers: {
        Authorization: access_Token, // Re-use the same access token
        'Cache-Control': 'no-cache'
    }
};
return requestOptions;

Implement the Polling Loop:
1. Drag a Forever Loop into your workflow.
2. In the loop's Condition, use the expression: finalResult.Status !== 'PENDING'. This means the loop will break as soon as the status is no longer 'PENDING'.
3. Inside the loop:
  1. Use a Wait activity (e.g., wait for 2 seconds) to avoid making too many API calls.
  2. Use the Rest Call Web Service activity with the configuration from the script above. Store the output in a temporary variable (e.g., pollResponse).
  3. Use a Set Variable activity to update finalResult.Status with pollResponse.status with pollResponse.
4. The loop will continue until the status in the API response changes to 'DONE'. Once it breaks out of the loop, finalResult.Data will contain the fully extracted invoice data!

You can now use the extracted data in the subsequent steps of your automation, such as filling a form or triggering another process.

The Flow will look like this:

Conclusion

Congratulations! You have successfully integrated SAP Document AI with SAP Build Process Automation. You have built an automation that can submit documents for processing and patiently wait for the AI to return the extracted structured data. This pattern can be applied to countless use cases involving invoices, contracts, forms, and more, unlocking significant potential for automation within your business processes.

This is just the beginning. You can explore creating custom schemas tailored to your specific document types for even more precise extraction.

Feel free to connect here or on LinkedIn for any questions or suggestions.

LinkedIn : Indrajit Shah https://www.linkedin.com/in/indrajit-shahkumar/
LinkedIn : Aditi Jain https://www.linkedin.com/in/aditi-jain-3b6b4a190/
Co-Author @Aditii_Jain

Question: When will my blog post become visible on SAP Community?

2025-10-07T09:50:03.016000+02:00

Hi everyone,
I submitted a blog post this past Sunday (October 5, 2025). As far as I know, blog posts on SAP Community go through a moderation process before being published.

How long does this usually take? My post hasn't been published yet.
Did I possibly miss something, or should I just wait a bit longer?

Thanks in advance 🙏

Unable to create SAP Document AI Trial instance (plan blocks_of_100)

2025-10-16T11:07:45.603000+02:00

Dear SAP Support Team,

We are currently testing the SAP Document AI Trial service and have encountered an issue when trying to create a new instance — either via Booster or manually through the BTP Cockpit.

Each attempt fails with the following message:
Couldn't create an instance of service 'SAP Document AI Trial' with plan 'blocks_of_100'.
Operation failed.

The same behavior occurs across multiple trial subaccounts and for different users. Additionally, previously created instances cannot be deleted and show the status “Deletion Failed”.

We understand this is a trial environment, but could you please advise whether this behavior is expected or if there is any known limitation or temporary issue affecting the SAP Document AI Trial service?

Any guidance or confirmation would be highly appreciated.

Thank you in advance for your help and time.

Tutorial Use Machine Learning to Extract Information from Documents with Swagger UI - error 500

2025-10-30T01:02:13.117000+01:00

Hi,

I'm following the tutorial Use Machine Learning to Extract Information from Documents with Swagger UI | SAP Tutorials.

When I try querying or posting data, it always returns error 500.

I'm using Trial subaccount - service instance SAP Document AI Trial - plan Default.

Has anyone encountered this issue?

Regards,

Tri

Best Practices for Training Custom Invoice Templates in SAP Document AI (Base Version)

2025-11-10T11:10:14.955000+01:00

📘 Summary:

Looking for best practices on how to handle schema updates and reuse trained templates in SAP Document Information Extraction (DOX) Base Version. After adding new fields through schema versioning, previous training and mappings are not retained. Need guidance on whether retraining is required or if there’s a migration/reuse option.

Scenario:

I’ve been working on a custom invoice extraction setup using SAP Document Information Extraction (DOX) – Base Version (non-generative).
The use case involves 500 invoices with different layouts.

I created:

One custom schema
Around 100 templates, each trained manually with “Setup Type = Manual” for all fields
Training done successfully for ~100 invoices

New Requirement:

Later, we received a request to add 5 new fields to the existing schema.
However, DOX doesn’t allow adding fields to an active schema version that already has templates.
To add new fields, we must:

Deactivate and delete the existing templates, or
Create a new schema version

I proceeded with creating a new schema version and added the new fields.

Challenge:

When uploading invoices using the new schema version, the existing training/mappings from version 1 are not available.
It behaves as a completely new schema — none of the previously trained fields or templates are reused.

Question / Clarification Needed:

Do I need to retrain all 500 invoices again for the new schema version, or
Is there any recommended approach or migration method to reuse the field mappings from the previous version?

🚀 Discussion Points / Best Practices (to confirm with SAP team):

Is there any way to migrate or copy field mappings between schema versions?
What’s the best practice when new fields need to be added to a schema with multiple templates?
Should we plan all required fields before starting training to avoid versioning issues?
Does SAP DOX Premium (Generative AI) handle such cases more flexibly?

Hello, what about SAP IA JOULE?

2025-11-14T11:34:09.879000+01:00

Can a sap consultant get the SAP AI Joule?

Outlook about SAP Document AI - Channels and workflow features

2025-11-14T15:21:06.408000+01:00

Hi Document AI experts

According to the latest update of Document AI documentation, "The integration of the Channels and Workflows features lets you set up inbound channels that route incoming emails to appropriately configured workflows, automating sophisticated document-processing tasks."

I have a request to make an automatic load of PDFs attached to emails in a outlook email folder and I was thinking about making a solution utilizing the MS-Graph API. Now it seems that SAP Document AI has build in functionality to handle this request, so I have put my MS-Graph idea on hold and hope that someone in here can share some experience and maybe even a guide how to set this up.

Kind Regards
Thomas Madsen Nielsen

SAP AI for SAP Transportation management

2025-12-11T13:25:51.909000+01:00

Hello,

Iam looking for an SAP AI just for the purpose of finding relevant information on SAP Transportation management relating to any issue or concept. That means i want to give a prompt and expects SAP AI to serach for information across sap ecosystem i.e. SAP Notes etc.

I have access to sap learning hub and practice system in the hub. Is it possible to use SAP AI in hub for the above question and if not how can i get a trial system of SAP AI for the above objective ?

rgds

Unexpected Line Item Extraction Behavior in Purchase Orders using SAP Document AI

2026-01-06T22:17:47.802000+01:00

Hi everyone,

We are currently using SAP Document Information Extraction (Premium Edition) and we have created a custom schema based on SAP’s standard Purchase Order schema.

The issue we are facing is related to line item extraction.
When we submit multi-page Purchase Orders, DOX often extracts line items from tables that are NOT the correct PO item table, and in many cases, those tables are located on different pages of the document. These “wrong” tables have a very similar structure to the real item table, which seems to confuse the detection.

What we have tried so far

1️⃣Edited documents in the DOX UI, corrected the fields, confirmed the results and submitted feedback (ground truth).
2️⃣Created a Template, associated the corrected documents to it, and executed extraction again using this template.

However, even after multiple training attempts, DOX still extracts line items from the wrong table/page in many cases.

Constraints

Splitting the PDF is not an option, because the “wrong” table is not always on the same page (for example, it is not always page 2). The page position is not consistent.

My questions

Shouldn’t this be resolved by using templates and providing repeated feedback?
Is this a limitation of DOX template training?
Is there any recommended approach or best practice to handle scenarios where multiple similar tables exist across different pages?

Any guidance, clarification, or alternative approach would be really appreciated.

Thanks in advance!