SAP Community - SAP Integration Suite

How to host static webpages through SAP CPI-Iflow

2024-04-22T15:26:00.691000+02:00

Before walking through the solution, We will glide through How it started.

Addressing the Integration community, We have a lot of systems to connect and indeed as an integration consultant we will be having POs and CPI systems to access them we will be using bookmarks and find it difficult to remember and open them at first glance.

As an enthusiastic integration consultant, I wanted to access the systems with in single click. I built a webpage with the help of Chat-GPT and wanted to host it but was not sure to host it on the internet and expose the domains.

Discussed the same with teammates and one of the teammates came up with a solution why can’t we do this with CPI, We will host it through CPI.

How to achieve this?

It is assumed that you have the HTML5 code in a single file already. You need to copy the HTML content and encode this to base64 (using an online tool) and copy the base64 encoded string to Notepad or any IDE of your choice.
Please save the Base64 code in a header as below.Content Modifier
COpy the below groovy script to decode the Base64 to HTML script.

import com.sap.gateway.ip.core.customdev.util.Message
import java.util.HashMap
import java.text.SimpleDateFormat
import java.util.Base64;
def Message processData(Message message) {
    
    def staticContent = message.getProperties().get("HTML_CONTENT");
    
    
    //Parse static content
    Base64.Decoder decoder = Base64.getDecoder();
	staticContent = new String(decoder.decode(staticContent));
   
    message.setBody(staticContent.getBytes("UTF-8"));
    message.setHeader("Content-Type", "text/html; charset=utf-8");
    return message
}

4.Deploy the iflow, copy the HTTPS URL, and hit it from the browser, you can access your webpage. 5. In case you're getting a CORS error, Please use API management to surpass it.

Let me know if you have any queries and please share your interesting findings as well! You can contact me on LinkedIn regarding the same! Until then stay tuned!

Supporting Multiple API Gateways with SAP API Management – using Azure API Management as example

2024-04-25T17:02:14.909000+02:00

Introduction

APIs are the cornerstone of modern business ecosystems and are crucial in today’s digital economy. The advent of action-based generative AI, which relies on accessible and well-defined APIs, underscores the importance of composable, API-centric software architectures.

Securing, governing, and managing APIs is essential. API Management has become a recognized discipline, supported by solutions from leading vendors such as SAP, Microsoft, Google, IBM, Salesforce, among others. API Management fosters an ecosystem where authorized developers can access and consume carefully curated and secured collections of APIs.

The essence of API management solutions lies in their ability to regulate access, authenticate and validate API calls through an API Gateway, a proxy solution deployed between API consumers and the service endpoints themselves.

Traditional API Management solutions relied on a single, central API gateway, set up in proximity to the APIs they safeguard. However, this simple architecture is less common today, due to the distributed nature of cloud software applications; customers deploy a growing number of distributed API gateways.

For example, SAP’s API Management solution, a capability of the SAP Integration Suite, provides API gateway functionality as a service available in over twenty regions around the world, as well as self-hosted options for on-premise or private-cloud deployments. This offers customers more flexibility on where to deploy API gateways.

Customers of SAP API Management may, additionally, use API gateways from other vendors and cloud providers like Microsoft Azure. But this presents a new challenge: while there may be multiple API gateways, customers still want to present the managed API information in a unified and centralized API catalog.

To address this, SAP and Microsoft have partnered, sharing programmatic interfaces to facilitate a cohesive API catalog.

This blog targets SAP Integration Suite customers utilizing the SAP API catalog (API Business Hub Enterprise) as their central repository for all APIs, accessible through a developer portal or directly via SAP Build development tools (SAP Build Apps, SAP Build Code, SAP Build Process Automation)

We introduce an extensible API import utility which streamlines and automates the process of importing, registering and publishing additional APIs into the central catalog, such as APIs managed by other vendors, or found in other external catalogs. Here, we focus on Microsoft’s Azure API Management as an information source, but the API import utility can be easily adapted to automate the import of APIs from any other source. For further details, please reach out to us.

A companion blog by our Microsoft friends details how APIs managed by the SAP API gateway can be imported to the Azure API Center, a new central catalog offering from Microsoft.

Automated API import

SAP API Management designates APIs that are not safeguarded by an SAP API gateway as “externally managed” or “unmanaged" APIs. Such APIs can be imported and listed in the API catalog and developer portal, and then optionally further enhanced or managed via API proxies, for instance to ensure that all APIs of the organization share the same security policies and common URL addressing.

While it is straightforward to manually specify a few individual unmanaged APIs and keep these specifications up to date, it becomes a daunting challenge for larger collections of APIs, especially those managed by a non-SAP API gateway.

This blog outlines an extensible API import utility that automates the process of importing, registering and publishing APIs into SAP API Management. The utility employs a “factory” design pattern implemented as an integration with Microsoft’s Azure API Management.

Available as a stand-alone open-source portable Java application, the API import utility can be executed from a desktop, but it is more likely that you will want to incorporate it into a regular workflow or CI/CD pipeline, in order to ensure that the API catalog remains up to date.

Accessing the API Import Utility

Essential tools for using the API Import Utility include:

Git
Java 11+
Maven 3.9+

The code is available here: https://github.com/SAP-samples/api-management-api-import-utility.

Overview

The API import utility was developed by our SAP colleagues Soumen Gosh and Bibin Thomas. It comprises two main components. The api-import code-tree contains the utility’s main class and entry point, in APIImportApplication.java, and the various functions for intefacing with the SAP API Management catalog.

The common interface (IRepositoryService) required by the “factory” in APIDiscoveryFactory.java is simple, and consists of three functions:

public List<API> getListOfApis()

public API getApi(String apiIdName)

public APIDefinition getApiDefinition(API api)

The implementation of these functions for Azure API Management is located in AAMApiRepository.java, in the AAM code-tree. The implementation adheres to the API service documented here: https://learn.microsoft.com/en-us/rest/api/apimanagement/api-management-service.

The following simplified illustration demonstrates the main flow:

The utility fetches the APIs from the Azure API Management and makes them available in SAP API Management as unmanaged APIs. In addition, by default, all imported APIs are added to a new product (a grouping of related APIs) called AAMProduct (you can change this default name in the AAM_product_description.json file in the conf folder).

Building the code

At the root of the code-tree, execute: mvn clean install
This creates a runtime folder: ./target/app with two subfolders, lib and conf.

Preparing Access to Azure API Management

Access to Azure API Management requires an Azure API Management account and the appropriate roles. You will need an Azure Entra ID (formerly known as Azure AD) service principal (for more information, see https://learn.microsoft.com/en-us/purview/create-service-principal-azure).

The command to create a service principal and configure its access to Azure resources is as follows:

az ad sp create-for-rbac --scopes /subscriptions/<subscriptionId> --role Contributor

For more information, see https://learn.microsoft.com/en-us/cli/azure/ad/sp?view=azure-cli-latest#az-ad-sp-create-for-rbac

The command response will be something like this:

Save this response in a file called AAM_connection_details.json, in the conf folder created in the build step above (replace the existing file placeholder).

Prepare access to the SAP API management catalog

Accessing SAP API management requires access to an SAP Business Technology Platform (BTP) subaccount, with an installed tenant of the SAP Integration Suite with activated API Management. You will need to generate a service key, which provides trusted access to the public APIs of SAP’s API Management. This process is explained here: https://help.sap.com/docs/integration-suite/sap-integration-suite/api-access-plan-for-api-portal . The resulting service key looks like this:

Save this result as a file called SAP_service_key.json, in the same conf folder created above (replace the existing file placeholder).

Running the API import utility

From the app folder, execute:

java -cp api-import.jar;lib/* com.apimgmt.gateway.APIImportApplication

The utility fetches the APIs and makes them available in SAP API Management as unmanaged APIs. In addition, by default, all imported APIs are added into a new product (a grouping of related APIs) called AzureAPIs.

It is up to you to decide if you want to take additional steps, such as adding an API proxy for any of the imported APIs.

Developers browsing the SAP API Management API catalog will see a new product tile representing this product grouping:

Conclusion

The API catalog of SAP API Management serves as a centralized listing for APIs managed across distributed and private-cloud API Gateways. The blog demonstrates how to seamlessly list “externally managed” APIs from various sources, enhancing them to conform to organizational security and compliance standards, and providing a consistent URL schema for all APIs.

The integration of APIs from SAP API Management with those from other gateways or repositories allows for the creation of intricate, distributed application architectures while preserving the vital role of a centralized API catalog.

Contacts

soumen.gosh@sap.com, chaim.bendelac@sap.com, mapankra@microsoft.com

Govern SAP APIs living in various API Management gateways in a single place with Azure API Center

2024-04-26T12:33:48.591000+02:00

Find the GitHub repos associated with this post on Azure API Center here.

Our engineering friends from SAP Integration Suite– in particular @Chaim_Bendelac – published a nice “sister blog” on supporting Azure API Management with the API Management capability of SAP Integration Suite here.

Dear community,

Many of you are heavily invested in APIs regarding your SAP ecosystem and the rest of your IT real estate. Given the integration specialization in the SAP space companies decide to use more than one integration tool to cater for SAP and non-SAP integrations. Gartner even says that 75% will use at least two different ones. For many of you that means SAP Integration Suite plus one for non-SAP.

Due to the fast-paced growth of APIs within organizations, inventory, governance, security, and management cannot keep up. The resulting fragmentation and inconsistency lead to adoption challenges, project delays, and security risks. Postman’s State of APIs report 2023 shows that API security incidents happen frequently.

These challenges are summed up under the term “API Sprawl” by the industry. Beware the API sprawl monster is upon you!

fig.1 Illustration of API Sprawl challenge

Key to survival is automatic discovery of available APIs and a single place to enforce guidelines from, or at least know these unmanaged APIs exist in your estate. Forgotten APIs are low hanging fruit for attackers. To drive home that argument: “Improper Inventory Management” made the OWASP top 10 list for API Security in 2023.

Besides that on the human side of things: Which developer likes to develop duplicate functionality just because of the lack of shared API inventory to discover existing stuff?

The API Sprawl monster🦖 much hungry! “Nomnom nomnom more food, yes more food!”.

Azure API Center embarked on the journey of taming the monster.

What API solutions can be registered to Azure API Center?

Azure API Center applies to any API and any API management solution out there. Always remember that API Center is not an API Gateway! It doesn’t expose the endpoints or apply policies to them. That stays with the API Management provider. API Center makes them discoverable and allows decorating APIs with additional info to improve governance.

Let that sink in.

My colleagues are building integrated experiences for the most interesting API and integration tool providers. However, API-based registration in API Center will always be possible.

Get it? APIs to register APIs to register APIs ... yah maybe to complicated for a joke.

fig.2 Azure API Center solution coverage overview

The focus of this blog post will be on inventorying APIs hosted by the API Management capability of SAP Integration Suite to mitigate SAP API sprawl. However, the approach described is applicable to all the other SAP APIs out there hosted on SAP Gateway, SAP Graph, SAP CAP, SAP RAP, CloudFoundry, Kyma, etc. too.

Another prominent SAP service would be SAP Cloud Integration (formerly CPI – Cloud Platform Integration). Many of you expose APIs internally or to business partners through SAP integration flows without fronting them with an API Management solution – you know who you are 😉. Those can be registered too. Unfortunately, there is no built-in option to retrieve the definition of such an endpoint. You may generate an API definition for your http trigger using payload samples for instance. I found this repo useful to get an overview on how to generate OpenAPI definitions from JSON payloads.

Even if you don’t, putting the available metadata on the Azure API Center inventory still improves discoverability and enterprise-wide governance by magnitudes.

But now on to SAP API Management.

Automagically registering SAP API Management APIs on Azure API Center

Our starting point is the SAP BTP service apimanagement-devportal. Check SAP’s docs on the setup process here. Make sure you don’t mistakenly choose apimanagement-apiportal.

The API “API Business Hub Enterprise - Discovery Service (CF)” enables querying all available APIs hosted on SAP API Management on that subaccount. It holds info about their OpenAPI definitions.

Authenticate on the service with any of the supported authentication mechanisms. I chose OAuth2 client credentials grant (instance secret – without payload).

See below response from “/apidiscovery/v1/apis” from my SAP BTP sandbox environment. Pay attention to the attributes of “apiDefinitions” and values for “oas-json”.

{
"@odata.context": "$metadata#apis",
  "value": [
    {
      "name": "GWSAMPLE_BASIC",
      "title": "GWSAMPLE_BASIC",
      "version": "1",
      "lastUpdated": "2024-01-24",
      "releaseStatus": "PUBLIC",
      "protocol": "ODATAV2",
      "entryPoints": [
        {
          "name": "GWSAMPLE_BASIC",
          "type": "PROD",
          "url": "https://msftapim.test.apimanagement.eu10.hana.ondemand.com:443/GWSAMPLE_BASIC"
        }
      ],
      "apiDefinitions": [
        {
          "type": "oas-json",
          "url": "https://eu10devportal.cfapps.eu10.hana.ondemand.com/odata/1.0/data.svc/APIMgmt.APIResourceDocumentations('2797A5F5-E18A-4FCC-826A-C833845303F5')/content/$value"
        },
        {
          "type": "edmx",
          "url": "https://msftapim.test.apimanagement.eu10.hana.ondemand.com:443/GWSAMPLE_BASIC/$metadata"
        }
      ]
}

For your convenience we have provided a sample repo that runs Infrastructure-as-Code scripting to register the SAP APIs using their OpenAPI definitions as highlighted above. On each SAP API definition we execute registration requests on Azure API Center.

You may also use Postman, or SAP Build Process Automation etc. to execute the REST API calls if you prefer. Find our collection here.

fig.3 Flow of automated API registration in Azure API Center

Discover all your APIs where you code – see VS Code and GitHub Copilot in action

We developers like to stay within our flow. So, having the API inventory available at my fingertips in VSCode is a good step into that direction. Also generating http requests to poke around the service and API clients is nice 😎Kiota supports a multitude of languages for SDK generation.

To get that going install the Azure API Center portal VSCode extension.

fig.4 Screenshot of VSCode extension with example OpenAPI definition

Please note that the authorize button (and respective authentication scheme) on the OpenAPI definition explorer is only available if present on the definition file. It looks like this for Basic Auth:

fig.5 Screenshot of auth definition in example OpenAPI spec for SAP OData service

When using the http file and the REST client extension of your choice, you may simply provide the authentication header with Bearer token etc.

Next to the Azure API Center extension view before, you can also use GitHub Copilot Chat to query available APIs from API Center. See Microsoft Learn for more samples. You may search for APIs by key words like so:

@apicenter /search business-partner

Cherry on the cake 🍰is the API Center portal for the classic developer portal experience across your whole registered API inventory wherever that is.

fig.6 Screenshot of Azure API Center portal API inventory view

So far so good on registering APIs and working off the info their definitions provide. But how about governance? I know how desperately everyone wants to plaster cost centers, line-of-business info, and security labels on your interfaces. 😏

Enforced API metadata is your second line of defense against API sprawl

In addition to simply registering APIs you may add custom properties to the object on Azure API Center. So, even if the info is not present on the API itself you can still govern it from Azure. See below sample that I created from the Microsoft Learn tutorial.

fig.7 Screenshot of Azure API Center metadata maintenance view

Knowing which APIs are public facing is useful, isn’t it?

For everyone looking for more sophisticated security with less human error surface, have a look at Defender for APIs. I like the alert rule for “un-authenticated APIs” and disabling endpoints that were not used in the past 60 days most – wait what? Those exist out there in the wild west of SAP on the Internet? 😲See the open-source automatic remediations repos here to mitigate for Azure API Management.

Defender for API integration with 42Crunch brings API security testing and hardening to your CI/CD pipeline.

API Linting gets you to the next level

OK, now let’s look at API style guide compliance. Is everyone playing by your rules? How do you make sure developers notice violations already during design phase rather than at later stages of deployment, release, or even months after the fact when audited?

Good automatic API linting creates much less hassle for everyone in the long run, less cost to fix API definitions after the fact, improved security posture, and a more rewarding experience for the people involved. See below video on the setup of the linting function for OpenAPI using Spectral linting engine.

Anyone aware of a great OData linter and would be curious to explore? Please share!

Get more details on API Linting for Azure API Center from this Microsoft Learn article.

Thoughts on production readiness

Azure API Center is in public preview but due for General Availability with the next wave of announcements, so completely ready for prime time. The same is true for the VS Code extensions and APIs used to orchestrate the integration between SAP API Management and Azure.

Intentionally registering APIs from SAP to Azure API Center improves API inventory management by magnitudes. However, shadow inventory thrives in places you don’t actively look. To mitigate even more effectively the team is building automated discovery from your GitHub org, Azure DevOps, and other popular sources.

SAP Fiori tools on VSCode provided by SAP SE enable usage of the approach described in this blog out of the box. The same is true for SAP CAP development in VSCode.

Final words

That’s a wrap🌯. You saw today how you can effectively counter API sprawl and its negative side effects that put your APIs and organizations at risk. A primary means to achieve that is creating a central API inventory hosted on all the different API Management solutions out there with Azure API Center.

This blog showed how to achieve that using the API Management capability of SAP Integration Suite as an example.

Furthermore, you learned about steps to improve API governance with custom properties and API linting. Eventually, you understood the difference between Azure API Center and an API Gateway.

Find the GitHub repos associated with this post here. It gets you started in no time.

Big #Kudos to Pascal van der Heiden – my brother in crime on this effort. And of course, last but not least to @UdoPaltzer and @vinayak_adkoli for the great collaboration! 🙌

Anyone curious to tap their toe into the waters where the API sprawl monster 🦖 lives, just reach out to me and Chaim or leave a comment.

Cheers

Martin

API Management: Unlocking OAuth Strategies

2024-04-30T15:30:43.947000+02:00

Introduction

In today's rapidly evolving enterprise landscape, the API-first approach has emerged as a cornerstone for organizations striving to stay competitive and relevant. At the heart of this approach lies API management, a central place for the orchestration, optimization, and security of APIs.

In this blog, I will elaborate the authorization considerations surrounding APIs, as well as the OAuth strategies I've investigated and encountered during my learning journey and professional experience.

If you are new to API management I highly recommend going through the help doc to get basic knowledge of API management capability before diving into this blog.

Through out the blog, I have added links to some learning journeys / blogs which you can refer. I thought this is better instead of rewriting the same content which is already well written.

I will structure the blog in below sections

Pre-requisites
Creating a simple integration flow
Securing integration flows
1. Exploring API Management OAuth Policy
2. Extending OAuth Beyond API Management
Testing steps to test both solution approaches.
Conclusion

Pre-requisites

SAP BTP account or trail account
Integration Suite Tenant with below capabilities
1. Cloud Integration
2. API Management (ensure you have proper roles assigned)
Process Integration Runtime service
Insomnia or Postman client

Creating simple integration flow

I created a simple integration flow and deployed it as a REST API in the cloud integration tenant. This integration flow does following things

1. Logs the incoming message as attachment

2. Persists the message in JMS queue

Note - This is the basic integration flow. Consider it as a mock iflow which can be replaced by any other iflow based on the business use case.

Figure-1: Integration Flow

Securing integration flows

In an enterprise landscape, you will always come across situations, that you have to expose your integration flows using API management. Ofcourse Integration suite provides API management capability which can seamlessly wrap the integration flows and create an API proxy.

One of the important question always comes up - How to secure your APIs/iflows? The obvious answer is OAuth 2.0 (Open Authorization 2.0).

Here I will showcase, 2 solution approaches to secure the API with OAuth 2.0.

Exploring API Management OAuth Policy: API Management provides security policies which can be used out of the box.

Create an API Proxy. Configure -- APIs -- OAuthService. This Service will be used to generate the token. Follow Step 1 (1 - Create the Token Endpoint) from this blog
Create another API - LogMessageAPI as a proxy to the iflow. We will add token validation in this proxy. Define the API by referring the Step 2: Define the API from this learning journey.

In the LogMessageAPI:

Add policy template to the LogMessageAPI as mentioned in Step 3, 4 and 5 of this learning journey

Add VerifyAccessToken policy as the first policy in the pre-flow of the ProxyEndpoint using OAuthV2 in the LogMessageAPI proxy.

<OAuthV2 async="false" continueOnError="false" enabled="true" xmlns="http://www.sap.com/apimgmt">
   <!-- this flag has to be set when you want to work with third-party access tokens -->
   <ExternalAuthorization>false</ExternalAuthorization>
   <!-- valid values are GenerateAccessToken, GenerateAccessTokenImplicitGrant, GenerateAuthorizationCode ,
    RefreshAccessToken , VerifyAccessToken , InvalidateToken , ValidateToken  -->
   <Operation>VerifyAccessToken</Operation>
   <GenerateResponse enabled="true"/><SupportedGrantTypes/>
   <Tokens/>
</OAuthV2>

Once you deploy the OAuthService and LogMessageAPI and publish the product to API Business Hub Enterprise, you will get the application key and application secret which can be used as client id and client secret to get the token.
In this way, you can use the inbuilt OAuth policy to generate the OAuth token and to validate the token for other API proxies.

Extending OAuth Beyond API Management: Let us dive deeper into another approach where we can use the process integration runtime OAuth server to generate the token through OAuthService API.
1. Create an API Proxy. Configure -- APIs -- OAuthService.
2. Import the policy template in the pre-flow of the ProxyEndpoint. I have exported my policy template and is available in GitHub - https://github.com/avinashvaidya09/apimanagementpolicytemplate
3. Each policy is listed and briefly described below -
  1. lookupOAuthToken - Using LookupCache policy, checks if access token is available in cache.
  2. extractRequestHeader - Using ExtractVariables policy, extracts the client id and client secret from the request headers.
  3. getBTPOAuthMetadata - Using KeyValueMapOperations policy, finds the BTP oauth token url saved in the key value map
  4. setAuthorization - Using BasicAuthentication policy, creates Authorization object.
  5. callBTPOAuthService - Using ServiceCallout policy, calls external BTP OAuth token url.
  6. raiseException - Using raiseException policy, raises exception if BTP OAuth token url returns exception.
  7. extractVariablesFromOAuthResponse - Using ExtractVariables policy, extracts token response.
  8. populateOAuthToken - Using PopulateCache policy, populates access token in cache which will be used by point 1.
4. Go to your BTP subaccount -- Create instance of Process Integration Runtime.
5. Create a service key and get the OAuth metadata
  1. client id
  2. client secret
  3. token url
6. Add the token url in the key value map in API configuration to make it externalized. This token url will be used in the service call out policy - callBTPOAuthService
7. Save and and deploy the OAuthService.
8. Ensure the integration flow is deployed. Get the endpoint URL.
9. Create another API - LogMessageAPI as a proxy to the iflow. Define the API by referring the Step 2: Define the API from this learning journey.
10. In this API, you do not have to define any access token validation policy, but the bearer token will be forwarded to the integration flow and will be validated directly by process integration runtime.

Test your APIs

Download any REST client like - Insomnia or Postman.
Deploy the API proxies and get the endpoints

Testing approach - 1:
1. Go to API Business Hub Enterprise. Create Application and add the product containing both the API proxies.
2. Get the application secret and application key after the application is created.
3. Create new HTTP request for API proxy OAuthService endpoint.
4. Add client id - client id will be the application key from point 2.
5. Add client secret - client secret will be the application secret from point 2.
6. Add grant type - client_credentials
7. Add Content-Type - application/x-www-form-urlencoded
8. Get the access token.
9. Create new HTTP request with API management LogMessageAPI endpoint.
  1. Add access token
  2. Add sample JSON payload
10. Execute the request.
11. Go to integration suite - Monitor - Integration & APIs. You should see a message in the queue. Click on the message id and you will land on the executed iflow with payload attached for reference.
Testing approach - 2:
1. Create new HTTP request with API management OAuthService endpoint.
2. Add client id - client id from process integration runtime service key.
3. Add client secret - client secret from process integration runtime service key.
4. Content-Type - application/x-www-form-urlencoded
5. Get the access token
6. Create new HTTP request with API management LogMessageAPI endpoint.
  1. Add access token
  2. Add sample JSON payload
7. Execute the request.
8. Go to integration suite - Monitor - Integration & APIs - Message Queues. You should see a messages in the queue. Click on the message id and you will land on the executed iflow with payload attached for reference.

Conclusion

Drawing from my experience, I must emphasize that these insights can serve as a starting point, though their applicability may vary based on specific scenarios.

This is just a glimpse of wide set of capabilities offered by SAP Integration Suite and API management. Stay curious! Keep learning!

To get more updates about this topic, please follow the below pages

Feel free to “like“, “Share“, “Add a Comment” and to get more updates about my next blogs follow me – avinash.vaidya@sap.com

SAP Emarsys Customer Engagement - Enhanced Lead Integration to SAP Sales Cloud Version 1

2024-05-03T07:46:16.745000+02:00

Introduction
Background Information
- Setup Steps in SAP Emarsys Customer Engagement
  - Create the Webhook Node Preset
  - Create the Automation Program
Integration Process
- SAP Integration Suite
  - Purpose
  - Integration Flow
- Connectivity Setup between Emarsys & SAP Integration Suite
  - Endpoint URL
  - Authentication
Conclusion

Introduction

The handover of marketing qualified leads to sales plays a crucial role in the marketing-driven lead process. It ensures that the sales team receives high-quality leads, which in turn improves sales efficiency and promotes collaboration between the marketing and sales departments.

This blog post describes how to built an integration between SAP Emarsys Customer Engagement (SAP Emarsys) and SAP Sales Cloud Version 1, leveraging the SAP Cloud Integration service included in SAP Integration Suite. This integration enables the creation of leads in SAP Sales Cloud with a broader range of information, exceeding the scope covered by the out-of-the box function.

Note: The SAP Emarsys Account Engagement (SAP Account Engagement) add-on enhances SAP Emarsys with B2B capabilities to run end-to-end processes being seamlessly integrated with SAP Sales Cloud. Our official SAP Account Engagement documentation highlights the out-of-the-box functionality of the SAP Emarsys Webhook which allows to create leads directly in SAP Sales Cloud. Creating leads directly in SAP Sales Cloud is limited to using a subset of fields represented in a flat structure.

The content described in this blog is custom-built and not part of any SAP pre-packaged integration. It can be used as a template for building a custom integration with SAP Emarsys Customer Engagement and SAP Sales Cloud Version 1. SAP Integration Suite is used for the message transformation for API-based data imports to SAP Sales Cloud Version 1.

Example:

Let's assume we are organizing a trade fair for which contacts can register online on an external landing page. Part of this registration form is the option to indicate interest in an expert meeting at the booth. To do so, contacts are asked to enter their topic of interest and a suitable time slot into the form notes. In addition, they have the option to provide their email opt-in.

When the form is submitted, contacts that previously did not exist yet in SAP Emarsys are created using the SAP Emarsys API. The additional form notes are written to a dedicated custom field.

To demonstrate the technical integration, we execute the following simplified scenario: We handover newly created SAP Emarsys contacts who have submitted form notes to SAP Sales Cloud so that sales representatives can further validate the lead. (Note that we are intentionally simplifying at this point. In a real-life scenario, the business logic for determining when a contact is a marketing-qualified contact can of course vary.)

In the following we will take a closer look at the required setup steps.

Background Information

To create the contacts gathered from the Web forms as leads in SAP Sales Cloud we need to use Webhooks as part of an SAP Emarsys Automation Program.

A Webhook is an event-based API endpoint responsible for triggering internal functions to look up information in real time when a specific event occurs. Webhooks are used to integrate different applications and third-party APIs, allowing for real-time communication between two applications without the need for additional coding or a complex setup. For more information, take a look at the SAP Emarsys help documentation under Webhook Node.

Note that the Webhook node is an advanced feature, for which you need some technical resources to integrate it into your application. It is your responsibility to make sure that the integrated service operates appropriately.

An Automation Program is a customer engagement journey that is automated and aims to lead your interaction with a customer towards a specific goal. It starts from a single entry point and can spread out to multiple branches of a decision tree until reaching an end point. For more information, take a look at the the SAP Emarsys help documentation under Automation.

Setup Steps in SAP Emarsys Customer Engagement

Create the Webhook Node Preset

You first have to create a Webhook preset that can be used as input for the Webhook node in your automation program.

To do so, open the menu and choose Automation > Webhook Node Presets > Create Preset:

Menu Path to "Webhook Node Presets"

Appearance Settings

Define a preset name - this will help to identify the configuration preset in the Webhook node settings and will appear as the name of the node.
Select a node icon - this icon will appear on the node after the preset has been selected.

SAP Emarsys Customer Engagement | Webhook Node - Appearance Settings

Connection Settings

The connection settings are described as part of the Integration Process (see below).

Data Settings

The Webhook node can send data of the triggering event to the API endpoint, as well as values of contact data fields and additional data with static values.

Event Data

Select this checkbox if you would like to forward the event payload that triggers the program. In our case, we don’t need to enable the “send triggering event data” flag as we are only considering the contact data that is coming from the external landing page through the SAP Emarsys API.

SAP Emarsys Customer Engagement | Webhook Node - Data Settings / Event Data

Contact Data

Here you can define key-value pairs to send contact data field values. In our case, we need to define the data that should be replicated for the lead creation in SAP Sales Cloud. Besides the SAP Emarsys system fields (see Emarsys System Fields), you can also use your own custom fields (see Creating Custom Fields). In the example below, we created the 'Form Notes' as a custom field which contains additional information gathered through the registration form.
Note: You can add maximum 20 contact fields when defining contact data key-value pairs.
Important: You must ensure to handover the contact identifier that was initially defined for the SAP Emarsys tenant. Handing over the identifier will allow SAP Sales Cloud to update the SAP Emarsys contact at a later point in time. For more details about contact matching with SAP Account Engagement, see Specifying a Contact Matching Method.

SAP Emarsys Customer Engagement |Webhook Node - Data Settings / Contact Data

Additional Data

These fields can contain static values that are sent with the requests so that your external system can identify their source (for example, the journey or scenario the event came from) and purpose.
Here we are using the following data:
- Name: No value defined - this allows the users that create the program to define the name
- Priority: Same values that are known in SAP Sales Cloud
- Origin Type: Same values that are known in SAP Sales Cloud (standard + custom fields)
- Qualification Level: Same values that are known in SAP Sales CloudSAP Emarsys Customer Engagement | Webhook Node - Data Settings / Additional Data

Operational Data

These fields are optional in the event data. You can also rename them. We recommend keeping the deduplication ID to make sure that the receiving system does not process duplicated messages. In our example, we don't map the 'ems_program_id', but it could be beneficial to map the programID where the Webhook nodes was triggered with a custom field in SAP Sales Cloud.

SAP Emarsys Customer Engagement | Webhook Node - Data Settings / Operational Data

Create the Automation Program

The example described above uses the transactional entry point "New Contact" to trigger the automation program. In addition to the new contact criteria, a segment filters on contacts that have submitted form notes as part of their fair trade registration.

Note: There are various options to customize this process by using additional filters to make sure that only dedicated contacts are replicated. If you are using the "New Contact" entry node without any additional filter criteria, every new contact is considered. Customers using the SAP Account Engagement add-on can use the pre-built "Replicate New Marketing Contact" tactic (see Replicate New Marketing Contact).

Create the Automation Program (Audience-focused journeys)
SAP Emarsys Customer Engagement | Automation Program Creation
Assign the Webhook Node
SAP Emarsys Customer Engagement | Automation Program - Webhook Node Assignment
Maintain 'Additional Data Settings'
As no value was defined for the 'Name' of the Webhook, the user creating the automation program can define the value.
SAP Emarsys Customer Engagement | Automation Program - Webhook Note Additional Data Settings

In the next step we will take a look at the integration process.

Integration Process

SAP Integration Suite

Purpose

SAP Integration Suite combines SAP Cloud Integration, API Management, Integration Advisor, Open Connectors, and other integration capabilities into a cohesive and simplified toolkit for enterprise integration.

The SAP Emarsys Webhook node only allows to pass flat structures to an API. For setting up the marketing permissions, the SAP Sales Cloud Lead-API requires sub-structures. Therefore, we use SAP Integration Suite to do this transformation.

Integration Flow

The integration flow was designed to make it easily transferable and adjustable. It consists of one Groovy-script step with the following functionality:

Mapping of field names and key values
Transformation to the required structure for the SAP Sales Cloud-Lead-API
Setting of the headers

SAP Cloud Integration | Integration Flow

Mapping

The mappings and transformations take place within the Groovy-script step. The entire code is provided at the end of this blog for reference. Within this script, the parsed JSON object from the Webhook payload is accessible via the Groovy Map 'mapJson'. Field names within 'mapJson' correspond to those in the Webhook within SAP Emarsys.
The target structure for calling the LeadCollection OData service is included in the Groovy-Map 'mapJson'. Here, it's crucial that the field names align precisely with those specified in the OData service definition. Upon completion of the mapping process, the 'mapJson' is converted into a JSON object and incorporated into the payload body. Encapsulation of value mappings is handled within the 'MapHelper' class, where static methods facilitate the mapping of field values.

SAP Cloud Integration | Groovy Script - Mapping

To include extra mappings such as linking the SAP Emarsys field 'MyAdditionalEmarsysField' to the SAP Sales Cloud field 'MyAdditionalSalesCloudField_KUT', simply insert the following line into the code:

mapJson.MyAdditionalSalesCloudField_KUT = inJson.MyAdditionalEmarsysField

Note: As mentioned above it is important to handover the contact identifier that was initially defined for the SAP Emarsys tenant. Handing over the identifier will allow SAP Sales Cloud to update the SAP Emarsys contact at a later point in time. For more details about contact matching with SAP Account Engagement, see Specifying a Contact Matching Method.

In our example, we created the SAP Sales Cloud extension field 'Emarsys Contact UID' and therefore maintained the following mapping:

mapJson.EmarsysContactUID_KUT = inJson.EmarsysUID

Inbound / Outbound Connectors

For both inbound and outbound we are using the HTTP-connector (since we have a JSON payload, we cannot use the Odata adapter for calling SAP Sales Cloud).

The adapter settings are the following:

Inbound

SAP Cloud Integration | Integration Flow - Inbound Connection

Outbound

SAP Cloud Integration | Integration Flow - Outbound Connection

Connectivity Setup between SAP Emarsys & SAP Integration Suite

Endpoint URL

After deploying the integration flow, the endpoint can be copied from SAP Integration Suite/SAP Cloud Integration

SAP Cloud Integration | Endpoint URL

to the Webhook in SAP Emarsys:

SAP Emarsys Customer Engagement | Webhook Endpoint URL

Authentication

The easiest way to authenticate the Webhook call at the integration flow endpoint is via OAuth. In the SAP Business Technology Platform cockpit make sure that the instance of the Process Integration Runtime contains the role ESBMessaging.send (see Setting Up Inbound HTTP Connections).

SAP BTP Cockpit | Instances View Parameter

Then create a service key and copy the clientid, clientsecret and the tokenurl

SAP BTP Cockpit | Instance Credentials

to the corresponding fields of the authentication settings of the Webhook in SAP Emarsys:

SAP Emarsys Customer Engagement | Webhook Authentication

Conclusion

Now let’s shed some light on what we have achieved with the utilization of SAP Integration Suite with regard to the integration of SAP Emarsys Customer Engagement with SAP Sales Cloud Version 1.

While the out-of-the-box SAP Emarsys Webhook functionality for lead creation in SAP Sales Cloud is limited to using a subset of fields represented in a flat structure, the use of SAP Integration Cloud removes this restriction and allows the incorporation of advanced lead information.

On the one hand, information has been successfully replicated to nested lead attributes, including form text to notes and email opt-in status at subnode level to the marketing permission:

SAP Sales Cloud - Lead Includes Notes and Email Opt-in

On the other hand, the “Additional data" including the priority, origin type, and qualification level of the lead was presented to the automation program creator in a descriptive format so that they do not have to rely solely on code values. Consequently, the process of selecting the appropriate values became noticeably more user-friendly.

In this way, the process of creating leads in SAP Sales Cloud has not only been enhanced by the use of SAP Integration Cloud, but has also been significantly simplified.

10+ ways to reshape your SAP landscape with SAP Business Technology Platform – Blog 7

2024-05-06T09:21:24.167000+02:00

Blog 7: The Role of Automation for Transformation Projects

In times of constant change, quickly emerging demands for digitalization and competitive markets, companies are looking for ways to further optimize their processes. Consequently, many organizations have turned to automation technologies, such as RPA tools, to automate mundane and repetitive tasks. The benefits of automation are quite obvious: Automated processes are faster, less prone to error and people can concentrate on more complex, higher value-added activities.

Especially for activities with high volumes the benefits can be huge. Let me illustrate this with an example: A few weeks ago, I talked to a company from the consumer products industry. While a lot of transactions with its suppliers run fully digital via B2B integration, they still receive roughly 4.000 documents per month such as invoices and purchase orders via email. Let us assume you need on average 2 minutes for processing one document, that is: Extracting and verifying the data, collecting approvals, creating it in the ERP system and so on. In total, this sums up to over 130 hours of manual work per month. Applying RPA tooling and AI for document understanding can drastically speed up the processing of these tasks, thereby freeing people from tedious copy & paste or data entry and export activities.

Why looking at automation only is not enough

To streamline their processes, many companies have also started looking into other technologies: Besides RPA, also workflow engines, integration middleware as well as tools for process mining and process analytics have risen. One big downside is that these technologies often are not well integrated and hence customers are struggling with setting up an aligned end-to-end approach.

With our Enterprise Automation offering we want to address this and provide a unified and interlinked approach. As described in the blog article by Angela Harvey, this concept is based on the combination of 3 products:

SAP Integration Suite: You cannot automate what you can’t connect to. Our suite provides unparalleled connectivity and integration for a variety of scenarios. Be it SAP / Non-SAP, Cloud / on-Premise applications, B2B, B2G, API-led or event-driven scenarios.
SAP Signavio: Before you optimize, you might need to understand where to start and uncover inefficiencies first. With SAP Signavio you can analyze, model and govern your business processes to drive operational change.
SAP Build: Drive automation of long-running processes with capabilities for app development, process automation and portal sites. Low-Code/No-Code and visual design tooling also empower citizen developers and business experts to contribute to the development process in an agile fashion.

It is important to note that Enterprise Automation is a flexible and modular concept. Hence, customers can start with whatever product they see fit. From a transformation perspective Enterprise Automation can provide significant benefits when modernizing your SAP ERP, so when moving from the classic SAP Business Suite to SAP S/4HANA. It can be applied at any stage, both before the migration as well as after the migration:

Before the move: Leverage Signavio’s “plug and gain” approach and LeanIX to analyze and clean up processes in advance and gain immediate benefits from applying automation (e.g. via pre-built content packages). Use the Integration Solution Advisory Methodology of the SAP Integration Suite to design and prepare a future-proof integration strategy.
After the move: Bring process excellence to the next level by combining process mining, integration and automation and enable a continuous improvement cycle. Enable a governed approach to accelerate and scale the building of process automations based on Low-Code/No-Code techniques.

Another advantage in terms of transformation is the versatility – similar to a Swiss Army Knife. Enterprise Automation allows process optimization across the enterprise and line of businesses: Be it sales, procurement, HR, finance, production or logistics. A comprehensive collection of use cases and end-to-end scenarios has been put together by Manuel Namyslo in his blog article: How to get started with Enterprise Automation.

The role of AI in Enterprise Automation

Let us take a closer look at the role of AI and how it fits into the picture of Enterprise Automation. When transforming business processes and modernizing enterprise applications, AI should be part of the agenda, due to the massive benefits it can provide. Within our Enterprise Automation offering, the AI capabilities are tightly embedded to make the consumption as easy, seamless and relevant as possible.

A few highlights of them include:

Use Document Information Extraction (DOX) within SAP Build Process Automation to grab data from documents like invoices, payment advices and purchase orders to automate their processing. We also plan to integrate the DOX premium edition to make the processing of arbitrary documents with varying structure and layout even simpler.
We plan to use AI to provide recommendations for workflows as proposals based on historic data. This will support and speed up the decision making for approvals and workflow items (see roadmap item).
Visual tools already cater towards ease-of-use and simplify the building of process automations by business experts. There are also plans to make the design process even more intuitive by leveraging Generative AI functions (see corresponding roadmap items). As illustrated in this preview video, users describe what they need or what they want to craft in natural language. This can be an entire process flow, but also single process steps such as a business rule. The system then automatically generates the respective building blocks. Apart from that the AI-based assistant will also help with the discovery of pre-built content by providing suitable proposals from the content catalog.

As you can see AI is a crucial factor in many ways. Not only for the automation of processes, but also to accelerate the design and development process in various aspects. In addition, Enterprise Automation can also help with infusing AI into the process layer, by providing the respective integration capabilities.

To conclude: Enterprise Automation should be considered as a strategic tool set in the light of any transformation initiative for ERP and enterprise application landscapes. Its modular style allows you to start small at any stage of your project: From simple workflow automation and high-level process modelling, up to a fully-fledged vehicle for continuous process excellence. While our offering is optimized for SAP applications and systems, we embrace openness and enable process excellence across heterogeneous IT landscapes.

Where to look next and where to start? I recommend checking out the blog article How to get started with Enterprise Automation to get further inspiration. In addition, Low-Code/No-Code tooling such as SAP Build is well suited for getting Hands-On experience. Therefore, feel free to reach out to your SAP account manager to experience the ease-of-use in a Hands-On workshop. Finally, start by looking into potential use cases and assess them with the help of SAP or partners on how to tackle them with Enterprise Automation.

Previous Blogs:

Blog 1: The Central Role of Clean Core and AI
Blog 2: The Role of BTP as Extension and Innovation Platform
Blog 3: How Clean Core can accelerate your SAP Business Transformation
Blog 4: Mastering the Data Dance: The Potential of Clean Core and AI
Blog 5: How hybrid Integration upholds the role of Clean Core
Blog 6: Clean Core Extensibility

Dynamically creating MPL custom headers without hardcoding in groovy script

2024-05-07T09:56:47.171000+02:00

Hello Everyone,

In our work with CPI (Cloud Platform Integration) MPL (Message Processing Logs), we often encounter the need to log specific values such as IDoc Numbers, PO Numbers, and Invoice Numbers as key-value pairs in custom headers. These custom headers allow us to enhance monitoring capabilities.

Traditionally, we would create headers or properties in a content modifier and then read those headers or properties in a custom Groovy script. However, this approach requires us to hardcode the property names both in the custom header and the Groovy script. As new requirements arise, we find ourselves repeatedly adding properties to the Groovy script, which can become cumbersome and error-prone.

To break out of this endless loop, I propose a solution:

Property Naming Convention:
- When creating properties for custom headers, start their names with “mpl-”. For example:
  - mpl-IDocNumber
  - mpl-PONumber
  - mpl-InvoiceNumber
Dynamic Groovy Script:
- Instead of hardcoding property names in the Groovy script, create a script that dynamically reads all properties starting with “mpl-” from your integration.
- After trimming the first 4 characters (i.e., “mpl-”), use the remaining part as the key for logging in the MPL custom header.

import com.sap.gateway.ip.core.customdev.util.Message;
import java.util.HashMap;

def Message processData(Message message) {
def body = message.getBody(java.lang.String) as String;
def mProp = message.getProperties(); //Read all the properies
def messageLog = messageLogFactory.getMessageLog(message);
if(messageLog != null)
{
mProp.each { key, value ->
if (key.startsWith("mpl-"))
{

def customHeaderName = key.substring(4); // Remove "mpl-"   // Extract the part after "mpl-"

messageLog.addCustomHeaderProperty(customHeaderName, value)   // Set custom header property
}
}

Output would be like:

Now, whenever you need to print something with custom header key-value pairs, you no longer have to hardcode any of the headers or properties within the Groovy code. Instead, you can use a single Groovy script for every interface in your tenant. 👍

By following this approach, you can log additional information without modifying the Groovy script each time a new requirement arises. It provides flexibility and simplifies maintenance.

Lets Send multipart/form-data to external Receiver: CPI to AI App

2024-05-08T17:41:14.873000+02:00

Let me start the story with the requirements I had then Ill come to the approach I opted and in the journey it is possible that I may have done some mistakes so I'll add a humble request to all my senior fellow to add there expertise in this blog, This will help everyone to get more insights.

Scenario:

We had to integrate an application which had a requirement to connect to and External AI application that has exposed some API to us with some documentation.

Challenge:

From the initial thoughts this looks easy, it just an API, as so as I thought , but as I mentioned above "documentation" gave us the real challenge. To complete all the call we had to send the multipart/form-data to this receiver.

Through the help of documentations we are able to successfully send the data from Postman. but when we tried to send it from CPI we faced challenges which lead us to the solution. So next I am sharing the steps we performed to send the form-data

Clearly, if such problem arrives instantly we come to SAP community and search for the answer, we found https://community.sap.com/t5/technology-q-a/sending-form-data-from-sap-cpi/qaq-p/12613755 solution that gives us a way and describe what form-data is and how well it can be passed- but solution given here was to change the type of data itself which was not a choice for us.

At least we got half way on understanding the way multipart/form-data will be constructed. To understand more about terms and functionality please refer this wonderful article https://community.sap.com/t5/technology-blogs-by-sap/what-is-form-data-and-how-to-send-it-from-sap-cloud-platform-integration/ba-p/13443126 by mattisebastian this will help us to create the exact frame of data we need to post and in correct way and how much important the envelope is.

Solution:

To Solve this it is very important to understand the few terms like Content-type, Content Disposition, Accept, Boundary and all the fields that are required in Form data.

To understand this format we had to mimic the Postman request we were directly send to API.

So we created a CPI Artifact to trace the look of the form data and we were able to get the exact same results as in screenshot , but we have to append an actual JSON which had a list of json data in format of string <binary> (File). And that was actually a file upload (that we were doing from postman).

till now if we send request without this JSON file, the AI app was accepting the request but with an error as no real data was found in envelope.

So now we had to find a way to send this file in Realtime to the receiver. and for that we tried various ways like to store the payload in data store and creating Tables and so on.

But issue persisted when we have to send it in Realtime. So to get a bit more Idea we again sent this request from Postman to CPI and traced it.

To get these headers understanding we also put some effort to create Multipart/Form-data from MIME script from the blog https://community.sap.com/t5/technology-blogs-by-sap/part-6-simplify-integration-of-data-from-sap-s-4hana-cloud-to-third-party/ba-p/13386862 from divyamary

and we created following results:

So now we know how the file data look like and what part in payload we are missing, as we had to send the binary data we changed the header Content-Transfer-Encoding: 7bit to

Content-Transfer-Encoding: Binary and same goes for file name.

and that's it.. it worked! Just some correction on payload customization and headers and we created multipart/form-data in Content Modifier only.

Briefs and Take-aways:

Mimic Postman Hits in CPI by artifacts , hit to CPI and traces the payload.
Set the Headers correctly specially Content-type and Boundary(specially spaces 🙂 )
Check if there is any media type for the type of data you are trying to send to receiver and set it correctly.
You can also try the MIME multipart script to simulate the outcome, it will give you some correct data formats and header options.
Add those steps and create a right payload in Content modifier.
After creating the form data always try to create a right envelope with right boundary (that worked for us)
Send with right adaptor configurations and ... it will be success.

Following are some key blogs from experts which we referred. a big thank them .

It was just my first blog so it will definitely have some misses, please add you comments and feedback so that ill improve.

Sap community is a Big sea and its just a drop that I added.

Thank you.

SAP Integration Suite - Design Guidelines in the integration flow editor of SAP Cloud Integration

2024-05-09T14:14:31.763000+02:00

Introduction

Design guidelines feature is available in the integration flow editor of SAP Cloud Integration with the 5.59.x/6.51.x to help integration developers to design and develop enterprise-grade integration flows. To know, you can refer the help documentation Design Guidelines | SAP Help Portal and Design Guidelines View | SAP Help Portal.

In this community blog, I will give a short overview on the design guidelines. Let us understand the fundamentals of the design guidelines prior having a deep dive.

What are Design Guidelines?

Integration flow design guidelines enable integration developers to design and develop integration flows, interfaces in a robust fashion to safeguard the company's mission critical business processes.

In the past, these design guidelines were recommendations that were published as product help and prepackaged content on SAP Business Accelerator Hub. You can find the guidelines in the Integration Flow Design Guidelines section. Now, the same guidelines are incorporated within the software as rules that help integration developers for easy consumption.

Why Design Guidelines?

Integration flow design guidelines help to improve the quality of the integration flows, interfaces by keeping them readable, easy to understand, avoid memory leaks, avoid performance overhead and helps handling error in a good way.

Motivation

Design guidelines helps understand the basic capabilities for modeling integration flows.
Provides guidelines to implement the basic enterprise integration patterns.
It helps to improve the performance of integration flows and reduce the possibility of the outages related to OOM, memory consumption etc.
It applies highest security standards.
It enables the integration flows ready for the productive usage.

Availability

Design guidelines are available in the SAP Cloud Integration across Standalone NEO, Cloud Foundry, and Integration suite.

Personas

Broadly there are two personas involved in the design guidelines configuration and execution process. Please check their roles and responsibility.

Persona	Responsibility
Tenant administrator/Integration Lead/Solution Architect	Configure the design guideline at the tenant level to make it available for all integration flows. Review the design guidelines compliance report and take a decision for the Go-Live in the production environment.
Integration Developer	Execute the design guideline on the integration flow. Download the compliance report and share with tenant administrator/integration lead for the review.

Workflow

Below diagram depicts the workflow that will benefit you in the usage of the design guidelines. Broadly there are two personas who will be involved in leveraging the benefit of design guidelines.

Tenant administrator, who will enable the design guidelines at the tenant level and integration developer who shall execute the guidelines on the integration flow to improve the quality.

Our recommendation is to impose the below process which will help to improve the quality of the integration flow and ensure the smooth transition for the go live in the production environment.

Let me try to explain the process by taking a following scenario into consideration.

Configure Design Guidelines

Configuring and enabling design guidelines is the responsibility of the Tenant Administrator/Integration Lead/Solution Architect.

We have a dedicated application role for the tenant administrator in NEO and Cloud Foundry. Please refer this documentation Persona | SAP Help Portal

However, tenant administrator can create a custom persona such as solution architect or Integration lead to manage and configure the design guidelines at the tenant level by using the role “WorkspaceDesignGuidelinesConfigure”.

User having a proper authorization will be able to see Design Guidelines tab in the integration settings page of the Integration Suite tenant.

Before you enable/disable design guidelines, understand every design guideline and its implications. Consume the in-app help available for each design guideline to learn more.

You can enable the design guidelines that you think are appropriate for your organization's requirements. Similarly, disable the ones that you think aren't appropriate anymore.

The design guidelines are logically grouped. For example, all transaction handling related guidelines are grouped under a single category. Some of the logical groupings are Handle errors gracefully and run an integration flow under well-defined boundary conditions.

Severity decides the criticality and importance of the design guidelines. Severity of the design guidelines is categorized as High, Medium, and Low.

Design guidelines which are switched-on will be applicable to the integration flows for the execution.

It is desirable that you like to follow the same design guidelines in your other development tenants, hence, to ease the work, we have offered export/import feature to export the design guidelines with the enabled/disabled status from one tenant and import the same in another tenant This way, you don't have to manually enable the same design guidelines in multiple tenants.

Execute Design Guidelines

As we discussed, design guidelines are rules that help you design robust integration flows. Your tenant administrators or integration leads would have enabled all or a subset of available design guidelines that they think are appropriate for your organization's business needs.

Integration developer responsibility would be to execute the design guidelines after they have developed the integration flow. You can also run the guidelines on the already developed integration flows as well.

Guidelines which are mandated by the tenant administrator or Integration lead will participate in the execution of the design guidelines on the selected integration flow.

Remember that not all enabled design guidelines are applicable for every integration flow that you create. The integration flow editor intelligently identifies the design guidelines that are applicable and validates the integration flow only against the applicable ones. Applicability – “Not Applicable” means the corresponding guidelines is not applicable to your integration flow.

Compliance status can be either “Compliant” or “Non-Compliant”. If the integration flow is not compliant to the guidelines, then analysis report will be provided which will educate integration developer how to fix the failed guidelines and ensure the quality of your interface. Analysis will also cover the model step where the fix must be applied. Easy navigation to the problematic step of the integration flow is offered to ease the job of the integration developer.

Once the design guidelines are made compliant, our recommendation is to download the report and share with the integration lead/Tenant administrator/Solution Architect so they can review the guidelines report and take a decision for transport of the integration content to the production environment for the Go-Live.

FAQs

In which product of SAP Cloud Integration, design guidelines feature is available?

Design guidelines is available in Cloud Integration NEO, Cloud Foundry, and Integration Suite all editions.

Who can configure, enable/disable design guidelines at the tenant level in the integration settings page?

Tenant administrator or the user having the application role – “WorkspaceDesignGuidelinesConfigure”.

Do the non-compliant design guidelines lead to integration flow deployment failure?

No.

Do the non-compliant design guidelines lead to content transport failure?

No.

Can I execute design guidelines on the standard pre-packaged integration content?

Yes, you can execute design guidelines on standard and custom integration content.

Can I execute design guidelines on the read-only Integration flow?

Yes.

Who can execute design guidelines on the integration flow?

Integration developer can execute the guidelines on the integration flow.

Do the support users are allowed to download design guidelines compliance report?

No, they cannot download the report. However, they can visualize the guidelines compliance result in the integration flow editor.

Can I know when the last time the design guidelines was executed on the integration flow?

Yes, we show the datetime information in the design guidelines tab of the integration flow after the guidelines are executed.

Can I know on which version of integration flow the design guidelines was executed?

Yes, we show the integration flow version in the design guidelines tab of the integration flow.

Can integration developer remove the design guidelines which are mandated by the tenant administrator?

No, guidelines which are mandated cannot be removed by the integration developer from the execution.

In which format I can download the design guidelines compliance report

Report is available in the xls format.

What’s coming next?

We have the following increments planned in the succession.

Increase the coverage by adding more design guidelines for the integration flow.
Skip design guidelines – If the design guidelines fail (i.e., non-compliant), we want to offer a feature for the integration developer to skip the failed design guidelines with his/her consent mainly to block the false positive guidelines failure.
Public Remote APIs for enabling CI/CD requirements.
Restrict deployment and transport of the integration flow but via a proper governance and transparency.
Enable design guidelines in the Script Collection, Message Mapping, and other applicable artifacts.
Custom design guidelines for the customers to write their own custom rules.

I am also planning to write a developer tutorial with an example. Stay tuned !

Hope you will get benefited with this feature. Please experience the feature and provide your valuable feedback.

Be a Cockroach: A Simple Guide to AI and SAP Full-Stack Development - Part I

2024-05-09T19:14:30.935000+02:00

Disclaimer: This blog delves into SAP full-stack development, incorporating SAP RAP for both front-end and back-end functionalities, Integration Suite for middleware tasks, TensorFlow for AI/ML modeling, and crafting a personalized system dashboard. Geared towards simplifying complex business systems through engaging stories, it's for those wanting to understand these concepts. If you're already well-versed, feel free to just stop here and explore other content.

I) Introduction (Story):

Most people came here to see what he has to say about cockroaches and their connection to SAP development. Some might be curious to learn about it. Few of you could feel grossed out and think cockroaches are yucky and ugly! But after reading this section, I hope you'll respect these special creatures called “COCKROACH”.

The world is more than 200,000 years old. Lots of animals have lived and expired over time. One of them was dinosaurs—they were huge and strong. Due to changes in the tectonic plates and environment, the Dinosaurs couldn't survive and disappeared. Now, you can see it in movies using computer-generated effects. However, cockroaches have been around for over 1,00,000 years and you can still come across them in your kitchen, living room, bathroom, trash can, and everywhere else. Cockroaches have adjusted to changes in their surroundings, which is why they still exist on the planet.

Even though companies produce items like HIT and Baygon to kill them, cockroaches survive by figuring out how to deal with those products. A study found that certain products cause some cockroaches to behave as if they're intoxicated. Their bodies adapt to these substances. This ability to change is known as adaptation.

There aren't any charities or groups dedicated to protecting cockroaches like there are for elephants and dolphins. Everyone wants to kill cockroach, yet these humble cockroaches keep on living and undefeated by human, and their existence is inevitable. Now that you understand, I trust you hold a greater appreciation for the modest cockroach. Similarly, Consultants must also adjust to shifts in market trends.

The IT market is going through changes like "Artificial Intelligence" and a higher demand for “SAP Full-Stack developers” and so on. In this blog we are go-through about SAP Full-Stack development.

(The story about cockroaches was inspired by a book called "11 Rules for Life" written by Chetan Bhagat)

II) Agenda:

Getting to Know SAP FullStack: Understanding the Full Stack
Background Insights: Delving into the Story Behind
System Design Visualization: Seeing the Technical Architecture
Practical Implementation: Learning the Practical Details Step-by-Step
Wrapping Up: Summarizing Key Points and Looking Ahead
Further Reading: Offering Links to More Research Materials

III) Getting to Know SAP Full-Stack: Understanding the SAP Full-Stack development

Many of our consultants believe that full stack development involves only front-end and back-end development. However, when it comes to SAP development, there's a third component: middleware.

In today's market, relying on just one system isn't enough. Data comes from various sources, and we need to consolidate it to generate reports. So, if you're interested in becoming an SAP full-stack developer, this blog will help you understand the basics of all three elements: front-end, middleware, and back-end, and explain them practically.

Fig1: Basic diagram for Full-Stack development

IV) Background Insights: Delving into the Story Behind

Sundhara Pandian is a big part of Kotravai Coffee Group in Queenstown, New Zealand. He makes really good coffee. His trick? He gets top-notch coffee beans by following a careful process.

Instead of just buying beans whenever he wants, Sundhara Pandian sends a request called a Purchase Order (PO) to a big coffee supplier in Bremen, Germany. But it's not as simple as filling out a form. The PO has to go through a smart system with AI and automation. This system checks the beans in the supplier's stock and confirms the order.

But Sundhara Pandian's job doesn't stop there. The results from the system are put into SAP S/4HANA Cloud. This helps keep track of orders and how much coffee is left.

Basically, Sundhara Pandian does more than just make coffee. He's good at handling complicated systems to make sure Queenstown always has enough beans for coffee lovers.

Let's dive into SAP full-stack development through Sundara Pandian's story as we build the SAP RAP App for PO cockpit, iFlows and AI/ML model.

V) Technical Architecture Diagram: Visualizing the System Design

Fig2: Architecture diagram for E2E – Full Stack development with SAP applications

VI) Technical Implementation: Step-by-Step Integration Details

AIM: (Custom Cockpit and Integrated Goods Receipts):

We're making a simple app for custom Purchase Orders (PO) with basic fields.
Obtaining the GR information electronically, then utilizing a machine learning algorithm to automatically update the GR within our application system (GR automation).
We're getting details about Goods Receipts (GR) from GR automation system and showing them in the custom PO form.

Target Audience:

Individuals with 0 to 4 years of experience in SAP.
Enthusiastic learners eager to explore new concepts and expand their knowledge base.

Before proceeding, please ensure the following prerequisites are met:

Familiarity with basic concepts of SAP S/4HANA Cloud and SAP RAP.
Activation of the ADT-Eclipse, Postman, Integration Suite API plan in your SAP BTP entitlement.
Understanding of fundamental AI and Automation concepts.
Knowledge related to Python, Javascript, System landscape and UI

Step 1: Install the ADT package in the Eclipse. You can refer the below link to download the ADT package in the eclipse.

ADT Link: https://tools.hana.ondemand.com/#abap

Step 2:

Step two has been divided into two sections, each detailed below:

Section-A: Connect to the S/4 HANA Cloud system.
Section-B: Create the modelling with SAP RAP

Go to help-> About Eclipse IDE -> Double check the below highlighted icon from your system.

Section-A: Connect to the S/4 HANA Cloud system

To connect the S/4 HANA Cloud system, please use the below steps.

Type the S/4 HANA Cloud web address: https://<Host>.s4hana.cloud.sap. Remember, select client 80. This client is used for making changes to the system.

Click the "Copy Logon URL to Clipboard" button. Then, open your web browser and paste the URL there. Enter your login details and press enter. After successful login, you'll see the screen below.

Click "Finish" and create your own package name. I've used "ZDEMO" here.

Part 1: Key Takeaway:

Gaining foundational knowledge of systems and landscapes through story-based learning.
Grasping the fundamentals of full-stack development with diagrams
Setting up Eclipse for SAP ADT and establishing a connection to the S/4 HANA Public Cloud system.

Part 2: Coming soon - Keep an eye out for an exciting story-based learning journey as we delve into creating SAP RAP applications.

Part 3: Coming soon - Embark on an exploration of AI and ML with our forthcoming model, create system with a dashboard for ( GR Automation )

Part 4: Coming soon - Learn how to connect the circuit using Integration Suite in our upcoming installment.

Author: If you find this information helpful, please consider clicking the "Like" button on this blog and sharing your thoughts in the comments section below. You can also connect with the author on their LinkedIn profile: [Author's LinkedIn Profile]

New Release Available: SAP Cloud Connector 2.17.0

2024-05-10T14:33:54.505000+02:00

We are happy to announce that the fresh version of the SAP Cloud Connector is now available for download. It is (as usual) packed with a host of new features and improvements. From bug fixes to enhancements, we've worked diligently to deliver an updated connector that addresses critical issues while also enhancing usability and functionality, which you can find more detailed in the release notes.
Moving onto enhancements, we've made changes to the underlying architecture and features of the Cloud Connector. One of the major changes is the switch from JavaWeb 3.x runtime on Tomcat 8.5 to JavaWeb 4.x, which operates on Tomcat 9. This new runtime container facilitates better performance and stability.
In addition, Cloud Connector 2.17 now supports the use of SapMachine 21 as Java runtime. This change can provide increased efficiency and flexibility for your operations.
One of the significant enhancements in this release is the addition of support for up to 3 LDAP servers for authentication.

This feature is essential for setups where the user base is spread across multiple LDAP user stores or multiple user bases in a single LDAP user store (find more in the documentation).
We've also introduced the option to configure a separate port for the HA-related communication between the master and shadow instances.

This new feature enables you to use HA in conjunction with certificate-based authentication to overcome the limitation from 2.16 (find more here)
Additional hardware monitoring REST APIs have been provided for disk and CPU status, allowing for more comprehensive system insights. Plus, you can now use the hardware monitor on the shadow instance as well.
To improve your operations, we’ve introduced for the access control settings a creation timestamp, providing more detailed and useful information for your operations:

Finally, we've improved the Cloud Connector UI by adding a session expiration progress bar. This new addition helps you keep track of your active session and alerts you when you need to log in again.

In summary, with this release, we've not only ensured the highest security levels but also worked on improving the overall user experience and functionality. Don't hesitate to upgrade your Cloud Connector to version 2.17.0 today (by downloading it from here) and explore these new features and improvements. For more detailed information, make sure to check out the official release notes. Enjoy the enhanced performance and functionality of the new SAP Cloud Connector!Happy Connecting!

Installing SAPRouter on Linux: A Step-by-Step Guide

2024-05-11T15:55:48.753000+02:00

What is SAP Router ?

SAPRouter is a software component used to secure communication between SAP systems and the internet. Installing SAPRouter on Linux is a crucial step in ensuring secure communication for your SAP landscape. This step-by-step guide will walk you through the installation process.

Prerequisites:

- Linux server (e.g., CentOS, Ubuntu)

- Root access to the server

- SAPRouter software package downloaded from the SAP Support Portal

Step 1: Download SAPRouter:

Download the SAPRouter software package from the SAP Support Portal. Ensure that you download the correct version for your operating system.

Step 2: Extract the SAPRouter Package:

Transfer the downloaded SAPRouter package to your Linux server. Use the following command to extract the package:

tar -xvf saprouter_<version>_linux_x86_64.tar.gz

Step 3: Create a Directory for SAPRouter:

Create a directory to store the SAPRouter files. You can use the following command to create the directory:

mkdir /usr/sap/saprouter

Step 4: Copy SAPRouter Files:

Copy the extracted SAPRouter files to the newly created directory:

cp -R <path_to_extracted_files>/saprouter /usr/sap/saprouter

Step 5: Create a Configuration File:

Create a configuration file named `saprouter.ini` in the `/usr/sap/saprouter` directory. Here's a basic example of the configuration file:

# SAProuter Configuration File

version = 39

httpport = 81

tracefile = /usr/sap/saprouter/saprouter.trc

authid = *

permit = *

Step 6: Set Permissions:

Ensure that the SAPRouter binary and configuration files have the correct permissions:

chmod 755 /usr/sap/saprouter/saprouter

chmod 644 /usr/sap/saprouter/saprouter.ini

Step 7: Start SAPRouter:

Start the SAPRouter using the following command:

/usr/sap/saprouter/saprouter -r -R /usr/sap/saprouter/saprouter.ini

Step 8: Verify SAPRouter Status:

Verify that SAPRouter is running and listening on the specified port (e.g., 81):

netstat -tuln | grep 81

Step 9: Configure Firewall:

Configure your firewall to allow incoming and outgoing traffic on the SAPRouter port (e.g., 81) to ensure proper communication.

Step 10: Configure SAP Systems:

Update the `secinfo` file of your SAP systems to include the SAPRouter details for communication through the SAPRouter.

Overall information:

By following these steps, you can successfully install SAPRouter on your Linux server. This will help secure communication between your SAP systems and the internet, ensuring the integrity and confidentiality of your SAP landscape.

#SAP #SAPRouter #Linux #Installation SAP Young Thinkers #Red Hat Enterprise Linux SUSE Linux Enterprise Server SAP Women in Tech SAP Integration Suite SAP Business Application Studio @Muthumayandi @Subramanian @Sap @sapsapsap

Technovation Summit 2024: Expand your AI and BTP Horizons

2024-05-13T15:21:27.786000+02:00

This week the SAPinsider Technovation Summit will commence its inaugural event, with the topics of Artificial Intelligence (AI) and SAP Business Technology Platform (BTP) the stars of the show. And following on just a few short months from the last SAPinsider EMEA conference, it will be great to see if the buzz and excitement around these two topics is just as palpable in Barcelona as it was in Copenhagen.

Technovation Summit 2024 is a 1.5-day event designed for anyone interested in AI or SAP BTP and offers an agenda rich in informational sessions and learning opportunities for any technology enthusiast seeking to expand their knowledge and understanding of the development, application and best practices in these two hot topics. And in addition to hearing from a wide range of speakers at the event, there will also be ample opportunity to network and dive into deeper technology discussions with technology experts and industry peers.

Barcelona of course is a wonderful city location for an event like this. After spending a long day inside, absorbing all the sessions that one can fit into a day, building your knowledge and expanding your horizons of the potential that AI and BTP has to offer your business, what better place could there be to then relax and reflect on the day by taking the time to visit the beach, enjoy a city walk, or unwind and eat Tapas in one of the many tavernas on Las Ramblas, while marveling at the diverse and unique architecture.

6 of the Best for Barcelona

The trouble with attending any event, is selecting which sessions to see that will prove to be of most value to you, especially when there are so many opportunities to learn new things. My advice is simple, try and plan as much in advance, and create your personal agenda as early as possible, so that when you get to the event you can spend less time planning and more time networking and seeking answers to the burning questions that are most important to you and your business.

So why don’t I help a little and start you off with some suggestions for your agenda. Let’s simply assume that everyone will plan to attend the day-1 and day-2 keynotes, to be delivered by Iver van de Zand (14 May, 9:30am) and Prof. Marek Kowalkiewicz (15 May, 9:30am) who will share their own insights and experiences of AI, and in the case of Iver BTP too, setting the scene and the tone for each day of the summit. But after the keynotes, what next? Here are my personal 6 picks for Barcelona:

Generating the Future: SAP Business AI and the Generative AI Roadmap, 14 May, 10:45am to 11:30am
AI Ethics – A Primer to Guide your Organization on the Opportunities and Risks, 14 May, 11:45am to 12:30pm
How to Automate Document Streams using SAP BTP and Machine Learning, 14 May, 1:30pm to 2:30pm
Case Study: Moving From On-Premises to The Cloud with SAP Integration Suite, 14 May, 3:00pm to 4:00pm
What's on your Mind? Maximize the Potential of SAP BTP & AI, 14 May, 4:15pm to 5:00pm
What's on your Mind? Unveiling the Magic of Digital Transformation, 15 May, 11:45am to 12:30pm

And, if you don’t mind, I’ll take a liberty and add one more to see. This isn’t strictly a session, but it will cover many topics that are BTP and AI related, and this is delivered in the form of a BTP: Technology News Update, which will take place at the SAP booth during the networking break at 10:15 on 14 May. Please head to the booth to hear from our presenter Carles, and then network with the SAP onsite team to ask them any questions that spring to mind. Don’t be afraid to challenge them – they’ll appreciate you spending the time to talk with them!

Of course, these are just a few suggestions, and there are many more sessions available for you to select from the agenda. Why not take a look at the SAPinsider Technovation Summit website and look at the event sessions list, or even better download the SAPinsider2024 event app to your smart device (available on the App Store and Google Play) and to see a full list of available sessions, and to create your own personal agenda.

Continue Your Learning After the Event

While I’m sure you’ll enjoy your experience and learning opportunities in Barcelona, I expect that you’ll also want to continue your own personal learning journey after the event too. In this case why not explore other learning opportunities from SAP. For example, find out how to build software applications, side-by-side extensions, and integrations to and from cloud applications by exploring SAP’s free learning content on SAP BTP. It is made for both integration designers and extension developers from all levels of expertise and will help you stay up to date with the latest SAP BTP innovations. And check out even more role-based learning resources and opportunities to get certified in one place on SAP Learning site.

For those of you travelling to Barcelona this week, I do hope that you find the SAPinsider Technovation Summit to be an informative and stimulating experience. My best wishes for safe travels, and I hope that you enjoy the event!

Chris Grundy

Product Marketing, SAP BTP

SAP Integration with SuccessFactors Through CPI: A Step-by-Step Handbook for Seamless Connectivity

2024-05-14T10:33:22.921000+02:00

Introduction:

This blog focuses on integrating SAP ERP with SuccessFactors through CPI, particularly addressing the configurable or standard Integration-Flow. Whether setting up initial connectivity with SAP or updating connectivity to a new SAP server, the steps outlined in this blog remain consistent across different SAP landscapes.

This blog will cover building the two-way communication channels:

SAP ERP to CPI Suite
CPI Suite to SAP ERP

Step :1 – Navigate to the CPI Suite and Copy the Name of the Integration Flow (I-Flow).

For Example: “Replicate Employee Master Data and Org. Assignments from SAP SuccessFactors Employee Central to SAP ERP or SAP S4HANA”

Step:2 – For all configurable Integration Flows (I-Flows), you can access the technical integration settings guide on the SAP Help Portal. Simply search on Google using the keyword "I-FlowName.pdf" to locate the relevant documentation.

For Example: “Replicate Employee Master Data and Org. Assignments from SAP SuccessFactors Employee Central to SAP ERP or SAP S4HANA.PDF”

This URL is applicable to the current situation https://help.sap.com/doc/2b73e3be50474cdc8cce76c3ff4860b6/2311/en-US/SF_EC_S4_Org_Objects_HCI_en-US.pdf

Under the "How to find Endpoint" section, locate the "Search Service definition." There, you'll find two procedures outlined in the SAP guide. Follow the steps provided in the guide accordingly.

1st Procedure - SAP ERP Communication to CPI:

Tips: Search for "SOAMANAGER" or "SM59" within the PDF guide to find the relevant web service definitions.

Step:3- Configure the URL in I-Flow Receivers

Step:4- Paste the URL into the Address section of the relevant Receiver.

2nd Procedure - SAP ERP Communication to CPI:

Tips: Look up "SOAMANAGER" or "SM59" in the PDF guide to find the relevant procedures.

Step:1 – Deploy the I-Flow

Step 2 - Copy the Endpoint URL or WSDL from Manage integration Content

Step 3 – Proceed to SOAMANAGER and Enter CPI SID in Username and Password

Then, navigate to the Transport Settings tab and enter the CPI Endpoint WSDL/URL.

Conclusion:

In conclusion, establishing seamless integration between SAP ERP and SuccessFactors through SAP Cloud Platform Integration (CPI) is critical for optimizing business processes and ensuring data consistency across the organization. This blog aims to provide a standard and comprehensive guide for configuring two-way communication regardless of the specific Integration flow (I-Flow).

Integrating SAP S/4HANA with Kafka via SAP Advanced Event Mesh: Part1 – Outbound connection

2024-05-14T16:23:10.601000+02:00

Introduction

In today's fast-paced business world, the ability to seamlessly communicate and exchange data between different systems is crucial. SAP Advanced Event Mesh (AEM) offers a robust solution for real-time event-driven communication across various SAP systems and external services. In this blog post, we'll explore how to integrate S/4HANA with Kafka using SAP AEM for data streaming and event-driven architecture.

If you’re new to SAP AEM, I highly recommend checking out this blog series to understand the fundamentals.

Step-by-Step Guide

Let's break down the connection process between S/4 HANA and Kafka using SAP AEM into 6 sections, each explaining a key part of the connection setup to help you easily understand and implement the process.

1. Login and Setup SAP AEM Service

First, log in to your BTP subaccount and create a subscription for AEM ensuring your user has the required roles. Please note that I've skipped the steps on how to create an AEM subscription, but you may refer to this blog for detailed steps. Once subscribed, log in to your SAP AEM tenant and navigate to Cluster Manager to create an Event Broker service. This service enables applications to publish or consume events. Below is the start page of SAP AEM after logging in.

Create an Event Broker Service by clicking on ‘Create Service’.

Provide a meaningful name for the service e.g. – ‘AEM_SERVICE_DEV’, select the service type, and choose the region. Click on "Create Service".

After the service is activated, you'll see the page.

Navigate to "Manage" and then "Authentication". Enable Client Certificate Authentication.

2. Establishing Trust between S/4 HANA and AEM

To implement client certificate-based authentication, you need to establish trust between S/4 HANA and the AEM service broker. In your S/4 HANA system, execute the STRUST transaction to open the Trust Manager. Export the certificates from SSL client (Standard) and upload them into AEM in the next step.

Go to "Manage" and then "Certificate Authorities". Upload the exported certificates by clicking on "Add Client Certificate Authority".

Once done, all the certificates will be displayed as shown below.

Now, import the certificate chain of the SAP AEM service broker host and BTP-IS Subaccount host in the SSL client (Standard) in the STRUST transaction code.

3. Broker Manager Configuration in AEM

Click on "Open Broker Manager" and log in using the "Management Editor Username" and "Management Editor Password". You can find these access details under the "Status" section of the broker service.

Once logged into Broker Manager, create a Queue which will serve as a storage mechanism for messages received by SAP AEM. When S/4HANA will generate any events or messages, they will be placed in the queue before being processed and forwarded to Kafka.

Provide a meaningful name for the Queue e.g. – ‘AEM_DEV’.
Assign a Subscription to the Queue. By creating a subscription, we ensure that our SAP AEM instance is subscribed to the relevant topics or events generated by S/4 HANA.

Go to "Access Control" and create a Client Username with the hostname from the leaf certificate maintained in SSL Client (Standard) in the STRUST.

4. Configure AEM to Kafka connection through Kafka Sender Bridge

The Kafka Sender Bridge is required to facilitate communication between AEM and the target Kafka cluster by converting AEM messages into Kafka events and propagating them to the remote Kafka cluster.
To establish client certificate authentication between AEM and the Kafka cluster, you'll need .jks files of the Keystore and Truststore from your target Kafka broker.
Open the command prompt and use the command 'keytool' to convert the .jks files into .p12 files. Here's how:

keytool -importkeystore -srckeystore C:\OpenSSL\<keystorefilename>.jks -destkeystore C:\OpenSSL\keystore.p12 -srcstoretype jks -deststoretype pkcs12

keytool -importkeystore -srckeystore C:\OpenSSL\<truststorefilename>.jks -destkeystore C:\OpenSSL\truststore.p12 -srcstoretype jks -deststoretype pkcs12

Once converted, copy these .p12 files to the OpenSSL -> Bin folder.
Now, navigate to the 'OpenSSL' directory and convert these .p12 files to .pem files using the commands below:

openssl pkcs12 -in keystore.p12 -out keystore.pem

openssl pkcs12 -in truststore.p12 -out truststore.pem

You'll need to set a passphrase during this process. Note: Remember this passphrase, as you'll need it for client certificate authentication.
From the ‘truststore.pem’ file, copy the content of root and the leaf certificates and save it as .cer files. Add those in our service broker under "Manage" -> "Certificate Authorities" -> "Domain Certificate Authorities".

Now, navigate inside Broker Manager to "Kafka Bridges" and create a "Kafka Sender".

Add the Kafka Broker Host and Port details in the ‘Bootstrap Address List’ and copy the contents of the 'keystore.pem' file and paste them under Client Certificate Authentication - > Content as shown below. Additionally, include the passphrase that we entered while converting the .p12 file to .pem in the ‘Password’.

Once the Kafka Sender is created, go inside, and click on "Queue Binding".

Select our queue – ‘AEM_DEV’ created in section 3.

Go inside the Queue Binding created in earlier step and add the topic name of the target Kafka cluster in the "Remote Topic".

Confirm whether the Kafka connection is up and running.

5. Configure S/4 HANA to SAP AEM connection

Now to establish a connection from S/4 HANA to AEM go to transaction code SM59, create a type-G RFC destination and enter the host and port of the SAP AEM service broker.

In transaction code /IWXBE/CONFIG, create Channel configuration in the S/4 HANA system by clicking on ‘via Service Key -> Advanced’ and assign the RFC destination created in the earlier step. In the ‘Service Key’ section enter the JSON content of the service key created using ‘aem-validation-service-plan’ instance in BTP cockpit. Please refer this blog on how to create a service key.

Save the above changes and activate the channel.
Create an outbound binding and assign any standard topic. For example, select "Business Partner". So whenever a Business Partner is newly created or modified, a standard event will be raised through this outbound channel.

6. Testing the end-to-end connection

To test the end-to-end connection, go to transaction code BP and create a Business Partner. Click on save.

Once saved, an event should be raised. You can check this by going to transaction code /IWXBE/EEE_SUPPORT and then to /IWXBE/R_EVENT_MONITOR.

Select your AEM channel.
You will find a list of all events that were raised and sent to AEM.

Now, go to AEM. In the Kafka sender, you can see the message count in the sent section has increased. This means that the message was successfully received by AEM and then pushed to the Kafka cluster. Additionally, verify the message at the Kafka end.

You can also navigate to the 'Try-Me' section where you can set up the sender and receiver connection. Subsequently, you can subscribe to our topic at the receiver end and observe the incoming message from S/4 HANA as shown below.

Conclusion

Through this blog, we've demonstrated the process of sending an event from SAP S/4HANA to Kafka via SAP AEM. Now, enterprises can leverage the power of event-driven architectures to drive innovation and efficiency in their operations.

Stay tuned for my next blog, where we'll explore how to establish an inbound connection from Kafka to S/4 HANA via SAP AEM, completing the full event-driven communication loop.

References

https://community.sap.com/t5/technology-blogs-by-sap/advanced-event-mesh-create-your-first-event-broker/ba-p/13571945

https://community.sap.com/t5/enterprise-resource-planning-blogs-by-sap/sap-s-4hana-integration-with-sap-integration-suite-advanced-event-mesh/ba-p/13577271

https://community.sap.com/t5/technology-blogs-by-sap/video-tutorial-sap-s-4hana-and-sap-integration-suite-advanced-event-mesh/ba-p/13575193

https://help.pubsub.em.services.cloud.sap/Cloud/cloud-lp.htm

https://help.pubsub.em.services.cloud.sap/Cloud/Broker-Manager/config-kafka-bridge.htm

SAP BTP Trial Account Creation and Enabling Integration Suite service(SAP CPI)

2024-05-16T09:35:26.608000+02:00

Create an SAP Account from SAP.com
It Creates the SAP Universal ID
Fill in the Required Fields and Click on Submit.

2. After Clicking on the Submit button, SAP will send an email to finalize your SAP account.

3. Open Your Mail and Click to activate your account

4. Now We have to create the SAP BTP Trial Account, go to account.hana.ondemand.com and click on sign in, give your email and password, and click on sign in.

5. After logging into the BTP Cockpit, Click on Trial Home.

6. Choose the region you want and click on Create Account

7. After clicking on Create Account, it creates a Global account and subaccount for you, then click on Continue.

8. Click on Go To Your Trial Account.

9. Click on Subaccount.

10. Click on Instances and Subscriptions

11. Here we can see the services we subscribed to.

To access the Integration Suite, first, we must Subscribe to it.

12. To Subscribe to the Integration Suite, click on Service Marketplace and search for Integration Suite.

13. Click on Integration Suite and click on Create.

14. Click on create.

15. To check Whether the Integration Suite is created or not click on View Subscription.

16. Now we can see the Integration suite in subscriptions.

Click on Integration Suite, it will open the Integration Suite.

17. It will Show you the Authorization error because to access the Integration Suite we have to assign an Integration_Provisioner Role for the User.

18. To assign an Integration_Provisioner Role, Go back to the BTP Cockpit click on Users, and select the User.

19. After Selecting the User click on Assign Role Collection.

20. Search for the Integration_Provisioner Role and click on Assign Role Collection.

21. After Assigning Role, Click on Instances and Subscriptions and then Click on Integration Suite.

22. Click on Add Capabilities.

Note: - Sometimes it will not show the add capabilities option, it will take time to resolve this Refresh the page or try sign-out and Sign in again.

23. After clicking on Add Capabilities, it will show you the list of Capabilities,

Select the capabilities that you need and click on next.

24. In capabilities if you select Manage APIs then this step will appear, select the two checkboxes and click on next.

25. Click on Activate.

26. To activate Capabilities of Integration Suite it will take some time.

27. After the activation click on ok.

28. To create and monitor the Iflow we need to assign some roles related to Process Integration.
Go back to BTP Cockpit->User->Assign the below Roles

29. Now open the Integration Suite from BTP cockpit->Instances and subscriptions.

Now You can create, edit, and monitor the Iflows.

How to use the OData-API for exchanging Messages from Logs and the Datastore in Integration Suite

2024-05-16T09:36:10.757000+02:00

In this blog I will show the basic steps and components to access the MessageProcessingLog and the attached messages, wether they are stored in the log itself, in the MessageStore or in the DataStore.

Later in further blogs I will go into details and show some specific implementations and will compare the different types of message-stores. So please be patient and check back for further information.

The starting point is in the SAP Business Accelerator Hub. There is an API package SAP Cloud Integration which offers some OData APIs in Version 2. We are focussing on the Message Processing Logs and download the definition of, what is called here a vdm - virtual data model. I don't want to go deeper in the edmx, only so far, if you search on your fav. search-engine you may get the following statement:

edmx file is an XML file that defines an Entity Data Model (EDM), describes the target database schema, and defines the mapping between the EDM and the database.

On the Business Accelerator Hub you can find tons of documentation about how to use the source code generator in maven, on how to build up (in this case) java-classes, test the API online, with a sandbox or with your own tenant and even code-snippets in many different languages can be downloaded.

In our scenario we build a client in the Netweaver Development Studio. Then maven-support is initially not part of it but can be added. Search for the keyword M2E. A good starting point on how to generate the client classes (vdm - virtual data model) and how to generate a project from a Maven archetype is on sap.github.io .

The overall framework for the implementation here is an OnPremise fat-client, but everything can run deployed on a BTP tenant, only the GUI frontend needs to be changed.

What you get when generating the sources for your client is the following helper-packages and classes and a MessageProcessingLogsService:

There are several steps to consider for enhancing your pom-file: Dependencies and plugins. E.g. here: SAP Cloud Application Programming Model: Deep Insert (5) Consume Remote Service with SAP Cloud SDK

The starting point here is the method getAllMessageProcessingLogs()

Depending on wether the code runs on your local laptop or in the cloud, you have to create a destination first:

import com.sap.cloud.sdk.cloudplatform.connectivity.AuthenticationType;
import com.sap.cloud.sdk.cloudplatform.connectivity.DefaultHttpDestination;
import com.sap.cloud.sdk.cloudplatform.connectivity.HttpDestination;
import com.sap.cloud.sdk.cloudplatform.security.BasicCredentials;

(...)
public DefaultHttpDestination getDestination() throws IllegalArgumentException	{
	if (destination == null)	{
		if (this instanceof BasicAuthentication)	{
			destination = DefaultHttpDestination.builder(tenant.getURL())
				.authenticationType(AuthenticationType.BASIC_AUTHENTICATION)
	   	    	.basicCredentials(new BasicCredentials(((BasicAuthentication)this).getUser(), ((BasicAuthentication)this).getPassword()))
	   	    	.keyStore(keystore)
	   	    	.build();
		}
	}
	return destination;
}

(On your tenant you can use the DestinationService)

With the destination described above we can request all packages and artifacts. As for now the request is sent by getAllIntegrationRuntimeArtifacts(). This includes iFlows but also Rest-APIs and more.

Unfortunately

runtime-artifacts do not point to any package
designtime-artifacts are called from a package-perspective but do not include some types like Rest-APIs.

(Status on 2024-05-24 is a response of HTTP 501 - not yet implemented. the dev is seems still working on putting everything in place)

Hence we have a gap of links between packages and artifacts, in case they are not iFlows (e.g. Rest-Api). The following code puts "everything" together as complete as possible and attaches the artifacts without a link to a package into a virtual package named "<unknown>"

//create Hashtable for storing our version of packages
designPackages = new Hashtable<String, DesignPackage>();

if (authentication != null)	{
	//get url
	String tenantURL = authentication.getTenant().getRuntimeURL();
	
	//create destination
	DefaultHttpDestination destination = authentication.getDestination();
	
	//create the services generated in the vdm
	IntegrationContentService iCS = new DefaultIntegrationContentService().withServicePath("/api/v1/");
	
	//create Hashtable for storing our version of runtime artifacts
	Hashtable<String, IntegrationRuntimeArtifact> ira_table = new Hashtable<String, IntegrationRuntimeArtifact>();
	
	//retrieve all packages, execute the network-request
	try {
		final List<IntegrationPackage> ip = iCS.getAllIntegrationPackages()
				.executeRequest(destination);
		
		//retrieve all runtime artifacts
		final List<IntegrationRuntimeArtifact> ira = iCS.getAllIntegrationRuntimeArtifacts()
				.executeRequest(destination);
		
		//and put them in the table
		for (IntegrationRuntimeArtifact ira_iter : ira)
			ira_table.put(ira_iter.getId(), ira_iter);
		
		//iterate over all packages
		for (IntegrationPackage ip_iter : ip) {
			
			//store it in our Hashtable
			DesignPackage dp = new DesignPackage(ip_iter);
			designPackages.put(dp.getName(), dp);
			
			//get artifacts for this package
			final List<IntegrationDesigntimeArtifact> ida = ip_iter.getIntegrationDesigntimeArtifactsOrFetch();
			
			//set artifacts as child to the package
			for (IntegrationDesigntimeArtifact ida_iter : ida) {
				String url = tenantURL + "/http/" + ida_iter.getId();
				Artifact a = new Artifact(ida_iter, url);
				dp.addArtifact(a);

				//remove from the list of runtime artifacts
				ira_table.remove(a.getId());
			}
		}
		
		//add remaining artifacts not linked to any package
		if (ira_table.size() > 0)	{
			
			//default package
			IntegrationPackage integrationPackage = new IntegrationPackage();
			integrationPackage.setName("<unknown>");
			defaultDesignPackage = new DesignPackage(integrationPackage);
			designPackages.put(defaultDesignPackage.getName(), defaultDesignPackage);
			
			Enumeration<String> t = ira_table.keys();
			while (t.hasMoreElements())	{
				String iraId = t.nextElement();
				Artifact a = new Artifact((IntegrationRuntimeArtifact)ira_table.get(iraId), tenantURL + "/http/" + iraId + "/*");
				defaultDesignPackage.addArtifact(a);
			}
		}
	} catch (com.sap.cloud.sdk.datamodel.odata.client.exception.ODataConnectionException e) {
	} catch (ODataException e) {
	} catch (Exception e) {
	}
}

Having all packages and artifacts in place, we can get a (filtered) set of log-entries:

/**
* Returns an ArrayList of LogEntries. 
* The list is based on the filtering.
* <p>
* This methods processing-time depends on the amount of data requested from BTP.
* The status should be set on the GUI for user-information.
* In future releases this method will work in a background-task.
*
*  authentication object relevant for login and serving a destination based on this creds
*  logListFilter the object for filtering the requested logs, i.e. datetime and number of entries
* @return ArrayList of LogEntries. 
*/
public ArrayList<LogEntry> updateMessageProcessingLog(Authentication authentication, LogListFilter logListFilter)	{
	
	ArrayList<LogEntry> logEntries = new ArrayList<LogEntry>();
	
	if (authentication != null)	{
		MessageProcessingLogFluentHelper mPLFH = mPLS.getAllMessageProcessingLogs();
		
		//logs should be filtered always by artifact-id (which is unique)
		if (logListFilter.getArtifact() != null)
			mPLFH = mPLFH.filter(MessageProcessingLog.INTEGRATION_FLOW_NAME.eq(logListFilter.getArtifact().getId()));
		
		//number of results limited by number?
		if (logListFilter.getCountFilterEnabled())	{
			mPLFH = mPLFH
			.skip(logListFilter.getCountFilterFrom() - 1)
			.top(logListFilter.getCountFilterTo());
		}
		
		//filtered by date?
		if (logListFilter.getDateFilterEnabled())	{
			//set start and end, logListFilter returns a @LocalDateTime 
			mPLFH = mPLFH.filter(MessageProcessingLog.LOG_START.gt(logListFilter.getDateTimeFilterStart()));
			mPLFH = mPLFH.filter(MessageProcessingLog.LOG_END.lt(logListFilter.getDateTimeFilterEnd()));
		}
		
		//the result is always ordered ascending by the LogEnd
		mPLFH = mPLFH.orderBy(MessageProcessingLog.LOG_END, Order.ASC);
		final List<MessageProcessingLog> mpl = mPLFH.executeRequest(authentication.getDestination());
		
		for (MessageProcessingLog mpl_iter : mpl)	{
			logEntries.add(new LogEntry(mpl_iter));
			
			try	{
				List<MessageStoreEntry> messageStoreEntries = mpl_iter.fetchMessageStoreEntries();
				if (messageStoreEntries != null)
					for (MessageStoreEntry mse : messageStoreEntries)	{
						List<MessageStoreEntryAttachment> mSEAs = mse.fetchAttachments();
						//in future releases <do_something> with the MessageStoreEntries and their attachments
					}
			}
			catch(com.sap.cloud.sdk.datamodel.odata.client.exception.ODataResponseException e)	{
				System.out.println("com.sap.cloud.sdk.datamodel.odata.client.exception.ODataResponseException");
			}
		}
	}
	
	return logEntries;
}

The result, shown as a Java-GUI is as following:

The java-object MessageProcessingLog and corresponding ..Attachment is based on the classes automatically created within the generator-plugin:

With that code we can retrieve the payload on any message starting from object MessageProcessingLog retrieve a MessageProcessingLogAttachment with the following code:

import com.sap.cloud.sdk.datamodel.namespaces.messageprocessinglogs.MessageProcessingLog;
import com.sap.cloud.sdk.datamodel.namespaces.messageprocessinglogs.MessageProcessingLogAttachment;

(...)

//the messageprocessinglog returns a java-List of attachments
List<MessageProcessingLogAttachment> mplAttachments = null;
try {
	mplAttachments = messageProcessingLog.fetchAttachments();
	
	//create a Hashtable which we use to store a local object version of attachment, keys derived from the attachment-name
	attachments = new Hashtable<String, LogAttachment>();
	for (MessageProcessingLogAttachment mpla_iter : mplAttachments) {
		LogAttachment logAttachment = new LogAttachment(mpla_iter);
		//use the name as key and put the attachment in the hashtable
		attachments.put(mpla_iter.getName(), logAttachment);
		
		//this logEntry has an attachment
		hasAttachment = true;
	}
	isPrepared = true;

} catch (Exception e) {
	// TODO Auto-generated catch block
	e.printStackTrace();
}

Different to the previous approaches, the attachment itself (payload) is not provided as a java-object, but by an InputStream. It will be stored in a String and returned by a getter:

//the constructor gets the 'mpla' when instantiated
public LogAttachment(MessageProcessingLogAttachment mpla)	{
   this.mpla = mpla;
}

//prepare the payload seperate from the getter (for caching purpose)	
public void preparePayload()	{

    InputStream is;
    try {
	is = mpla.fetchMediaStream();
		
	if ( is instanceof ByteArrayInputStream)	{
	  int n;
	  try {
	    n = is.available();
	    byte[] bytes = new byte[n];
	    is.read(bytes, 0, n);
	    payload = new String(bytes, StandardCharsets.UTF_8);
	  } catch (IOException e) {
	    e.printStackTrace();
	  }
        }
    } catch (ODataException e) {
      // TODO Auto-generated catch block
      e.printStackTrace();
    }
}
//getter for returning the payload as a String 
//Todo for a future release: what happens if a payload is 'big'? Let's say > 10mb. 
//We don't want to store that in memory in a String!?
public String getPayload()	{
	
    if (payload == null)
      preparePayload();
	
    return payload;
}

Based on the code shown in this blog any app can download, show and also re-send the payload to it's origin:

Stay tuned for further blogs describing details. Don't hesitate to request details in which you are interested.

Best regards Jörg Hopmann

Enabling In-Order processing with SAP Integration Suite, advanced event mesh Partitioned Queues

2024-05-16T13:18:43.926000+02:00

This blog has been co-authored by -

Prashant Rakheja, SAP
Hari Rangarajan, Solace

This blog covers SAP Integration Suite, advanced event mesh’s support for Partitioned Queues (henceforth referred to as PQs) and how this feature can be used to support several critical business needs such as in-order & parallel processing of messages. It also covers a special use-case involving “exactly-once-in-order” processing of messages with SAP Cloud Integration & AEM, by leveraging PQs.

Overview of Partitioned Queues

In AEM, there are two access types for Queues:

1) Exclusive

In an exclusive Queue, only one consumer can actively connect and consume messages from the Queue, while others remain on standby.

2) Non-exclusive

In a non-exclusive Queue, multiple consumers can actively connect to the Queue and consume in a round-robin fashion.

To learn more about configuring AEM Queues, you can refer to the AEM help doc.

A partitioned queue is a special type of non-exclusive Queue where each partition represents a logical subdivision within the Queue.

Key aspects of PQ:

Broker distributes incoming messages to partitions in a PQ based on a ‘partition key’. All messages with the same ‘partition key’ are handled by the same partition.

The following table provides API information regarding how the ‘partition key’ can be set on corresponding API for each client type:

Client API

API Parameter

JMS
JMSXGroupID
HTTP
JMSXGroupID
MQTT5
JMSXGroupID
SMF
QUEUE_PARTITION_KEY
AMQP
group-id

For up-to-date information on details about the exact key refer to this link.

2. If no partition key is specified, the message would be randomly distributed to any of the partitions.

3. Each partition connects to exactly one consumer.

4. Each partition in a PQ can handle more than one partition key. All messages within a partition are delivered in-order. However, there is no guarantee of message ordering across different partitions in a Queue.

5.One of other benefits of Partitioned Queues is that it enables parallel consumption of messages from the Queue, thereby improving egress throughput without the need for additional ‘exclusive access’ queues.

In the above diagram, there is a non-exclusive Queue that contains three partitions.

G1 and G2 are messages that belong to the same group (same partition key). But H1, H2 (same group) also reside in the same partition. This point is important because while setting the partition key ensures that messages that carry the same partition key end up in the same partition, a given partition can also have messages that carry a different partition key. All the messages (in the same partition) would be delivered in-order irrespective of their group.

As far as consumption is concerned, there are 2 consumers for 3 partitions (refer to the diagram above).

Each partition maintains a sticky connection to the consumer, and in case the consumer goes down, connection is reconciled to another consumer. A partition can have at-most 1 active consumer, which is important from the point-of-view of maintaining in-order processing.

Exactly-Once-In-Order (EOIO) processing

In addition to in-order processing, if a scenario requires ‘exactly-once’ quality, it is important to understand some edge cases which cannot be handled by AEM Broker alone.

AEM’s Partitioned Queues described above can be leveraged along with SAP Integration Suite’s Cloud Integration capability to build scenarios involving “exactly-once-in-order” processing. By combining the power of AEM for in-order processing and the power of Cloud Integration for ‘exactly-once’ processing, there is a possibility to handle EOIO.

Why is 'exactly-once' messaging important for your business?

In some cases, it is important that the messages are delivered exactly-once due to the inability of the consuming system to process duplicates, or the transaction is not differentiable as a duplicate or different transaction.

For example, a FinTech company processing loyalty points using an event-driven-architecture -- Imagine the consequences if some of those events are delivered more than once, when they were not supposed to be.

In any case, it's important that the middleware system can support this.

Building EOIO scenarios in Integration Suite (Cloud Integration and AEM)

With the new AEM adapter, it becomes easy to connect Cloud Integration with AEM. The AEM adapter is a key component which makes it simple to consume AEM’s native features such as Partitioned Queues etc. in Cloud Integration. It is also possible to connect to AEM with standard adapters like AMQP, but it is recommended to go with AEM adapter itself since it is tailor made for AEM.

Publishing events via Cloud Integration

For publishing, AEM broker provides the following APIs: JMS/AMQP/HTTP/MQTT. If there is a need to have other client types, Cloud Integration can be leveraged. The iflow below uses an HTTP endpoint for the client to publish messages. It shall be possible to use other adapters following a similar approach.

This section is optional if AEM broker’s APIs are directly used.

The above iFlow can be used as a reference model for publishing messages in-order to AEM.

The sequence diagram above depicts how the messages flow into Cloud Integration using the HTTP adapter and get published in AEM broker using AEM adapter.

While publishing messages to AEM broker, following steps are needed to set the ‘partition key’ in AEM adapter:

1. Use message property to dynamically assign the ‘partition key’ -- the actual value can be set via something like a groovy script step.

This is a small snippet where groupID is extracted from the message payload (note: the message payload needs to contain such data) and setting that as a message property to provide context to Cloud Integration.

In Advanced Event Mesh adapter ‘user properties’ configuration, create a new key-value pair as shown below to set the partition key.

Since the adapter uses SMF protocol, the key should be ‘JMSXGroupID’ -- you should use QUEUE_PARTITION_KEY (which is constant with the value ‘JMSXGroupID’) while using the programmatic client.

The most important consideration during publishing messages to cloud integration is that the client must wait for acknowledgement before publishing the next message. This is critical for ensuring that events are persisted in-order in the AEM queue.

Consuming from AEM in Cloud Integration

The above iFlow modelling presents a reference for consuming messages from AEM using the AEM adapter and forwarding to a target system with “exactly-once” quality.

To enable exactly once processing of messages, the following steps are needed:

AEM sender adapter channel configuration

While consuming from AEM, in the AEM adapter configuration, the acknowledgement mode should be ‘Automatic on Exchange Complete,’ this is important since the adapter waits for the entire exchange to be completed and does not return a response immediately.

The consumer mode should be set to ‘Guaranteed.’

The iflow needs to implement idempotent behaviour

This can be achieved by using local idempotent process call step. Once the message flows to Cloud Integration, the message ID can be set using groovy scripting.

The approach defined above for setting message property can be used here as well.

The reason for setting a message ID in the consumption iFlow is to provide some context to Cloud Integration to determine the uniqueness of the message. The duplicate check is performed with the help of the local idempotent process call. Before executing the local idempotent process call, Cloud Integration checks if the message ID has been processed before, if yes, it is not processed again. Hence, even if the client publishes the messages multiple times, the message would only be processed once.

If the message has not been processed before, Cloud Integration will execute the local idempotent process call and thus, forward the message.

It is up to the end user to build their own logic for determining the uniqueness of the message, in this case, the message ID was part of the message payload itself.

Handling failures

If there is a failure at the camel exchange level, e.g something goes wrong with the groovy script “setMessageID” as shown above or the HTTP call to S4HANA fails, the message would be retried as per the processing configuration of the adapter.

As you can see in the above configuration, we are setting the ‘Maximum message processing attempts’ to 5, this means the adapter attempts the delivery five times.

However, after the maximum attempts of the adapter, when the message is returned to the broker, the queue level configuration kicks in.

If you have set the ‘Redelivery’ to ‘Try forever’ at the Queue level on the broker, the message is again picked up for processing by the adapter and this loop continues forever!

However, there are other strategies of dealing with this –

(i) You can set a finite redelivery count for your queue and then the message won’t block the queue throughput. The message would be discarded once the redelivery count is exhausted.

(ii) It is also possible to use a 'dead-message-queue (DMQ)' in conjunction with the redelivery count on the broker. The message moves to the DMQ after the retry attempts are exhausted.

There could be more ways of handling failures/retries but that this beyond the scope of this post.

Ensuring order during consumption

The “in-order” processing quality is ensured while publishing by waiting for acknowledgement before sending a new message. Fortunately, on the consumption side, since we are leveraging AEM adapter, the ordering is taken care of by the AEM adapter itself! (given that you have set acknowledgement mode to ‘Automatic on exchange complete’ and consumer mode to ‘Guaranteed’ as described above)

The Cloud Integration worker nodes internally maintain a sticky connection to the Partitioned Queues which ensures that messages from the same partition are consumed in order.

All of this combined in SAP Cloud Integration & AEM gives us the magic of “exactly-once-in-order” messaging.

Scale with Partitions or Scale with Queues

The ability to scale consumers effectively is a critical aspect of ensuring optimal performance and throughput. Since this blog concerns with “in-order” processing of events, we will be looking at a couple of strategies which can be used namely:

Scaling with partitions:

As discussed earlier, Partitioned queues divide message traffic into multiple partitions based on a partition key, which is defined by the producing client application. Each partition operates independently, allowing messages to be processed concurrently across multiple consumer instances. This parallel processing capability is essential for achieving high throughput and scalability in distributed messaging environments.

Partitioned queues facilitate horizontal scaling by allowing multiple instances of a consumer application to process messages in parallel. Each consumer instance is responsible for processing messages from one or more partitions, enabling linear scalability as the number of consumer instances increases.

It’s important to note that auto-scaling capabilities (e.g. Kubernetes Event-driven Autoscaling [KEDA]) can also be leveraged to automatically adjust the number of consumer instances based on workload demand. Cloud-based deployment platforms and container orchestration frameworks provide built-in support for elastic scaling, enabling consumer applications to dynamically scale up or down in response to changing message traffic patterns. Scaling policies and thresholds based on different criteria can be defined to implement proactive scaling strategies to anticipate workload spikes and ensure timely resource provisioning.

Scaling with queues:

In SAP AEM, multiple queues and pattern matching in subscriptions can be leveraged to achieve consumer scaling while maintaining in-order processing.

Multiple queues can be used to distribute message traffic across multiple consumers, facilitating parallel message processing and horizontal scalability. Pattern matching subscriptions allow consumers to subscribe to multiple queues simultaneously based on subscription patterns, enabling dynamic routing of messages to matching queues. By combining these features, organizations can achieve both scalability and in-order message processing in complex messaging scenarios.

To read more about pattern matching in SAP AEM, you can refer to the docs here: Wildcard characters in topic subscriptions

To configure multiple queues with pattern matching, you need to create each queue separately and assign it to a specific consumer. Each consumer then processes messages from its assigned queue. The number of queues you create will depend on the level of parallelism you want to achieve.

When a new consumer is added, a new queue can be created and assigned to it. If a consumer fails or is removed, its queue can be reassigned to another consumer or deleted. This dynamic assignment and reassignment of queues enable consumer scaling and ensure high availability.

While multiple queues and pattern matching enable high levels of consumer scaling, there are some performance considerations to keep in mind. As the number of queues increases, so does the overhead of managing these queues. Therefore, it's important to find a balance between the level of parallelism and the management overhead.

Sincerely hoping that this helps you & your organisation level up your integration game with SAP Integration Suite!

Deploying SAP Edge Integration Cell Playground

2024-05-18T17:14:05.051000+02:00

Cat deploying SAP Edge Integration Cell during server room night-shift

Who should read this blog

You read about SAP Edge Integration Cell (EIC) and want to try it out yourself.
You want to deploy it with the least effort and cost.

Note: This is my personal learning journey - this is by no means a deployment recommendation or replacing proper sizing or SAP recommendations stated for this product. It might or might not work for you.

What is SAP Edge Integration Cell

Welcome to the world of SAP Edge Integration Cell (EIC), a cutting-edge solution designed to bridge the gap between cloud innovation and on-premises data security. EIC is an integral part of the SAP Integration Suite, offering a hybrid deployment model that allows businesses to design integration content in the cloud and deploy it securely within their local environment. This powerful tool addresses the critical need for data compliance and security while enabling seamless data connectivity across diverse systems. Whether you're dealing with sensitive data that cannot be hosted in the cloud or looking to leverage the agility of cloud-based services without compromising on-premises security, EIC provides the best of both worlds.

Overview of deployment

As stated, we want to have a playground for EIC at the end! For that we'll run through these steps:

Create a virtual machine (VM)
Install Docker on VM and configure VM
Deploy K3S on VM & Configure K3S
Activate EIC on SAP Integration Suite
Deploy and run Edge Lifecycle Management & EIC on K3S
Test for successful installation

Let's start!

1. Create a virtual machine

If you have a 6+ CPU-core machine sitting around on which you can install Linux and which is reachable through DNS from the internet, you can use this device! I had no free machine at home that meets these specs, so I used my Google Cloud Platform (GCP) account to set up a VM. Why 6 cores? This is the minimal value you can choose! Each deployment the ELM will trigger later has a CPU and Memory requirement. If you add this up, you end up with a value below 6 but greater than 4. You can find all the details in the official guideline, remember, this blog is not a recommended path for a test or production setup!

I chose 6 vCPUs and 16GB memory together with a 40GB boot disk. I recommend to select the VM availability policy from "Standard" to "Spot" which brings down the cost per month significantly, 40% of the standard cost in my case. I don't care if Google shuts down the VM in peak times, but it never happened so far anyway. As for the OS I selected Debian Bookworm. Once the VM is provisioned, you can SSH into it.

You want the VM to be reachable from the internet (so that Integration Suite finds it). Since I go cheap, there's only a dynamic public IP and for that I use Duck DNS to provide me with a fixed URL that SAP IS can use later.

Just follow the "installation instructions" for cron on Linux which is super simple and you have a cron-job at the end that keeps the IP always updated.

2. Install Docker on VM and configure VM

Next, we'll get docker up and running. I recommend to work with the official Docker documentation. As for me I had to add the current user I used for SSH to the docker user group after installation:

sudo usermod -aG docker $USER

Then activate the change:

newgrp docker

3. Deploy K3S on VM & Configure K3S

Now we're good to run K3S inside Docker as a very lean and quick way to provide for Kubernetes. I used the Rancher image which you deploy like shown below in your SSH session.

docker run -d --restart=unless-stopped   -p 80:80 -p 443:443   --privileged   rancher/rancher:latest

I found this description very helpful. Once it runs, open the Rancher UI. You need to derive the initial password through this command (just replace the "your docker container name" with the name of the container without the double quotes. You get the container name (or ID) through docker ps.

docker logs "your docker container name" 2>&1 | grep "Bootstrap Password:"

Finally you should see something like this (likely without that many pods running).

What's missing is a storage class on your new K3S cluster. Without that, no persistence and the installation of Edge Lifecycle Management (ELM) will fail. In the upper right area of the 'local' page of Rancher you'll find the option to open a shell.

Enter below command and wait for the deployment to finish.

> kubectl apply -f https://raw.githubusercontent.com/rancher/local-path-provisioner/v0.0.26/deploy/local-path-storage.yaml

Check that the provisioning of local-path storage class was successful in the same shell:

> kubectl -n local-path-storage get pod
NAME                                     READY   STATUS    RESTARTS      AGE
local-path-provisioner-dbdfc4849-g9wqj   1/1     Running   1 (23d ago)   23d

You should also see it under Storage➡StorageClasses in the UI. The SAP ELM just requests storage but doesn't specify which class. Therefore, set the new StorageClass as default:

ELM and EIC make also use of Kubernetes' scaling capabilities. Install the horizontal pod autoscaler on the K3S cluster as a next activity. Like before, open the shell inside(!) the UI of Rancher and execute these commands:

> helm upgrade --install metrics-server metrics-server/metrics-server

check for installation correctness:

> kubectl get --raw "/apis/metrics.k8s.io/"
{"kind":"APIGroup","apiVersion":"v1","name":"metrics.k8s.io","versions":[{"groupVersion":"metrics.k8s.io/v1beta1","version":"v1beta1"}],"preferredVersion":{"groupVersion":"metrics.k8s.io/v1beta1","version":"v1beta1"}}
>

Our cluster is ready!

4. Activate EIC on SAP Integration Suite

As next step, the EIC needs activation in your subscription. There can be cost impacts to this step, please do whatever is necessary on your end you don't run into trouble before this!

Activate EIC in Settings➡Runtime. Wait for it to complete. This is only making the required settings in your SAP Integration Suite Account. It has no effect on the EIC setup.

5. Deploy and run Edge Lifecycle Management & EIC on K3S

This is the most time-intensive step in that sense that - given you've set up all up so far - you have to wait most of the time. It can take up to one hour (usually 30-40 mins) until it's finally deployed on K3S. I'd assume it might be faster with a bigger-sized cluster. It's exciting though to see the services spinning up so with a cup of coffee or tea there's some entertainment provided.

First, click on the link that was provided when you activated the EIC (see image above). Then click on "Add Edge Node". A window with several steps pops up. Check all the steps. You need to create a P-User (with according role assignment. You should now have all what you need already by now. Click "Next". Set the name of the EIC, no need to check anything (it would increase the EIC's size).

In the next window you'll assign the users for the BTP (technical P-User) and the Repo-Access user. Later you can check it here and it should look similar to that. Make sure to have the right roles assigned in the BTP account!

So, no "Logging and Monitoring" and no "Local Container Registry" as well as "HTTP Proxy". Unless you need a proxy of course.

Next, download the kubeconfig file from Rancher. It's the button to the left of the "kubectl shell" button you clicked earlier to set up the StorageClass and HPA. A file will be downloaded. It should already contain the duckdns URL.

Move back to the setup of the ELM/EIC page and it should look similar to this before you upload the kubeconfig file:

Upload the file and assign a password. Now you can download the bootstrapping file. Chrome might block the download, so check it and allow it.

Follow the 4 steps now. It will set up the ELM which is the basis for the EIC deployment. It will take a while and you can watch how the services spin up in the Rancher UI. After maybe 10mins of waiting, you should see successful deployment of the components below:

Let's now deploy the EIC on top of ELM. You click "Deploy Solution" and select the version to deploy, I've always chosen the latest, it will look therefore different for you when you read the blog.

On the Rancher side of things this will create another 3 namespaces. Here's how it looks for me.

All of the above are created through ELM and then the upper 3 through EIC solution deployment. Time again for a coffee ☕! If you get asked about "Loadbalancer" choose "AWS" and "Internal".

Once complete, you should see a page similar to this:

Congratulations! You won't see the version with exclamation marks, that's just for me as I didn't upgrade since the last deployment 23 days ago (I should do that now after the blog 😃).

At the end of the installation, add a keystore for the EIC in the Integration Suite itself. (Manage Security➡Keystore➡Create for EIC, it's the "Add" button on the Keystores screen. It takes a while to replicate, maybe 3mins). Then create a keypair with the alias name and CN the virtual host.

6. Test for successful installation

Time to test for successful installation! Let's move to the SAP Integration Suite page.

Settings➡Integration should show an active instance of the EIC by now.

Let's now do a simple connectivity test.

Select the EIC, and hit the tile "Connectivity Tests". On the following page select "TLS" test and enter www.google.com (or anything else). It should be successful.

Successful FTP Test to a test server

And we can do the same for FTP or any other protocol.

Successful TLS test to Google.com

From here, you can now start developing iFlows like you would with SAP Integration Suite but use the runtime of EIC with all benefits that this solution provides.

Closing

In conclusion, our journey through setting up a tiny, yet functional SAP Edge Integration Cell (EIC) environment has demonstrated that with the right tools and guidance, you can create a playground for exploring the capabilities of EIC. This hands-on experience is for understanding how EIC can fit into your organization's integration strategy, providing the agility of cloud services while maintaining the security of on-premises data handling. I hope this guide has been a helpful starting point for your adventures with EIC.

I'm eager to hear about your experiences and any insights you've gained along the way. Please share your thoughts and questions in the comments below – your feedback is not only welcome but essential for my learning as well. For further reading and to deepen your understanding of SAP Edge Integration Cell and SAP Edge Lifecycle Management, be sure to explore the following resources:

Thank you for joining me on this technical excursion, and I look forward to our continued dialogue as we delve deeper into the world of SAP integrations.

SAP Tech Bytes: SAP S/4HANA Cloud + SAP Integration Suite, AEM + SAP Build Process Automation

2024-05-20T13:25:58.571000+02:00

In this blog post, I share a couple of SAP Tech Bytes on how to enable eventing between SAP S/4HANA Cloud and SAP Build Process Automation via SAP Integration Suite, advanced event mesh. We will configure and trigger events from SAP S/4HANA Cloud, which will then be received by SAP Integration Suite, advanced event mesh, and finally forwarded to SAP Build Process Automation.

Enabling the end-to-end event-driven integration scenario between SAP S/4HANA Cloud and SAP Build Process Automation involves many steps and requires extensive documentation. Sometimes, it might be best to process all that information in video format.

If you're interested in learning how to enable the same scenario using SAP Event Mesh, check out this blog post: https://community.sap.com/t5/technology-blogs-by-sap/sap-tech-bytes-deliver-events-from-sap-eventmesh-to-sap-build-process/ba-p/13574547.

SAP Tech Bytes: Configuring SAP S/4HANA Cloud and SAP Integration Suite, advanced event mesh

Steps shown in the video:

Now that we've configured the communication between SAP S/4HANA Cloud and SAP Integration Suite, advanced event mesh, we can proceed to configure the delivery of events to SAP Build Process Automation.

SAP Tech Bytes: Delivering events from SAP S/4HANA Cloud to SAP Build Process Automation via AEM

Steps shown in the video:

In this SAP Tech Bytes, we’ve seen what’s required to configure eventing between SAP S/4HANA Cloud and SAP Integration Suite, advanced event mesh. Also, we created and configured a REST Delivery Point in AEM in order to forward events to SAP Build Process Automation. The REST Delivery Point functionality is not limited to SAP Build Process Automation; you can also deliver events to other systems that expose an HTTP endpoint to receive them. I hope you enjoy this video on event-driven integration, as there is more to come on this topic in the near future….

Happy eventing!

Client API	API Parameter
JMS	JMSXGroupID
HTTP	JMSXGroupID
MQTT5	JMSXGroupID
SMF	QUEUE_PARTITION_KEY
AMQP	group-id

SAP Community - SAP Integration Suite

How to host static webpages through SAP CPI-Iflow

Supporting Multiple API Gateways with SAP API Management – using Azure API Management as example

Introduction

Automated API import

Accessing the API Import Utility

Overview

Building the code

Preparing Access to Azure API Management

Prepare access to the SAP API management catalog

Running the API import utility

Conclusion

Contacts

Govern SAP APIs living in various API Management gateways in a single place with Azure API Center

What API solutions can be registered to Azure API Center?

Automagically registering SAP API Management APIs on Azure API Center

Discover all your APIs where you code – see VS Code and GitHub Copilot in action

Enforced API metadata is your second line of defense against API sprawl

API Linting gets you to the next level

Thoughts on production readiness

Final words

API Management: Unlocking OAuth Strategies

Introduction

Pre-requisites

Creating simple integration flow

Securing integration flows

Test your APIs

Conclusion

SAP Emarsys Customer Engagement - Enhanced Lead Integration to SAP Sales Cloud Version 1

Table of Contents

Introduction

Background Information

Setup Steps in SAP Emarsys Customer Engagement

Create the Webhook Node Preset

Appearance Settings

Create the Automation Program

Integration Process

SAP Integration Suite

Purpose

Integration Flow

Mapping

Inbound / Outbound Connectors

Connectivity Setup between SAP Emarsys & SAP Integration Suite

Endpoint URL

Authentication

Conclusion

10+ ways to reshape your SAP landscape with SAP Business Technology Platform – Blog 7

Blog 7: The Role of Automation for Transformation Projects

Why looking at automation only is not enough

The role of AI in Enterprise Automation

Dynamically creating MPL custom headers without hardcoding in groovy script

Lets Send multipart/form-data to external Receiver: CPI to AI App

SAP Integration Suite - Design Guidelines in the integration flow editor of SAP Cloud Integration

Introduction

What are Design Guidelines?

Why Design Guidelines?

Motivation

Availability

Personas

Workflow

Configure Design Guidelines

Execute Design Guidelines

FAQs

What’s coming next?

Be a Cockroach: A Simple Guide to AI and SAP Full-Stack Development - Part I

New Release Available: SAP Cloud Connector 2.17.0

Installing SAPRouter on Linux: A Step-by-Step Guide

Technovation Summit 2024: Expand your AI and BTP Horizons

6 of the Best for Barcelona

Continue Your Learning After the Event

SAP Integration with SuccessFactors Through CPI: A Step-by-Step Handbook for Seamless Connectivity

Integrating SAP S/4HANA with Kafka via SAP Advanced Event Mesh: Part1 – Outbound connection

SAP BTP Trial Account Creation and Enabling Integration Suite service(SAP CPI)

How to use the OData-API for exchanging Messages from Logs and the Datastore in Integration Suite

Enabling In-Order processing with SAP Integration Suite, advanced event mesh Partitioned Queues

Deploying SAP Edge Integration Cell Playground

Who should read this blog

What is SAP Edge Integration Cell

Overview of deployment

1. Create a virtual machine