SAP Community - SAP Event Broker for SAP cloud applications

Automating Advanced Event Mesh Broker Creation

2024-02-08T09:07:53.387000+01:00

Introduction

The need to reduce costs associated with operating and maintaining an active message broker led to the creation of a tool that automates the generation and configuration any time the need for asynchronous event-based messages arises.

This article will showcase a method of creating an event broker in Advanced Event Mesh by leveraging the REST endpoints provided by Solace and SEMP configuration. The solution will allow the system to request a broker creation, monitor the status, re-generate the queues and topics and extract the endpoints for sending or receiving data. Besides that, a UI has been set up that simplifies the broker configuration and allows users to persist or modify configurations.

Scenario Overview

All the necessary steps for broker creation are done in SAP Integration Suite using multiple iFlow artifacts for each process step, JMS Queues for handling connection errors, timeouts and message routing, Datastore for saving broker configurations and a main page that acts as a user interface for setting up everything.

Architecture Overview

Picture 1 - Architecture Overview

UI Configuration Page

Picture 2 - UI Configuration Page

The configuration page is a HTML page exposed as a REST endpoint, containing multiple sections that allow the user to generate new configurations, use existing ones and even check the status of the already-deployed broker creation jobs.

Picture 3 - Deployment Page

On this page, we can see the status of the active or pending brokers on the cluster. I also added the ability to delete the broker on demand.

Technical Setup

The process of creating an event broker has been detailed in the Managing Services with the SAP Integration Suite, Advanced Event Mesh REST API section and it consists of several REST calls in order to create a broker, get the connection details and check the creation status.

The most important parameters that allow us to continue with the next steps are :

serviceId - the id of the service cluster generated
msgVpnName - the internal id of the broker
defaultManagementHostname - the hostname that will be used for queue, topic and rest delivery point creation

In order to create an event broker :

HTTP POST https://api.solace.cloud/api/v0/services

{
   "name": "Broker Name",
   "datacenterId": "<Data Center Id>",
   "serviceClassId":"developer",
   "serviceTypeId": "developer",
   "adminState": "start"
}

The authentication for this endpoint is made via an authorization token generated in the Cluster Manager.

Picture 4 - API Token Management

In order to create a queue object :

HTTP POST https://<defaultManagementHostname>/SEMP/v2/config/msgVpns/<msgVpnId>/queues

{
"accessType": "exclusive",
"egressEnabled": true,
"ingressEnabled": true,
"msgVpnName": "<msgVpnId>",
"owner": "all",
"permission": "consume",
"queueName": "<queue-name>"
}

In order to create a topic subscription object :

HTTP POST https://<defaultManagementHostname>/SEMP/v2/config/msgVpns/<msgVpnId>/queues/<queue-name>/subscriptions

{
"msgVpnName": "<msgVpnId>",
"queueName": "<queue-name>",
"subscriptionTopic": "<topic/name>"
}

In order to create a rest delivery point object :

HTTP POST https://<defaultManagementHostname>/SEMP/v2/config/msgVpns/<msgVpnId>/restDeliveryPoints

{
    "clientProfileName": "default",
    "enabled": true,
    "msgVpnName": "<msgVpnNameId>",
    "restDeliveryPointName": "<restDeliveryPointId>"
}

If everything is correctly triggered, we should be able to see in the Cluster Manager our newly created broker, and as well in the broker console the queues and topics.

Picture 5 - Cluster Manager

Picture 6 - Queues

Picture 7 - Topics subscribed to a queue

Adapter for inserting/consuming events

By using the recently published adapter for Advanced Event Mesh integrations, we can insert and consume events from our brokers. Below is a sample iFlow artifact that connects to a specific topic and consumes messages from there.

Picture 8 - iFlow artifact for topic message consumption

Conclusions

Using the SAP Help documentation was the key behind the success, because the product is rather new on the market and we did not really heard of other use cases yet. By following the steps we were able to easily connect two Business Technology Platform products and we were able to leverage event-based integration far more powerful than the traditional SAP Event Mesh.

For any questions or suggestions, feel free to post here or in the special Q&A Section.

Thank you!

CloudEvents at SAP 🌁

2024-03-04T07:13:27.857000+01:00

TLDR; SAP is an active contributor to the CloudEvents specification and it has adopted the specification for describing event data in common formats to provide interoperability across its services/applications/systems. In the SAP Business Accelerator Hub, you can find all the events exposed by a total of 17 different SAP services that publish events following the specification, including its flagship ERP products, SAP S/4HANA Cloud public edition and SAP S/4HANA, exposing more than 600+ events combined. To enable seamless event-based integrations between SAP applications, SAP applications must conform to the CloudEvents specification. SAP also offers three services, part of the SAP Business Technology Platform (SAP BTP), that can help customers embrace event-driven architectures, these are: SAP Event Broker for SAP cloud applications, SAP Event Mesh, and SAP Integration Suite, advanced event mesh.

Before we dive into how CloudEvents have been adopted within SAP, we might need to talk a bit about what an Event-Driven architecture is and what a CloudEvent is as well. Let's get started.

📢A new Developer Challenge will be announced later this week. The challenge will focus on CloudEvents and Event-driven architectures. Stay tuned for more 📺.

Event-Driven Architectures

Long gone are the days when a system (aka target system) will constantly poll to check if there are any changes in another system, e.g. a new customer created in a master data system. Traditionally, the target system will only know this by programming a routine that will poll a file share and check for any files produced by this system or a web service exposed in the source system, every X minutes/hours/days. The expectation nowadays is that systems are integrated and that the data exchanged between these systems will be immediately available in the target system(s) if any data is created/changed in the source system. Enter Event-Driven architectures.

An Event-Driven Architecture is a software architecture paradigm concerning the production and consumption of events. An event can be defined as a significant change in the state of an object within a system[1]. For example, when a customer/supplier/employee (business object) is created/updated/deleted ( action) in a system. Translating this to the SAP world, when a Business Partner is created/changed in SAP S/4HANA (source system), SAP S/4HANA can notify that there was a change in a business object and target system(s) interested in the Business Partner object can then react and trigger follow-up activities in their systems.

How do source and target systems communicate?

Now, if the source system lets other systems know of any changes happening in its business objects, it will not be sustainable to create a new programming routine within the source system every time we want to notify a new target system of any changes. Traditionally we would have some form of middleware, e.g. SAP Cloud Integration, and configure our source system, an SAP S/4HANA system, to send notifications of these events to the middleware and then use the middleware to distribute these messages, e.g. we would add target system(s) as needed. Now, we are moving the problem from the source system to a sort of middleware but ideally, there will be a way for the source system to notify others without the need to make any changes. Enter the event broker.

An event broker is message-oriented middleware that enables the transmission of events between different components of a system, acting as a mediator between publishers and subscribers. It is the cornerstone of event-driven architecture, and all event-driven applications use some form of event broker to send and receive information[2].

By introducing an event broker in our landscapes, we can configure our source systems to publish their events to this message-oriented middleware. The source system will specify the class of the message (aka topic). Then, systems interested in the changes happening, e.g. in a particular business object in the source system, can subscribe to the event(s), via the event broker, by specifying the topic they are interested (topic-based filtering) in or it can also be based on the content of the message (content-based filtering). There are two keywords important here, publish and subscribe (PubSub), this is a well-known messaging pattern used to decouple systems/applications and allow asynchronous communication between them.

What is the PubSub messaging pattern?

Publish-subscribe is a communication pattern that is defined by the decoupling of applications, where applications publish messages to an intermediary broker rather than communicating directly with consumers (as in point-to-point)[3]. In a way, publishers and consumers do not need to know each other; they simply publish (produce) or consume (receive) the events. When following this messaging pattern we move from the traditional polling mechanism to know if there have been any changes in the source system to reacting to real-time events (notifications) the moment something happens in the source system.

A subscriber (consumer), is generally only interested in a subset of the messages published. Normally, a message-oriented middleware will provide subscribers with a mechanism to set filters on the data that they want to receive. There are two common ways of filtering: topic-based and content-based[4].

Topic-based filtering

We mentioned before how target systems can subscribe to events by specifying a topic they are interested in.... some event brokers will allow subscribers to subscribe to topics by using wildcards (*) and they will be able to receive only the events they are interested in, which can be for different topics. For example, let's assume we have an SAP S/4HANA with the name S4D and it publishes the Business Partner create and change on the following topics: sap/S4HANAOD/S4D/ce/sap/s4/beh/businesspartner/v1/BusinessPartner/Created/v1 and sap/S4HANAOD/S4D/ce/sap/s4/beh/businesspartner/v1/BusinessPartner/Changed/v1. A subscriber system could subscribe to both topics using a wildcard, e.g. sap/S4HANAOD/S4D/ce/sap/s4/beh/businesspartner/v1/BusinessPartner/*/v1 and receive the messages for both event types. In this case, we are filtering the message by topics.

Content-based filtering

In some other cases, the event broker will allow the subscriber to specify which messages they want to receive, based on the attributes or content of the message. For example, the message sent to the event broker can contain attributes that describe the message and we can do some filtering based on the values of those attributes. Note: An example is shared later, on how attributes in a message (CloudEvent) can be used to define filters.

SAP offers different services that can act as event brokers. We will discuss this further in the SAP's event-driven portfolio section.

With many systems in our landscapes and each one being developed by different vendors/teams, it would be good if there was a standard way of structuring these events to simplify how systems create/handle/process these messages right? Enter CloudEvents.

What is CloudEvents?

Today's system landscapes are very complex and we need to deal with many systems communicating with each other, ideally as close to real-time as possible. Nowadays, a system can publish events to notify other systems of the changes happening within the objects of their system. Given that we are talking of many systems, ideally, there will be a common way of describing the data produced by these systems. The CloudEvents specification can help us with this. We can leverage it to provide a consistent way for how our systems can communicate with others about these events.

As mentioned on the CloudEvents website... CloudEvents is a specification for describing event data in a common way. It's goal is to simplify event declaration and delivery across services, platforms and beyond! The specification is now under the Cloud Native Computing Foundation.

Below is an example of what a CloudEvent message will look like:

{
  "specversion": "1.0",
  "type": "com.github.pull_request.opened",
  "source": "https://github.com/cloudevents/spec/pull",
  "subject": "123",
  "id": "A234-1234-1234",
  "time": "2018-04-05T17:31:00Z",
  "comexampleextension1": "value",
  "comexampleothervalue": 5,
  "datacontenttype": "text/xml",
  "data": "<much wow=\"xml\"/>"
}

You'll notice that the example above is composed of many attributes, the event context. The context describes the event and is independent of the event data. Meaning that we can somehow process/inspect the event without needing to process its data. Now, let's dive a bit into the message itself.

For more information on the history, development and design rationale behind the specification, see the CloudEvents Primer document.

CloudEvents message format

A CloudEvent message is mainly composed of context attributes and data.

Context Attributes

A number of attributes can be included within the message, these attributes are known as Context Attributes and the idea is that these context attributes can be used to describe the event. We can think of these context attributes as the header information of our event that can be used for filtering and routing. Let's explore some of the attributes available.

Name	Required	Description	Example
id	✅	Identifies the event. Producers MUST ensure that source + id is unique for each distinct event.	63d6a150-c6a1-4c5b-bcc3-27d90c07941c
source	✅	Identifies the context in which an event happened.	/default/sap.s4.beh/244572008
specversion	✅	The version of the CloudEvents specification which the event uses.	1.0
type	✅	Describes the type of the event	sap.s4.beh.businesspartner.v1.BusinessPartner.Changed.v1
datacontenttype		Content type of the value in data.	application/json

Extension Context Attributes

A CloudEvent message may also include additional context attributes, which are not defined as part of the specification. These additional attributes are known as "extension attributes" and can be used by the producer systems or intermediaries to include additional metadata to an event, similar to how we can use HTTP custom headers.

For example, in the SAP Digital Vehicle Hub Business Events package, we can see that the event raised when a vehicle changes - sap.dmo.dvh.Vehicle.Changed.v1, contains the extension context attribute sappassport, which is an SAP specific tracing identifier.

{
  "specversion": "1.0",
  "type": "sap.dmo.dvh.Vehicle.Changed.v1",
  "source": "/eu10/sap.dmo.dvh",
  "subject": "808E6E30B65149978A443429B29FB300",
  "id": "a823e884-5edc-4194-a81a-f3a3632417ee",
  "time": "2018-04-08 08:31:00",
  "datacontenttype": "application/json",
  "sappassport": "string",
  ....
}

Data

A CloudEvent message may include a payload but this is not required. If included it will be in the format specified in the datacontenttype context attribute. Although it is not required, we will generally have a payload in messages. Below we can see an example of an event message that contains a payload. In this case, a Business Partner changed event generated by an SAP S/4HANA Cloud Public Edition.

{
    "type": "sap.s4.beh.businesspartner.v1.BusinessPartner.Changed.v1",
    "specversion": "1.0",
    "source": "/default/sap.s4.beh/244572008",
    "id": "63d6a150-c6a1-4c5b-bcc3-27d90c07941c",
    "time": "2024-02-26T10:53:06Z",
    "datacontenttype": "application/json",
    "data": {
        "BusinessPartner": "1000667"
    }
}

Check out the Business Partner changed event in the SAP Business Accelerator Hub - https://hub.sap.com/event/CE_BUSINESSPARTNEREVENTS/resource.

SAP's adoption of CloudEvents

SAP is an active contributor to the CloudEvents specification and one of its many adopters. Evidence of that is the different products (event packages) that we can find documented in the SAP Business Accelerator Hub > Events category.

The products listed below document their events in the form of event packages in the SAP Business Accelerator Hub. All their events follow the CloudEvents specification:

SAP Agricultural Origination Portal
SAP Batch Release Hub for Life Sciences
SAP Cell and Gene Therapy Orchestration
SAP Cloud for Utilities Foundation
SAP Digital Vehicle Hub
SAP GRC Cloud - Issue Management and Remediation Service
SAP Industry Process Framework
SAP Intelligent Clinical Supply Management for Operations
SAP Intelligent Clinical Supply Management for Planning
SAP Landscape Management Cloud
SAP Marketing Cloud
SAP Order Management Foundation
SAP S/4HANA
SAP S/4HANA Cloud Public Edition
SAP S/4HANA Utilities for Customer Engagement
SAP S/4HANA for Procurement Planning Cloud
SAP Subscription Billing

SAP's flagship ERP products, SAP S/4HANA Cloud Public edition and SAP S/4HANA, expose more than 600+ events combined. As you might have noticed from the events that we use as examples above, these events follow the CloudEvents specification. Below we can see the BusinessPartner Created event generated by an SAP S/4HANA Cloud Public Edition and we can see how it follows the CloudEvents specification.

{
    "type": "sap.s4.beh.businesspartner.v1.BusinessPartner.Created.v1",
    "specversion": "1.0",
    "source": "/default/sap.s4.beh/244572008",
    "id": "194780e0-b5db-1ede-b58a-4550178dff9e",
    "time": "2024-02-26T09:50:00Z",
    "datacontenttype": "application/json",
    "data": {
        "BusinessPartner": "1000667"
    }
}

Kyma

Kyma contains an Eventing module, which provides functionality to publish and subscribe to CloudEvents. It follows the PubSub messaging pattern and all events in Kyma conform to the CloudEvents specification[5]. At the moment, NATS (provided by the NATS module) and SAP Event Mesh are supported as backends for eventing and it is recommended to use the CloudEvents specification for sending and receiving events. Even the legacy events are automatically converted to CloudEvents.

SAP's event-driven portfolio

SAP offers various services that can help customers embrace event-driven architectures. The different offerings can meet customers where they are in their EDA adoption/implementation journey. These are:

SAP Event Broker for SAP cloud applications
SAP Event Mesh
SAP Integration Suite, advanced event mesh

SAP Event Broker for SAP cloud applications

SAP Event Broker is the latest service released by SAP that focuses on event-driven architecture and it supports event-based integration between SAP cloud and BTP-based applications. It is a fully managed cloud service that allows SAP cloud applications to communicate asynchronously through events. Meaning it can be used to distribute events between SAP cloud applications. For example, this service can be leveraged to enable seamless integration between SAP S/4HANA Cloud public edition and SAP Subscription Billing. When an integration scenario leverages SAP Event Broker, it eliminates having to configure an event storage or infrastructure such as customer-defined topics or queues to enable the event-based integration between systems.

Integration between SAP S/4HANA Cloud and SAP Subscription Billing

CloudEvents are first-class citizens in SAP Event Broker, as events that are distributed between SAP cloud applications using SAP Event Broker all conform to the CloudEvent Specification[6]. Evidence of this is how we can filter SAP event subscriptions by defining filters based on the CloudEvent context attributes, e.g. by filter by type, subject, or extension context attribute. Note: This is an example of content-based filtering taking place in the message-oriented middleware.

If all your event needs are within the SAP cloud world, e.g. exchange events between SAP cloud applications and SAP BTP, this service can fulfill your needs.

SAP Event Mesh

SAP Event Mesh allows applications to communicate through asynchronous events. It can be used to publish and consume business events from SAP and non-SAP sources across hybrid landscapes from the digital core (ERP products, e.g. SAP S/4HANA Cloud public edition and SAP S/4HANA) to extension applications through event-driven architecture. It is a fully managed centralised event-oriented service part of SAP BTP.

The service supports the use of REST APIs for Events, for publishing and consuming events that are compliant with the CloudEvents specification. Related to event-based integrations, events that flow through SAP Event Mesh between SAP applications must all conform to the CloudEvents specification.

You can check out how to enable the consumption of SAP S/4HANA Cloud public edition events in your SAP BTP account - https://youtu.be/4LspIL3NxqQ

SAP Integration Suite, advanced event mesh (AEM)

SAP Integration Suite, advanced event mesh (AEM) is a complete event streaming, event management, and monitoring platform that incorporates best practices, expertise, and technology for event-driven architecture (EDA) on a single platform. With AEM you can deploy event broker services, create event meshes, and optimise and monitor your event-driven system.

AEM offers these categories of services:

Event Streaming: It provides event streaming services and gives you access to event broker services that provide advanced features like the publish-subscribe messaging pattern, queuing, streaming, replay, and request-reply, while doing all the heavy lifting to dynamically deploy, upgrade, patch, and operate the service. It extends beyond the cloud and supports infrastructure and EDAs across hybrid cloud solutions and your on-premises deployments.
Event brokers can be deployed to: public clouds administered by SAP (Public Regions or Dedicated Regions) and customer's own regions in the cloud or on-premises (Customer-Controlled Regions)
Event Management: It provides event management services. The Event Portal provides powerful tools to create, design, share, and manage various aspects of your EDA based on event brokers or other streaming technologies. It fully supports the AsyncAPI 2.0.0 specification so that you can generate code, documentation, validate events, and apply API management policies for applications in your EDA.
You might have noticed that some event specification in the SAP Business Accelerator Hub are documented using the AsyncAPI specification, e.g. Marketing Campaign Events part of SAP Marketing Cloud.
Event Monitoring and Insights: Availability out-of-the-box of dashboards and easy-to-understand visualizations based on historical and real-time metrics, and timely notifications about the event broker services deployed.

Similar to SAP Event Mesh, it can be used to publish and consume business events from SAP and non-SAP sources across hybrid landscapes. Meaning, if your event data needs to exit the SAP cloud world, you can use any of these two services.

Thanks for making it this far, we've covered a lot in this blog post and I hope it gives you a general understanding of CloudEvents, its importance in the SAP world, how it can enable integration across SAP applications and the different SAP offerings that can enable your EDA journey.

Special thanks to Klaus Deissner (@klaus_deissner) for reviewing and contributing to this blog post.

Event-Driven architecture: link ↩️
What is an event broker?: link ↩️
What is the publish-subscribe messaging pattern?: link ↩️
Publish-subscribe pattern - Message filtering: link ↩️
Eventing module in Kyma: link ↩️
SAP Event-Driven integrations: link ↩️

Originally published at ajmaradiaga.com.

Unleash the Power of Real-Time Business Insights with SAP Advanced Event Mesh

2024-03-26T03:57:44.968000+01:00

Unleash the Power of Real-Time Business Insights with SAP Advanced Event Mesh

In today's ever-changing business landscape, being able to respond quickly is crucial. SAP Business Technology Platform (BTP) lays the groundwork for this agility by providing a comprehensive integration platform. But what if you could harness the power of instant reactions to every customer interaction? That's where SAP Advanced Event Mesh (AEM) comes in - a revolutionary tool that unlocks the true potential of SAP BTP. By utilizing the event-driven architecture championed by SAP BTP, AEM empowers businesses to make real-time, data-driven decisions - a vital capability in today's fast-paced environment.

Imagine a world where every customer interaction triggers an immediate response, enabling businesses to make informed choices in real-time. This is the power of SAP Advanced Event Mesh (AEM), a revolutionary tool that transforms how businesses react to events and unlock hidden opportunities .

(Image Source: SAP)

The Grocery Store Scenario: A Glimpse into the Future of Inventory Management

Consider a grocery store where every purchase instantly triggers an event that is relayed to a central system. This eliminates the need for traditional end-of-day reports, allowing for immediate insights into:

Product popularity: Identify fast-selling items and ensure they are always in stock.
Inventory optimization: Replenish stock as soon as it runs low, preventing lost sales and customer frustration.

This is just one example of how AEM streamlines business processes and enables proactive decision-making across various industries.

Key Features of SAP AEM:

Event-driven communication: Respond to events as they happen, creating a dynamic and responsive system.
Scalability and flexibility: Easily adapt to changing business needs and data volumes.
Unified event management: Gain centralized control and visibility over all events within your SAP landscape.
Seamless integration: Integrate seamlessly with SAP and Non-SAP solutions.

Components of SAP AEM:

Mission Control: Easily deploy and manage event brokers, monitor their performance, and visualize your event-driven architecture.

Event Portal: Design your event-driven setup using a full set of tools, including an overview, designer, catalog, and event manager for running events.

Insights: Get real-time insights into the health and performance of your event network, ensuring your applications work well.

Deployment Options:

You can choose from different deployment options for SAP AEM:

Public Regions: This allows you to use the infrastructure managed by SAP, which is simple and easy to use.
Dedicated Regions: This gives you more control and isolation within a dedicated SAP cloud environment.
Customer-Controlled Regions: This lets you deploy AEM on your own Kubernetes cluster, giving you ultimate customization and control

Security:

SAP AEM takes security very seriously at every level, with features like:

Secure cloud architecture with various deployment options.
VPC/VNet isolation for keeping data segregated securely.
Multi-factor authentication and authorization to have comprehensive access control.

March Developer Challenge - CloudEvents: It's a wrap!

2024-04-01T06:08:23.211000+02:00

Yesterday, we concluded March's developer challenge in the SAP Community - All about CloudEvents and event-driven architectures. Thanks to everyone who took part in it :-). I hope you had as much fun participating in the challenge as I had preparing it.

In case you missed the developer challenge but you are interested in knowing what it was all about... you can get started here: March Developer Challenge - CloudEvents: Week 1.

If you want to learn more about CloudEvents and event-driven integrations, I would recommend keeping an eye on the content of the Event-driven integration CodeJam - https://github.com/SAP-samples/events-driven-integrations-codejam, which is now close to being finalised :-). Also, remember that you can request/host an SAP CodeJam event in your city or company, and we will be happy to help you run it - https://community.sap.com/t5/sap-codejam-blog-posts/so-you-want-to-host-a-codejam-everything-you-need-to-know/ba-p/221415.

Are you wondering about what will be April's Developer challenge? It is live now 🚀... April (Citizen) Developer Challenge – SAP Build Apps: Task 0 - https://community.sap.com/t5/sap-builders-discussions/april-citizen-developer-challenge-sap-build-apps-task-0/m-p/13654312

Now, some cool stats on community members' participation:

A total of 36 SAP Community members took part in the challenge.
Unique SAP community members participation per week:
Week Total submissions
Week 1 31
Week 2 26
Week 3 23
Week 4 16
Total 96
Total submissions per week. Note: In week 2 a couple of participants submitted more than one solution and in weeks 3 and 4, I'm counting as valid submissions any valid message received. Many participants submitted more than one message.
Week Total submissions
Week 1 31
Week 2 28
Week 3 70
Week 4 98
Total 227
Last but certainly not least... a special mention to all the SAP Community members who completed the challenge every single week 🎉
@Alpesa1990
@MatLakaemper
@MioYasutake
@Nagarajan-K
@PriyankaChak
@Ruthiel
@ajos
@emiliocampo
@geek
@pamoli_banerjee
@r00k13d3v
@saitgunacorel
@spirit2681
@xavisanse

If you are interested in the event-driven integrations/architectures topic.... 📺 stay tuned as there is way more coming later this year in this space.

Week	Total submissions
Week 1	31
Week 2	26
Week 3	23
Week 4	16
*Total*	96