SAP Community - SAP HANA

Where can I acquire the skills to use monitor and performance tools within SAP HANA Cockpit?

2024-09-23T13:37:17.641000+02:00

I recommend to access our Using Monitoring and Performance Tools in SAP HANA learning journey

Overview
This SAP Learning Journey equips you with skills to use monitor and performance tools, troubleshoot performance issues, investigate SQL statements, address hardware performance issues, manage workload, and use capture and replay using SAP HANA cockpit. You will gain practical knowledge through hands-on exercises and real-world scenarios. By the end, you'll be proficient in investigating system performance within your organization. Get ready to embark on this exciting and comprehensive journey!

Learning objectives

Develop proficiency in using monitor and performance tools within SAP HANA Cockpit for effective system performance analysis.
Acquire the skills to troubleshoot performance issues, including investigating SQL statements and addressing hardware performance issues.
Gain expertise in workload management to ensure efficient system operation and resource utilization.
Learn how to utilize capture and replay functionalities to simulate and analyze system workloads, enabling informed decisions for performance improvements.

Prerequisites

Installing and Administering SAP HANA

Please ask a question related to the digital learning Journey in the Q&A area.

Our SAP Learning Experts will get back to you as soon as possible! We are here to support you.

I appreciate your feedback and we will make sure to continue sharing interesting topics.

Kind regards
Margit

Product Updates for SAP Build Code – September 2024 Edition

2024-09-26T14:49:39.442000+02:00

Dear SAP Community,

As we wrap up another exciting month, we wanted to take a moment to share some of the latest updates with you. With September coming to a close, we are getting ready for two major events in the SAP Community - Devtoberfest and TechEd. For more information, be sure to read the rest of this release information.

Support SAP S/4HANA Cloud Service Integration for Data Model and App Logic

You can use our generative AI assistant Joule in SAP Build Code to help you to write code quickly and easily by providing natural language prompts. As you know, we are constantly listening to your feedback to make our tools even better. One request we got was to enable AI support for external service scenarios. We are excited to tell you that it is now possible to use Joule to generate code for your integrated SAP S/4HANA Cloud services.

You start by creating your full-stack application, importing an external SAP S/4HANA Cloud service, and defining the data model and services. (With Joule, you can easily create your data model and services.)

Data Model and Service creation with the help of Joule

With our new feature, it is now possible to use Joule to enhance the data model you created before: You can add associations between the external entity and local entities, and you can expose the external entity to a local service via a projection.

Using Joule to add an association between the entity Books and the external entity A_BusinessPartner

We've also added support for custom logic generation to read the associated external SAP S/4HANA Cloud entity and to manipulate the exposed external entity.

Using Joule to add a custom logic reading data from the external entity A_BusinessPartner

SQL Notebooks for SAP HANA

SAP Build Code offers notebook functionality, similar to the well-known Jupyter notebooks. Since Q3 2024 the SAP HANA tools in SAP Build Code offer a notebook experience for executing SQL statements on an SAP HANA Cloud database.

To get started, create an SAP HANA dev space. Inside the dev space, create a file with the .notebook extension and open it. The file will be opened with the default notebook editor. Next, create a code block. The default code block type should be “SAP HANA Connection Management”. This code block type allows opening, closing, and selecting a database connection. Typing ‘#’ opens the value help with the available commands.

Choose the “List database connections” Command

Choose the “List database connections” command and execute the code block. A list of the available database connections from the SAP HANA Database Explorer view will be shown. You can choose a database connection to use for the notebook by clicking on its name in the list. The selected database will be highlighted with a different background color.

Select Database Connection

Now you can create another code block for executing SQL statements. Be sure to change the code block type to “SAP HANA SQL” by clicking on the “SAP HANA Connection Management” indicator in the bottom right corner and choosing “SAP HANA SQL” from the subsequent dialog.

Execute SQL Queries

Add one or more SQL statements and execute the code block. The results will be shown below the code block.

Add SQL Statements and Execute the Code Block

You can add more code and markdown blocks as needed.

Use the Joule functionality to enhance your MDK projects in SAP Build Code

Remember back in June when we announced you could create and manage your mobile development kit (MDK) apps directly from the SAP Build lobby and continue their development using the Storyboard in SAP Build Code? We're thrilled to share another exciting update on that front!

You can now take advantage of the generative AI assistant Joule to enhance your existing MDK projects in SAP Build Code. For further details, I recommend to read this blogpost written by my colleague Jitendra.

Devtoberfest and TechEd

Get ready for an exciting journey of learning and collaboration with Devtoberfest and SAP TechEd Virtual!

From September 23rd to October 25th, 2024, join us for Devtoberfest, a celebration of coding and collaboration within the SAP Community. This virtual event is designed to prepare you for the highly anticipated SAP TechEd. My colleague Ravi summarizes all you need to know about SAP Build Code at Devtoberfest. For further information, read the Devtoberfest 2024 Welcome blog post and join the Devtoberfest group in SAP Community to take part in this great event!

And don't forget to mark your calendar for SAP TechEd Virtual on October 8-9, 2024, your gateway to the latest innovations and advancements in SAP technologies! The session catalog is now live, featuring a wide range of topics across various tracks, including the Application Development and Automation and the Data and Analytics tracks, where you'll find dedicated sessions on SAP Build Code. You are looking for a summary where to find SAP Build Code at SAP TechEd? Then I recommend to read this blog post.

Don't miss these opportunities to enhance your technical expertise and connect with the SAP community.

Let Us Hear Your Voice!

We prepared a short survey, focused on improving the product to better fit your needs. By participating, you can help us improve SAP Build Code and SAP Business Application Studio. It should only take 5 minutes to complete. Please take the survey now.

For further information on new features, please check our roadmaps: roadmap for SAP Build Code, SAPUI5 Road Map. And have a look at What's New in SAP Build Code, the What's New Viewer for SAPUI5 and the SAP Cloud Application Programming Model Release Notes. If you want to stay up-to-date with news, learning resources, and product and strategy updates for SAP Build Code: follow our tag.

We look forward to seeing how you use these new capabilities to create innovative solutions and drive your projects forward. Stay tuned for more updates and happy coding! 😊

Guide to use HANA NSE in the BW/4HANA system

2024-09-30T10:39:42.626000+02:00

It is critical for the HANA systems not to overload the memory, because loads/unloads are affecting the performance drastically.

We are constantly working on reducing the memory footprint to improve the performance and avoid buying additional memory since that could be complicated by various reasons like hardware costs, unavailability of particular memory planks in the market, a necessity to reconfigure the current landscape etc.

So, in addition to archiving, we decided also to implement a HANA NSE.

Native storage extension is quite a flexible tool to reduce the data memory footprint, which is not so complicated to set up.

NSE makes much sense e.g. in the SAP BW/4HANA environment, since you usually have quite a lot of data, plus it supports a scale-out scenario, the only thing you need to know, is that HANA does not consider the NSE table for the redistributions, so you need to take care of them manually if needed.

You can put to the NSE tables, table Indexes, table partitions, or single table columns, they will be kept in the storage, and only the required pages will be loaded, not the whole table/column.

This has an effect also on the storage sizing, the page loadable objects require more storage space and also will be a bit slower for the querying, so this should be taken into account

There could be several ways to identify it, there is a recommended way from SAP to run the NSE Advisor, but as I know some customers do not want to create additional load on HANA with the statistics collection, so they do it with the commonsense analysis, e.g. checking the EWA for the biggest tables which is not accessed too much, or put to the NSE the data which is not needed to be queried/reloaded often, e.g. the data for the previous years in the reporting which needs

In our experience, we did not find a significant increase in the HANA load with the Advisor activated, so we decided to go with the NSE advisor

You cannot run NSE load unit conversion DDLs on the following types of SAP HANA system tables:

Tables that were created on SAP HANA versions 2.0 SPS 00 or earlier, and never rewritten or merged into later/higher SAP HANA versions.

In our scenario we used HANA 2.0 SP5, so we followed the note 2927591, but for the different SPs you need to follow the different notes

So first of all, you need to convert the tables into the HANA 2.0 format, in our case there were ~1000 tables with the old format, and the conversion took around 2 hours. There were no problems in the normal BW/HANA operations, only many alerts appeared during the conversions of these tables, but you can ignore them. In general, you can test this process in the quality system to ensure there are no problems during the conversions

Before that, you need to find all of them, by running the following select:

select * from M_CS_COLUMNS_PERSISTENCE where PERSISTENCE_TYPE = 'VIRTUAL_FILE' or PERSISTENCE_TYPE = 'VIRTUAL_PAGED'

It also makes sense to run the delta merge for the tables to be converted:

select count(*) from <table_name>

merge delta of <table_name> force rebuild

Download the stored procedure for your SP from the relevant SAP note, e.g. for HANA 2.0 SP7 you would need SAP Note 3320095

After you create the procedure in HANA, you need to execute the following:

CALL PREPARE_PAGE_LOADABLE_PERSISTENCE('<schema_name>', ?)

After the procedure successfully executed you need to start the NSE advisor, and let it run for quite some time, depends on your scenario. E.g. if you have only the monthly reports you cannot do any analysis before the whole month passed, the data would not be relevant

Start NSE_ADVISOR:

ALTER SYSTEM ALTER CONFIGURATION ('indexserver.ini','system') SET ('cs_access_statistics','collection_enabled') = 'true' WITH RECONFIGURE;

After you have collected all the necessary statistics you can disable it:

ALTER SYSTEM ALTER CONFIGURATION ('indexserver.ini','system') SET

('cs_access_statistics','collection_enabled') = 'false'

WITH RECONFIGURE;

In our scenario we did not disable that, so it keeps running, and we have new inputs every 2-3 months for the objects which we could put into NSE

The $trexexternalkey$ values stand for the indexes, which HANA recommend to put into the NSE, since they are not being accessible too often.

Putting the data into NSE:

You can run it for the table:

alter table <schema_name>.<table_name>_BOOKM page loadable

for the column:

alter table <schema_name>."/BI0/SCO_ITEM_NO" alter ("SID" ALTER PAGE LOADABLE)

for putting the index of the table to NSE you need at first finding the name of it

SELECT SCHEMA_NAME, TABLE_NAME, INDEX_NAME, INDEX_TYPE, CONSTRAINT FROM INDEXES WHERE TABLE_NAME = <table_name>;

After finding the index name:

ALTER INDEX <schema_name>."_SYS_TREE_CS_#1816272_#0_#P0" PAGE LOADABLE;

to get it back to the default behavior you need to run the following command:

ALTER TABLE "<table_name>" DEFAULT LOADABLE CASCADE;

ALTER INDEX <schema_name>."_SYS_TREE_CS_#1816272_#0_#P0" default LOADABLE;

To check the tables in the NSE you can run the following command:

select * from tables where load_unit = 'PAGE'

To check the indexes in the NSE you can run the following command:

select * from indexes where load_unit = 'PAGE'

To check the columns in the NSE you can run the following command:

select * from m_cs_all_columns where load_unit = 'PAGE'

It's also important to know, how much memory you saves by putting the data into the NSE, and here are 2 ways of doing it

Usually, in the NSE Advisor, you see the amount of memory you will save putting the table to the NSE, with summing up the memory size in the main memory columns

select * from Sys.M_CS_NSE_ADVISOR

if you are not using advisor, you can run the select * from m_cs_all_columns where table_name = 'RSBATCHDATA' and check memory_size_in_total, for e.g. a column or index which you want to put into NSE

In our system, we saved around 2 TB of memory, which helped a lot in improving the HANA performance and satisfaction of our users.

Where can I learn configuring the Python machine learning client for SAP HANA (hana-ml)?

2024-09-30T15:16:52.971000+02:00

I recommend to access our Developing Regression Models with the Python Machine Learning Client for SAP HANA learning journey.

Overview
In this enablement you will apply the machine learning workflow using the Python machine learning client for SAP HANA, also known as hana-ml. You will look at the steps from feature extraction to model evaluation, using the California housing dataset as scenario.

Learning objectives

Configuring the Python machine learning client for SAP HANA (hana-ml) and connecting to SAP HANA Cloud
Accessing and visually exploring SAP HANA data using HANA dataframes via Python
Training a regression model for the California housing dataset using hana-ml Predictive Analysis Library (PAL) functions
Performing Model Evaluation to show the model accuracy obtained

Prerequisites

Basic Structured Query Language (SQL) knowledge is highly desirable.
If you need some foundational information, you can also refer to:
Developing AI Models with the Python Machine Learning Client for SAP HANA

Please ask a question related to the digital learning Journey in the Q&A area.

Our SAP Learning Experts will get back to you as soon as possible! We are here to support you.

I appreciate your feedback and we will make sure to continue sharing interesting topics.

Kind regards
Margit

Partition of ACR Table (DRC-Statutory Reporting) in SAP S/4HANA 2021 (On Premise)

2024-10-04T15:19:01.941000+02:00

Title: Partition of ACR Table (DRC-Statutory Reporting) in SAP S/4HANA 2021 (On-Premise)

Introduction:
Hi Friends!

This is my second blog post and aims to explain the concept of partitioning the ACR table for DRC (Statutory Reporting) in SAP S/4HANA 2021, focusing on its significance, how it differs from other partitioning approaches, and potential issues during implementation. I will divide the discussion into three key areas: like what is partition and How its different partition from other, implementation and other issues.

In SAP S/4HANA, partitioning refers to the process of splitting large database tables into smaller, more manageable pieces, called partitions. This enhances database performance, particularly for large tables, by distributing the data across different sections and improving query efficiency, load balancing, and parallel processing.

There are two main types of partitioning in SAP HANA:
1. Row Store Partitioning:
• Row-based partitioning is less common in S/4HANA because SAP HANA primarily operates as a columnar database. Row-based partitioning might still be used in specific cases where the nature of the data requires it.
2. Column Store Partitioning:
• This is the most common type in SAP HANA, as it aligns with its in-memory, columnar design. Column store tables are split into partitions based on certain criteria, typically by ranges of values or hash-based partitioning.
• Types of Column Store Partitioning:
o Range Partitioning: The table is split based on value ranges of a particular column (e.g., date or ID ranges).
o Hash Partitioning: Rows are distributed across partitions using a hash function, spreading the data evenly.
o Round Robin Partitioning: Data is distributed in a cyclic manner across partitions

Why do we need to partition tables in HANA ?
As HANA can not hold more than 2 billion records per column store table or per partition , any table which has crossed more than 1.5 billion rows are valid candidates of partitioning . It also helps in parallel procession of queries , if they are distributed across nodes in case of HANA scale out scenario producing results faster .

Enable partitioning:
In this section, I’ll explain how to enable partitioning for the SRF_REP_RUN_DATA table, which is a key component in SAP's Advanced Compliance Reporting (ACR) and Statutory Reporting. Partitioning is essential for managing large datasets, ensuring system performance, and preventing table overflow.
1. Enabling Partitioning through TCI Note 3354120
The partitioning for the SRF_REP_RUN_DATA table can be activated after implementing the TCI (Transport-based Correction Instructions) note 3354120. This note consolidates fixes and improvements for Statutory Reporting and introduces partitioning capabilities to manage large data volumes more effectively.
2. Steps to Enable Partitioning
After implementing the TCI note, follow these steps to activate partitioning for the SRF_REP_RUN_DATA table:
a. Implement TCI Note 3354120
Ensure that your system has applied TCI 3354120, which includes various improvements for Statutory Reporting, including partitioning support. This is a consolidated fix and comes with the necessary tools for enabling partitioning.
b. Execute Program SRF_ACTIVATE_TABLE_PARTITIONS
Once the TCI is implemented, the system introduces a new program: SRF_ACTIVATE_TABLE_PARTITIONS. You can run this program to enable partitioning for the SRF_REP_RUN_DATA table.
o Navigate to the transaction code SE38 and execute the program SRF_ACTIVATE_TABLE_PARTITIONS.
o This will partition the table, optimizing data handling, and enhancing system performance, especially for large datasets.
c. Post-Execution Check
After running the program, you can verify whether the partitioning has been successfully applied by checking the table's structure in SE14 or DBACOCKPIT.

Before Partition :

After Partition:

we have enabled partition through program "SRF_ACTIVATE_TABLE_PARTITIONS " in the test systems.

1. We observed that running this program has automatically partitioned table "SRF_REP_RUN_DATA". Is this program applicable only for this table or is there any impact on any other DRC table? Yes only SRF_REP_RUN_DATA

2. Do we need to follow any specific steps to run this program in production? e.g. Down time etc.---- No. we can do as its like transport movement

Issues :
In this section will explain the issue after post implementation
Issue 1 :

Post implementation of the TCI 3354120 , we observed that all the logs of the prior runs are not visible . Even the logs are not showing for the new runs
Before TCI Note : 3354120 :

After TCI note : Logs lost for all previous runs and New Runs

As per SAP suggestion kindly implement the note 3367884 clear the cache as mentioned in the note 2319491 then logs appeared (Immediately will not appear wait for some time)

Issue 2: Record Count is same, but entries are increasing

so total entries in table increased but count of entries in partition is same.

There are 2 partitions seen in HANA. But all records are in Partition 1 . Please clarify after report run for partition , are there any further post actions. And, when will records go into Partition 2 with zero records ?

For SRF_REP_RUN_DATA table I can see that in SE16N there are 1,707,345,316 and at the same time in DB02/DBACOCKPIT transaction I can see below 5 partitions

Checking it from application layer would be ideal as it abstracts all the backend HANA computations which will always be in a flux due to data being written to it. You could try to refresh the screen in HANA and check, but the still recommend checking the data count from application layer SE16N/DB02/DBACOCKPIT.
Tcode : DB02

Select Diagnostics and Table

Click on execute

Is SAP automatically creating these sub-partitions?
SAP: Yes, SAP automatically creates the sub-partition.system will do automatically partition once threshold limit of 50 percentage

Issue 3 : We have observed one thing once successfully at the time of refresh of system . we got an error “ EC0x006: Table partitioning not enabled on SRF_REP_RUN_DATA “. To resolve this issue to we added manually table entries mentioned in the below image to the table SRF_PARTITION

Conclusion:

This blog post addressed Partition of ACR table (SRF_REP_RUN_DATA) and some known issues with their solutions. GenAI was used to help generate this content. Stay Tuned!.

Thank you for going through the post. Your feedback and comments will help me make it more relevant to the community.

First Blog : https://community.sap.com/t5/enterprise-resource-planning-blogs-by-members/predictive-accounting-in-sap-s-4hana-1909/ba-p/13507148

Regards,

Durga Sai Ram

SIOS LifeKeeper for High Availability of SAP ASCS /ERS HA and HANA Database on Azure

2024-10-07T09:12:26.327000+02:00

Introduction

SAP HANA is a high-performance in-memory database that supports real-time analytics and transactions. It is widely used by enterprises to run their critical business applications and processes. However, SAP HANA systems also require high availability and disaster recovery solutions to ensure business continuity and data protection in case of failures or disasters.

Azure is a cloud platform that offers a range of services and features to host, manage, and scale SAP HANA systems. Azure provides high-performance virtual machines, storage, networking, and security options that can meet the requirements of SAP HANA workloads. However, Azure does not offer a native high availability solution for SAP HANA systems, which means that customers need to use third-party tools or custom scripts to achieve high availability for their SAP HANA databases on Azure.

SIOS LifeKeeper is a proven high availability and disaster recovery solution that can protect SAP HANA systems on Azure. SIOS LifeKeeper can monitor, detect, and recover from failures of SAP HANA instances, virtual machines, storage, network, or Azure services. SIOS LifeKeeper can also replicate data between primary and secondary SAP HANA systems, and perform automatic or manual failover and failback operations. SIOS LifeKeeper can integrate with Azure services such as Azure Load Balancer, Azure Traffic Manager, and Azure Site Recovery to provide a comprehensive and reliable high availability solution for SAP HANA systems on Azure.

Prerequisites

Before you can configure and deploy a highly available SAP HANA system on Azure using SIOS LifeKeeper, you need to have the following prerequisites:

An Azure subscription and an Azure account with sufficient permissions to create and manage resources.
A SAP HANA license and installation media.
A SIOS LifeKeeper license and installation media.
Two or more Azure virtual machines that meet the SAP HANA hardware and software requirements. You can use the Azure Marketplace to create SAP HANA certified virtual machines.
A shared storage solution that supports SAP HANA data and log volumes. You can use Azure NetApp Files, Azure Files, or Azure Shared Disks as shared storage options.
A network solution that allows communication between the SAP HANA systems and the SIOS LifeKeeper nodes. You can use Azure Virtual Network, Azure VPN Gateway, or Azure ExpressRoute as network options.
A load balancer solution that can distribute traffic to the SAP HANA systems and the SIOS LifeKeeper nodes. You can use Azure Load Balancer, Azure Application Gateway, or Azure Traffic Manager as load balancer options.

Configuration Steps

After you have prepared the prerequisites, you can follow these steps to configure and deploy a highly available SAP HANA system on Azure using SIOS LifeKeeper:

Install SAP HANA on the primary virtual machine and configure the SAP HANA data and log volumes on the shared storage.
Install SAP ASCS on the primary virtual machine and SAP ERS on Secondary Virtual Machine and /sapmnt on the shared storage.
Install SIOS LifeKeeper on the primary virtual machine and join it to a SIOS LifeKeeper cluster.
Create a SIOS LifeKeeper SAP HANA resource hierarchy on the primary virtual machine and configure the SAP HANA instance details and the shared storage paths.
Install SAP HANA on the secondary virtual machine and configure the SAP HANA data and log volumes on the shared storage.
Install SIOS LifeKeeper on the secondary virtual machine and join it to the same SIOS LifeKeeper cluster as the primary virtual machine.
Extend the SIOS LifeKeeper SAP HANA resource hierarchy from the primary virtual machine to the secondary virtual machine and configure the replication mode and the failover policies.
Configure the load balancer to direct traffic to the SAP HANA systems and the SIOS LifeKeeper nodes.
Test the high availability and disaster recovery scenarios by simulating failures and verifying the failover and failback operations.

Architecture Design

SIOSLV-RG

IP address Assignmnet

Component	West US 2	IP-ADDRESS
VNET	vnet-sioslv-westus2	10.3.0.0/16
Snet-app-sioswestus2	snet-app-siosz1-westus2	10.3.0.0/24
Snet-hdb-sios-westus2	snet-app-siosz2-westus2	10.3.1.0/24
s4hdiagsios
Azure File share (NFS 4.1)	s4hsiosnfstest
AzureFirewallSubnet	AzureFirewallSubnet
AzureBastionSubnet	AzureBastionSubnet	10.3.2.0/26
Azure File share nfs4.1	s4hsiosfsdiag
ASCS Cluster Node1	vmslvascs1	10.3.0.4
ASCS Cluster Node2	vmslvascs2	10.3.0.5
Witnesss server for SAP in Zone3	vmslvwit1	10.3.0.6
SAP PAS	vmslvpas	10.3.0.7
VIP-SAP DB	S4sascs s4sers	10.3.0.20 10.3.0.21
VIP-HANA DB	sioshdb-lb	10.3.1.10
HANA DB Cluster Node 1	vmslvhdb1	10.3.1.4
HANA DB Cluster Node 2	vmslvhdb2	10.3.1.5

Created sioslv-rg

Created VNET/SN

SUSE SLES for SAP Applications 15 SP5 with 24x7

Installation of HANA DB on node1 and node2

Please perform the same for Node2

Download the software

Download lkwmc

./setup

Install licenskey

We don’t need to select Likekeeper startup

Setup failed due to

Mistped the location of licensekey

Now perform the same step on VM2

vmslvhdb2:/misc/SIOS-9.8.1/siossoft981 # mount /misc/SIOS-9.8.1/sps.img /misc/SIOS-9.8.1/siossoft981 -t iso9660 -o loop

vmslvhdb2:/misc/SIOS-9.8.1/siossoft981 # suseconnect -p sle-module-legacy/15.5/x86_64

Registering system to registration proxy https://smt-azure.susecloud.net

Updating system details on https://smt-azure.susecloud.net ...

Activating sle-module-legacy 15.5 x86_64 ...

-> Adding service to system ...

-> Installing release package ...

Successfully registered system

Install LKWMC

WITNESS SERVER

Pre-requisite :

vi /etc/default/LifeKeeper

NOBCASTPING=1

Now browse the LKWMC via link below for

vmslvhdb1:/misc/SIOS-9.8.1/lkwmc/sios-lkwmc-1.1.0 # /opt/LifeKeeper/bin/lcdstatus -q

LifeKeeper does not appear to be running.

vmslvhdb1:/misc/SIOS-9.8.1/lkwmc/sios-lkwmc-1.1.0 # ./lkstart

-bash: ./lkstart: No such file or directory

vmslvhdb1:/misc/SIOS-9.8.1/lkwmc/sios-lkwmc-1.1.0 # /opt/LifeKeeper/bin/lkstart

Created symlink /etc/systemd/system/lifekeeper-graphical.target.requires/lifekeeper.service → /usr/lib/systemd/system/lifekeeper.service.

Created symlink /etc/systemd/system/lifekeeper-multi-user.target.requires/lifekeeper.service → /usr/lib/systemd/system/lifekeeper.service.

vmslvhdb1:/misc/SIOS-9.8.1/lkwmc/sios-lkwmc-1.1.0 #

Broadcast message from systemd-journald@vmslvhdb1 (Thu 2024-04-11 19:00:22 UTC):

lcdinit[5788]: EMERG:lcd.lcdchkseml:::011138:The LifeKeeper product on this system is using an evaluation license key which will expire at midnight on 05/11/24. To continue functioning beyond that time, a permanent license key must be obtained.

vmslvhdb1:/misc/SIOS-9.8.1/lkwmc/sios-lkwmc-1.1.0 # /opt/LifeKeeper/bin/lcdstatus -q

LOCAL TAG ID STATE PRIO PRIMARY

vmslvhdb1:/misc/SIOS-9.8.1/lkwmc/sios-lkwmc-1.1.0 #

Now lauch the LKWCM again

To connect to the LifeKeeper Web Management Console from a web browser:

https://vmslvhdb1:5110

Create communication between all systems

Click add new server to add vmslvhdb2

Click create another communication path to Witness server

Create communication between vmslvhdb2 to vmslvwit1

Click on Resource tree for creating the resource IP

Click extend

Click to create the resource for – LB health checkup

click Extend

Before creating the below resource enable HSR

Take Backup of SYSTEMDB and TENANT DB on both nodes

Copy the below ssfs files to secondary HDB

s4sadm@vmslvhdb1:/usr/sap/S4S/SYS/global/security/rsecssfs> ls

data key

s4sadm@vmslvhdb1:/usr/sap/S4S/SYS/global/security/rsecssfs> scp -rp /usr/sap/S4S/SYS/global/security/rsecssfs/* s4sadm@vmslvhdb2:/usr/sap/S4S/SYS/global/security/rsecssfs/

s4sadm@vmslvhdb2's password:

SSFS_S4S.DAT 100% 2960 855.0KB/s 00:00

SSFS_S4S.KEY 100% 187 127.9KB/s 00:00

Now lets create the HANA resource – lets start the HANA system on both system

Now enable the replication between them

s4sadm@vmslvhdb1:/usr/sap/S4S/home> hdbnsutil -sr_enable --name=s4sz1

nameserver is active, proceeding ...

successfully enabled system as system replication source site

done.

s4sadm@vmslvhdb1:/usr/sap/S4S/home>

hdbnsutil -sr_register --remoteHost=vmslvhdb1 --remoteInstance=00 --replicationMode=syncmem --operationMode=logreplay --name=s4sz2 –online

Creating SAP HANA resource

Click Close

Failover test on HANA DB.

Trying the takeover by rightclick on standby – in sync resource

Current status

SIOS cluster will detect and start it and reregister it as secondary

Failback

Now lets work on SAP ASCS / ERS

NFS shares

/sapmnt

/ASCS00

/ERS01

/SYS

Local

/usr/sap

Goto SAP system and mount them

sudo mkdir -p /usr/sap/S4S/ascs00

sudo mount -t nfs siossapnfs.file.core.windows.net:/siossapnfs/ascs00 /usr/sap/S4S/ascs00 -o vers=4,minorversion=1,sec=sys,nconnect=4

sudo mkdir -p /usr/sap/S4S/ers01

sudo mount -t nfs siossapnfs.file.core.windows.net:/siossapnfs/ers01 /usr/sap/S4S/ers01 -o vers=4,minorversion=1,sec=sys,nconnect=4

sudo mkdir -p /sapmnt/S4S

sudo mount -t nfs siossapnfs.file.core.windows.net:/siossapnfs/sapmnt /sapmnt -o vers=4,minorversion=1,sec=sys,nconnect=4

sudo mkdir -p /usr/sap/S4S/sys

sudo mount -t nfs siossapnfs.file.core.windows.net:/siossapnfs/sys / -o vers=4,minorversion=1,sec=sys,nconnect=4

Lets mount and install SIOS

Pre-requisite

vmslvascs1:~ # suseconnect -p sle-module-legacy/15.5/x86_64

Registering system to registration proxy https://smt-azure.susecloud.net

Updating system details on https://smt-azure.susecloud.net ...

Activating sle-module-legacy 15.5 x86_64 ...

-> Adding service to system ...

-> Installing release package ...

Successfully registered system

vmslvascs1:/misc/SIOS-9.8.1 # ls

lkwmc siossoft981 sps.img

vmslvascs1:/misc/SIOS-9.8.1/siossoft981 # mount /misc/SIOS-9.8.1/sps.img /misc/SIOS-9.8.1/siossoft981 -t iso9660 -o loop

mount: /misc/SIOS-9.8.1/siossoft981: WARNING: source write-protected, mounted read-only.

vmslvascs1:/misc/SIOS-9.8.1/siossoft981 # ./setup

Lets install LKGUI

To connect to the LifeKeeper Web Management Console from a web browser:

https://vmslvascs1:5110

vmslvascs1:/misc/SIOS-9.8.1/lkwmc/sios-lkwmc-1.1.0 # /opt/LifeKeeper/bin/lcdstatus -q

LOCAL TAG ID STATE PRIO PRIMARY

Now perform the same vmslvascs2

Lets perform LKGUI

To connect to the LifeKeeper Web Management Console from a web browser:

https://vmslvascs2:5110

Now lets launch the LK wmc and setup communication between resources and create LB

Click communication path and under operations – create communication path

Click ADD A NEW SERVER

It will ask for login info of vmslvascs2

Now create another communication between

Vmslvascs1àvmslvascs2 -- Done

Vmslvascs1àvmslvwit1

Vmslvascs2 à vmslvwit1

Now lest create the resources

LB Health Checkup

SAP

10.3.0.20

Click Extend

Click Close

Lets add ERS IP

Click Close

Now lets add LB Health Check for SAPASCS / SAPERS

Now perform for vmslvascs2 from vmslvascs1

Now install SAP ASCS

Ping virtual ips

10.3.0.20 s4sascs

10.3.0.21 s4sers

vmslvascs1:/misc/s4hana2023-includesall/SWPM # ping -a 10.3.0.20

PING 10.3.0.20 (10.3.0.20) 56(84) bytes of data.

64 bytes from 10.3.0.20: icmp_seq=1 ttl=64 time=0.034 ms

64 bytes from 10.3.0.20: icmp_seq=2 ttl=64 time=0.033 ms

./sapinst SAPINST_USE_HOSTNAME=s4sascs

ERS on node2

vmslvascs2:/misc/s4hana2023-includesall/SWPM # ./sapinst SAPINST_USE_HOSTNAME=s4sers

[==============================] | extracting... done!

INFO: Native locale set to C.UTF-8

INFO 2024-04-22 21:44:45.283 (mainThread) [sixxcreate.cpp:349]

********************************************************************************

Initial log directory: /root/.sapinst/vmslvascs2/13944

********************************************************************************

WARNING 2024-04-22 21:44:45.284 (root/root) (startInstallation) [syxxcnwmgt.cpp:61] id=syslib.network.overriddenHostName void CSyNetworkMgtImpl::overrideHostName(const iastring & "

https://vmslvascs2:4237/sapinst/docs/index.html

/usr/sap/S4S/SYS

Installing PAS

vmslvpas:/usr/sap/s4hana2023-includesall/SWPM # ./sapinst

Now install PAS on same server

Creating a HA cluster for your SAP ASCS/ERS servers in AWS - Google Docs

Copy the sapservices file between node1 and node2

vmslvascs2:/opt/LifeKeeper/bin # more /usr/sap/sapservices

systemctl --no-ask-password start SAPS4S_01 # sapstartsrv pf=/usr/sap/S4S/SYS/profile/S4S_ERS01_s4sers

systemctl --no-ask-password start SAPS4S_00 # sapstartsrv pf=/usr/sap/S4S/SYS/profile/S4S_ASCS00_s4sascs

Also perform below - Verify that the rules for both your ASCS and ERS instances are in the /etc/polkit-1/rules.d/ directory. If not copy the ERS file to the ASCS server and vice versa:

Make sure you copy the files between system

/etc/systemd/system/

SAPS4S_00.service

SAPS4S_00.service.d

SAPS4S_01.service

SAPS4S_01.service.d

Lets create the SAP resource for ASCS/ERS on SIOS

Now create ERS resource

Create dependency

SAP as parent resource

Child as VIP

Child as LBHC

Confirm

===Now for SAP ERS resource

Lets delete the old license

vmslvascs2:/opt/LifeKeeper/bin # cd /var/LifeKeeper/license/

vmslvascs2:/var/LifeKeeper/license # ls

20240422.lic 20240429.lic

vmslvascs2:/var/LifeKeeper/license # ls -ltr

total 16

-rw-r--r-- 1 root root 6121 Apr 22 15:19 20240422.lic

-rw-r--r-- 1 root root 6121 Apr 29 14:02 20240429.lic

vmslvascs2:/var/LifeKeeper/license # rm 20240422.lic

Lest crash the node1

vmslvascs1:/etc/systemd/system # echo c > /proc/sysrq-trigger

Node1 is hanging

ASCS moved to Node2 and the locks are still there and SAP is able to connect with out any issue.

After few seconds the VM1 did not come up and we see the ERS is failing over to Node1

Restarted the VM manually –once the VM is up and running

SIOS will automatically move the ERS to another node , I see the status as below

Manually switchover the ASCS to Node1

Rightclick and make it as in service

Done Tests are successful.

Unlocking the True Potential of Data in Files with SAP HANA Database SQL on Files in SAP HANA Cloud

2024-10-08T09:30:00.027000+02:00

In today's data-driven world, enterprises collect more data than ever before. Unlocking the potential hidden insights within this data, while seamlessly connecting the dots among both SAP and non-SAP applications, is not only essential for adapting to an ever-changing business environment but also crucial for differentiating your business from competitors.

Imagine having a treasure trove of business data stored in common formats like CSV, Parquet, and Delta table (Open Table Format). Each holds a vast volume of valuable information. Starting with SAP HANA Cloud QRC 03/2024, I'm thrilled to announce that SAP HANA Database offers native SQL on Files capabilities. This unlocks these chests with powerful analytical query processing capabilities, like what has been done with SAP HANA Database before. This enhancement makes your structured data in files instantly accessible and incredibly powerful while leveraging all the existing capabilities of SAP HANA Cloud.

This new native SQL on Files capability allows you to directly access and read data in SAP HANA Cloud, data lake Files, especially in CSV, Parquet, and Delta table (Open Table Format), without moving or loading them into SAP HANA Database. This will also enable better control of the cost-performance balance for the workload, with increased flexibility. In a nutshell, this new capability includes the ability to:

Link your SAP HANA Cloud, SAP HANA Database to SAP HANA Cloud, data lake Files with one click in SAP HANA Cloud Central (only available in multi-environment edition).
Access data in files stored in SAP HANA Cloud, data lake Files on-the-fly by creating read-only virtual tables that point to the files or directory, and then executing SQL queries.
Benefit from support for file formats such as CSV, Parquet, and Delta table (Delta table is an Open Table Format table based on Parquet files). Since Delta table provides table semantics, it is the preferred choice for query processing.
Execute time travel queries on Delta table by specifying the version or timestamp in the SELECT statement.

Prerequisites and considerations

Please ensure that all the following prerequisites are met before following the steps in this blog.

Both SAP HANA Database and data lake Files are provisioned and running in SAP HANA Cloud. Please refer to Provisioning a SAP HANA Cloud, SAP HANA Database and Adding a Standalone Data Lake to SAP HANA Cloud (creating a data lake instance with the data lake Relational Engine disabled will provision a file-based data lake).
SAP HANA Database and data lake Files should be in the same subaccount.
The version of the SAP HANA Database in SAP HANA Cloud is QRC 03/2024 or later.
Files are already stored in the data lake Files using hdlfscli. If not, please refer to the Getting Started with Data Lake Files HDLFSCLI guide and upload the files.

When uploading files, if both Parquet and Delta Lake formats are used, it is highly recommended to separate directories because both formats basically use the same file extension (.parquet), while Delta Lake has an additional _delta_log folder for the transaction log.

Getting started with SAP HANA Database Native SQL on Files in SAP HANA Cloud

If the above conditions are met and the files are ready in data lake Files, let's configure the connection between the SAP HANA Database and the data lake Files with a simple one-click process in SAP HANA Cloud Central.

Link your SAP HANA Cloud, SAP HANA Database to SAP HANA Cloud, data lake Files:

If you have provisioned the SAP HANA Database and data lake Files in SAP HANA Cloud, you can see both in SAP HANA Cloud Central. To use SQL on Files, a secure connection between the SAP HANA Database and data lake Files should be created. For this, if you click on your SAP HANA Database in SAP HANA Cloud, you will now see a new tab named Linked Data Lake Files, which lists all the Data Lake Files instances in the same subaccount, as shown below. Please note that this feature is only available in the multi-environment edition of SAP HANA Cloud Central. To use the multi-environment edition, please follow the instruction in Subscribing to the SAP HANA Cloud Administration Tools.

In this screen, all you need to do is simply click Link, and it will handle everything, including certificates, trust, authorization, and remote source creation.

With this one click, as you can see in the example below, my demo-hdb is linked to demo-hdlf via an auto-generated remote source called demo-hdlf_rs.

You can also unlink and delete the auto-generated remote source by simply clicking Unlink. However, it will drop the remote source and all objects such as virtual tables that depend on it in the SAP HANA Database. Even though this action will not change anything in your data lake Files, careful operation is required as it cannot be undone. And if you link data lake Files via SAP HANA Cloud Central, please use unlink and do not drop the auto-generated remote source manually.

Please also refer to the links below for further details.

SAP HANA Cloud, SAP HANA Database SQL on Files Guide: Create Links to Data Lake Files Instances

Alternatively, the simple one-click process described above can also be achieved through SQLs and manual configurations. However, since using the one-click SAP HANA Cloud Central way is the recommended method, I will not address this alternative method in this blog. If you're interested in this alternative method, please refer to the links below.

SAP HANA Cloud, SAP HANA Database SQL on Files Guide: Set Up Certificates for Linking to Data Lake Files
SAP HANA Cloud, SAP HANA Database SQL on Files Guide: Create an SQL on Files Remote Source

Create virtual tables:

Once you link the SAP HANA Database to data lake Files, virtual tables can be created by pointing to files stored in the data lake Files. All examples in this blog are based on TLC Trip Record Data.

Let's start with a simple CSV file. To create a virtual table by pointing to a CSV file, you can use the CREATE VIRTUAL TABLE statement. In this case, column names and data types should be defined as in the example below.

-- create a virtual table by pointing to a CSV file stored in data lake Files
CREATE VIRTUAL TABLE DEMO.TAXI_ZONE (
    "LocationID" INT PRIMARY KEY,
    "Borough" NVARCHAR(20),
    "Zone" NVARCHAR(50),
    "service_zone" NVARCHAR(20)
    ) AT "demo-hdlf_rs"."/csv/"
  AS CSV;

In this case, you should make sure that the CSV file does not have column headers, and the defined data type should be large enough to read the data. Otherwise, you will see the "too large value for column" error when executing a query on the virtual table, even though the virtual table is successfully created.

Please also find the example result in the screenshot below.

In SQL on Files, you can also specify the file name (e.g., /csv/taxi_zone_lookup.csv) instead of just the directory (/csv/). Using the directory allows for the automatic detection of any additionally uploaded files, while using the specific file name will not auto-detect any additionally uploaded files.

Now, let’s exclude the second column from the virtual table. Please see the below example.

-- create a virtual table by pointing to a CSV file without second column
CREATE VIRTUAL TABLE DEMO.TAXI_ZONE_NO_BOROUGH (
    "LocationID" INT PRIMARY KEY,
    "Zone" NVARCHAR(50),
    "service_zone" NVARCHAR(20)
    ) AT "demo-hdlf_rs"."/csv/"
  AS CSV FILE COLUMN LIST(1, 3, 4);

Please also find the example result in the screenshot below.

Here, as you can see, the second column Borough is excluded by using FILE COLUMN LIST(1, 3, 4), which means that only the first, third, and fourth columns are referred to when creating a virtual table.

Alright, now let’s see a Parquet example. For Parquet and Delta table, you can use GET_REMOTE_SOURCE_FILE COLUMNS built-in procedure to retrieve column information. Please note that this built-in procedure cannot be used for CSV files.

-- retrieve column information (parquet, delta table only)
CALL GET_REMOTE_SOURCE_FILE_COLUMNS ('demo-hdlf_rs', '/parquet/', 'PARQUET');

Please also find the example result in the screenshot below.

As you can see, you can find the recommended data types for each column in the SQL_DATA_TYPE. For more information, please refer to the link below.

SAP HANA Cloud, SAP HANA Database SQL on Files Guide: GET_REMOTE_SOURCE_FILE_COLUMNS

With this information, I can easily write a CREATE VIRTUAL TABLE statement like the one below.

-- create a virtual table by pointing to Parquet files
CREATE VIRTUAL TABLE DEMO.YELLOW_TRIP (
    "VendorID" INT,
    "tpep_pickup_datetime" TIMESTAMP,
    "tpep_dropoff_datetime" TIMESTAMP,
    "passenger_count" BIGINT,
    "trip_distance" DOUBLE,
    "RatecodeID" BIGINT,
    "store_and_fwd_flag" NVARCHAR(1),
    "PULocationID" INT,
    "DOLocationID" INT,
    "payment_type" BIGINT,
    "fare_amount" DOUBLE,
    "extra" DOUBLE,
    "mta_tax" DOUBLE,
    "tip_amount" DOUBLE,
    "tolls_amount" DOUBLE,
    "improvement_surcharge" DOUBLE,
    "total_amount" DOUBLE,
    "congestion_surcharge" DOUBLE,
    "Airport_fee" DOUBLE
    ) AT "demo-hdlf_rs"."/parquet/"
  AS PARQUET;

In this example, since I know the store_and_fwd_flag column only has Y or N, I set this as NVARCHAR(1).
Please also find the example result in the screenshot below.

If needed, FILE COLUMN LIST can also be used for Parquet, as shown in the example below.

-- create a virtual table by pointing to Parquet files with 3 columns
CREATE VIRTUAL TABLE DEMO.YELLOW_TRIP (
    "VendorID" INT,
    "congestion_surcharge" DOUBLE,
    "Airport_fee" DOUBLE
    ) AT "demo-hdlf_rs"."/parquet/"
  AS PARQUET FILE COLUMN LIST(1, 18, 19);

For CSV and Parquet, directory-based partitioning is supported for better performance by pruning unnecessary files during query execution. For more information, please refer to the link below.

SAP HANA Cloud, SAP HANA Database SQL on Files Guide: Create a Virtual Table

Last but not least, let's see a Delta table example. For this example, the Parquet files shown above are converted into a Delta table. Also, the timestampNtz data types are converted to timestamp because currently, Reader Version 1 is the only version supported in SQL on Files.

For reading a Delta table, the entire Delta table path should be uploaded. Please also refer to the Converting from Parquet to Delta Lake. Different from CSV and Parquet, a virtual table can be created without defining the columns and corresponding data types, as shown in the example below.

-- create a virtual table by pointing to a delta table
CREATE VIRTUAL TABLE DEMO.YELLOW_TRIP_DELTA AT "demo-hdlf_rs"."/delta/" AS DELTA;

If necessary, you can also define data types for each column, but different from CSV or Parquet, the column name must be the same as the Delta table column name. Additionally, FILE COLUMN LIST and directory-based partitioning are not supported with a Delta table.

For a Delta table, time travel queries are supported, as shown in the examples below.

-- the latest version
SELECT COUNT(*) FROM DEMO.YELLOW_TRIP_DELTA;

-- version 1
SELECT COUNT(*) FROM DEMO.YELLOW_TRIP_DELTA FOR VERSION AS OF '1';

In addition to the version (FOR VERSION AS OF), a timestamp (FOR SYSTEM_TIME AS OF) can also be used. For more information, please refer to the link below.

SAP HANA Cloud, SAP HANA Database SQL on Files Guide: Time Travel Queries on Delta Tables

Additionally, you can check the latest version of the Delta table with the DELTA_LAKE_TABLE_VERSION built-in procedure and retrieve the change data feed between given versions of a Delta table with the DELTA_LAKE_TABLE_CHANGES built-in procedure. However, to use the DELTA_LAKE_TABLE_CHANGES built-in procedure, the enableChangeDataFeed must be set for the Delta table. Please also refer to the links below.

SAP HANA Cloud, SAP HANA Database SQL on Files Guide: DELTA_LAKE_TABLE_VERISON
SAP HANA Cloud, SAP HANA Database SQL on Files Guide: DELTA_LAKE_TABLE_CHANGES

Recommendations and supported data types

Now that you have understood the basic operations of using the SAP HANA Database Native SQL on Files capability in SAP HANA Cloud, please carefully review the recommendations and supported data types provided in the links below before proceeding further.

SAP HANA Cloud, SAP HANA Database SQL on Files Guide: Recommendations
SAP HANA Cloud, SAP HANA Database SQL on Files Guide: Supported Data Types

Monitoring, performance analysis, and trouble shooting

For monitoring, there are various system views available to monitor SQL on Files operations. Please refer to the link provided below.

SAP HANA Cloud, SAP HANA Database SQL on Files Guide: System Views for SQL on Files Monitoring

For further performance analysis, including EXPLAIN PLAN and SQL Analyzer Tool for SAP HANA, please refer to the detailed information provided in the link below.

SAP HANA Cloud, SAP HANA Database SQL on Files Guide: Performance Analysis

For troubleshooting, traces can be used to address issues. Please also find information on how to set trace levels in the link below.

SAP HANA Cloud, SAP HANA Database SQL on Files Guide: Troubleshooting

FAQs

Q: To use this SQL on Files feature, do I need to subscribe to any additional service or pay more?
A: No, all you need is a right-sized SAP HANA Database and data lake Files in SAP HANA Cloud. It requires proper sizing of memory and computing for the SAP HANA Database, as well as the amount of data read from the system during a specific month (API calls) of data lake files. Please refer to the SAP HANA Cloud Capacity Unit Estimator.

Q: Are there any limitations on the file size or volume that SQL on Files can handle?
A: There is no hard limitation, but it is limited by the provisioned SAP HANA Database size in SAP HANA Cloud. If the required resources to execute an SQL on Files query are higher than the provisioned SAP HANA Database size, such queries cannot be executed and scaling up of SAP HANA Database should be considered. To overcome the limitation of a single SAP HANA Database instance, we have a plan to support elastic compute nodes in the future.

Q: This SQL on Files feature is read-only from the SAP HANA Cloud Database perspective. If I want to execute DMLs on a Delta table, how can I do this?
A: Since Apache Spark can be used with the data lake Files, DMLs are expected to be executed separately via Apache Spark. Please refer to the link below.

SAP HANA Cloud, SAP HANA Data Lake User Guide for Data Lake Files: Data Lake Files with Apache Spark

Q: It seems that it shares the same virtual table or federation framework as SAP HANA smart data access (SDA). Does this mean it also supports toggling to replica tables or creating a view on it for static result cache?
A: Yes, toggling between virtual tables and replica tables is supported, but only snapshot replicas are supported. Please refer to the link below.

SAP HANA Cloud, SAP HANA Database SQL on Files Guide: Toggle Between Virtual Tables and Replica Tables

Creating a view on a virtual table and then creating a static result cache on the view is also supported.

Q: Is it recommended to create statistics for better performance?
A: SQL on Files virtual tables offer runtime data statistics collection, so you don't need to explicitly create statistics with the CREATE STATISTICS statement in most cases since this process happens automatically. However, please note that the runtime estimation uses approximated statistics via sampling. While this approach generally provides the optimal performance, it might not guarantee the best performance in some corner cases with highly skewed datasets. Please refer to the link below.

SAP HANA Cloud, SAP HANA Database SQL on Files Guide: Statistics

Conclusion

In conclusion, the new SAP HANA Database Native SQL on Files capability in SAP HANA Cloud marks a significant advancement in unlocking the potential of data within different file formats. This enhancement not only provides instant access to valuable business data in files but also seamlessly integrates them with existing data and capabilities in SAP HANA Database.

With this direct and instant access to data lake Files in SAP HANA Cloud without ingestion, it will offer new possibilities for enterprises seeking to leverage their data for strategic decision-making, while enabling a modern data analytics and data engineering ecosystem.

Continuously, we are planning for further enhancements and additional features in the future, such as Delta Sharing support and access to external object storages. Please visit our SAP Road Map Explorer to stay tuned for the upcoming updates.

A Sneak Peek into the Upcoming Dynamic Elasticity (Elastic Compute Node) in SAP HANA Cloud

2024-10-08T09:30:00.155000+02:00

Already featured on the SAP Road Map Explorer and announced at SAP TechEd, the upcoming SAP HANA Cloud, SAP HANA database QRC 04/2024 release will introduce Elastic Compute Node (ECN), a new dynamic elasticity feature.

As the name indicates, ECN is a read-only compute node (computeserver) that can be easily provisioned and deprovisioned based on incoming workloads or customer demands. With ECN, customers can efficiently address expected peak workloads, such as heavy planning jobs and Black Friday, while optimizing the size of the SAP HANA database in SAP HANA Cloud based on average workloads because ECNs can be spun up only when needed and then removed.

ECN has already been successfully used by SAP Integrated Business Planning (IBP) and SAP Datasphere for the purpose of addressing peak workloads while optimizing TCO, and now, it will be available for SAP HANA Cloud standalone use cases from QRC 04/2024. If you want to know more about how IBP benefits from the gained elasticity, then have a look at this blogpost and learn about IBP’s journey to SAP HANA Cloud.

Please find a nutshell of this new capability below:

A read-only compute node where the workload can be distributed.
Guided elasticity via REST API, btp Command Line Interface (CLI), or cf (Cloud Foundry) CLI.
Basically, equivalent to the indexserver but no persistency to store data permanently.
Replica tables and temporary tables can be created and stored in ECNs to accelerate query performance. These tables are removed when the ECNs are deprovisioned.
Since ECNs are not included in High Availability (HA) or Disaster Recovery (DR), this means there are no replicas for ECNs themselves for the purpose of HA/DR, resulting in a smaller footprint and lower TCO.
Workload Class-based dynamic workload routing.

As illustrated in the diagram above, while a 16 vCPUs and 256 GB SAP HANA database is intended for average workloads, a provision of 8 vCPUs and 128 GB ECN can be made on-demand to address peak workloads and then deprovisioned as needed. Essentially, the size of the ECN can be larger than the main SAP HANA database in SAP HANA Cloud, and multiple ECNs can also be provisioned. However, detailed restrictions, such as the maximum size of the ECN and the maximum number of parallel ECNs, will be announced with QRC 04/2024.

In conclusion, the upcoming introduction of ECN in SAP HANA Cloud represents a significant step forward in dynamic elasticity. This new feature will allow for greater flexibility and efficiency in addressing peak workloads, offering a promising solution for optimizing the size of the SAP HANA database while enhancing overall performance. As this is just a sneak peek and SAP HANA Cloud is continuously evolving, we encourage you to stay tuned for the upcoming updates.

Where can a database administrators start learning SAP HANA user management?

2024-10-11T14:15:59.389000+02:00

I recommend to access our Implementing SAP HANA User Management learning journey

Overview
In this learning journey, you will learn about users, user groups, role, role groups, privileges in SAP HANA Cloud and SAP HANA Platform. This learning journey equips database administrators with essential skills to create, modify, and delete users, manage roles, and control access.

Learning objectives
After completing this learning journey, you will be able to:

Perform SAP HANA user management, including creation, modification, and deletion of users, and assignment of appropriate roles and privileges.
Manage roles and privileges to control access to critical data and functions within SAP HANA.

Please ask a question related to the digital learning Journey in the Q&A area.

Our SAP Learning Experts will get back to you as soon as possible! We are here to support you.

I appreciate your feedback and we will make sure to continue sharing interesting topics.

Kind regards
Margit

Navigating the Latest Bug Fixes in SAP HANA 2.0 SPS7 Rev 78 and 79: Essential Updates Before Upgrade

2024-10-24T11:44:02.383000+02:00

SAP HANA continuously evolves, and recent updates in version 2.0 SPS7 Rev 78 and 79 bring critical bug fixes that can impact your environment. Below, we highlight notable issues addressed, along with their corresponding SAP Notes and applicability, to help you evaluate your upgrade path.

Category	Issue Number	Description	SAP Note	Fixed in Rev	Parent Note	Applicability	Affected Release/ Version
BW/OLAP	310837	Fixed an issue that caused the BW query to fail with the error message "Failed to lookup path '$transient_path,B<n>': calcEngine search on olapIndex failed" as a result of incorrect handling of the hierarchy join attribute during the optimization process.	3466327	78	https://me.sap.com/notes/3447850/E	BW	SAP HANA 2: Revisions <= 059.12 (SPS05) Revisions of SPS06 Revisions <= 077.00 (SPS07)
	322338	Fixed an issue that could have caused a query to fail with the error 'Attribute Engine method not implemented' when OlapEngine joins on TIMESTAMP type attributes with compressed attributes. This issue can be worked around by using "WITH HINT( NO_USE_OLAP_PLAN )".		78	https://me.sap.com/notes/3447850/E	BW	SAP HANA 2: Revisions <= 059.12 (SPS05) Revisions of SPS06 Revisions <= 077.00 (SPS07)
Data Provisioning	323021	Resolved missing records after data load with ODataAdapter and SAP SuccessFactors remote source.	3453818	78		SF to BW	SAP HANA 2.0 SPS 06 (2.00.067.00)
	323380	Fixed incomplete data issue with SuccessFactors API for SAP BW Extraction.	3456531	78		SF to BW	SAP HANA DATABASE Version 2 2.00.076
High Availability	318016	Fixed race condition preventing SQL port opening after timetravel.		78			SAP HANA 2: Revisions <= 059.12 (SPS05)
	320246	Resolved long HSR takeover times leading to errors when primary remained in "stopping" state.		78			SAP HANA 2: Revisions <= 059.12 (SPS05)
Scale Out	317146	Fixed landscape redistribution failure with HDI tables.	3410692	78			SAP HANA 2: Revisions of SPS00 - SPS06 Revisions <= 077.00 (SPS07)
Smart Data Access	286631	Fixed indexserver crash when a federated query was canceled.	3467714	78		sda	SAP HANA 2: Revisions of SPS00 - SPS04 Revisions <= 059.12 (SPS05) Revisions <= 077.00 (SPS07)
	313635	Fixed PSE encrypted SDA connections failure.	3397488	78	s4,bw	sda	SAP HANA 2: Revisions >= 070.00 (SPS07)
	321175	Resolved hanging issue during thread callstacks collection from third-party libraries.	3439753	78		thread locking situation	SAP HANA 2: Revisions 071.00 - 077.00 (SPS07)
XS Engine	322233	Fixed potential deadlock in threads from xsodata batch requests.		78		BW
General	287968	Resolved indexserver failure on startup after upgrade due to unregistered conversion rates provider.	3326809	78		all indexserver	SAP HANA 2: Revisions (SPS01 - SPS05) Revisions <= 066.00 (SPS06)
	297054	Fixed crash at ESStatistics check for invalid log_mode value.	3284126	78		all log_mode	SAP HANA 2: Revisions of SPS03 Revisions of SPS04 Revisions <= 059.13 (SPS05) Revisions of SPS06 Revisions <= 077.00 (SPS07)
	310220	Fixed long execution times for Savepoints with HANA Fast Restart Option.	3466826	78		all Savepoint Slowness	Affected Releases: SAP HANA 2: All Revisions of SPS04 Revisions <= 059.12 (SPS05) All Revisions of SPS06 Revisions <= 077.00 (SPS07)
	311518	Fixed auto delta merge failures due to outdated CPU load checks.		78			< 2.00.058.00
	318143	Resolved webdispatcher crash when changing profile parameters.	3416859	78		all HANA Web Dispatcher	SAP HANA 2: Revisions of SPS00 - SPS04 Revisions <= 059.12 (SPS05) Revisions of SPS06 Revisions <= 077.00 (SPS07
	319152	Removed confusing error message after upgrading HANA revisions.		78
	319313	Fixed issues with statements not being canceled properly.		78		all Cancel requested	< 2.00.078.00
	320559	Fixed issue with alias name insertion in base aliases setting.		78			< 2.00.078.00
	321431	Fixed invalid column name error during query optimization.		78			< 2.00.058.00
	321579	Fixed delayed delta merges causing garbage collection issues.	3449331	78		all Delta Merge	SAP HANA 2: Revisions of SPS00 - SPS04 Revisions <= 059.12 (SPS05) Revisions of SPS06
	321748	Fixed incorrect TSTMP_IS_VALID function results for leap day timestamps.		78			< 2.00.078.00
	321758	Fixed assertion failure when handling NULL values in the ESX engine.		78			< 2.00.078.00
	321773	Fixed long takeover duration on secondary site during ongoing table compression.		78			< 2.00.058.00
	321905	Resolved long query runtimes on column store tables with concatenated indexes.		78			< 2.00.078.00
	322463	Fixed indexserver crash at TREX search add operation.	3471504	78		all indexserver	Affected Releases: SAP HANA 2: Revisions of SPS00 - SPS06 Revisions <= 077.00 (SPS07)
	323800	Fixed indexserver crash due to allowed rowcount discrepancy.	3462484	78		all indexserver	SAP HANA 2: Revisions of SPS00 - SPS04 Revisions <= 059.13 (SPS05) Revisions of SPS06 Revisions <= 077.00 (SPS07) Prerequisites:
	321503	Fixed query failure on joined parameterized SQL views.	3483974	78			SAP HANA 2: Revisions <= 077.00 (SPS07)
	323344	Fixed indexserver crash related to synchronization locks on IBM Power architecture.		78			< 2.00.058.00
	320762	Fixed suboptimal insert performance with Inverted Individual indexes.		78		All	< 2.00.078.00
	319948	Fixed mem_phys value not refreshing in M_HOST_INFORMATION with cgroups-v1 enabled.	3452648	78		all IBM Power machines	RHEL8.x: impacted RHEL9.x: no issue
	319233	Fixed issue preventing LOB cleanup initiation based on global settings.		78		All	< 2.00.058.00
	321696	Fixed memory increase during SQL parsing without proper reclamation.		78		All	< 2.00.058.00
BW/OLAP	322966	Fixed an issue in pruning optimization that could lead to long runtime and high memory consumption for a query.		79	https://me.sap.com/notes/3470932/E	BW
	324526	Fixed an issue that may have caused unexpected query results in the OLAP engine when applying filters to an inner joined table.		79		BW
Backup & Recovery	320536	Fixed an issue that previously caused recovery to fail due to indexserver crash on "Backup::Channel::DecompressionChannel::waitForDecompressionJob()" when out-of-memory events occurred during backup writing.		79		All
	322793	Fixed an issue that caused the recovery of the Tenant DB to fail due to the SSFS access not being initialized.		79		All	affecting to SPS07 only lower than revision 78
	323138	Fixed an issue that caused an indexserver crash with a crashdump when the backup via backint was unexpectedly aborted. The stack trace of the crash was similar to: "Double concatenation exception throw location: 0: ltt::exception::append_exp 1: Diagnose::ExceptionTransfer::registerException 2: Execution::JobFinalObjectImpl::transferError ... exception: Error during stream send: Error: multistream data transfer terminated during send after transferring ... instead of ... bytes exception throw location: 0: Stream::Multiplexer::doCopy() 1: Backup::BackupSrc_Copier::doCopy 2: Backup::BackupSrc_Copier::doMultistreamCopy 3: Backup::MultistreamBackupCoordinator::run() 4: Backup::BackupExe_Job::run()".	3509465	79		all data compression	SAP HANA 2: Revisions of SPS06 Revisions <= 078.00 (SPS07)
	324556	Fixed an issue that caused hdbbackint files used as pipes of Backint to not be cleaned up after completing or canceling a backup.	3465987	79		all hdbbackint files	SAP HANA 2: Revisions of SPS00 - SPS06 Revisions <= 078.00 (SPS07)
	324910	Fixed an issue that caused an old backup file not to be deleted at the end of backup due to some reference counting issue when it did not reach zero.		79		all
Database Client	323226	Fixed an issue that caused an XA join request to fail when using ECN with the error message: "failed to join XA session due to active user transaction".		79		all	Affected only Hana Cloud Edition
Data Provisioning	314926	Fixed an issue that caused the candidate select function to return all rows from the master data set instead of only returning the rows that matched the break keys when enabled with pre-cleansed data.	3482523	79		all	SAP HANA 2.0 SP05 Rev 59.13 and below SAP HANA 2.0 SP07 Rev 78 and below
Enterprise Search	322672	Fixed an issue that caused the cancellation of ESH queries with an "*" sign to fail with a timeout error.		79		all	have been fixed in 2.00.078 or higher. affected to SPS07 only lower than revision 78
High Availability	297664	Fixed an issue that could have caused the nameserver service to not respond to requests anymore due to exhausting its threads. This could have occurred on hosts where one or more filesystems had been mounted that did not respond to a "stat" call.	3434285	79		all nameserver	SAP HANA 2: Revisions of SPS00 - SPS04 Revisions <= 059.10 (SPS05) Revisions <= 067.03 (SPS06) Revisions <= 073.00 (SPS07)
	322107	Fixed an issue that caused the host failover to the standby node to fail, due to the old coordinator not responding to the timestamp synchronization request, causing post-commit operations to be blocked.		79		all
Live Cache	322905	Fixed an issue that could have led to an indexserver crash at OMS_Hash due to the exception "Cannot delete Block [...] (maybe double delete)" caused by a rare race condition when running liveCache SAPAPO_CLEANUP.		79		all	Issue has been already fixed in SPS05 revision 058 or higher
Scale Out	322580	Fixed an issue that caused a table redistribution to fail with the error "replica index does not exist" when the table contained a column of type TEXT and there was a FULLTEXT index on that column.		79		S4,BW,PW	fixed in 2.00.078 or higher Affecting to SPS07 only lower than revision 78
Security	322110	Fixed an issue that caused unexpected performance when granting privileges on schemas in data warehouse scenarios.		79		all	issue has already been fixed in SPS05 revision 058 or higher
Smart Data Access	306575	Fixed an issue that prevented cancellation in certain timing scenarios if the remote statement execution did not reach the remote server (for example, in the Federation layer, in the client, on the network).		79			Affected only Hana Cloud Edition
	322428	Fixed an issue that caused the linked database feature to fail with the error message "SAP DBTech JDBC: [7]: feature not supported: Linked Database feature not supported on this remote source" for a Google BigQuery remote source when several catalogs were available.		79			HANA 2.0 SPS05 or higher. Relevant fixes have been implemented in 2.00.059.13 or higher.
	324712	Fixed an issue that led to an indexserver crash due to the presence of invalid characters in the remote table intended for replication.	3475314	79		all indexserver	SAP HANA 2: Revisions of SPS00 - SPS04 Revisions <= 059.13 (SPS05) Revisions of SPS06 Revisions <= 078.00 (SPS07)
	325214	Fixed an issue that could lead to the error "Failed to create virtual table: cannot use duplicate table name: LT0000000000000000" when using the linked database feature.		79			Reported from HANA 2.0 SPS05 or higher, Relevant fixes have been implemented in 2.00.059.13 or higher.
Smart Data Integration	324674	Fixed an issue related to SAP HANA Smart Data Quality (SDQ) Match candidate selection.	3482523	79		all	SAP HANA 2.0 SP05 Rev 59.13 and below SAP HANA 2.0 SP07 Rev 78 and below
Supportability	324622	Improved the format used to store error messages that are returned by the technical consistency check in the column 'ERROR_MESSAGE' of the monitoring view 'M_CONSISTENCY_CHECK_HISTORY_ERRORS'. The error message might have contained redundant line breaks.		79		all	Reported in SPS06 or higher, and have been fixed in 2.00.078 or higher
	303777	Fixed an SQL performance issue related to SQL queries with CAST or SUBSTRING predicates when an argument of those SQL functions was a column of a remote data source and the ABAP VARCHAR mode was enabled.	3333831	79		Near line Storage- BW External remote non-HANA data source	SAP HANA 2: Revisions of SPS06 Revision = 070.00 (SPS07)
	316947	Fixed an issue that caused long query runtime and high memory consumption when using 'count(distinct)'.		79			Affected only Hana Cloud Edition
	318529	Fixed an issue with the Query Optimizer where evaluations of a function were missing support for a data type.		79			Affected only Hana Cloud Edition
	318908	Fixed an issue that caused small column store tables to experience increasing disk size consumption in 'M_TABLE_PERSISTENCE_STATISTICS'.		79			Affected only Hana Cloud Edition
	319110	Fixed an issue that could lead to an indexserver crash at "Task::TaskOperationStatistics::getTaskStatus".		79			Affected only Hana Cloud Edition
	320157	Fixed an issue that led to a secondary indexserver crash with the error message "Assertion failed: m_pEncryptionManager->encryptionActive(getLoadOrFlushSavepointVersion()) == isFlushEncrypted()".	3442541	79		all HANA secondary indexserver crashes	SAP HANA 2: All Revisions of SPP06 Revisions <= 078.00 (SPS07)
	322172	Fixed an issue that caused unexpected results when using the group by clause in queries on remote sources.		79			Affected only Hana Cloud Edition
	322197	Fixed an issue that caused the planning engine thread to stay in "Running (CANCEL REQUESTED)" status for a long time, even after the job had been canceled from the application.	2987850	79		all	SAP HANA 2: All revisions of SPS05 - SPS06 Revisions <= 078.00 (SPS07)
	322361	Fixed an issue that caused memory to remain in the 'Pool/JoinEvaluator/JERequestedAttributes/Results/Latemat' allocator after exporting a column store table as CSV.		79		all
	322730	Fixed an issue that caused the index server to crash when accessing the monitoring view 'M_TEMPORARY_VIEW_COLUMNS'.	3485471	79		all Indexserver M_TEMPORARY_VIEW_COLUMNS	All revisions of SPS05 – SPS06 Revisions <= 078.00 (SPS07)
	323102	Fixed an issue that could have led to an indexserver crash due to garbage collection after the DDL execution of altering a table.		79		all	Reported from HANA 2.0 SPS05 or higher, but the relevant fixes have been implemented in 2.00.059.13 or higher.
	323635	Fixed an issue that caused the query on table function to fail with an assertion error.		79			Affected only Hana Cloud Edition
	323875	Fixed an issue that caused the indexserver to not start up with "Missing savepoint/snapshot record for savepoint/snapshot ID".		79		all	reported from HANA 2.0 SPS05 or higher, but the relevant fixes have been implemented in 2.00.059.13 or higher.
	324657	Fixed an issue that caused the indexserver to crash on "ESX::EsxFieldExpressionTrexTypes::FixedAttributeValue8::evaluate" due to incorrect type casting.		79			Affected only Hana Cloud Edition
	325035	Fixed an issue that caused DML statements to unexpectedly fail with the error message: "inserted value too large for column".	3471848	79		all DML	SAP HANA 2: Revisions <= 059.13 (SPS05) All revisions of SPS06 Revisions <= 079.00 (SPS07)
	325369	Fixed an issue that could have caused an indexserver crash at "JoinEvaluator::CJEGraph::effectMove" during the creation or alteration of a complex view.	3482555	79		all indexserver	SAP HANA 2: Revisions of SPS00 - SPS06 Revisions <= 078.00 (SPS07)
	322999	Fixed an issue that caused a catalog import to hang after an error occurred while importing a table.		79		all
	323230	Fixed an issue that caused no entries to be written in 'M_SERVICE_THREAD_SAMPLES' due to a missing service reconfiguration.		79			Affected only Hana Cloud Edition
	316442	Fixed an issue that caused unexpected results when sorting in the Row engine and ESX engine using the locale parameter.	3501346	79		all	SAP HANA 2: Revisions of SPS00-SPS06 Revisions <= 078.00 (SPS07)
	323024	Fixed an issue that could have caused tables with no data in any main fragment of their partitions to be skipped by the NSE Advisor.	3475978	79		NSE Advisor	SAP HANA 2: Revisions of SPS00 - SPS04 Revisions <= 059.13 (SPS05) Revisions of SPS06 Revisions <= 078.00 (SPS07)
	324780	Fixed an issue that caused the particular column and the UT metadata check to fail during an import with mismatching ids when the column contained whitespaces at the end of the name.		79		all	known issue in HANA 2.0 SPS5 Rev59.09 reported from HANA 2.0 SPS05 or higher

SAP HANA Cloud sessions at Devtoberfest 2024

2024-10-28T15:37:22.785000+01:00

Hello SAP HANA Enthusiasts!

Devtoberfest 2024 has concluded, and we hope you were able to join some of the sessions. If you missed any, no worries! Here is an overview of the SAP HANA Cloud sessions for your convenience.

🔵 Embed the latest from SAP HANA’s AI functions into your SAP BTP Application with @ChristophMorgen

In this session, Christoph introduced newly added Machine Learning and AI functions designed to enhance your SAP BTP Intelligent Data Applications.

The features include data-parallel AutoML (such as segmented AutoML TimeSeries), functions for vector data analysis, multi-label classification and regression, and more from the latest SAP HANA Cloud releases, as well as the Predictive Analysis Library (PAL) and the Machine Learning client for SAP HANA.

🔵 SAP HANA Cloud Multi-Model with @mfath

In this session, Markus focused on the Multi-Model capabilities of SAP HANA Cloud, with a particular emphasis on the vector and graph engines. Discover how a robust and scalable data management platform accelerates AI projects through rapid data processing, real-time analytics, and advanced algorithms.

🔵 Advanced Strategies for Intelligent Data Applications on SAP HANA Cloud with @VolkerSaggau , @shraddhashetty and @breglerj

This session explored advanced features of HDI and HANA Tooling. Learn from Jonathan, Volker, and Shraddha about optimizing deployment with HDI parameters, utilizing the Fabric Virtual Table in HDI, and leveraging advanced HANA tooling features, including AI-generated SQL and CDS.

Are you new to Devtoberfest?

Check more about the event at https://community.sap.com/t5/devtoberfest-blog-posts/devtoberfest-2024-welcome/ba-p/9358 by @thomas_jung

Are you looking for sessions from previous editions?

Please check the Data && Analytics sessions at Devtoberfest 2023.

Enjoy your Devtoberfest 2024 experience!

--Vitaliy

Manhattan WMS integration with SAP S/4HANA

2024-11-05T13:06:52.848000+01:00

Manhattan WMS integration with SAP S/4HANA

In the complex world of modern supply chains, businesses often use a combination of software systems to manage various processes. One common scenario involves integrating the Manhattan Warehouse Management System (WMS) with SAP S/4HANA. This integration allows businesses to leverage the strengths of both systems and achieve seamless warehouse operations across their supply chains.

Here’s an in-depth look at the integration process, benefits, and key considerations when integrating Manhattan WMS with SAP S/4HANA.

Overview of Manhattan WMS and SAP S/4HANA

Manhattan WMS:

Manhattan Associates offers a comprehensive Warehouse Management System (WMS) that supports advanced distribution and logistics management. Known for its flexibility and scalability, the system provides tools for labour management, inventory visibility, and slotting optimization.

Manhattan WMS is a highly scalable, feature-rich solution used by businesses worldwide to manage and optimize their warehouse operations. It is recognized for its advanced capabilities in picking, packing, and inventory management, effectively addressing complex warehousing requirements through configurable workflows.

SAP S/4HANA:

SAP S/4HANA is an integrated enterprise resource planning (ERP) system developed by SAP. Built on SAP's in-memory database, HANA, it offers real-time data processing and analytics, streamlining business operations across various functions like finance, supply chain, sales, and procurement. Its core features include simplified data models, enhanced performance, and cloud, on-premise, or hybrid deployment options. S/4HANA is designed to help businesses transition to digital, enabling them to be more agile, efficient, and data-driven.

Both systems offer distinct advantages, prompting businesses to consider integration to harmonize overall supply chain and logistics operations.

Why Integrate Manhattan WMS with SAP S/4HANA HANA?

Integrating Manhattan WMS with SAP S/4HANA offers several benefits, especially for businesses with diverse warehouse management needs. Here are some key reasons for integration:

a) Leverage Specialized Functionalities:

Manhattan WMS is known for its robust warehouse management features, while SAP S/4HANA excels at integrating warehouse operations with broader business processes, such as finance and procurement. Integrating both systems allow businesses to utilize the strengths of each solution.

b) Unified Data Flow:

Integration ensures seamless data flow between Manhattan WMS and SAP’s ecosystem, allowing real-time updates on inventory, order fulfilment, and stock movements. This creates a more synchronized supply chain, minimizing errors, and improving visibility.

c) Enhance Scalability and Flexibility:

Manhattan WMS manages highly complex warehouse operations, while SAP S/4HANA ensures integration with ERP systems. Together, they allow businesses to scale their warehouse operations across multiple regions or distribution centers without sacrificing efficiency.

d) Support for Complex Warehousing Models:

For businesses operating multiple warehouses or using third-party logistics providers, integrating Manhattan WMS and SAP S/4HANA helps maintain unified and consistent operations.

Integration Methods: Key Approaches

Integrating Manhattan WMS with SAP S/4HANA requires proper planning and leveraging either standard or custom integration methods. Here are the common approaches to integration:

a) Application Programming Interfaces (APIs):

APIs provide a flexible and scalable method to exchange data between Manhattan WMS and SAP S/4HANA. Many modern WMS solutions, including Manhattan, offer APIs to connect with ERP and other systems. SAP S/4HANA also supports APIs for smooth communication with external systems.

How it works:

Data from Manhattan WMS, such as order status, stock levels, or shipment details, is sent via APIs to SAP S/4HANA.
SAP S/4HANA processes this data and sends relevant updates back to Manhattan WMS through the API channels.

b) Middleware Integration:

A middleware solution (such as SAP PI/PO or a third-party integration platform) acts as a bridge between the two systems. The middleware transforms and transfers data between Manhattan WMS and SAP S/4HANA, ensuring compatibility and real-time synchronization.

Advantages:

Centralized data mapping and transformation.
Supports integration with other business systems, such as CRM or transportation management.

c) Direct Database Integration:

In some cases, businesses can integrate the databases of Manhattan WMS and SAP S/4HANA directly. While this approach requires advanced customization and technical expertise, it enables real-time data sharing.

Considerations:

Custom development might be needed.
Data integrity and compatibility checks are essential to avoid inconsistencies.

d) File-Based Integration (IDoc/BAPI):

SAP S/4HANA can integrate with external systems using IDocs (Intermediate Documents) or BAPI (Business Application Programming Interface) to send and receive messages. This method is often used for exchanging bulk data such as orders, inventory movements, and stock status updates.

Process:

Manhattan WMS sends data in structured file formats, such as XML or CSV, which is then converted into IDoc or BAPI format and processed by SAP S/4HANA.
Similarly, SAP S/4HANA can generate files and send them to Manhattan WMS to update warehouse operations.

Key Data Flows in Manhattan WMS and SAP S/4HANA Integration

Integrating Manhattan WMS with SAP S/4HANA involves the seamless exchange of critical data to ensure efficient warehouse operations. Some of the key data flows between Manhattan WMS and SAP S/4HANA during integration include:

Order Management:

Data related to order creation, picking, packing, and shipping is shared between both systems to ensure accurate order fulfilment and inventory synchronization.

Inventory Management:

Inventory movements, stock counts, and replenishments are updated in real-time, ensuring both SAP ERP and Manhattan WMS always reflect the same stock levels.

Warehouse Tasks:

Tasks like picking, packing, and putaway are optimized through real-time data synchronization, enhanced inventory visibility, and efficient task management.

Integration between Manhattan Warehouse Management System (WMS) and SAP S/4HANA requires the seamless exchange of key data flows to ensure efficient warehouse operations. Below are some of the most important data flows between the two systems, focusing on key processes like inbound and outbound logistics, inventory management, and more.

Outbound Process/Order Fulfilment

1. Master Data Synchronization

Products/Materials: Master data related to products, materials, and stock keeping units (SKUs) must be synchronized between Manhattan WMS and SAP S/4HANA to ensure consistency. This includes attributes such as weight, dimensions, product category, and handling instructions.
Business Partners: Data for business partners like customers, suppliers, and carriers must be shared across both systems for streamlined inbound and outbound logistics.
Warehouse Structure: Layouts such as storage bins, storage types, and areas need to be consistent between the two systems.

2. Inventory Data Flow

Inventory Balances: Real-time updates of stock levels in different locations or bins must be synchronized between Manhattan WMS and SAP S/4HANA.
Stock Adjustments: Any changes in stock due to damages, cycle counts, or other adjustments must be updated and communicated between both systems.
Inventory Transfers: Movement of goods between different warehouse locations should trigger the update of inventory levels in both systems.

3. Inbound Process (Goods Receipt)

Advanced Shipping Notices (ASN): When inbound deliveries or shipments are received, ASNs from SAP S/4HANA may be sent to Manhattan WMS to trigger the receipt process.
Goods Receipt Confirmation: Once goods are received and checked in Manhattan WMS, confirmation is sent to SAP S/4HANA to update inventory levels and trigger subsequent processes like quality inspections or putaway.
Putaway Instructions: Manhattan WMS may handle putaway instructions based on SAP S/4HANA data, such as predefined storage bins.

4. Outbound Process (Order Fulfilment)

Sales Orders/Outbound Deliveries: Sales orders created in SAP are transferred to Manhattan WMS, which handles picking, packing, and shipping processes.
Pick Confirmation: Once the order is picked, a confirmation is sent back to SAP S/4HANA, which updates the stock level and outbound delivery status.
Shipping Confirmation: Manhattan WMS communicates shipping confirmations back to SAP S/4HANA, which triggers invoicing and shipment tracking.

5. Returns Handling

Returns Order Creation: SAP S/4HANA or SAP ERP may generate a returns order, which is passed to Manhattan WMS for processing the physical return.
Goods Receipt for Return: Returned goods are processed in Manhattan WMS, and the information is sent back to SAP S/4HANA to update the inventory and returns status.

6. Transportation Data Flow

Shipping Instructions: Shipping or transportation orders generated in SAP S/4HANA are sent to Manhattan WMS for shipment planning and execution.
Carrier Data: Information about the carriers, tracking numbers, and shipping labels is updated in SAP S/4HANA once the shipment is confirmed in Manhattan WMS.

7. Handling Unit Management

HU Data Synchronization: Handling unit data such as pallets, cartons, etc. and their statuses must be consistently updated between both systems, especially during outbound and inbound processes.

8. Reporting and Analytics

Operational Data: Manhattan WMS sends operational data such as order status, inventory levels, and performance metrics to SAP S/4HANA for analysis and reporting.
Warehouse Performance: KPIs like order accuracy, pick rates, and stock levels are exchanged to create comprehensive reports.

9. Error Handling and Reconciliation

Error Logging: Any errors in data transfer (e.g., stock mismatches, failed transactions) between the two systems should trigger error logs for reconciliation.
Reconciliation Reports: Data reconciliation ensures that inventory levels, order statuses, and other critical data match between Manhattan WMS and SAP S/4HANA.

Integration Technologies

Middleware/Integration Platform: Middleware platforms (e.g., SAP PI/PO, MuleSoft) or API-based integration are commonly used to manage data flows between Manhattan WMS and SAP S/4HANA.
IDocs/EDI: Standard formats like IDocs, XML, or EDI are used to transfer data related to orders, shipments, and inventory between systems.

Summary of Key Data Flows:

Master Data: Products, partners, warehouse structure.
Inventory Data: Stock levels, adjustments, and transfers.
Inbound Logistics: ASN, goods receipt, putaway.
Outbound Logistics: Orders, picking, shipping confirmation.
Returns: Return orders and stock updates.
Transportation: Shipping instructions and carrier updates.
Handling Units: Handling unit updates and synchronization.
Reporting: Operational metrics and performance data.
Error Handling: Reconciliation and error logging.

These data flows ensure smooth warehouse operations, inventory accuracy, and alignment of order fulfilment processes across both systems.

Communication Ports and Protocols:

Integration between SAP S/4HANA and Manhattan Warehouse Management System (WMS) typically involves communication via middleware or direct API calls over specific communication ports. Common communication protocols include:

SAP S/4HANA often uses standard ports for communication over protocols like HTTP(S), SOAP, REST, IDoc, or RFC for message exchanges.
Manhattan WMS may have different integration approaches, but it commonly relies on APIs (SOAP/REST) or message queues like JMS for communication.

Some commonly used ports:

HTTP (80) / HTTPS (443):
- Used for REST and SOAP API calls for communication between Manhattan WMS and SAP S/4HANA via web services.
RFC (SAP-specific, typically over ports 3300-3399):
- SAP S/4HANA uses Remote Function Calls (RFC) for external system integration. Middleware may be required to translate these to compatible API calls for Manhattan WMS.
IDoc (Intermediate Document, typically over ports like 3300-3399 for RFC):
- SAP S/4HANA may send or receive IDocs to/from Manhattan WMS. Ports used are typically similar to RFC communication.
JMS (Java Message Service):
- If Manhattan WMS uses a message queue, SAP S/4HANA might need to communicate over a JMS queue using middleware like IBM MQ or TIBCO. The port could vary based on the specific queue implementation, often 61616 for ActiveMQ.
FTP/SFTP (Ports 21 for FTP, 22 for SFTP):
- File-based data exchange between SAP S/4HANA and Manhattan WMS can be done via FTP/SFTP.

Example Integration:

Middleware platforms like SAP PI/PO, Dell Boomi, or Mulesoft are often used to translate between SAP S/4HANA formats (IDoc, BAPI, RFC) and the APIs or message formats used by Manhattan WMS.
Both systems can communicate by exchanging data over a predefined integration protocol like XML, JSON, or flat file.

Specific Ports to Configure:

SAP Gateway/Dispatcher: Ports 3300+Instance Number for RFC communication.
Web services/HTTP: Ports 80/443 for web-based communication (SOAP/REST).
FTP/SFTP: Ports 21/22 for file transfers.

To determine the exact ports, the middleware or direct integration protocol needs to be evaluated, and port configurations should align with security policies and firewall settings of both systems.

Real-World Example: A Success Story

A global retail giant successfully integrated Manhattan WMS with SAP S/4HANA to optimize operations across distribution centers in multiple regions. The integration enabled real-time inventory updates, reduced order processing times, and improved labour productivity. By using middleware for seamless communication between the systems, they were able to handle large volumes of data without compromising operational accuracy.

Conclusion

Integrating Manhattan WMS with SAP S/4HANA provides businesses with a powerful solution that enhances warehouse efficiency, data accuracy, and end-to-end supply chain visibility. However, to ensure success, businesses must carefully plan, adopt best practices, and address potential challenges head-on. Embracing this integration can drive efficiency, minimize errors, and enable growth in an increasingly competitive landscape.

How to handle UUID in APL

2024-11-05T16:19:55.577000+01:00

As the adoption of UUIDs (Universally Unique Identifier) and GUIDs (Globally Unique Identifier) becomes prevalent, APL (Automated Predictive Library) users may face the case where their training dataset contains such IDs. These unique identifiers act as a primary key to differentiate each row in your invoices table for example, or your customers table. This blog shows how APL can work with input data containing a UUID. We will cover the two phases of machine learning: Train and Predict.

The UUID in SAP HANA

If you run the following SQL in SAP HANA:

select SYSUUID as "MY_UUID" from dummy

you obtain a big alpha-numeric value:

Now, create a view from it just to check the datatype:

create view MY_APL_VIEW as
select SYSUUID as "MY_UUID" from dummy;

select view_name, column_name, data_type_name
from view_columns where view_name = 'MY_APL_VIEW';

The UUID is stored as a VARBINARY, but this datatype is not supported in APL, therefore we will convert the UUID into a hexadecimal string. This conversion can be done with the function: bintohex().

Training dataset with a UUID

We need to train a classification model against this dataset:

Because the table contains a UUID, we must convert that column using the script below:

select 
 BINTOHEX("UUID") as "Id",
 "Days_to_Report",
 "Bodily_Injury", 
 "Property_Damage",
 "Age",
 "Gender",
 "Income",
 "Fraud" 
from 
 "APL_SAMPLES"."ACCIDENT_CLAIMS"

Make the conversion either via a SQL view if you are working with APL SQL, or via a dataframe if you are working with APL Python. An example of dataframe based on a select statement is shown here:

https://community.sap.com/t5/technology-blogs-by-sap/multiclass-classification-with-apl-automated-predictive-library/ba-p/13545072

The SQL view for training the APL model looks like this:

Inference dataset with a UUID

The inference dataset requires the same conversion before you can run the APL predictions:

select 
 BINTOHEX("UUID") as "Id",
 "Days_to_Report",
 "Bodily_Injury", 
 "Property_Damage",
 "Age",
 "Gender",
 "Income"
from 
 "APL_SAMPLES"."ACCIDENT_NEW_CLAIMS"

The APL output from predicting claims fraud contains Id values that are in hexadecimal:

We must convert back the Id so that the APL output table can be joined to the input table:

select N.*, "gb_decision_Fraud" as "Prediction" 
from APPLY_OUT O, APL_SAMPLES.ACCIDENT_NEW_CLAIMS N
where HEXTOBIN(O."Id") = N."UUID"

To know more about APL

Archiving in Commercial Project Management

2024-11-07T09:12:39.905000+01:00

Introduction

When an application data reaches end-of-purpose and is no longer required for the ongoing business, the data in the database system is moved to an external storage file system via archiving. SAP recommends this process of data archiving for improved system's performance and reduced maintenance cost.

In SAP Commercial Project Management (CPM), every business object has its own archiving object. There may be dependencies between them. For this reason, some objects can be archived only when you have completely archived the dependent data. For more details, refer the section 'Validation during Archiving Process'. Below is the list of archiving objects in CPM.

Archiving Object

Archiving Object	Description
/CPD/PWS_M	Master Projects
/CPD/PFP_P	Plan Header
/CPD/MC_H	Milestone Checklist Header
/CPD/MC_I	Milestone Checklist Items
/CPD/RM_I	Risk Management Archiving Object
/CPD/RM_A	Risk Action Items
/PICM/BO_I	Issue and Change Request
/PICM/BO_A	Activities

The archiving process is mainly divided into three parts:

Preprocessing
Writing
Deleting

Preprocess: This step applies the check program that selects the data to be archived. If the selected data of the business object pass all the checks, the status is set to 'Archiving in Process'.

Write: This step consists of the actual archiving session. The data of the objects with the status 'Archiving in Process' is written sequentially to the archive files.

Delete: This step removes the newly archived data from the database. The system reads the archive files created during the write process, and if this is successful, the system removes the data from the database. The system also generates an archive index during this process to allow faster read access.

When the Infostructure is activated in CPM, a transparent table with prefix ‘ZARIXCA*’ is generated. In CPM, the Field catalogs and Infostructures exists with the same name as archiving objects. For example, the archiving object '/CPD/PWS_M' has an infostructure '/CPD/PWS_M' and the field catalog '/CPD/PWS_M'.

Validation during Archiving Process

To archive a business object, all the dependent objects should be archived. Below table shows the dependency relation of the business objects:

Business Object	Dependent Business Object
Commercial Project	PS Project, Financial Plan, Milestone Checklist, Issue & Change Request, Risk
Milestone Checklist	Milestone Checklist Activity
Issue & Change Request	Issue & Change Request Activity
Risk	Risk Activity

Additionally there are validation checks for Status and End date as well.

Residence Period

The residence period is the minimum span of time that data must spend in the database before it meets the archivability criteria. You may configure the residence period for relevant business objects in CPM in the following path in SPRO. In the given example the residence period is '0' which indicates that the data can be archived immediately provided it fulfills the criteria. This period can be configured based on the business requirement.

Cross-Application Components → Processes and Tools for Enterprise Applications → Reusable Objects and Functions for BOPF Environment → Archiving Adapter → Maintain BO-Specific Residence Periods

Fig. 1: Residence Period Overview

Persistence of Project Hierarchy

For an ongoing operative commercial project, The Project Hierarchy is not stored in CPM rather it is fetched at runtime from Project Systems or Portfolio and Project Management. During archiving, it is crucial to store the hierarchy otherwise hierarchy may change and some information might get lost. To prevent that CPM implements an additional logic.

During execution of preprocessing program of Financial Plan archiving, the hierarchy is fetched at the runtime and is pushed to a temporary table '/CPD/D_ARCH_HIER'. When the delete program is executed, the same hierarchy is pushed to archive file and is deleted from the temporary table.

For Commercial Project, since there is a validation that the PS project should be archived first. During the execution of PS Project archive deletion program, a CPM API '/CPD/ARCH_HIERARCHY_SET' is called which sets hierarchy in the temporary table. The data in the table later gets pushed to the archived files during commercial project archive.

The feature mentioned is part of S/4 HANA 2023 FPS02, 2023 FPS02, 2022 FPS04, 2021 FPS07, and 2020 FPS09.

Conclusion

With the new development for persisting project hierarchy, the complete information will be available in the archive file which can be referred at any point of time. The new development also enables multiple validation and streamlines the entire archiving process in CPM.

Guided Wizards for JWT/SAML identity providers in HANA Cloud Central and SAP HANA Cockpit: Part 1

2024-11-12T15:07:57.449000+01:00

As you may know, within HANA Tooling (either it HANA Cloud Central* or HANA Cockpit) you have an opportunity to set up your own identity providers to make Single-Sign-On possible.

From now on, to simplify this journey and give you a smooth and approachable way of setting them up, we are introducing Guided Provider Creation of those identity providers for JWT and SAML.

The guided app wizard streamlines identity provider integration by automatically reading configurations, certificates, and keys from OpenID providers, reducing manual effort and errors. It combines the functions of the JWT provider, Certificates, and Certificate Collections apps, simplifying identity management and enhancing security in one unified solution. This consolidation minimizes operational complexity and costs, allowing businesses to efficiently manage identity configurations at scale.

For more details on Identity Providers, please see:

„Configuration Workflow for Authentication” section of SAP HANA Cloud Security documentation.

„Managing Identity Providers for User Authentication” section of SAP HANA Administration with SAP HANA Cockpit documentation.

Illustration of the wizard usage

In this guide, we will use the “JWT identity provider” setup as an example for our demo. In principle, the SAML provider setup follows the same logic.

As usual, this guided provider app you can find within “User and Authorization Management” app in the section “Identity providers” (see screenshot below).

Step 0. To make sure you have access to this application, the following system privileges are required when you use SAP HANA Cloud Central:

CREATE JWT PROVIDER

CERTIFICATE ADMIN

TRUST ADMIN

If you use SAP HANA Database 2 on-premise and SAP HANA Cockpit accordingly, the following privileges are required:

USER ADMIN

CERTIFICATE ADMIN

TRUST ADMIN

On Step 1, the required system privileges are checked. If privileges are missing, an error message shows which ones are missing.

Step 1. If the user has all the required privileges mentioned before, you need to provide a “JWT Identity Provider Name.”

While you type the name in, it is checked for validity and whether there is already a JWT identity provider with the given name. You cannot proceed to the next step if the name is invalid or duplicate — you will be notified about that.

Step 2. Provide the URL to the JWT OpenID Configuration of your OpenID provider

This is the URL where the OpenID configuration of your JWT provider will be read from, including available claims and certificates or public keys. As the wizard mentions, usually, this is the issuer URL followed by the ‘/.well-known/openid-configuration’ path. The full URL your input provides (with mandatory scheme and URL ending) is displayed for you to check.

When pressing the button for the next step, the configuration is read and evaluated. If the configuration cannot be read with the provided link, or if required information is missing in the read result, an error shows.

The progress is reset to the "JWT OpenID Configuration" step.

If the configuration can be read successfully, the wizard progresses to Step 3.

Step 3. Additional settings

Some additional settings for the JWT identity provider must be supplied like “Priority” and “JWT identity Claim”:

Priority: used to determine the order in which the identity providers with the same issuer URL are checked when an incoming token is mapped to an identity provider in the instance.

The priority value is between 1 (lowest) and 255 (highest). By default, the value is set to 100.

The identity provider with the highest priority set (255) is checked first.

If there is already JWT identity provider defined for the Issuer URL, the new provider has to have a different priority.

JWT Identity Claim: contains the user's unique identifier, such as username or email. It is used for user authentication and mapping. It can vary depending on the provider.

It can be selected from the list of supported claims as they are defined in the OpenID configuration

Step 4. You can also provide “Additional claims”

The claims in the section "Other" list must all be unique and for each line with a selected claim, a value must be entered, otherwise you will not be able to proceed. You do not have to worry about the empty lines – they will be simply ignored. With that, we can go the step of setting up certification collection.

Step 5. Here, a new certificate collection will be created for the JWT provider

The creation of a certificate collection is needed for verifying the signature of the JWT provider tokens.

The certificates and/or public keys for this collection will be taken from the OpenID configuration, imported into the SAP HANA Cloud and added to the new certificate collection by the wizard application.

A standard name for the collection will be proposed, but you can change it to your liking. The input will be validated to determine whether the certificate collection with this name already exists.

On top of that, you can select whether your certificate collection will be managed and tick the appropriate box for that. We will not do it this time. If you like to know what is going to happen when you tick the box, I invite you to check out the 2nd part of this blog post 😉

Step 6. Depending on what the OpenID server provides, either the “Public Keys” or the “Certificates” Step will show up.

It might also be possible that the step "Certificates" appears first, then the step "Public Keys".

In our case, we only have “Public Keys” Step:

This step only comes up if there are public key definitions provided by the configuration that do not explicitly contain a certificate (which would then be displayed in step "Certificates”).

The given names are validated so there are no empty name entries, no two names are the same, and no name of an already imported public key is entered.

Review and creation of identity provider

After you press the “review” button, all the information is checked.

If this check finds an error in one or several steps due to invalid changes after a step was already left for its subsequent step, a message popover is displayed indicating which steps contain invalid input.

You can navigate from these messages to the corresponding steps to correct the input.

After the final check is completed, all specified data is listed in the Review page. You still can edit the settings by pressing one of the step "Edit" buttons.

After successful execution, a summary page is displayed listing the performed actions:

Deleting the identity provider

In case you decide to delete JWT identity provider, you will have to options: restrict or cascade deletion.

If you choose:

“restrict”: only specified JWT identity provider will be deleted

“cascade”: required only when the JWT provider is referenced by a Certification collection’s (or in short PSE's**) purpose object. The provider is removed from any PSE referencing it along with any user mappings for users authenticating with the provider. For PSEs where it is the only provider assigned, the purpose of that PSE is also removed.

More about dropping JWT Provider you can read here.

Big thanks to the SAP HANA Development and Design team, without the help of whom this blog post would not have been possible!

Additional resources:
*With the migration of SAP HANA Cockpit functionality in SAP HANA Cloud Central, all of the documentation for these features has also been migrated to the SAP HANA Cloud docu set.

Moreover, SAP HANA Cloud Central has in-app help that can be accessed by clicking the “?” icon in the shellbar. This in-app help is extensive and also contains links off to relevant content in the cloud docu set.

**PSE stands for Personal Security Environment. PSEs are containers for security credentials, specifically certificates and cryptographic keys, used to establish secure communications and authenticate users or systems. PSEs store these certificates and keys in a structured, encrypted format, enabling SAP HANA Cloud to support secure communication protocols such as SSL/TLS for encrypted data transfer.

In SAP HANA Cloud, PSEs are often used in scenarios involving secure connections to identity providers, third-party applications, or other SAP systems, ensuring data integrity and confidentiality in distributed environments.

The following links may also be useful:

 For SAP data lake (SAP HANA Cloud) administration: SAP HANA Cloud, Data Lake Administration Guide for Data Lake Relational Engine

For SAP HANA Cloud administration: SAP HANA Cloud Administration Guide

For SAP HANA database (SAP HANA Cloud) administration: SAP HANA Cloud, SAP HANA Database Administration Guide

 Also, the SAP HANA Cloud Database Administration with SAP HANA Cockpit still exists, but will not be updated or maintained going forward.

Guided Wizards for JWT/SAML identity providers in HANA Cloud Central and SAP HANA Cockpit: Part 2

2024-11-12T15:19:12.067000+01:00

As you may know, within HANA Tooling (either it HANA Cloud Central* or HANA Cockpit) you have an opportunity to set up your own identity providers to make Single-Sign-On possible.

From now on, to simplify this journey and give you a smooth and approachable way of setting them up, we are introducing Guided Provider Creation of those identity providers for JWT and SAML.

For more details on Identity Providers, please see:

„Configuration Workflow for Authentication” section of SAP HANA Cloud Security documentation.

„Managing Identity Providers for User Authentication” section of SAP HANA Administration with SAP HANA Cockpit documentation.

Guided Wizard with managed Certificate Collections (or PSEs**)

In the 1st part of the blogpost, we were setting up JWT identity provider for SSO with the help of Guided Wizard. We finished on Step 5, where we decided not to tick the box for “Managed Certificate Collection” or in short “Managed PSEs”.

So, in this part, we will explore a bit more what is this feature, why it is needed and how to finish an identity provider setup with “Managed PSEs”.

What is it all about and why is it simplifying the management of certificates and public keys?

As we have already learnt, for the guided creation of a JWT/SAML identity providers, a wizard-based application was released. For one of the steps, the wizard walks through the creation of a certificate collection for verifying the signature of the JWT provider tokens.

Up to now, the current certificates and/or public keys for this collection were taken from the OpenID configuration, imported into the SAP HANA Cloud and added to the new certificate collection by the wizard application.

Usually, the certificates and/or public keys in the OpenID configuration are updated regularly and would have to be updated in the certificate collection accordingly each time this happens.

With this feature, a managed certificate collection can be created instead for verifying the JWT provider token signatures, and everything is configured so that this certificate collection is automatically updated periodically.

Finishing up the wizard set up (the rest of the setup you can see in the 1st part of the blog-post)

Step 5. Here a new certificate collection will be created for the JWT provider

Standard name for collection will be proposed, but you can change this name to your liking. The input will be validated whether the certificate collection with this name already exists.

On top of that, you will select the option when your certificate collection is managed and tick the according box for that.

The option for unmanaged certificate collection is needed for the trust of the automatic, periodic manager URL evaluation by the SAP HANA Cloud.

Step 6. Trust for Managed Certificate Collection

If this global root certificate collection exists, you can choose that this one should be used (recommended), or you can create a new certificate collection for the trust. This is what we are going to do.

You will need to specify the name of the new trust collection. A default name is suggested for the collection, which should be different from the name of the managed certificate collection from the step before.

Step 7. Certificate for Trust Certificate Collection

In this step, you can specify which certificate is to be added to the trust certificate collection.

If there is already a suitable root certificate imported in the HANA, you can choose to use this certificate. A suitable root certificate is a certificate with subject = issuer (hence "root"), and this certificate has to be the uppermost certificate in the certificate chain of the manager URL evaluation.

A suitable certificate will be added to the trust certificate collection, and you can go to review step to finish up setting up the JWT identity provider.

If you do not have a suitable root certificate already imported in the SAP HANA Cloud, you can upload and import a new certificate:

Review and creation of identity provider

After you press “review” button, all the information is checked.

You can navigate from these messages to the corresponding steps to correct the input.

After the final check is completed, all specified data is listed in the Review page. You still can edit the settings by pressing one of the step "Edit" buttons.

After successful execution, a summary page is displayed listing the performed actions:

You can now close the summary with the "Close" button and analyse, why the managed certificate collection could not be updated - e.g. a wrong certificate was used.

You can easily navigate from the identity provider details page, which is displayed after closing the summary page, to the details of the managed certificate collection. There you can try another manual update, navigate to the certificate collection for the manager URL trust to correct it, etc:

With that, we are concluding our 2 parts blogposts about guided wizard for identity provider set up for SAP HANA Cloud and SAP HANA onprem.

Big thanks to the SAP HANA Development and Design team, without the help of whom this blog post would not have been possible!

Additional resources:

*With the migration of SAP HANA Cockpit functionality in SAP HANA Cloud Central, all of the documentation for these features has also been migrated to the SAP HANA Cloud docu set.

The following links may also be useful:

For SAP data lake (SAP HANA Cloud) administration: SAP HANA Cloud, Data Lake Administration Guide for Data Lake Relational Engine

For SAP HANA Cloud administration: SAP HANA Cloud Administration Guide

For SAP HANA database (SAP HANA Cloud) administration: SAP HANA Cloud, SAP HANA Database Administration Guide

 Also, the SAP HANA Cloud Database Administration with SAP HANA Cockpit still exists, but will not be updated or maintained going forward.

Handling SAP On-Prem Financials with IBM Apptio

2024-11-18T12:16:49.303000+01:00

Simplifying SAP On-Prem Financials with Clarity, Efficiency, and Insights

Managing SAP on-premise financials can be complex, especially when addressing areas like vendor management, IT tower management, and IT services management. With IBM Apptio, organizations can simplify these processes through seamless data integration, enhanced visibility, and automated financial analysis. This blog covers how IBM Apptio’s integration with SAP on-premise systems can streamline these critical financial functions, using real-world use cases to illustrate the impact.

Value Proposition

IBM Apptio’s Datalink integrates with SAP on-premise systems, offering enhanced visibility into cost management and financial performance. By automating data collection and providing powerful reporting capabilities through TBM Studio, Apptio enables organizations to make better-informed financial decisions while reducing manual effort in financial management.

Streamlined Financial Data Aggregation and Reporting: Apptio Datalink efficiently gathers and reports SAP financial data, reducing manual effort and time.
Enhanced Cost Visibility: Integration with SAP financials enables Apptio to provide actionable insights, helping stakeholders understand and manage costs more effectively.
Targeted Data Mapping in TBM Studio: With data visualization capabilities in TBM Studio, users can map financial data schemas to specific organizational needs, improving analysis accuracy.

Architecture Overview

The integration between SAP on-premise systems and IBM Apptio comprises several key components:

Apptio Frontdoor > Datalink: Acts as the gateway for Apptio’s data collection.

Datalink Agents: Fetch and transmit SAP cost data, securing access through tokens and proxies.

Datalink Connections: Support seamless data flow and integration with SAP financial sources.

Each component plays a vital role in capturing and managing financial data for streamlined reporting and analysis.

IBM Apptio Offerings

IBM Apptio’s offerings are designed to address these challenges. Datalink sits within the SAP on-premise systems, acting as a powerful data bridge that works inside or outside firewall constraints. Key functionalities include:

Datalink Agent: Positioned within SAP environments, Datalink gathers cost reports seamlessly and provides a central repository for data aggregation.
Authentication Tokens and Proxies: Securely establish connections to SAP data sources, enabling easy access to essential financial metrics.
Streamlined Data Management: Apptio consolidates and organizes data from SAP financials, making it accessible for advanced financial analysis and management in TBM Studio.

Planning, Staging, and Reporting in TBM Studio

IBM Apptio’s TBM Studio offers a robust platform for planning and staging reports before they go live. The process includes:

Report Development: Building and testing reports to capture essential SAP financials.
Error Resolution: Correcting any issues before final deployment.
Production Release: Moving reports from development to staging and, finally, to production.

Use Cases

IBM Apptio’s integration with SAP on-premise financials provides valuable insights and streamlined management for key financial areas such as:

Vendor Management: Tracking and analyzing vendor-related costs to improve budgeting and alignment with business goals.
IT Tower Management: Centralizing and optimizing costs for IT resources like networking, storage, and compute.
IT Services Management: Managing the financials of IT services to ensure efficient budgeting and planning.

Conclusion

Managing SAP on-premise financials doesn’t have to feel like solving a giant jigsaw puzzle. With IBM Apptio, the pieces start to fall into place, giving you a clear picture of your financials.

See the Bigger Picture: Gain real-time insights into your financial data, bringing much-needed clarity and transparency to the table.
Say Goodbye to Manual Hassles: Automate the heavy lifting of cost aggregation, freeing up time for smarter budgeting and planning.
Turn Data into Decisions: Use TBM Studio to uncover actionable insights that help you make confident financial choices.

By integrating IBM Apptio with SAP on-prem systems, you can demystify financial management—whether it’s tracking vendor expenses, optimizing IT tower costs, or managing IT services. It’s not just about numbers anymore; it’s about turning those numbers into strategies that make sense for your business.

SAP HANA 2.0 SPS 08 Released – Safeguarding Customer Investments and Maximizing Value

2024-11-20T10:00:00.035000+01:00

With the general availability of SAP HANA 2.0 SPS 08 as of November 20, 2024, we’re introducing significant enhancements to improve scalability, reliability, user experience, and reduce the total cost of ownership. The SAP HANA database currently supports over 70,000 customers, powering some of the world’s most critical applications and workloads. Our customers, partners, and internal teams value SAP HANA for its stability, reliability, sustainability, ease of use, and cost-effectiveness. With SAP HANA 2.0 SPS 08, we’ve focused on these priorities to protect our customers’ investments, while continuing to drive value in this leading translytical and multi-model database platform.

Broad Range of Innovative Features and Benefits

SAP HANA 2.0 SPS 08 introduces a range of valuable new features that make the in-memory computing environment more accessible and user-friendly. These enhancements empower more team members to obtain rapid insights and make data-driven decisions. Additional improvements focus on reducing the total cost of ownership, boosting scalability and data reliability, enhancing user experience, and more.

Here are just some of the many significant new features, capabilities, and enhancements you’ll find in SAP HANA 2.0 SPS 08:

Backup & Recovery now supports AES-256-CTR encryption for persistency, log and backup.
SAP HANA Native Storage Extension (NSE) now automatically applies recommendations and supports dynamic aging for range partitions, which automatically creates new range partitions.
Partitioning now includes three-level partitioning with Range-Range-Hash and Range-Range-Range type of partitions and comes with an enhanced multistep partitioning wizard in the SAP HANA Cockpit.
SAP HANA now restricts the connections between the user group members using IP / Network addresses, applications, and authentication method conditions.
X.509 certificates now supports additional wildcard values.
The SAP HANA Database Explorer is now available as an extension for Visual Studio Code.
Enhanced Machine Learning capabilities in multiple areas, like classification, time series, forecasting, and regression, to support the development of next-generation intelligent applications.
Updated SAP HANA Smart Data Access & Smart Data Integration adapter certifications and new data provisioning capabilities, allowing for improved and seamless data federation and replication capabilities.
Modeling enhancements for column-based pruning and greedy pruning, along with the support of user-defined functions in expressions of calculated columns and filters, which provide better performance and more data modeling flexibility.
Updated XSA Runtimes for TomCat, Node.js, and SAPMachine provided for continued deployment flexibility.
Enhanced SAP HANA on-premises to SAP HANA Cloud migration assistant in SAP Build now supports XSC to XSA and CAP migrations.

Future Release & Maintenance Strategy

Starting with SAP HANA 2.0 SPS 08, we’re planning to deliver new SAP HANA 2.0 versions every 2 years and always in Q4 of the respective calendar year. In addition, each SAP HANA 2.0 version will provide a 4-year maintenance period. This will provide a clear and predictable SAP HANA release and maintenance approach. In addition, customers will always have a 2-year overlap between two consecutive versions, offering sufficient time to plan and upgrade to the most recent version.

Want to learn more about the new SAP HANA 2.0 SPS 08 release? If so, please register for the upcoming “What’s New in SAP HANA 2.0 SPS 08 Overview Session” delivered by our SAP HANA Product Management team.

Check out the detailed What’s New Sessions that are scheduled on YouTube Live to learn more about the various enhancements.

If you have questions, please visit the SAP Community Q&A for SAP HANA or submit your question directly.

New Machine Learning features in SAP HANA 2.0 SPS 08

2024-11-22T18:01:15.165000+01:00

With the SAP HANA 2.0 SPS 08 release, several new features have been released in the SAP HANA Cloud Predictive Analysis Library (PAL), an enhancement summary is available in the What’s new document for Predictive Analysis Library (PAL) enhancements overview with SAP HANA Platform 2.0 SPS 08.

The feature highlights for the current release are described in more detail below

Classification and regression enhancements

A new multi-task multilayer perceptron (MLP) function is introduced, enabling multi-label classification or multi-target regression scenarios. Using a multi-task learning neural network, a single ML model allows to predict multiple, related target columns at once, as the model captures both common features across tasks as well as task-specific information with the same prediction model.

It leverages the commonalities between related tasks to improve the performance, generalization, and training efficiency, and moreover enables efficient use of data, better feature extraction, knowledge transfer, regularization, and end-to-end learning.
Furthermore, the function supports early stopping using validation data to avoid overfitting.
Users of the new multi-task MLP also benefit from shorter MLP training times and models with improved accuracy.

The new function provides unique prediction model capabilities for example scenarios like

automated multi-field value proposals or pre-filling of forms (e.g. Sales Order Automation)
or predicting multiple price-/sales-targets (average, minimum, etc.) in a single model.

For a more detailed introduction to the new algorithm see the following blog post Advancing to Multi-task Multilayer Perceptron: a new Neural Network design in HANA Machine Learning https://community.sap.com/t5/technology-blogs-by-sap/advancing-to-multi-task-multilayer-perceptron-a-new-neural-network-design/ba-p/13730435

Unified regression and -classification now support permutation feature importance,

a new and trending method in global explainability to evaluate the contribution of individual features to the overall predictive power of a model.
This is achieved by measuring the decrease of a model’s performance when a feature‘s values are being shuffled around. A detailed explanation and examples are also given in this blog Global Explanation Capabilities in SAP HANA Machine Learning.

Classic feature importance vs permutation feature importance reports

The unified regression function is enhanced with prediction- and confidence interval

Interval estimates and the point estimate of response values in the unified prediction or scoring regression procedure. The prediction interval predicts distribution of individual future observation of the target, while the confidence interval predicts the mean statistic of the future target population.
Now supported with Generalised Linear Models (GLM), Multiple Linear Regression (MLR), Random Decision Trees (RDT), and Hybrid Gradient Boosting Tree (HGBT).
Complementing this, a new interval quality function is added, which aids to evaluate the quality of an interval

A new outlier detection for regression function is added,

providing the ability to detect point outliers in data used when training linear or tree-based regression models using MLR and HGBT regressors.
Outliers in training data get identified based on residual analysis and outlier-score evaluation, then can be excluded for model training and allowing to build improved regression model predictions.

A new FairML classification and regression function is added

New function aiding to mitigate “bias” and potential “unfairness” in ML model predictions related to affected groups of individuals
Binary classification or regression models based on Hybrid Gradient Boosting Tree (HGBT) are supported
Fairness evaluation is aided with metrics like demographic parity, equalized odds, and more
For a detailed use case and function example see the following blog https://blogs.sap.com/2023/12/08/fairness-in-machine-learning-a-new-feature-in-sap-hana-pal/

The Hybrid Gradient Boosting Tree (HGBT) function has been improved

New objective function Huber Loss and reweighted square for more robust regression models
Improved classification models by support of F1, recall, precision as cross validation metrics and weight scaling of target values to address imbalanced classes or weight scale values related to different costs
For improved early stopping during model optimization, the validation metric for early stopping can now be explicitly set.

Further regression and classification enhancements included

Improved KNN search, by matrix-enabled search method
Option to limit results to t op N classes in multi-class predictions with unified classification
Enhanced one-hot encoding of categorial features, supports aggregating infrequent values, into a single output for each feature (SVM, MLP, MLinR, MCLogR)
Data-parallel / massive execution support with Multiple Linear-, Exponential-, Bi-Variate Natural Logarithmic-, Bi-Variate Geometric-, Polynomial-Regression

Time series analysis and forecasting enhancements

New outlier detection function in time series analysis

Automatic or manual time series residual analysis, incl. intermittent time series detection, and missing value handling, auto smoothing of time series
Automatic selection of the outlier detection algorithm from the residual series incl. Z1, Z2, Inter-Quartile-Range, Mean absolute deviation, Isolation Forest and DBSCAN
Support of a voting ensemble across multiple outlier scoring metrics

Improved seasonality- and trend decomposition in time series analysis

Seasonal and trend decomposition using the STL-method (season, trend, low-pass window), based on LOESS (locally estimated scatterplot smoothing)
and a super smoother function
Allowing for seasonality tests with more robust decomposition and less sensitive to outliers

Time series analysis permutation-based external feature importance evaluation

a new and trending method in global explainability to evaluate the contribution of individual features to the overall forecast model accuracy.
This is achieved by measuring the decrease of a model’s performance when a feature‘s values are being shuffled around. A detailed explanation and examples are also given in this blog Global Explanation Capabilities in SAP HANA Machine Learning.
Supported with the following time series algorithms ArimaX, Bayesian Structural Time Series (BSTS), Long term time series (LTSF), Additive model time series analysis (AMTSA)

Forecast prediction interval evaluation

a new interval quality function is added, which aids to evaluate the quality the forecasted prediction interval

Long term time series (LTSF) enhancements

New Neural Network model types (NLinear, DLinear, XLinear) for improved speed and forecast accuracy
Rigorous decomposition support of neural network time series models into TREND, SEASONAL and EXOGENOUS_CONTRIBUTIONS parts

New data-parallel, massive execution time series function support

Online BCPD, Forecast Accuracy Measure, Time series outlier detection, Single Exponential Smoothing, Double Exponential Smoothing, Triple Exponential Smoothing, Brown Exponential Smoothing, Croston's Method, Linear Regression with Damped Trend and Seasonal Adjust, White Noise Test, Seasonality Test, Trend Test, Change-Point Detection, Intermittent Time Series Forecast (ITSF)
Allowing segmented time series analysis and forecasting and fastest result when modelling thousands of time series in parallel, identified by a segmentation, grouping column

AutoML and ML pipelines enhancements

General AutoML configuration and optimization improvements

Optimized pipeline operator selection, considering operator sequence and importance using operator connection constraints
Finetuning phase of best pipeline after initial AutoML optimization, by further evaluation the remaining parameter combination options and without changing the selected pipeline operators itself
Support of random search optimization for small AutoML configurations and faster results, for example with simple time series scenarios
Faster AutoML pipeline predictions interpretability based on a more light-weight shapley explainer model (SHAPGlobal surrogate). More details can be found in the following blog post Demystifying Pipeline Explanation for Time Series Data.
AutoML scenario configuration verification option
Pipeline predictions supporting more algorithm specific parameters
Score function for hold-out sample based classification / regression model evaluation
Improved AutoML progress logging in Python ML client, with full log information

AutoML Time Series enhancements

Support for auto exponential smoothing, polynomial feature generation,
HGBT-/ MLinR- regression operators for time series
SPEC as a new evaluation metric supported

New data-parallel AutoML and ML Pipeline modeling functions allow for developing and executing multiple models in parallel by

building AutoML classification, regression or time series models in parallel on data subsets, identified by a grouping column,
using the new massive AutoML and Pipeline SQL procedures

This allows for fastest parallel AutoML modeling, like e.g. segmented AutoML time series. At the same time SAP HANA Cloud’s database workload management is best applied to control and limit compute resource consumed.

Further AI function enhancements

New Quantile Transform data preprocessing

The Quantile Transform automatically applies a non-linear transform, that maps numeric input columns of varying data distributions, into a uniform or normal target distribution making it more suitable as an input for many ML models.
Preserves the rank order of the data, and reduces the impact of outliers, makes data less sensitive to extreme values

New financial data analysis functions

Benford Analysis, trending algorithm used to detect anomalies in financial data
- One of the (not so) well-known statistical observations is the fact that in many datasets the leading significant digits are not equally distributed. If all digits were represented equally, then they would appear 11.1 percent (1/9TH) of the time. However, when analyzing real-world datasets, it is revealed that the distribution of the leading digits follows the Bedford’s law, also known as the first-digit law.
- It is now very easy to validate if a dataset obeys Bedford’s law or not. A first step means, very commonly applied in financial applications is to detect unexpected value distributions and e.g. potential fraudulent transaction data.

Hull-White Model, trending used in modeling the time evolution of interest rates, rates, which are required for price estimation of financial instruments based on interest rate derivatives.
- To apply the Hull-White model it first needs to be adopted to match existing market conditions (interest rates). This is achieved by providing the values of the drift term of the Hull-White model as a time series as input table. The simulation will then provide the mean value for a given number of simulation paths (also specified as an input parameter), their variance, as well as the upper and lower bounds.

New MLP Recommender function

Powerful neural network function to predict binary targets such as click-through rates, a fundamental part in recommender systems.
The MLP recommender function does not require specific user and item details, and can include extra data features, unrelated features without any user connection.
The algorithm employs a dual-stream MLP framework, capable of handling larger-scale and “complicated” input data, designed to boost the recommendation performance.

Text processing enhancements

New RDT model-based text classification
Improved TF-IDF providing control over stopword usage

Improved Clustering

Pre-c alculated d istance m atrices supported with agglomerate hierarchical- and spectral clustering with the unified clustering function
Improved one-hot encoding support with spectral clustering

Data-parallel / massive execution support

Isolation Forest, Variance Test, Inter-Quartile Range, Fast Fourier Transform, K-Optimal Rule Discovery (KORD), K-Means, FP-Growth

Automated Predictive Library

Automated Predictive Library (APL) 2425 release enhancements

Reduced execution time around 15% for binary classification or regression models (gradient boosting), and around 35% for multi-class classification.
Better estimates and faster calculation of variable interactions
See what's New 2311
Local explanations for time series See what's New 2403

Python machine learning client enhancements

The full list of new methods and enhancements within the hana_ml releases 2.17-2.22 are summarized in the changelog for hana-ml as part of the documentation.

Selected key enhancements include

AutoML and pipeline modeling improvements

Visual editor support for the AutoML scenario configuration

Massive, data parallel AutoML support

PipelineProgressMonitor python runtime_platform specification for jupyter, vscode, bas, console
AutoML optimization enhancement with connection constraints and connection visualizations

More detailed references to the described enhancements can also be found in the New Machine Learning features in SAP HANA Cloud - SAP Community blog posts.

Furthermore you can find example notebooks illustrating the highlighted feature enhancements under hana-ml-samples/Python-API/pal/notebooks at main · SAP-samples/hana-ml-samples · GitHub, like for example H2SPS08_AutoML.ipynb, H2SPS08_RegressionClassification.ipynb, H2SPS08_TimeSeries.ipynb, H2SPS08_MiscEnhancements.ipynb and H2SPS08_Design-time code generation.ipynb.

Explore and enjoy all the new capabilities!

Product Updates for SAP Build Code – November 2024 Edition

2024-11-28T08:36:58.413000+01:00

Dear SAP Community,

Welcome to our November release update! We're excited to present you many powerful new features. Whether you're a seasoned developer or just starting your journey, we have something for you. This update features a range of exciting new capabilities, such as adding explanations to SQL statements for SAP HANA Cloud or an update to our Generative AI capabilities for MDK development.

Join us as we explore the details of what's new!

SAP HANA Cloud SQL Statement Explanation

The SAP HANA Database Explorer SQL Console in SAP Build Code offers a new feature for adding explanations to SQL statements for SAP HANA Cloud. The explanations can make it easier to understand complex or unfamiliar SQL queries.

The statement explanation feature is triggered by selecting an SQL statement in the SQL console and choosing “Explain SQL Statement using AI” from the menu.

Screenshot: Choosing “Explain SQL Statement using AI” from the Menu

The explanations are added to a copy of the original statement into the SQL console. The copy that contains the explanations is delimited by “AI generated SQL query explanation begins/ends” block comments.

Screenshot: AI Generated SQL Query Explanation

As with all AI generated content the generated comments should be reviewed and checked for validity. If the comments are useful, the original statement may be discarded or combined with the comments as needed.

Generative AI Capabilities for MDK

We are thrilled to announce an exciting update to our Generative AI capabilities for MDK (Mobile Development Kit) development. Previously, these capabilities were only available when a single project was opened at the folder level. We have now enhanced this functionality to support more scenarios. Our latest updates now enable you to use Generative AI features when working with multiple projects listed either in the Projects Explorer or at the workspace level.

For example, if you have multiple projects open in the Projects Explorer and want to generate a page in a specific project using the MDK page generation genie, it will now prompt you to select a service file from various service files in multiple projects and a page will be generated in that specific project. This enhancement provides greater flexibility and convenience when working with multiple projects in MDK development.

Screenshot: MDK page generation with Joule

Further readings

Announcing a New Guide: Deep Dive into SAPUI5 Apps on SAP BTP

We are excited to share a new comprehensive guide crafted by our colleague Matthias Schmalz, offering profound technical insights into SAPUI5 applications running on the SAP Business Technology Platform (BTP). In this guide, Matthias delves into essential topics such as how to structure complex SAPUI5 applications to avoid common pitfalls, how multiple apps work with a shared reuse library and how to integrate business solutions as content providers into SAP Build Work Zone.

For further details, dive into Matthias' guide Designing UI5 Apps as Business Solution for SAP Build Work Zone and elevate your expertise in SAPUI5 development.

Explore Our Recent SAP Fiori Tools Release

For a deeper dive into the latest updates and features in SAP Fiori tools, be sure to check out the following blog post: SAP Fiori Tools 2411 Update: Enhanced Full-Stack Development, AI Integration, and More Open Source. This post provides a comprehensive overview of the new features and improvements that combine frontend and backend development, broaden AI-driven functionalities, and further the integration of open-source technologies.

How to Upgrade Your Existing Standalone Service Plans

SAP Build Code offers different types of service plans in SAP BTP: trial, free and standard. For the initial setup, a booster helps you set up SAP Build Code for the plan you want to use. But what if you already use an existing standalone service plan and want to upgrade to one of the SAP Build Code service plans?

We created the following group: Update from Current Service Plan to Build-Code Service Plan. This group consists of tutorials that guide you step by step through the upgrade process:

Our help documentation provides you a list of different current and target service plans and supported migration scenarios.

Webinar Series What's New in SAP HANA 2.0 SPS 08

Interested in discovering the latest innovations in SAP HANA 2.0 SPS 08? Then join our "What's New in SAP HANA 2.0 SPS 08" webinar series! This series is divided into seven captivating sessions, each filled with insights from over a dozen experts straight from our SAP HANA & Database Product Management team. Dive deep into key updates and features while gaining practical industry knowledge.

We look forward to seeing how you use these new capabilities to create innovative solutions and drive your projects forward. Stay tuned for more updates and happy coding! 😊