SAP Community - SAP HANA

Automated Test Suite: RHEL HA Solution for SAP S/4HANA ENSA 2 using Ansible

2026-02-25T13:33:15.204000+01:00

Introduction

Testing of High Availability Clusters used for managing SAP S/4HANA application servers is critical for ensuring their reliability and resilience. Many issues can be avoided in the first place during the pre-go-live stage by carrying out certain tests.

While manual verification of an HA cluster setup is possible, these tests can also be automated using Ansible. Developing custom roles and playbooks allows for a modular, reusable, and scalable design to automate each test case. This approach is particularly effective for tests like simulating a node failure to verify failover and resource migration.

Manual testing becomes tedious, time-consuming, and error-prone, especially in HA cluster environments where multiple nodes must be constantly observed during every test.

What is the Automated Test suite?

This Automated Test suite boosts quality assurance efficiency, test coverage, and accelerates bug reproduction/verification and post-maintenance, leading to faster production-ready RHEL HA cluster environments for ENSA 2.

The Automated Test suite requires a dedicated, out-of-cluster RHEL Ansible Control node to execute and monitor tests—including failovers and node crashes—on the SAP HA cluster. Using Ansible makes the tests transparent and easy to understand, and its built-in plugins allow saving logs for future review and audits.

Which tests are performed?

This automation test suite also covers the tests for the SAP HA Interface for SAP application servers. Each test carries out specific tasks to verify the cluster configuration and current state, conducts the actual test, verifies the output, and compares it with the expected results.

The following table outlines what the test suite actually verifies on the systems being tested:

Test Name	Test Title	Minimum no. of HA Nodes
Test01	Name and version of HA software	2
Test02	HA configuration showing no errors	2
Test03	Shared Library (HA-Interface) loads without any errors	2
Test04	Manual move of ASCS works correctly with lock data	2
Test05	Irrecoverable outage of the Enqueue Server (ES) 2 is handled correctly	2
Test06	Outage of the Enqueue Replicator (ER) 2 is handled correctly	>2
Test07	ASCS moves correctly under load without specifying the destination node (Skipped in the current version)	>2
Test08	ASCS moves correctly in case of hardware or OS failure (Note: Node crash execution)	>2
Test09	Recoverable outage of the Message Server is handled correctly (if the SAP Profile Parameter “Restart_Program” is used for the Message Server) (Note: Backup and Auto-config)	>2
Test10	Irrecoverable outage of the Message Server is handled correctly (if the SAP Profile Parameter “Start_Program” is used for the Message Server) (Note: Backup and Auto-config)	>2

Please note that test07 is skipped in the current version and may be implemented and updated in future releases. Additional test cases are planned for inclusion in future releases to enhance coverage.

Prerequisites

1. A minimum of 2 node cluster configuration with ASCS and ERS resources to start with the initial tests, but 3 nodes are recommended.
Refer to the following document for the guidelines to configure the cluster that can be tested with the Ansible playbooks described in this blog: Configuring HA clusters to manage SAP NetWeaver or SAP S/4HANA Application server instances using the RHEL HA Add-On | Red Hat Enterprise Linux for SAP Solutions (Only ENSA2). Your pacemaker cluster should look like the following:

[root@s4node01: ~]# pcs cluster status
Cluster Status:
.....
Node List:
  * Online: [ s4node01 s4node02 s4node03 ]

PCSD Status:
  s4node01: Online
  s4node03: Online
  s4node02: Online

[root@s4node01: ~]# pcs resource status
……
  * Resource Group: s4h_ASCS20_group:
    * s4h_lvm_ascs20    (ocf:heartbeat:LVM-activate):     Started s4node03
    * s4h_fs_ascs20    (ocf:heartbeat:Filesystem):     Started s4node03
    * s4h_vip_ascs20    (ocf:heartbeat:IPaddr2):     Started s4node03
    * s4h_ascs20    (ocf:heartbeat:SAPInstance):     Started s4node03
  * Resource Group: s4h_ERS29_group:
    * s4h_lvm_ers29    (ocf:heartbeat:LVM-activate):     Started s4node01
    * s4h_fs_ers29    (ocf:heartbeat:Filesystem):     Started s4node01
    * s4h_vip_ers29    (ocf:heartbeat:IPaddr2):     Started s4node01
    * s4h_ers29    (ocf:heartbeat:SAPInstance):     Started s4node01

This is also a precondition before running any test.

2. Ensure that the SAP HA Interface for SAP ABAP application server instances is configured as mentioned here: How to enable the SAP HA Interface for SAP ABAP application server instances managed by the RHEL HA Add-On? - Red Hat Customer Portal.

3. Ensure that the sap.sap_operations collection for host_info and pcs_status_info module is installed on the ansible control node.

Getting started

1. Clone the community.sap_ha_cluster_qa repository and enter into that directory

# git clone https://github.com/sap-linuxlab/community.sap_ha_cluster_qa.git
# cd community.sap_ha_cluster_qa.git

2. Verify that the ansible.cfg, inventory and playbook files match your specific Ansible environment

3. Make sure that the inventory file contains at least the hostnames of all the reachable cluster nodes that you want to test:

For example:

# cat tests/inventory/x86_64.yml

---
all:
  children:
    s4hana-3n:
      hosts:
        s4node01:
        s4node02:
        s4node03:

4. Ensure that the nodes are reachable via ansible ping, for example:

# ansible all -m ping -i tests/inventory/x86_64.yml
s4node01 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": false,
    "ping": "pong"
}
s4node02 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": false,
    "ping": "pong"
}
s4node03 | SUCCESS => {
    "ansible_facts": {
        "discovered_interpreter_python": "/usr/bin/python3"
    },
    "changed": false,
    "ping": "pong"
}

How to run the tests

1. While in the community.sap_ha_cluster_qa directory run the playbook as follows for test01 to verify name and version of HA software.

# ansible-playbook -i tests/inventory/x86_64.yml  ansible_collections/sap/cluster_qa/playbooks/test01.yml -v

PLAY [Playbook to run test01 test case on ASCS and ERS instances] ***************************************************************************

TASK [Collect necessary gather_facts] ***************************************************************************
ok: [s4node01]
ok: [s4node03]
ok: [s4node02]

......

TASK [sap.cluster_qa.test01 : Print test results completing test01 test case for current instance] ***************************************************************************
ok: [s4node02] => {
    "msg": {
        "changed": false,
        "failed": false,
        "ha_get_failoverconfig_info": {
            "HAActive": true,
            "HAActiveNode": "s4node02",
            "HADocumentation": "https://github.com/ClusterLabs/sap_cluster_connector",
            "HANodes": "",
            "HAProductVersion": "Pacemaker",
            "HASAPInterfaceVersion": "sap_cluster_connector"
        }
    }
}

......

TASK [sap.cluster_qa.test01 : Print test results completing test01 test case for current instance] ***************************************************************************
ok: [s4node03] => {
    "msg": {
        "changed": false,
        "failed": false,
        "ha_get_failoverconfig_info": {
            "HAActive": true,
            "HAActiveNode": "s4node03",
            "HADocumentation": "https://github.com/ClusterLabs/sap_cluster_connector",
            "HANodes": "",
            "HAProductVersion": "Pacemaker",
            "HASAPInterfaceVersion": "sap_cluster_connector"
        }
    }
}

2. Similarly, you can run playbooks for the next test case by replacing the playbook. In this case test02.yml to verify HA configuration shows no errors

# ansible-playbook -i tests/inventory/x86_64.yml ./ansible_collections/sap/cluster_qa/playbooks/test02.yml -v


………
TASK [sap.cluster_qa.test02 : Print the results completing TEST02 test case for ERS node] ***************************************************************************
skipping: [s4node02] => {}
skipping: [s4node03] => {}
ok: [s4node01] => {
    "msg": {
        "changed": false,
        "failed": false,
        "ha_check_config_info": [
            {
                "category": "SAPControl-SAP-CONFIGURATION",
                "comment": "0 ABAP instances detected",
                "description": "Redundant ABAP instance configuration",
                "state": "SAPControl-HA-SUCCESS"
            },
            {
                "category": "SAPControl-SAP-CONFIGURATION",
                "comment": "All Enqueue server separated from application server",
                "description": "Enqueue separation",
                "state": "SAPControl-HA-SUCCESS"
            },
            {
                "category": "SAPControl-SAP-CONFIGURATION",
                "comment": "All MessageServer separated from application server",
                "description": "MessageServer separation",
                "state": "SAPControl-HA-SUCCESS"
            },
            {
                "category": "SAPControl-SAP-STATE",
                "comment": "SCS instance status ok",
                "description": "SCS instance running",
                "state": "SAPControl-HA-SUCCESS"
            },
            {
                "category": "SAPControl-SAP-CONFIGURATION",
                "comment": "SAPInstance includes is-ers patch",
                "description": "SAPInstance RA sufficient version (s4ascs_S4H_20)",
                "state": "SAPControl-HA-SUCCESS"
            },
            {
                "category": "SAPControl-SAP-CONFIGURATION",
                "comment": "Enqueue replication enabled",
                "description": "Enqueue replication (s4ascs_S4H_20)",
                "state": "SAPControl-HA-SUCCESS"
            },
            {
                "category": "SAPControl-SAP-STATE",
                "comment": "Enqueue replication active",
                "description": "Enqueue replication state (s4ascs_S4H_20)",
                "state": "SAPControl-HA-SUCCESS"
            },
            {
                "category": "SAPControl-SAP-CONFIGURATION",
                "comment": "SAPInstance includes is-ers patch",
                "description": "SAPInstance RA sufficient version (s4ers_S4H_29)",
                "state": "SAPControl-HA-SUCCESS"
            }
        ]
    }
}

PLAY RECAP ***************************************************************************
s4node01                   : ok=32   changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0  
s4node02                   : ok=30   changed=0    unreachable=0    failed=0    skipped=4    rescued=0    ignored=0  
s4node03                   : ok=32   changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0

3. Similarly, you can run the tests for each test case by replacing the command with the corresponding test03 playbook as shown below. Test03 verifies the shared Library (HA-Interface) is being loaded without any errors.

# ansible-playbook -i tests/inventory/x86_64.yml ./ansible_collections/sap/cluster_qa/playbooks/test03.yml -v
.......
TASK [sap.cluster_qa.test03 : Printing SAP HA trace logs for ERS instance] ***************************************************************************
ok: [s4node01] => {
    "msg": [
        "SAP HA Trace: Thu Jan 22 15:59:09 2026",
        "SAP HA Trace: --- SAP_HA_FindSAPInstance Exit-Code: SAP_HA_OK ---",
        "SAP HA Trace: Thu Jan 22 15:59:09 2026",
        "SAP HA Trace: === SAP_HA_StartCluster ===",
        "SAP HA Trace: Fire system command /usr/bin/sap_cluster_connector cpa ...",
        "SAP HA Trace: SAP_HA_StartCluster: FOUND PENDING ACTION -> SAP_HA_START_IN_PROGRESS",
        "SAP HA Trace: Thu Jan 22 15:59:09 2026",
        "SAP HA Trace: --- SAP_HA_StartCluster Exit-Code: SAP_HA_START_IN_PROGRESS ---",
        "SAP HA Trace: Mon Jan 26 13:35:13 2026",
        "SAP HA Trace: === SAP_HA_FindSAPInstance ===",
        "SAP HA Trace: Fire system command /usr/bin/sap_cluster_connector lsr ...",
        "SAP HA Trace: searchClusterFile: S4H:29 found",
        "SAP HA Trace: Mon Jan 26 13:35:13 2026",
        "SAP HA Trace: --- SAP_HA_FindSAPInstance Exit-Code: SAP_HA_OK ---",
        "SAP HA Trace: Mon Jan 26 13:35:13 2026",
        "SAP HA Trace: === SAP_HA_StopCluster ===",
        "SAP HA Trace: Fire system command /usr/bin/sap_cluster_connector cpa ...",
        "SAP HA Trace: SAP_HA_StopCluster: FOUND PENDING ACTION -> SAP_HA_STOP_IN_PROGRESS",
        "SAP HA Trace: Mon Jan 26 13:35:13 2026",
        "SAP HA Trace: --- SAP_HA_StopCluster Exit-Code: SAP_HA_STOP_IN_PROGRESS ---",
        "SAP HA Trace: Mon Jan 26 13:35:17 2026",
        "SAP HA Trace: === SAP_HA_FindSAPInstance ===",
        "SAP HA Trace: Fire system command /usr/bin/sap_cluster_connector lsr ...",
        "SAP HA Trace: searchClusterFile: S4H:29 found",
        "SAP HA Trace: Mon Jan 26 13:35:17 2026",
        "SAP HA Trace: --- SAP_HA_FindSAPInstance Exit-Code: SAP_HA_OK ---",
        "SAP HA Trace: Mon Jan 26 13:35:17 2026",
        "SAP HA Trace: === SAP_HA_StartCluster ===",
        "SAP HA Trace: Fire system command /usr/bin/sap_cluster_connector cpa ...",
        "SAP HA Trace: SAP_HA_StartCluster: FOUND PENDING ACTION -> SAP_HA_START_IN_PROGRESS",
        "SAP HA Trace: Mon Jan 26 13:35:17 2026",
        "SAP HA Trace: --- SAP_HA_StartCluster Exit-Code: SAP_HA_START_IN_PROGRESS ---",
        "SAP HA Trace: Mon Jan 26 13:36:20 2026",
        "SAP HA Trace: === SAP_HA_CheckConfig ===",
        "SAP HA Trace: Fire system command /usr/bin/sap_cluster_connector hcc ...",
        "SAP HA Trace: Mon Jan 26 13:36:20 2026",
        "SAP HA Trace: --- SAP_HA_CheckConfig Exit-Code: SAP_HA_OK ---",
        "SAP HA Trace: Mon Jan 26 13:36:20 2026",
        "SAP HA Trace: === SAP_HA_FreeConfigCheck ===",
        "SAP HA Trace: Mon Jan 26 13:36:20 2026",
        "SAP HA Trace: --- SAP_HA_FreeConfigCheck Exit-Code: SAP_HA_OK ---",
        "SAP HA Trace: Mon Jan 26 13:36:21 2026",
        "SAP HA Trace: === SAP_HA_CheckConfig ===",
        "SAP HA Trace: Fire system command /usr/bin/sap_cluster_connector hcc ...",
        "SAP HA Trace: Mon Jan 26 13:36:21 2026",
        "SAP HA Trace: --- SAP_HA_CheckConfig Exit-Code: SAP_HA_OK ---",
        "SAP HA Trace: Mon Jan 26 13:36:21 2026",
        "SAP HA Trace: === SAP_HA_FreeConfigCheck ===",
        "SAP HA Trace: Mon Jan 26 13:36:21 2026",
        "SAP HA Trace: --- SAP_HA_FreeConfigCheck Exit-Code: SAP_HA_OK ---"
    ]
}
skipping: [s4node02] => {}
skipping: [s4node03] => {}

PLAY RECAP ***************************************************************************
s4node01                   : ok=32   changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0  
s4node02                   : ok=30   changed=0    unreachable=0    failed=0    skipped=4    rescued=0    ignored=0  
s4node03                   : ok=32   changed=0    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0

4. Test04 for verifying manual ASCS move:

# ansible-playbook -i tests/inventory/x86_64.yml ./ansible_collections/sap/cluster_qa/playbooks/test04.yml -v
......
TASK [sap.cluster_qa.test04 : Asserting the locks of ASCS and ERS after move completing the TEST04 Test Case] ***************************************************************************
ok: [s4node01] => {
    "changed": false,
    "msg": "All assertions passed"
}
ok: [s4node02] => {
    "changed": false,
    "msg": "All assertions passed"
}
ok: [s4node03] => {
    "changed": false,
    "msg": "All assertions passed"
}

PLAY RECAP ***************************************************************************
s4node01                   : ok=69   changed=1    unreachable=0    failed=0    skipped=8    rescued=0    ignored=0   
s4node02                   : ok=66   changed=0    unreachable=0    failed=0    skipped=10   rescued=0    ignored=0   
s4node03                   : ok=70   changed=2    unreachable=0    failed=0    skipped=6    rescued=0    ignored=0

Lock table data is compared and asserted 3 times in this test run: ASCS and ERS before move, ASCS before and after move, ASCS and ERS after move.

5. Test05 for verifying irrecoverable outage of the Enqueue Server (ES) 2 is handled correctly.

# ansible-playbook -i tests/inventory/x86_64.yml ./ansible_collections/sap/cluster_qa/playbooks/test05.yml -v
......
TASK [sap.cluster_qa.test05 : Verifying ASCS not on the same node] ***************************************************************************
ok: [s4node01] => {
    "changed": false,
    "msg": "All assertions passed"
}
ok: [s4node02] => {
    "changed": false,
    "msg": "All assertions passed"
}
ok: [s4node03] => {
    "changed": false,
    "msg": "All assertions passed"
}

PLAY RECAP ***************************************************************************
s4node01                   : ok=38   changed=1    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0   
s4node02                   : ok=38   changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
s4node03                   : ok=36   changed=1    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0

6. Test06 for verifying outage of the Enqueue Replicator 2 is handled correctly

# ansible-playbook -i tests/inventory/x86_64.yml ./ansible_collections/sap/cluster_qa/playbooks/test06.yml -v
......
TASK [sap.cluster_qa.test06 : Verifying ERS not on the same node as ASCS] ***************************************************************************
ok: [s4node01] => {
    "changed": false,
    "msg": "ERS successfully moved to s4node01, different from ASCS node s4node02"
}
ok: [s4node02] => {
    "changed": false,
    "msg": "ERS successfully moved to s4node01, different from ASCS node s4node02"
}
ok: [s4node03] => {
    "changed": false,
    "msg": "ERS successfully moved to s4node01, different from ASCS node s4node02"
}

PLAY RECAP ***************************************************************************
s4node01                   : ok=53   changed=2    unreachable=0    failed=0    skipped=0    rescued=0    ignored=0   
s4node02                   : ok=49   changed=1    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0   
s4node03                   : ok=49   changed=1    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0

7. Test07 (skipped in the current version)

8. Test08 to verify ASCS moves correctly in case of hardware or OS failure. Please note that in this test the primary ASCS node will be crashed using the “echo c > /proc/sysrq-trigger” command.

# ansible-playbook -i tests/inventory/x86_64.yml ./ansible_collections/sap/cluster_qa/playbooks/test08.yml -v
......
TASK [sap.cluster_qa.test08 : Verifying ASCS not on the same node] ***************************************************************************
ok: [s4node01] => {
    "changed": false,
    "msg": "ASCS successfully moved from s4node03 to s4node02"
}

PLAY RECAP ***************************************************************************
s4node01                   : ok=40   changed=1    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0   
s4node02                   : ok=33   changed=0    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0   
s4node03                   : ok=18   changed=1    unreachable=0    failed=0    skipped=2    rescued=0    ignored=0

9. Test09: Recoverable outage of the Message Server is handled correctly (if the SAP Profile Parameter “Restart_Program” is used for the Message Server). Note that this test will check for the “Restart_Program” parameter in the instance profile. If not found, it will be inserted, and the instances will restart before performing the actual test.

# ansible-playbook -i tests/inventory/x86_64.yml ./ansible_collections/sap/cluster_qa/playbooks/test09.yml -v
......
TASK [sap.cluster_qa.test09 : Display test summary] ***************************************************************************
ok: [s4node01] => {
    "msg": [
        "===============================================",
        "              TEST09 SUMMARY",
        "===============================================",
        "Message Server killed: 6 times",
        "Initial ASCS location: s4node02",
        "Final ASCS location: s4node02",
        "Initial ERS location: s4node01",
        "Final ERS location: s4node01",
        "HA Action taken: ASCS Restart on same node",
        "ASCS/ERS separation maintained: YES",
        "==============================================="
    ]
}

PLAY RECAP *********************************************************************
s4node01                   : ok=105  changed=1    unreachable=0    failed=0    skipped=77   rescued=0    ignored=0   
s4node02                   : ok=123  changed=6    unreachable=0    failed=0    skipped=22   rescued=0    ignored=0   
s4node03                   : ok=81   changed=0    unreachable=0    failed=0    skipped=64   rescued=0    ignored=0

Please note this test may take approximately 7 to 10 minutes since the message server is killed repeatedly up to 6 times or until HA software intervenes to perform a failover.

10. Test10: Recoverable outage of the Message Server is handled correctly (if the SAP Profile Parameter “Start_Program” is used for the message server). Note that this test will check for the “Start_Program” parameter in the instance profile. If not found, it will be inserted and the instances will restart before performing the actual test.

# ansible-playbook -i tests/inventory/x86_64.yml ./ansible_collections/sap/cluster_qa/playbooks/test10.yml -v
......
TASK [sap.cluster_qa.test10 : Display test summary] ***************************************************************************
ok: [s4hana17] => {
    "msg": [
        "===============================================",
        "              TEST10 SUMMARY",
        "===============================================",
        "Test: Irrecoverable Message Server outage",
        "Profile Parameter: Start_Program (not Restart_Program)",
        "Message Server killed: YES",
        "Initial ASCS location: s4hana19",
        "Final ASCS location: s4hana18",
        "Initial ERS location: s4hana17",
        "Final ERS location: s4hana17",
        "HA Action taken: ASCS Failover",
        "ASCS/ERS separation maintained: YES",
        "==============================================="
    ]
}

PLAY RECAP ***************************************************************************
s4hana17                   : ok=136  changed=4    unreachable=0    failed=0    skipped=10   rescued=0    ignored=0   
s4hana18                   : ok=119  changed=0    unreachable=0    failed=0    skipped=6    rescued=0    ignored=0   
s4hana19                   : ok=124  changed=1    unreachable=0    failed=0    skipped=1    rescued=0    ignored=0 rescued=0    ignored=0

Refer to the README.md file of each test role for more details about the tests. The README.md for each test role can be found in the following location of the same directory: ansible_collections/sap/cluster_qa/roles/<test-name>/README.md.

Authors: Amir Memon

SAP HANA System Replication Takeover and Client Connectivity

2026-02-25T18:00:00.017000+01:00

Documented Scope and Architectural Responsibilities

Introduction
This article summarizes publicly available SAP documentation and reflects the documented behavior of SAP HANA System Replication at the time of writing.

SAP HANA System Replication is widely used in productive landscapes to improve availability and to enable takeover to a secondary system.

In operational environments it may be observed that after a takeover the database role changes while application connectivity depends on the surrounding client access path.

This article does not provide configuration or implementation guidance.
Instead, it consolidates SAP documentation to clarify the documented scope of System Replication and the responsibilities described in related high-availability integration documentation.

The purpose is to align operational expectations with documented behavior.

Documented Purpose of System Replication
According to SAP documentation, SAP HANA System Replication continuously transfers database changes from a primary system to a secondary system and supports a role change (takeover).

SAP documentation describes different consistency and data-loss characteristics depending on the selected replication mode.

Sources
SAP HANA System Replication Guide
https://help.sap.com/doc/c81e9406d08046c0a118c8bef71f6bdc/2.0.07/en-US/SAP_HANA_System_Replication_Guide_en.pdf
SAP Help Portal – System Replication Overview
https://help.sap.com/docs/SAP_HANA_PLATFORM/6b94445c94ae495c83a19646e7c3fd56/676844172c2442f0bf6c8b080db05ae7.html

After takeover, the secondary system becomes the new primary database system from the database perspective.
The documentation therefore describes System Replication primarily in terms of database availability and data consistency.

Documented Behavior During Takeover
During a takeover, SAP HANA changes the database role of the participating systems.

According to SAP documentation, once the takeover command completes, the former secondary system becomes the new active primary system. The documentation further describes that the system replays the last transaction logs and starts to accept queries.

The System Replication Guide documents the database role change and service startup behavior.
The documentation describes database state and service availability, while network routing and client access handling are described separately in high-availability integration documentation.

Actions After Takeover in HA Integration Documentation
SAP documentation describes integration mechanisms that allow external components to react to a role change.

SAP HANA provides a HA/DR provider hook that enables external high-availability components to execute actions after a takeover.

Documented examples include service endpoint switching or cluster-related actions.

Sources
Implementing SAP HANA HA/DR Providers
https://help.sap.com/docs/SAP_HANA_PLATFORM/6b94445c94ae495c83a19646e7c3fd56/1200ab8ef0c54c54be2d0e7f5327f7ed.html
Installing and Configuring HA/DR Provider Hook
https://help.sap.com/docs/SAP_HANA_PLATFORM/6b94445c94ae495c83a19646e7c3fd56/2962efcfdd6740689d5705bdabe9a2d5.html

The HA/DR integration documentation describes coordination with external high-availability mechanisms following a database role change.

Database Role and Client Connectivity in Documented Architecture
System Replication changes the database role.
Client connectivity depends on the access path used by the application.

Typical environments may use infrastructure components such as:

virtual IP redirection
DNS-based redirection
cluster management software
application-server-side failover configuration

SAP System Replication documentation describes the database role and persistence synchronization, while HA integration documentation describes coordination with external availability mechanisms.
These documents therefore address different parts of the overall availability architecture.

Interpreting Connection Failures After Takeover
In operational situations, a takeover may be completed from the database perspective while application connections still fail if the client access path still points to the former primary host or has not yet been updated by surrounding infrastructure.

From a documentation perspective, this reflects different layers of the availability architecture rather than a single mechanism.

SAP documentation separates:

database role change (System Replication)
external availability coordination (HA integration)

Reference
SAP Note 1999880 - FAQ: SAP HANA System Replication

Relationship to High Availability Architecture
SAP HANA high availability consists of multiple documented components, including:

System Replication
cluster or HA integration
infrastructure routing mechanisms

System Replication addresses database availability.
Other components coordinate access paths and service endpoints in the surrounding infrastructure.

Conclusion
SAP HANA System Replication provides database availability and role takeover capabilities, while consistency and potential data loss characteristics depend on the configured replication mode.

Client connectivity depends on surrounding high-availability and infrastructure mechanisms that are documented separately from System Replication.

Understanding the documented scope of these components helps distinguish database state from client access routing and supports clearer troubleshooting during incidents.

For implementation or configuration guidance, always follow the official SAP documentation and SAP recommendations.

Disclaimer
This article summarizes publicly available SAP documentation and does not replace official SAP configuration, sizing, or implementation guidance. This article describes documented architectural behavior and is not an SAP support statement.

Avoiding HANA Connection Exhaustion: A Practical Guide to Hikari & Multi-Tenant Scaling

2026-02-26T09:53:05.868000+01:00

Introduction

Modern SAP applications running on SAP BTP with CAP Java and SAP HANA rely heavily on efficient database connection management.

Opening a new database connection for every request is expensive:

It increases latency
It consumes database resources
It reduces scalability

To solve this, we use connection pools.

In this blog, we’ll walk through how connection pooling works in CAP Java, what can go wrong in multi-tenant systems, and how to configure it correctly to avoid production outages.

What is a Connection Pool?

A connection pool is a cache of reusable database connections maintained by your application.

open connection → execute query → close connection

becomes:

borrow connection → execute query → return to pool

Benefits

Faster response times
Reduced database load
Reuse of expensive DB sessions
Better scalability

How CAP Java Uses Connection Pools

In CAP Java:

Database access uses JDBC connection pooling via Spring Boot
The default pool implementation is HikariCP
Each application instance maintains its own connection pool

This means that if your application is scaled horizontally, the number of connections increases with the number of instances.

Why This Becomes Critical in Production

In the basic mode, the HANA database is provisioned with 2 vCPUs, which allows roughly 1000 concurrent connections.

Now consider a typical CAP multi-tenant setup:

Multiple services
Multiple tenants
Multiple app instances (autoscaling)
Each instance maintaining its own pool

The total DB connections can grow very quickly if not controlled.

Multi-Tenant Scaling Problem

Without optimization, the connection count grows like this:

connections = instances × tenants × pool size

Example:

4 tenants
3 application instances
default pool size = 10

4 × 3 × 10 = 120 connections

Now multiply this across multiple services and sidecars, you can quickly reach the HANA limit.

The Solution: Shared Pools with combinePools

CAP Java provides a very important optimization:

cds:
  multi-tenancy:
    datasource:
      combinePools:
        enabled: true

This ensures that instead of creating pools per tenant, the application uses:

One shared connection pool per HANA database per application instance

Now the formula becomes:

connections = instances × pool size

This makes connection usage predictable and scalable.

HikariCP Configuration (Production)

A production-ready CAP Java configuration looks like this:

cds:
  multi-tenancy:
    datasource:
      combinePools:
        enabled: true
      hikari:
        maximum-pool-size: 40
        minimum-idle: 10
        idle-timeout: 600000
        max-lifetime: 3300000

Parameter Description

maximumPoolSize	Maximum connections per instance
minimumIdle	Warm connections kept ready
idleTimeout	Idle connection eviction time
maxLifetime	Maximum lifetime before connection refresh

How to Size Your Pool Correctly

Use this simple guideline:

maximumPoolSize ≈ concurrent DB requests per instance

And always validate:

instances × maximumPoolSize < HANA DB connection limit

For example:

3 instances
pool size 40

3 × 40 = 120 connections

Well within a 1000 connection limit.

Monitoring & Observability

You should continuously monitor:

Active DB connections
Idle connections
Pool utilization
Total HANA connection usage

Set alerts at 70–80% of DB capacity.

Common Pitfalls

Not enabling combinePools in multi-tenant apps
Oversizing connection pools
Multiple services sharing same HANA DB without coordination
No monitoring of DB connection usage
Assuming default values are safe for production

Summary

Area Recommendation

Connection Pool	Use HikariCP
Multi-Tenancy	Enable combinePools
Pool Sizing	Based on concurrency
Monitoring	Mandatory
Scaling	Validate against DB limits

Final Thoughts

Connection pooling is often invisible, until it fails in production.

With the right configuration in CAP Java and SAP HANA, you can:

avoid outages
improve performance
scale safely in multi-tenant environments

If you are running CAP Java in production, take a few minutes to review your pool configuration, it can prevent your next incident.

References

Understanding SAP HANA Disaster Recovery - Technical Scope, Distance, and Data Protection Boundaries

2026-03-01T18:00:00.018000+01:00

Introduction
SAP HANA provides documented mechanisms to support disaster recovery (DR) scenarios in which an entire site, data center, or availability zone becomes unavailable.

From a technical perspective, disaster recovery addresses failure domains that exceed the scope of local high-availability mechanisms. Its primary objective is to restore database operations after site-level outages, while preserving data consistency according to architecturally defined recovery objectives.

Disaster recovery in SAP HANA is implemented through a combination of documented mechanisms such as system replication, backup-based recovery, distance-aware configurations, and controlled takeover procedures.

This article consolidates SAP-documented SAP HANA disaster-recovery mechanisms into a single technical view.

Its purpose is to make scope, distance considerations, and data-protection boundaries explicit, supporting sound DR architecture design and precise technical discussions.

All statements are derived from official SAP Help Portal documentation.

1. Disaster Recovery in SAP HANA - Technical Definition
In SAP HANA documentation, disaster recovery refers to mechanisms that enable database availability after site-level failures, such as:

complete data-center outages
regional infrastructure failures
loss of connectivity between sites

Although some SAP HANA mechanisms such as system replication can be used for both high availability and disaster recovery scenarios, disaster recovery addresses broader failure domains such as site-level outages.

This separation of objectives is consistently reflected in SAP HANA administration and architecture documentation.

Key boundary
In this context, disaster recovery focuses on broader failure domains such as site-level outages rather than isolated single-component failures.

2. Failure Domains and Distance Considerations
A defining characteristic of disaster recovery is physical and logical separation between systems.

From a technical standpoint, DR architectures introduce:

network latency
increased failure isolation
replication latency considerations

Distance directly influences:

feasible replication modes
achievable Recovery Point Objective (RPO)
achievable Recovery Time Objective (RTO)

Distance and latency influence the choice of replication mode and DR architecture.

Key boundary
Distance and latency are key architectural design considerations that influence the feasible replication mode and the achievable recovery characteristics.

3. SAP HANA System Replication as a DR Mechanism
3.1 Technical Role in Disaster Recovery
SAP HANA System Replication (SR) is a primary building block for replication-based disaster recovery architectures.

In DR scenarios:

primary and secondary systems are deployed in separate sites
replication protects against site-level failures
takeover is initiated when the primary site is unavailable

System replication continuously transfers data changes from the primary system to the secondary system, maintaining a replica of the current database state.

SAP Help Portal reference
SAP HANA System Replication - Overview

3.2 Replication Modes and Distance
SAP HANA system replication supports synchronous and asynchronous replication modes.
From a disaster-recovery perspective:

Synchronous replication
is typically constrained by distance and latency and is therefore limited to shorter distances
Asynchronous replication
is commonly used for long-distance DR scenarios and accepts some replication lag

The selected replication mode directly influences:

potential data loss during failover
achievable RPO values

This behavior is an inherent consequence of documented SR semantics.

SAP Help Portal reference
SAP HANA System Replication - Replication Modes

Key boundary
Replication mode determines data-loss characteristics, not historical recovery capability.

4. Multitier and Multitarget System Replication
SAP HANA supports multitier system replication in chained setups as well as multitarget system replication for replication to more than one secondary system.

In disaster-recovery architectures, multitier replication can be used to:

combine high availability and disaster recovery objectives
separate near-site and far-site replicas
increase isolation between failure domains

Each replication tier operates within documented technical constraints, including supported replication paths and takeover rules.

SAP Help Portal reference
SAP HANA Multi-Tier System Replication
Supported Replication Modes Between Systems

Key boundary
Multitier replication can support multi-site architectures and extended failure-domain separation; however, it does not replace backup-based recovery mechanisms.

5. Backup-Based Disaster Recovery
5.1 Technical Model
In addition to replication-based approaches, disaster recovery can be implemented using backup-based recovery.

In this model:

data and log backups are stored in a remote or isolated location
recovery is performed by restoring backups at the target site
point-in-time recovery is possible, depending on backup and log availability

Backup-based DR provides:

strong isolation between sites
access to historical restore points

It typically involves longer recovery times compared to replication-based DR.

SAP Help Portal reference
Planning Your Backup and Recovery Strategy

5.2 Replication-Based vs. Backup-Based DR
From a technical standpoint:

Replication-based DR
prioritizes continuity with limited data loss
Backup-based DR
prioritizes isolation and historical recovery

Replication-based and backup-based approaches can be combined to address different availability and recovery requirements.

6. Disaster Recovery vs. High Availability
Although both HA and DR may use system replication, their objectives and assumptions differ fundamentally.

High availability
handles component-level failures within a site
Disaster recovery
handles complete loss of a site or failure domain

Disaster recovery designs emphasize:

physical separation
controlled takeover procedures
explicitly defined data-loss boundaries

Key boundary
DR is validated against site-level failure scenarios, not host failures.

7. RPO and RTO in Disaster Recovery Context
In SAP HANA DR architectures:

RPO
is influenced by replication mode or backup frequency
RTO
is influenced by takeover procedures, recovery execution, and operational readiness

SAP HANA provides the technical mechanisms required to support different RPO/RTO targets.
Actual values are derived from architecture and configuration, not configured as product parameters.

8. Design Boundaries and Architectural Implications
From a disaster-recovery architecture perspective:

no single mechanism covers all site-level failure scenarios
replication-based DR trades isolation for continuity
backup-based DR trades speed for historical recovery
distance and latency impose hard technical constraints

Understanding these documented boundaries is essential for realistic DR expectations and technically sound architecture design.

Summary
SAP HANA provides documented technical mechanisms to support disaster-recovery scenarios involving site-level failures.

Disaster recovery is not defined by a single feature, but by the combination of system replication, backup mechanisms, distance-aware configuration, and controlled recovery procedures.

By understanding the documented scope, distance considerations, and data-protection boundaries of SAP HANA disaster-recovery mechanisms, architectures can be designed with clear, realistic, and technically defensible recovery expectations.

BI Remediation Checks during ECC to S/4HANA Migration

2026-03-02T07:22:03.809000+01:00

As SAP prepares to end support for ECC by 2027, many organizations are making the crucial move to SAP S/4HANA. The shift can unlock improved capabilities and performance, but it can also introduce changes that ripple through the BI landscape if we don’t plan for them early.

From a BI perspective, there are multiple aspects to consider during the transition. This blog focuses on the BI remediation checks that help protect data trust, reporting, and decision-making continuity during and after the migration.

Why do we need BI remediation?
The migration from SAP ECC to SAP S/4HANA introduces significant changes to the underlying data structures. These changes impact how data is extracted, modelled, and consumed in the BI system. Because of that, reporting remediation becomes critical to ensure data consistency, accuracy, and continuity in analytics after the migration.

With remediation, we ensure:

BI models and reports remain functional and aligned with the new S/4HANA structures
Disruption is minimized for business operations and reporting processes

Data model simplifications in S/4HANA (what changes underneath)
When SAP S/4HANA was first introduced, the SAP ERP 6.0 data model underwent major restructuring. Redundancies were removed, and many smaller tables were consolidated into fewer, larger ones.
Figure 1 can help visualize the concept and why “what worked in ECC” may not behave the same way in S/4HANA.

FIGURE 1

Impact of S/4HANA migration on the BI system (what to check)

1) Impact on BW Datasources / Extractors
If you run SAP BW, one of the first questions is:
“After ECC becomes S/4HANA, will my extractors and DataSources still be supported and behave the same way?”
SAP note 2500202 provides an Excel sheet produced by SAP that details the impact of S/4HANA on BW Datasources. This sheet is periodically updated. The same note also contains a PPT that guides us on how to interpret and use the Excel list.

A proven approach is:

Inventory your active extractors (e.g., via ROOSOURCE).
Identify which are truly used (process chains / BI statistics where available)
Cross-check them against the SAP note 2500202 list and categorize outcomes (no change / restricted / obsolete / alternative required)

2) Tables with Replacement Objects (don’t rely on “classic tables” blindly)
If, as part of your BI remediation strategy, you plan to continue using certain classic ECC tables, for example, relying on MARD instead of adopting the newer MATDOC structure, you must depend on the corresponding Replacement Object rather than accessing the table directly.
With the data model simplification in SAP S/4HANA, many traditional ECC tables are no longer the authoritative “single source of truth.” In many cases, data is aggregated or derived via CDS views that join and enrich information from multiple sources. Because of this shift, it becomes essential to transition from reading classic tables (such as MARC, MARD, and others) to using their designated Replacement Objects, ensuring that analytics and integrations receive accurate, up‑to‑date data.

In Native HANA, it is advisable to verify whether a table has a replacement object or whether it can still be used directly.
In SAP BW, the datasource typically absorbs most of the underlying data model changes. However, in hybrid scenarios, particularly when calculation views are built on ECC base tables or when custom datasources built on ECC base tables, it becomes essential to thoroughly assess the impact on the affected tables.

Figure 2 shows an example for table MARD. NSDM_E_MARD serves as the replacement view for the classic MARD table. Although data continues to reside in MARD, those entries generally represent older documents migrated to S/4HANA or new S/4HANA records that are stored without measure fields. To obtain the full set of information (both characteristics and measures) for transactions created in S/4HANA, you must use the replacement object, in this case NSDM_E_MARD, rather than relying on MARD directly.

FIGURE 2

3) Replacement of obsolete tables
Some ECC tables are replaced or made obsolete, and direct reporting on such tables is no longer recommended or supported. Examples tables include:

KONV, VBUP, VBUK, EIKP, EIPO

Obsolete tables may still exist, but they can behave as compatibility views. For accuracy and performance, they should be switched to transparent tables where applicable.

SAP Note 3250204: describes about the replacement table PRDC_ELEMENTS for KNOV table.
SAP Note 2198647: talks about tables VBUP, VBUK and SD data model changes.
SAP Note 3482527: covers about new S/4HANA foreign trade.

4) Key fields changed (primary key shifts can break pipelines)
In S/4HANA, the primary keys of some tables have changed compared to ECC. Some examples are:

VBFA
FAAV_ANLC

These changes directly affect how data is stored, identified, and retrieved. That can break:

Joins in HANA models
Deduplication logic
Delta logic or key-based transformations in BW

Action

Review where these tables are used in extraction/transformation layers
Re-check join conditions, uniqueness assumptions, and surrogate key logic
Re-test deltas and aggregations end-to-end

5) Field length extensions (small change, big downstream effect)
Some fields were extended in S/4HANA to support broader functional requirements, enhanced integration needs, and future extensibility.
One such example is:

VBAP-VBTYP (Char1) Document Category replaced by VBAP-VBTYPL (Char4)

SAP Note 2495386 details the BW datasource that uses this object.

Another major example:

Material Number (SAP Note 2215424) changed from char(18) to char(40)

A key design decision regarding material numbers is whether your organization will:

Adopt a true 40‑character material number, introducing a new business meaning, or
Retain the existing 18‑character logic, while simply expanding the technical field length to 40 characters.

The chosen approach directly influences how material numbers flow through your BI data models and determines the adjustments required in downstream reporting and integrations.

6) Centralized Master Data via Business Partner (Customer/Vendor integration)
SAP fundamentally changed how master data is managed by introducing a unified, centralized model for customers, vendors, and business partners. This replaces the traditional ECC siloed structures with a harmonized object: the Business Partner (BP).
It’s not a “technical-only” change, it’s a semantic change in how master data is governed
Please check the SAP Note 2265093 note for more information.

7) Redirecting ECC Virtual Tables in HANA Calculation Views
When your BI models use calculation views that reference ECC base tables, these views must be updated after the migration to ensure they point to the corresponding objects in the S/4HANA system. Redirecting them is essential to prevent calculation view failures and to avoid inconsistent or incorrect analytical results after the migration. Please refer the blog for mitigation of this impact.

Final thoughts: ECC to S/4HANA migration is not only an ERP transformation, but also a BI continuity challenge. The safest approach is to treat BI remediation as a structured workstream, validating extractors, replacing obsolete dependencies, and proactively addressing table/key/field changes before they surface as production reporting issues.

If this post helps you structure your BI remediation checks, please consider sharing your own experiences or additions in the comments.

Migrating to SAP HANA Cloud: What Actually Gets Better (Part 1 of 2)

2026-03-02T16:18:10.052000+01:00

SAP COMMUNITY | TECHNICAL GUIDE | SAP HANA CLOUD | PART 1 OF 2

Migrating to SAP HANA Cloud: What Actually Gets Better

Part 1 of 2: The genuine capabilities that open up when you move from on-premise HANA to HANA Cloud — HDI containers, SAP AI Core, and the BTP application ecosystem.

By Deepa Dorairaj | SAP AI Solution Architect | HANA Data Engineering | Published 2026

This is Part 1 of a two-part series on SAP HANA Cloud migration. Part 1 covers what genuinely improves when you move to the cloud. Part 2 covers the pain points and what to watch out for. Both perspectives are necessary for an honest picture of the migration journey.

TL;DR: Moving from on-premise HANA to HANA Cloud is not just an infrastructure change. It unlocks three capabilities that are difficult or impossible to replicate on-premise: a modern database development workflow through HDI containers and Git, access to SAP AI Core for production-grade AI deployments on your SAP data, and a rich application ecosystem through BTP that connects your HANA data to S/4HANA, Fiori, Integration Suite, and beyond.

Why This Article Exists

Most migration content falls into one of two traps: vendor-produced material that glosses over the real challenges, or war stories that focus exclusively on what broke. The honest picture is more nuanced than either.

Part 2 of this series covers the friction — the tooling gaps, scripting compatibility issues, and security model changes that will slow your team down. You should read that too, especially if you are in the planning phase.

But this article is about the other side of that equation: what genuinely gets better. Not what SAP's marketing says gets better — what actually changes for the teams doing real work in real HANA Cloud environments. The three areas covered here represent capabilities that were either unavailable, severely limited, or operationally painful in on-premise. In HANA Cloud they become first-class, production-ready patterns.

IMPROVEMENT 1 OF 3

HDI Containers: A Modern Database Development Workflow

What HDI Actually Is — and Why It Changes Everything

The HANA Deployment Infrastructure (HDI) container model is the single biggest workflow improvement that HANA Cloud brings for development teams. On-premise HANA, most teams deploy database objects directly against schemas — DDL scripts run manually or through transport management, objects exist in a shared schema that everyone touches, and version control is an afterthought bolted on top of an inherently stateful system.

HDI flips this model entirely. Database objects are defined as source artifacts — .hdbtable, .hdbview, .hdbprocedure files — that live in a Git repository alongside application code. The HDI container is deployed from those artifacts, and the deployment engine handles dependency resolution, delta deployment, and environment isolation automatically.

The result is that your database development workflow starts to look like your application development workflow: version controlled, reviewable, deployable, and reproducible.

Git Integration: Version Control as a First-Class Citizen

On-premise HANA development rarely had true version control for database objects. Teams used SAP Transport Management for moving objects between landscapes, but transport management is not version control — it doesn't give you branching, pull requests, history, or the ability to roll back to any point in time with confidence.

HANA Cloud's HDI model integrates directly with Git repositories through SAP Business Application Studio and the HANA Cloud toolchain. Every database object is a file in a repository. Changes go through the same review process as application code. Deployments are triggered from specific commits or branches.

Feature branches for database changes — developers work on isolated branches without affecting the shared schema, exactly as they would for application code.
Pull request reviews for schema changes — DDL changes are reviewed by peers before deployment, catching errors before they reach development or production environments.
Full commit history for every database object — the ability to see exactly what changed, when, and who made the change, with the ability to revert to any previous state.
Branch-based environment promotion — development, QA, and production environments are deployed from specific branches, making the promotion process auditable and repeatable.

This is the change that development teams feel most immediately after migration. The frustration of untraceable schema changes, overwritten objects, and transport conflicts disappears when database artifacts live in Git like everything else.

Environment Isolation and Team Collaboration

On-premise HANA landscapes typically have a development system, a QA system, and a production system — three full HANA instances that are expensive to maintain and still don't provide true isolation between individual developers working simultaneously.

HDI containers provide lightweight, isolated development environments within a single HANA Cloud instance. Each developer or team can have their own container — a complete, isolated copy of the database schema — without requiring a separate HANA system. Changes in one container have zero impact on another.

Developer containers — each developer works in their own isolated container, testing changes without risk of interfering with colleagues.
Feature containers — a complete isolated environment for a specific feature or project, spun up and torn down as needed.
Automated dependency management — the HDI deployment engine resolves object dependencies automatically. If a view depends on a table that doesn't exist yet, the deployment fails with a clear error rather than creating an inconsistent state silently.
Simplified landscape transport — promoting changes from development to QA to production means deploying the same Git artifacts to different containers, not managing SAP transport requests.

Cleaner Deployment and Versioning

One of the most underappreciated HDI benefits is idempotent deployment. On-premise, re-running a DDL script against an existing object requires DROP and recreate logic, ALTER statements, or manual intervention. HDI's deployment engine compares the desired state (your artifact files) against the current state (what's in the container) and applies only the necessary changes — automatically, safely, and repeatably.

-- On-premise: manual drop and recreate required
DROP TABLE my_schema.my_table;
CREATE TABLE my_schema.my_table (id INT, name VARCHAR(100));

-- HDI: define once in .hdbtable artifact, deploy repeatedly
-- HDI engine handles delta — no manual DROP/CREATE needed
COLUMN TABLE my_table (id INT, name NVARCHAR(100));

Capability	What It Unlocks
Git version control	Full history, branching, pull request reviews, and rollback for every database object
Developer isolation	Individual HDI containers eliminate shared schema conflicts between developers
Automated dependency resolution	HDI deployment engine manages object dependencies — no manual sequencing of DDL scripts
Idempotent deployment	Deploy the same artifacts repeatedly without manual DROP/CREATE logic
Landscape transport	Git-based promotion replaces SAP transport requests for database objects
Feature branching	Database changes developed and tested in isolation before merging to main branch

IMPROVEMENT 2 OF 3

SAP AI Core: Production-Grade AI on Your SAP Data

Why AI Core Changes the Enterprise AI Equation

On-premise HANA has machine learning capabilities — PAL (Predictive Analysis Library) and APL (Automated Predictive Library) provide a solid foundation for classical ML workloads that run close to the data. But these capabilities have a fundamental limitation: they are designed for traditional ML patterns and do not provide a path to deploying modern foundation models, RAG systems, or generative AI against your SAP data in a production-ready, governed way.

SAP AI Core, accessible through BTP and natively integrated with HANA Cloud, is a different category of capability entirely. It is a managed AI runtime that handles the infrastructure, scaling, and lifecycle management of AI workloads — allowing you to focus on the AI logic rather than the infrastructure that runs it.

For SAP architects and data engineers, the critical insight is this: AI Core is where your HANA Cloud data becomes the foundation for genuinely intelligent enterprise applications, not just analytical ones.

Vector Embeddings and RAG for SAP Data

HANA Cloud includes native vector storage capabilities — the ability to store and query vector embeddings directly in the database alongside your structured SAP data. This is the foundational capability for Retrieval-Augmented Generation (RAG) systems that can answer questions about your SAP landscape using live data rather than static training.

The architecture this enables is significant: instead of sending sensitive SAP business data to an external AI service for every query, you store embeddings of your SAP content in HANA Cloud's vector store and retrieve only the relevant context for each query. The foundation model gets the context it needs without your raw SAP data leaving your controlled environment.

-- Create a vector store table in HANA Cloud
CREATE TABLE sap_knowledge_embeddings (
  id          BIGINT PRIMARY KEY,
  content     NCLOB,
  source_type NVARCHAR(100),
  embedding   REAL_VECTOR(1536)
);

-- Semantic similarity search against SAP knowledge base
SELECT TOP 5 id, content, source_type,
  COSINE_SIMILARITY(embedding, TO_REAL_VECTOR(?)) AS score
FROM sap_knowledge_embeddings
ORDER BY score DESC;

This pattern — storing SAP process documentation, BAPI descriptions, master data definitions, and business rules as searchable embeddings — is the foundation of a genuine enterprise AI assistant that knows your specific SAP landscape, not just generic SAP knowledge.

Connecting Foundation Models via AI Core APIs

AI Core provides governed, auditable access to foundation models — including SAP's own generative AI models and third-party models through the Generative AI Hub — with the enterprise controls that on-premise environments cannot provide: usage tracking, cost allocation, rate limiting, and compliance logging built in.

The connection between HANA Cloud data and AI Core models enables patterns that were genuinely not achievable on-premise:

Intelligent document processing — extracting structured data from unstructured SAP documents (purchase orders, invoices, contracts) using foundation models and storing results directly in HANA Cloud tables.
Natural language interfaces to SAP data — allowing business users to query HANA Cloud analytics in plain language, with AI Core translating natural language to SQL and grounding responses in live data.
Automated exception handling — AI agents that monitor SAP business processes, identify anomalies in HANA Cloud data, and take corrective actions through SAP APIs without human intervention.
Fine-tuned models on SAP data — training or fine-tuning foundation models on your organization's specific SAP data patterns through AI Core's managed training infrastructure.

Grounding AI in Live SAP Business Data

The most powerful aspect of the HANA Cloud and AI Core combination is the ability to ground AI outputs in live, governed SAP business data. Generic AI models hallucinate when asked about your specific business — your customer master, your material catalog, your financial postings. Grounded models, retrieval systems, and tool-calling agents that query HANA Cloud directly produce outputs that are accurate, current, and auditable.

The architecture pattern that is emerging as the standard for enterprise SAP AI: HANA Cloud as the data foundation and vector store → AI Core as the model runtime and orchestration layer → SAP Business AI applications as the user-facing interface. Each layer does what it is best at, and the combination is genuinely more capable than any of its parts.

Capability	What It Unlocks
Native vector storage	Store and query embeddings directly in HANA Cloud alongside structured SAP data
RAG on SAP data	Build retrieval systems that answer questions using live SAP content without external data exposure
Foundation model access	Governed, auditable access to SAP and third-party foundation models through Generative AI Hub
Fine-tuning capability	Train models on your specific SAP data patterns through AI Core's managed infrastructure
Grounded AI outputs	AI responses anchored to live HANA Cloud data — accurate, current, and auditable
Intelligent automation	AI agents that monitor SAP processes and act on HANA data without human intervention

IMPROVEMENT 3 OF 3

The BTP Application Ecosystem: Connecting HANA Cloud to Everything

From Database to Platform

On-premise HANA is primarily a database — an exceptional one, but a database. Applications connect to it, consume its data, and leave. The integration patterns are well established but fundamentally passive: HANA stores and processes data, applications request it.

HANA Cloud on BTP is part of a platform ecosystem that changes this dynamic. The connections between HANA Cloud and the broader SAP and non-SAP landscape are richer, more standardized, and more capable than what on-premise environments support. Your HANA Cloud instance is not just a database — it is a data foundation for a platform that can build, connect, and deploy enterprise applications at a speed that on-premise infrastructure cannot match.

Connecting HANA Cloud to S/4HANA and ECC

The integration between HANA Cloud and SAP's core ERP systems — S/4HANA and ECC — is one of the most practically valuable improvements for organizations running mixed landscapes. On-premise, cross-system data access required custom RFC connections, ALE configurations, or ETL pipelines that were expensive to build and fragile to maintain.

Through BTP's Integration Suite and HANA Cloud's federation capabilities, S/4HANA and ECC data becomes accessible in HANA Cloud with significantly less custom integration work. This enables analytical and AI use cases that span transactional ERP data and the broader data landscape without requiring full data replication.

Real-time operational reporting on S/4HANA data through HANA Cloud without impacting ERP system performance.
Cross-system analytics combining ERP data with data from non-SAP sources in a single HANA Cloud analytical layer.
AI and ML workloads that consume live ERP data through governed, auditable connections rather than stale data extracts.

Building Fiori and BTP Applications on HANA Cloud Data

SAP Business Application Studio — the cloud-based IDE that replaces the Eclipse-based tooling of on-premise development — provides a first-class development environment for building Fiori applications, CAP (Cloud Application Programming model) services, and full-stack BTP applications directly on top of HANA Cloud data.

The CAP model in particular represents a significant improvement over traditional on-premise application development patterns. CAP services expose HANA Cloud data through OData and REST APIs with built-in authorization, query handling, and SAP Fiori integration — dramatically reducing the boilerplate code required to build enterprise-grade applications on SAP data.

-- CAP service definition exposing HANA Cloud entity
service AnalyticsService {
  @readonly entity SalesOrders as
    SELECT from db.SalesOrders {
      order_id, customer, amount, status,
      created_at, region
    } where status = 'ACTIVE';
}

-- Automatically generates OData V4 endpoint
-- with filtering, sorting, and pagination built in

OData and REST API Exposure

Exposing HANA Cloud data through standardized OData and REST APIs — consumable by Fiori applications, third-party tools, mobile applications, and external systems — is significantly more streamlined on BTP than on-premise. The CAP framework handles the API layer, and BTP's API Management provides governance, rate limiting, and monitoring for those APIs across all consumers.

For organizations building an enterprise data mesh or API-first data strategy, this is one of the most compelling HANA Cloud advantages — your HANA data becomes a governed, discoverable, consumable API product rather than a database that specific applications happen to know how to query.

SAP Integration Suite and Side-by-Side Extensibility

SAP Integration Suite on BTP provides pre-built integration flows between HANA Cloud and the broader SAP and non-SAP ecosystem — Salesforce, Microsoft, ServiceNow, and hundreds of other systems — without custom middleware development. On-premise integration required custom development, dedicated middleware servers, and ongoing maintenance of bespoke integration code.

Side-by-side extensibility — the BTP pattern for extending S/4HANA and other SAP systems without modifying core — relies on HANA Cloud as its data persistence layer. Extension applications built on BTP consume S/4HANA data through governed APIs, process and store results in HANA Cloud, and surface insights back to SAP users through Fiori — all without touching the core ERP system.

Side-by-side extensibility is the pattern that makes HANA Cloud most valuable for organizations that need to innovate on top of S/4HANA without the risk and cost of core modifications. HANA Cloud is the data foundation that makes this pattern production-ready.

Capability	What It Unlocks
S/4HANA and ECC integration	Cross-system analytics and AI workloads on live ERP data without custom RFC development
CAP application development	Build Fiori and full-stack BTP applications on HANA Cloud data with dramatically less boilerplate
OData and REST APIs	Expose HANA Cloud data as governed, discoverable API products consumable by any client
SAP Integration Suite	Pre-built integration flows to SAP and non-SAP systems without custom middleware
Side-by-side extensibility	Extend S/4HANA through BTP applications backed by HANA Cloud without core modifications
BTP ecosystem	Access to the full BTP service catalog — AI, analytics, integration, automation — from a single platform

The Honest Summary

The three improvements covered in this article are not incremental upgrades to what on-premise HANA already does. They are qualitatively different capabilities that change what is possible for development teams, data engineers, and enterprise architects working with SAP data.

HDI containers with Git integration bring database development into the modern software development workflow — version controlled, isolated, reviewable, and reproducible. SAP AI Core unlocks production-grade AI on your SAP data — RAG systems, foundation model access, intelligent automation — that simply cannot be built with the same reliability and governance on-premise. And the BTP application ecosystem transforms HANA Cloud from a database into a platform foundation for the full range of SAP and cross-system enterprise applications.

None of this makes the migration easy. Part 2 of this series is honest about the friction — and there is real friction. But the capabilities waiting on the other side of that friction are substantial enough to make the journey worth taking.

Part 2 of this series covers the pain points in detail: HANA Studio to Cloud Central tooling gaps, stored procedure compatibility issues, and security model changes that will catch your team off guard. Read both before finalizing your migration plan.

Good to know: "HASH Partitioning" vs "RANGE Partitioning" in context of SAP HANA database

2026-03-04T08:09:16.242000+01:00

last updated: 2026-03-04

Why was this blog post created?

The area of partitioning is a very complex topic when it comes to the SAP HANA database (in case of all kind of database). The blog post aim is to summarize the facts and explain the different types on high level (pro, cons). Deeper understanding is available in the referenced documents. A high-level understanding is a critical step in order to choose the correct and most effective partitioning in your actual need/situation.

What are the differences?

The following tables contains the differences. Please note that this is a general categorization, both HASH and RANGE type of partitioning method are more complex and intricate than this table mentions. The table uses a simplified categorization.

	HASH	RANGE
Amount of data	Typically less data.	Typically huge amount of data.
When to use?	Solves the 2 billion limit.	Next to partitioning (2 billion limit) there are further requirements (e.g. performance needs).
Focus	Focus is on the partitioning itself.	Focus is not only the partitioning but further requirements also.
Implementation	Easier setup.	Preparation and planning necessary, more complex task.
Implementation Type	Mainly technical task. Planning is also necessary and depending on selectivity.	More deeper and detailed planning & technical tasks.
Example	3281773 - What cause the non uniform data distribution in HASH partitioned table in SAP HANA?	2289491 - Best Practices for Partitioning of Finance Tables
Involvement	Mainly a DBA task but data owner participation can be required.	Need further teams (e.g. application knowledge).

Where can I find the most important information about the topic of SAP HANA partitioning?

SAP Knowledge Base Article(s):

2044468 - FAQ: SAP HANA Partitioning
3146645 - What is the best approach in partitioning tables on SAP HANA?
3307500 - How to decide which partitioning type and column(s) should been used to partition a table in SAP HANA?

SAP Community Article:

Collected information regarding partitioning in SAP HANA (with examples)

Complexity

The complexity is also indicated by the fact that all relevant information can be found in the central documentation (2044468), however, based on experience, further explanations are needed for the interpretation and use of this documentation, which is why additional supporting documents were created (e.g. 3146645 or 3307500). Furthermore, the Blog listed above was created to present specific examples and to compile the supporting documents.

What is the conclusion?

Understanding is the key element to do the proper partitioning of the table based on different requirements. This document explains a top down approach and gives an overview about the topic and the necessary documentations.

Do you have further questions?

Please do not hesitate to contact me if you have question or observation regarding the article.
Q&A link for SAP HANA: https://answers.sap.com/tags/73554900100700000996

Contribution

If you find any missing information belongs to the topic, please let me know. I am happy to add the new content. My intention is to maintain the content continuously to keep the info up-to-date.

Release Information

Release Date	Description
2026.03.04	First/initial Release of the SAP Blog Post documentation (Technical Article).

Enterprise AI in Action: SAP Joule Delivers Where Others Don’t

2026-03-04T12:33:21.937000+01:00

AI is becoming part of everyday enterprise work, from drafting emails to designing complex processes. While ChatGPT and Gemini can generate functional specs and ABAP code, the real question is: “Why use SAP Joule?”
The difference is context – Joule is embedded in S/4HANA, understanding system configuration, business rules, workflows, and live data to deliver outputs that are accurate, compliant, and ready for real-world use. Unlike general-purpose AI, it doesn’t just produce well-written text; it grasps the technical and business landscape to ensure every recommendation aligns with your enterprise environment.
To understand this difference in practical terms, let’s look at two common S/4HANA scenarios: a Purchase Order compliance check and a Sales Order credit validation. On the surface, both appear straightforward – define the requirement, identify the enhancement spot, document the logic, and generate the necessary ABAP code.
A general-purpose AI tool can certainly help structure the functional specification and outline the technical approach. However, the real challenge in enterprise projects is not drafting the document – it is ensuring that the design aligns with the system’s actual configuration, enhancement framework, workflow setup, authorization model, and business data. This is where the distinction between generic AI and embedded, SAP-native intelligence becomes clear.

1. Purchase Order Compliance Check
Business Requirement: Block Purchase Order creation if:
• Vendor is not approved in a custom Z-table
• PO value exceeds 1,000,000
• Purchasing group = “A01”
• Mandatory attachment is missing
Using Chatgpt or Gemini:
• Enhancement via BAdI ME_PROCESS_PO_CUST
• Logic to check vendor approval, PO value, purchasing group, and attachment
• Error messages and test cases
Pros: Clear FS structure, general logic explained
Cons: Generic — it won’t know your actual Z-table structure, workflow configuration, or document type setup. You must feed these details manually.
Using SAP Joule:
• Identifies the correct BAdI or workflow in your S/4 system
• Pulls actual vendor master and custom table fields
• Checks your approval strategy and document type configuration
• Generates FS logic aligned with your live system setup
Pros: System-grounded, accurate, reduces rework, supports compliance

2. Sales Order Credit Validation
Business Requirement: Prevent Sales Order creation if a customer’s credit limit is exceeded.
Using Chatgpt or Gemini:
• Identify a relevant BAdI or user exit
• Check credit exposure against the limit
• Trigger an error message if the limit is exceeded
• Suggest test cases for validation
Pros: Clean structure, well-written, and easy to understand
Cons: Generic — it doesn’t know your actual credit control area, company code setup, or the exact fields and BAdIs in your S/4HANA system. All system-specific details must be manually added.
Using SAP Joule:
• Detect the active BAdI or enhancement spot in your system
• Pull the correct credit control configuration and field names
• Align logic with company code settings and authorization roles
• Suggest tests based on your live system data
Pros: Context-aware, system-aligned, lower risk of errors

Key Differentiators

Verified SAP Knowledge Base: Responses are grounded in SAP's official documentation and verified sources.
SAP-Specific Methodology: Understands SAP implementation approaches, configuration requirements, and technical constraints.
Industry-Specific Compliance: Aligned with SAP's consulting standards and best practices.
SAP Ecosystem Integration: Deep understanding of how SAP modules, technologies, and processes interconnect.

Comparison Table

Capability	SAP Joule for Consultants	ChatGPT	Gemini	Notes
SAP Domain Expertise	Deep, verified SAP knowledge from official sources	General SAP knowledge from public training data	General SAP knowledge with real-time search	Joule is trained on SAP-verified documentation
Template Generation	SAP-specific templates and methodologies	Generic business templates	Generic business templates with Google integration	Templates follow SAP best practices and standards
Functional Specification Writing	SAP implementation-focused with proper technical structure	General functional spec approaches	Structured but generic approaches	Joule understands SAP-specific requirements and constraints
Industry Standards Compliance	SAP best practices and implementation standards	General industry standards	General standards with research capabilities	Aligned with SAP's consulting methodologies
Technical Documentation Structure	SAP-aligned formats (ABAP, configuration paths, transaction codes)	Generic technical documentation	Structured technical docs	Joule provide SAP-specific technical details and paths
Integration Knowledge	Deep SAP ecosystem and module integration expertise	Broad but surface-level SAP integration knowledge	General integration knowledge	Joule understand SAP module interdependencies and best practices
Business Process Understanding	SAP module-specific processes and workflows	General business process knowledge	General business knowledge with research	Joule trained on SAP business process documentation

For functional specification writing in SAP contexts, Joule is the superior choice because:

Accuracy: knowledge comes from verified SAP sources rather than general internet training data.
Relevance: understands SAP-specific technical requirements, configuration paths, and implementation constraints.
Structure: can provide proper SAP documentation formats, including transaction codes, customization paths, and technical specifications.
Best Practices: aligned with SAP's consulting methodologies and implementation standards.

We’ve seen how generic AI can draft specs and code, but context matters. In your S/4HANA projects, where has AI helped — and where has it fallen short? Share your experiences, challenges, or tips below!

Source:

SAP Help Portal | SAP Online Help

Joule

Other Blogs:

From Conversational AI to Business AI: Why SAP Jou... - SAP Community

System Conversion to SAP S/4HANA - SAP Community

Knowledge Graphs on SAP HANA: From Zero to Enterprise RAG with Triple Store

2026-03-05T11:37:21.007000+01:00

Introduction

If you’ve ever worked with Retrieval-Augmented Generation (RAG) systems, you know the drill, you embed your documents into vectors, store them in a vector database, and at query time, you retrieve the top-K most similar chunks to feed as context to an LLM. It works. But there’s a problem.

Vector search alone is semantically “flat.” It tells you which chunks look similar to a query, but it doesn’t understand the relationships between entities in your data. It cannot reason about connections, follow chains of facts, or understand how entity A relates to entity B through entity C.

This is where Knowledge Graphs come in.

In this blog, I’ll walk you through — from absolute scratch — how i built a Knowledge Graph layer on SAP HANA using a custom Triple Store, and how i combined it with vector embedding-based retrieval to create a Hybrid RAG system that gives our AI agents both relevance (from vectors) and reasoning (from graphs).

Everything you see is from POC codebase. Let’s begin.

What is a Knowledge Graph?
What are RDF Triplets?
Why Do We Need Knowledge Graphs in RAG?
Extracting Triplets from Text
- 4a. Rule-Based Extraction with spaCy
- 4b. LLM-Based Extraction
The Multi-Agent Triplet Processing Pipeline
Storing Triplets in SAP HANA Triple Store
The Complete Ingestion Pipeline
Querying the Knowledge Graph
Hybrid Retrieval: Vector + Graph
The Full RAG Pipeline
Exposing via MCP for AI Agents
Architecture Summary
Conclusion

1. What is a Knowledge Graph?

A Knowledge Graph is a structured representation of real-world entities and the relationships between them. Think of it as a web of connected facts.

For example, consider this sentence:

“SAP SE is headquartered in Walldorf, Germany and develops enterprise software.”

A human reading this immediately understands three facts: - SAP SE → is_headquartered_in → Walldorf, Germany - SAP SE → develops → enterprise software - Walldorf → is_located_in → Germany

A Knowledge Graph captures exactly these facts as structured, queryable data. Each fact is stored as a triplet (more on this next).

Why is this different from a regular database?

Feature Relational Database Knowledge Graph

Structure	Fixed schema (tables, columns)	Flexible (entities + relationships)
Relationships	JOINs across tables	First-class citizens (edges)
Schema changes	ALTER TABLE needed	Just add new triplets
Query style	SQL with JOINs	Graph traversal / pattern matching
Best for	Structured, predictable data	Connected, evolving knowledge

A Knowledge Graph is schema-flexible — you don’t need to define all possible relationships upfront. You can keep adding new types of facts without changing any table structure.

2. What are RDF Triplets?

RDF stands for Resource Description Framework. It’s a standard model for representing knowledge as triplets — the atomic unit of a Knowledge Graph.

Every triplet has three parts:

(Subject, Predicate, Object)

Subject: The entity we’re talking about (e.g., “SAP SE”)
Predicate: The relationship or property (e.g., “is_headquartered_in”)
Object: The value or target entity (e.g., “Walldorf”)

Visual Representation

    [SAP SE] ---is_headquartered_in--→ [Walldorf]
       |                                    |
       |---develops--→ [Enterprise Software] |
                                            |
                       [Germany] ←--is_located_in---┘

Each arrow is a triplet. A collection of triplets forms a graph — hence “Knowledge Graph.”

Why Triplets?

Triplets are powerful because they are: 1. Atomic — Each triplet is a self-contained fact 2. Composable — You can combine millions of triplets into a unified graph 3. Queryable — You can traverse relationships: “What does SAP SE develop?” or “Where is the company headquartered?” 4. Language-agnostic — The structure is universal regardless of source language

3. Why Do We Need Knowledge Graphs in RAG?

Let’s understand this with a concrete example.

The Problem with Vector-Only RAG

Imagine your enterprise has documents about employees, projects, and departments. A user asks:

“Who is the project lead for Project Phoenix and which department does that person belong to?”

With vector-only RAG, the system: 1. Embeds the query into a vector 2. Finds the top-K most similar document chunks 3. Hopes that one single chunk contains all the information

But what if: - Chunk A says: “John Smith is the project lead for Project Phoenix” - Chunk B says: “John Smith works in the Cloud Engineering department”

Vector search might retrieve Chunk A (it’s most similar to the query), but miss Chunk B entirely. The LLM then cannot answer the full question.

How Knowledge Graphs Solve This

With a Knowledge Graph, we have these triplets stored:

(John Smith, is_project_lead_of, Project Phoenix)
(John Smith, works_in, Cloud Engineering)
(Cloud Engineering, is_department_of, SAP)

When the system retrieves Chunk A via vector search and finds “John Smith,” it can traverse the graph to discover that John Smith works in Cloud Engineering — even though that second fact was in a completely different document chunk.

The Power of Hybrid Retrieval

Capability Vector Search Knowledge Graph Hybrid (Both)

Semantic similarity	✅	❌	✅
Relationship traversal	❌	✅	✅
Multi-hop reasoning	❌	✅	✅
Handles unseen phrasing	✅	❌	✅
Entity connections	❌	✅	✅

By combining both, we get the best of both worlds: semantic understanding from vectors AND structured reasoning from graphs.

4. Extracting Triplets from Text

The first step in building a Knowledge Graph is extracting triplets from unstructured text. i implemented two approaches in our system.

4a. Rule-Based Extraction with spaCy (NLP Approach)

Our first approach uses spaCy, a natural language processing library, to extract Subject-Verb-Object (SVO) patterns from sentences.

Here’s the code from our triplets_service.py:

import re
from typing import List, Tuple
import spacy

_nlp = None

def get_nlp():
    global _nlp
    if _nlp is None:
        _nlp = spacy.load("en_core_web_sm")
    return _nlp

def clean(text: str) -> str:
    """Basic text cleaning function"""
    return re.sub(r"\s+", " ", text).strip()

def extract_svo_spacy(sentence: str):
    nlp = get_nlp()
    doc = nlp(sentence)
    triplets = []
    for token in doc:
        if token.pos_ == "VERB":  # processing every verb
            # subject can be to the left or as a child
            subj = [w for w in token.children if w.dep_ in ("nsubj", "nsubjpass")]
            objs = [w for w in token.children if w.dep_ in ("dobj", "attr", "dative", "oprd")]

            # prepositional objects
            for prep in [w for w in token.children if w.dep_ == "prep"]:
                pobj = [w for w in prep.children if w.dep_ == "pobj"]
                objs.extend(pobj)

            if subj and objs:
                for o in objs:
                    triplets.append((clean(subj[0].text), clean(token.lemma_), clean(o.text)))
    return triplets

How This Works — Step by Step

Load the spaCy model (en_core_web_sm) — a pre-trained English NLP model
Parse the sentence — spaCy tokenizes it and identifies parts of speech (POS) and dependency relations
Find every VERB — verbs are potential predicates in our triplets
Find subjects — tokens with dependency labels nsubj (nominal subject) or nsubjpass (passive nominal subject)
Find objects — tokens with dependency labels dobj (direct object), attr (attribute), etc.
Handle prepositional objects — “works in engineering” → the object is “engineering” via the preposition “in”
Build triplets — (subject_text, verb_lemma, object_text)

Example:
Input: "SAP develops enterprise software"
Output: [("SAP", "develop", "software")]

Batch Processing

i also have a batch function in triplets_service.py to process an entire corpus:

def extract_corpus_triplets(corpus: List[str]) -> List[List[Tuple[str, str, str]]]:
    """extracting triplets from a list of text chunks"""
    results = []
    for text in corpus:
        sent = text.strip()
        if not sent:
            continue
        triplets = extract_svo_spacy(sent)
        results.append(triplets)
    return results

Limitations of Rule-Based Extraction

While fast and deterministic, spaCy-based extraction has limitations: - Misses implicit relationships (“Headquartered in Walldorf since 1972” — the subject “SAP” isn’t in this fragment) - Produces shallow predicates (just the verb lemma, not semantic relationship names) - Struggles with complex sentences and nested clauses

This is why i built a second, more powerful approach.

4b. LLM-Based Extraction (AI-Powered Approach)

Our primary extraction method uses a Large Language Model (via SAP AI Core) to understand text semantically and produce high-quality triplets.

Here’s the code from our knowledge_graph_service.py:

async def generate_triplets(text_chunk: str):
    """
    Uses an LLM via the Generative AI Hub SDK to extract RDF triplets
    from a given text chunk.
    """
    prompt_for_triplets = f"""
    Extract key factual statements from the text as RDF triplets.

    STRICT REQUIREMENTS:
    1. Return ONLY a valid JSON object
    2. Use exactly this format: {{"triplets": [["subject", "predicate", "object"]]}}
    3. Each triplet must have exactly 3 elements
    4. Use clear, concise predicates (e.g., "is", "has", "uses", "located_in")
    5. Maximum 15 triplets per response
    6. No explanations, no extra text outside JSON

    Text: "{text_chunk}"

    JSON:
    """

    try:
        response_content = await get_llm_response_async(prompt=prompt_for_triplets)
        response_json = json.loads(response_content)
        triplets = response_json.get("triplets", [])
        # converting the inner lists to tuples for consistency
        return [tuple(triplet) for triplet in triplets]

    except json.JSONDecodeError:
        print("Failed to parse JSON response from LLM.")
        return []
    except Exception as e:
        print(f"An error occurred: {e}")
        return []

How This Works

Craft a precise prompt — We instruct the LLM to extract facts as [subject, predicate, object] arrays in JSON format
Call the LLM asynchronously — via get_llm_response_async() which hits SAP AI Core’s inference API
Parse the JSON response — extract the triplets array
Convert to tuples — for consistency with the rest of our pipeline
Handle errors gracefully — JSON parse failures and API errors return empty lists

Why LLM-Based Extraction is Superior

Aspect spaCy (Rule-Based) LLM (AI-Powered)

Implicit relationships	❌Misses them	✅Infers from context
Semantic predicates	❌Raw verb lemmas	✅Meaningful labels like “is_headquartered_in”
Complex sentences	❌Often fails	✅Handles well
Consistency	✅Deterministic	⚠️May vary (hence we validate)
Speed	✅Very fast	⚠️API call overhead
Cost	✅Free	⚠️LLM API costs

Batch Processing with Async Concurrency

Processing an entire corpus (potentially hundreds of chunks) sequentially would be extremely slow. i use asyncio.gather for concurrent processing:

async def convert_corpus_to_triplets_async(
    corpus: list,
    use_orchestrator: bool = True,
) -> List[List[Tuple[str, str, str]]]:
    """Convert entire corpus to triplets using async concurrency"""
    if not corpus:
        return []

    try:
        if use_orchestrator:
            orchestrator = TripletOrchestrator()
            results = await orchestrator.process_corpus(corpus)
            return results
        else:
            # Run all LLM calls concurrently
            tasks = [generate_triplets(text) for text in corpus]
            results = await asyncio.gather(*tasks, return_exceptions=True)

            processed_results = []
            for i, result in enumerate(results):
                if isinstance(result, Exception):
                    print(f"Error processing chunk {i}: {result}")
                    processed_results.append([])
                else:
                    processed_results.append(result)
            return processed_results

    except Exception as e:
        print(f"Error in convert_corpus_to_triplets_async: {e}")
        return [[]] * len(corpus)

Notice the use_orchestrator flag — when True, i use our Multi-Agent Pipeline for higher quality triplets. Let’s dive into that next.

5. The Multi-Agent Triplet Processing Pipeline

Raw LLM output isn’t always perfect. The model might produce inconsistent predicate names, duplicate triplets, or factually questionable relationships. To solve this, i built a multi-agent processing pipeline where specialized agents collaborate to refine triplets.

Architecture Overview

┌──────────────────────────────────────────────────────────┐
│                   TripletOrchestrator                     │
│                                                          │
│  ┌────────────┐   ┌───────────┐   ┌───────────┐        │
│  │  Analyzer   │──→│  Semantic  │──→│ Validator  │        │
│  │   Agent     │   │  Cleaner   │   │   Agent    │        │
│  └────────────┘   └───────────┘   └───────────┘        │
│        │                                    │            │
│        │          ┌───────────┐              │            │
│        └─error──→ │   JSON     │              │            │
│                   │  Repair    │              │            │
│                   └───────────┘              │            │
│                                    ┌────────▼────────┐  │
│                                    │   Aggregator     │  │
│                                    │     Agent        │  │
│                                    └─────────────────┘  │
└──────────────────────────────────────────────────────────┘

The State Object

All agents share a common state object defined in state_schema.py:

class TripletState(BaseModel):
    """State management for triplet processing pipeline"""
    raw_text: str
    initial_triplets: List[List[str]] = []
    cleaned_triplets: List[List[str]] = []
    validated_triplets: List[List[str]] = []
    final_triplets: List[List[str]] = []

    # metadata and tracking
    processing_state: str = "initial"
    error_messages: List[str] = []
    quality_scores: Dict[str, float] = {}
    retry_count: int = 0
    max_retries: int = 3

    # agent outputs
    analyzer_feedback: Optional[str] = None
    cleaner_feedback: Optional[str] = None
    validator_feedback: Optional[str] = None

This is a shared state pattern — each agent reads from and writes to specific fields, and the orchestrator controls the flow.

Stage 1: Analyzer Agent

The AnalyzerAgent in analyzer_node.py performs the initial extraction with a quality assessment:

class AnalyzerAgent:
    """Analyzes text and extracts initial triplets with quality assessment"""

    async def process(self, state: TripletState) -> TripletState:
        try:
            prompt = self._create_analysis_prompt(state.raw_text)
            response = await get_llm_response_async(prompt)
            analysis_result = self._parse_response(response)

            if analysis_result["success"]:
                state.initial_triplets = analysis_result["triplets"]
                state.quality_scores["analyzer"] = analysis_result["quality_score"]
                state.analyzer_feedback = analysis_result["feedback"]
                state.processing_state = "analyzed"
            else:
                state.error_messages.append(f"Analyzer failed: {analysis_result['error']}")
        except Exception as e:
            state.error_messages.append(f"Analyzer exception: {str(e)}")
        return state

The analyzer’s prompt asks the LLM to not only extract triplets but also rate the quality (0-1 score) and provide feedback about text complexity. This metadata helps downstream agents.

Stage 2: Semantic Cleaner Agent

The SemanticCleanerAgent in semantic_cleaner_agent.py standardizes the triplets:

class SemanticCleanerAgent:
    """Cleans and standardizes triplets for consistency"""

    def _create_cleaning_prompt(self, triplets: List[List[str]]) -> str:
        triplets_str = json.dumps(triplets)
        return f"""
        Clean and standardize the following triplets for better semantic consistency.

        CLEANING TASKS:
        1. Normalize entity names (remove extra spaces, fix capitalization)
        2. Standardize predicates (use consistent naming like "is_a", "has_property", "located_in")
        3. Remove duplicate or very similar triplets
        4. Fix obvious semantic issues
        5. Ensure subjects and objects are meaningful entities

        REQUIREMENTS:
        - Maintain factual accuracy
        - Use snake_case for predicates
        - Remove triplets with generic or meaningless entities

        Original triplets: {triplets_str}
        """

This agent ensures that predicates like "is located in", "located_in", and "is_in" all become a consistent "located_in". This consistency is critical for reliable graph queries later.

Stage 3: Validator Agent

The TripletValidatorAgent in triplet_validator_agent.py cross-checks triplets against the original text:

class TripletValidatorAgent:
    """Validates triplets for factual accuracy and relevance"""

    def _create_validation_prompt(self, triplets, original_text):
        return f"""
        Validate the following triplets against the original text for factual accuracy.

        VALIDATION CRITERIA:
        1. Factual accuracy: Are the relationships stated correctly?
        2. Semantic validity: Do the predicates make sense?
        3. Completeness: Are important facts missing?
        4. Consistency: Are there contradictions?
        5. Relevance: Are all triplets relevant to the source text?

        Original text: "{original_text}"
        Triplets to validate: {json.dumps(triplets)}
        """

This is a crucial fact-checking step — the validator has access to both the triplets AND the original text, so it can verify that no hallucinated facts slipped through.

Stage 4: Aggregator Agent

The AggregatorAgent in aggregator_node.py selects the best output and calculates an overall quality score:

class AggregatorAgent:
    """Aggregates results from all agents and determines final output"""

    def _select_best_triplets(self, state: TripletState) -> List[List[str]]:
        """Select the best quality triplets from processing pipeline"""
        if state.validated_triplets:
            return state.validated_triplets
        elif state.cleaned_triplets:
            return state.cleaned_triplets
        elif state.initial_triplets:
            return state.initial_triplets
        else:
            return []

    def _calculate_overall_quality(self, state: TripletState) -> float:
        """Weighted average quality score"""
        weights = {
            "analyzer": 0.3,
            "cleaner": 0.3,
            "validator": 0.4   # validator has higher weight
        }
        weighted_sum = 0.0
        total_weight = 0.0
        for agent, weight in weights.items():
            if agent in state.quality_scores:
                weighted_sum += state.quality_scores[agent] * weight
                total_weight += weight
        return weighted_sum / total_weight if total_weight > 0 else 0.0

Notice the graceful degradation — if validation fails, it falls back to cleaned triplets; if cleaning fails, it uses the initial triplets. The system never completely fails.

The Orchestrator

The TripletOrchestrator in orchestrator.py wires everything together with retry logic:

class TripletOrchestrator:
    def __init__(self):
        self.analyzer = AnalyzerAgent()
        self.cleaner = SemanticCleanerAgent()
        self.validator = TripletValidatorAgent()
        self.aggregator = AggregatorAgent()
        self.json_repair = JSONRepairAgent()

    async def preprocess_text_chunk(self, text_chunk: str) -> List[Tuple[str, str, str]]:
        state = TripletState(raw_text=text_chunk)
        try:
            # Stage 1: Initial analysis (with retries)
            while True:
                state = await self.analyzer.process(state)
                if not self._should_retry(state):
                    break
                state.retry_count += 1

            # Stage 2: Semantic cleaning
            if state.initial_triplets and not self._has_critical_errors(state):
                while True:
                    state = await self.cleaner.process(state)
                    if not self._should_retry(state):
                        break
                    state.retry_count += 1

            # Stage 3: Validation
            if (state.cleaned_triplets or state.initial_triplets) and not self._has_critical_errors(state):
                while True:
                    state = await self.validator.process(state)
                    if not self._should_retry(state):
                        break
                    state.retry_count += 1

            # Stage 4: Aggregation (no retries needed)
            state = self.aggregator.process(state)
            return [tuple(triplet) for triplet in state.final_triplets]

        except Exception as e:
            return []

Each stage has retry logic — if an agent fails (e.g., LLM returns malformed JSON), the orchestrator retries up to max_retries (3) times before moving to the next stage.

6. Storing Triplets in SAP HANA Triple Store

Now that we can extract high-quality triplets, we need somewhere to store them. We built a Triple Store as a column table in SAP HANA.

Table Schema

From our hana_repository.py, here’s the table creation:

def ensure_triple_store():
    """Create TRIPLE STORE table and indexes if they don't exist"""
    conn = get_hana_db()
    try:
        # Check if table exists
        exists_sql = """
        SELECT 1 FROM SYS.TABLES
        WHERE TABLE_NAME = 'TRIPLE_STORE'
        AND SCHEMA_NAME = CURRENT_SCHEMA
        """
        result = conn.sql(exists_sql).collect()
        if not result.empty:
            return

        # Create the table
        create_sql = """
        CREATE COLUMN TABLE TRIPLE_STORE (
            ID BIGINT GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
            EMB_REF_ID NVARCHAR(36) NOT NULL,
            CHUNK_INDEX INTEGER,
            SUBJECT NVARCHAR(500),
            PREDICATE NVARCHAR(200),
            OBJECT NVARCHAR(1000),
            CREATED_AT TIMESTAMP DEFAULT CURRENT_UTCTIMESTAMP,
            FOREIGN KEY (EMB_REF_ID) REFERENCES DOCUMENTS_EMBEDDING(ref_id)
        )
        """
        conn.execute_sql(create_sql)

        # Create indexes for fast lookups
        conn.execute_sql("CREATE INDEX IDX_TRIPLE_SUBJ ON TRIPLE_STORE (SUBJECT)")
        conn.execute_sql("CREATE INDEX IDX_TRIPLE_PRED ON TRIPLE_STORE (PREDICATE)")
        conn.execute_sql("CREATE INDEX IDX_TRIPLE_OBJ ON TRIPLE_STORE (OBJECT)")
        conn.execute_sql("CREATE INDEX IDX_TRIPLE_REF ON TRIPLE_STORE (EMB_REF_ID, CHUNK_INDEX)")
    except Exception as e:
        print(f"error ensuring triple store: {e}")

Understanding the Schema

Column Type Purpose

`ID`	BIGINT (auto)	Unique triplet identifier
`EMB_REF_ID`	NVARCHAR(36)	Links to the embedding table — this is the bridge between vectors and graph
`CHUNK_INDEX`	INTEGER	Which chunk within a document this triplet came from
`SUBJECT`	NVARCHAR(500)	The subject entity
`PREDICATE`	NVARCHAR(200)	The relationship
`OBJECT`	NVARCHAR(1000)	The object entity
`CREATED_AT`	TIMESTAMP	When the triplet was created

The EMB_REF_ID foreign key is the critical design decision. It links every triplet back to its source embedding in the DOCUMENTS_EMBEDDING table. This means: - Given a vector search result, we can find its associated triplets - Given a triplet, we can find the original text chunk - The two systems (vector + graph) are tightly integrated

Indexes for Performance

i created four indexes for fast querying: - IDX_TRIPLE_SUBJ — look up triplets by subject - IDX_TRIPLE_PRED — look up triplets by predicate - IDX_TRIPLE_OBJ — look up triplets by object - IDX_TRIPLE_REF — look up triplets by source embedding (composite index)

The Embedding Table (for reference)

The companion DOCUMENTS_EMBEDDING table stores the vector embeddings:

CREATE COLUMN TABLE DOCUMENTS_EMBEDDING (
    id INTEGER GENERATED BY DEFAULT AS IDENTITY PRIMARY KEY,
    document_text NVARCHAR(5000),
    embedding REAL_VECTOR(3072),
    chunk_metadata NVARCHAR(1000),
    ref_id NVARCHAR(36) UNIQUE NOT NULL
)

REAL_VECTOR(3072) is SAP HANA’s native vector type — it stores 3072-dimensional embeddings (from OpenAI’s text-embedding-3-large model) and supports cosine similarity searches natively.

Inserting Triplets

def insert_triplets(triplets_rows: list):
    """
    Insert triplets with their metadata
    triplets_rows: list of (ref_id, chunk_index, subject, predicate, object) tuples
    """
    ensure_triple_store()
    conn = get_hana_db()

    if not triplets_rows:
        return False

    sql = """
        INSERT INTO TRIPLE_STORE (EMB_REF_ID, CHUNK_INDEX, SUBJECT, PREDICATE, OBJECT)
        VALUES (?, ?, ?, ?, ?)
    """
    try:
        cur = conn.connection.cursor()
        cur.executemany(sql, triplets_rows)
        conn.connection.commit()
        cur.close()
        return True
    except Exception as e:
        return False

We use executemany for batch insertion — inserting hundreds of triplets in a single database round-trip instead of one-by-one.

7. The Complete Ingestion Pipeline

Let’s see how everything comes together in our document_processing_service.py. This is the end-to-end flow: from a raw document URL to stored embeddings AND triplets.

async def process_and_embed_file_from_url(file_url: str):
    """
    Download file, extract text, create embeddings and triplets,
    store everything in HANA
    """
    # Step 1: Download and extract text
    text_content = process_file_from_url(file_url)

    # Step 2: Split into overlapping chunks
    chunks = split_text_into_chunks(text_content)

    # Step 3: Preprocess chunks
    preprocessed_chunks = preprocess_text_chunks(chunks)

    # Step 4: Generate unique reference IDs (linking embeddings ↔ triplets)
    ref_ids = [str(uuid.uuid4()) for _ in preprocessed_chunks]

    # Step 5: Create embeddings for all chunks (concurrent)
    embeddings = await embedding_service.get_embeddings_batch(preprocessed_chunks, max_workers=5)

    # Step 6: Extract triplets from all chunks (multi-agent pipeline)
    triplets_per_chunk = await convert_corpus_to_triplets_async(preprocessed_chunks)

    # Step 7: Insert embeddings FIRST (ref_ids must exist for FK constraint)
    rows = []
    for i, (chunk, embedding_vector) in enumerate(zip(preprocessed_chunks, embeddings)):
        metadata_json = json.dumps({
            "chunk_index": i,
            "chunk_size": len(chunk),
            "source_url": file_url,
            "ref_id": ref_ids[i],
        })
        embedding_string = str(embedding_vector)
        rows.append((chunk, embedding_string, metadata_json, ref_ids[i]))

    batch_insertion_embedding(rows=rows)

    # Step 8: Insert triplets (linked via ref_ids)
    triplets_rows = []
    for chunk_index, (ref_id, chunk_triplets) in enumerate(zip(ref_ids, triplets_per_chunk)):
        for t in chunk_triplets:
            if isinstance(t, dict):
                head, relation, tail = t.get("subject"), t.get("predicate"), t.get("object")
            else:
                head, relation, tail = t
            triplets_rows.append((ref_id, chunk_index, head, relation, tail))

    if triplets_rows:
        insert_triplets(triplets_rows)

The Text Chunking Strategy

def split_text_into_chunks(text: str, chunk_size: int = 1000, overlap: int = 200):
    """Split text into overlapping chunks for better embedding"""
    chunks = []
    start = 0
    while start < len(text):
        end = start + chunk_size
        chunk = text[start:end]
        chunks.append(chunk)
        start = end - overlap
    return chunks

i used overlapping chunks (200 characters overlap) so that facts spanning chunk boundaries aren’t lost. Each chunk is 1000 characters with a 200-character overlap with the next chunk.

The API Endpoint

This entire pipeline is exposed via a FastAPI endpoint in genai_hub.py:

_router.post("/create-store-embedding", response_model=EmbeddingKGResponse)
async def create_and_store_embedding(
    request: EmbeddingKGRequest,
    background_tasks: BackgroundTasks
) -> EmbeddingKGResponse:
    """Create and store embeddings using file URL"""
    # Processing in background (non-blocking)
    background_tasks.add_task(process_and_embed_file_from_url, request.file_url)

    return EmbeddingKGResponse(
        success=True,
        message=f"Successfully processed and embedded document from {request.file_url}"
    )

The processing runs as a background task — the API responds immediately while the heavy work (embedding + triplet extraction) happens asynchronously.

8. Querying the Knowledge Graph

Now for the exciting part — retrieving knowledge from our graph. i have two query strategies in knowledge_graph_service.py.

Strategy 1: Chunk-Based Retrieval

Given a set of embedding reference IDs (from vector search results), fetch all associated triplets:

def get_triplets_by_chunks(ref_ids: list, query: str):
    """Getting triplets from specific chunks + optional query expansion"""
    conn = get_hana_db()

    ref_placeholders = ",".join(f"'{ref}'" for ref in ref_ids)
    base_sql = f"""
    SELECT SUBJECT, PREDICATE, OBJECT
    FROM TRIPLE_STORE
    WHERE EMB_REF_ID IN ({ref_placeholders})
    ORDER BY CHUNK_INDEX, ID
    LIMIT 200
    """

    triplets = []
    try:
        df = conn.sql(base_sql).collect()
        for _, row in df.iterrows():
            triplets.append((row["SUBJECT"], row["PREDICATE"], row["OBJECT"]))
    except Exception as e:
        print(f"An error occurred: {e}")

    # Optional: expand with query-related triplets
    if query and triplets:
        expanded = search_related_triplets(
            query, existing_entities=[t[0] for t in triplets[:10]]
        )
        triplets.extend(expanded)

    return triplets

This function does two things: 1. Direct retrieval — gets all triplets linked to the vector search results 2. Graph expansion — discovers additional related triplets based on entities found

Strategy 2: Keyword + Entity Expansion

The search_related_triplets function implements a two-phase search:

def search_related_triplets(query: str, existing_entities: list, limit: int = 50):
    """Searching triplets by keyword + entity expansion"""
    conn = get_hana_db()
    triplets = []

    # Phase 1: Keyword search in triplets
    query_clean = query.replace("'", "''")
    keyword_sql = f"""
    SELECT SUBJECT, PREDICATE, OBJECT
    FROM TRIPLE_STORE
    WHERE SUBJECT LIKE '%{query_clean}%'
    OR OBJECT LIKE '%{query_clean}%'
    OR PREDICATE LIKE '%{query_clean}%'
    LIMIT {limit}
    """
    try:
        df = conn.sql(keyword_sql).collect()
        for _, row in df.iterrows():
            triplets.append((row["SUBJECT"], row["PREDICATE"], row["OBJECT"]))
    except Exception as e:
        print(f"error in keyword triplet search: {e}")

    # Phase 2: Entity expansion
    if existing_entities:
        entities_clean = [e.replace("'", "''") for e in existing_entities[:5]]
        entities_placeholders = "','".join(entities_clean)
        expansion_sql = f"""
        SELECT SUBJECT, PREDICATE, OBJECT
        FROM TRIPLE_STORE
        WHERE SUBJECT IN ('{entities_placeholders}')
        OR OBJECT IN ('{entities_placeholders}')
        LIMIT {limit}
        """
        try:
            df = conn.sql(expansion_sql).collect()
            for _, row in df.iterrows():
                triplets.append((row["SUBJECT"], row["PREDICATE"], row["OBJECT"]))
        except Exception as e:
            print(f"error in entity expansion triplet search: {e}")

    return triplets

Phase 1 (Keyword Search): Finds triplets where the query terms appear anywhere in subject, predicate, or object. This catches direct mentions.

Phase 2 (Entity Expansion): Takes entities discovered in Phase 1 and finds ALL triplets mentioning those entities. This is the “graph traversal” — following connections one hop outward.

This is how we achieve multi-hop reasoning: Vector search finds relevant chunks → chunk triplets reveal entities → entity expansion reveals connected facts from other chunks.

9. Hybrid Retrieval: Vector + Graph

The ContextService in context_service.py is where vector search and knowledge graph come together:

class ContextService:
    """Context service with cache for RAG Pipeline"""

    async def hybrid_search_context(self, query: str, top_k: int = 5, expand_graph: bool = True) -> str:
        """
        Hybrid retrieval: vector similarity + graph retrieval
        Returns combined textual context for RAG
        """
        # Step 1: Generate query embedding
        query_embedding = await embedding_service.get_embedding(query)

        # Step 2: Vector similarity search
        vector_results = await search_similiar_documents(query_embedding, top_k=top_k)

        if vector_results is None or vector_results.empty:
            return ""

        context_parts = []
        ref_ids = []

        # Step 3: Extract results and collect ref_ids
        for _, row in vector_results.iterrows():
            ref_id = row.get("REF_ID") or row.get("ref_id")
            chunk_text = row.get("DOCUMENT_TEXT", '')
            similarity = row.get("SIMILARITY", 0)

            context_parts.append(f"Chunk (similarity: {similarity:.4f}): {chunk_text}")
            if ref_id:
                ref_ids.append(ref_id)

        # Step 4: Graph expansion using ref_ids from vector results
        if expand_graph and ref_ids:
            graph_context = get_triplets_by_chunks(ref_ids, query)
            if graph_context:
                context_parts.append(f"\nRelated Facts: {graph_context}")

        return "\n\n".join(context_parts)

The Hybrid Flow Visualized

User Query: "Who leads Project Phoenix?"
        │
        ▼
┌─────────────────────────┐
│  1. Embed Query          │  → [0.023, -0.156, 0.891, ...]
└────────────┬────────────┘
             │
             ▼
┌─────────────────────────┐
│  2. Vector Search        │  → Top 5 similar chunks
│  (COSINE_SIMILARITY)     │     with ref_ids
└────────────┬────────────┘
             │
             ▼
┌─────────────────────────────────────────┐
│  3. Graph Expansion                      │
│  - Fetch triplets for those ref_ids      │
│  - Keyword search for "Project Phoenix"  │
│  - Entity expansion for found entities   │
└────────────┬────────────────────────────┘
             │
             ▼
┌─────────────────────────┐
│  4. Combine Context      │
│  - Chunk texts           │  → "Chunk (similarity: 0.92): ..."
│  - Related facts         │  → "Related Facts: [(John Smith, leads, Project Phoenix), ...]"
└────────────┬────────────┘
             │
             ▼
┌─────────────────────────┐
│  5. Send to LLM          │  → Final answer with full context
└─────────────────────────┘

10. The Full RAG Pipeline

Everything comes together in the /chat endpoint in RAG_pipeline.py:

_pipeline_router.post("/chat", response_model=RAGChatResponse)
async def rag_chat(request: RAGChatRequest) -> RAGChatResponse:
    """Complete RAG pipeline: Retrieve + Augment + Generate"""
    try:
        # Check cache first
        cache_key = context_service.cache.make_key(
            "rag_response",
            _make_query_hash(request.query, request.temperature, request.max_tokens),
        )
        cached_response = await context_service.cache.get(cache_key)
        if cached_response:
            cached_response["from_cache"] = True
            return RAGChatResponse(**cached_response)

        # Hybrid retrieval (vector + graph)
        context = await context_service.hybrid_search_context(
            request.query, top_k=request.k, expand_graph=True
        )

        if context == "":
            return RAGChatResponse(
                success=True, query=request.query,
                answer="I couldn't find any relevant documents to answer your question.",
            )

        # Generate answer with augmented context
        answer = await generate_rag_response(
            request.query, context, request.temperature, request.max_tokens
        )

        response_data = {
            "success": True, "query": request.query,
            "answer": answer, "context_used": context, "from_cache": False
        }

        # Cache for 1 hour
        await context_service.cache.set(cache_key, response_data, ttl=3600)
        return RAGChatResponse(**response_data)
    except Exception as e:
        raise HTTPException(status_code=500, detail={"success": False, "error": str(e)})

The prompt sent to the LLM for generation:

async def generate_rag_response(query, context, temperature=0.1, max_tokens=500):
    prompt = f"""
    Based on the following context, answer the question:

    Context:
    {context}

    Question: {query}

    Answer based only on the context provided:
    """
    response = await llm_service.get_llm_response_async(prompt=prompt)
    return response

The context now contains both similar text chunks (from vectors) and structured facts (from the knowledge graph). The LLM gets the richest possible context to generate its answer.

11. Exposing via MCP for AI Agents

i also expose our hybrid RAG as an MCP (Model Context Protocol) tool in mcp/server.py, so AI agents can use it directly:

from fastmcp import FastMCP

mcp = FastMCP("AI-RAG-Service")

@mcp.tool()
async def rag_chat(query: str, top_k: int = 3, temperature: float = 0.1) -> str:
    """Chat with RAG pipeline - retrieves context and generates answer"""
    # Get context using hybrid search (vector + knowledge graph)
    context = await context_service.hybrid_search_context(
        query, top_k=top_k, expand_graph=True
    )
    if not context:
        return "I couldn't find any relevant documents to answer your question."

    prompt = f"""Based on the following context, answer the question:
    Context: {context}
    Question: {query}
    Answer based only on the context provided:"""

    answer = await llm_service.get_llm_response_async(prompt=prompt)
    return answer

This means any MCP-compatible agent (like those built with LangChain, LangGraph, or Claude) can call our rag_chat tool and get knowledge-graph-augmented answers automatically.

12. Architecture Summary

Here’s the complete system architecture:

┌─────────────────────────────────────────────────────────────────────┐
│                        INGESTION PIPELINE                          │
│                                                                     │
│  Document URL → Download → Text Extraction → Chunking (1000/200)   │
│       │                                                             │
│       ├──→ Embedding Service ──→ DOCUMENTS_EMBEDDING table          │
│       │    (text-embedding-3-large, 3072 dims)                      │
│       │                                                             │
│       └──→ Multi-Agent Triplet Pipeline ──→ TRIPLE_STORE table      │
│            (Analyzer → Cleaner → Validator → Aggregator)            │
│                                                                     │
│  Tables linked by ref_id (UUID) ←─ Foreign Key relationship        │
└─────────────────────────────────────────────────────────────────────┘

┌─────────────────────────────────────────────────────────────────────┐
│                        RETRIEVAL PIPELINE                           │
│                                                                     │
│  User Query → Embed Query → Cosine Similarity Search (HANA Vector) │
│       │                                                             │
│       ├──→ Top-K similar chunks (text + ref_ids)                    │
│       │                                                             │
│       └──→ Graph Expansion (ref_id triplets + keyword + entities)   │
│                                                                     │
│  Combined Context → LLM Prompt → Generated Answer                   │
│                                                                     │
│  Exposed via: FastAPI REST + MCP Tools for AI Agents                │
└─────────────────────────────────────────────────────────────────────┘

13. Conclusion

Knowledge Graphs are not just an academic concept — they’re a practical, powerful enhancement to any enterprise RAG system. By combining vector embeddings with structured knowledge triplets stored in SAP HANA’s Triple Store, i achieved:

Better answer quality — The LLM receives both semantically similar text AND structured facts
Multi-hop reasoning — Entity expansion allows discovering connected facts across different documents
Production-grade reliability — Multi-agent pipeline with retries, validation, and graceful degradation
Tight integration — Foreign key linking between embeddings and triplets enables seamless hybrid retrieval
Agent-ready — MCP tools let any AI agent leverage the knowledge graph instantly

Key Takeaways for Your Team

Start with LLM-based triplet extraction — it produces far better triplets than rule-based NLP
Always validate — the multi-agent pipeline (analyze → clean → validate → aggregate) catches errors early
Link your stores — the ref_id foreign key between embeddings and triplets is what makes hybrid retrieval possible
Index aggressively — Subject, Predicate, Object, and composite indexes make graph queries fast
Expand the graph — entity expansion is where the real magic happens, discovering connections the vector search alone would miss

The future of enterprise AI isn’t just “more data”, it’s more connected data. Knowledge Graphs give your AI agents not just context, but direction.

Have questions or want to discuss further? Reach out to me

Quick Financial Plan - Key Updates in SAP Commercial Project Management S/4 HANA 2025 FSP01 release

2026-03-06T11:59:26.620000+01:00

The S/4 HANA 2025 FSP01 release brings significant enhancements to Quick Financial Plan (QFP) capabilities within SAP S/4 HANA Commercial Project Management (CPM), reinforcing SAP`s commitment to smarter, faster and more integrated financial planning. In this blog, we explore the key innovations delivered in the S/4 HANA 2025 FSP01 release and how they help in streamlining the planning process using QFP`s capability as a comprehensive financial planning component in CPM.

Enhanced Distribution with addition of Rate (Cost) and Rate (Revenue) fields for Quantity-Driven Resource Types

The Distribute functionality has been enhanced for quantity-driven resource types. The pop-up dialog now allows planners to enter Rate (Cost) and Rate (Revenue) in addition along with Quantity.

Fig 1: Rate (Cost) and Rate (Revenue) added for Quantity-driven Resource Types in QFP

The system automatically applies the Rate across all planning periods along with the allocated Quantities. Financial values like Cost (Transaction), Cost (Plan), Revenue (Transaction), Revenue (Plan) are also calculated by the system, thereby eliminating manual intervention.

Fig 2: Rate automatically distributed across all periods in Distribution

Addition of Import Functionality

The QFP page for planning now includes a dedicated Import button. Planning data can be brought in from associated Project System projects into Commercial Projects.

Fig 3: Import button added in QFP

Upon selecting Import Data under Import button, the system initiates the import of data from PS project. The import process is executed based on the import method and import strategies configured in SPRO. During execution, the system automatically determines the mapping between the PS elements and the corresponding CPM structure. The progress of the import can be monitored by clicking the Refresh button. By expanding the page header, the current status of the import can be viewed in the Import Status indicator.Fig 4: Check status of Import using the Import Status indicator

Once the Import Status indicator turns green, the imported data can be immediately accessed in the QPF page. The system updates the data in real time, so there is no need to reload the application to view the imported data. Fig 5: Select structure element from Financial Summary to view Import results

The Import Log can be reviewed to access details of the imported data, including the import methods and strategies applied. Additionally, there is an option to simulate the import using Simulate Data function and review the details in Simulate Log. This allows verification of import process before executing it in live environment.

Fig 6: Additional functionalities available under Import button in QFP

Output Based Planning

Output Based planning is now enabled on QFP for monthly and fiscal breakdown scenarios. Upon selecting the Plan button in Financial Plan, the system displays the Output Based planning option.

Fig 7: Output Based Planning added under Plan in Financial Plan

QFP allows output quantity to be planned to statistical key figure in Output Based planning. In addition, the standard options to Distribute and Delete the entries are present, similar to any other financial plan.Fig 8: Select statistical key figure to link output quantities in QFP

Once the output quantity is planned, the corresponding Input Quantity, Cost needed to achieve the planned outputs can be planned in Input Resource Plan. The standard options of Calculate Cost, Delete and Distribute are available in the input plan. Additionally, a new Calculate quantity button has been introduced to compute the input quantity based on Productivity (Plan) and planned output quantity. Once the input plan is entered, the system automatically populates the Unit price and Productivity fields in the output plan, streamlining the planning process and reducing manual effort. Fig 9: Define Input Resource Plan values to achieve the output quantities in QFP

Cash Flow Planning

Cash Flow planning is now embedded within QFP for monthly and fiscal breakdown scenarios, giving organizations a clear view of the incoming and outgoing payments of projects based on the financial plan. This makes it easier to manage resources efficiently and provides a comprehensive view of project financial health.

Consider the below monthly planning:

Fig 10: Monthly planning to review Cash Flow

In order to access Cash Flow in QFP, select Cash Flow under Plan in Financial Plan. When the QFP Page opens, a new button Cash Flow is displayed. Fig 11: Cash Flow added under Plan in Financial Plan

Fig 12: Cash Flow button added in QFP

Once the Cash Flow button is executed, it triggers the valuation strategy and method associated to the resource type in planning. The incoming and outgoing payment values are then copied to the fields present in the valuation method configured in SPRO.

Fig 13: Select structure element from Financial Summary to view Cash Flow results

QFP also allows to adjust the timings of payments using period shifts. By specifying values in period shift field, the system can postpone incoming or outgoing payments to a later date. Fig 14: Period shift in Cash Flow

In the above example the period shift for March is set to 31 and Cash Flow is executed again. The payments are shifted by 31 days i.e. to April. This feature helps prevent timing mismatches and ensures that cash flow projections remain accurate and up to date.

Fig 15: Select structure element from Financial Summary to view Period Shift results

Version Comparison Report

The Version Comparison Report in QFP provides a comparative view of financial plan versions, including baseline, active plan, and forecast snapshots. This consolidated report assists in performing a rapid assessment of the project’s financial health and planning deviations.

In order to access Version Comparison Report in QFP, select ComparePlanVersions under Reports in Financial Plan.

Fig 16: Version Comparison Report added under Reports in Financial PlanFig 17: Compare versions in Financial Plan

Additional information

For more details, you may refer below links:

From HANA Studio to HANA Cloud: The Pain Points Nobody Warns You About

2026-03-06T12:00:00.026000+01:00

SAP COMMUNITY | TECHNICAL GUIDE | SAP HANA CLOUD MIGRATION

From HANA Studio to HANA Cloud: The Pain Points Nobody Warns You About

A practitioner's guide to the friction points that slow down on-premise to HANA Cloud migrations — covering tooling gaps, scripting compatibility, and security model changes that will catch your team off guard.

By Deepa Dorairaj | SAP AI Solution Architect | HANA Data Engineering | Published 2026

TL;DR: Migrating from SAP HANA Studio on-premise to HANA Cloud is not a lift-and-shift. The tooling changes, the security model changes, and a significant portion of your existing stored procedures will need rework before they run. This article covers the three areas where teams consistently underestimate the effort — and exactly what to watch out for in each.

The Migration Gap Nobody Talks About

SAP HANA Cloud is the right destination for most organizations still running on-premise HANA landscapes. The scalability, the managed infrastructure, the BTP integration story — the strategic case is clear. What's less clear, until you're in the middle of it, is how much the day-to-day operational reality changes between HANA Studio on-premise and HANA Cloud Central.

Teams that have spent years building muscle memory in HANA Studio — navigating schemas, editing calculation views graphically, managing users through a familiar interface, writing stored procedures against a predictable execution context — will find that the cloud environment is not a 1:1 replacement. It is a different tool, with different constraints, different patterns, and different failure modes.

This article is not a migration how-to. It is a field guide to the specific pain points that will slow your team down, based on direct experience with this transition. The goal is to give you the information early enough to plan for it rather than discover it mid-project.

PAIN POINT 1 OF 3

HANA Studio Features That Don't Exist in HANA Cloud Central

The Schema Browser and Object Navigation Gap

HANA Studio's system and schema browser is one of its most underappreciated features. Developers and architects who have used it for years navigate schema structures, object dependencies, and table definitions almost by instinct — expanding trees, right-clicking to view DDL, drilling into column definitions without leaving their context.

HANA Cloud Central's database explorer is capable, but the navigation paradigm is meaningfully different. The object hierarchy is presented differently, context menus behave differently, and the workflow for tasks as simple as viewing a table's full column list or checking an index definition requires more steps than Studio users are accustomed to. This sounds minor until you factor in the cumulative friction across a team of developers doing this hundreds of times a day.

Watch out: Teams consistently underestimate the productivity dip in the first 4-6 weeks after cutover. Budget time for your team to rebuild navigation muscle memory in the new tool. What takes 30 seconds in Studio may take 3 minutes in Cloud Central until the team is fluent.

Specific navigational differences to prepare for:

Schema-level object counts and quick overviews that were visible at a glance in Studio require explicit queries or additional clicks in Cloud Central.
The right-click 'Open DDL' shortcut that Studio developers rely on is not directly replicated — you will need to query SYS.OBJECTS or use the SQL console to retrieve DDL for many object types.
Cross-schema object browsing that was seamless in Studio requires explicit privilege grants to be visible in Cloud Central's explorer, which surfaces privilege gaps earlier but creates friction during initial setup.

The Graphical Calculation View Editor

This one causes more frustration than almost any other tooling difference. Teams that built complex calculation views in HANA Studio's graphical editor — star schemas, multi-join hierarchies, calculated columns, input parameters — are accustomed to a specific visual workflow that has been refined over many versions of the on-premise product.

The calculation view editor in SAP Business Application Studio (BAS), which replaces the Studio graphical editor for HANA Cloud, is architecturally different. It is web-based, operates through HDI containers rather than directly against the database schema, and has a different set of supported node types and behaviors.

Critical: Calculation views that reference classic schema objects directly cannot be opened in the BAS graphical editor without first migrating them to HDI container-based deployment. This is not a minor adjustment — it affects the entire deployment and change management workflow for your calculation view landscape.

What this means in practice:

Views that were deployed and edited directly against the HANA schema in Studio must be converted to HDI artifacts (.hdbcalculationview files) before they can be managed in the Cloud toolchain.
The visual editor in BAS handles most standard patterns well but has gaps for certain advanced features that Studio supported — particularly around some legacy join types and specific aggregation node configurations.
Teams using vibe coding approaches for script conversion will find calculation view migration the hardest area to automate reliably — the graphical representation does not translate cleanly to text-based artifacts without manual review.

Practical recommendation: Before cutover, audit your calculation view landscape and categorize views by complexity. Simple star schema views migrate cleanly. Complex multi-join hierarchies with custom aggregation behaviors need hands-on migration effort, not automated conversion.

Watch Out For	Why It Matters
Schema browser navigation	Muscle memory built in Studio does not transfer — plan for a productivity dip and explicit team training time
DDL viewing workflow	Right-click DDL access is not replicated — teams need to learn alternative query-based approaches
Calculation view editor	Studio graphical editor → BAS editor is not a direct replacement; HDI container migration required first
Advanced view features	Some Studio-supported node types and behaviors have gaps in BAS — audit before migrating complex views

PAIN POINT 2 OF 3

Stored Procedure and Scripting Compatibility Issues

Deprecated Syntax That Silently Worked On-Premise

HANA Cloud runs on a more recent version of the HANA engine than most on-premise landscapes. Syntax and behaviors that were technically deprecated in earlier HANA versions but continued working on-premise — because the on-premise version hadn't yet enforced the deprecation — will fail in HANA Cloud with no grace period.

The challenge is that these failures are not always obvious during planning. A stored procedure that has run reliably for three years on-premise, never touched because it works, may fail immediately in HANA Cloud because it uses syntax that was deprecated in HANA 2.0 SPS04 and the on-premise landscape never enforced the removal.

The most common deprecated patterns we encountered: EXEC statements used where EXECUTE IMMEDIATE is the correct modern syntax, certain FOR loop constructs that behave differently in the Cloud SQL engine, and ARRAY handling patterns that were partially supported on-premise but have stricter enforcement in Cloud.

The vibe coding approach to manual script conversion helps significantly here — using AI-assisted code review to scan procedure bodies for known deprecated patterns before migration is faster than manual review at scale. But it requires a human review pass on the output because the conversion suggestions are not always context-aware enough to handle complex procedure logic correctly.

Implicit Type Conversion Differences

HANA Cloud is stricter about implicit type conversions than on-premise HANA in several specific scenarios. Code that relied on HANA silently casting between numeric types, or between date and string types in certain contexts, will produce errors or subtly different results in Cloud.

-- On-premise: this worked with implicit STRING to DATE cast
-- HANA Cloud: requires explicit CAST or TO_DATE conversion
WHERE change_date > '2024-01-01'

-- HANA Cloud safe version:
WHERE change_date > TO_DATE('2024-01-01', 'YYYY-MM-DD')

The more dangerous scenario is numeric type conversion. HANA Cloud's stricter enforcement of arithmetic precision in certain contexts means that calculations which produced correct results on-premise may produce different results in Cloud — not errors, but silently wrong numbers. This is harder to catch in testing because the procedure runs successfully.

Watch out: Implicit conversion issues in financial calculations are particularly risky. Any procedure that performs arithmetic across mixed numeric types (INTEGER, DECIMAL, DOUBLE) should be explicitly reviewed and tested against known expected outputs before cutover.

Procedure Privileges and Execution Context Changes

On-premise HANA procedures run with a well-understood privilege model that many teams have not had to think about deeply — the procedure owner's privileges, or explicitly granted execution rights, determine what the procedure can access. HANA Cloud introduces execution context behaviors that are different in the HDI container model and stricter about cross-container and cross-schema access.

Procedures that accessed objects in multiple schemas by relying on the executing user's broad schema privileges will break in HANA Cloud if those cross-schema access patterns are not explicitly replicated through the new privilege model.

DEFINER vs INVOKER rights behavior differs between on-premise and Cloud in specific scenarios — procedures that relied on implicit DEFINER rights behavior need explicit review.
Cross-schema procedure calls that worked on-premise through broad user grants will require explicit synonym or privilege configuration in the Cloud environment.
Procedures that called other procedures across schema boundaries need the full dependency chain mapped before migration — missing one intermediate privilege breaks the entire call chain.

Watch Out For	Why It Matters
Deprecated syntax	Procedures that ran for years on-premise may fail immediately in Cloud — scan all procedure bodies before migration
Implicit type conversions	Stricter enforcement means silent wrong results, not always errors — test financial calculations against known outputs
FOR loop and EXEC patterns	Common on-premise scripting patterns have changed behavior or stricter enforcement in HANA Cloud SQL engine
Cross-schema procedure calls	Privilege chains that worked on-premise need explicit remapping in the Cloud privilege model

PAIN POINT 3 OF 3

Security and User Management Changes

The Role and Privilege Model Is Fundamentally Different

On-premise HANA security administration through HANA Studio is direct and familiar — create users, assign roles, grant privileges, done. The privilege model is granular but the tooling is transparent. What you see in the security editor is what exists in the database.

HANA Cloud introduces a layered security model that operates differently depending on whether you are working in the HDI container context, the plain schema context, or through BTP's identity and access management layer. Teams that migrate assuming the privilege model works the same way will create security configurations that appear to work in testing but fail in production under specific access patterns.

The most common mistake: Granting broad schema-level privileges in HANA Cloud the same way they were granted on-premise, then discovering that certain operations still fail because the HDI container security layer has separate privilege requirements that aren't covered by schema-level grants.

HDI Containers Replacing Classic Schema-Based Access

If your on-premise landscape uses classic schema-based deployment — objects created directly in database schemas, accessed directly by users with schema privileges — the HDI container model is a significant conceptual shift, not just a technical one.

In HDI containers, database objects are owned by the container's technical user, not by the schema owner or the deploying user. Access to those objects by application users goes through a specific HDI access role pattern that is different from the direct privilege grants your team is used to managing.

Users and roles that worked on-premise cannot be directly imported into HANA Cloud — the privilege model differences mean a fresh role design is almost always necessary.
Application users that previously connected directly to the HANA schema need to be restructured to go through the appropriate HDI access pattern or plain schema equivalent in Cloud.
The tooling for managing HDI container privileges is in BTP Cockpit and HANA Cloud Central, not in a single location — teams need to understand which layer manages which aspect of the privilege model.

Practical recommendation: Before migration, document every user, role, and privilege assignment in your on-premise landscape. Do not attempt a direct import or recreation — use that documentation as a reference to design a new privilege model that fits the HANA Cloud architecture. The effort is significant but unavoidable.

SSO and Identity Provider Configuration

On-premise HANA landscapes often use database-native user authentication or LDAP integration managed directly in the HANA system. HANA Cloud on BTP uses SAML-based SSO through BTP's Identity Authentication Service (IAS) or a connected corporate identity provider.

Teams that relied on database-native authentication for both human users and technical system users will find that HANA Cloud's integration with BTP's identity layer requires a different configuration approach for each user type. Technical users — RFC connections, ETL tool connections, reporting tool connections — often cannot use SAML-based SSO and require specific certificate or password-based authentication configuration that is handled differently in Cloud.

Watch out: Every system that connects to your on-premise HANA — ETL tools, reporting platforms, RFC connections, middleware — needs its authentication method reviewed against HANA Cloud's supported patterns before cutover. This is frequently the last item tested and the most common cause of post-cutover incidents.

Loss of OS-Level Access

On-premise HANA gives administrators OS-level access to the underlying server — the ability to check file system paths, review trace files directly, restart services, and perform low-level diagnostics that go beyond what the database interface exposes.

HANA Cloud is a managed service. OS-level access does not exist. All diagnostics and administration must go through HANA Cloud Central, the SQL console, or the available monitoring views. Teams that have built operational runbooks around OS-level access — checking trace file directories, manually managing backup files, scripting OS-level health checks — need to rebuild those runbooks entirely against the Cloud-native toolset.

Trace file access moves to HANA Cloud Central's diagnostic tools — the content is the same but the access method is completely different.
Backup and recovery operations that were managed through OS-level scripts need to be rebuilt using HANA Cloud's backup catalog and BTP-native tooling.
Custom OS-level monitoring scripts used by your operations team will not function and need Cloud-native equivalents built before cutover.

Watch Out For	Why It Matters
Role and privilege model	On-premise privilege assignments cannot be directly migrated — design a new model for Cloud from scratch
HDI container access	Classic schema-level grants do not cover HDI container object access — separate privilege configuration required
SSO configuration	Every connecting system needs authentication method reviewed against Cloud's supported patterns
OS-level access	All OS-level runbooks and monitoring scripts must be rebuilt using HANA Cloud Central and Cloud-native tools

What This Means for Your Migration Plan

The three areas covered in this article — tooling gaps, scripting compatibility, and security model changes — are the ones most consistently underestimated in HANA Cloud migration projects. They share a common characteristic: they are invisible during the planning phase because the on-premise system works fine, and they surface as a cluster of friction points in the first weeks after cutover.

The teams that navigate this transition most successfully do two things differently. First, they run a structured pre-migration audit — cataloging calculation views by complexity, scanning stored procedures for deprecated patterns, and mapping every privilege assignment before touching the Cloud environment. Second, they treat the first month in Cloud Central as a productivity investment, not a productivity loss — explicitly budgeting time for the team to build new tooling fluency before expecting output at on-premise velocity.

The migration is worth doing. HANA Cloud's operational model, scalability, and BTP integration story are genuinely better than maintaining on-premise infrastructure. But it is not a lift-and-shift, and approaching it as one is the single most reliable way to turn a manageable transition into a painful one.

If your team is in the planning phase of this migration and you want to discuss specific aspects of the audit process or privilege model redesign, drop a comment below. These are the conversations worth having before cutover, not after.

SAP CodeJam Roadshow 2026 – Brazil Edition: La Comunidad SAP Paraguay Estuvo Ahí

2026-03-09T04:46:28.639000+01:00

¡Hola comunidad SAP Paraguay! En marzo de 2026 tuvimos la oportunidad de participar en el SAP CodeJam Roadshow 2026 – Brazil Edition, dos jornadas intensas de aprendizaje hands-on en São Paulo, Brasil. Y como siempre, Paraguay no se quedó afuera.

¿Qué es el SAP CodeJam?
Para quienes aún no lo conocen: el SAP CodeJam es un evento práctico de entre 5 y 6 horas, respaldado por SAP, donde los participantes exploran tecnologías reales en un ambiente colaborativo y sin presión. No se trata de escuchar presentaciones — se trata de hacer, experimentar y aprender juntos.

Esta edición fue parte de un roadshow por Brasil, con paradas en la oficina de SAP en São Paulo y Exed Consulting. Los instructores fueron los Developer Advocates Antonio Maradiaga y Kevin Riedelsheimer — un lujo para cualquier developer SAP.

Los dos dias ¿Qué vivimos?
✔️3 de marzo – Joule Studio: Create Joule Skills and Agents Without Coding
El primer día nos introdujo a Joule Studio, la nueva herramienta dentro de SAP Build que permite extender a Joule, el asistente digital de SAP, con flujos propios.

✔️ 4 de marzo – SAP Build: Create Event-Based Processes
El segundo día fue igualmente intenso. En Exed Consulting, trabajamos con SAP Build Process Automation y SAP Integration Suite, advanced event mesh para construir procesos disparados por eventos.

Lo mas destacado
Más allá del contenido técnico, lo que hace especial a un CodeJam es la gente. Compartir mesa con profesionales de Brasil, resolver ejercicios juntos, debatir enfoques distintos — eso no tiene precio.
Un agradecimiento especial a Kevin Riedelsheimer, Antonio Maradiaga y Antonia Zorn por la confianza y la oportunidad

SAP Community = Conexiones Reales
Eventos como este no llegan solos. Detrás de cada CodeJam hay una comunidad activa que lo hace posible: profesionales que publican en el SAP Community Blog, que comparten sus experiencias, que asisten a eventos en la región, que conectan con Developer Advocates y que demuestran que hay una comunidad con interés real.

No alcanza con querer que suceda: hay que ser parte activa. Eso significa:

Crear una cuenta en SAP Community y participar con preguntas, respuestas y blogs
Seguir y conectar con los Developer Advocates en LinkedIn y SAP Community
Compartir las experiencias de los eventos regionales para que Paraguay aparezca en el radar
Asistir a CodeJams en países vecinos y llevar de vuelta el conocimiento.

Desde la SAP Community Paraguay seguimos trabajando para que Ciudad del Este y Asunción sean paradas reales en el mapa de la comunidad SAP de la región.

¿Qué tema te gustaría ver en un CodeJam en Paraguay? ABAP Cloud, Joule Studio, SAP Build, BTP, UI5 — cuéntanos en los comentarios. Cada opinión cuenta para hacerlo realidad.

¡Sigamos construyendo comunidad! 🚀

End-to-End SAP GUI Logon Automation with Codex and Python: Customizing + Reporting

2026-03-09T08:50:54.651000+01:00

Introduction

Many SAP users still perform repetitive tasks in SAP GUI manually — logging into systems, navigating transactions, adjusting customizing parameters, or extracting reports.With the rise of AI-assisted development and automation frameworks, it is now possible to orchestrate these operations programmatically.

In this article, I present an end-to-end automation scenario that combines:

Python
SAP GUI Scripting
AI-assisted coding with Codex

The solution demonstrates how an AI agent can:

Log into SAP GUI automatically
Execute customizing steps
Navigate transactions
Extract data and generate reports

By integrating AI-assisted scripting with SAP GUI automation, organizations can significantly accelerate operational tasks, reduce human error, and improve productivity.

This approach illustrates how AI agents can augment SAP consultants and administrators, enabling faster execution of routine activities and creating a foundation for more advanced intelligent automation in SAP landscapes.

Accelerate with AI Agents: Build, Adapt, and Report in SAP at Speed

1.What You Can Automate

First of all, I would like to thank you for inspiring me on this topic @Hrb24
Claude Code + Python + SAP GUI Scripting: Your AI Agent for Any SAP Transaction

With Codex + SAP GUI scripting, you can automate both daily operations and advanced SAP configuration work in a practical, scalable way:

Perform SAP customizing tasks directly from scripts
Execute mass customizing by providing Excel-based input files to Codex and applying changes in bulk
Run operational and analytical reports automatically (for example via SE16N-driven workflows)
Validate existing configurations and compare system state against expected setup rules
Build guided simulations for training and decision support (for example, simulate and identify the best price scenario for a specific material)

In short, you can move from manual, screen-by-screen execution to controlled, repeatable, and auditable automation across reporting, configuration, validation, and simulation use cases.

Prerequisites (Before Any Connection)

2.1 SAP-side prerequisites

SAP GUI for Windows must be installed
SAP Logon entry must exist and be tested manually
SAP GUI Scripting must be enabled:
- SAP GUI → Options → Accessibility & Scripting → Scripting
- Enable scripting (client-side)
Server-side scripting parameter must allow scripting:
- sapgui/user_scripting = TRUE (Basis side)
Your user needs authorization for:
- target transactions (SE16N, OVX4, OVX5, etc.)
- customizing save + transport assignment (if doing config)

2.2 Local environment prerequisites

Python 3.10+ installed
Install required packages:
python -m pip install pywin32 openpyxl pypdf
Keep SAP Logon open before script execution (recommended)
Codex installed ( paid subscription required )

3. Project Structure (Recommended)

Use a simple folder structure:

automation/ sap_connect.py

sap_connect.py: connection/bootstrap layer

4. Connection Script Design (Core)

Your sap_connect.py should handle:

Attach to running SAP GUI:
- GetObject("SAPGUI")
Start SAP Logon if not running
Open connection by exact SAP Logon entry name
Wait until session window is ready
Detect login screen vs already authenticated menu
Handle post-login popups (information/multiple logon)
Verify status bar for login errors
Return a reusable session wrapper
Example Script:

"""Open or create an SAP GUI session and authenticate (SSO or password).


"""

import argparse
import getpass
import logging
import os
import subprocess
import sys
import time
from dataclasses import dataclass
from typing import Any, Dict, List, Optional

import win32com.client

try:
    from sap_scripting import SapSession
except Exception:
    class SapSession:
        def __init__(self):
            self.connection_index = 0
            self.session_index = 0
            self.application = None
            self.connection = None
            self.session = None

        def get_session_info(self) -> Dict[str, Any]:
            info = self.session.Info
            return {
                "system": getattr(info, "SystemName", ""),
                "client": getattr(info, "Client", ""),
                "user": getattr(info, "User", ""),
                "transaction": getattr(info, "Transaction", ""),
                "response_time": getattr(info, "ResponseTime", ""),
            }


logging.basicConfig(
    level=logging.INFO,
    format="%(asctime)s [%(levelname)s] %(message)s",
    datefmt="%H:%M:%S",
)
log = logging.getLogger("sap_connect")


@dataclass(frozen=True)
class SapConnectConfig:
    default_system: str = os.getenv("SAP_SYSTEM", "")
    default_client: str = os.getenv("SAP_CLIENT", "")
    default_user: str = os.getenv("SAP_USER", "")
    default_language: str = os.getenv("SAP_LANGUAGE", "EN")
    default_sso: bool = os.getenv("SAP_SSO", "true").lower() in ("1", "true", "yes")
    login_wait: float = 2.0
    popup_wait: float = 0.5
    startup_timeout: int = 30
    session_ready_timeout: float = 30.0
    saplogon_candidates: tuple = (
        r"C:\Program Files (x86)\SAP\FrontEnd\SAPgui\saplogon.exe",
        r"C:\Program Files\SAP\FrontEnd\SAPgui\saplogon.exe",
        r"C:\Program Files (x86)\SAP\SAPLogon\saplogon.exe",
    )


CONFIG = SapConnectConfig()

DEFAULT_SYSTEM = CONFIG.default_system
DEFAULT_CLIENT = CONFIG.default_client
DEFAULT_USER = CONFIG.default_user
DEFAULT_LANGUAGE = CONFIG.default_language
DEFAULT_SSO = CONFIG.default_sso

LOGIN_SCREEN_FIELDS = (
    "wnd[0]/usr/txtRSYST-MANDT",
    "wnd[0]/usr/txtRSYST-BNAME",
)


def _mask(text: str, enabled: bool) -> str:
    if not enabled:
        return text
    if not text:
        return text
    if len(text) <= 2:
        return "*" * len(text)
    return text[0] + ("*" * (len(text) - 2)) + text[-1]


def _find_element(session: Any, element_id: str) -> Optional[Any]:
    try:
        return session.findById(element_id)
    except Exception:
        return None


def _attach_scripting_engine() -> Any:
    rot_entry = win32com.client.GetObject("SAPGUI")
    return rot_entry.GetScriptingEngine


def _ensure_saplogon_running() -> Any:
    try:
        app = _attach_scripting_engine()
        log.info("SAP Logon already running. Active connections: %s", app.Children.Count)
        return app
    except Exception:
        log.info("SAP Logon not detected - attempting to start it ...")

    exe_path = next((p for p in CONFIG.saplogon_candidates if os.path.isfile(p)), None)
    if exe_path is None:
        raise ConnectionError(
            "SAP Logon is not running and saplogon.exe was not found in:\n"
            + "\n".join(f"  {p}" for p in CONFIG.saplogon_candidates)
        )

    log.info("Launching SAP Logon from: %s", exe_path)
    subprocess.Popen([exe_path])

    deadline = time.time() + CONFIG.startup_timeout
    while time.time() < deadline:
        try:
            app = _attach_scripting_engine()
            log.info("SAP Logon started successfully.")
            return app
        except Exception:
            time.sleep(1.0)

    raise ConnectionError(
        f"SAP Logon did not become ready within {CONFIG.startup_timeout} seconds. "
        "Check SAP GUI scripting settings."
    )


def _wait_for_session_ready(connection: Any) -> Any:
    deadline = time.time() + CONFIG.session_ready_timeout
    while time.time() < deadline:
        try:
            session = connection.Children(0)
            _ = session.findById("wnd[0]").Text
            return session
        except Exception:
            time.sleep(0.5)
    raise RuntimeError("SAP window did not become ready within timeout after OpenConnection().")


def _detect_screen_state(session: Any) -> str:
    if _find_element(session, LOGIN_SCREEN_FIELDS[0]) is not None:
        return "LOGIN"

    try:
        tcode = session.Info.Transaction.strip()
        if tcode and tcode != "LOGIN":
            return "MENU"
    except Exception:
        pass

    try:
        if session.findById("wnd[0]").Text.strip():
            return "MENU"
    except Exception:
        pass

    return "UNKNOWN"


def _do_login(
    session: Any,
    client: str,
    user: str,
    password: str,
    language: str,
    sso: bool,
) -> None:
    mandt = _find_element(session, "wnd[0]/usr/txtRSYST-MANDT")
    if mandt and mandt.Changeable and client:
        mandt.text = client
        log.info("Client set.")

    if not sso:
        bname = _find_element(session, "wnd[0]/usr/txtRSYST-BNAME")
        bcode = _find_element(session, "wnd[0]/usr/pwdRSYST-BCODE")
        langu = _find_element(session, "wnd[0]/usr/txtRSYST-LANGU")

        if bname:
            bname.text = user
        if bcode:
            bcode.text = password
        if langu and langu.Changeable:
            langu.text = language

        log.info("Credentials filled (user/password mode).")
    else:
        log.info("SSO mode - skipping username/password fields")

    session.findById("wnd[0]").sendVKey(0)
    log.info("Login submitted (Enter)")


def _handle_multiple_logon_popup(session: Any) -> None:
    log.info("Multiple logon detected - selecting OPT2 (keep existing sessions).")
    opt2 = _find_element(session, "wnd[1]/usr/radMULTI_LOGON_OPT2")
    if opt2:
        opt2.select()
    confirm = _find_element(session, "wnd[1]/tbar[0]/btn[0]")
    if confirm:
        confirm.press()
    else:
        session.findById("wnd[1]").sendVKey(0)


def _handle_post_login_popups(session: Any) -> None:
    time.sleep(CONFIG.popup_wait)
    for _ in range(5):
        popup = _find_element(session, "wnd[1]")
        if popup is None:
            break

        log.info("Popup detected: '%s'", popup.Text.strip())
        try:
            if _find_element(session, "wnd[1]/usr/radMULTI_LOGON_OPT1"):
                _handle_multiple_logon_popup(session)
            else:
                popup.sendVKey(0)
        except Exception as exc:
            log.warning("Popup handling failed: %s", exc)
        time.sleep(CONFIG.popup_wait)


def _verify_login(session: Any) -> None:
    sbar = _find_element(session, "wnd[0]/sbar")
    if not sbar:
        return

    msg_type = getattr(sbar, "MessageType", "")
    msg_text = getattr(sbar, "Text", "")
    if msg_type in ("E", "A"):
        raise RuntimeError(f"Login failed [{msg_type}]: {msg_text}")
    if msg_type == "W":
        log.warning("Login warning [%s]: %s", msg_type, msg_text)
    elif msg_type == "S" and msg_text:
        log.info("Login status [%s]: %s", msg_type, msg_text)


def _wrap_existing_session(application: Any, conn_idx: int, session_index: int = 0) -> SapSession:
    sap = object.__new__(SapSession)
    sap.connection_index = conn_idx
    sap.session_index = session_index
    sap.application = application
    sap.connection = application.Children(conn_idx)
    sap.session = sap.connection.Children(session_index)
    return sap


def connect_to_system(
    system: str = DEFAULT_SYSTEM,
    client: str = DEFAULT_CLIENT,
    user: str = DEFAULT_USER,
    password: str = "",
    language: str = DEFAULT_LANGUAGE,
    sso: bool = DEFAULT_SSO,
) -> SapSession:
    if not sso:
        if not user:
            user = input(f"SAP user for {system}/{client}: ").strip()
        if not password:
            password = getpass.getpass(f"Password for {user}@{system}: ")

    app = _ensure_saplogon_running()
    log.info("SAP Logon ready. Active connections before: %s", app.Children.Count)
    log.info("Opening connection to '%s' (SSO=%s) ...", system, sso)

    try:
        connection = app.OpenConnection(system, True)
    except Exception as exc:
        raise ConnectionError(
            f"Cannot open connection to '{system}': {exc}\n"
            "Check SAP Logon entry name (case-sensitive)."
        ) from exc

    session = _wait_for_session_ready(connection)
    log.info("Session opened.")

    state = _detect_screen_state(session)
    log.info("Screen state after OpenConnection: %s", state)
    if state == "LOGIN":
        _do_login(session, client, user, password, language, sso)
        time.sleep(CONFIG.login_wait)
    elif state == "MENU":
        log.info("SSO authenticated automatically - skipped login screen.")
    else:
        log.warning("Unexpected screen state '%s' - attempting to continue.", state)

    _handle_post_login_popups(session)
    _verify_login(session)

    conn_idx = app.Children.Count - 1
    sap = _wrap_existing_session(app, conn_idx, session_index=0)
    info = sap.get_session_info()
    log.info(
        "Connected - system=%s, client=%s, user=%s, transaction=%s",
        info["system"],
        info["client"],
        _mask(info["user"], True),
        info["transaction"],
    )
    return sap


def list_logon_entries(mask_sensitive: bool = True) -> List[Dict[str, Any]]:
    try:
        app = _attach_scripting_engine()
        active: List[Dict[str, Any]] = []
        for i in range(app.Children.Count):
            conn = app.Children(i)
            for j in range(conn.Children.Count):
                sess = conn.Children(j)
                active.append(
                    {
                        "conn_idx": i,
                        "sess_idx": j,
                        "system": sess.Info.SystemName,
                        "client": _mask(sess.Info.Client, mask_sensitive),
                        "user": _mask(sess.Info.User, mask_sensitive),
                        "tcode": sess.Info.Transaction,
                    }
                )
        return active
    except Exception as exc:
        log.warning("list_logon_entries failed: %s", exc)
        return []


def _build_parser() -> argparse.ArgumentParser:
    parser = argparse.ArgumentParser(
        description="Open a new SAP session from SAP Logon and log in.",
        formatter_class=argparse.RawDescriptionHelpFormatter,
        epilog=(
            "Examples:\n"
            "  python sap_connect.py\n"
            "  python sap_connect.py --system MYSYS --client 100\n"
            "  python sap_connect.py --no-sso --user YOUR_USER\n"
            "  python sap_connect.py --list\n"
            "  python sap_connect.py --list --no-mask"
        ),
    )
    parser.add_argument("--system", default=DEFAULT_SYSTEM, help="SAP Logon entry description")
    parser.add_argument("--client", default=DEFAULT_CLIENT, help="SAP client number")
    parser.add_argument("--user", default=DEFAULT_USER, help="SAP username (ignored for SSO)")
    parser.add_argument("--language", default=DEFAULT_LANGUAGE, help="Logon language")
    parser.add_argument("--no-sso", dest="sso", action="store_false", help="Use password login")
    parser.set_defaults(sso=DEFAULT_SSO)
    parser.add_argument("--list", action="store_true", help="List active SAP sessions and exit")
    parser.add_argument("--no-mask", action="store_true", help="Do not mask user/client in output")
    return parser


def main() -> None:
    args = _build_parser().parse_args()

    if args.list:
        sessions = list_logon_entries(mask_sensitive=not args.no_mask)
        if not sessions:
            print("No active SAP sessions found.")
            return
        print(f"\nActive SAP sessions ({len(sessions)}):")
        for s in sessions:
            print(
                f"  [{s['conn_idx']}:{s['sess_idx']}]  "
                f"{s['system']}/{s['client']}  user={s['user']}  tcode={s['tcode']}"
            )
        return

    mode = "SSO" if args.sso else f"password (user={args.user or 'prompt'})"
    print(f"\nConnecting to {args.system} / client {_mask(args.client, not args.no_mask)}  [{mode}] ...")

    try:
        sap = connect_to_system(
            system=args.system,
            client=args.client,
            user=args.user,
            language=args.language,
            sso=args.sso,
        )
        info = sap.get_session_info()
        print(f"\n[OK] Connected: {info['system']} / {_mask(info['client'], not args.no_mask)} / {_mask(info['user'], not args.no_mask)}")
        print(f"  Transaction : {info['transaction']}")
        print(f"  Server      : {info['response_time']} ms response")
    except (ConnectionError, RuntimeError) as exc:
        log.error("%s", exc)
        sys.exit(1)


if __name__ == "__main__":
    main()

5. How to Connect (Execution)

5.1 List active sessions

python sap_connect.py --list

5.2 Connect with SSO

python sap_connect.py --system "Your SAP Logon Entry Name" --client 100

5.3 Connect with username/password

python sap_connect.py --system "Your SAP Logon Entry Name" --client 10

Important: --system must match SAP Logon entry description exactly (case-sensitive in many setups).

6. After Connection: Reporting Flow (SE16N)

A standard reporting script should do this:

StartTransaction("SE16N")
Set table (for example VBAK / VBRK)
Set filters in selection fields (date/material/org/etc.)
Execute (F8)
Read ALV grid rows/columns
Save output to Excel

Example use cases

Last 6 months sales report
Open request list (E070)
Material-level pricing checks

7. Excel Output Best Practices

For business users, generate:

Sheet 1: summary table (monthly totals/KPIs)
Sheet 2: raw data extract
Charts:
- pie chart for distribution
- bar chart for monthly trend
Metadata row:
- generation timestamp
- source table
- filter range

8. After Connection: Customizing Flow

For customizing automation scripts:

Start transaction (for example OVX4, OVXI, OVXB)
Enter Change mode if needed
Use New Entries / Copy As consistently
Fill required fields only
Save
Handle transport popup:
- assign existing request or create new one
Read status bar and validate success

9. Transport Handling Pattern

During save, scripts should:

Detect “Prompt for Customizing Request”
If request field is empty, set existing request ID
Confirm popup
Close follow-up information popup
Verify status = success (MessageType = S)

10. Validation Strategy

Always validate after script execution:

Re-open transaction and check entry exists
Cross-check table data in SE16N
Capture:
- status type
- status text
- key fields created/updated
Export validation result if needed

Conclusion

By combining Codex for rapid script generation and SAP GUI Scripting for execution, teams can standardize repetitive SAP work: from reporting to customizing. The key to reliability is a strong connection layer, deterministic popup handling, and strict post-run validation.

SAP HANA Backup and Recoverability: Backup Completion vs. Recovery

2026-03-11T17:00:00.028000+01:00

A successful backup confirms data was written, it does not, by itself, confirm recovery readiness.

Introduction
This article summarizes publicly available SAP documentation and SAP Knowledge Base Articles and reflects the documented behavior of SAP HANA backup and recovery.

In many SAP HANA landscapes, backup monitoring is treated as the primary indicator of data protection. Backup jobs run regularly, verification checks succeed, and storage systems confirm that backup files are readable and complete.

However, SAP documentation and SAP Knowledge Base Articles describe situations in which a database recovery cannot be performed even though:

• data backups exist
• backup jobs reported success
• backup files are accessible
• backup integrity checks do not report errors

During a recovery attempt, the restore procedure may fail with errors indicating missing encryption material.

SAP Knowledge Base Articles document recovery failures when required encryption root keys are not available in the target system environment.

This behavior is not related to a malfunction of the backup operation itself and does not indicate a storage failure. It is consistent with the documented recovery procedures of SAP HANA.

This article does not provide configuration or implementation guidance.
Its purpose is to consolidate SAP documentation and SAP KBAs to clarify an important technical distinction:

In SAP HANA, a successful backup operation alone does not necessarily confirm that a database system is recoverable in a newly installed or rebuilt environment.

Understanding this distinction helps interpret restore failures correctly and align recovery expectations with the documented system behavior.

1. What SAP HANA Actually Backs Up

According to the SAP HANA documentation, a data backup captures the database persistence at a specific point in time.

SAP Help Portal – Backups (SAP HANA Platform)
https://help.sap.com/docs/SAP_HANA_PLATFORM/6b94445c94ae495c83a19646e7c3fd56/158071e0c455487a89ea56ac53ad4b31.html

The documentation explains that data backups store the database data pages, while log backups record subsequent database changes.

SAP Help Portal – Log Backups
https://help.sap.com/docs/SAP_HANA_PLATFORM/6b94445c94ae495c83a19646e7c3fd56/c3bb7e33bb571014a03eeabba4e37541.html

From the documentation perspective, backups capture database content.
However, SAP documentation differentiates between database data, database metadata, and system security information.

The backup files contain the database persistence and its change history through log backups.
Recovery, however, requires the database system to interpret that persistence correctly.

2. The Restore Dependency Documented in SAP KBAs

SAP Knowledge Base Articles document recovery failures even when backup files are valid and accessible.

KBA 3558019
Recovering a database backup fails with error: encryption root keys and backup password are missing
https://me.sap.com/notes/3558019

KBA 3250470
Recovery of tenant database fails due to missing encryption root keys
https://me.sap.com/notes/3250470

In these documented situations:

• backup files are intact
• storage is reachable
• the backup catalog exists

The restore still fails because a required dependency is missing in the target system environment.

The missing dependency is the encryption root key material required for decryption during recovery.

3. Encryption in SAP HANA Is Also a Recovery Dependency

SAP HANA uses an internal key hierarchy for encryption.
When encrypted persistence or encrypted backups are involved, recovery requires the database to decrypt stored content.

SAP documentation describes handling of encryption material and system operations in administrative procedures and system replication operations.

SAP HANA System Replication Guide
https://help.sap.com/doc/c81e9406d08046c0a118c8bef71f6bdc/2.0.07/en-US/SAP_HANA_System_Replication_Guide_en.pdf

From the documented restore behavior, recovery does not only read backup files.
The system must reconstruct the database using security metadata belonging to the original database system.

Therefore, recovery typically requires multiple components:
• backup data files
• backup catalog
• log chain (depending on recovery type)
• encryption root key material

The encryption root key material required to decrypt encrypted backup content is not contained in the backup data files and must be available in the target system environment.

4. Why a Valid Backup Can Still Not Be Restored

This leads to a fundamental distinction.

A backup operation verifies that database persistence was successfully written.

A recovery operation verifies that the database system can interpret that persistence.

SAP HANA recovery errors documented in the KBAs show that the restore process depends on system security material outside the backup chain.

In these cases, restore failure is not caused by:

• backup corruption
• storage failure
• an unsuccessful backup job

It occurs because required system security state is not available in the recovery environment.

5. Typical Recovery Scenarios

Documented recovery problems appear when a new system environment is created, for example after:

• host rebuild
• new system installation
• landscape re-provisioning
• disaster recovery activation
• ransomware recovery

Administrators correctly preserve backups but the target system is not identical to the original database system from a security perspective.

Even if SID, instance number, and database name are identical, the required encryption root key material is not available in the target system environment.

As a result, the recovery process cannot decrypt the persistence and the database cannot be reconstructed.

6. Interpreting the Documented Behavior

SAP documentation and SAP KBAs together indicate an important technical understanding:

The documented recovery behavior shows that successful backup creation alone is not sufficient to guarantee that recovery can be executed in a different or newly installed system environment.

The backup protects the stored data.
The encryption material enables the database to interpret that data.

Without the corresponding system security material, database persistence cannot be reconstructed even though valid backups exist.

Conclusion

SAP documentation and SAP Knowledge Base Articles document that encrypted backup recovery requires the corresponding encryption material to be present in the restore environment.

Therefore:

A successful SAP HANA backup confirms that the backup operation completed successfully.

It does not, by itself, confirm that a database recovery can be performed in a newly installed or rebuilt system.

Understanding this distinction helps correctly interpret restore failures and reduces recovery analysis time.

For implementation or configuration procedures, always follow the official SAP documentation and SAP Notes.

What This Means for Administrators

The documented recovery behavior shows that backup monitoring alone does not fully represent recovery readiness.

When planning disaster recovery procedures, administrators should ensure that recovery tests validate the complete restore procedure and not only the existence or readability of backup files.

Only a successfully executed recovery test confirms that the documented recovery dependencies are fulfilled in the target environment.

For operational procedures and configuration details, always follow official SAP documentation and SAP Notes.

Disclaimer

This article is an interpretation of publicly available SAP documentation and SAP Knowledge Base Articles.
Only official SAP documentation and SAP Notes are authoritative.

Real-Time Architecture in SAP CAP: Mastering Asynchronous Jobs, Gateways with WebSockets

2026-03-12T21:26:14.975000+01:00

In my previous article (SAP CAP on HANA XSA), we explored how the SAP Cloud Application Programming Model (CAP) on SAP HANA 2.0 XSA provides the raw power needed for high-performance industrial applications, specifically focusing on mass data ingestion via native Stored Procedures.

However, processing high volumes of data in real-tme is only half the battle. In a mission-critical Industrial Data Application, the user interface must stay in sync with these background processes. If a mass import takes several minutes, a standard synchronous request will lead to Gateway Timeouts (HTTP 502) and a frustrated user.

This second part of our series focuses on how to close this gap by combining CAP, the SAP Job Scheduler, and Express WebSockets into a responsive real-time architecture.

While the SAP Cloud Application Programming Model (CAP) is most commonly associated with the SAP Business Technology Platform (BTP), it offers immense productivity on-premise as well. However, the challenge lies in harmonizing long-running backend jobs (e.g., mass data imports via Stored Procedures) with the need for real-time updates in the frontend (Angular UI) under XSA security and routing constraints (Approuter). Relying on purely synchronous requests in this context risks gateway timeouts and a poor user experience.

In this post, I present a responsive real-time architecture that closes these gaps by combining CAP, SAP Job Scheduler, Angular WebSockets, and native HANA Stored Procedures.

Architecture at a Glance

Our high-level architecture diagram divides the Industrial Data Application into logical layers. The central data flow is categorized into color-coded zones representing the Client, Gateway, Service, Data, and External Systems.

Core Components of the Architecture:

Client Layer (Purple): An Angular UI providing real-time charting dashboards. It communicates with the backend via HTTPS OData APIs and WSS (WebSockets) Event subscriptions.
Gateway Layer (Green): The SAP Approuter serves as the central entry point and gateway. It manages API destinations, handles JWT tokens for authentication (linked to the XSUAA/UAA provider), and routes both HTTPS (OData/Actions) and WebSocket traffic.
Service Layer (Blue): An SAP CAP Node.js backend extended with express-ws. It contains the standard OData CRUD layer, business logic, and specialized mass data handlers. Central to this is the Express WebSocket Endpoint with a client registry to manage push notifications. This layer is also integrated with the SAP Job Scheduler.
Data Layer (Orange): The SAP HANA 2.0 XSA on-premise platform. An HDI Container stores the primary application data in the Column Store. Native HANA Stored Procedures (SQLScript) are used to execute the actual mass data calls and imports.
External Systems (Amber): External industrial APIs, historians, or sensor interfaces that act as the source for massive data streams. Native HANA procedures communicate with these systems for data ingestion.

Technical Implementation: WebSocket Registry and Broadcasting

To transmit real-time events efficiently to the Angular UI, we use express-ws within the CAP Node.js backend. This requires a structured registry to manage WebSocket clients based on their specific interests (event types or channels).

In our project, we defined the WebSocket channels as follows. This registry utilizes Sets to store unique WebSocket connections for each business event type:

// db/src/ws/wsRegistry.js (Example location)
'use strict';

// Business-specific registration of WebSocket channels
const wsClients = {
    'validation-status': new Set(), // Critical for the numbered workflow
    'import-status': new Set(),
    'committing-status': new Set(),
    'industrial-trade': new Set(),
    'systemStatusChanged': new Set(),
    'UserChanged': new Set(),
    'process_step': new Set(),
    'job_scheduler': new Set(),
    'program-input-mode': new Set(),
    'batch-job-status': new Set(),
};

module.exports = { wsClients };

The Asynchronous Job Workflow

The most critical part of this architecture is the asynchronous job flow, numbered 1–6 in the diagram. This flow is a best practice for industrial scenarios where jobs cannot be completed synchronously. It demonstrates how CAP and WebSockets prevent gateway timeouts.

Let’s break down the steps:

Angular UI triggers CAP Action (Client -> Service): The process starts when the Angular UI triggers a CAP Action (or service call) via HTTPS. This follows the standard HTTP OData/Actions path.
Fast HTTP 202 Response (Service -> Client): This is a decisive step for UX and timeout prevention. Instead of waiting for the job to finish, the CAP service responds immediately with an HTTP 202 (Accepted). This signals to the frontend that the job has been queued and is running in the background.
CAP Logic calls Stored Procedure (Service -> Data): The CAP backend hands the heavy lifting off to the native HANA layer. This is done via a CALL proc (SQL/ODBC) that triggers the native HANA Stored Procedure (SQLScript).
Native HANA Job Execution (Data -> External -> Data): The Stored Procedure executes the native job. It connects to external industrial APIs, fetches measurement data, processes it, and saves it into the HDI container’s Column Store. This can take seconds or minutes without blocking the UI.
Job Completion & Event Push (Data -> Service) -> WebSocket Trigger: Once the HANA procedure finishes, it sends a status trigger back to the CAP service. The CAP handler then triggers the WebSocket event.

Below is the code snippet showing how the business logic triggers the event using the wsClients registry. In this example, we send a status update to the validation-status channel:

// srv/handlers/validationHandler.js (Example location)
const { wsClients } = require('../ws/wsRegistry'); // Import the registry

// ... Inside business logic, after successful job completion ...
const jobId = result.jobId; 
const final = true; 
const isValid = true; 

// Trigger wsSend with business payload
wsSend("validation-status", {
    jobId,
    status: final ? "SUCCESS" : "IN_PROGRESS",
    request_Id: result?.CheckRequest || result?.Request || result?.request || null,
    isValid: final ? isValid : null,
    detailStatus: final ? 200 : 202,
    resultStatus: final
      ? (isValid ? 'VALIDATED' : 'NOT_VALIDATED')
      : 'PENDING'
});

6. 'JobCompleted' Event Sent (Service -> Client): The Express WebSocket endpoint receives the trigger and uses the registry (wsClients['validation-status']) to actively push the JobCompleted event to the connected Angular UI via the WSS path.

Here is the core wsSend broadcasting method. Note how it cleans up closed connections automatically:

// srv/ws/wsBroadcaster.js
const { wsClients } = require('./wsRegistry');

async function wsSend(type, payload) {
  try {
    const clients = wsClients[type];
    const idx = Number(process.env.CF_INSTANCE_INDEX || process.env.INSTANCE_INDEX || 0);
    console.log(`[wsSend] broadcast on instance ${idx}, channel ${type}`);

    if (!clients || clients.size === 0) return false;

    const msg = JSON.stringify({
      event: 'statusEvent',
      data: { type, ...payload }
    });

    for (const ws of clients) {
      try {
        if (ws.readyState === ws.OPEN) {
          ws.send(msg); 
        } else {
          clients.delete(ws); // Cleanup
        }
      } catch (err) {
        clients.delete(ws);
      }
    }
    return true;
  } catch (err) {
    return false;
  }
}

The UI Reaction (Ensuring Consistency)

In response to the incoming JobCompleted event, the Angular UI performs an OData Refresh. This ensures the most recent data is pulled from the CAP service via the standard OData CRUD path, keeping the UI perfectly in sync.

Technical Notes: package.json Setup

To enable this in your CAP Node.js backend, you'll need express-ws. This allows you to integrate WebSockets directly into the Express app that CAP sits on:

{
  "dependencies": {
    "express-ws": "5.0.2",
    "express": "^4"
  }
}

Conclusion

By combining the productivity of SAP CAP with the raw power of HANA Stored Procedures and the push capabilities of WebSockets, we’ve built a resilient real-time architecture for on-premise industrial applications. This asynchronous workflow with HTTP 202 effectively kills gateway timeouts and ensures a fluid user experience.

Key Benefits:

Peak Performance: Native HANA procedures handle massive data imports without Node.js overhead.
Enhanced UX: Immediate 202 responses and real-time status updates prevent UI lockups.
Maintainability: A structured WebSocket registry allows for a clean separation of business channels.
Data Sovereignty: The entire stack remains securely on your on-premise HANA XSA infrastructure.

How do you handle mass data jobs and real-time requirements in your CAP apps? Are you already utilizing asynchronous patterns? I’d love to hear your thoughts in the comments!

Free migration assessments for modernization with SAP BDC

2026-03-27T14:59:14.022000+01:00

SAP is currently offering a limited-time complimentary migration assessment service for modernization with SAP Business Data Cloud. All customers running the applications listed below, either on-premises or in a private cloud, are eligible to take advantage of this free service.

SAP BW 7.x (all databases) and all versions of BW/4HANA.
SAP BusinessObjects (BOBJ), all versions
SAP Business Planning and Consolidation (BPC), all versions
SAP HANA and SAP IQ databases running custom-built (non-SAP) applications

These assessments provide deep insights into active and inactive objects, offer guidance on what can be modernized, and provide an actionable modernization plan with best practices.

To better understand the process, diagram below illustrates the process.

Key illustrated steps include

Submit an online request, which takes a couple of minutes. SAP will review the requests and approve them on a first-come, first-served basis. Customers will be provided with a few scripts to gather necessary information, which usually takes a few minutes to run on the production systems.
Upon sharing with SAP, expert teams will review the information and generate an assessment report.
Then, the SAP expert team will deliver a readout session, sharing insights into the system's usage patterns and a personalized transformation journey with modernization options to build a scalable solution and prepare for the era of AI with SAP BDC.

Additionally, the video below walks through the steps to request the free assessment service.

Refer to the links below for additional information and to request free assessments.

Join the SAP Data Professional Community for the latest on Data & AI topics.

Good to know: "SQL Statement Collection" the Swiss army knife for SAP HANA database administrators

2026-04-04T11:21:44.192000+02:00

🎓The blog contains SAP Learning references. 📺

last updated: 2026-04-04

Why was this blog post created?

The SAP HANA database comes with a set of tools that support the work of database administrators, developers, and other support teams. The tools are available both internally (SAP Employees) and externally. Knowledge of this set of tools is essential for SAP HANA database operators.

Where can I find the most important information about SQL Statement Collection reports?

The following SAP Note is the central source of the toolkit 🔧 called "SQL Statement Collection".

📘 1969700 - SQL Statement Collection for SAP HANA

There are huge number of reports available for different purposes. There is a SAP Knowledge Base Article created to explain the reports one-by-one in details.

📕 3311408 - Bookmark of SQL Statement Collection reports for SAP HANA

The SAP Knowledge Base Article is under construction and will contain more reports in the future.

SAP Help Portal - SAP HANA Troubleshooting and Performance Analysis Guide - Using the SQL Statement Collection for Analysis and Health Checks

Where to find learning materials?

SAP Learning - SAP HANA - Installation and Administration - Using the SAP HANA Statement Library

Do you have further questions?

Please do not hesitate to contact me if you have question or observation regarding the article.
Q&A link for SAP HANA: https://answers.sap.com/tags/73554900100700000996

Contribution

If you find any missing information belongs to the topic, please let me know. I am happy to add the new content. My intention is to maintain the content continuously to keep the info up-to-date.

Release Information

Release Date	Description
2026.04.04	First/initial Release of the SAP Blog Post documentation (Technical Article).

Strategies for Downtime-Optimized DMO : doDMO, doC, DMOVE2S4 for large databases

2026-04-06T14:14:14.510000+02:00

Context and Challenges:
Recently I got to migrate a very large non-HANA database of 50+ TB to HANA at AWS using Downtime-Optimized DMO (doDMO)
While this was a shift from on-premise to AWS , the customer sought to migrate on HANA at AWS first i.e. to validate data compression, operations at 32TB U7inh EC2 instance of AWS etc, and decided to change the Suite subsequently later. Thus, the plan for using DMO move to SAP S/4HANA (DMOVE2S4) approach along with Downtime-Optimized Conversion (doC) had to be dropped

Operating in R3load pipeline mode ,doDMO is much faster compared to the usual DMO with system move approach that operates at file mode only. However, since downtime-optimized options (doDMO and doC) do not support System Move, thus, in order to migrate only the database using doDMO , a customer confronts the challenges:
1. No Application migration : Customer has to install separate SAP application, and make necessary changes and connect the new SAP application at migrated database.
2. No usability of Source DB upon migration : Customer has to restore the Source database or scrap it as the database cannot be used for an SAP system, since DMO artifacts remain in it even upon Database migration is completed.

Do DMOVE2S4 and DMO with System Move solve these problems?
DMOVE2S4 is just an approach that can be incorporated with DMO, doDMO and doC based migrations. It is not a "Selectable" choice in the SUM UI ,but a method that combines DMO without letting SUM to know that this approach is actually used. With DMOVE2S4, there is no change in application-specific tasks such Simplification Item-Check and activities such as FIN Data Conversion and they are performed in uptime/downtime based on the chosen options i.e. Standard or Downtime-Optimised.
Unlike usual SUM DMO, DMOVE2S4 lets SUM operate on the application, installed in Target environment (which initially is connected on Source database).
With the completion of DMOVE2S4 based migration, ASCS is also moved to target environment where application finds itself already connected to new database of DMO.

While DMOVE2S4 approach tries to solve the first problem to some extent, it comes with its own technical and infrastructure requirements such as Latency < 20 ms & Bandwidth > 400 MBit/s, and limitation that target must be S/4HANA.
DMO with System move option was incorporated in DMO to solve both the above problems, but since it operates at file mode, it is not very fast and downtime friendly.
Even parallel mode (using rsync for files) of DMO with System move may not match downtime-optimised DMO or even the normal DMO which also uses memory pipes for database migration.
Thus, for a data-center migration of a large database & application aiming no-S/4HANA,we still need to make some necessary compromises e.g. install and connect SAP applications, restore the source database (if needed for any use) etc.

Available choices for a customer

With the introduction of doDMO and doC, along with DMOVE2S4 approach, customers running SAP ERP or S/4HANA have multiple paths to transition their ERP to S/4HANA on hyperscalers such as AWS and Azure.
To the journey of S/4HANA, multiple options focused on reducing the Conversion or migration-downtime brings us the situations to decide between the two questions : possible and optimal.
The diagram below demonstrates the optimal choice that is possible for the different cases of migrations, keeping an eye on business-downtime.
Factors such as cost and complexity can often be inversely proportional to the downtime-optimized by different ways, and they must be traded-off well.

HANA Roadmap

In case SUM DMO is not supported, we always have option for classical - export import of flat files for both homogeneous and heterogeneous migration using SWPM.

Transaction cannot be revived after delayed abortion(RAP_RUNTIME 019) - Must ROLLBACK After Failures

2026-04-08T11:52:21.989000+02:00

1. Symptom (ST22 / Dump Keywords)

A typical dump looks like this (same category as in your case):

Runtime Error: RAISE_SHORTDUMP
Exception: CX_SADL_DUMP_APPL_MODEL_ERROR
T100: RAP_RUNTIME / 019
Short Text: Transaction cannot be revived after delayed abortion (BO: I_ENTERPRISEPROJECTTP_2)

When debugging you may also notice RAP internal state like:

CL_RAP_BHV_PROCESSOR->IF_RAP_LEGACY_TRANSACTION~MV_ABORT = abap_true

Meaning: RAP has already marked the logical transaction as aborted (delayed abortion). It must not be reused.

2. Root Cause: RAP Transaction State + Delayed Abortion

A RAP EML “logical transaction” typically consists of two phases:

Transaction phase

MODIFY ENTITIES runs validations/determinations/checks.
Errors are usually returned in FAILED / REPORTED

Save phase

COMMIT ENTITIES triggers the save sequence (finalize/save, etc.).
Errors again come back via FAILED / REPORTED (in the commit response this is the LATE response).

If anything fails in either phase, RAP can enter delayed abortion:
Your ABAP code continues running,but the RAP logical transaction becomes unusable.
Any subsequent EML using that aborted transaction context may lead to RAP_RUNTIME 019.

MODIFY ENTITIES OF <BO> ... FAILED ls_failed_tx REPORTED ls_reported_tx.
COMMIT ENTITIES ... FAILED ls_failed_save REPORTED ls_reported_save.

" If commit failed but you don't rollback here,
" next EML call can dump (RAP_RUNTIME 019)
MODIFY ENTITIES OF <BO> ...

Key point: Never continue with another EML step after a failure without rollback.

5. Correct Example Using `I_EnterpriseProjectTP_3`

*&---------------------------------------------------------------------*
*& Report zdemo_rap_eml_epp3
*&---------------------------------------------------------------------*
*&
*&---------------------------------------------------------------------*
REPORT zdemo_rap_eml_epp3.

PARAMETERS:
      p_uuid TYPE sysuuid_c36 OBLIGATORY,
      p_desc TYPE string      DEFAULT 'Demo update from EML'.

START-OF-SELECTION.

  DATA lv_uuid_x16 TYPE sysuuid_x16.

  TRY.
              cl_system_uuid=>convert_uuid_c36_static(
                  EXPORTING
                      uuid     = p_uuid
                  IMPORTING
                      uuid_x16 = lv_uuid_x16 ).
        CATCH cx_uuid_error.
              WRITE: / 'Invalid UUID format. Use 36-char UUID (xxxxxxxx-xxxx-xxxx-xxxx-xxxxxxxxxxxx).'.
              RETURN.
  ENDTRY.

  DATA:
        ls_failed_tx     TYPE RESPONSE FOR FAILED   i_enterpriseprojecttp_3,
        ls_reported_tx   TYPE RESPONSE FOR REPORTED i_enterpriseprojecttp_3,
        ls_failed_save   TYPE RESPONSE FOR FAILED   LATE i_enterpriseprojecttp_3,
        ls_reported_save TYPE RESPONSE FOR REPORTED LATE i_enterpriseprojecttp_3.

  TRY.
              " Transaction phase
              MODIFY ENTITIES OF i_enterpriseprojecttp_3
                  ENTITY EnterpriseProject
                  UPDATE FIELDS ( ProjectDescription )
                  WITH VALUE #(
                      ( %key-ProjectUUID      = lv_uuid_x16
                          ProjectDescription    = p_desc )
                  )
                  FAILED   ls_failed_tx
                  REPORTED ls_reported_tx.

              IF ls_failed_tx IS NOT INITIAL.
                    WRITE: / 'MODIFY ENTITIES failed (transaction phase) -> ROLLBACK ENTITIES'.
                    ROLLBACK ENTITIES.
                    RETURN.
              ENDIF.

              " Save phase
              COMMIT ENTITIES
                  BEGIN RESPONSE OF i_enterpriseprojecttp_3
                  FAILED   ls_failed_save
                  REPORTED ls_reported_save.
              COMMIT ENTITIES END.

              IF ls_failed_save IS NOT INITIAL.
                    WRITE: / 'COMMIT ENTITIES failed (save phase) -> ROLLBACK ENTITIES'.
                    ROLLBACK ENTITIES.
                    RETURN.
              ENDIF.

              WRITE: / 'COMMIT ENTITIES OK.'.

        CATCH cx_root INTO DATA(lx_root).
              WRITE: / 'Unexpected exception:', lx_root->get_text( ).
              " Keep the EML contract: if anything goes wrong, end the logical transaction.
              ROLLBACK ENTITIES.
  ENDTRY.

Two important details in this example

COMMIT ENTITIES requires LATE response types

RESPONSE FOR FAILED LATE i_enterpriseprojecttp_3
RESPONSE FOR REPORTED LATE i_enterpriseprojecttp_3

“Fail fast” with rollback

If FAILED is not initial in either phase, rollback and end the logical transaction.
Start the next business step in a new logical transaction context.

8. FAQ: Why not `ROLLBACK WORK`?

In RAP EML, use the EML-compatible statement:

Errors in EML logical transaction → ROLLBACK ENTITIES

ROLLBACK WORK is lower-level LUW handling and can lead to semantic mismatches with RAP’s logical transaction handling. The dump you saw is fundamentally about the RAP logical transaction being aborted and still being reused.

Exploring Model Lifecycle Management with Python Machine Learning Client for SAP HANA

2026-04-09T08:42:17.110000+02:00

1. Introduction

This blog post introduces key model lifecycle management features in the SAP HANA Predictive Analysis Library (PAL) through the Python Machine Learning Client for SAP HANA (hana_ml). PAL provides rich in-database machine learning capabilities, while hana_ml exposes these capabilities in Python so that users can build, execute, and operate model workflows directly on SAP HANA data with minimal data movement.

Model lifecycle development usually starts from a fit task, followed by some scoring tasks to evaluate the performance of a derived model. After repeated cycles of fitting and scoring tasks to get an acceptable model, the model can be delivered for continual prediction tasks and optional drift detection tasks. Drift detection is used to answer the question of whether the currently deployed model is still performing acceptably, since patterns in prediction data may change from time to time. Once drift is confirmed, a fit task on the latest data is required, and therefore a new cycle of model development begins.

Experiment tracking is a feature that automatically traces and persists some information for the input and output of PAL procedure execution. Each PAL procedure call is identified by a tracking ID. The tracking ID consists of two parts: an experiment ID and a run ID, representing one experiment and one run respectively. Logically, the relationship between experiment and run is one-to-many. One experiment usually targets a single task such as fitting, predicting, or scoring, with a selected algorithm. Users may have multiple runs for the same experiment while exploring different hyperparameters and/or datasets.

The persisted information for experiment tracking consists of two parts: tracking metadata and tracking logs. Different APIs are available to retrieve them. Tracking metadata records information about the invoked PAL procedure and its current running status. Tracking logs are persisted in chronological order, each of them has a log type, log value and log timestamp.

The available log types include:

Parameter: PAL configuration for each procedure. This is essential information for rerunning the procedure.
Dataset Metadata: Metadata of the input data, including the dataset source and descriptions of the fields in the dataset.
Model Signature: Information about how to use the model, including model input data format, accepted configuration parameters, and output data structure.
Metric: Metric data that records statistical information for the model.
Figure: Discrete and continuous figure data used for plotting and analysis.

Reproducing the exact same result with a given dataset and configuration is a fundamental requirement in model development. It can be used for validating findings, debugging models, and ensuring consistent behavior over time. The logs of input parameters, dataset metadata, and tracking metadata are essential information for achieving this purpose.

The model signature log can guide users in executing prediction and scoring tasks with a derived model. Metric and figure logs can be used for analyzing model performance.

Model Storage is another important feature for model development. Users can identify a model by model name and model version. Model storage serves as the standard source for machine learning models when committing prediction, scoring and evaluation tasks. Besides, newly generated models can be pushed into model storage with a selected model name and model version.

Drift detection is a turning point in model development and runtime operations. It alerts users that model degradation may happen or has happened. This can be achieved with the features of drift detection combined with metric and/or figure track entities described above. Currently hana_ml provides visual charts to compare metrics between different experiment runs to verify whether such a drift has happened.

Finally, automation drives the workflow across model development activities. With the Scheduled Execution feature, users can not only automate single tasks such as model fit, model prediction, and model scoring, but also orchestrate related tasks in a defined order or in parallel. All automated tasks can be scheduled as background jobs either immediately or by time-frequency settings.

With all these features in hana_ml, users can construct flexible model development workflows, including model fitting, model testing, model storage, model serving, and drift detection.

2. Use Case: End-to-End Model Lifecycle Management

This use case shows a compact end-to-end model lifecycle workflow based on the public Pima Indians Diabetes dataset. This example is intended for demonstration and teaching purposes only. It uses SAP HANA Predictive Analysis Library (PAL) together with the Python Machine Learning Client for SAP HANA (hana_ml) to illustrate how a model is trained, tracked, stored, operationalized, and monitored over time. The notebook example for this use case can be downloaded from the ml-lifecycle-examples folder.

Lifecycle phases covered:

Baseline Training & Experiment Tracking: Train an initial model and log all parameters, metrics, and artifacts.
Model Storage & Management: Save, version, load, and manage the model.
Scheduled Inference: Automate predictions on a regular schedule.
Monitoring & Retraining: Simulate monitoring for data drift and retraining decisions.

Dataset note: this example uses the Pima Indians Diabetes dataset, a binary classification task based on clinical measurements such as glucose concentration, BMI, and age.

Step 0: Environment and Data Preparation

We start by importing the core libraries and creating a connection to SAP HANA. This connection context is the entry point for loading data, running PAL algorithms, and managing lifecycle artifacts in the database. It is the required first step because all later actions reuse it.

from hana_ml import dataframe
from hana_ml.algorithms.pal.utility import DataSets
from hana_ml.algorithms.pal.unified_classification import UnifiedClassification
from hana_ml.artifacts.tracking.tracking import MLExperiments, delete_experiment_log, get_tracking_log
from hana_ml.visualizers.tracking import ExperimentMonitor, ScheduledTaskMonitor
from hana_ml.model_storage import ModelStorage

# Establish a connection to SAP HANA
conn = dataframe.ConnectionContext(url='<host>', port=<port>, user='<user>', password='<pwd>')

Replace ‘host’, ‘port’, ‘user’, and ‘pwd’ with your SAP HANA instance details.

Next, use train_test_val_split to create the main working subsets: df_train for baseline fitting, df_score for holdout evaluation, and df_inference for scheduled batch prediction. This keeps training, evaluation, and operational inference clearly separated.

from hana_ml.algorithms.pal.partition import train_test_val_split
df_full, _, _, _ = DataSets.load_diabetes_data(conn)
df_train, df_score, _ = train_test_val_split(data=df_full, partition_method='random', random_seed=23, 
                                            training_percentage=0.8, testing_percentage=0.2, validation_percentage=0.0, id_column="ID")
# Save tables in HANA
df_train.save("PIMA_INDIANS_DIABETES_TRAIN_TBL")
df_score.save("PIMA_INDIANS_DIABETES_SCORE_TBL")

print("Train sample shape : ", df_train.shape)
print("The first 3 rows: ")
print(df_train.head(3).collect())

print("The first 3 rows of inference sample (no label column):")
df_inference = df_score.deselect("CLASS")
print(df_inference.head(3).collect())

Fig.1 Data samples

To mimic a simple monitoring scenario, we then construct weekly batches with different label distributions. These batches keep the label column so later scoring runs can be compared for drift.

# Simulated weekly batches for drift observation (different label mix).
week20_hdf = df_score.filter('ID < 100')
week21_hdf = df_score.filter('CLASS = 1')
week22_hdf = df_score.filter('CLASS = 0')

week20_hdf.save('DIABETES_WEEK20')
week21_hdf.save('DIABETES_WEEK21')
week22_hdf.save('DIABETES_WEEK22')

print(week20_hdf.shape)
print(week21_hdf.shape)
print(week22_hdf.shape)

Fig 2. Shape of simulated data

Step 1. Baseline Training and Experiment Tracking

This step establishes the initial reference model. We train a Hybrid Gradient Boosting Tree classifier, use MLExperiments to log parameters and metrics automatically, and keep the run history auditable. Training and tracking are created together.

We begin by creating a dedicated tracking session identified by a unique EXPERIMENT_ID. Within that experiment, training and scoring are recorded as separate runs.

# Constants for the workflow
EXPERIMENT_ID = "BLOG_DIABETES_TRACKING"
MODEL_NAME = "BLOG_DIAB_HGBT"
TASK_ID = "DIABETES_WEEKLY_PREDICT"

# Optional, clear previous tracking logs so each run starts from a clean state.
delete_experiment_log(conn, EXPERIMENT_ID)

# Initialize the experiment tracker
tracker = MLExperiments(
    connection_context=conn,
    experiment_id=EXPERIMENT_ID,
    experiment_description="diabetes experiment")

# Define the hyperparameter grid for model search.
param_values = {
    "learning_rate": [0.1, 0.4],
    "n_estimators": [5, 10],
    "split_threshold": [0.1, 0.3]}

We configure the model with grid search and cross-validation, then enable autologging. This captures the hyperparameters, source dataset, and generated artifacts under the run name “Diagnosis_classifier-fit”. The key value is reproducibility: the tracked baseline can later be compared with monitoring runs.

uhgbt = UnifiedClassification(
    func="HybridGradientBoostingTree",
    param_search_strategy="grid",
    resampling_method="cv",
    evaluation_metric="error_rate",
    ref_metric=["auc"],
    fold_num=5,
    random_state=123,
    param_values=param_values)

# Enable automatic tracking of parameters, metrics, and artifacts.
tracker.autologging(
    model=uhgbt,
    run_name="Diagnosis_classifier-fit",
    dataset_name="diabetes",
    dataset_source="PIMA_INDIANS_DIABETES_TRAIN_TBL")
# Train the model using stratified partitioning
uhgbt.fit(data=df_train, key="ID", label="CLASS", partition_method="stratified",
    partition_random_state=5, stratified_column="CLASS")

# Log a separate scoring run
tracker.autologging(
    model=uhgbt,
    run_name="Diagnosis_classifier-score",
    dataset_name="diabetes",
    dataset_source="PIMA_INDIANS_DIABETES_SCORE_TBL")
score_pred, score_stats, score_cm, score_metrics = uhgbt.score(
    data=df_score, key="ID", label="CLASS")

After execution, we can retrieve and inspect the tracking artifacts for the current run:

tracking_id = tracker.get_current_tracking_id()
print(f"tracking id: {tracking_id}")
print(tracker.get_tracking_metadata_for_current_run().collect())
print(get_tracking_log(connection_context=conn, tracking_id).head(5).collect())

Fig. 3 Tracking metadata and log

We also provide a dashboard view of tracked runs, metrics, and artifacts.

experiment_monitor = ExperimentMonitor(connection_context=conn, experiment_ids=[EXPERIMENT_ID])
experiment_monitor.start()

Fig. 4 Experiement dashboard

In the Experiment Monitor, the experiment named “BLOG_DIABETES_TRACKING” contains two runs. Opening a run shows its tracked details, including parameters, metrics, and visual artifacts. For example, the run labeled “Diagnosis_classifier-score” shows charts such as ROC and cumulative gains plots.

Fig.5 Continuous figure

Step 2. Model Storage & Management

In this step, we persist the selected model in SAP HANA. This creates a versioned model artifact that can be loaded for later use. The ModelStorage class handles save, list, load, and delete operations. In the notebook, this step shows how the tracked baseline is promoted into a reusable deployment artifact.

# Example decision: choose uhgbt as the baseline operational model.
candidate_model = uhgbt
# Assign model identity fields before persisting.
candidate_model.name = MODEL_NAME
candidate_model.version = 1
model_storage = ModelStorage(connection_context=conn)
# if_exists='replace' overwrites an existing model with the same name/version.
model_storage.save_model(model=candidate_model, if_exists="replace") 

# List the models
model_storage.list_models(name=MODEL_NAME)

deployed_model = model_storage.load_model(name=MODEL_NAME)
print(f"Deployed model from storage: {MODEL_NAME}")

Fig. 6 Model list

The following command is optional cleanup for demo artifacts.

model_storage.delete_models(name=MODEL_NAME)

Step 3. Operationalize with Scheduler

This step moves the stored model into an operational workflow by creating scheduled inference. The scheduler runs predictions on a defined cadence, such as weekly, without manual execution. In other words, deployment is not only about storing a model, but also about defining how it will run repeatedly.

# Import and initialize the scheduler
from hana_ml.algorithms.pal.scheduler import ScheduledExecution
sexec = ScheduledExecution(conn)

# Define a prediction task using the deployed model
sexec.create_predict_task(
    obj=deployed_model,
    predict_params={"data": df_inference, "key": "ID"},
    task_id=TASK_ID,
    force=True)

# Schedule the task to run automatically
weekly_cron = "* * * mon 8 0 0"
schedule_info = sexec.create_task_schedule(
    task_id=TASK_ID,
    cron=weekly_cron,
    force=True)
schedule_info.collect()

Fig.7 Schedule information

# Launch the scheduler monitoring dashboard
scheduled_task_monitor = ScheduledTaskMonitor(connection_context=conn, task_ids=[TASK_ID])
scheduled_task_monitor.start()

Fig.8 Scheduled Task Monitor

The following commands are included for reference only. In the notebook, they are optional inspection and validation utilities rather than required steps.

Query Schedule & Logs: The function query_task_schedule(task_id) returns two DataFrames. The first describes the schedule definition, such as the cron pattern, and the second contains execution logs for historical runs, which are useful for auditing and debugging.
Trigger a Manual Validation Run: The function create_one_off_task_schedule(task_id) triggers an immediate out-of-schedule execution of the task. This is useful for validating the setup before relying on the automated schedule.
Remove the Schedule: The function remove_task_schedule(task_id) removes the schedule binding for the specified task ID.

Step 4. Monitor Drift Signals and Decide Whether to Retrain

This final step closes the model lifecycle loop by simulating production monitoring. We reuse the weekly batches prepared earlier, score them with the deployed model, and compare whether the tracked metrics remain stable or start to drift. The main idea is not the exact metric values in this toy example, but the monitoring pattern: repeated scoring of later batches against the same deployed model.

# Optional
WEEKLY_EXPERIMENT_ID = "WEEKLY_HGBT_TRACK"
delete_experiment_log(conn, WEEKLY_EXPERIMENT_ID)

# Create a dedicated experiment for production-like weekly monitoring.
MLModel_weekly_tracking = MLExperiments(
    connection_context=conn,
    experiment_id=WEEKLY_EXPERIMENT_ID,
    experiment_description="Monitor of HGBT model weekly"
)

# Reuse the weekly slices prepared earlier and log one score run per week.
weekly_batches = [
    ("week20-score", week20_hdf, "diabetes_week20", "DIABETES_WEEK20"),
    ("week21-score", week21_hdf, "diabetes_week21", "DIABETES_WEEK21"),
    ("week22-score", week22_hdf, "diabetes_week22", "DIABETES_WEEK22"),
]

for run_name, weekly_batch, dataset_name, dataset_source in weekly_batches:
    MLModel_weekly_tracking.autologging(
        model=deployed_model,
        run_name=run_name,
        dataset_name=dataset_name,
        dataset_source=dataset_source
    )
    score_pred, score_stats, score_cm, score_metrics = deployed_model.score(
        weekly_batch,
        key="ID",
        label="CLASS"
    )

You can visualize these trends, such as accuracy or AUC, in the Experiment Monitor dashboard. For example, in the simulated scenarios for Weeks 20, 21, and 22, you can select the accuracy metric, choose the three corresponding weekly runs, and click Compare to open a detailed comparison view.

Fig. 9 Weekly monitor

In the figure below, you can observe fluctuations in accuracy across the weeks, for example from 0.78 to 0.61 and then to 0.81. A sharp shift such as the drop in Week 21 would be a reasonable drift signal. In a real workflow, that signal would trigger a review and could lead to actions such as retraining the model.

Fig. 10 Model Drift

3. Summary

This blog post shows the key features and an end-to-end model lifecycle workflow use case with SAP HANA PAL and hana_ml. Conceptually, model lifecycle management in SAP HANA PAL with hana_ml is a closed-loop process that connects model development, model registration, model operations, and post-deployment monitoring in one traceable workflow. In this article, we use a compact scenario to explain that progression from baseline training to operational monitoring.

References

Notebook download folder: ml-lifecycle-examples

Product Documentation:

Most Relevant Blog Posts:

Other Related Blog Posts:

SAP Community - SAP HANA

Automated Test Suite: RHEL HA Solution for SAP S/4HANA ENSA 2 using Ansible

Introduction

What is the Automated Test suite?

Which tests are performed?

Test Name

Test Title

Minimum no. of HA Nodes

Prerequisites

Getting started

How to run the tests

SAP HANA System Replication Takeover and Client Connectivity

Avoiding HANA Connection Exhaustion: A Practical Guide to Hikari & Multi-Tenant Scaling

Introduction

What is a Connection Pool?

Benefits

How CAP Java Uses Connection Pools

Why This Becomes Critical in Production

Multi-Tenant Scaling Problem

The Solution: Shared Pools with combinePools

HikariCP Configuration (Production)

How to Size Your Pool Correctly

Monitoring & Observability

Common Pitfalls

Summary

Final Thoughts

References

Understanding SAP HANA Disaster Recovery - Technical Scope, Distance, and Data Protection Boundaries

BI Remediation Checks during ECC to S/4HANA Migration

Migrating to SAP HANA Cloud: What Actually Gets Better (Part 1 of 2)

Why This Article Exists

HDI Containers: A Modern Database Development Workflow

SAP AI Core: Production-Grade AI on Your SAP Data

The BTP Application Ecosystem: Connecting HANA Cloud to Everything

The Honest Summary

Good to know: "HASH Partitioning" vs "RANGE Partitioning" in context of SAP HANA database

Why was this blog post created?

What are the differences?

Where can I find the most important information about the topic of SAP HANA partitioning?

Complexity

What is the conclusion?

Other articles

Do you have further questions?

Contribution

Enterprise AI in Action: SAP Joule Delivers Where Others Don’t

Knowledge Graphs on SAP HANA: From Zero to Enterprise RAG with Triple Store

Introduction

Table of Contents

1. What is a Knowledge Graph?

Why is this different from a regular database?

2. What are RDF Triplets?

Visual Representation

Why Triplets?

3. Why Do We Need Knowledge Graphs in RAG?

The Problem with Vector-Only RAG

How Knowledge Graphs Solve This

The Power of Hybrid Retrieval

4. Extracting Triplets from Text

4a. Rule-Based Extraction with spaCy (NLP Approach)

How This Works — Step by Step

Batch Processing

Limitations of Rule-Based Extraction

4b. LLM-Based Extraction (AI-Powered Approach)

How This Works

Why LLM-Based Extraction is Superior

Batch Processing with Async Concurrency

5. The Multi-Agent Triplet Processing Pipeline

Architecture Overview

The State Object

Stage 1: Analyzer Agent

Stage 2: Semantic Cleaner Agent

Stage 3: Validator Agent

Stage 4: Aggregator Agent

The Orchestrator

6. Storing Triplets in SAP HANA Triple Store

Table Schema

Understanding the Schema

Indexes for Performance

The Embedding Table (for reference)

Inserting Triplets

5. Correct Example Using `I_EnterpriseProjectTP_3`

8. FAQ: Why not `ROLLBACK WORK`?