SAP Community - Python

🚀Building Agents for a Simple Microservice Architecture with FastAPI (Part 2)

2025-08-10T09:01:21.368000+02:00

Previous Blog : Building Collaborative Microservices in Python with FastAPI: Echo & Reverse Agents (Beginner -Part1)

Microservices are a powerful way to design scalable and maintainable applications.

In this blog, we will explore a minimal yet effective microservice setup using FastAPI, perfect for learning and experimentation. This will help to you build better Microservices and deploy in SAP BTP - Kyma

Sample Use Case

A client sends a city name to the Weather Agent. The agent fetches enrichment data from the Data Enricher, generates fake weather data, and returns a combined report. This mimics real-world API composition and data aggregation.

Overview

It consists of two core services:

Fake Weather Agent (weather_agent.py)
Data Enricher (data_enricher.py)

A shell script (run.sh) is included to launch both services on separate ports, simulating a real-world microservice environment.

🌦️ 1. Fake Weather Agent (weather_agent.py)

Purpose: Generates a fake weather report for a given city.

API Endpoint: POST /weather — Accepts a JSON payload with a city name.

How It Works:

Receives a city name from the client.
Optionally calls the Data Enricher service to fetch additional info (e.g., population, country).
Generates random weather data:
- Temperature
- Condition (e.g., sunny, rainy)
- Humidity
- Wind speed
Returns a combined weather report, enriched with city metadata if available.

Tech Stack:

FastAPI for API development
Pydantic for data validation
httpx for asynchronous HTTP calls

from fastapi import FastAPI, HTTPException
from pydantic import BaseModel
import httpx
import os
import random

PORT = int(os.getenv("PORT", 8002))
TARGET = os.getenv("TARGET_URL", "http://localhost:8003")   # downstream agent

app = FastAPI(title="Fake-Weather-Agent")

class Location(BaseModel):
    city: str

class WeatherReport(BaseModel):
    source: str
    city: str
    temperature: float   # °C
    condition: str
    humidity: int        # %
    wind_kmh: float

CONDITIONS = ["Sunny", "Cloudy", "Rain", "Snow", "Thunderstorm"]

@app.post("/weather", response_model=WeatherReport)
async def get_weather(loc: Location):
    """Generate a fake weather report for the given city."""
    # Optionally call another agent (e.g. a “data-enrichment” service)
    async with httpx.AsyncClient() as client:
        try:
            r = await client.post(
                f"{TARGET}/enrich",
                json={"city": loc.city}
            )
            r.raise_for_status()
            extra = r.json()
        except Exception:
            extra = {}

    return WeatherReport(
        source="Fake-Weather-Agent",
        city=loc.city,
        temperature=round(random.uniform(-10, 40), 1),
        condition=random.choice(CONDITIONS),
        humidity=random.randint(20, 95),
        wind_kmh=round(random.uniform(0, 40), 1),
        **extra
    )

🏙️ 2. Data Enricher (data_enricher.py)

Purpose: Provides additional metadata about a city.

API Endpoint: POST /enrich — Accepts a JSON payload with a city name.

How It Works:

Looks up the city in a fake in-memory database.
Returns population and country if found.
If not found, returns default placeholder values.

Tech Stack:

FastAPI
Pydantic

from fastapi import FastAPI
from pydantic import BaseModel

app = FastAPI(title="Data-Enricher")

class EnrichRequest(BaseModel):
    city: str

class EnrichResponse(BaseModel):
    population: int
    country: str

FAKE_DB = {
    "london": {"population": 9_000_000, "country": "UK"},
    "paris":  {"population": 2_100_000, "country": "France"},
    "tokyo":  {"population": 14_000_000, "country": "Japan"},
}

@app.post("/enrich", response_model=EnrichResponse)
def enrich(req: EnrichRequest):
    city = req.city.lower()
    if city not in FAKE_DB:
        return EnrichResponse(population=0, country="Unknown")
    return FAKE_DB[city]

🖥️ 3. Running the Services (run.sh)

Purpose: Starts both services using uvicorn, FastAPI’s ASGI server.
A shell script (run.sh) is provided to run both services on different ports.

How It Works:

Launches Fake Weather Agent on port 8002
Launches Data Enricher on port 8003
Each service runs in its own terminal window

# Terminal 1
uvicorn fake_weather:app --port 8002 --reload

# Terminal 2
uvicorn data_enricher:app --port 8003 --reload

Key Points :

Microservice Communication:
The Weather Agent calls the Data Enricher via HTTP to demonstrate service-to-service communication.
Extensibility:
Easy to add more enrichment services or expand the fake database.
FastAPI Features:
Shows how to use Pydantic models, async endpoints, and response models.
Local Development:
Simple to run both services locally for testing and learning.

Easy way to move zeroes in SAP BTP ABAP(Steampunk), JS & Python

2025-08-10T15:27:33.552000+02:00

Introduction

This is part of the Easy way to write algorithms in ABAP: Series 01. For more algorithms, please check the main blog-post.

Problem

Given an integer array nums, move all 0's to the end of it while maintaining the relative order of the non-zero elements.

Note that you must do this in-place without making a copy of the array.

Example 1:

Input: nums = [0,1,0,3,12]
Output: [1,3,12,0,0]

Example 2:

Input: nums = [0]
Output: [0]

Constraints:

1 <= nums.length <= 104
-231 <= nums[i] <= 231 - 1

Follow up: Could you minimize the total number of operations done?

Solution

Time Complexity: O(n)
Space Complexity: O(1)

ABAP

CLASS zmove_zeroes DEFINITION
  PUBLIC
  FINAL
  CREATE PUBLIC .

  PUBLIC SECTION.
    INTERFACES if_oo_adt_classrun.

  PROTECTED SECTION.
  PRIVATE SECTION.
    " Define a table type for integers
    TYPES ty_nums TYPE STANDARD TABLE OF i WITH EMPTY KEY.

    " Method to move zeroes in-place
    METHODS moveZeroes
      CHANGING lt_nums TYPE ty_nums.

ENDCLASS.

CLASS zmove_zeroes IMPLEMENTATION.

  METHOD if_oo_adt_classrun~main.
    " Initialize the number array with some zeroes and non-zeroes
    DATA(lt_nums) = VALUE ty_nums( ( 0 ) ( 1 ) ( 0 ) ( 3 ) ( 12 ) ).

    " Output the array before moving zeroes
    out->write( |Array before moving zeroes: | ).
    LOOP AT lt_nums INTO DATA(lv_num).
      out->write( lv_num ).
    ENDLOOP.

    " Call the method to move zeroes to the end
    moveZeroes( CHANGING lt_nums = lt_nums ).

    " Output the array after moving zeroes
    out->write( |Array after moving zeroes: | ).
    LOOP AT lt_nums INTO lv_num.
      out->write( lv_num ).
    ENDLOOP.

  ENDMETHOD.

  METHOD moveZeroes.

    DATA(lv_count) = 0. " Counter for non-zero elements

    " First pass: Move all non-zero elements to the front
    LOOP AT lt_nums ASSIGNING FIELD-SYMBOL(<lf_num>).
      IF <lf_num> <> 0.
        " Place the non-zero element at the next available position
        lt_nums[ lv_count + 1 ] = <lf_num>.
        lv_count += 1.
      ENDIF.
    ENDLOOP.

    " Second pass: Fill the rest of the array with zeroes
    WHILE lv_count < lines( lt_nums ).
      lt_nums[ lv_count + 1 ] = 0.
      lv_count += 1.
    ENDWHILE.

  ENDMETHOD.

ENDCLASS.

JavaScript

function moveZeroes(nums) {
    let left = 0;
    for (let right = 0; right < nums.length; right++) {
        if (nums[right] !== 0) {
            // Swap nums[left] and nums[right]
            let temp = nums[left];
            nums[left] = nums[right];
            nums[right] = temp;
            left++;
        }
    }
    return nums;
}

Python

def moveZeroes(nums):
    left = 0
    for right in range(len(nums)):
        if nums[right] != 0:
            nums[left], nums[right] = nums[right], nums[left]
            left += 1
    return nums

N.B: For ABAP, I am using SAP BTP ABAP Environment 2309 Release.

Happy Coding! 🙂

SAP Datasphere Automation : Creating Database user in SAP Datasphere using Datasphere CLI & Python

2025-08-12T08:06:51.294000+02:00

Introduction

One way to access datasphere hana database is via database users. Each database user is linked to one space (except database analysis user). We can create database user vai GUI however, its a long process and prone to errors. In this blog, i will share my work on how i automated the process using SAP Datasphere CLI and Python without the need to login into datasphere GUI.

Old Process:

If you want to create a database user via GUI, you need to perform the below steps.

Login into the datasphere tenant. Make sure you have the required application roles to perform the task.
Go to Space management and select the space. Scroll down and click create in database user.
Provide the username and deploy the space. Save the password and share it with the user.

Automated Process

To overcome the long process, i have divided the python script to perform all these task in sequential manner.

Datasphere CLI Setup : Set the CLI and login into datasphere
Reading the metadata: Read the space metadata using SAP Datasphere CLI space read command
Modifying JSON : Enhance the output JSON with the new user details and deploy it using the space create command
Password Reset : Reset the database user password and save it in a file for sharing.

Set up SAP Datasphere CLI (First time only)

To start working with Datasphere CLI, we need to perform few one time configuration like setting host, login in and setting cache. If you want to login via a different tenant, you need to change the CLIENT_ID and CLIENT_SECRETS as per the tenant.

datasphere config set host {"host_url"}
datasphere login --client-id {"client_id"} --client-secret {"client_secret"}
datasphere config set cache --client-id {"client_id"} --client-secret {"client_secret"}

Once you are done with the initial setup of CLI, we can start working on the main code which will create database users.

Reading space metadata

Since we do not have a direct command to create DB user, we extract the space metadata JSON using CLI. This command output the space metadata JSON into console. I have stored this into a variable that i will use later in modifying the JSON

space_Json = datasphere space read -space "{space"} --definations

Modifying the JSON and deploying it in datasphere

To create a new databased user in space, we need to add the new user details in the dbuser object of the JSON. We can store the space metadata JSON into our local machine and modify it or we can modify it on the go using tempfiles. I have used the later approach as it eliminates the need of JSON management

#Concatenating the space and username
new_db_user = f"{space}#{username}"

#Appending the new_db_user details into the space metadata JSON
space_JSON[space]["spaceDefinition"]["dbusers"][new_db_user] = {
"ingestion":{
        "auditing":{
          "dppRead":{
            "retentionPeriod":21
            "isAuditPolicyActive":True
          },
         
        }
      },
      "consumption":{
        "consumptionWithGrant":false,
        "spaceSchemaAccess":True,
        "scriptServerAccess":false,
        "localSchemaAccess":false,
        "hdiGrantorForCupsAccess":false
}

Once the JSON is modified, write the new JSON into a temporary file. I have stored the file path into a variable called tmp_file_path. I will push this file path to datasphere tenant.

#Pushing the modified JSON to datasphere tenant
datasphere space create --file-path "{tmp_file_path}" -- force-defination-deployment

If you want to delete a database user, remove the entry from the JSON and run the same command and add --enforce-database-user-deletion in the end. If this command is not added, then the deletion will not work.

Password Reset

Once the database user is created, we need to reset the password to store it in a local file for sharing. Below is the command to perform that action

datasphere dbusers password reset --space{"space"} --databaseuser {"new_db_user"}

(Tip: Add a 30 seconds wait time between creating the database user and resetting the password commands because the space deployment takes some time and we cannot reset the password before that).

I have also added an automatic email creation in my workflow and inserting database user details into a local table for future analysis.

Conclusion:

With this automation, we can create database users in datasphere tenant without even login into datasphere. This workflow provides a more robust way to handle creation and maintenance database users.

I would love to know your thoughts on this workflow in the comments below. Lets explore more opportunities of automation using datasphere CLI.

Predictive Stock Transfer & Automatic Purchase Re-Order: Plant-to-Plant A2A Orchestration

2025-08-16T14:04:54.954000+02:00

Use-case: Predictive Stock Transfer & Automatic Purchase Re-Order (Plant-to-Plant A2A scenario driven by real-time APIs)

A fast-moving material in Plant A is kept in stock by an end-to-end, automated flow that

checks forecast demand,
looks for internal surplus in nearby plants, and
automatically creates stock transfers or purchase requisitions as needed.

Prerequisities and needed

SAP S/4HANA Cloud (Inventory & MRP) – On-hand stock, stock in transit, MRP items
SAP Integrated Business Planning (IBP) – Demand forecast for FG-100 at Plant A
SAP Extended Warehouse Management (EWM) – Real-time on-hand stock incl. quarantine
SAP Ariba – Supplier catalog and pricing for external options
External supplier catalog (Ariba-like) – External pricing and lead times
SAP Integration Suite / SAP Event Mesh – Orchestrates the end-to-end flow and publishes events

Sequence – how the APIs interact end-to-end

Step 0 – Trigger
A nightly iFlow in SAP Integration Suite (or SAP Event Mesh) starts the orchestration.

Step 1 – Demand forecast
GET /api/ibp/v1/demandplanning/forecast?material=FG-100&plant=A&weeks=4
→ Returns 1,200 pcs forecasted demand.

Step 2 – Current & projected stock in Plant A
GET /sap/opu/odata/sap/API_MATERIAL_STOCK_SRV/MaterialStock?material=FG-100&plant=A
→ 180 pcs unrestricted, 40 pcs blocked.
GET /sap/opu/odata/sap/API_MRP_COCKPIT_SRV/MrpItems?material=FG-100&plant=A
Result: confirmed receipts 600 pcs
→ Confirmed receipts 600 pcs, so net shortage = 1,200 – 180 – 600 = 420 pcs.

Step 3 – Locate surplus stock in the network
Parallel calls (one per plant):
GET /sap/opu/odata/sap/API_MATERIAL_STOCK_SRV/MaterialStock?material=FG-100&plant=B
→ 300 pcs unrestricted.
GET /sap/opu/odata/sap/API_MATERIAL_STOCK_SRV/MaterialStock?material=FG-100&plant=C
→ 250 pcs unrestricted.

Step 4 – Check ATP (available-to-transfer) incl. transit time
POST /sap/opu/odata/sap/API_ATP_CHECK_SRV/CheckAvailability
Body:

{ 
"material": "FG-100", 
"plant": "B", 
"demandQty": 300, 
"requiredDate": "2024-06-25" 
}

→ Confirms 300 pcs can be delivered by 2024-06-23.
Same for Plant C → 120 pcs available by 2024-06-24.

Step 5 – Decide cheapest internal option
Cost service (custom REST on S/4):
POST /internal/transferCost

{ 
"fromPlants": ["B","C"], 
"toPlant": "A", 
"qty": [300,120] 
}

→ Plant B has the lowest freight cost (€0.05/pc vs €0.07/pc).

Step 6 – Create stock transport order (STO)
POST /sap/opu/odata/sap/API_STOCK_TRANSPORT_ORDER_SRV/A_StockTransportOrder
Body:

{ 
"SupplyingPlant": "B", 
"ReceivingPlant": "A", 
"Material": "FG-100", 
"OrderQuantity": 300, 
"DeliveryDate": "2024-06-23" 
}

→ STO 4500012345 created.

Step 7 – Remaining uncovered quantity
Shortage after internal transfer = 420 – 300 = 120 pcs.
Call Ariba to get best external price:
GET /v2/suppliers/catalog?material=FG-100&qty=120&currency=EUR
→ Supplier S-987 offers €2.30/pc, lead time 7 days.

Step 8 – Create purchase requisition in S/4
POST /sap/opu/odata/sap/API_PURCHASEREQ_PROCESS_SRV/A_PurchaseRequisitionHeader
Body:

{ 
"Material": "FG-100", 
"Plant": "A", 
"Quantity": 120, 
"DeliveryDate": "2024-06-27", 
"SupplierHint": "S-987" 
}

→ PR 1000123456 created.

Step 9 – Notify planners
Publish event to SAP Event Mesh topic /business/plantA/stockReplenished
Payload:

{ 
"material": "FG-100", 
"sto": "4500012345", 
"pr": "1000123456", 
"status": "covered" 
}

Outcome
Plant A will receive 300 pcs from Plant B via an automatically created STO and 120 pcs via a purchase requisition with the cheapest external supplier—no manual intervention, no stock-out, and minimal freight cost.

complete code

#!/usr/bin/env python3
"""
End-to-end A2A flow:
1. Read demand forecast (IBP)
2. Check stock & MRP in Plant A
3. Search surplus stock in Plants B/C
4. Run ATP check
5. Create STO (cheapest internal)
6. Create PR for remaining qty (Ariba best price)
"""

import os, json, math
from datetime import datetime, timedelta
from dotenv import load_dotenv
from requests import Session
from requests_oauthlib import OAuth2Session
from oauthlib.oauth2 import BackendApplicationClient

load_dotenv()

# ---------- CONFIG ----------
MATERIAL = "FG-100"
PLANT_A  = "A"
PLANTS_SURPLUS = ["B", "C"]
DEMAND_WEEKS   = 4
TRANSPORT_DAYS = 2
# ----------------------------

s = Session()

# ---------- 0.  OAUTH TOKEN for S/4 ----------
def s4_token():
    url = f"{os.getenv('S4_HOST')}/oauth/token"
    r = s.post(url,
               auth=(os.getenv('S4_USER'), os.getenv('S4_PASSWORD')),
               data={'grant_type':'client_credentials'})
    r.raise_for_status()
    return r.json()['access_token']

s.headers.update({'Authorization': f"Bearer {s4_token()}"})

# ---------- 1.  IBP – demand forecast ----------
def ibp_forecast():
    url = (f"{os.getenv('S4_HOST')}/sap/opu/odata/sap/"
           f"API_DEMAND_PLANNING_SRV/DemandForecast")
    params = {
        "$filter": f"Material eq '{MATERIAL}' and Plant eq '{PLANT_A}'",
        "$select": "DemandQuantity",
        "$format": "json"
    }
    r = s.get(url, params=params)
    r.raise_for_status()
    total = sum(item['DemandQuantity'] for item in r.json()['d']['results'])
    return total

demand_qty = ibp_forecast()
print("Demand forecast:", demand_qty)

# ---------- 2.  Stock & MRP ----------
def plant_stock(plant):
    url = (f"{os.getenv('S4_HOST')}/sap/opu/odata/sap/"
           f"API_MATERIAL_STOCK_SRV/MaterialStock")
    params = {
        "$filter": f"Material eq '{MATERIAL}' and Plant eq '{plant}' and "
                   f"StorageLocation ne ''",
        "$format": "json"
    }
    r = s.get(url, params=params)
    r.raise_for_status()
    return sum(item['UnrestrictedStockQuantity']
               for item in r.json()['d']['results'])

def plant_receipts(plant):
    url = (f"{os.getenv('S4_HOST')}/sap/opu/odata/sap/"
           f"API_MRP_COCKPIT_SRV/MrpItems")
    params = {
        "$filter": f"Material eq '{MATERIAL}' and Plant eq '{plant}' and "
                   f"MrpElementCategory eq 'AR'",
        "$format": "json"
    }
    r = s.get(url, params=params)
    r.raise_for_status()
    return sum(item['Quantity'] for item in r.json()['d']['results'])

stock_A = plant_stock(PLANT_A)
receipts_A = plant_receipts(PLANT_A)
shortage = max(0, demand_qty - stock_A - receipts_A)
print("Shortage:", shortage)
if shortage == 0:
    print("No action needed.")
    exit()

# ---------- 3.  Surplus stock ----------
surplus = {}
for p in PLANTS_SURPLUS:
    surplus[p] = plant_stock(p)
print("Surplus:", surplus)

# ---------- 4.  ATP check ----------
def atp_ok(plant, qty, req_date):
    url = (f"{os.getenv('S4_HOST')}/sap/opu/odata/sap/"
           f"API_ATP_CHECK_SRV/CheckAvailability")
    body = {
        "Material": MATERIAL,
        "Plant": plant,
        "DemandQuantity": str(qty),
        "RequiredDate": req_date.isoformat()
    }
    r = s.post(url, json=body)
    r.raise_for_status()
    return r.json()['d']['ConfirmedQuantity'] == str(qty)

best_internal = None
needed = shortage
for plant, qty in surplus.items():
    take = min(qty, needed)
    req = datetime.utcnow().date() + timedelta(days=TRANSPORT_DAYS)
    if atp_ok(plant, take, req):
        best_internal = (plant, take)
        break
if best_internal:
    plant, qty = best_internal
    print(f"Best internal: {qty} from {plant}")

    # ---------- 5.  Create STO ----------
    url = (f"{os.getenv('S4_HOST')}/sap/opu/odata/sap/"
           f"API_STOCK_TRANSPORT_ORDER_SRV/A_StockTransportOrder")
    body = {
        "SupplyingPlant": plant,
        "ReceivingPlant": PLANT_A,
        "Material": MATERIAL,
        "OrderQuantity": str(qty),
        "DeliveryDate": req.isoformat()
    }
    r = s.post(url, json=body)
    r.raise_for_status()
    sto = r.json()['d']['StockTransportOrder']
    print("STO created:", sto)
    shortage -= qty

# ---------- 6.  Ariba – best external price ----------
if shortage > 0:
    client = BackendApplicationClient(client_id=os.getenv('ARIBA_CLIENT_ID'))
    oauth = OAuth2Session(client=client)
    token = oauth.fetch_token(
        token_url=os.getenv('ARIBA_TOKEN_URL'),
        client_id=os.getenv('ARIBA_CLIENT_ID'),
        client_secret=os.getenv('ARIBA_CLIENT_SECRET')
    )
    url = "https://api.ariba.com/v2/suppliers/catalog"
    params = {
        "material": MATERIAL,
        "qty": shortage,
        "currency": "EUR"
    }
    r = oauth.get(url, params=params)
    r.raise_for_status()
    best = min(r.json()['offers'], key=lambda x: float(x['price']))
    print("Best supplier:", best['supplier'], best['price'])

    # ---------- 7.  Create PR ----------
    url = (f"{os.getenv('S4_HOST')}/sap/opu/odata/sap/"
           f"API_PURCHASEREQ_PROCESS_SRV/A_PurchaseRequisitionHeader")
    body = {
        "Material": MATERIAL,
        "Plant": PLANT_A,
        "Quantity": str(shortage),
        "DeliveryDate": (datetime.utcnow().date()
                         + timedelta(days=int(best['leadtime']))).isoformat(),
        "SupplierHint": best['supplier']
    }
    r = s.post(url, json=body)
    r.raise_for_status()
    pr = r.json()['d']['PurchaseRequisition']
    print("PR created:", pr)

print("Flow finished.")

SAP Datasphere & Python : One click to export data of multiple views in Excel/CSV

2025-08-19T08:22:39.531000+02:00

Introduction

Currently, SAP Datasphere only allows data export through Analytical Models. That means for every fact view, one has to create a separate Analytical Model just to download the data. It’s not ideal, especially when you have many views. Exporting them one by one becomes slow and repetitive.

To solve this, Python script was developed that connects to SAP Datasphere, runs a query on each view from list, and saves the results as Excel files in a local drive path which is predefined. Now just run the script, and it exports everything in one go—no manual effort needed.

Requirements

Before running the script, install the following Python packages:

These packages enable secure connectivity to SAP Datasphere and support efficient data handling.

pip install hdbcli
pip install sqlalchemy
pip install sqlalchemy-hana

Creating a Database User in SAP Datasphere

To enable Python connectivity, create a database user:

Navigate to Space Management in SAP Datasphere
Select the relevant space
Click on Database Access
Create a new user with read access
Copy the host, port, username, and password

Helpful links : Create DB User in Datasphere Space

Python Script

# 📦 Install required packages (run these in your terminal or notebook)
# pip install hdbcli              # SAP HANA database client
# pip install sqlalchemy          # SQL toolkit and ORM for Python
# pip install sqlalchemy-hana     # SAP HANA dialect for SQLAlchemy

# 📚 Import necessary libraries
import pandas as pd               # For data manipulation and Excel export
from hdbcli import dbapi          # SAP HANA DBAPI for direct connection
import warnings                   # To suppress unnecessary warnings
import os                         # For file path and directory handling

# 🚫 Suppress warnings for cleaner output
warnings.filterwarnings('ignore')

# 🔐 Define SAP Datasphere connection parameters
# 👉 Replace the placeholders below with your actual connection details
db_user = '<your_database_user>'           # User with access to target schema
db_password = '<your_secure_password>'     # Password (handle securely)
db_host = '<your_datasphere_host_url>'     # Host URL (e.g., xyz.hanacloud.ondemand.com)
db_port = 443                               # Default HTTPS port for SAP HANA Cloud
db_schema = '<your_schema_name>'           # Target schema containing views

# 📁 Ensure output folder exists for Excel exports
output_folder = r'C:\Datasphere\Excel export'  # Update path as needed
os.makedirs(output_folder, exist_ok=True)

# 📋 Define list of views to extract data from
# 👉 Add or modify view names based on your schema
view_list = ['VIEW_1', 'VIEW_2']  # Example views

try:
    # 🌐 Establish secure connection to SAP Datasphere
    connection = dbapi.connect(
        address=db_host,
        port=db_port,
        user=db_user,
        password=db_password,
        encrypt=True,
        sslValidateCertificate=True
    )
    print("✅ Connected to SAP Datasphere")

    cursor = connection.cursor()

    # 🔁 Loop through each view and export its data
    for view_name in view_list:
        try:
            # 📊 Construct and execute SQL query
            sql_query = f'SELECT * FROM "{db_schema}"."{view_name}"'
            print(f"📊 Executing query: {sql_query}")
            cursor.execute(sql_query)

            # 📥 Fetch results and convert to DataFrame
            rows = cursor.fetchall()
            columns = [desc[0] for desc in cursor.description]
            df = pd.DataFrame(rows, columns=columns)

            # 📤 Export DataFrame to Excel
            output_path = os.path.join(output_folder, f'{view_name}.xlsx')
            df.to_excel(output_path, index=False)
            print(f"✅ Data from '{view_name}' saved to: {output_path}")

        except dbapi.Error as view_err:
            print(f"❌ Error querying '{view_name}': {view_err}")

except dbapi.Error as db_err:
    print(f"❌ Database error: {db_err}")
except Exception as ex:
    print(f"⚠️ Unexpected error: {ex}")
finally:
    # 🔒 Ensure connection is closed gracefully
    if 'connection' in locals():
        connection.close()
        print("🔒 Connection closed")

Script Capabilities

Establishes secure connection to SAP Datasphere
Executes queries on each listed view
Saves data from each view into a separate Excel file
Stores all files in a defined folder

Only connection details and view names need to be updated. The script handles the rest.

One-Click Execution with a .bat File

To simplify execution, create a RunExport.bat file to run the Python script with a double-click.

Double-clicking the file will automatically export all views to Excel without opening a terminal

 off
REM Activate Python and run the Export script

REM Change to the script directory
cd /d "C:\Datasphere\Excel export"

REM Run the Python script
python Export.py

REM Pause to keep the window open (optional)
pause

Example

Before Execution

Double Click on "RunExcel.bat" file

Post Execution

Conclusion
This automation simplifies data exports from SAP Datasphere, especially when working with multiple views.

It reduces manual effort
improves consistency and saves time.
ideal for recurring tasks or scheduled jobs.

For setup support or customization, feel free to connect.

Thanks

Vikas Parmar

SAP Datasphere SAP Business Data Cloud Python

SAP Databricks in SAP Business Data Cloud – a Typical Machine Learning Workflow

2025-09-04T04:16:17.227000+02:00

With SAP Databricks you have access to an amazing set of capabilities to work with your BDC Data Products and other data.

This blog post is part of a series exploring SAP Databricks in SAP Business Data Cloud:

Part 1 – SQL analytics with SAP Data Products
Part 2 – Build and deploy Mosaic AI and Agent Tools
Part 3 – How to use AutoML to forecast sales data
Part 4 – Connect SAP Data Products with non-SAP data from AWS S3
Part 5 – End-to-end integration: SAP Databricks, SAP Datasphere, and SAP Analytics Cloud
Part 6 – Create inferences for application integration with SAP Build

Part 7 - SAP Databricks in SAP Business Data Cloud – a Typical Machine Learning Workflow

In this blog post we’ll look at the typical workflow you would undertake when trying to train a machine learning model.

Visualise and understand your data
Optimise for hyperparameters to tune your model
Explore hyperparameter sweep results with MLflow
Register the best performing model in MLflow
Apply the registered model with batch inference and Databricks Model Serving

We’ll use a classic machine learning dataset to predict whether a wine is of high quality or not (a data classification problem). Of course you have access to a range of SAP Data Products, but by using this dataset you don’t even need a connected S/4HANA system to follow along. The dataset also comes built-in with SAP Databricks.

Wine Quality Classification

We will train a binary classification model to predict the quality of Portuguese "Vinho Verde" wine based on the wine's physicochemical properties.

The dataset is from the UCI Machine Learning Repository, presented in Modelling wine preferences by data mining from physicochemical properties [Cortez et al., 2009]. And the good news is that this dataset comes preloaded with your Databricks system.

Let’s create a new notebook in our SAP Databricks system and in a new cell we will install some module dependencies.

%pip install --upgrade -Uqqq mlflow>=3.0 xgboost hyperopt
%restart_python

Installs:

The latest mlflow for experiment tracking and general MLOps
xgboost being a fantastic and very popular machine learning model architecture (it dominates many Kaggle competitions)
hyperopt is a python library used for hyperparameter optimisation. It intelligently searches for the optimal hyperparameters to use when training a machine learning model.
The `%restart_python` magic command it necessary in Databricks notebooks because they use long running Python processes and this ensures that the system path and any newly installed python packages are being used.

Next, we create a cell to connect MLFlow to the Databricks Unity Catalog (it would otherwise use an sqlite data store) and create a few constants that will be used later:

import mlflow
mlflow.set_registry_uri("databricks-uc")

CATALOG_NAME = "workspace"
SCHEMA_NAME = "default"
MODEL_NAME = "wine_quality_classifier"

Read and Understand the DATA

Read the white wine quality and red wine quality CSV datasets and merge them into a single DataFrame. Note theses datasets come with your Databricks system.

import pandas as pd

white_wine = pd.read_csv("/databricks-datasets/wine-quality/winequality-white.csv", sep=";")
red_wine = pd.read_csv("/databricks-datasets/wine-quality/winequality-red.csv", sep=";")

Merge the two DataFrames into a single dataset, with a new binary feature "is_red" that indicates whether the wine is red or white.

red_wine['is_red'] = 1
white_wine['is_red'] = 0

data = pd.concat([red_wine, white_wine], axis=0)

# cast to float as a best practice (avoids dtype issues with NaN's later)
data["is_red"] = data["is_red"].astype("float32")

# Remove spaces from column names
data.rename(columns=lambda x: x.replace(' ', '_'), inplace=True)

Now we have one dataset with a mix of white and red wines.

Visualize data

Before training a model, explore the dataset using popular charting libraries: Seaborn and Matplotlib.

Plot a histogram of the dependent variable, quality.

import seaborn as sns
sns.displot(data.quality, kde=False)

Looks like quality scores are normally distributed between 3 and 9. Define a wine as high quality if it has quality >= 7.

high_quality = (data.quality >= 7)
data.quality = high_quality

# cast to float as a best practice (avoids dtype issues with NaN's later)
data["quality"] = data["quality"].astype("float32")

Box plots are useful for identifying correlations between features and a binary label. Create box plots for each feature to compare high-quality and low-quality wines. Significant differences in the box plots indicate good predictors of quality.

import matplotlib.pyplot as plt

dims = (3, 4)

f, axes = plt.subplots(dims[0], dims[1], figsize=(25, 15))
axis_i, axis_j = 0, 0
for col in data.columns:
  if col == 'is_red' or col == 'quality':
    continue # Box plots cannot be used on indicator variables
  sns.boxplot(x=high_quality, y=data[col], ax=axes[axis_i, axis_j])
  axis_j += 1
  if axis_j == dims[1]:
    axis_i += 1
    axis_j = 0

In the above box plots, a few variables stand out as good univariate predictors of quality.

In the alcohol box plot, the median alcohol content of high-quality wines is greater than even the 75th quantile of low-quality wines. High alcohol content is correlated with quality.
In the density box plot, low quality wines have a greater density than high quality wines. Density is inversely correlated with quality.

Preprocess data

Before training a model, check for missing values and split the data into training and validation sets.

data.isna().any()

There are no missing values.

Note: Often you will need to take advantage of feature engineering at this step. This is where you can build new features as combinations of your existing data… for example multiplying two existing feature columns together to create a new column may enable the model to find better patterns in the data.

For time series data it can be very helpful to generate a new feature column called “Qtr” for example to show which qtr of the year that data point is in based on a date. You will need to experiment with this…

Prepare the dataset to train a baseline model

Split the input data into 3 sets:

Train (60% of the dataset used to train the model)
Validation (20% of the dataset used to tune the hyperparameters)
Test (20% of the dataset used to report the true performance of the model on an unseen dataset)

from sklearn.model_selection import train_test_split

X = data.drop(["quality"], axis=1)
y = data.quality

# Split out the training data
X_train, X_rem, y_train, y_rem = train_test_split(X, y, train_size=0.6, random_state=123)

# Split the remaining data equally into validation and test
X_val, X_test, y_val, y_test = train_test_split(X_rem, y_rem, test_size=0.5, random_state=123)

Train a baseline model

This task seems well suited to a random forest classifier, since the output is binary and there may be interactions between multiple variables.

Build a simple classifier using scikit-learn and use MLflow to keep track of the model's accuracy and save the model for later use.

Note: When this cell is executed an MLFlow Experiment will be created by default (automatically) using the full path of this Notebook. In production experiments its best practice to set the Experiment name with mlflow.set_experiment() because you may work on the problem over multiple Notebooks and/or users.

import mlflow.pyfunc
import mlflow.sklearn
import numpy as np
import sklearn
from sklearn.ensemble import RandomForestClassifier
from sklearn.metrics import roc_auc_score
from mlflow.models.signature import infer_signature
from mlflow.utils.environment import _mlflow_conda_env
import cloudpickle
import time

# The predict method of sklearn's RandomForestClassifier returns a binary classification (0 or 1). 
# The following code creates a wrapper function, SklearnModelWrapper, that uses 
# the predict_proba method to return the probability that the observation belongs to each class. 

class SklearnModelWrapper(mlflow.pyfunc.PythonModel):
  def __init__(self, model):
    self.model = model
    
  def predict(self, context, model_input):
    return self.model.predict_proba(model_input)[:,1]

# mlflow.start_run creates a new MLflow run to track the performance of this model. 
# Within the context, you call mlflow.log_param to keep track of the parameters used, and
# mlflow.log_metric to record metrics like accuracy.
with mlflow.start_run(run_name='rf_baseline_n10'):
  n_estimators = 10
  model = RandomForestClassifier(n_estimators=n_estimators, random_state=np.random.RandomState(123))
  model.fit(X_train, y_train)

  # predict_proba returns [prob_negative, prob_positive], so slice the output with [:, 1]
  predictions_test = model.predict_proba(X_test)[:,1]
  auc_score = roc_auc_score(y_test, predictions_test)
  mlflow.log_param('n_estimators', n_estimators)
  # Use the area under the ROC curve as a metric.
  mlflow.log_metric('auc', auc_score)
  wrappedModel = SklearnModelWrapper(model)
  
  # MLflow contains utilities to create a conda environment used to serve models.
  # The necessary dependencies are added to a conda.yaml file which is logged along with the model.
  conda_env = _mlflow_conda_env(
        additional_conda_deps=None,
        additional_pip_deps=["cloudpickle=={}".format(cloudpickle.__version__), "scikit-learn=={}".format(sklearn.__version__)],
        additional_conda_channels=None,
    )

  # Here we log the model and register it to Unity Catalog in one go.
  # MLflow automatically generates model signatures when you provide
  # an `input_example` during model logging. This works for all model 
  # flavors and is the recommended approach for most use cases.
  # By registering this model to Unity Catalog, you can easily reference
  # the model from anywhere within Databricks.
  # 
  sample_input = X_train.head(5)

  model_version = mlflow.pyfunc.log_model(
    name="rf_baseline",
    python_model=wrappedModel,
    conda_env=conda_env,
    input_example=sample_input,
    registered_model_name=MODEL_NAME,
  )

Note how the trained model is registered when logging it to MLFlow. This can be done separately as we will see later. If you are running many experiments there is no need to register every model but only the best model.

Review the learned feature importances output by the model. As illustrated by the previous boxplots, alcohol and density are important in predicting quality.

feature_importances = pd.DataFrame(model.feature_importances_, index=X_train.columns.tolist(), columns=['importance'])
feature_importances.sort_values('importance', ascending=False)

You logged the Area Under the ROC Curve (AUC) to MLflow. Click the Experiment icon in the right sidebar to display the Experiment Runs sidebar. The model achieved an AUC of 0.854. A random classifier would have an AUC of 0.5, and higher AUC values are better.

The ROC AUC is a good evaluation metric for binary classification problems like we have here (is good quality / is not good quality).

Next, assign this model the "Best" tag, and load it into this notebook from Unity Catalog.

from mlflow.tracking import MlflowClient

client = MlflowClient()
client.set_registered_model_alias(MODEL_NAME, "Best", model_version.registered_model_version)

In Unity Catalog, the model version now has the tag "Best". You can now refer to the model using the path models:/{model_name}@Best.

Experiment with a new model

The random forest model performed well even without hyperparameter tuning.

Let's now try and do better and use the xgboost library to train a more accurate model. Run a hyperparameter sweep to train multiple models in parallel, using Hyperopt and Trials. As before, MLflow tracks the performance of each parameter configuration.

Note: We must use Trials and not SparkTrials in SAP Databricks, because SparkTrials tries to access the underlying JVM which is not supported on serverless compute.

We use the validation dataset here for hyperparameter search.

Note this training cell takes over 20mins to complete!

from hyperopt import fmin, tpe, hp, Trials, STATUS_OK
from hyperopt.pyll import scope
from math import exp
import mlflow.xgboost
import numpy as np
import xgboost as xgb

search_space = {
  'max_depth': scope.int(hp.quniform('max_depth', 4, 100, 1)),
  'learning_rate': hp.loguniform('learning_rate', -3, 0),
  'reg_alpha': hp.loguniform('reg_alpha', -5, -1),
  'reg_lambda': hp.loguniform('reg_lambda', -6, -1),
  'min_child_weight': hp.loguniform('min_child_weight', -1, 3),
  'objective': 'binary:logistic',
  'seed': 123, # Set a seed for deterministic training
}

def train_model(params):
  # With MLflow autologging, hyperparameters and the trained model are automatically logged to MLflow.
  mlflow.xgboost.autolog()
  with mlflow.start_run(nested=True):
    train = xgb.DMatrix(data=X_train, label=y_train)
    validation = xgb.DMatrix(data=X_val, label=y_val)
    # Pass in the validation set so xgb can track an evaluation metric. XGBoost terminates training when the evaluation metric
    # is no longer improving.
    booster = xgb.train(params=params, dtrain=train, num_boost_round=1000,
                        evals=[(validation, "validation")], early_stopping_rounds=50)
    validation_predictions = booster.predict(validation)
    auc_score = roc_auc_score(y_val, validation_predictions)
    mlflow.log_metric('auc', auc_score)

    # Don't register the model in one-step here - let the hyperparameter search find the best one first.
    #signature = infer_signature(X_train, booster.predict(train))
    #mlflow.xgboost.log_model(booster, name="xgboost", signature=signature)
    sample_input = X_train.head(5)
    mlflow.xgboost.log_model(booster, name="xgboost", input_example=sample_input)
    
    # Set the loss to -1*auc_score so fmin maximizes the auc_score
    return {'status': STATUS_OK, 'loss': -1*auc_score, 'booster': booster.attributes()}

# Use Trials instead of SparkTrials
trials = Trials()

# Run fmin within an MLflow run context so that each hyperparameter configuration is logged as a child run of a parent
# run called "xgboost_models" .
with mlflow.start_run(run_name='xgboost_models'):
  best_params = fmin(
    fn=train_model, 
    space=search_space, 
    algo=tpe.suggest, 
    max_evals=96,
    trials=trials,
  )

Use MLflow to view the results

Open up the Experiments sidebar to see the MLflow runs. Click on Date next to the down arrow to display a menu, and select 'auc' to display the runs sorted by the auc metric. The highest auc value is ~0.90. Remember that this is against the validation data.

Hyperparameter tuning (runs) score against the validation dataset!

MLflow tracks the parameters and performance metrics of each run. Click the External Link icon at the top of the Experiment Runs sidebar to navigate to the MLflow Runs Table.

Update the best version of the wine_quality_classifier model

Earlier, you saved the baseline model to Unity Catalog with the name wine_quality_classifier. Now you can update wine_quality_classifier to a more accurate model from the hyperparameter sweep. Because you used MLflow to log the model produced by each hyperparameter configuration, you can use MLflow to identify the best performing run and save the model from that run to Unity Catalog.

best_run = mlflow.search_runs(order_by=['metrics.auc DESC']).iloc[0]
print(f'AUC of Best Run: {best_run["metrics.auc"]}')

new_model_version = mlflow.register_model(f"runs:/{best_run.run_id}/model", MODEL_NAME)

# Registering the model takes a few seconds, so add a small delay
import time
time.sleep(15)

Click Models in the left sidebar to see that the wine_quality_classifier model now has a new versions. Assign the "Best" alias to the new version.

from mlflow.tracking import MlflowClient

client = MlflowClient()
client.set_registered_model_alias(MODEL_NAME, "Best", new_model_version.version)

Clients that call load_model() using the "Best" alias now get the new model.

>> Let's get the AUC score against the Test data:

model = mlflow.pyfunc.load_model(f"models:/{MODEL_NAME}@Best")

from sklearn.metrics import roc_auc_score
print(f'AUC: {roc_auc_score(y_test, model.predict(X_test))}')

The new version achieved a better score (AUC = 0.90) on the test set.

Batch inference

There are many scenarios where you might want to evaluate a model on a corpus of new data. For example, you may have a fresh batch of data or may need to compare the performance of two models on the same corpus of data.

Evaluate the model on data stored in a Delta table, using Spark to run the computation in parallel.

# To simulate a new corpus of data, save the existing X_train data to a Delta table. 
# In the real world, this would be a new batch of data.
spark_df = spark.createDataFrame(X_train)

table_name = f"{CATALOG_NAME}.{SCHEMA_NAME}.wine_data"

(spark_df
  .write
  .format("delta")
  .mode("overwrite")
  .option("overwriteSchema",True)
  .saveAsTable(table_name)
)

Load the model into a Spark UDF, so it can be applied to the Delta table.

apply_model_udf = mlflow.pyfunc.spark_udf(spark, f"models:/{MODEL_NAME}@Best")

# Read the "new data" from the Unity Catalog table
new_data = spark.read.table(f"{CATALOG_NAME}.{SCHEMA_NAME}.wine_data")

from pyspark.sql.functions import struct

# Apply the model to the new data
udf_inputs = struct(*(X_train.columns.tolist()))

new_data = new_data.withColumn(
  "prediction",
  apply_model_udf(udf_inputs)
)

display(new_data)

Each row now has an associated prediction. Note that the xgboost function is using the objective "binary:logistic" so the predictions shown here are probabilities.

We also add a is_good_quality column:

from pyspark.sql.functions import col, when

new_data = new_data.withColumn("prediction", col("prediction")[0])

new_data = new_data.withColumn(
  "is_good_quality",
  when(col("prediction") > 0.5, True).otherwise(False)
)
display(new_data)

Overwrite the table with the new columns:

(new_data
  .write
  .format("delta")
  .mode("overwrite")
  .option("overwriteSchema", True)
  .saveAsTable(table_name)
)

# Enable Change Data Feed for the table
# Seems that we can only add this option via SQL!!
spark.sql(f"ALTER TABLE {table_name} SET TBLPROPERTIES (delta.enableChangeDataFeed = true)")

Serve the model

To productionize the model for low latency predictions, use Mosaic AI Model Serving to deploy the model to an endpoint. The following cell shows how to use the MLflow Deployments SDK to create a model serving endpoint (which can also be done view the Serving menu on the left).

First of all, let’s just show the current model’s name and best version

# Get the model vesion we tagged as @Best
from mlflow.tracking import MlflowClient
best_ver = MlflowClient().get_model_version_by_alias(MODEL_NAME, "Best").version
print(MODEL_NAME, best_ver)

Check if any versions of this model are already being served and delete them

from mlflow.deployments import get_deploy_client

client = get_deploy_client("databricks")
endpoints = client.list_endpoints()

deployed = False
deployed_versions = []
for ep in endpoints:
    ep_detail = client.get_endpoint(ep["name"])
    for entity in ep_detail.get("config", {}).get("served_models", []):
        if entity.get("model_name") == MODEL_NAME or entity.get("model_name").endswith(MODEL_NAME):
            deployed = True
            deployed_versions.append(str(entity.get("model_version")))
            # Delete the serving endpoint if the model is already deployed
            client.delete_endpoint(ep["name"])

if deployed_versions:
    deployed_versions_str = ", ".join(deployed_versions)
else:
    deployed_versions_str = ""

display(spark.createDataFrame([{"model_name": MODEL_NAME, "deployed": deployed, "deployed_versions": deployed_versions_str, "action": "deleting..."}]))

Creating the endpoint can take 5+ minutes...

# the "name" property can't include special chars so we drop the catalog and schema from the model_name

from mlflow.deployments import get_deploy_client

client = get_deploy_client("databricks")
endpoint = client.create_endpoint(
    name="wine-model-endpoint",
    config={
        "served_entities": [
            {
                "name": MODEL_NAME,
                "entity_name": f"{CATALOG_NAME}.{SCHEMA_NAME}.{MODEL_NAME}",
                "entity_version": best_ver,
                "workload_size": "Small",
                "scale_to_zero_enabled": True
            }
        ],
      }
)

Test the Model Serving Endpoint

Navigate to User Settings -> Developer and create an Access Token for calling the serving endpoint.

Ensure the model is being served as this can take 5-10 mins.

In the below cells the notebook will:

Ask for your access token
Setup a payload (the required inputs for your model)
Call the model serving endpoint!

from getpass import getpass
token = getpass("🔑  Paste your Databricks token: ")

Setup the api call request payload with sample wine quality data:

payload = {
  "dataframe_split": {
    "columns": [
      "fixed_acidity", "volatile_acidity", "citric_acid", "residual_sugar",
      "chlorides", "free_sulfur_dioxide", "total_sulfur_dioxide",
      "density", "pH", "sulphates", "alcohol", "is_red"
    ],
    "data": [
      [7.3, 0.19, 0.27, 1.6, 0.027, 35, 136, 0.99248, 3.38, 0.54, 11, 0],
      [7.8, 0.88, 0.00, 2.6, 0.098, 25, 67, 0.9968, 3.20, 0.68, 9.8, 1]
    ]
  }

Use the python requests package to make an api call to the SAP Databricks Model Serving Endpoint.

Make sure to update the endpoint uri below to match your current SAP Databricks system!

import os, json, requests

url   = "https://<uri>.cloud.databricks.com/serving-endpoints/wine-model-endpoint/invocations"

resp = requests.post(
    url,
    headers={
        "Authorization": f"Bearer {token}",
        "Content-Type": "application/json"
    },
    data=json.dumps(payload),
    timeout=60
)

if resp.status_code == 404:
    print("The endpoint is still deploying.")
else:
    print(resp.json())
    for i, score in enumerate(resp.json()["predictions"], start=1):
        is_good = score >= 0.5          # quality flag
        print(f"Row {i}: {score:.3f}  ➜  Good quality? {is_good}")

You can use this API endpoint to perform inference from your own applications – as is done with blog post : “Part 6 – Create inferences for application integration with SAP Build ” in the series.

Conclusion

We’ve seen in this notebook, if you have followed along, the typical pattern of training a machine learning model.

It always starts with understanding the data available. Visualising the data with histograms and box plots as shown here can be a great help. Use tools like ChatGPT to assist in the best ways to flesh out information about your data
It can often be helpful to create a quick baseline model just to see that we can do better than random luck with the training data
Use a hyperparameter optimisation tool to help search for the ideal parameters to tune the best model. Be very careful with the split of training, validation and test data and ensure that there can never be any overlap. Research how to do this if using time-series data
Use MLFlow to log training experiments and their generated models. Assign tags to highlight specific or “best” models.
Look at Batch Inference or Model Serving.
- The former (batch) being ideal if you want to batch score a table of data and potentially share it back to BDC to be used in analytics models. Make use of scheduled notebooks to keep the data up to date and to train the model on new data
- Use Model Serving to expose an endpoint for real-time applications to make predictions.

Hello Python: My First Script in SAP BAS Connecting to HANA Cloud

2025-09-26T13:05:26.454000+02:00

Credit: @Vitaliy-R Your startup blogs kindled my interest to explore working with Python in SAP ecosystem. Python in BAS and Jupyter in BAS

When I first started exploring SAP Business Application Studio (BAS), I was curious about how Python could fit into the SAP landscape. I’ve mostly associated BAS with HANA artefacts(SQLScript, hdbcalculationview, hdbreptask etc.) and CAP artefacts, so writing a Python script inside BAS felt like venturing into new territory. My goal was simple: write a basic script and connect it to SAP HANA Cloud. What I discovered along the way is that Python not only works smoothly in BAS but also makes it easy to interact with HANA Cloud, opening up opportunities for data exploration, automation, and integration in a way that feels both modern and approachable.

Before jumping into the Python script, I had to get my environment ready in SAP Business Application Studio (BAS). Here’s what I set up:

A BAS dev space with a full-stack cloud application space since it supports multiple runtimes, including Python. I had a space with HANA Native Application type. Since the Python tools extension is not added by default, I edited the space to select the Python tools in the additional extension options.

HANA Dev Space Python extension

Note: For initial steps to check the Python version, Jupyter notebook and set ups refer to the blogs listed at the start.

Use Case: I attempted to achieve the following:

Establish a connection to HANA Cloud
Execute an SQL query on a table/view
Display the results

In the BAS, I created a project from Template: SAP HANA Database Project

Next step: Create a notebook file.

My guide to connect to HANA Cloud: Connect to HANA Cloud

When I first tried importing hdbcli into my Jupyter Notebook within BAS, I ran into the same ModuleNotFoundError. Even though I had already installed hdbcli In the terminal, the notebook kernel wasn’t recognizing it. On some search and prompting with GPT( 😁), I understood that it's a common issue because Jupyter can run in a different Python environment than the terminal. The fix was simple: I ran

import sys
!{sys.executable} -m pip install hdbcli

directly in a notebook cell. This ensures that the HANA client is installed in the same environment as the notebook kernel. After this step, I could successfully import dbapi and connect to HANA Cloud without any errors. It was a small but important lesson about Python environments in BAS, especially when using Jupyter.

With the hdbcli package installed and working in my Jupyter Notebook, I was ready to write my first Python script to connect to SAP HANA Cloud.

In the next cell, I imported hdbcli in this notebook.

import hdbcli
print(hdbcli.__file__)

The next step was to gain access to the dbapi interface, which allows you to establish connections, execute SQL queries, and fetch results from your HANA Cloud instance. This simple import is the gateway to working with HANA directly from Python.

from hdbcli import dbapi

The next step is to establish a connection to your HANA Cloud instance. This requires specifying the host, port, username, and password.

After connecting, you can create a cursor object to execute SQL statements. An SQL statement, preferably a Select Query to test the retrieval of data from HANA Cloud. In my case, I used a Select with count on the number of records in a view. Once the variables were ready, execute the connection cursor object.

Note: in the SQL variable, use single quotes and a semicolon at the end of the query. (beginner tip 🙂 )

Now is the time to test the data retrieval from the script and compare it with the Database Explorer.

Drum roll....🥁

Hurray 🎉

Completing my first Python script in SAP Business Application Studio and connecting it to HANA Cloud was an exciting milestone. From the initial curiosity to the small hurdles like installing hdbcli in the notebook and finally seeing my script return results, every step felt like a mini victory.

That simple output from HANA Cloud made all the effort worthwhile and gave me a real sense of accomplishment.

This experience has sparked my curiosity to explore more complex queries, data analysis, and automation using Python in SAP.

I hope my journey inspires others to take that first step and discover how fun and powerful working with Python and HANA Cloud can be.

Chao.

SAP CPQ 2511 - Scripting for Custom Quote Actions & Quote Item Custom Fields Access Control

2025-10-26T14:38:16.450000+01:00

This blog introduces the scripting enhancements in SAP CPQ 2511, tested and verified in my pre-release version. The scripting examples provided here are optimized for implementation and serve as a valuable resource for CPQ developers, functional consultants, and integration specialists seeking for practical guidance.

Special thanks to Nikola & Pavithran for upgrading to 2511 for pre-release testing

Follow for more updates from : Yogananda Muthaiah

Dynamic Access Control for Quote Item Custom Fields

Access level permissions on Quote Item Custom Fields can now be dynamically set through scripting in Quote 2.0. This feature allows users to control the editability, read-only status, and visibility of fields, enhancing customization and security in managing quotes.

Here's an example script:

from Scripting.Quote import QuoteFieldAccessLevel

for item in context.PagedItems:
    QuoteFieldAccessContext.SetAccessLevel(item, 'TargetPrice', QuoteFieldAccessLevel.Hidden)
    QuoteFieldAccessContext.SetAccessLevel(item, 'TargetPrice', QuoteFieldAccessLevel.Readonly)
    QuoteFieldAccessContext.SetAccessLevel(item, 'TargetPrice', QuoteFieldAccessLevel.Editable)


eachItem = context.Quote.GetItemByItemNumber(1)
QuoteFieldAccessContext.SetAccessLevel(eachItem, 'TargetPrice', QuoteFieldAccessLevel.ReadOnly)  

QuoteFieldAccessContext.SetColumnAccessLevel('TargetPrice', QuoteFieldAccessLevel.ReadOnly)


QuoteFieldAccessContext.SetAccessLevelForProductType(39, 'TargetPrice', QuoteFieldAccessLevel.ReadOnly)

QuoteFieldAccessContext.SetAccessLevelForSection('ProductType', 'TargetPrice', QuoteFieldAccessLevel.Editable)

QuoteFieldAccessContext.SetAccessLevelForCartTotal

Executing Custom Quote Actions via Scripting

As of the 2511 release, you can execute custom quote actions in Quote 2.0 within the Workflow context through scripting.

The SAP CPQ system checks all the permissions and conditions defined in the Workflow for these actions to ensure that only available custom quote actions are executed. For each available action, the system performs pre-actions and post-actions and sends notifications. The list of available custom quote actions is also returned in scripting.

Here's an example script:

customQuoteactionAvailable = WorkflowExecutor.IsActionAvailableForQuote(3102, context.Quote.Id)
WorkflowExecutor.ExecuteActionOnQuote(3102, context.Quote.Id)

Enhanced ExtenalItemID Editing via API and Scripting

As of 2511 release, extenalItemID can be edited, updated and accessed through API and scripting. This enhancement enables more flexible and efficient data management and integration scenarios.

to enable ExternalItemId to be changed from the Scripting

Here's an example script:

Items = context.Quote.GetAllItems()
Items[0].ExternalItemId = 'Testing from Yoga to update ExternalItemId '

Trace.Write(Items[0].ExternalItemId)

Improved RestClient Methods for Decompressed Responses

RestClients previously returned unreadable strings for compressed responses. To address this issue and support the decompression of the response body for compressed responses using Gzip or Deflate compression methods, a new decompressResponse parameter was added. By default, it is set to False, maintaining current functionality.

Setting it to True automatically decompresses responses into readable JSON, allowing users to handle compressed responses without changing existing calls unless needed.

Here's an example script:

authorization_token = RestClient.GetBasicAuthenticationHeader("YMUTHAIAH", "XXXXXXXXXXXXXXX")
headers = { "Authorization": authorization_token}
token_url = "https://XXXXXXXXX.de1.demo.crm.cloud.sap/sap/c4c/api/v1/iam-service/token"
token = RestClient.Get(token_url,headers )
 
aa = token.value.access_token
 
url = "https://XXXXXXXXX.de1.demo.crm.cloud.sap/sap/c4c/api/v1/document-service/documents/"
headers1 = { "Authorization": "Bearer " + str(aa),
           'Accept': 'application/json; charset=UTF-8'}
 
json_body = {
    'isSelected':'false',
    'isDisplayDocument':'true',
    'fileName': 'CPQ-Document.xlsx',
    'category': 'DOCUMENT',
    'type': '10001'
}
 
newdata=JsonHelper.Deserialize(JsonHelper.Serialize(json_body))
response = RestClient.Post(url, newdata, headers1, True)

Script Fix for Quote 2.0 Alternatives

The IronPython script issue in Quote 2.0, where newly added alternative items were incorrectly flagged and excluded from calculations, has been resolved.

This fix ensures accurate total and product type calculations, making it essential for users who rely on scripting to manage quote alternatives efficiently.

Here's an example script:

quote = QuoteHelper.Get("00021088")
item = quote.GetItemByItemNumber(1)
altProduct = ProductHelper.CreateProduct('00021088',item.QuoteItem)
alternativeItem = quote.AddItem(altProduct,1).AsMainItem
alternativeItem.ChangeItemTypeToAlternative(item.Id)

From SAP Datasphere to a Local LLM (Llama 3.1) — Hands-On Tutorial

2025-10-29T12:05:30.405000+01:00

Introduction

This post documents a small, reproducible pattern for bringing SAP Datasphere data to a local large language model (LLM) for lightweight analysis. The goal is simple: keep modeling and governance in Datasphere, pull a view into a Jupyter notebook with pandas, and let a local LLM produce machine-readable JSON that you can filter, join, or visualize. The prototype runs on a CPU-only laptop so anyone can follow along without special hardware.

What this is:
- A step-by-step walkthrough that uses hdbcli to query a Datasphere view and Transformers to run Meta Llama 3.1 locally.
- A row-by-row prompt pattern that returns compact JSON (total, average, flags) you can plug back into your DataFrame. A row-by-row prompt pattern that returns compact JSON (total, average, flags) you can plug back into your DataFrame.
- A row-by-row prompt pattern that returns compact JSON (total, average, flags) you can plug back into your DataFrame.

What this is not:
- A benchmarking or performance guide. CPU runs are slow but convenient for learning.
- A recommendation to hard-code credentials. The POC mirrors the original notebook for clarity; use environment variables or a vault in real projects.A recommendation to hard-code credentials. The POC mirrors the original notebook for clarity; use environment variables or a vault in real projects.
- A pattern for sending sensitive data to external endpoints. Use test or masked data if you move past local inference. A pattern for sending sensitive data to external endpoints. Use test or masked data if you move past local inference.

What you will build:

A compact flow: SAP Datasphere View -> Python/Jupyter (hdbcli + pandas) -> row-level prompt -> Local LLM (Transformers) -> JSON back to DataFrame. The same prompts can be pointed to managed inference later for production.

Prerequisites

1. SAP Datasphere space with permission to create a Database User and a SQL View
2. Python 3.10+ with Jupyter
3. Libraries: pandas, hdbcli, transformers, torch, accelerate, ipython
4. Hugging Face account + access token (accept access for the Llama 3.1 model)

Security note: The POC code below uses inline credentials to mirror the original run. In real work, put secrets in environment variables or a vault and keep TLS validation enabled.

Part A — SAP Datasphere

A1. Enable database access for the space

1. Open SAP Datasphere -> Spaces -> select your space.
2. Go to Database Access and confirm SQL access is enabled for the space

A2. Create a Database User

1. Database Access -> Database Users -> Create.Database Access -> Database Users -> Create.
2. Grant only read privileges/necessary privileges to the schema/view you will query.
3. Copy the SQL Endpoint (host) and port 443 for Python connectivity. Make sure you copy password and host details and store it in a safe place.

A3. Create a demo view with a few rows

Create a SQL View (or graphical view). Here we have taken "ACN_DWC"."DemoView_SETHIR_PY" with columns:
Stud_ID, Stud_Fname, Stud_Lname, Stud_DOB, Maths, Physics, Chemistry, Total, Stud_Addr, Stud_Faname for the demo.
Make sure the view is exposed for Consumption.
Optional seed SQL you can adapt:

SELECT * FROM (
  SELECT 101 AS Stud_ID, 'Arjun'     AS Stud_Fname, 'Singh'    AS Stud_Lname, DATE'2012-04-06' AS Stud_DOB,
         80 AS Maths, 75 AS Physics, 80 AS Chemistry, 235 AS Total, 'Chandigarh' AS Stud_Addr, 'Sukhdev' AS Stud_Faname
  UNION ALL
  SELECT 102, 'Harpreet', 'Kaur',  DATE'2010-09-08', 85, 78, 85, 248, 'Ludhiana',  'Gurinder'
  UNION ALL
  SELECT 103, 'Gursimran','Gill',  DATE'2011-07-09', 95, 85, 90, 270, 'Amritsar',  'Balwinder'
  UNION ALL
  SELECT 104, 'Manpreet', 'Sidhu', DATE'2014-01-01', 90, 85, 95, 270, 'Patiala',   'Harjit'
  UNION ALL
  SELECT 105, 'Jasleen',  'Dhillon',DATE'2012-03-02', 89, 90, 75, 254, 'Jalandhar', 'Paramjit'
) AS t;
SELECT * FROM (
  SELECT 101 AS Stud_ID, 'Arjun'     AS Stud_Fname, 'Singh'    AS Stud_Lname, DATE'2012-04-06' AS Stud_DOB,
         80 AS Maths, 75 AS Physics, 80 AS Chemistry, 235 AS Total, 'Chandigarh' AS Stud_Addr, 'Sukhdev' AS Stud_Faname
  UNION ALL
  SELECT 102, 'Harpreet', 'Kaur',  DATE'2010-09-08', 85, 78, 85, 248, 'Ludhiana',  'Gurinder'
  UNION ALL
  SELECT 103, 'Gursimran','Gill',  DATE'2011-07-09', 95, 85, 90, 270, 'Amritsar',  'Balwinder'
  UNION ALL
  SELECT 104, 'Manpreet', 'Sidhu', DATE'2014-01-01', 90, 85, 95, 270, 'Patiala',   'Harjit'
  UNION ALL
  SELECT 105, 'Jasleen',  'Dhillon',DATE'2012-03-02', 89, 90, 75, 254, 'Jalandhar', 'Paramjit'
) AS t;

SAP Datasphere DB user :
SAP Datasphere DB user screen

Demo View :

Part B — Local environment (quick path)

1. Create a project folder and venvCreate a project folder and venv

mkdir datasphere-local-llm && cd datasphere-local-llm
python -m venv .venv
# Windows: .venv\Scripts\activate
# macOS/Linux:
source .venv/bin/activate

2. Install requirements

pip install hdbcli
pip install sqlalchemy
pip install sqlalchemy-hana
pip install pandas
pip install hdbcli
pip install transformers torch accelerate
pip install ipython

3. Launch Jupyter

jupyter notebook

Part C — Original POC notebook

1. Open a new notebook and write the code.

# --- Imports and setup (original)
import pandas as pd
from hdbcli import dbapi
import warnings
from transformers import pipeline
from IPython.display import display
warnings.filterwarnings('ignore')

# --- Inline credentials (POC-style; replace with environment variables for real use)
db_user = 'ACN_DWC#SETHIR_DB'
db_password = 'secret'
db_host = 'secret'
db_port = 443
db_schema = 'ACN_DWC'


connection = dbapi.connect(
    address = db_host,
    port = db_port,
    user = db_user,
    password = db_password,
    encrypt = True,
    sslValidCertificate = False   # POC-only convenience; prefer True in real use
)
print("Connected to SAP Datasphere - confirmation")

2. If you see the output - "Connected to SAP Datasphere - confirmation" -- this implies you are on right track.

# --- Query the view
view_name = 'DemoView_SETHIR_PY'
sql_query = f'SELECT * FROM "{db_schema}"."{view_name}"'
print(f"Executing query: {sql_query}")

cursor = connection.cursor()
cursor.execute(sql_query)
rows = cursor.fetchall()
columns = [desc[0] for desc in cursor.description]

df = pd.DataFrame(rows, columns=columns)
display(df.head())

3. At this point, you should see the outcome of the view that you have. The last statement is used for displaying the data in the form of pandas dataFrame.

4. Now we need to prepare our data , so that the LLM can process it. This can be done by converting our data in the form of string format.

# ==============================================================================
#Prepare Data and Interact with OpenLLM
# ==============================================================================

# Convert DataFrame to a string
data_as_string = df.to_string(index=False)
print("\n--- Data Prepared for LLM ---")
print("The DataFrame has been converted to the following string format:")
print(data_as_string[:300] + "\n...")  # snippet preview

5. Prepare data pipeline :

# --- LLM imports
import os
import torch
import json
from transformers import AutoTokenizer, AutoModelForCausalLM, BitsAndBytesConfig, pipeline

# --- LLM Configuration
MODEL_ID = "meta-llama/Meta-Llama-3.1-8B-Instruct"

def load_llama_pipeline():
    """
    Loads the quantized Llama 3 model, tokenizer, and the text-generation pipeline.
    """
    print("\n--- Loading Llama 3 Model ---")
    hf_token = "secret"  # replace with your HF token; accept model access first

    quantization_config = BitsAndBytesConfig(
        load_in_4bit=True,
        bnb_4bit_quant_type="nf4",
        bnb_4bit_compute_dtype=torch.bfloat16
    )

    tokenizer = AutoTokenizer.from_pretrained(MODEL_ID, token=hf_token)
    model = AutoModelForCausalLM.from_pretrained(
        MODEL_ID,
        token=hf_token,
        quantization_config=quantization_config,
        torch_dtype=torch.bfloat16,
        device_map="auto",
    )

    print("Model loaded. Creating text generation pipeline...")
    return pipeline(
        "text-generation",
        model=model,
        tokenizer=tokenizer,
    )

6. Build the prompt for the LLM model :

# --- Prompt builder 
def build_student_analysis_prompt(tokenizer, student_data: pd.Series) -> str:
    """
    Builds a structured prompt for Llama 3 to analyze student marks.
    """
    input_text = (
        f"Student {student_data['Stud_Fname']} {student_data['Stud_Lname']} "
        f"(ID: {student_data['Stud_ID']}) scored {student_data['Maths']} in Maths, "
        f"{student_data['Physics']} in Physics, and {student_data['Chemistry']} in Chemistry."
    )

    messages = [
        {
            "role": "system",
            "content": (
                "You analyze one student's marks and return ONLY a valid JSON object. "
                "Output must start with '{' and end with '}'. No commentary, no markdown, no backticks.\n\n"
                "Schema (keys and types MUST match exactly):\n"
                "{\n"
                '  "total_marks": <int>,\n'
                '  "average_percentage": <float>,\n'
                '  "is_top_performer": <boolean>\n'
                "}\n\n"
                "Rules:\n"
                "1) total_marks = Maths + Physics + Chemistry (each out of 100).\n"
                "2) average_percentage = total_marks / 3.\n"
                "3) Round average_percentage to TWO decimals.\n"
                "4) is_top_performer = true if average_percentage > 80.0; else false.\n"
                "5) Use lowercase true/false for booleans.\n"
                "6) Do not include extra keys. Do not include trailing commas.\n"
                "7) Return JSON only."
            )
        },
        {
            "role": "user",
            "content": input_text
        }
    ]
    return tokenizer.apply_chat_template(messages, tokenize=False, add_generation_prompt=True)

7. Do the analysis :

# --- Row-by-row analysis
def analyze_student_data(df, text_pipeline):
    """
    Analyzes the student DataFrame row by row using the LLM pipeline.
    """
    print("\n--- Starting Student Performance Analysis with Llama 3 ---")
    results = []
    for index, row in df.iterrows():
        print(f"\nAnalyzing student ID: {row['Stud_ID']}...")

        prompt = build_student_analysis_prompt(text_pipeline.tokenizer, row)

        raw_output = text_pipeline(
            prompt,
            max_new_tokens=128,
            do_sample=False,  # deterministic
            temperature=None,
            top_p=None,
        )[0]['generated_text']

        json_response_str = raw_output[len(prompt):].strip()
        try:
            analysis_result = json.loads(json_response_str)
            results.append(analysis_result)
            print("Analysis successful.")
        except json.JSONDecodeError:
            print(f"  > Failed to decode JSON from model output.")
            print(f"  > Raw model output: {json_response_str}")
            results.append({"error": "Invalid JSON output", "raw_output": json_response_str})

    print("\n--- Analysis Complete ---")
    return pd.DataFrame(results)

# --- Main
if __name__ == "__main__":
    if df is not None and not df.empty:
        try:
            llm_pipeline = load_llama_pipeline()
            analysis_df = analyze_student_data(df, llm_pipeline)

            if analysis_df is not None:
                final_df = pd.concat([df.reset_index(drop=True), analysis_df.reset_index(drop=True)], axis=1)

                print("\n--- Full Data with LLM Analysis ---")
                display(final_df)

                print("\n--- Top Performing Students (Average > 80%) ---")
                top_performers = final_df[final_df['is_top_performer'] == True]

                if not top_performers.empty:
                    display(top_performers[['Stud_ID', 'Stud_Fname', 'Stud_Lname', 'total_marks', 'average_percentage']])
                else:
                    print("No students found with an average greater than 80%.")

        except Exception as e:
            print(f"\nAn unexpected error occurred during the analysis process: {e}")
    else:
        print("\nDataFrame is empty. Cannot proceed with analysis.")

8. Example output (simulated for the final cell)

Full Data with LLM Analysis (excerpt)

Stud_ID  Stud_Fname  Stud_Lname  Maths  Physics  Chemistry  total_marks  average_percentage  is_top_performer
101      Arjun       Singh       80     75       80         235          78.33               false
102      Harpreet    Kaur        85     78       85         248          82.67               true
103      Gursimran   Gill        95     85       90         270          90.00               true
104      Manpreet    Sidhu       90     85       95         270          90.00               true
105      Jasleen     Dhillon     89     90       75         254          84.67               true

Top Performing Students (Average > 80%)

Stud_ID  Stud_Fname  Stud_Lname  total_marks  average_percentage
102      Harpreet    Kaur        248          82.67
103      Gursimran   Gill        270          90.00
104      Manpreet    Sidhu       270          90.00
105      Jasleen     Dhillon     254          84.67

Part D — Production path (same pattern, managed inference)

When performance or scale matters, swap the local pipeline for a hosted endpoint and keep the Datasphere extraction and prompts identical. When performance or scale matters, swap the local pipeline for a hosted endpoint and keep the Datasphere extraction and prompts identical.

1. Databricks Model Serving / Foundation Model Endpoints (REST).
2. Hugging Face Inference Endpoints (private endpoints; REST with your HF token)
3. SAP AI Core (containerized model hosting; REST)SAP AI Core (containerized model hosting; REST)

Minimal REST skeleton --

import requests, os, json
endpoint = os.getenv("INFERENCE_URL")
token = os.getenv("INFERENCE_TOKEN")
r = requests.post(endpoint, headers={"Authorization": f"Bearer {token}"},
                  json={"inputs": "your_prompt_here", "parameters": {"max_new_tokens": 128, "temperature": 0.0}})
print(r.json())

Conclusion ---

This walkthrough demonstrated how to extract data from SAP Datasphere, process it in pandas, and apply a local LLM for row-level analysis that returns clean, machine-readable JSON. The pattern is intentionally small and portable: you can keep iterating locally to refine prompts and outputs, then swap the model call for a managed endpoint (Databricks, Hugging Face, or SAP AI Core) when performance, cost control, or governance call for it. From here, natural next steps include batching larger datasets, persisting results back to a database table, wiring the outputs to SAP Analytics Cloud dashboards, and adding guardrails around data privacy and prompt consistency.

I would be excited to hear how you adapt this to your solutions. Feel free to reach out to me or comment.

SAP Datasphere : Export Data of AnalyticalModel via Odata URL & Oauth Client of type Technical User

2025-11-05T07:58:30.289000+01:00

In this blog, I have explained how to export data from an analytical model into CSV file securely using an OData URL and a technical user OAuth client.

Step - 1) Create an Oauth Client with Purpose as Technical User and select required roles. get client ID and secret and save it. How to Guide

Step - 2) Create Odata_Tech_User_OauthClient.py file with below code

# --- IMPORTS ---
import requests              # For making HTTP requests to token and OData endpoints
import pandas as pd          # For handling tabular data from the OData response
import os                    # For file path resolution and environment variable access
from dotenv import load_dotenv  # For loading credentials from a .env file

# --- LOAD ENV VARIABLES ---
load_dotenv()                # Load environment variables from .env file into the runtime

# --- CONFIG: Read credentials and endpoints from environment ---
odata_url = os.getenv("ODATA_URL")             # OData service endpoint
token_url = os.getenv("TOKEN_URL")             # OAuth token endpoint
client_id = os.getenv("CLIENT_ID")             # OAuth client ID
client_secret = os.getenv("CLIENT_SECRET")     # OAuth client secret

# --- GET TOKEN: Request access token using client credentials ---
token_payload = {
    "grant_type": "client_credentials",        # OAuth flow type
    "client_id": client_id,                    # Injected from .env
    "client_secret": client_secret             # Injected from .env
}
token_resp = requests.post(token_url, data=token_payload)  # POST request to token endpoint
token_resp.raise_for_status()                 # Raise error if token request fails
access_token = token_resp.json()["access_token"]  # Extract access token from response

# --- CALL ODATA SERVICE: Fetch data using bearer token ---
headers = {"Authorization": f"Bearer {access_token}"}  # Auth header with token
response = requests.get(odata_url, headers=headers)    # GET request to OData endpoint
response.raise_for_status()                            # Raise error if data fetch fails

# --- PARSE RESULTS: Convert JSON payload to DataFrame ---
data = response.json()["value"]         # Extract 'value' list from OData response
df = pd.DataFrame(data)                 # Convert list of records to pandas DataFrame

# --- DISPLAY OR EXPORT: Show preview and optionally save to CSV ---
print()
print("🔍 Displaying top rows for quick inspection:")
print()
print(df.head())                        # Display top rows for quick inspection
print()

# --- Extract view name from OData URL ---
view_name = odata_url.rstrip("/").split("/")[-1]  # Gets 'AM_EXPORT' from the URL

# --- Construct filename ---
filename = f"{view_name}.csv"

# --- Display and save ---
df.to_csv(filename, index=False)
print(f"📁 Exported data Saved to: {os.path.abspath(filename)}")

Step - 3) Create .env file with all values of variables.

ODATA_URL : Copy the Odata link from analytical model

TOKEN_URL : get it from Datasphere -> System -> App Integration page

CLIENT ID & Secret : Get it from Step-1

Step- 4) Create .bat file with blow code

 off

chcp 65001 >nul

REM ───────────────────────────────────────────────
REM 🚀 ODATA TECH USER OAUTH CLIENT EXECUTION SCRIPT
REM ───────────────────────────────────────────────


echo.
echo ==============================================
echo 🔄 Starting OData export process...
echo ==============================================

REM --- Navigate to script directory ---
cd /d "%~dp0"

REM --- Run the Python script ---
echo 🐍 Running Python script: Odata_Tech_User_OauthClient.py
python Odata_Tech_User_OauthClient.py

REM --- Completion message ---
echo.
echo ✅ Script execution completed.
echo ==============================================

REM --- Keep window open ---
pause

Keep all files in same directory

Once everything is set up, just double-click the .bat file to run the process. It will execute the Python script and, once finished, generate a .csv file name exactly same name as the analytical model name. As part of the execution, the first five rows of data will also be displayed on screen for quick preview

Thanks

Vikas Parmar

Want to explore developing AI workflows with the Python Machine Learning Client for SAP HANA?

2025-11-07T16:40:45.744000+01:00

I recommend to access our Developing AI workflows with the Python Machine Learning Client for SAP HANA learning journey.

Leaning Objectives
By the end of this learning journey, learners will be able to:

Know how to install and configure the Python Machine Learning Client (hana-ml) and connect securely to SAP HANA Cloud.
Apply HANA DataFrames to access, prepare, and explore SAP HANA data for machine learning tasks.
Think critically to build, train, and deploy machine learning models using PAL functions for regression, classification, and time-series forecasting.
Impact business outcomes by using real-world datasets (e.g., housing prices, employee churn) to complete end-to-end ML workflows.
Evaluate models using appropriate metrics to assess performance, robustness, and business relevance.

Goals

Get started with SAP: Building the basics
Develop your expertise: Skill deepening learning
Excel in your expertise: Advanced specialization learning

Prerequisites

Basic python expertise

Please post you question related to the digital learning Journey in the Q&A area.

Our SAP Learning Experts will get back to you as soon as possible!
We are here to support you.

I appreciate your feedback and we will make sure to continue sharing interesting topics.

Kind regards
Margit

How to Reduce Prompt Token Costs Using Toon = Save Money

2025-11-12T17:34:51.069000+01:00

As you already might know prompt tokens are the backbone of communication with large language models (LLMs). However, as usage scales, token costs can quickly become a significant expense. If you’re using Toon—a tool designed for optimizing prompt workflows—you can dramatically cut down on these costs without sacrificing performance.

Why Token Costs Matter

Every interaction with an LLM consumes tokens. These tokens represent:

Input tokens: The text you send to the model.
Output tokens: The text generated by the model.

The more tokens you use, the higher your bill. For businesses running thousands of prompts daily, even small inefficiencies can lead to big costs.

Example:

Original prompt: “Please summarize the following text in a clear and concise manner, highlighting the key points and provide a RACI Matrix for S/4 HANA”
Compressed prompt: “Summarize key points.”

What is TOON?

TOON is a compact, human-readable serialization format designed for passing structured data to LLMs with significantly reduced token usage. It acts as a lossless, drop-in representation of JSON, optimized for token efficiency.

Why Use TOON?

Token-efficient: Saves 30–60% tokens compared to formatted JSON for large uniform arrays.
LLM-friendly: Explicit lengths and fields improve parsing and validation.
Minimal syntax: Removes redundant punctuation (braces, quotes).
Tabular arrays: Declare keys once, stream data as rows.
Optional key folding: Collapses nested chains into dotted paths for fewer tokens.

Benchmarks

TOON uses 39.6% fewer tokens than JSON while improving retrieval accuracy (73.9% vs 69.7%).
For uniform tabular data, TOON is slightly larger than CSV (+6%) but far smaller than JSON (-58%).

When NOT to Use TOON

Deeply nested or non-uniform structures → JSON may be better.
Pure tabular data → CSV is smaller.
Latency-critical apps → Benchmark first; compact JSON might be faster.

How to Use TOON

NPM Lib
npm install -format/toon

Python
https://github.com/xaviviro/python-toon

https://github.com/toon-format/toon

https://github.com/toon-format/spec

Why TOON Matters while your working for different SAP Product LoBs

Token Cost Reduction
- SAP S/4 HANA systems handle large datasets (e.g., thousands of line items in a purchase order).
- JSON representation of these datasets is expensive in token terms.
- TOON compresses this data by 30–60%, reducing LLM API costs significantly.
Performance Gains
- Smaller prompts mean lower latency and faster response times.
- This is critical for real-time SAP applications like Joule for procurement or HR assistants.
Improved Accuracy
- TOON’s explicit structure (e.g., tabular arrays with declared keys) helps LLMs parse data better.
- This reduces hallucinations in SAP workflows like financial reconciliation or compliance checks.

High Level Flow

Integration Approach

Middleware Layer: Convert SAP OData/JSON responses to TOON before sending to LLM.
SAP BTP Extension: Implement TOON conversion in CAP, Cloud Foundry apps or Kyma runtime or SAP Databricks, AI Core Models you have selected.
Prompt Wrapping: Always wrap TOON in fenced code blocks for LLM clarity

Best Practices

Use tab-delimited rows for extra token savings.
Benchmark TOON vs JSON for your specific SAP dataset.
Cache static context to avoid repeated token costs.

Conclusion

TOON offers a simple yet powerful way to reduce LLM costs in your SAP landscape environments. By compressing structured data without losing meaning, SAP teams can achieve:

Up to 60% cost savings
Faster response times
Improved AI accuracy

As SAP continues its AI journey, adopting TOON can make intelligent automation more scalable and cost-effective.

SAP RPT-1 Context Model vs. Training Classical Models: The Models Battle (Python Hands-on)

2025-11-20T07:50:27.670000+01:00

💥The Models Battle

Predictive modeling is becoming a built-in capability across SAP, improving how teams handle forecasting, pricing, and planning. Many SAP professionals, however, aren’t machine-learning specialists, and traditional models often demand extensive setup, tuning, and repeated training, which slows down new ideas.

SAP RPT-1 offers a simpler path. It’s a pretrained model from SAP, also available in an OSS version, that lets developers and consultants produce predictions with far less technical effort, no deep ML background required.

I've explored SAP RPT-1 hands-on, comparing it with traditional regressors using Python and a real public vehicles price dataset.

Goal: To see (as a non Data Scientist) how SAP RPT-1 behaves in practice, what advantages and limits it shows, and when it could make sense in a predictive scenario.

Usually for real-world scenario, the right approach would be consume the SAP RPT-1 though the available and simplified API, but for studies proposal and fair comparision over othe traditional ML models, the OSS fits perfectly for it:

🤔 SAP RPT-1 vs Traditional Machine Learning - Core Differences

Before diving into the code, let’s quickly revisit how traditional ML models work:

Training-based models like Random Forest, LightGBM, and Linear Regression learn patterns directly from data.
They require hundreds or thousands of examples to tune their internal parameters.
Their performance depends heavily on data quantity and quality.
The more relevant examples they see, the smarter they get.

On the other hand, SAP RPT-1 follows a different philosophy. It’s part of the RPT (Representational Predictive Transformer) family, pretrained on a wide variety of business and contextual data. This means:

You don’t "train" it in the traditional sense. Instead, it uses context embeddings to predict outcomes.
It can be used immediately, even with smaller datasets.
The OSS version allows developers to experiment directly in Python.
No special SAP backend required.

Outcome: Traditional ML models learn from high amount of data. SAP RPT-1 already knows how to deal with small context amount of data.

🖥 The Experiment - Setup & Dataset

Spoiler

Don't worry on "playing puzzles" copying + pasting code below. The full version is available for download at end!

To make this comparison tangible, I built a simple yet realistic Python experiment to predict vehicle selling prices using a public dataset containing car attributes like make, model, year, transmission, and mileage.

Why vehicle pricing? Because it’s an intuitive example where both traditional machine learning and pretrained AI models can be applie, and it helps visualize how prediction quality evolves as the sample size grows.

This entire analysis runs on a local Python environment with the following stack:

import os
import gc
import warnings
import pandas as pd
import numpy as np
import matplotlib.pyplot as plt
from datetime import datetime
from sklearn.model_selection import train_test_split
from sklearn.metrics import r2_score
from sklearn.ensemble import RandomForestRegressor
from sklearn.preprocessing import LabelEncoder
from sklearn.linear_model import LinearRegression
from sap_rpt_oss import SAP_RPT_OSS_Regressor
import lightgbm as lgb

pandas and numpy for data manipulation
scikit-learn for classical ML regressors (Random Forest, Linear Regression)
LightGBM for gradient boosting comparison
sap_rpt_oss — the open-source Python version of SAP’s RPT-1 model
matplotlib for all visualizations

SAP RPT-1 OSS can be downloaded installed following official Hugging Face: https://huggingface.co/SAP/sap-rpt-1-oss?library=sap-rpt-1-oss . Python can be installed with executable download on Windows, or via Home Brew for Mac and apt commands for Linux. Libraries dependencies can be downloaded with pip commands. Googling it may not be a road blocker.

We use a sample vehicle sales dataset. The complete file is about to 88Mb but for such experiment a restricted sample of 20k as it's more than enough to prove our the concept, still it's faster and consuming less computing resources.

Feature	Description
`year`	Vehicle model year
`make`	Brand (e.g., Toyota, Ford, BMW)
`model`	Specific model name
`body`	Type (SUV, Sedan, etc.)
`transmission`	Gear type
`odometer`	Vehicle mileage
`color`, `interior`	Visual attributes
`sellingprice`	The target variable to predict

📊 Dataset Download: https://www.kaggle.com/datasets/syedanwarafridi/vehicle-sales-data?resource=download

The dataset is loaded and preprocessed in a few simple steps:

df = pd.read_csv("car_prices.csv").sample(n=20000, random_state=42)

# Fill missing values for categorical columns
fill_defaults = {
    'make': 'Other', 'model': 'Other', 'color': 'Other',
    'interior': 'Unknown', 'body': 'Unknown', 'transmission': 'Unknown'
}
for col, val in fill_defaults.items():
    df[col] = df[col].fillna(val)

X = df[["year", "make", "model", "body", "transmission", "odometer", "color", "interior"]]
y = df["sellingprice"]

At this point, the stage is set:

The data is clean.
The environment is ready.
All models, traditionals and SAP RPT-1, are ready to be tested under identical conditions.

🤖 Training the Models - Three different ones

With the dataset ready, the next step is to run each model under the same conditions: same features, same target, same train/test split and same random seed. This ensures the comparison is fair and repeatable.

We evaluate prediction performance using R² (coefficient of determination), which indicates how much of the price variation the model can explain (1.0 = perfect prediction).

Training Model #1 - Random Forest

Random Forest is often the first model used in tabular ML. It works by creating many decision trees and averaging their predictions. Before training, categorical variables need to be label-encoded into numbers, a common requirement for classical ML models:

def train_random_forest(X, y):
    X = X.copy()
    cat_cols = ["make", "model", "body", "transmission", "color", "interior"]
    le = LabelEncoder()

    for col in cat_cols:
        X[col] = le.fit_transform(X[col].astype(str).fillna("Unknown"))

    X_train, X_test, y_train, y_test = train_test_split(
        X, y, test_size=default_test_size, random_state=42
    )

    model = RandomForestRegressor(
        n_estimators=150, max_depth=20, random_state=42, n_jobs=-1
    )

    try:
        model.fit(X_train, y_train)
        preds = model.predict(X_test)
        r2 = r2_score(y_test, preds)
    except Exception as e:
        preds, r2 = np.zeros_like(y_test), 0

    return [preds, r2, y_test]

Up to 50 rows:

Up to 7067 rows:

Live view

Training Model #2 - LightGBM

LightGBM is one of the most powerful models for tabular data. Unlike Random Forest (many independent trees), LightGBM builds trees sequentially, each correcting the errors of the previous one. It supports categorical features natively, which simplifies preprocessing.

def train_lightgbm(X, y):
    X = X.copy()
    cat_cols = ["make", "model", "body", "transmission", "color", "interior"]
    for col in cat_cols:
        X[col] = X[col].astype(str).fillna("Unknown").astype("category")

    X_train, X_test, y_train, y_test = train_test_split(
        X, y, test_size=default_test_size, random_state=42
    )

    model = lgb.LGBMRegressor(
        n_estimators=500, learning_rate=0.05, num_leaves=31,
        subsample=0.8, colsample_bytree=0.8, random_state=42
    )

    try:
        model.fit(X_train, y_train, categorical_feature=cat_cols)
        preds = model.predict(X_test)
        r2 = r2_score(y_test, preds)
    except Exception:
        preds, r2 = np.zeros_like(y_test), 0

    return [preds, r2, y_test]

Up to 50 rows:

Up to 7067 rows:

Live view

Training Model #3 - Linear Regression

Not fancy and even not complex, Linear Regression provides a baseline that shows: “If the relationship between attributes and price is roughly linear, how well can a simple model perform?”

def train_linear_model(X, y):
    X = X.copy()
    cat_cols = ["make", "model", "body", "transmission", "color", "interior"]
    for col in cat_cols:
        X[col] = LabelEncoder().fit_transform(X[col].astype(str).fillna("Unknown"))

    X_train, X_test, y_train, y_test = train_test_split(
        X, y, test_size=default_test_size, random_state=42
    )

    model = LinearRegression()
    X_train = X_train.fillna(X_train.mean(numeric_only=True))
    X_test = X_test.fillna(X_test.mean(numeric_only=True))

    try:
        model.fit(X_train, y_train)
        preds = model.predict(X_test)
        r2 = r2_score(y_test, preds)
    except Exception:
        preds, r2 = np.zeros_like(y_test), 0

    return [preds, r2, y_test]

Up to 50 rows:

Up to 7067 rows:

Live view

🏁 SAP RPT-1 OSS: Context Model

This is where things get interesting. SAP RPT-1 does not rely on learning patterns from the dataset. Instead, it uses a pretrained transformer architecture to infer relationships directly through context embeddings. Lean and simple, "for non-Data Science PhD":

def train_sap_rpt1(X, y):
    X_train, X_test, y_train, y_test = train_test_split(
        X, y, test_size=default_test_size, random_state=42
    )

    model = SAP_RPT_OSS_Regressor(max_context_size=8192, bagging=8)
    model.fit(X_train, y_train)

    preds = model.predict(X_test)
    r2 = r2_score(y_test, preds)

    return [preds, r2, y_test]

Up to 50 rows:

Up to 2055 rows:

Live view

🔎 Running Experiments at Multiple Sample Sizes

This section breaks down how the iterative experiment loop works, why the SAP RPT-1 OSS model has a max-context limit, and how performance changes as we scale up the dataset. By running the same models across several sample sizes, we can see where traditional ML shines, where RPT-1 stays competitive, and how both behave as the data grows.

sample_sizes = np.linspace(50, len(X), 200, dtype=int)
results, max_r2_rpt1, max_sample_rpt1 = [], 0, 0

for n in sample_sizes:
    idx = np.random.choice(len(X), n, replace=False)
    X_sample, y_sample = X.iloc[idx], y.iloc[idx]


    # SAP RPT-1 OSS (limited sample size)
    if n <= rpt1_limit:
        rpt_res = train_sap_rpt1(X_sample, y_sample)
        fn = plot_predictions(rpt_res[2], rpt_res[0], rpt_res[1], "SAP_RPT1", n)
        video_frames["SAP_RPT1"].append(fn)
        r2_rpt1 = rpt_res[1]
        max_r2_rpt1 = max(max_r2_rpt1, r2_rpt1)
    else:
        r2_rpt1 = max_r2_rpt1
        if max_sample_rpt1 == 0:
            max_sample_rpt1 = n

    # Train and plot models
    rf_res = train_random_forest(X_sample, y_sample)
    fn = plot_predictions(rf_res[2], rf_res[0], rf_res[1], "RandomForest", n)
    video_frames["RandomForest"].append(fn)

    lgb_res = train_lightgbm(X_sample, y_sample)
    fn = plot_predictions(lgb_res[2], lgb_res[0], lgb_res[1], "LightGBM", n)
    video_frames["LightGBM"].append(fn)

    lin_res = train_linear_model(X_sample, y_sample)
    fn = plot_predictions(lin_res[2], lin_res[0], lin_res[1], "LinearModel", n)
    video_frames["LinearModel"].append(fn)

    results.append((n, rf_res[1], r2_rpt1, lgb_res[1], lin_res[1]))

    # Early stop if traditional model reaches SAP RPT-1
    if rf_res[1] >= max_r2_rpt1 or lgb_res[1] >= max_r2_rpt1 or lin_res[1] >= max_r2_rpt1:
        break
    gc.collect()

This loop compares SAP RPT-1 OSS with traditional ML models as sample sizes increase. Each iteration randomly selects a subset of the data and trains all models on the same slice for a fair comparison. SAP RPT-1 can only run up to its max-context limit, so once the sample size exceeds that threshold, it stops retraining and simply carries forward its best R². The traditional models continue training at every step. The loop ends early when any traditional model matches or surpasses RPT-1’s best score, making the experiment efficient while showing how performance evolves as data grows.

🔚 Conclusion and Final Thoughts

SAP RPT-1 OSS stands out because it performs well with small datasets, requires minimal code, and can generate useful predictions with just an API call and a bit of context. This makes it ideal for jump-starting predictive use cases early on, delivering fast business value without a full ML pipeline. Traditional models, however, still shine when projects mature, data grows, and fine-tuned control becomes important. It’s not about choosing one over the other, but understanding where each approach brings the most value.

Aspect	SAP RPT-1 OSS	Traditional ML (RF, LGBM, Linear)
Data Requirements	Low (performs well with small samples)	Medium/High (performance scales with data
Setup Effort	Minimal (API call + context)	Higher (preprocessing, encoding, tuning)
Training Process	None (pretrained context model)	Full training pipeline required
Speed to Insights	Very fast	Moderate to slow
Best Use Case	Early-stage predictive cases, quick baselines	Mature pipelines, high control and customization
Flexibility	Limited tuning / plug-and-play	Highly customizable
Business Value	Immediate, fast, accessible	Strong when optimized and scaled

This experiment highlights a simple truth: SAP RPT-1 isn’t here to replace traditional ML, it jump-starts it. With a pretrained, context-driven approach, RPT-1 delivers fast, reliable insights with very little data and almost no setup. Traditional models still excel in mature, data-rich scenarios, but RPT-1 shines as a rapid accelerator and early-value generator inside SAP landscapes.

💬Open for Exchange

If you're testing RPT-1, exploring predictive cases, or want the full code, feel free to reach out.
Happy to connect, compare experiences, and push this topic forward together.

Querying RDF Graphs with SAP HANA Cloud Knowledge Graph Engine

2025-12-12T03:01:55.876000+01:00

SAP HANA Cloud’s multi-model platform brings together vector, graph, text, spatial, and relational data natively. It enables developers and data teams to build smarter, more context-aware AI solutions — directly on operational data.

SAP HANA Cloud uniquely supports:

Vector data for semantic and similarity search
Graph data for explicit relationship modeling and knowledge graphs
Text and spatial data for real-world context
Relational data for structured operations and analytics

Rather than sending data across disparate services, you can store and process all of it in one place, accelerating time-to-value while reducing the risk of misalignment. This is multi-model done right, and it is the foundation for powerful AI workloads that scale.

This blog demonstrates how to query RDF knowledge graphs in SAP HANA Cloud, both:

Directly from the HANA Cloud Central SQL Console, and
Programmatically using Python

We will also show how RDF graphs can be combined seamlessly with vector similarity search, spatial filtering, and SQL analytics to create intelligent, real‑world use cases. The blog expands on this post.

From supplier notes to AI‑ready supply‑chain intelligence

To ground the concepts, we start with a simple but realistic scenario: turning unstructured supplier feedback into AI‑ready supply‑chain intelligence.

Step 1: Create a supply‑chain schema and table

We first create a dedicated schema and table to store supplier identifiers, short operational reports, vector embeddings, and geographic locations.

CREATE SCHEMA KG_SUPPLYCHAIN;

CREATE TABLE KG_SUPPLYCHAIN.SUPPLIER_REPORTS_LOOKUP (
  supplier_uri       NVARCHAR(200),
  report_text        NVARCHAR(1000),
  report_embedding   REAL_VECTOR(768),
  geo_location       ST_POINT(4326)
);

Step 2: Insert natural‑language supplier reports

The reports represent real‑world observations such as customs delays or smooth clearance. At this stage, AI embeddings and spatial attributes are not yet populated.

INSERT INTO KG_SUPPLYCHAIN.SUPPLIER_REPORTS_LOOKUP VALUES
('http://example.org/supplier/AlphaGmbH', 'Consistently on time. No customs issue.', NULL, NULL),
('http://example.org/supplier/BetaGmbH',  'Severe customs delays reported last month.', NULL, NULL),
('http://example.org/supplier/GammaLogistics', 'Smooth operations, cleared customs quickly.', NULL, NULL);

Step 3: Add geographic context

Each supplier is enriched with a geographic location using WGS84 coordinates. This enables proximity analysis and regional risk assessment.

UPDATE KG_SUPPLYCHAIN.SUPPLIER_REPORTS_LOOKUP
SET geo_location = ST_GeomFromText('POINT(8.6821 50.1109)', 4326)
WHERE supplier_uri = 'http://example.org/supplier/AlphaGmbH';

Step 4: Generate vector embeddings inside the database

SAP HANA Cloud generates vector embeddings directly in‑database using SAP’s native embedding model. This enables semantic understanding of supplier reports.

UPDATE KG_SUPPLYCHAIN.SUPPLIER_REPORTS_LOOKUP
SET report_embedding = VECTOR_EMBEDDING(
  report_text,
  'QUERY',
  'SAP_NEB.20240715'
);

At this point, we have a single AI‑ready table that supports:

Semantic similarity search
Spatial analysis
Generative AI grounding

—all without exporting data outside SAP HANA Cloud.

Adding semantic supplier knowledge with SAP HANA Knowledge Graph

Relational tables capture operational facts well, but business meaning and rules are better expressed semantically. This is where the SAP HANA Cloud Knowledge Graph engine comes in.

Step 5: Initialize the RDF graph

We start by removing any existing graph to ensure a clean load.

CALL SPARQL_EXECUTE('DROP GRAPH <kg_supplychain>', '', ?, ?);

Step 6: Insert semantic supplier knowledge

We then insert RDF triples describing suppliers and their business attributes. Each supplier uses the same URI as the relational table, creating a natural bridge between SQL and RDF.

CALL SPARQL_EXECUTE(
'
INSERT DATA {
  GRAPH <kg_supplychain> {
    <http://example.org/supplier/AlphaGmbH> a <Supplier> ;
      <hasCertification> "ISO 9001" ;
      <hasCarbonTaxRate> "low" ;
      <isFlaggedForDelays> false .
  }
}',
'', ?, ?);

Within the Knowledge Graph, suppliers are enriched with structured, machine‑readable facts such as certifications, carbon‑tax exposure, and compliance flags.

The result is a dual‑layer architecture:

Relational layer: text, embeddings, spatial data
Semantic layer: business meaning, rules, and relationships

Both are connected by shared URIs and executed in the same database.

Querying across graph, vector, spatial, and SQL models

Query 1: Filter Knowledge Graph data directly in SQL

SAP HANA Cloud allows SPARQL queries to be consumed as relational tables using SPARQL_TABLE, enabling seamless integration with SQL analytics.

SELECT * FROM SPARQL_TABLE('
  SELECT ?supplier ?certification ?carbontax ?flag
  FROM <kg_supplychain>
  WHERE {
    ?supplier a <Supplier> .
    ?supplier <hasCertification> ?certification .
    ?supplier <hasCarbonTaxRate> ?carbontax .
    ?supplier <isFlaggedForDelays> ?flag .
    FILTER(
      ?certification = "ISO 9001" &&
      ?carbontax = "low" &&
      STR(?flag) = "false"
    )
  }
');

This bridges semantic reasoning and relational analytics in a single query flow.

Query 2: Hybrid SPARQL + SQL with vector and spatial filtering

This advanced query combines:

Geospatial filtering (nearby suppliers)
Vector similarity search on supplier reports
Knowledge Graph constraints

CALL SPARQL_EXECUTE(
'
SELECT *
FROM <kg_supplychain>
WHERE {
  SQL_TABLE("SELECT \"uri_str\", \"REPORT_TEXT\", \"SCO\" FROM (
    SELECT *, sco - FIRST_VALUE(sco) OVER(ORDER BY sco DESC) AS diff
    FROM (
      SELECT \"SUPPLIER_URI\" AS \"uri_str\", \"REPORT_TEXT\",
      COSINE_SIMILARITY(
        \"REPORT_EMBEDDING\",
        VECTOR_EMBEDDING(''no customs delay'', ''QUERY'', ''SAP_NEB.20240715'')
      ) AS sco
      FROM KG_SUPPLYCHAIN.SUPPLIER_REPORTS_LOOKUP
      WHERE \"GEO_LOCATION\".ST_Distance(
        ST_GeomFromText(''POINT(8.6821 50.1109)'', 4326)
      ) < 50000
      ORDER BY sco DESC
    )
  ) WHERE diff > -0.2") .
}
ORDER BY DESC(?SCO)
LIMIT 10
',
'Accept: application/sparql-results+csv Content-Type: application/sparql-query',
?, ?);

This single query performs AI‑driven supplier discovery, ranking results by semantic meaning while respecting spatial and business rules.

Querying and managing Knowledge Graphs with Python

Beyond the SQL console, Knowledge Graphs can be managed programmatically using Python and the HANA DBAPI.

The following example shows how to:

Upload an RDF ontology (Turtle .ttl format)
Verify graph load success
Query RDF data programmatically

from hdbcli import dbapi

conn = dbapi.connect(
    address='XXX.hana.prod-ap10.hanacloud.ondemand.com',
    port=443,
    user='XXX',
    password='XXX'
)

cursor = conn.cursor()

Load an ontology into SAP HANA Cloud Knowledge Graph

# Populate a new RDF in HANA Cloud with the turtle file content. We put the ontology into a specific graph.
ttl_filename = "/Users/materials_ontology.ttl"
graph_name = "materials_ontology"

# Load the ontology into HANA Cloud KG
print("Loading ontology...")

try:
    with open(ttl_filename, 'r') as ttlfp:
        request_hdrs = ''
        request_hdrs += 'rqx-load-protocol: true' + '\r\n'            # required header for upload protocol
        request_hdrs += 'rqx-load-filename: ' + ttl_filename + '\r\n' # optional header
        request_hdrs += 'rqx-load-graphname: ' + graph_name + '\r\n'   # optional header to specify name of the graph
        
        print(f"Loading file: {ttl_filename}")
        print(f"Graph name: {graph_name}")
        
        # Execute the load
        result = conn.cursor().callproc('SPARQL_EXECUTE', (ttlfp.read(), request_hdrs, '?', None))
        print("Materials ontology loaded successfully!")
        print(f"Result: {result}")
        
except Exception as e:
    print(f"Error loading materials ontology: {e}")
    print(f"Error type: {type(e)}")

Query the ontology

Do note that you will have to adjust the query based on your ontology structure.

# Verify the materials ontology was loaded
print("Verifying materials ontology load...")

try:
    # Count triples in the materials graph
    query = f"""
    SELECT (COUNT(*) as ?Triples) 
    WHERE 
      {{ GRAPH <{graph_name}> 
          {{ ?s ?p ?o }} 
      }}
    """
    
    resp = conn.cursor().callproc('SPARQL_EXECUTE', (query, 'Accept: application/sparql-results+csv', '?', None))
    print("Materials ontology statistics:")
    print(resp[2])
    
    # Query for some material instances
    query2 = f"""
    PREFIX mat: <http://example.com/materials/>
    PREFIX matprop: <http://example.com/materials/property/>
    
    SELECT ?material ?materialId ?level2 ?level3
    FROM <{graph_name}>
    WHERE {{
        ?material a mat:Material .
        ?material matprop:materialId ?materialId .
        OPTIONAL {{ ?material matprop:level2 ?level2 . }}
        OPTIONAL {{ ?material matprop:level3 ?level3 . }}
    }}
    LIMIT 5
    """
    
    resp2 = conn.cursor().callproc('SPARQL_EXECUTE', (query2, 'Accept: application/sparql-results+csv', '?', None))
    print("\n Sample material instances:")
    print(resp2[2])
    
except Exception as e:
    print(f"Error verifying materials ontology: {e}")

Summary

SAP HANA Cloud enables true multi‑model intelligence by allowing relational data, vector embeddings, spatial context, and RDF knowledge graphs to coexist and execute together.

By combining in‑database vector embeddings, semantic Knowledge Graphs, and SQL, SPARQL, Python access, organizations can build powerful AI applications that are context‑aware, explainable, and operationally grounded—all from a single platform.

SAP HANA Cloud is not just a database for AI. It is where data, meaning, and intelligence converge.

Implementing Thread-Safe Structured Logging for Python FastAPI

2025-12-18T05:41:59.266000+01:00

With the focous on bussiness AI scenarios and high throughput applications, fastapi has come to be the de facto for standard python developement due to it's native support for async I/0 and pydantic based validations ensuring type safety. However, deploying fastapi applications to SAP BTP CF enviroment gives us issues with observalibity.

the standard library provided by SAP, sap-cf-logging relies heavily on WSGI middleware and thread-local storage to manage requests contexts. in ASGI enviroment, a single thread can manage multiple concurrent requests via an event loop.

relying on thread-local storage in this enviroment can lead to Context Leakage where logs from one Request A appear with Coorelation id of Request B, or worse context is lost entirely due to await calls.

to ensure this doesn't occur in our application and observalibity is not effected in Kibana without compromising on non-blocking operations, there needed to be a custom solution using python contextvars

Architecture: Context-Local vs Thread-Local

to maintain the integrity of correlation id across async boundaries, there was a need for moving away from global state and implement context aware logger. this has mainly three components:-

context variables: to hold the state safley across asyncio event loop
custom json formatter: to serialize logs into strict JSON format required by SAP cloud logging
ASGI Middleware: to handle lifecycle management of the correlation id

1. Async-safe Context

first we need to define a ContextVar, unlike threading.local , contextvars are natively supported by asyncio. when a task awaits a coroutine then context is preserved for that specific chain of execution, ensuring concurrent reqeusts never corrupt each other's state.

loggger.py

import logging
import sys
import json
import uuid
from contextvars import ContextVar
from typing import Optional

# ContextVar to store Correlation ID safely (Async safe!)
# This works per-task, ensuring concurrent requests preserve their unique ID.
correlation_id_ctx: ContextVar[Optional[str]] = ContextVar("correlation_id", default=None)

2. JSON Formatter

SAP BTP's log ingestion pipeline expects specifc fields (mgs, level, correlation_id, written_at) to index logs correctly in Kibana. we need to extend the standard python logging.Formatter to intercept every log record adn inject the current context dynamically.

logger.py

class JSONFormatter(logging.Formatter):
    """
    Custom formatter that outputs logs as a JSON string.
    Automatically injects the Correlation ID from the async context.
    """
    def format(self, record):
        # Retrieve the current correlation ID (or None if outside a request)
        cid = correlation_id_ctx.get()

        # Build the structured log dictionary required by SAP Cloud Logging
        log_record = {
            "level": record.levelname,
            "msg": record.getMessage(),
            "logger": record.name,
            "correlation_id": cid,
            "written_at": self.formatTime(record, self.datefmt),
        }

        # Include exception info if present (essential for stack traces in Kibana)
        if record.exc_info:
            log_record["exc_info"] = self.formatException(record.exc_info)

        return json.dumps(log_record)

def setup_logging():
    logger = logging.getLogger()
    if logger.handlers:
        logger.handlers = []
    
    # Stream to stdout is crucial for Cloud Foundry loggregator
    handler = logging.StreamHandler(sys.stdout)
    handler.setFormatter(JSONFormatter())
    
    logger.addHandler(handler)
    logger.setLevel(logging.INFO)

3. Middleware injection

we need to implement a BaseHTTPMiddleware to intercept every incoming request. this middleware handles the extraction of the X-CorrelationID header or generates a new UUID V4 if one is missing.

logger.py

from fastapi import Request
from starlette.middleware.base import BaseHTTPMiddleware

class CorrelationMiddleware(BaseHTTPMiddleware):
    async def dispatch(self, request: Request, call_next):
        # 1. Extract ID from header OR generate a new one
        correlation_id = request.headers.get("X-CorrelationID") or str(uuid.uuid4())
        
        # 2. Set the ID in the ContextVar and capture the Token
        token = correlation_id_ctx.set(correlation_id)
        
        try:
            # 3. Process the request
            response = await call_next(request)
            
            # 4. Propagate the ID back to the client (UI) via headers
            response.headers["X-CorrelationID"] = correlation_id
            return response
            
        finally:
            # 5. Clean up: Reset the context to prevent leakage in thread pooling
            correlation_id_ctx.reset(token)

4. "Background-Task" Conundrum

a specfic challenge encountered was handlign FASTAPI BackgroundTasks. since background tasks run after the HTTP reponse is returned, the middleware describes above has already finished execution and called reset(token). consequence being that background tasks start wtih an empty context, causing logs to show correlation_id: null

to solve this, we need to implementa Context Injection Pattern. we need to explicitily capture the ID while the requrst is still active and pass it to a wrapper function that re-hydrates the context inside the background thread.

api.py

@Api_router.post("/extract")
def extract(data: ExtractRequest, background_tasks: BackgroundTasks):
    # Capture the ID from the current context before the response is sent
    current_cid = correlation_id_ctx.get()
    
    # Pass it explicitly to the background task wrapper
    background_tasks.add_task(process_contract_sync, data.contract_id, current_cid)
    
    return {"message": f"Extracting data for contract ID: {data.contract_id}"}

def process_contract_sync(contract_id: str, correlation_id: str):
    # RE-INJECT the ID into the context for this isolated thread
    token = correlation_id_ctx.set(correlation_id)
    
    try:
        # Run the business logic with full logging context
        asyncio.run(process_contract(contract_id))
    except Exception as e:
        logging.exception(f"Critical failure in background task for {contract_id}")
    finally:
        # Strict cleanup using the token
        correlation_id_ctx.reset(token)

Conclusion

by leaving syncchronous sap-cf-logging library in favour of a native contextvars approach, we acheived:

1. full traceability: every log entry, including those in disconnected background tasks is tagged with a consistent Correlation ID

2. Stability: eliminated "socket hang up" errors caused by WSGI middleware incompabilties

3. Observability: logs appear in Kibana as structured JSON, allowing for deep filtering on specific correlation id or error states.

this approach offers a blueprint for any team aiming to utilize FastAPI alongside the strict enterprise observability requirements of SAP BTP.

New Machine Learning, NLP and AI features in SAP HANA Cloud 2025 Q4

2025-12-22T08:08:25.334000+01:00

With the SAP HANA Cloud 2025 Q4 release, several new embedded Machine Learning / AI functions have been released with the with the Predictive Analysis Library (PAL), the Automated Predictive Library (APL) and the NLP Services in SAP HANA Cloud.

Key new capabilities to be highlighted include

A new unified time series procedure, serving developers to utilize the same interface across different times series algorithms
text embedding model enhancements, supporting output vector dimensionality reduction, while maintaining retrieval accuracy
a new cross encoder model with the NLP services, for accurately re-ranking search results
text column input, text embedding operators with AutoML classification and regression models
text tokenization enhancements supporting regular expression token filtering and a new text log parsing function for detecting and determining new log pattern
the HANA ML experiment monitor UI now supports visual model monitoring and drift analysis

An enhancement summary is available in the What’s new document for SAP HANA Cloud database 2025.40 (QRC 4/2025).

Time series enhancements

Introducing Unified Time Series interfaces

Unified time series is a newly introduced interface for a simplified use of multiple time series algorithms (ARIMA, Exponential Smoothing, Bayesian Structural Time Series (BSTS) and Additive Model Analysis (aka prophet)).

it allows for application developers to flexibly consume different time series analysis algorithms using the same procedure interface and a harmonized the handling of time series data patterns, avoiding algorithm-specific data preparation.
It also supports massive, data-parallel time series modeling for maximum parallelism and performance when modeling thousands of time series in parallel

A more detailed introduction to the new function is given in the following blog post https://community.sap.com/t5/technology-blog-posts-by-sap/simplifying-time-series-analytics-with-unified-time-series-interface/ba-p/14292218

AutoML time series now support Prediction Intervals

AutoML time series predict function, in addition to all regular time series functions, now supports prediction intervals for probabilistic forecasting

the uncertainty associated with a forecast is quantified by providing a range (lower/upper bounds) into which a future observation likely falls with a specific confidence level.
For example a 95% prediction interval contains the true value 95% of the time

Text embedding model enhancements (NLP services)

Output vector dimension reduction

The Text Embedding models in SAP HANA Cloud have been enhanced with an attached a linear layer, derived from PCA trainings, to allow for the output vector dimensionality reduction

The arget dimension cardinality can be flexibly set to 128, 256 384, 512, 768 dimensions using PCA_DIM_NUM parameter
A near-original retrieval task accuracy is sustained with 256 dimensions, at 1/3rd of the original vector size (768 dimensions)
While lower dimension values than 128 would lead to significant and critical-level accuracy loss for retrieval tasks, hence 256 dimensions is recommended for efficiency & performance

Now text embeddings of significantly lower dimensionality can be utilized at a minimal information loss for retrieval tasks as well as machine learning.

Moreover smaller vector sizes unlock much faster machine learning processing times and may also serve vector retrieval queries

A more detailed introduction to the output vector dimensionality reduction is given in the following blog post New Cross Encoder and Text Embedding support Dimensionality Reduction in HANA NLP Service 2025 Q4- SAP Community blog post.

Text feature data support with AutoML models

Text column data and Text Embedding Operator in AutoML models

With the introduction of the a new text embedding operator for AutoML models, text columns can be directly used as input feature data and benefit from automatic, and optimized text vectorization utilizing SAP HANA Clouds text embedding models.

Text columns can be processed as feature, specified with the new parameter TEXT_VARIABLE
In addition, the new TextEmbedding operator supports the new target dimension reduction with the parameter PCA_DIM_NUM
The enhancement are available with the SQL interface as well as hana-ml 2.27 in Python

With that, the semantic insights from text columns can get automatically unlocked when building AutoML classification / regression models.

Cross Encoder Model (NLP services)

Accurate re-ranking of search results

The family NLP services in SAP HANA Cloud has now been enriched with a new cross encoder model and respective PAL functions. The cross encoder model

Processes pairs/sets of text sentences (query, candidate results) together
Therefore allows for a more precise semantic similarity re-ranking of search results, based on the full contextual interaction analysis between the query and the input candidate set (e.g. a an initial result set retrieved from a vector engine similarity search)
Thus much higher accurate and relevant-ranked similarity search results can be achieved

Moreover the use of cross encoder models allows to combine multiple result sets retrieved from for example classic text search and vector engine similarity search queries into a hybrid search result, which can be passed into the cross encoder for its overall and combined re-ranking.

Custom AI, Retrieval Augmented Generation Applications (RAG) can now fully be served by Text Embedding Models, Vector Engine with Similarity Search and the Cross Encoder model, managing context privacy from the SAP HANA Cloud database.

A more detailed introduction to the new cross encoder model is given in the following blog post New Cross Encoder and Text Embedding support Dimensionality Reduction in HANA NLP Service 2025 Q4- SAP Community blog post.

Text tokenization enhancements and new automated log text analysis

Text tokenization support for regular expressions

The text pre-processing function Text Tokenize function for splitting input text into smaller units called tokens, has now been enhanced and supports regular expression for filtering token output patterns.

Custom filtering (removal/keeping) of text patterns can be applied
List of regular expressions, matching tokens will be kept / excluded
Typical filtering examples include extracting e-mail addresses, URLs, or other token patterns for domain-specific needs

Automatic pattern detection from log texts

A new analysis function for log text documents Text Log Parse has been added to the Predictive Analysis Library, which allows

For an automatic extraction of new log patterns and derive templates for new log patterns
High-performant processing of log texts for log classification and automated log analysis, the ability to detect and alert for new log patterns

Real-time prediction performance improvements

Using PAL stateful ML models for real-time prediction performance

When a PAL ML model state is created, the model is parsed only once and kept as a runtime in-memory object (see PAL documentation on state-enabled-real-time-scoring-functions)

The actual prediction-function references to the PAL ML model by its STATE_ID
The repeated overhead of PAL ML model parsing with every predict-function call can be avoided in scenarios
- with rather complex and larger PAL ML models with significant model parsing time proportion
- the prediction runtime shall be as minimal as possible and near real-time
The prediction runtime performance has now been improved from ~100ms to ~20ms in exemplary use cases (see example PAL_ML_model_for_real-time_predictions.sql)

ML experiment tracking and task scheduling enhancements

Experiment tracking enhancements

Tracking of experiments, now supports custom track entity tags

Standard track entities generated in PAL Training, Forecast, etc. can now be enriched with custom tags like business related information associated with related track entity
Tag is name-value pair for annotation or note, binding extra information to existing entities

A more detailed introduction is provided in the following blog post comprehensive-guide-to-mltrack-in-sap-hana-cloud-end-to-end-machine

Moreover the Experiment Monitor in the Python Machine Learning client (hana-ml) supports for visually monitoring ML model performance degradation and drift.

One-off scheduling of PAL tasks

The PAL procedure scheduling interfaces has been enhance with a One-OFF schedule option, allowing for

ad-hoc, automatic one-off scheduled execution of PAL task procedures with dynamic setting of time-frequency information based on current UTC timestamp
it triggers a scheduled job to execute immediately, and the corresponding one-off schedule is removed right away and doesn’t require to be manually maintained.

Python ML client (hana-ml) enhancements

The full list of new methods and enhancements with hana_ml 2.27 is summarized in the changelog for hana-ml 2.27 as part of the documentation. You can find an examples notebook illustrating the highlighted feature enhancements here 25QRC04_2.27.ipynb.

The key enhancements in this release include the following:

Time series data outlier detection with threshold support

The method for time series outlier detection in the Predictive Analysis Library has added support outlier threshold settings, in addition to the outlier voting capability using different outlier evaluation methods incl. Z1 score, Z2 score, IIQR score, MAD score, IsolationForest and DBSCAN

If the absolute value of outlier score is beyond the threshold, the respective data point will be considered as an outlier.

Time series reports for massive, data-parallel model scenarios

Massive AutoML Time Series modeling scenarios often utilize random-search with hyperband as the fastest optimization, potentially with larger number of time series data segment groups to be processed and forecasted in parallel, each time series segment group again with a significant number of forecast models to be explored.

Hence the display of forecasts which have been explored by AutoML within each time series segment group is collapsed by default and can be expanded for review.

Classification and regression function enhancements

Support Vector Machine (SVM) model training is computationally expensive, and computational costs are specifically sensitive to the number of training points, which makes SVM models often impractical for large datasets.

The SVM algorithm now supports Coreset Sampling

which allows to automatically sample small, representative subsets (the "coreset") from larger datasets,
enabling faster, more efficient training and processing while maintaining similar model accuracy as using the full data.

This enhancement significantly reduces SVM training time with minimal impact on accuracy.

The model report for classification tasks now supports a percentage display in confusion matrix for easier visual interpretation of classification results.

High-dimensional feature data reduction using UMAP

UMAP (Uniform Manifold Approximation and Projection) is a non-linear dimensionality reduction algorithm used to simplify complex, high-dimensional feature spaces, while preserving its essential structure. It is widely considered the modern gold standard for visualizing targeted dimension reduction of large-scale datasets, because it balances computational speed with the ability to maintain both local and global relationships.

It reduces thousands of variables (dimensions) into 2D or 3D scatter plots that humans can easily interpret.
Unlike comparable methods like t-SNE, UMAP is better at preserving global structure, meaning the relative positions between different clusters remain more meaningful.
It is significantly faster and more memory-efficient than t-SNE, capable of processing datasets with millions of points in a reasonable timeframe.
It can be used as a "transformer" preprocessing step in Machine Learning scenarios to reduce large feature spaces before applying clustering (e.g., k-means, HDBSCAN) or classification models, often improving their performance.

Calculating pairwise distances

Many algorithms, for example clustering algorithms utilize distance matrixes as a preprocessing step, often inbuild the functions. Often there is the wish to decouple though the distance matrix calculation from the follow-up task like the actual clustering. Moreover, if decoupled custom calculated matrixes can be fed into algorithms as input.

Most PAL clustering functions support to feed-in a pre-calculated similarity matrix

Now, a pairwise distance calculation function is provided

It supports distance metrics like Manhattan, Euclidien, Minkowski, Chebyshey as well as Levenshtein
The Levenshtein distance (or edit distance) is a distance metric specifically targeting distance between text-columns. It calculates the minimum number of single-character edits (insertions, deletions, or substitutions) needed to transform one word into another, acting as a measure of their similarity. A lower distance indicates a higher similarity.

Applicable use cases

It is useful in data cleaning, table column similarity analysis between columns of the same data type.
After calculating the column similarity across all data types, clustering like K-Means can be applied to group similar fields and propose mappings for fields within the same cluster

Again, an incredible set of enhancements in the SAP HANA Cloud database AI engine and NLP services!

Enjoy to try out all the enhancements and let us know what you think here!

Leveraging SAP AI Core to Build Custom AI Agents with CrewAI

2025-12-24T06:41:18.594000+01:00

Introduction

AI agents are becoming the go-to pattern for building modular, autonomous assistants that can think, act, and collaborate. CrewAI gives you a developer-friendly way to orchestrate these agents, while SAP AI Core provides enterprise-grade access to LLMs with proper governance, scalability, and security.

This quickstart shows you how to connect CrewAI to SAP AI Core, create a custom LLM interface, and build your first working agent — all in under 10 minutes.

By the end, you’ll have a simple CrewAI agent powered by an LLM hosted in SAP AI Core.

How SAP Facilitates Building Custom Agents

CrewAI handles the agent orchestration layer, while SAP AI Core hosts and manages your LLMs.

Connecting them allows you to use SAP’s AI infrastructure directly inside your CrewAI workflows — no external API juggling.

Here’s the high-level flow:

User Prompt → CrewAI Agent → Custom LLM Class → SAP AI Core Endpoint → Model Response

Flow summary:

User sends a prompt → CrewAI Agent receives it.
Agent forwards it to a custom LLM wrapper (`SAPAILLM`).
The wrapper authenticates using the token and sends the request to SAP AI Core.
SAP AI Core executes the model and returns a response to the agent.
CrewAI handles the output and delivers the final answer.

This setup ensures enterprise-level reliability with full control over which models your agents use.

Prerequisites

Before jumping in, make sure you have:

Access to SAP AI Core with deployed LLM models (via AI Launchpad)
Python 3.10+
VS Code or your favorite IDE
Installed libraries:

pip install ipykernel
pip install crewai
pip install crewai_tools
pip install generative-ai-hub-sdk[all]
pip install langchain_community

Your SAP AI Core Service Key downloaded as a .json file
Your A-game to create some amazing agents

*If you want to learn how to deploy your LLM model on launchpad, you can go through this course.

Step-by-Step Guide

Let us now dive into building our own custom agent which utilizes SAP's AI Core.

Step 1: Get Your SAP AI Core Credentials

In your SAP BTP Cockpit → AI Core Instance → Service Keys, create a new key and save it locally as credentials.json.

Example structure:

{
    "clientid": "your-client-id",
    "clientsecret": "your-client-secret",
    "url": "https://<your-region>.authentication.sap.hana.ondemand.com",
    "serviceurls": {
        "AI_API_URL": "https://***.aws.ml.hana.ondemand.com"
    }
}

This file contains everything needed to authenticate and access your deployed LLM.

Step 2: Get Your BTP LLM Access Token

We’ll use this token to authenticate each request to SAP AI Core.

import json, requests

# open credentials file
with open("<path-to-file>/credentials.json", "r") as key_file:
    svcKey = json.load(key_file)
authUrl = svcKey["url"]
clientid = svcKey["clientid"]
clientsecret = svcKey["clientsecret"]
apiUrl = svcKey["serviceurls"]["AI_API_URL"]

# request token
params = {"grant_type": "client_credentials" }
resp = requests.post(f"{authUrl}/oauth/token",
                    auth=(clientid, clientsecret),
                    params=params)

BtpLlmAccessToken = resp.json()["access_token"]
print("Token retrieved successfully!")

Tokens usually expire after 1 hour. Make sure to refresh them before making multiple requests.

Step 3: Create a Custom LLM Class Template for CrewAI

CrewAI lets you define your own LLM wrappers, which means we can make one that talks directly to SAP AI Core.

In this class, we can define the procedure to use the deployed models from SAP AI Core.

You can now import this class into any CrewAI workflow to use SAP’s hosted LLMs.

from crewai import BaseLLM
from typing import Any, Dict, List, Optional, Union

class CustomLLM(BaseLLM):
    def __init__(self, model: str, api_key: str, endpoint: str, temperature: Optional[float] = None):
        super().__init__(model=model, temperature=temperature)

        self.api_key = api_key
        self.endpoint = endpoint

    def call(
        self,
        messages: Union[str, List[Dict[str, str]]],
        tools: Optional[List[dict]] = None,
        **kwargs
    ) -> Union[str, Any]:
        """Call the LLM with the given messages."""
        # Convert string to message format if needed
        if isinstance(messages, str):
            messages = [{"role": "user", "content": messages}]

        payload = {
            "messages": messages,
            "temperature": self.temperature,
            "max_tokens": 1000 # This can be modified as per your use case. You can parameterize this as well.
        }

        headers = {
            'AI-Resource-Group': "<your-resource-group-name>",
            'Content-Type': 'application/json',
            'Authorization': f'Bearer {BtpLlmAccessToken}',
        }

        # Make API call
        response = requests.post(
            self.endpoint,
            headers= headers,
            json=payload,
            timeout=30
        )
        response.raise_for_status()

        result = response.json()
        return result["choices"][0]["message"]["content"]

Step 4: Initialize your Custom LLM Class

Alright, time to bring that class to life.

The cool thing here is that you can hook it up to any model you’ve deployed in SAP AI Core, whether it’s GPT-4o, Claude, or your own fine-tuned beast.

In this example, we’ll wire it up to a **GPT-4o** deployment and let CrewAI start chatting through it. You can get the deployment endpoint from your SAP AI Launchpad. You can find the payload formats for different models here.

deployment_url = "<deployment URL>" + "/chat/completions?api-version=2024-02-01"

# instantiating Custom LLM object
custom_llm = CustomLLM(
    model="gpt-4o",
    api_key=BtpLlmAccessToken,
    endpoint=deployment_url,
    temperature=0.7
)

Step 5: Build a Simple CrewAI Agent

Now that your LLM class is ready, let’s create a basic Research CrewAI agent that uses it.

from crewai import Agent, Task, Crew

# Load credentials
agent = Agent(
    role="Research Assistant",
    goal="Find and analyze information",
    backstory="You serve as a research assistant, helping to gather and analyze information about SAP tools and related topics.",
    llm=custom_llm
)

# Create and execute tasks
task = Task(
    description="Research the latest developments in SAP AI Core",
    expected_output="A comprehensive summary",
    agent=agent
)

crew = Crew(agents=[agent], tasks=[task])
result = crew.kickoff()

If all goes well, you’ll see a clean response from your SAP-hosted LLM, directly through your CrewAI agent. Here is a sample output:

CrewOutput(raw="The latest developments in SAP AI Core focus on enhancing the capabilities for deploying and managing AI models in a scalable and efficient manner. SAP AI Core is part of SAP Business Technology Platform (BTP) and is designed to integrate with other SAP applications to streamline AI operat....ding tools that enhance productivity, and driving innovation through AI-powered solutions.", pydantic=None, json_dict=None, agent='Research Assistant', output_format=<OutputFormat.RAW: 'raw'>)], token_usage=UsageMetrics(total_tokens=0, prompt_tokens=0, cached_prompt_tokens=0, completion_tokens=0, successful_requests=0))

Conclusion

And that’s it — you just built your first CrewAI agent powered by SAP AI Core.

From here, you can:

Add tools for the agent to interact with SAP APIs (e.g., fetching invoice data).
Add memory and multi-agent orchestration.
Deploy it on SAP BTP or your internal infrastructure for enterprise use.

This quick start gives you the foundation — the next step is making your agent truly yours.

References

New Machine Learning, NLP and AI features in SAP HANA Cloud 2025 Q3

2026-01-09T12:54:46.437000+01:00

With the SAP HANA Cloud 2025 Q3 release, several new embedded Machine Learning / AI functions have been released with the SAP HANA Cloud Predictive Analysis Library (PAL) and the Automated Predictive Library (APL).

An enhancement summary is available in the What’s new document for SAP HANA Cloud database 2025.28 (QRC 3/2025).

Time series analysis and forecasting function enhancements

Threshold support in timeseries outlier detection

In time series, an outlier is a data point that is different from the general behavior of remaining data points. In the PAL time series outlier detection function, the outlier detection task is divided into two steps

In step 1 the residual values are derived from the original series,
In step 2, the outliers are detected from the residual values.

Multiple methods are available to evaluate a data point to be an outlier or not.

Including Z1 score, Z2 score, IIQR score, MAD score, IsolationForest, DBSCAN
If used in combination, outlier voting can be applied for a combined evaluation.

Now, new and in addition, thresholds values for outlier scores are supported

New parameter OUTPUT_OUTLIER_THRESHOLD
Based on the given threshold value, if the time series value is beyond the (upper and lower) outlier threshold for the time series, the corresponding data point as an outlier.
Only valid when outlier_method = 'iqr', 'isolationforest', 'mad', 'z1', 'z2'.

Classification and regression function enhancements

Corset sampling support with SVM models

Coreset sampling is a machine learning technique to

select a small, representative subset (the "coreset") from larger datasets,
enabling faster, more efficient training and processing while maintaining similar model accuracy as using the full data.
It works by identifying the most "informative" samples, filtering out redundant or noisy data, and allowing complex algorithms to run on a manageable dataset sizes.

Therefore SVM in the Predictive Analysis Library has been enhanced and now

offers embedded coreset sampling capabilities
enabled with the new parameters USE_CORESET and CORESET_SCALE as the sampling ratio when constructing coreset.

This enhancement significantly reduces SVM training time with minimal impact on accuracy.

AutoML and pipeline function enhancements

Target encoding support in AutoML

The PAL AutoML framework introduces a new pipeline operator for target encoding of categorial features

Categorical data is often required to be preprocessed and required to get converted from non-numerical features into formats suitable for the respective machine learning algorithm, i.e. numeric values
- Examples features: text labels (e.g., “red,” “blue”) or discrete categories (e.g., “high,” “medium,” “low”)
One-hot encoding converts each categorial feature value into a binary column (0 or 1), which works well for features with a limited number of unique values. PAL already applies an optimized one-hot encoding method aggregating very low frequent values.
Target encoding replaces the categorial values with the mean of the target / label column for high-cardinality features, which avoids to create large and sparse one-hot encoded feature matrices
- Example of a high cardinality feature: “city” column with hundreds-thousands of unique values, postal code, product IDs etc.

The PAL AutoML engine will analyze the input feature cardinality and then automatically decide if to apply target encoding or another encoding method. For medium to high cardinality categorial features, target encoding may improve the performance significantly.

By automating target encoding, the PAL AutoML engine aims to improve model performance and generalization, especially when dealing with complex, high-cardinality categorical features, without requiring manual intervention.

In addition, the AutoML and pipeline function now also support columns of type half precision vector.

Misc. Machine Learning and statistics function enhancements

High-dimensional feature data reduction using UMAP

It reduces thousands of variables (dimensions) into 2D or 3D scatter plots that humans can easily interpret.
Unlike comparable methods like t-SNE, UMAP is better at preserving global structure, meaning the relative positions between different clusters remain more meaningful.
It is significantly faster and more memory-efficient than t-SNE, capable of processing datasets with millions of points in a reasonable timeframe.
It can be used as a "transformer" preprocessing step in Machine Learning scenarios to reduce large feature spaces before applying clustering (e.g., k-means, HDBSCAN) or classification models, often improving their performance.

The following new functions are introduced

_SYS_AFL.PAL_UMAP with the most important parameters N_NEIGHBORS, MIN_DIST, N_COMPONENTS, DISTANCE_LEVEL

_SYS_AFL.PAL_TRUSTWORTHINESS, used to measure the structure similarity between original high dimensional space and embedded low dimensional space based on K nearest neighbors.

Calculating pairwise distances

Many algorithms, for example clustering algorithms utilize distance matrixes as a preprocessing step, often inbuild to the functions. While often there is the wish to decouple though the distance matrix calculation from the follow-up task like the actual clustering. Moreover, if decoupled custom calculated matrixes can be fed into algorithms as input.

Most PAL clustering functions support to feed-in a pre-calculated similarity matrix

Now, a dedicated pairwise distance calculation function is provided

It supports distance metrics like Manhattan, Euclidien, Minkowski, Chebyshey as well as Levenshtein
The Levenshtein distance (or “edit distance”) is a distance metric specifically targeting distance between text-columns.
- It calculates the minimum number of single-character edits (insertions, deletions, or substitutions) needed to transform one word into another, acting as a measure of their similarity. A lower distance indicates a higher similarity.

Applicable use cases

It is useful in data cleaning, table column similarity analysis between columns of the same data type.
After calculating the column similarity across all data types, clustering like K-Means can be applied to group similar fields and propose mappings for fields within the same cluster

Real Vector data type support

The following PAL functions have been enhanced to support columns of type real vector

Spectral Clustering
Cluster Assignment
Decision tree
Sampling

In addition the AutoML and pipeline function now also support columns of type half precision vector.

Creating Vector Embeddings enhancements

The SAP HANA Database Vector Engine function VECTOR_EMBEDDING() has added support for remote, SAP AI Core exposed embedding models. Detailed instruction are given in the documentation at Creating Text Embeddings with SAP AI Core | SAP Help Portal

Python ML client (hana-ml) enhancements

The full list of new methods and enhancements with hana_ml 2.26 is summarized in the changelog for hana-ml 2.26 as part of the documentation. The key enhancements in this release include

New Functions

Added text tokenization API.
Added explainability support with IsolationForest Outlier Detection
Added constrained clustering API.
Added intermittent time series data test in time series report.

Enhancements

Support time series SHAP visualizations for AutoML Timeseries model explanations

You can find an examples notebook illustrating the highlighted feature enhancements here 25QRC03_2.26.ipynb.

Deploy Machine Learning Model as Fast API to Cloud Foundry BTP Trial

2026-01-14T18:17:09.330000+01:00

Deploy Machine Learning Model as Fast API to Cloud Foundry BTP Trial

Objective: This Blog helps an SAP developer who is new to Machine Learning and want to learn how a python machine learning model can be deployed to BTP trial account.

Later this model api can be consumed to SAP UI5 application.

Project structure was the challenging part for me being an on-prem ABAP consultant 😀 with zero knowledge in BTP deployment.

Note: This model is not an enterprise grade machine learning model. This is a beginner friendly model for learning purpose.

Pre-requisite:

BTP trial account and Business application Studio.

Create Project Structure:

Step 1: Create simple machine learning python program using scikit learn and expose it using FAST API.

My Project structure:

mypython/

|-- app.py # FastAPI python code

|-- requirements.txt # Python Dependencies

|-- Procfile # Command for CF(Cloud Foundry) to start the web application

|--manifest.yml # CF deployment config

File 1: app.py

This is a sample python program to create REST API that serves a machine learning model for classifying Iris flowers using The Gaussian Naive Bayes classifier.

File 2: manifest.yml

File 3: Procfile

File 4: requirement.txt

Local Testing:

Test the program before deployment to cloud foundry.

Click on the app.py file and right-click -> Open in integrated Terminal. Terminal will open with current project directory.

Run command pip install -r requirements.txt

Run command to test the fast api locally.

Code written is used for running the fast api locally

# Run locally with: app.py

if __name__ == "__main__":

import uvicorn

uvicorn.run(app, host="0.0.0.0", port=8000)

Hover to the link and click on follow link.

Below links will open.

Add postfix \docs to test the application in browser.

Deployment to Cloud Foundry

In Business application studio, First login to cloud foundry

Press ctrl+shift+P to login to cloud foundry and login to cloud foundry.

Push command to final deploy to cloud foundry

In case deployment is fails logs can be checked by

cf logs < CF app name e.g. > --recent

Check the deployed app in cloud foundry

Go to the sub-account and the dev space where you have deployed the app you can get all the necessary details.

You can click on the api link and check test the api by added \docs postfix.

Conclusion:

You got the basic understanding how python api gets deployed to cloud foundry which can consumed by the UI5 or CAP application for integrating it to business application.

Happy Learning!

Reference:

Create an Application with Cloud Foundry Python Buildpack | SAP Tutorials

Displaying SAP Analytics Cloud KPI Tiles from Stories Using REST APIs

2026-02-04T08:51:58.622000+01:00

Introduction

SAP Analytics Cloud (SAC) is widely used by organizations to provide interactive storytelling and track the business KPIs with advanced visualizations. However, companies often need to obtain KPI information in ways other than the SAC user interface, including in custom web apps. Although SAC does not support the direct embedding of KPI tiles into external apps, it does provide REST APIs that allow programmatic access to widget-level data from SAC stories. These APIs can be used to collect and display KPI tile data, including number (value), number state (status), title, and subtitle, in a bespoke user interface.

In this blog, I walk through a detailed, end-to-end implementation that illustrates how to fetch KPI tile data from a SAP Analytics Cloud story using the widgetquery/getWidgetData REST API. The approach uses Python for backend processing and Flask as a lightweight web framework to securely call SAC APIs and output KPI values on a web page.

Configuration of the Project

We may test API access and retrieve KPI tile data using a straightforward Python script before developing the Flask application. This program shows you how to:

Use OAuth 2.0 to authenticate with SAC
Acquire a token of access
Use the SAC widgetquery/getWidgetData RESTAPI.
Show the console’s KPI values

import requests
import webbrowser
import urllib.parse

# ---------------- CONFIG ----------------
TENANT_URL = "https://<your-tenant>.hanacloudservices.cloud.sap"

CLIENT_ID = "<YOUR_CLIENT_ID>"
CLIENT_SECRET = "<YOUR_CLIENT_SECRET>"

AUTHORIZATION_ENDPOINT = "https://<your-tenant>.hana.ondemand.com/oauth/authorize"
TOKEN_ENDPOINT = "https://<your-tenant>.hana.ondemand.com/oauth/token"

REDIRECT_URI = "https://your-app-domain.com/oauth/callback"  # used only to capture code manually

STORY_ID = "<your-storyid>"
WIDGET_IDS = [
    "Chart_1", "Chart_2", "Chart_3",
    "Chart_4", "Chart_5", "Chart_6", "Chart_8"
]

# ---------------- STEP 1: LOGIN ----------------
params = {
    "response_type": "code",
    "client_id": CLIENT_ID,
    "redirect_uri": REDIRECT_URI
}

auth_url = AUTHORIZATION_ENDPOINT + "?" + urllib.parse.urlencode(params)

print("\n Opening browser for SAC login...")
webbrowser.open(auth_url)

code = input("\n Paste the authorization code here: ").strip()

# ---------------- STEP 2: TOKEN ----------------
payload = {
    "grant_type": "authorization_code",
    "code": code,
    "redirect_uri": REDIRECT_URI,
    "client_id": CLIENT_ID,
    "client_secret": CLIENT_SECRET
}

token_resp = requests.post(
    TOKEN_ENDPOINT,
    data=payload,
    headers={"Content-Type": "application/x-www-form-urlencoded"}
)

token_resp.raise_for_status()
access_token = token_resp.json()["access_token"]

print("\n Access token received")

# ---------------- STEP 3: FETCH KPI ----------------
headers = {
    "Authorization": f"Bearer {access_token}",
    "Accept": "application/json"
}

print("\n KPI VALUES\n" + "-" * 40)

for widget_id in WIDGET_IDS:
    url = f"{TENANT_URL}/widgetquery/getWidgetData"
    params = {
        "storyId": STORY_ID,
        "widgetId": widget_id,
        "type": "kpiTile"
    }

    r = requests.get(url, headers=headers, params=params)
    if r.ok:
        data = r.json()
        number = data.get("number", "N/A")
        title = data.get("title", widget_id)
        print(f"{title}: {number}")
    else:
        print(f"{widget_id}:  Error")
print("\n Done")

How this operate

Login Step: Launches a web browser to obtain the permission code and log into SAC.
Token Step: Provides an access token in exchange for the authorization code.
Fetch KPI Step: Prints the KPI number and title after contacting the SAC REST API for each widget ID.

Code of Application

The full Flask application that is used to retrieve KPI tile data and authenticate with SAP Analytics Cloud is shown below. This code manages widget data fetching, token retrieval, login, and creates an eye-catching KPI dashboard in the browser.

HTML_TEMPLATE = """
<!DOCTYPE html>
<html lang="en">
<head>
<meta charset="UTF-8">
<meta name="viewport" content="width=device-width, initial-scale=1.0">
<title>SAC KPI Dashboard</title>
<style>
    body {
        font-family: 'Segoe UI', Tahoma, Geneva, Verdana, sans-serif;
        background: #f4f6f8;
        margin: 0; padding: 0;
    }
    h1 { text-align: center; margin-top: 20px; color: #333; }
    .container {
        display: flex;
        flex-wrap: wrap;
        justify-content: center;
        margin: 40px auto;
        max-width: 1200px;
        gap: 20px;
    }
    .card {
        color: #fff;
        width: 250px;
        height: 150px;
        border-radius: 16px;
        box-shadow: 0 10px 20px rgba(0,0,0,0.2);
        display: flex;
        flex-direction: column;
        justify-content: center;
        align-items: center;
        transition: transform 0.3s, box-shadow 0.3s;
        cursor: pointer;
        text-align: center;
        padding: 10px;
    }
    .card:hover {
        transform: translateY(-10px);
        box-shadow: 0 20px 30px rgba(0,0,0,0.3);
    }
    .number {
        font-weight: bold;
        white-space: nowrap;
        overflow: hidden;
        text-overflow: ellipsis;
        max-width: 90%;
    }
    .title {
        font-size: 1em;
        margin-top: 10px;
        color: #e0e0e0;
    }
    /* Rainbow colors for cards */
    .rainbow-0 { background: linear-gradient(135deg, #ff6b6b, #f06595); }
    .rainbow-1 { background: linear-gradient(135deg, #feca57, #ff9f43); }
    .rainbow-2 { background: linear-gradient(135deg, #1dd1a1, #10ac84); }
    .rainbow-3 { background: linear-gradient(135deg, #54a0ff, #2e86de); }
    .rainbow-4 { background: linear-gradient(135deg, #5f27cd, #341f97); }
    .rainbow-5 { background: linear-gradient(135deg, #ee5253, #c0392b); }
    .rainbow-6 { background: linear-gradient(135deg, #48dbfb, #00d2d3); }
</style>
<script>
    // Adjust font size based on length
    function adjustFontSize() {
        const numbers = document.querySelectorAll('.number');
        numbers.forEach(num => {
            const length = num.innerText.length;
            if(length <= 5) num.style.fontSize = '2.5em';
            else if(length <= 8) num.style.fontSize = '2em';
            else num.style.fontSize = '1.5em';
        });
    }
    window.onload = adjustFontSize;
</script>
</head>
<body>
<h1>SAP Analytics Cloud - RESTAPI Fetched Sales KPI Dashboard</h1>
<div class="container">
    {% for kpi in kpis %}
    <div class="card rainbow-{{ loop.index0 % 7 }}">
        <div class="number">{{ kpi.number }}</div>
        <div class="title">{{ kpi.title }}</div>
    </div>
    {% endfor %}
</div>
</body>
</html>
"""

from flask import Flask, render_template_string
import requests
import urllib.parse
import webbrowser

# ---------------- CONFIG ----------------
TENANT_URL = "https://yourtenant.hanacloudservices.cloud.sap"
CLIENT_ID = "<YOUR_CLIENT_ID>"
CLIENT_SECRET = "<YOUR_CLIENT_SECRET>"
AUTHORIZATION_ENDPOINT = "https://yourtenant.hana.ondemand.com/oauth/authorize"
TOKEN_ENDPOINT = "https://yourtenant.hana.ondemand.com/oauth/token"
REDIRECT_URI = "https://your-app-domain.com/oauth/callback"

STORY_ID = "<your_storyid>"
WIDGET_IDS = [
    "Chart_1", "Chart_2", "Chart_3",
    "Chart_4", "Chart_5", "Chart_6", "Chart_8"
]

# ---------------- FLASK APP ----------------
app = Flask(__name__)

def get_access_token():
    # Step 1: login manually
    params = {"response_type": "code", "client_id": CLIENT_ID, "redirect_uri": REDIRECT_URI}
    auth_url = AUTHORIZATION_ENDPOINT + "?" + urllib.parse.urlencode(params)
    print("\nOpen this URL in browser to login to SAC:")
    print(auth_url)
    webbrowser.open(auth_url)
    code = input("\nPaste the authorization code here: ").strip()

    # Step 2: get token
    payload = {
        "grant_type": "authorization_code",
        "code": code,
        "redirect_uri": REDIRECT_URI,
        "client_id": CLIENT_ID,
        "client_secret": CLIENT_SECRET
    }
    r = requests.post(TOKEN_ENDPOINT, data=payload, headers={"Content-Type": "application/x-www-form-urlencoded"})
    r.raise_for_status()
    return r.json()["access_token"]

def fetch_kpis(token):
    headers = {"Authorization": f"Bearer {token}", "Accept": "application/json"}
    kpis = []
    for widget_id in WIDGET_IDS:
        url = f"{TENANT_URL}/widgetquery/getWidgetData"
        params = {"storyId": STORY_ID, "widgetId": widget_id, "type": "kpiTile"}
        r = requests.get(url, headers=headers, params=params)
        if r.ok:
            data = r.json()
            kpis.append({
                "title": data.get("title", widget_id),
                "number": data.get("number", "N/A")
            })
        else:
            kpis.append({"title": widget_id, "number": "Error"})
    return kpis
# HTML Code will be written Here
@app.route("/")
def dashboard():
    token = get_access_token()
    kpis = fetch_kpis(token)
    return render_template_string(HTML_TEMPLATE, kpis=kpis)

if __name__ == "__main__":
    app.run(debug=True,port=8000)

An explanation of the code

/route logic
- To manually retrieve an OAuth token, use get_access_token().
- Uses fetch_kpis() to retrieve KPI tile data.
- Uses the HTML_TEMPLATE to render all KPIs with rainbow-colored cards.
Managing OAuth (get_access_token)
- Launches the browser's SAC login.
- The authorization code is pasted by the user. To call SAC REST APIs, code is exchanged for access tokens.
Data retrieval for widgets (fetch_kpis)
- Repeats over every WIDGET_IDS.
- For every widget, the widgetquery/getWidgetData endpoint is called.
- Gathers the number and title for the display.
Using HTML Templates for UI rendering
- Flex arrangement is used for tiles.
- Each KPI card has a rainbow gradient background.
- For a contemporary appearance, use shadow and hover effects.
- The length of the integer automatically modifies the font size.

Result/Output

The KPI data is retrieved and shown in a personalized web dashboard after the program has launched and the user has successfully logged in using SAP Analytics Cloud OAuth.

SAP Analytics Cloud Story KPI Tiles

The original KPI tiles as they appear in the SAP Analytics Cloud narrative are depicted in the image below. Business users in SAC are in charge of configuring and maintaining these KPIs.

Custom Web Dashboard with KPI Tiles

The widgetquery/getWidgetData REST API is used to retrieve the same KPI values, which are then shown in a specially created web application. Custom Web Dashboard Screenshot

Important Findings

The KPI numbers in the SAC story and the web dashboard are same.
Every KPI tile shows:

Value or Number
The title

The website's dashboard uses the following to improve visualization:

Gradient cards with rainbows
Hover animations and shadows
Adaptable design

This proves to the secure consumption and reuse of SAC KPI data outside of the SAC user interface without requiring the duplication of business logic.

Conclusion

In this blog post, we shown a simple yet effective technique for extracting SAP REST APIs are used to tile Analytics Cloud KPI data and display it on a special webpage.

Important findings:

SAC KPIs can be consumed using custom dashboards outside of the SAC UI.
Python provides an adaptable and lightweight backend for API integration. REST APIs enable KPI tracking in real-time, while front-end. Style increases visibility.
Setting up the environment and security is essential for a safe execution.
SAC insights can be incorporated into executive dashboards, intranet portals, or external web apps, businesses are able to give users a consolidated view of key performance indicators without having to Launch SAC directly.