SAP Community - SAP Data Services

BODS migration

2024-05-14T10:39:48.281000+02:00

In this blog, I have detailed the steps taken for SAP BODS application migration from existing on premises to HANA database including the DR Testing Setup.

The Purpose of this blog is to provide a guidance path for migration of the BODS Application to HANA.

Existing Landscape Design:
1. The IPS and DS version remain the same as on premises.

In this landscape NW JAVA is used as underlying Web Application instead of Tomcat.
This landscape had additional WACS installation.
There is a single Central repository across the landscape.

Considerations prior to Migration:
1. The Version history for Objects cannot be retained in case of a Migration of BODS application to another Database. SAP KBA 2768127.

The migration is performed using the ATL code import mechanism.
Central Repository migration must be performed via ATL code import into local repository. i.e. An additional local repository needs to be created on prem for this activity.
For Installation and build the <SID>adm user should be using csh shell for IPS and NW JAVA.
For Installation and build the <SID>adm user should be using the bash shell for DS.

Migration Steps:

Since the landscape consists of NW JAVA – Import / Export for NW JAVA via SWPM tool is used.
For the Database HANA MCD Setup is used. Corresponding users are created for each repository and the CMS & AUD repositories.
HANA Encryption performed as per the SAP Blog SAP HANA Database Encryption - SAP Community
For HANA Database connectivity, UNIXodbc must be installed on all application servers for BODS.
1. Can be checked by using ./odbcinst -j as root user.
2. Edit the odbc.ini file to allow for the connections to the HANA database from the BODS Application servers. The DSN entry could be for each user for the Repository / Single DSN used across the repositories, but the <CMS> and <AUD> DSN are mandatory. Examples below:
3. The connection can be checked using the command as below as root user.
  1. Command – isql <DSN> <USERNAME> <PASSWORD>

Each BODS Application server requires its own IPS and DS installation.

Few important steps during the installation which needs to be considered:

The IPS and DS Installation can be run directly using the patch download for the required version.
The Installation for IPS must be run with <SID>adm user under csh shell.
Once the CMS and AUD Repositories are confirmed on the HANA Database, select the option to manually deploy Web Applications

The SIA name and port details need to be confirmed at this time to use <hostname> and port 6400 or any other port. We will cover cluster name change and cluster setup for IPS under step <f>.
In case of multiple Application servers for BODS – as mentioned above each server will require its own IPS and DS installation. While performing the second IPS installation for the CMS Information step, you will need to provide the hostname for the first IPS installation.
For changing the cluster name in case of multiple applications once the steps till <e> are completed, stop the IPS as per SAP note 2143877 - What is the correct start and stop order for Data Services components. Follow the SAP note 3169978 - How to change CMS cluster name in Linux for BI 4.x to update the cluster name. The input needed will be <hostname:port> while providing the new Cluster name – which will translate to <@hostname:port> while logging onto BOE/CMC & DSMC and Designer tool.
For DS Installation as well, run with <SID>adm user. The destination folder for the DS Installation can be pre-created.
For the CMS Logon during DS installation – provide each individual <hostname:port> information. Execute Step <f> at the end.
For our purpose we had re-used the DSConfig.txt from the on-premises, during the DS Installation – this helps to cover any added parameters into the DSCOnfig.txt file from on premises but does not help with the job server and repositories assignments – which must be repeated to be covered after step <l and m>
Since we had selected Web Application to be manually deployed, we will have to run Wdeploy to create the configuration files to be imported to NW JAVA via SUM Tool.
1. Edit the file corresponding to the NW version under path <BODS Installation Directory>/enterprise_xi40/wdeploy/conf. The file name would be in the format config.sapappsvr<XX>
2. The inputs into this File would be against “as_sid”, “as_instance”, “as_admin_username” and “as_admin_password” fields
3. Run the command as ./wdeploy.sh sapappsvr<XX> predeployall
4. This creates the required files under path <BODS Installation Directory>/ enterprise_xi40/wdeploy/workdir/sapappsvr<XX>/application
5. Copy over these files to /usr/sap/trans/EPS/in
6. Run the SUM tool and import the components to the NW JAVA

For WACS installation run the ./modifyOrRemoveProducts.sh under BODS installation directory and select the Information Platform Services to modify. Make sure to not remove any selections already showing – but only add the selection for “Web Application Container Service” in the Step and complete the installation providing the same information as provided during the IPS installation for each server.

Create the Repositories corresponding to the Users created on the HANA Database using the below command.

Command to be run from <BODS installation Directory>/bin

./repoman -c -tlocal -U<USERNAME> -P<Password> -S<FQDN for the DB> -p<Tenant Port> -s -NHANA -Q<SID>

./repoman -c -tcentral -U<USERNAME> -P<Password> -S<FQDN for the DB> -p<Tenant Port> -s -NHANA -Q<SID>

For Central Repository the “Secure” option can be activated using the Repository Manager tool.

The Tenant port can be checked using the command from the SYSTEM Database

SELECT DATABASE_NAME, SQL_PORT FROM SYS_DATABASES.M_SERVICES

Re-create the Job server and assign Repositories.
1. Edit the DSConfig.txt file to remove the entries for Job Server, Access Server coming from On Premises. Find the Section [AL_JobServer] and remove the entries against fields “AL_JobServerName1” & “AL_JobServerPort1” and any more entries. Keep the Same Job Server name and port while recreating.
2. Find the Section [Access_Server] and remove the entry against “Connection_ID”
3. Once the above two are completed, when you run ./svrcfg it will prompt you to start creating Job Servers and assign Repositories against them. Re-create on each BODS Application server exactly as it was on Premises with the same port details.
4. For the usage of “Server Group” in DSMC – you will need to assign the same repository to multiple job servers on each BODS Application server.
You can source the al_env.sh and odbc.sh into the .bashrc file for the <SID>adm user by adding the below to entries to the file:

source /<BODS Installation Directory>/dataservices/DataDirect/odbc/odbc.sh

source /<BODS Installation Directory>/dataservices/bin/al_env.sh

Once the above steps are completed – You can Export the LCMBIAR and ATL from the On-premises application and Import into the Target Environment.
1. For LCMBIAR Export and Import Note that during the process, you will need to select each page and use the “select all” option above for each page. If not done, there could be some objects missed out.
2. Check the total objects being promoted from on premises and make sure the count is at least the same (could be more for target).
3. If the option “Hot Backup” is enabled under CMC in Settings each time you try to run the LCMBIAR File import and check the final count on promotable objects in the target it will show double – this is normal behaviour. Suggest deleting existing job and run a new job every time.
Login to CMC using the System details as <@Clustername:port> and under Data Services – you will need to delete and re-create each repository with the new hostname and port details.

Disaster Recovery:

The Disaster Recovery strategy used here is to have a pre-installed BODS Application and Database and during DR scenarios the latest available HANA backup is used to recover the Database on the DR Site. To achieve this, below steps were used.

BODS Application including NW JAVA as described in the above section is pre-built on the DR VM provisioned along with the HANA tenant on the DR site.
Steps for LCMBIAR & ATL export/import, WACS Install, creation of repositories and Job Servers, Edit of the DSConfig.txt file are excluded in this build.
A VM Backup and HANA Database Backup is retained at this stage for future re-use.

As part of the Disaster Recovery Test the below Steps are performed, without disturbing the existing Production Landscape.

HANA backup from Production and Restore onto the DR Database.
Follow SAP note 1275068 to recreate SIA for the application.
1. Select * from cms_infoobjects7 where parented=16 or parented=59;
2. Delete from cms_infoobjects7 where parented=16 or parented=59;
Re-create SIA using ./serverconfig.sh following SAP note 2206006. You will require the Cluster key and the CMS DSN user credentials to connect to the database from Production.
Change Cluster Name as detailed in the above section under step <f>.
For NW JAVA as the Database is coming in from Production backup – the only option is to uninstall – SAP Systems or Single Instances.

Re-Install NW JAVA using the option Standard Installation under HANA Database, but using “Homogeneous System Copy” instead of “Standard System Copy” used during build.

Provide the Existing Production Database SYSTEM and Schema details in the next steps along with the schema password and select “Data has already been Loaded”.

New Service Plan: Lower Pricing for SAP Data Quality Management, microservices for location data

2024-06-13T23:55:31.949000+02:00

Introduction:

We are thrilled to announce an exciting update to SAP Data Quality Management, microservices for location data. In our ongoing efforts to provide you with the best value and meet your evolving needs, we will be introducing a new, more affordable service plan available under the SAP BTP Enterprise Agreement (BTPEA) or Pay-As-You-Go for SAP BTP. The new Professional plan will replace our existing Standard plan, which will be gradually phased out in the coming months. We understand that change can be unsettling, but we assure you that this transition will bring numerous benefits and cost savings for all our valued customers.

Why the Change?

As a customer-centric company, we continuously evaluate our offerings to ensure they align with your requirements and provide the best possible experience. After careful consideration and feedback from our customers, we have developed a new service plan that offers the same quality and features you love, but at a significantly lower cost. Our goal is to make our services more accessible and affordable to a wider range of customers.

What to Expect from the New Service Plan:

Our new Professional plan will offer all the essential features and benefits of the existing Standard plan, but at a reduced price point. The service will ensure backward compatibility with our service APIs, so you can simply migrate to the new service plan to work with your existing systems and workflows without any compatibility issues. However, the Professional plan offers an opportunity for growth and expansion, so we encourage you to consider migrating to this plan when you are ready to leverage the benefits of future innovation.

Service Region Availability:

We are happy to share the news that our service recently expanded its service operation in North America. We have now established operations in the United States and Germany, enabling customers in these regions to benefit from improved performance. By connecting to the service nearest to your environment, you can enjoy faster response times and a more seamless experience. The Professional and Standard plans are available in these regions; however, please remember that our future investment will only be in our Professional plan.

How to get started:

Getting started with the new service plan is simple and my previous article can still be a useful reference for you. If you have already tried our free plan, you can upgrade to the professional plan. Alternatively, you can start directly with the Professional plan by creating a new service instance in your subaccount on the SAP BTP Cockpit.

SAP BTP Cockpit Service Instance Creation

What does this mean for existing customers?

Don't worry, you can continue using your existing service plan until you are ready to switch over to the new service instance. This means that you can explore the new service plan at your own pace, without any disruptions to your current setup.

To update the service plan within your subaccount

When you are ready to migrate to the new service plan, the simplest way is to update your existing service instance to the new service plan within your subaccount. You can follow these simple but important steps to update your service plan correctly. The Professional plan is available only in the Cloud Foundry environment. The update within your subaccount will retain your existing configurations and usage history.

SAP BTP Cockpit Service Instance Update

To migrate to another subaccount

If you want to migrate your service to another subaccount, you can create a new service instance in the target subaccount, configure the authentication method, and export any configuration files from your existing subaccount to your new subaccount if there is any custom configuration.

This process may be for your case if you are migrating from the Neo environment to the Cloud Foundry environment or moving to another data center.

To migrate from one subaccount to another, you can refer to our Neo to Cloud Foundry migration documentation. The process is also similar when moving between Cloud Foundry based subaccounts. However, please note that the transaction history data will not be transferred to the new subaccount. If you wish to retain a record of your past activities, you can optionally export the data before migrating.

Data Quality Service UI Configuration Export

Transition Timeline:

We will soon announce the official deprecation date for the existing Standard plan, and it will no longer be available for new customers. This means that new customers seeking to purchase our service plan will need to opt for the new Professional plan under the SAP BTP Enterprise Agreement or Pay-As-You-Go for SAP BTP.

If you are using the service in the Cloud Foundry environment:

Existing customers, however, can continue using the existing service plan until their contractual agreement is renewed. We value our long-standing customers and want to ensure a smooth transition for them. We suggest planning for the transition as soon as possible to take advantage of the new pricing we offer with the new service plan.

If you are using the service in the Neo environment:

Warning! We recommend you transition to the new environment as soon as possible, as the Neo environment will be deprecated in 2028. We will not add any new features to our deployments in the Neo environment. Please find additional details regarding the Neo deprecation in this article.

Conclusion:

We are excited for the launch of our new service plan for SAP Data Quality Management, microservices for location data. The service is available in North America and Europe regions with attractive pricing options under SAP BTP Enterprise Agreement or Pay-As-You-Go for SAP BTP. We believe the new lower-cost service plan with improved performance will provide significant value to our customers.

We encourage our existing customers to explore the benefits of our service plan while we are committed to supporting their existing service operations based on the deprecation schedule outlined in their contractual agreement. If you have any questions or concerns about the transition to the new service plan, please contact our customer support team. Thank you for your continued trust and support.

ABAP CDS Select from dynamic data source

2024-07-08T14:20:49.192000+02:00

Hello friends,

I faced a little problem while reporting in Fund Management (FM) module as the tables are generated automatically while creating derivation rules with dynamic names and differs by system /client.

The rule entries are stored in the generated table "FMFMOAXXXYYYZZZZ", where XYZ symbols means as follows:

XXX is the system specification (e.g., DE1 or QE1)
YYY is the client number (e.g., 120 or 200)

ZZZZ is the number of the derivation rule (which you should have as per your table 31, 51, 64, or 61)

To be able to move my report / solution I had to selecting from dynamic source which is not supported in CDS!

After investing some time I came to try it using AMDP with table function

Now let's explain it as simple as possible.

*Step 1 create a table function *

@EndUserText.label: 'demo table function' 
define table function zcds_amdp_demo_tf with parameters @Environment.systemField:#CLIENT clnt5 : abap.clnt, derivation : char4 
returns { 
MANDT : mandt ; 
SOUR1_FROM : ps_posid ;
SOUR1_TO : ps_posid ;
VALID_FROM : abaintab ;
TARGET1 : fistl;
DELETE_FLG : abadrdelflag;
ADDED_BY : abadrerfasser;
ADDED_ON : erdat; 

} 
implemented by method zcl_amdp_demo=>get_data;

*Step 2 Create the class zcl_amdp_demo*

CLASS zcl_amdp_demo DEFINITION PUBLIC FINAL CREATE PUBLIC.
PUBLIC SECTION.
INTERFACES: if_amdp_marker_hdb, if_oo_adt_classrun.

CLASS-METHODS get_data FOR TABLE FUNCTION zcds_amdp_demo_tf.
PROTECTED SECTION.
PRIVATE SECTION.
ENDCLASS.

CLASS zcl_amdp_demo IMPLEMENTATION. 
  METHOD get_data BY DATABASE FUNCTION FOR HDB LANGUAGE SQLSCRIPT OPTIONS READ-ONLY USING tadir.
    declare lv_fm_tab string; 
    declare tablename string; 
    declare clientnumber string; 
    declare systemid string; 
    declare deriv string;
*must be the same data types as consuming cds also in same order 
declare it_fm_tab table ( 
  MANDT nvarchar(3), 
  SOUR1_FROM nvarchar(24), 
  SOUR1_TO nvarchar(24), 
  VALID_FROM nvarchar(10), 
  TARGET1 nvarchar(16), 
  DELETE_FLG nvarchar(1), 
  ADDED_BY nvarchar(12),
  added_on nvarchar(10) ); 
  
* fmfmoaxxxyyyzzzz 
*fields from parameters 
clientnumber = clnt5; 
select srcsystem into systemid from tadir where pgmid = 'HEAD' and object = 'SYST';

*'COST' => Cost Element = Commitment Item 
*'WBSE' => wbs Element = Funds Center

  if derivation = 'COST' then
    if systemid = 'DS1' then deriv = '0051'; end if; 
    if systemid = 'QS1' then deriv = '0062'; end if; 
    if systemid = 'PS1' then deriv = '0051'; end if; 
  else
    if systemid = 'DS1' then deriv = '0052'; end if;
    if systemid = 'QS1' then deriv = '0064'; end if;
    if systemid = 'PS1' then deriv = '0041'; end if; 
  end if; 
  
  tablename = 'FMFMOA' || systemid || clientnumber || deriv; 
  
  EXECUTE immediate 'select MANDT, SOUR1_FROM, SOUR1_TO, VALID_FROM, TARGET1, DELETE_FLG, ADDED_BY, ADDED_ON from ' || tablename into it_fm_tab ;
  
  RETURN SELECT * FROM :it_fm_tab; ENDMETHOD.
ENDCLASS.

*Step 3 and final Create a sonsumable CDS view*

@AbapCatalog.sqlViewName: 'ZCDS_AMDP_DEMO_V' 
@AbapCatalog.compiler.compareFilter: true 
@AbapCatalog.preserveKey: true 
@AccessControl.authorizationCheck: #NOT_REQUIRED 
@EndUserText.label: 'CDS that consuming the TF' 
define view zcds_amdp_demo as select from zcds_amdp_demo_tf (clnt5 : $session.client, derivation : 'COST' ) 
//COST => Cost Element = Commitment Item 
//WBSE =>WBS Element = Funds Center 
{ 
* 
}

Output:

A Closer look: if you change the parameter from 'COST' to 'WBSE' the return changes which makes it dynamic selection now 🙂 .. congratulations!

Please feel free to comment any observations, enhancements or recommendations.

Regards,
Abdelrahman Zaki

Statement of Direction - SAP Data Services as of July 2024

2024-07-16T23:57:19.052000+02:00

Statement of Direction – SAP Data Services

July 2024

Introduction

As our matured products continues to have consistent traction in the market with emerging technologies and data platforms, it is essential for us to focus on extending its versions to meet the evolving needs of our customers. This Statement of Direction outlines our strategic approach to extending product versions, ensuring that we maintain our competitive edge in the on-premises data integration space as well as on-cloud and deliver value to our customers. Our primary objective is to understand and address the specific requirements of our customers, for the matured products as well as new and emerging technologies in the cloud. We will actively engage with our user base through feedback sessions, support channels, and market research to identify their pain points, challenges, and emerging needs.

Executive Summary

This document provides guidance to our customers of SAP Data Management portfolio, with respect to SAP Data Services (DS) solution, on the product directions. For Information Steward we will have a separate communication made available. This document is an extension of our blog post made on November 3, 2023, focused on SAP Data Services

SAP Data Management portfolio expanded further in the cloud with the introduction of SAP Datasphere in March 2023. SAP Datasphere is a comprehensive data service built only on SAP® Business Technology platform (SAP BTP) that enables every data professional to deliver seamless and scalable access to mission critical business data. It is the next generation of SAP® Data Warehouse Cloud, with new capabilities that enhance data discovery, modeling, and distribution. With a unified experience for data integration, data cataloging, semantic modeling, data warehousing, data federation, and data virtualization. SAP Datasphere enables data professionals to easily distribute authoritative business data with business context and logic preserved across the data landscape.

As we define the future of SAP Data Management portfolio, with our stated data management strategy, current on-premise SAP Data Services customers can in fact leverage all their existing investments and efforts, repurposing those time-tested and trusted artifacts and configurations that are being used today in SAP Data Services (DS), while also extending the reach of their data management to the cloud, with SAP Datasphere, a native cloud solution. Yes, we are committed to our existing SAP Data Services and customer base too. In the current release of SAP Data Services, we do have the connector to SAP Datasphere to consume the raw, semi-processed as well as fully processed data from SAP Data Services connections. This capability enables to reduce the gap of heterogeneous sources and targets between SAP Data Services and SAP Datasphere.

“In 2022 we did a minor version upgrade of SAP Data Services to 4.3, allowing the compatibility with Business Intelligence BusinessObjects 4.3. As a result, our support policy extended the end of mainstream maintenance timeline to end of 2025 and priority 1 support to end of 2027 for both SAP Data Services and SAP Information Steward. We have made a change in Q3 of 2024, wherein the SAP Data Services and SAP Information Steward 4.3 timeline and extended the mainstream support till end of December 2026. Priority 1 support has been replaced with CSM (Customer Specific Maintenance). However, with the mainstream maintenance of version 4.3 of SAP Data Services suite ending in 2026, we do not want to leave our large customer base behind, and therefore we have decided to release an updated version, code named SAP Data Services 2025 to provide more time to our on-premises customers to adopt our rapidly evolving, state-of-the-art cloud data integration scenarios. As more of our customers shift their on-premises workloads to the cloud, we also offer SAP Data Services, private cloud edition (PCE), which runs the ETL and Data Quality solutions in a managed cloud environment operated by SAP. Many of our largest SAP Data Services customers already run both solutions side-by-side, during their transition to the managed cloud deployment.”

SAP Data Services 2025

Our current plan to release a new version of the SAP Data Services in 2025 as a successor of SAP Data Services 4.3 remains unchanged and will be...

code named SAP Data Services 2025.
available on-premises and through managed cloud,
focused on the most widely adopted on-premises scenarios: SAP Data Services for native ETL and Data Quality,

o Existing architecture and component map will be followed including Designer, Repository, Job Server, supported web applications and BI module.

o Compatible with proposed BI 2025 suite and planned to be released a quarter after BI 2025

o We have plans to release SAP Data Services 2025 by the end of Q2 2025 followed by ‘Fix Packages’ as needed.

o We have plans to release a new version every alternate year and include continuous enhancements through patches and fix-packs. This plan is aimed for faster enablement of improved and newly added functionalities.

o We will provide clear use case migration paths/workarounds for the components for the version we plan the end of support beyond 2027.

o Continue to support Windows and Linux platforms only. AIX and Solaris support has already been discontinued in the 4.3 version of SAP Data Services.

o SAP private cloud edition (PCE) will continue to be primary option of availability for SAP Data Services on managed cloud as well as on premise. The option to host on customers choice of private cloud and AWS EC3, Azure VM etc. remains unchanged.

o We do not foresee any impact to solutions and partners using SAP Data Services in an OEM mode.

o We also have plans to discontinue the support of few sources and targets based on their sunset timelines, market demand and popularity score.

We are committed to support you.

We recognize the importance of continuously improving and innovating our product and yet manage a balance between our on-premises and cloud-based solutions. We will invest in research and development to explore new technologies, industry trends, and market demands yet support our existing customer base for the mature products. By fostering a culture of innovation and collaboration, we will ensure that our product versions remain cutting-edge and relevant in the ever-changing business landscape. We are committed to help and support our customers.

Many customers rely on SAP Data Services for critical use cases and will need to do so beyond 2025. Our support plan for SAP Data Services is as follows and our message is consistent. Please refer to the respective GA versions Product Availability Matrix for the support timelines. Please refer to SAP Note 3486924 for description of 'Customer Specific Maintenance'.

Your choice of deployment

With SAP Data Services, private cloud edition (PCE), we offer a predefined package operated by SAP on the main cloud platform providers, including software, support, technical managed services, and infrastructure. This also, opens an opportunity to shift from a maintenance contract to a subscription model while leveraging your existing investments in SAP Data Services solution.

Here are some of the key benefits:

The latest version of SAP Data Services is always available, thanks to the periodical upgrades.
3-tier landscape: development/test, quality assurance and production by default, with optional additional tiers.
Pre-configured hardware landscapes optimized for performance,
Choice of cloud platform providers infrastructure includes AWS, Google Cloud Platform, Microsoft Azure.
End-to-end SLA 99.7%, backup and restore included within standard service scope.

In the future, customers will continue to have the choice for their infrastructure for SAP Data Services solution, either on-premises or in a managed cloud operated by SAP. Additionally, our directions for the on-premises data integration and data governance scope with respect to SAP Data Services and SAP Information Steward remains mutually exclusive and unchanged due to the data integration and data governance scope within SAP’s cloud solutions.

We have plans to stop or reduce the scope of few functionalities.

PeopleTools for PeopleSoft Applications
JD Edwards Application connectivity
SAP Sybase Replication Server
Siebel
SalesForce Adapter
Attunity aka Qlik Data Integration Platform

WE WANT TO HEAR FROM YOU

SAP Data Services remains as a key component of our Data Management portfolio, and we are committed to our customers. Please help us fine tune our roadmaps of SAP Data Services providing your input through the dedicated inbox:

SAP Data Services: sap.dataservices@sap.com

We appreciate your feedback and will answer your questions via our official communication channels.

Unleashing the Power of CDS Views in SAP HANA

2024-08-02T20:16:19.570000+02:00

Introduction to CDS Views in SAP HANA

Core Data Services (CDS) views are a crucial aspect of SAP's data modeling capabilities, allowing for efficient and flexible data management. In this blog post, we'll explore what CDS views are, how they work, and why they are essential for SAP users and developers.

Pre-requisites:SAP HANA Studio or Eclipse with ABAP Development Tools (ADT)

ABAP Backend System
Basic Knowledge of SQL and ABAP

What are CDS Views?

Core Data Services (CDS)

CDS is a framework for defining and consuming data models on SAP HANA, providing a powerful way to manage and retrieve data efficiently. Data models are fundamental in designing databases, ensuring data integrity, and optimizing data retrieval and manipulation. They serve as blueprints for how data is stored, accessed, and utilized across different SAP applications and modules.

Key Features and Benefits of CDS Views

Enhanced Performance: CDS views optimize data access, making it faster and more efficient.
Simplified Data Modelling: With CDS views, you can define complex data models using simple annotations and syntax.
Seamless Integration: CDS views integrate seamlessly with other SAP tools and applications.

Note: In ABAP, during select queries, it works like SELECT – SE11 – DB – SE11 – Query. In CDS, it works like SELECT – CDS – Query. Select options cannot be used in CDS Views; instead, annotations are required.

Components of CDS

CDS Views: These are like database views but more powerful, allowing complex joins, associations, and calculations. They are defined using the CDS language and can be consumed in ABAP programs.
CDS Entities: These are the core building blocks of CDS, representing data models that include tables, views, and associations.
Annotations: Annotations provide metadata about the CDS elements, such as descriptions, semantics, and UI information. They help in enhancing the functionality and integration with other SAP components.
Associations: Associations define relationships between CDS entities, making it easier to navigate and query related data.

Step-by-Step Guide to ABAP CDS View Annotations and Buffering

@AbapCatalog.sqlViewName: 'ZVSQL_SAMPLE_01' // In this ABAP catalog, we can set the name for the DDIC.
@EndUserText.label: 'This is my first DDIC-based CDS' // Here we can set the description for this SQL view.
@AbapCatalog.preserveKey: true // With the help of this, we can set the key field as per our requirement.

@@@@@@@@@@@@@@@@@ If we Add Buffering then @@@@@@@@@@@@@@
@AbapCatalog: {
buffering: {
status: Buffering,
type: #NONE,
numberOfKeyFields: 000
},
dbHints: [{
dbSystem: '', // Here we can add a specific database
hint: ''
}],
ViewEnhancementCategory: [], // Here we can allow or not for CDS enhancement.
sqlViewName: '',
preserveKey: true,
compiler: {
compareFilter: true
}
}

Explanation

@AbapCatalog.sqlViewName: Sets the name for the DDIC.
@EndUserText.label: Sets the description for the SQL view.
@AbapCatalog.preserveKey: Allows setting the key field as per requirements.
Buffering Section:
- status: Buffering status.
- type: Type of buffering (#NONE means no buffering).
- numberOfKeyFields: Number of key fields to be buffered.
dbHints: Database hints section:
- dbSystem: Specifies the database system.
- hint: Provides hints for the database.
ViewEnhancementCategory: Allows or disallows CDS enhancements.
compiler.compareFilter: Compares the filter for the compiler.

Creating CDS Views in SAP

Open the SAP Development Environment: Launch SAP HANA Studio or Eclipse with ABAP Development Tools.
Create a New CDS View: Right-click on your project and select New > Other ABAP Repository Object > Data Definition.
Define the CDS View: Enter the required details, such as the name and description of the CDS view. Define the SQL-like syntax to specify the data model.

@AbapCatalog.sqlViewName: 'ZC_MY_CDS_VIEW'
@AbapCatalog.compiler.compareFilter: true
@AccessControl.authorizationCheck: #NOT_REQUIRED
@EndUserText.label: 'My CDS View'
define view ZC_My_CDS_View as select from SFLIGHT {
    carrid,
    connid,
    fldate,
    price
}

.Activate the CDS View: Once you've defined the view, activate it to make it available for use.
Test the CDS View: Test the CDS view by executing it in the development environment to ensure it retrieves the expected data.

CDS View Entity with Join and Literals

define view ZDDLS_SAMPLE as select from vbak inner join vbap on vbak.vbeln = vbap.vbeln {
    key vbak.vbeln as Sales_Order,
    vbak.vkorg as Sales_org,
    vbak.vkgrp,
    vbap.posar,
    vbap.matnr, 
    0.23 as num_lit,
    'CDS' as char_lit
}
where vbak.vbeln = '0237259939'

CDS View with Input Parameters

define view ZDDLS_SAMPLE with parameters p_vbeln : vbeln as select from vbak inner join vbap on vbak.vbeln = vbap.vbeln {
    key vbak.vbeln as Sales_Order,
    vbak.vkorg as Sales_org,
    vbak.vkgrp,
    vbap.posar,
    vbap.matnr, 
    0.23 as num_lit,
    'CDS' as char_lit
}
where vbak.vbeln = $parameters.p_vbeln

Key Points to Remember

We can give the SQL view name up to 16 characters.
Annotations provide vital metadata and enhance CDS functionality.
CDS views support various join types: inner join, left outer join, right outer join, union, and union all.

Using Session Variables in CDS Views.

In SAP CDS views, session variables such as the current user or date are handled differently compared to traditional ABAP programming. When you need to use session variables like sy-uname (current user) or sy-datum (current date), you utilize the $session prefix in CDS views.

define view ZDDLS_SAMPLE
as select from vbak
inner join vbap on vbak.vbeln = vbap.vbeln
{
    key vbak.vbeln as Sales_Order,
    vbak.vkorg as Sales_org,
    vbak.vkgrp,
    vbap.posar,
    vbap.matnr,
    0.23 as num_lit,
    'CDS' as char_lit,
    $session.user as current_user,
    $session.client as current_client,
    $session.system_language as current_language,
    $session.system_date as current_date
}

Explanation of Session Variables

$session.user: Retrieves the current user's ID.
$session.client: Retrieves the current client number.
$session.system_language: Retrieves the system language.
$session.system_date: Retrieves the current system date.

Make sure you use the $session prefix followed by the appropriate keyword. Pressing Ctrl + Space after $session. will show you the available options, such as user, client, system_language, and system_date.

SAP CDS View String Operations

1. Concatenating Strings

In CDS views, you can concatenate strings using the CONCAT keyword. However, by default, you can only concatenate two fields at a time. To concatenate more than two fields, you need to perform concatenation multiple times.

define view ZCDS_CONCATENATION
as select from vbak
{
    key vbeln,
    erdat,
    ernam,
    concat(concat(vkorg, spart), 'additional_text') as concatenated_field
}

2. Trimming Leading Characters

The LTRIM function removes leading spaces or specific leading characters from a string.

@AbapCatalog.sqlViewName: 'ZCDS_string_oper'
@AbapCatalog.compiler.compareFilter: true
@AbapCatalog.preserveKey: true
@AccessControl.authorizationCheck: #NOT_REQUIRED
@EndUserText.label: 'CDS string operations'
define view ZCDS_STRING_OPERATIONS 
as select from vbak
{
    key ltrim(vbeln, '0') as sales_order_no,
    erdat,
    ernam,
    ltrim(vkorg, '0') as vkorg,
    spart
}

3. Trimming Trailing Characters

The RTRIM function removes trailing spaces or specific trailing characters from a string..

@AbapCatalog.sqlViewName: 'ZCDS_string_oper'
@AbapCatalog.compiler.compareFilter: true
@AbapCatalog.preserveKey: true
@AccessControl.authorizationCheck: #NOT_REQUIRED
@EndUserText.label: 'CDS string operations'
define view ZCDS_STRING_OPERATIONS 
as select from vbak
{
    key ltrim(vbeln, '0') as sales_order_no,
    erdat,
    ernam,
    ltrim(vkorg, '0') as vkorg,
    spart,
    rtrim(vkorg, '0') as vkorg_trimmed
}

4. Combining LTRIM and RTRIM

You can combine both LTRIM and RTRIM functions to clean up a string by removing leading and trailing characters.

@AbapCatalog.sqlViewName: 'ZCDS_string_oper'
@AbapCatalog.compiler.compareFilter: true
@AbapCatalog.preserveKey: true
@AccessControl.authorizationCheck: #NOT_REQUIRED
@EndUserText.label: 'CDS string operations'
define view ZCDS_STRING_OPERATIONS 
as select from vbak
{
    key ltrim(vbeln, '0') as sales_order_no,
    erdat,
    ernam,
    ltrim(vkorg, '0') as vkorg,
    spart,
    rtrim(vkorg, '0') as vkorg_trimmed,
    ltrim(rtrim(vbeln, '0'), '0') as cleaned_vbeln
}

5. Finding Substrings

The INSTR function is used to find the position of a substring within a string.

@AbapCatalog.sqlViewName: 'ZCDS_string_oper'
@AbapCatalog.compiler.compareFilter: true
@AbapCatalog.preserveKey: true
@AccessControl.authorizationCheck: #NOT_REQUIRED
@EndUserText.label: 'CDS string operations'
define view ZCDS_STRING_OPERATIONS 
as select from vbak
{
    key ltrim(vbeln, '0') as sales_order_no,
    erdat,
    ernam,
    ltrim(vkorg, '0') as vkorg,
    spart,
    rtrim(vkorg, '0') as vkorg_trimmed,
    ltrim(rtrim(vbeln, '0'), '0') as cleaned_vbeln,
    instr('Avinash Vasnat Borate', 'Borate') as position_of_substring
}

6. Extracting Left Substrings

The LEFT function extracts a specified number of characters from the beginning of a string.

@AbapCatalog.sqlViewName: 'ZCDS_string_oper'
@AbapCatalog.compiler.compareFilter: true
@AbapCatalog.preserveKey: true
@AccessControl.authorizationCheck: #NOT_REQUIRED
@EndUserText.label: 'CDS string operations'
define view ZCDS_STRING_OPERATIONS 
as select from vbak
{
    key ltrim(vbeln, '0') as sales_order_no,
    erdat,
    ernam,
    ltrim(vkorg, '0') as vkorg,
    spart,
    rtrim(vkorg, '0') as vkorg_trimmed,
    ltrim(rtrim(vbeln, '0'), '0') as cleaned_vbeln,
    instr('Avinash Vasnat Borate', 'Borate') as position_of_substring,
    left('SAP ABAP HANA', 3) as left_substring
}

7. Extracting Right Substrings

The RIGHT function extracts a specified number of characters from the end of a string.

@AbapCatalog.sqlViewName: 'ZCDS_string_oper'
@AbapCatalog.compiler.compareFilter: true
@AbapCatalog.preserveKey: true
@AccessControl.authorizationCheck: #NOT_REQUIRED
@EndUserText.label: 'CDS string operations'
define view ZCDS_STRING_OPERATIONS 
as select from vbak
{
    key ltrim(vbeln, '0') as sales_order_no,
    erdat,
    ernam,
    ltrim(vkorg, '0') as vkorg,
    spart,
    rtrim(vkorg, '0') as vkorg_trimmed,
    ltrim(rtrim(vbeln, '0'), '0') as cleaned_vbeln,
    instr('Avinash Vasnat Borate', 'Borate') as position_of_substring,
    left('SAP ABAP HANA', 3) as left_substring,
    right('SAP ABAP S4 HANA', 4) as right_substring
}

8. Padding Strings

The RPAD (Right Pad) function pads a string on the right side with a specified character until the string reaches a desired length.

@AbapCatalog.sqlViewName: 'ZCDS_string_oper'
@AbapCatalog.compiler.compareFilter: true
@AbapCatalog.preserveKey: true
@AccessControl.authorizationCheck: #NOT_REQUIRED
@EndUserText.label: 'CDS string operations'
define view ZCDS_STRING_OPERATIONS 
as select from vbak
{
    key ltrim(vbeln, '0') as sales_order_no,
    erdat,
    ernam,
    ltrim(vkorg, '0') as vkorg,
    spart,
    rtrim(vkorg, '0') as vkorg_trimmed,
    ltrim(rtrim(vbeln, '0'), '0') as cleaned_vbeln,
    instr('Avinash Vasnat Borate', 'Borate') as position_of_substring,
    left('SAP ABAP HANA', 3) as left_substring,
    right('SAP ABAP S4 HANA', 4) as right_substring,
    rpad('EWM team', 6, 'XY') as padded_string
}

9. Replacing Substrings

The REPLACE function replaces all occurrences of a specified substring within a string with another substring.

@AbapCatalog.sqlViewName: 'ZCDS_string_oper'
@AbapCatalog.compiler.compareFilter: true
@AbapCatalog.preserveKey: true
@AccessControl.authorizationCheck: #NOT_REQUIRED
@EndUserText.label: 'CDS string operations'
define view ZCDS_STRING_OPERATIONS 
as select from vbak
{
    key ltrim(vbeln, '0') as sales_order_no,
    erdat,
    ernam,
    ltrim(vkorg, '0') as vkorg,
    spart,
    rtrim(vkorg, '0') as vkorg_trimmed,
    ltrim(rtrim(vbeln, '0'), '0') as cleaned_vbeln,
    instr('Avinash Vasnat Borate', 'Borate') as position_of_substring,
    left('SAP ABAP HANA', 3) as left_substring,
    right('SAP ABAP S4 HANA', 4) as right_substring,
    rpad('EWM team', 6, 'XY') as padded_string,
    replace('Avinash Borate', 'Avinash', 'Vicky') as updated_name
}

10. Extracting Substrings

The SUBSTRING function extracts a specific portion of a string, starting from a given position and optionally for a specified length.

@AbapCatalog.sqlViewName: 'ZCDS_string_oper'
@AbapCatalog.compiler.compareFilter: true
@AbapCatalog.preserveKey: true
@AccessControl.authorizationCheck: #NOT_REQUIRED
@EndUserText.label: 'CDS string operations'
define view ZCDS_STRING_OPERATIONS 
as select from vbak
{
    key ltrim(vbeln, '0') as sales_order_no,
    erdat,
    ernam,
    ltrim(vkorg, '0') as vkorg,
    spart,
    rtrim(vkorg, '0') as vkorg_trimmed,
    ltrim(rtrim(vbeln, '0'), '0') as cleaned_vbeln,
    instr('Avinash Vasnat Borate', 'Borate') as position_of_substring,
    left('SAP ABAP HANA', 3) as left_substring,
    right('SAP ABAP S4 HANA', 4) as right_substring,
    rpad('EWM team', 6, 'XY') as padded_string,
    replace('Avinash Borate', 'Avinash', 'Vicky') as updated_name,
    substring(erdat, 5, 2) as month,
    substring(erdat, 1, 4) as year,
    substring(erdat, 7, 2) as date
}

Calling RFC in BODS job (Data Encryption in BODS)

2024-08-07T15:44:10.092000+02:00

Scenario was to call the RFC which is used to create a custom key generation for encryption of data and use the custom key in BODS for certain fields for data encryption.

Steps to be done before using RFC in BODS,

Create an RFC in given system with ABAP code.
Import the RFC in Data services repo, which is datastores in data services.

After import of RFC,

To use the RFC in BODS , we require to use Row_Generation Transformation. This will act as a dummy input, After Row_Generation use Query Transformation in which RFC call can be made using normal function call in Query_encryption transformation as below,

Here, pass the required input parameters and get the output parameter, as shown below.

Then after Query transformation for encryption, map the input data and the generated key transformation to new Query Transformation as below,

Encrypt_aes is the standard Function available in BODS which accepts input as plain text, passphrase and AES key length.

In this way above custom logic for encryption can be utilized in BODS for encryption of data.

Refer Complete BODS job flow,

Here /BIC**291 is the table data from BW system and in Query_2 Transformation, key and data is available to use.

After execution output will look like below,

Statement of Direction – SAP Information Steward

2024-08-16T16:29:03.565000+02:00

August 2024

Introduction

As our matured products continue to have consistent traction in the market with emerging technologies and data platforms, it is essential for us to focus on extending its versions to meet the evolving needs of our customers. This Statement of Direction outlines our strategic approach to extending product versions, ensuring that we maintain our competitive edge in the on-premises data stewardship, data quality, metadata, and glossary space as well as on-cloud and deliver value to our customers. Our primary objective is to understand and address the specific requirements of our customers, for the matured products as well as new and emerging technologies in the cloud. We will actively engage with our user base through feedback sessions, support channels, and market research to identify their pain points, challenges, and emerging needs.

Executive Summary

This document provides guidance to our customers of SAP Data Management portfolio, with respect to SAP Information Steward (IS) solution, on the product directions. For Data Services (DS) we have a separate blog communication made available. This document is an updated version of our blog post made on November 3, 2023, which focused on SAP Information Steward and SAP Data Services.

SAP Data Management portfolio expanded further in the cloud with the introduction of SAP Datasphere in March 2023. SAP Datasphere is a comprehensive data service built only on SAP® Business Technology platform (SAP BTP) that enables every data professional to deliver seamless and scalable access to mission critical business data. It is the next generation of SAP® Data Warehouse Cloud, with new capabilities that enhance data discovery, data catalog, metadata, modeling, and distribution. With a unified experience for data cataloging, semantic modeling, data integration, data warehousing, data federation, and data virtualization. SAP Datasphere enables data professionals to easily distribute authoritative business data with business context and logic preserved across the data landscape.

As we define the future of SAP Data Management portfolio, with our stated data management strategy, current on-premise SAP Information Steward customers can leverage their existing investments and efforts, repurposing those time-tested and trusted artifacts and configurations that are being used today in SAP Information Steward (IS), while also extending the reach of their data management to the cloud, with SAP Datasphere, a native cloud solution. We are committed to our existing SAP Information Steward and customer base too.

In 2022 we did a minor version upgrade of SAP Information Steward to 4.3, allowing the compatibility with Business Intelligence BusinessObjects 4.3. As a result, our support policy extended the end of mainstream maintenance timeline to end of 2026 with Customer Specific Maintenance (CSM) to end of 2028 for both SAP Information Steward and SAP Data Services 4.3. However, with the mainstream maintenance of version 4.3 of SAP Information Steward ending in 2026, we do not want to leave our large customer base behind, and therefore we have decided to release an updated version, code named SAP Information Steward 2025 to provide more time to our on-premises customers to adopt our rapidly evolving, state-of-the-art cloud data quality and data governance scenarios. Many of our largest SAP Information Steward customers already run both solutions side-by-side, during their transition to the managed cloud deployment.

SAP Information Steward 2025

Our current plan to release a new version of the SAP Information Steward in 2025 as a successor of SAP Information Steward 4.3 remains unchanged and will be...

Code named SAP Information Steward 2025
Available on-premises
Focused on the most widely adopted on-premises scenarios: SAP Information Steward for data stewardship, metadata, glossary, and data quality
Compatible with proposed BI 2025 suite and planned to be released a quarter after BI 2025
We have plans to release the first version by the end of Q3 2025 followed by ‘Fix Packages’ as needed.
We have plans to release a new version every alternate year and reduce the frequency of SAP Information Steward, (SP) Service Pack releases. The ‘Service Pack’ release has been planned to be deprecated.
Continue to support Windows and Linux platforms only.
We do not foresee any impact to solutions and partners using SAP Information Steward in an OEM mode.

We are committed to support you.

Many customers rely on SAP Information Steward for critical use cases and will need to do so beyond 2025. Our support plan for SAP Information Steward is as follows and our message is consistent. Please refer to the respective GA versions Product Availability Matrix for the support timelines.

WE WANT TO HEAR FROM YOU

SAP Information Steward remains as a key component of our Data Management portfolio, and we are committed to our customers. Please help us fine tune our roadmaps of SAP Information Steward providing your input through the dedicated inbox:

SAP Information Steward: sap.informationsteward.enhance@sap.com

We appreciate your feedback and will answer your questions via our official communication channels.

DataServices Salesforce Adapter retirement - a free alternative solution within BODS

2024-12-24T16:04:08.405000+01:00

If you are here, you may already know the issue, but for the rest:

The problem: The current out-of-the-box way of connecting to Saleforce with DataServices won’t work after Salesforce retires its REST API v21, in the summer of '25.

Regarding the end of Salesforce compatibility SAP is citing legal restrictions in the related SAP Note:

3004914 - SAP Data Services and Salesforce.com compatibility

Cause:

The Adapter for SFDC as a part of the Data Services using is indeed on SFDC API version 21 only. Due to
the legal restrictions, SAP applications do not connect to SFDC directly or provide direct interface into
SFDC systems. Thus SAP Data Services cannot connect directly to SFDC as well. The supported version
21 is prior to the restrictions and thus it is being provided in an ‘as is’ state and no further enhancements
will be provided.

SAP proposes two possible solutions:

with BODS: purchase a 3rd party adapter instead of the current one: Simba Salesforce Adapter for SAP Data Services
leave BODS and resolve the Salesforce integration with SAP Data Intelligence Cloud.

In this post I describe a solution which is free* and is within BODS. If you have other data sources besides Salesforce and have a large invesment in BODS; you may prefer to stick with it. With the below solution you can do it without additional cost.

The key building blocks will be:

user defined transform + python + simple-salesforce + Salesfroce Bulk API 2.0 (+ CSV)

User defined transform (UDT)

In a nutshell, User Defined Transform is the way to extend your BODS out-of-the-box capabilities with whatever you want. Well… as long as it fits into 255 varchar columns. This is quite a limitation in 2024, so the work-around we will apply is to dump our Salesforce resultset into a local CSV. It is less elegant, but it works with longer fields as well.

Simple-salesforce

Simple-salesforce is a Python library for... you guessed it, Salesforce. You may use the Salesforce API directly, but I found this library quite ~~simple~~ useful. The latest version supports Bulk API 2.0 as well. To be able to use it within BODS, we would need to import it to our environment as follows:

Adding simple_salesforce to local python:
Add to the Environment variables:

E:\SAP BusinessObjects\Data Services\DataQuality\python
E:\SAP BusinessObjects\Data Services\DataQuality\python\Scripts
E:\SAP BusinessObjects\Data Services\bin

Download and install pip

Documentation: https://pip.pypa.io/en/stable/installation/#get-pip-py

1. Download the script from https://bootstrap.pypa.io/get-pip.py

2. Command prompt: python get-pip.py

Install simple_salesforce:

python -m pip install --target="E:\SAP BusinessObjects\Data Services\DataQuality\python\Lib\site-packages" simple_salesforce

Python code of the User Defined Transform

from simple_salesforce import Salesforce, SalesforceLogin, SFType

# set salesforce connection details
username = 'aaa'       # !!!
password = 'bbb'       # !!!
security_token = 'ccc' # !!!
domain = 'xxx'         # !!!
sf_version = '59.0'

# login and get session
session_id, instance = SalesforceLogin(username=username,
password=password, security_token=security_token, sf_version=sf_version, domain=domain)
sf = Salesforce(instance=instance, session_id=session_id)

queryName = record.GetField(u'i_queryName')
querySOQL = record.GetField(u'i_querySOQL')

# execute query
results = sf.bulk2.Account.query(
query=querySOQL, max_records=None, column_delimiter='PIPE', line_ending='CRLF'
)

# write results to local drive as CSV -- !!!customize to your local folder!!!
for i, data in enumerate(results):
  with open(f"F:/FileDatasource/results/{queryName}-part-{i}.csv", "w", encoding="utf-8") as bos:
  bos.write(data)

o_result = queryName + ' loaded successfully'
record.SetField(u'o_result', o_result)

del queryName
del querySOQL
del o_result

A few notes regarding the above code:

sf.bulk2.Account.query – Here Account doesn’t seem to matter at all. I have tried with sf.bulk2.blalba.query – and it worked the same way. What seem to matter is the SOQL query. Consequently, you can solve different Salesforce table loads with the same single UDT.
I used PIPE as column delimiter, and CRLF as line ending for the CSV. These are not the default settings for CSV-s in BODS. Keep it in mind when defining your CSV source. I used these becuase these are less frequently used characters in Salesforce content. It did the trick for me, but keep in mind that this is still not a bulletproof solution. You may want to check for these characters in your content before dumping as CSV.
Don't forget to customise the UDT to your environment:
- username
- password
- token
- domain
- CSV local folder
Unfortunately I have experienced that BODS crashed with ntdll.dll error, after closing the Python editor when opening from the Repository\Transforms (ie modifying the main object) BUT less so when modifying an instance of the same UDT object inside a dataflow. The trace I have found in the Windows event log points to: C:\Windows\SYSTEM32\ntdll.dll
A related SAP Note on the topic: 3485456 - Designer crashes frequently without error message -
SAP BusinessObjects Data Services
Eventually, my recommendation is to develop Python code outside of BODS and later import the UDT from XML or ATL.
Unicode Getfield crash: 1256795 - User Defined Transform with Python code is crashing - Data
Services

Attached you can find the export of the User Defined Transform.

Summary

With the above UDT you can easily build dataflows to extract data from Salesforce:

You may start with a Row Generator.
with a Query transform you may set the two incoming parameters:
1. a query name - just to identify your query
2. the query itself
The next step is the UDT itslelf: you need to connect the inputs, and the output back to the dataflow. This transform should produce a single column with '[query] loaded successfully' after the results were written as CSV to the local drive.
With a second dataflow you would need to process the CSV

*If you find the above solution useful please leave a comment with your country / city. I would like to create a map with pins - hopefully - around the globe.

Archiving: The forgotten hero of S4HANA. The Tool Guide You Want to Have.

2025-02-01T17:16:04.162000+01:00

Forget robots just showing up – they're now creating and multiplying data robots! Deepseek's arrival is our AI Sputnik moment, a surprise launch into a whole new era. Could Deepseek be our "set-it-and-forget-it" solution for S/4HANA archiving in future ? Imagine: automatic data archiving that slashes costs on cleanup and keeps your S/4HANA projects running lean and mean.

💥Could there be SAP Data's Sputnik Moment? 💥

"Stop fearing your SAP data swamp. What if Deepseek could be the dynamite that blasts SAP Data open, cleaning and mopping the way for a data superhighway? Turn data nightmares into data sweet deep seek dreams?."

While dreams are exciting, let's get down to business. This blog dives into a comparison of different SAP archiving tools, offering a clear perspective on their strengths and weaknesses. Keep in mind, these are my personal insights, independent of any company affiliation. Ultimately, the best tool for your needs requires your careful evaluation.

Background

For example, DeepSeek’scould analyze SAP data logs and suggest which files should be archived based on usage patterns, while DeepSeek’s AI-powered search could instantly pull up archived records using context rather than keywords.. SAP archiving tools like SNP Archiving, OpenText, SAP ILM, and Proceed Group are playing crucial roles in optimizing storage, improving compliance, and maintaining SAP system performance. Let’s break down how these tools stack up against each other based on real-world use cases.

Feeling the weight of years of data slowing down your SAP system? You're not alone. Many businesses struggle with bloated databases, sluggish performance, and the ever-present fear of non-compliance.

But with so many archiving options available, choosing the right one can feel overwhelming. That's where this guide comes in. We'll break down the top contenders – SNP Archiving, Proceed Group, OpenText, SAP ILM, Data Lakes, and In-System Archiving – so you can find the perfect fit for your needs.

Understanding Your Archiving Needs

Before diving into the tools, let's clarify your goals. Ask yourself:

What's slowing us down? Is it slow report generation, long system backups, or something else?

What kind of data are we dealing with? Is it mostly structured data within SAP, or do we have lots of documents, emails, and other unstructured data?

How important is compliance? Are we in a heavily regulated industry with strict data retention requirements?

What's our budget? Archiving solutions range from affordable to enterprise-grade.

Comparing the Contenders

Now, let's meet the archiving all-stars:

1. SNP Archiving: The Streamlined Solution

Overview: SNP is a well-known SAP partner specializing in data management, system migration, and archiving. SNP’s archiving solution is specifically designed to optimize SAP system performance while ensuring compliance with data retention requirements.

Key Features:

SAP-Optimized: SNP focuses on delivering archiving solutions tailored to SAP systems, helping businesses remove old data from live environments and store it in a way that it remains accessible for compliance or auditing.

Automated Archiving: The solution automates the archiving process, making it easier to migrate data from SAP systems to storage systems.

Integration with SAP: Seamless integration with SAP S/4HANA, SAP ECC, and other SAP modules, ensuring minimal disruption to business operations.

Pros:

Designed specifically for SAP environments.

Strong focus on compliance and audit-ability.

Easy integration with other SAP modules.

Cons:

Limited flexibility for non-SAP data.

May not be as versatile for broader, cross-platform archiving needs.

2. OpenText: The Content Management Powerhouse

Overview: OpenText is one of the leaders in enterprise content management (ECM) and offers comprehensive archiving solutions for SAP users. OpenText’s solutions go beyond simple archiving, incorporating information lifecycle management, document management, and workflow optimization.

Key Features:

Enterprise Content Management (ECM): OpenText provides a broad suite of ECM tools, including document management, records management, and data archiving.

SAP Integration: Seamlessly integrates with SAP applications (such as SAP S/4HANA), helping businesses manage their SAP documents and records efficiently.

Scalability: OpenText’s archiving solution can scale with enterprise growth, offering flexible data storage options for high volumes of information.

Compliance and Security: Strong features around compliance, audit trails, and security, ensuring that all archived data meets regulatory requirements.

Pros:

Rich ECM capabilities beyond simple data archiving.

Extensive support for compliance and security.

Strong SAP integration.

Cons:

Might be overkill for organizations seeking a simple SAP-only archiving solution.

Higher cost due to comprehensive feature set.

3. Proceed Group: Tailored Solutions

Overview: Proceed Group offers archiving solutions that are integrated into SAP environments and focus on optimizing data storage and retrieval, as well as improving overall system performance.

Key Features:

End-to-End Integration: Proceed Group’s solution integrates directly into SAP and enhances the performance of SAP systems by archiving obsolete or redundant data while ensuring the system stays efficient.

Data Segmentation: Allows businesses to archive data in segments, making it easier to manage large datasets while retaining the ability to retrieve specific pieces of information quickly.

Cost-Efficiency: Designed to reduce operational costs by offloading data from expensive, high-performance systems to lower-cost storage solutions.

Pros:

Effective for businesses looking for cost-effective SAP archiving solutions.

Simple integration with SAP landscapes.

Focus on maintaining system performance and accessibility.

Cons:

May not offer as advanced features as other specialized archiving solutions.

4. SAP ILM (Information Lifecycle Management): The Native Solution

Overview: SAP ILM is an SAP-native solution that offers an enterprise-grade approach to managing data throughout its lifecycle. It’s specifically designed to help businesses comply with legal data retention requirements while managing the storage of SAP data.

Key Features:

Data Lifecycle Management: SAP ILM helps businesses manage the entire lifecycle of data, from creation through retention to deletion, ensuring compliance with legal and regulatory requirements.

Archiving and Data Destruction: Features archiving as well as the automated destruction of data when it is no longer required by regulations.

Seamless SAP Integration: ILM integrates natively with SAP systems, including SAP HANA and SAP S/4HANA, ensuring smooth operation without the need for external systems.

Retention Management: Customizable retention policies allow businesses to define how long data should be kept, making it easier to comply with both internal and external requirements.

Pros:

Native integration with SAP systems.

Comprehensive lifecycle management of data.

Strong regulatory compliance features.

Cons:

Focused mainly on SAP environments; not as flexible for non-SAP data.

The learning curve can be steep for users unfamiliar with SAP.

5. Data Lakes: The Data Insight Reservoir

Overview: A Data Lake is an evolving concept in data storage, where data is stored in its raw form in a large, centralized repository. Although it’s not an archiving solution in the traditional sense, data lakes are increasingly being used for long-term storage and analysis of big data.

Key Features:

Storage of Raw Data: Data lakes store data in its native format (structured, semi-structured, or unstructured), which means all data is available for future analysis without needing to be processed first.

Scalability: Data lakes are built to scale horizontally, handling vast amounts of data with low-cost storage solutions, such as Hadoop or cloud-based services like AWS S3.

Data Exploration: Provides businesses with the ability to run advanced analytics and machine learning algorithms on large datasets without the need for complex data transformation processes.

Flexibility: Can store data from multiple sources, including SAP and non-SAP systems, and provide analytics and insights.

Pros:

Ideal for handling large-scale structured, unstructured, or semi-structured data.

Highly flexible and scalable.

Can be used for data exploration and advanced analytics.

Cons:

Not designed specifically for archiving; lacks out-of-the-box features for compliance or regulatory management.

Requires specialized skills for managing and querying large datasets.

Potentially high storage and data management costs without proper governance.

6. In-System Archiving: The Lightweight Option

Overview: In-System Archiving stores archived data within the SAP system itself (database or storage infrastructure), simplifying management in SAP-centric environments.

Key Features:

Data Stored in SAP

Optimized for SAP Performance

Transparency

Simplified Data Management

Pros:

Seamless SAP Integration

Lower Costs for Smaller Systems

Compliance and Security

Simplicity

Cons:

Scalability limitations for large data volumes

Limited flexibility for non-SAP environments

Less advanced analytics capabilities

The optimal choice for your organisation will depend on a variety of factors, including the size of your company, the volume of data you need to archive, your compliance requirements, and the complexity of your IT environment and hope this guide is useful for you as a starting point.

HTTP Status 500-Internal Server Error in SAP BusinessObjects Intelligence Platform and Data Services

2025-03-14T20:53:06.584000+01:00

Issue: HTTP Status 500-Internal Server Error
We have often seen this error in the SAP BI platform or Data Services, which rely on Tomcat services. This issue is observed when you try to access the application via the central management console. There might be a scenario where a few of the options work fine and a few give the below error message.

Even though sufficient memory is available on the VM to be utilized, this issue is still encountered. A quick resolution is to restart the Tomcat, which is not a sustainable solution.

Cause: The main reason for this issue is the heap memory utilization of Tomcat.

Tomcat is an essential component in SAP BusinessObjects (BO) architecture, particularly for hosting web applications.

Role of Tomcat in SAP BusinessObjects (BO)
Apache Tomcat acts as the web application server for deploying SAP BO web applications such as:

Central Management Console (CMC) — for managing and configuring the SAP BO platform.
BI Launch Pad — for accessing reports, dashboards, and analytics.
Web Intelligence (WebI) — for interactive report creation.
Crystal Reports Viewer — for viewing detailed Crystal Reports.

Inefficient configuration of memory leads to crashes or out-of-memory errors.

It is a bit challenging to figure out the optimum configuration, hence, the issue of monitoring the heap memory utilization becomes critical.

A tool "jconsole" is offered by SAPJVM, which is available in the Business Objects executable.

Steps to set up the console:

Set the below parameters in Tomcat configuration

-Dcom.sun.management.jmxremote
-Dcom.sun.management.jmxremote.port=8008
-Dcom.sun.management.jmxremote.authenticate=false
-Dcom.sun.management.jmxremote.ssl=false

Restart the Tomcat for it to take effect.

Running the console

Browse to the location of sapjvm, it is in the BI installation location - <install_location>\SAP BusinessObjects Enterprise XI 4.0\win64_x64\sapjvm\bin
Execute the jconsole - jconsole.exe <hostname>:8008 (Here, 8008 is the port which is set in property)

Using the various tabs, real-time monitoring can be done, and it will lead to setting up the optimum memory parameters for the system to run smoothly without any issues. An ad-hoc garbage collection can also be performed from here.

Conclusion: It is a great offering within the SAPJVM which empowers us as admins to monitor the Tomcat heap memory and fine-tune the parameters for optimum performance.

How to Extract Long Text from ECC and Prepare a DMC Template for Loading Long Text into S4/HANA

2025-03-18T17:06:53.690000+01:00

Introduction

Extracting long text from SAP systems may seem straightforward, but it often involves complexities that go beyond simple extraction methods. When traditional approaches fall short, leveraging Remote Function Calls (RFCs) can provide an effective solution for retrieving and integrating long text into SAP Business Objects Data Services (BODS) workflows.

In this blog, we’ll explore how to use the RFC_READ_TEXT function module to efficiently extract long text from SAP systems and seamlessly incorporate it into BODS for further processing and analysis. Whether you're dealing with technical challenges or looking for a practical guide to streamline your data integration, this bog will help you leverage RFCs effectively to ensure a smoother and efficient workflow.

Key Functionalities of RFC_READ_TEXT

- Retrieval of long text data stored in SAP tables.
- Filtering of text data based on object type and other parameters.
- Returning retrieved text for further processing or display.

Understanding the Output Structure

The output of RFC_READ_TEXT is structured in a tabular format with the following key fields:

TDLINE: Contains the actual text lines retrieved from SAP.
TDOBJECT: Indicates the object type (e.g., 'MAKT' for material descriptions, 'VBAK' for sales order headers, etc.)
TDID: Identifies the specific object within the given object type.
TDNAME: Specifies the type of long text (e.g., 'LTXT' for general long text, 'ATXT' for header text).
TDSPRAS: Specifies the language of the retrieved text.

Steps to Implement RFC_READ_TEXT in SAP BODS

- Create the structure based on the number of unique IDs available.
- Group the dataset under a unique value to optimize processing.

Fig 1: Creating the extraction structure and splitting data based on multiple IDs.

Fig_2: Standard query inputs that need to be maintained.

Fig_3: Standard schema structure.

Fig_4: Query read text input and RFC calling in BODS program.

Fig_5: Input parameters for calling RFC_READ_TEXT.

Fig_6: Output parameters of RFC_READ_TEXT.

Fig_7: Unnesting the query.

OUTPUT

The RFC_READ_TEXT function returns a table containing long text along with key metadata fields. Here’s a breakdown of the main output fields:

TDLINE: Holds the actual text lines, with each entry representing a separate line.
TDOBJECT: Specifies the object type associated with retrieved text (e.g., MAKT for materials, VBAK for sales orders).
TDID: Uniquely identifies the text object (e.g., material number, sales order number).
TDNAME: Defines the type of long text (LTXT for general long text, ATXT for header text).
TDSPRAS: Indicates the language of the retrieved text.

In essence, the RFC_READ_TEXT output provides the requested long text data along with essential metadata like the object type, object ID, text type, and language. This output structure is crucial for correctly interpreting and utilizing the retrieved long text data within your application.

Fig_8: Output Data

Concatenating Rows in SAP BODS: A Custom Function Approach

When working with SAP Business Objects Data Services (BODS), a common challenge is combining multiple rows into a single row. While BODS provides a built-in reverse pivot transform for this purpose, it requires a predefined number of rows to concatenate. This constraint becomes a hurdle when the row count is unknown or variable.

Let’s explore a more flexible and efficient solution to this problem by leveraging custom functions in BODS. We’ll also discuss the limitations of traditional methods, the performance implications of database-level XML functions, and how a custom function can provide a more robust and scalable solution.

The Problem: Concatenating Rows with Unknown Counts

In many data integration scenarios, multiple rows need to be concatenated into a single row based on a unique identifier (UNIQ_ID). The built-in reverse pivot transform in BODS works only if the number of rows is fixed, making it unsuitable for dynamic datasets.

Traditional Workarounds and Their Limitations

Some common alternatives include database-level SQL functions, such as XMLAGG and XMLELEMENT. These functions can dynamically aggregate multiple rows into a single XML string, which can then be parsed and processed. While this method works, it has significant drawbacks:

Performance Overhead: XML manipulation at database level can be resource-intensive, especially when dealing with large datasets, slowing down job execution and increasing memory consumption.
Scalability Issues: Large datasets (millions of records) can lead to database bottlenecks, making XML functions impractical for high-volume data processing.

A more efficient approach is to create a custom function within BODS to handle row concatenation dynamically. This offers several benefits:

Flexibility: No need to predefine the number of rows to be concatenated.
Improved Performance: Potentially faster execution than database-level XML functions, especially for large datasets.
Reduced Memory Usage: Optimized memory management for better performance.

Solution:

A BODS Custom Function allows concatenation of multiple rows into a single row, even when the number of rows is undefined.

Advantages of Using a Custom Function:

No need to predefine the number of rows.
Enhanced job performance.
Reduced memory usage for cache.
Simple and straightforward implementation.

Input Data:

The source table consists of four columns: UNIQ_ID, EBELN, TDID, and GEN_ROW_NO. The source data is structured as follows:

Pre-Requisites:

Define parameters, local and global variables.
Generate a sequence number for each UNIQ_ID.
Group the dataset based on a unique value.

Output Data:

Data Loading into S4/HANA

Once the extracted and transformed data is generated, it is stored in a structured file format at the designated file path. Since DMC is used as the loading mechanism, the file must strictly adhere to the DMC format. The generated content should be copied and pasted into the corresponding views within the DMC template.

Once the file is created, it can be loaded into the system using the Data Migration Cockpit, after which the data can be reviewed in the S4HANA using transaction code ME23N.

Our references: Credits : Sami Mohammed

Ensure Smooth Integration: Your Quick Guide to SAP Data Services Compatibility ✅

2025-07-17T20:46:58.318000+02:00

📢Installation & Upgrade Guide for SAP Data Services 🛠️

💡When installing or upgrading, compatibility questions are bound to arise since we deal with different databases, drivers, applications and other components.

Compatibility Checklist

✅ Ensuring a Certified Environment To confirm compatibility with your target (new) version of SAP Data Services, verify the following:

🔍Information platform services (IPS) / Business Intelligence Platform (BIP)
🔍 Operating System
🔍 Database (Repository / Source & Target)
🔍 Drivers
🔍 Browser
🔍 Cloud Platforms
🔍 Cloud Storage
🔍 Message Client Library
🔍 Etc.

Understanding the Compatibility Legend

🌟Make sure you understand its legend, as it is crucial to proceed:

For example: If item y was certified in version 4.x SPx Patch x, it will be considered as supported/compatible for that version and above until the version being mentioned as deprecated in the PAM in the future.

🖥Designer Client Tool

Application client tool, named as Designer, needs to be under the same version, support pack and Patch of SAP Data Services Job Server to avoid any compatibility within the Job Server and CMS (component from IPS/BI application).

Because of this need for harmony, the Designer tool can't share a machine with another BI/IPS installation, even if you're not using that CMS connection for SAP Data Services. But don't worry! If you need IPS/BI installed, just make sure it's compatible according to our compatibility matrix.

🔗 3322026 - Can BOE/BI/IPS Platform Client Tools and Designer Client Tool coexist in the same machine? - SAP Data Services

📚Useful Resources

Each SAP product has its Product Availability Matrix (PAM) and Maintenance Strategy. For more information on how to access these, check out the KBA below:

🔗 1338845 - How to find Product Availability Matrix (PAM) / Supported Platforms Documentation for SAP BusinessObjects products (BI/IPS, DS, IS)?

Visit the SAP Data Services for detailed information on Installation, Upgrade, and Migration steps. Remember to change the documentation version to your desired SAP Data Services version.

Recommendation

💡 We strongly recommend installing/upgrading to the latest certified version between IPS/BIP and SAP Data Services to avoid any known issues that have been fixed.

If upgrading is not possible, before proceeding with the installation or upgrade, check the Release Notes for that version under the SAP Knowledge Base. There might be known issues that could affect your daily work.
This will help you identify and be prepared to apply any available workaround!

🌟If you have any questions or need assistance, feel free to reach out to SAP Support and search under our Knowledge Base!

📌If you found this helpful or interesting, I would really appreciate a 'Kudo' to let me know! Your feedback means a lot. 😊

Avoiding RFC Timeout Issues During SAP Table Job Extraction in SAP Data Services

2025-08-19T04:29:54.905000+02:00

Avoiding RFC Timeouts: Executing SAP Table Extractions Jobs in Background via RFC Batch

Introduction

Extracting large volumes of data from SAP systems using SAP Data Services (BODS) can be a real challenge — especially when working with standard (non-ABAP) dataflows. One of the most common issues encountered is the dreaded RFC timeout, which occurs when a data extraction takes too long to return results, and the SAP session times out.

In many environments, using ABAP dataflows to offload processing to SAP is restricted due to governance or lack of ABAP resources. So, how can we extract large datasets without ABAP Dataflow, and still avoid timeouts?

In this blog, we explore how executing jobs in RFC batch background mode can help solve this problem.

Understanding RFC Timeouts

An RFC (Remote Function Call) allows SAP Data Services to call SAP functions like RFC_READ_TABLE to extract data. However, when these calls are executed in dialog mode (default), they inherit the standard timeout settings of an interactive SAP user session — typically 600 to 1800 seconds (10–30 minutes).

When a query takes too long — due to high data volume, slow performance, or unfiltered extraction — the RFC session times out, causing the job to fail.

What Is "Execute in Background Job in RFC Batch"?

"Execute in background job in RFC batch" means:

The RFC-connected extraction is not executed in foreground (interactive user session).
Instead, SAP submits the job to the background job scheduler, and the logic is run as a batch job.
This background job is still triggered via RFC but runs asynchronously, bypassing timeout limitations.

Benefits of Executing in RFC Batch Background

Avoids dialog session timeouts
Allows longer-running data extraction jobs
Reduces job failures in SAP Data Services
No need for ABAP code or ABAP dataflows

Steps to Enable Background Job Execution for SAP Table Extraction

Double-click on the SAP Table:
- In SAP Data Services Designer, double-click on the SAP table you want to extract (for example, EKKO) to open its properties.
Open Source Table Editor:
- This opens the Source Table Editor options for that table.
Enable Background Execution:
- In the Source Table Editor, locate and enable the option Execute in Background (Batch) .
Save the Job:
- Save your changes to the dataflow or job.
Run the Job:
- Execute the job. The extraction will now run as a background RFC batch job, preventing timeout errors typical of foreground execution.

How to use Run_Batch_Job web services call?

2025-09-23T18:52:52.889000+02:00

Before proceeding, check the documentation below:

Using SAP Data Services as a SOAP Web Service provider | SAP Help Portal

Let's use SoapUI as a third-party trigger for our job without the need of exposing the Job directly, but before using the Run_Batch_Job operation, it is important to check if our Session Security is enabled from Data Services Management Console (DSMC).

From DSMC, access the Administrator tab, go to "Web Services" and "Web Services Configuration".

If Session Security is enabled, you must perform a Logon operation to get a Session ID to provide to the Run_Batch_Job web service call, also perform the Logout to close that session.
Session Security is enabled by default and we will be using it during this post.

On DSMC, let's select "View WSDL" from Web Services Status tab:

It will open a new browser tab, showing the WSDL xml file:

Copy the URL to use on SoapUI. In my case, this is my URL:

http://localhost:8080/DataServices/servlet/webservices?ver=2.1&wsdlxml

Since I will be testing from SoapUI installed on the same machine as the web tier application, I will be using localhost, but it would be the SERVER NAME / HOST instead of "localhost".

From SoapUI, right-click on Projects and select "New SOAP Project":

Give a name to your project and provide the URL previously collected on DSMC. Click "OK":

It will provide all the Operations available on the WSDL:

We will be using three operations for this test: Logon, Run_Batch_Job and Logout.

Double-click under the request created during the project creation or create a new request, input the CMS information for the connection (same as your CMC login credentials) and execute the Request.

Check: Logon | SAP Help Portal

You might face an error in case you need to use the cms_system port depending of the application version.
Refer to SAP Note: 3412752 - SAP Data Services Logon web service does not take port in the cms_system parameter.

Logon operation will return a Session ID as previously mentioned in this post:

Now, let's execute a request of Run_Batch_Job.
Check: Run_Batch_Job | SAP Help Portal

I will remove some sections (jobParameters and globalVariables) and the serverGroup parameter, since it does not apply for my test and we will reuse the SessionID from our Logon call:

It successfully ran:

From Data Services Management Console (DSMC) > Administrator > Status / Batch > Training_DS (This is your repository section) > Batch Job Status, you will be able to check the execution history:

After this execution, we will perform a Logout call.
Check: Logout | SAP Help Portal

Note: Tests and screenshots taken from SAP Data Services 2025 version.

Every Web Services call must work outside of SAP Data Services, and in this post, your learned how to applied on SoapUI.
For third party schedulers or tools that could be used to trigger our jobs, you must check with its provider what are the required information to proceed.

🌟If you have any questions or need assistance, feel free to reach out to SAP Support and search under our Knowledge Base!

📌If you found this helpful or interesting, I would really appreciate a 'Kudo' to let me know! Your feedback means a lot. 😊

Designer Installation: Step-by-Step Guide for SAP Data Services Client Tool 💻

2025-10-06T15:56:33.291000+02:00

Make sure you download Designer setup with the same version as your Data Services Job Server machine, which consequently makes the Designer certified with the CMS (BI/IPS) version.

In case you have BI/IPS application or client installed on this machine, make sure it is also certified with your Designer application, even if it is not being used as CMS Server:

3322026- Can BOE/BI/IPS Platform Client Tools and Designer Client Tool coexist in the same machine? - SAP Data Services

Download Designer using the steps below:

Software Center - SAP for Me > SUPPORT PACKAGES & PATCHES > By Alphabetical Index (A-Z) > D > SAP DATA SERVICES > SAP DATA SERVICES 4.x > SAP DS CLIENT TOOLS 4.x > Select a compatible version and download it

NOTE: This post is using Designer Client 2025 (14.4.0.165) as screenshots reference.

INSTALLATION STEPS 🚀

After downloading the setup, let's start the installation.

Run the installer as Administrator user to avoid any problems.
Make sure all the prerequisites which are Critical type are on Succeeded status and click on Next.
Read and select the "I accept the License Agreement". Click on Next.
Add the same valid License Key as your Data Services application server and click on Next:

Additional content:
1977090- Is a new license key required when upgrading to Data Services 4.2, 4.3 or 2025?
Confirm the installation path and click on Next.
Confirm the %DS_COMMON_DIR% directory of SAP Data Services Designer and click on Next:
Select any additional language and click on Next:
Select "Skip CMS", since this is the Designer Client tool installation and not the application server and click on Next:
Uncheck the "Data Services Server" box and click on Next:
Select "Skip Configuration", since this is your Designer client machine and the DSConfig.txt files will be under your Job Server machine. Click on Next.
Start the installation. Click Next:
Installation is complete. Click on Finish.
Reboot machine is required. Only click on "Yes" after confirming that you can restart your machine right now.
Your Designer should be accessible:

Additional content:

"The base keycode for the Data Services platform was not found or has expired. Obtain a valid keycode and use the License Manager to update your installation. (BODI-1116106)" after Designer installation / upgrade:

Check the KBA below:
1395080- How to install or maintain License/Product keys - SAP Data Services
For SAP Data Services 2025, missing ncs.dll:
3641815 - Designer fails to launch with keycode error - SAP Data Services 2025
For SDK Failure when accessing Designer tool:
It might be compatibility between your Designer tool and the CMS Server (BI/IPS), but also bi-directionally communication between Designer <-> CMS Server.
Make sure you can ping and telnet from Designer to the CMS Server and vice versa (CMS Server to Designer).
In case it is compatible and telnet/ping works bi-directionally, check and apply the KBA below:

2450650 - Logon failed to the specified system. Check system name, port number, user name, password, authentication method, and try again. Additional information: <Session: SDK failure>. - SAP Data Services
Designer client is slow:
3034996- SAP Data Services Designer slow performance - SAP Data Services

🌟If you have any questions or need assistance, feel free to reach out to SAP Support and search under our Knowledge Base!

📌If you found this helpful or interesting, I would really appreciate a 'Kudo' to let me know! Your feedback means a lot. 😊

Can SAP’s Real-Time Analytics Redefine Decision-Making for the C-Suite?

2025-10-07T11:34:00.294000+02:00

The Changing Role of Leaders

Today’s CEOs and CTOs face more data than ever before. Numbers flow in from finance, supply chain, and customers. The challenge is not access. The challenge is speed. Leaders cannot wait days for reports. They need answers now.

Why Real-Time Matters

Traditional reporting works on past data. By the time insights reach the boardroom, the market may have moved. Real-time analytics flips this model. It brings live data from operations straight to decision-makers. A CEO can see global sales by region as they happen. A CTO can monitor system health before an outage hits.

How SAP Enables This

SAP HANA: stores and processes data in-memory for instant results.
SAP Analytics Cloud (SAC): gives live dashboards and predictive models.
Embedded Analytics: in S/4HANA allows leaders to view insights while working inside core processes.

These tools cut delays and remove the guesswork.

Practical Example

Imagine a manufacturer facing a sudden raw material shortage. Without real-time analytics, the CEO learns of it after a weekly report. With SAP, alerts appear the moment supply drops. Leaders can adjust sourcing, update prices, and inform partners—all within hours, not days.

Best Practices for the C-Suite

Ask sharper questions: Real-time data works best when leaders know what to track.
Balance speed with accuracy: Not all data needs live updates. Focus on what drives action.
Build trust in data: If executives doubt the system, they will fall back on instinct.

Challenges to Watch

Real-time analytics is powerful, but not simple. Common issues include:

Integrating data from multiple systems.
Ensuring security and access control.
Training leaders to read dashboards, not just slides.

The Bigger Picture

Real-time analytics does not replace leadership. It sharpens it. CEOs still need vision. CTOs still need strategy. SAP’s role is to provide clear, trusted signals so leaders can act without delay.

Engage Further

The question for the C-suite is simple: Are you ready to let real-time data guide your next move? If you want to explore practical steps, connect with experts, or join discussions in the SAP Community.

Use of DSEDIT tool to create entry in sql.ini file for Sybase ASE database

2025-12-05T15:41:19.023000+01:00

SAP Data Services installation stalls as the installer is unable to identify the CMS and Audit repositories.

Environment for this installation setup

Product - SAP Data Services 4.3
Database - Sybase ASE 16.0
Operating system - Windows Server 2019 Standard

Initial investigation took us to the ODBC connections and it worked fine and successfully connected to the respective database repositories. However on deeper analysis, it was observed that for the backend Sybase ASE database the connection doesn’t utilizes the ODBC connectors, it requires a Sybase client to be installed on the application machine and the resulting sql.ini file needs to be updated with the entries for CMS and AUDIT repositories.

SAP Data Services or Business Intelligence installation on Sybase ASE database needs the following client to be installed on the application host

After the installation below shown "sql.ini" file is created which needs to be amended with the entries of the repositories required for connection.

The format for the entries are

[CMS]
master=TCP,<Host/IP_of_DataBase_Host>,<Port_of_DataBase>
query=TCP,<Host/IP_of_DataBase_Host>,<Port_of_DataBase>

[AUDIT]
master=<Host/IP_of_DataBase_Host>,<Port_of_DataBase>
query=TCP,<Host/IP_of_DataBase_Host>,<Port_of_DataBase>

Another way of updating the sql.ini file is to use the DSEDIT tool provided by the client.

Launch the tool with administrator access

Click Ok

Click on the ServerObject and select Add

Enter the name of the repository, this should be exactly the same which will be used in the installation, in this example “CMS”

Double click on the server address bar on the right and click on the add button

On the TCP protocol in the network address bar enter the hostname/IP address followed by the port number at which the Sybase ASE DB is reachable separated by a comma

Once the entry is added test the connection

Right click and select Ping Server

Click on Ping

A connect succeeded message will prove the connection is successfully established, and an entry will get created in the sql.ini file with right syntax and parameters.

Now the installer will identify the CMS and AUDIT database repositories

Hedonic Pricing: What Do Investors Pay For?

2025-12-26T22:55:40.142000+01:00

Executive summary
Investors do not primarily pay for short-term margin spikes or isolated growth initiatives. Instead, over a 35-year horizon, the market has consistently rewarded one thing above all else: persistency. This is the term Goldman Sachs attaches to the capital structure discipline, specifically leverage (Assets/Equity). We find that leverage explains the majority of the variation in Home Depot’s Market Cap / Equity, our proxy for the Price/Book ratio. This finding closely replicates Goldman Sachs’ famous analysis of what markets pay for, but applies to a single company and validates using long-horizon time-series data. Profitability and efficiency matter — but they come second.

This note describes our findings in three simple blocks:

data preparation and transformation
results and interpretation
what it means for Home Depot and investors.

1. Data preparation: looking at Home Depot the way markets do

This analysis starts from a simple premise:

If we want to understand how investors value Home Depot, we must model Home Depot the way investors observe it.

We assembled 144 quarters (1990–2025 YTD) of data covering both market valuation metrics (P/B ratio, Market Cap, Enterprise Value) and accounting fundamentals (assets, equity, leverage, margins, asset turnover).

This is important: unlike cross-sectional studies across many firms, this model is time-series for a single company. We are not asking why Home Depot is valued differently than Lowe’s. We are asking a more precise question:

What has historically driven changes in Home Depot’s valuation over time?

Log transformations: elasticity, not levels

All core variables are expressed in logarithmic form. This is essential for interpretation:

Coefficients become elasticities
Relationships become scale-free
Long-term growth dynamics dominate short-term noise

In plain English: we are modeling percentage changes investors respond to, not absolute dollar movements.

The dependent (predicted) variable is the

ln(Market Cap / Equity), a proxy for the P/B ratio, directly links market valuation to balance-sheet fundamentals — exactly how long-term equity investors think.

Three independent (predictor) variables are

Leverage = (Assets - Goodwill) / Stockholders Equity where we specifically remove Goodwill to account for “tangible leverage” only,

Net Margin = Net Income / Revenue,

Turnover = Revenue / (Assets - Goodwill) here too, assets are net of goodwill for "tangible turnover."

In one quarter, Home Depot reported negative net income. In 9 quarters the company reported negative equity - initially in 2019 as a result of the adoption of the FASB ASU No. 2016-02, "Leases (Topic 842)". This is why the lines in the plot below exhibit discontinuities: natural logarithm of a negative number is undefined.

In log terms, valuation and operating fundamentals move in clear and persistent co-movement over time. Sustained multiple expansion coincides with rising operating efficiency, greater leverage capacity, and improving margins, while periods of valuation compression align with broad-based fundamental deterioration. Valuation itself behaves as a slow-moving state variable, reflecting investors’ long-run assessment of economic capacity, whereas operating metrics adjust at higher frequency around that state. Even during externally imposed disruptions—such as recessions and market panics—temporary dislocations give way to re-convergence, underscoring that market multiples are ultimately disciplined by fundamentals rather than driven by sentiment alone.

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------

This systematic co-movement directly motivates a hedonic valuation, in which observed market multiples are modeled as the equilibrium price investors assign to distinct operating characteristics rather than as standalone time-series outcomes.

-----------------------------------------------------------------------------------------------------------------------------------------------------------------------------

2. The hedonic pricing regression: what explains Home Depot's valuation?

Conceptually, the model mirrors the DuPont identity, which decomposes Return on Equity into three elements:

Leverage (Assets / Equity)
Profitability (Net Margin)
Efficiency (Asset Turnover)

Our model - with the log-transformed data - is:

To accommodate periods of negative net margin and negative stockholders’ equity, we introduced dummy variables that flag sign changes (0 for positive values, 1 for negative). The inclusion of dummy variables allows the model to accommodate sign changes without materially altering coefficient magnitudes or inference.

The regression, run in R Studio, produced a formula where most coefficients are statistically significant at the 5% level (p<0.05). The ln_net_margin variable returned a p-value of 0.0744, indicating a 7.4% probability of observing this result (or one more extreme) if there were actually no relationship between ln_net_margin and ln_mktcap_over_equity.

We drop the decimals:

Here is the most intuitive interpretation of the model’s coefficients and log-terms: elasticity. These bars represent standardized coefficients, not raw elasticities, and are intended for relative magnitude and intuition, not literal percentage translation.

	Home Depot’s valuation is most sensitive to changes in asset turnover, indicating that improvements in capital efficiency drive market multiples more powerfully than margin expansion alone.
In raw log terms, leverage can be interpreted as follows: a 1% increase in leverage increases Market Cap / Equity by approximately 1%.
Net margins exhibit a comparatively modest elasticity, suggesting that accounting profitability, while important, does not independently command valuation premiums.

Taken together, the results align closely with how long-term equity investors think: sustainable valuation expansion is rooted in efficient deployment of capital, not short-term margin optimization.

The key result: persistency dominates

The model has R2 of 94.05% and Adjusted R2 of 93.91%. If we decompose the individual predictors’ contribution to this explained variance in Market Cap/Equity, the key conclusion emerges:

Visually, this is striking: the largest block by far is leverage. Most of what investors paid for Home Depot over 35 years is captured by three variables of which Leverage is the most significant. This is what Goldman Sachs found in their panel data analysis calling the hedonic pricing element persistency.

Persistency, in Goldman’s framework, means:

Stable capital structure
Predictable balance-sheet scaling
Consistency in how equity is multiplied into assets and cash flows

What we have done here is replicate that insight, but:

For one company
Using 35 years of quarterly data
With a fully transparent regression framework

The alignment is almost exact.

Notably, the model contains no explicit time trends, macroeconomic variables, or sentiment indicators—yet it explains over 94% of valuation variation, underscoring how consistently markets price the same economic fundamentals.

In Home Depot’s case, leverage alone explains roughly two-thirds of the market’s valuation signal. Profitability matters — but it is secondary, less persistent, and less trusted by investors as a long-term signal.

Both actual and predicted series represent the log market-cap-to-equity multiple where

The chart shows that a simple three-variable hedonic model tracks Home Depot’s valuation remarkably well over more than three decades. Even though the model has no awareness of time, cycles, or sentiment, its predicted valuation closely follows the actual market valuation across multiple regimes. Deviations between the two lines are concentrated in periods of market stress or speculative excess, while long-run trends are captured cleanly.

In other words, investors consistently price the same economic fundamentals, though they might temporarily lose discipline during panics and recessions.

This directly supports the executive intuition captured in the quote by Dilip Shanghvi, the founder and CEO of Sun Pharmaceuticals:

“I will never risk capital; but I will risk profit.”

Markets agree.

3. How solid is our model? Is it "too clean"?

No serious business model should avoid this question.

Residual behavior (what we checked)

We explicitly reviewed:

Residual distribution
Residuals vs. fitted values
Time clustering during stress periods

With F-statistics 684.4 on 3 and 130 degrees of freedom and p-value < 2.2 x 10-16, the model is structurally sound, though markets still panic occasionally — and then revert.

That is exactly what we would expect from real investors.

What do residuals tell us

Residuals widen during periods of macro-financial stress, including the Global Financial Crisis, COVID, and the post-2022 rate shock. This is expected: markets temporarily price fear, liquidity constraints, and narrative risk beyond firm-level fundamentals.

Outside stress regimes, residuals oscillate tightly around zero with no persistent bias. Deviations cluster in time rather than by sign, indicating regime effects rather than model misspecification. Overvaluation episodes—most notably the late 1990s—mean-revert as market discipline reasserts itself.

Bottom line: the model “fails” when markets panic — and reliably re-anchors valuation once sentiment normalizes.

* * *

So, what does this actually mean for Home Depot?

Insight #1: capital discipline is the valuation anchor

Home Depot’s high valuation multiple is not fragile. It is anchored in:

Consistent leverage
Predictable scaling of equity
Transparent capital return policy

Any unexpected shift in capital structure — aggressive deleveraging or sudden debt expansion — would matter far more to valuation than a bad margin quarter.

Insight #2: profitability matters — but only if it is repeatable

Operational improvements (margin, turnover, ROA) explain about one quarter of valuation variation. That is meaningful — but markets discount one-off wins. Investors prefer:

Incremental improvements
Predictable execution
Fewer surprises

Spectacular quarters impress analysts. Consistent ones move valuation.

Insight #3: macro matters less than headlines suggest

Despite constant focus on rates, housing cycles, and sentiment, macro variables explain only a single-digit share of long-term valuation variation. Home Depot’s valuation history is fundamentally a company-driven story, not a macro-driven one.

Once full-year 2025 financials are available, we will:

Re-estimate the model
Test coefficient stability
Re-evaluate valuation implications under the new regime and compare/contrast the hedonic pricing model with the traditional DCF valuation model

If history is any guide, the conclusion is unlikely to change — but the path may.

Final thought

Markets are often accused of being short-term and irrational. Over 35 years of Home Depot data suggests the opposite.

Investors are remarkably consistent in what they reward.

They pay for persistency.

High Impact SAP Data Objects to Clean for an S/4HANA Selective Transition from ECC

2026-01-15T13:38:05.506000+01:00

*Views Expressed here are my own and doesn't represent any entity.

Introduction

Moving from SAP ECC to S/4HANA using a Selective Data Transition is not just a technical system conversion,it is a data transformation journey. While selective carve outs allow organizations to migrate only relevant organizational units, time slices, or data sets, poor data quality can jeopardize the entire S/4HANA transformation.S/4HANA’s simplified data model, stricter checks, and real-time processing mean that legacy inconsistencies tolerated in ECC will cause errors, functional gaps, or performance issues in S/4HANA.

This blog outlines the SAP data objects that require mandatory data cleaning before a selective S/4HANA migration, along with practical examples and rationale for each.

Why Data Cleaning Is Mandatory in a Selective Transition

In a selective approach:

Only chosen company codes, plants, or years are migrated
Historical inconsistencies become more visible
Data is restructured, not simply converted

S/4HANA:

Enforces business partner (BP) as a single master
Removes many legacy tables
Uses real-time reconciliation
Requires consistent master and transactional data

Without cleaning, you risk:

Migration failures
Reconciliation mismatches
Broken business processes
Delayed go-lives

1. Business Partner (Customer & Vendor)

What Needs Cleaning

Duplicate customers/vendors
Inconsistent master data across company codes
Missing mandatory fields (address, tax, bank details)
Vendors acting as customers (and vice versa)

ECC Example

Vendor 100045 exists for AP
Customer 200078 exists for AR
Same legal entity, different tax IDs and addresses

Why This Is Mandatory

In S/4HANA:

Customers and Vendors are merged into Business Partner
Roles (Customer, Supplier) are assigned to a single BP number
Inconsistent data causes CVI (Customer Vendor Integration) errors

Rationale

Selective transitions often migrate only certain company codes, making duplicates harder to reconcile post-migration. Cleaning ensures:

Smooth BP conversion
Accurate AR/AP reporting
Compliance with S/4 data model

2. Material Master Data

What Needs Cleaning

Obsolete or unused materials
Incomplete views (MM, SD, PP)
Invalid units of measure
Duplicate material numbers

ECC Example

Material created but never used in 8 years
Sales view missing for materials with open SD history
Base UoM inconsistent with alternative UoM

Why This Is Mandatory

S/4HANA:

Performs real-time MRP
Uses simplified inventory tables (MATDOC)
Enforces stricter checks on material consistency

Rationale

Migrating unused or inconsistent materials:

Increases migration volume
Slows performance
Causes MRP and ATP errors

Selective transitions benefit most by reducing material footprint.

3. Chart of Accounts & G/L Accounts

What Needs Cleaning

Blocked or unused G/L accounts
Accounts without proper account type
Inconsistent descriptions across company codes
Legacy reconciliation accounts

ECC Example

G/L 799999 created for testing, never used
Same G/L has different names in different company codes

Why This Is Mandatory

S/4HANA:

Introduces Universal Journal (ACDOCA)
FI and CO are merged
Reconciliation accounts behave differently

Rationale

Dirty G/L data leads to:

Conversion errors
Financial reconciliation issues
Incorrect financial statements

In selective transitions, only active and relevant accounts should be migrated.

4. Open Financial Transactions (AR/AP)

What Needs Cleaning

Cleared documents not properly closed
Open items with incorrect statuses
Residual items with zero balance
Old parked documents

ECC Example

Invoice marked as open but cleared manually
Zero-balance open items still flagged as open

Why This Is Mandatory

S/4HANA:

Performs real-time reconciliation
No longer tolerates technical inconsistencies
Universal Journal requires exact balances

Rationale

Open AR/AP items are always migrated in selective scenarios. Incorrect open items cause:

Balance mismatches
Failed migrations
Post-go-live reconciliation chaos

5. Asset Accounting (FI-AA)

What Needs Cleaning

Assets without acquisition values
Assets without depreciation areas
Retired assets still marked as active
Incomplete asset master records

ECC Example

Asset created but never capitalized
Depreciation area missing for a company code

Why This Is Mandatory

S/4HANA:

Uses New Asset Accounting only
Fully integrated into Universal Journal
Legacy asset inconsistencies cannot be converted

Rationale

Selective transitions often migrate only active assets. Cleaning ensures:

Correct depreciation
Accurate asset balances
Compliance with S/4 FI-AA model

6. Open Logistics Transactions

What Needs Cleaning

Open purchase orders with no activity
Incomplete deliveries
Open production orders never closed
Inconsistent goods movements

ECC Example

PO created 6 years ago, no GR, no IR
Production order technically complete but not closed

Why This Is Mandatory

S/4HANA:

Uses MATDOC for inventory
Enforces strict document flow
Real-time stock valuation

Rationale

Open logistics documents are technically migrated, not recreated. Dirty data causes:

Stock inconsistencies
Migration failures
Incorrect inventory valuation

7. Customizing and Organizational Data

What Needs Cleaning

Unused plants, storage locations
Inactive company codes
Invalid assignments (plant to company code)
Legacy controlling areas

ECC Example

Plant exists but no postings in 10 years
Company code included but not in scope

Why This Is Mandatory

Selective transitions rely on precise scoping. Invalid org data:

Complicates data slicing
Causes migration logic conflicts
Increases testing effort

Rationale

Clean org structures ensure:

Correct selective extraction
Lean S/4 system
Lower total cost of ownership

8. Custom Code & Z-Objects (Indirect Data Impact)

What Needs Cleaning

Z-tables with obsolete data
Custom programs using removed tables
Hard-coded company codes or years

ECC Example

Z-table referencing BKPF/BSEG directly
Custom report using VBFA incorrectly

Why This Is Mandatory

While not migrated as data:

Custom code processes migrated data
Dirty Z-data can break business processes

Rationale

Selective data migration often includes selective Z-data. Cleaning avoids:

Runtime errors
Incorrect reporting
Post-go-live defects

Understand your Business "WHY" for DATA CLEARLY

A Selective S/4HANA transition magnifies data quality issues because only parts of the system are migrated. S/4HANA’s simplified architecture does not tolerate inconsistencies that ECC allowed for decades.

Mandatory Data Cleaning Ensures:

Successful migration runs
Accurate financial reconciliation
Stable business processes
Long-term S/4HANA performance

Key Takeaway:

Data cleansing is not a technical task—it is a business transformation enabler.

#S4HANA #DATA #DATACLEANSING #MandatoryS4HANADATAOBJECTS #SAPECCtoS4HANA #SelectiveTransformation #DataQuality #ERPTransformation

#LegacyModernization

Helpful tips and tricks to save time when developing in SAP Data Services - Macro Substitutions

2026-02-09T12:15:24.597000+01:00

Did you ever have to deal with a wide table or an input file format containing dozens—or even hundreds—of columns, each storing different attributes of a single record?

And did you then find yourself cleaning those fields, applying functions, adjusting data types, and mapping everything to a target structure one column at a time?

If so, you’re definitely not alone. Fortunately, SAP Data Services provides a built-in feature that can dramatically reduce the effort required for exactly these kinds of scenarios.

The Challenge: Wide Files and Manual Effort

Let’s assume you are tasked with processing large and wide input files—such as .txt, .csv, or even .xlsx—using SAP Data Services.
Your goal is to load this data into a database in a clean, uniform, and standardized format, so it can be processed further downstream.

Manually applying cleansing logic and data type adjustments to each column quickly becomes inefficient and error-prone when dealing with large schemas.

The Solution: Bulk Column Adjustments in a Query Transform

The SAP Data Services Designer Client includes a built-in solution for bulk modifications of multiple attributes at once.

To make use of it, insert a Query transform directly after your file or table input and pull all input attributes into the output schema.

Now you can select the columns you want to adjust:

Click the first attribute, scroll down, and Shift-click the last one
Or simply use Ctrl + A to select all output columns
If you only want to work on a subset, use Ctrl-click to select individual attributes

Once the desired columns are selected, right-click on any highlighted column and choose “Properties…”:

Multiple Column Properties Window

You are now in the Multiple Column Properties window. All previously selected attributes are grouped together, allowing you to apply changes to all of them in one go.

In this example, let’s assume we want to:

Change the data type of all selected columns to VARCHAR(80)
Apply a basic cleansing logic to every field

To do this:

Enable the Data type checkbox and set the length to 80
Disable Content type and Description
Keep the Mapping checkbox enabled

Applying Identical Logic Using Macro Substitutions

In the Mapping section, you can apply identical transformation logic to all selected columns while keeping their original names intact.

This is achieved using the {original_expression} macro.

Example mapping logic:

ltrim(rtrim(upper(nvl(FF_PCO_EMS_EXTRACT.ExpectedClosingDate,'')),' '), ' ')

This expression:

Trims leading and trailing spaces
Converts values to uppercase
Replaces NULL values using NVL (also known as COALESCE)

Once you click OK, the mapping logic and data type changes are applied to all selected output columns at once.

The Result

Voilà!
You have just mass-updated data types and mapping logic for a large number of fields in seconds.

This approach is especially useful when:

Working with very wide files or tables
Applying standard cleansing rules early in a dataflow
Grouping columns into subsets that require different treatments (e.g. different data types or functions)

If you have ever relied on copy & paste or manual adjustments for each attribute in the Designer Client, this technique will save you a significant amount of time—and frustration.

Give it a try. I hope it helps anyone out there who is struggling with wide tables or file formats!

Julian