SAP Community - SAP Information Steward

#DataToOucome Chief Data Officer (CDO) and Data Governance Sessions are Available On Demand!

2020-09-18T08:29:21+02:00

Did you miss the #DataToOutcome event live, or do you want to deep dive into your favorite session? If so, our replays have you covered! Our SAP teams wanted to make sure that you can access this interactive, virtual event at your convenience.

#DataToOutcome kicks it up a notch from your basic online event by incorporating keynotes, demos, customer discussions, live Q&A, and persona-based tracks. This innovative platform spices things up and gives you the opportunity to replay the sessions that best fit your needs!

Five sessions are focused on the Chief Data Officer (CDO) and data governance, the first being, Using Intelligent Technology to Secure Green Energy Investment and Combat Climate Change. In this session the CEO and founder of Kaiserwetter Energy, Hanno Schoklitsch, discusses how Kaiserwetter leveraged SAP Data Intelligence Cloud to create their solution, Aristoteles. Hanno goes into depth about how Aristoteles helps them innovate to combat climate change and invest in the future of green energy.

The next session, Supply Chain Data Governance Strategy at Corning, gives you the opportunity to hear from Mustafa Dadawalla, global data governance manager at Corning Incorporated. Mustafa walks through Corning’s journey implementing a data governance strategy and the key ways they are using SAP Information Steward and SAP Data Services for data governance business rules. He also gives recommendations on how to get started with data governance, as well as how to introduce it to an executive sponsor.

Mirjam Heijnen-Keultjes, head of group master data management at Döhler, joined us for a session on, One Data Management at Döhler. During this session Mirjam talks about Döhler’s implementation of their master data governance program and how their desire to be a digital leader in the natural food and beverages industry helped them gain support from their board. Mirjam also addresses how they use SAP Master Data Governance data quality management capabilities to improve their data processes and increase insight into their master data quality across the organization.

Two SAP data quality experts, Sue Waite, senior director platform and data management Center of Excellence (CoE) at SAP, and David Steinbruck, senior director enterprise architect CoE at SAP, joined us for our session, All Good? Understand the Truth with a Data Quality Assessment. Sue and David discuss the benefits of a data quality assessment and how many organizations may be surprised with their results. They also emphasize the value of building a solid data strategy and provide resources to help you get started building yours.

Our last CDO and data governance centered session is, Data Quality and Migration: How to Prepare the Move to S/4HANA. In this session Sheila McCarthy, solution manager and data strategist at SAP, walks through the process of moving to SAP S/4HANA and what you need set up prior to cleansing your data for the migration. Sheila also addresses the first steps you should take if you don’t have a data migration strategy, along with what solutions are needed for the migration to help ensure the best results.

The #DataToOutcomes event provides a variety of sessions to make sure your data needs and questions are met. To learn more about what this event additionally has to offer, check out Ina’s blog and access all of the replays on our virtual platform!

Shared Terms – A Hybrid Use Case with SAP Data Intelligence and SAP Information Steward

2020-12-22T02:56:32+01:00

This blog introduces the second hybrid use case between SAP Data Intelligence and SAP Information Steward that gives the opportunity to leverage an on-premise deployment and connectivity from SAP Information Steward with SAP Data Intelligence Cloud.

Data stewardship solution provides data profiling, validation and quality rules, glossary terms, monitoring, metadata management, and information policy management capabilities to easily analyze, improve, and monitor the data integrity of your enterprise data. All while also gaining continuous insight into the quality of your enterprise information; and understand the impact of data quality on your processes and enhance your operational, analytical, and data governance initiatives.

In our shared terms hybrid use case, existing SAP Information Steward customers’ terms, terms that they created, defined definitions that could include internal and external links, terms that have gone through an approval process, and are being used to build transparency, understanding, and governance today can now be reused in SAP Data Intelligence. SAP Data Intelligence can reuse the same approved terms from SAP Information Steward for various new data sources, including cloud sources. Check out this video, showing how quick and easy we can share terms from SAP Information Steward with SAP Data Intelligence.

Benefits of shared rules, include:

Enables glossary terms to apply to a widely expansive cloud and on premise of sources and applications

Allows substantial reduction of time to redevelop terms while gaining faster insight into newly added sources and applications

Improves governance and transparency with common understanding and agreement on business taxonomies across all sources and applications

Promotes proactive data governance with shared glossary terms to maintain, manage, and govern all terms

Increases collaboration between users, as well as SAP applications

Burst computation with cloud deployment with pay as you go model

Shared terms hybrid scenario increases collaboration between users, as well as SAP applications. Give sharing terms from SAP Information Steward with your SAP Data Intelligence projects a try today and get that quick start to improving your governance initiatives. Keep your eye out for more hybrid scenarios between SAP Data Intelligence and SAP Information Steward.

DATA PROFILING OVERVIEW – SAP INFORMATION STEWARD

2021-04-19T08:40:23+02:00

This Blog Post will give the overview on Data Profiling technique within SAP Information Steward data quality tool. I will start with overview in this post and will explain the commonly used data profiling technique in SAP Information Steward which is column profiling.

This article will guide you through step by step procedure and will give you the complete idea on usage of column profiling.

So let's start by understanding what is Data Profiling?

What is Data Profiling?

It is the process of examining the data available from an existing information source (SAP, Database, File) and collecting statistics or informative summaries about that data

Use profiling to examine data so you can understand its content, structure, and data quality dependencies.

Data Sources for Data Profiling

Benefits of Data Profiling :

Types of Data Profiling :

Column Profiling – Determines the values and characteristics of data elements such as :

•Value, string length, and completeness

•Number of distinct values or patterns in a column

•Distribution of distinct words in a column

Dependency Profiling – Identifies attribute-level relationships in the data by finding the values in one or more dependent columns that rely on the value in a primary column.

Address Profiling – Determines the quality of addresses by determining whether the address is:

•Valid

•Correctable

•Invalid

Redundancy Profiling – Determines the degree of overlapping data values or duplications between two sets of columns.

Uniqueness Profiling – Returns the percentage and counts of rows that contain unique data for the set of selected columns.

Content Type Profiling – Returns the information about the type of data exist in the columns of table.

Uniqueness vs Redundancy

Uniqueness profiling talks about finding duplicate values and checking uniqueness within the column/s of same table Whereas, Redundancy profiling talks about finding overlapping between the pair of columns of two different tables.

Above are the different types of profiling techniques available with in SAP Information Steward.

Now begin with explaining them in detail, I will start with column profiling.

Consider the below data set as an example to explain the column profiling-

Data set example

Here are some key points to remember when you are performing column profiling in SAP Information Steward :

Key Points

Column profiling as the name implies, it helps the user in understanding the Data stored within the columns in table’s/view’s.

This profiling feature help in determining Values, String lengths, Completeness and Distribution across the columns.

Column profiling can be performed on table/view

How to perform the column profiling?

To perform the column profiling on Table/View:

Just select the view/table and hit on Column profiling from the profiling options in workspace section of SAP Information Steward. Window shown in screenshot will get pop up. (See the screenshot)

Check the boxes w/ types of column profile to perform and leave other values to default

Hit Save and Run Now button to execute the column profiling

column profiling task creation

Important values to keep in mind :

Input Sampling Rate- How you want the records chosen. For example, if you chose a Max input size of 1,000 records and you enter a rate of 1, then the first 1000 records will be profiled. If you enter a rate of 2, then every second record of the total records in the table, up to 1000 records, will be profiled, and so on.

Filter Condition-: You can add filter condition while creating task also using filter condition option.

Distribution Option : If you have requirements for checking the distribution of data or if you want to see how the data is spread across the primary columns, you can use the median and distribution and word distribution options.

Note: this option should only be used for the columns for which you want to see distribution results as this option takes lot of time to give you the results when you have huge data.

You can check the task status in Task section of SAP Information Steward, once task is complete results can be viewed in workspace section.

Reading the results generated from column profiling-

Column Profiling gives us the statistical analysis of data which can be very useful in understanding the data which exists in the system. Below are the explanation of each result.

Output/Results of Column Profiling

Value (Analysis of values populated in the columns)

Min- Minimum value for that column

Max- Maximum value for that column

Average- Average value for that column(all records)

Median – Centre value of that column

String Length (Analysis of lengths of values populated in the column)

Here also it gives us the min, max, average and median of lengths of values available in column.

Completeness (Provides analysis around what percentage of data is populated within the columns)

Null %- Percentage of Null values present in the column

Blank %- Percentage of Blank values present in the column

Zero %-Percentage of Zero(0) values present in the column

Distribution (Analysis of how data is spread across columns)

Value- Shows distribution of records across the column

Pattern- Shows different data patterns followed across the column

Word- Tells different usage of words across the columns

Here I complete the detailed explanation of first type of data profiling which is column profiling in SAP Information Steward. I will be covering all other types in my next posts, so be connected.

Please do provide your valuable feedback on this post in comments section, this will help me in improving my content and share more knowledge with this community.

Thanks and Happy learning!

Uniqueness Profiling Overview- SAP Information Steward

2021-04-19T18:49:20+02:00

This Blog Post will give the overview on Uniqueness Data Profiling technique within SAP Information Steward data quality tool. I will explain uniqueness data profiling technique step by step in SAP Information Steward.

This article will guide you through step by step procedure and will give you the complete idea on usage of uniqueness profiling techniques.

Now Let's begin with explaining that in detail, I will start with Uniqueness profiling.

Consider the below data set as an example-

Here are some key points to remember when you are performing uniqueness profiling in SAP Information Steward :

Key Points

Uniqueness profiling is useful for identifying duplicate data within the columns in tables/views.

This profiling feature shows the duplicate as well as non duplicate records with Pie chart graph analysis.

Also, very important It will give you the duplicate count for those non unique records

How to perform the Uniqueness profiling?

To perform the uniqueness profiling on Table/View:

Just select the view/table and hit on Uniqueness profiling from the profiling options in workspace section of SAP Information Steward. Window shown in screenshot will get pop up. (See the screenshot)

Check the boxes against the column names for which you want to check the uniqueness

Hit Save and Run Now button to execute the uniqueness profiling

Uniqueness Profiling Task Details

Uniqueness can be defined on individual columns or set of columns. If you will select multiple columns in uniqueness profiling task then it would consider it as a set of columns and analyze the records for uniqueness based on those set of columns.

If you want to perform for individual columns then you need select one column at a time and create multiple instances of the task.

You can check the task status in Task section of SAP Information Steward, once task is complete results can be viewed in workspace section.

Reading the results generated from Uniqueness profiling :

To check the results of uniqueness profiling, you need to follow below steps-

Go to workspace and select the view/table for which you performed uniqueness profiling

On the right hand top corner, click on advanced tab

Under advanced tab section you would see different columns with different profiling technique names

Under uniqueness profiling section, you would see an green tick beside your view, click on that and you should be able to see the results as shown below.

Result -1 - Unique vs Non Unique records

Result - 2 - Duplicate Count for non unique records

Here I complete the detailed explanation of Uniqueness data profiling technique in SAP Information Steward. I will be covering all other types in my next posts, so be connected.

Please do provide your valuable feedback on this post in comments section, this will help me in improving my content and share more knowledge with this community.

Thanks and Happy learning!

SAP Data Intelligence Expert Services Q2-Q3 2021

2021-04-22T00:52:31+02:00

If you are

an existing SAP Data Services user, you may wonder how SAP Data Intelligence can help you do more with SAP Data Services for all your data integration and data quality requirements

or an SAP Information Steward user, you may think how to expand your data stewardship capabilities when you use SAP Data Intelligence with SAP Information Steward.

or If you are an early adopter of SAP Data Hub and/or SAP Data Intelligence, you like to know how SAP Data Intelligence has evolved since you last attended a session or worked with SAP Data Intelligence, particularly how public API in metadata management is used by a partner, BigID to enhance SAP Data Intelligence Data Catalog and how data ingestion in SAP Data Warehouse cloud is served.

Irrespective of whatever of these categories you belong to, we invite you to attend our new SAP Data Intelligence Expert Services scheduled for 2021 Q2 and Q3.

Here is a list of webinars scheduled.

The way forward SAP Information Steward and SAP Data Intelligence, Wednesday, June 23, 2021, 10:00 AM EDT

Strategy and latest updates for SAP Data Intelligence, Thursday, June 24, 2021, 11:00 AM EDT

SAP Data Warehouse Cloud and SAP Data Intelligence: Better Together, Monday, June 28, 2021, 10:00 AM EDT

How SAP Data Mapping and Protection by BigID and SAP Data Intelligence work together to explore, discover, and protect your data, Thursday, July 01, 2021, 10:00 AM EDT

The way forward SAP Data Services and SAP Data Intelligence, Thursday, September 30, 2021, 12:00 PM EDT

Select one or more of the following webcasts and complete registration. Please check out registration page for these upcoming webinars and please register to attend!

If you are new to SAP Data Intelligence, please check out for available SAP Data Intelligence resources.

https://blogs.sap.com/2021/02/04/sap-data-intelligence-hands-on-video-tutorials/

Dependency Profiling Overview- SAP Information Steward

2021-04-27T17:06:54+02:00

This Blog Post will give the overview on Dependency Data Profiling technique within SAP Information Steward data quality tool. I will explain dependency data profiling technique step by step in SAP Information Steward.

This article will guide you through step by step procedure and will give you the complete idea on usage of Dependency profiling technique.

Now Let’s begin with explaining that in detail, I will start with Dependency profiling.

Consider the below data set as an example-

Here are some key points to remember when you are performing dependency profiling in SAP Information Steward :

Dependency profiling helps the user in identifying relationship between the columns of same table/view by defining the primary and dependent columns

Results of this profiling feature is in the form of values for both primary and dependent columns. Values for primary columns are shown as header which can be drilled down to get values for dependent columns

How to perform the Dependency profiling?

To perform the dependency profiling on Table/View:

Just select the view/table and hit on dependency profiling from the profiling options in workspace section of SAP Information Steward. Window shown in screenshot will get pop up. (See the screenshot)

Define primary and dependents column, add the column to primary section which you want to treat as a master column and add other columns to dependent section for which you want to check the dependency/relationship with primary column

Hit Save and Run Now button to execute the dependency profiling

Note : Dependency profiling can be performed on single table/view. Also, only one column is allowed to be added in primary column section. You can add multiple columns to dependent section. This means with dependency profiling you can check 1:1 or 1:n relationships

Dependency Profiling Task Details

Important values to keep in mind :

Input Sampling Rate– How you want the records chosen. For example, if you chose a Max input size of 1,000 records and you enter a rate of 1, then the first 1000 records will be profiled. If you enter a rate of 2, then every second record of the total records in the table, up to 1000 records, will be profiled, and so on.

Max distinct values-: This field allows you to set how many values you want to display as sample in output. maximum and default is 50.

You can check the task status in Task section of SAP Information Steward, once task is complete results can be viewed in workspace section.

Reading the results generated from Dependency profiling :

To check the results of dependency profiling, you need to follow below steps-

Go to workspace and select the view/table for which you performed dependency profiling

On the right hand top corner, click on advanced tab

Under advanced tab section you would see different columns with different profiling technique names

Under dependency profiling section, you would see an green tick beside your table/view, click on that and you should be able to see the results as shown below.

Dependency Profiling Result

Interpreting the results :

You would be able to see the results in the form of header-item kind of structure, where header represents the primary column value and item represents the dependent column values(multiple values get separated by commas)

Number associated with header value gives the count of dependent records on that primary value
- eg- In screenshot, Mumbai(2) represents that Mumbai city has 2 dependent employee records.

Here I complete the detailed explanation of Dependency data profiling technique in SAP Information Steward. I will be covering other types in my next posts, so be connected.

Please do provide your valuable feedback on this post in comments section, this will help me in improving my content and share more knowledge with this community.

Thanks and Happy learning!

Redundancy Profiling Overview- SAP Information Steward

2021-07-26T12:48:16+02:00

This Blog Post will give the overview on Redundancy Data Profiling technique within SAP Information Steward data quality tool. I will explain Redundancy data profiling technique step by step in SAP Information Steward.

This article will guide you through step by step procedure and will give you the complete idea on usage of Redundancy profiling technique.

Now Let’s begin with explaining that in detail, I will start with Redundancy profiling.

Consider the below data set as an example-

Data Example

Here are some key points to remember when you are performing redundancy profiling in SAP Information Steward :

Redundancy profiling helps the user in identifying relationship between the columns of different table/view by giving the details around overlapping of data between those columns.

Results of this profiling feature is in the form of venn diagram showing overlap between the columns. Also, gives the count of distinct values for that column and the number can be clicked to view records.

How to perform the Redundancy profiling?

To perform the redundancy profiling on Table/View:

Just select the view/table and hit on redundancy profiling from the profiling options in workspace section of SAP Information Steward. Window shown in screenshot will get pop up. (See the screenshot)

Select columns from main as well as comparison table for which you want to perform redundancy profiling

Hit Save and Run Now button to execute the redundancy profiling

Note : Redundancy profiling can be performed on 2 tables/views and a single column needs to be selected from both the tables for comparison.

Data Redundancy Profile

You would be able to see the results in the form of venn diagram where it will show the overlap between columns if they have common values else 2 separate circles if the values do not match.

You will be able to see the count of records overlapping the values and when you click on the number
- eg- In screenshot, City column has overlapped values with other city column in different table and when you click on the number you will be able to see those records.

Redundancy Profile Results

Here I complete the detailed explanation of Redundancy data profiling technique in SAP Information Steward.

Please do provide your valuable feedback on this post in comments section, this will help me in improving my content and share more knowledge with this community.

Thanks and Happy learning!

SAP Data Services, SAP Information Steward, and SAP Data Intelligence: strategy, maintenance, and future vision of SAP Data Management products

2021-08-03T19:28:04+02:00

An update on May 3, 2022

SAP Data Services 4.3 is released.

Update on April 19, 2022

Our current GA release is DS 4.2 SP 14 Patch x and IS 4.2 SP 14 Patch x, and its planned end of mainstream maintenance is May 2023. Please refer to page 5 of DS PAM and page 5 of IS PAM for all details concerning active versions, support policy and timeline by release. For more information on support policies please refer to Appendix 3 of the respective PAM documents.

DS 4.3 and IS 4.3 release plan has been revised from Q1, 2022 to middle of Q2, 2022 . With the release of SAP Data Services and SAP Information Steward 4.3 support plan gets extended to end 2027. Please refer to chapter 2.3.1 and 2.3.2 in SAP Maintenance and Support Policy for details.

There will be a one-year adoption time available to upgrade from DS 4.2 to DS 4.3 and IS 4.2 to IS 4.3.

In case of any delay in releasing DS 4.3 and IS 4.3, adoption time of upcoming version and maintenance of the current GA version will be extended accordingly, to ensure continuity.

Abstract: In this blog post you’ll find the following information

SAP’s Data Management portfolio direction with respect to SAP Data Services (DS) and SAP Information Steward (IS) in conjunction with SAP Data Intelligence

Product release and support strategy, and links to published documents

Go forward solution for utilizing SAP Data Services and SAP Information Steward assets at existing user bases

Data Management portfolio direction

SAP launched SAP Data Intelligence as a comprehensive data management solution, conceived and built since day one on a cloud-native containerized architecture. As the data orchestration layer of SAP’s Business Technology Platform (BTP), it transforms distributed data sprawls into vital data insights, delivering innovation at scale. It can be consumed both as-a-Service in SAP’s BTP, or BYOL on several hyperscaler and on-premise Kubernetes distributions, enabling complete flexibility of deployment.

While investing heavily to provide cutting-edge cloud-native data management, SAP is also fully committed to supporting hybrid landscapes, enabling enterprises to fully leverage their existing on-premise data sources and data management tools as they transition to the cloud. SAP’s hybrid data management strategy is designed to help existing SAP Data Services and Information Steward customers to leverage the power of SAP Data Intelligence to extend their reach and capabilities.

With our stated hybrid data management strategy, current on-premise DS and IS customers can in fact leverage all their existing investments and efforts, reusing those time-tested and trusted artifacts and configurations that are being used today in DS and IS, while also extending the reach of their data management to the cloud, with SAP Data Intelligence, a native cloud solution.

The key principles of this strategy are the following:

Reuse existing DS and IS setups for all on-prem sources and workloads that are still relevant, for all the time they will still be in use, without requiring any rip and replace nor migration

Implement all new cloud hybrid, or big data-oriented workloads on SAP Data Intelligence, seamlessly extending and reusing existing DS and IS workloads.

Please check out the following expert sessions to learn more about this hybrid data management strategy.

Product release and support strategy

For the coming years, SAP’s innovation investments will mostly prioritize SAP Data Intelligence, our flagship cloud-native data management solution. At the same time, and despite the fact that the transition to the cloud is clearly SAP’s strategic direction, it is also clear that such transition will not be black and white and will happen gradually over time, with on-premise landscapes that will still be part of the overall enterprise data landscape for several years.

For this reason, SAP will continue to maintain and innovate also its two main on-premise solutions, DS and IS, as well as upgrading the underlying platform - SAP BusinessObjects Business Intelligence (BI) or Information Platform Services (IPS). The dependency and product support information are shared via the respective product availability matrix (PAM). The DS/IS PAM document is updated regularly matching the GA version of the software. Maintenance model is published in the respective PAM documents.

Our current GA release is DS 4.2 SP 14 Patch x and IS 4.2 SP 14 Patch x, and its end of mainstream maintenance is March 2023. Please refer to page 5 of DS PAM and page 5 of IS PAM for all details concerning active versions, support policy and timeline by release. For more information on support policies please refer to Appendix 3 of the respective PAM documents.

DS 4.3 and IS 4.3 has been planned to be released in the end of Q1 2022. With this new release, mainstream maintenance will then be guaranteed until 2025 at least, and priority 1 support until 2027 at least. Please refer to chapter 2.3.1 and 2.3.2 in SAP Maintenance and Support Policy for details.

There will be a one-year adoption time available to upgrade from DS 4.2 to DS 4.3 and IS 4.2 to IS 4.3

In case of any delay in releasing DS 4.3 and IS 4.3, mainstream maintenance of the current GA version will be extended accordingly, to ensure continuity.

Concerning SAP Data Intelligence, the current release strategy is to issue a new version every quarter for SAP Data Intelligence, cloud edition, and once/twice a year for the BYOL/on-prem edition of the product. Given the frequent releases, the BYOL/on-prem edition of SAP Data Intelligence is normally maintained only for the minimum period of two years since the release to customer date of each major release, but retaining the possibility to extend such maintenance period if required, according to the planned released date of the subsequent version.

For example, for DI 3.x versions, end of mainstream maintenance was initially set to 15.03.2022 (two years since release of DI 3.0), but in April 2021 SAP decided to extend it to 15.03.2023, given that we are not shipping a new major version in 2021. Every minor version is then supported for at least six months after the release of its successor version: i.e. patches for version 3.n are shipped until 6 months after release of version 3.(n+1). Please refer to page 6 of the Data Intelligence on-prem PAM for more details, and to the PAM summary for the details of the current end of mainstream maintenance dates.

Go forward solution

In summary, SAP will release new version(s) of SAP DS and IS to all existing and new on-premise DS and IS customers. DS and IS customers can reuse existing investment with SAP Data Intelligence and to assist customers in migrating to the cloud. SAP customers can receive the necessary support information directly at the published PAM document.

For users opting to do Data Integration and Data Governance in cloud with a PaaS method Data Intelligence is the way forward

For existing DS/IS on premise users our upcoming solution of Hybrid Data Management is offered to use both data management products and DI as bridged solution, which allows them to retain their current DS/IS contents and manage them from Data Intelligence as cross functional components. This also allows them to increase their ability of cloud-based data integration with additional cloud-based data and application system connectivity, and machine learning capabilities which cannot be provided by on premise data management solutions.

SAP Data Intelligence is available both as a public cloud service and as a BYOL/on-prem product. The latter is issued on the same code base and feature scope as the public cloud service, on a fully containerized cloud-native technical architecture. For this reason, Data Intelligence BYOL/on-prem edition is not meant to replace DS/IS for traditional data management workloads, but it rather complements DI cloud edition for hybrid scenarios and big data scenarios.

If a user doesn’t want to go either cloud or Hybrid, they can keep using DS/IS with available functionalities within the scope of on-premise data integration and computing.

Useful Links

SAP Data Services 4.2 PAM

SAP Data Services 4.3 PAM

SAP Information Steward 4.2 PAM

SAP Information Steward 4.3 PAM

SAP Data Intelligence on-prem PAM

SAP Maintenance and Support Policy

SAP Community

Building a data quality pipeline using SAP Information Steward

2021-09-23T15:42:56+02:00

Data quality is relevant for all applications especially it can define the success of data migration or data reports above configuration above several aspects. It can be handled in several ways with several tools.

This article will explain with a few examples what are the main data quality issues and it will sketch a rough architecture of an SAP-based solution that identifies data quality issues using SAP Information Steward. Usually, Information Steward is used in combination with SAP Data Services. SAP Data Services can use the information provided by SAP Information Steward to perform mitigating measures on the database itself. This would be a second part article to this content.

The main data quality issues

First of all, the folk wisdom of Data Quality divides issues into seven dimensions, and we will follow that structure (according to this Source/Inspiration😞

Completeness: it captures instances where there are missing data and on which columns it's incomplete;

Accuracy: it identifies how well the data you collected reflects the actual occurrence. One example would be a date for a past event recorded as 01-01-9999;

Consistency: how consistent is a specific piece of information across your IT landscape? For instance, if the marital status of a customer is consistent in two databases;

Uniqueness: it detects duplicate entries. For instance, when there are two records for the same customer;

Integrity: it ensures that data isn't missing essential relationship linkages. One example can be if data is stored in a shorter form in different systems, making it harder to establish a direct connection across systems.

Timeliness: it captures instances where data isn't sufficiently up to date for some task. One example is data from the previous business day is loaded at noon of the following day, and we have a report is being executed at 10am;

Validity: it identifies data that does not necessarily obey the rules we expect. For instance, a telephone number should not contain letters;

Analyzing SAP objects in SAP Information Steward

So, in this article, we are using SAP Information Steward to profile and assess the data according to the seven dimensions.

The first tab is the workspace, where you can run rules and profile the data.

The following highlighted tab is the rule tab, where Rules are created, published, and approved.

The next highlight is the Scorecard setup tab, where scorecards are created, using rules and applying them to tables.

The final tab is the Tasks tab, where you can view and rerun past tasks, such as profiles and rules.

To import the data, click the "Add" button in the menu.

Select Tables…

Select the connection and in the following screen, expand the folder where the tables are located. Next, select the tables to import and add to the project.

Once imported, you can locate the tables in the workspace.

You can then create views and apply joining conditions and additional filters or transformations to the data. Finally, let's advance to the actual data quality work.

To profile each table or view, select the checkbox and expand the dropdown list under the profile icon

There are six different profiles available.

The column profiling is considered basic and can be seen within this view tab. All others have the results in the advanced tab.

When you run a profile or a rule, the tool will tell you to create a task in a popup window. Save and run.

The result of a column profile can be seen below, expanding the table to see the columns.

It will give some information on the columns data distribution, for instance, in the highlighted area, we can see it states the low cardinality of some fields, and the material field is unique (potential key field). Then, the advisor will suggest some primary rules to clean the data (identifying the nature of the field by the header). These rules can either be accepted or declined. The other columns will display some statistics about the data, extreme values,…

The data content profile will identify the data type of the field and will be useful when there are generic data types, such as telephone numbers or addresses. These common types have standard SAP cleansing advisors, so dealing with these columns will be less time-consuming.

Other types of profiling can be run and seen in the Advanced view of the workspace.

Namely, to see if there are any duplicates records, select the Uniqueness profile, and it will display the number of duplicate records and the degree of repetition.

To see redundant data – data duplicated across tables – run the redundancy profile. It will compare selected columns and will present a Venn diagram with the two datasets. Each of the subsets can be viewed upon clicking.

To see the dependency between two or more fields, you select the dependency profile. It will present the N:M relation occurring between those fields. For instance, if a value for one field has three values mapped to it.

The Rule tab will present the rules divided by the data quality domain. It will also show the score of the rules that were already run.

To create a rule, click "New".

Enter and name ($<name>)some of the parameters that will later be bound to specific table columns.

Also, define the expressions associated with those parameters. For instance, $parameter IS NOT NULL. Any record that has a null on the column the rule is reading will fail the rule.

Also, one must specify the rule approver and add a Quality dimension to the rule (described briefly at the beginning of the article).

It is important to be coherent when classifying rules by the quality dimension. This will be helpful because on scorecards, the quality dimensions will be analyzed separately, and results will have better quality.

In addition, the advanced rule editor has some added features, which enable more complex rules. All of the functions available are described and have syntax examples.

Also, one can quantify the cost per failed record. This is done in Financial Impact per failure.

You can categorize the type of cost accordingly and add a dollar value to each instance where it was identified. This will help Business estimate the impact of data quality and raise red flags where needed.

After this step is done, rules have to be bound.

After that, the parameters for the rules will have to be mapped to a particular field of the table.

Also, the quality threshold is customizable on a 10 point scale, depending on the specific requirements of the dataset. Also, notice that the same rule can be bound to several tables.

The next step is the creation of scorecards.

Each of the scorecards has a Key Data domain. Name accordingly to the tables you want to classify.

The next step is to add a quality dimension. If more than one is added, you can weigh them according to their relevance for the data domain.

For each of the quality dimensions, rules can be entered in the data domain. Again, for each of the rules belonging to the same quality dimension, a specific weight can be attributed.

The final step is to select, for each of the rules entered, the tables they are bound to. And the weight of these bindings is also customizable.

Once the scorecard is prepared, the rules are run in the workspace tab, and the scorecard gets populated.

Selecting on view or table and then clicking on the Calculate Score button will present the trend of the rule score – once it's executed more than once – and a sample of the failed data, which can also be filtered or exported. Notice that the score's color is based on the thresholds defined earlier. If the table is expanded, you can see a detailed view of the score of each field.

Bear in mind that the entire subset of data should be exported to tables in the database assigned in the Central Management Console.

To view the scorecard, change the Workspace tab to the Scorecard tab.

There the trend for Quality Domain can be seen dependent on the rule you wish to see. It's important to distinguish how to Show the score. 'As of Now' will present manual rule execution; 'Last Run' will show the execution of the rule as a scheduled CMC scorecard run.

As in the Rule view, we can see the failed data but in the scorecard view, we can see the business value analysis.

This feature can be seen on the screen below:

Once clicked, it will present information about the object, synonyms, and keywords.

Let me know if this topic is interesting to you in the comments if you want to see how Data Services would mitigate the information extracted by Information Steward or if you have specific feedback about this topic.

SAP Data Services 4.2 SP13 Upgrade steps

2021-10-15T16:05:18+02:00

Purpose:

The purpose of this document is to up-grade SAP Data Services from 4.2 SP12 to SAP Data Services 4.2 SP13

Overview:

Environment Details:

Operating system: Windows Server 2016 64 Bit

Database: Microsoft SQL Server 2012 R2

Web Application: Tomcat 8

SAP Business Objects Tools: Information Platform Services 4.2 SP7; SAP Data Services 4.2 SP13

Migration Tools: Data Services Repository Manager

Repository version: BODI 11.7 or BODS 3.X or BODS 4.X

Installation & Configuration steps to upgrade SAP BODS 3.X/4.X to SAP BODS 4.2 SP13:

Pre Installation Check List:

Backup of repositories, configuration files & Data Cleanse files:

Backup of Local, Profile & Central Repository

Backup of Following Configuration Files:
- admin.xml
- sapconnections.xml
- as.xml
- dsconfig.txt

Backup of Data Cleanse Files

Create a Check list of Number of Batch Jobs/Real Time Jobs, Number of Access Server, Number of Job Server & Configuration Path (Datastore) of Source & Target Systems

Create a Check list of Jobs Scheduling in SAP Business Objects Data Services Management Console

Create a Check list of Groups & Users Available in SAP Business Objects Data Services Management Console

Post Installation Check List:

Installation of EIM(Enterprise Information Management) or IPS(Information Platform Services) Package in New Landscape

Installation of SAP BODS 4.2 SP13

Installation & Configuration of Address Cleansing Package 4.2

Best Practices says use the same Database that you have use in SAP BODS 4.x make a cloned copy and Use in SAP BODS 4.2 SP13 environment

Up-gradation of Local /Central repository Using Data Services Repository Manager

Configuration of Repository in EIM or IPS (Package Name) Central management Console.

Configuration of SAP RFC in Data Services Management Console 4.2 SP13

Configuration of Adapters in Data Services Management Console 4.2 SP13

Apply Rights & Security according to your requirement in BIM or IPS (Package Name) Central management Console

Now Login in SAP BODS Designer 4.2 SP13 and configure Jobs (Real Time/Batch Jobs)

Configuration of Datastore (SAPR3,SAP BW, RDBMS), Substitution Parameters, Job Server, Access Server, Format Data Type (Flat Files, CSV)

Now validate the Jobs at Top Level and if error comes fix those errors

Now Execute the Jobs using Scheduler or other BI Tools

SAP Business Objects Information Platform Services 4.x SPxx Installation & Configuration Documents:

Product Infrastructure Dependency:

Data Services (DS) requires SAP BusinessObjects Business Intelligence (BI) or Information Platform Services (IPS) for security capability. The table below shows the BI/IPS version and the corresponding compatible version of Data Services.

For more information on the compatibility of Data Services with BI platform or IPS versions, and compatible patch information, please see SAP Note 1740516

https://service.sap.com/sap/support/notes/1740516

1789731 – What version is compatible with Data Services 4.X? – Data Quality Management for SAP

Supported Platform Link:

http://service.sap.com/PAM

Validation of Current Environment before and after application upgrade:

For Validation of Jobs, Access Server, Adapter & Web Services I am using script to populate the data in text file, Please refer the o/p for your reference.

S.No	Validation Activity	Count
1	Count of Local Repository	1
2	Count of Secure Central Repository	1
3	Count of Job Server	1
4	Count of Access Server	1
5	Count of Batch Job	18
6	Count of Real Time Job	18
7	Count of Adapter	1
8	Count of Web Services	18

Count of Repository: Currently I have 1 “Local & 1 “Secure Central repository” highlighted in Yellow color.

* For validation you can also use “Local repository” metadata for above details. Here I used BATCH script to check all above parameter in my current systems, Please refer the below screenshot for your reference.

Count of Job Server:

Count of Access Server:

Count of Batch & Real Time Job:

Same details you can get from batch script for Adapter & Web Services.

Up gradation & Configuration Documents:

When the prerequisite system check completes successfully, click Next

On the “SAP BusinessObjects Information platform services 4.2 SP7 Patch 7 setup” page, click Next.

Accept the License Agreement, and click Next.

Select Language (English) By Default, and click on Next

Type a password for the CMS Administrator account, and click Next

Now Select Installation mode as “Standard Installation” and click on Next

To start the installation, click Next.

Installation is in progress

Finish

Up gradation & Configuration Documents of Data Services:

When the prerequisite system check completes successfully, click Next

On the “SAP BusinessObjects Data Services 4.2 SP12 setup” page, click Next.

Accept the License Agreement, and click Next.

Click Next to accept the default path to the folder where the program will be installed.

Click Next so that it can use the same location where SAP BusinessObjects Data services related files are deployed, once upgrade is done please validate the configuration

Click Next as “English” is by default Language, You can select as per your requirement

Update SAP BusinessObjects Administrator credential to update the Metadata w.r.t new setup and click on Next

Click on Yes

SAP BusinessObjects Data Services Installation is started, Click on Next

Installation is in progress

Finish

SAP Business Objects Data Services 4.2 SP13 Configuration Documents:

Before upgrade of local repository below are the error in SAP BusinessObjects Data Services Designer/Central Management console

So before login upgrade the repository using SAP Business Objects Repository manager.Below are the steps for upgrade of local repository details:

Below are the steps for upgrade of local repository & upgrade of secure central repository

Upgrade of Local Repository using SAP Data Services Repository manager:

Open SAP Data Services repository manager and check the version before migrating, You can also check in repository metadata through SQL

And after validating click on upgrade tab

Upgrade of Secure Central Repository:

Do the same validation for Secure central repository

And after validating click on upgrade tab

After Installation & Up-gradation of SAP business Objects BOE/IPS Server, SAP Business Objects Data Services 4.2SP13 below are the status that i upgraded in VM Ware

S.No	Repository Name	Repository Version	Upgraded Version	Status
1	LOCAL_REPO	14.2.3	14.2.13	Completed
2	CENTRAL_REPO	14.2.3	14.2.13	Completed

Then Login to Central Management Console -> Go to Data Services

Then Click on Data Services and check the repository version details.

Now try to login in SAP BusinessObjects Data Services Designer with SAP Business Objects Credential

Click on Logon and Click on “ok”

Now login in SAP BusinessObjects Data Services management console

Click on “Administrator” tab

Relevant Systems

These enhancements were successfully implemented in the following systems:

SAP Information Platform Services 4.2 SP7

SAP BusinessObjects Data Services 4.2 SP13

This document is relevant for

SAP BusinessObjects Data Services Administrator

This blog does not cover:

SAP BusinessObjects Data Quality Management Up-Gradation

Upgrade installation of SAP Data Services 4.2 SP13:- Some items to keep in mind when you install this upgrade:

The installer migrates the objects in the DSConfig.txt file to the new Data Services 4.2 SP9 locations. For more information about these changes, see User Access Control (UAC) support.

If, during the installation, you choose to configure a Data Services repository with an Oracle database, the installer will not accept the ojdbc5.jar file as a supported Oracle JDBC driver, and the installation will be blocked. The ojdbc5.jar file is not a supported Oracle JDBC driver in Data Services 4.2 SP13, so use ojdbc6.jar instead.

While you are performing an upgrade installation, if the existing Data Services product keys are already expired, the installation will be blocked.

For Windows, the upgrade installer contains an option to change the default location of the Data Services common directory (DS_COMMON_DIR). The new directory must be local and be accessible to all applicable users. Additionally the new directory must have sufficient disk space for the configuration and log files that reside in the common directory. For more information, see the Installation Guide for Windows (Installation, Interactive installation using default configuration, Change DS_COMMON_DIR location).

Whether you are upgrading from Data Services 4.x or some other prior version, keep in mind the following, because they apply to both upgrade scenarios.

- Data Services Profiler configuration: When you upgrade to Data Services 4.2 SP13, the Profiler configuration parameters are set to default values. If you had previously modified any of the Profiler configuration parameters, after installing 4.2 SP13, you need to log on to the Management Console and change the Profiler configuration parameters to your custom settings.

Oracle users: Before upgrading to version 4.2 SP13, be sure to grant the database account the create any sequence privilege

Creating Data Quality Rule In SAP Information Steward

2022-03-02T07:28:17+01:00

This Blog Post will give the overview on creating data quality rule in SAP Information Steward data quality tool. I will start with overview in this post and will explain the steps to create a rule in SAP IS.

So let's start by understanding what is Data Quality Rule?

What is Data Quality Rule?

Data is always been the core data asset for any company or enterprise. The bad data always leads to inconsistent business processes. To mitigate the bad data and make sure your data quality is fit for use, you need to create data validation to measure the quality of data.

SAP IS data quality tool helps in defining and building these rules, below are the steps to create a rule in SAP IS

To create a rule in SAP IS we use rules section in Data insight module of information steward.

Once you are in the rules section you would see different operations available at the top like creation, copy etc, I will explain them in detail. Refer below image, we will go from left to right starting with '+' option

Rules Menu

Creating New Rules - This option is used to create a new rule

Add Public Rules- This option is used to add existing public rules available in information steward rules repository

View/Edit- Option to display or edit rule depending on the user roles

Review Rule- option to review the rules and then approve or reject based on review

Publish - Option to publish the rule for other IS projects

Copy - Option to copy rule

Binding - Option used to bind the data set on which the rule would be executed

Delete- Option to delete the selected rule (Note : Bindings should be deleted before you try to delete the rule)

Rule History - Option to see the rule changes

Show usage-Option will help in checking where all the rule is being used currently

Share- This will generate a URL for that rule which can be shared with other users

Refresh - This Option will refresh the rules section

Steps for Creating Rule :

Click on create new rule option, Rule editor window will get open

Fill the mandatory and other important details
- Name - Here you can add meaningful name to the DQ rule
- Description - Specify the description for the rule, preferably it is good to add rule validation requirement in this
- Approver - Select the approver from the list who will review the rule
- Observer- If you want some other user to keep eye on the rule, you can add that in the observer field
- Quality Dimension - Specify the dimension for data quality rule. This is basically a rule category based on type of validation you are adding to rule. eg- If you are confirming some length or format of a field you can specify the dimension as Conformity

Parameters - This section is used to add parameters for the rule which would be used for binding fields on which rule will be executed

Expression Definition - This section would be used to write the actual validation/logic as per the requirement. you can use the inbuilt functions provided to help you write the code/logic

Expression filters - Here you can specify the filters for your data set if they are specific to rule.

Test - Test option allows you to test your logic by giving some dummy inputs to check if your rule is giving you correct results or not

Once you have completed all the details, logic and testing you can save the rule or you can also send it for review.

Rule Editor

Binding the Rule:

Above steps are just to create a rule in IS but to get the rule work, we need to bind it to our dataset and fields for which validation needs to be performed

To bind the rule,

Select the binding option from rules menu after selecting the rule for which you want to add binding

select the table/view on which you want to run your rule to check the validation

bind the fields corresponding to parameters

Specify the threshold value for rule results and then save

Execution of Rule:

To execute the rule we need to create task for the same.

For creating task, go to workspace section and then Rule results tab in top right corner.

Select the table/view which you used during binding

You will find calculate score option the top menu, click on that.

It will open a window, where you can set the execution properties and give a name to task.

Click on save and run now button, this will trigger the rule task and status can be monitored in tasks section

Once task is completed, come back to rules section and if you select the rule you should be able to see the score out of 10, failed records and total records adjacent to your dataset (table/view)

Rule Score

I have added few screenshots for your reference, it may help in understanding the details provided here.

Here I complete the detailed explanation of creating the rule in SAP Information Steward.

Please do provide your valuable feedback on this post in comments section, this will help me in improving my content and share more knowledge with this community.

Thanks and Happy learning! Do follow for more contents on data migration and Quality tools and technologies.

Enterprise Data Analytics & Data Management: A reference architecture

2022-04-07T14:39:49+02:00

Before explaining how a specific SAP analytics or data management product can address a dedicated problem, it is quite important to understand the overall problem using a product agnostic approach. A reference architecture typically helps in the identification of areas and business pain points that need to be addressed. Together with a world wide team of SAP domain experts for analytics, data warehousing and data management we've created the below content in 2021.

Target Audience

Data Management & Data Analytics architects from customers and/or services partners that are responsible for technical environments used for enterprise wide analytics & data management solutions.

The below content provides a summary of the content that was created. Detailed documents with descriptions of all capabilities, the use cases and their relations exists. We are happy to exchange and discuss them with you in a 1:1 conversation!

BUILDING BLOCKS

If you think about Enterprise Data Analytics & Data Management as a DOMAIN, this can be broken down in the following 5 SUB-DOMAINS.

Reference Architecture Block Diagram

Data & Analytics

The Analytics sub-domain is the process of discovering, interpreting, and communicating significant patterns in data. Quite simply, analytics helps us see insights and meaningful data that we might not otherwise detect.

The Data Warehousing sub-domain is the storage layer for any analytics solution. It is designed to hold data extracted from transaction systems, operational data stores and external sources. The warehouse combines data in an aggregate, summary form suitable for enterprise wide data analysis and reporting for predefined business needs.

Data Management

The Metadata Management sub-domain provides an end-to-end process and governance framework for creating, controlling, enhancing, attributing, defining and managing a metadata schema, model or other structured aggregation system, either independently or within a repository and the associated supporting processes

The Data Quality Management sub-domain contains capabilities related to Data Quality Dashboards, Data Analytics, Data Processing, Data Governance and Data Quality Services. Any data quality solution is comprised of one or more capabilities from this list.

The Data Integration sub-domain comprises the practices, architectural techniques and tools for achieving the consistent access and delivery of data across the spectrum of data subject areas and data structure types in the enterprise to meet the data consumption requirements of all applications and business processes.

SUB-DOMAIN AREAS

Analytics Sub-Domain AREAS

Any analytics solution is comprised of capabilities that can be grouped into one of the 8 areas outlined here. A detailed Analytics Reference Architecture document describes all areas and capabilities in detail.

Data Warehouse Sub-Domain AREAS

Any data warehouse solution is comprised of capabilities that can be grouped into one of the 6 areas outlined here. A detailed Data Warehouse Reference Architecture document describes all areas and capabilities in detail.

Metadata Management Sub-Domain AREAS

Any meta data management solution is comprised of capabilities that can be grouped into one of the 5 areas outlined here. A detailed Metadata Management Reference Architecture document describes all areas and capabilities in detail.

Data Quality Sub-Domain AREAS

Any data quality solution is comprised of capabilities that can be grouped into one of the 6 areas outlined here. A detailed Data Quality Management Reference Architecture document describes all areas and capabilities in detail.

Data Integration Sub-Domain AREAS

Any data integration solution is comprised of capabilities that can be grouped into one of the 6 areas outlined here. A detailed Data Integration Reference Architecture document describes all areas and capabilities in detail.

USE CASES

Reference architectures are usually something monolithic and static and it is hard for users to understand what this is all good for. A set of sample use cases help and show how the different building blocks and capabilities work together to support specific user roles.

Below is an overview of use cases which we've identified as "most relevant". Each use case is documented in detail including user roles and flow diagrams.

16 (most relevant) Analytics & Data Warehousing Use Cases

Based on the Analytics Reference Architecture and Data Warehouse Reference Architecture a number of use case have been outlined and defined in detail. Each use case also contains a mapping to relevant SAP products & solutions.

17 (most relevant) Data Management Use Cases

Based on the Data Management Reference Architectures a number of use case have been outlined and defined in detail. Each use case also contains a mapping to relevant SAP products & solutions.

SUMMARY

The aim of this set of documents is to provide common definitions and a common notation for all capabilities that are relevant for data analytics and data management solutions. It is for sure a working set of documents that may change over time.

Call-to-Action

Please provide feedback about your view and experience related to reference architectures for analytics and data management! Either contact me directly or post or comment below.

Additional Assets

Additional assets with regards to data architectures, data management, big data, and analytics can be found here:

Quick briefing decks for new database and data management trends

A future-proof Big Data Architecture

Does Your Organization Have a Data Strategy?

Digital Transformation and the Data Warehouse

Master Data Quality Management with SAP Master Data Governance on SAP S/4HANA 1909

Week 6: Why you should move your data management to the cloud!

2022-05-01T10:04:03+02:00

This article is part of our series “Give Data Purpose Weekly” . In this article I share my thoughts why traditional on-prem data management is not sufficient for the management of cloud data sources.

INTRODUCTION: DATA MOVES TO THE CLOUD

Many organizations have moved core applications to the cloud. With every application moving to the cloud, there is a significant amount of critical company data moved alongside.

According to Gartner, 75% of all databases will be deployed or migrated to a cloud platform by 2022 (Gartner: “Gartner Says the Future of the Database Market Is the Cloud”). Companies and the heads of data management in particular, are being confronted with complex hybrid data landscapes comprising both on-prem & cloud data stores. In this document we outline the challenges and solutions for a successful cloud data management transformation.

Cloud data is different

Access and processing of these new cloud data stores, however, is quite different from fully controlled, operated, and managed on-prem RDBMS systems. It requires new methods.

Furthermore, additional cloud datatypes and significant amount of data coming from social media, web sites, IoT and streaming devices are available. They exceed the traditional amount of data processed by companies today and need new concepts of data processing and data storage.

Managing cloud data leads to two key questions:

Are my current data management solutions sufficient for new data complexity?

What is the right data strategy: (a) Extend my current landscape to the cloud or (b) recreate it from scratch in the cloud?

If you're experiencing the same challenges and like to identify your Cloud Data Management approach, please get in touch with our SAP EXPERTS:
eMail us to arrange a Cloud Data Management workshop appointment!

CLOUD DATA MANAGEMENT FOR CLOUD DATA

Follow the data gravity

Now that most data is moving to the cloud, the adjacent data management topics need to be pulled to the cloud as well to be the new center for all data and data management functions.

“The center of data gravity is shifting to Cloud” said Adam Ronthal from Gartner in a 2021 Webinar.

Why on-prem data management is no longer sufficient?

Most data management solutions are from the early 2000s with a strong focus on data in on-prem applications. They have mainly been designed to integrate, improve, and synchronize data between on-prem applications. Such traditional ETL/ELT, replication, synchronization and data quality tools run in local data centers.

Although these traditional data management solutions still have a significant relevance, their reach to cloud data is "limited" for the following reasons:

Cloud applications provide new & different APIs

Cloud applications (usually) do not allow database level access

Cloud applications run in a different network

Cloud applications require new ways of authorization & authentication

A new approach is needed to manage on-premise and cloud data in a unified way.

DATA MANAGEMENT IN THE CLOUD: CHANGE IS REQUIRED!

Cloud data scalability is tempting

PaaS offerings tempt companies to generate large data stores without a strategy to process and apply this scale of data. Today we find data lakes of petabyte size in many companies storing unused “dormant” data. Since they lack a proper cloud data management strategy and software tools, this data remains unused and does not generate any value for the company.

Realtime self-service expectation

The intuitive real-time UX of modern mobile apps, cloud applications, and internet services has developed an according user expectation for data management tools. Users expect self-service and fast availability of any data product instead of predefined, static data marts. Users want secure, business user friendly tools to access corporate data sources (on-premise & cloud). Accordingly, data management must shift from an IT-centric tooling to a business user focus.

THE GOAL:
CLOUD DATA MANAGEMENT FOR ALL DATA SOURCES

Only a Cloud Data Management platform can address all demands to data management for both cloud and on-prem data without disruption. A Cloud Data Management platform is comprised of four main capabilities:

Cloud Data Integration

Source/Target processing with multiple exchange patterns: ETL/ELT, Replication, Virtualization, Streaming

Cloud Data Quality

Data Object profiling and improvement utilizing a data quality rule framework

Cloud Data Catalog

Metadata sourcing, discovery, storage, search and data lineage

Cloud Data Store

Used to persist all relevant data as a source for analytics or new Cloud applications

All capabilities should utilize artificial intelligence wherever meaningful. Self-service is an overall supported pattern to reduce dependency from IT experts and increase agility.

GO "HYBRID" IN DATA MANAGEMENT

“Hybrid” in the context of data management means expanding an existing on-prem data management to the Cloud to leverage the best of two worlds.

A hybrid data management environment complements the existing on-prem data management solutions with a Cloud Data Management platform containing new cloud data capabilities and functionalities to orchestrate, invoke and re-use components from the on-prem data management solutions. Data virtualization plays a key role in a hybrid data management strategy. A virtual database with a unified data model and a single interface for accessing data residing in different distributed data models provides quick access to on-prem data sources and avoids complex data integration effort.

Any new data management requirement should be implemented and configured in the Cloud Data Management platform ("Cloud first" approach). The legacy data management platform will continue to serve “on-prem only” requirements.

Artefacts from the legacy data management platform will be migrated over time whenever appropriate.

Eventually the on-prem data management platform can be switched-off without any risk and the Cloud Data Management solution will be the single future-ready platform to handle on-prem and cloud data management requirements.

SAP SOLUTIONS FOR HYBRID DATA MANAGEMENT

SAP Data Intelligence Cloud

SAP Data Intelligence Cloud is SAP’s comprehensive Cloud Data Management solution. As the data orchestration layer of SAP’s Business Technology Platform, it transforms distributed data sprawls into vital data insights, delivering innovation at scale.

It consolidates data from heterogeneous on-prem and cloud sources, provides management of IoT data streams, and facilitates scalable machine learning. SAP Data Intelligence enables business applications to deliver the intelligent enterprise vision and provides a holistic, unified way to manage, integrate, and process all your enterprise data

SAP HANA Cloud

SAP HANA Cloud provides a single place to access, store, and process all enterprise data in real time.

It is a cloud-native platform that reduces the complexity of multi-cloud or hybrid system landscapes. SAP HANA Cloud provides all of the advanced SAP HANA technologies for multi-model data processing in-memory or on disk. You can benefit from cloud qualities such as automatic software updates, elasticity, and low total cost of ownership by using SAP HANA Cloud either as a stand-alone solution or as an extension to your existing on-prem environment.

Orchestrate & re-use artefacts from SAP Data Services, SAP Information Steward &
SAP Landscape Transformation Replication Server

SAP HANA Cloud and SAP Data Intelligence Cloud provide the best-in-class connectivity to existing on-prem data management solutions from SAP. SAP Data Intelligence can orchestrate any ETL job build in SAP Data Services. Business terms and data quality rules form SAP Information Steward can be imported into SAP Data Intelligence to be re-used for cloud native data sources. Data from on-premise ABAP data sources can be replicated to cloud native targets utilizing SAP Data Intelligence replication flows together with on-prem SAP Landscape Transformation Replication Server.

GET ENGAGED!

e Mail us to arrange a Cloud Data Management workshop appointment!

We'd like to discuss your current data management challenges, your landscape and your roadmap towards the Cloud to identify your best Cloud Data Management journey!

SOMETHING TO READ?

General overview

What is SAP Data Intelligence Cloud?

What is SAP HANA Cloud?

Additional BLOGS

SAP Data Services, SAP Information Steward, and SAP Data Intelligence: Strategy …

SAP Data Services & SAP Data Intelligence: Optimized Data Management with Hybrid Solution

A Hybrid Use Case with SAP Data Intelligence and SAP Information Steward

SAP HANA Cloud Demo:
Data Virtualization & Relational Data Lake with AWS & Azure services

Free Software Trial

SAP Data Intelligence | SAP HANA Cloud

SAP Community

SAP Data Intelligence | SAP HANA Cloud

openSAP Micro Learning

SAP Data Intelligence | SAP HANA Cloud

Do you also want to give more purpose to your data?

Have a look in our popular paper “12 ways to give data purpose in a multi-cloud world”, where we provided a quick preview into 12 main ways to overcome typical hurdles that keep companies from using their data.

If you have any questions about SAP Business Technology Platform or data & analytics, leave a question in SAP Community Q&A or visit our SAP Community topic page.

Statement of Direction - SAP Data Services and SAP Information Steward

2022-09-09T02:07:47+02:00

Update as of November 2023 - Please refer to the latest Statement of Direction that overwrites this blog post that was published in September 2022

© 2022 SAP SE or an SAP affiliate company. All rights reserved.

No part of this publication may be reproduced or transmitted in any form or for any purpose without the express permission of SAP SE or an SAP affiliate company.

The information contained herein may be changed without prior notice. Some software products marketed by SAP SE and its distributors contain proprietary software components of other software vendors. National product specifications may vary.

These materials are provided by SAP SE or an SAP affiliate company for informational purposes only, without representation or warranty of any kind, and SAP or its affiliated companies shall not be liable for errors or omissions with respect to the materials. The only warranties for SAP or SAP affiliate company products and services are those that are set forth in the express warranty statements accompanying such products and services, if any. Nothing herein should be construed as constituting an additional warranty.

In particular, SAP SE or its affiliated companies have no obligation to pursue any course of business outlined in this document or any related presentation, or to develop or release any functionality mentioned therein.
This document, or any related presentation, and SAP SE’s or its affiliated companies’ strategy and possible future developments, products, and/or platform directions and functionality are all subject to change and may be changed by SAP SE or its affiliated companies at any time for any reason without notice. The information in this document is not a commitment, promise, or legal obligation to deliver any material, code, or
functionality. All forward-looking statements are subject to various risks and uncertainties that could cause actual results to differ materially from expectations. Readers are cautioned not to place undue reliance on these forward-looking statements, and they should not be relied upon in making purchasing decisions.

SAP and other SAP products and services mentioned herein as well as their respective logos are trademarks or registered trademarks of SAP SE (or an SAP affiliate company) in Germany and other countries. All other product and service names mentioned are the trademarks of their respective companies. See www.sap.com/trademark for additional trademark information and notices.

This document provides guidance to our customers of SAP Data Management portfolio, with respect to SAP Data Services (DS) and SAP Information Steward (IS) solutions, on the product directions. This document is an extension of our blog posting made on August 3^rd, 2021

SAP Data Management portfolio is the strategic Data Integration offering from SAP, and it is changing the way our customers run their business needs around Data Integration. SAP launched SAP Data Intelligence as a comprehensive data management solution, conceived and built since day one on a cloud-native containerized architecture. As the data orchestration layer of SAP’s Business Technology Platform (BTP), it transforms distributed data sprawls into vital data insights, delivering innovation at scale. It can be consumed both as-a-Service in SAP’s BTP, or BYOL on several cloud platforms and on-premise Kubernetes distributions, enabling complete flexibility of deployment.

As we define the future of SAP Data Management portfolio, with our stated data management strategy, current on-premise SAP Data Services and SAP Information Steward customers can in fact leverage all their existing investments and efforts, reusing those time-tested and trusted artifacts and configurations that are being used today in SAP Data Services (DS) and SAP Information Steward (IS), while also extending the reach of their data management to the cloud, with SAP Data Intelligence, a native cloud solution. Yes, we are committed to our existing SAP Data Services and SAP Information Steward customer base too.

In 2021 we added the hybrid data management capabilities that enables, existing SAP Data Services and SAP Information Steward setups for all native sources and workloads that are still relevant, for all the time they will still be in use, without requiring any rip and replace nor migration. Additionally, implement all new cloud hybrid, or big data-oriented workloads, including data streaming and data pipelining on SAP Data Intelligence, and seamlessly extending, and reusing existing DS and IS workloads.

In 2022 we did a minor version upgrade of SAP Data Services and Information Steward to 4.3, allowing the compatibility with Business Intelligence BusinessObjects 4.3. As a result, our support policy extended the end of mainstream maintenance timeline to end of 2025 and priority 1 support to end of 2027 for both SAP Data Services and SAP Information Steward. However, with the mainstream maintenance of version 4.3 of SAP Data Services and SAP Information Steward suite ending in 2025, we do not want to leave our large customer base behind, and therefore we have decided to release an updated version, code named SAP Data Services 2024 and SAP Information Steward 2024 to provide more time to our on-premises customers to adopt our rapidly evolving, state-of-the-art cloud data integration scenarios.

SAP Data Services 2024 and SAP Information Steward 2024

o We are planning to release a new version of the SAP Data Services and SAP Information Steward in 2025

code named SAP Data Services 2024 and SAP Information Steward 2024, respectively

available on-premises and through managed cloud,

focused on the most widely adopted on-premise scenarios: SAP Data Services and SAP Information Steward for native ETL, Data Quality and Data Governance,

o   Existing architecture and component map will be followed including Designer, repository, Job Server, supported web applications and BI module

o   Compatible with proposed SAP BusinessObjects BI 2024 suite

o   We will provide clear use case migration paths for the components for which we plan the end of support beyond 2027

o   Our intention is to only support Windows and Linux platforms

o   No change is anticipated in the Hybrid Data Management capabilities with Data Intelligence cloud

As more of our customers shift their on-premises workloads to the cloud, we also offer SAP Data Services and SAP Information Steward, private cloud edition (PCE), which runs the ETL and Data Quality solutions in a managed cloud environment operated by SAP. Many of our largest SAP Data Services and SAP Information Steward customers already run both solutions side- by-side, during their transition to the managed cloud deployment.

Many of our SAP Data Services and SAP Information Steward suite customers have now upgraded to the version 4.3. This version enables the compatibility with BI 4.3 as well as allows a smooth transition as there are no architectural changes, and code migration and upgrade is without any additional configuration. The enhancement around the major RDBMS connectivity with x.509 support gives the users a better user experience for secured data access.

We know that many customers rely on SAP Data Services and SAP Information Steward solutions for critical use cases and will need to do so beyond 2025. Therefore, we are planning to release a new version in the first half of 2025, code named SAP Data Services 2024 and SAP Information Steward 2024, respectively, that will extend the maintenance timeline for on-premise use cases. Although it is a long time from now, but we have plans to extend the mainstream support through 2030 and priority 1 through 2032

As we did for the upgrade to version 4.3, we will make sure that the upgrade to the new release is as smooth as possible and will provide enough overlap so that each customer can upgrade at their own pace. This new release will be an opportunity for us to continue the modernization started in version 4.3 (user experience, latest 3rd party components versions, system library updates) and streamline the ETL, Data Quality and Data Governance to the core requirements of our active user base

In the meantime, and as version 4.2 reaches end of mainstream maintenance at the end of May 2023 – followed by priority one support until end of December 2024 – we strongly recommend all customers to upgrade to version 4.3 to benefit from the latest features, up-to-date platforms, connectivity, and security standards.

With SAP Data Services and SAP Information Steward, private cloud edition, we offer a predefined package operated by SAP on the main cloud platform providers, including software, support, technical managed services, and infrastructure. This is a great opportunity to shift from a maintenance contract to a subscription model while leveraging your existing investments in SAP Data Services and SAP Information Steward solutions.

Here are some of the key benefits:

The latest version of SAP Data Services and SAP Information Steward is always available, thanks to yearly upgrades,

3-tier landscape: development/test, quality assurance and production by default, with optional additional tiers,

Pre-configured hardware landscapes optimized for performance,

Choice of cloud platform providers infrastructure includes AWS, Google Cloud Platform, Microsoft Azure,

End-to-end SLA 99.7%, backup and restore included within standard service scope,

In the future, customers will continue to have the choice for their infrastructure for SAP Data Services and SAP Information Steward solutions, either on-premises or in a managed cloud operated by SAP. Additionally, our directions for the on-premise data integration and data governance scope with respect to SAP Data Services and SAP Information Steward remains mutually exclusive and unchanged due to the data integration and data governance scope within SAP's cloud solutions.

The following are planned to be removed from SAP Data Services 2024:

AIX and Solaris operating systems

Application layer connectivity of reduced market share applications

Java based adapters for which we have already provided native connectivity

SAP Data Services and SAP Information Steward remains a key component of our Data Management portfolio and we are committed to our customers. Please help us fine tune our roadmaps of SAP Data Services and SAP Information Steward by providing your input through the dedicated inboxes respectively:

SAP Data Services: sap.dataservices@sap.com

SAP Information Steward: sap.informationsteward.enhance@sap.com

We appreciate your feedback and will answer your questions via our official communication channels

SAP Data Migration Cockpit - Nutshell

2023-02-19T20:45:52+01:00

Hello Learners,

Hope you all are doing Great!

Welcome back to my data migration blog series.

Purpose of this Blog :

With introduction of S4 HANA, SAP has introduced us to many new tools built in the system. Today we will talk about one such tool used for data migration in S4 HANA, SAP Migration Cockpit.

I will try to cover the details which I have got over the period by working on this tool.

What’s in it for you?

The blog will explain the different pieces of migration cockpit which will give you the very good start and understanding of tool.

Let’s get into the blog!

Overview -

Migration cockpit is SAP provided tool in S4 Hana for data migration purpose. This tool has different approaches for migrating data from SAP or Non SAP system through files, tables or direct transfer. Tool comes with built in content/objects for data migration like products, business partners etc.

Tool Access -

Tool is available to be accessed in GUI through tcode as well as FIORI app in recent S4 versions.There have been significant changes in features between both the options but the end to end process is same.

Tcode Way-

LTMC (Legacy Transfer Migration Cockpit) is the tcode to access the tool in SAP GUI, user interface will open up in interner explorer browser. (see screenshot) Using this interface you can work with projects you created or you can create new projects also.

LTMC Tcode

Fiori Way -

You can access the fiori GUI using tcode (/n/ui2/flp) or through direct link. Once the fiori is up and running you can look for app called "Migrate Your Data" to access migration cockpit.

Migration Cockpit main screen- Fiori

What to use from above?

Old versions including S4 hana 2020 and prior will allow you to use LTMC in full mode. SAP introduced fiori app from 2020 version. So 2020 version you will have both options available to you for using migration cockpit but versions released after 2020 will only allow fiori app method to access migration cockpit and LTMC will only be available as read only option.

Note : There is one more important tcode associated with migration cockpit which is LTMOM used for customization purposes, will have more details in the later section of blog.

Data Migration Approaches -

There are widely 3 migration approaches in migration cockpit-

File Based Approach -
- In this approach, SAP provides in built templates for the data migration for various sap data objects.
- This is suitable when you have less complex data migration requirements and volume is not huge.
- This supports migration from NON SAP/SAP to S4 Hana

Staging Tables based Approach -
- In this approach, staging tables are created in HANA DB for data objects which can be populated using ETL tool.
- This is suitable for any complex data migration requirements and for high volumes of data.
- This supports migration from NON SAP/SAP to S4 Hana

Direct Transfer Approach
- In this approach, direct connection is created between SAP system to S4 Hana system for data transfer

Based on data migration requirement for business suitable approach is selected.

To use any approach you would need to create project first and set it up correctly based on approach requirements. select the data objects as per scope and make sure dependecies are also taken care.

Data Migration Process -

4 step guided migration process is built in migration cockpit for carrying out the data migration.

Step 1- Validate Data - This step validates the data from templates or tables and throws any errors encountered in configs etc

Step 2 - Convert Values-

This step is used for performing value mapping for fields from source to target. You can view, edit any source to target value mappings for fields in use.

Correct mapping of all values is must in this step

You can export import value mappings when you have larget set of source to target values. LTMC generates semi colon separated csv file while exporting and it expects same format for import whereas fiori has predefined xml template for value mappings also which could be used.

Step 3 - Simulation of Data - Migration cockpit does data simulation before actual load to see if there any more data validation issues before the the final import of data

Step 4- Import Data -

This is the step where actually the data is migrated to system, you may see some errors on this step also

if any errors encontered in this step, those records with errors will be skipped and rest of the data will be loaded to system.

A delta file with failed records will be automatically created and would be available for download in case of file approach

Please see, the sequence of steps is imp and before moving to next step prior step should be completed.

You may notice various types of messages/notifications in each step like below -

Error, Warnings, Information etc.

LTMOM (Customization of Migration project)

LTMOM is very important piece of migration cockpit which works with both approaches LTMC and Fiori.

Using LTMOM tcode you can customize your data migration project for any migration apporach.

You can customize objects by adding, deleting fields or adding custom fields also

You can create your own objects from scratch

You can write complex logics for field mappings if required or you can create your own rules also.

Migration cockpit does not end here and has lot to offer specially with new fiori app but this is the nutshell of it and would provide a good start if you are trying to learn this. I will cover more advanced topics in upcoming blogs so stay tuned!

Please do share and subscribe the blog and share your valuable feedback so that I can improve my content and help readers get more better learning experience around SAP data migration and data quality.

Happy Learning and reading!! 🙂

Why is it Vital to Have a Data Catalog?

2023-02-28T21:17:01+01:00

Within this blog post we will define what a catalog is, SAP’s vision for a catalog, what data-driven decision-making means, and the importance of having a catalog.

What is a catalog?

The heart of a catalog is the foundation of data intelligence, it is the collection of metadata to serve as an inventory of your organization’s data. A present-day catalog is for everyone – technical and non-technical users, with the capability to share and collaborate with each other. Catalogs help users discover and preview, analyze, understand the quality and trustworthiness, and govern the data. A catalog gives key stakeholders the ability to identify actionable insights, allowing them to formulate data-driven decision making with confidence.

SAP’s vision for a catalog

SAP’s vision is that a “Catalog is a comprehensive solution that collects and organizes data and metadata to enable business and technical users to make confident data-driven decisions. Catalog improves productivity and efficiency by building trust in enterprise metadata through consistent data quality and governance. SAP vision is to build a catalog rich with SAP business applications data and provide an open ecosystem to integrate with leading 3^rd party catalog vendors to power data intelligence.”

What does data-driven decision-making mean?

A “data-driven” approach means you and your organization are using data to make informed and strategic decisions to drive business growth that aligns with your organization’s strategy. Making strategic decisions based off the organization’s data is truly realizing the full value of your data while removing any bias. Empowering all lines of business, such as, business analyst, data scientist, human resource manager, marketing managers, and so on to make smarter and more informed decisions every day based on the data.

Making data-driven decisions is important because it allows organizations to gain insights and predictions to reach their goals. Here are a few of the great benefits to making data-driven decisions:

Improved productivity and efficiency

Transparency; removing any predisposition and making confident decisions

Aligning with and achieving your organization’s strategy and goals

Becoming proactive; leveraging the data to identify new business opportunities and identifying risks and threats before they get out of hand

Increase collaboration among teams; everyone working towards the common goal

Why it’s vital to have a catalog?

As mentioned above a catalog is the heart and foundation for your data intelligence and is for all users within an organization to unlock the full potential of enterprise data and to make data-driven decisions with confidence. Here are five benefits of why it is vital to have a catalog:

Empower user to be data driven – with a unified view, easy access to all organizational data, and an understanding about the trustworthiness of the data allows users to spend their time analyzing and gaining insights to make decisions in line with the organization’s goals and strategies.

Increase data literacy – every organization has approved, and standardized vocabulary terms and concepts used daily in communication. Business glossaries promotes proactive data governance with common understanding and agreement on business terms and classifications, as well as promoting better communication. Granting users, the power to read, work with, analyze, and communicate clearly and concisely.

Improve data efficiency – easily, proficiently, and correctly locate the correct data set that will meet your needs. Saving you from spending excessive time trying to discover that correct data set. This is especially important when data is literally stored everywhere and in a variety of formats.

Boost trust in your data – offering the ability to preview and profile data – to validate the completeness, accuracy, and quality of the data. Ultimately ensuring trust in the data and delivering intelligence through data-driven insight and innovation.

Advance data compliance – governing data availability to ensure compliance with laws, regulations, and organizational policies around the data. Providing the ability to follow lineage and safeguard confidentiality and integrity around the data. Protect the data.

Conclusion

Data is an extremely valuable asset, that is, until it becomes a liability. To protect against the data becoming a liability, embody a data catalog. A catalog solution collects and inventories your data, giving you a holistic view of your data regardless of where it resides or what format the data is in. Catalogs provide meaningful insights about the data and permits you to make data-driven decisions from your trusted data. All while improving collaboration and communication within your organization. Catalogs are vital to your organization.

Be sure to join us on March 8, 2023 for our SAP Data Unleashed event, with Julia White, Chief Marketing and Solutions Officer and Executive Board Member, SAP and Juergen Mueller, Chief Technology Officer and Executive Board Member, SAP; Register today. This is your opportunity to be the first to hear about the latest SAP advancements – watch live demos, see new capabilities in action, and get a deep dive from customers on how they unleash the power of their business data across their data landscape.

SAP Data Services: Session SDK Failure Error while installing SAP Information Steward

2023-03-28T14:14:40+02:00

Overview

This blog refers to the Session SDK Failure Error which might occur at the SAP Data Services Designer Repository Login when installing SAP IS on top of SAP DS and SAP BI/IPS platform. The error popped when SAP IS Installation was triggered, everything had been working fine initially without SAP IS.

There are many solutions already provided to mitigate this issue. But when none of those solutions worked, there came a need to analyse the system in detail from the scratch. The problem is quite basic so at times may be overlooked or neglected. Thus, the purpose of this blog is to bring basics forefront and to highlight how this error is linked with the SAP IS installation.

Furthermore, the sections of this blog are divided as follows:

Description of the issue

Summary of analysis in terms of the various SAP KBA/SAP Note

System Application Details: An example

Root Cause

Resolutions

Abbreviations

A. Description:

When trying to log on to the SAP DS Designer, it throws the Session SDK Failure error.

Additionally, the login to CMC has also been disabled.

The SIA is also down and not running.

The CMS wouldn’t be starting.

The Database is up and running.

This error appeared when the installation of SAP IS was triggered.

Fig.1. Session: SDK Failure (Source: SAP Data Services Dialog)

Fig.2. CMC Error (Source: CMC Login Page)

B. Summary of analysis in terms of the various SAP KBA/SAP Note

Table 1. List of SAP notes related to Session SDK error

Sr. No.	SAP KBA / SAP Note	Description
1.	2151044	Could not reach CMS ‘hostname:6400’. Specify the correct host and port and check for network issues
2.	1952120	Best Practices & pre-requisites on Windows while Install/Update/Patching BI
3.	1862954	Unable to log in to Designer – Data Services 4.x
4.	1989019	Unable to log in to Data Services Designer
5.	2473022	unable to log in Designer with SSL - <Session: SDK failure> - SAP Data Services 4.x
6.	2450650	Get error when logging on Data Services Designer from Data Services client machine
7.	1897030	Cannot log into Designer using FQDN - Data Services 4.x
8.	1601655	Session: SDK failure error when trying to log on to Designer - Data Services

The summary of the resolutions provided from these SAP KBA/ SAP note were (please refer to the Table 1):

Perform manual tests to verify if the issue lies in the network (for example: ping, telnet).

The request port between SAP DS client machine and the CMS server machine might be blocked by firewall or other network issues.

Ensure whether the CMS can be accessed.

The database, server name, port numbers, CMS name, must all be synced up.

SSL protocol have not been configured on Client machine in order to communicate with BI/IPS environment which one is using SSL for any communication with the CMS.

Note: But none of the above helped with the error. Please refer the following sections for the root cause and the resolutions.

C. Application Details: An example

Fig.3. High-Level System Landscape

Consider the above figure (Fig.3.) as an example of a High-level System landscape. It depicts the underlying Repository, the SAP IPS/BI, SAP DS, and SAP IS in a single stack.

D. Root Cause

After conducting all the analysis mentioned over the SAP Note and KBA's. There came a need to start the analysis from the scratch and that's when this compatibility check came in which are mentioned over in the following subsections Part 1 and Part 2. Please refer the SAP DS and SAP IS PAM guides for the reference.

SAP DS 4.3 PAM

SAP IS 4.3 PAM

Part 1:

Fig.4. Compatibility Check for SAP BI/IPS and SAP DS

From the above compatibility table (Fig.4.) for SAP DS and SAP BI/IPS, the SAP DS v4.3 SP0 P01 is COMPATIBLE with BI/IPS v4.3 SP2 P01.

So, there is no problem in the compatibility of SAP DS and BI/IPS.

Part 2:

From the below compatibility table (Fig.5.) for SAP IS and SAP BI/IPS, the SAP IS v4.3 SP01 P03 is NOT COMPATIBLE with BI/IPS v4.3 SP2 P01.

So, there is problem in the compatibility of SAP IS and BI/IPS.

And that is why when the SAP IS installation has been triggered, the SIA went down and everything was down in accordance.

Fig.5. Compatibility Check for SAP BI/IPS and SAP IS

E. Resolutions

Knowing the Root Cause of the issue, the following two resolution options are being suggested.

Option 1:

Restoring from the backup the SAP BI/IPS and SAP DS without the SAP IS installation process.

After this, choosing the compatible version of SAP IS.

But the only compatible version is SAP IS v4.3 SP00 and whose state is inactive.

The term INACTIVE means that, ‘This particular DS/IS version is no longer in active development in accordance with the DS/IS Maintenance Strategy'.

So, this will be again a decision to make whether to go forward with it or not.

Option 2:

Starting from scratch.

Planning for all the three components wisely i.e., SAP BI/IPS, SAP DS, and SAP IS. Their versions, their compatibility based on the dependencies.

Also, choosing the platform i.e., SAP BI or SAP IPS should be used (please refer to the documentation on SAP Help for further information on this).

Conclusion

Make sure to do the version compatibility checks for any installation.

In case of any further recommendations, please feel free to add in the comments section so that it can reach out to the wider audience. Thanks.

Abbreviations

IPS	SAP Information Platform Services
BI	SAP BusinessObjects Business Intelligence
DS	SAP Data Services
IS	SAP Information Steward
CMS	Central Management Server
CMC	Central Management Console
SIA	Server Intelligence Agent
KBA	Knowledge Base Article

Call-to-Action

Please provide feedback about your view and experience related to this error or installation! Feel free to ask question by contacting me directly or commenting below.

Reference Links

Database and Data Management Community Content

Statement of Direction - SAP Data Services and SAP Information Steward

SAP Information Steward Upgrade Guide v4.3 SP02

SAP Information Steward Email Notification Utility

2023-04-07T22:25:05+02:00

Overview

How to activate e-mail alerts for utility service monitor?

Usage

E-mail notifications are preferred for the following processes:

Match review tasks

Rule tasks

Rules approval process

Terms

Policy sets

Policy statements

Scorecards

General Steps

Appropriate permissions are required to perform this task (For example: Administrator rights to CMC).

There are three steps which needs to be performed in CMC:

(Configuration of e-mail notifications)
(i) CMC Users and Groups: User e-mail address for notifications set-up.
(ii) CMC Servers: E-mail notification configuration set-up.

(Adding applicable e-mail addresses)
(iii) CMC Information Steward Application: Activating e-mail alerts for utilities.

(Optional) The final step has to be performed in SAP IS, if the default settings are not being changed.
- By default, the email notification is being enabled in SAP IS.
- SAP IS Manage: Set-up general e-mail notification preferences.

Part 1: Setting up user's e-mail address for notification (CMC Users and Groups)

Pre-requisite: An administrator rights of CMC is required to add user's e-mail information.

Steps:

1. Log in to the CMC with Administrator privileges.

Fig.1. SAP BusinessObjects CMC (Source: CMC Homepage)

2. From the Organize section (or dropdown menu) in the CMC homepage, select Users and Groups.

Fig.2. Users and Groups (Source: CMC Homepage)

3. Add a new user, provide all the details here.

Fig.3. Adding a New User (Source: CMC Homepage (Users and Groups))

Or go to the appropriate user name (from the User List) or group (from the group list) and click Manage. Now, go to the Properties option.

Fig.4. Selecting appropriate user or group (Source: CMC Users and Groups)

4. In the E-mail textbox, fill in the appropriate contact information.

Fig.5. Adding User's E-mail Details (Source: CMC Users and Groups)

5. Finally, click on Save & Close.

Next Steps: Recipient user's e-mail address has been successfully added for notifications. Once this step is completed, now the next step is about configuring the e-mail notification.

Part 2: Setting up e-mail notification configuration (EIM Adaptive Job Server Destination e-mail: CMC Servers)

Pre-requisites: The recipient user's email address must be added in the Users and Groups configuration from CMC. Ensure the appropriate permissions to CMC are present.

Steps:

1. Log in to the CMC with Administrator privileges.

Fig.6. SAP BusinessObjects CMC (Source: CMC Homepage)

2. From the Organize section (or dropdown menu) in the CMC homepage, select Servers.

Fig.7. Servers (Source: CMC Homepage)

3. On the left side, click on the Servers list. This will display a list of all the existing servers.

Fig.8. Servers Selection (Source: CMC Servers)

4. Now, from this Servers list, select the Adaptive Job Server.

Fig.9. Servers List (Source: CMC Servers)

5. Click Manage. Now, go to the Properties option and the respective dialog box will open.

Fig.10. Adaptive Job Server Selection - Properties (Source: CMC Servers)

6. From the left side, select Destination.

Fig.11. Adaptive Job Server Properties - Destination (Source: CMC Servers)

7. Here, a list of e-mail options will appear. Select the respective E-mail and click on Add.

Fig.12. Email Selection (Source: CMC Servers)

8. Add the E-mail related details here.

Fig.13. E-mail details (Source: CMC Servers)

9. Click on Save & Close.

Next Steps: Once the EIM Adaptive Job Server Destination Email has been configured with all the details (like SMTP domain, host, port, message content, etc) then the next step is about activating e-mail alerts.

Part 3: Activating e-mail alerts for Utilities (CMC Information Steward Application)

Prerequisites: Setting up recipient user's e-mail and configuring the EIM Adaptive Job Server destination e-mail.

Steps:

1. Log in to the CMC with Administrator privileges.

Fig.14. SAP BusinessObjects CMC (Source: CMC Homepage)

2. From the Manage section (or dropdown menu) in the CMC homepage, select Applications.

Fig.15. CMC Applications (Source: CMC Homepage)

3. From the Applications menu, double-click on Information Steward Application.

Fig.16. CMC Information Steward Application (Source: CMC Applications)

4. On the navigation pane at the left side, select the 'Configure Application'.

Fig.17. CMC Information Steward Configure Application (Source: CMC Applications)

5. Scroll at the bottom most part where you can find the Email settings.

Fig.18. Email Settings (Source: CMC Applications)

6. Now, in the To Email Address option, enter the recipient user's email address.

7. Finally, click on Save.

(Optional) Part 4: Setting general email notification (SAP Information Steward)

Pre-requisite: Please follow these steps, if someone has changed the default settings in SAP IS and if you would like to enable email notification.

Steps:

1. Go to SAP Information Steward application.

Fig.19. SAP Information Steward Application (Source: SAP IS URL Page)

2. After successful login, go to right-hand side of the SAP IS homepage. Select Manage and click on Preferences.

Fig.20. SAP IS Homepage (Source: SAP IS)

3. Now, scroll to the Email Notification Settings section, select the appropriate option for the Send Notifications. Also, from the Locale menu, select the option. This setting is being done to determine the language in which the email is written. And the language options are being available and set up during the installation.

Fig.21. Email Notification Settings (Source: SAP IS)

4. Click on Save.

Abbreviations

CMC	Central Management Console
SAP IS	SAP Information Steward

Call-to-Action

Please provide feedback about your view and experience related to this E-mail Utility Configuration In SAP Information Steward! Feel free to ask question by contacting me directly or commenting below.

Reference Links

Database and Data Management Community Content

SAP Information Steward Help Page

Data Strategy with SAP - Data Governance

2023-06-03T13:42:25+02:00

Currently I see many topics in my projects, where the right Data Governance would do a good job. So in this blog today, there are no fancy graphics or a to clear structure as I mainly write down things just going through my head at the moment. Happy to share them with you and possibly get into discussion about.

It is always helpful to start with a definition:

Data governance is the specification of decision rights and an accountability framework to ensure the appropriate behavior in the valuation, creation, consumption and control of data and analytics.

Source: Gartner, 2023

While I would say Data Governance is not just about data and analytics, it becomes typically pretty relevant in this area. If you have a data strategy in place and there is no word about Data Governance, you finally have NO data strategy.

As most tech people (I assume you are more tech than business here) are more interested in tools than processes and organization, let's start with the tool side.

In Data Governance we find typically these solutions:

Data Quality Management - e. g. SAP Information Steward

Master Data Management - e. g. SAP Master Data Governance

Metadata Management - e. g. Data Catalog in SAP Datasphere or SAP Data Intelligence but also technical solutions like SAP PowerDesigner

Business Glossary Management - e. g. often also connected to Data Catalog solutions but for SAP context bluetelligence Enterprise Glossary is also a good example

Data Catalog Management - see Metadata Management and Business Glossary Management

Data Lineage Management - e. g. Manta but can also be found in the context of Data Catalog solutions, strongly connected with Data Lineage is Impact Analysis

Now, let's come to the motivation. A lot of companies are aware, that Data Governance is important. Typically they faced some situations where they learned that one or the other solution categories listed here, make sense because of security breaches, bad decisions based on poor data quality, long development processed for data and analytics projects due to missing transparency of data, long discussions about specific meaning of an KPI of why we have so much different KPI values with the same name. And at the latest if you hear yourself askting "Who is responsible for … requirements, implementation, data assets, data protection, data quality, KPI definitions" often - Data Governance is there for you!

Some further reasons for Data Governance are:

If a regulatory compliance is very important, you should already have a good Data Governance in place.

Decentralization and more and more autonomuous functional areas and business units additionaly drive the need for a federated governance.

You want to foster Self-Service BI, Data Literacy and Data Democratization in general - lead soon to chaos, high costs and bad decisions without a good Data Governance strategy and strong principles how to handle data within your company.

Maybe these challenges make clear, Data Governance makes sense and, done right, saves a lot of money or even helps earning more money e. g. via a better customer satisfaction. A data-driven company runs on the right Data Governance and is unthinkable without. To find the right balance and way is the art to master as there is not the one way.

So far, we see we should care about Data Governance. But why are many companies do not? Indeed there are several reasons, why it can be a little bit complicated:

There is possibly no initial value - Who pays for a project (while Data Governance shouldn't be just a project), if no direct value can be seen? Most company just start after something happens with a high impact on business.

It is driven by IT only or tool-oriented ("let's buy xyz for data quality" or "a data catalog for being data driven") - Business involvement is key and many IT-driven initiatives die very fast, if you can not find together.

It is driven by business only - Business department often starts from their perspective, but often miss a holistic view and are not aware, what is possible to do with technology today.

You want it all and you want it now - Boiling the ocean kills many initiatives. You are maybe motivated doing the right thing, but doing to much and to fast, so that your organization can not follow and typically it gets to expensive.

Some recommendations from my experience:

Think big, start small - Identify important data domains for your business (often customer or product) and start here. Learn during the way.

See the change necessary and take the people with you - You will need new roles, new processes, the right tools and the right collaboration model between IT and business.

Identify the right use cases and the value behind - Start with the most valuable first and build lighthouses.

Clarify the mandat - If you have to convince all the people, you will need forever to establish a Data Governance. To rule them all on the other side, is typically also not the way and leads to low acceptance and resistance.

Many things can be said about Data Governance. It is worth to care about if you want to create value from your data. To do the right things the right way is important.

What is your experience with Data Governance? Do you care about?

Data Quality Audit

2024-02-21T12:46:34.652000+01:00

One of the indispensables, essential and dominant parts of our lives is our smart phone. You can almost say there is nothing a smart phone cannot do nowadays. However, technological advancements do not stop there, we have continuous innovations been introduced into our surroundings.

Over 20 years, we saw the first mobile phones with antennas that evolved into faster, lighter, graphical, and more intuitive devices. This is a splendid example of technological transformation.

We started off with just voice communication over analog signals which transformed into so much more. We can now access the world’s news and information at our fingertips, communicate and network within our circle - even with strangers based on our geographic location - shop, trade, manage finances and organize our lives globally from wherever we may be.

As mobiles get smarter and smarter, think about the data that has been transferred from one device to another. And these transfers came with several implications and adjustments to the data over the years. For instance,

Compliance wise – with the onset of international calling, phone numbers were suddenly required to be stored in international format.
Competition wise – amongst varied brands, more features kept pouring in terms of speed, additional fields, and memory to store more contacts and addresses.
Looking at technology trends – devices now have the power of internet telephony, connecting themselves to self-driving cars and comprehensive mobile applications to become a pocket computer.

Data quality is the least of our concern as we are focused on the outcome to move to the new device and get started, by just ensuring that we retain the data in the new device in some form. The advanced intelligent machines assume implicitly that the data transferred to them is suitable for their intended purposes. We all know that transfers across platforms and vendors are limited by data models and technical limitations.

Consider this scenario: during the transfer of data between different device platforms, envision a situation where contact numbers or addresses are jumbled. You may have been consuming this data over years across devices manually, knowing the limitations and issues in the data, and deciding accordingly e.g., Mobile phone and Office phone fields have data interchanged during transfer, and postcode missed in the addresses during transfer from Apple to Google platform. Now, imagine relying on this data on your mobile device with a self-driving car to navigate you to your destination e.g., AI considering only street name to navigate and deciding between multiple options!

How can you trust the data that has been moved from your old devices into the newest device via multiple devices and platforms?

Would you be able to use the full capability of the new AI self-driving cars?

Is the data that we have been maintaining for decades fit to be used in these innovations without human intervention?

Or more simply, is the data fit for the new intended purpose?

Now coming back to the corporate world and considering the business point of view for customers who have moved their enterprise data assets across multiple platforms over last few decades to SAP ECC or any other ERP system. Most of these customers are now in the process of transforming their technical landscape and business processes to get ready for the future, i.e., Cloud! Whether these organizations are moving to private or public cloud, RISE with SAP or GROW with SAP, going greenfield or brownfield; the transition mirrors the evolution we discussed earlier with mobile phones but, in this case, businesses are shifting from their familiar ERP ECC systems to the futuristic landscape of SAP S/4HANA. It involves migrating from mixed landscapes, including SAP, non-SAP, and legacy systems, into the innovative SAP S/4HANA platform. The AI and Generative AI capabilities embedded in SAP S/4HANA mimics driverless cars we mentioned before. So, we find ourselves asking the same questions once again:

Does the data make sense for the S/4HANA system? Can we take advantage of the GenAI capabilities?

Can we rely on and trust the data for the system to predict your business outcomes or take business decisions?

In the end, is the data fit for intended purpose?

Data quality (DQ) is always the key focus and concern from the business at the beginning of the transformation, however as program evolves, DQ topic fades as the business priority moves to ensure processes are designed and tested. The challenge is that no one could really quantify and establish the problem to define a business case to invest in data cleansing to save costs for the organization. By the time business users realize the scope of issues through migration, it is too late to fix this problem and they are already testing the processes. If at the end they are successful in migration, they still end up with technically correct and not “business correct” data set.

This helps us see those businesses, like individuals with mobile phones, need to make sure their data is ready for the changes and innovations in their systems.

How do we ensure data is fit for purpose?

Let us address this topic again with the same mobile phone analogy. Consider this scenario: you are moving from an old Nokia phone to an iPhone. The former mobile phone saved contacts separately and did not have any tagging features to link landline or mobiles numbers of the same person together. However, the new iPhone has the feature to add multiple numbers with tags to a single contact. Now when you are moving to the new phone quite literally, the contacts can become redundant in your new device and cause confusion. How would you be able to tell apart which number is landline and which is mobile?

Similarly, what if some mobile numbers were transferred incorrectly and lead to missing information in your contacts? Or during transfers and conversion of formats to comply with international codes, incorrect patterns were transferred, or the numbers truncated?

This would affect your new device’s ability to function as expected and for you to utilise innovative features on your contacts.

Many such examples can come up in various parts of the transfer journey between the devices especially when you are moving between technically diverse landscapes. How do we ensure we do not move junk or bad data into our systems? Or if we already have moved without auditing the data first, how can we identify inferior quality data in our current landscapes?

The first step to check if data is fit for purpose is to understand the depth and extent of your data issues. To understand your data issues, you need to audit the data. The data may appear clean, but the question is if it’s fit for the intended purpose.

In the business world, the information spread across your systems is undeniably the most valuable asset. Achieving 100% data quality is a challenging task for any organization, but depending on how crucial each aspect is to the business, you can prioritize and make it feasible over time. For example, if there are 100 fields or attributes that require attention, and only 70% of them are essential for your business operations, you can identify key areas to concentrate on and begin planning for improvement. So, where do we begin?

What is Data Quality Audit?

Data Quality Audit, a service offered by SAP, assesses, and checks the actual data stored in systems to uncover the existing issues and determine fitness for target systems like S/4HANA, C4C, ARIBA, MDG etc. Irrespective of the number of systems you have, or at which stage you are in your transformation journey, Data Quality Audit checks the ‘quality’ of the data in terms of ideal principles set by your business and generates a ‘scorecard’ along with a business impact ‘cost analysis’. This allows organizations to assess the status of their data and then help formulate their cleansing, resource and migration plans accordingly.

Let us briefly look at the steps that this service follows to achieve the results:

Once scoping is established, data quality ‘rules’ are defined on the data with business or IT. The ‘rules’ originate from ideal principles of data quality: Accuracy, Completeness, Conformity, Consistency, Integrity, Timeliness, Uniqueness. SAP offers best practices content of data quality rules as a QuickStart to any quality audit.

These ‘rules’ are then executed on the data, and ‘scorecards’ are established in a dashboard-like view. The scorecards also depict the associated cost impact analysis for the poor data quality. This then helps enterprises to assess their data, constantly review their data during the cleansing activities, supporting control and governance.

To conclude,

Data Quality Audit is a service that is pivotal in the initial 'Get Clean' phase of your transformation. Unfortunately, these aspects are often underestimated, leading to the failure or delay of numerous transformations. Taking these steps seriously is crucial, as neglecting them can jeopardize the success of your large-scale investment, impede growth, and hinder the identification of business issues.

Consider these three key takeaways:

Recognize the vital role of Reference Data Quality Audit and Data Quality Audit in achieving a clean transformation. Look out for a similar blog on Reference Data Quality Audit - stay tuned!
Be mindful of the commonly underestimated nature of these steps, as it can significantly impact the success of your initiative.
Formulate robust 'Stay Clean' and ‘Get Clean’ plans based on the insights gained, avoiding pitfalls that have derailed many transformations."

SAP Community - SAP Information Steward

#DataToOucome Chief Data Officer (CDO) and Data Governance Sessions are Available On Demand!

Shared Terms – A Hybrid Use Case with SAP Data Intelligence and SAP Information Steward

DATA PROFILING OVERVIEW – SAP INFORMATION STEWARD

Uniqueness Profiling Overview- SAP Information Steward

SAP Data Intelligence Expert Services Q2-Q3 2021

Dependency Profiling Overview- SAP Information Steward

Redundancy Profiling Overview- SAP Information Steward

SAP Data Services, SAP Information Steward, and SAP Data Intelligence: strategy, maintenance, and future vision of SAP Data Management products

Building a data quality pipeline using SAP Information Steward

The main data quality issues

Analyzing SAP objects in SAP Information Steward

SAP Data Services 4.2 SP13 Upgrade steps

Creating Data Quality Rule In SAP Information Steward

Enterprise Data Analytics & Data Management: A reference architecture

Target Audience

BUILDING BLOCKS

SUB-DOMAIN AREAS

USE CASES

SUMMARY

Call-to-Action

Additional Assets

Week 6: Why you should move your data management to the cloud!

INTRODUCTION: DATA MOVES TO THE CLOUD

CLOUD DATA MANAGEMENT FOR CLOUD DATA

DATA MANAGEMENT IN THE CLOUD: CHANGE IS REQUIRED!

THE GOAL: CLOUD DATA MANAGEMENT FOR ALL DATA SOURCES

GO "HYBRID" IN DATA MANAGEMENT

SAP SOLUTIONS FOR HYBRID DATA MANAGEMENT

GET ENGAGED!

eMail us to arrange a Cloud Data Management workshop appointment!

SOMETHING TO READ?

Do you also want to give more purpose to your data?

Statement of Direction - SAP Data Services and SAP Information Steward

Update as of November 2023 - Please refer to the latest Statement of Direction that overwrites this blog post that was published in September 2022

SAP Data Services 2024 and SAP Information Steward 2024

SAP Data Migration Cockpit - Nutshell

Why is it Vital to Have a Data Catalog?

What is a catalog?

SAP’s vision for a catalog

What does data-driven decision-making mean?

Why it’s vital to have a catalog?

Conclusion

SAP Data Services: Session SDK Failure Error while installing SAP Information Steward

Overview

Description of the issue

Summary of analysis in terms of the various SAP KBA/SAP Note

System Application Details: An example

Root Cause

Resolutions

Abbreviations

A. Description:

B. Summary of analysis in terms of the various SAP KBA/SAP Note

C. Application Details: An example

D. Root Cause

Part 1:

Part 2:

E. Resolutions

Option 1:

Option 2:

Conclusion

Abbreviations

Call-to-Action

Reference Links

SAP Information Steward Email Notification Utility

Overview

Usage

General Steps

Part 1: Setting up user's e-mail address for notification (CMC Users and Groups)

Part 2: Setting up e-mail notification configuration (EIM Adaptive Job Server Destination e-mail: CMC Servers)

Part 3: Activating e-mail alerts for Utilities (CMC Information Steward Application)

(Optional) Part 4: Setting general email notification (SAP Information Steward)

Abbreviations

Call-to-Action

Reference Links

Data Strategy with SAP - Data Governance

Data Quality Audit

THE GOAL:
CLOUD DATA MANAGEMENT FOR ALL DATA SOURCES

e Mail us to arrange a Cloud Data Management workshop appointment!