Tutorial: How to monitor Business Partner updates?

Overview

Use Case

This use case describes the recommended use of CDQ Cloud Services for establishing continuous data quality monitoring for Business Partner data.

MBPDQ Overview

info

A complete use case description is available on the Wiki.

Learning Goals

mbpdq process

In this tutorial, the user will be focused on:

Remember

This tutorial is based on local data and specific naming. The presented results will be different from yours.

Prerequisites

Login

Log into the CDQ Cloud Apps

Authorization

If you have no access to the CDQ Cloud App, ask your internal point of contact to create a CDQ dedicated account. Account detail will be sent by email.

Data Mirror

info

To proceed with this tutorial the Data Mirror is needed. Go to this page to get more information.

Step 1: Data Quality Monitor Configuration

Create a configuration for the Data Quality Rules Engine that is used for monitoring

Application to be used

Data Validation Configurator - allows customizing the validation process by certain rules and parameters.

  1. Pick Data Validation Configurator from accessible apps
  2. Click create for a new configuration
  3. Name the new configuration Standard
  4. Click create
createconf
  1. Check the Data Validation Configuration table below. Every configuration has a unique number assigned.
conflist confid
  1. Select newly created configuration for an edition
  2. Edit the parameters in the "Details of Standard configuration" section:
    • select Standard Validation Profile from the list
    • set the Rule Status to Release
Validation profile Rule status
profilesetup rulefilter
info

To read more about profiles visit this wiki page

  1. Limit validation results with the county filter in the "Data Quality Rules" section
    • Use filter by country and set DE-Germany
    • Set Release state of the rule
Setting filter Rule state
countryfilter rulefilter
Congratulation!

You have created a configuration for the Data Quality Rules Engine!

Create a data quality monitor and activate it using the previously created configuration

Application to be used

Data Monitoring Configurator - allows schedule regular running jobs on the data mirror.

  1. Pick Data Monitor Configurator from accessible apps
  2. Click Add New Data Monitor
newmonitor
  1. In the creation windows pick:
    • Data Mirror created before ( Prerequisites )
    • Configuration created in the Step 1
    • Component from the list
    • Schedule plan
    • Click Create New Data Monitor
createmonitor

Thanks to the schedule set during creation, the validation process of the pointed data mirror will run automatically. The report will be sent directly to the mailbox to be assigned to the user.

Congratulation!

You have created a data quality monitor!

Step 2: Data Quality Measurement Results Analysis

Analyze Data Quality in the Data Quality Dashboard

Application to be used

Data Quality Dashboard - allows for UI guided analysis of the data quality performance, customizing dynamically result reports and enables to track the progress over time.

To analise the quality of the data, it is important to prepare quality dashboard:

  1. Select your workspace where the desired data source is located
DQD workspace
  1. Open the filter by clicking on the icon in the upper-left corner of the page
DQD filter
  1. Set data source and select the countries you want to validate the data for
  2. Select Apply button
DQD filtersettings
  1. Check the data in the updated dashboard
DQD dash1
  1. Check the Number of Distinct Records and Records with ERROR Violation increase/decrease indicator compared to previous data quality monitor run
Number of Distinct Records Records with ERROR Violation
DQD changeindi DQD violation
  1. Verify the rest of the charts
Congratulation!

You have set dashboard for displaying desired data for visual analysis!

Download Data Quality Reports

Application to be used

Batch Data Validation - run a data quality profiling on data storage and receive data quality reports.

Batch Data Validation app will be used as a report reader-generated from created data monitor. When the job based on the Data Monitor schedule starts the status of completion will be visible in the Data Validation Jobs.

  1. Validate the job status:
Job ID Job in progress Job is done
jobid jobinprogress jobfinished
  1. If it's finished, scroll down to the Data Validation Reports
valreps
  1. Check the completion status of the report
  2. Select Show Results
showres
  1. Select one of the results provided by the job and download it
download
Pop-up Window

Remember to allow the browser to display Pop-ups

The Job result page provides a list of all generated reports and the Dashboard with charts.

list
Congratulation!

Now you have the report of your data quality!


Your opinion matters!

We are constantly working on providing an outstanding user experience with our products. Please share your opinion about this tutorial!

Mail our developer-portal team: developer-portal@cdq.com