How to generate reports in the Data Clinic?

Overview

Use Case

This use case describes the recommended use of CDQ Cloud Services for generating reports in Data Clinic cloud app.

Learning Goals

In this tutorial, the user will focus on the following:

  • Activating data sources that can be searched
  • Creating a configuration for the Augmentation Monitor
  • Creating the Augmentation Monitor
  • Creating a configuration for the Data Quality Profiling Monitor
  • Creating the Data Quality Profiling (Validation) Monitor
  • Generating reports in the Data Clinic app
Remember

This tutorial is based on local data and specific naming. The presented results will be different from yours.

Step 1: Login

To set up the Augmentation Monitoring, you have to log in to the CDQ Cloud Apps.

image01login

Click on Sign In and enter your username and password.

image02login

Step 2: Activating Data Source

CDQ is integrated with a variety of data sources, ranging from business and tax registers to commercial data providers such as D&B or Moody's. For a comprehensive list of all available data sources, please refer to a list of Data Sources.

Before these data sources can be used in CDQ products, they must be explicitly activated. This activation is a one-time process for the entire organization, and once completed, the activated data sources can be used across all workspaces.

  1. Navigate to the Global Settings,

img001

info

Here is a list of all supported data sources. You can filter the data sources by country of origin, name, and activation status.

  1. Click on the Settings icon activated required data source,

img002

attention

Some data sources may require authentication. Depending on the authentication mechanisms of the data source, you will need to enter your credentials, tokens, or API keys.

  1. Modify the settings:
    • Provide the necessary credentials (if needed),
    • Activate a data source using the activation toggle,
    • Click the Save credentials button,

img003

Data sources are not unlimited and are subject to quotas. The organization level quota can be controlled by CDQ or your organization itself (if you authenticate with your credentials, e.g., D&B).

Additionally, quotas can be assigned to each workspace. This helps to regulate the use of data sources and prevents unintentional and expensive downloads of data from commercial providers such as D&B.

img004

Step 3: Creating an Augmentation Configuration

The Business Partners monitoring required setting up a configuration:

  1. Sellect Augmentation Configurator cloud app,

img005

  1. Click the Create new configuration button:
    • Name the new configuration,
    • Click the Create button,

img007

  1. Check the Augmentation Configurator table if every configuration has a unique number assigned
  2. Select the newly created configuration for the edition,
info

Note that you can only select data sources that you have activated globally for your organization. See Step 2.

img02

  1. Scroll down to activate Reference Data Sources section,

img008

  1. Select data sources for Lookup to define which sources should be used for augmentation,

img009

  • Use the Lookup checkbox to choose all data sources displayed on the page

img0010

  • Use All Lookup options on all pages checkbox to choose all available data sources (all pages)
attention

Using these checkboxes can activate commercial (paid) data sources. Make sure only required data sources are activated.

  1. Go to the Curation settings tab
  2. Choose reuired profile based on the table below for Curation Profile.

img0011

  1. Click the "Save configuration" button

Curation Profiles

info

A list of all curation profiles and features is described here: Curation API profiles

Report Name Report Description Monitor type Requirements
Data Defects The report outlines all errors found within the dataset, presented line by line with brief violation messages, helping businesses identify critical issues in Business Partner Data. Data Quality Profiling Global settings: Activated Reference Data Sources.

Validation configuration: Choose EU_VAT_QUALIFICATION or WORDLWIDE_IDENTIFIER_QUALIFICATION - or add to STANDARD profile qualification feature.

Monitor turned on
Defective Records The report provides Business Partners with their validation status and the number of rules violated per each record. Data Quality Profiling Global settings: Activated Reference Data Sources.

Validation configuration: Default or custom rules for required countries. Profile: STANDARD/ADDRESS.

Monitor turned on
Identifier Qualification For a given Business Partner, the qualification is made for name, address, and identifier, whether they are really associated with this identifier. Data Quality Profiling Global settings: Activated Reference Data Sources.

Validation configuration: Choose EU_VAT_QUALIFICATION or WORDLWIDE_IDENTIFIER_QUALIFICATION - or add to STANDARD profile qualification feature.

Monitor turned on
Identifier Qualification Per Decision For a given Business Partners attribute, the company's name and address are checked, whether they are really associated with this identifier. Data Quality Profiling Global settings: Activated Reference Data Sources.

Validation configuration: Choose EU_VAT_QUALIFICATION or WORDLWIDE_IDENTIFIER_QUALIFICATION - or add to STANDARD profile qualification feature.

Monitor turned on
Update Report The report outlines all individual updates for a given Business Partners attribute made according to changes in the reference source. Augmentation Global Settings: Activated Reference Data Sources.

Augmentation configuration: Activated Reference Data Sources for update monitoring, Business Partner from mirror subscribed.

Monitor turned on
Natural Person Screening For a given Business Partner in-depth analysis of natural person screening is made as well as politically exposed person identification results. Augmentation Global Settings: Activated Reference Data Sources.

Augmentation configuration: Activated Reference Data Sources for lookup.

Curation Profile: NATURAL_PERSON_SCREENING or added FeatureOn: Enrich Categories.

Monitor turned on
Address Curation Report that offers insights into the results of address cleansing activities (standardization, enrichment, cleansing, translation, and geo-coding of addresses). Augmentation Global Settings: Activated Reference Data Sources.

Augmentation configuration: Activated Reference Data Sources for lookup.

Curation Profile: AddressOnly or added Curation featuresOn from this [list](https://meta.cdq.com/API/DataCurationAPI/Profile/ADDRESSONLY).

Monitor turned on
Legal Entity Report that provides full information about BP found in reference data sources inc. status, registered name, legal form, legal address, and tax identifiers. Augmentation Global Settings: Activated Reference Data Sources.

Augmentation configuration: Activated Reference Data Sources for lookup.

Curation Profile: Business Partner Only or added Curation featuresOn from this list.

Monitor turned on
Data Mirror Dump Provide simplified information about Business Partners in the mirror + raw record. Not required Business Partner uploaded and correctly mapped in the Data Mirror.
Subscription Report Return information if Business Partners were linked with reference data sources. Augmentation Global Settings: Activated Reference Data Sources.

Augmentation configuration: Activated Reference Data Sources for updates monitoring.

Monitor turned on
Overlap Report Provides a list of 10 first lookup results for each Business Partner based on input data. Augmentation Global Settings: Activated Reference Data Sources.

Augmentation configuration: Activated Reference Data Sources for lookup.

Curation featuresOn: PERSIST_LOOKUP_RESULTS, suggested.

Curation Profile: Golden Record or Standard.

Monitor turned on

Step 4: Augmetation Monitor

Create an Augmentation Monitor and activate it using the previously created configuration:

  1. Go to the Data Clinic cloud app,

img0028

  1. Click the Add New Data Monitor button,

img0012

  1. Configure new monitor:
    • Select Data Sources for which the monitor should be turned on,
    • Select Augmentation for Monitor Type,
    • Choose just created augmentation configuration,
    • Select Validity span (optional),
info

Thanks to Validity Span setup, the Business Process will be refreshed if no reevaluation trigger occurs within the selected period.

img0013

  1. Click Create New Data Monitor button to save
  2. Wait until the job for the new monitor is finished

img0014

Step 5: Data Validation (Quality Profiling) Configuration

Create a configuration for the Data Quality Rules Engine used for monitoring:

  1. Select Data Validation Configurator cloud app,

img0030

  1. Select the Create button,

img0029

  1. Create new configuration:
    • Provide a name for the configuration,
    • Click the Create button,

img0015

  1. Check the Data Validation Configuration table below,
info

Each configuration has a unique number assigned.

  1. Select the newly created configuration for an edition and scroll down,

img0016

  1. Edit the parameters in the "Details of Standard configuration" section:
    • Select Validation Profile from the list based on report requirements,
    • Set the Rule Status to Release ,

img0017

  1. Use "Data Quality Rules" section to limit the validation results (optional):
    • Select the country,
    • Set the Rule Status to Release ,

img0018

img0019

It's possible to modify the criticality of selected rules, customize violation messages, or disable rule execution.

img0020

  1. Click the "Save changes" button.

Step 6: Data Quality Profiling Monitor

Create a Data Quality Profiling Monitor and activate it using the configuration that was previously created:

  1. Go to the Data Clinic cloud app

img0028

  1. Click the Add New Data Monitor button
  2. Configure new monitor
    • Select Data Sources for which the monitor should be turned On
    • Select Data Quality Profiling for Monitor Type
    • Choose just created validation configuration
    • Select Validity span (optional)
info

Thanks to Validity Span setup, the Business Process will be refreshed if no reevaluation trigger occurs within the selected period.

img0021

  1. Click Create New Data Monitor button to save
  2. Wait until the job for the new monitor is finished

img0022

Step 7: Generate the report

  1. To generate the report, go to the Reports tab of the Data Clinic cloud app,

img0023

  1. Click the Generate new report button,
  2. Configure the report:
    • Provide Report title,
    • Choose Natural Person Screening Report for a report type,
    • Select data sources (the same as for Data Monitor in case of reports based on Data Quality Profiling or Augmentation Monitors),
    • Create a report only for specific countries, if needed,
    • Choose a file format (optional)

img0024

  1. Click the „Generate” button to start a job with the report creation
  2. Wait until the job is finished

img0025

  1. Download the report;

img0026

  1. Open the report to see the results:

img0027

CONGRATULATION

You have generated the Data Quality report.


Your opinion matters!

We are constantly working on providing an outstanding user experience with our products. Please share your opinion about this tutorial!

Mail our developer-portal team: developer-portal@cdq.com