SAP ODM: How to do bulk matching and enrichment?

Overview

Bulk matching and enrichment is a process that allows you to match a large number of Business Partners against the CDQ database. This process is useful when you have a large dataset of Business Partners that need to be matched and enriched.

Learning Goals

In this tutorial, you will learn how to:

Prepare input data using standard SAP Business Partner objects.
Perform bulk lookup to match Business Partners against the CDQ database.
Review the candidate list and select the best candidates for further processing.
Download and enrich matched records.
Load data into the target environment.

Pre-requisites

Authorization

Before trying CDQ APIs, user must be authenticated:

Paste the API Key in the console's security bar into the X-API-KEY field.

After pasting the API Key, the green padlock will appear.

Be careful

Green padlock doesn't mean that the API Key was pasted correctly.

Check your API key for missing characters or extra space before trying.

No API Key?

Check how to get one on authentication page
Follow the steps above.

Step 1: Prepare Input Data Using Standard SAP Business Partner Objects

Gather a list of Business Partner addresses and identification numbers, and ensure that they are structured correctly for processing. Use the standard SAP Business Partner objects to format the Business Partner Master Data. To facilitate the matching process, it is essential to install the necessary content pack in the CDQ environment. After installation, the customer can pass the data to the CDQ system. The matching process will then use all available data sources configured within the CDQ Cloud Apps.

info

For further details, you can refer to the documentation provided here

Step 2: ODM Bulk Lookup

Bulk Lookup is a process that allows you to match a large number of Business Partners against the CDQ database. This process is useful when you have a large dataset of Business Partners that need to be matched and enriched.

Create Bulk Lookup

Initiate the matching process by creating a Bulk Lookup job. The collected list of Business Partner data, formatted according to SAP standards, is sent to the CDQ system where the matching occurs.

Use the Create Bulk Lookup endpoint,
Use Create Bulk Business Partner Lookup with Golden Record example,

Features to be used in Bulk Lookup:

Feature	Description
`SHOW_GOLDENRECORD_STANDARD`	Activates the Golden Record.
`GOLDENRECORD_INLINE_SORTED`	Sorts the Golden Record by its matching score within the list of other hits.
Optional: `GOLDENRECORD_INLINE`	to always have the Golden Record appear at the top of the list.

Send the request,

Check the response:
- Save id for the next step,
- See the status of the job.

Copy

Copied

{
    "id": "fa6a7348-98e6-480b-9ff2-fa3a3432a7c8",
    "status": {
        "code": 200,
        "technicalKey": "OK",
        "details": [{
            "id": 200000006,
            "message": "The bulk is currently being executed.",
            "technicalKey": "BULK_RUNNING"
        }
    ]
  }
}

info

For more details check the endpoint's documentation

Read Bulk Lookup

Monitor the progress of the matching job as it runs on the CDQ side. Use the Read Bulk Lookup function to track the completion percentage of the job.

Use the Read Bulk Lookup endpoint,
Adjust the Parameter section:
- Use id from the previous subchapter,
- Make sure that startAfter value is set empty,
Send the request,

Get the results of the matching job once it is completed.

Additional information about pagination.

attention

The results are provided in pages, with each page containing up to 1,000 records. This pagination is managed by the limit parameter within the request. To retrieve later pages, you must set the startAfter parameter in the next request to the value of nextStartAfter from the previous request. This allows you to get all matching responses in batches until the entire dataset is retrieved.

info

For more details check the endpoint's documentation

Step 3: Review Candidate List

To select the best candidates for further processing, review the list of matched Business Partners.

Prioritize the CDQ Golden Record for its comprehensive data summary.
Avoid selecting low-scoring candidates. Define an acceptable score threshold with guidance from your CSM.
If needed, prioritize international sources based on specific corporate requirements.

attention

For better readability, this candidate list was extracted from the Bulk Lookup API into a table. It is not the default output of the API. Choose your desired candidate or refine the lookup and start over.

customerNumber	matchingScore	cdqId
1234567890	0.99	GR.DE.RC:B8537_HRB739690
2345678901	0.98	GR.PL.NOBR:8992790965

Step 4: Download and Enrich Matched Records

Once the best candidates have been selected, the next step is to download and enrich the matched records.

Execute ODM Bulk Fetch

The Bulk Fetch process requires the candidates from the previous steps got with Create Bulk Lookup and Read Bulk Lookup endpoints. The output of these methods, combined with the sourcing strategy, generates a list of identifiers for each chosen hit or the address data itself, along with the identifier details provided in the lookup output.

Use the Create Bulk Fetch endpoint,
Use Create Bulk Business Partner Fetch with Golden Record example,
Send the request,

Check the response:

Save id for the next step,
See the status of the job.

Additional information about the fetch strategy.

info

Using CDQ-IDs: The simpler approach is to feed the "Create Bulk Fetch" function with a list of CDQ-IDs provided by the lookup results. Each CDQ-ID should be specified as cdqId in the company enrichment request for Create Bulk Fetch. This method directly leverages the identifiers provided by CDQ, making the process straightforward.
Using Addresses and Identifiers: In scenarios where you need to pass addresses and identifiers instead of CDQ-IDs to the fetch function, the CDQ-ID must not be handed over to Create Bulk Fetch. Instead, a fetch strategy needs to be specified. It is recommended to use the "USE_FETCH_STRATEGY_GOLDEN_RECORD" strategy, which calculates an updated version of the CDQ Golden Record based on the given data and all available data sources.

info

For more details check the endpoint's documentation

Monitor and Receive Enriched Data

The first is to monitor the progress of the fetch job as it runs on the CDQ side. By using the Read Bulk Fetch function, you can track the completion percentage of the job in real time, ensuring you are aware of how far along the process is and when you can expect the results.

To monitor the progress of the bulk fetch job:

Use the Read Bulk Fetch endpoint,
Adjust the Parameter section:

Use id from the previous subchapter,
Make sure that startAfter value is set empty,

Send the request,

Get the results of the fetch job once it is completed.

attention

For better readability, this download list was extracted from the Bulk Fetch API into a table. It is not the default output of the API.

status_code	status_detail_id	status_technical_key	external_id	cdq_id
200	200000000	DEFAULT_OK	1234567890	GR.DE.RC:B8537_HRB739690
200	200000000	DEFAULT_OK	2345678901	GR.PL.NOBR:8992790965

info

For more details check the endpoint's documentation

Step 5: Load Data into Target Environment

Now the target environment, such as SAP MDG, can be populated with the enhanced Business Partner data. Before finalizing the import, it is crucial for data integrity and consistency checks to be performed to ensure that all necessary standards and requirements are met by the data. Once validated, the enhanced data is loaded into the target system, completing the process.

Your opinion matters!

We are constantly working on providing an outstanding user experience with our products. Please share your opinion about this tutorial!

Mail our developer-portal team: developer-portal@cdq.com