# SAP ODM: How to do bulk matching and enrichment?

## Overview

Bulk matching and enrichment is a process that allows you to match a large number of Business Partners against the CDQ
database. This process is useful when you have a large dataset of Business Partners that need to be matched and enriched.

**Learning Goals**

In this tutorial, you will learn how to:

- Prepare input data using standard SAP Business Partner objects.
- Perform bulk lookup to match Business Partners against the CDQ database.
- Review the candidate list and select the best candidates for further processing.
- Download and enrich matched records.
- Load data into the target environment.


## Pre-requisites

### Authorization

Before trying CDQ APIs, user must be authenticated:

1. Paste the API Key in the console's security bar into the `X-API-KEY` field.


![](/assets/auth7.0d4fb05fa55bc16d828c76e9976f979930ffadb949cfc149a6617cc6518ab57d.6fde6558.png)

1. After pasting the API Key, the green padlock will appear.


![](/assets/auth8.28e02515b78db57ec32c07fd6bd7d296484e64f65ef6a3bc5296520cb1da8d65.6fde6558.png)

Be careful
Green padlock doesn't mean that the API Key was pasted correctly.

1. Check your API key for missing characters or extra space before trying.


#### No API Key?

1. Check how to get one on [authentication page](/documentation/instructions/authentication).
2. Follow the steps above.


## Step 1: Prepare Input Data Using Standard SAP Business Partner Objects

Gather a list of Business Partner addresses and identification numbers, and ensure that they are structured correctly
for processing. Use the standard SAP Business Partner objects to format the Business Partner Master Data. To facilitate
the matching process, it is essential to install the necessary content pack in the CDQ environment. After installation,
the customer can pass the data to the CDQ system. The matching process will then use all available data sources configured
within the **CDQ Cloud Apps**.

Info
For further details, you can refer to the documentation provided [here](https://community.sap.com/t5/technology-blogs-by-members/trusted-business-partner-data-in-sap-master-data-governance-cloud-edition/ba-p/13583615)

## Step 2: ODM Bulk Lookup

Bulk Lookup is a process that allows you to match a large number of Business Partners against the CDQ database. This process
is useful when you have a large dataset of Business Partners that need to be matched and enriched.

### Create Bulk Lookup

Initiate the matching process by creating a **Bulk Lookup** job. The collected list of Business Partner
data, formatted according to SAP standards, is sent to the CDQ system where the matching occurs.

1. Use the **Create Bulk Lookup** endpoint,
2. Use **Create Bulk Business Partner Lookup with Golden Record** example,


Features to be used in **Bulk Lookup**:

| Feature | Description |
|  --- | --- |
| `SHOW_GOLDENRECORD_STANDARD` | Activates the Golden Record. |
| `GOLDENRECORD_INLINE_SORTED` | Sorts the Golden Record by its matching score within the list of other hits. |
| **Optional**: `GOLDENRECORD_INLINE` | to always have the Golden Record appear at the top of the list. |


1. Send the request,


1. Check the response:


* Save `id` for the next step,
* See the status of the job.



```json
{
    "id": "fa6a7348-98e6-480b-9ff2-fa3a3432a7c8",
    "status": {
        "code": 200,
        "technicalKey": "OK",
        "details": [{
            "id": 200000006,
            "message": "The bulk is currently being executed.",
            "technicalKey": "BULK_RUNNING"
        }
    ]
  }
}
```

For more details check the endpoint's [documentation](https://developer.cdq.com/reference-docs/sap-odm/V3/tag/Bulk-Lookup/#tag/Bulk-Lookup/paths/~1referencedata~1rest~1v4~1businesspartners~1lookup~1bulks/post)

### Read Bulk Lookup

Monitor the progress of the matching job as it runs on the CDQ side. Use the **Read Bulk Lookup** function to track the
completion percentage of the job.

1. Use the **Read Bulk Lookup** endpoint,
2. Adjust the **Parameter** section:


* Use `id` from the previous subchapter,
* Make sure that `startAfter` value is set empty,


1. Send the request,


1. Get the results of the matching job once it is completed.


details
summary
Additional information about pagination.
The results are provided in pages, with each page containing up to `1,000` records. This pagination is managed by the `limit`
parameter within the request. To retrieve later pages, you must set the `startAfter` parameter in the next request to the
value of `nextStartAfter` from the previous request. This allows you to get all matching responses in batches until the
entire dataset is retrieved.

For more details check the endpoint's [documentation](https://developer.cdq.com/reference-docs/sap-odm/V3/tag/Bulk-Lookup/#tag/Bulk-Lookup/paths/~1referencedata~1rest~1v4~1businesspartners~1lookup~1bulks~1%7Bid%7D/get)

## Step 3: Review Candidate List

To select the best candidates for further processing, review the list of matched Business Partners.

* Prioritize the CDQ Golden Record for its comprehensive data summary.
* Avoid selecting low-scoring candidates. Define an acceptable score threshold with guidance from your CSM.
* If needed, prioritize international sources based on specific corporate requirements.


Congratulation
Now you have the report of your data quality!

| customerNumber | matchingScore | cdqId |
|  --- | --- | --- |
| 1234567890 | 0.99 | GR.DE.RC:B8537_HRB739690 |
| 2345678901 | 0.98 | GR.PL.NOBR:8992790965 |


## Step 4: Download and Enrich Matched Records

Once the best candidates have been selected, the next step is to download and enrich the matched records.

### Execute ODM Bulk Fetch

The Bulk Fetch process requires the candidates from the previous steps got with **Create Bulk Lookup** and **Read Bulk Lookup** endpoints.
The output of these methods, combined with the sourcing strategy, generates a list of identifiers for each chosen hit or
the address data itself, along with the identifier details provided in the lookup output.

1. Use the **Create Bulk Fetch** endpoint,
2. Use **Create Bulk Business Partner Fetch with Golden Record** example,
3. Send the request,


1. Check the response:


* Save `id` for the next step,
* See the status of the job.


details
summary
Additional information about the fetch strategy.
* **Using CDQ-IDs**: The simpler approach is to feed the "Create Bulk Fetch" function with a list of `CDQ-IDs` provided by
the lookup results. Each `CDQ-ID` should be specified as `cdqId` in the company enrichment request for **Create Bulk Fetch**.
This method directly leverages the identifiers provided by CDQ, making the process straightforward.
* **Using Addresses and Identifiers**: In scenarios where you need to pass addresses and identifiers instead of `CDQ-IDs`
to the fetch function, the `CDQ-ID` must not be handed over to **Create Bulk Fetch**. Instead, a fetch strategy needs to be
specified. It is recommended to use the "`USE_FETCH_STRATEGY_GOLDEN_RECORD`" strategy, which calculates an updated version
of the CDQ Golden Record based on the given data and all available data sources.


For more details check the endpoint's [documentation](https://developer.cdq.com/reference-docs/sap-odm/V3/tag/Bulk-Fetch/#tag/Bulk-Fetch/paths/~1referencedata~1rest~1v4~1businesspartners~1fetch~1bulks/post)

### Monitor and Receive Enriched Data

The first is to monitor the progress of the fetch job as it runs on the CDQ side. By using the **Read Bulk Fetch** function,
you can track the completion percentage of the job in real time, ensuring you are aware of how far along the process is
and when you can expect the results.

To monitor the progress of the bulk fetch job:

1. Use the **Read Bulk Fetch** endpoint,
2. Adjust the **Parameter** section:


* Use `id` from the previous subchapter,
* Make sure that `startAfter` value is set empty,


1. Send the request,


1. Get the results of the fetch job once it is completed.


For better readability, this download list was extracted from the **Bulk Fetch** API into a table. It is not the default output of the API.

| status_code | status_detail_id | status_technical_key | external_id | cdq_id |
|  --- | --- | --- | --- | --- |
| 200 | 200000000 | DEFAULT_OK | 1234567890 | GR.DE.RC:B8537_HRB739690 |
| 200 | 200000000 | DEFAULT_OK | 2345678901 | GR.PL.NOBR:8992790965 |


For more details check the endpoint's [documentation](https://developer.cdq.com/reference-docs/sap-odm/V3/tag/Bulk-Fetch/#tag/Bulk-Fetch/paths/~1referencedata~1rest~1v4~1businesspartners~1fetch~1bulks~1%7Bid%7D/get)

## Step 5: Load Data into Target Environment

Now the target environment, such as SAP MDG, can be populated with the enhanced Business Partner data. Before finalizing
the import, it is crucial for data integrity and consistency checks to be performed to ensure that all necessary standards
and requirements are met by the data. Once validated, the enhanced data is loaded into the target system, completing the
process.

## Your opinion matters!

We are constantly working on providing an outstanding user experience with our products. Please share your opinion about this tutorial!