Setup and Synchronize a Data Mirror

Overview

The sequence of activities for setting up a data mirror and continuously synchronization with the local source system.

Scenario

In this instruction you will learn how to:

1. Create a Data Source
2. Synchronize your data with the CDQ Data Model

Prerequisites

Data Storage

Step 1: Data Mirror Configuration

To configure a Data Mirror a Data Source and a Data Mapper Definition must be set.

To create Data Source:

1. Use the Create Data Source endpoint to send the below request.

2. Check the above example in the Try It Console now!
3. Set value of:
   • dataSource to YOUR_DATASOURCE_ID
   • dataMapperDefinitionId to default
   • storageId to YOUR_STORAGE_ID in Parameters section

Details of created Data Source are provided:

{
    "id": "YOUR_DATASOURCE_ID",
    "name": "YOUR_DATASOURCE_NAME",
    "dataMapperDefinitionId": "default"
}

Step 2: Synchronize Data

Data synchronization process is dependent to the size of the data to be updated:

1. If the update contains many Business Partners and the file size <= 10 MB read: File upload instruction.
2. If the update contains many Business Partners and the file size > 10 MB read: Request upload instruction.
3. If the update is for one Business Partner go to Single record update.

First time data upload

To start the import job of a csv or xlsx file:

1. Use the Start Import Job endpoint to send the below request.

curl -i -X POST \
  https://api.cdq.com/data-exchange/rest/jobs/importjobs \
  -H 'Content-Type: multipart/form-data' \
  -H 'X-API-KEY: YOUR_API_KEY_HERE' \
  -F file=CH-import.csv \
  -F url=customer-uploads/CH-import.csv \
  -F storageId=72d6900fce6b326088f5d9d91049e3e6 \
  -F dataSource=648824a691d8d2503d65103e \
  -F externalIdColumn=BP_EXTERNAL_ID \
  -F dataMapperDefinitionId=6400955811c68a034bcef311 \
  -F featuresOn=UPSERT_BY_EXTERNAL_ID

2. Check the above example in the Try It Console now!
3. Set value of:
   • dataMapperDefinitionId to default
   • dataSource to YOUR_DATASOURCE_NAME
   • featuresOn to UPSERT_BY_EXTERNAL_ID
   • storageId to YOUR_STORAGE_ID
4. Point the file path.

In the response you will find 3 important parameters:

• The id with the value representing YOUR_JOB_ID - the unique number of every upload job.
• The progress: 0 represents the progress of the job.
• The status: SCHEDULED - the job is in the job queue.

Mass data update

To perform mass data update follow the same steps from the First time data upload subchapter.

Single record update

To update existing records:

1. Use the Upsert Business Partners endpoint to send the below request.

2. Check the above example in the Try It Console now!
3. Select example Single record update.
4. Set the YOUR_DATASOURCE_ID.
5. Replace below values with the Business Partner data to be updated:
   • BP_EXTERNAL_ID
   • COMPANY_NAME
   • COUNTRY_CODE
   • CITY
   • STREET
   • POSTAL_CODE
6. Set the YOUR_STORAGE_ID in Parameters section.

Results

3 types of results can be expected:

1. New record created.
2. Existing record updated.
3. Something went wrong:
   • The response includes a message about the root of the issue.

{
"numberOfAccepted": 1,
"numberOfInserts": 1,
"numberOfUpdates": 0,
"numberOfFailed": 0
}

{
"numberOfAccepted": 1,
"numberOfInserts": 0,
"numberOfUpdates": 1,
"numberOfFailed": 0
}

{
"numberOfInserts": 0,
"numberOfUpdates": 0,
"numberOfFailed": 1,
"failures":
[

    {
        "externalId": "0053",
        "message": "Incorrect record quote escaping"
    }

]
}

Additional activities

Read updated Business Partner

To start the update of the existing records:

1. Use the Read Business Partners endpoint to send the below request.

curl -i -X GET \
  'https://api.cdq.com/data-exchange/rest/v4/storages/{storageId}/businesspartners?businessPartnerId=63e635235c06b7396330fe40&countryCode=CH&curationLogResultStatus=LEVEL_1&dataSource=%22CUSTOM_DATA_SOURCE%22%20or%20%22648824a691d8d2503d65103e%22&externalId=The%20ID%20managed%20in%20the%20customer'\''s%20SAP%20systems.&featuresOff=APPLY_CURATION_DECISIONS&featuresOn=APPLY_CURATION_DECISIONS&fromJobProcessedAt=2025-03-31T14%3A03%3A32Z&fromProcessedAt=2025-03-31T14%3A03%3A32Z&fromTriggerProcessedAt=2025-03-31T14%3A03%3A32Z&hasRawData=false&limit=100&logResultTriggerProvenanceTechnicalKeys=VIES&logResultTriggerType=CREATED&modifiedAfter=2025-03-31T14%3A03%3A32Z&modifiedBefore=2025-03-31T14%3A03%3A32Z&reviewStatus=REVIEWED&sharingStatus=UNDER_CONSIDERATION&startAfter=5712566172571652&tagTypeTechnicalKey=WAREHOUSE&tagValues=Warehouse&toJobProcessedAt=2025-03-31T14%3A03%3A32Z&toProcessedAt=2025-03-31T14%3A03%3A32Z&toTriggerProcessedAt=2025-03-31T14%3A03%3A32Z&typeTechnicalKeys=LEGAL_ENTITY&updateMonitoring=true&validationLogResultStatus=WARNING&validationSharingLogResultStatus=WARNING' \
  -H 'X-API-KEY: YOUR_API_KEY_HERE'

2. Check the above example in the Try It Console now!
3. Set values of:
   • storageId to YOUR_STORAGE_ID
   • dataSource to YOUR_DATASOURCE_ID
   • externalId to BUSINESSPARTNER_EXTERNALID

Clear a complete data source

To clear a data source:

1. Use the Clear Data Source endpoint to send the below request.

curl -i -X POST \
  'https://api.cdq.com/data-exchange/rest/storages/{storageId}/datasources/{dataSourceId}/clear' \
  -H 'X-API-KEY: YOUR_API_KEY_HERE'

2. Check the above example in the Try It Console now!
3. Set values of:
   • storageId to YOUR_STORAGE_ID
   • dataSource to YOUR_DATASOURCE_ID

Delete a data source

To delete a data source:

1. Use the Delete Data Source endpoint to send the below request.

curl -i -X DELETE \
  'https://api.cdq.com/data-exchange/rest/storages/{storageId}/datasources/{dataSourceId}' \
  -H 'X-API-KEY: YOUR_API_KEY_HERE'

2. Check the above example in the Try It Console now!
3. Set values of:
   • storageId to YOUR_STORAGE_ID
   • dataSource to YOUR_DATASOURCE_ID

Delete a Business Partner Record

To delete a data source:

1. Use the Delete Business Partners endpoint to send the below request.

2. Check the above example in the Try It Console now!
3. Select Delete Business Partner Based on External Id example.
4. Set values of:
   • externalId to BP_EXTERNAL_ID
   • dataSource to YOUR_DATASOURCE_ID
5. Set the YOUR_STORAGE_ID in Parameters section.

Your opinion matters!

We are constantly working on providing an outstanding user experience with our products. Please share your opinion about this tutorial!

Mail our developer-portal team: developer-portal@cdq.com