# Setup and Synchronize a Data Mirror

**Overview**

The sequence of activities for setting up a data mirror and continuously synchronization with the local source system.

This instruction is based on [Setup and Synchronize a Data Mirror](https://meta.cdq.com/Use_case/Data_Mirror_Setup_and_Synchronization) use case published on the Wiki.

**Scenario**

In this instruction you will learn how to:

1. Create a Data Source
2. Synchronize your data with the CDQ Data Model


## Prerequisites

**Data Storage**

To create a Data Source a [Storage](/documentation/instructions/uploading-cvs-xls-data#step-1-create-a-storage) or Data Mirror is needed! Create one before further step and use its storageId value as `YOUR_STORAGE_ID` in the next steps of this instruction.

Every created Storage or Data Mirror has a default Data Source.

## Step 1: Data Mirror Configuration

To configure a Data Mirror a Data Source and a Data Mapper Definition must be set.

To create Data Source:

1. Use the **Create Data Source** endpoint to send the below request.
2. Check the above example in the **Try It Console** now!
3. Set value of:
  * *dataSource* to `YOUR_DATASOURCE_ID`
  * *dataMapperDefinitionId*  to `default`
  * *storageId* to `YOUR_STORAGE_ID` in **Parameters** section


Details of created Data Source are provided:


```json
{
    "id": "YOUR_DATASOURCE_ID",
    "name": "YOUR_DATASOURCE_NAME",
    "dataMapperDefinitionId": "default"
}
```

Congratulations
Your Data Source is created!

## Step 2: Synchronize Data

Data synchronization process is dependent to the size of the data to be updated:

1. If the update contains many Business Partners and the **file size** <= 10 MB read: [File upload instruction](/documentation/instructions/uploading-cvs-xls-data).
2. If the update contains many Business Partners and the **file size** > 10 MB read: [Request upload instruction](/documentation/instructions/url-upload).
3. If the update is for one Business Partner go to [Single record update](#single-record-update).


### First time data upload

To start the import job of a csv or xlsx file:

1. Use the **Start Import Job** endpoint to send the below request.
2. Check the above example in the **Try It Console** now!
3. Set value of:
  * *dataMapperDefinitionId*  to `default`
  * *dataSource* to `YOUR_DATASOURCE_NAME`
  * *featuresOn* to `UPSERT_BY_EXTERNAL_ID`
  * *storageId* to `YOUR_STORAGE_ID`
4. Point the file path.


In the response you will find 3 important parameters:

* The **id** with the value representing `YOUR_JOB_ID` - the unique number of every upload job.
* The **progress**: `0` represents the progress of the job.
* The **status**: `SCHEDULED` - the job is in the job queue.


Note down received `YOUR_JOB_ID` for polling the job status. Check how to poll the job status [here](/documentation/instructions/uploading-cvs-xls-data#pool-for-the-completion-status).

### Mass data update

To perform mass data update follow the same steps from the [First time data upload](#first-time-data-upload) subchapter.

The key to successfully uploading is `UPSERT_BY_EXTERNAL_ID` feature. The update file with correctly mapped **External Id** column after upload will result in data update for matching `externalIds` and new record creation for new `externalIds`.

### Single record update

Before the update of the existing records the Business Partner's `externalId` is needed. Be sure that the `UPSERT_BY_EXTERNAL_ID` feature is active.

To update existing records:

1. Use the **Upsert Business Partners** endpoint to send the below request.


Check the below example based on `default` mapping.

1. Check the above example in the **Try It Console** now!
2. Select example **Single record update**.
3. Set the `YOUR_DATASOURCE_ID`.
4. Replace below values with the Business Partner data to be updated:
  * `BP_EXTERNAL_ID`
  * `COMPANY_NAME`
  * `COUNTRY_CODE`
  * `CITY`
  * `STREET`
  * `POSTAL_CODE`
5. Set the `YOUR_STORAGE_ID` in **Parameters** section.


**Results**

3 types of results can be expected:

1. New record created.
2. Existing record updated.
3. Something went wrong:
  * The response includes a message about the root of the issue.



```json
{
"numberOfAccepted": 1,
"numberOfInserts": 1,
"numberOfUpdates": 0,
"numberOfFailed": 0
}
```


```json
{
"numberOfAccepted": 1,
"numberOfInserts": 0,
"numberOfUpdates": 1,
"numberOfFailed": 0
}
```


```json
{
"numberOfInserts": 0,
"numberOfUpdates": 0,
"numberOfFailed": 1,
"failures":
[

    {
        "externalId": "0053",
        "message": "Incorrect record quote escaping"
    }

]
}
```

Congratulations
Now you know how to update a single record.

## Additional activities

### Read updated Business Partner

To start the update of the existing records:

1. Use the **Read Business Partners** endpoint to send the below request.
2. Check the above example in the **Try It Console** now!
3. Set values of:
  * *storageId* to `YOUR_STORAGE_ID`
  * *dataSource* to `YOUR_DATASOURCE_ID`
  * *externalId* to `BUSINESSPARTNER_EXTERNALID`


### Clear a complete data source

To clear a data source:

1. Use the **Clear Data Source** endpoint to send the below request.


Remember that your **ALL** data within cleared Data Source will be deleted.

1. Check the above example in the **Try It Console** now!
2. Set values of:
  * *storageId* to `YOUR_STORAGE_ID`
  * *dataSource* to `YOUR_DATASOURCE_ID`


### Delete a data source

To delete a data source:

1. Use the **Delete Data Source** endpoint to send the below request.


Remember that your Data Source and its data will be deleted.

1. Check the above example in the **Try It Console** now!
2. Set values of:
  * *storageId* to `YOUR_STORAGE_ID`
  * *dataSource* to `YOUR_DATASOURCE_ID`


### Delete a Business Partner Record

To delete a data source:

1. Use the **Delete Business Partners** endpoint to send the below request.


Remember that your Business Partner record will be deleted permanently.

1. Check the above example in the **Try It Console** now!
2. Select **Delete Business Partner Based on External Id** example.
3. Set values of:
  * *externalId* to `BP_EXTERNAL_ID`
  * *dataSource* to `YOUR_DATASOURCE_ID`
4. Set the `YOUR_STORAGE_ID` in **Parameters** section.


## Your opinion matters!

We are constantly working on providing an outstanding user experience with our products. Please share your opinion about this tutorial!