A Data Mirror is the copy of your business partner data that CDQ holds and keeps in step with your own systems. CDQ services do not reach into your ERP. They work on the mirror, and everything you get back - curated records, duplicate links, quality findings, qualification results - is written back into the mirror for you to collect.
That single idea has a large consequence. The mirror, not any individual product, is what you integrate with. Once data flows into it and results flow out of it, every CDQ product is reachable through the same two directions of travel, and adding a product later becomes a configuration change rather than a new integration project.
In this article, you will understand:
- What a Data Mirror holds, and how storages, data sources and business partners relate
- What identifies a record, and why that makes every write an update rather than a duplicate
- How to decide your data source design: by process stage, by source system, and by partner role
- How data and results move around the mirror over the life of an engagement
Available in:
- CDQ Cloud Apps - set up and inspect the mirror through a user interface
- Data Exchange API - the programmatic interface
This article covers the concepts and the design decisions. For endpoints, payloads and the mechanics of moving data, see Data Mirror Integration.
Three objects carry the whole model.
| Object | What it is | Why it matters |
|---|---|---|
| Storage | The container that holds your business partner data in CDQ. A storage usually corresponds to one workspace. | It is the boundary of your data, and it appears in every call an integration makes. |
| Data source | A named partition inside a storage. A storage can hold up to 100. | The unit you write to, the unit you filter by when reading, and the unit most configuration is scoped to. |
| Business partner | One record: names, addresses, identifiers, bank accounts, classifications. | Identified by its external ID within a data source, not by a CDQ-generated ID. |
A record is identified by the pair external ID + data source. In practice the external ID is the key the record already has in your system, most often the SAP business partner number.
This has a consequence worth stating plainly. CDQ also assigns its own id to every record, but that is not the key you build on. If the same external ID arrives twice into the same data source, the second arrival updates the first record rather than creating a second one. Writes behave as updates, and the external ID is what makes that safe to repeat.
If your source system does not have a stable key per record, resolve that before connecting it. An external ID that changes between loads produces duplicate records in the mirror, and no amount of downstream duplicate detection fully repairs it.
A single data source is enough to hold data. Almost nobody uses only one, because the data source is the natural place to express a split that already exists in the business.
Three splits account for most designs, and they combine rather than compete.
| Split by | Example | What it buys you |
|---|---|---|
| Process stage | input and output | The record you sent and the record CDQ produced both survive, so you can compare them. |
| Source system | s4, salesforce, legacy-erp | Each system keeps its own mapping and its own key space, and you know where a record came from. |
| Partner role | suppliers and customers | Different rules, different checks and different owners for populations that are not comparable. |
The first split, and the one worth taking even on a small engagement:
┌──────────────────── YOUR STORAGE ────────────────────┐
│ │
│ ┌────────────────┐ ┌────────────────┐ │
│ │ input │ │ output │ │
│ │ raw records │ │ curated records│ │
│ │ as your system │ │ as CDQ returns │ │
│ │ holds them │ │ them │ │
│ └───────┬────────┘ └────────▲───────┘ │
│ │ │ │
└───────────┼──────────────────────────────┼───────────┘
│ │
▼ │
┌──────────────────────────────────────────┐
│ CDQ services │
│ augmentation · validation · duplicates │
│ qualification · curation │
└──────────────────────────────────────────┘Keeping them apart means the record you sent and the record CDQ produced both survive. You can compare them, decide per field whether to accept a change, and re-run the process without having overwritten your own starting point. If input and output share one data source, the original value is gone after the first write back.
Most organisations do not keep business partners in one place. The same supplier exists in S/4, in Salesforce, and often in a legacy system nobody has retired yet. Giving each system its own data source is the normal design, for three reasons.
Each data source carries its own mapping. A data source is bound to a data transformation definition, which describes how that system's fields become CDQ attributes. S/4 and Salesforce do not name anything the same way, and one mapping cannot serve both. Separate data sources let each system keep its own layout and arrive without pre-translation.
Each system keeps its own key space. A record is identified by external ID within a data source, so the S/4 number and the Salesforce ID never collide even when they describe the same company. That matters, because they often are the same company.
You keep provenance. When results flow back, you filter by data source, so each system collects the records it is responsible for. Without the split you would need a field of your own to remember which system a record came from, and keep it correct forever.
The same company appearing in several data sources is not a modelling mistake. It is the honest representation of your landscape, and it is what makes cross-system duplicate detection possible: a duplicate monitor configured over several data sources searches for candidates across all of them, so the S/4 record and the Salesforce record are found as a pair. Collapsing both systems into one data source would hide exactly the overlap you want measured.
The other common split is by what the partner is: suppliers in one data source, customers in another, sometimes employees or carriers alongside them.
The reason is that the two populations are not judged the same way. A supplier usually needs bank account validation, sanction screening and ownership checks. A customer usually needs none of those and needs something else instead. The completeness bar differs, the stewardship owner differs, and a quality score computed across both at once describes nobody.
This is enforceable rather than conventional, because a data monitor is configured against a set of data sources. The same mirror can run a strict configuration over suppliers and a lighter one over customers, and report on each separately, without either set of rules reaching the other population.
The splits combine. A mature setup often has an input and an output data source per source system, with roles distinguished inside them or by further sources again.
The cost is configuration. Each data source may carry its own mapping, and each monitor has to know which sources it covers. Splitting along a line that does not change how records are treated adds that cost and returns nothing.
A useful test: if two groups of records will be mapped the same way, checked by the same rules, and collected by the same system, they do not need separate data sources.
Alongside your own data sources, a storage also sees CDQ-managed reference data sources: the community data pool, business registers, commercial providers and tax authority databases. You do not write to these. They are what CDQ matches your records against, and which of them are active is a configuration of your workspace rather than part of your integration.
Putting both directions together gives the shape that a CDQ engagement follows, whether it runs for a week or across a multi-year migration.
your system input DS CDQ services output DS your system
│ │ │ │ │
│ records in │ │ │ │
│ ─────────────────►│ │ │ │
│ │ augmentation, │ │ │
│ │ ───────────────────►│ │ │
│ │ curation, │ results written │ │
│ │ duplicates, DQ │ ──────────────────►│ │
│ │ │ │ results out │
│ │ │ │ ────────────────►│
│ │
│◄────────────────── repeat for changed records only ───────────────────────────│The loop matters more than any single pass. A first load is rarely the end of the work: records change in the source system, curation decisions get revisited, and new rules get applied to data that was already processed. A connection designed around "what changed since last time" handles all three without special cases.
How often the mirror is updated is a business decision before it is a technical one.
- On every change. Each create, update and delete in your system reaches CDQ immediately. Results become available shortly after, which is what you want when a user is waiting for a validation or a sanction check before onboarding a partner.
- Daily. Collect the day's changes and send them in one go. Cheaper to run and simpler to operate, at the cost of results being a day behind.
- On demand. Load when a project needs it. Normal for migration and cleansing work, where data moves in distinct rounds rather than continuously.
A live connection and a periodic bulk load are complements rather than alternatives. The live connection keeps the mirror current; a periodic load catches anything the event stream missed. Deletions in particular need their own handling, because a record that disappears from your system does not disappear from the mirror unless you say so.
Where an engagement delivers curated values alongside the originals, the convention is to carry both on the same record under paired field names: one set holding the value as you supplied it, one holding the value as CDQ curated it.
This is what lets a data steward decide per field and per record whether to accept a change, rather than accepting or rejecting a whole load. Which fields are paired, and what the pairs are called, is agreed per engagement rather than fixed by the platform.
The Data Mirror is the copy of your business partner data that CDQ works on, and it is the surface every CDQ product is reached through. A storage holds data sources, a data source holds business partners, and a record is identified by its external ID within a data source, which is what makes repeated writes safe.
How you cut the mirror into data sources is a design decision worth making deliberately rather than inheriting. Separating input from output is what lets you compare what you sent against what CDQ returned. Separating by source system keeps each system's mapping and key space its own, and is what makes cross-system duplicate detection meaningful. Separating by partner role is what lets suppliers and customers be held to different rules. The three combine, and the limit on how far to take them is the configuration each one carries.
- Data Mirror Integration
- How to synchronize data with the Data Mirror
- How to Define and Use Data Transformation Definitions
- Data Exchange API
We are constantly working on providing an outstanding user experience with our products. Please share your opinion about this tutorial!