Skip to main content

MDM Test Data: Generating Data to Test Match, Merge, and Golden Record Logic

· Updated

MDM test data is a synthetic dataset built to validate match/merge and golden-record logic — records that vary in known ways (typos, formatting differences, exact duplicates) tagged with a Master ID and Duplicate Type so you can test each step of the validation sequence: data quality checks, standardization, matching, survivorship, reconciliation, and golden record.

The MDM testing validation sequence

Master data management systems resolve duplicate and near-duplicate records to one canonical "golden record" through a sequence of stages (per Datagaps' MDM validation structure). Each stage needs a different kind of test data to actually exercise it:

MDM testing validation sequence
StageWhat kind of test data exercises it
Data quality checksRecords with realistic gaps and formatting issues to validate against.
StandardizationFormatting variants of the same field (phone/address formats) to normalize.
MatchingTypo/phonetic/reordered variants of the same identity to test match rules.
Survivorship rulesMultiple source-system versions of the same record to pick a winner from.
ReconciliationConflicting field values across source systems to resolve.
Golden recordA known-correct merged output to check the pipeline against.

What MDM test data needs to do

Master data management systems exist to catch duplicate and near-duplicate records — the same person or entity showing up under slightly different spellings, addresses, or account references — and resolve them to one canonical record. Testing that logic means the test data has to carry the kind of internal structure real records carry: a name tied to an address, an address tied to an account, a phone number that belongs to the same person as the email. Rows where every field is independent of every other field don't exercise match/merge logic at all; there's nothing for it to reconcile.

Why correlated fields matter for match/merge testing

Field groups solve this directly. Instead of generating a name column and an address column separately, you link the fields you need correlated into a group, and they generate together as one coherent unit per row. That's what gives an MDM test set something to actually match against — synthetic records where a person's or entity's attributes hang together the way a matching algorithm expects them to, rather than a name field with no relationship to the address sitting next to it.

How to generate MDM test data with known duplicates

For match/merge and survivorship testing specifically, mark the fields that should carry duplicates exact_matching and/or fuzzy_matching in the schema builder — the generator produces a master record per identity, then variant rows tagged back to it with a Master ID, so you have a documented, known answer for what should and shouldn't merge. Add a source_system field (CRM/ERP/Billing, etc.) to also test survivorship across systems, the way the Customer (MDM) catalog dataset does. Small worked example — one identity across three source systems:

Worked example: one identity, three source systems
Master IDDuplicate TypeSource SystemNameEmail
701MASTERCRMDominic Falk[email protected]
701EXACTERPDominic Falk[email protected]
701ADDRESS_TYPOBillingDominic Falk[email protected]

For the full downloadable column reference and answer key, see the duplicate customer records dataset.

Building MDM test data with ai_generated field groups

Start with the schema you need for MDM testing: name, address, phone, employer, entity ID, account reference — whatever your matching logic keys on. For each field, write a free-text prompt describing what a value should look like; that's the ai_generated field type generating the values through an AI provider. Then link the fields that need to stay consistent with each other into a group, so they generate together instead of independently. The setup — prompt syntax, how groups link fields — is covered in set up AI-generated correlated fields. Once the schema and groups are in place, generate a dataset from it. The qualifier holds throughout: this is synthetic data, not de-identified real data. Every field is produced by an AI provider from the prompt you wrote — nothing here started as a real person's or organization's record. If you need person-only fixtures without the entity/account layer MDM testing adds, see related: person test data set. For string-similarity-specific testing (typo/phonetic/token variants), see fuzzy matching test data.

Frequently asked questions

What is MDM test data?

A synthetic dataset built to test match, merge, and golden-record logic — records that vary in known ways (typos, formatting differences, exact duplicates) tagged with a Master ID and Duplicate Type, and often a source_system field, so each stage of the validation sequence has real data to work against. Not de-identified real data.

How do I test a golden-record/match-merge process?

Generate records that vary across the stages your pipeline needs to handle: formatting differences for standardization, typo/phonetic variants for matching, multiple source-system versions of the same identity for survivorship, and a known-correct merged output to check reconciliation against.

What's the difference between MDM and entity resolution?

Entity resolution is the general technique for deciding which records refer to the same real-world entity — matching, blocking/indexing, deterministic vs. probabilistic scoring. MDM is one applied domain of it, specifically the business process of governing a "golden record" for master data (customers, products, vendors) across source systems. See fuzzy matching test data for the underlying technique.

Back to all guides