Case studyCase 03
One record per customer across a million rows
- Operations team
- 1M+ records
- Under 30 minutes
Results
What changed, and what each number rests on.
1M+
Records resolved
Basis: One full run in production
< 30 min
End to end
Basis: Measured on the full file
90%+
Match accuracy
Basis: Sampled against a person's decision
Side by side
Uncertain pairs go to a person
Basis: By design
Source-traced
Every merge keeps its origin records
Basis: By design
Repeatable
Runs again as new records arrive
Basis: By design
Company profile
- Company
- An operations team with a million-row customer file
- Stack
- The CRM, the billing system, the support desk, and a shared export of all three
- Operating need
- Know which records are the same customer before any follow-up, quoting, or reporting could be trusted.
The challenge
Where the work was stuck.
The same customer existed three times: once in the CRM under a nickname, once in billing under a company email, and once in the support desk under a phone number. Every report double-counted, every follow-up risked writing to the wrong record, and nobody could say how many customers the company actually had.
Manual cleanup had been tried twice. Each time the file was clean for a week and then drifted, because new records kept arriving through the same three doors. The team needed a system that could resolve the whole file quickly, do it again on a schedule, and never merge two records on a guess.
The audit
Six systemic patterns.
- Three systems each created customers on their own, with no shared identity between them.
- Names, emails, and phone numbers were entered by hand, so the same person appeared under dozens of spellings.
- Exact matching caught the easy duplicates and missed the ones that mattered.
- Cleanup was a project, not a process, so the file drifted back within weeks.
- Nobody owned the customer record, so no system was allowed to be the source of truth.
- Every downstream automation the team wanted depended on this one being right first.
The data entry was not the problem. The team was missing the identity layer every other system sits on.
The architecture
Five components, installed in order.
- 01NormalizationNames, emails, phone numbers, and addresses cleaned into one canonical form per field, so the same value written three ways compares as one.
- 02BlockingRecords grouped by cheap keys such as email domain and phone prefix, so the system compares candidates instead of every record against every other record.
- 03Candidate generationFuzzy matching inside each block that proposes pairs likely to be the same customer, with the reason for each proposal.
- 04ScoringA learned score on each pair. High scores merge on their own, low scores stay separate, and the middle band waits for a person.
- 05Adjudication queueUncertain pairs shown to a person with both records side by side. Every decision is logged and feeds the next run.
Anonymization
The company is not named. The pipeline and the numbers are as they ran.
- Names withheld: the company, its account names, and its customers are not identified.
- Numbers from the operating baseline: run time and accuracy are measured on the real file, not on a sample built for a demo.
- Uncertain records go to a person: no pair in the middle band is merged without a person's decision.
Implementation
Four stages, one set price.
| Stage | Focus | Key results |
|---|---|---|
| Map | Where customers were created, which fields were reliable, how duplicates were found today, and who would own the record. | An operating map, a baseline duplicate rate, and a written scope with one set price. |
| Build and review mode | Normalization, blocking, candidate generation, and scoring on the full file, with every proposed merge reviewed by the team. | The first full run, with the team approving merges and setting the thresholds. |
| Controls | Merge thresholds, the adjudication queue, source tracing on every merge, and the schedule for re-runs. | High-confidence merges on their own. The middle band to a person, side by side. |
| Launch and run | Scheduled runs as new records arrive, monitored in production, reported weekly. | 1M+ records in under 30 minutes at 90%+ accuracy. |
Ownership
What the client keeps.
- Every outcome: the resolved customer file and everything built on it belong to the company.
- The data: the file, the match decisions, and the logs stay in the company's environment.
- The documentation: the matching rules, the thresholds, and the runbook for re-runs.
- The named owner: the operations lead who owns the adjudication queue and reads the weekly report.
The next phase
What this makes possible next.
The planned next phase is Follow-up on the resolved file: aged leads and unpaid invoices worked on schedule, now that every message reaches the right customer exactly once. Identity is already shared, so it is an addition, not a second build.
Related analysis
The CRM automation guideThe company's identity is withheld. The numbers are from the system running in production.
Your first department
Bring the process you still run by hand.
One mapping call. You leave with the system design, the price in writing, and the date it goes live.
How it works
From the first call to a system in production.
The six steps every installation follows, with the output and the control at each one, and the 90-day shape.
See the process