The Data Hygiene Playbook

Hygiene is maintenance, not a project

The distinction matters because it determines how the work gets funded and staffed. A project has an end date, a budget, and a completion criterion. Data hygiene has none of those — it is closer to server maintenance, and treating it as a project is why the same cleanup gets commissioned every eighteen months.

The framing that works: seven recurring jobs, each with a schedule, a named owner, and an alert when it fails. The data quality standard defines what good looks like; this is the operational work that holds it.

The seven jobs

Deduplication done properly

The order of operations matters more than the matching algorithm.

1. 01
   Match at creation, before the record exists

   Preventing a duplicate costs nothing. Merging one that has three months of activity, two tasks, and an opportunity attached is slow, lossy, and occasionally destructive. Most systems support creation-time matching and most teams leave it off.

2. 02
   Use layered match rules

   Exact email, then normalised email ignoring plus-addressing and dots, then domain plus normalised name, then fuzzy name within a domain. Each layer catches what the previous one misses.

3. 03
   Auto-merge only on high confidence

   Exact email matches can merge automatically. Fuzzy matches go to a review queue. Auto-merging fuzzy matches is how two genuinely different people at the same company become one record.

4. 04
   Preserve the merge trail

   Record which records were merged, when, and on what rule. When someone asks why an activity is attached to an unexpected contact, this is the only way to answer.

Suppression is hygiene

Suppression lists are the least glamorous hygiene job and the one whose failure is most visible externally. Emailing an existing customer a cold pitch, or contacting an account with an open opportunity through an unrelated sequence, costs credibility in a way no internal data problem does.

• Existing customers and their whole domain, not just the contacts you know about.
• Open opportunities — the account, and every contact at it.
• Recent touches within a defined window, across all channels rather than per tool.
• Hard bounces, unsubscribes, and complaints, permanently, and enforced at the pipeline level.
• Competitors and partners, maintained as a named list somebody actually owns.
• Explicit do-not-contact requests, which are frequently a legal obligation rather than a courtesy.

The critical design point: suppression must be enforced in the pipeline before the send, not configured separately inside each engagement tool. Per-tool suppression means the list is only as good as the tool you happened to use, and that is how the embarrassing send happens.

Fixing a database that is already bad

The instinct is to fix everything at once, which takes months and blocks all other work. Triage instead by what is blocking a decision this quarter.

1. 01
   Stop the inflow first

   Validation at every entry point and creation-time deduplication. There is no point cleaning a database that is still filling with the same problems, and this step takes days rather than weeks.

2. 02
   Fix the active segment only

   The accounts you will contact this quarter. Not the whole database. This gets you a usable dataset in about a week instead of an unusable one in three months.

3. 03
   Rebuild the suppression lists

   Highest external risk, lowest effort, and it can be done in an afternoon. Do it before the next campaign, not after.

4. 04
   Archive the long tail

   Records with no engagement in 24 months, archived rather than deleted. This shrinks the working set dramatically and makes every subsequent job faster and cheaper.

5. 05
   Then start the recurring jobs

   Once inflow is controlled and the active segment is clean, the seven jobs above keep it that way. Starting them before the first two steps means running maintenance against a moving target.

Archive, do not delete

Deleting records feels like progress and removes information you will want. Archive to a separate table or a cold store instead, keeping the identifier, the reason for archiving, and the date.

Two situations make this worthwhile. A previously dormant company shows a buying signal and you want its history rather than a blank record. And someone asks why the database shrank by 40,000 records last quarter, which is a question you want a precise answer to. Archiving costs almost nothing and preserves both — and it keeps the pipelines that read from the database operating on a smaller, faster working set.