What bad data actually means

The meeting starts with a calm question that is not calm at all: “Why are these numbers wrong?” Everyone looks at the analyst. The CRM says 12,400 active customers. Finance says 11,900. Marketing’s campaign export says something else again. Someone mutters “bad data,” as if that were a diagnosis instead of a shrug.

Bad data is real. It is also vague. When you cannot name what is wrong, you default to panic cleaning: fill blanks, delete “weird” rows, force a match, and hope the slide deck holds. That is how you fix the wrong thing, quietly break history, and still get blamed next quarter.

This is Part 1 of Data quality for people who ship numbers. The series is for analysts who own dashboards, handoffs, and Monday morning explanations, not for enterprise master data theater. We start with language: the dimensions that turn “this feels off” into a fixable ticket.

What you’ll learn

• Five practical quality dimensions: completeness, accuracy, consistency, timeliness, and uniqueness
• One workplace example for each, so you can recognize them under pressure
• Why naming the dimension before you “clean” saves hours and credibility
• A simple table you can paste into a ticket or Slack thread
• How this series connects to SQL, Python, and spreadsheet habits you already have

“Bad data” is not a root cause

In everyday work, “bad data” usually means one of three feelings: the total does not match another system, a chart looks implausible, or a stakeholder found a counterexample in two minutes. Those feelings matter. They are not a plan.

Data quality frameworks (DAMA-style practice, vendor guides, and a lot of dusty governance decks) break quality into dimensions: measurable ways data can fail relative to a use case. You do not need fifteen dimensions on day one. You need a short list you can say out loud in a stand-up without sounding like a white paper.

Here is the decision rule for this whole series:

Name the dimension before you fix the row. Cleaning without a named failure mode is gambling with history.

If you have been living in Excel hell, the same instinct shows up in From spreadsheets to real data: structure the problem, then transform. Quality work is the same idea applied to trust, not just tidy columns.

Five dimensions you can use on Monday

We will use five dimensions that show up constantly when people ship numbers: completeness, accuracy, consistency, timeliness, and uniqueness. Validity (does the value obey a format or allowed set?) often rides along with accuracy and consistency. Integrity (do keys and references hang together?) shows up hard in joins and dedupes later in the series.

Each dimension answers a different question. Mixing them up is how you “fix completeness” by inventing zeros and call it accuracy.

1. Completeness: is the required field present?

Example:

Name the dimension first:

Completeness is about whether values that should exist for a purpose actually exist. A blank region on an optional survey field is not the same crisis as a blank order_id on a paid invoice export.

Workplace example: You build a conversion funnel from form submissions. Ten percent of rows have no utm_source. Marketing wants “share of leads by channel.” If you drop those rows, conversion rates rise for every named channel and you hide a real acquisition path (direct, untagged, broken tags). If you bucket them as “Unknown,” the chart stays honest about missing tags. Completeness is the issue. Inventing a channel is not accuracy; it is fiction.

Completeness is always relative to a required set. Write that set down: for “billable order,” maybe order id, customer id, amount, and order date are mandatory. Everything else can be incomplete without killing the metric.

2. Accuracy: does the value match reality (or a trusted source)?

Accuracy is correctness against the real world or an agreed source of truth. The cell can be complete, valid-looking, and still wrong: a phone number with the right length but the wrong digits; a status of “Closed” when the ticket is still open.

Workplace example: Support tickets show average handle time of 4 minutes. Shadow a few tickets and you discover agents close and reopen cases to meet SLAs, so duration restarts. The field is filled (complete). It is consistent across exports (same broken logic everywhere). It is still inaccurate for “true customer wait.” Accuracy needs a definition of truth, not just a non-null column.

Analysts cannot always visit the warehouse floor. You can still improve accuracy by checking samples, reconciling to finance ledgers, or comparing system events to user-visible outcomes. When you cannot verify, say so: “We measure system close time, not lived experience.” That sentence is quality work.

3. Consistency: do related values agree across places and rules?

Consistency is agreement: the same fact should not disagree with itself across systems, tables, or business rules without a documented reason. Customer email in CRM vs billing. Status “Shipped” while inventory still shows “In warehouse.” Region “EMEA” in one table and “Europe” in another when both claim the same hierarchy.

Workplace example: Sales pipeline reports $2.1M for the quarter. Finance’s booking report shows $1.8M. After an hour of defensiveness, you learn Sales counts “Commit” stage while Finance counts signed contracts with a booking date. Both can be “right” for different definitions. That is a consistency of definition problem as much as a data problem. Until you align grain and rules, every join and reconciliation will look like a fight.

This is where Analytics foundations pays off: the question and the metric definition come before the chart polish.

4. Timeliness: is the data fresh enough for the decision?

Timeliness is fitness for when you decide. Yesterday’s inventory snapshot can be perfect for a weekly board pack and useless for same-day warehouse picks. “Late” is not a moral failing. It is a mismatch between refresh lag and decision speed.

Workplace example: Leadership asks for “current churn” during a 9 a.m. stand-up. Your churn table refreshes at 10 a.m. from overnight batches. You present yesterday’s number without a timestamp. Someone checks a live product admin panel and sees different counts. The data is not necessarily inaccurate for its pipeline; it is untimely for a live operational debate. Label the as-of time. If the decision needs faster data, escalate the pipeline, do not silently stretch a batch metric into a real-time claim.

5. Uniqueness: is each real-world entity represented once where it should be?

Uniqueness means one entity, one row (or one key) at the grain you claim. Duplicate customers, double-counted invoices, and repeated event rows destroy rates and totals. Sometimes duplicates are legitimate history (every status change is a row). The bug is claiming unique customers while counting event rows.

Workplace example: A “unique users this week” tile doubles after a tracking change emits two pageview events per load. Completeness looks better (more events!). Accuracy of “users” collapses. Uniqueness at the wrong grain is how growth charts go vertical for one sprint and then require a quiet restatement.

Part 3 of this series goes deep on deduping without shredding history. For now, remember: uniqueness is a claim about grain, not a reason to delete every repeated string you dislike.

The dimensions at a glance

Keep this table near your keyboard. When someone says “bad data,” force a row choice.

Worked example: one messy export, five labels

Imagine a weekly CSV from a ticketing tool used for a “customers helped” metric. Here is a tiny slice (fake but familiar):

Walk the rows with dimensions, not vibes:

• Completeness: Ticket 1002 has no email. If the metric is “helped customers,” you cannot attribute a person. Flag it; do not invent an email.
• Accuracy: Ticket 1004 is open but has closed_at and a handle time. Something is wrong relative to status rules. Investigate before averaging handle time.
• Consistency: Regions US vs usa, statuses closed vs Closed. Same concepts, different spellings. Mapping belongs later (Part 4); naming the issue is consistency of representation.
• Timeliness: If this file landed at 9 a.m. with a 24-hour lag, do not title the chart “Live support load.”
• Uniqueness: Ticket 1001 appears twice. Counting rows inflates volume. Decide whether this is a load bug or intentional history.

A quick SQL profile (concept you will deepen in Part 2) already separates dimensions:

SELECT
  COUNT(*) AS row_count,
  COUNT(DISTINCT ticket_id) AS distinct_tickets,
  COUNT(*) - COUNT(customer_email) AS missing_email,
  COUNT(*) FILTER (WHERE status ILIKE 'closed' AND closed_at IS NULL) AS closed_without_time,
  COUNT(*) FILTER (WHERE status ILIKE 'open' AND closed_at IS NOT NULL) AS open_with_close_time
FROM tickets_raw;

If row_count exceeds distinct_tickets, you have a uniqueness conversation. If missing_email is high, completeness. If open tickets have close times, accuracy or process bugs. The query does not “fix” anything. It assigns names. That is the win.

In Python, the same idea looks like the profiling habits from the Python for analytics series: count nulls, value counts, and duplicates before you mutate.

import pandas as pd

df = pd.read_csv("tickets_raw.csv")
print("rows", len(df))
print("distinct ticket_id", df["ticket_id"].nunique())
print("null email", df["customer_email"].isna().sum())
print(df["status"].value_counts(dropna=False))
print(df["region"].value_counts(dropna=False))
print("dup ticket rows", df.duplicated(subset=["ticket_id"]).sum())

Name it before you fix it: a mini playbook

When the Slack thread turns red, use this sequence:

1. Restate the decision. What choice does this number support? Hiring? Spend? SLA breach?
2. State the grain. One row equals what? Ticket? Customer-day? Invoice line?
3. Pick a primary dimension. You can have secondary issues, but lead with one.
4. Show a metric for that dimension. Percent null, percent mismatch, lag hours, duplicate rate.
5. Propose a fix path that matches the dimension. Completeness: capture or document. Accuracy: correct source or redefine. Consistency: map or align definitions. Timeliness: label lag or speed up. Uniqueness: keys and merge rules (carefully).

This is the opposite of “I’ll clean it in the notebook so the chart looks fine.” Silent notebook cleaning is how two teams ship two truths.

Common mistakes

• Treating all quality as null-filling. Nulls are one symptom class. Wrong values and duplicate keys are not cured by fillna(0).
• Confusing validity with accuracy. A date of 2099-01-01 can pass a type check and still be nonsense for “last purchase.”
• Declaring one system always right. Consistency problems need a defined source of truth per field, not tribal loyalty to the CRM.
• Hiding lag. Stakeholders will invent their own “live” comparison. Put as-of timestamps on exports and dashboards.
• Deleting duplicates on sight. Some repeats are history. Uniqueness fixes need keys and lineage (coming in Part 3).
• Skipping definitions. Many “data quality” fights are metric definition fights wearing a data costume.

How to practice this week

• Pick one dashboard people argue about. Write five bullets, one per dimension, even if a bullet is “not the issue here.”
• Add an as-of timestamp to one export you send regularly.
• Compute three rates on a raw table: percent null on a key field, percent duplicate keys, hours since max event time.
• When someone says “bad data,” reply with a dimension name and a single number. Watch how the conversation changes.
• Browse the hands-on paths on Learn if you need a refresher on SQL or Python before Part 2’s profiling drills.

Quick recap

• “Bad data” is a feeling. Dimensions are a diagnosis language.
• Completeness, accuracy, consistency, timeliness, and uniqueness cover most analyst pain.
• Each dimension implies a different fix. Mixing them creates quiet damage.
• Name the dimension, measure it, then clean. That order protects trust.
• Next: profile before you polish so you stop cleaning blind.

If you already work in SQL, keep the SQL series nearby for the queries. Quality is not a separate career track. It is how reliable numbers get shipped.

Sources

• DAMA International, Data Management Body of Knowledge (DMBOK) overview: https://www.dama.org/cpages/body-of-knowledge
• DAMA NL research paper on dimensions of data quality (PDF): https://www.dama-nl.org/wp-content/uploads/2020/09/DDQ-Dimensions-of-Data-Quality-Research-Paper-version-1.2-d.d.-3-Sept-2020.pdf
• IBM documentation on data quality dimensions: https://www.ibm.com/docs/en/ws-and-kc?topic=quality-data-dimensions
• Collibra, “The 6 dimensions of data quality”: https://www.collibra.com/blog/the-6-dimensions-of-data-quality
• Wang, R. Y., & Strong, D. M. (1996). Beyond accuracy: What data quality means to data consumers. Journal of Management Information Systems. (Classic academic framing of quality as fitness for use.)

Merch on Etsy

Open store →

You're On Mute White Ceramic Coffee Mug, Data Science Mug, Data Mug, Analytics Mug, Statistics Mug, Programming Mug, Gift Mug, Coffee Mug View on Etsy → Y Hat Mouse Pad, Data Science Mouse Pad, Analytics Mouse Pad, Y Pred Mouse Pad View on Etsy → Imbalanced Classes Mouse Pad, Data Science Mouse Pad, Analytics Mouse Pad, Imbalanced Mouse Pad View on Etsy → Boxplot Mouse Pad, Data Science Mouse Pad, Analytics Mouse Pad, Box Plot, Whisker Plot View on Etsy →