This path is for analysts who ship numbers other people will quote. Parts go from what bad data means, through profiling, deduping without destroying history, names and dates, checks you can automate, and a note so the next person trusts you. Start with the first part on this page.
- 1 What does bad data mean? How to name the problem so you can fix it Bad data is a shrug, not a diagnosis. Learn five practical quality dimensions (completeness, accuracy, consistency, timeliness, uniqueness) with workplace examples so you name the failure mode before you clean and make things quietly worse.
- 2 How to profile a messy data table before you clean it Cleaning before you look is how silent bias sneaks into dashboards. Learn a lightweight profile of counts, nulls, ranges, and weird categories with SQL and Python so every polish step has a reason and a named quality dimension.
- 3 How to remove duplicate customer records without losing their history Duplicates are identity problems, not a delete button. Learn natural versus surrogate keys, soft deletes, merge lineage, and validation checks so you collapse false twins without shredding history, audits, or revenue totals that should stay invariant.
- 4 How to standardize messy category names in your data Category chaos turns dashboards into spelling bees. Learn canonical values, mapping tables, alias lists, and how to escape Other hell so groupbys stay honest and unmapped rates stay visible without enterprise MDM theater.
- 5 How to handle dates, time zones, and fiscal calendars in your reports Time zones and fiscal calendars quietly break dashboards people trust. Learn to store UTC, display local time, separate calendar months from fiscal periods, and cast dates safely in SQL and Python before you ship a number.
- 6 How to automate data validation checks so bad data never reaches your reports Automated checks catch bad data before leadership does. Build small scripts for row counts, referential integrity, freshness, and yesterday versus last week sanity so every number you ship has a real safety net.
- 7 How to document data quality so other teams trust your tables Trust needs a light paper trail, not a bureaucracy. Publish a simple data quality scorecard with owner, metrics, known issues, and next check so teammates can rely on your numbers without guessing in a crisis.