The unglamorous majority of the job

There's a well-worn saying that data scientists spend 80% of their time getting and cleaning data and the other 20% complaining about it. It's only half a joke. Models and charts get the glory, but the quiet work of turning messy, real-world data into something trustworthy is where projects are actually won or lost. Garbage in, garbage out is not a cliché here — it's the daily reality.

Getting the data

Data arrives from many places: CSV and Excel files, databases (via SQL), web APIs that return JSON, and sometimes web scraping. In Python, pandas reads most of them in a line — read_csv, read_excel, read_json. The first questions to ask of any source: how fresh is it, how was it collected, and can I trust it?

The common cleaning tasks

Real data is messy in predictable ways:

• Missing values — decide deliberately: drop the rows, or fill (impute) with a mean, median, or sensible default. Never ignore them silently.
• Wrong types — numbers stored as text, dates as strings. Convert them so math and sorting work.
• Duplicates — find and drop repeated rows that would skew counts.
• Inconsistent categories — "NY," "N.Y.," and "New York" are one place; standardize them.
• Impossible values — an age of 200 or a negative price is a data error, not a discovery.

Why it deserves real care

Every problem you leave in the data silently corrupts everything downstream — your charts mislead, your statistics lie, your model learns nonsense. Cleaning isn't busywork before the real analysis; it's the foundation the analysis stands on.

Trust your conclusions only as much as you trust your data. The hours spent cleaning are the hours that make every later result believable.

Try this: Load a real, messy public dataset and just inventory its problems — missing values, odd types, duplicates, inconsistent labels — without fixing anything yet. Simply seeing how much cleanup real data needs reframes what data science actually is.