Smart analysis, wrong conclusion

You can run every step correctly and still reach a confidently wrong answer, because the dangerous mistakes in data science are traps in reasoning, not bugs in code. Knowing the classic ones by name is the best defense — once you can spot them, you stop falling for them.

The traps that catch everyone

• Correlation mistaken for causation — ice cream sales and drownings rise together; neither causes the other (summer does). Before claiming X causes Y, ask what else could drive both.
• Selection and survivorship bias — analyzing only the data that "survived" skews everything. Studying only successful startups can't tell you what causes success; the failures you ignored might have done the same things.
• Simpson's paradox — a trend that appears within groups can reverse when you combine them. Always check whether a hidden subgroup is flipping your result.
• Data leakage — letting information you wouldn't have at decision time sneak into the analysis, producing results too good to be true.
• p-hacking — testing many things and reporting only what looked significant. Torture the data long enough and it confesses to anything.

Bias in, bias out

Data reflects the world that produced it, including its biases. A hiring model trained on past decisions learns past prejudices; a survey of app users can't speak for people who never installed it. Ask not just "what does the data say?" but "who or what is missing from it?"

Stay skeptical of your own results

The most important skepticism points inward. A result you want to be true deserves more scrutiny, not less.

When a finding looks surprising or too good, suspect a pitfall before you celebrate. The analyst's job isn't to produce exciting conclusions — it's to produce ones that survive scrutiny.

Try this: Pick a recent "data-driven" claim you've seen and try to argue the opposite using the traps above — could it be reverse causation, survivorship bias, or a hidden subgroup? Practicing the counter-argument is how you build a reflex for catching these in your own work.