Data beats algorithm

Beginners obsess over picking the perfect algorithm. Experienced practitioners know a quieter truth: the features usually matter more than the model. A great algorithm on poor features loses to a simple model on well-crafted ones. Most of the real work — and the real gains — in classical ML happen before you call fit, in shaping the data.

Feature engineering

Features are the input columns the model sees, and you often have to create the useful ones:

• Derive new columns — from a timestamp, extract day-of-week, hour, or "is weekend." From a price and a cost, compute margin.
• Encode categories — models need numbers, so turn "red/green/blue" into numeric form (one-hot encoding).
• Combine — a ratio or interaction such as price per square foot can carry more signal than either raw column.

Good features encode your domain knowledge in a form the model can use.

Preprocessing essentials

A few cleanups apply almost every time:

• Scaling — put numeric features on comparable ranges, since many algorithms stumble when one column is in millions and another in fractions.
• Missing values — decide deliberately: impute with a sensible value or flag them; don't let them silently break things.
• Outliers — investigate extreme values; sometimes they're errors, sometimes the most important cases.

The cardinal rule

Fit all preprocessing on the training data only, then apply it to validation and test — otherwise you leak information and inflate your scores (the leakage trap again). scikit-learn's Pipeline bundles preprocessing and model so this happens correctly by construction.

When a model underperforms, your time is almost always better spent improving the features than swapping the algorithm. Better inputs beat fancier math more often than not.

Try this: Take any dataset with a date column and engineer three features from it — day of week, month, and a weekend flag. Even that small bit of feature work often moves a model's accuracy more than switching algorithms does.