Stage 02 — ML Fundamentals

Before deep learning, there was just machine learning: take features, define a loss, optimize parameters, evaluate. Modern AI is built on these primitives, and skipping them produces engineers who can fine-tune a 70B model but can’t read a confusion matrix.

Prerequisites

Stage 01 (linear algebra, probability, calculus)

Learning ladder

Supervised learning — regression, classification, the train/val/test split
Unsupervised learning — clustering, dim reduction, density estimation
Loss functions & optimization — MSE, cross-entropy, SGD/Adam
Evaluation & metrics — accuracy, precision/recall, F1, AUC, calibration
Regularization & generalization — bias/variance, L1/L2, early stopping
Classical algorithms — linear/logistic regression, trees, ensembles, kNN, SVMs

MVU

You can:

State the bias–variance tradeoff in one sentence
Choose between accuracy, F1, and AUC for a given problem and defend it
Explain why a 99% accurate model can be useless (class imbalance)
Describe the difference between training, validation, and test sets — and what data leakage means

Exercise

Train a logistic regression on the UCI Adult dataset (predict income > $50k). Compute precision, recall, F1, and AUC by hand from a confusion matrix. Compare with scikit-learn’s classification_report.

Why classical ML still matters

Tabular data. Gradient-boosted trees (XGBoost, LightGBM, CatBoost) still beat deep nets on most tabular problems.
Baselines. Always start with logistic regression / a tree. If a complex model can’t beat it, you have a feature problem, not a model problem.
Interpretability. Linear models are inspectable; trees produce rules. Sometimes that matters more than 2 extra points of accuracy.
Speed and cost. A scikit-learn model trains in seconds, runs on CPU, and costs ~nothing to serve.