ai-explained

Stage 02 — ML Fundamentals

Before deep learning, there was just machine learning: take features, define a loss, optimize parameters, evaluate. Modern AI is built on these primitives, and skipping them produces engineers who can fine-tune a 70B model but can’t read a confusion matrix.

Prerequisites

  • Stage 01 (linear algebra, probability, calculus)

Learning ladder

  1. Supervised learning — regression, classification, the train/val/test split
  2. Unsupervised learning — clustering, dim reduction, density estimation
  3. Loss functions & optimization — MSE, cross-entropy, SGD/Adam
  4. Evaluation & metrics — accuracy, precision/recall, F1, AUC, calibration
  5. Regularization & generalization — bias/variance, L1/L2, early stopping
  6. Classical algorithms — linear/logistic regression, trees, ensembles, kNN, SVMs

MVU

You can:

  • State the bias–variance tradeoff in one sentence
  • Choose between accuracy, F1, and AUC for a given problem and defend it
  • Explain why a 99% accurate model can be useless (class imbalance)
  • Describe the difference between training, validation, and test sets — and what data leakage means

Exercise

Train a logistic regression on the UCI Adult dataset (predict income > $50k). Compute precision, recall, F1, and AUC by hand from a confusion matrix. Compare with scikit-learn’s classification_report.

Why classical ML still matters

  • Tabular data. Gradient-boosted trees (XGBoost, LightGBM, CatBoost) still beat deep nets on most tabular problems.
  • Baselines. Always start with logistic regression / a tree. If a complex model can’t beat it, you have a feature problem, not a model problem.
  • Interpretability. Linear models are inspectable; trees produce rules. Sometimes that matters more than 2 extra points of accuracy.
  • Speed and cost. A scikit-learn model trains in seconds, runs on CPU, and costs ~nothing to serve.

See also