demo · 02-ml-fundamentals
Decision tree — growing split by split
Each step adds one axis-aligned cut, chosen by maximum Gini impurity reduction. Colored regions fill in as the tree commits to decisions. Slide max depth to 6 and watch perfect training accuracy appear alongside completely overfit boundaries.
Why it matters
Decision trees are the building block for random forests and gradient boosting — the models that dominated tabular ML before deep learning. The key insight is that every split maximises information gain: it picks the cut that most cleanly separates classes in the resulting sub-regions. Deep trees memorise; shallow trees generalise. The tradeoff is identical to polynomial degree in regression.
The math
Gini(node) = 1 − Σ pᵢ² # pᵢ = fraction of class i
Gain(split) = Gini(parent)
− (nL/n)·Gini(left)
− (nR/n)·Gini(right)
Best split = argmax Gain over all features and thresholds
Anchored to 02-ml-fundamentals/supervised-learning.