Stage 01 — Math Foundations

“ML is mostly linear algebra wearing a trench coat.”

You don’t need to be a mathematician. You need fluency in four things:

1. Linear algebra — the language of every operation in ML
2. Probability & statistics — what models are uncertain about and why
3. Calculus & optimization — how learning happens
4. Information theory — what “loss” actually measures

This stage covers each at the depth needed to read papers, debug models, and stop nodding politely when someone says “covariance matrix.”

Prerequisites

• High-school algebra
• A Python environment with NumPy

Learning ladder

Read in this order:

1. Linear algebra — vectors, matrices, dot products, eigendecomposition
2. Probability & statistics — distributions, expectation, MLE, Bayes
3. Calculus & optimization — derivatives, gradients, gradient descent
4. Information theory — entropy, cross-entropy, KL divergence

The order matters: probability uses linear algebra; calculus is the bridge to optimization; information theory ties it all to loss functions.

Minimum viable understanding (MVU)

Before moving to Stage 02 you should be able to:

• Compute a dot product, matrix-vector product, and matrix-matrix product by hand on small examples.
• Explain why softmax outputs are non-negative and sum to 1.
• Write the gradient descent update rule from memory: θ ← θ − η · ∇L(θ).
• Define entropy in plain English and compute it for a 2-outcome distribution.
• Explain what “negative log-likelihood = cross-entropy” means.

Exercises

1. Dot product by hand. For a = [1,2,3] and b = [4,5,6], compute a·b. Then compute it in NumPy. Match.
2. Softmax intuition. Implement softmax on [1,2,3] and [101,102,103]. Same output? Why or why not?
3. Gradient descent on a parabola. Minimize f(x) = (x−3)² starting at x=0 with η=0.1. Plot x and f(x) over 50 steps.
4. Entropy by hand. Compute the entropy of a fair coin and a 90/10 coin. Which is higher?

Common pitfalls

• Skipping it because “I’ll learn it as I go.” You won’t. You’ll cargo-cult papers and never quite trust your own debugging.
• Going too deep too soon. You don’t need measure theory or category theory. Resist the rabbit hole.
• Not coding the math. Math you’ve only read fades. Math you’ve implemented sticks.

Where this stage feeds

• Stage 02 (ML fundamentals) uses linear algebra and calculus end-to-end.
• Stage 03 (neural networks) is calculus + linear algebra at scale.
• Stage 06 (transformers) lives or dies on how well you understand matrix products and softmax.

See also

• Stage 02 — ML Fundamentals
• Stage 03 — Neural Networks