This course traces the path from foundational mathematics to modern large language model training. We begin with the linear algebra and calculus that underpin all of machine learning, build intuition for neural networks, and progressively work toward understanding how LLMs like GPT and Claude are trained at scale.
Each unit includes video lectures from top educators. Work through them in order — later material builds on earlier concepts.
What are vectors, and why are they the language of machine learning? This lesson introduces vectors geometrically and numerically — the building block for everything that follows.
▶ Watch: Vectors — Essence of Linear Algebra, Ch. 1 (3Blue1Brown)How vectors combine to fill space, and what it means for a set of vectors to form a basis.
▶ Watch: Linear Combinations, Span, and Basis Vectors — Essence of Linear Algebra, Ch. 2 (3Blue1Brown)Matrices as transformations of space. The core operation behind every neural network layer.
▶ Watch: Linear Transformations and Matrices — Essence of Linear Algebra, Ch. 3 (3Blue1Brown)How matrix math works and why dot products measure similarity — a concept central to attention mechanisms.
▶ Watch: Matrix Multiplication as Composition — Essence of Linear Algebra, Ch. 4 (3Blue1Brown)A visual, intuitive introduction to neural networks: layers, neurons, weights, biases, and activation functions. No prior ML knowledge required.
▶ Watch: But What Is a Neural Network? — Deep Learning, Ch. 1 (3Blue1Brown)How neural networks learn: computing gradients and updating weights to minimize error.
▶ Watch: Gradient Descent, How Neural Networks Learn — Deep Learning, Ch. 2 (3Blue1Brown)The chain rule applied to networks — the mathematical engine behind training.
How language gets converted into numbers. From one-hot encoding to dense vector representations like Word2Vec.
Early approaches to processing language in order: RNNs, LSTMs, and their limitations.
The breakthrough idea: letting a model focus on different parts of the input when producing each output. Query, key, and value vectors explained.
Putting it all together: multi-head attention, positional encoding, feed-forward layers, and residual connections.
▶ Watch: Attention Is All You Need — Paper Explained (Yannic Kilcher)How LLMs learn language by predicting the next word, trained on massive text corpora.
Why bigger models trained on more data perform better, and how training is parallelized across thousands of GPUs. Sutskever's NeurIPS retrospective covers a decade of lessons on what actually drives progress in neural networks.
▶ Watch: Sequence to Sequence Learning with Neural Networks: What a Decade — NeurIPS 2024 (Ilya Sutskever)Aligning a pre-trained model to follow instructions and be helpful: supervised fine-tuning and reinforcement learning from human feedback.
Rich Sutton's influential 2019 essay argues that general methods leveraging computation (search and learning) have always ultimately outperformed approaches that try to encode human knowledge. A key philosophical underpinning of the scaling era.
▶ Read: The Bitter Lesson (Rich Sutton, 2019)A quick, accessible introduction to neural networks from the Computerphile channel. We were unable to verify the exact video URL — search Computerphile's YouTube channel for their neural network introductions.