Toppertube › Programming & Tech
Building makemore Part 4: Becoming a Backprop Ninja
Andrej Karpathy · Programming & Tech · 1 h 55 min · 391 thousand views · published 2022-10-11
We take the 2-layer MLP (with BatchNorm) from the previous video and backpropagate through it manually without using PyTorch autograd's loss deep learning backpropagation neural network language model chain rule tensors
Watch this lesson free on Toppertube, take notes beside the video, and add it to a course to track your progress and earn a certificate.
More from Andrej Karpathy
- How I use LLMs Andrej Karpathy
- Deep Dive into LLMs like ChatGPT Andrej Karpathy
- Let's reproduce GPT-2 (124M) Andrej Karpathy
- Let's build the GPT Tokenizer Andrej Karpathy
- [1hr Talk] Intro to Large Language Models Andrej Karpathy
- Let's build GPT: from scratch, in code, spelled out. Andrej Karpathy
- Building makemore Part 5: Building a WaveNet Andrej Karpathy
- Building makemore Part 3: Activations & Gradients, BatchNorm Andrej Karpathy
More in Programming & Tech
- The Agents Were Fine. Everything Around Them Broke. - Joey Wang, Harvey AI Engineer
- 60% of Our Data PRs Now Merge With No Human Review - Chetas Joshi, Robinhood AI Engineer
- An Agent That Fixes Its Own Mistakes - Saloni Gandhi & Ziwei Guo, LinkedIn AI Engineer
- Stop Chatting With Spreadsheets — Francesco Galletta & Felippe Felisola Caso, Monzo AI Engineer
- Your Self-Hosted LLM Needs to Say No - Harshul Jain & Parv Khatri, Audible & SAP AI Engineer
- OpenAI spent $1.2 million hacking Linux... and it was a bargain Fireship
- How Gemini 4 Argon & Carbon were built (ft. DeepMind’s Head of Gemini Product) Google Cloud Tech
- How Are People Making Games Like This? Traversy Media