Toppertube › Programming & Tech
Let's build GPT: from scratch, in code, spelled out.
Andrej Karpathy · Programming & Tech · 1 h 56 min · 8.2 million views · published 2023-01-17
We build a Generatively Pretrained Transformer (GPT), following the paper "Attention is All You Need" and OpenAI's GPT-2 / GPT-3. We talk ab deep learning neural network language model pytorch gpt chatgpt openai generati
Watch this lesson free on Toppertube, take notes beside the video, and add it to a course to track your progress and earn a certificate.
More from Andrej Karpathy
- How I use LLMs Andrej Karpathy
- Deep Dive into LLMs like ChatGPT Andrej Karpathy
- Let's reproduce GPT-2 (124M) Andrej Karpathy
- Let's build the GPT Tokenizer Andrej Karpathy
- [1hr Talk] Intro to Large Language Models Andrej Karpathy
- Building makemore Part 5: Building a WaveNet Andrej Karpathy
- Building makemore Part 4: Becoming a Backprop Ninja Andrej Karpathy
- Building makemore Part 3: Activations & Gradients, BatchNorm Andrej Karpathy
More in Programming & Tech
- The Agents Were Fine. Everything Around Them Broke. - Joey Wang, Harvey AI Engineer
- 60% of Our Data PRs Now Merge With No Human Review - Chetas Joshi, Robinhood AI Engineer
- An Agent That Fixes Its Own Mistakes - Saloni Gandhi & Ziwei Guo, LinkedIn AI Engineer
- Stop Chatting With Spreadsheets — Francesco Galletta & Felippe Felisola Caso, Monzo AI Engineer
- Your Self-Hosted LLM Needs to Say No - Harshul Jain & Parv Khatri, Audible & SAP AI Engineer
- OpenAI spent $1.2 million hacking Linux... and it was a bargain Fireship
- How Gemini 4 Argon & Carbon were built (ft. DeepMind’s Head of Gemini Product) Google Cloud Tech
- How Are People Making Games Like This? Traversy Media