1989–, Computer scientist
Alec Radford is an American computer scientist who, at OpenAI, has been the first author of a long string of foundational papers: GPT (2018), GPT-2 (2019), CLIP (2021) and Whisper (2022). His career trajectory tracks the history of generative AI in the late 2010s and early 2020s with unusual clarity.
The 2018 GPT paper Improving Language Understanding by Generative Pre-Training (with Karthik Narasimhan, Tim Salimans and Ilya Sutskever) introduced the decoder-only autoregressive Transformer pre-training paradigm. GPT-2 (2019) scaled this up and demonstrated that staged release was warranted given misuse concerns. CLIP (2021) introduced contrastive image-text pretraining that became the foundation of multimodal models. Whisper (2022) demonstrated that the same recipe extended to speech recognition.
Radford's productivity, technical taste and lack of public profile make him one of the most consequential AI researchers most non-specialists have never heard of. He left OpenAI in December 2024 to pursue independent research.
Video
Related people: Ilya Sutskever, Sam Altman
Works cited in this book:
- Unsupervised Representation Learning with Deep Convolutional Generative Adversarial Networks (2015) (with Luke Metz, Soumith Chintala)
- Proximal Policy Optimization Algorithms (2017) (with John Schulman, Filip Wolski, Prafulla Dhariwal, Oleg Klimov)
- Improving Language Understanding by Generative Pre-Training (2018) (with Karthik Narasimhan, Tim Salimans, Ilya Sutskever)
- Language Models are Unsupervised Multitask Learners (2019) (with Jeffrey Wu, Rewon Child, David Luan, Dario Amodei, Ilya Sutskever)
- Language Models are Few-Shot Learners (2020) (with Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Ilya Sutskever, Dario Amodei)
- Scaling Laws for Neural Language Models (2020) (with Jared Kaplan, Sam McCandlish, Tom Henighan, Tom B. Brown, Benjamin Chess, Rewon Child, Scott Gray, Jeffrey Wu, Dario Amodei)
- Learning Transferable Visual Models From Natural Language Supervision (2021) (with Jong Wook Kim, Chris Hallacy, Aditya Ramesh, Gabriel Goh, Sandhini Agarwal, Girish Sastry, Amanda Askell, Pamela Mishkin, Jack Clark, Gretchen Krueger, Ilya Sutskever)
- Robust Speech Recognition via Large-Scale Weak Supervision (2023) (with Jong Wook Kim, Tao Xu, Greg Brockman, Christine McLeavey, Ilya Sutskever)
Discussed in:
- Chapter 13: Attention & Transformers, Attention and Transformers