Tom B. Brown, Benjamin Mann, Nick Ryder, Melanie Subbiah, Jared Kaplan, Prafulla Dhariwal, Arvind Neelakantan, Pranav Shyam, Girish Sastry, Amanda Askell, Sandhini Agarwal, Ariel Herbert-Voss, Gretchen Krueger, Tom Henighan, Rewon Child, Aditya Ramesh, Daniel M. Ziegler, Jeffrey Wu, Clemens Winter, Christopher Hesse, Mark Chen, Eric Sigler, Mateusz Litwin, Scott Gray, Benjamin Chess, Jack Clark, Christopher Berner, Sam McCandlish, Alec Radford, Ilya Sutskever, & Dario Amodei (2020)
arXiv.
DOI: https://doi.org/10.48550/arxiv.2005.14165
Abstract. Introduces GPT-3, a 175-billion-parameter language model that exhibits strong few-shot and in-context learning: given a few examples in the prompt, the model performs novel tasks without any parameter updates. GPT-3 established in-context learning as an emergent property of scale.
Tags: transformer gpt-3 language-models in-context-learning