Jason Wei, Yi Tay, Rishi Bommasani, Colin Raffel, Barret Zoph, Sebastian Borgeaud, Dani Yogatama, Maarten Bosma, Denny Zhou, Donald Metzler, Ed H. Chi, Tatsunori Hashimoto, Oriol Vinyals, Percy Liang, Jeff Dean, & William Fedus (2022)
arXiv.
DOI: https://doi.org/10.48550/arxiv.2206.07682
Abstract. Catalogues a variety of LLM capabilities, arithmetic, multi-step reasoning, translation, that appear to 'emerge' abruptly at particular scale thresholds, arguing that such phase transitions distinguish large from small models.
Tags: scaling language-models emergence