机器学习与数据科学博士生系列论坛（第六十七期）—— An Introduction to Scaling Laws of Deep Learning Models

摘要：With the development of deep learning techniques, there has been a consensus that increasing the number of parameters and the size of data in neural networks may lead to better performance. However, the question of how to effectively scale computational resources to improve performance has been a persistent challenge. In order to address this issue, researchers have proposed scaling laws to describe the relationship between the optimal performance of a model and the size of its parameters and dataset.

In this talk, I will introduce the scaling law research of deep learning models and show some connections with the emergent abilities of large language models.

论坛简介：该线上论坛是由张志华教授机器学习实验室组织，每两周主办一次（除了公共假期）。论坛每次邀请一位博士生就某个前沿课题做较为系统深入的介绍，主题包括但不限于机器学习、高维统计学、运筹优化和理论计算机科学。