大模型训练-优化器整理

【论文阅读】{MegaScale}:Scaling Large Language Model Training to More Than 10,000 {GPUs}