共计 27 篇文章

---

2025

12-28

【论文阅读】ByteScale:Efficient Scaling of LLM Training with a 2048K Context Length on More Than 12,000

12-28

【Megatron-LM源码分析（四）】-DDP数据并行

12-26

【Megatron-LM源码分析（三）】-性能分析

12-22

【论文阅读】ScheMoE:An Extensible Mixture-of-Experts Distributed Training System with Tasks Scheduling

12-22

【Megatron-LM源码分析（二）】-GPT模型pretrain流程

12-14

【Megatron-LM源码分析（一）】-环境配置与训练示例跑通

12-08

【论文阅读】Reducing Activation Recomputation in Large Transformer Models

12-07

【论文阅读】Megatron-LM论文阅读

07-12

【pytorch-fsdp 源代码阅读（二）】-参数流转

07-02

【pytorch-fsdp 源代码阅读（一）】-全流程概览