共计 45 篇文章
2025
【论文阅读】ScheMoE:An Extensible Mixture-of-Experts Distributed Training System with Tasks Scheduling
【Megatron-LM源码分析（二）】-GPT模型pretrain流程
【Megatron-LM源码分析（一）】-环境配置与训练示例跑通
【论文阅读】Rail-only:A Low-Cost High-Performance Network for Training LLMs with Trillion Parameters
【论文阅读】Reducing Activation Recomputation in Large Transformer Models
【论文阅读】Megatron-LM论文阅读
【pytorch-fsdp 源代码阅读（二）】-参数流转
【pytorch-fsdp 源代码阅读（一）】-全流程概览
【Picotron-Tutorial】上下文并行
【Picotron-Tutorial】流水线并行