Efficient Large-scale Language Model Training On Gpu Clusters Using Megatron-lm | Jared Casper

2024 24:04
Synopsis
In this talk we present how we trained a 530B parameter language model on a DGX SuperPOD with over 3000 A100 GPUs and a ...

You Might Also Like

RAS: Efficient Large-Scale Language Model Training on GPU Clusters Using Megatron-LM - G. Perrotta 37:36
Ras: Efficient Large-scale Language Model Training On Gpu Cl...
2023
Efficient Large-Scale Language Model Training on GPU Clusters 22:58
Efficient Large-scale Language Model Training On Gpu Cluster...
2022
Training LLMs at Scale - Deepak Narayanan | Stanford MLSys #83 56:00
Training Llms At Scale - Deepak Narayanan | Stanford Mlsys #...
2024
Efficient Large Scale Language Model Training on GPU Clusters Using Megatron LM 8:17
Efficient Large Scale Language Model Training On Gpu Cluster...
2026
Megatron-LM: Mastering Multi-Billion Parameter Language Models 10:52
Megatron-lm: Mastering Multi-billion Parameter Language Mode...
2024
Ultimate Guide To Scaling ML Models - Megatron-LM | ZeRO | DeepSpeed | Mixed Precision 1:22:58
Ultimate Guide To Scaling Ml Models - Megatron-lm | Zero | D...
2023
The Ultra-Scale Playbook: Training LLMs on GPU Clusters 1:25:56
The Ultra-scale Playbook: Training Llms On Gpu Clusters
2026
Scaling Language Training to Trillion-parameter Models on a GPU Cluster 59:42
Scaling Language Training To Trillion-parameter Models On A ...
2023
From Data to Deployment: Building European AI at Scale 59:54
From Data To Deployment: Building European Ai At Scale
2026
DeepSpeed: All the tricks to scale to gigantic models 39:42
Deepspeed: All The Tricks To Scale To Gigantic Models
2022
Running GPT-3 Models with NVIDIA® NeMo Megatron on Azure 11:32
Running Gpt-3 Models With Nvidia® Nemo Megatron On Azure
2024
Download Options
Choose a download method below. All links open in new tabs.
Service Features Action
Ssvid
MP4 & MP3 • HD Quality • Browser Extension Available
Download
SaveFrom
MP4 & MP3 • HD Quality • Browser Extension Available
Download
Security Notice: These are third-party services. We recommend using antivirus software and being cautious of pop-up ads.