Large Language Model Parallelization
22 repos
Training and inference frameworks for distributed large language models, with emphasis on model parallelism techniques across multiple GPUs and TPUs. These repositories implement strategies for splitting transformer models across devices, optimizing communication patterns, and scaling LLM training to billions of parameters. The cluster centers on production systems like Megatron-LM and its variants, alongside supporting tools for distributed training, reinforcement learning from human feedback, and multi-node orchestration.
Python · 20
Jupyter Notebook · 1
speech-translation ·36,832
speaker-recognition ·36,832
speech-synthesis ·36,832
neural-networks ·36,832
deeplearning ·36,832
generative-ai ·36,832
asr ·36,832
speaker-diariazation ·36,832
machine-translation ·36,832
tts ·36,832
