DeepSpeed
Deep learning optimization library from Microsoft for training and running very large models efficiently across multiple GPUs.
Overview
DeepSpeed implements memory- and communication-efficient techniques - like ZeRO optimizer state partitioning - that let teams train models far larger than would otherwise fit in available GPU memory, and speeds up training across multiple GPUs and machines. It integrates with PyTorch (also in this catalog) rather than replacing it.
DeepSpeed is a fit for teams training large models that push the limits of available hardware, a specialized optimization layer on top of PyTorch rather than a general-purpose framework - most teams reach for it specifically when standard single-GPU or naive multi-GPU training isn't enough.
- Categories
- AI & Machine Learning
- Languages
- Python
- License
- Apache-2.0
Related Projects
PyTorch
BSD-3-ClauseOpen source machine learning framework that accelerates the path from research prototyping to production deployment.
- AI & Machine Learning
TensorFlow
Apache-2.0End-to-end open source platform for machine learning, from research to production.
- AI & Machine Learning
Unsloth
Apache-2.0Library for fine-tuning large language models significantly faster and with less memory than standard training approaches.
- AI & Machine Learning
CAMEL
Apache-2.0Open-source framework for research and building systems of multiple communicating, role-playing AI agents that collaborate autonomously.
- AI & Machine Learning
Spotted an error? Suggest an edit on GitHub.