https://huggingface.co/docs/accelerate/en/usage_guides/fsdp
Fully Sharded Data Parallel · Hugging Face
We’re on a journey to advance and democratize artificial intelligence through open source and open science.
fully shardeddata parallel
https://huggingface.co/papers/2304.11277
Paper page - PyTorch FSDP: Experiences on Scaling Fully Sharded Data Parallel
Join the discussion on this paper page
fully shardedpaperpytorchdata