Tensor Parallelism with JAX + Shard Map
-
Updated
Sep 29, 2023 - Python
Tensor Parallelism with JAX + Shard Map
Two implementations of ZeRO-1 optimizer sharding in JAX
Add a description, image, and links to the pjit topic page so that developers can more easily learn about it.
To associate your repository with the pjit topic, visit your repo's landing page and select "manage topics."