Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Cut a release branch for GPU runs + flash attention WIP #525

Draft
wants to merge 14 commits into
base: main
Choose a base branch
from
Draft

Cut a release branch for GPU runs + flash attention WIP #525

wants to merge 14 commits into from

Commits on Feb 28, 2024

  1. Make small changes for running it via XPK.

    @yangyuwei
    yangyuwei committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    4307ed2 View commit details
    Browse the repository at this point in the history

  2. Revert changes to docker_build_dependency_image.sh.

    @yangyuwei
    yangyuwei committed Feb 28, 2024
    Configuration menu
    Copy the full SHA
    6b105eb View commit details
    Browse the repository at this point in the history

Commits on Feb 29, 2024

  1. Refactor the code for MaxText training on H100 GPUs via XPK.

    @yangyuwei
    yangyuwei committed Feb 29, 2024
    Configuration menu
    Copy the full SHA
    7888f6a View commit details
    Browse the repository at this point in the history

Commits on Mar 13, 2024

  1. Fix the issue of creating 8 processes per GPU.

    @yangyuwei
    yangyuwei committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    fc0f9cf View commit details
    Browse the repository at this point in the history

  2. Fix the issue of creating 8 processes per GPU for gpu_multi_process_r… 

    …un.sh
    @yangyuwei
    yangyuwei committed Mar 13, 2024
    Configuration menu
    Copy the full SHA
    1046886 View commit details
    Browse the repository at this point in the history

Commits on Mar 14, 2024

  1. Constraint dependency versions 

    Workaround of #516

Also pin other dependencies for mostly reproducible container build
    @chajath
    chajath committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    b259042 View commit details
    Browse the repository at this point in the history

  2. Allow NCCL_DEBUG override

    @chajath
    chajath committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    3437425 View commit details
    Browse the repository at this point in the history

  3. adding helm configs to run llama 7b

    @tejasnamagoogle
    tejasnamagoogle committed Mar 14, 2024
    Configuration menu
    Copy the full SHA
    ddc6b6e View commit details
    Browse the repository at this point in the history

Commits on Mar 15, 2024

  1. Install TransformerEngine needed for FlashAttention.

    @yangyuwei @chajath
    yangyuwei authored and chajath committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    61950fd View commit details
    Browse the repository at this point in the history

  2. Add flash attention

    @chajath
    chajath committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    3fe081c View commit details
    Browse the repository at this point in the history

  3. changing the image

    @tejasnamagoogle
    tejasnamagoogle committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    a19f879 View commit details
    Browse the repository at this point in the history

  4. adding flash attention

    @tejasnamagoogle
    tejasnamagoogle committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    d554870 View commit details
    Browse the repository at this point in the history

  5. minor nit

    @tejasnamagoogle
    tejasnamagoogle committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    39751d9 View commit details
    Browse the repository at this point in the history

  6. changing name to 7b

    @tejasnamagoogle
    tejasnamagoogle committed Mar 15, 2024
    Configuration menu
    Copy the full SHA
    e7b0df2 View commit details
    Browse the repository at this point in the history