Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Adding the new feature of FPDT #6462

Open
wants to merge 71 commits into
base: master
Choose a base branch
from
Open

Adding the new feature of FPDT #6462

wants to merge 71 commits into from
+1,432 −2

Commits on Aug 3, 2024

  1. fix the bug of deepspeed sequence parallel working with batch size la… 

    …rger than 1
    Jinghan Yao committed Aug 3, 2024
    Configuration menu
    Copy the full SHA
    c076827 View commit details
    Browse the repository at this point in the history

Commits on Aug 6, 2024

  1. Merge branch 'master' into master

    @samadejacobs
    samadejacobs authored Aug 6, 2024
    Configuration menu
    Copy the full SHA
    1b8a8c1 View commit details
    Browse the repository at this point in the history

Commits on Aug 7, 2024

  1. apply yapf formatting

    Jinghan Yao committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    ed34e89 View commit details
    Browse the repository at this point in the history

  2. Formatting fixes

    @loadams
    loadams committed Aug 7, 2024
    Configuration menu
    Copy the full SHA
    89b119e View commit details
    Browse the repository at this point in the history

Commits on Aug 28, 2024

  1. Merge branch 'microsoft:master' into master

    @YJHMITWEB
    YJHMITWEB authored Aug 28, 2024
    Configuration menu
    Copy the full SHA
    7db5798 View commit details
    Browse the repository at this point in the history

Commits on Aug 29, 2024

  1. add FPDT

    Jinghan Yao committed Aug 29, 2024
    Configuration menu
    Copy the full SHA
    0beff24 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into master

    @YJHMITWEB
    YJHMITWEB authored Aug 29, 2024
    Configuration menu
    Copy the full SHA
    4522ed7 View commit details
    Browse the repository at this point in the history

Commits on Sep 6, 2024

  1. Merge branch 'master' into master

    @tjruwase
    tjruwase authored Sep 6, 2024
    Configuration menu
    Copy the full SHA
    c15d1d8 View commit details
    Browse the repository at this point in the history

Commits on Sep 24, 2024

  1. modify streams

    Jinghan Yao committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    69f3892 View commit details
    Browse the repository at this point in the history

  2. modify streams

    Jinghan Yao committed Sep 24, 2024
    Configuration menu
    Copy the full SHA
    8ef9f5a View commit details
    Browse the repository at this point in the history

Commits on Sep 27, 2024

  1. Merge branch 'master' into master

    @loadams
    loadams authored Sep 27, 2024
    Configuration menu
    Copy the full SHA
    b43c5ec View commit details
    Browse the repository at this point in the history

Commits on Oct 7, 2024

  1. remove duplication of alltoall

    Jinghan Yao committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    a55d1f5 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' of github.com:YJHMITWEB/DeepSpeed

    Jinghan Yao committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    1cbd59d View commit details
    Browse the repository at this point in the history

  3. remove duplication of pos

    Jinghan Yao committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    6bfd76f View commit details
    Browse the repository at this point in the history

  4. fix format

    Jinghan Yao committed Oct 7, 2024
    Configuration menu
    Copy the full SHA
    4eeadca View commit details
    Browse the repository at this point in the history

Commits on Oct 10, 2024

  1. Merge branch 'master' into master

    @tohtana
    tohtana authored Oct 10, 2024
    Configuration menu
    Copy the full SHA
    8994991 View commit details
    Browse the repository at this point in the history

  2. fix format and add unit test for fpdt

    Jinghan Yao committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    128286c View commit details
    Browse the repository at this point in the history

  3. Merge branch 'master' of github.com:YJHMITWEB/DeepSpeed 

    fix format and add unit test for fpdt
    Jinghan Yao committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    386f606 View commit details
    Browse the repository at this point in the history

  4. add einops

    Jinghan Yao committed Oct 10, 2024
    Configuration menu
    Copy the full SHA
    ebea5b0 View commit details
    Browse the repository at this point in the history

Commits on Oct 11, 2024

  1. add flashattn

    Jinghan Yao committed Oct 11, 2024
    Configuration menu
    Copy the full SHA
    5c8eec8 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' into master

    @tohtana
    tohtana authored Oct 11, 2024
    Configuration menu
    Copy the full SHA
    a7e175a View commit details
    Browse the repository at this point in the history

Commits on Oct 14, 2024

  1. add requirements for flash-attn in FPDT

    Jinghan Yao committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    764a572 View commit details
    Browse the repository at this point in the history

  2. Merge branch 'master' of github.com:YJHMITWEB/DeepSpeed 

    add requirements for flash-attn in FPDT
    Jinghan Yao committed Oct 14, 2024
    Configuration menu
    Copy the full SHA
    14e91b8 View commit details
    Browse the repository at this point in the history

Commits on Oct 17, 2024

  1. Merge branch 'master' into master

    @tohtana
    tohtana authored Oct 17, 2024
    Configuration menu
    Copy the full SHA
    77dcd38 View commit details
    Browse the repository at this point in the history

Commits on Oct 22, 2024

  1. Merge branch 'master' into master

    @tohtana
    tohtana authored Oct 22, 2024
    Configuration menu
    Copy the full SHA
    7a5c29c View commit details
    Browse the repository at this point in the history

Commits on Nov 5, 2024

  1. skip test when fa is unavailable

    @tohtana
    tohtana committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    534cb93 View commit details
    Browse the repository at this point in the history

  2. formatting

    @tohtana
    tohtana committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    972ddda View commit details
    Browse the repository at this point in the history

  3. add workflow to run a6000 tests

    @tohtana
    tohtana committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    37bc694 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'master' into FPDT

    @tohtana
    tohtana committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    8f8aaa0 View commit details
    Browse the repository at this point in the history

  5. revert world sizes for tests

    @tohtana
    tohtana committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    ac7baf6 View commit details
    Browse the repository at this point in the history

  6. Merge pull request #1 from YJHMITWEB/tohtana/merge_FPDT 

    Fix for tests on DeepSpeed repo
    @tohtana
    tohtana authored Nov 5, 2024
    Configuration menu
    Copy the full SHA
    0d2b624 View commit details
    Browse the repository at this point in the history

  7. update workflow

    @tohtana
    tohtana committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    8935529 View commit details
    Browse the repository at this point in the history

  8. update image version

    @tohtana
    tohtana committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    edd2e05 View commit details
    Browse the repository at this point in the history

  9. remove --no-build-isolation

    @tohtana
    tohtana committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    464d117 View commit details
    Browse the repository at this point in the history

  10. remove requirements file for flash-attn

    @tohtana
    tohtana committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    7389f66 View commit details
    Browse the repository at this point in the history

  11. remove flash-attn requirements from setup.py

    @tohtana
    tohtana committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    5f859be View commit details
    Browse the repository at this point in the history

  12. fix pip command

    @tohtana
    tohtana committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    56cb647 View commit details
    Browse the repository at this point in the history

  13. modify unit test for fpdt

    Jinghan Yao committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    164f459 View commit details
    Browse the repository at this point in the history

  14. modify unit test for fpdt

    Jinghan Yao committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    3eb816d View commit details
    Browse the repository at this point in the history

  15. modify unit test for fpdt

    Jinghan Yao committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    2ae68dc View commit details
    Browse the repository at this point in the history

  16. modify unit test for fpdt

    Jinghan Yao committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    b1b2688 View commit details
    Browse the repository at this point in the history

  17. modify unit test for fpdt

    Jinghan Yao committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    67aa3df View commit details
    Browse the repository at this point in the history

  18. modify unit test for fpdt

    Jinghan Yao committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    42461d2 View commit details
    Browse the repository at this point in the history

  19. modify unit test for fpdt

    Jinghan Yao committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    d637d60 View commit details
    Browse the repository at this point in the history

  20. modify unit test for fpdt

    Jinghan Yao committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    907c79d View commit details
    Browse the repository at this point in the history

  21. modify unit test for fpdt

    Jinghan Yao committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    8f5d039 View commit details
    Browse the repository at this point in the history

  22. modify unit test for fpdt

    Jinghan Yao committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    02c2fbf View commit details
    Browse the repository at this point in the history

  23. modify unit test for fpdt

    Jinghan Yao committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    f570213 View commit details
    Browse the repository at this point in the history

  24. add condition for using fpdt offloading

    Jinghan Yao committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    5b8c419 View commit details
    Browse the repository at this point in the history

  25. add condition for using fpdt offloading

    Jinghan Yao committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    bd090c8 View commit details
    Browse the repository at this point in the history

  26. add flash-attn version check

    Jinghan Yao committed Nov 5, 2024
    Configuration menu
    Copy the full SHA
    e48e85b View commit details
    Browse the repository at this point in the history

Commits on Nov 6, 2024

  1. Merge branch 'master' into master

    @tohtana
    tohtana authored Nov 6, 2024
    Configuration menu
    Copy the full SHA
    af24777 View commit details
    Browse the repository at this point in the history

  2. add unit test directory as test trigger

    @tohtana
    tohtana committed Nov 6, 2024
    Configuration menu
    Copy the full SHA
    ebaf56c View commit details
    Browse the repository at this point in the history

  3. add cron for test and reporting for nightly CI failures

    @tohtana
    tohtana committed Nov 6, 2024
    Configuration menu
    Copy the full SHA
    9e811b8 View commit details
    Browse the repository at this point in the history

Commits on Nov 7, 2024

  1. add multiGPU fpdt unit test

    Jinghan Yao committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    a7522da View commit details
    Browse the repository at this point in the history

  2. add multiGPU fpdt unit test

    Jinghan Yao committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    209adab View commit details
    Browse the repository at this point in the history

  3. add multiGPU fpdt unit test

    Jinghan Yao committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    dbeea8a View commit details
    Browse the repository at this point in the history

  4. add multiGPU fpdt unit test

    Jinghan Yao committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    845e42d View commit details
    Browse the repository at this point in the history

  5. add multiGPU fpdt unit test

    Jinghan Yao committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    8b2549c View commit details
    Browse the repository at this point in the history

  6. add multiGPU fpdt unit test

    Jinghan Yao committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    058c973 View commit details
    Browse the repository at this point in the history

  7. add multiGPU fpdt unit test

    Jinghan Yao committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    0dcc234 View commit details
    Browse the repository at this point in the history

  8. add multiGPU fpdt unit test

    Jinghan Yao committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    d1be5d3 View commit details
    Browse the repository at this point in the history

  9. add multiGPU fpdt unit test

    Jinghan Yao committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    3a0feba View commit details
    Browse the repository at this point in the history

  10. add multiGPU fpdt unit test

    Jinghan Yao committed Nov 7, 2024
    Configuration menu
    Copy the full SHA
    8c57812 View commit details
    Browse the repository at this point in the history

Commits on Nov 8, 2024

  1. add multiGPU fpdt unit test

    Jinghan Yao committed Nov 8, 2024
    Configuration menu
    Copy the full SHA
    43decf6 View commit details
    Browse the repository at this point in the history

  2. add multiGPU fpdt unit test

    Jinghan Yao committed Nov 8, 2024
    Configuration menu
    Copy the full SHA
    d39585c View commit details
    Browse the repository at this point in the history

  3. add multiGPU fpdt unit test

    Jinghan Yao committed Nov 8, 2024
    Configuration menu
    Copy the full SHA
    389b1a3 View commit details
    Browse the repository at this point in the history

  4. add multiGPU fpdt unit test

    Jinghan Yao committed Nov 8, 2024
    Configuration menu
    Copy the full SHA
    958f3bf View commit details
    Browse the repository at this point in the history

  5. add multiGPU fpdt unit test

    Jinghan Yao committed Nov 8, 2024
    Configuration menu
    Copy the full SHA
    af025c5 View commit details
    Browse the repository at this point in the history

  6. Merge branch 'master' into master

    @tohtana
    tohtana authored Nov 8, 2024
    Configuration menu
    Copy the full SHA
    2230377 View commit details
    Browse the repository at this point in the history

Commits on Nov 25, 2024

  1. Merge branch 'master' into master

    @tohtana
    tohtana authored Nov 25, 2024
    Configuration menu
    Copy the full SHA
    7636690 View commit details
    Browse the repository at this point in the history