Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

Support Eleuther LM-Eval-Harness in Levanter #675

Open
wants to merge 41 commits into
base: main
Choose a base branch
from
Open

Support Eleuther LM-Eval-Harness in Levanter #675

wants to merge 41 commits into from

Commits on May 20, 2024

  1. wip

    @dlwh
    dlwh committed May 20, 2024
    Configuration menu
    Copy the full SHA
    8d1f3a6 View commit details
    Browse the repository at this point in the history

Commits on May 22, 2024

  1. Merge remote-tracking branch 'origin/main' into eval_harness

    @dlwh
    dlwh committed May 22, 2024
    Configuration menu
    Copy the full SHA
    a91afd9 View commit details
    Browse the repository at this point in the history

Commits on May 23, 2024

  1. ok it runs. garbage but it runs?

    @dlwh
    dlwh committed May 23, 2024
    Configuration menu
    Copy the full SHA
    628c525 View commit details
    Browse the repository at this point in the history

  2. don't require imports

    @dlwh
    dlwh committed May 23, 2024
    Configuration menu
    Copy the full SHA
    d4bf9e2 View commit details
    Browse the repository at this point in the history

Commits on May 28, 2024

  1. wio

    @dlwh
    dlwh committed May 28, 2024
    Configuration menu
    Copy the full SHA
    a32308a View commit details
    Browse the repository at this point in the history

  2. wip

    @dlwh
    dlwh committed May 28, 2024
    Configuration menu
    Copy the full SHA
    319eb6a View commit details
    Browse the repository at this point in the history

Commits on Jun 12, 2024

  1. Merge remote-tracking branch 'origin/main' into eval_harness

    @dlwh
    dlwh committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    d0a1560 View commit details
    Browse the repository at this point in the history

  2. almost there

    @dlwh
    dlwh committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    f05b189 View commit details
    Browse the repository at this point in the history

  3. Merge remote-tracking branch 'origin/main' into eval_harness

    @dlwh
    dlwh committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    7ba7f47 View commit details
    Browse the repository at this point in the history

  4. maybe we are there?

    @dlwh
    dlwh committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    19ac049 View commit details
    Browse the repository at this point in the history

  5. launcher

    @dlwh
    dlwh committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    f5dec31 View commit details
    Browse the repository at this point in the history

  6. fix (?) logging of loading time etc.

    @dlwh
    dlwh committed Jun 12, 2024
    Configuration menu
    Copy the full SHA
    12c0b06 View commit details
    Browse the repository at this point in the history

Commits on Jun 13, 2024

  1. wip

    @dlwh
    dlwh committed Jun 13, 2024
    Configuration menu
    Copy the full SHA
    463331e View commit details
    Browse the repository at this point in the history

Commits on Jun 21, 2024

  1. wip

    @dlwh
    dlwh committed Jun 21, 2024
    Configuration menu
    Copy the full SHA
    2d4d0d8 View commit details
    Browse the repository at this point in the history

Commits on Jul 25, 2024

  1. Merge remote-tracking branch 'origin/main' into eval_harness

    @dlwh
    dlwh committed Jul 25, 2024
    Configuration menu
    Copy the full SHA
    f9ccebb View commit details
    Browse the repository at this point in the history

Commits on Nov 15, 2024

  1. Merge remote-tracking branch 'origin/main' into eval_harness

    @dlwh
    dlwh committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    3040956 View commit details
    Browse the repository at this point in the history

  2. off by one

    @dlwh
    dlwh committed Nov 15, 2024
    Configuration menu
    Copy the full SHA
    c7c5f70 View commit details
    Browse the repository at this point in the history

Commits on Nov 21, 2024

  1. Merge remote-tracking branch 'origin/main' into eval_harness

    @dlwh
    dlwh committed Nov 21, 2024
    Configuration menu
    Copy the full SHA
    50c20c0 View commit details
    Browse the repository at this point in the history

Commits on Nov 24, 2024

  1. move logging and types to util to make python's module resolution hap… 

    …pierq
    @dlwh
    dlwh committed Nov 24, 2024
    Configuration menu
    Copy the full SHA
    eb441e3 View commit details
    Browse the repository at this point in the history

  2. hijack HF's download so it works with gcs etc.

    @dlwh
    dlwh committed Nov 24, 2024
    Configuration menu
    Copy the full SHA
    353caa6 View commit details
    Browse the repository at this point in the history

  3. missed some renames?

    @dlwh
    dlwh committed Nov 24, 2024
    Configuration menu
    Copy the full SHA
    72fa689 View commit details
    Browse the repository at this point in the history

  4. rename maybe_fused_next_token_loss

    @dlwh
    dlwh committed Nov 24, 2024
    Configuration menu
    Copy the full SHA
    16195b0 View commit details
    Browse the repository at this point in the history

  5. add some more tests to make sure different seq lens work

    @dlwh
    dlwh committed Nov 24, 2024
    Configuration menu
    Copy the full SHA
    e2cab79 View commit details
    Browse the repository at this point in the history

  6. bump jax version

    @dlwh
    dlwh committed Nov 24, 2024
    Configuration menu
    Copy the full SHA
    4c34eec View commit details
    Browse the repository at this point in the history

  7. depend on my fork

    @dlwh
    dlwh committed Nov 24, 2024
    Configuration menu
    Copy the full SHA
    4ecc630 View commit details
    Browse the repository at this point in the history

  8. eval_harness is about there

    @dlwh
    dlwh committed Nov 24, 2024
    Configuration menu
    Copy the full SHA
    bcfc225 View commit details
    Browse the repository at this point in the history

Commits on Nov 26, 2024

  1. refactor

    @dlwh
    dlwh committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    e0ef6f8 View commit details
    Browse the repository at this point in the history

  2. pad number of requests to proper length

    @dlwh
    dlwh committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    e52b61d View commit details
    Browse the repository at this point in the history

  3. sigh

    @dlwh
    dlwh committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    babaee2 View commit details
    Browse the repository at this point in the history

  4. revert

    @dlwh
    dlwh committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    dbf5341 View commit details
    Browse the repository at this point in the history

  5. Optim config drop stable and add decay (#818)

    @blahBlahhhJ @dlwh
    blahBlahhhJ authored and dlwh committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    f898399 View commit details
    Browse the repository at this point in the history

  6. Bump fsspec (#824)

    @dlwh
    dlwh committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    5235415 View commit details
    Browse the repository at this point in the history

  7. add cycle_length (#825)

    @dlwh
    dlwh committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    4d78749 View commit details
    Browse the repository at this point in the history

  8. Merge remote-tracking branch 'origin/main' into eval_harness

    @dlwh
    dlwh committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    2a2a98f View commit details
    Browse the repository at this point in the history

  9. stack_tree

    @dlwh
    dlwh committed Nov 26, 2024
    Configuration menu
    Copy the full SHA
    ae9624a View commit details
    Browse the repository at this point in the history

Commits on Nov 29, 2024

  1. ok i think we're good

    @dlwh
    dlwh committed Nov 29, 2024
    Configuration menu
    Copy the full SHA
    e97697c View commit details
    Browse the repository at this point in the history

Commits on Nov 30, 2024

  1. ok good enough

    @dlwh
    dlwh committed Nov 30, 2024
    Configuration menu
    Copy the full SHA
    65a661c View commit details
    Browse the repository at this point in the history

  2. kk

    @dlwh
    dlwh committed Nov 30, 2024
    Configuration menu
    Copy the full SHA
    aeb8095 View commit details
    Browse the repository at this point in the history

  3. remove max_examples

    @dlwh
    dlwh committed Nov 30, 2024
    Configuration menu
    Copy the full SHA
    3179f67 View commit details
    Browse the repository at this point in the history

  4. initialize

    @dlwh
    dlwh committed Nov 30, 2024
    Configuration menu
    Copy the full SHA
    69ea6b1 View commit details
    Browse the repository at this point in the history

  5. remove none

    @dlwh
    dlwh committed Nov 30, 2024
    Configuration menu
    Copy the full SHA
    5a4e6ce View commit details
    Browse the repository at this point in the history