Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

[Feature] Integrated Training and Inference -- Part 1 #532

Open
wants to merge 32 commits into
base: main
Choose a base branch
from
Open

[Feature] Integrated Training and Inference -- Part 1 #532

wants to merge 32 commits into from

Commits on Mar 29, 2024

  1. [Feature] Support Sequence parallel (InternLM#456) 

    * support sequence

* add configs

* add sp example to custom dataset

* WIP

* add dispatch utils

* delete useless codes

* move xtuner/engine/sequence_parallel to xtuner/parallel/sequence

* fix lint

* fix lint

* add init_dist to xtuner and add trust_remote_code=True to AutoConfig

* add internlm2 custom_dataset sp4 config

* Sequence Parallel doc V1

* Sequence Parallel doc V1

* Sequence Parallel doc V1

* fix bugs in llama_varlen_attn_forward

* rename indexes to position_ids

* add attn_implementation to config

* delete useless codes

* fix lint

* refine default_collate_fn

* refine doc

* refine doc

* refine doc

* delete replace_internlm2_rote

* add repeat_kv_bshd

* fix apply_rotary_pos_emb bug

* add enable_sequence_parallel flag

* refine doc

* assert {'input_ids', 'labels'}.issubset(dataset.column_names)

* refine doc
    @HIT-cwh @pppppM
    HIT-cwh authored and pppppM committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    360391a View commit details
    Browse the repository at this point in the history

  2. integrated norm chat finetune and inference

    @pppppM
    pppppM committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    5fffd8c View commit details
    Browse the repository at this point in the history

  3. Merge branch 'main' into refactor-llm

    @pppppM
    pppppM committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    0f31481 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into refactor-llm

    @pppppM
    pppppM committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    f1111a9 View commit details
    Browse the repository at this point in the history

  5. remove encoder

    @pppppM
    pppppM committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    7edb76d View commit details
    Browse the repository at this point in the history

  6. add open-source dataset convert tool

    @pppppM
    pppppM committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    c5f38d2 View commit details
    Browse the repository at this point in the history

  7. fix shard count

    @pppppM
    pppppM committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    e0ab003 View commit details
    Browse the repository at this point in the history

  8. fix dataset bugs

    @pppppM
    pppppM committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    b0e71b1 View commit details
    Browse the repository at this point in the history

  9. add alpaca example

    @pppppM
    pppppM committed Mar 29, 2024
    Configuration menu
    Copy the full SHA
    5da532c View commit details
    Browse the repository at this point in the history

Commits on Apr 1, 2024

  1. refactored the inheritance hierarchy

    @pppppM
    pppppM committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    5cfd71f View commit details
    Browse the repository at this point in the history

  2. adjust dir structure

    @pppppM
    pppppM committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    2e1d238 View commit details
    Browse the repository at this point in the history

  3. add BaseAlogrithm docstrings

    @pppppM
    pppppM committed Apr 1, 2024
    Configuration menu
    Copy the full SHA
    046e943 View commit details
    Browse the repository at this point in the history

Commits on Apr 7, 2024

  1. add dataset docstring

    @pppppM
    pppppM committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    68afab4 View commit details
    Browse the repository at this point in the history

  2. add pack dataset docstrings

    @pppppM
    pppppM committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    cf8e8af View commit details
    Browse the repository at this point in the history

  3. remove old collate fn

    @pppppM
    pppppM committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    9dc1142 View commit details
    Browse the repository at this point in the history

  4. Merge branch 'main' into refactor-llm

    @pppppM
    pppppM committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    662cebb View commit details
    Browse the repository at this point in the history

  5. add new chat hook

    @pppppM
    pppppM committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    7ac1e0f View commit details
    Browse the repository at this point in the history

  6. add gradient disable interface

    @pppppM
    pppppM committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    0ad84f2 View commit details
    Browse the repository at this point in the history

  7. add llava dataset example

    @pppppM
    pppppM committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    e85d176 View commit details
    Browse the repository at this point in the history

  8. batch_infer is no longer an abstract method

    @pppppM
    pppppM committed Apr 7, 2024
    Configuration menu
    Copy the full SHA
    9f39627 View commit details
    Browse the repository at this point in the history

Commits on Apr 9, 2024

  1. support auto model

    @pppppM
    pppppM committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    c0655a1 View commit details
    Browse the repository at this point in the history

  2. rename

    @pppppM
    pppppM committed Apr 9, 2024
    Configuration menu
    Copy the full SHA
    fd9ecca View commit details
    Browse the repository at this point in the history

Commits on Apr 11, 2024

  1. update auto model

    @pppppM
    pppppM committed Apr 11, 2024
    Configuration menu
    Copy the full SHA
    15860f9 View commit details
    Browse the repository at this point in the history

Commits on Apr 15, 2024

  1. refactor dataset

    @pppppM
    pppppM committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    7236d40 View commit details
    Browse the repository at this point in the history

  2. enhance dataset convert

    @pppppM
    pppppM committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    4db2955 View commit details
    Browse the repository at this point in the history

  3. remove useless code

    @pppppM
    pppppM committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    aad9ee3 View commit details
    Browse the repository at this point in the history

  4. diff files support diff sample ratios

    @pppppM
    pppppM committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    8daafcb View commit details
    Browse the repository at this point in the history

  5. unified naming

    @pppppM
    pppppM committed Apr 15, 2024
    Configuration menu
    Copy the full SHA
    df60d91 View commit details
    Browse the repository at this point in the history

Commits on Apr 16, 2024

  1. Merge branch 'main' of github.com:InternLM/xtuner into refactor-llm

    @HIT-cwh
    HIT-cwh committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    6185c9b View commit details
    Browse the repository at this point in the history

  2. support sp in TextFinetune

    @HIT-cwh
    HIT-cwh committed Apr 16, 2024
    Configuration menu
    Copy the full SHA
    ca272bf View commit details
    Browse the repository at this point in the history

Commits on Apr 19, 2024

  1. Merge pull request #2 from HIT-cwh/refactor-llm 

    Support sp
    @pppppM
    pppppM committed Apr 19, 2024
    Configuration menu
    Copy the full SHA
    b658d76 View commit details
    Browse the repository at this point in the history

Commits on May 22, 2024

  1. Merge branch 'main' into refactor-llm

    @pppppM
    pppppM committed May 22, 2024
    Configuration menu
    Copy the full SHA
    d2f1002 View commit details
    Browse the repository at this point in the history