Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Jump to bottom

merge lora权重后的模型无法推理 #3738

Closed
1 task done
ironlionliu opened this issue May 14, 2024 · 0 comments
Closed
1 task done

merge lora权重后的模型无法推理 #3738

ironlionliu opened this issue May 14, 2024 · 0 comments
Labels
wontfix This will not be worked on

Comments

@ironlionliu
Copy link

Reminder

  • I have read the README and searched the existing issues.

Reproduction

UDA_VISIBLE_DEVICES=0 API_PORT=8000 llamafactory-cli api examples/inference/llama3_vllm.yaml
当配置文件中为llama3的时候是正常的，当我用自己merge过lora权重后的模型时，会卡在图中这一步
image

Expected behavior

No response

System Info

No response

Others

No response
@hiyouga hiyouga added pending This problem is yet to be addressed. wontfix This will not be worked on and removed pending This problem is yet to be addressed. labels May 24, 2024
@hiyouga hiyouga closed this as not planned Won't fix, can't repro, duplicate, stale May 29, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Assignees
No one assigned
Labels
wontfix This will not be worked on
Projects
None yet
Milestone
No milestone
Development

No branches or pull requests
2 participants
@hiyouga @ironlionliu