Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[BUG] v1.3.8 - custom model (open ai compatible server) returning 400 #1352

Closed
kresimirfijacko opened this issue May 14, 2024 · 1 comment
Closed

Comments

@kresimirfijacko
Copy link

kresimirfijacko commented May 14, 2024

Bug Description
v1.3.8 update doesn't work for using it with vLLM openai compatible server (custom model llama3).
v1.3.6 was working good.

Actual Results
API Error: Status Code 400, {"object":"error","message":"[{'type': 'string_type', 'loc': ('body', 'messages', 0, 'typed-dict', 'content'), 'msg': 'Input should be a valid string', 'input': [{'type': 'text', 'text': 'Current model: casperhansen/llama-3-70b-instruct-awq\\n\\nYou are a helpful assistant. You can help me by answering my questions. You can also ask me questions.'}]}, {'type': 'literal_error', 'loc': ('body', 'messages', 0, 'typed-dict', 'role'), 'msg': \"Input should be 'user'\", 'input': 'system', 'ctx': {'expected': \"'user'\"}}, {'type': 'literal_error', 'loc': ('body', 'messages', 0, 'typed-dict', 'role'), 'msg': \"Input should be 'assistant'\", 'input': 'system', 'ctx': {'expected': \"'assistant'\"}}, {'type': 'string_type', 'loc': ('body', 'messages', 0, 'typed-dict', 'content'), 'msg': 'Input should be a valid string', 'input': [{'type': 'text', 'text': 'Current model: casperhansen/llama-3-70b-instruct-awq\\n\\nYou are a helpful assistant. You can help me by answering my questions. You can also ask me questions.'}]}, {'type': 'string_type', 'loc': ('body', 'messages', 0, 'typed-dict', 'content'), 'msg': 'Input should be a valid string', 'input': [{'type': 'text', 'text': 'Current model: casperhansen/llama-3-70b-instruct-awq\\n\\nYou are a helpful assistant. You can help me by answering my questions. You can also ask me questions.'}]}, {'type': 'literal_error', 'loc': ('body', 'messages', 0, 'typed-dict', 'role'), 'msg': \"Input should be 'tool'\", 'input': 'system', 'ctx': {'expected': \"'tool'\"}}, {'type': 'missing', 'loc': ('body', 'messages', 0, 'typed-dict', 'tool_call_id'), 'msg': 'Field required', 'input': {'role': 'system', 'content': [{'type': 'text', 'text': 'Current model: casperhansen/llama-3-70b-instruct-awq\\n\\nYou are a helpful assistant. You can help me by answering my questions. You can also ask me questions.'}]}}, {'type': 'string_type', 'loc': ('body', 'messages', 0, 'typed-dict', 'content'), 'msg': 'Input should be a valid string', 'input': [{'type': 'text', 'text': 'Current model: casperhansen/llama-3-70b-instruct-awq\\n\\nYou are a helpful assistant. You can help me by answering my questions. You can also ask me questions.'}]}, {'type': 'missing', 'loc': ('body', 'messages', 0, 'typed-dict', 'name'), 'msg': 'Field required', 'input': {'role': 'system', 'content': [{'type': 'text', 'text': 'Current model: casperhansen/llama-3-70b-instruct-awq\\n\\nYou are a helpful assistant. You can help me by answering my questions. You can also ask me questions.'}]}}, {'type': 'literal_error', 'loc': ('body', 'messages', 0, 'typed-dict', 'role'), 'msg': \"Input should be 'function'\", 'input': 'system', 'ctx': {'expected': \"'function'\"}}]","type":"BadRequestError","param":null,"code":400}

Desktop (please complete the following information):

  • Operating System: linux ubuntu 22.04
  • Application Version: 1.3.8
  • vLLM v0.4.2, openai v1.25.1
@Bin-Huang
Copy link
Owner

Bin-Huang commented May 14, 2024

Fixed in v1.3.9
https://github.com/Bin-Huang/chatbox/releases

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

No branches or pull requests

2 participants