Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

[Bug]: 自定义模型没有使用自定义api_host #1117

Open
1 of 2 tasks
zodiacg opened this issue May 11, 2024 · 2 comments
Open
1 of 2 tasks

[Bug]: 自定义模型没有使用自定义api_host #1117

zodiacg opened this issue May 11, 2024 · 2 comments
Labels
bug Something isn't working

Comments

@zodiacg
Copy link

zodiacg commented May 11, 2024

这个bug是否已存在现有issue了?

  • 我确认没有已有issue,且已阅读常见问题

错误表现

使用自定义模型添加模型:

"extra_model_metadata": {
        "GLM-4": {
            "model_type": "OpenAIVision",
            "model_name": "glm-4",
            "multimodal": false,
            "api_host": "http://one-api:3000/",
            "api_key": "sk-XXXXXXXXXXXXXXXXXX",
         }
}

选择模型进行使用时,会报错:

{    "error": {        "message": "Incorrect API key provided: sk-XX*****************************XXX. You can find your API key at https://platform.openai.com/account/api-keys.",        "type": "invalid_request_error",        "param": null,        "code": "invalid_api_key"    }}

经检查发现OpenAPIVision中第40行处,虽然使用api_host生成了新的API URL,但并没有使用。实际使用中使用的是shared.state中的对应OpenAI原始URL

复现操作

  1. Docker部署Chuanhu Chat和OneAPI
  2. 使用自定义模型添加模型:
"extra_model_metadata": {
        "GLM-4": {
            "model_type": "OpenAIVision",
            "model_name": "glm-4",
            "multimodal": false,
            "api_host": "http://one-api:3000/",
            "api_key": "sk-XXXXXXXXXXXXXXXXXX",
         }
}
  1. 使用时选择自定义模型
  2. 控制台报错Incorrect API key

错误日志

No response

运行环境

  • OS: Ubuntu 22.04
  • Browser: Firefox
  • Gradio version:
  • Python version:

帮助解决

  • 我愿意协助解决!

补充说明

No response

@zodiacg zodiacg added the bug Something isn't working label May 11, 2024
@GaiZhenbiao
Copy link
Owner

我看看

@vc12345679
Copy link
Contributor

vc12345679 commented May 13, 2024

举例: extra_model_metadata 设置如下时

"extra_model_metadata": {
      "DeepSeek V2": {
            "model_name": "deepseek-chat",
            "description": "This is DeepSeek's powerful open source LLM.",
            "model_type": "OpenAIVision",
            "multimodal": false,
            "api_host": "https://api.deepseek.com",
            "api_key": "sk-xxxxxxxx",
            "token_limit": 4096,
            "system": "You are a helpful AI assistant.",
            "placeholder": {
                "logo": "https://avatars.githubusercontent.com/u/148330874",
            }
      },
},

OpenAIVision.py 里 __init__(), _get_response(), _single_query_at_once() 三个函数中的变量分别为

self.api_host				https://api.deepseek.com
self.chat_completion_url		https://api.deepseek.com/v1/chat/completions
self.images_completion_url		https://api.deepseek.com/v1/images/generations
self.openai_api_base			https://api.deepseek.com/v1
self.balance_api_url			https://api.deepseek.com/dashboard/billing/credit_grants
self.usage_api_url			https://api.deepseek.com/dashboard/billing/usage
shared.state.chat_completion_url	https://api.openai.com/v1/chat/completions
CHAT_COMPLETION_URL			https://api.openai.com/v1/chat/completions

requests.post() 时的 url 为 shared.state.chat_completion_url,应更正为 self.chat_completion_url #1125 。更正后运行正常

chat_DeepSeek-V2

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
bug Something isn't working
Projects
None yet
Development

No branches or pull requests

3 participants