Error starting worker manager: model llama-cpp@llama.cpp(10.10.12.118:5670) start failed, Traceback (most recent call last) #1479

webhxx · 2024-04-30T03:35:32Z

Search before asking

I had searched in the issues and found no similar issues.

Operating system information

Linux

Python version information

=3.11

DB-GPT version

main

Related scenes

Installation Information

Device information

Intel(R) Xeon(R) CPU E5-2620 0 @ 2.00GHz * 2
2个物理CPU，12个物理核心，24个逻辑核心

Models information

model_name: llama-cpp
model_path: /home/guest/DB-GPT/models/ggml-model-q4_0.gguf
device: cpu
model_type: llama.cpp
prompt_template: vicuna-13b-v1.5

What happened

服务不能启动

What you expected to happen

运行日志如下：
=========================== WebServerParameters ===========================

host: 0.0.0.0
port: 5670
daemon: False
controller_addr: None
model_name: llama-cpp
share: False
remote_embedding: False
log_level: INFO
light: False
log_file: dbgpt_webserver.log
tracer_file: dbgpt_webserver_tracer.jsonl
tracer_storage_cls: None
disable_alembic_upgrade: False
awel_dirs: None
default_thread_pool_size: None

======================================================================

2024-04-30 11:28:01 debian dbgpt.app.base[46348] INFO Database dbgpt already exists
2024-04-30 11:28:01 debian dbgpt.component[46348] INFO Register component with name dbgpt_unified_metadata_db_manager_factory and instance: <dbgpt.storage.metadata.db_factory.UnifiedDBManagerFactory object at 0x7fa26cb05150>
/home/guest/miniconda3/envs/dbgpt_env/lib/python3.10/site-packages/pydantic/_internal/fields.py:160: UserWarning: Field "model_name" has conflict with protected namespace "model".

You may be able to resolve this warning by setting model_config['protected_namespaces'] = ().
warnings.warn(
2024-04-30 11:28:03 debian dbgpt.component[46348] INFO Register component with name dbgpt_thread_pool_default and instance: <dbgpt.util.executor_utils.DefaultExecutorFactory object at 0x7fa266558d00>
2024-04-30 11:28:03 debian dbgpt.component[46348] INFO Register component with name dbgpt_default_scheduler and instance: <dbgpt.app.initialization.scheduler.DefaultScheduler object at 0x7fa2665b0b50>
2024-04-30 11:28:03 debian dbgpt.component[46348] INFO Register component with name dbgpt_model_controller and instance: <dbgpt.model.cluster.controller.controller.ModelControllerAdapter object at 0x7fa273b01330>
2024-04-30 11:28:03 debian dbgpt.component[46348] INFO Register component with name dbgpt_connector_manager and instance: <dbgpt.datasource.manages.connector_manager.ConnectorManager object at 0x7fa26640fd00>
2024-04-30 11:28:03 debian dbgpt.component[46348] INFO Register component with name dbgpt_plugin_hub and instance: <dbgpt.serve.agent.hub.controller.ModulePlugin object at 0x7fa2672bba60>
2024-04-30 11:28:03 debian dbgpt.component[46348] INFO Register component with name dbgpt_multi_agents and instance: <dbgpt.serve.agent.agents.controller.MultiAgents object at 0x7fa26710cd60>
2024-04-30 11:28:03 debian dbgpt.app.initialization.embedding_component[46348] INFO Register local LocalEmbeddingFactory
2024-04-30 11:28:03 debian dbgpt.model.adapter.embeddings_loader[46348] INFO [EmbeddingsModelWorker] Parameters of device is None, use cpu
2024-04-30 11:28:03 debian dbgpt.app.initialization.embedding_component[46348] INFO

=========================== EmbeddingModelParameters ===========================

model_name: text2vec
model_path: /home/guest/DB-GPT/models/text2vec-large-chinese
device: cpu
normalize_embeddings: None

======================================================================

2024-04-30 11:28:04 debian sentence_transformers.SentenceTransformer[46348] INFO Load pretrained SentenceTransformer: /home/guest/DB-GPT/models/text2vec-large-chinese
2024-04-30 11:28:04 debian sentence_transformers.SentenceTransformer[46348] WARNING No sentence-transformers model found with name /home/guest/DB-GPT/models/text2vec-large-chinese. Creating a new one with MEAN pooling.
2024-04-30 11:28:07 debian dbgpt.component[46348] INFO Register component with name embedding_factory and instance: <dbgpt.app.initialization.embedding_component.LocalEmbeddingFactory object at 0x7fa26640f5e0>
2024-04-30 11:28:07 debian dbgpt.component[46348] INFO Register component with name dbgpt_model_cache_manager and instance: <dbgpt.storage.cache.manager.LocalCacheManager object at 0x7fa22353d660>
2024-04-30 11:28:07 debian dbgpt.component[46348] INFO Register component with name dbgpt_awel_trigger_manager and instance: <dbgpt.core.awel.trigger.trigger_manager.DefaultTriggerManager object at 0x7fa22353c550>
2024-04-30 11:28:07 debian dbgpt.component[46348] INFO Register component with name dbgpt_awel_dag_manager and instance: <dbgpt.core.awel.dag.dag_manager.DAGManager object at 0x7fa22353de10>
2024-04-30 11:28:07 debian dbgpt.component[46348] INFO Register component with name dbgpt_agent_manager and instance: <dbgpt.agent.core.agent_manage.AgentManager object at 0x7fa22353df60>
2024-04-30 11:28:07 debian dbgpt.component[46348] INFO Register component with name dbgpt_app_editor_service and instance: <dbgpt.app.openapi.api_v1.editor.service.EditorService object at 0x7fa22353dfc0>
2024-04-30 11:28:07 debian dbgpt.component[46348] INFO Register component with name dbgpt_serve_prompt and instance: <dbgpt.serve.prompt.serve.Serve object at 0x7fa22353e020>
2024-04-30 11:28:07 debian dbgpt.component[46348] INFO Register component with name dbgpt_serve_prompt_service and instance: <dbgpt.serve.prompt.service.service.Service object at 0x7fa22353e050>
2024-04-30 11:28:07 debian dbgpt.component[46348] INFO Register component with name dbgpt_serve_conversation and instance: <dbgpt.serve.conversation.serve.Serve object at 0x7fa21883ae60>
2024-04-30 11:28:07 debian dbgpt.component[46348] INFO Register component with name dbgpt_serve_conversation_service and instance: <dbgpt.serve.conversation.service.service.Service object at 0x7fa21883ad70>
2024-04-30 11:28:07 debian dbgpt.component[46348] INFO Register component with name dbgpt_serve_flow and instance: <dbgpt.serve.flow.serve.Serve object at 0x7fa218802e00>
2024-04-30 11:28:07 debian dbgpt.component[46348] INFO Register component with name dbgpt_dbgpts_loader and instance: <dbgpt.util.dbgpts.loader.DBGPTsLoader object at 0x7fa21e116080>
2024-04-30 11:28:07 debian dbgpt.component[46348] INFO Register component with name dbgpt_serve_flow_service and instance: <dbgpt.serve.flow.service.service.Service object at 0x7fa218802ef0>
2024-04-30 11:28:07 debian dbgpt.component[46348] INFO Register component with name dbgpt_rag and instance: <dbgpt.serve.rag.serve.Serve object at 0x7fa2186d6170>
2024-04-30 11:28:07 debian dbgpt.component[46348] INFO Register component with name dbgpt_rag_service and instance: <dbgpt.serve.rag.service.service.Service object at 0x7fa2186d7d90>
2024-04-30 11:28:07 debian dbgpt.component[46348] INFO Register component with name dbgpt_datasource and instance: <dbgpt.serve.datasource.serve.Serve object at 0x7fa218732680>
2024-04-30 11:28:07 debian dbgpt.component[46348] INFO Register component with name dbgpt_datasource_service and instance: <dbgpt.serve.datasource.service.service.Service object at 0x7fa218765000>
2024-04-30 11:28:07 debian dbgpt.app.base[46348] WARNING For safety considerations, MySQL Database not support DDL init and upgrade. "
"1.If you are use DB-GPT firstly, please manually execute the following command to initialize,
mysql -h127.0.0.1 -uroot -p{your_password} < ./assets/schema/dbgpt.sql "
"2.If there are any changes to the table columns in the DB-GPT database,
it is necessary to compare with the DB-GPT/assets/schema/dbgpt.sql file
and manually make the columns changes in the MySQL database instance.
Model Unified Deployment Mode!
2024-04-30 11:28:07 debian dbgpt.model.cluster.worker.manager[46348] INFO Worker params:

=========================== ModelWorkerParameters ===========================

model_name: llama-cpp
model_path: /home/guest/DB-GPT/models/ggml-model-q4_0.gguf
worker_type: None
model_alias: None
worker_class: None
model_type: huggingface
host: 0.0.0.0
port: 5670
daemon: False
limit_model_concurrency: 5
standalone: True
register: True
worker_register_host: None
controller_addr: None
send_heartbeat: True
heartbeat_interval: 20
log_level: None
log_file: dbgpt_model_worker_manager.log
tracer_file: dbgpt_model_worker_manager_tracer.jsonl
tracer_storage_cls: None

======================================================================

2024-04-30 11:28:07 debian dbgpt.model.cluster.worker.manager[46348] INFO Run WorkerManager with standalone mode, controller_addr: http://127.0.0.1:5670
2024-04-30 11:28:07 debian dbgpt.model.adapter.model_adapter[46348] INFO Use DB-GPT old adapter
2024-04-30 11:28:07 debian dbgpt.model.adapter.old_adapter[46348] INFO Found llm model adapter with model name: llama-cpp, <dbgpt.model.adapter.old_adapter.LlamaCppAdapater object at 0x7fa21818a0e0>
Get model chat adapter with model name llama-cpp, <dbgpt.app.chat_adapter.LlamaCppChatAdapter object at 0x7fa218189ae0>
2024-04-30 11:28:07 debian dbgpt.model.cluster.worker.default_worker[46348] INFO model_name: llama-cpp, model_path: /home/guest/DB-GPT/models/ggml-model-q4_0.gguf, model_param_class: <class 'dbgpt.model.parameter.LlamaCppModelParameters'>
2024-04-30 11:28:07 debian dbgpt.model.cluster.worker.default_worker[46348] INFO [DefaultModelWorker] Parameters of device is None, use cpu
2024-04-30 11:28:07 debian dbgpt.model.cluster.worker.manager[46348] INFO Init empty instances list for llama-cpp@llm
2024-04-30 11:28:07 debian dbgpt.component[46348] INFO Register component with name dbgpt_worker_manager_factory and instance: <dbgpt.model.cluster.worker.manager._DefaultWorkerManagerFactory object at 0x7fa2180efc10>
2024-04-30 11:28:07 debian dbgpt.util.dbgpts.loader[46348] INFO Found 0 dbgpts packages from /home/guest/.dbgpts/packages/2a64323cb20d04b70d0eb0a4f0955fe6
INFO: Started server process [46348]
INFO: Waiting for application startup.
2024-04-30 11:28:08 debian dbgpt.core.awel.dag.loader[46348] INFO Importing /home/guest/DB-GPT/examples/awel/simple_chat_dag_example.py
2024-04-30 11:28:08 debian dbgpt.core.awel.dag.loader[46348] INFO Found dag DAG(dag_id=dbgpt_awel_simple_dag_example) from mod <module 'unusual_prefix_87257987419ac5d0cc1bf718afef7ae70617c34c_simple_chat_dag_example' from '/home/guest/DB-GPT/examples/awel/simple_chat_dag_example.py'> and model file /home/guest/DB-GPT/examples/awel/simple_chat_dag_example.py
2024-04-30 11:28:08 debian dbgpt.core.awel.dag.loader[46348] INFO Importing /home/guest/DB-GPT/examples/awel/simple_chat_history_example.py
2024-04-30 11:28:08 debian dbgpt.core.awel.dag.loader[46348] INFO Found dag DAG(dag_id=dbgpt_awel_simple_chat_history) from mod <module 'unusual_prefix_ccd03f69ef1749b8f132062b283f9c813eee0a43_simple_chat_history_example' from '/home/guest/DB-GPT/examples/awel/simple_chat_history_example.py'> and model file /home/guest/DB-GPT/examples/awel/simple_chat_history_example.py
2024-04-30 11:28:08 debian dbgpt.core.awel.dag.loader[46348] INFO Importing /home/guest/DB-GPT/examples/awel/data_analyst_assistant.py
2024-04-30 11:28:08 debian dbgpt.core.awel.dag.loader[46348] INFO Found dag DAG(dag_id=dbgpt_awel_data_analyst_assistant) from mod <module 'unusual_prefix_65b0f4947ed3b44d36f2129daadd8b5084e3e622_data_analyst_assistant' from '/home/guest/DB-GPT/examples/awel/data_analyst_assistant.py'> and model file /home/guest/DB-GPT/examples/awel/data_analyst_assistant.py
2024-04-30 11:28:08 debian dbgpt.core.awel.dag.loader[46348] INFO Importing /home/guest/DB-GPT/examples/awel/simple_rag_rewrite_example.py
2024-04-30 11:28:08 debian dbgpt.core.awel.dag.loader[46348] INFO Found dag DAG(dag_id=dbgpt_awel_simple_rag_rewrite_example) from mod <module 'unusual_prefix_deb8233492ae48c88e904879656fa15665b08bad_simple_rag_rewrite_example' from '/home/guest/DB-GPT/examples/awel/simple_rag_rewrite_example.py'> and model file /home/guest/DB-GPT/examples/awel/simple_rag_rewrite_example.py
2024-04-30 11:28:08 debian dbgpt.core.awel.dag.loader[46348] INFO Importing /home/guest/DB-GPT/examples/awel/simple_rag_summary_example.py
2024-04-30 11:28:08 debian dbgpt.core.awel.dag.loader[46348] INFO Found dag DAG(dag_id=dbgpt_awel_simple_rag_summary_example) from mod <module 'unusual_prefix_b42bd9a8533723db47607b5516bdd353a2221c5e_simple_rag_summary_example' from '/home/guest/DB-GPT/examples/awel/simple_rag_summary_example.py'> and model file /home/guest/DB-GPT/examples/awel/simple_rag_summary_example.py
2024-04-30 11:28:08 debian dbgpt.core.awel.dag.loader[46348] INFO Importing /home/guest/DB-GPT/examples/awel/simple_dag_example.py
2024-04-30 11:28:08 debian dbgpt.core.awel.dag.loader[46348] INFO Found dag DAG(dag_id=simple_dag_example) from mod <module 'unusual_prefix_b76a5d0a9fd65b03ff7f30bb4ec522182a87c3d1_simple_dag_example' from '/home/guest/DB-GPT/examples/awel/simple_dag_example.py'> and model file /home/guest/DB-GPT/examples/awel/simple_dag_example.py
2024-04-30 11:28:08 debian dbgpt.core.awel.dag.loader[46348] INFO Importing /home/guest/DB-GPT/examples/awel/simple_llm_client_example.py
2024-04-30 11:28:08 debian dbgpt.core.awel.dag.loader[46348] INFO Found dag DAG(dag_id=dbgpt_awel_simple_llm_client_generate) from mod <module 'unusual_prefix_be97d6bab8ddceb0f005712ae772a54ec9ea4332_simple_llm_client_example' from '/home/guest/DB-GPT/examples/awel/simple_llm_client_example.py'> and model file /home/guest/DB-GPT/examples/awel/simple_llm_client_example.py
2024-04-30 11:28:08 debian dbgpt.core.awel.dag.loader[46348] INFO Found dag DAG(dag_id=dbgpt_awel_simple_llm_client_count_token) from mod <module 'unusual_prefix_be97d6bab8ddceb0f005712ae772a54ec9ea4332_simple_llm_client_example' from '/home/guest/DB-GPT/examples/awel/simple_llm_client_example.py'> and model file /home/guest/DB-GPT/examples/awel/simple_llm_client_example.py
2024-04-30 11:28:08 debian dbgpt.core.awel.dag.loader[46348] INFO Importing /home/guest/DB-GPT/examples/awel/simple_nl_schema_sql_chart_example.py
2024-04-30 11:28:08 debian dbgpt.core.awel.dag.loader[46348] INFO Found dag DAG(dag_id=simple_nl_schema_sql_chart_example) from mod <module 'unusual_prefix_910bf7f3edc34d51d6f19df118fd62a4d63e3747_simple_nl_schema_sql_chart_example' from '/home/guest/DB-GPT/examples/awel/simple_nl_schema_sql_chart_example.py'> and model file /home/guest/DB-GPT/examples/awel/simple_nl_schema_sql_chart_example.py
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.trigger_manager[46348] INFO Register trigger HttpTrigger(node_id=e0674696-3177-4b86-bb30-085b5dce96d5)
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.http_trigger[46348] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7fa218703a30>(AWEL_trigger_route__examples_simple_chat), endpoint: /examples/simple_chat, methods: ['POST']
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.http_trigger[46348] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/simple_chat
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.trigger_manager[46348] INFO Include router <fastapi.routing.APIRouter object at 0x7fa22353d810> to prefix path /api/v1/awel/trigger
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.trigger_manager[46348] INFO Register trigger HttpTrigger(node_id=33fa79d0-3c5d-407b-93a1-90ba7cafec40)
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.http_trigger[46348] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7fa2107ac670>(AWEL_trigger_route__examples_simple_history_multi_round_chat_completions), endpoint: /examples/simple_history/multi_round/chat/completions, methods: ['POST']
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.http_trigger[46348] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/simple_history/multi_round/chat/completions
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.trigger_manager[46348] INFO Include router <fastapi.routing.APIRouter object at 0x7fa22353d810> to prefix path /api/v1/awel/trigger
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.trigger_manager[46348] INFO Register trigger HttpTrigger(node_id=eb6f1ee1-c2dc-4431-a6d9-cfb1f4cded70)
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.http_trigger[46348] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7fa2107acca0>(AWEL_trigger_route__examples_data_analyst_copilot), endpoint: /examples/data_analyst/copilot, methods: ['POST']
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.http_trigger[46348] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/data_analyst/copilot
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.trigger_manager[46348] INFO Include router <fastapi.routing.APIRouter object at 0x7fa22353d810> to prefix path /api/v1/awel/trigger
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.trigger_manager[46348] INFO Register trigger HttpTrigger(node_id=82098d63-d37d-4d06-a6cb-9c1523d9bfbf)
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.http_trigger[46348] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7fa2107ad2d0>(AWEL_trigger_route__examples_rag_rewrite), endpoint: /examples/rag/rewrite, methods: ['POST']
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.http_trigger[46348] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/rag/rewrite
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.trigger_manager[46348] INFO Include router <fastapi.routing.APIRouter object at 0x7fa22353d810> to prefix path /api/v1/awel/trigger
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.trigger_manager[46348] INFO Register trigger HttpTrigger(node_id=5c39846e-9bf9-4160-9090-146de8208070)
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.http_trigger[46348] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7fa2107add80>(AWEL_trigger_route__examples_rag_summary), endpoint: /examples/rag/summary, methods: ['POST']
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.http_trigger[46348] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/rag/summary
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.trigger_manager[46348] INFO Include router <fastapi.routing.APIRouter object at 0x7fa22353d810> to prefix path /api/v1/awel/trigger
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.trigger_manager[46348] INFO Register trigger HttpTrigger(node_id=a9ae994c-c011-4987-a66d-c68b80034000)
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.http_trigger[46348] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function_get at 0x7fa2107ae710>(AWEL_trigger_route__examples_hello), endpoint: /examples/hello, methods: ['GET']
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.http_trigger[46348] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/hello
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.trigger_manager[46348] INFO Include router <fastapi.routing.APIRouter object at 0x7fa22353d810> to prefix path /api/v1/awel/trigger
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.trigger_manager[46348] INFO Register trigger HttpTrigger(node_id=5e1bba00-7539-4be1-bafa-49322a0f369c)
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.http_trigger[46348] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7fa2107ae8c0>(AWEL_trigger_route__examples_simple_client_chat_completions), endpoint: /examples/simple_client/chat/completions, methods: ['POST']
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.http_trigger[46348] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/simple_client/chat/completions
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.trigger_manager[46348] INFO Include router <fastapi.routing.APIRouter object at 0x7fa22353d810> to prefix path /api/v1/awel/trigger
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.trigger_manager[46348] INFO Register trigger HttpTrigger(node_id=9dfacd73-994f-4931-9a58-ce9a33895684)
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.http_trigger[46348] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7fa2104b8790>(AWEL_trigger_route__examples_simple_client_count_token), endpoint: /examples/simple_client/count_token, methods: ['POST']
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.http_trigger[46348] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/simple_client/count_token
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.trigger_manager[46348] INFO Include router <fastapi.routing.APIRouter object at 0x7fa22353d810> to prefix path /api/v1/awel/trigger
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.trigger_manager[46348] INFO Register trigger HttpTrigger(node_id=20bcbfb3-1f0e-4340-881d-d5077d84e73d)
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.http_trigger[46348] INFO mount router function <function HttpTrigger._create_route_func..create_route_function..route_function at 0x7fa2107acc10>(AWEL_trigger_route__examples_rag_schema_linking), endpoint: /examples/rag/schema_linking, methods: ['POST']
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.http_trigger[46348] INFO Mount http trigger success, path: /api/v1/awel/trigger/examples/rag/schema_linking
2024-04-30 11:28:08 debian dbgpt.core.awel.trigger.trigger_manager[46348] INFO Include router <fastapi.routing.APIRouter object at 0x7fa22353d810> to prefix path /api/v1/awel/trigger
2024-04-30 11:28:09 debian dbgpt.model.cluster.worker.manager[46348] INFO Begin start all worker, apply_req: None
2024-04-30 11:28:09 debian dbgpt.model.cluster.worker.manager[46348] INFO Apply req: None, apply_func: <function LocalWorkerManager._start_all_worker.._start_worker at 0x7fa21040e4d0>
2024-04-30 11:28:09 debian dbgpt.model.cluster.worker.manager[46348] INFO Apply to all workers
INFO: Application startup complete.
2024-04-30 11:28:09 debian dbgpt.model.cluster.worker.default_worker[46348] INFO Begin load model, model params:

=========================== LlamaCppModelParameters ===========================

model_name: llama-cpp
model_path: /home/guest/DB-GPT/models/ggml-model-q4_0.gguf
device: cpu
model_type: llama.cpp
prompt_template: vicuna-13b-v1.5
max_context_size: 4096
num_gpus: None
max_gpu_memory: None
cpu_offloading: False
load_8bit: True
load_4bit: False
quant_type: nf4
use_double_quant: True
compute_dtype: None
trust_remote_code: True
verbose: False
seed: -1
n_threads: None
n_batch: 512
n_gpu_layers: 1000000000
n_gqa: None
rms_norm_eps: 5e-06
cache_capacity: None
prefer_cpu: False

======================================================================

INFO: Uvicorn running on http://0.0.0.0:5670 (Press CTRL+C to quit)
2024-04-30 11:28:09 debian dbgpt.model.llm.llama_cpp.llama_cpp[46348] INFO Llama.cpp use cpu
2024-04-30 11:28:09 debian dbgpt.model.llm.llama_cpp.llama_cpp[46348] INFO Llama.cpp use cpu
2024-04-30 11:28:09 debian dbgpt.model.llm.llama_cpp.llama_cpp[46348] INFO Cache capacity is 0 bytes
2024-04-30 11:28:09 debian dbgpt.model.llm.llama_cpp.llama_cpp[46348] INFO Load LLama model with params: {'model_path': '/home/guest/DB-GPT/models/ggml-model-q4_0.gguf', 'n_ctx': 4096, 'seed': -1, 'n_threads': None, 'n_batch': 512, 'use_mmap': True, 'use_mlock': False, 'low_vram': False, 'n_gpu_layers': 1000000000, 'n_gqa': None, 'logits_all': True, 'rms_norm_eps': 5e-06}
llama_model_loader: loaded meta data with 22 key-value pairs and 403 tensors from /home/guest/DB-GPT/models/ggml-model-q4_0.gguf (version GGUF V3 (latest))
llama_model_loader: Dumping metadata keys/values. Note: KV overrides do not apply in this output.
llama_model_loader: - kv 0: general.architecture str = llama
llama_model_loader: - kv 1: general.name str = vicuna-13b-v1.5
llama_model_loader: - kv 2: llama.block_count u32 = 40
llama_model_loader: - kv 3: llama.context_length u32 = 4096
llama_model_loader: - kv 4: llama.embedding_length u32 = 5120
llama_model_loader: - kv 5: llama.feed_forward_length u32 = 13824
llama_model_loader: - kv 6: llama.attention.head_count u32 = 40
llama_model_loader: - kv 7: llama.attention.head_count_kv u32 = 40
llama_model_loader: - kv 8: llama.attention.layer_norm_rms_epsilon f32 = 0.000010
llama_model_loader: - kv 9: general.file_type u32 = 1
llama_model_loader: - kv 10: llama.vocab_size u32 = 32000
llama_model_loader: - kv 11: llama.rope.dimension_count u32 = 128
llama_model_loader: - kv 12: tokenizer.ggml.model str = llama
llama_model_loader: - kv 13: tokenizer.ggml.pre str = default
llama_model_loader: - kv 14: tokenizer.ggml.tokens arr[str,32000] = ["", "~~", "~~", "<0x00>", "<...
llama_model_loader: - kv 15: tokenizer.ggml.scores arr[f32,32000] = [0.000000, 0.000000, 0.000000, 0.0000...
llama_model_loader: - kv 16: tokenizer.ggml.token_type arr[i32,32000] = [2, 3, 3, 6, 6, 6, 6, 6, 6, 6, 6, 6, ...
llama_model_loader: - kv 17: tokenizer.ggml.bos_token_id u32 = 1
llama_model_loader: - kv 18: tokenizer.ggml.eos_token_id u32 = 2
llama_model_loader: - kv 19: tokenizer.ggml.padding_token_id u32 = 0
llama_model_loader: - kv 20: tokenizer.ggml.add_bos_token bool = true
llama_model_loader: - kv 21: tokenizer.ggml.add_eos_token bool = false
llama_model_loader: - type f32: 121 tensors
llama_model_loader: - type f16: 282 tensors
llm_load_vocab: special tokens definition check successful ( 259/32000 ).
llm_load_print_meta: format = GGUF V3 (latest)
llm_load_print_meta: arch = llama
llm_load_print_meta: vocab type = SPM
llm_load_print_meta: n_vocab = 32000
llm_load_print_meta: n_merges = 0
llm_load_print_meta: n_ctx_train = 4096
llm_load_print_meta: n_embd = 5120
llm_load_print_meta: n_head = 40
llm_load_print_meta: n_head_kv = 40
llm_load_print_meta: n_layer = 40
llm_load_print_meta: n_rot = 128
llm_load_print_meta: n_embd_head_k = 128
llm_load_print_meta: n_embd_head_v = 128
llm_load_print_meta: n_gqa = 1
llm_load_print_meta: n_embd_k_gqa = 5120
llm_load_print_meta: n_embd_v_gqa = 5120
llm_load_print_meta: f_norm_eps = 0.0e+00
llm_load_print_meta: f_norm_rms_eps = 1.0e-05
llm_load_print_meta: f_clamp_kqv = 0.0e+00
llm_load_print_meta: f_max_alibi_bias = 0.0e+00
llm_load_print_meta: f_logit_scale = 0.0e+00
llm_load_print_meta: n_ff = 13824
llm_load_print_meta: n_expert = 0
llm_load_print_meta: n_expert_used = 0
llm_load_print_meta: causal attn = 1
llm_load_print_meta: pooling type = 0
llm_load_print_meta: rope type = 0
llm_load_print_meta: rope scaling = linear
llm_load_print_meta: freq_base_train = 10000.0
llm_load_print_meta: freq_scale_train = 1
llm_load_print_meta: n_yarn_orig_ctx = 4096
llm_load_print_meta: rope_finetuned = unknown
llm_load_print_meta: ssm_d_conv = 0
llm_load_print_meta: ssm_d_inner = 0
llm_load_print_meta: ssm_d_state = 0
llm_load_print_meta: ssm_dt_rank = 0
llm_load_print_meta: model type = 13B
llm_load_print_meta: model ftype = F16
llm_load_print_meta: model params = 13.02 B
llm_load_print_meta: model size = 24.24 GiB (16.00 BPW)
llm_load_print_meta: general.name = vicuna-13b-v1.5
llm_load_print_meta: BOS token = 1 ''
llm_load_print_meta: EOS token = 2 ''
llm_load_print_meta: UNK token = 0 ''
llm_load_print_meta: PAD token = 0 ''
llm_load_print_meta: LF token = 13 '<0x0A>'
llm_load_tensors: ggml ctx size = 0.20 MiB
llama_model_load: error loading model: done_getting_tensors: wrong number of tensors; expected 403, got 363
llama_load_model_from_file: failed to load model
2024-04-30 11:28:09 debian dbgpt.model.cluster.worker.manager[46348] ERROR Error starting worker manager: model llama-cpp@llama.cpp(10.10.12.118:5670) start failed, Traceback (most recent call last):
File "/home/guest/DB-GPT/dbgpt/model/cluster/worker/manager.py", line 507, in _start_worker
await self.run_blocking_func(
File "/home/guest/DB-GPT/dbgpt/model/cluster/worker/manager.py", line 105, in run_blocking_func
return await loop.run_in_executor(self.executor, func, *args)
File "/home/guest/miniconda3/envs/dbgpt_env/lib/python3.10/concurrent/futures/thread.py", line 58, in run
result = self.fn(*self.args, **self.kwargs)
File "/home/guest/DB-GPT/dbgpt/model/cluster/worker/default_worker.py", line 116, in start
self.model, self.tokenizer = self.ml.loader_with_params(
File "/home/guest/DB-GPT/dbgpt/model/adapter/loader.py", line 127, in loader_with_params
return llamacpp_loader(llm_adapter, model_params)
File "/home/guest/DB-GPT/dbgpt/model/adapter/loader.py", line 417, in llamacpp_loader
model, tokenizer = LlamaCppModel.from_pretrained(model_path, model_params)
File "/home/guest/DB-GPT/dbgpt/model/llm/llama_cpp/llama_cpp.py", line 88, in from_pretrained
result.model = Llama(**params)
File "/home/guest/miniconda3/envs/dbgpt_env/lib/python3.10/site-packages/llama_cpp/llama.py", line 323, in init
self._model = _LlamaModel(
File "/home/guest/miniconda3/envs/dbgpt_env/lib/python3.10/site-packages/llama_cpp/_internals.py", line 55, in init
raise ValueError(f"Failed to load model from file: {path_model}")
ValueError: Failed to load model from file: /home/guest/DB-GPT/models/ggml-model-q4_0.gguf

INFO: Shutting down
INFO: Waiting for application shutdown.
2024-04-30 11:28:09 debian dbgpt.model.cluster.worker.manager[46348] INFO Stop all workers
2024-04-30 11:28:09 debian dbgpt.model.cluster.worker.manager[46348] INFO Apply req: None, apply_func: <function LocalWorkerManager._stop_all_worker.._stop_worker at 0x7fa21040e4d0>
2024-04-30 11:28:09 debian dbgpt.model.cluster.worker.manager[46348] INFO Apply to all workers
2024-04-30 11:28:09 debian dbgpt.model.cluster.worker.manager[46348] WARNING Stop worker, ignored exception from deregister_func: All connection attempts failed
2024-04-30 11:28:09 debian dbgpt.model.cluster.worker.default_worker[46348] WARNING Model has been stopped!!
2024-04-30 11:28:09 debian dbgpt.model.cluster.worker.manager[46348] WARNING Stop worker, ignored exception from deregister_func: All connection attempts failed
INFO: Application shutdown complete.
INFO: Finished server process [46348]
in order to avoid chroma db atexit problem

How to reproduce

=========================== WebServerParameters ===========================