OpenAPI intermittently unresponsive

[dc-vin]

Self Checks

• This is only for bug report, if you would like to ask a quesion, please head to Discussions.
• I have searched for existing issues search for existing issues, including closed ones.
• I confirm that I am using English to submit this report (我已阅读并同意 Language Policy).
• Pleas do not modify this template :) and fill in all the required fields.

Dify version

0.6.5

Cloud or Self Hosted

Self Hosted (Docker)

Steps to reproduce

When I use Postman or Apifox to request the /v1/chat-messages endpoint, using stream mode, the interface intermittently fails to respond and is unstable. From the dify console logs, I can see that the request results are normal, but they are not properly returned to Apifox.

✔️ Expected Behavior

No response

❌ Actual Behavior

No response

[dosubot]

The intermittent unresponsiveness you're encountering with the /v1/chat-messages endpoint in stream mode could be due to several factors, particularly how streaming responses are managed by your HTTP clients (Postman or Apifox) and the configuration of your self-hosted Dify setup in Docker.

For HTTP clients like Postman or Apifox, ensure they are correctly configured to handle streaming responses. This involves setting the request to include a parameter or configuration that specifies the response mode as 'streaming'. Misconfiguration here or network latency issues could lead to the intermittent unresponsiveness observed.

On the Docker side, network configurations, resource limitations, or Docker's internal networking can affect the stability and responsiveness of streaming endpoints. Optimizing Docker container resources and configurations for real-time streaming data is crucial. This includes adjusting GUNICORN_TIMEOUT, using gevent for SERVER_WORKER_CLASS, and configuring the number of API server workers appropriately. Additionally, consider Docker-specific optimizations such as Redis for caching, database scalability configurations, vector store configurations, using Nginx as a reverse proxy, and environment configurations for performance tuning.

These steps should help mitigate the issues you're facing with the /v1/chat-messages endpoint. If the problem persists, providing more details about your setup and the specific issues encountered could help in further troubleshooting.

Details

Sources

• Self Hosted (Docker), api call /chat-messages with response_mode: streaming, the message_end response lasts up to 5 seconds.
• sdks/python-client/dify_client/client.py
• sdks/php-client/dify-client.php
• en/getting-started/install-self-hosted/environments.md
• web/docker/pm2.json
• docker/docker-compose.yaml
• api/docker/entrypoint.sh

About Dosu
This response is meant to be useful and save you time. It is not meant to be a precise solution, but rather a starting point for your own research.

Help me be more useful! Please leave a 👍 if this is helpful and 👎 if it is irrelevant.

To continue the conversation, mention @dosu.