Thanks. It'll be nice to adjust the retry time according to the error msg.
One workaround is to set flaml.oai.retry_time = 60 in your code for now if that's the most common retry time required.

I also get this error:

[autogen.oai.completion: 09-27 12:50:43] {236} INFO - retrying in 10 seconds...
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/autogen/oai/completion.py", line 206, in _get_response
response = openai_completion.create(**config)
File "/usr/local/lib/python3.10/dist-packages/openai/api_resources/chat_completion.py", line 25, in create
return super().create(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/openai/api_resources/abstract/engine_api_resource.py", line 155, in create
response, _, api_key = requestor.request(
File "/usr/local/lib/python3.10/dist-packages/openai/api_requestor.py", line 299, in request
resp, got_stream = self._interpret_response(result, stream)
File "/usr/local/lib/python3.10/dist-packages/openai/api_requestor.py", line 710, in _interpret_response
self._interpret_response_line(
File "/usr/local/lib/python3.10/dist-packages/openai/api_requestor.py", line 775, in _interpret_response_line
raise self.handle_error_response(
openai.error.RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-6oXqS68sE8UL8MSONa3w2IpY on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method.
INFO:autogen.oai.completion:retrying in 10 seconds...
Traceback (most recent call last):
File "/usr/local/lib/python3.10/dist-packages/autogen/oai/completion.py", line 206, in _get_response
response = openai_completion.create(**config)
File "/usr/local/lib/python3.10/dist-packages/openai/api_resources/chat_completion.py", line 25, in create
return super().create(*args, **kwargs)
File "/usr/local/lib/python3.10/dist-packages/openai/api_resources/abstract/engine_api_resource.py", line 155, in create
response, _, api_key = requestor.request(
File "/usr/local/lib/python3.10/dist-packages/openai/api_requestor.py", line 299, in request
resp, got_stream = self._interpret_response(result, stream)
File "/usr/local/lib/python3.10/dist-packages/openai/api_requestor.py", line 710, in _interpret_response
self._interpret_response_line(
File "/usr/local/lib/python3.10/dist-packages/openai/api_requestor.py", line 775, in _interpret_response_line
raise self.handle_error_response(
openai.error.RateLimitError: Rate limit reached for default-gpt-3.5-turbo in organization org-6oXqS68sE8UL8MSONa3w2IpY on requests per min. Limit: 3 / min. Please try again in 20s. Contact us through our help center at help.openai.com if you continue to have issues. Please add a payment method to your account to increase your rate limit. Visit https://platform.openai.com/account/billing to add a payment method.

Is there a configuration to set the max numbers of requests that can be made per minute? This would help to avoid hitting rate limits for Requests Per Minute. 😅

@Pavel-hb @RobertWeaver I added some answers in #53 . Could you take a look and let me know if it answers your question?

I also observed the same behavior that retry occurs with 10 seconds, it definitely should retry based on the time in the error message.
Also is there a way to dynamically reduce max tokens? I have seen in some instances with GPT 3.5 Turbo 16k that the output in one chat goes beyond the token limit- one should calculate the token count in system and user prompt and then set max token based on the model - max token limit- the token used in user prompt and system prompt?

The retry time behavior has been changed in v0.2. https://microsoft.github.io/autogen/docs/Installation#python
Dynamically reducing max tokens is an idea @kevin666aa may want to take a note.

The retry time behavior has been changed in v0.2. https://microsoft.github.io/autogen/docs/Installation#python Dynamically reducing max tokens is an idea @kevin666aa may want to take a note.

The "calculate the token count in system and user prompt" are already the features of CompressibeAgent.

I think "dynamically reducing max tokens" is not necessary to have. The similar thing is achieved by the "TERMINATE" mode of CompressibeAgent: when the token count is smaller than max token limit, the completion will be allowed. OpenAI will automatically return when max token limit of the model is reached. After that, the next completion will be terminated due to token count limit. So, no retry due to token limit will be happen.

@raghavgrover13 Can you checkout https://github.com/microsoft/autogen/blob/main/notebook/agentchat_compression.ipynb to see if it helps with your problem?