Unable to run chatbot-rag-app using Azure OpenAI model

[thaddeuselastic]

I attempted to run the chatbot-rag-app using an Azure OpenAI model, and found that when I entered a question in the prompt box in the GUI, the service would never produce a response.

I was able to successfully use the docker run --rm --env-file .env chatbot-rag-app flask create-index to index the sample documents to my Elasticsearch cluster, so all ES configuration elements are good.

I assumed I was passing a bad var value in the .env file, so I:

• Double-checked my OPENAI_API_KEY
• Attempted multiple permutations of OPENAI_VERSION - the model resource overview page in the Azure portal shows a model version of 0613, but the annotation in the project docs indicates that the version should appear like 2023-05-15, so I tried using both 0613 and 2023-06-13, but got the same results with both
• Double checked the OPENAI_BASE_URL var, using the Endpoint value shown on the Keys & Endpoints page in the azure portal, both with and without the tailing /

Attached are a redacted copy of my .env file (see above for the additional variations I tried), as well as the docker logs from a container instance where this behavior occurred

redacted_dot_env_file.txt
azureopenai_error_logs.txt

[miguelgrinberg]

The error that you get is this:

openai.error.InvalidRequestError: Resource not found

This would suggest that the configuration values that you provided are incorrect. I have just re-tested Azure to confirm that it works. These are my settings:

LLM_TYPE=azure
OPENAI_VERSION=2023-03-15-preview
OPENAI_BASE_URL=https://ent-search-demo.openai.azure.com/
OPENAI_ENGINE=ent-search-gpt4
OPENAI_API_KEY=<redacted>

Do you want to try with these? I'm not sure if the version and URL are current or out of date, but you can play with the settings after you get it to work.

[joemcelroy]

closing this for now. feel free to re-open if you still unable to resolve it :)

[thaddeuselastic]

Ok, looking to reopen this:

I've had some time to re-attempt this and I'm still unable to get the azure openai mode to work

here's my .env LLM env vars:

LLM_TYPE=azure
OPENAI_VERSION=0613
OPENAI_BASE_URL=https://sa-openai.openai.azure.com/
OPENAI_API_KEY=
OPENAI_ENGINE=gpt-4

Here's the model resource details from the azure-cli:
{
"kind": "OpenAI",
"model": {
"baseModel": null,
"callRateLimit": null,
"capabilities": {
"FineTuneTokensMaxValue": "2000000000",
"FineTuneTokensMaxValuePerExample": "8192",
"chatCompletion": "true",
"scaleType": "Standard"
},
"deprecation": {
"fineTune": "2024-09-30T00:00:00Z",
"inference": "2024-09-30T00:00:00Z"
},
"finetuneCapabilities": null,
"format": "OpenAI",
"isDefaultVersion": true,
"lifecycleStatus": "GenerallyAvailable",
"maxCapacity": 3,
"name": "gpt-4",
"skus": [
{
"capacity": {
"default": 10,
"maximum": 10000,
"minimum": null,
"step": null
},
"deprecationDate": "2024-09-30T00:00:00+00:00",
"name": "Standard",
"rateLimits": [
{
"count": 1.0,
"key": "request",
"renewalPeriod": 10.0,
"rules": null
},
{
"count": 1000.0,
"key": "token",
"renewalPeriod": 60.0,
"rules": null
}
],
"usageName": "OpenAI.Standard.gpt-4"
},
{
"capacity": {
"default": null,
"maximum": 30000,
"minimum": 100,
"step": 100
},
"deprecationDate": "2024-09-30T00:00:00+00:00",
"name": "ProvisionedManaged",
"rateLimits": [
{
"count": 20.0,
"key": "request",
"renewalPeriod": 10.0,
"rules": null
}
],
"usageName": "OpenAI.ProvisionedManaged.gpt-4"
}
],
"source": null,
"systemData": {
"createdAt": "2023-06-19T00:00:00+00:00",
"createdBy": "Microsoft",
"createdByType": "Application",
"lastModifiedAt": "2023-06-19T00:00:00+00:00",
"lastModifiedBy": "Microsoft",
"lastModifiedByType": "Application"
},
"version": "0613"
},
"skuName": "S0"
}

[yansavitski]

Hey
Are u sure about enigne-name='gpt-4'?
For Azure OPENAI_ENGINE means deployment name in Azure not a basic openAI model.
And OPENAI_VERSION is supposed to be likeThe API version to use for this operation. This follows the YYYY-MM-DD format according to Azure API
Could you check those values?
Link to Azure Open AI authentication