You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
If a deployment has 0 failed requests, it should be skipped from the "Top 5 Deployments with Most Failed Requests" report.
Similarly, if a deployment has 0s/token latency, it should be skipped from the "Top 5 Slowest Deployments" report.
Example daily report sent in Slack today:
Here are today's key metrics :chart_with_upwards_trend::
:exclamation: Top 5 Deployments with Most Failed Requests:
1. Deployment: azure/gpt-35-turbo-0125, Failed Requests: 0, API Base: https://xyz1.openai.azure.com
2. Deployment: azure/gpt-35-turbo-0125, Failed Requests: 0, API Base: https://xyz2.openai.azure.com
3. Deployment: azure/gpt-35-turbo-0125, Failed Requests: 0, API Base: https://xyz3.openai.azure.com
4. Deployment: azure/gpt-35-turbo-0125, Failed Requests: 0, API Base: https://xyz4.openai.azure.com
5. Deployment: azure/gpt-35-turbo-0125, Failed Requests: 0, API Base: https://xyz5.openai.azure.com
:sweat_smile: Top 5 Slowest Deployments:
1. Deployment: azure/gpt-35-turbo-0125, Latency per output token: 0s/token, API Base: https://xyz1.openai.azure.com
2. Deployment: azure/gpt-35-turbo-0125, Latency per output token: 0s/token, API Base: https://xyz2.openai.azure.com
3. Deployment: azure/gpt-35-turbo-0125, Latency per output token: 0s/token, API Base: https://xyz3.openai.azure.com
4. Deployment: azure/gpt-35-turbo-0125, Latency per output token: 0s/token, API Base: https://xyz4.openai.azure.com
5. Deployment: azure/gpt-35-turbo-0125, Latency per output token: 0s/token, API Base: https://xyz5.openai.azure.com
Expected report:
Scenario 1: No failed requests and all latency 0s: should skip sending the report completely (or can send the report but explicitly say "None" for both categories - my personal preference would be to skip to keep noise to minimum)
Scenario 2: One of the categories has all 0s, other has non-zero values:
Here are today's key metrics :chart_with_upwards_trend::
:exclamation: Top Deployments with Most Failed Requests:
None
:sweat_smile: Top Slowest Deployments:
1. Deployment: azure/gpt-35-turbo-0125, Latency per output token: 2.5s/token, API Base: https://xyz1.openai.azure.com
2. Deployment: azure/gpt-35-turbo-0125, Latency per output token: 1.2s/token, API Base: https://xyz2.openai.azure.com
3. Deployment: azure/gpt-35-turbo-0125, Latency per output token: 0.7s/token, API Base: https://xyz3.openai.azure.com
4. Deployment: azure/gpt-35-turbo-0125, Latency per output token: 0.5s/token, API Base: https://xyz4.openai.azure.com
Relevant log output
No response
Twitter / LinkedIn details
No response
The text was updated successfully, but these errors were encountered:
What happened?
If a deployment has 0 failed requests, it should be skipped from the "Top 5 Deployments with Most Failed Requests" report.
Similarly, if a deployment has 0s/token latency, it should be skipped from the "Top 5 Slowest Deployments" report.
Example daily report sent in Slack today:
Expected report:
Scenario 1: No failed requests and all latency 0s: should skip sending the report completely (or can send the report but explicitly say "None" for both categories - my personal preference would be to skip to keep noise to minimum)
Scenario 2: One of the categories has all 0s, other has non-zero values:
Relevant log output
No response
Twitter / LinkedIn details
No response
The text was updated successfully, but these errors were encountered: