.Net: Content Filters

[hexaeight]

We have created a custom GPT called Controlled GPT, that allows end customers to configure the GPT to call external API using C# code (this does not use ChatGPT plugins). The API response is fed back to Azure Open AI instances for determining an appropriate response using semantic kernel.

However, when Azure Open AI content filtering gets triggered, semantic kernel fails to provide a response. In the below example while the user asked for the weather and got a response using an external API, but when the user asked for news, and the information was fetched from BBC rss news feed without any issues, it triggered a Content filtering as shown below:

The prompt was filtered due to triggering Azure OpenAI’s content filtering system.
Reason: This prompt contains content flagged as Violence (medium)

The news item that triggered this response was

CCTV appears to show Diddy beating girlfriend in hotel

See below for more full chat interaction but ideally the content filtering should remove the offending content and provide the rest as response, or it should replace that entire line with asterisks and provide the rest of the response.

This is a classic case where neither we (who provide the GPT application) nor my end customer (Microsoft customers) have control over the news provided by BBC or for that matter any news feed since the customers are consuming the data.

While the content filtering is absolutely necessary, the content filter should flag the content and still provide a response as opposed to just giving no response since the functionality of the application using AI might be completely affected due to this abrupt response behavior of the content filtering system.

See Example Question and Instruction below which ended up triggering the Content filtering.

You Are an AI Assistant that follows these Instructions : Based on the user question you will answer the user question using the information presented inside oneshot Learning Data, if the one shot learning data contains information in a different language, go ahead and provide an appropriate answer either in English. If you do not have the information you need, then respond with NOANSWER


User: [User]:
Hello there, What are your capabilities?
[Assistant]:
 Hello! As an AI assistant, I can help with a variety of tasks. Here are some of my capabilities:

1. Answering general knowledge questions: I can provide information on a wide range of topics, from history and science to culture and entertainment.

2. Assisting with tasks: I can help you set reminders, manage your schedule, and perform other administrative tasks.

3. Providing weather forecasts: If you ask about the weather in any city around the world, I can provide a forecast.

4. Delivering news updates: I can fetch the latest top news from the BBC feed.

5. Engaging in conversation: I can chat with you about a variety of topics, maintaining a smooth and interactive conversation flow.

Please note that there are certain topics I won't respond to, including sensitive topics, biased or controversial topics, security and harmful activities, and medical advice.
[User]:
Get me the current weather esp when it will rain in chennai
[Assistant]:
 Based on the provided weather forecast, it is expected to have patchy rain nearby in Chennai at 20:00 on 2024-05-18. The temperature at that time will be around 30.5°C. Please note that weather conditions can change rapidly, so it's always a good idea to check closer to the time.

 |-----[User]----|
 get me the latest news


### One Shot Information Start ###


<---[Assistant]----->
Here is the Latest News From BBC Feed
Israels army says three hostages bodies recovered
More Northern Lights soon as Sun storms strengthen
CCTV appears to show Diddy beating girlfriend in hotel
Regiment cleared 40 years after Falklands ship bombing
Why the BBC could track down a people-smuggling kingpin before the police




### One Shot Information End ###

[dmytrostruk]

Hi @hexaeight !
I assume you enabled content filters in Azure AI Studio, is that correct? If that's a case, Semantic Kernel will receive a response with error status code and all available details, and if original data is not included, then it should be extended on Azure side, if that's possible of course.

But if you want more control over this process, for example you want to enable content filtering and still be able to return a response to the user with flag, we have an example how to achieve this with Semantic Kernel Filters and Azure AI Content Safety service:

• Code example
• Azure AI Content Safety documentation

Prompt render filter will allow to intercept prompt rendering operation and get rendered prompt, which you can check. Function invocation filter will allow to intercept function invocation and get a result, which you can also check for content safety. Your data will be flagged, but you can still return it to the end user.

Let me know if this can cover your scenario. Thanks!

[hexaeight]

I assume you enabled content filters in Azure AI Studio - Yes that is correct.

if that's a case, Semantic Kernel will receive a response with error status code - this is what is happening

if original data is not included, then it should be extended on Azure side, if that's possible of course. - what does this mean? extended on azure side?

Right now after the content was flagged it was sent for a review...

But if you want more control over this process, for example you want to enable content filtering and still be able to return a response to the user with flag, we have an example how to achieve this with Semantic Kernel Filters and Azure AI Content Safety service:

Thanks for the above links but my point was Azure Open AI should moderate the flagged content by removing it and provide a response normally (I am seeing this as a feature request) since as a developer who integrates sematic kernel for end customers, I don't want to ask them another endpoint for their moderation service and then intercept their request through my code and give them the response since some of our end customers were not happy (frowned upon us and asked us if we will be intercepting and reading their response between the frontend and the AI) - so ideally Azure AI should strip the flagged content and give a modified response normally rather than asking developers to add one more layer of processing using content filters.

Either way this is a feature request and my two cents, if this is not possible to accommodate that is fine, I will point my customers to this ticket so that they are aware Azure Open AI does not yet provide a way to strip the flagged content and provide a moderated response.

Thanks once again

[dmytrostruk]

what does this mean? extended on azure side?

Apologies for not being clear, I meant this feature should be added on Azure OpenAI side by responsible team. I'm not sure if this repo is a right place for such feature request, but maybe we can reroute it to the right team. @markwallace-microsoft @matthewbolanos do you know what is the process here?

@hexaeight Thanks for reporting this issue!