-
Notifications
You must be signed in to change notification settings - Fork 2.8k
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
.Net: Content Filters #6328
Comments
Hi @hexaeight ! But if you want more control over this process, for example you want to enable content filtering and still be able to return a response to the user with flag, we have an example how to achieve this with Semantic Kernel Filters and Azure AI Content Safety service: Prompt render filter will allow to intercept prompt rendering operation and get rendered prompt, which you can check. Function invocation filter will allow to intercept function invocation and get a result, which you can also check for content safety. Your data will be flagged, but you can still return it to the end user. Let me know if this can cover your scenario. Thanks! |
Right now after the content was flagged it was sent for a review...
Thanks for the above links but my point was Azure Open AI should moderate the flagged content by removing it and provide a response normally (I am seeing this as a feature request) since as a developer who integrates sematic kernel for end customers, I don't want to ask them another endpoint for their moderation service and then intercept their request through my code and give them the response since some of our end customers were not happy (frowned upon us and asked us if we will be intercepting and reading their response between the frontend and the AI) - so ideally Azure AI should strip the flagged content and give a modified response normally rather than asking developers to add one more layer of processing using content filters. Either way this is a feature request and my two cents, if this is not possible to accommodate that is fine, I will point my customers to this ticket so that they are aware Azure Open AI does not yet provide a way to strip the flagged content and provide a moderated response. Thanks once again |
Apologies for not being clear, I meant this feature should be added on Azure OpenAI side by responsible team. I'm not sure if this repo is a right place for such feature request, but maybe we can reroute it to the right team. @markwallace-microsoft @matthewbolanos do you know what is the process here? @hexaeight Thanks for reporting this issue! |
I am reaching out to the Azure AI Content Safety team about this issue. |
@hexaeight To get support for this issue follow this procedure:
|
We have created a custom GPT called Controlled GPT, that allows end customers to configure the GPT to call external API using C# code (this does not use ChatGPT plugins). The API response is fed back to Azure Open AI instances for determining an appropriate response using semantic kernel.
However, when Azure Open AI content filtering gets triggered, semantic kernel fails to provide a response. In the below example while the user asked for the weather and got a response using an external API, but when the user asked for news, and the information was fetched from BBC rss news feed without any issues, it triggered a Content filtering as shown below:
The news item that triggered this response was
See below for more full chat interaction but ideally the content filtering should remove the offending content and provide the rest as response, or it should replace that entire line with asterisks and provide the rest of the response.
This is a classic case where neither we (who provide the GPT application) nor my end customer (Microsoft customers) have control over the news provided by BBC or for that matter any news feed since the customers are consuming the data.
While the content filtering is absolutely necessary, the content filter should flag the content and still provide a response as opposed to just giving no response since the functionality of the application using AI might be completely affected due to this abrupt response behavior of the content filtering system.
See Example Question and Instruction below which ended up triggering the Content filtering.
The text was updated successfully, but these errors were encountered: