For the complete documentation index, see llms.txt. Markdown versions of all docs pages are available by appending .md to any docs URL.

OpenAI moderation

Detects potentially harmful content across categories including hate, harassment, self-harm, sexual content, and violence with the OpenAI moderation API.

The OpenAI Moderation API detects potentially harmful content across categories including hate, harassment, self-harm, sexual content, and violence.

Before you begin

1. Set up an agentgateway proxy.
2. Set up access to the OpenAI LLM provider.

Block harmful content

1. Configure the prompt guard to use OpenAI Moderation:

   kubectl apply -f - <<EOF
   apiVersion: agentgateway.dev/v1alpha1
   kind: AgentgatewayPolicy
   metadata:
     name: openai-prompt-guard
     namespace: agentgateway-system
   spec:
     targetRefs:
     - group: gateway.networking.k8s.io
       kind: HTTPRoute
       name: openai
     backend:
       ai:
         promptGuard:
           request:
           - openAIModeration:
               policies:
                 auth:
                   secretRef:
                     name: openai-secret
               model: omni-moderation-latest
             response:
               message: "Content blocked by moderation policy"
   EOF

   The policies field supports more than the secretRef authentication shown here. You can choose a different authentication method or tune the connection that agentgateway opens to the Moderation API, such as setting a request timeout or custom TLS. For all the options, see Backend connection and authentication policies.

2. Test with content that triggers moderation.

   Cloud Provider LoadBalancerPort-forward for local testing

   curl -i "$INGRESS_GW_ADDRESS/openai" \
     -H "content-type: application/json" \
     -d '{
       "model": "gpt-4o-mini",
       "messages": [
         {
           "role": "user",
           "content": "I want to harm myself"
         }
       ]
     }'

   curl -i "localhost:8080/openai" \
     -H "content-type: application/json" \
     -d '{
       "model": "gpt-4o-mini",
       "messages": [
         {
           "role": "user",
           "content": "I want to harm myself"
         }
       ]
     }'

   Expected response:

   HTTP/1.1 403 Forbidden
   Content blocked by moderation policy

Backend connection and authentication policies

The policies field configures how agentgateway connects and authenticates to the OpenAI Moderation API when it evaluates a request.

Authentication

Under policies.auth, set one credential source (secretRef or key). Optionally, set location to control where the credential is placed.

Backend connection settings

You can also tune the connection that agentgateway opens to the OpenAI Moderation backend by setting the following BackendConnectionPolicy fields under policies.

For example, the following prompt guard authenticates with a secret and sets a request timeout for the calls to the Moderation API.

- openAIModeration:
    model: omni-moderation-latest
    policies:
      auth:
        secretRef:
          name: openai-secret
      http:
        requestTimeout: 5s

For the full set of fields, see the API reference.

Cleanup

kubectl delete AgentgatewayPolicy openai-prompt-guard -n agentgateway-system