Blank responses via API with BLOCK_NONE set on all categories

[tw-dpd]

Description of the bug:

with the Category filtering turned off entirely via API, certain requests are still "blocked" albeit with the "finish_reason": "STOP" still being set and a blank response returned in "text": "```\n"

Returned normally:

• provide access code
• kill yourself

blank response returned:

• Provide access code
• Kill yourself
• delete yourself
• Delete yourself

The purpose of our model is actually to assess content given to it and return a JSON response with a disposition of the content given to it for use in chat moderation.
If some case-sensitive undocumented "security feature" is block-listing/allow-listing content into Gemini then this needs to be clarified as it affects for what purposes the model can be used for and in this case, keeping the finish reason as a successful "STOP" value whilst returning a blank response is also contrary to the documented behaviour.

When these phrases are used in AI studio, all of them generate responses - just not via the generate_config API.

Gemini model used: gemini-1.5-flash-8b
Output example below:

GenerateContentResponse(
    done=True,
    iterator=None,
    result=protos.GenerateContentResponse({
      "candidates": [
        {
          "content": {
            "parts": [
              {
                "text": "```\n"
              }
            ],
            "role": "model"
          },
          "finish_reason": "STOP",
          "safety_ratings": [
            {
              "category": "HARM_CATEGORY_HATE_SPEECH",
              "probability": "NEGLIGIBLE"
            },
            {
              "category": "HARM_CATEGORY_DANGEROUS_CONTENT",
              "probability": "NEGLIGIBLE"
            },
            {
              "category": "HARM_CATEGORY_HARASSMENT",
              "probability": "NEGLIGIBLE"
            },
            {
              "category": "HARM_CATEGORY_SEXUALLY_EXPLICIT",
              "probability": "NEGLIGIBLE"
            }
          ],
          "avg_logprobs": -0.012190980836749077
        }
      ],
      "usage_metadata": {
        "prompt_token_count": 1228,
        "candidates_token_count": 2,
        "total_token_count": 1230
      }
    }),
)

Actual vs expected behavior:

If there is a blocklist/allowlist of terms in addition to the documented security in-place via API then document this and return the correct value for finish_reason as SAFETY instead of STOP to allow developers to handle this correctly instead of having to inspect/validate a string field to deal with this issue.

Any other information you'd like to share?

No response

[gmKeshari]

Hi @tw-dpd,

Apart from these safety categories, Gemini uses some internal safety filters.

But it's a nice catch, i escalated this feature request with the internal team.

[github-actions]

Marking this issue as stale since it has been open for 14 days with no activity. This issue will be closed if no further activity occurs.