Gemini 2.0 Flash spatial reasoning severly degraded with Image size 1024

[fayezsalka]

Description of the bug:

I was trying to reproduce results I get with the spatial reasoning applet found in the AI studio, using the spatial reasoning notebook. However, I noticed for the same prompt and same image, applet produces far better results with 2.0 Flash.

Upon further investigation, it seems that in the notebook the image is resized to 1024, but in the applet the image is resized to 640. Using images with size of 1024 significantly degrades the quality of the boxes generated. E.g a lot of the bounding boxes are misplaced and detected incorrectly. Changing the sizing to use maximum 640 for width in the notebook fixes the issue.

Any intuition on why image sizes 1024 degardes the spatial reasoning?

Actual vs expected behavior:

No response

Any other information you'd like to share?

No response

[fayezsalka]

Here is the comparison between 640 and 1024 image size:

640:

1024:

The degraded quality is not specific to this image / prompt. Same issue happens with all the examples found in spatial reasoning. Cupcakes, Fox, etc

[Giom-V]

Thanks for reporting that behavior! I'll check with the Deepmind researchers but I'd say that it's because there is too much information when the image is not resized.

[lucianommartins]

Hi @fayezsalka,

Can you share a colab with the experiment you did? I tried reproducing the misbehavior you are facing using the following image but it consistently worked with resize() set to [1024, 768, 640, 512, 256]:

Thanks!!