[Bug]: benchmark_app defaults to bf16 when explicitly given -ip f32

[azhuvath]

OpenVINO Version

2024.5

Operating System

Ubuntu 20.04 (LTS)

Device used for inference

CPU

Framework

None

Model used

Marqo/marqo-fashionSigLIP

Issue description

benchmark_app defaults to bf16 when explicitly given -ip f32. The system used was Intel Xeon Scalable Sapphire Rapids which support BF16. The command used is as follows.

nohup benchmark_app -m marqo-fashionSigLIP-fp32.xml -data_shape "input_ids[7, 64],pixel_values[1, 3, 224, 224]" -b 1 -api sync -ip f32 -pc > fp32_output.log 2>&1 &

Step-by-step reproduction

Convert the model Marqo/marqo-fashionSigLIP from Hugging Face and try benchmark_app as given in the description.

Relevant log output

No response

Issue submission checklist

• I'm reporting an issue. It's not a question.
• I checked the problem with the documentation, FAQ, open issues, Stack Overflow, etc., and have not found a solution.
• There is reproducer code and related data files such as images, videos, models, etc.

[Wovchena]

How do I convert https://huggingface.co/Marqo/marqo-fashionSigLIP?

[azhuvath]

How do I convert https://huggingface.co/Marqo/marqo-fashionSigLIP?

https://docs.openvino.ai/2024/notebooks/clip-zero-shot-classification-with-output.html

Another option - Just change the image path to some image in the below code

import openvino as ov
from PIL import Image
from pathlib import Path
from transformers import AutoModel, AutoProcessor

model_path = 'Marqo/marqo-fashionSigLIP'

Load the model and processor

model = AutoModel.from_pretrained(model_path, trust_remote_code=True)
processor = AutoProcessor.from_pretrained(model_path, trust_remote_code=True)

Prepare input data

image = [Image.open("images/27206.jpg")]
text = ["Accessories", "Apparel", "Footware", "Free Items", "Home", "Personal Care", "Sporting Goods"]
inputs = processor(text=text, images=image, padding='max_length', return_tensors="pt")

fp32_model_path = Path("marqo-fashionSigLIP-fp32.xml")
model.config.torchscript = True

ov_model = ov.convert_model(model, example_input=dict(inputs))
ov.save_model(ov_model, fp32_model_path, compress_to_fp16=False)