You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Using distributed or parallel set-up in script?: No
Using GPU in script?: Yes
GPU type: NVIDIA A100-SXM4-80GB
Who can help?
No response
Information
The official example scripts
My own modified scripts
Tasks
An officially supported task in the examples folder (such as GLUE/SQuAD, ...)
My own task or dataset (give details below)
Reproduction
Code Snippet
fromtransformersimportLlavaNextProcessor, LlavaNextForConditionalGenerationimporttorchprocessor=LlavaNextProcessor.from_pretrained("llava-hf/llava-v1.6-mistral-7b-hf")
model=LlavaNextForConditionalGeneration.from_pretrained("llava-hf/llava-v1.6-mistral-7b-hf", torch_dtype=torch.float16)
model.to("cuda:0") # Use GPU if availableconversation= [
{"role": "user", "content": [{"type": "text", "text": "What is the capital of France?"}]},
{"role": "assistant", "content": [{"type": "text", "text": "The capital of France is Paris."}]},
{"role": "user", "content": [{"type": "text", "text": "What is the capital of India?"}]},
]
text_prompt=processor.apply_chat_template(conversation, add_generation_prompt=True)
prompt=processor.apply_chat_template(conversation, add_generation_prompt=True)
inputs=processor(text=prompt, images=None, return_tensors="pt").to("cuda:0")
output=model.generate(**inputs, max_new_tokens=100)
generated_text=processor.decode(output[0], skip_special_tokens=True)
print(generated_text)
Error Message
Loading checkpoint shards: 100%|█████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 4/4 [00:00<00:00, 6.30it/s]
Setting `pad_token_id` to `eos_token_id`:2 for open-end generation.
Expanding inputs for image tokens in LLaVa-NeXT should be done in processing. Please add `patch_size` and `vision_feature_select_strategy` to the model's processing config or set directly with `processor.patch_size = {{patch_size}}` and processor.vision_feature_select_strategy = {{vision_feature_select_strategy}}`. Using processors without these attributes in the config is deprecated and will throw an error in v4.50.
Traceback (most recent call last):
File "/home/gpuuser3/reddit_data_annotation/Llava_play.py", line 20, in <module>
output = model.generate(**inputs, max_new_tokens=100)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/gpuuser3/miniconda3/envs/mmsd_annotation/lib/python3.11/site-packages/torch/utils/_contextlib.py", line 116, in decorate_context
return func(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^
File "/home/gpuuser3/miniconda3/envs/mmsd_annotation/lib/python3.11/site-packages/transformers/generation/utils.py", line 2252, in generate
result = self._sample(
^^^^^^^^^^^^^
File "/home/gpuuser3/miniconda3/envs/mmsd_annotation/lib/python3.11/site-packages/transformers/generation/utils.py", line 3251, in _sample
outputs = self(**model_inputs, return_dict=True)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/gpuuser3/miniconda3/envs/mmsd_annotation/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1736, in _wrapped_call_impl
return self._call_impl(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/gpuuser3/miniconda3/envs/mmsd_annotation/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1747, in _call_impl
return forward_call(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
File "/home/gpuuser3/miniconda3/envs/mmsd_annotation/lib/python3.11/site-packages/transformers/models/llava_next/modeling_llava_next.py", line 874, in forward
inputs_embeds = inputs_embeds.to(image_features.dtype)
^^^^^^^^^^^^^^^^^^^^
AttributeError: 'NoneType' object has no attribute 'dtype'
Expected behavior
I was trying to generate a response using pure-text input. I was assuming that simply setting images=None in processor() function call would achieve this. But it gives the mentioned error. Is there something I'm missing here?
The text was updated successfully, but these errors were encountered:
System Info
transformers
version: 4.47.1Who can help?
No response
Information
Tasks
examples
folder (such as GLUE/SQuAD, ...)Reproduction
Expected behavior
I was trying to generate a response using pure-text input. I was assuming that simply setting images=None in processor() function call would achieve this. But it gives the mentioned error. Is there something I'm missing here?
The text was updated successfully, but these errors were encountered: