Does not work in Oobabooga

[Kaszebe]

Hello,

I downloaded this last night:

VPTQ-community_Meta-Llama-3.1-405B-Instruct-v16-k65536-1024-woft

I ran it in Oobabooga. It loaded fine. But when I tried to talk to the model (chat-instruct) nothing happened. I ran nvidia-smi and it looked like the model loaded but no inferencing was going on.

I also did the same for other V8 and V16 models and they did not work in Oobabooga as well.

I don't have to use Oobabooga....I can use other programs such as Kobold if they work better?

[YangWang92]

Please hold on, I'll take a closer look. Thank you.