make it work with ollama and I will dockerise it for you

[fahadshery]

dont want to use openAI
already have ollama installed

[MR-444]

I don't know Ollama, but I guess it is a self-hosted server with an exposed API?
Edit: I looked on Ollama, it seems that this it is used to run a text-to-text LLM locally, but we need also an image-to-text system, if both are working in Ollama it would be possible, running a text-to-text LLM only would not work.

If these requirements are met, make it easy for us and give the API endpoints description(s) (+parameters) you need, then it would be easy to include it.

[fahadshery]

Well ollama has a built-in openai compatibility. https://ollama.com/blog/openai-compatibility

image to text could be done via models like llava using ollama?

[MR-444]

Thank you, this is interesting, I did not see it yesterday.
This can really do it:
https://ollama.com/library?q=llava
We need to add another endpoint pointing to the oolama server api (I hope).

But I am not sure how good the image-to-text results would be, in my experience GPT-Vision is the best so far, maybe the Claude system is on par, followed by Gemini.

Well, I can't promise anything other than to try to find some time to investigate.

[MarkusR1805]

I use Ollama and love it!
You can quickly install a model, try it out, and if it's not good, you can uninstall it pretty quickly.
Here is a LLava model. I can't get along with it, but I installed it because I'm trying to do something with it in Python.
To generate prompts I use llama3.2:3b.