Let's bring it all home

[JamesStallings]

Is it not possible to use phidata with local LLMs runningg under e.g., ollama, and keep this all local and opensource? and please, without docker? No interest in dodging the limits on free token quotas with the cloud LLMs, and no interest in sending them money.

Docker has too much overhead, from a resource perspective and a complexity perspective. Too many moving parts, too much to go wrong, too much misdirection.

Give us the simple recipe, let the docker wizards do their thing with it.

[ysolanky]

Hello @JamesStallings!

You can use phidata with Ollama without docker: https://docs.phidata.com/llms/ollama

Though we use a postgres server running on docker for memory and knowledge base for the Assistant. But it is not required and depends on your use case. So, docker is not needed to get an LLM working with phidata + Ollama.

[Welborn09]

I am new here and just found you. The OP had the same idea I had, are there any documents that specify how to set it up to use Ollama? I already have an environment I am using with open-webui but would love to try this out.

Edit: I did check out the link you provided above. The tutorial still references openAI and doesn't specify how to point it to ollama and chose a model.