Other LLMs instead of Openai

[codelauncher444]

Is it possiblew to use custom LLMs instead of Openai api?

[farrael004]

Although not the ideal solution, for now you can just modify the create_chat_completion function here to implement any LLM you want. If you want to use something like obabooga's text-generation-webui, you'd have to use the API in code.

[vineetvermait]

this results in page not found. Anything changed??

cant find the LLM_utils

[farrael004]

Yeah, they changed where the definition is. Now you can find it here.

[LanceLake]

If I have oobabooga/text-generation-webui running with an API for OpenAPI calls, can you give me an example of what values I would put in the file to make it connect to this?

[ntindle]

We're making progress towards this with the rearch. In the meantime, you can use one of the many tools that reimplement the OpenAI Api and use the base_url config option to point to the tool you're using.

[LanceLake]

When I do this, I get this error..

AutoGPT: Failed to update API costs: KeyError: 'mayaeary_pygmalion-6b-4bit-128g'
Error parsing JSON response with literal_eval invalid syntax (, line 1)

Traceback (most recent call last):
File "", line 198, in _run_module_as_main
File "", line 88, in run_code
File "C:\Coderoot\AI\AutoGPT\Pure\autogpt_main.py", line 5, in
autogpt.cli.main()
File "C:\Python311\Lib\site-packages\click\core.py", line 1130, in call
return self.main(*args, **kwargs)
^^^^^^^^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\site-packages\click\core.py", line 1055, in main
rv = self.invoke(ctx)
^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\site-packages\click\core.py", line 1635, in invoke
rv = super().invoke(ctx)
^^^^^^^^^^^^^^^^^^^
File "C:\Python311\Lib\site-packages\click\core.py", line 1404, in invoke
return ctx.invoke(self.callback, **ctx.params)
^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^

Oggba: Exception: When loading characters/instruction-following/None.yaml: FileNotFoundError(2, 'No such file or directory')
Warning: Loaded default instruction-following template for model.
Warning: too many messages for context size, dropping 1 oldest message(s).
Warning: Ignoring max_new_tokens (910), too large for the remaining context. Remaining tokens: -304
Warning: Set max_new_tokens = -304
Output generated in 2.63 seconds (0.00 tokens/s, 0 tokens, context 2252, seed 130161750)
127.0.0.1 - - [18/Jul/2023 14:54:53] "GET /v1/models HTTP/1.1" 200 -