GPT-3.5 doesn't seem good enough for this

[barrymac]

I have the impression that the context length of GPT3.5 doesn't handle these use cases very well. I gave this fairly simple un-engineered :-O! prompt, far too simple most likely, and it got stuck in a loop for a long time. I gave it a continuous 100 iterations before I left the house.

I'm looking forward to a day when we can use tools like this with some open models that may perform better on such tasks like vicuna maybe. More and more better ones seem to be coming out all the time.

ai_goals:
- find the best bang for buck in terms of machine learning processing capability benchmarks for the price on ebay for a dell high density 1U GPU server like a C4130 or C4140 with 4 Nvidia vlink GPUs
ai_name: GPUnodeResearcher
ai_role: an AI designed to research the best price for a dell GPU server with 32Gb ram GPUS

CRITICISM:  constructive self-criticism
NEXT ACTION:  COMMAND = command name ARGUMENTS = {'arg name': 'value'}
SYSTEM:  Command command name returned: Unknown command 'command name'. Please refer to the 'COMMANDS' list for available commands and only respond in the specified JSON format.
 THOUGHTS:  thought
REASONING:  reasoning
PLAN: 
-  short bulleted
-  list that conveys
-  long-term plan
CRITICISM:  constructive self-criticism
NEXT ACTION:  COMMAND = command name ARGUMENTS = {'arg name': 'value'}
SYSTEM:  Command command name returned: Unknown command 'command name'. Please refer to the 'COMMANDS' list for available commands and only respond in the specified JSON format.
 THOUGHTS:  thought
REASONING:  reasoning
PLAN: 
-  short bulleted
-  list that conveys
-  long-term plan
CRITICISM:  constructive self-criticism
NEXT ACTION:  COMMAND = command name ARGUMENTS = {'arg name': 'value'}
SYSTEM:  Command command name returned: Unknown command 'command name'. Please refer to the 'COMMANDS' list for available commands and only respond in the specified JSON format.
 THOUGHTS:  thought
REASONING:  reasoning
PLAN: 
-  short bulleted
-  list that conveys
-  long-term plan
CRITICISM:  constructive self-criticism
NEXT ACTION:  COMMAND = command name ARGUMENTS = {'arg name': 'value'}
SYSTEM:  Command command name returned: Unknown command 'command name'. Please refer to the 'COMMANDS' list for available commands and only respond in the specified JSON format.
 THOUGHTS:  thought
REASONING:  reasoning
PLAN: 
-  short bulleted
-  list that conveys
-  long-term plan
CRITICISM:  constructive self-criticism
NEXT ACTION:  COMMAND = command name ARGUMENTS = {'arg name': 'value'}
SYSTEM:  Command command name returned: Unknown command 'command name'. Please refer to the 'COMMANDS' list for available commands and only respond in the specified JSON format.
 THOUGHTS:  thought
REASONING:  reasoning
PLAN: 
-  short bulleted
-  list that conveys
-  long-term plan

[sutty5]

Once GPT4 32k Context length model is available, we will see massive improvements in what Auto-GPT can do. One of the biggest pitfalls at the moment is the fact that data has to be summarized or compressed down so detail gets lost. A larger context window will really open up a new world of possibilities. The larger the context window, the deeper the neural network has to be, so we are not likely to see large context windows in smaller models, but who knows, the pace of change is rapid, never know what next week will bring.

[GoMightyAlgorythmGo]

Is auto GPT a neuronal network? Are you sure? There is machinelerning in autopt at all?

[barrymac]

I'm not using autogpt to find gpu servers to run autogpt, which of course in default configuration uses external inference engines like openAI APIs. Among other things, I'd like to use a private instance of something like a fine tuned vicuna as a back end to be to run in continuous mode.

[sutty5]

AutoGPT is fundamentally based on OpenAI's GPT3.5/GPT-4 Language Models, along with some other smaller models for carrying out various tasks as agents. There is no 'active' machine learning as all the models are pre-trained. any information learned during a session is not learned by the language models themselves, they are simply stored in local or external memory to be accessed at a later date, but this is fundamentally different and less than ideal compared to a Model that could actively learn and expand it's knowledge, and such a model does not yet exist.

[barrymac]

Yes that is understood, but I believe there is a strong motivation to extend autogpt, or fork it to support Open source models which can be used via APis. It seems likely that openAIs api would become popular as a standard that self hosted inference engine would adopt, in order to support portability from the most popular API. I believe progress in this direction will be impressive, given the power that these tools can give us to work faster on coding tasks.

I assume however that openAI will maintain their edge with features such as extended context length as you mentioned that could prove cost prohibitive to self host. That doesn't mean that a self hosted model could become good enough for some very useful tasks in the near future and I would look forward to that possibility.

[LecJackS]

I understand that when GPT4 32k goes live things will be MUCH better. But is that the only way? Maybe I'm dreaming too much, but could it be possible to use much less powerful models in a more intelligent / guided way, so we can leverage their original possibilities?

Every day I'm seeing more and more projects where big open-source models are reduced in requirements. If a local model can be run on any computer, and/or several small models can be run/queried at the same time (in a fast way), does that show us another path where auto-gpt could shine, leveraging their mediocre possibilities to a much useful level?

The only (but not small) thing that's holding this back is that the models we have now cannot be improved, no matter how well we guide them in the process of answering/asking questions. Their parameter size is fixed, and no more knowledge can be put in there.

Hope to see more research in this direction in the coming times.

[endolith]

Yeah it's never successfully accomplished anything for me, just gets lost in loops. I feel like it relies too heavily on the smartness of GPT4, but could be made to work with GPT3 with better prompting and management of agents to check each other's work, etc.?

[barrymac]

I think it's possible to get it to do useful things, but like you say, you need to work pretty hard on the prompts and give it quite a lot of help. I am dying waiting for GPT-4 API access! I'm even considering spinning up an Azure account just to get access to the service through there.

[sutty5]

GPT-4 does not really improve Auto-GPT aside from the larger context window, I haven't managed to get it to do anything even remotely complex without it getting stuck in a loop. it's very much hit and miss. But I do find it to be much more useful inside ChatGPT.

[gl2007]

I am waitlist for GPT4. Can this work with GPT3.5 API key?

[endolith]

It "works" but it can't actually do any tasks, it just thinks itself in circles.

[MeatlHead]

Gpt-3 does not in anyway reflect the use cases for this model...I only hope that in the future we can use other AI APIs rather than openAIs coz openAI seems to be constricted and very limited to handle any relevant use case...this is my opinion I think you should test other AI versions for this...or maybe we can tweak the token or the limits of the AI by enhancing competition between the platforms by engaging different AI approaches hoping we get opensource access to developing and newly developed AI models.