How to form/activate our R&D wing?

[p-i-]

Vision

Imagine: AI engineers are community-engaging to develop fresh concepts, informed by collectively keeping on top of Agent-related AI advances, and there is a smooth connection through the information-flow of the AutoGPT team so that novel concepts can be explored by leveraging AutoGPT, and winning concepts make it into the master branch.

What do we need for this?

• We've got the raw talent within the community.
• We need a R&D team -- an effective community-structure for these engineers to interact.
• We need a knowledgebase this team can keep up to date.
• We need to strengthen connections between core maintainers and the R&D team.

Motivation

The main purpose of AutoGPT is to provide an ecosystem for advancing the SotA of autonomous agents.

Thus far focus has been mostly on cleaning up the project, and this has resulted in a massive buildout. Code has been refactored, functionality added, plugins have been split out, bugs have been fixed, we have seriously advanced CI now.

But we haven't actually evolved the project significantly conceptually.

Since starting AutoGPT, interesting papers have come out, e.g.:

• Unleashing Infinite-Length Input Capacity for Large-scale Language Models with Self-Controlled Memory System
• Tree of Thoughts: Deliberate Problem Solving with Large Language Models
• VOYAGER: An Open-Ended Embodied Agent with Large Language Models

... but until now we haven't been ready to implement them. We've been too busy getting our collective shit together.

And technologies have sprung up, e.g. these 3 do pretty much the same thing:

• LMQL
• GuardRails
• Guidance
  ... but we've had a hard time choosing whether to use ANY of them (let alone which) as the core team lacks expertise.

But that expertise is present in our community! Just today samdcbu let a ReadingGroup discussion on TreeOfThoughts with an attendence of 15.

We want to be firing on all cylinders. How to activate our R&D wing? How do we create a buzzing R&D sub-community?

Initial proposals

• we need some Discord structure

  • Current Discord structure

    • reading-group badge
    • COMMUNITY/
      • resources
      • research
    • ReadingGroup
      • about
      • news
      • reading-group
      • Reading Group (VoiceChat)

  • Revised Discord structure

    • r&d badge as well as reading-group badge?
    • R&D section
      • ReadingGroup section should be removed, and channels moved into this section
      • I think both resources/research should be in R&D section
        • probably renamed as it's not currently clear what content should go in what channel
      • What DO we want in this section?
        • We want feeds of papers, github-projects, news-articles
        • We want (non--AutoGPT-centric) discussions, separate from the feeds
        • We want the reading-group (announcements, discussion, voiceChat)
        • I think we want to reuse the same voiceChat for the ReadingGroup and general R&D chat)
        • Maybe a channel specifically for AutoGPT-related R&D

• Also we want a knowledgebase for:

  • maintaining tidy lists of resources
  • structuring information (e.g. resources on transformer architecture)
  • maybe projects like .ipynb files (maybe just linking to gists / github-repos

Crosslinking with Discord thread

[dschonholtz]

@p-i- i love this effort.
now that we have a dedicated group interested in research I think the next step may be to actually do research. This would likely involve writing research papers.

I am currently getting advised by David Bau who is famous for LLM interpretability on how to submit something to ICLR.

He would really like to see a paper get submitted focusing on AutoGPT autonomy and perhaps explaining our benchmarking effort and how it could be applied more broadly to other agents.

[Boostrix]

We also need to look at our own strengths and weaknesses, and relate those to similar projects and their pros&cons to be able to prioritize certain types of research (evaluation) - e.g. referring specifically to the "Tree of Thoughts" paper (since we're lacking proper planning support), but also nvidia's voyager paper - since it's basically re-implemented Auto-GPT and came up with something that seems superior in various areas.

In other words, we need to understand our own limitations and map out relevant research efforts (or similar projects) to explore the solution space properly. That applies even more so, because in many people's eyes, the re-arch effort caused the project to stagnate, and indeed over the course of the last 4-6 weeks, a number of compelling projects have stepped out of the shadows, some of which are providing interesting solutions where Auto-GPT in its current form isn't exactly shining (yet).

[samuelbutler]

I would appreciate a separate channel in the R&D section just for papers, and then a separate one for all other other resources/github-projects/videos. I currently use the paper-suggestions channel as a way to collect suggestions for papers to read and would like to continue to have a dedicated channel just for that purpose.

As for badges, I think moving to just an R&D badge makes sense. I think the only potential reason to maintain a separate reading-group badge is if there are people who want to contribute to contribute to r&d but don't want to be pinged by the bot at the start of every reading group meeting.

[ThyFriendlyFox]

Yes exactly, a separate repository. Should have tags though. I grew fond of using Mendeley with Word, if there was a version similar to that that could maintain these references automatically that anyone knows of, let me know

[samuelbutler]

I've used Zotero for citation management, but I don't think any traditional citation management software could do that for us from within Discord. If we had a logger Discord bot monitor the channel and export all of the messages (or ideally send them to a webhook), we could then write some scripts to generate the BibTeX citations for all of the papers and then generate a static site with all of the papers and their citations, organized into categories by tags. We could then host that static site as a GitHub page.

Let me know if anyone has any other ideas about how to accomplish this or something similar.

[7anooch]

I use zotero for citation management, although I've also been using a database on notion (just much nicer interface to work with than zotero is) for more thorough note taking/labelling and then when I need the citation I just export that from zotero. Though I think shared notion pages require a paid subscription? not sure.

Edit: I haven't used benty fields extensively, but its used in one of the journal clubs I was part of briefly, and might be suitable for our purposes: https://www.benty-fields.com/

[ThyFriendlyFox]

I'm reposting from the discord a summary of our discussion:

-First paper should be about forming a benchmark evaluation test for LLMs and Auto-GPT through challenges.
-A future paper about measurable autonomy in LLM based agents could be explored further down the line

-Research should always be conducted with intent to integrate into the main branch of Auto-GPT in the future.
-Research papers should follow the general template as follows:
-Abstract
-Introduction to Field
-Literature Review of Prior Publications in the Field
-Presentation of Gaps in Prior Research/Identification of Novel Purpose
-Methodology
-Acquired Data & Analysis
-Conclusion & Future Outlook
-Abstracts, either summary of completed research of summary of intent to research, are required to be submitted to journals. They are accepted on the basis of novel methodology, novel theory, or novel data. For research to progress, an idea must be developed into an abstract able to satisfy at the minimum one of those three requirements.
-For research to be considered complete internally, it should be able to be passed into the next phase of implementation into the Auto-GPT framework/architecture.

A suggestion I have is to make a separate repository of paper that we can tag, so they are easy to obtain and reference in the papers we write.

[ThyFriendlyFox]

Through a paper? Can you elaborate?

[Boostrix]

As in, looking for, filtering through and evaluating research relevant to the project

[samuelbutler]

Worth keeping in mind, some folks seem to be more interested in working through research papers while others seem eager to create papers...

Well working through the research literature is a necessary prerequisite to writing papers. Doing a thorough literature review is an important part of writing a paper, authors need to be intimately familiar with the current SoTA literature in order to be able to contribute to the field. Paper authors are inherently a subset of those who want to work through the papers, as that is a core part of what is involved in authoring original research.

[Boostrix]

Right, but with the exception of @dschonholtz I have not seen many people who repeatedly expressed an interest in literally publishing - most efforts so far were seemingly aimed at evaluating existing research with a focus on improving agpt

[ThyFriendlyFox]

I would like to literally publish. Engagement with journals requires peer review, which is a way to get info from other projects that lead the field

[ScarletPan]

1. We can utilize an existing wiki as our knowledge base, such as the Awesome Resources page, which already contains valuable projects, papers, and talks. We can reorganize this information in a more structured manner to make it more accessible and useful.

2. For paper tracking and reproduction, we can categorize our efforts based on specific purposes: (1) Expanding our horizons and rethinking the production of Auto-GPT, such as through projects like Voyager, and (2) Enhancing Auto-GPT's modules, including memory and reflection improvements. The primary goal is to iterate on our algorithm flow to make it more efficient and accurate. To achieve this, we need to establish an evaluation benchmark for Auto agents. Once the benchmark is in place, we can collaborate to discuss and reproduce new algorithms to test their potential for improving our workflows.

3. Regarding paper writing, we can consider submitting system, demonstration, or industrial papers to top-tier conferences. For example, we can write a paper introducing our entire system, similar to Transformers: State-of-the-Art Natural Language Processing. If our system proves to be highly accurate, powerful, scalable, and efficient, it could make a significant impact in the academic field. Building an open-source framework and publishing it in a top-tier conference is a feasible approach, based on my previous experience.

4. We should also create a dedicated sub-channel for developing our own models. Many users, including myself, face challenges when using AutoGPT due to the need to access the expensive and limited GPT-4 API. A more user-friendly solution for our community would be to fine-tune our own model, allowing for inference on home GPUs. With a sufficient amount of well-curated data, we can potentially achieve results comparable to GPT-4 (or at least better than GPT-3.5).

[ScarletPan]

Here's a summary of this section from discord channel

1. The development of our custom model is essential, which could be achieved using a 7B or 13B model. There is also potential for creating a suite of smaller agents, each fine-tuned for specific tasks. For instance, a 13B model could be fine-tuned for reflection. Gorilla-7B may serve as a temporary solution.

2. It is crucial to establish a benchmark for evaluating not only llms but also the whole system of autonomous agents, to optimize each module, E.g. Planning part, relfexion part, even the quality of the tools. A fundamental Auto-GPT benchmark should assess the following metrics:

   • Original query and its difficulty
   • Steps required for convergence
   • Milestones achieved
   • Pass/Fail status
   • Solution quality
   • Additional factors such as consistency, robustness, and sensitivity

3. The benchmark's source questions and goals can be derived from various sources and directions, such as proof competitions, millennium problems, FaithfulQA/WebQA, and more.

4. The flakiness of challenges in Auto-GPT and finding ways to improve its performance is also very important. A perturbed prompt generator, the possibility of using math proofs as non-flaky answers, Wolfram's problem generator, and Tree of thoughts may reduce the flakiness.

[Boostrix]

A perturbed prompt generator

That is something that I tinkered with inspired by @merwanehamadi's idea for dynamic prompting. I am basically taking a prompt and asking the LLM to come up with N versions of the same prompt and then use a loop to evaluate the performance of each prompt to weigh them in comparison, to determine which prompts have a higher rate of success. It's basically a form of TDD to come up with LLM prompts that work better than others.

The development of our custom model is essential,

Just thinking out loud: Given the number of papers suggested on Discord, for the R&D wing - might it actually make sense to train a custom model on research specifically related to autonomous agents/LLMs ?

Just like the ChatPDF service/plugin currently works, but with a focus on ingesting all suggested papers to grow an LLM based knowledge base that's fed all recent papers to help with navigating all publications.

That way, we could use a Discord bot to feed papers into the "papers LLM", and the R&D wing could use this to help preview/prioritize papers. This may involve supporting proper citations directly.

Any thoughts / ideas ?

[ScarletPan]

Cool idea. ``PaperGPT'' can help us for our workflow a lot if it comes true and is accurate.

Here the custom model I mention is the one would enable us to rely less on GPT-4 API, making advanced AI technology more accessible and affordable for a wider range of individuals.

[Boostrix]

yes, this was repeatedly brought up on github in the context of using a local/central LLM as a "proxy" to "cache" queries, but also to train itself by keeping track of queries and responses.
Also, there's at least one issue that I recall where someone suggested to use such a setup to gather project specific usage data for troubleshooting purposes.
Basically, Auto-GPT would optionally use a central LLM like chat.agpt.co which would OPTIONALLY gather all sorts of data to better understand how people are using the system and what types of problems they're running into.

Will edit this response once I have looked up the corresponding issues.

Here's the proxy LLM idea:

• 💡Fully Air-Gapped Offline Auto-GPT #348 (comment)

And here's the idea to use a central server for self-improvement

• Shared information to speed up common task and to help drive self-improvment #1749
• Sharing insights regarding prompts #203

Thinking about it "PaperGPT" would sound like a compelling standalone tool/project, with us primarily interested in using it for AI/LLM related research, but other folks possibly wating to use it to collate research in other areas. I suppose, there must already be an existing project to train a pre-trained LLM using our own PDF files - so it's probably not that far-fetched to come up with a corresponding service. I supppose we need to do some ... "research" to see what's already available tooling-wise :-)

Related issue:

• Adding Arxiv Search #826

[holyr00d]

loving everything I am seeing here.

my suggestions are orientated around essentially adding to the team. what are peoples thoughts on having individuals from VR, AR, Politics, Computer Science & Design, Social Science, Neuroscience, Linguistics and Natural Language Processing, Ethics and Philosophy, Psychology and Human-Computer Interaction, Arts and Creative Fields? and any other aspect of how we grow. I feel the more individuals we have the greater not only our understanding will grow but the AI too.

my 2 cents...

Holy

[7anooch]

All for interdisciplinary work, but so far it seems like everyone in the server is some variety of software dev. Would be nice to have others involved, especially folks from neuroscience/linguistics/psychology.

[holyr00d]

I honestly feel we should look to build a true AI dream team. not talking lots of folk 2-5? from each area should do it as I am sure they will speak with their peers at work just as we do, well I do..
Cybersecurity, Robotics and Automation, Healthcare and Medicine, Education, Economics and Data Science, Law and Regulation, Anthropology and Cultural Studies. are other areas.

I am sure some of us will know people who work in these areas. I talk to teachers a lot. and I have a model that was kinda trained on around 377 from a library called the Signet Library, famous Law library here in Edinburgh. Which will grow as long as GPT gets the data it needs and it knows the book. Will divulge anything.

[7anooch]

The dream team is probably already employed somewhere in the bay area haha. I do agree though that we need a more interdisciplinary approach than I've seen so far, there's so much untapped potential at the moment and the only bottleneck I see is that I don't want to spend thousands of my own money on api calls for gpt-4. We also need to figure out funding if we're going to dream big.

[holyr00d]

yep I have an idea about funding. Gonna fly it all by Pi first though.

[7anooch]

Honestly I don't think an LLM finetuned on LLM papers will be of much use. As nice as it would be to have a machine capable of doing our research for us, I don't think we can yet dispense with human thought to do research. An LLM would be able to tell us what papers to look at for a particular idea, but not much beyond that at the moment. We can do that by creating vector embeddings for the set of papers we want.

[Boostrix]

the point was not dispending with human thought, but using an LLM trained on papers as an assistant to have a "dialog" with that is heavily influenced by the LLM being trained on recent, and relevant, research.
This was largely inspired by us having a reading group, and discussing maybe one paper a week, but by folks posting dozens of papers - so we need a way to drink from that firehose - even if only to weigh/prioritize different papers, and cross-reference those.

For instance, the original Tree of Thoughts code that we came up with was entirely based on using the OpenAI ChatPDF plugin to process the PDF and then generate a self-contained prototype in Python.
This actually ended up spitting out working code.

Also, it's a great way to ask questions specific to certain research and cross reference that research with similar papers.

So I would not underestimate the power of this approach, nobody said that we want to dispend with human thinking ...