feat: S.M.A.R.T (Specific Multi Agent Recipe Tracking) Prompt POC (Proof of Concept) #2426
+329
−34
Add this suggestion to a batch that can be applied as a single commit.
This suggestion is invalid because no changes were made to the code.
Suggestions cannot be applied while the pull request is closed.
Suggestions cannot be applied while viewing a subset of changes.
Only one suggestion per line can be applied in a batch.
Add this suggestion to a batch that can be applied as a single commit.
Applying suggestions on deleted lines is not supported.
You must change the existing code in this line in order to create a valid suggestion.
Outdated suggestions cannot be applied.
This suggestion has been applied or marked resolved.
Suggestions cannot be applied from pending reviews.
Suggestions cannot be applied on multi-line comments.
Suggestions cannot be applied while the pull request is queued to merge.
Suggestion cannot be applied right now. Please check back later.
Background
Enhancing LLMs with Multi-Step Cognitive Missions with S.M.A.R.T (Specific Multi-Agent Recipe Tracking) Prompt:
Seeking Collaboration on a Proof of Concept
Overview
In exploring the application of Large Language Models (LLMs) across a spectrum of cognitive tasks—from straightforward Q&A to fully autonomous agents—we observe a significant performance variance. While LLMs excel in simple dialogues, their efficacy diminishes as tasks approach autonomous execution.
Current Challenges and Proposed Solutions
Introducing a Novel Approach
I have developed a proof of concept (POC) that sits between these existing strategies, aiming to validate its viability in a production environment. This solution is designed not merely as a user co-pilot but as a mission co-pilot, guiding the LLM through a multi-step chat where each phase has a distinct prompt, goal, and completion criteria. This structured approach allows for complex interactions to be broken down into manageable, sequential steps, thereby enhancing the LLM's performance even with lower-quality models.
Concept Details:
MoodBooster Example
In the MoodBooster scenario, each stage has defined outcomes—some rely on user approval, others on self-assessment. Preliminary tests with GPT-3.5 and GPT-4 show promising results, with models autonomously advancing through the recipe's stages and adhering closely to provided instructions.
Challenges
Managing a 3-Way Chat (LLM, Recipe, User)
The technique introduced necessitates handling a 3-way chat, involving the LLM, the recipe, and the user—a scenario for which it was not originally optimized. Typically, a "system prompt" might guide the recipe, but this scenario proves more intricate. To navigate this complexity, the approach divides instructions into two distinct sets and two levels, enhancing the dynamics of interaction within this multi-end chat environment. This structured method includes:
Finding a General Way via a Prompt to Maintain and Follow the State
Effective stage transitions require a state machine-like behavior within the LLM. Each state necessitates that agents "load" a new personality, aligning their responses with the current context. Instructing the agent to greet the user as a 'NamedAgent' has significantly improved its understanding of the state, enhancing coherence and contextual awareness.
Future Directions
Call for Collaboration From LibreChat contributers
Since this is only a proof of concept (POC), currently there is no way for users to upload their recipes to the code. While the Planner and the Improver tools can be incorporated into the code as tools, the specific recipes should probably be uploaded by the user via the UI.
@danny-avila @fuegovic @berry-13,
I would appreciate your feedback on this idea for implementation and improvement based on the LibreChat code when you have a moment. It would also be helpful if this could be added to the roadmap.
Call for Collaboration From other readers
This project is still in its early stages, having started just a week ago, with the recipe integration and tool descriptions needing further refinement to accommodate a broader range of scenarios. If this initiative resonates with you, especially researchers interested in co-authoring a paper, I welcome your expertise and collaboration to advance this concept towards real-world application.
Fun Fact
Summary of the Code changes
Created a new tool, "agent-coordinator" and 2 recipes.
The motivation and some details on the implementation can be found in the background introduction above.
Change Type
Testing
Mainly manual.
Checklist