Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

Support fully autonomous mode for agent #1798

Open
xingyaoww opened this issue May 14, 2024 · 0 comments
Open

Support fully autonomous mode for agent #1798

xingyaoww opened this issue May 14, 2024 · 0 comments
Labels
agent framework strategies for prompting, agent, etc architecture related to architecture, including frontend and backend enhancement New feature or request frontend issues related to frontend severity:low Minor issues, code cleanup, etc

Comments

@xingyaoww
Copy link
Collaborator

What problem or use case are you trying to solve?

Discussion starts here: #1755 (comment)

This isn't for testing purpose - I believe it's for SWE-Bench evaluation, see here.
Looking at that giant PR, it looks like we could make this prompt as a built-in one?
'Please continue working on the task on whatever approach you think is suitable.\n'
'If you think you have modified the code in a way that fixes the issue, please run the following command: <execute_bash> exit </execute_bash>.\n'
'IMPORTANT: YOU SHOULD NEVER ASK FOR HUMAN HELP OR USE THE INTERNET TO SOLVE THIS TASK.\n'

The agent could have a full-autonomous mode so that it doesn't ask for user inputs, and instead, it uses the above prompt. @xingyaoww what do you think? I could help on that if you need a hand.

Originally posted by @li-boxuan in #1755 (comment)

Describe the UX of the solution you'd like

Maybe add a checkbox in the front-end that allows the (frontend? backend?) system to automatically provide the prompt ("Please continue working on the task on whatever approach you think is suitable...") whenever the model asks for human inputs.

The user may want it to run autonomously for a while, then uncheck the box; they expect the model will go back to the mode that requests human input.

When autonomous mode finishes, it may or may not solve the user's task (but will emit the AgentFinish action). We need to consider how to handle this - should we allow users to follow up given the same context?

cc @li-boxuan
Do you have thoughts on the technical implementation?

Describe alternatives you've considered

Additional context

@xingyaoww xingyaoww added the enhancement New feature or request label May 14, 2024
@xingyaoww xingyaoww changed the title Support fully autonomous model for agent Support fully autonomous mode for agent May 14, 2024
@xingyaoww xingyaoww added architecture related to architecture, including frontend and backend agent framework strategies for prompting, agent, etc labels May 14, 2024
@SmartManoj SmartManoj added the severity:low Minor issues, code cleanup, etc label May 18, 2024
@SmartManoj SmartManoj added the frontend issues related to frontend label Jun 3, 2024
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
agent framework strategies for prompting, agent, etc architecture related to architecture, including frontend and backend enhancement New feature or request frontend issues related to frontend severity:low Minor issues, code cleanup, etc
Projects
None yet
Development

No branches or pull requests

2 participants