Skip to content
New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

feat: cookbook entry to auto-merge/close select Dependabot PRs #724

Open
wants to merge 8 commits into
base: main
Choose a base branch
from

Conversation

cyai
Copy link

@cyai cyai commented Jan 5, 2024

feat: cookbook entry to auto-merge/close select Dependabot PRs

This commit includesPython script, dependabot_auto_evaluation_pr.py, which automates the process of handling Dependabot PRs. The script authenticates with GitHub, retrieves open PRs from Dependabot, uses marvin ai function to determine heuristics (confidence to merge) to evaluate the PRs, and then decides whether to merge or close the PRs based on the evaluation results.

closes #700

Copy link
Collaborator

@zzstoatzz zzstoatzz left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💙 this is awesome - thanks for the PR!

just a couple comments / requests

.github/workflows/dependabot-auto-evaulate-pr.yaml Outdated Show resolved Hide resolved
def heruistics(pr):
client = OpenAI(api_key=os.getenv("OPENAI_API_KEY"))

def evaluate_heuristic(pr, labels, description, files_changed):
Copy link
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

can we type-hint this since it'll be something we serialize and show to an LLM?

.github/workflows/dependabot-auto-evaulate-pr.yaml Outdated Show resolved Hide resolved
@zzstoatzz
Copy link
Collaborator

@cyai it would also be great if you could run pre-commit install and make sure ruff-format gets a chance to run so we pass static analysis

Comment on lines +3 to +6
on:
pull_request:
paths:
- "cookbook/**/*"
Copy link
Collaborator

@zzstoatzz zzstoatzz Jan 6, 2024

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I don't think we want this to run on all pull requests, perhaps a cron would be more appropriate? or maybe an if that checks the username / branch upon a new pull request?

Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Checking the username / branch upon a new pull request is a great idea.

@cyai
Copy link
Author

cyai commented Jan 17, 2024

Hey, @zzstoatzz I've added commits for the requested changes and also ran pre-commit install and made sure code is according to ruff-format. Please review and let me know if there is anything more to do (:

Copy link

@jsugg jsugg left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The inline comment in the GitHub Actions workflow is clear, but the Python script itself would benefit from more in-line comments explaining the logic, especially around decision points like confidence score thresholds.

return repo.get_pulls(state="open", head="dependabot")


def heruistics(pr) -> float:
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

heruistics != heuristics


def notify_maintainer_merger(pr, confidence_score: float) -> None:
pr.create_issue_comment(
f"Hey @@zzstoatzz! I have evaluated this PR and am {confidence_score*100}%"
Copy link

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The script hardcodes the GitHub username @zzstoatzz in the notification messages. It's better to use a dynamic value or an environment variable for flexibility.

@jsugg
Copy link

jsugg commented Feb 6, 2024

Adding docstrings explaining the purpose, parameters, and return values of functions would greatly improve code readability and maintainability.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment
Labels
None yet
Projects
None yet
Development

Successfully merging this pull request may close these issues.

add cookbook entry to auto-merge/close select Dependabot PRs
3 participants