Skip to content

Issues: princeton-nlp/SWE-bench

New issue

Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.

Sign up for GitHub

By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.

Already on GitHub? Sign in to your account

29 Open 49 Closed
29 Open 49 Closed
Author
Filter by author
Label
Filter by label
Use alt + click/return to exclude labels
or ⇧ + click/return for logical OR
Projects
Filter by project
Milestones
Filter by milestone
Assignee
Filter by who’s assigned
Sort
Sort by
Newest Oldest Most commented Least commented Recently updated Least recently updated Best match
Most reactions
👍 👎 😄 🎉 😕 ❤️ 🚀 👀

Issues list

model_name_or_path is None when running models without adapters, causing an error in run_evaluation.py bug Something isn't working
#126 opened May 25, 2024 by rucnyz
Using uv pip instead of pip for significant speedup enhancement New feature or request
#120 opened May 15, 2024 by klieret
1
Has anyone successfully ran an eval on patches against early versions of astropy, sympy, scipy etc? I'm really struggling to run things from earlier python versions documentation Improvements or additions to documentation
#119 opened May 14, 2024 by PandelisZ
2
swe-bench eval stops running after a point bug Something isn't working
#118 opened May 13, 2024 by ssh-randy
2
run_live.py: clone_repo() takes 3 positional arguments but 5 were given bug Something isn't working
#117 opened May 11, 2024 by stevensu1977
Is it possible to evaluate the train set? documentation Improvements or additions to documentation
#116 opened May 10, 2024 by chriscremer
Dockerization of run_evaluation.py
#114 opened May 2, 2024 by aorwall
5
Reproducer Docker image
#113 opened May 1, 2024 by zygi
environment is lost when running pip install bug Something isn't working
#111 opened Apr 30, 2024 by waterson
1
improve eval performance by caching per-repo/version conda environments
#104 opened Apr 26, 2024 by waterson
1
Dataset field & set up reliable environment documentation Improvements or additions to documentation
#99 opened Apr 21, 2024 by Hodge931
Repository not found while running python3 create_text_dataset.py inference This issue is related to running inference
#98 opened Apr 18, 2024 by hsm1997
1
/bin/sh: pytest: command not found when running evaluations bug Something isn't working
#96 opened Apr 17, 2024 by psykhi
2
Remove pre_install from install specs evaluation This issue is related to running evaluation
#95 opened Apr 11, 2024 by carlosejimenez
Adding LLM tokens to the generated inference for cost calculation inference This issue is related to running inference
#91 opened Apr 10, 2024 by moresearch
Running create_text_dataset.py gets Killed and takes too long inference This issue is related to running inference
#88 opened Apr 7, 2024 by vishwa27yvs
2
inference issue run live inference This issue is related to running inference
#87 opened Apr 7, 2024 by kneeraj-AIDE
1
Issue with Django gold patch results not evaluating to "Resolved" when logs say "All tests passed" evaluation This issue is related to running evaluation
#80 opened Apr 5, 2024 by brombaut
@john-b-yang
7
Error when running evaluation - ModuleNotFoundError: No module named 'conda' evaluation This issue is related to running evaluation
#77 opened Apr 4, 2024 by brombaut
11
Unable to replicate basic results evaluation This issue is related to running evaluation
#74 opened Apr 3, 2024 by bytesuji
2
Upper bound score by skilled human?
#72 opened Apr 2, 2024 by paul-gauthier
7
Failed to install low versions of Python with Conda evaluation This issue is related to running evaluation
#71 opened Apr 2, 2024 by abmfy
1
PASS_TO_PASS tests failing on original program evaluation This issue is related to running evaluation
#68 opened Apr 1, 2024 by yuntongzhang
1
How can I run_evalution? What is the args swe_bench_tasks,I need to create by myself? evaluation This issue is related to running evaluation
#59 opened Mar 22, 2024 by Ottomachine1
3
Difficulty Reproducing BM25 Results inference This issue is related to running inference
#58 opened Mar 20, 2024 by VecherVhatuX
1
ProTip! Follow long discussions with comments:>50.