swebench

This project publishes no release notes. Between v4.0.4 and v4.1.0 there were 32 commits, 31 of them substantive:

• Doc: Link mini (#445)
• Fix inference pyproject extras deps
• Fix #444
• Port over blog from mini
• Restructure some of the tabs
• Correct URL (swebench instead of swe-bench)
• Rename we-bench tab
• Fix formatting in FAQ
• Add authors.yml file
• Update blog text
• Set page url back to swe-bench.com
• Docs: Deploy with regular mkdocs deploy, not mike
• Doc(blog): Add proper path to roulette config
• Update plot
• Doc(blog): Fix broken links
• default to x86 images always
• Minor fix: Add shorthands for run_evaluation
• Nicer logging for run_evaluation
• Update tests
• Update pytest workflow (#464)

…and 11 more.

This project publishes no release notes. Between v4.0.1 and v4.0.4 there were 21 commits, 17 of them substantive:

• Fix docs
• Update PR template
• Add more informative log for prepare_images script
• fixes #58 - overestimating recall for small k (#409)
• Update namespace arg type for argparser to do null-conversion implicitly
• Add clean step for requirements / environments for python constants per #399
• Update README.md
• fixes #421 - fix mention of private dockerhub link in readme (#422)
• Fix: Allow empty namespace correctly. Ref #423 (#424)
• update modal docs + fix deprecations (#426)
• Fix mm dev p5js log parsing
• Fix chartjs log parsing
• support return exceptions (#431)
• CI: Queue up doc pushes (#428)
• Fix docs for prediction format
• fix(build): fix python base images requirement types-setuptools incor… (#417)
• Add tests for harness utils

This project publishes no release notes. Between v3.0.17 and v4.0.1 there were 13 commits, 12 of them substantive:

• Update README.md
• Doc: Add links to other github repos (#387)
• Doc: Fix closing div
• CI: Remove griffe-pydantic from mkdocs extensions (#391)
• Fix loading of jsonl data (#390)
• Support multilingual evaluation (#392)
• Fix: the pbar doesn't update immediately when futures fail unless there are subsequent successful futures. (#370)
• fix: preserve all issue references with same keyword in PRs (#358)
• Add test for PR #358
• Update README.md (#388)
• Urgent, there is a bug when generate prompt_col in create_text_dataset.py #368 (#369)
• Minor #369 fix

This project publishes no release notes. Between v3.0.15 and v3.0.17 there were 11 commits, 10 of them substantive:

• Update collection.md
• catch-all exception for docker pull (#366)
• Clean diff after setup (#353)
• Simplify installation guidelines for inference submodule
• Fixes #368
• Fix/missing text column (#376)
• Docs (#381)
• deploy docs
• Update pytest workflow python version
• Update docs link

This project publishes no release notes. Between v3.0.14 and v3.0.15 there were 4 commits, 4 of them substantive:

• Minor eval log parsing fix
• fixed qhull issue for matplotlib images. (#329)
• Fix logic error in old-style pytest parsing (#338)
• Pausing support on SWE-bench task creation

This project publishes no release notes. Between v3.0.13 and v3.0.14 there were 6 commits, 6 of them substantive:

• Minor simplification of grading
• Fix broken link in README.md (#326)
• Cleanup (remove unused funcs, reorg) (#348)
• Fix: modify unbound variable REPO_NAME to NEW_REPO_NAME (#340)
• Add test_spec back into parsing
• Fix pytest specs for pytest-dev__pytest-8250

This project publishes no release notes. Between v3.0.12 and v3.0.13 there were 2 commits, 2 of them substantive:

• Remove redundant log
• Move swe-bench repos to new org

This project publishes no release notes. Between v3.0.11 and v3.0.12 there were 1 commits, 1 of them substantive:

• Add base_image_key arg to env images