// Evaluate & guard
SWE-benchv5.0.1
SWE-bench: Can Language Models Resolve Real-world Github Issues?
Open source Maintained For developers
IN ONE PARAGRAPH
SWE-bench is SWE-bench: Can Language Models Resolve Real-world Github Issues?.
Good for: Developers and teams who want to evaluate & guard.
01How to start
- 1Read the official quick-start (link above).
- 2Install from the official package registry:
pip install swebench - 3Try it on a small example before wiring it into a project.
02Alternatives & comparison
Compare side by side → OSWorld [NeurIPS 2024] OSWorld: Benchmarking Multimodal Agents for Open-Ended Tasks in Real Computer Environments ★ 3.2k lm-evaluation-harness A framework for few-shot evaluation of language models. ★ 14k opencompass OpenCompass is an LLM evaluation platform, supporting a wide range of models from OpenAI, Anthropic, Gemini, Qwen, GLM, DeepSeek, etc, across 100+ datasets covering knowledge, reasoning, coding, science, language, long-context, and safety. ★ 7.5k tau2-bench τ-Bench: A Benchmark for Tool-Agent-User Interaction in Real-World Domains ★ 2.2k lmms-eval One-for-All Multimodal Evaluation Toolkit Across Text, Image, Video, and Audio Tasks ★ 4.5k