openai/SWELancer-Benchmark

This repo contains the dataset and code for the paper "SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering?"

1,430

55 commits

updated Jul 18, 2025

See the code

README

SWELancer

The SWE-Lancer codebase has been merged into https://github.com/openai/preparedness!

Please see https://github.com/openai/preparedness to run SWELancer.

Significant stargazers

Jangwon Park

873 followers · starred Feb 2025

Nguyen Bach

14 followers · starred Mar 2025

Ciro Santilli (三西猴) OurBigBook.com sponsor me: cirosantilli.com/sponsor, opinions my own 2020冠状病毒审查，2020郝海东，2018新疆改造中心，1989六四事件，1999法轮功，2019 996.ICU, 2018包子露宪，2015 709大抓捕，2015巴拿马文件邓家贵，2017低端人口，2008西藏骚乱<script>alert(1)</script>

2,231 followers · starred Mar 2025

Shenggui Li

408 followers · starred Apr 2025

Similar repos

bytedance/FullStackBench

Official repository for our paper "FullStack Bench: Evaluating LLMs as Full Stack Coders"

122

Vaibhavi1707/unibench_code

SWEAgents and CodeLLM Evaluation Framework

SWE-Perf/SWE-Perf

Dataset

SWE-Perf: Can Language Models Optimize Code Performance on Real-World Repositories?

RUCAIBox/SWE-Master

SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Training

108

datacurve/deep-swe

Dataset

Boom-Bamboo/VLM-Benchmark

ByteDance/FullStackBench

Dataset

Official repository for our paper "FullStack Bench: Evaluating LLMs as Full Stack Coders"

SWE-Perf/SWE-Perf

openai/SWELancer-Benchmark

This repo contains the dataset and code for the paper "SWE-Lancer: Can Frontier LLMs Earn $1 Million from Real-World Freelance Software Engineering?"

1,430

55 commits

updated Jul 18, 2025

See the code

README

SWELancer

The SWE-Lancer codebase has been merged into https://github.com/openai/preparedness!

Please see https://github.com/openai/preparedness to run SWELancer.

Significant stargazers

Jangwon Park

873 followers · starred Feb 2025

Nguyen Bach

14 followers · starred Mar 2025

2,231 followers · starred Mar 2025

Shenggui Li

408 followers · starred Apr 2025

Similar repos

bytedance/FullStackBench

Official repository for our paper "FullStack Bench: Evaluating LLMs as Full Stack Coders"

122

Vaibhavi1707/unibench_code

SWEAgents and CodeLLM Evaluation Framework

SWE-Perf/SWE-Perf

Dataset

SWE-Perf: Can Language Models Optimize Code Performance on Real-World Repositories?

RUCAIBox/SWE-Master

SWE-Master: Unleashing the Potential of Software Engineering Agents via Post-Training

108

datacurve/deep-swe

Dataset

Boom-Bamboo/VLM-Benchmark

ByteDance/FullStackBench

Dataset

Official repository for our paper "FullStack Bench: Evaluating LLMs as Full Stack Coders"

SWE-Perf/SWE-Perf