Senior Software Engineer – Open Source & SWE-Bench Evaluation (Fully Remote)

Anyone AI Inc.; United States; Remote; USD 83,000 - 165,000; Part time

Job summary

Anyone AI is seeking a software engineer experienced in code review and testing to assess real GitHub issues and PRs. You will analyze task reproducibility, dependencies, and test quality to guide acceptance decisions.

This remote, part-time project-based engagement focuses on technical evaluation of unit tests, coverage, and environment compatibility across multi-file codebases. Strong Git/GitHub experience and Python familiarity are valued.

Qualifications

• 3+ years of professional software engineering experience with multi-file codebases.
• Strong understanding of unit testing, test coverage, and dependency management.
• Experience with Git and GitHub workflows; ability to analyze complex problems.

Responsibilities

• Review coding tasks derived from real GitHub issues and PRs to assess technical soundness and reproducibility.
• Evaluate unit tests for correctness, coverage, and robustness, identifying flaky tests, missing dependencies, version conflicts, and environment issues.
• Determine reproducibility across environments and provide clear recommendations on acceptance.

Skills

Software engineering

Unit testing

Git/GitHub workflows

Tools

Python frameworks

Docker

CI/CD

pip

conda

Job description

• Review coding tasks derived from real GitHub issues and pull requests to assess technical soundness and reproducibility.
• Evaluate unit tests for correctness, coverage, and robustness, identifying flaky tests, missing dependencies, version conflicts, and environment issues.
• Determine whether tasks can be reliably reproduced across environments and provide clear recommendations on acceptance.

Requirements:

• 3+ years of professional software engineering experience, with strong experience in multi-file codebases.
• Strong understanding of unit testing, test coverage, and dependency management.
• Experience with Git and GitHub workflows, and ability to analyze complex technical problems.

Nice to Have:

• Contributions to open-source projects or experience with SWE-Bench benchmarks.
• Experience with Python projects like Django, Flask, scikit-learn, or SymPy.
• Knowledge of Docker, CI/CD, pip, conda, or dependency pinning.

Engagement:

• Work Type: Remote, part-time project-based consulting.
• Focus on software engineering, open-source code review, testing, and technical evaluation.

Anyone AI

Anyone AI is a company that focuses on AI-related projects, particularly in evaluating software engineering tasks. They are a smaller organization with a culture that values technical expertise and attention to detail.